From tjreedy at udel.edu  Sun Jan  1 09:24:59 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 01 Jan 2012 03:24:59 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
Message-ID: <jdp58r$b11$1@dough.gmane.org>

On 12/31/2011 5:57 PM, Nathan Schneider wrote:

> The necessary resolution of key conflicts is what makes "+" feel less
> natural to me for dicts than it does for sequences, where the order of
> the operands is transparently reflected in the output. For dicts, in
> contrast, we are talking about a "lossy" union/combination where what
> is lost depends on the order of the operands?this is less transparent
> in the output. However it is spelled, though, I would be thrilled if
> such an operation were built in. :)

You mean like dict.update?

-- 
Terry Jan Reedy




From jeanpierreda at gmail.com  Sun Jan  1 09:40:01 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 1 Jan 2012 03:40:01 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <jdp58r$b11$1@dough.gmane.org>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<jdp58r$b11$1@dough.gmane.org>
Message-ID: <CABicbJJz0ohcHBROAwVtxcypYBZm6ChnWYtoxUY5p3mmMHrgjg@mail.gmail.com>

On Sun, Jan 1, 2012 at 3:24 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> You mean like dict.update?

Nah nah, that's "+= "; "+" is dict(d1, **d2).

-- Devin

On Sun, Jan 1, 2012 at 3:24 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/31/2011 5:57 PM, Nathan Schneider wrote:
>
>> The necessary resolution of key conflicts is what makes "+" feel less
>> natural to me for dicts than it does for sequences, where the order of
>> the operands is transparently reflected in the output. For dicts, in
>> contrast, we are talking about a "lossy" union/combination where what
>> is lost depends on the order of the operands?this is less transparent
>> in the output. However it is spelled, though, I would be thrilled if
>> such an operation were built in. :)
>
>
> You mean like dict.update?
>
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From julien at tayon.net  Mon Jan  2 11:51:01 2012
From: julien at tayon.net (julien tayon)
Date: Mon, 2 Jan 2012 11:51:01 +0100
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
Message-ID: <CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>

2011/12/31 Nathan Schneider <nathan at cmu.edu>:
> On Sat, Dec 31, 2011 at 4:55 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 12/31/2011 11:07 AM, julien tayon wrote:
>>
>>> The question is a dict the same as a vector or as a set ?
>>
>>
>> A dict is a mapping, which is to say, a functional set of key,value pairs.
>> 'Functional' means that the keys are unique. This is a key property (pun
>> intended). A vector can be regarded as a functional set of count,value
>> pairs. Some languages use dicts for vectors. Even in Python, people use
>> dicts for sparse arrays of various dimensions, with unspecified pairs
>> assumed to have a 0 value.
Yes, a potentialy indefinite sparse value vector on a base at least
greater. For which the dimension is the path of keys to a value.
And I lamely admitted these were orthogonal in my mind to one another.
(therefore dicthing coupling problematics in the first time since I
will think of matrix later)

That is the reason, I intend to back down on any argumentation on
vectoriel meaning of dict without proper code to back up my claims :)
.

>
> It occurs to me that any argument for a vector interpretation of
> dicts, with element-wise operations, should apply equally well to
> lists and tuples, which are perhaps even more natural as
> representations for vectors. But of course, the + operator for
> sequences is interpreted as concatenation, not element-wise addition;
> it would be strange if the reverse were true for dicts.

I do admit I came to the fact you are right. And, I think that if two
keys were equals but not the value it should raise a collision
exception.

I could provide a consistent "set addition" for dict as an amend for
making so much noise :)

I would also provide sub.

I see a way to provide a dynamic dict to sets translator that would be
quite weiredl.

>
> The necessary resolution of key conflicts is what makes "+" feel less
> natural to me for dicts than it does for sequences, where the order of
> the operands is transparently reflected in the output. For dicts, in
> contrast, we are talking about a "lossy" union/combination where what
> is lost depends on the order of the operands?this is less transparent
> in the output. However it is spelled, though, I would be thrilled if
> such an operation were built in. :)
>
I can write my module  for this. :) (I need to document myself on magic methods)

Well, if dict are vectors I must define :

- matrix (projection of vectors in an another base)
- distance (cosine similarity)

I will therefore be able to define the notion of proximity once done
that will tell if something is almost another thing

a word counter of a text should be close to the word counters of the
keyword it triggers.

PS I did not quite catch the functionnal thingy.

Cheers
-- 
jul


From jamesghutchison at gmail.com  Mon Jan  2 19:09:13 2012
From: jamesghutchison at gmail.com (James Hutchison)
Date: Mon, 2 Jan 2012 11:09:13 -0700
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
Message-ID: <CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>

Here are some "utility" functions I had to write in my current project:

# add entries from d2 to d1
# supports only dict and numeric types for now
# recursive, applies function to all subdicts
def addDictEntries(d1, d2)

# take the max of entries from d2 to d1
# supports only dict and numeric types for now
# recursive, applies function to all subdicts
def maxDictEntries(d1, d2):

# take the min of entries from d2 to d1
# supports only dict and numeric types for now
# recursive, applies function to all subdicts
def minDictEntries(d1, d2)

# merges entries from d2 to d1 like update does
# if an entry already exists and the values differ then throws an exception
# recursive, applies function to all subdicts
def mergeDictEntriesSafe(d1, d2)

# traverses through the dict and converts all defaultdicts to dicts
# this is done so that missing elements will throw an exception rather
than creating a default entry
# also, defaultdicts aren't picklable so this is required in order to
serialize a dict
def unDefaultDict(d)

# same as unDefaultDict but isn't recursive. Only applies to top level keys
def unDefaultDictShallow(d)

The usage for all this varies but I found that combining / adding
dicts together becomes very prevalent when you split datasets up for
multihreading/multiprocessing purposes and need to recombine it later.

so I propose that a "combine" function is added to the dict class and
all dict subclasses. The function would have the following arguments:

def combine(self, d2, conflictFunc=None):

If combineFunc is a function, then that function is called and the
result is used. val = conflictFunc(k1,k2);
If combineFunc is a string, then self's key value calls conflictFunc
and the new value is the return value

    def combine(self, d2, combineFunc=None):
        if combineFunc == None:
            for k,i in d2.items():
                if(not k in self):
                    self[k] = copy.deepcopy(i);
                else:
                    if(hasattr(i,'keys')):
                        self[k].combine(i);
                    else:
                        if self[k] != i:
                            raise someUsefulException();
            return;
        if (type(combineFunc) == str):
            for k,i in d2.items():
                if(not k in self):
                    self[k] = copy.deepcopy(i);
                else:
                    if(hasattr(i,'keys')):
                        self[k].combine(i,combineFunc);
                    else:
                        self[k] = getattr(self[k],combineFunc)(i);
        else:
            for k,i in d2.items():
                if(not k in self):
                    self[k] = copy.deepcopy(i);
                else:
                    if(hasattr(i,'keys')):
                        self[k].combine(i,combineFunc);
                    else:
                        self[k] = combineFunc(self[k],i);

examples:

d.combine(d2); # update with d2 but throw an exception if any keys
have different values
d.combine(d2, myCustomFunc); # if conflict, then result is =
myCustomFunc(v1,v2) which would be defined elsewhere
d.combine(d2, '__add__'); # if conflict, then result is = v1 + v2

>>print(d1)
{'a': 'b', 'c': {'g': 5}}
>>print(d2)
{'a': 'ecause', 'c': {'g': 10}}
>>d1.combine(d2,'__add__');
>>print(d1)
{'a': 'because', 'c': {'g': 15}}

I think doing so would be more powerful and less ambiguous than trying
to give dict a '+'

A second function I suggest adding is asDict(). This would return a
dictionary with all sub dictionaries-types converted to the dict type.
I.E. all defaultdicts and counters become just plain dicts. This would
be the same as my unDefaultDict function. It could take an optional
argument to do the conversion only to the shallow keys

def asDict(self, shallow=False)


From goodman.m.w at gmail.com  Mon Jan  2 23:49:16 2012
From: goodman.m.w at gmail.com (goodman.m.w at gmail.com)
Date: Mon, 2 Jan 2012 14:49:16 -0800
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
Message-ID: <CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>

It's funny, I joined this list several days ago to ask about this very
problem. Perhaps I can add something (there's code below if you want
to skip the discussion):

I do research in computational linguistics, and adding/incrementing
dictionary values is a very common task (e.g. counting unique words in
a corpus). A defaultdict and a for loop have served me well toward
this end (and Counter may be nice, but I haven't tried it), but I've
always wanted something more functional. I found in Haskell a nice
method Map.fromListWith that takes a list of (key, value) pairs and a
function describing how to deal with collisions. It's useful and
flexible.

Summing up what has been said, as I understand it, it's not obvious
what a good behaviour for + is with dicts, and we can't assure it
follows algebraic properties like commutativity. As Guido hinted at in
the first messages, addition over disjoint dicts (and we can include
the identity dict) should be the same no matter the operation. That
is:
{k1:v1} + {k2:v2} = {k1:v1, k2:v2}
{k1:v1} + {} = {k1:v1}

The only interesting cases are when we have the same key:
{k1:v1} + {k1:v2} = {k1:op(v1,v2)}
where op is some binary function.

While dict doesn't have __add__ defined, it does resolve conflicts by
just using the latter value (i.e. op=lambda x, y: y), as in the
following three statements:
d = dict([(k1,v1), (k1,v2)])  # result: {k1:v2}
d[k1] = v3  # result: {k1:v3}
d.update([(k1,v4)  # result: {k1:v4}

I think the latter two, at least, should not be counted as addition,
but rather stay as assignment. Someone brought up the idea of a
__collision__ method that is called when __setitem__ is called on an
existing key. I like this idea, and there's probably some use for it,
but for implementing addition it might not be practical (how, then,
would you re-assign a key's value without forcing "op" to apply on the
new and existing value?).

I propose something like the following proof-of-concept:

class AccumulationDict(dict):
    def __init__(self, accumulator, *args, **kwargs):
        if not callable(accumulator):
            raise TypeError('Accumulator must be callable.')
        self.accumulator = accumulator
        self.accumulate(*args, **kwargs)

    def __additem__(self, key, value):
        if key in self:
            self[key] = self.accumulator(self[key], value)
        else:
            self[key] = value

    def __add__(self, other):
        result = AccumulationDict(self.accumulator, self) # check if
other's accumulator is same?
        result.accumulate(other)
        return result

    def accumulate(self, *args, **kwargs):
        for arg in args:
            if isinstance(arg, list):
                for (key, value) in arg:
                    self.__additem__(key, value)
            elif isinstance(arg, dict):
                for (key, value) in arg.items():
                    self.__additem__(key, value)
            else:
                raise TypeError('Argument must be of type list or dict.')
        for key in kwargs:
            self.__additem__(key, kwargs[key])

Which has the following properties:
* takes a binary function for accumulating existing and new values
* otherwise can be instantiated like a dict
* accumulate() function is like update(), but uses the accumulator to
deal with conflicts
* KeyErrors still occur when getting a non-existent key
* addition is supported and returns a new object with the merged dicts
* __setitem__ and update() are unchanged so they can be used to assign values

>>> d = AccumulationDict(operator.add, [('a',1),('b',1),('a',1)], b=1)
>>> d
{'a': 2, 'b': 2}
>>> d['a'] = 0
>>> d
{'a': 0, 'b': 2}
>>> d + {'a':3,'b':1}
{'a': 3, 'b': 3}
>>> d
{'a': 0, 'b': 2}
>>> d['c']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'c'
>>> d.update({'c':1})
>>> d
{'a': 0, 'c': 1, 'b': 2}
>>> d.accumulate({'c':1})
>>> d
{'a': 0, 'c': 2, 'b': 2}

I hope this contributes to the discussion, and I'd be happy to hear
your thoughts.

Thanks,

-- 
-Michael Wayne Goodman


From eliben at gmail.com  Tue Jan  3 04:08:27 2012
From: eliben at gmail.com (Eli Bendersky)
Date: Tue, 3 Jan 2012 05:08:27 +0200
Subject: [Python-ideas] working on a PEP for the __preview__ package
Message-ID: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>

Hello,

I plan to take the ideas discussed in the thread on the
__experimental__/__preview__ package (1), and further elaborated by
Nick Coghlan in (2), and turn it into a PEP. Further, I'm willing to
invest the time resources to make the package release-worthy before
Alpha 1 of 3.3, with at least one module in it. This work will be done
under the mentorship of Nick Coghlan.

Any ideas, suggestions or objections are welcome.

Eli

----
(1) http://mail.python.org/pipermail/python-ideas/2011-August/011317.html
(2) http://readthedocs.org/docs/ncoghlan_devs-python-notes/en/latest/pep_ideas/preview_namespace.html


From ncoghlan at gmail.com  Tue Jan  3 05:32:12 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 3 Jan 2012 14:32:12 +1000
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
Message-ID: <CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>

On Tue, Jan 3, 2012 at 1:08 PM, Eli Bendersky <eliben at gmail.com> wrote:
> Hello,
>
> I plan to take the ideas discussed in the thread on the
> __experimental__/__preview__ package (1), and further elaborated by
> Nick Coghlan in (2), and turn it into a PEP. Further, I'm willing to
> invest the time resources to make the package release-worthy before
> Alpha 1 of 3.3, with at least one module in it. This work will be done
> under the mentorship of Nick Coghlan.
>
> Any ideas, suggestions or objections are welcome.

I think one thing that needs to be made crystal clear (both in the PEP
and in the documentation for the namespace) is that anyone attempting
to combine the preview namespace with persistence mechanisms that rely
on module names are setting themselves up for migration problems in
the future.

It isn't just the case that the __preview__ namespace APIs *might*
change: they're actually *guaranteed* to change, when they move to
their final location (probably, but not necessarily, in the next
release).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From anacrolix at gmail.com  Tue Jan  3 15:57:26 2012
From: anacrolix at gmail.com (Matt Joiner)
Date: Wed, 4 Jan 2012 01:57:26 +1100
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
Message-ID: <CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>

Just a suggestion:

Golang uses an "exp" namespace, this would translate to a exp package
for grouping like modules in a similar way that xml, http and
concurrent are currently used.
http://golang.org/pkg/exp/

The name is unoffensive, and somewhat more palatable than __preview__,
which really isn't a protocol, or a bag for special interpreter
switches like __future__.

Under this scheme, the currently proposed modules would be:

exp.regex
exp.daemon
exp.ipaddr

Rather than

__preview__.regex
__preview__.daemon
__preview__.ipaddr

Either way I really look forward to something for PEP 3153 being trialed.
http://www.python.org/dev/peps/pep-3153/

Matt

On Tue, Jan 3, 2012 at 3:32 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Jan 3, 2012 at 1:08 PM, Eli Bendersky <eliben at gmail.com> wrote:
>> Hello,
>>
>> I plan to take the ideas discussed in the thread on the
>> __experimental__/__preview__ package (1), and further elaborated by
>> Nick Coghlan in (2), and turn it into a PEP. Further, I'm willing to
>> invest the time resources to make the package release-worthy before
>> Alpha 1 of 3.3, with at least one module in it. This work will be done
>> under the mentorship of Nick Coghlan.
>>
>> Any ideas, suggestions or objections are welcome.
>
> I think one thing that needs to be made crystal clear (both in the PEP
> and in the documentation for the namespace) is that anyone attempting
> to combine the preview namespace with persistence mechanisms that rely
> on module names are setting themselves up for migration problems in
> the future.
>
> It isn't just the case that the __preview__ namespace APIs *might*
> change: they're actually *guaranteed* to change, when they move to
> their final location (probably, but not necessarily, in the next
> release).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
?_?


From digitalxero at gmail.com  Tue Jan  3 16:06:09 2012
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Tue, 3 Jan 2012 10:06:09 -0500
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
Message-ID: <CAMPUAFPUcHhZfguroX9SEgb0K5zHtE4HhQdxGbEM2EQdxiW09A@mail.gmail.com>

On Tue, Jan 3, 2012 at 9:57 AM, Matt Joiner <anacrolix at gmail.com> wrote:
> The name is unoffensive, and somewhat more palatable than __preview__,
> which really isn't a protocol, or a bag for special interpreter
> switches like __future__.

It can be a bag for special interpreter switches or packages that are
up for api review. Also not using a dunder name space indicates
stability so it isnt an option


From techtonik at gmail.com  Tue Jan  3 16:47:47 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 3 Jan 2012 18:47:47 +0300
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
Message-ID: <CAPkN8x+6qcRKQMaoGZoMWDbuR86B4-NC3bZtC0Q7Oxx6ebcjYg@mail.gmail.com>

On Tue, Jan 3, 2012 at 6:08 AM, Eli Bendersky <eliben at gmail.com> wrote:

> Hello,
>
> I plan to take the ideas discussed in the thread on the
> __experimental__/__preview__ package (1), and further elaborated by
> Nick Coghlan in (2), and turn it into a PEP. Further, I'm willing to
> invest the time resources to make the package release-worthy before
> Alpha 1 of 3.3, with at least one module in it. This work will be done
> under the mentorship of Nick Coghlan.
>
> Any ideas, suggestions or objections are welcome.
>
> Eli
>
> ----
> (1) http://mail.python.org/pipermail/python-ideas/2011-August/011317.html
> (2)
> http://readthedocs.org/docs/ncoghlan_devs-python-notes/en/latest/pep_ideas/preview_namespace.html


Before attempting any library refactoring or addition, I would first think
about placing a feedback mechanism in place to collect public response
about satisfaction with current stdlib contents, and collecting proposals
about parts that should be included into stdlib (or another bundle shipped
with Python).
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120103/6e9318b5/attachment.html>

From robert.kern at gmail.com  Tue Jan  3 17:27:35 2012
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 03 Jan 2012 16:27:35 +0000
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
Message-ID: <jdva9o$6fa$1@dough.gmane.org>

On 1/3/12 2:57 PM, Matt Joiner wrote:
> Just a suggestion:
>
> Golang uses an "exp" namespace, this would translate to a exp package
> for grouping like modules in a similar way that xml, http and
> concurrent are currently used.
> http://golang.org/pkg/exp/
>
> The name is unoffensive, and somewhat more palatable than __preview__,
> which really isn't a protocol, or a bag for special interpreter
> switches like __future__.
>
> Under this scheme, the currently proposed modules would be:
>
> exp.regex
> exp.daemon
> exp.ipaddr
>
> Rather than
>
> __preview__.regex
> __preview__.daemon
> __preview__.ipaddr

The nice thing about the dunders is that no one will make a module with a 
conflicting name. That's really the common factor behind all uses of dunders: 
reserving a name for Python's use. Whether something is a protocol or bag of 
switches doesn't really enter into it.

Like "new" and "random", "exp" or anything else that means something like 
"experimental" is going to trigger some problems when people use that name for 
*their* experimental modules.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From nathan.alexander.rice at gmail.com  Tue Jan  3 17:59:45 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Tue, 3 Jan 2012 11:59:45 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
Message-ID: <CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>

This is slightly tangential, but I've always wondered... Why aren't
set operations implemented on dicts?  It is fairly natural to view a
dictionary as a set of (key, value) pairs.  Things like
subset/superset checking (with concomitant operator support) make
sense.  I have written stuff like set(dict1.items()) <
set(dict2.items()) many times.

I don't even know what rich comparison operators on dictionaries do
now, it isn't intuitive at all.

Nathan


From robert.kern at gmail.com  Tue Jan  3 18:10:51 2012
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 03 Jan 2012 17:10:51 +0000
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
Message-ID: <jdvcqr$pd5$1@dough.gmane.org>

On 1/3/12 4:59 PM, Nathan Rice wrote:
> This is slightly tangential, but I've always wondered... Why aren't
> set operations implemented on dicts?  It is fairly natural to view a
> dictionary as a set of (key, value) pairs.  Things like
> subset/superset checking (with concomitant operator support) make
> sense.  I have written stuff like set(dict1.items())<
> set(dict2.items()) many times.

The values are unrestricted Python objects. They do not have to be hashable or 
sortable. The set operations you describe would have to be require one or both 
(or else do something algorithmically horrendous).

Further, you cannot treat dicts as sets of (key, value) pairs because dicts have 
unique keys, not unique (key, value) pairs.

> I don't even know what rich comparison operators on dictionaries do
> now, it isn't intuitive at all.

The rich comparison operators are only defined for == and !=. In Python 2.x, 
there is a legacy implementation of __cmp__ that does something more 
complicated. I recommend ignoring it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From solipsis at pitrou.net  Tue Jan  3 18:29:42 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Jan 2012 18:29:42 +0100
Subject: [Python-ideas] working on a PEP for the __preview__ package
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org>
Message-ID: <20120103182942.1552481e@pitrou.net>

On Tue, 03 Jan 2012 16:27:35 +0000
Robert Kern <robert.kern at gmail.com> wrote:
> On 1/3/12 2:57 PM, Matt Joiner wrote:
> > Just a suggestion:
> >
> > Golang uses an "exp" namespace, this would translate to a exp package
> > for grouping like modules in a similar way that xml, http and
> > concurrent are currently used.
> > http://golang.org/pkg/exp/
> >
> > The name is unoffensive, and somewhat more palatable than __preview__,
> > which really isn't a protocol, or a bag for special interpreter
> > switches like __future__.
> >
> > Under this scheme, the currently proposed modules would be:
> >
> > exp.regex
> > exp.daemon
> > exp.ipaddr
> >
> > Rather than
> >
> > __preview__.regex
> > __preview__.daemon
> > __preview__.ipaddr
> 
> The nice thing about the dunders is that no one will make a module with a 
> conflicting name. That's really the common factor behind all uses of dunders: 
> reserving a name for Python's use. Whether something is a protocol or bag of 
> switches doesn't really enter into it.

Plus, writing "from __preview__ import regex" looks cooler than
"from exp import regex".

Regards

Antoine.




From nathan.alexander.rice at gmail.com  Tue Jan  3 18:39:48 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Tue, 3 Jan 2012 12:39:48 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <jdvcqr$pd5$1@dough.gmane.org>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvcqr$pd5$1@dough.gmane.org>
Message-ID: <CAOFbRmJQC+j9zsW1GwszfYZNs_kdVAoVee45tdO4JXvnL6UnrQ@mail.gmail.com>

On Tue, Jan 3, 2012 at 12:10 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 1/3/12 4:59 PM, Nathan Rice wrote:
>>
>> This is slightly tangential, but I've always wondered... Why aren't
>> set operations implemented on dicts? ?It is fairly natural to view a
>> dictionary as a set of (key, value) pairs. ?Things like
>> subset/superset checking (with concomitant operator support) make
>> sense. ?I have written stuff like set(dict1.items())<
>> set(dict2.items()) many times.
>
>
> The values are unrestricted Python objects. They do not have to be hashable
> or sortable. The set operations you describe would have to be require one or
> both (or else do something algorithmically horrendous).

I haven't had any problems with using set(somedict.items()).  I will
admit that I primarily do this in simple contexts.

This brings me to another curiosity... Why do mutable items not
implement hash using id()?

> Further, you cannot treat dicts as sets of (key, value) pairs because dicts
> have unique keys, not unique (key, value) pairs.

I'm confused.  Because keys are unique, (key, value) pairs are unique
as well.  I realize the values are not unique but the tuple is
guaranteed to be.

>> I don't even know what rich comparison operators on dictionaries do
>> now, it isn't intuitive at all.
>
>
> The rich comparison operators are only defined for == and !=. In Python 2.x,
> there is a legacy implementation of __cmp__ that does something more
> complicated. I recommend ignoring it.

Recommendation definitely noted :D

Nathan


From robert.kern at gmail.com  Tue Jan  3 19:45:56 2012
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 03 Jan 2012 18:45:56 +0000
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAOFbRmJQC+j9zsW1GwszfYZNs_kdVAoVee45tdO4JXvnL6UnrQ@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvcqr$pd5$1@dough.gmane.org>
	<CAOFbRmJQC+j9zsW1GwszfYZNs_kdVAoVee45tdO4JXvnL6UnrQ@mail.gmail.com>
Message-ID: <jdvid4$2th$1@dough.gmane.org>

On 1/3/12 5:39 PM, Nathan Rice wrote:
> On Tue, Jan 3, 2012 at 12:10 PM, Robert Kern<robert.kern at gmail.com>  wrote:
>> On 1/3/12 4:59 PM, Nathan Rice wrote:
>>>
>>> This is slightly tangential, but I've always wondered... Why aren't
>>> set operations implemented on dicts?  It is fairly natural to view a
>>> dictionary as a set of (key, value) pairs.  Things like
>>> subset/superset checking (with concomitant operator support) make
>>> sense.  I have written stuff like set(dict1.items())<
>>> set(dict2.items()) many times.
>>
>>
>> The values are unrestricted Python objects. They do not have to be hashable
>> or sortable. The set operations you describe would have to be require one or
>> both (or else do something algorithmically horrendous).
>
> I haven't had any problems with using set(somedict.items()).  I will
> admit that I primarily do this in simple contexts.
>
> This brings me to another curiosity... Why do mutable items not
> implement hash using id()?

Usually because they do not implement __eq__ using id(). The invariant that 
needs to be maintained is that if two objects compare equal, then they need to 
hash equal. Many mutable objects compare by value, and thus two non-identical 
objects can compare equal.

>> Further, you cannot treat dicts as sets of (key, value) pairs because dicts
>> have unique keys, not unique (key, value) pairs.
>
> I'm confused.  Because keys are unique, (key, value) pairs are unique
> as well.  I realize the values are not unique but the tuple is
> guaranteed to be.

Yes, the dict.items() would be a valid, unique set, but set operations on those 
sets may not give valid dicts because the same key could point to different 
values in the two dict operands. For example:

[~]
|7> a = {'one': 1}

[~]
|8> b = {'one': '1'}

[~]
|9> set(a.items()) | set(b.items())
set([('one', '1'), ('one', 1)])

[~]
|10> dict(set(a.items()) | set(b.items()))
{'one': 1}

You've lost a value there.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From joshua.landau.ws at gmail.com  Tue Jan  3 19:55:45 2012
From: joshua.landau.ws at gmail.com (Joshua Landau)
Date: Tue, 3 Jan 2012 18:55:45 +0000
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <jdvid4$2th$1@dough.gmane.org>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvcqr$pd5$1@dough.gmane.org>
	<CAOFbRmJQC+j9zsW1GwszfYZNs_kdVAoVee45tdO4JXvnL6UnrQ@mail.gmail.com>
	<jdvid4$2th$1@dough.gmane.org>
Message-ID: <CAN1F8qUTYnxnQgLKbjks5ggi8uP6bcwAVGUqNwgQopfH_1uTew@mail.gmail.com>

On 3 January 2012 18:45, Robert Kern <robert.kern at gmail.com> wrote:

> [~]
> |7> a = {'one': 1}
>
> [~]
> |8> b = {'one': '1'}
>
> [~]
> |9> set(a.items()) | set(b.items())
> set([('one', '1'), ('one', 1)])
>
> [~]
> |10> dict(set(a.items()) | set(b.items()))
> {'one': 1}
>

I don't know what you've done here:

>>> a = {'one': 1}
>>> b = {'one': 1}
>>> set(a.items()) | set(b.items())
{('one', 1)}
>>> dict(set(a.items()) | set(b.items()))
{'one': 1}
>>>

(
However, your point remains:

>>> a = {'number': 1}
>>> b = {'number': 2}
>>> set(a.items()) | set(b.items())
{('number', 1), ('number', 2)}
>>> dict(set(a.items()) | set(b.items()))
{'number': 2}
>>>
)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120103/a036523d/attachment.html>

From robert.kern at gmail.com  Tue Jan  3 19:58:46 2012
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 03 Jan 2012 18:58:46 +0000
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAN1F8qUTYnxnQgLKbjks5ggi8uP6bcwAVGUqNwgQopfH_1uTew@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvcqr$pd5$1@dough.gmane.org>
	<CAOFbRmJQC+j9zsW1GwszfYZNs_kdVAoVee45tdO4JXvnL6UnrQ@mail.gmail.com>
	<jdvid4$2th$1@dough.gmane.org>
	<CAN1F8qUTYnxnQgLKbjks5ggi8uP6bcwAVGUqNwgQopfH_1uTew@mail.gmail.com>
Message-ID: <jdvj56$7qq$1@dough.gmane.org>

On 1/3/12 6:55 PM, Joshua Landau wrote:
> On 3 January 2012 18:45, Robert Kern
> <robert.kern at gmail.com
> <mailto:robert.kern at gmail.com>> wrote:
>
>     [~]
>     |7> a = {'one': 1}
>
>     [~]
>     |8> b = {'one': '1'}
>
>     [~]
>     |9> set(a.items()) | set(b.items())
>     set([('one', '1'), ('one', 1)])
>
>     [~]
>     |10> dict(set(a.items()) | set(b.items()))
>     {'one': 1}
>
>
> I don't know what you've done here:
>
>  >>> a = {'one': 1}
>  >>> b = {'one': 1}
>  >>> set(a.items()) | set(b.items())
> {('one', 1)}
>  >>> dict(set(a.items()) | set(b.items()))
> {'one': 1}
>  >>>

Sorry, I was a little too clever for my own good:

   a = {'one': 1}
   b = {'one': '1'}

The first has the integer 1; the second has the string '1'.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From nathan.alexander.rice at gmail.com  Tue Jan  3 21:37:42 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Tue, 3 Jan 2012 15:37:42 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <jdvid4$2th$1@dough.gmane.org>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvcqr$pd5$1@dough.gmane.org>
	<CAOFbRmJQC+j9zsW1GwszfYZNs_kdVAoVee45tdO4JXvnL6UnrQ@mail.gmail.com>
	<jdvid4$2th$1@dough.gmane.org>
Message-ID: <CAOFbRmJXcBCq4j88a_RFusxixo7WSrfyHyrdJjM93o+KvWBbZA@mail.gmail.com>

On Tue, Jan 3, 2012 at 1:45 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 1/3/12 5:39 PM, Nathan Rice wrote:
>> I haven't had any problems with using set(somedict.items()). ?I will
>> admit that I primarily do this in simple contexts.
>>
>> This brings me to another curiosity... Why do mutable items not
>> implement hash using id()?
>
>
> Usually because they do not implement __eq__ using id(). The invariant that
> needs to be maintained is that if two objects compare equal, then they need
> to hash equal. Many mutable objects compare by value, and thus two
> non-identical objects can compare equal.

Ok.  I missed that invariant in the data model (and a couple of other
places, apparently).  Thanks.

>>> Further, you cannot treat dicts as sets of (key, value) pairs because
>>> dicts
>>> have unique keys, not unique (key, value) pairs.
>>
>>
>> I'm confused. ?Because keys are unique, (key, value) pairs are unique
>> as well. ?I realize the values are not unique but the tuple is
>> guaranteed to be.
>
>
> Yes, the dict.items() would be a valid, unique set, but set operations on
> those sets may not give valid dicts because the same key could point to
> different values in the two dict operands. For example:
>
> [~]
> |7> a = {'one': 1}
>
> [~]
> |8> b = {'one': '1'}
>
> [~]
> |9> set(a.items()) | set(b.items())
> set([('one', '1'), ('one', 1)])
>
> [~]
> |10> dict(set(a.items()) | set(b.items()))
> {'one': 1}
>
> You've lost a value there.

I have been following the earlier discussion of + in the context of
dictionaries, and I agree that many operators do not cleanly map over.
 It seems to me that the comparison operators are well behaved in this
regard though.

As an additional aside, perhaps the return type of elements in the
items call could be made into a nice "Item" namedtuple.  I always
unpack my items into (k, v) or something similar when possible, but
some people aren't as considerate.  Having item.key and item.value
available (and used in examples) encourages nice code; even if someone
chose a horrible name for their items() unpacking, it is easier to
grok "badname.key ... badname.value" than "badname[0] ... badname[1]".

Nathan


From jeanpierreda at gmail.com  Tue Jan  3 21:50:51 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 3 Jan 2012 15:50:51 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <jdvcqr$pd5$1@dough.gmane.org>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvcqr$pd5$1@dough.gmane.org>
Message-ID: <CABicbJLDryDOdyZVV=koRs2CXyn-n-29KLaAFQw8gSFoFCHqAg@mail.gmail.com>

> The values are unrestricted Python objects. They do not have to be hashable
> or sortable. The set operations you describe would have to be require one or
> both (or else do something algorithmically horrendous).

He only describes <, which can be implemented in linear time as:

    def __lt__(self, d2):
        if not isinstance(d2, dict):
            return NotImplemented

        return all(key in d2 and d2[key] == value for key, value in
self.items())

Which set operations are you thinking of?

-- Devin

On Tue, Jan 3, 2012 at 12:10 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 1/3/12 4:59 PM, Nathan Rice wrote:
>>
>> This is slightly tangential, but I've always wondered... Why aren't
>> set operations implemented on dicts? ?It is fairly natural to view a
>> dictionary as a set of (key, value) pairs. ?Things like
>> subset/superset checking (with concomitant operator support) make
>> sense. ?I have written stuff like set(dict1.items())<
>> set(dict2.items()) many times.
>
>
> The values are unrestricted Python objects. They do not have to be hashable
> or sortable. The set operations you describe would have to be require one or
> both (or else do something algorithmically horrendous).
>
> Further, you cannot treat dicts as sets of (key, value) pairs because dicts
> have unique keys, not unique (key, value) pairs.
>
>
>> I don't even know what rich comparison operators on dictionaries do
>> now, it isn't intuitive at all.
>
>
> The rich comparison operators are only defined for == and !=. In Python 2.x,
> there is a legacy implementation of __cmp__ that does something more
> complicated. I recommend ignoring it.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
> ?that is made terrible by our own mad attempt to interpret it as though it
> had
> ?an underlying truth."
> ?-- Umberto Eco
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From robert.kern at gmail.com  Tue Jan  3 21:57:25 2012
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 03 Jan 2012 20:57:25 +0000
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CABicbJLDryDOdyZVV=koRs2CXyn-n-29KLaAFQw8gSFoFCHqAg@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvcqr$pd5$1@dough.gmane.org>
	<CABicbJLDryDOdyZVV=koRs2CXyn-n-29KLaAFQw8gSFoFCHqAg@mail.gmail.com>
Message-ID: <jdvq3l$n90$1@dough.gmane.org>

On 1/3/12 8:50 PM, Devin Jeanpierre wrote:
>> The values are unrestricted Python objects. They do not have to be hashable
>> or sortable. The set operations you describe would have to be require one or
>> both (or else do something algorithmically horrendous).
>
> He only describes<, which can be implemented in linear time as:
>
>      def __lt__(self, d2):
>          if not isinstance(d2, dict):
>              return NotImplemented
>
>          return all(key in d2 and d2[key] == value for key, value in
> self.items())

Actually, this implements __le__. For __lt__, you need to exclude the case where 
they are exactly equal.

You're right though that it wouldn't be algorithmically horrendous to do this.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From tjreedy at udel.edu  Tue Jan  3 22:02:04 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 03 Jan 2012 16:02:04 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
Message-ID: <jdvqcg$sd8$1@dough.gmane.org>

On 1/3/2012 11:59 AM, Nathan Rice wrote:
> This is slightly tangential, but I've always wondered... Why aren't
> set operations implemented on dicts?  It is fairly natural to view a
> dictionary as a set of (key, value) pairs.  Things like
> subset/superset checking (with concomitant operator support) make
> sense.  I have written stuff like set(dict1.items())<
> set(dict2.items()) many times.

As a said earlier in this thread, being able to view dicts as sets is 
such a good idea that it was already added a few years ago.

 >>> {1:1}.items() < {1:1, 2:2}.items()
True

> I don't even know what rich comparison operators on dictionaries do
> now, it isn't intuitive at all.

TypeError: unorderable types: dict() < dict()

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Tue Jan  3 22:10:30 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 03 Jan 2012 16:10:30 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <jdvid4$2th$1@dough.gmane.org>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvcqr$pd5$1@dough.gmane.org>
	<CAOFbRmJQC+j9zsW1GwszfYZNs_kdVAoVee45tdO4JXvnL6UnrQ@mail.gmail.com>
	<jdvid4$2th$1@dough.gmane.org>
Message-ID: <jdvqsa$vuf$1@dough.gmane.org>

On 1/3/2012 1:45 PM, Robert Kern wrote:

> |7> a = {'one': 1}
> |8> b = {'one': '1'}
> |9> set(a.items()) | set(b.items())
> set([('one', '1'), ('one', 1)])

In Python 3, the subject of this list, converting set views to sets is 
redundant.

 >>> a.items() | b.items()
{('one', '1'), ('one', 1)}
 >>> set(a.items()) | set(b.items())
{('one', '1'), ('one', 1)}

-- 
Terry Jan Reedy



From barry at python.org  Tue Jan  3 22:18:43 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 3 Jan 2012 16:18:43 -0500
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
References: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
Message-ID: <20120103161843.25044187@resist.wooz.org>

On Dec 27, 2011, at 09:01 AM, anatoly techtonik wrote:

>As you may know, the python-ideas list is opened only to subscribers. This 
>is inconvenient, because:
>1. it requires three step subscription process
>2. it is impossible to post a reply to existing thread/idea
>
>There is a web-interface in Google Groups at 
>https://groups.google.com/forum/#!forum/python-ideas that can solve 
>problems above and provide some more nifty features such as embedded 
>search.  But there comes another problem that messages posted through the 
>group doesn't end in list, because list requires subscription.  I've 
>already tried to find a solution, but run out of time, so I summarized the 
>proposal at http://wiki.python.org/moin/MailmanWithGoogleGroups

I disagree with some of the statements on that wiki page.

Most if not all of our mailing lists are mirrored on Gmane and very likely
many other public archive sites.  You can read and post to mailing lists
through Gmane, after an initial verification dance, just like I'm doing here
to this message.

The problem with search and "thread notifications" (if I understand the latter
correctly) is a problem with Pipermail, not specifically Mailman even though
the former is bundled with the latter.  I've been begging people for at least
a decade to help out with modernizing Pipermail, but the truth is that the
state of the art in open source archivers has been abysmal for years.

OpenID/OpenAuth (maybe BrowserID, etc.) - come join us in the Mailman 3
project, and help us get things polished so we can release it.

I'd certainly welcome a Google Groups mirror of any of our mailing lists, just
as I welcome mirrors on Gmane, The Mail Archive, etc.  I do think our core
technology ought to be open source, but that doesn't mean we shouldn't
maximize the reach of these valuable assets of ours.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120103/43759305/attachment.pgp>

From barry at python.org  Tue Jan  3 22:22:58 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 3 Jan 2012 16:22:58 -0500
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
References: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
	<jddenk$gr1$1@dough.gmane.org>
	<14270238.69.1325067063613.JavaMail.geo-discussion-forums@yqip20>
Message-ID: <20120103162258.0978cd1e@resist.wooz.org>

On Dec 28, 2011, at 02:11 AM, anatoly techtonik wrote:

>Does Mailman support confirmation for messages with high SPAM ratio?

Not out of the box, no.

First, spam detection is not Mailman's mission.  This is better done with
other tools in integration with an MTA.  There could be some integration
points with Mailman here, but it's not clear they are worth it, especially
when other more interesting work could be done (e.g. DKIM, OpenPGP signatures,
etc.).  Of course, I welcome anyone interested to join us on
mailman-developers at python.org if you want to discuss stuff like this further.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120103/6add81ad/attachment.pgp>

From nathan.alexander.rice at gmail.com  Tue Jan  3 22:25:44 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Tue, 3 Jan 2012 16:25:44 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <jdvqcg$sd8$1@dough.gmane.org>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvqcg$sd8$1@dough.gmane.org>
Message-ID: <CAOFbRmKFQ0CuNvN8Yfq4bs-8-Ls_O6NX3nqo2p0YmHopE6cUPw@mail.gmail.com>

On Tue, Jan 3, 2012 at 4:02 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 1/3/2012 11:59 AM, Nathan Rice wrote:
>>
>> This is slightly tangential, but I've always wondered... Why aren't
>> set operations implemented on dicts? ?It is fairly natural to view a
>> dictionary as a set of (key, value) pairs. ?Things like
>> subset/superset checking (with concomitant operator support) make
>> sense. ?I have written stuff like set(dict1.items())<
>> set(dict2.items()) many times.
>
>
> As a said earlier in this thread, being able to view dicts as sets is such a
> good idea that it was already added a few years ago.
>
>>>> {1:1}.items() < {1:1, 2:2}.items()
> True

Yes, my apologies, I knew about views but I missed the part where they
have set operations.  I was under the impression when I wrote that
they were just plain iterators.  I try to keep current on Py3 nuances,
but since my work ties me to Py2, I will miss subtle but important
points like that from time to time.  I still don't think there is
anything wrong with having the comparison operations present on the
dictionary directly, it seems intuitive *to me*.  Of course I
understand that I may be an outlier in this (and it certainly wouldn't
be the first time :P).

Nathan


From ncoghlan at gmail.com  Wed Jan  4 02:29:37 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 4 Jan 2012 11:29:37 +1000
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <20120103182942.1552481e@pitrou.net>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
Message-ID: <CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>

On Wed, Jan 4, 2012 at 3:29 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 03 Jan 2012 16:27:35 +0000
> Robert Kern <robert.kern at gmail.com> wrote:
>> The nice thing about the dunders is that no one will make a module with a
>> conflicting name. That's really the common factor behind all uses of dunders:
>> reserving a name for Python's use. Whether something is a protocol or bag of
>> switches doesn't really enter into it.
>
> Plus, writing "from __preview__ import regex" looks cooler than
> "from exp import regex".

Dunder names also alert people that something special is going on (in
this case, it helps alert them to the fact that the modules in this
namespace *will* be moved in the future).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Wed Jan  4 02:34:22 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 4 Jan 2012 11:34:22 +1000
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CAPkN8x+6qcRKQMaoGZoMWDbuR86B4-NC3bZtC0Q7Oxx6ebcjYg@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CAPkN8x+6qcRKQMaoGZoMWDbuR86B4-NC3bZtC0Q7Oxx6ebcjYg@mail.gmail.com>
Message-ID: <CADiSq7eikhzVrNV0afFu-cZUCfFXDQ827jMNS=bQRPhTayKyXw@mail.gmail.com>

On Wed, Jan 4, 2012 at 1:47 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> Before attempting any library refactoring or addition, I would first think
> about placing a feedback mechanism in place to collect public response about
> satisfaction with current stdlib contents, and collecting proposals about
> parts that should be included into stdlib (or another bundle shipped with
> Python).

Sorry anatoly, you're not going to win that one. We're well aware you
don't like the use of mailing lists, the issue tracker, IRC, blogs,
Reddit, Hacker News, Stack Overflow, etc for feedback and want
something else that is formal and structured, but isn't the issue
tracker (since you object to that for reasons I don't really
understand). If you want to set up your own feedback mechanism,
encourage people to use it, and act as a conduit between your new
mechanism and the channels the current core devs are already paying
attention to, feel free.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From solipsis at pitrou.net  Wed Jan  4 02:37:08 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 4 Jan 2012 02:37:08 +0100
Subject: [Python-ideas] working on a PEP for the __preview__ package
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
	<CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
Message-ID: <20120104023708.75e25be8@pitrou.net>


(and by the way I think __preview__ is a *very* interesting idea!)

cheers

Antoine.


On Wed, 4 Jan 2012 11:29:37 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Jan 4, 2012 at 3:29 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Tue, 03 Jan 2012 16:27:35 +0000
> > Robert Kern <robert.kern at gmail.com> wrote:
> >> The nice thing about the dunders is that no one will make a module with a
> >> conflicting name. That's really the common factor behind all uses of dunders:
> >> reserving a name for Python's use. Whether something is a protocol or bag of
> >> switches doesn't really enter into it.
> >
> > Plus, writing "from __preview__ import regex" looks cooler than
> > "from exp import regex".
> 
> Dunder names also alert people that something special is going on (in
> this case, it helps alert them to the fact that the modules in this
> namespace *will* be moved in the future).
> 
> Cheers,
> Nick.




From stephen at xemacs.org  Wed Jan  4 03:48:03 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 04 Jan 2012 11:48:03 +0900
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
In-Reply-To: <20120103162258.0978cd1e@resist.wooz.org>
References: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
	<jddenk$gr1$1@dough.gmane.org>
	<14270238.69.1325067063613.JavaMail.geo-discussion-forums@yqip20>
	<20120103162258.0978cd1e@resist.wooz.org>
Message-ID: <87mxa441zw.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:
 > On Dec 28, 2011, at 02:11 AM, anatoly techtonik wrote:
 > 
 > >Does Mailman support confirmation for messages with high SPAM ratio?
 > 
 > Not out of the box, no.

But it would be trivial to implement efficiently, I think, even in
MM2, as long as there is a front-end (eg, spamassassin) that does the
spam-checking efficiently.

 > First, spam detection is not Mailman's mission.

A big +1 to that, but I've had problems on the XEmacs lists with
marginally spammy stuff.  (Eg, one frequent contributor had a
French-castle-guard+some-even-more-awful-Aussie-slang version of
spook.el hooked up to Gnus, and he regularly ended up snagged by the
"male potency enhancers" filter.  But actual spam outnumbered his
posts about 99 to 1. :^)

So you might want to let some spammy stuff through *to* Mailman, yet
still have Mailman hold the spammy-ER stuff among that.

I don't know if Python lists ever have that kind of problem, though.

 > There could be some integration points with Mailman here, but it's
 > not clear they are worth it,

For mailman-developers, I think it's preferable to teach people to
fish (here, write and install simple Handlers), and provide a contrib
page on the wiki for posting them.



From techtonik at gmail.com  Wed Jan  4 11:45:52 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 4 Jan 2012 13:45:52 +0300
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
	<CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
Message-ID: <CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>

On Wed, Jan 4, 2012 at 4:29 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Wed, Jan 4, 2012 at 3:29 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > On Tue, 03 Jan 2012 16:27:35 +0000
> > Robert Kern <robert.kern at gmail.com> wrote:
> >> The nice thing about the dunders is that no one will make a module with
> a
> >> conflicting name. That's really the common factor behind all uses of
> dunders:
> >> reserving a name for Python's use. Whether something is a protocol or
> bag of
> >> switches doesn't really enter into it.
> >
> > Plus, writing "from __preview__ import regex" looks cooler than
> > "from exp import regex".
>
> Dunder names also alert people that something special is going on (in
> this case, it helps alert them to the fact that the modules in this
> namespace *will* be moved in the future).


I am afraid that dunders will become a standard thing in many Python
packages, because of everybody's natural desire to use latest features that
work right. That means that new people to Python will have to deal with
Python magic right from the start, which will make language more
complicated for them.

A correct way IMO would be to introduce a versioned exp27, exp271 packages:
1. It makes it explicitly clear when imported package version was
considered experimental
2. It allows to switch between multiple experimental package versions
3. It will provide future compatibility if a package released from `exp`
breaks API
5. as a sum of 2+3 above - it provides a development process and extends
time to stabilize API with public testing and feedback before freezing the
package to die in stdlib

-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120104/5535e754/attachment.html>

From techtonik at gmail.com  Wed Jan  4 12:23:55 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 4 Jan 2012 14:23:55 +0300
Subject: [Python-ideas] Public feedback tools (Was: working on a PEP for the
 __preview__ package)
Message-ID: <CAPkN8xKs9CHSetHoqZ42K0YqDztGkveLbvxBP391vqHbnOiS2w@mail.gmail.com>

On Wed, Jan 4, 2012 at 4:34 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Wed, Jan 4, 2012 at 1:47 AM, anatoly techtonik <techtonik at gmail.com>
> wrote:
> > Before attempting any library refactoring or addition, I would first
> think
> > about placing a feedback mechanism in place to collect public response
> about
> > satisfaction with current stdlib contents, and collecting proposals about
> > parts that should be included into stdlib (or another bundle shipped with
> > Python).
>
> Sorry anatoly, you're not going to win that one. We're well aware you
> don't like the use of mailing lists, the issue tracker, IRC, blogs,
> Reddit, Hacker News, Stack Overflow, etc for feedback and want
> something else that is formal and structured, but isn't the issue
> tracker (since you object to that for reasons I don't really
> understand).


I want something that will accumulate and summarize the outcome of all
these informal activities in a structured way. With a tight time limits it
is impossible to read and follow all discussions, so there should be an
easy way to say +1 or -1 to filter useful stuff. There is nothing wrong
with Reddit, Stack Overflow and etc., except that it is very hard to figure
out trends in Python development.

Without visible trends (or Roadmap FWIW) - you can't see what is going to
be changed and how can you help (in accordance with your own skills and
motivations). Even if you don't want to (or can't) help, you at least want
to place your +1, -1 and see what other people think is right or wrong
about Python. At least you will learn something new that will help to
change things you've become aware of in the future.

If you want to set up your own feedback mechanism,
> encourage people to use it, and act as a conduit between your new
> mechanism and the channels the current core devs are already paying
> attention to, feel free.


If only I could finish anything alone..
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120104/f406ac86/attachment.html>

From stephen at xemacs.org  Wed Jan  4 12:29:33 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 04 Jan 2012 20:29:33 +0900
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
	<CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
	<CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>
Message-ID: <878vln4sf6.fsf@uwakimon.sk.tsukuba.ac.jp>

anatoly techtonik writes:

 > I am afraid that dunders will become a standard thing in many Python
 > packages, because of everybody's natural desire to use latest features that
 > work right.

__preview__ is for the latest features that have not been confirmed to
work right (yet).  Where's the problem?  Newbies certainly should not
be using such in production applications, although it's an excellent
way to learn everything you always wanted to know about Python
internals but were afraid to ask (and more!) <wink/>



From techtonik at gmail.com  Wed Jan  4 12:36:27 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 4 Jan 2012 14:36:27 +0300
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <878vln4sf6.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
	<CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
	<CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>
	<878vln4sf6.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPkN8xK8mjPj6sVVuQF48MiX-i-8wHc40Az+W1-aBHVRBOxttQ@mail.gmail.com>

On Wed, Jan 4, 2012 at 2:29 PM, Stephen J. Turnbull <stephen at xemacs.org>wrote:

> anatoly techtonik writes:
>
>  > I am afraid that dunders will become a standard thing in many Python
>  > packages, because of everybody's natural desire to use latest features
> that
>  > work right.
>
> __preview__ is for the latest features that have not been confirmed to
> work right (yet).  Where's the problem?  Newbies certainly should not
> be using such in production applications, although it's an excellent
> way to learn everything you always wanted to know about Python
> internals but were afraid to ask (and more!) <wink/>
>

What's the point in developing code you won't use? __preview__ features
don't have a Roadmap, so you may wait forever to release you package into
production. With indefinite release cycle __preview__ will become useless
for the most people, who will quickly lose enthusiasm to use __preview__
and continuously update their code for API breaks.
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120104/e1901180/attachment.html>

From jeanpierreda at gmail.com  Wed Jan  4 12:49:11 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 4 Jan 2012 06:49:11 -0500
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
	<CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
	<CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>
Message-ID: <CABicbJ+JGEmN9=Cy7xAy=7qqd-8KfNkC9RN_3COSupHyZHBPTg@mail.gmail.com>

> A correct way IMO would be to introduce a versioned exp27, exp271 packages:

+2. Rather than silently introducing subtle semantics changes,
wouldn't it be better for older imports to fail with an ImportError?
Since this is just for toying around, this shouldn't break any play,
but it _should_ discourage (very strongly) people using experimental
stuff in their published code.

Perhaps a dunder name like __preview32__ ? __preview__._32 ?

> 1. It makes it explicitly clear when imported package version was considered
> experimental

I agree with this too. It's a good benefit.

> 2. It allows to switch between multiple experimental package versions
> 3. It will provide future compatibility if a package released from `exp`
> breaks API

Less sold on these. This isn't the purpose of __preview__ as I
understand it, and including every revision of every __preview__
package could increase Python's size very dramatically over time.

-- Devin

On Wed, Jan 4, 2012 at 5:45 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> On Wed, Jan 4, 2012 at 4:29 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> On Wed, Jan 4, 2012 at 3:29 AM, Antoine Pitrou <solipsis at pitrou.net>
>> wrote:
>> > On Tue, 03 Jan 2012 16:27:35 +0000
>> > Robert Kern <robert.kern at gmail.com> wrote:
>> >> The nice thing about the dunders is that no one will make a module with
>> >> a
>> >> conflicting name. That's really the common factor behind all uses of
>> >> dunders:
>> >> reserving a name for Python's use. Whether something is a protocol or
>> >> bag of
>> >> switches doesn't really enter into it.
>> >
>> > Plus, writing "from __preview__ import regex" looks cooler than
>> > "from exp import regex".
>>
>> Dunder names also alert people that something special is going on (in
>> this case, it helps alert them to the fact that the modules in this
>> namespace *will* be moved in the future).
>
>
> I am afraid that dunders will become a standard thing in many Python
> packages,?because of everybody's natural desire to use?latest features that
> work right. That means that new people to Python will have to deal with
> Python magic right from the start, which will make language more complicated
> for them.
>
> A correct way IMO would be to introduce a versioned exp27, exp271 packages:
> 1. It makes it explicitly clear when imported package version was considered
> experimental
> 2. It allows to switch between multiple experimental package versions
> 3. It will provide future compatibility if a package released from `exp`
> breaks API
> 5. as a sum of 2+3 above - it provides a development process and extends
> time to stabilize API with public testing and feedback before freezing the
> package to die in stdlib
>
> --
> anatoly t.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From julien at tayon.net  Wed Jan  4 12:54:15 2012
From: julien at tayon.net (julien tayon)
Date: Wed, 4 Jan 2012 12:54:15 +0100
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAOFbRmKFQ0CuNvN8Yfq4bs-8-Ls_O6NX3nqo2p0YmHopE6cUPw@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
	<CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>
	<CAFpLVkyL_zgeG5-r4ygiynx-DAkveXkMwYwfKZrh=HTZpWAiGA@mail.gmail.com>
	<CAEKRZgMZBd1gwhNYZMoNpyY1MwQPt-m3rqHONa=rHen9joMDxQ@mail.gmail.com>
	<CAGXBFArQf9tP1G96Q3ZUmKH=mjbO=4nL6VQmFG8v=sVqTkLTNQ@mail.gmail.com>
	<CAOFbRm+_1Bbo0LCsbw2V14vjD0BZK6HfhnnWBaiYQokhhtRz_A@mail.gmail.com>
	<jdvqcg$sd8$1@dough.gmane.org>
	<CAOFbRmKFQ0CuNvN8Yfq4bs-8-Ls_O6NX3nqo2p0YmHopE6cUPw@mail.gmail.com>
Message-ID: <CAFpLVkx8GYxnjVdjQ6FHy5m9J_oHD8GxLfcXZSKiWRq94KccGg@mail.gmail.com>

2012/1/3 Nathan Rice <nathan.alexander.rice at gmail.com>:
>>
>> As a said earlier in this thread, being able to view dicts as sets is such a
>> good idea that it was already added a few years ago.
>>
>>>>> {1:1}.items() < {1:1, 2:2}.items()
>> True
>
Making me think of funny stuffs  :

>>> dict( a = .1 * .1 )  > dict( a = .01 )
True
( value compared, floats are behaving as they always do)

>>> dict( a = lambda x : x  ) == dict( a = lambda x  : x )
False
( value compared, and lambda are id compared I guess)

And since I  always mixed up truth and lies :
>>> False , True = True, False ==  True and "to be" or not "to be" if  "logic" is "insane"  else 42
-- 
Jul


From jkbbwr at gmail.com  Wed Jan  4 13:03:59 2012
From: jkbbwr at gmail.com (Jakob Bowyer)
Date: Wed, 4 Jan 2012 12:03:59 +0000
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CABicbJ+JGEmN9=Cy7xAy=7qqd-8KfNkC9RN_3COSupHyZHBPTg@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
	<CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
	<CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>
	<CABicbJ+JGEmN9=Cy7xAy=7qqd-8KfNkC9RN_3COSupHyZHBPTg@mail.gmail.com>
Message-ID: <CAA+RL7GGqE5XwWzYqwprJU4JvTDkgeKdJhOi7Y2xe2FvwQKqjA@mail.gmail.com>

Surely preview should only contain whats being previewed for the next
release then when its released with the next version they carry deprecation
warnings and are removed in the following version e.g.

v1
__preview__.magic
v2
deprecate(__preview__.magic)
v3
There is no longer __preview__.magic in __preview__


On Wed, Jan 4, 2012 at 11:49 AM, Devin Jeanpierre <jeanpierreda at gmail.com>wrote:

> > A correct way IMO would be to introduce a versioned exp27, exp271
> packages:
>
> +2. Rather than silently introducing subtle semantics changes,
> wouldn't it be better for older imports to fail with an ImportError?
> Since this is just for toying around, this shouldn't break any play,
> but it _should_ discourage (very strongly) people using experimental
> stuff in their published code.
>
> Perhaps a dunder name like __preview32__ ? __preview__._32 ?
>
> > 1. It makes it explicitly clear when imported package version was
> considered
> > experimental
>
> I agree with this too. It's a good benefit.
>
> > 2. It allows to switch between multiple experimental package versions
> > 3. It will provide future compatibility if a package released from `exp`
> > breaks API
>
> Less sold on these. This isn't the purpose of __preview__ as I
> understand it, and including every revision of every __preview__
> package could increase Python's size very dramatically over time.
>
> -- Devin
>
> On Wed, Jan 4, 2012 at 5:45 AM, anatoly techtonik <techtonik at gmail.com>
> wrote:
> > On Wed, Jan 4, 2012 at 4:29 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >>
> >> On Wed, Jan 4, 2012 at 3:29 AM, Antoine Pitrou <solipsis at pitrou.net>
> >> wrote:
> >> > On Tue, 03 Jan 2012 16:27:35 +0000
> >> > Robert Kern <robert.kern at gmail.com> wrote:
> >> >> The nice thing about the dunders is that no one will make a module
> with
> >> >> a
> >> >> conflicting name. That's really the common factor behind all uses of
> >> >> dunders:
> >> >> reserving a name for Python's use. Whether something is a protocol or
> >> >> bag of
> >> >> switches doesn't really enter into it.
> >> >
> >> > Plus, writing "from __preview__ import regex" looks cooler than
> >> > "from exp import regex".
> >>
> >> Dunder names also alert people that something special is going on (in
> >> this case, it helps alert them to the fact that the modules in this
> >> namespace *will* be moved in the future).
> >
> >
> > I am afraid that dunders will become a standard thing in many Python
> > packages, because of everybody's natural desire to use latest features
> that
> > work right. That means that new people to Python will have to deal with
> > Python magic right from the start, which will make language more
> complicated
> > for them.
> >
> > A correct way IMO would be to introduce a versioned exp27, exp271
> packages:
> > 1. It makes it explicitly clear when imported package version was
> considered
> > experimental
> > 2. It allows to switch between multiple experimental package versions
> > 3. It will provide future compatibility if a package released from `exp`
> > breaks API
> > 5. as a sum of 2+3 above - it provides a development process and extends
> > time to stabilize API with public testing and feedback before freezing
> the
> > package to die in stdlib
> >
> > --
> > anatoly t.
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > http://mail.python.org/mailman/listinfo/python-ideas
> >
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120104/e69354b7/attachment.html>

From ncoghlan at gmail.com  Wed Jan  4 13:09:46 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 4 Jan 2012 22:09:46 +1000
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CABicbJ+JGEmN9=Cy7xAy=7qqd-8KfNkC9RN_3COSupHyZHBPTg@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
	<CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
	<CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>
	<CABicbJ+JGEmN9=Cy7xAy=7qqd-8KfNkC9RN_3COSupHyZHBPTg@mail.gmail.com>
Message-ID: <CADiSq7di2Y4pO7NGqAOuc=KQmJc=iB-WQHuQzO_5vsJ6c9dxtQ@mail.gmail.com>

On Wed, Jan 4, 2012 at 9:49 PM, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
>> A correct way IMO would be to introduce a versioned exp27, exp271 packages:
>
> +2. Rather than silently introducing subtle semantics changes,
> wouldn't it be better for older imports to fail with an ImportError?
> Since this is just for toying around, this shouldn't break any play,
> but it _should_ discourage (very strongly) people using experimental
> stuff in their published code.

__preview__ isn't just for toying around: so long as you don't need to
support multiple Python versions, it will be just as stable and well
supported as the rest of the standard library. That means people
deploying to controlled environments (such as the folks in corporate
and governmental environments that are intended to be the primary
beneficiaries) can use it quite happily (although keeping in mind the
potential for changes when finally upgrading in the future). That's
the entire point of the exercise.

It's unlikely that things will be in the __preview__ namespace for
more than one release - I expect that most packages that are of an
adequate standard to get added in the first place will make it through
their preview release period without significant changes. The process
just gives us one final chance to get broader feedback before we flip
the switch and lock in the API indefinitely.

For folks that have an easier time of dependency management, it's
likely a better bet for them to depend on a PyPI release of a package
rather than using the __preview__ version, but such folks don't depend
as much on the standard library in the first place.

People also shouldn't read too much into the relaxation of the
backwards compatibility guarantees - we aren't going to go nuts and
just change things on a whim. We'll just be a little less conservative
when we identify problems in the preview APIs that need to be fixed
before they're added to the main part of the standard library.

Deciding whether or not to rely on __preview__ modules, or to depend
on packages that in turn use __preview__ modules will still be a
decision developers need to make for themselves. That's no different
from any other form of due diligence that professional developers need
to conduct on their dependencies today.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Wed Jan  4 13:20:17 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 4 Jan 2012 22:20:17 +1000
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CAA+RL7GGqE5XwWzYqwprJU4JvTDkgeKdJhOi7Y2xe2FvwQKqjA@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
	<CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
	<CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>
	<CABicbJ+JGEmN9=Cy7xAy=7qqd-8KfNkC9RN_3COSupHyZHBPTg@mail.gmail.com>
	<CAA+RL7GGqE5XwWzYqwprJU4JvTDkgeKdJhOi7Y2xe2FvwQKqjA@mail.gmail.com>
Message-ID: <CADiSq7cANPi2GUQLPE-erw1DFM_yQSPRsY-HCanvDhW_QUjJ5A@mail.gmail.com>

On Wed, Jan 4, 2012 at 10:03 PM, Jakob Bowyer <jkbbwr at gmail.com> wrote:
> Surely preview should only contain whats being previewed for the next
> release then when its released with the next version they carry deprecation
> warnings and are removed in the following version e.g.
>
> v1
> __preview__.magic
> v2
> deprecate(__preview__.magic)
> v3
> There is no longer __preview__.magic in __preview__

The expected progression is:

3.X
__preview__.example
3.X+1
There is no longer a __preview__.example (What's New will say whether
to find it in the standard library or on PyPI)

I think there's a useful set of guidelines to come out of this
discussion, though:

1. All __preview__ candidates must be available as supported modules on PyPI
2. Entry into __preview__ marks the start of the transition to the
standard library. Issues may be reported both via project specific
channels and core Python channels.
3. If insurmountable problems are encountered during the preview
release (for example, buildbot instability on par with that
historically exhibited by bsddb), the project will be removed from
__preview__ and revert to standalone status
4. If no such problems are encountered, the module fully enters the
standard library in the subsequent release, with the PyPI release
remaining available for use in earlier Python versions

So multi-version code (such as other packages published on PyPI)
should generally depend on the PyPI version, but __preview__ becomes
an option if you can't use the PyPI version for some reason.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From jeanpierreda at gmail.com  Wed Jan  4 13:30:51 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 4 Jan 2012 07:30:51 -0500
Subject: [Python-ideas] working on a PEP for the __preview__ package
In-Reply-To: <CADiSq7di2Y4pO7NGqAOuc=KQmJc=iB-WQHuQzO_5vsJ6c9dxtQ@mail.gmail.com>
References: <CAF-Rda_x3Fu8-Uu+FT_1nGQ+OG=qc_eo8v=GahP=9jdFs3BHWA@mail.gmail.com>
	<CADiSq7cPubTpvgPoBd+NKPOi7BK2F3yrmL24_oL1x9ZDH5J-ig@mail.gmail.com>
	<CAB4yi1PW9E9H7FvETzOH1ofA5hs8h7vFsUkEcaQQDjqt3kYY-Q@mail.gmail.com>
	<jdva9o$6fa$1@dough.gmane.org> <20120103182942.1552481e@pitrou.net>
	<CADiSq7eht9us45cg552D-uvYdKVjsQB8o9eww_3pfxzjNPaoYA@mail.gmail.com>
	<CAPkN8x+ugTGRTvEqc_WaZ0Q-VFC1ZQD8aC_33GXqHK=gLFr_OQ@mail.gmail.com>
	<CABicbJ+JGEmN9=Cy7xAy=7qqd-8KfNkC9RN_3COSupHyZHBPTg@mail.gmail.com>
	<CADiSq7di2Y4pO7NGqAOuc=KQmJc=iB-WQHuQzO_5vsJ6c9dxtQ@mail.gmail.com>
Message-ID: <CABicbJJuCQKJF5OCePxi=6+CE3c6BA3HUyQA_Q7Wz7JGAVqQVw@mail.gmail.com>

> It's unlikely that things will be in the __preview__ namespace for
> more than one release - I expect that most packages that are of an
> adequate standard to get added in the first place will make it through
> their preview release period without significant changes. The process
> just gives us one final chance to get broader feedback before we flip
> the switch and lock in the API indefinitely.

Ah, this would remove much of the point of explicit version numbering
wouldn't it?

Your other paragraphs make other good points against it too. It was a
good response.

I am no longer +2 on explicit versioning. I still find it nice for
"experimental" code to be a bit clear about which experiment you're
referring to, but it isn't very important if it doesn't last long and
doesn't go through that many breaking changes. In fact, the way the
use case is shaping up, it isn't important at all. I suspect I
misremembered or only selectively remembered the original __preview__
proposal.

I suppose this is a good reason to get a PEP out, huh.

-- Devin

On Wed, Jan 4, 2012 at 7:09 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Jan 4, 2012 at 9:49 PM, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
>>> A correct way IMO would be to introduce a versioned exp27, exp271 packages:
>>
>> +2. Rather than silently introducing subtle semantics changes,
>> wouldn't it be better for older imports to fail with an ImportError?
>> Since this is just for toying around, this shouldn't break any play,
>> but it _should_ discourage (very strongly) people using experimental
>> stuff in their published code.
>
> __preview__ isn't just for toying around: so long as you don't need to
> support multiple Python versions, it will be just as stable and well
> supported as the rest of the standard library. That means people
> deploying to controlled environments (such as the folks in corporate
> and governmental environments that are intended to be the primary
> beneficiaries) can use it quite happily (although keeping in mind the
> potential for changes when finally upgrading in the future). That's
> the entire point of the exercise.
>
> It's unlikely that things will be in the __preview__ namespace for
> more than one release - I expect that most packages that are of an
> adequate standard to get added in the first place will make it through
> their preview release period without significant changes. The process
> just gives us one final chance to get broader feedback before we flip
> the switch and lock in the API indefinitely.
>
> For folks that have an easier time of dependency management, it's
> likely a better bet for them to depend on a PyPI release of a package
> rather than using the __preview__ version, but such folks don't depend
> as much on the standard library in the first place.
>
> People also shouldn't read too much into the relaxation of the
> backwards compatibility guarantees - we aren't going to go nuts and
> just change things on a whim. We'll just be a little less conservative
> when we identify problems in the preview APIs that need to be fixed
> before they're added to the main part of the standard library.
>
> Deciding whether or not to rely on __preview__ modules, or to depend
> on packages that in turn use __preview__ modules will still be a
> decision developers need to make for themselves. That's no different
> from any other form of due diligence that professional developers need
> to conduct on their dependencies today.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From jimjjewett at gmail.com  Wed Jan  4 15:41:19 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 4 Jan 2012 09:41:19 -0500
Subject: [Python-ideas] Proposed PEP on concurrent programming support
Message-ID: <CA+OGgf6_xSQkzV8Tz4C3FvkDV+Sn4zX_Vy7Z2OLGm2nOqeCqAg@mail.gmail.com>

(I've added back python-ideas, because I think that is still the
appropriate forum.)

>.... A new
> suite type - the ``transaction`` will be added to the language. The
> suite will have the semantics discussed above: modifying an object in
> the suite will trigger creation of a thread-local shallow copy to be
> used in the Transaction. Further modifications of the original will
> cause all existing copies to be discarded and the transaction to be
> restarted. ...

How will you know that an object has been modified?

The only ways I can think of are

(1)  Timestamp every object -- or at least every mutable object -- and
hope that everybody agrees on which modifications should count.

(2)  Make two copies of every object you're using in the suite; at the
end, compare one of them to both the original and the one you were
operating on.  With this solution, you can decide for youself what
counts as a modification, but it still isn't straightforward; I would
consider changing a value to be changing a dict, even though
nothing in the item (header) itself changed.

-jJ


From barry at python.org  Wed Jan  4 16:18:12 2012
From: barry at python.org (Barry Warsaw)
Date: Wed, 4 Jan 2012 10:18:12 -0500
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
In-Reply-To: <87mxa441zw.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
	<jddenk$gr1$1@dough.gmane.org>
	<14270238.69.1325067063613.JavaMail.geo-discussion-forums@yqip20>
	<20120103162258.0978cd1e@resist.wooz.org>
	<87mxa441zw.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20120104101812.64d32e8e@limelight.wooz.org>

On Jan 04, 2012, at 11:48 AM, Stephen J. Turnbull wrote:

>Barry Warsaw writes:
> > On Dec 28, 2011, at 02:11 AM, anatoly techtonik wrote:
> > 
> > >Does Mailman support confirmation for messages with high SPAM ratio?
> > 
> > Not out of the box, no.
>
>But it would be trivial to implement efficiently, I think, even in
>MM2, as long as there is a front-end (eg, spamassassin) that does the
>spam-checking efficiently.

Right.  If the front-end added some header containing a pattern that could be
parsed for spamminess, it would be nearly trivial to write a handler that
could make that determination and hold/discard/reject such a message.  Skip
(IIRC) and I once had a moderately-well working spambayes plugin for Mailman.

>I don't know if Python lists ever have that kind of problem, though.

I don't think we have much of a spam problem on the lists these days.  It used
to be python-list got most of its spam through the Usenet gateway.  Probably
in another year or so all of the terms "Usenet", "spam", and "email" will
produce a reaction in our young intarweb users much like the terms "vinyl
records" and "land-line" do today.  There's no problem that doesn't disappear
if you ignore it long enough <wink>.

> > There could be some integration points with Mailman here, but it's
> > not clear they are worth it,
>
>For mailman-developers, I think it's preferable to teach people to
>fish (here, write and install simple Handlers), and provide a contrib
>page on the wiki for posting them.

Like all open source, contributions are welcome. :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120104/eeb50d28/attachment.pgp>

From tjreedy at udel.edu  Wed Jan  4 19:11:21 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 04 Jan 2012 13:11:21 -0500
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
In-Reply-To: <20120104101812.64d32e8e@limelight.wooz.org>
References: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
	<jddenk$gr1$1@dough.gmane.org>
	<14270238.69.1325067063613.JavaMail.geo-discussion-forums@yqip20>
	<20120103162258.0978cd1e@resist.wooz.org>
	<87mxa441zw.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120104101812.64d32e8e@limelight.wooz.org>
Message-ID: <je24od$utc$1@dough.gmane.org>

On 1/4/2012 10:18 AM, Barry Warsaw wrote:

>> I don't know if Python lists ever have that kind of problem, though.
>
> I don't think we have much of a spam problem on the lists these days.  It used
> to be python-list got most of its spam through the Usenet gateway.

It still gets a few spam messages a day.

-- 
Terry Jan Reedy



From t.landschoff at gmx.net  Wed Jan  4 23:32:04 2012
From: t.landschoff at gmx.net (Torsten Landschoff)
Date: Wed, 04 Jan 2012 23:32:04 +0100
Subject: [Python-ideas] Sampling Profiler for Python
Message-ID: <4F04D364.8030404@gmx.net>

Hello world,

since I moved from Java development to Python development, I really miss 
the sampling profiler of jvisualvm (http://visualvm.java.net/). Perhaps 
it is only me but especially for long running GUI applications, I would 
really like to know why it sometimes slows down to a crawl.

cProfile is not really useful for this as it makes the software 
unresponsive alone. Also if using multiple threads (like doing some I/O 
in background, when using http://www.lag.net/paramiko/) it will give 
meaningless results.

I looked for an existing profiler and found statprof, mentioned in this 
blog: http://wingolog.org/archives/2005/10/28/profiling

However, this still fails profiling multiple threads. I don't think one 
can do any better without touching the interpreter core. So that's what 
I tried...

My approach implements the profiler from the following parts:

   1. Sample Trigger: A timer that triggers taking samples of all
      threads at fixed intervals.
   2. Sampling function injection: Ensures that a sampling function is
      called on each thread at the first opportunity after the sample
      trigger fired.
   3. Sampling function: A function that assigns the current time slice
      to the executing code on the thread it is called on.

Only the second part has to be implemented in the Python interpreter 
core, the sample trigger and sampling functions can be provided by 
external modules. My implementation of (2) can be found at 
https://bitbucket.org/bluehorn/sampling_prof

I also prepared a first implementation of (1) and (3) at 
https://bitbucket.org/bluehorn/profiler_util. Using that library, I can 
profile pybench with a 3 % slowdown (taking 100 profiling samples per 
second):

$ export PYTHON_PATH=$HOME/python-work/profiler_util
$ python3 -m sampling_profiler -- Tools/pybench/pybench.py -c 
stock.pybench -f prof.pybench
[...]
Totals:                          3119ms  3072ms   +1.5%  3198ms  
3108ms   +2.9%

This also shows that the string benchmark takes the most time:

Thread MainThread (3842 samples, 38.434401 seconds)
------------------------------------------------------------------------
cpu_time    (cum)  ticks  (cum) samples (cum) 
filename:lineno                          function
    0.548    0.548     55     55     55     55 
Tools/pybench/Strings.py:455             test
    0.448    0.448     45     45     45     45 
Tools/pybench/Strings.py:568             calibrate
    0.428    0.428     43     43     43     43 
Tools/pybench/Constructs.py:484          test
    0.380    0.380     38     38     38     38 
Tools/pybench/Calls.py:445               f
    0.324    0.324     34     34     34     34 
Tools/pybench/Calls.py:122               f

The output is explained as follows:

cpu_time: cpu time in seconds accounted to that line.
(cum): This is the cpu time including the cumulative time of the 
functions called on that line. Gathering that data means that the 
sampling function has to walk the stack for each sample, which will 
cause a dramatic slowdown for deeply recursive calls. There should be a 
switch to disable this.
ticks: Number of sampling profiler ticks accounted to that line. For my 
example (where sample ticks are equidistant in wall time) this is a 
measure of the time spent in that line, including any blocking calls 
(especially I/O).
(cum): ... including called functions
samples: Number of times the sampling profiler function was called for 
that line. This should equal ticks unless execution of that line took 
longer than a profiler tick.

The remaining output should be obvious.

I have to admit that this is far from finished, but I hope to get some 
feedback and perhaps even help by posting it here.

What do you think?

Greetings, Torsten

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120104/664a8811/attachment.html>

From sumerc at gmail.com  Thu Jan  5 07:45:29 2012
From: sumerc at gmail.com (=?ISO-8859-1?Q?S=FCmer_Cip?=)
Date: Thu, 5 Jan 2012 08:45:29 +0200
Subject: [Python-ideas] Sampling Profiler for Python
In-Reply-To: <4F04D364.8030404@gmx.net>
References: <4F04D364.8030404@gmx.net>
Message-ID: <CAOkQLaBsGsXDrKm9eO9eJ6j_L12ne6zVpbLfxpwjthf=j_WLwQ@mail.gmail.com>

Have you looked at yappi? yappi.start() accepts a sampling parameter.

Regards,

On Thu, Jan 5, 2012 at 12:32 AM, Torsten Landschoff <t.landschoff at gmx.net>wrote:

> **
> Hello world,
>
> since I moved from Java development to Python development, I really miss
> the sampling profiler of jvisualvm (http://visualvm.java.net/). Perhaps
> it is only me but especially for long running GUI applications, I would
> really like to know why it sometimes slows down to a crawl.
>
> cProfile is not really useful for this as it makes the software
> unresponsive alone. Also if using multiple threads (like doing some I/O in
> background, when using http://www.lag.net/paramiko/) it will give
> meaningless results.
>
> I looked for an existing profiler and found statprof, mentioned in this
> blog: http://wingolog.org/archives/2005/10/28/profiling
>
> However, this still fails profiling multiple threads. I don't think one
> can do any better without touching the interpreter core. So that's what I
> tried...
>
> My approach implements the profiler from the following parts:
>
>    1. Sample Trigger: A timer that triggers taking samples of all threads
>    at fixed intervals.
>    2. Sampling function injection: Ensures that a sampling function is
>    called on each thread at the first opportunity after the sample trigger
>    fired.
>    3. Sampling function: A function that assigns the current time slice
>    to the executing code on the thread it is called on.
>
> Only the second part has to be implemented in the Python interpreter core,
> the sample trigger and sampling functions can be provided by external
> modules. My implementation of (2) can be found at
> https://bitbucket.org/bluehorn/sampling_prof
>
> I also prepared a first implementation of (1) and (3) at
> https://bitbucket.org/bluehorn/profiler_util. Using that library, I can
> profile pybench with a 3 % slowdown (taking 100 profiling samples per
> second):
>
> $ export PYTHON_PATH=$HOME/python-work/profiler_util
> $ python3 -m sampling_profiler -- Tools/pybench/pybench.py -c
> stock.pybench -f prof.pybench
> [...]
> Totals:                          3119ms  3072ms   +1.5%  3198ms  3108ms
> +2.9%
>
> This also shows that the string benchmark takes the most time:
>
> Thread MainThread (3842 samples, 38.434401 seconds)
> ------------------------------------------------------------------------
> cpu_time    (cum)  ticks  (cum) samples (cum)
> filename:lineno                          function
>    0.548    0.548     55     55     55     55
> Tools/pybench/Strings.py:455             test
>    0.448    0.448     45     45     45     45
> Tools/pybench/Strings.py:568             calibrate
>    0.428    0.428     43     43     43     43
> Tools/pybench/Constructs.py:484          test
>    0.380    0.380     38     38     38     38
> Tools/pybench/Calls.py:445               f
>    0.324    0.324     34     34     34     34
> Tools/pybench/Calls.py:122               f
>
> The output is explained as follows:
>
> cpu_time: cpu time in seconds accounted to that line.
> (cum): This is the cpu time including the cumulative time of the functions
> called on that line. Gathering that data means that the sampling function
> has to walk the stack for each sample, which will cause a dramatic slowdown
> for deeply recursive calls. There should be a switch to disable this.
> ticks: Number of sampling profiler ticks accounted to that line. For my
> example (where sample ticks are equidistant in wall time) this is a measure
> of the time spent in that line, including any blocking calls (especially
> I/O).
> (cum): ... including called functions
> samples: Number of times the sampling profiler function was called for
> that line. This should equal ticks unless execution of that line took
> longer than a profiler tick.
>
> The remaining output should be obvious.
>
> I have to admit that this is far from finished, but I hope to get some
> feedback and perhaps even help by posting it here.
>
> What do you think?
>
> Greetings, Torsten
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


-- 
--
Sumer Cip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120105/5a1d0759/attachment.html>

From t.landschoff at gmx.net  Sun Jan  8 02:01:04 2012
From: t.landschoff at gmx.net (Torsten Landschoff)
Date: Sun, 08 Jan 2012 02:01:04 +0100
Subject: [Python-ideas] Sampling Profiler for Python
In-Reply-To: <CAOkQLaBsGsXDrKm9eO9eJ6j_L12ne6zVpbLfxpwjthf=j_WLwQ@mail.gmail.com>
References: <4F04D364.8030404@gmx.net>
	<CAOkQLaBsGsXDrKm9eO9eJ6j_L12ne6zVpbLfxpwjthf=j_WLwQ@mail.gmail.com>
Message-ID: <4F08EAD0.20802@gmx.net>

Hi S?mer,

On 01/05/2012 07:45 AM, S?mer Cip wrote:
> Have you looked at yappi? yappi.start() accepts a sampling parameter.
no, I haven't because I googled for sampling profiler or statistical 
profiler. I compared running pybench with yappi vs. my sampling profiler 
(with 100 samples/sec).

First, the run time increases in both cases vs. the original performance 
of python. The figures are different because I backported to Python2 
again for the comparison (is yappi available for Python3?)

yappi:
Totals:                          4692ms  3317ms  +41.5%  4831ms  3448ms  
+40.1%

sampling_profiler:
Totals:                          3491ms  3269ms   +6.8%  3638ms  
3435ms   +5.9%

These are the hot spots as reported by yappi:
name                                 #n       tsub       ttot       tavg
Tools/pybench/Calls.py.test:439      10       1.709853   1.709853   0.170985
Tools/pybench/Strings.py.test:467    10       1.383400   2.133651   0.213365
Tools/pybench/With.py.test:16        10       1.179130   1.614330   0.161433
Tools/pybench/Calls.py.test:176      10       1.088372   1.885379   0.188538
Tools/pybench/Unicode.py.test:388    10       1.062785   1.729250   0.172925
Tools/pybench/Calls.py.test:118      10       1.008680   1.284282   0.128428
Tools/pybench/With.py.test:64        10       0.999503   1.431072   0.143107
Tools/pybench/Lists.py.test:9        10       0.995866   1.394482   0.139448
Tools/pybench/Calls.py.test:9        10       0.979050   1.505686   0.150569
Tools/pybench/Dict.py.test:351       10       0.959278   1.290247   0.129025

Compared with the sampling_profiler:
cpu_time    (cum)  ticks  (cum) samples (cum) 
filename:lineno                          function
    0.552    4.224     55    419     55    419 
Tools/pybench/Calls.py:446               f
    0.512    0.512     52     52     52     52 
Tools/pybench/Unicode.py:452             calibrate
    0.436    0.436     44     44     44     44 
Tools/pybench/Unicode.py:542             calibrate
    0.436    0.436     44     44     44     44 
Tools/pybench/Strings.py:562             calibrate
    0.372    0.372     37     37     37     37 
Tools/pybench/Constructs.py:485          test
    0.348    0.348     36     36     36     36 
Tools/pybench/Calls.py:122               f
    0.272    0.272     27     27     27     27 
Tools/pybench/Lists.py:339               test
    0.248    0.248     25     25     25     25 
Tools/pybench/Calls.py:445               f
    0.228    0.228     22     22     22     22 
Tools/pybench/Strings.py:247             calibrate
    0.212    0.212     21     21     21     21 
Tools/pybench/With.py:14                 __exit__

Let's look at leading hot-spots:
1) yappi reports Calls.py line 439 as the hot spot. This refers to this 
code:

     def test(self):                 # <-- line 439
         global f
         def f(x):
             if x > 1:
                 return f(x-1)       # <-- line 446
             return 1
         for i in xrange(self.rounds):
             f(10)
             f(10)
             ...

Here yappi identified the outer function, the sampling profiler points 
directly to the line where most of the time is spent.

2) Most other hot spots reported by sampling_profiler lie in the 
calibrate functions. I wondered by that might be and noticed that 
pybench calls these 20 times per default. As the sampling_profiler 
reports the hot spots based on lines vs. the functions in yappi, this 
means we can not really compare these results.


So let's compare the behaviour of the different profilers on this code:

from time import time

def fib(n):
     if n > 1:
         return fib(n-1) + fib(n-2)
     else:
         return n

def fib2(n):
     stack = [n]
     while stack:
         n = stack.pop()
         if n > 1:
             stack.append(n-1)
             stack.append(n-2)

s = time()
fib(34)
print "fib: %s seconds" % (time() - s)
s = time()
fib2(34)
print "fib2: %s seconds" % (time() - s)

Here are two variants of the fibonnaci function. The first works using 
recursion and returns the correct result. The second is just for 
simulating the same behaviour without recursion by using a local stack. 
The result is not computed since it is not required here - I just want 
to compare how this behaves wrt. our profilers.

When I run this without any profiler, I get

fib: 2.87187790871 seconds
fib2: 5.26651716232 seconds

With yappi, I get this:
fib: 6.42756795883 seconds
fib2: 14.6631779671 seconds

name                                 #n       tsub       ttot       tavg
demo.py.<module>:1                   1        6.427689   21.090864  
21.090864
demo.py.fib2:9                       1        10.353305  14.663173  
14.663173
..atprof/Lib/posixpath.py.dirname:11 1        0.000004   0.000005   
0.000005
..prof/Lib/threading.py.setprofile:8 1        0.000001   0.000001   
0.000001
..n2.6/site-packages/yappi.py.stop:4 1        0.000000   0.000000   
0.000000
demo.py.fib:3                        184549.. 0.000000   0.000000   
0.000000

The slow down for fib is about what I expected, but I have no idea why 
fib2 is slowed down that much. AFAIK, the usual profiling hooks of 
Python are called on entry and exit of each function, not on each line?!

The results are incorrect for the fib function for reasons unknown to me 
- yappi thinks that no time is spent in that function, while the program 
output makes it obvious that more than 6 seconds are spent there.

I tried using the timing_sample parameter to yappi.start. When set to 2, 
all of tsub, ttot and tavg are 0 in the output for this example. I can 
speed up the run to 4.97 seconds for fib and 9.33 s for fib2 using a 
high timing_sample value of 200 or more. That is still quite some 
overhead for keeping it active at all times.

With the sampling profiler, the output is
fib: 3.10812282562 seconds
fib2: 5.91388201714 seconds
Thread MainThread (902 samples, 9.036564 seconds)
------------------------------------------------------------------------
cpu_time    (cum)  ticks  (cum) samples (cum) 
filename:lineno                          function
    0.000    9.005      0    902      0    902 
ython-work/statprof/Lib/runpy.py:122     _run_module_as_main
    0.000    9.005      0    902      0    902 
tatprof/Lib/sampling_profiler.py:203     main
    0.000    9.005      0    902      0    902 
tatprof/Lib/sampling_profiler.py:209 <module>
    0.000    9.005      0    902      0    902 
ython-work/statprof/Lib/runpy.py:34      _run_code
    0.000    5.912      0    592      0    592 demo.py:21 <module>
    2.068    3.092    211    310    211    310 
demo.py:5                                fib
    0.000    3.092      0    310      0    310 demo.py:18 <module>
    2.196    2.196    217    217    217    217 
demo.py:12                               fib2
    1.576    1.576    157    157    157    157 
demo.py:15                               fib2
    0.980    0.980    101    101    101    101 
demo.py:14                               fib2
    0.900    0.900     87     87     87     87 
demo.py:4                                fib
    0.644    0.644     67     67     67     67 
demo.py:11                               fib2
    0.516    0.516     50     50     50     50 
demo.py:13                               fib2
    0.124    0.124     12     12     12     12 
demo.py:7                                fib

Just a small slow down here. And as you can see, this even tells you on 
what lines the most cpu time is spent. While I look at this, I am 
missing per-function totals, but that should be easy to add.
As you can see, the profiler correctly detects that line 21 and line 18 
of the demo code take most of the time (which are the module-level calls 
to fib2 and fib).

For completeness, let's look what cProfile finds out:
torsten at pulsar:~/python-work/statprof$ ./python run_cprofile.py demo.py
fib: 5.83289813995 seconds
fib2: 11.2586638927 seconds
          55364794 function calls (36909866 primitive calls) in 17.092 
CPU seconds

    Ordered by: standard name

    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
         1    0.000    0.000   17.092   17.092 <string>:1(<module>)
         1    0.000    0.000   17.092   17.092 demo.py:1(<module>)
18454929/1    5.833    0.000    5.833    5.833 demo.py:3(fib)
         1    7.831    7.831   11.259   11.259 demo.py:9(fib2)
  18454928    1.421    0.000    1.421    0.000 {method 'append' of 
'list' objects}
         1    0.000    0.000    0.000    0.000 {method 'disable' of 
'_lsprof.Profiler' objects}
  18454929    2.007    0.000    2.007    0.000 {method 'pop' of 'list' 
objects}
         4    0.000    0.000    0.000    0.000 {time.time}

I am impressed how cProfile detects the recursive calls of the fib 
function (I guess that's the info in the ncalls column). Anyway, the 
slow down here is much more noticeable for the same information.


So, why would I prefer the sampling profiler?
a) Less slow down (could still use some optimization)
b) Points directly to the code lines where the hot spots are (based on 
equidistant time samples).
c) Does not distort run time by being active.

Greetings, Torsten



From mwm at mired.org  Sun Jan  8 02:07:13 2012
From: mwm at mired.org (Mike Meyer)
Date: Sat, 7 Jan 2012 17:07:13 -0800
Subject: [Python-ideas] Boolean value of file object?
Message-ID: <20120107170713.3f3f1b2d@bhuda.mired.org>

Just an off-the-wall thought.

Is there any reason a file object's boolean value shouldn't be false
once it's been closed? This would allow replacing "if f and f.closed:" (to
make sure you have a file and it's open) with just "if f:". Is there
some use case where you want to verify that an object is a file
object instead of None (or another false value), and don't care if
it's closed?

My own thought is that this is a case where status quo wins, but
thought someone else might think the idea has more merit.

	<mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From solipsis at pitrou.net  Sun Jan  8 02:20:32 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 8 Jan 2012 02:20:32 +0100
Subject: [Python-ideas] Boolean value of file object?
References: <20120107170713.3f3f1b2d@bhuda.mired.org>
Message-ID: <20120108022032.2ca233f5@pitrou.net>

On Sat, 7 Jan 2012 17:07:13 -0800
Mike Meyer <mwm at mired.org> wrote:
> Just an off-the-wall thought.
> 
> Is there any reason a file object's boolean value shouldn't be false
> once it's been closed? This would allow replacing "if f and f.closed:" (to
> make sure you have a file and it's open) with just "if f:". Is there
> some use case where you want to verify that an object is a file
> object instead of None (or another false value), and don't care if
> it's closed?

That sounds just too smart. It is not obvious that a file should become
"false" once it is closed. Writing "f.closed" OTOH is explicit and
obvious.

Regards

Antoine.




From ncoghlan at gmail.com  Sun Jan  8 02:18:30 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 8 Jan 2012 11:18:30 +1000
Subject: [Python-ideas] Boolean value of file object?
In-Reply-To: <20120107170713.3f3f1b2d@bhuda.mired.org>
References: <20120107170713.3f3f1b2d@bhuda.mired.org>
Message-ID: <CADiSq7dgqVNttbqe193W20pFvKedJDGWAXHmDcXPxJwOdYRw1A@mail.gmail.com>

On Sun, Jan 8, 2012 at 11:07 AM, Mike Meyer <mwm at mired.org> wrote:
> My own thought is that this is a case where status quo wins, but
> thought someone else might think the idea has more merit.

Unfortunately, there are way too many "file-like objects" in the wild
for it to ever be feasible to rely on such a shorthand. Cute idea,
though.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From steve at pearwood.info  Sun Jan  8 03:06:51 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 08 Jan 2012 13:06:51 +1100
Subject: [Python-ideas] Boolean value of file object?
In-Reply-To: <20120107170713.3f3f1b2d@bhuda.mired.org>
References: <20120107170713.3f3f1b2d@bhuda.mired.org>
Message-ID: <4F08FA3B.40204@pearwood.info>

Mike Meyer wrote:
> Just an off-the-wall thought.
> 
> Is there any reason a file object's boolean value shouldn't be false
> once it's been closed?

Is there any reason it should?

There's nothing false-like about a closed file object. Truthy and falsy 
objects should map to "something" vs "nothing" or "non-empty" vs "empty", not 
"some arbitrary flag that might be useful sometimes".


-- 
Steven



From raymond.hettinger at gmail.com  Sun Jan  8 03:08:30 2012
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sat, 7 Jan 2012 18:08:30 -0800
Subject: [Python-ideas] Boolean value of file object?
In-Reply-To: <20120107170713.3f3f1b2d@bhuda.mired.org>
References: <20120107170713.3f3f1b2d@bhuda.mired.org>
Message-ID: <9904C12C-7DD7-4C54-B1C3-14BBDCF87B6F@gmail.com>


On Jan 7, 2012, at 5:07 PM, Mike Meyer wrote:

> Is there any reason a file object's boolean value shouldn't be false
> once it's been closed? 

Because that would make it harder to learn what things in Python can be False and what that implies.

Currently, None is always false, numbers are false when they are zero, and containers are false when they are empty.

Files objects don't fit into that model.  Some would question whether a file could be considered a container.  Even if a file was considered a container, there is still an important distinction between files that are closed versus files that are empty (i.e. they have a length of zero).

Lastly, the file API is adopted by many objects, so we would need to change them all.



Raymond

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120107/085d162f/attachment.html>

From benjamin at python.org  Sun Jan  8 17:17:30 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 8 Jan 2012 16:17:30 +0000 (UTC)
Subject: [Python-ideas] adding a casefold() method to str
Message-ID: <loom.20120108T170852-634@post.gmane.org>

Hi,
Casefolding (Unicode Standard 3.13) is a more aggressive version of lowercasing.
It's purpose to assist in the implementation of caseless mapping. For example,
under lowercase "?" -> "?" but under casefolding "?" -> "ss". I propose we add a
casefold() method. So, case-insensitive matching should really be
"one.casefold() == two.casefold()"
rather than "one.lower() == two.lower()".

Regards,
Benjamin



From steve at pearwood.info  Sun Jan  8 17:58:17 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 09 Jan 2012 03:58:17 +1100
Subject: [Python-ideas] adding a casefold() method to str
In-Reply-To: <loom.20120108T170852-634@post.gmane.org>
References: <loom.20120108T170852-634@post.gmane.org>
Message-ID: <4F09CB29.1000106@pearwood.info>

Benjamin Peterson wrote:
> Hi,
> Casefolding (Unicode Standard 3.13) is a more aggressive version of lowercasing.
> It's purpose to assist in the implementation of caseless mapping. For example,
> under lowercase "?" -> "?" but under casefolding "?" -> "ss". I propose we add a
> casefold() method. So, case-insensitive matching should really be
> "one.casefold() == two.casefold()"
> rather than "one.lower() == two.lower()".

+1 in principle, but in practice case folding is more complicated than a 
single method might imply. The most obvious complication is treatment of 
dotted and dotless I.

See, for example:

http://unicode.org/Public/UNIDATA/CaseFolding.txt
http://www.w3.org/International/wiki/Case_folding
http://en.wikipedia.org/wiki/Letter_case#Unicode_case_folding_and_script_identification

So while having proper Unicode case-folding is desirable, I don't know how 
simple it is to implement.

Would it be appropriate for casefold() to take an optional argument as to 
which mappings to use? E.g. something like:

str.casefold()  # defaults to simple folding
str.casefold(string.SIMPLE & string.TURKIC)
str.casefold(string.FULL)

or should str.casefold() only apply simple folding, with the others 
combinations relegated to a function in a module somewhere?

I count 4 possible functions:

simple casefolding, without Turkic I
full casefolding, without Turkic I
simple casefolding, with Turkic I
full casefolding, with Turkic I




-- 
Steven


From benjamin at python.org  Sun Jan  8 18:47:24 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 8 Jan 2012 17:47:24 +0000 (UTC)
Subject: [Python-ideas] adding a casefold() method to str
References: <loom.20120108T170852-634@post.gmane.org>
	<4F09CB29.1000106@pearwood.info>
Message-ID: <loom.20120108T184555-569@post.gmane.org>

Steven D'Aprano <steve at ...> writes:
> or should str.casefold() only apply simple folding, with the others 
> combinations relegated to a function in a module somewhere?

Yes, I think so. str does not have any other features dependent on locale.
Section 3.3 defines "Default casefolding" which is what the casefold() method
should use.






From simon.sapin at kozea.fr  Sun Jan  8 22:58:05 2012
From: simon.sapin at kozea.fr (Simon Sapin)
Date: Sun, 08 Jan 2012 22:58:05 +0100
Subject: [Python-ideas] A key parameter for heapq.merge
Message-ID: <4F0A116D.3030202@kozea.fr>

Hi,

According to its own documentation, the merge() function in the heapq 
module is similar to sorted(). However, it has none of the key and 
reverse parameters that sorted() has. I think they could be just as 
useful as in sorted().

http://docs.python.org/dev/library/heapq.html#heapq.merge
http://docs.python.org/dev/library/functions.html#sorted


First of all what do you think of the idea?

I?m working on a patch for a key parameter. I think it can be pretty 
straightforward, but I?ll measure if the "no key" case becomes slower 
(calls to lambda x: x) At worst we can always duplicate the loop.

However, I am not sure how to implement reverse. Not all values have an 
"opposite value" that reverses order, _nsmallest and _nlargest are quite 
different, and merge uses neither.


Anyway, if I get this to work, it will be my first contribution to 
CPython. I?m trying to follow the Developer?s guide, but is there 
something else I should be aware of?

http://docs.python.org/devguide/

Regards,
-- 
Simon Sapin


From steve at pearwood.info  Mon Jan  9 00:46:48 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 09 Jan 2012 10:46:48 +1100
Subject: [Python-ideas] A key parameter for heapq.merge
In-Reply-To: <4F0A116D.3030202@kozea.fr>
References: <4F0A116D.3030202@kozea.fr>
Message-ID: <4F0A2AE8.6090608@pearwood.info>

Simon Sapin wrote:

> I?m working on a patch for a key parameter. I think it can be pretty 
> straightforward, but I?ll measure if the "no key" case becomes slower 
> (calls to lambda x: x) At worst we can always duplicate the loop.

In my experience, it is *much* faster to test against None every time through 
the loop than to call a do-nothing function. Instead of this:

if key is None:
     key = lambda x: x
for value in heap:
     do_something_with(key(value))

this is much faster:

for value in heap:
     if key is None:
         do_something_with(value)
     else:
         do_something_with(key(value))

and this is faster still:

if key is None:
     for value in heap:
         do_something_with(value)
else:
     for value in heap:
         do_something_with(key(value))

YMMV; I encourage you to benchmark.




-- 
Steven



From simon.sapin at kozea.fr  Mon Jan  9 08:17:24 2012
From: simon.sapin at kozea.fr (Simon Sapin)
Date: Mon, 09 Jan 2012 08:17:24 +0100
Subject: [Python-ideas] A key parameter for heapq.merge
In-Reply-To: <4F0A2AE8.6090608@pearwood.info>
References: <4F0A116D.3030202@kozea.fr> <4F0A2AE8.6090608@pearwood.info>
Message-ID: <4F0A9484.7050903@kozea.fr>

Le 09/01/2012 00:46, Steven D'Aprano a ?crit :
> and this is faster still:
>
> if key is None:
>       for value in heap:
>           do_something_with(value)
> else:
>       for value in heap:
>           do_something_with(key(value))

Yes, this is what I meant by duplicating the loop, though 
do_something_with() is a bit longer:

http://hg.python.org/cpython/file/56e9d025078d/Lib/heapq.py#l336

 > YMMV; I encourage you to benchmark.

I have a working patch with tests. I?ll do a benchmarks next. After that 
is filing an issue?

Thanks,
-- 
Simon Sapin


From raymond.hettinger at gmail.com  Mon Jan  9 08:36:41 2012
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 9 Jan 2012 07:36:41 +0000
Subject: [Python-ideas] A key parameter for heapq.merge
In-Reply-To: <4F0A116D.3030202@kozea.fr>
References: <4F0A116D.3030202@kozea.fr>
Message-ID: <73792DF0-F128-437E-8AC8-A9F34D042FF4@gmail.com>


On Jan 8, 2012, at 9:58 PM, Simon Sapin wrote:

> According to its own documentation, the merge() function in the heapq module is similar to sorted(). However, it has none of the key and reverse parameters that sorted() has. I think they could be just as useful as in sorted().

A key-argument could be useful but a reverse parameter doesn't make as much sense.

If you want to make a patch, add it to the tracker and assign it to me for review.


Raymond



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120109/cd43fa4d/attachment.html>

From techtonik at gmail.com  Mon Jan  9 09:35:17 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 9 Jan 2012 11:35:17 +0300
Subject: [Python-ideas] Boolean value of file object?
In-Reply-To: <9904C12C-7DD7-4C54-B1C3-14BBDCF87B6F@gmail.com>
References: <20120107170713.3f3f1b2d@bhuda.mired.org>
	<9904C12C-7DD7-4C54-B1C3-14BBDCF87B6F@gmail.com>
Message-ID: <CAPkN8x+_9SyEKC2yrHo4wDeXWR8K-PBD92T8Ys1jxiJop0FhiQ@mail.gmail.com>

On Sun, Jan 8, 2012 at 5:08 AM, Raymond Hettinger <
raymond.hettinger at gmail.com> wrote:

>
> there is still an important distinction between files that are closed
> versus files that are empty (i.e. they have a length of zero).
>

and `file`s that are None. When you check f.closed explicitly - the None
objects throw an exception.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120109/30275d6c/attachment.html>

From techtonik at gmail.com  Mon Jan  9 11:30:55 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 9 Jan 2012 13:30:55 +0300
Subject: [Python-ideas] Pythonic buffering in Py3 print()
Message-ID: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>

In Python 2 "print 'something', statement calls were unbuffered and
immediately appeared on screen.
In Python 3 "print('something', end='')" calls are buffered. This breaks
progress bars and other stuff.

1. What is more "Pythonic" and why?
2. Should Python 3 be fixed ASAP?
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120109/b0f43294/attachment.html>

From simon.sapin at kozea.fr  Mon Jan  9 11:46:46 2012
From: simon.sapin at kozea.fr (Simon Sapin)
Date: Mon, 09 Jan 2012 11:46:46 +0100
Subject: [Python-ideas] A key parameter for heapq.merge
In-Reply-To: <73792DF0-F128-437E-8AC8-A9F34D042FF4@gmail.com>
References: <4F0A116D.3030202@kozea.fr>
	<73792DF0-F128-437E-8AC8-A9F34D042FF4@gmail.com>
Message-ID: <4F0AC596.60906@kozea.fr>

Le 09/01/2012 08:36, Raymond Hettinger a ?crit :
>
> A key-argument could be useful but a reverse parameter doesn't make as
> much sense.
>
> If you want to make a patch, add it to the tracker and assign it to me
> for review.
>

I just opened http://bugs.python.org/issue13742 , but I can?t assign it. 
(New account on the tracker.)

Thanks,
-- 
Simon Sapin


From eric at trueblade.com  Mon Jan  9 11:51:34 2012
From: eric at trueblade.com (Eric V. Smith)
Date: Mon, 09 Jan 2012 05:51:34 -0500
Subject: [Python-ideas] A key parameter for heapq.merge
In-Reply-To: <4F0AC596.60906@kozea.fr>
References: <4F0A116D.3030202@kozea.fr>
	<73792DF0-F128-437E-8AC8-A9F34D042FF4@gmail.com>
	<4F0AC596.60906@kozea.fr>
Message-ID: <4F0AC6B6.2040403@trueblade.com>

On 1/9/2012 5:46 AM, Simon Sapin wrote:
> Le 09/01/2012 08:36, Raymond Hettinger a ?crit :
>>
>> A key-argument could be useful but a reverse parameter doesn't make as
>> much sense.
>>
>> If you want to make a patch, add it to the tracker and assign it to me
>> for review.
>>
> 
> I just opened http://bugs.python.org/issue13742 , but I can?t assign it.
> (New account on the tracker.)

I assigned it to Raymond.

Eric.



From stefan_ml at behnel.de  Mon Jan  9 11:51:49 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 09 Jan 2012 11:51:49 +0100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
Message-ID: <jeegs5$jh$1@dough.gmane.org>

anatoly techtonik, 09.01.2012 11:30:
> In Python 2 "print 'something', statement calls were unbuffered and
> immediately appeared on screen.
> In Python 3 "print('something', end='')" calls are buffered. This breaks
> progress bars and other stuff.
> 
> 1. What is more "Pythonic" and why?

I find the Py3 behaviour more pythonic: flushing should be explicit (which
is better than implicit). The mere fact that the user did *not* provide a
terminator shows that there are likely other things to follow. A progress
bar is just one specific case where flushing is actually desired. In other
cases, it may not be desired and would rather lead to performance issues.
Making it implicit if a flush happens or not prevents users from
controlling it.


> 2. Should Python 3 be fixed ASAP?

IMHO, no.

Stefan



From jeanpierreda at gmail.com  Mon Jan  9 11:53:09 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Mon, 9 Jan 2012 05:53:09 -0500
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
Message-ID: <CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>

> In Python 2 "print 'something', statement calls were unbuffered and
> immediately appeared on screen.

No, they weren't.

Python doesn't do any extra buffering or flushing. Usually your
terminal emulator line-buffers stdout. If you don't print a newline
(in Python 2 OR 3) then it doesn't show up until you either flush
(sys.stdout.flush()) or send a newline.

-- Devin

On Mon, Jan 9, 2012 at 5:30 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> In Python 2 "print 'something', statement calls were unbuffered and
> immediately appeared on screen.
> In Python 3 "print('something', end='')" calls are buffered. This breaks
> progress bars and other stuff.
>
> 1. What is more "Pythonic" and why?
> 2. Should Python 3 be fixed ASAP?
> --
> anatoly t.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From techtonik at gmail.com  Mon Jan  9 13:16:07 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 9 Jan 2012 15:16:07 +0300
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <jeegs5$jh$1@dough.gmane.org>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<jeegs5$jh$1@dough.gmane.org>
Message-ID: <CAPkN8x+1skUomSo353JKK7F72JYk1=upWCdvCtVfwg-Ceev0Hg@mail.gmail.com>

On Mon, Jan 9, 2012 at 1:51 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> anatoly techtonik, 09.01.2012 11:30:
> > In Python 2 "print 'something', statement calls were unbuffered and
> > immediately appeared on screen.
> > In Python 3 "print('something', end='')" calls are buffered. This breaks
> > progress bars and other stuff.
> >
> > 1. What is more "Pythonic" and why?
>
> I find the Py3 behaviour more pythonic: flushing should be explicit (which
> is better than implicit). The mere fact that the user did *not* provide a
> terminator shows that there are likely other things to follow.


Do you find pythonic that `print` statement should output string to the
screen (according to docs), but it doesn't do this?
Do you find ability to stack chars into screen buffer before the output
(instead of doing stacking to a string) more important than hiding the low
level output implementation details from the users?

A progress
> bar is just one specific case where flushing is actually desired. In other
> cases, it may not be desired and would rather lead to performance issues.
> Making it implicit if a flush happens or not prevents users from
> controlling it.
>

Can you give some examples of these 'other cases'? Maybe 95% of users are
using print for progress bars, and enabling flushing by default will make
it more pythonic by making code more beautiful (which is better than ugly).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120109/8789392b/attachment.html>

From techtonik at gmail.com  Mon Jan  9 13:21:31 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 9 Jan 2012 15:21:31 +0300
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
Message-ID: <CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>

On Mon, Jan 9, 2012 at 1:53 PM, Devin Jeanpierre <jeanpierreda at gmail.com>wrote:

> > In Python 2 "print 'something', statement calls were unbuffered and
> > immediately appeared on screen.
>
> No, they weren't.
>

 --- cut py2print.py ---
from time import sleep

while 1:
  sleep(1)
  print ".",
--- cut ---

This produces one dot every second with Python 2.7.2 on Windows.

Python doesn't do any extra buffering or flushing. Usually your
> terminal emulator line-buffers stdout. If you don't print a newline
> (in Python 2 OR 3) then it doesn't show up until you either flush
> (sys.stdout.flush()) or send a newline.


I assume you use Linux. Do you agree that for cross-platform language some
behavior should be made consistent?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120109/e362d7da/attachment.html>

From stefan_ml at behnel.de  Mon Jan  9 13:42:34 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 09 Jan 2012 13:42:34 +0100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAPkN8x+1skUomSo353JKK7F72JYk1=upWCdvCtVfwg-Ceev0Hg@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<jeegs5$jh$1@dough.gmane.org>
	<CAPkN8x+1skUomSo353JKK7F72JYk1=upWCdvCtVfwg-Ceev0Hg@mail.gmail.com>
Message-ID: <jeenbq$mel$1@dough.gmane.org>

anatoly techtonik, 09.01.2012 13:16:
> On Mon, Jan 9, 2012 at 1:51 PM, Stefan Behnel wrote:
>> anatoly techtonik, 09.01.2012 11:30:
>>> In Python 2 "print 'something', statement calls were unbuffered and
>>> immediately appeared on screen.
>>> In Python 3 "print('something', end='')" calls are buffered. This breaks
>>> progress bars and other stuff.
>>>
>>> 1. What is more "Pythonic" and why?
>>
>> I find the Py3 behaviour more pythonic: flushing should be explicit (which
>> is better than implicit). The mere fact that the user did *not* provide a
>> terminator shows that there are likely other things to follow.
> 
> Do you find pythonic that `print` statement should output string to the
> screen (according to docs), but it doesn't do this?

Well, it does it if you tell it to. Lacking an explicit request, the rest
is platform specific, as usual with multi-level output buffering.

Actually, print() is often the first place where new users get in touch
with I/O "specialties" such as flush() and buffering/queuing, and it's
helpful to learn about them. Not only in the context of terminal I/O but
also for disk or network I/O.


> Do you find ability to stack chars into screen buffer before the output
> (instead of doing stacking to a string) more important than hiding the low
> level output implementation details from the users?

All I'm saying is that there is no "right" way to do it, so choosing to
flush implicitly will a) break user code and b) keep some (or many?) users
from getting the behaviour they want. And that without good reason and
without making it "better" or "more correct" through such a change.


>> A progress
>> bar is just one specific case where flushing is actually desired. In other
>> cases, it may not be desired and would rather lead to performance issues.
>> Making it implicit if a flush happens or not prevents users from
>> controlling it.
> 
> Can you give some examples of these 'other cases'? Maybe 95% of users are
> using print for progress bars

... or maybe not ...

> and enabling flushing by default will make
> it more pythonic by making code more beautiful (which is better than ugly).

print() is just one way of pushing output "somewhere", and often not the
best one. It's helpful for debugging and mostly acceptable in short
scripts, but it's otherwise useless for most applications. That being said,
I certainly don't claim to have an idea of what "most" users use print()
for and what behaviour "most" users expect...

Stefan



From p.f.moore at gmail.com  Mon Jan  9 13:50:23 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 9 Jan 2012 12:50:23 +0000
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
Message-ID: <CACac1F_MLW8Lif5+0nTBHmcNSZK5t7GaydCGHnPKK4fGk_9UCQ@mail.gmail.com>

On 9 January 2012 12:21, anatoly techtonik <techtonik at gmail.com> wrote:
>> Python doesn't do any extra buffering or flushing. Usually your
>> terminal emulator line-buffers stdout. If you don't print a newline
>> (in Python 2 OR 3) then it doesn't show up until you either flush
>> (sys.stdout.flush()) or send a newline.
>
> I assume you use Linux. Do you agree that for cross-platform language some
> behavior should be made consistent?

No. (At least not for 2.x). The Python 2.x position has always been
that the IO layer is a relatively thin wrapper over the platform
behaviour, and so this sort of platform dependent behaviour should not
come as a surprise to anyone.

For Python 3.x, the IO layer was reworked to be more platform-neutral,
and I would expect consistent behaviour across platforms there. And
the behaviour I would expect (given that stdout is a buffered text
stream) is that output from print would be buffered until an implicit
or explicit flush. And that is exactly what happens.

So I'd say that you're 100% right in what you say as regards Python 3,
and that is exactly what is happening. The difference with Python 2 is
precisely one of those backward-incompatible breaks that required the
2->3 major version change.

Characterising a carefully thought out change implemented in a way
that minimises backward compatibility issues as a "bug", is at best
unfair and dismissive of the work that was put into this.

Paul.


From sergio at gruposinternet.com.br  Mon Jan  9 14:05:57 2012
From: sergio at gruposinternet.com.br (=?ISO-8859-1?Q?S=E9rgio?= Surkamp)
Date: Mon, 9 Jan 2012 11:05:57 -0200
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
Message-ID: <20120109110557.75e23bc3@icedearth.gruposinternet.com.br>

Em Mon, 9 Jan 2012 13:30:55 +0300
anatoly techtonik <techtonik at gmail.com> escreveu:

> In Python 2 "print 'something', statement calls were unbuffered and
> immediately appeared on screen.

Only if Python is called with -u command line option, from the man page:

-u     Force  stdin,  stdout  and  stderr to be totally unbuffered.

Regards,
-- 
  .:''''':.
.:'        `     S?rgio Surkamp | Administrador de Redes
::    ........   sergio at gruposinternet.com.br
`:.        .:'
  `:,   ,.:'     *Grupos Internet S.A.*
    `: :'        R. Lauro Linhares, 2123 Torre B - Sala 201
     : :         Trindade - Florian?polis - SC
     :.'
     ::          +55 48 3234-4109
     :
     '           http://www.gruposinternet.com.br


From jeanpierreda at gmail.com  Mon Jan  9 14:25:23 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Mon, 9 Jan 2012 08:25:23 -0500
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
Message-ID: <CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>

> This produces one dot every second with Python 2.7.2 on Windows.

Weird. Must be a windows thing.

> I assume you use Linux. Do you agree that for cross-platform language some
> behavior should be made consistent?

Yes. I guess that's why they changed the behavior of print on Windows
in Python 3.

-- Devin

On Mon, Jan 9, 2012 at 7:21 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> On Mon, Jan 9, 2012 at 1:53 PM, Devin Jeanpierre <jeanpierreda at gmail.com>
> wrote:
>>
>> > In Python 2 "print 'something', statement calls were unbuffered and
>> > immediately appeared on screen.
>>
>> No, they weren't.
>
>
> ?--- cut py2print.py ---
> from time import sleep
>
> while 1:
> ? sleep(1)
> ? print ".",
> --- cut ---
>
> This produces one dot every second with Python 2.7.2 on Windows.
>
>> Python doesn't do any extra buffering or flushing. Usually your
>> terminal emulator line-buffers stdout. If you don't print a newline
>> (in Python 2 OR 3) then it doesn't show up until you either flush
>> (sys.stdout.flush()) or send a newline.
>
>
> I assume you use Linux. Do you agree that for cross-platform language some
> behavior should be made consistent?


From victor.stinner at haypocalc.com  Mon Jan  9 15:14:37 2012
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 9 Jan 2012 15:14:37 +0100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
Message-ID: <CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>

Read also the discussion on the Python bug tracker:
http://bugs.python.org/issue11633

Victor


From guido at python.org  Mon Jan  9 18:52:21 2012
From: guido at python.org (Guido van Rossum)
Date: Mon, 9 Jan 2012 09:52:21 -0800
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
Message-ID: <CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>

This has nothing to do with print or print(). There are no flush() calls
implied by print/print() calls, nor should there be(*). print/print()
simply calls sys.stdout.write() one or more times, the final call usually
being sys.stdout.write('\n') which will trigger a flush() if the file is
line-buffered -- but if you use a trailing comma (Python 2) or end=...
(Python 3) the last thing written is not '\n' and no flush() will be
triggered.

Whether writing \n flushes or not depends on the platform, the Python
version and implementation, and the environment (e.g. most platforms
automatically turn on line buffering for files known to be associated with
a terminal emulator or other interactive device, assuming there's an easy
way to tell).

__________
(*) There are plenty of apps that construct their output using many
print/print() calls, and for which flushing after each call would be quite
unnecessary and potentially cause lots of redundant I/O operations,
ultimately slowing down the app. Apps that need flushing should call
flush().

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120109/c29bf90c/attachment.html>

From tjreedy at udel.edu  Mon Jan  9 20:59:40 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 09 Jan 2012 14:59:40 -0500
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
Message-ID: <jefgvo$ptm$1@dough.gmane.org>

On 1/9/2012 12:52 PM, Guido van Rossum wrote:
> This has nothing to do with print or print(). There are no flush() calls
> implied by print/print() calls, nor should there be(*). print/print()
> simply calls sys.stdout.write() one or more times, the final call
> usually being sys.stdout.write('\n') which will trigger a flush() if the
> file is line-buffered -- but if you use a trailing comma (Python 2) or
> end=... (Python 3) the last thing written is not '\n' and no flush()
> will be triggered.
>
> Whether writing \n flushes or not depends on the platform, the Python
> version and implementation, and the environment (e.g. most platforms
> automatically turn on line buffering for files known to be associated
> with a terminal emulator or other interactive device, assuming there's
> an easy way to tell).
>
> __________
> (*) There are plenty of apps that construct their output using many
> print/print() calls, and for which flushing after each call would be
> quite unnecessary and potentially cause lots of redundant I/O
> operations, ultimately slowing down the app.

Thank you for the clarification. I noted on
http://bugs.python.org/issue11633
that there should be no code change

> Apps that need flushing should call flush().

and that saying something like this in the print doc entry is the only 
possible change.

-- 
Terry Jan Reedy



From songofacandy at gmail.com  Tue Jan 10 06:52:39 2012
From: songofacandy at gmail.com (INADA Naoki)
Date: Tue, 10 Jan 2012 14:52:39 +0900
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <jefgvo$ptm$1@dough.gmane.org>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
	<jefgvo$ptm$1@dough.gmane.org>
Message-ID: <CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>

What about adding flush=False keyword argument to print()?

On Tue, Jan 10, 2012 at 4:59 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 1/9/2012 12:52 PM, Guido van Rossum wrote:
>>
>> This has nothing to do with print or print(). There are no flush() calls
>> implied by print/print() calls, nor should there be(*). print/print()
>> simply calls sys.stdout.write() one or more times, the final call
>> usually being sys.stdout.write('\n') which will trigger a flush() if the
>> file is line-buffered -- but if you use a trailing comma (Python 2) or
>> end=... (Python 3) the last thing written is not '\n' and no flush()
>> will be triggered.
>>
>> Whether writing \n flushes or not depends on the platform, the Python
>> version and implementation, and the environment (e.g. most platforms
>> automatically turn on line buffering for files known to be associated
>> with a terminal emulator or other interactive device, assuming there's
>> an easy way to tell).
>>
>> __________
>> (*) There are plenty of apps that construct their output using many
>> print/print() calls, and for which flushing after each call would be
>> quite unnecessary and potentially cause lots of redundant I/O
>> operations, ultimately slowing down the app.
>
>
> Thank you for the clarification. I noted on
> http://bugs.python.org/issue11633
> that there should be no code change
>
>
>> Apps that need flushing should call flush().
>
>
> and that saying something like this in the print doc entry is the only
> possible change.
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
INADA Naoki? <songofacandy at gmail.com>


From cmjohnson.mailinglist at gmail.com  Tue Jan 10 09:49:32 2012
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Mon, 9 Jan 2012 22:49:32 -1000
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
	<jefgvo$ptm$1@dough.gmane.org>
	<CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>
Message-ID: <BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>


On Jan 9, 2012, at 7:52 PM, INADA Naoki wrote:

> What about adding flush=False keyword argument to print()?

I would expect flush=False to mean "please don't flush this right now, even if I do use a newline." 

flush=None?


From songofacandy at gmail.com  Tue Jan 10 10:05:00 2012
From: songofacandy at gmail.com (INADA Naoki)
Date: Tue, 10 Jan 2012 18:05:00 +0900
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
	<jefgvo$ptm$1@dough.gmane.org>
	<CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>
	<BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
Message-ID: <CAEfz+Ty61fdf6j4XVKO2vr6aXUYXyNJA5vBy24HFzj7-syE2YQ@mail.gmail.com>

On Tue, Jan 10, 2012 at 5:49 PM, Carl M. Johnson
<cmjohnson.mailinglist at gmail.com> wrote:
>
> On Jan 9, 2012, at 7:52 PM, INADA Naoki wrote:
>
>> What about adding flush=False keyword argument to print()?
>
> I would expect flush=False to mean "please don't flush this right now, even if I do use a newline."

print() doesn't flushes even if there is newline. So the keyword argument means
"do or don't call flush() method of the file."

>
> flush=None?
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
INADA Naoki? <songofacandy at gmail.com>


From anacrolix at gmail.com  Tue Jan 10 10:15:27 2012
From: anacrolix at gmail.com (Matt Joiner)
Date: Tue, 10 Jan 2012 20:15:27 +1100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAEfz+Ty61fdf6j4XVKO2vr6aXUYXyNJA5vBy24HFzj7-syE2YQ@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
	<jefgvo$ptm$1@dough.gmane.org>
	<CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>
	<BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
	<CAEfz+Ty61fdf6j4XVKO2vr6aXUYXyNJA5vBy24HFzj7-syE2YQ@mail.gmail.com>
Message-ID: <CAB4yi1MvZF8-tFF6DBaZ5SZ9G5WO3QjRzq60MgXZ2nXHMZmGWQ@mail.gmail.com>

I don't think it would be straight forward or reasonable to _prevent_
flushing from occurring. Forcing a flush, is only useful when the file
object to which you're printing isn't already handy, and this only
occurs for sys.stdout. It's not worth adding a "flush-in-case" flag
just for this one stream, especially if it's unable to prevent
flushing.

On Tue, Jan 10, 2012 at 8:05 PM, INADA Naoki <songofacandy at gmail.com> wrote:
> On Tue, Jan 10, 2012 at 5:49 PM, Carl M. Johnson
> <cmjohnson.mailinglist at gmail.com> wrote:
>>
>> On Jan 9, 2012, at 7:52 PM, INADA Naoki wrote:
>>
>>> What about adding flush=False keyword argument to print()?
>>
>> I would expect flush=False to mean "please don't flush this right now, even if I do use a newline."
>
> print() doesn't flushes even if there is newline. So the keyword argument means
> "do or don't call flush() method of the file."
>
>>
>> flush=None?
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>
>
>
> --
> INADA Naoki? <songofacandy at gmail.com>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From barry at python.org  Tue Jan 10 10:34:27 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 10 Jan 2012 10:34:27 +0100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<jeegs5$jh$1@dough.gmane.org>
	<CAPkN8x+1skUomSo353JKK7F72JYk1=upWCdvCtVfwg-Ceev0Hg@mail.gmail.com>
Message-ID: <20120110103427.285de748@rivendell>

On Jan 09, 2012, at 03:16 PM, anatoly techtonik wrote:

>Can you give some examples of these 'other cases'? Maybe 95% of users are
>using print for progress bars, and enabling flushing by default will make
>it more pythonic by making code more beautiful (which is better than ugly).

I guess I'm in that other 5% <wink>.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120110/50f6c89c/attachment.pgp>

From masklinn at masklinn.net  Tue Jan 10 11:15:58 2012
From: masklinn at masklinn.net (Masklinn)
Date: Tue, 10 Jan 2012 11:15:58 +0100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAEfz+Ty61fdf6j4XVKO2vr6aXUYXyNJA5vBy24HFzj7-syE2YQ@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
	<jefgvo$ptm$1@dough.gmane.org>
	<CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>
	<BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
	<CAEfz+Ty61fdf6j4XVKO2vr6aXUYXyNJA5vBy24HFzj7-syE2YQ@mail.gmail.com>
Message-ID: <B07A7173-1553-4EE1-8835-E5EE8E92BBE2@masklinn.net>

On 10 janv. 2012, at 10:05, INADA Naoki <songofacandy at gmail.com> wrote:
> On Tue, Jan 10, 2012 at 5:49 PM, Carl M. Johnson
> <cmjohnson.mailinglist at gmail.com> wrote:
>> 
>> On Jan 9, 2012, at 7:52 PM, INADA Naoki wrote:
>> 
>>> What about adding flush=False keyword argument to print()?
>> 
>> I would expect flush=False to mean "please don't flush this right now, even if I do use a newline."
> 
> print() doesn't flushes even if there is newline.
Ni, but the underlying stream may flush itself on newlines. 

> So the keyword argument means
> "do or don't call flush() method of the file."
That's not clear from its name since the flushing behavior can depend on the underlying stream type. force_flush would be closer to the actual meaning of the param. 

From phd at phdru.name  Tue Jan 10 12:22:16 2012
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 10 Jan 2012 15:22:16 +0400
Subject: [Python-ideas] Posting style
Message-ID: <20120110112216.GA21299@iskra.aviel.ru>

Hello!

   I very much hope I do not embarrass the audience but people can we
return back to email etiquette? Inline quoting instead of top-posting,
proper trimming instead of overquoting, an empty line to separate quoted
text from the reply.

   Sorry for the noise but can I ask at least? Yes, I am an old fart
joined the Net at 1991, but I find netiquette really useful for effective
communication and deviations from it inconvenient and annoying.

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From ncoghlan at gmail.com  Tue Jan 10 12:53:05 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Jan 2012 21:53:05 +1000
Subject: [Python-ideas] Posting style
In-Reply-To: <20120110112216.GA21299@iskra.aviel.ru>
References: <20120110112216.GA21299@iskra.aviel.ru>
Message-ID: <CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>

On Tue, Jan 10, 2012 at 9:22 PM, Oleg Broytman <phd at phdru.name> wrote:
> Hello!
>
> ? I very much hope I do not embarrass the audience but people can we
> return back to email etiquette? Inline quoting instead of top-posting,
> proper trimming instead of overquoting, an empty line to separate quoted
> text from the reply.

Unfortunately, you're fighting against the default behaviour of many
current email clients. As far as I can tell, they're written on the
assumption of conversational email chains amongst small numbers of
people rather than potentially sprawling permanently archived mailing
list threads with multiple branches and a large number of
participants. Even the Gmail web client starts with the cursor above
the quoted text (although it places the signature block after it).

The big pain though is answering email on a (not-so-)smartphone - I
know the Android Gmail client places both cursor and signature above
the quoted message, and fixing the formatting is annoying enough that
the only two realistic choices are either top posting or not replying
at all (and I'll often choose to wait until I'm back at a machine with
a full keyboard for exactly that reason).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From barry at python.org  Tue Jan 10 13:01:14 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 10 Jan 2012 13:01:14 +0100
Subject: [Python-ideas] Posting style
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
Message-ID: <20120110130114.7cb8d59e@rivendell>

On Jan 10, 2012, at 09:53 PM, Nick Coghlan wrote:

>Unfortunately, you're fighting against the default behaviour of many
>current email clients.

We've been fighting damaged email clients for probably as long as there has
been email.  Yes it's a futile fight despite that mail client authors have
known these things for a bazillion years.  Oleg's probably not as old or farty
as me, but I appreciate him at least reminding people of these conventions.
Being aware of them, perhaps we'll all take just a few extra seconds to make
sure our posts are more readable for everyone.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120110/7dcb397e/attachment.pgp>

From phd at phdru.name  Tue Jan 10 13:13:28 2012
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 10 Jan 2012 16:13:28 +0400
Subject: [Python-ideas] Posting style
In-Reply-To: <CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
Message-ID: <20120110121328.GB21299@iskra.aviel.ru>

On Tue, Jan 10, 2012 at 09:53:05PM +1000, Nick Coghlan wrote:
> On Tue, Jan 10, 2012 at 9:22 PM, Oleg Broytman <phd at phdru.name> wrote:
> >    I very much hope I do not embarrass the audience but people can we
> > return back to email etiquette? Inline quoting instead of top-posting,
> > proper trimming instead of overquoting, an empty line to separate quoted
> > text from the reply.
> 
> Unfortunately, you're fighting against the default behaviour of many
> current email clients.

   I do this with the assumption that subscribers to python mailing
lists are developers and powerful users who understand what's good and
what's bad and can configure their software accordingly. Defaults are
for mere mortals. ;-)

> The big pain though is answering email on a (not-so-)smartphone

   I know and I can bear this particular situation. People usually don't
participate in long and complex discussions using small screen clients,
and short top-posted notes are bearable. Especially if the rest is
trimmed.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From jeanpierreda at gmail.com  Tue Jan 10 13:15:07 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 10 Jan 2012 07:15:07 -0500
Subject: [Python-ideas] Posting style
In-Reply-To: <20120110112216.GA21299@iskra.aviel.ru>
References: <20120110112216.GA21299@iskra.aviel.ru>
Message-ID: <CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>

>   Sorry for the noise but can I ask at least? Yes, I am an old fart
> joined the Net at 1991, but I find netiquette really useful for effective
> communication and deviations from it inconvenient and annoying.

Is replying like this acceptable?

-- Devin

On Tue, Jan 10, 2012 at 6:22 AM, Oleg Broytman <phd at phdru.name> wrote:
> Hello!
>
> ? I very much hope I do not embarrass the audience but people can we
> return back to email etiquette? Inline quoting instead of top-posting,
> proper trimming instead of overquoting, an empty line to separate quoted
> text from the reply.
>
>   Sorry for the noise but can I ask at least? Yes, I am an old fart
> joined the Net at 1991, but I find netiquette really useful for effective
> communication and deviations from it inconvenient and annoying.
>
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?
> A: Top-posting.
> Q: What is the most annoying thing in e-mail?
>
> Oleg.
> --
> ? ? Oleg Broytman ? ? ? ? ? ?http://phdru.name/ ? ? ? ? ? ?phd at phdru.name
> ? ? ? ? ? Programmers don't die, they just GOSUB without RETURN.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From phd at phdru.name  Tue Jan 10 13:43:54 2012
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 10 Jan 2012 16:43:54 +0400
Subject: [Python-ideas] Posting style
In-Reply-To: <CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
Message-ID: <20120110124354.GC21299@iskra.aviel.ru>

On Tue, Jan 10, 2012 at 07:15:07AM -0500, Devin Jeanpierre wrote:
> >   Sorry for the noise but can I ask at least? Yes, I am an old fart
> > joined the Net at 1991, but I find netiquette really useful for effective
> > communication and deviations from it inconvenient and annoying.
> 
> Is replying like this acceptable?

   I am not going to mandate anything (first, I don't have power).
Anything is acceptable. But decide for yourself what is convenient *for
you*. Imagine you are reading through a big and complex discussion with
many threads. Or think about looking for information in the mailing
list(s) archive. How much excessive information can you bear? What
posing style would be the best in that cases?

   It is easy to see the dilemma. It's convenient for a poster to do a
quick reply - hence top-posting without trimming. But that's
inconvenient for readers and especially for archive browsers. So if you
are going to be a long-term participator wouldn't it be in the public
interests if you spend a few additional seconds properly formatting your
replies? It's in your interests too because sooner or later you will
return to your own discussions in the archive.

   That's my humble opinion.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From jeanpierreda at gmail.com  Tue Jan 10 14:13:42 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 10 Jan 2012 08:13:42 -0500
Subject: [Python-ideas] Posting style
In-Reply-To: <20120110124354.GC21299@iskra.aviel.ru>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
	<20120110124354.GC21299@iskra.aviel.ru>
Message-ID: <CABicbJ+_ZOnbDiwn+E+54Brbktv6s3Y8wMaOy5UrNZgA9c+OuA@mail.gmail.com>

On Tue, Jan 10, 2012 at 7:43 AM, Oleg Broytman <phd at phdru.name> wrote:
> On Tue, Jan 10, 2012 at 07:15:07AM -0500, Devin Jeanpierre wrote:
>> > ? Sorry for the noise but can I ask at least? Yes, I am an old fart
>> > joined the Net at 1991, but I find netiquette really useful for effective
>> > communication and deviations from it inconvenient and annoying.
>>
>> Is replying like this acceptable?
>
> ? I am not going to mandate anything (first, I don't have power).
> Anything is acceptable. But decide for yourself what is convenient *for
> you*. Imagine you are reading through a big and complex discussion with
> many threads. Or think about looking for information in the mailing
> list(s) archive. How much excessive information can you bear? What
> posing style would be the best in that cases?

I don't really understand the objection. My mail client hides unbroken
quotes at the bottom of an email, so that I don't see anything except
what you quoted (if I read my own email), plus my signature. I take it
yours doesn't do this?

-- Devin


From techtonik at gmail.com  Tue Jan 10 14:16:09 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 10 Jan 2012 16:16:09 +0300
Subject: [Python-ideas] An etherpad for every idea
Message-ID: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>

The last thread about "Posting style" makes me think that effective
communication with email is impossible at all in modern world, where time
pressure doesn't allow to read responses, not speaking about properly
formatting them or following guidelines. The problem with email threads
that they can be hijacked, polluted with irrelevant info (context garbage),
personal opinions etc. If the aim of the thread is to "sell" some idea, it
is almost impossible to track when people change their minds. And the
biggest problem is that *email threads are stateless*. Meaning that to get
the current state of the problem everybody needs to waste a lot of time to
reread the thread and filter it. Sometimes it just takes too much time.

So I propose to have a summary for each thread in Etherpad -
http://en.wikipedia.org/wiki/Etherpad - that will provide current current
summary of email thread, describing original problem, its evolution,
arguments and counter-arguments.

-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120110/b644b2ae/attachment.html>

From arnodel at gmail.com  Tue Jan 10 14:20:04 2012
From: arnodel at gmail.com (Arnaud Delobelle)
Date: Tue, 10 Jan 2012 13:20:04 +0000
Subject: [Python-ideas] Posting style
In-Reply-To: <CABicbJ+_ZOnbDiwn+E+54Brbktv6s3Y8wMaOy5UrNZgA9c+OuA@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
	<20120110124354.GC21299@iskra.aviel.ru>
	<CABicbJ+_ZOnbDiwn+E+54Brbktv6s3Y8wMaOy5UrNZgA9c+OuA@mail.gmail.com>
Message-ID: <CAJ6cK1aNVgpL5L1-aQbQPbUihQ-0AA9WEB2amvFds8ofcDCBFw@mail.gmail.com>

On 10 January 2012 13:13, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
> On Tue, Jan 10, 2012 at 7:43 AM, Oleg Broytman <phd at phdru.name> wrote:
>> ? I am not going to mandate anything (first, I don't have power).
>> Anything is acceptable. But decide for yourself what is convenient *for
>> you*. Imagine you are reading through a big and complex discussion with
>> many threads. Or think about looking for information in the mailing
>> list(s) archive. How much excessive information can you bear? What
>> posing style would be the best in that cases?
>
> I don't really understand the objection. My mail client hides unbroken
> quotes at the bottom of an email, so that I don't see anything except
> what you quoted (if I read my own email), plus my signature. I take it
> yours doesn't do this?

People don't usually look at the archives with their email client.

-- 
Arnaud


From techtonik at gmail.com  Tue Jan 10 14:27:24 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 10 Jan 2012 16:27:24 +0300
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
Message-ID: <CAPkN8xKPZP2cA7OHmu5d76dh3uzomx-8VmfLLQjMfsq2MWStuQ@mail.gmail.com>

I see that Python 3 behavior is consistent on Windows too meaning it
doesn't output anything on the screen when end=''

It seems that basic function print() that was created to just "print a
string" got over-engineered breaking unix principle of "doing one thing".
It reminds me of subprocess.Popen which was also designed to just "run a
process". I would keep it as simple as possible for printing stuff to the
screen without any implicit buffering. You called print() - you get the
output. KISS. If people need more control with output buffering - provide a
pointer to sys.stdout with all gory details about streams etc.

To get back on topic, let me give a definition. To me "pythonic" means
"intuitive" in the first place. And "intuitive" means "up to user
expectations". What does user expect from "print()" function? Writing to a
file? I don't think so. Output to printer? Maybe. Printing to the screen?
Definitely.

"intuitive" also means that users won't have to consult user documentation
for every basic thing it needs, especially for such basic thing such as
getting some stuff printed to the screen output - the stuff that is
essential in the coding process. It is already less convenient in Py3k,
because you need to type extra Shift-9 and Shift-0 in the process, and it
risks to become completely weird:

  print('calling external process... ', end='')
  sys.output.flush()
  ...
  print('done')


So, py3k print() is not pythonic. To make it pythonic, the output should be
immediately available after you call it. If you need to buffer output - do
this explicitly, because print() doesn't guarantee that it will be buffered
anyway. If you care about performance - use sys.stdout directly and tune it
explicitly.


On Mon, Jan 9, 2012 at 4:25 PM, Devin Jeanpierre <jeanpierreda at gmail.com>wrote:

> > This produces one dot every second with Python 2.7.2 on Windows.
>
> Weird. Must be a windows thing.
>
> > I assume you use Linux. Do you agree that for cross-platform language
> some
> > behavior should be made consistent?
>
> Yes. I guess that's why they changed the behavior of print on Windows
> in Python 3.
>
> -- Devin
>
> On Mon, Jan 9, 2012 at 7:21 AM, anatoly techtonik <techtonik at gmail.com>
> wrote:
> > On Mon, Jan 9, 2012 at 1:53 PM, Devin Jeanpierre <jeanpierreda at gmail.com
> >
> > wrote:
> >>
> >> > In Python 2 "print 'something', statement calls were unbuffered and
> >> > immediately appeared on screen.
> >>
> >> No, they weren't.
> >
> >
> >  --- cut py2print.py ---
> > from time import sleep
> >
> > while 1:
> >   sleep(1)
> >   print ".",
> > --- cut ---
> >
> > This produces one dot every second with Python 2.7.2 on Windows.
> >
> >> Python doesn't do any extra buffering or flushing. Usually your
> >> terminal emulator line-buffers stdout. If you don't print a newline
> >> (in Python 2 OR 3) then it doesn't show up until you either flush
> >> (sys.stdout.flush()) or send a newline.
> >
> >
> > I assume you use Linux. Do you agree that for cross-platform language
> some
> > behavior should be made consistent?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120110/8ee3f125/attachment.html>

From phd at phdru.name  Tue Jan 10 14:30:47 2012
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 10 Jan 2012 17:30:47 +0400
Subject: [Python-ideas] Posting style
In-Reply-To: <CABicbJ+_ZOnbDiwn+E+54Brbktv6s3Y8wMaOy5UrNZgA9c+OuA@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
	<20120110124354.GC21299@iskra.aviel.ru>
	<CABicbJ+_ZOnbDiwn+E+54Brbktv6s3Y8wMaOy5UrNZgA9c+OuA@mail.gmail.com>
Message-ID: <20120110133047.GD21299@iskra.aviel.ru>

On Tue, Jan 10, 2012 at 08:13:42AM -0500, Devin Jeanpierre wrote:
> I don't really understand the objection. My mail client hides unbroken
> quotes at the bottom of an email, so that I don't see anything except
> what you quoted (if I read my own email), plus my signature. I take it
> yours doesn't do this?

   When I do googling through web-archives I use web browser to read
what I found. Google, in my experience, produces the most relevant
results.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From solipsis at pitrou.net  Tue Jan 10 14:31:56 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Jan 2012 14:31:56 +0100
Subject: [Python-ideas] An etherpad for every idea
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
Message-ID: <20120110143156.1188db76@pitrou.net>

On Tue, 10 Jan 2012 16:16:09 +0300
anatoly techtonik <techtonik at gmail.com>
wrote:
> 
> So I propose to have a summary for each thread in Etherpad -
> http://en.wikipedia.org/wiki/Etherpad - that will provide current current
> summary of email thread, describing original problem, its evolution,
> arguments and counter-arguments.

Are you willing to maintain it? Feel free to create and update etherpad
summaries for a few of our current discussion threads, so that we see
if that's useful.

Regards

Antoine.




From anacrolix at gmail.com  Tue Jan 10 18:08:49 2012
From: anacrolix at gmail.com (Matt Joiner)
Date: Wed, 11 Jan 2012 04:08:49 +1100
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <20120110143156.1188db76@pitrou.net>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
Message-ID: <CAB4yi1NfJvWkCiknuvbE49K8+31LBB2qswBhwWzug723H7oOcQ@mail.gmail.com>

Are these already in an etherpad clone or...?

On Wed, Jan 11, 2012 at 12:31 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 10 Jan 2012 16:16:09 +0300
> anatoly techtonik <techtonik at gmail.com>
> wrote:
>>
>> So I propose to have a summary for each thread in Etherpad -
>> http://en.wikipedia.org/wiki/Etherpad - that will provide current current
>> summary of email thread, describing original problem, its evolution,
>> arguments and counter-arguments.
>
> Are you willing to maintain it? Feel free to create and update etherpad
> summaries for a few of our current discussion threads, so that we see
> if that's useful.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From anacrolix at gmail.com  Tue Jan 10 18:12:37 2012
From: anacrolix at gmail.com (Matt Joiner)
Date: Wed, 11 Jan 2012 04:12:37 +1100
Subject: [Python-ideas] Posting style
In-Reply-To: <20120110133047.GD21299@iskra.aviel.ru>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
	<20120110124354.GC21299@iskra.aviel.ru>
	<CABicbJ+_ZOnbDiwn+E+54Brbktv6s3Y8wMaOy5UrNZgA9c+OuA@mail.gmail.com>
	<20120110133047.GD21299@iskra.aviel.ru>
Message-ID: <CAB4yi1P6DmmCqaB8rY0uca0janpvaq4Qr5j0uzcvE8BqHcNJXg@mail.gmail.com>

lol

On Wed, Jan 11, 2012 at 12:30 AM, Oleg Broytman <phd at phdru.name> wrote:
> On Tue, Jan 10, 2012 at 08:13:42AM -0500, Devin Jeanpierre wrote:
>> I don't really understand the objection. My mail client hides unbroken
>> quotes at the bottom of an email, so that I don't see anything except
>> what you quoted (if I read my own email), plus my signature. I take it
>> yours doesn't do this?
>
> ? When I do googling through web-archives I use web browser to read
> what I found. Google, in my experience, produces the most relevant
> results.
>
> Oleg.
> --
> ? ? Oleg Broytman ? ? ? ? ? ?http://phdru.name/ ? ? ? ? ? ?phd at phdru.name
> ? ? ? ? ? Programmers don't die, they just GOSUB without RETURN.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From ethan at stoneleaf.us  Tue Jan 10 18:04:28 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 10 Jan 2012 09:04:28 -0800
Subject: [Python-ideas] Posting style
In-Reply-To: <CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
Message-ID: <4F0C6F9C.50409@stoneleaf.us>

Devin Jeanpierre wrote:
>>   Sorry for the noise but can I ask at least? Yes, I am an old fart
>> joined the Net at 1991, but I find netiquette really useful for effective
>> communication and deviations from it inconvenient and annoying.
> 
> Is replying like this acceptable?

It would be better if you deleted the post following your reply.  It's 
already been read, no need to consume the bandwidth on the wire nor on 
the screen.  :)

~Ethan~


From ubershmekel at gmail.com  Tue Jan 10 19:36:04 2012
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Tue, 10 Jan 2012 20:36:04 +0200
Subject: [Python-ideas] Posting style
In-Reply-To: <4F0C6F9C.50409@stoneleaf.us>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
	<4F0C6F9C.50409@stoneleaf.us>
Message-ID: <CANSw7KwTskQGwVY7+qFXZow3L5s+u+VocPKM6C3CKnhQijxk=w@mail.gmail.com>

On Tue, Jan 10, 2012 at 7:04 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> Devin Jeanpierre wrote:
>
>> Is replying like this acceptable?
>>
>>
> It would be better if you deleted the post following your reply.  It's
> already been read, no need to consume the bandwidth on the wire nor on the
> screen.  :)
>
>
Gmail collapses that so I would have never even known he kept the post
following following his reply.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120110/011a25b4/attachment.html>

From phd at phdru.name  Tue Jan 10 20:05:18 2012
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 10 Jan 2012 23:05:18 +0400
Subject: [Python-ideas] Posting style
In-Reply-To: <CANSw7KwTskQGwVY7+qFXZow3L5s+u+VocPKM6C3CKnhQijxk=w@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
	<4F0C6F9C.50409@stoneleaf.us>
	<CANSw7KwTskQGwVY7+qFXZow3L5s+u+VocPKM6C3CKnhQijxk=w@mail.gmail.com>
Message-ID: <20120110190518.GA30986@iskra.aviel.ru>

On Tue, Jan 10, 2012 at 08:36:04PM +0200, Yuval Greenfield wrote:
> On Tue, Jan 10, 2012 at 7:04 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> > Devin Jeanpierre wrote:
> >> Is replying like this acceptable?
> >>
> > It would be better if you deleted the post following your reply.  It's
> > already been read, no need to consume the bandwidth on the wire nor on the
> > screen.  :)
> >
> Gmail collapses that so I would have never even known he kept the post
> following following his reply.

   And of course everyone and his brother use GMail?

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From mwm at mired.org  Tue Jan 10 20:25:21 2012
From: mwm at mired.org (Mike Meyer)
Date: Tue, 10 Jan 2012 11:25:21 -0800
Subject: [Python-ideas] Posting style
In-Reply-To: <CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
Message-ID: <20120110112521.12d7531c@mikmeyer-vm-fedora>

On Tue, 10 Jan 2012 21:53:05 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> The big pain though is answering email on a (not-so-)smartphone - I
> know the Android Gmail client places both cursor and signature above
> the quoted message, and fixing the formatting is annoying enough that
> the only two realistic choices are either top posting or not replying
> at all (and I'll often choose to wait until I'm back at a machine with
> a full keyboard for exactly that reason).

The android client K-9 is better behaved. I find it more usable in
general than the Gmail client (or Google's mail client, for matter
that).

To me, the real issue is proper editing more than where the quote
winds up. If you're just going to fire off a quick quip, editing the
quote makes it a lot more work. If you're actually taking time to
think about what you're saying, then editing the quote is a minor bit
of work, and may well help with the composition of the message.

Basically, top posting says "I didn't think about this, just shot off
a reply." Bottom posting beneath an unedited quote says "I'm an old
fart who didn't think about this, just shot off a reply." Taking the
time to properly edit the quoted text says "I thought about what I'm
saying, and took the time to remove the irrelevant."

Final comment: I saw a number of "My mail reader hides the quotes"
replies. If "get a better mailer" is an acceptable response, then
consider it used if your mailer makes top posting the easy option.

     <mike


From ben+python at benfinney.id.au  Tue Jan 10 21:21:14 2012
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 11 Jan 2012 07:21:14 +1100
Subject: [Python-ideas] Posting style
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
Message-ID: <87mx9v8g1x.fsf@benfinney.id.au>

Nick Coghlan <ncoghlan at gmail.com> writes:

> On Tue, Jan 10, 2012 at 9:22 PM, Oleg Broytman <phd at phdru.name> wrote:

> > I very much hope I do not embarrass the audience but people can we
> > return back to email etiquette? Inline quoting instead of
> > top-posting, proper trimming instead of overquoting, an empty line
> > to separate quoted text from the reply.
>
> Unfortunately, you're fighting against the default behaviour of many
> current email clients. [?] Even the Gmail web client starts with the
> cursor above the quoted text (although it places the signature block
> after it).

But that's the right behaviour: for deleting unnecessary quoted text and
writing inline responses, the natural way to work is from top to bottom;
so starting the cursor at the top is correct.

(only half-joking :-)

> The big pain though is answering email on a (not-so-)smartphone [?]
> (and I'll often choose to wait until I'm back at a machine with a full
> keyboard for exactly that reason).

Yes. If one knows enough to feel the need to apologise for one's email
client, then one also knows enough to wait until a better email client
is available. No apology necessary.

-- 
 \       ?Know what I hate most? Rhetorical questions.? ?Henry N. Camp |
  `\                                                                   |
_o__)                                                                  |
Ben Finney



From alexander.belopolsky at gmail.com  Tue Jan 10 21:56:41 2012
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 10 Jan 2012 15:56:41 -0500
Subject: [Python-ideas] Posting style
In-Reply-To: <87mx9v8g1x.fsf@benfinney.id.au>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
	<87mx9v8g1x.fsf@benfinney.id.au>
Message-ID: <CAP7h-xam0VmadAx62_JmB0U0RQqWYTKUNL0JN5wsvMRD8j7M0Q@mail.gmail.com>

On Tue, Jan 10, 2012 at 3:21 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> The big pain though is answering email on a (not-so-)smartphone [?]
>> (and I'll often choose to wait until I'm back at a machine with a full
>> keyboard for exactly that reason).
>
> Yes. If one knows enough to feel the need to apologise for one's email
> client, then one also knows enough to wait until a better email client
> is available. No apology necessary.

+1

In good old times your usenet client (nn) would say:

This program posts news to thousands of machines throughout the entire
civilized world. Your message will cost the net hundreds if not
thousands of dollars to send everywhere. Please be sure you know what
you are doing. Are you absolutely sure that you want to do this? [y /
n]

The economic realities of the Internet changed since that program was
in wide use, but it is still true that any time you save by getting
your reply out sooner is not worth it if it will cost untold hours to
the global community for the rest of the life of the list archive.

A hint to Gmail users: if you highlight a portion of the message
before hitting reply, only that portion will be quoted.


From guido at python.org  Tue Jan 10 22:36:28 2012
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2012 13:36:28 -0800
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAPkN8xKPZP2cA7OHmu5d76dh3uzomx-8VmfLLQjMfsq2MWStuQ@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAPkN8xKPZP2cA7OHmu5d76dh3uzomx-8VmfLLQjMfsq2MWStuQ@mail.gmail.com>
Message-ID: <CAP7+vJJDc8D6L8p6+1tN=WPsNRceGbbEP1zB76EE=UZv-5jR6A@mail.gmail.com>

On Tue, Jan 10, 2012 at 5:27 AM, anatoly techtonik <techtonik at gmail.com>wrote:

> So, py3k print() is not pythonic. To make it pythonic, the output should
> be immediately available after you call it. If you need to buffer output -
> do this explicitly, because print() doesn't guarantee that it will be
> buffered anyway. If you care about performance - use sys.stdout directly
> and tune it explicitly.


That's utter bullshit.

However, I would be fine with adding a new keyword argument to print() in
3.3 to force a flush, as long as it defaults to off.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120110/96a7b240/attachment.html>

From jxo6948 at rit.edu  Wed Jan 11 00:50:55 2012
From: jxo6948 at rit.edu (John O'Connor)
Date: Tue, 10 Jan 2012 18:50:55 -0500
Subject: [Python-ideas] Posting style
In-Reply-To: <CAP7h-xam0VmadAx62_JmB0U0RQqWYTKUNL0JN5wsvMRD8j7M0Q@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
	<87mx9v8g1x.fsf@benfinney.id.au>
	<CAP7h-xam0VmadAx62_JmB0U0RQqWYTKUNL0JN5wsvMRD8j7M0Q@mail.gmail.com>
Message-ID: <CABCbifW5twh3vLRg3tK_0XKoF_ahfdBkRzhC0_rnDh81maSffA@mail.gmail.com>

On Tue, Jan 10, 2012 at 3:56 PM, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
> A hint to Gmail users: if you highlight a portion of the message
> before hitting reply, only that portion will be quoted.

Great tip!

Note that you will also have to enable it from the Gmail Labs.


From p.f.moore at gmail.com  Wed Jan 11 01:28:44 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 11 Jan 2012 00:28:44 +0000
Subject: [Python-ideas] Posting style
In-Reply-To: <CABCbifW5twh3vLRg3tK_0XKoF_ahfdBkRzhC0_rnDh81maSffA@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
	<87mx9v8g1x.fsf@benfinney.id.au>
	<CAP7h-xam0VmadAx62_JmB0U0RQqWYTKUNL0JN5wsvMRD8j7M0Q@mail.gmail.com>
	<CABCbifW5twh3vLRg3tK_0XKoF_ahfdBkRzhC0_rnDh81maSffA@mail.gmail.com>
Message-ID: <CACac1F8psaAnv=umaQYnOijCZS+8PKCvBE8jRCXf0R5QKSsbmQ@mail.gmail.com>

On 10 January 2012 23:50, John O'Connor <jxo6948 at rit.edu> wrote:
> Note that you will also have to enable it from the Gmail Labs.

Yes, that's the trick :-) Thnks.
Paul


From cs at zip.com.au  Wed Jan 11 01:40:32 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 11 Jan 2012 11:40:32 +1100
Subject: [Python-ideas] Posting style
In-Reply-To: <CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
References: <CADiSq7cyk1rT5UMRhudKg035OEOO2J_xXGiiXh_yoMKBw0bbJA@mail.gmail.com>
Message-ID: <20120111004032.GA30538@cskk.homeip.net>

On 10Jan2012 21:53, Nick Coghlan <ncoghlan at gmail.com> wrote:
| On Tue, Jan 10, 2012 at 9:22 PM, Oleg Broytman <phd at phdru.name> wrote:
| > ? I very much hope I do not embarrass the audience but people can we
| > return back to email etiquette? Inline quoting instead of top-posting,
| > proper trimming instead of overquoting, an empty line to separate quoted
| > text from the reply.
| 
| Unfortunately, you're fighting against the default behaviour of many
| current email clients. [...] Even the Gmail web client starts with the
| cursor above the quoted text [...]

At Google one convention was that this is a cue to start at the top of
the quoted text and trim rubbish aggressively, and inline one's replies.
Of course that is a workflow mindset chosen to overcome this bad UI
choice:-)

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Would you remember a one-line .sig?     - Paul Thompson, thompson at apple.com


From cs at zip.com.au  Wed Jan 11 01:44:43 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 11 Jan 2012 11:44:43 +1100
Subject: [Python-ideas] Posting style
In-Reply-To: <20120110133047.GD21299@iskra.aviel.ru>
References: <20120110133047.GD21299@iskra.aviel.ru>
Message-ID: <20120111004443.GA31682@cskk.homeip.net>

On 10Jan2012 17:30, Oleg Broytman <phd at phdru.name> wrote:
| On Tue, Jan 10, 2012 at 08:13:42AM -0500, Devin Jeanpierre wrote:
| > I don't really understand the objection. My mail client hides unbroken
| > quotes at the bottom of an email, so that I don't see anything except
| > what you quoted (if I read my own email), plus my signature. I take it
| > yours doesn't do this?
| 
|    When I do googling through web-archives I use web browser to read
| what I found. Google, in my experience, produces the most relevant
| results.

It is worth mentioning (since I routinely see it missed it these
discussions) that leaving huge chunks of untrimmed quotes makes it much
harder to find the _relevant_ archived message because of all the
irrelevant keyword hits.

Quote trimming helps put relevant stuff in the email flow and keeps
irrelevant stuff out.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

I swear to god, officer, I'm fixing this bridge. Just go divert traffic.


From greg.ewing at canterbury.ac.nz  Tue Jan 10 23:31:08 2012
From: greg.ewing at canterbury.ac.nz (Greg)
Date: Wed, 11 Jan 2012 11:31:08 +1300
Subject: [Python-ideas] Posting style
In-Reply-To: <CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
Message-ID: <4F0CBC2C.8070901@canterbury.ac.nz>

Devin Jeanpierre wrote:
>>  Sorry for the noise but can I ask at least? Yes, I am an old fart
>>joined the Net at 1991, but I find netiquette really useful for effective
>>communication and deviations from it inconvenient and annoying.
> 
> 
> Is replying like this acceptable?
> 
> -- Devin
> 
> [rest of message quoted here]

Not entirely, because I have to scroll through the rest of the message
anyway if I want to be sure there aren't any further comments in it,
despite the embedded signature.

-- 
Greg


From wuwei23 at gmail.com  Wed Jan 11 02:43:54 2012
From: wuwei23 at gmail.com (alex23)
Date: Tue, 10 Jan 2012 17:43:54 -0800 (PST)
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAPkN8xKPZP2cA7OHmu5d76dh3uzomx-8VmfLLQjMfsq2MWStuQ@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAPkN8xKPZP2cA7OHmu5d76dh3uzomx-8VmfLLQjMfsq2MWStuQ@mail.gmail.com>
Message-ID: <95040435-d494-4507-a5c0-446256eff84e@n6g2000vbz.googlegroups.com>

On Jan 10, 11:27?pm, anatoly techtonik <techto... at gmail.com> wrote:
> To get back on topic, let me give a definition. To me "pythonic" means
> "intuitive" in the first place. And "intuitive" means "up to user
> expectations".

No, what "pythonic" means here is "up to anatoly's expectations".

If I could kill one idea dead about Python it's this persistent belief
that you shouldn't need to read any documentation in order to be able
to use it.

> "intuitive" also means that users won't have to consult user documentation
> for every basic thing it needs

Well, clearly that's not true, as your intuition about print's
behaviour was wrong. Other people's "intuitive" understanding of
mutable default arguments gets them into trouble; you can either "fix"
this behaviour so it's inconsistent with the rest of Python's object
model, or you can recommend they read the docs so their understanding
changes and they know what it means. The latter is the only sane
method, IMO.

Similarly, every time your intuition is wrong  doesn't make it a
regression or bug.


From ncoghlan at gmail.com  Wed Jan 11 02:59:02 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Jan 2012 11:59:02 +1000
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <B07A7173-1553-4EE1-8835-E5EE8E92BBE2@masklinn.net>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
	<jefgvo$ptm$1@dough.gmane.org>
	<CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>
	<BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
	<CAEfz+Ty61fdf6j4XVKO2vr6aXUYXyNJA5vBy24HFzj7-syE2YQ@mail.gmail.com>
	<B07A7173-1553-4EE1-8835-E5EE8E92BBE2@masklinn.net>
Message-ID: <CADiSq7ccAy-feM7UDgPdKivwUHs1owcGw_vnHXnYVn3OvtyoUQ@mail.gmail.com>

On Tue, Jan 10, 2012 at 8:15 PM, Masklinn <masklinn at masklinn.net> wrote:
>> So the keyword argument means
>> "do or don't call flush() method of the file."
> That's not clear from its name since the flushing behavior can depend on the underlying stream type. force_flush would be closer to the actual meaning of the param.

+0 for being able to write print("whatever", force_flush=True) instead
of having to do:

import sys # somewhere in the file
print("whatever")
sys.stdout.flush()

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From python at mrabarnett.plus.com  Wed Jan 11 03:36:23 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 11 Jan 2012 02:36:23 +0000
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CADiSq7ccAy-feM7UDgPdKivwUHs1owcGw_vnHXnYVn3OvtyoUQ@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
	<jefgvo$ptm$1@dough.gmane.org>
	<CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>
	<BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
	<CAEfz+Ty61fdf6j4XVKO2vr6aXUYXyNJA5vBy24HFzj7-syE2YQ@mail.gmail.com>
	<B07A7173-1553-4EE1-8835-E5EE8E92BBE2@masklinn.net>
	<CADiSq7ccAy-feM7UDgPdKivwUHs1owcGw_vnHXnYVn3OvtyoUQ@mail.gmail.com>
Message-ID: <4F0CF5A7.2080504@mrabarnett.plus.com>

On 11/01/2012 01:59, Nick Coghlan wrote:
> On Tue, Jan 10, 2012 at 8:15 PM, Masklinn<masklinn at masklinn.net>
> wrote:
>>> So the keyword argument means "do or don't call flush() method of
>>> the file."
>> That's not clear from its name since the flushing behavior can
>> depend on the underlying stream type. force_flush would be closer
>> to the actual meaning of the param.
>
> +0 for being able to write print("whatever", force_flush=True)
> instead of having to do:
>
> import sys # somewhere in the file
 > print("whatever")
> sys.stdout.flush()
>
Given that 'print' is a function in Python 3, it's easy to redefine it
to flush. I've done it on occasion.


From guido at python.org  Wed Jan 11 04:14:11 2012
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2012 19:14:11 -0800
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <4F0CF5A7.2080504@mrabarnett.plus.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
	<jefgvo$ptm$1@dough.gmane.org>
	<CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>
	<BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
	<CAEfz+Ty61fdf6j4XVKO2vr6aXUYXyNJA5vBy24HFzj7-syE2YQ@mail.gmail.com>
	<B07A7173-1553-4EE1-8835-E5EE8E92BBE2@masklinn.net>
	<CADiSq7ccAy-feM7UDgPdKivwUHs1owcGw_vnHXnYVn3OvtyoUQ@mail.gmail.com>
	<4F0CF5A7.2080504@mrabarnett.plus.com>
Message-ID: <CAP7+vJ+8sqLuF+KJ=GMV9ZUvcf5pA+KbnEx_d4WctVBL9LxRmA@mail.gmail.com>

On Tue, Jan 10, 2012 at 6:36 PM, MRAB <python at mrabarnett.plus.com> wrote:

> On 11/01/2012 01:59, Nick Coghlan wrote:
>
>> On Tue, Jan 10, 2012 at 8:15 PM, Masklinn<masklinn at masklinn.net**>
>> wrote:
>>
>>> So the keyword argument means "do or don't call flush() method of
>>>> the file."
>>>>
>>> That's not clear from its name since the flushing behavior can
>>> depend on the underlying stream type. force_flush would be closer
>>> to the actual meaning of the param.
>>>
>>
>> +0 for being able to write print("whatever", force_flush=True)
>> instead of having to do:
>>
>> import sys # somewhere in the file
>>
> > print("whatever")
>
>> sys.stdout.flush()
>>
>>  Given that 'print' is a function in Python 3, it's easy to redefine it
> to flush. I've done it on occasion.
>

Heh, that's exactly why we made it a function. So is adding e.g. a
force_flush flag -- though personally I would be fine calling it "flush"
despite the possibility that the underlying stream might still flush when
it is not explicitly requests.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120110/d3fb2ad0/attachment.html>

From python at mrabarnett.plus.com  Wed Jan 11 04:47:21 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 11 Jan 2012 03:47:21 +0000
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAP7+vJ+8sqLuF+KJ=GMV9ZUvcf5pA+KbnEx_d4WctVBL9LxRmA@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAMpsgwb_EoVaCVAeEczPQyiG_rPDY=2p1YMbEvHXaZO=5AW4qA@mail.gmail.com>
	<CAP7+vJKFj5n6WSi5eRcBE2mj4230BDWfPSHFtsz43J+53fBX3A@mail.gmail.com>
	<jefgvo$ptm$1@dough.gmane.org>
	<CAEfz+Tx6AQ_x78tG+8AEqCtVMuqDWbMCDtWfcDAAh2KYUkvc0A@mail.gmail.com>
	<BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
	<CAEfz+Ty61fdf6j4XVKO2vr6aXUYXyNJA5vBy24HFzj7-syE2YQ@mail.gmail.com>
	<B07A7173-1553-4EE1-8835-E5EE8E92BBE2@masklinn.net>
	<CADiSq7ccAy-feM7UDgPdKivwUHs1owcGw_vnHXnYVn3OvtyoUQ@mail.gmail.com>
	<4F0CF5A7.2080504@mrabarnett.plus.com>
	<CAP7+vJ+8sqLuF+KJ=GMV9ZUvcf5pA+KbnEx_d4WctVBL9LxRmA@mail.gmail.com>
Message-ID: <4F0D0649.6020400@mrabarnett.plus.com>

On 11/01/2012 03:14, Guido van Rossum wrote:
> On Tue, Jan 10, 2012 at 6:36 PM, MRAB <python at mrabarnett.plus.com
> <mailto:python at mrabarnett.plus.com>> wrote:
>
>     On 11/01/2012 01:59, Nick Coghlan wrote:
>
>         On Tue, Jan 10, 2012 at 8:15 PM, Masklinn<masklinn at masklinn.net
>         <mailto:masklinn at masklinn.net>__>
>         wrote:
>
>                 So the keyword argument means "do or don't call flush()
>                 method of
>                 the file."
>
>             That's not clear from its name since the flushing behavior can
>             depend on the underlying stream type. force_flush would be
>             closer
>             to the actual meaning of the param.
>
>
>         +0 for being able to write print("whatever", force_flush=True)
>         instead of having to do:
>
>         import sys # somewhere in the file
>
>      > print("whatever")
>
>         sys.stdout.flush()
>
>     Given that 'print' is a function in Python 3, it's easy to redefine it
>     to flush. I've done it on occasion.
>
>
> Heh, that's exactly why we made it a function. So is adding e.g. a
> force_flush flag -- though personally I would be fine calling it "flush"
> despite the possibility that the underlying stream might still flush
> when it is not explicitly requests.
>
Even a buffered stream flushes sometime. :-)


From stephen at xemacs.org  Wed Jan 11 06:27:30 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 11 Jan 2012 14:27:30 +0900
Subject: [Python-ideas] Posting style
In-Reply-To: <CABicbJ+_ZOnbDiwn+E+54Brbktv6s3Y8wMaOy5UrNZgA9c+OuA@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
	<20120110124354.GC21299@iskra.aviel.ru>
	<CABicbJ+_ZOnbDiwn+E+54Brbktv6s3Y8wMaOy5UrNZgA9c+OuA@mail.gmail.com>
Message-ID: <87hb02ajwd.fsf@uwakimon.sk.tsukuba.ac.jp>

Devin Jeanpierre writes:

 > I don't really understand the objection. My mail client hides unbroken
 > quotes at the bottom of an email, so that I don't see anything except
 > what you quoted (if I read my own email), plus my signature. I take it
 > yours doesn't do this?

Mine does not, and I like it that way, because almost all the mail I
receive is either very short threads where it doesn't matter, or
related to fora where the vast majority of participants follow the
standard netiquette conventions of interlinear response and fairly
aggressive trimming.  It is often the case that the trailing text is
useful for establishing context (because if it wasn't the poster would
have trimmed it).

I believe that the top-posting style works well in business because
threads are either short, or almost everybody is almost always on the
same page anyway, and occasionally refers to something buried deeper
in the thread.  Also, in most cases any conflicts will be settled more
or less arbitrarily by the boss's intuition, so extended analysis with
precise rebuttals is rare.

It sucks in technical discussion, because people have interesting
things to say (aka, conflicting analyses that go beyond Snappy Answers
to Stupid Questions one-liners).



From cs at zip.com.au  Wed Jan 11 07:15:31 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 11 Jan 2012 17:15:31 +1100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
References: <BDE209E9-03D4-46E4-9325-927E5F725AE8@gmail.com>
Message-ID: <20120111061531.GA2804@cskk.homeip.net>

On 09Jan2012 22:49, Carl M. Johnson <cmjohnson.mailinglist at gmail.com> wrote:
| On Jan 9, 2012, at 7:52 PM, INADA Naoki wrote:
| > What about adding flush=False keyword argument to print()?
| 
| I would expect flush=False to mean "please don't flush this right now,
| even if I do use a newline."
| 
| flush=None?

+1 from me, if None means "honour the file's default flushing
behaviour".
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Since I've mentioned the subject of geneology, I'll repeat a story I heard
about a poor fellow over on airstrip one.  Seems he spent the most recent
thirty years of his life tracking down his family history.  Spent hundreds
of pounds, traveled, devoted his life to it.  Then, last month, a cousin
told him he was adopted.  Ahhh, sweet irony.
        - Tim_Mefford <tim at physics.orst.edu>


From g.brandl at gmx.net  Wed Jan 11 09:28:25 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 11 Jan 2012 09:28:25 +0100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAP7+vJJDc8D6L8p6+1tN=WPsNRceGbbEP1zB76EE=UZv-5jR6A@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAPkN8xKPZP2cA7OHmu5d76dh3uzomx-8VmfLLQjMfsq2MWStuQ@mail.gmail.com>
	<CAP7+vJJDc8D6L8p6+1tN=WPsNRceGbbEP1zB76EE=UZv-5jR6A@mail.gmail.com>
Message-ID: <jejh0q$clv$1@dough.gmane.org>

Am 10.01.2012 22:36, schrieb Guido van Rossum:
> On Tue, Jan 10, 2012 at 5:27 AM, anatoly techtonik <techtonik at gmail.com
> <mailto:techtonik at gmail.com>> wrote:
> 
>     So, py3k print() is not pythonic. To make it pythonic, the output should be
>     immediately available after you call it. If you need to buffer output - do
>     this explicitly, because print() doesn't guarantee that it will be buffered
>     anyway. If you care about performance - use sys.stdout directly and tune it
>     explicitly. 
> 
> 
> That's utter bullshit.
> 
> However, I would be fine with adding a new keyword argument to print() in 3.3 to
> force a flush, as long as it defaults to off.

I've created an issue to track this:

http://bugs.python.org/issue13761

Georg



From victor.stinner at haypocalc.com  Wed Jan 11 09:46:24 2012
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 11 Jan 2012 09:46:24 +0100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <jejh0q$clv$1@dough.gmane.org>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAPkN8xKPZP2cA7OHmu5d76dh3uzomx-8VmfLLQjMfsq2MWStuQ@mail.gmail.com>
	<CAP7+vJJDc8D6L8p6+1tN=WPsNRceGbbEP1zB76EE=UZv-5jR6A@mail.gmail.com>
	<jejh0q$clv$1@dough.gmane.org>
Message-ID: <CAMpsgwbw4V-67cztw9XsYKfKDoWp-X=Ka66ryWO6Yx_boKoXKA@mail.gmail.com>

There is already an option for flush: -u command line option. Can't we
change print() to flush if end is set and -u option is used, instead
of introducing a new option?

I understand that a command line option is less practical than a new
keyword, especially on Windows.


From techtonik at gmail.com  Wed Jan 11 14:55:04 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 11 Jan 2012 16:55:04 +0300
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <20120110143156.1188db76@pitrou.net>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
Message-ID: <CAPkN8x+N9ck99pbJXQXEU=TWy6FRwqe-qHxiwRA7+PnmMZSxzA@mail.gmail.com>

On Tue, Jan 10, 2012 at 4:31 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Tue, 10 Jan 2012 16:16:09 +0300
> anatoly techtonik <techtonik at gmail.com>
> wrote:
> >
> > So I propose to have a summary for each thread in Etherpad -
> > http://en.wikipedia.org/wiki/Etherpad - that will provide current
> current
> > summary of email thread, describing original problem, its evolution,
> > arguments and counter-arguments.
>
> Are you willing to maintain it? Feel free to create and update etherpad
> summaries for a few of our current discussion threads, so that we see
> if that's useful.
>

The idea is about collaborative service, not a  "community job offer" for
single person, responsible to maintain and update summaries for email
threads on some wiki page. The architecture should be inspiring for people
to provide summaries when they start to find threads confusing, need to
organize ideas around the threads or keep the discussion focused.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120111/dee2a9b1/attachment.html>

From masklinn at masklinn.net  Wed Jan 11 15:02:58 2012
From: masklinn at masklinn.net (Masklinn)
Date: Wed, 11 Jan 2012 15:02:58 +0100
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <CAPkN8x+N9ck99pbJXQXEU=TWy6FRwqe-qHxiwRA7+PnmMZSxzA@mail.gmail.com>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
	<CAPkN8x+N9ck99pbJXQXEU=TWy6FRwqe-qHxiwRA7+PnmMZSxzA@mail.gmail.com>
Message-ID: <894BC1E9-5D57-4702-A758-45E29A1F2AF5@masklinn.net>

On 2012-01-11, at 14:55 , anatoly techtonik wrote:
> On Tue, Jan 10, 2012 at 4:31 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
>> On Tue, 10 Jan 2012 16:16:09 +0300
>> anatoly techtonik <techtonik at gmail.com>
>> wrote:
>>> 
>>> So I propose to have a summary for each thread in Etherpad -
>>> http://en.wikipedia.org/wiki/Etherpad - that will provide current
>> current
>>> summary of email thread, describing original problem, its evolution,
>>> arguments and counter-arguments.
>> 
>> Are you willing to maintain it? Feel free to create and update etherpad
>> summaries for a few of our current discussion threads, so that we see
>> if that's useful.
>> 
> 
> The idea is about collaborative service, not a  "community job offer" for
> single person, responsible to maintain and update summaries for email
> threads on some wiki page.

That does not preclude maintaining:
1. Somebody needs to handle the hardware and software stack, check that the load is acceptable, test software updates and deploy them, handle ACLs if any, ?
2. Information is not going to self-organize over the whole platform, people will need to keep track of the various documents being created and provide an entry point for the service (linking to various sub-trees)
3. Open-edition systems (wikis, etherpad instances, ?) are subject to vandalism, spam and edit wars, there needs to be people and tools able to manage both issues, cleanup the crap in the first two cases and potentially temporarily lock documents in the latter.

You're the one who brought it up and suggested it would be a valuable tool, the onus is on you to demonstrate it and it makes sense that you'd be the one maintaining it until then.

Hence Antoine's question/suggestion.

>  The architecture should be inspiring for people
> to provide summaries when they start to find threads confusing, need to
> organize ideas around the threads or keep the discussion focused.

Good luck with that.

From manuel at enigmage.de  Wed Jan 11 16:08:20 2012
From: manuel at enigmage.de (=?ISO-8859-1?Q?Manuel_B=E4renz?=)
Date: Wed, 11 Jan 2012 16:08:20 +0100
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
Message-ID: <4F0DA5E4.40308@enigmage.de>

I propose two new control flows for Python:

"while ... try":

while expr try:
    suite1
except SomeException:
    suite2
else:
    suite3

This executes suite1 as long as handled exceptions are thrown and expr
is True.
* If an unhandled exception is thrown, it passes the exception on to the
surrounding or the stack.
* If no exception occurs, life goes on as normal, suite3 is executed and
execution goes on afterwards.

The control flow is thus equivalent to:

while expr:
    try:
        suite1
    except SomeException:
        suite2
    else:
        suite3
        break

But it's neater, very natural (in my opinion) and saves an indentation
level.

One further enhancement: If expr is encountered to be False, some
special exception "NoMoreTriesException" could be raised. It can be
catched in the same "while ... try" block.

Usecase:

while network_is_up() try:
    connect_to_server()
except ConnectError:
    time.sleep(timeout)
except NoMoreTriesException:
    print("Couldn't establish connection")
else:
    download_stuff()
finally:
    make_sure_resource_is_freed()

Another usecase:

while receive_packet() try:
    check_packet()
except ChecksumError:
    print("You sent the wrong thing. Try again.")
except NoMoreTriesException:
    print("I couldn't get a single useful packet from you :(")
else:
    process_packet()
finally:
    close_connection()


A similar thing could be made with "for ... try":

for password in passwords_i_remember try:
    connect(password)
except WrongPassError:
    pass # No pun intended
except NoMoreTriesException:
    print("Not a single one worked.")
else:
    check_mailbox()

The advantages are the same as for "while ... try".

Cheers, Manuel


From anacrolix at gmail.com  Wed Jan 11 16:11:18 2012
From: anacrolix at gmail.com (Matt Joiner)
Date: Thu, 12 Jan 2012 02:11:18 +1100
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <4F0DA5E4.40308@enigmage.de>
References: <4F0DA5E4.40308@enigmage.de>
Message-ID: <CAB4yi1ORtxGtatRpmai0dXBV8SJtAUCEttOSWpb_gCvVPgMDUQ@mail.gmail.com>

I don't like it, you're merging 2 statements at the expense of readability.

It's like egyptian exception handling. -1

2012/1/12 Manuel B?renz <manuel at enigmage.de>:
> I propose two new control flows for Python:
>
> "while ... try":
>
> while expr try:
> ? ?suite1
> except SomeException:
> ? ?suite2
> else:
> ? ?suite3
>
> This executes suite1 as long as handled exceptions are thrown and expr
> is True.
> * If an unhandled exception is thrown, it passes the exception on to the
> surrounding or the stack.
> * If no exception occurs, life goes on as normal, suite3 is executed and
> execution goes on afterwards.
>
> The control flow is thus equivalent to:
>
> while expr:
> ? ?try:
> ? ? ? ?suite1
> ? ?except SomeException:
> ? ? ? ?suite2
> ? ?else:
> ? ? ? ?suite3
> ? ? ? ?break
>
> But it's neater, very natural (in my opinion) and saves an indentation
> level.
>
> One further enhancement: If expr is encountered to be False, some
> special exception "NoMoreTriesException" could be raised. It can be
> catched in the same "while ... try" block.
>
> Usecase:
>
> while network_is_up() try:
> ? ?connect_to_server()
> except ConnectError:
> ? ?time.sleep(timeout)
> except NoMoreTriesException:
> ? ?print("Couldn't establish connection")
> else:
> ? ?download_stuff()
> finally:
> ? ?make_sure_resource_is_freed()
>
> Another usecase:
>
> while receive_packet() try:
> ? ?check_packet()
> except ChecksumError:
> ? ?print("You sent the wrong thing. Try again.")
> except NoMoreTriesException:
> ? ?print("I couldn't get a single useful packet from you :(")
> else:
> ? ?process_packet()
> finally:
> ? ?close_connection()
>
>
> A similar thing could be made with "for ... try":
>
> for password in passwords_i_remember try:
> ? ?connect(password)
> except WrongPassError:
> ? ?pass # No pun intended
> except NoMoreTriesException:
> ? ?print("Not a single one worked.")
> else:
> ? ?check_mailbox()
>
> The advantages are the same as for "while ... try".
>
> Cheers, Manuel
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From manuel at enigmage.de  Wed Jan 11 16:17:31 2012
From: manuel at enigmage.de (=?UTF-8?B?TWFudWVsIELDpHJlbno=?=)
Date: Wed, 11 Jan 2012 16:17:31 +0100
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <CAB4yi1ORtxGtatRpmai0dXBV8SJtAUCEttOSWpb_gCvVPgMDUQ@mail.gmail.com>
References: <4F0DA5E4.40308@enigmage.de>
	<CAB4yi1ORtxGtatRpmai0dXBV8SJtAUCEttOSWpb_gCvVPgMDUQ@mail.gmail.com>
Message-ID: <4F0DA80B.3060901@enigmage.de>

On 11.01.2012 16:11, Matt Joiner wrote:
> I don't like it, you're merging 2 statements at the expense of readability.
You might find it unreadable because you're not used to it. What is so
unreadable about it? I find it more readable because it's flat.
> It's like egyptian exception handling. -1
What is that supposed to mean?


From techtonik at gmail.com  Wed Jan 11 16:45:33 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 11 Jan 2012 18:45:33 +0300
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <CAB4yi1NfJvWkCiknuvbE49K8+31LBB2qswBhwWzug723H7oOcQ@mail.gmail.com>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
	<CAB4yi1NfJvWkCiknuvbE49K8+31LBB2qswBhwWzug723H7oOcQ@mail.gmail.com>
Message-ID: <CAPkN8x+XVbYqiG6xmzn689Y9G-M_NaDKBpOfq7463120TAshMA@mail.gmail.com>

Created a page for the current thread:
http://piratepad.net/python-ideas-an-etherpad-for-every-idea

No clone of Etherpad sources. The best way would be to start with Etherpad
Lite http://etherpad.org/2012/01/02/a-short-etherpad-lite-talk/ if we are
talking about implementation.
-- 
anatoly t.


On Tue, Jan 10, 2012 at 8:08 PM, Matt Joiner <anacrolix at gmail.com> wrote:

> Are these already in an etherpad clone or...?
>
> On Wed, Jan 11, 2012 at 12:31 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > On Tue, 10 Jan 2012 16:16:09 +0300
> > anatoly techtonik <techtonik at gmail.com>
> > wrote:
> >>
> >> So I propose to have a summary for each thread in Etherpad -
> >> http://en.wikipedia.org/wiki/Etherpad - that will provide current
> current
> >> summary of email thread, describing original problem, its evolution,
> >> arguments and counter-arguments.
> >
> > Are you willing to maintain it? Feel free to create and update etherpad
> > summaries for a few of our current discussion threads, so that we see
> > if that's useful.
> >
> > Regards
> >
> > Antoine.
> >
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > http://mail.python.org/mailman/listinfo/python-ideas
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120111/d554288d/attachment.html>

From stefan_ml at behnel.de  Wed Jan 11 17:03:34 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 11 Jan 2012 17:03:34 +0100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CAMpsgwbw4V-67cztw9XsYKfKDoWp-X=Ka66ryWO6Yx_boKoXKA@mail.gmail.com>
References: <CAPkN8x+Kpm8KQcU-Fk1dzS5RYEL8xiMQQ+UtzVcALD-Mn-nvFw@mail.gmail.com>
	<CABicbJJHq9H97=yD8+B4iJS1h+BYm5+yS10cKBUcBd0F5vq_KA@mail.gmail.com>
	<CAPkN8x+A0-apR+i0AGyG=awsNuaqbAmwJcgmugKX_hA4zvjT3w@mail.gmail.com>
	<CABicbJLEQs8iqs=MrQyGEtctmz58NqajxVFm3qh_vbJRBTXazw@mail.gmail.com>
	<CAPkN8xKPZP2cA7OHmu5d76dh3uzomx-8VmfLLQjMfsq2MWStuQ@mail.gmail.com>
	<CAP7+vJJDc8D6L8p6+1tN=WPsNRceGbbEP1zB76EE=UZv-5jR6A@mail.gmail.com>
	<jejh0q$clv$1@dough.gmane.org>
	<CAMpsgwbw4V-67cztw9XsYKfKDoWp-X=Ka66ryWO6Yx_boKoXKA@mail.gmail.com>
Message-ID: <jekbsm$oal$1@dough.gmane.org>

Victor Stinner, 11.01.2012 09:46:
> There is already an option for flush: -u command line option. Can't we
> change print() to flush if end is set and -u option is used, instead
> of introducing a new option?
> 
> I understand that a command line option is less practical than a new
> keyword, especially on Windows.

For one, it doesn't allow the program code to explicitly enforce the flush
at a specific point if it depends on a command line switch.

Note that the caller of print() may not even know what "print" really is at
a given point in the execution, and what (if any) output stream it will
write to. Doesn't have to be sys.stdout. So a new keyword argument makes
sense to me (and I agree that calling it "flush" should be enough).

Stefan



From techtonik at gmail.com  Wed Jan 11 17:30:21 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 11 Jan 2012 19:30:21 +0300
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <894BC1E9-5D57-4702-A758-45E29A1F2AF5@masklinn.net>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
	<CAPkN8x+N9ck99pbJXQXEU=TWy6FRwqe-qHxiwRA7+PnmMZSxzA@mail.gmail.com>
	<894BC1E9-5D57-4702-A758-45E29A1F2AF5@masklinn.net>
Message-ID: <CAPkN8xKARo29R=aA1GmyfCcBwLnD8fNa+6sUBkgarXiCsBpY4A@mail.gmail.com>

On Wed, Jan 11, 2012 at 5:02 PM, Masklinn <masklinn at masklinn.net> wrote:

> On 2012-01-11, at 14:55 , anatoly techtonik wrote:
> > On Tue, Jan 10, 2012 at 4:31 PM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> >
> >> On Tue, 10 Jan 2012 16:16:09 +0300
> >> anatoly techtonik <techtonik at gmail.com>
> >> wrote:
> >>>
> >>> So I propose to have a summary for each thread in Etherpad -
> >>> http://en.wikipedia.org/wiki/Etherpad - that will provide current
> >> current
> >>> summary of email thread, describing original problem, its evolution,
> >>> arguments and counter-arguments.
> >>
> >> Are you willing to maintain it? Feel free to create and update etherpad
> >> summaries for a few of our current discussion threads, so that we see
> >> if that's useful.
> >>
> >
> > The idea is about collaborative service, not a  "community job offer" for
> > single person, responsible to maintain and update summaries for email
> > threads on some wiki page.
>
> That does not preclude maintaining:

1. Somebody needs to handle the hardware and software stack, check that the
> load is acceptable, test software updates and deploy them, handle ACLs if
> any, ?
>

Does that mean that the idea is good enough to start talking about problems
with infrastructure and maintenance? =)


> 2. Information is not going to self-organize over the whole platform,
> people will need to keep track of the various documents being created and
> provide an entry point for the service (linking to various sub-trees)
>

That's right. The idea is to provide an automatic service tied to each
email thread, meaning that email threads must be identified first. Then
- you're right - there must be a web interface of the entry point
that lists threads with pointers to the corresponding Etherpad page. This
can further list auxiliary flags, for example if thread was updated after
the last change in Etherpad. Etherpad interface then can be tweaked further
to implement "approval" system that let's every user place a "tick" next to
the reference to a message that the user believes is integrated into
summary correctly (or flag messages that are completely missing).


> 3. Open-edition systems (wikis, etherpad instances, ?) are subject to
> vandalism, spam and edit wars, there needs to be people and tools able to
> manage both issues, cleanup the crap in the first two cases and potentially
> temporarily lock documents in the latter.
>

In this case it is possible to limit access to python.org accounts. Is it
possible? Is there a central login server for python.org?


> You're the one who brought it up and suggested it would be a valuable
> tool, the onus is on you to demonstrate it and it makes sense that you'd be
> the one maintaining it until then.
>

I only proposed the idea. If you think this tool won't be valuable then I
am open to discuss why. If it is only because I failed to make it a
valuable tool then I can make it even more obvious by saying - yes, I will
not be able to create a valuable tool out of it alone.
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120111/10f8486f/attachment.html>

From jimjjewett at gmail.com  Wed Jan 11 17:58:40 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 11 Jan 2012 11:58:40 -0500
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <4F0DA5E4.40308@enigmage.de>
References: <4F0DA5E4.40308@enigmage.de>
Message-ID: <CA+OGgf4uxvpPrVSS-MjYMs2R1WX=OOXk+-v+p-nxDfBnnEsT6Q@mail.gmail.com>

For me, control flow matching indentation is probably the single most
important feature of python.

What I would support is a more convenient way to inject an exception
handler into an expression.  For example, (a more complicated version
of) your first use case would be improved by a way to say

    (except ConnectError) ==>?(time.sleep(timeout); continue)

and your password checker example would be improved by a way to say

    (except WrongPassError) ==> (pass)

More details below.

2012/1/11 Manuel B?renz <manuel at enigmage.de>:

> while expr try:
> ? ?suite1
> except SomeException:
> ? ?suite2
> else:
> ? ?suite3

Without your explanation, I would still have assumed that the
exception terminated the while.


> while network_is_up() try:
> ? ?connect_to_server()
> except ConnectError:
> ? ?time.sleep(timeout)
> except NoMoreTriesException:
> ? ?print("Couldn't establish connection")
> else:
> ? ?download_stuff()
> finally:
> ? ?make_sure_resource_is_freed()

This would be a lot easier for me to follow with the extra indents.

As best I can tell, under normal circumstances, that will just keep
making new connections, and never get to the download until the
network itself goes down.  Or does the else bind to the try, rather
than the while -- in which case it is still an infinite loop, but at
least succeeds.  I'm also not sure which resource needs to be freed,
so I'll just call it special_resource.  For what I think you wanted, I
can read it a lot easier as:

    with special_resource:
        for keep_trying in range(3):  # not infinity
            if not network_is_up():    # I would prefer this elsewhere ...
                time.sleep(timeout)
                continue
            try connect_to_server():
                download_stuff()
                break
            except ConnectError:
                time.sleep(timeout)
        else:
            print("Couldn't establish connection")

which I would in turn prefer to reduce to:

    retry=WaitPolicy('sleep_and_retry', 3)

    with ensure_network(onfail=retry) as network:
        with connect_to_server(transport=network, onfail=retry):
            download_stuff()

> while receive_packet() try:
> ? ?check_packet()
> except ChecksumError:
> ? ?print("You sent the wrong thing. Try again.")
> except NoMoreTriesException:
> ? ?print("I couldn't get a single useful packet from you :(")
> else:
> ? ?process_packet()
> finally:
> ? ?close_connection()

This time, it looks like only the final packet gets processed.  If
else binds to the try, it makes more sense, but still complains about
not getting a single useful packet even after getting and processing a
dozen.  I would prefer:

    with connection:
        while receive_packet():
            if OK == check_packet():
                process_packet()
            else:
                print("You sent the wrong thing. Try again.")

or at least (following your API more closely):

    with connection:
        got_any = False
        while receive_packet():
            try check_packet():
                process_packet()
                got_any = True
            except ChecksumError:
                print("You sent the wrong thing. Try again.")
        if not got_any:
            print("I couldn't get a single useful packet from you :(")

> A similar thing could be made with "for ... try":
>
> for password in passwords_i_remember try:
> ? ?connect(password)
> except WrongPassError:
> ? ?pass # No pun intended
> except NoMoreTriesException:
> ? ?print("Not a single one worked.")
> else:
> ? ?check_mailbox()

I honestly don't see that as an improvement over


for password in passwords_i_remember:
    try?connect(password):
 ? ?    check_mailbox()
        break
    except WrongPassError:
 ? ?    pass # No pun intended
else:
 ? ?print("Not a single one worked.")


From arnodel at gmail.com  Wed Jan 11 18:06:58 2012
From: arnodel at gmail.com (Arnaud Delobelle)
Date: Wed, 11 Jan 2012 17:06:58 +0000
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <4F0DA5E4.40308@enigmage.de>
References: <4F0DA5E4.40308@enigmage.de>
Message-ID: <CAJ6cK1ZMCdvgBsVvH818ozPt77C54hgBKcA5vo1RPY+wYfcU4w@mail.gmail.com>

2012/1/11 Manuel B?renz <manuel at enigmage.de>:
> I propose two new control flows for Python:
>
> "while ... try":
>
> while expr try:
> ? ?suite1
> except SomeException:
> ? ?suite2
> else:
> ? ?suite3
>
> This executes suite1 as long as handled exceptions are thrown and expr
> is True.
> * If an unhandled exception is thrown, it passes the exception on to the
> surrounding or the stack.
> * If no exception occurs, life goes on as normal, suite3 is executed and
> execution goes on afterwards.
>
> The control flow is thus equivalent to:
>
> while expr:
> ? ?try:
> ? ? ? ?suite1
> ? ?except SomeException:
> ? ? ? ?suite2
> ? ?else:
> ? ? ? ?suite3
> ? ? ? ?break

Aside from anything else, there is the problem that

while:
    suite
else:
    suite

is already valid Python

Your proposed syntax changes the semantics of "else" after "while"
(and the same applies to "for ...  else".

-- 
Arnaud


From jimjjewett at gmail.com  Wed Jan 11 18:21:05 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 11 Jan 2012 12:21:05 -0500
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <CAPkN8xKARo29R=aA1GmyfCcBwLnD8fNa+6sUBkgarXiCsBpY4A@mail.gmail.com>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
	<CAPkN8x+N9ck99pbJXQXEU=TWy6FRwqe-qHxiwRA7+PnmMZSxzA@mail.gmail.com>
	<894BC1E9-5D57-4702-A758-45E29A1F2AF5@masklinn.net>
	<CAPkN8xKARo29R=aA1GmyfCcBwLnD8fNa+6sUBkgarXiCsBpY4A@mail.gmail.com>
Message-ID: <CA+OGgf6vXjS89C=xQ-OfsfvgUmGcGQK0dvrj63X+zTp=oCphtw@mail.gmail.com>

Quick Summary:  There used to be python-dev summaries.  They were
great, but too much work.

>> >> On Tue, 10 Jan 2012 16:16:09 +0300
>> >> anatoly techtonik <techtonik at gmail.com> wrote:

>> >>> So I propose to have a summary for each thread in Etherpad -
>> >>> http://en.wikipedia.org/wiki/Etherpad - that will provide current
>> >>> current summary of email thread, describing original problem,
>> >>> its evolution, arguments and counter-arguments.

>> > On Tue, Jan 10, 2012 at 4:31 PM, Antoine Pitrou <solipsis at pitrou.net>
>> > wrote:

>> >> Are you willing to maintain it? Feel free to create and update
>> >> etherpad summaries for a few of our current discussion threads,
>> >> so that we see if that's useful.

On Wed, Jan 11, 2012 at 11:30 AM, anatoly techtonik <techtonik at gmail.com> wrote:

> I only proposed the idea. If you think this tool won't be?valuable?then
> I am open to discuss why.

(1)  Going to a separate website -- let alone one not at python.org --
is a pain.
(2)  Creating and maintaining the summaries is a pain.

For individual discussions, someone will occasionally post a summary,
or even write up a PEP.  For the discussions where that doesn't
happen, perhaps no one is both sufficiently overwhelmed to need a
summary and sufficiently motivated to provide one.

Python-ideas was spun off from python-dev.  For the python-dev mailing
list as a whole, various people have provided biweekly summaries in
the past.  The summaries were quite helpful.  Alas, they were almost
never complete, and they all soon slipped into "late" and then got
discontinued, because they were too much work to create.

Why won't the same thing happen again?  Is there something about the
Etherpad platform that makes the job much easier?

-jJ


From ethan at stoneleaf.us  Wed Jan 11 17:47:49 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 11 Jan 2012 08:47:49 -0800
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <CAB4yi1ORtxGtatRpmai0dXBV8SJtAUCEttOSWpb_gCvVPgMDUQ@mail.gmail.com>
References: <4F0DA5E4.40308@enigmage.de>
	<CAB4yi1ORtxGtatRpmai0dXBV8SJtAUCEttOSWpb_gCvVPgMDUQ@mail.gmail.com>
Message-ID: <4F0DBD35.2000309@stoneleaf.us>

Matt Joiner wrote:

[snip]

Please trim your replies.

~Ethan~


From ctb at msu.edu  Wed Jan 11 18:43:53 2012
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 11 Jan 2012 09:43:53 -0800
Subject: [Python-ideas] Netiquette issues (was Re: "while ... try" -
	block	or "for ... try" - block)
In-Reply-To: <4F0DBD35.2000309@stoneleaf.us>
References: <4F0DA5E4.40308@enigmage.de>
	<CAB4yi1ORtxGtatRpmai0dXBV8SJtAUCEttOSWpb_gCvVPgMDUQ@mail.gmail.com>
	<4F0DBD35.2000309@stoneleaf.us>
Message-ID: <20120111174353.GC13614@idyll.org>

Folks,

I'm a moderator.  Volunteer, true, but a moderator nonetheless.  And
I've had enough.

So:

Netiquette discussions are always fascinating, but please, let's stop talking
about how other people should post.  Most of my e-mail from this list over the
last few days has been from people on this list nattering about how gmail foo
and mailer blah, and it is OFF TOPIC and NO LONGER WELCOME.  (Frankly, you
should have figured this out for yourselves.)

If you feel absolutely compelled to tell people that they are lousy netizens
and/or correct them on their posting style, please do so PRIVATELY and not
on this list.

Please e-mail me PRIVATELY if you disagree with this.  Brett Cannon is the
other list admin, and you should feel free to tell him how much you dislike
me & my overbearing moderator ways.

Further e-mails on netiquette will result in those people no longer being
able to post without moderation.

thanks,
--titus

p.s. This is not aimed especially at Ethan -- it was just the latest e-mail
that pushed me over the edge :)
-- 
C. Titus Brown, ctb at msu.edu


From ubershmekel at gmail.com  Wed Jan 11 21:00:38 2012
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Wed, 11 Jan 2012 22:00:38 +0200
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <CA+OGgf6vXjS89C=xQ-OfsfvgUmGcGQK0dvrj63X+zTp=oCphtw@mail.gmail.com>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
	<CAPkN8x+N9ck99pbJXQXEU=TWy6FRwqe-qHxiwRA7+PnmMZSxzA@mail.gmail.com>
	<894BC1E9-5D57-4702-A758-45E29A1F2AF5@masklinn.net>
	<CAPkN8xKARo29R=aA1GmyfCcBwLnD8fNa+6sUBkgarXiCsBpY4A@mail.gmail.com>
	<CA+OGgf6vXjS89C=xQ-OfsfvgUmGcGQK0dvrj63X+zTp=oCphtw@mail.gmail.com>
Message-ID: <CANSw7KwWkJ0QKZYiUfnMJocVbF6nNDiVfL3njYdB6S7fVN_5sQ@mail.gmail.com>

E.g. for this discussion: http://openetherpad.org/JlL56IJFYo

The farthest this idea can go in my opinion is:

1. Someone makes a bot that auto posts the etherpads to each idea in
python-ideas.
or
2. Whoever cares enough about an idea will make an etherpad for it if
he/she likes to.

I don't mind experimenting though.

Yuval Greenfield
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120111/e5c79e38/attachment.html>

From cs at zip.com.au  Thu Jan 12 00:02:51 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 12 Jan 2012 10:02:51 +1100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <CACac1F-7mkBhoSbEOgMEo7j25QxivOW26597f254Qgeb-wHe-Q@mail.gmail.com>
References: <CACac1F-7mkBhoSbEOgMEo7j25QxivOW26597f254Qgeb-wHe-Q@mail.gmail.com>
Message-ID: <20120111230251.GA23059@cskk.homeip.net>

[ Taken back on list - I can't see anyway personal here, and the
  objection is a good one. - Cameron
]

On 11Jan2012 08:41, Paul Moore <p.f.moore at gmail.com> wrote:
| On 11 January 2012 06:15, Cameron Simpson <cs at zip.com.au> wrote:
| > | flush=None?
| >
| > +1 from me, if None means "honour the file's default flushing
| > behaviour".
| 
| The trouble with this is that we then have:
|  - no parameter (default None) => default behaviour
|  - flush=True => Always flush
| 
| and that begs the question of what an explicit flush=False means, and
| people will assume it means "never flush" even thought you can't force
| that as the underlying stream might auto-flush (e.g., at a newline).
| 
| +1 for flush=True/False, with a default of False (meaning do the
| stream's default behaviour).

Hmm. Good point about False suggesting "never flush", but I think it should
just be considered invalid. What about flush=3, or flush=tuple?

Better if we only specify behaviour for True (flush now) and
None/default/missing (flush default) and make it slightly clear that
other values do not have defined behaviour. That leaves scope later
for defining a sane (or at least clear) meaning for False if a _useful_
one is ever agreed upon. Call it "reserved".

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Copyright and Patents: To prevent the Progress of Science and useful Arts, by
securing for unlimited Times to Authors and Inventors and Trolls the
exclusive Right to all Writings and Discoveries.
- http://science.slashdot.org/story/11/04/26/211258/Copyright-Law-Is-Killing-Science


From guido at python.org  Thu Jan 12 00:33:33 2012
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jan 2012 15:33:33 -0800
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <20120111230251.GA23059@cskk.homeip.net>
References: <CACac1F-7mkBhoSbEOgMEo7j25QxivOW26597f254Qgeb-wHe-Q@mail.gmail.com>
	<20120111230251.GA23059@cskk.homeip.net>
Message-ID: <CAP7+vJJ52UrsqnkC0sdOaNnUTKvS8Mw-uFQPTEG4S_zSDYoYdQ@mail.gmail.com>

See what I added to the tracker.

On Wed, Jan 11, 2012 at 3:02 PM, Cameron Simpson <cs at zip.com.au> wrote:

> [ Taken back on list - I can't see anyway personal here, and the
>  objection is a good one. - Cameron
> ]
>
> On 11Jan2012 08:41, Paul Moore <p.f.moore at gmail.com> wrote:
> | On 11 January 2012 06:15, Cameron Simpson <cs at zip.com.au> wrote:
> | > | flush=None?
> | >
> | > +1 from me, if None means "honour the file's default flushing
> | > behaviour".
> |
> | The trouble with this is that we then have:
> |  - no parameter (default None) => default behaviour
> |  - flush=True => Always flush
> |
> | and that begs the question of what an explicit flush=False means, and
> | people will assume it means "never flush" even thought you can't force
> | that as the underlying stream might auto-flush (e.g., at a newline).
> |
> | +1 for flush=True/False, with a default of False (meaning do the
> | stream's default behaviour).
>
> Hmm. Good point about False suggesting "never flush", but I think it should
> just be considered invalid. What about flush=3, or flush=tuple?
>
> Better if we only specify behaviour for True (flush now) and
> None/default/missing (flush default) and make it slightly clear that
> other values do not have defined behaviour. That leaves scope later
> for defining a sane (or at least clear) meaning for False if a _useful_
> one is ever agreed upon. Call it "reserved".
>
> Cheers,
> --
> Cameron Simpson <cs at zip.com.au> DoD#743
> http://www.cskk.ezoshosting.com/cs/
>
> Copyright and Patents: To prevent the Progress of Science and useful Arts,
> by
> securing for unlimited Times to Authors and Inventors and Trolls the
> exclusive Right to all Writings and Discoveries.
> -
> http://science.slashdot.org/story/11/04/26/211258/Copyright-Law-Is-Killing-Science
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120111/ce7115c0/attachment.html>

From steve at pearwood.info  Thu Jan 12 01:00:20 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 12 Jan 2012 11:00:20 +1100
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <4F0DA5E4.40308@enigmage.de>
References: <4F0DA5E4.40308@enigmage.de>
Message-ID: <4F0E2294.40806@pearwood.info>

Manuel B?renz wrote:
> I propose two new control flows for Python:
> 
> "while ... try":
> 
> while expr try:
>     suite1
> except SomeException:
>     suite2
> else:
>     suite3
> 
> This executes suite1 as long as handled exceptions are thrown and expr
> is True.


This conflates two naturally distinct operations, looping and exception 
handling, into a single construct that tries to do both. "Loop" is a natural 
operation. "Catch exceptions" is a natural operation. "Loop and catch 
exceptions" is not, it is two operations and so should be written as two 
operations.

It isn't obvious that handled exceptions remain in the loop. Because there are 
two equally good behaviours -- handled exceptions remain in the loop, or 
handled exceptions exit the loop -- people will get confused by the construct 
and repeatedly be surprised it doesn't do what they expect.

Your proposed syntax doesn't allow the user to write code which couldn't be 
written before, it adds no new power or expressiveness to Python. The sole 
advantage is that it saves one line and one indent level, which is a trivial 
advantage: there are no shortages of either newlines or indents, and if anyone 
is writing such deeply nested code that they are regularly worried about 
indents, their code almost certainly is in desperate need of refactoring.

Disadvantages include that it increases complexity of the language: the 
compiler becomes more complex, there is another feature for people to learn. 
Every time somebody writes a loop with a try, they will have to decide whether 
to write it the old way or the new way. The try clause can be easily missed 
while skimming code, making the construct inelegant. The syntax also clashes 
with existing while...else.

What about nested try blocks? If we introduce this, will people try writing this?

while expr try try:
         ...
     except InnerException:
         ...
     ...
except OuterException: ...

This seems like a reasonable thing to do. I know that's exactly what I'd try.

It also raises one special case above all other cases. An arbitrary loop 
containing a try block looks like this:

while expr:
     preamble # 0 or more lines before the try block
     try block
     postscript  # 0 or more lines after the try block

Your proposed syntax only covers the case where both the preamble and the 
postscript are 0 lines. What is so special about that case that it needs extra 
syntax? It's not that common. Existing syntax is consistent, natural and 
obvious. Your proposal looks like you've picked two block constructs, a while 
loop and a try block, and nailed them together.

-1 on this suggestion.


[...]
> One further enhancement: If expr is encountered to be False, some
> special exception "NoMoreTriesException" could be raised. It can be
> catched in the same "while ... try" block.

That use-case is already handled in existing Python by the while...else form. 
There is no need to add an additional built-in exception for this.



-- 
Steven


From ubershmekel at gmail.com  Thu Jan 12 01:13:09 2012
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Thu, 12 Jan 2012 02:13:09 +0200
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <4F0E2294.40806@pearwood.info>
References: <4F0DA5E4.40308@enigmage.de>
	<4F0E2294.40806@pearwood.info>
Message-ID: <CANSw7KxOwLat4Byge_u4j1bBd2x=jTYCrmnchXKWfXO=ym2wsg@mail.gmail.com>

I also only see trouble with this construct.

I initially assumed you meant:

    while expr try:
        # ...
    except IOError:
        print 'error'

would actually equal:

    try:
        while expr:
            # ...
    except IOError:
        print 'error'

This makes sense because the "except" is after the while loop suite so it
should mean that being in the except suite removes the possibility of
reentering the while suite.

Anyhow, because of the ambiguity, and because we'll immediately need
for/try if/try and try comprehensions (some of which have contradicting
"else" semantics) I'm -2 on this.

Yuval Greenfield
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120112/8cb1ed71/attachment.html>

From tjreedy at udel.edu  Thu Jan 12 01:15:03 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 11 Jan 2012 19:15:03 -0500
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <20120111230251.GA23059@cskk.homeip.net>
References: <CACac1F-7mkBhoSbEOgMEo7j25QxivOW26597f254Qgeb-wHe-Q@mail.gmail.com>
	<20120111230251.GA23059@cskk.homeip.net>
Message-ID: <jel8mm$bb0$1@dough.gmane.org>

On 1/11/2012 6:02 PM, Cameron Simpson wrote:

> Better if we only specify behaviour for True (flush now) and
> None/default/missing (flush default) and make it slightly clear that
> other values do not have defined behaviour.

That was the intent of my suggested doc rewording -- file controls 
flushing unless flush=True is given. What I wrote on the issue was a 
first try and I suspect that there are other and perhaps better concise 
wordings. Post one on the issue if you have one. (But writing in 'doc 
style' is not always easy ;-).

-- 
Terry Jan Reedy



From cs at zip.com.au  Thu Jan 12 02:40:17 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 12 Jan 2012 12:40:17 +1100
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <jel8mm$bb0$1@dough.gmane.org>
References: <jel8mm$bb0$1@dough.gmane.org>
Message-ID: <20120112014017.GA14688@cskk.homeip.net>

On 11Jan2012 19:15, Terry Reedy <tjreedy at udel.edu> wrote:
| On 1/11/2012 6:02 PM, Cameron Simpson wrote:
| >Better if we only specify behaviour for True (flush now) and
| >None/default/missing (flush default) and make it slightly clear that
| >other values do not have defined behaviour.
| 
| That was the intent of my suggested doc rewording -- file controls
| flushing unless flush=True is given. What I wrote on the issue was a
| first try and I suspect that there are other and perhaps better
| concise wordings. Post one on the issue if you have one. (But
| writing in 'doc style' is not always easy ;-).

Do I comment here, or in #11633 (which is closed) or in #13761?
We seem to have multiple streams of discussion.

BTW, regarding #11633, I don't like your sentence:

  Use ``sys.stdout.flush()`` to ensure immediate appearance on a screen

Shouldn't that be "*file*.flush()" ? I would _not_ want to give a
shallow reader the idea that flushing stdout magicly sorts their
problems with a specific file. And I've seen that level of logical gap
in various questions on python-list (and elsewhere, of course).

I'd also change "ensure immediate appearance on a screen" to "ensure
immediate output, for example to ensure appearance on a screen".
Again, it changes the wording from "to perform this specific special
case call flush" to "calling flush does blah, for example to aid this
specific case".

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

When in doubt, gas it. It may not solve the problem, but it ends the suspense.
        - Steve Moonitz (92 Ducati 900ss) <stevem at ew.timeinc.com>


From wuwei23 at gmail.com  Thu Jan 12 02:44:59 2012
From: wuwei23 at gmail.com (alex23)
Date: Wed, 11 Jan 2012 17:44:59 -0800 (PST)
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <CANSw7KwWkJ0QKZYiUfnMJocVbF6nNDiVfL3njYdB6S7fVN_5sQ@mail.gmail.com>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
	<CAPkN8x+N9ck99pbJXQXEU=TWy6FRwqe-qHxiwRA7+PnmMZSxzA@mail.gmail.com>
	<894BC1E9-5D57-4702-A758-45E29A1F2AF5@masklinn.net>
	<CAPkN8xKARo29R=aA1GmyfCcBwLnD8fNa+6sUBkgarXiCsBpY4A@mail.gmail.com>
	<CA+OGgf6vXjS89C=xQ-OfsfvgUmGcGQK0dvrj63X+zTp=oCphtw@mail.gmail.com>
	<CANSw7KwWkJ0QKZYiUfnMJocVbF6nNDiVfL3njYdB6S7fVN_5sQ@mail.gmail.com>
Message-ID: <70c88004-a082-4f59-805d-1713c7c92728@gj9g2000pbc.googlegroups.com>

On Jan 12, 6:00?am, Yuval Greenfield <ubershme... at gmail.com> wrote:
> 1. Someone makes a bot that auto posts the etherpads to each idea in
> python-ideas.
> or
> 2. Whoever cares enough about an idea will make an etherpad for it if
> he/she likes to.

And _both_ options split attention between the list and the etherpad.
Which is primary? What happens if someone chooses to participate only
on one and not the other?

What happens in 6 months when someone else decides that whatever
technology du jour they find is cool should be supported as well?


From wuwei23 at gmail.com  Thu Jan 12 03:19:38 2012
From: wuwei23 at gmail.com (alex23)
Date: Wed, 11 Jan 2012 18:19:38 -0800 (PST)
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <4F0DA5E4.40308@enigmage.de>
References: <4F0DA5E4.40308@enigmage.de>
Message-ID: <240e47e1-e2cf-4bd0-a4f8-f72d94ca8cba@g4g2000pbi.googlegroups.com>

On Jan 12, 1:08?am, Manuel B?renz <man... at enigmage.de> wrote:
> I propose two new control flows for Python:

Guido recently said:

"There seems to be a trend in proposing more random
variants of the existing compound statements. Please don't do this.
The existing complement of compound statements is quite sufficient
and
the new proposals do nothing but complicate the parser, the
documentation, and the learning process for occasional users."

https://groups.google.com/group/python-ideas/msg/5b884fd1caef2f74


From tjreedy at udel.edu  Thu Jan 12 04:03:49 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 11 Jan 2012 22:03:49 -0500
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <20120112014017.GA14688@cskk.homeip.net>
References: <jel8mm$bb0$1@dough.gmane.org>
	<20120112014017.GA14688@cskk.homeip.net>
Message-ID: <jelij4$3e6$1@dough.gmane.org>

On 1/11/2012 8:40 PM, Cameron Simpson wrote:
> On 11Jan2012 19:15, Terry Reedy<tjreedy at udel.edu>  wrote:
> | On 1/11/2012 6:02 PM, Cameron Simpson wrote:
> |>Better if we only specify behaviour for True (flush now) and
> |>None/default/missing (flush default) and make it slightly clear that
> |>other values do not have defined behaviour.
> |
> | That was the intent of my suggested doc rewording -- file controls
> | flushing unless flush=True is given. What I wrote on the issue was a
> | first try and I suspect that there are other and perhaps better
> | concise wordings. Post one on the issue if you have one. (But
> | writing in 'doc style' is not always easy ;-).
>
> Do I comment here, or in #11633 (which is closed)

After much discussion that included the possibility of a behavior 
change, that issue became about changing the print doc, especially for 3.2.

> or in #13761?

After Guido's approval of leaving the default behavior as is but adding 
a parameter to change it, that was opened for the behavior change for 
3.3. That change will require further doc change for 3.3.

> BTW, regarding #11633, I don't like your sentence:
>
>    Use ``sys.stdout.flush()`` to ensure immediate appearance on a screen
>
> Shouldn't that be "*file*.flush()" ?

That was my original idea but I changed it after internal debate. But I 
was at most 60-40 for what I did.

> I would _not_ want to give a
> shallow reader the idea that flushing stdout magicly sorts their
> problems with a specific file. And I've seen that level of logical gap
> in various questions on python-list (and elsewhere, of course).
>
> I'd also change "ensure immediate appearance on a screen" to "ensure
> immediate output, for example to ensure appearance on a screen".
> Again, it changes the wording from "to perform this specific special
> case call flush" to "calling flush does blah, for example to aid this
> specific case".

If one of the doc people agree with you enough to refine the effectively 
temporary change for 3.2, fine, but I do not think it worth the effort. 
I suggest you focus on the doc further change for 3.3 after the behavior 
change. That will potentially be in the docs for several future 
versions. For that, post a post-change suggestion to #13761.

-- 
Terry Jan Reedy



From eliben at gmail.com  Thu Jan 12 08:50:55 2012
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 12 Jan 2012 09:50:55 +0200
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <4F0DA5E4.40308@enigmage.de>
References: <4F0DA5E4.40308@enigmage.de>
Message-ID: <CAF-Rda-vHRsKhLADNCYcLQXcCOcWm5YAJfZpqmLLO2Prgtg_UQ@mail.gmail.com>

2012/1/11 Manuel B?renz <manuel at enigmage.de>

> I propose two new control flows for Python:
>
> "while ... try":
>
> while expr try:
>    suite1
> except SomeException:
>    suite2
> else:
>    suite3
>
<snip>

Manuel,
Thanks for the improvement idea, but I must admit I'm -1 on this. Having
first looked at the proposed syntax without your explanation, I found it
hard to be sure what it does, and this goes against the spirit of Python.
The compound statement it purports to replace is just one indent deeper and
much clearer.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120112/543620f4/attachment.html>

From techtonik at gmail.com  Thu Jan 12 11:04:58 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 12 Jan 2012 13:04:58 +0300
Subject: [Python-ideas] Posting style
In-Reply-To: <CAJ6cK1aNVgpL5L1-aQbQPbUihQ-0AA9WEB2amvFds8ofcDCBFw@mail.gmail.com>
References: <20120110112216.GA21299@iskra.aviel.ru>
	<CABicbJ+dBzeB8Jcd_YL63Vas9ApRJWpVLZgSW9bCh9=4JUm+LA@mail.gmail.com>
	<20120110124354.GC21299@iskra.aviel.ru>
	<CABicbJ+_ZOnbDiwn+E+54Brbktv6s3Y8wMaOy5UrNZgA9c+OuA@mail.gmail.com>
	<CAJ6cK1aNVgpL5L1-aQbQPbUihQ-0AA9WEB2amvFds8ofcDCBFw@mail.gmail.com>
Message-ID: <CAPkN8x+jiRi1Bn1z76LacDbe6x4eQdNnDs0UyYGg626GV1OLPw@mail.gmail.com>

On Tue, Jan 10, 2012 at 4:20 PM, Arnaud Delobelle <arnodel at gmail.com> wrote:

> On 10 January 2012 13:13, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
> > On Tue, Jan 10, 2012 at 7:43 AM, Oleg Broytman <phd at phdru.name> wrote:
> >>   I am not going to mandate anything (first, I don't have power).
> >> Anything is acceptable. But decide for yourself what is convenient *for
> >> you*. Imagine you are reading through a big and complex discussion with
> >> many threads. Or think about looking for information in the mailing
> >> list(s) archive. How much excessive information can you bear? What
> >> posing style would be the best in that cases?
> >
> > I don't really understand the objection. My mail client hides unbroken
> > quotes at the bottom of an email, so that I don't see anything except
> > what you quoted (if I read my own email), plus my signature. I take it
> > yours doesn't do this?
>
> People don't usually look at the archives with their email client.


I agree that Pipermail (Mailman's default interface) suxx for browsing
threads and archives, but Google Groups are able to handle that pretty well
https://groups.google.com/forum/#!forum/python-ideas
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120112/066a2da4/attachment.html>

From manuel at enigmage.de  Thu Jan 12 11:51:24 2012
From: manuel at enigmage.de (=?ISO-8859-1?Q?Manuel_B=E4renz?=)
Date: Thu, 12 Jan 2012 11:51:24 +0100
Subject: [Python-ideas] "while ... try" - block or "for ... try" - block
In-Reply-To: <CA+OGgf4uxvpPrVSS-MjYMs2R1WX=OOXk+-v+p-nxDfBnnEsT6Q@mail.gmail.com>
References: <4F0DA5E4.40308@enigmage.de>
	<CA+OGgf4uxvpPrVSS-MjYMs2R1WX=OOXk+-v+p-nxDfBnnEsT6Q@mail.gmail.com>
Message-ID: <4F0EBB2C.9020609@enigmage.de>


On 11.01.2012 17:58, Jim Jewett wrote:
> 2012/1/11 Manuel B?renz <manuel at enigmage.de>:
>
>> while expr try:
>>    suite1
>> except SomeException:
>>    suite2
>> else:
>>    suite3
> Without your explanation, I would still have assumed that the
> exception terminated the while.
> #Steve's comment:
> people will get confused by the construct 
> and repeatedly be surprised it doesn't do what they expect.
Ok, you convinced me that this is not intuitive for most people.
Normally, one expects the try block to work and not to fail. So
continuing the loop on failure and not on success is counterintuitive.
>     retry=WaitPolicy('sleep_and_retry', 3)
>
>     with ensure_network(onfail=retry) as network:
>         with connect_to_server(transport=network, onfail=retry):
>             download_stuff()
That's a nice and Pythonic suggestion. Thanks.

On 12.01.2012 01:00, Steven D'Aprano wrote:
> This conflates two naturally distinct operations, looping and exception 
> handling, into a single construct that tries to do both. "Loop" is a natural 
> operation. "Catch exceptions" is a natural operation. "Loop and catch 
> exceptions" is not, it is two operations and so should be written as two 
> operations.
In these words, I wanted to point out that "Loop until it works" is a
natural operation and should have a Python control flow.
> The syntax also clashes with existing while...else.
Not syntactically:

>>> while True try:
  File "<stdin>", line 1
    while True try:
                 ^
SyntaxError: invalid syntax

The interpreter could easily distinguish the two, since try is a
reserved keyword..
> The try clause can be easily missed 
> while skimming code,
Very good point.
> Your proposal looks like you've picked two block constructs, a while 
> loop and a try block, and nailed them together.
Sorry, I'll have to defend against that a bit. This idea really came
from a (simple) usecase. I did a performance test of the
functools.lru_cache decorator against a naive self-written memoize class
and found me writing:

        try:
            result = self.cache[n]
        except KeyError:
            result = self.cache[n] = self.f(n)
        return result

I would have wanted to write something like:

while True try:
    return self.cache[n]
except KeyError:
    self.cache[n] = self.f(n)

That's how I came up with the idea.


But you presented enough reasonable arguments against it such that I
won't consider it anymore.
Thanks for your comments.


From julien at tayon.net  Thu Jan 12 14:05:44 2012
From: julien at tayon.net (julien tayon)
Date: Thu, 12 Jan 2012 14:05:44 +0100
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <70c88004-a082-4f59-805d-1713c7c92728@gj9g2000pbc.googlegroups.com>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
	<CAPkN8x+N9ck99pbJXQXEU=TWy6FRwqe-qHxiwRA7+PnmMZSxzA@mail.gmail.com>
	<894BC1E9-5D57-4702-A758-45E29A1F2AF5@masklinn.net>
	<CAPkN8xKARo29R=aA1GmyfCcBwLnD8fNa+6sUBkgarXiCsBpY4A@mail.gmail.com>
	<CA+OGgf6vXjS89C=xQ-OfsfvgUmGcGQK0dvrj63X+zTp=oCphtw@mail.gmail.com>
	<CANSw7KwWkJ0QKZYiUfnMJocVbF6nNDiVfL3njYdB6S7fVN_5sQ@mail.gmail.com>
	<70c88004-a082-4f59-805d-1713c7c92728@gj9g2000pbc.googlegroups.com>
Message-ID: <CAFpLVky_b8OEBHNCaq+uK_R7VuQEzfv4sQxzqBDXTgNYQ3k3Gw@mail.gmail.com>

Hello

2012/1/12 alex23 <wuwei23 at gmail.com>:
> On Jan 12, 6:00?am, Yuval Greenfield <ubershme... at gmail.com> wrote:
>> 1. Someone makes a bot that auto posts the etherpads to each idea in
>> python-ideas.
>> or
>> 2. Whoever cares enough about an idea will make an etherpad for it if
>> he/she likes to.
>
> And _both_ options split attention between the list and the etherpad.
> Which is primary? What happens if someone chooses to participate only
> on one and not the other?
>
> What happens in 6 months when someone else decides that whatever
> technology du jour they find is cool should be supported as well?

Well, I may be dumb, but ideas seems like non structured draft of PEP.
(like IETF RFC drafts), and by looking at PEP 1, I read that it
describes a very workflow on how to do things.

Maybe python-ideas should only accept ideas in the form a link to a
PEP draft and people would have to maintain there draft as long as it
is not accepted.

http://www.python.org/dev/peps/pep-0001/

So why duplicates already existing procedure and infrastructure ?
Maybe amending PEP 1 could be a good way to achieve the aformentioned
goal.

By not adding a PEP draft that is not accepted in a documentation
database, we achieve one of the very first goal of a documentation
database which is lowering the Noise Signal Ratio.


My .02 cents,

-- 
Julien


From fuzzyman at gmail.com  Thu Jan 12 16:02:52 2012
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 12 Jan 2012 15:02:52 +0000
Subject: [Python-ideas] os.listdir default to '.'
Message-ID: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>

Hello all,

I frequently use Python from the interactive interpreter as a shell. It is
a frequent, but mild, irritant that I have to type os.listdir('.') instead
of just os.listdir().

How about making '.' a default argument for os.listdir ?

All the best,

Michael

-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120112/45be498b/attachment.html>

From rob.cliffe at btinternet.com  Thu Jan 12 16:14:46 2012
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 12 Jan 2012 15:14:46 +0000
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
References: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
Message-ID: <4F0EF8E6.6030002@btinternet.com>

In Python 2.5 you can at least say os.listdir('') which had the same effect.
In Python 2.7 you can't.

Strictly speaking I think the default value should be os.path.curdir, 
not specifically '.'

+1 from me
+1 on allowing the empty string argument again (may avoid breaking some 
old code)

Rob Cliffe

On 12/01/2012 15:02, Michael Foord wrote:
> Hello all,
>
> I frequently use Python from the interactive interpreter as a shell. 
> It is a frequent, but mild, irritant that I have to type 
> os.listdir('.') instead of just os.listdir().
>
> How about making '.' a default argument for os.listdir ?
>
> All the best,
>
> Michael
>
> -- 
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
>
> May you share freely, never taking more than you give.
> -- the sqlite blessinghttp://www.sqlite.org/different.html
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120112/f2d8c8de/attachment.html>

From masklinn at masklinn.net  Thu Jan 12 16:22:42 2012
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 12 Jan 2012 16:22:42 +0100
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
References: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
Message-ID: <82AB40F9-9C34-460F-A7CD-9E30BB3BF0C4@masklinn.net>

On 2012-01-12, at 16:02 , Michael Foord wrote:
> Hello all,
> 
> I frequently use Python from the interactive interpreter as a shell. It is
> a frequent, but mild, irritant that I have to type os.listdir('.') instead
> of just os.listdir().
> 
> How about making '.' a default argument for os.listdir ?
I was going to say I don't think '.' is a good idea for `os.listdir`, but
then?

> ls
Doc               LICENSE           Makefile          Misc              PC                Python            build             configure         pybuilddir.txt    python.exe
Grammar           Lib               Makefile.pre      Modules           PCbuild           README            config.log        configure.in      pyconfig.h        python.exe-gdb.py
Include           Mac               Makefile.pre.in   Objects           Parser            Tools             config.status     install-sh        pyconfig.h.in     setup.py
> python3.2
Python 3.2.2 (default, Nov  7 2011, 20:32:23) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.listdir()
['.bzrignore', '.gitignore', '.hg', '.hgcheck', '.hgeol', '.hgignore', '.hgtags', 'build', 'config.log', 'config.status', 'configure', 'configure.in', 'Doc', 'Grammar', 'Include', 'install-sh', 'Lib', 'LICENSE', 'Mac', 'Makefile', 'Makefile.pre', 'Makefile.pre.in', 'Misc', 'Modules', 'Objects', 'Parser', 'PC', 'PCbuild', 'pybuilddir.txt', 'pyconfig.h', 'pyconfig.h.in', 'Python', 'python.exe', 'python.exe-gdb.py', 'README', 'setup.py', 'Tools']

looks like '.' is already the default argument for os.listdir in Python 3.2 (but not in 3.1, no idea about 3.3)

From massimo.dipierro at gmail.com  Thu Jan 12 16:28:13 2012
From: massimo.dipierro at gmail.com (Massimo Di Pierro)
Date: Thu, 12 Jan 2012 09:28:13 -0600
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <4F0EF8E6.6030002@btinternet.com>
References: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
	<4F0EF8E6.6030002@btinternet.com>
Message-ID: <6B78951D-9135-4C87-B702-EFF05C5A62DA@gmail.com>

+1

On Jan 12, 2012, at 9:14 AM, Rob Cliffe wrote:

> In Python 2.5 you can at least say os.listdir('') which had the same effect.
> In Python 2.7 you can't.
> 
> Strictly speaking I think the default value should be os.path.curdir, not specifically '.'
> 
> +1 from me
> +1 on allowing the empty string argument again (may avoid breaking some old code)
> 
> Rob Cliffe
> 
> On 12/01/2012 15:02, Michael Foord wrote:
>> 
>> Hello all,
>> 
>> I frequently use Python from the interactive interpreter as a shell. It is a frequent, but mild, irritant that I have to type os.listdir('.') instead of just os.listdir().
>> 
>> How about making '.' a default argument for os.listdir ?
>> 
>> All the best,
>> 
>> Michael
>> 
>> -- 
>> http://www.voidspace.org.uk/
>> 
>> May you do good and not evil
>> May you find forgiveness for yourself and forgive others
>> 
>> May you share freely, never taking more than you give.
>> -- the sqlite blessing http://www.sqlite.org/different.html
>> 
>> 
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120112/08d35db7/attachment.html>

From phd at phdru.name  Thu Jan 12 16:39:27 2012
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 12 Jan 2012 19:39:27 +0400
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
References: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
Message-ID: <20120112153927.GA30594@iskra.aviel.ru>

On Thu, Jan 12, 2012 at 03:02:52PM +0000, Michael Foord wrote:
> I frequently use Python from the interactive interpreter as a shell. It is
> a frequent, but mild, irritant that I have to type os.listdir('.') instead
> of just os.listdir().

   Not exactly what you've asked but let me show you my config for
interactive sessions: http://phdru.name/Software/dotfiles/init.py.html
   You can type

>>> cd('/etc')
>>> ls

   and got back the list. (-:

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From benjamin at python.org  Thu Jan 12 16:56:07 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 12 Jan 2012 15:56:07 +0000 (UTC)
Subject: [Python-ideas] os.listdir default to '.'
References: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
Message-ID: <loom.20120112T165556-345@post.gmane.org>

Michael Foord <fuzzyman at ...> writes:

> 
> 
> Hello all,I frequently use Python from the interactive interpreter as a shell.
It is a frequent, but mild, irritant that I have to type os.listdir('.') instead
of just os.listdir().How about making '.' a default argument for os.listdir ?All
the best,Michael-- 

$ ./python 
Python 3.3.0a0 (default:2db3ca05fbb7+, Jan 11 2012, 20:46:04) 
[GCC 4.5.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.listdir()
['autom4te.cache', 'pybuilddir.txt', 'PC', 'Tools', 'pyconfig.h', 'python',
'.hgtags', 'Makefile.pre.in', 'install-sh', 'configure.in', '.hg',
'python-gdb.py', 'README', 'Parser', 'setup.py', 'config.log', 'Makefile',
'Doc', 'Lib', 'Objects', 'build', 'LICENSE', 'pyconfig.h.in', 'Python',
'Makefile.pre', '.hgeol', 'Include', 'libpython3.3m.a', '.hgignore',
'.bzrignore', 'PCbuild', 'Modules', 'config.status', 'Misc', 'configure',
'.gitignore', 'Grammar', 'Mac']







From brett at python.org  Thu Jan 12 17:35:00 2012
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Jan 2012 11:35:00 -0500
Subject: [Python-ideas] Netiquette issues (was Re: "while ... try" -
 block or "for ... try" - block)
In-Reply-To: <20120111174353.GC13614@idyll.org>
References: <4F0DA5E4.40308@enigmage.de>
	<CAB4yi1ORtxGtatRpmai0dXBV8SJtAUCEttOSWpb_gCvVPgMDUQ@mail.gmail.com>
	<4F0DBD35.2000309@stoneleaf.us> <20120111174353.GC13614@idyll.org>
Message-ID: <CAP1=2W4dyZyTh0sduq5=-DMRb038BkY9LKKcNF1JQNoamndRvA@mail.gmail.com>

On Wed, Jan 11, 2012 at 12:43, C. Titus Brown <ctb at msu.edu> wrote:

> Folks,
>
> I'm a moderator.  Volunteer, true, but a moderator nonetheless.  And
> I've had enough.
>
> So:
>
> Netiquette discussions are always fascinating, but please, let's stop
> talking
> about how other people should post.  Most of my e-mail from this list over
> the
> last few days has been from people on this list nattering about how gmail
> foo
> and mailer blah, and it is OFF TOPIC and NO LONGER WELCOME.  (Frankly, you
> should have figured this out for yourselves.)
>
> If you feel absolutely compelled to tell people that they are lousy
> netizens
> and/or correct them on their posting style, please do so PRIVATELY and not
> on this list.
>
> Please e-mail me PRIVATELY if you disagree with this.  Brett Cannon is the
> other list admin, and you should feel free to tell him how much you dislike
> me & my overbearing moderator ways.
>

And just FYI to people, I wouldn't bother complaining to me because I
support Titus on this one; stuff has gone off the rails here, but I have
been flat-out ignoring the list as a whole because of it instead of getting
worked up like Titus. He's just a more "delicate flower" than me when comes
to this and apparently cares more. =)

-Brett


>
> Further e-mails on netiquette will result in those people no longer being
> able to post without moderation.
>
> thanks,
> --titus
>
> p.s. This is not aimed especially at Ethan -- it was just the latest e-mail
> that pushed me over the edge :)
> --
> C. Titus Brown, ctb at msu.edu
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120112/4b34da08/attachment.html>

From bruce at leapyear.org  Thu Jan 12 17:57:56 2012
From: bruce at leapyear.org (Bruce Leban)
Date: Thu, 12 Jan 2012 08:57:56 -0800
Subject: [Python-ideas] An etherpad for every idea
In-Reply-To: <CAFpLVky_b8OEBHNCaq+uK_R7VuQEzfv4sQxzqBDXTgNYQ3k3Gw@mail.gmail.com>
References: <CAPkN8xLD=B6Uw3qSELH73f+3M_9A8XWrJVdW=ae58VnVMJdfpA@mail.gmail.com>
	<20120110143156.1188db76@pitrou.net>
	<CAPkN8x+N9ck99pbJXQXEU=TWy6FRwqe-qHxiwRA7+PnmMZSxzA@mail.gmail.com>
	<894BC1E9-5D57-4702-A758-45E29A1F2AF5@masklinn.net>
	<CAPkN8xKARo29R=aA1GmyfCcBwLnD8fNa+6sUBkgarXiCsBpY4A@mail.gmail.com>
	<CA+OGgf6vXjS89C=xQ-OfsfvgUmGcGQK0dvrj63X+zTp=oCphtw@mail.gmail.com>
	<CANSw7KwWkJ0QKZYiUfnMJocVbF6nNDiVfL3njYdB6S7fVN_5sQ@mail.gmail.com>
	<70c88004-a082-4f59-805d-1713c7c92728@gj9g2000pbc.googlegroups.com>
	<CAFpLVky_b8OEBHNCaq+uK_R7VuQEzfv4sQxzqBDXTgNYQ3k3Gw@mail.gmail.com>
Message-ID: <CAGu0AnsCdaaSr5Dv0VrDGmm50ZViW+ENcKktKcQFY+kn5bjyAQ@mail.gmail.com>

On Thu, Jan 12, 2012 at 5:05 AM, julien tayon <julien at tayon.net> wrote:

> Maybe python-ideas should only accept ideas in the form a link to a
> PEP draft and people would have to maintain there draft as long as it
> is not accepted.
>
> http://www.python.org/dev/peps/pep-0001/
>

Seems like you didn't read PEP 1 carefully enough:

"Vetting an idea publicly before going as far as writing a PEP is meant to
save the potential author time. Many ideas have been brought forward for
changing Python that have been rejected for various reasons...."

Asking people to write a PEP before discussing ideas would stifle
discussion and innovation.


> By not adding a PEP draft that is not accepted in a documentation
> database, we achieve one of the very first goal of a documentation
> database which is lowering the Noise Signal Ratio.
>

python-ideas is not a documentation database. It's a discussion list.

--- Bruce
Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120112/bf8aff84/attachment.html>

From grosser.meister.morti at gmx.net  Thu Jan 12 19:46:12 2012
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Thu, 12 Jan 2012 19:46:12 +0100
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
References: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
Message-ID: <4F0F2A74.2090800@gmx.net>

IIRC for Python 2 under Windows os.listdir(".") does something different than os.listdir(u"."). 
Because I think there are two filesystem APIs under Windows, one using char and the other wchar_t. 
(I don't use Windows so I could be mistaken.)

But anyway, under any OS Python 2 returns something else whether you pass a unicode or str object to 
listdir (a list of unicode or string objects). Under Python 3 there is no unicode because str is 
what under Python 2 was unicode.

So under Python 2 os.listdir() would be not good. What should it return? strs or unicodes?

On 01/12/2012 04:02 PM, Michael Foord wrote:
> Hello all,
>
> I frequently use Python from the interactive interpreter as a shell. It is a frequent, but mild,
> irritant that I have to type os.listdir('.') instead of just os.listdir().
>
> How about making '.' a default argument for os.listdir ?
>
> All the best,
>
> Michael
>
> --
>
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
>
> May you share freely, never taking more than you give.
> -- the sqlite blessinghttp://www.sqlite.org/different.html
>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From masklinn at masklinn.net  Thu Jan 12 21:00:32 2012
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 12 Jan 2012 21:00:32 +0100
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <4F0F2A74.2090800@gmx.net>
References: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
	<4F0F2A74.2090800@gmx.net>
Message-ID: <7CE9E4B7-F04F-41BB-9C77-E70E021FA173@masklinn.net>

On 2012-01-12, at 19:46 , Mathias Panzenb?ck wrote:
> 
> But anyway, under any OS Python 2 returns something else whether you pass a unicode or str object to listdir (a list of unicode or string objects). Under Python 3 there is no unicode because str is what under Python 2 was unicode.
> 
> So under Python 2 os.listdir() would be not good. What should it return? strs or unicodes?
I understand Python 2 is in maintenance-only status anyway, so there's no way
python-ideas will apply to it.

And since Python 3 seems to already have os.listdir default to '.'...

From tjreedy at udel.edu  Thu Jan 12 21:34:12 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 12 Jan 2012 15:34:12 -0500
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
References: <CAKCKLWyReUfU7v6J5-7rnwe8xYaSDdCyGSW=X56_xUTm75U8wg@mail.gmail.com>
Message-ID: <jeng4k$hd4$1@dough.gmane.org>

On 1/12/2012 10:02 AM, Michael Foord wrote:

> How about making '.' a default argument for os.listdir ?

 From 3.2 doc: "os.listdir(path='.')"

In general, to find out what is already in future versions, the 
development docs (now 3.3a0, but later 3.4a0, etc) are at 
http://docs.python.org/dev/

Enhancement issues on the tracker show existing proposals.

-- 
Terry Jan Reedy



From nathan.alexander.rice at gmail.com  Thu Jan 12 21:45:48 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Thu, 12 Jan 2012 15:45:48 -0500
Subject: [Python-ideas] Symbolic expressions (or: partials and closures from
	the inside out)
Message-ID: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>

Greetings,

I have been writing a lot of code lately that involves creating
symbolic expressions of one form or another, which are then fully
evaluated at a later time.  Examples of this include Elementwise,
where I create expressions that act on every member of an iterable
(there is a much improved new version coming soon, by the way), and a
design by contract/validation lib I'm working on (which shall remain
nameless :D) that uses symbolic expressions in the *args of the
metaclass __new__ method to generate a constraint class which
validates input using __instancecheck__.  I do most of this with
lambdas, a little hacking with closures and FunctionType(), and
chainable objects.  I am very impressed that python is this flexible,
but there are some issues with the approach that I would like to
rectify, namely:

1.  Because of the early binding behavior of most things in Python, if
I want to include isinstance(X, someclass) in a symbolic expression, I
have to wrap it in a lambda (or use .apply(), in the case of
Elementwise).  This is not a huge deal for me, but it forces me to
create wrappers for lots of functions (e.g. isinstance_(X, someclass))
and/or have users wrap every such function they want to use in a
symbolic expression.   Having to do this also bloats the code a lot;
the github version of Elementwise is over 3,000 LoC at this point
(including prodigious documentation, but still...).

2.  Python expects that certain functions, such as int(), str(), etc,
will have a specific return type.  While in general I agree with this,
it makes Elementwise somewhat inconsistent (and it will do the same to
anything else that wants to work with symbolic expressions).

I'm interested in fixing both issues. I believe both issues I've had
could be solved by having a robust "symbolic object".  These objects
would basically usable like ordinary objects, however upon any
attribute access or other form of interaction, the object would
basically short circuit the calling function, and return a symbolic
object directly to the outer scope.  The symbolic object would behave
like a generator function frozen at the point of attribute access, and
upon send()-ing (or whatever method), it would behave exactly as if
the values sent had been the ones passed in originally (ideally
without consuming the generator).

I have thought about ways to approximate this behavior python
currently, and while I could hack something together using inspect to
pull relevant info from the stack, then break out using a special
exception (potentially passing a generator with state as an exception
arg), this approach strikes me as VERY brittle, implementation
dependent, ugly and difficult to work with.  Additionally, you would
need to catch the special exception somewhere in the stack, so this
trick wouldn't work on the first thing in an expression to be
evaluated.

As an aside, I'd like to solicit some feedback on the validation
syntax I've been working on.  Currently, I have code that support
things like:

X = SymbolicObject()

const = Constraints(X * 2 + 1 >= 5, X % 2 != 0)
const2 = Constraints(X[-1] == "h")
const3 = Constraints(X[-1].upper() == "H")

>>> print isinstance(3, const)
True

>>> print isinstance(2, const)
False

>>> print isinstance(1, const)
False

>>> print isinstance("bleh", const2)
True

>> print isinstance("bleh", const3)
True

Callables are supported as well, so if you wanted to do something like:

Constraints(isinstance(X.attr, someclass), somefunc(X[-2].attr, args))

You could approximate that with:

Constraints(lambda x: isinstance(x.attr, someclass), lambda x:
somefunc(x[-2].attr, args))

As I mentioned in the first paragraph, Constraints is a metaclass, so
your validations are checked using __instancecheck__.  I'm also
considering having __init__ generate mock objects (for certain
straight-forward cases, anyhow).

Thanks for your time,

Nathan


From cs at zip.com.au  Thu Jan 12 22:32:23 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 13 Jan 2012 08:32:23 +1100
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <4F0EF8E6.6030002@btinternet.com>
References: <4F0EF8E6.6030002@btinternet.com>
Message-ID: <20120112213223.GA7309@cskk.homeip.net>

On 12Jan2012 15:14, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
| In Python 2.5 you can at least say os.listdir('') which had the same effect.
| In Python 2.7 you can't.
| 
| Strictly speaking I think the default value should be
| os.path.curdir, not specifically '.'
| +1 from me

I think it should be '.'.

It is possible to arrange directory permissions so that you can't open
curdir because it traverses a directory to which one lacks permissions,
but you can still open '.'.  Correspondingly, there can be times you can
opendir('.') but not compute curdir (for access reasons); an unpleasant
situation but possible (maybe not on platforms where the OS hands you
curdir like Linux; historically UNIX platforms compute it by consulting
the mount table and walking some directories.

| +1 on allowing the empty string argument again (may avoid breaking
| some old code)

I used to be a fan of the '' UNIX string referring to the current
directory (on the tenuous but cool logic that '' lets the kernel hand you
the current directory directly, while '.' implies _opening_ the current
directory and finding its '.' entry, and opening that:-) but not any
more - it makes more trouble than its worth (and I think it went away
in POSIX).

'.' has better lexical properties than '' - it makes dirname and basename and
similar operations easier to code and think about because it is not an empty
string.

opendir() should be as thin a glue around the OS's opendir as possible.
On that basis I'm good with a default argument value but bad on explicitly
supporting '' as meaning the current directory when the OS may not;
it would at least mean wiring special knowledge into os.opendir for
that string.

I guess I'm:
  +1 for '.' being opendir's default.
  -0.5 for '' being opendir's default.
  -1 for os.curdir() being opendir's default.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

I'M NOT afraid of insects talking over the world, and you know why?  It would
take about a billion ant just to aim a gun at me, let alone fire it.  And
know what I'm doing while they're aiming ti at me?  I just sort of slip off
to the side, and then suddenly run up and kick the gun out of their hands.
        - Jack Handey


From ben+python at benfinney.id.au  Thu Jan 12 23:26:10 2012
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Jan 2012 09:26:10 +1100
Subject: [Python-ideas] os.listdir default to '.'
References: <4F0EF8E6.6030002@btinternet.com>
	<20120112213223.GA7309@cskk.homeip.net>
Message-ID: <87ipkg7e2l.fsf@benfinney.id.au>

Cameron Simpson <cs at zip.com.au> writes:

> I think it should be '.' [instead of ?os.path.curdir?].
>
> It is possible to arrange directory permissions so that you can't open
> curdir because it traverses a directory to which one lacks permissions,
> but you can still open '.'.

What distinction are you drawing?

    >>> os.path.curdir
    '.'

Do you get a different result?

-- 
 \       ?Working out the social politics of who you can trust and why |
  `\      is, quite literally, what a very large part of our brain has |
_o__)                                   evolved to do.? ?Douglas Adams |
Ben Finney



From sven at marnach.net  Thu Jan 12 23:11:36 2012
From: sven at marnach.net (Sven Marnach)
Date: Thu, 12 Jan 2012 22:11:36 +0000
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <20120112213223.GA7309@cskk.homeip.net>
References: <4F0EF8E6.6030002@btinternet.com>
	<20120112213223.GA7309@cskk.homeip.net>
Message-ID: <20120112221136.GA32550@pantoffel-wg.de>

Cameron Simpson schrieb am Fr, 13. Jan 2012, um 08:32:23 +1100:
> On 12Jan2012 15:14, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
> | Strictly speaking I think the default value should be
> | os.path.curdir, not specifically '.'
[...]
>   -1 for os.curdir() being opendir's default.

There's a misunderstanding here.  You are confusing 'os.getcwd()' and
'os.curdir'.

The default value for the parameter of 'os.listdir()' should probably
be 'os.curdir', but since it is defined in the different
platform-specific OS modules like posixmodule.c, it probably is
'os.curdir'.  Moreover, I'm not aware of a platform where
os.curdir != '.'

 -- Sven


From tjreedy at udel.edu  Fri Jan 13 00:03:17 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 12 Jan 2012 18:03:17 -0500
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
 from the inside out)
In-Reply-To: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
Message-ID: <jenos5$g8d$1@dough.gmane.org>

On 1/12/2012 3:45 PM, Nathan Rice wrote:
> Greetings,
>
> I have been writing a lot of code lately that involves creating
> symbolic expressions of one form or another, which are then fully
> evaluated at a later time.  Examples of this include Elementwise,

Python is designed for concrete, rather than symbolic computation. But 
the latter has been done.

> where I create expressions that act on every member of an iterable
> (there is a much improved new version coming soon, by the way), and a
> design by contract/validation lib I'm working on (which shall remain
> nameless :D) that uses symbolic expressions in the *args of the
> metaclass __new__ method to generate a constraint class which
> validates input using __instancecheck__.  I do most of this with
> lambdas, a little hacking with closures and FunctionType(), and
> chainable objects.  I am very impressed that python is this flexible,
> but there are some issues with the approach that I would like to
> rectify, namely:
>
> 1.  Because of the early binding behavior of most things in Python,

I think you may be confusing name-resolution time with execution time. 
They are distinct, though somewhat tied together in Python. For example: 
if I hand you a book and say "Read this", I intend that you immediately 
resolve 'this' to the book I just handed you and immediately 'read' it. 
If I instead say, "Tomorrow morning, read this", I still intend that you 
immediately resolve 'this' to a particular object, but now intend that 
you package the object with the action for later execution. I believe 
you want this this mixture of resolution now with action later.

If so,  your problem is that executing 'read(this)' or 'this.read()' 
does both things now, while "def f(): read(this)" or "lambda: 
read(this)" puts off both things until later.

Python has several ways to binds objects now with actions later. Let 
<book> be a reference to 'the book Terry handed me, which I stored 
wherever'.

0: rewrite the text -- a bit awkward in Python.

action = compile("read({this})".format(this=<book>), 'xxx', 'eval')

1. default args -- has to be done when the function is defined.

def action(this = <book>): read(this)

2. closures (nested functions) -- also requires a planned-ahead definition.

make_read(x):
   return lambda: read(x)
action = make_read(<book>)

3. bound methods -- only works for classes with methods.

Class Book:
     def read(self): pass
action = Book(<book>).read

4. partial binding of function params -- generalizes bound methods; 
works for any function and argument.

from functools import partial
action = partial(read, <book>)

> if I want to include isinstance(X, someclass) in a symbolic expression,

Consider using partial, which can totally bind all needed args *now* for 
*later* action.

 >>> from functools import partial
 >>> t = partial(isinstance, 1, int)
 >>> t()
True
 >>> f = partial(isinstance, 1, float)
 >>> f()
False

> I have to wrap it in a lambda

Are you sure that partial will not work for you? Since partial is 
written in Python, you can grab and adjust the code to your needs. It 
amounts to adding default args after the fact by using a generic closure.

 > (or use .apply(), in the case of
> Elementwise).  This is not a huge deal for me, but it forces me to
> create wrappers for lots of functions (e.g. isinstance_(X, someclass))
> and/or have users wrap every such function they want to use in a
> symbolic expression.   Having to do this also bloats the code a lot;
> the github version of Elementwise is over 3,000 LoC at this point
> (including prodigious documentation, but still...).
>
> 2.  Python expects that certain functions, such as int(), str(), etc,
> will have a specific return type.  While in general I agree with this,

People expect that class constructors produce an instance of the class. 
It is surprising when one does otherwise ;-). Builtin classes like int, 
bool, and str are classes just like ones you write.

> it makes Elementwise somewhat inconsistent (and it will do the same to
> anything else that wants to work with symbolic expressions).
>
> I'm interested in fixing both issues. I believe both issues I've had
> could be solved by having a robust "symbolic object".  These objects
> would basically usable like ordinary objects, however upon any
> attribute access or other form of interaction, the object would
> basically short circuit the calling function, and return a symbolic
> object directly to the outer scope.  The symbolic object would behave
> like a generator function frozen at the point of attribute access, and
> upon send()-ing (or whatever method), it would behave exactly as if
> the values sent had been the ones passed in originally (ideally
> without consuming the generator).
>
> I have thought about ways to approximate this behavior python
> currently, and while I could hack something together using inspect to
> pull relevant info from the stack, then break out using a special
> exception (potentially passing a generator with state as an exception
> arg), this approach strikes me as VERY brittle, implementation
> dependent, ugly and difficult to work with.  Additionally, you would
> need to catch the special exception somewhere in the stack, so this
> trick wouldn't work on the first thing in an expression to be
> evaluated.
>
> As an aside, I'd like to solicit some feedback on the validation
> syntax I've been working on.  Currently, I have code that support
> things like:
>
> X = SymbolicObject()
>
> const = Constraints(X * 2 + 1>= 5, X % 2 != 0)
> const2 = Constraints(X[-1] == "h")
> const3 = Constraints(X[-1].upper() == "H")

Using a special class is a standard way to delay concrete execution.

>>>> print isinstance(3, const)
> True

A Contraints instance defines a set. 'const' is the set 'odd_ge_3'
It would look better if you used standard syntax and do the inclusion 
check in a __contains__ method.

 >>> 3 in odd_ge_3
True

>>>> print isinstance(2, const)
> False

2 in odd_ge_3
1 in odd_ge_3

>>>> print isinstance(1, const)
> False
>
>>>> print isinstance("bleh", const2)
> True
>
>>> print isinstance("bleh", const3)
> True

"bleh" in ends_in_h
"bleh" in ends_in_h_or_H

> Callables are supported as well, so if you wanted to do something like:
>
> Constraints(isinstance(X.attr, someclass), somefunc(X[-2].attr, args))
>
> You could approximate that with:
>
> Constraints(lambda x: isinstance(x.attr, someclass), lambda x:
> somefunc(x[-2].attr, args))
>
> As I mentioned in the first paragraph, Constraints is a metaclass,

This is your first mention, actually.

 > so your validations are checked using __instancecheck__.

But it is a fake check in that 3 is not really an instance of the class, 
which has no instances. It is hard for me to believe that you cannot put 
the same constraint data in an instance of Constraint as a class and the 
inclusion logic in __contains__. If you customize __instancecheck__ for 
each instance of Constraints, then write

   def__contains__(self, ob): return self._check(ob)

where _check does the same as your current __instancecheck__

Even if you *have* to make Constraints a metaclass, for other reasons, I 
believe you could still give it the same __contains__ method. A 
metaclass *is* a class, and if its class instances represent 
collections, inclusion in the colleciton should be tested in the 
standard way.

> I'm also
> considering having __init__ generate mock objects (for certain
> straight-forward cases, anyhow).

-- 
Terry Jan Reedy



From guido at python.org  Fri Jan 13 00:11:24 2012
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jan 2012 15:11:24 -0800
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <20120112221136.GA32550@pantoffel-wg.de>
References: <4F0EF8E6.6030002@btinternet.com>
	<20120112213223.GA7309@cskk.homeip.net>
	<20120112221136.GA32550@pantoffel-wg.de>
Message-ID: <CAP7+vJJr5RPSRSiyw3i442PQdY2tJXMAFdtWZh=HLGcS5uwUQQ@mail.gmail.com>

On Thu, Jan 12, 2012 at 2:11 PM, Sven Marnach <sven at marnach.net> wrote:

> Cameron Simpson schrieb am Fr, 13. Jan 2012, um 08:32:23 +1100:
> > On 12Jan2012 15:14, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
> > | Strictly speaking I think the default value should be
> > | os.path.curdir, not specifically '.'
> [...]
> >   -1 for os.curdir() being opendir's default.
>
> There's a misunderstanding here.  You are confusing 'os.getcwd()' and
> 'os.curdir'.
>
> The default value for the parameter of 'os.listdir()' should probably
> be 'os.curdir', but since it is defined in the different
> platform-specific OS modules like posixmodule.c, it probably is
> 'os.curdir'.  Moreover, I'm not aware of a platform where
> os.curdir != '.'
>

The default for os.listdir() should be the current directory by whatever
name it is known. os.curdir is ':' in macpath.py (which applies to MacOS
1-9, not OS X).

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120112/a35ffdfc/attachment.html>

From cs at zip.com.au  Fri Jan 13 00:24:59 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 13 Jan 2012 10:24:59 +1100
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <87ipkg7e2l.fsf@benfinney.id.au>
References: <87ipkg7e2l.fsf@benfinney.id.au>
Message-ID: <20120112232459.GA32459@cskk.homeip.net>

On 13Jan2012 09:26, Ben Finney <ben+python at benfinney.id.au> wrote:
| Cameron Simpson <cs at zip.com.au> writes:
| > I think it should be '.' [instead of ?os.path.curdir?].
| >
| > It is possible to arrange directory permissions so that you can't open
| > curdir because it traverses a directory to which one lacks permissions,
| > but you can still open '.'.
| 
| What distinction are you drawing?
| 
|     >>> os.path.curdir
|     '.'

A totally bogus distinction. I had os.getcwd() in my head instead.
Please ignore my post:-(

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

This signature was originally recorded on analog equipment. While we have
attempted to preserve the meaning of the original, the high resolution of
this media may reveal limitations of the source.
        - paul-michael agapow <agapow at latcs1.oz.au>


From cs at zip.com.au  Fri Jan 13 00:26:04 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 13 Jan 2012 10:26:04 +1100
Subject: [Python-ideas] os.listdir default to '.'
In-Reply-To: <20120112221136.GA32550@pantoffel-wg.de>
References: <20120112221136.GA32550@pantoffel-wg.de>
Message-ID: <20120112232604.GA914@cskk.homeip.net>

On 12Jan2012 22:11, Sven Marnach <sven at marnach.net> wrote:
| Cameron Simpson schrieb am Fr, 13. Jan 2012, um 08:32:23 +1100:
| > On 12Jan2012 15:14, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
| > | Strictly speaking I think the default value should be
| > | os.path.curdir, not specifically '.'
| [...]
| >   -1 for os.curdir() being opendir's default.
| 
| There's a misunderstanding here.  You are confusing 'os.getcwd()' and
| 'os.curdir'.

Yes.

Please ignore the rest of that post, based as it is on a brain fart on
my part.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

It must be public fact, because I'm not the only one who knows about it.
        - Stefan A. Werner <iczer-1!saw at uunet.uu.net>


From pyideas at rebertia.com  Fri Jan 13 07:22:26 2012
From: pyideas at rebertia.com (Chris Rebert)
Date: Thu, 12 Jan 2012 22:22:26 -0800
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
 from the inside out)
In-Reply-To: <jenos5$g8d$1@dough.gmane.org>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<jenos5$g8d$1@dough.gmane.org>
Message-ID: <CAMZYqRQ1XbdyU9=AzgJe92z3JoqM4a9aHKRtq2PM83RgVkGKcw@mail.gmail.com>

On Thu, Jan 12, 2012 at 3:03 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 1/12/2012 3:45 PM, Nathan Rice wrote:
<snip>
>>>>> print isinstance(3, const)
>>
>> True
>
>
> A Contraints instance defines a set. 'const' is the set 'odd_ge_3'
> It would look better if you used standard syntax and do the inclusion check
> in a __contains__ method.
>
>>>> 3 in odd_ge_3
>
> True

But what are types but abstract sets of values? Phrasing it as a
typecheck is perfectly sensible from a type-theoretic standpoint.
Also, the problem of representing `isinstance(X.attr, someclass)` [for
non-Constraint someclass, e.g. str] in a Constraint would still
remain, since there's no "__lcontains__" (thus precluding `X.attr in
str`).

<snip>
>> so your validations are checked using __instancecheck__.
>
> But it is a fake check in that 3 is not really an instance of the class,
> which has no instances.

The same can be true for abstract base classes, which have been
sufficiently accepted to warrant adding __instancecheck__() in the
first place and also to be added to the std lib (witness the `abc` and
`collections` modules). It may seem unfamiliar, but then again it was
only made possible starting with v2.6, which is pretty recent.

Cheers,
Chris
--
http://rebertia.com


From nathan.alexander.rice at gmail.com  Fri Jan 13 07:24:35 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Fri, 13 Jan 2012 01:24:35 -0500
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
 from the inside out)
In-Reply-To: <jenos5$g8d$1@dough.gmane.org>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<jenos5$g8d$1@dough.gmane.org>
Message-ID: <CAOFbRm+c49hE_fruQOgYH=ijcp4ZbcwSCHe0xvf2k5UXafQpXw@mail.gmail.com>

>> I have been writing a lot of code lately that involves creating
>> symbolic expressions of one form or another, which are then fully
>> evaluated at a later time. ?Examples of this include Elementwise,
>
>
> Python is designed for concrete, rather than symbolic computation. But the
> latter has been done.

Being able to create abstract expressions that are later realized is
super useful and neat.  You can do this decently right now, but life
would be better if you didn't have to jump through so many hoops.
Having symbolic variables override *anything* would also make lambdas
obsolete and greatly increase the potential for lazy evaluation.

> 0: rewrite the text -- a bit awkward in Python.
>
> action = compile("read({this})".format(this=<book>), 'xxx', 'eval')

Yeah, that is something I'd expect to see in Perl code :)

> 1. default args -- has to be done when the function is defined.
>
> def action(this = <book>): read(this)
>
> 2. closures (nested functions) -- also requires a planned-ahead definition.
>
> make_read(x):
> ?return lambda: read(x)
> action = make_read(<book>)

I use this extensively in Elementwise.

> 3. bound methods -- only works for classes with methods.
>
> Class Book:
> ? ?def read(self): pass
> action = Book(<book>).read
>
> 4. partial binding of function params -- generalizes bound methods; works
> for any function and argument.
>
> from functools import partial
> action = partial(read, <book>)
>
>> if I want to include isinstance(X, someclass) in a symbolic expression,
>
>
> Consider using partial, which can totally bind all needed args *now* for
> *later* action.

The issue isn't so much that I *can't* do things as they are more
trouble than they should be, and it makes the end user interface for
the stuff I write less elegant.

For example, if I want to work with a symbolic object, but include a
function that is not well behaved, or if I was creating a constraint
on the result of a function applied to a symbolic object, I have to
know ahead of time everything I want to do, so I can wrap the whole
expression in a lambda.  Once I wrap it, the nice generative chaining
property disappears and I'm stuck with a callable.

>>>> from functools import partial
>>>> t = partial(isinstance, 1, int)
>>>> t()
> True
>>>> f = partial(isinstance, 1, float)
>>>> f()
> False
>
>
>> I have to wrap it in a lambda
>
>
> Are you sure that partial will not work for you? Since partial is written in
> Python, you can grab and adjust the code to your needs. It amounts to adding
> default args after the fact by using a generic closure.

In some cases it would, in some cases it wouldn't.  Since I basically
never do */** expansion on wrappers, lambdas tend to be my go-to more
often.

>> (or use .apply(), in the case of
>>
>> Elementwise). ?This is not a huge deal for me, but it forces me to
>> create wrappers for lots of functions (e.g. isinstance_(X, someclass))
>> and/or have users wrap every such function they want to use in a
>> symbolic expression. ? Having to do this also bloats the code a lot;
>> the github version of Elementwise is over 3,000 LoC at this point
>> (including prodigious documentation, but still...).
>>
>> 2. ?Python expects that certain functions, such as int(), str(), etc,
>> will have a specific return type. ?While in general I agree with this,
>
>
> People expect that class constructors produce an instance of the class. It
> is surprising when one does otherwise ;-). Builtin classes like int, bool,
> and str are classes just like ones you write.

type/str/int/etc as types is definitely semi-coherent, since the
language really treats them more like functions.  They are treated
that way all over the docs as well.

>From the data model page:

"object.__str__(self)
Called by the str() built-in function and by the..."

"object.__nonzero__(self)
Called to implement truth value testing and the built-in operation bool()"

"object.__complex__(self)
object.__int__(self)
object.__long__(self)
object.__float__(self)
Called to implement the built-in functions complex(), int(), long(),
and float(). Should return a value of the appropriate type."

So clearly this is an area that needs some polish :)

>> X = SymbolicObject()
>>
>> const = Constraints(X * 2 + 1>= 5, X % 2 != 0)
>> const2 = Constraints(X[-1] == "h")
>> const3 = Constraints(X[-1].upper() == "H")
>
>
> Using a special class is a standard way to delay concrete execution.

Standard, and currently a pain in the butt, starting from the fact
that operators don't hook into __getattribute__ and getting
progressively worse from there.

>>>>> print isinstance(3, const)
>>
>> True
>
>
> A Contraints instance defines a set. 'const' is the set 'odd_ge_3'
> It would look better if you used standard syntax and do the inclusion check
> in a __contains__ method.

Used standard syntax?  Can you elaborate please?

Also, a set is one of many things a Constraints instance could
logically be represented with, as well as a discontinuous interval, a
class in the colloquial sense, etc.  The nice thing about
__instancecheck__ is that every possible constraint reduces to a type
check.  Of course, you could rephrase all type checking in terms of
set membership easily, but I don't think it is quite as intuitive to
most folks.  Your preference has been noted though.

>> As I mentioned in the first paragraph, Constraints is a metaclass,
>
>
> This is your first mention, actually.

Feeling feisty?  I'm actually curious to see what sort of rational you
come up with to back up that statement; should be interesting :)

>
>> so your validations are checked using __instancecheck__.
>
> But it is a fake check in that 3 is not really an instance of the class,
> which has no instances. It is hard for me to believe that you cannot put the
> same constraint data in an instance of Constraint as a class and the
> inclusion logic in __contains__. If you customize __instancecheck__ for each
> instance of Constraints, then write

It is a fake check, like any abstract base class instancecheck is a fake check.

> ?def__contains__(self, ob): return self._check(ob)
>
> where _check does the same as your current __instancecheck__
>
> Even if you *have* to make Constraints a metaclass, for other reasons, I
> believe you could still give it the same __contains__ method. A metaclass
> *is* a class, and if its class instances represent collections, inclusion in
> the colleciton should be tested in the standard way.

It could be any sort of callable.  __instancecheck__ is the only
reason it is a metaclass.  Otherwise, I probably wouldn't bother with
classes at all; returning a check inner function with constraints in
the closure would be easy.

Cheers,

Nathan


From jeanpierreda at gmail.com  Fri Jan 13 14:45:38 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Fri, 13 Jan 2012 08:45:38 -0500
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
 from the inside out)
In-Reply-To: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
Message-ID: <CABicbJJG60vQnMDM3_6JYuvmWY5OtOdptDdqVP4HC2O-TEiF=Q@mail.gmail.com>

On Thu, Jan 12, 2012 at 3:45 PM, Nathan Rice
<nathan.alexander.rice at gmail.com> wrote:
> I'm interested in fixing both issues. I believe both issues I've had
> could be solved by having a robust "symbolic object".  These objects
> would basically usable like ordinary objects, however upon any
> attribute access or other form of interaction, the object would
> basically short circuit the calling function, and return a symbolic
> object directly to the outer scope.  The symbolic object would behave
> like a generator function frozen at the point of attribute access, and
> upon send()-ing (or whatever method), it would behave exactly as if
> the values sent had been the ones passed in originally (ideally
> without consuming the generator).

I find the way you've formalized this a bit "weird". It looks like
you're suggesting adding laziness to Python.

If that's what you want, maybe you should try PyPy and the thunk object space:

http://doc.pypy.org/en/latest/objspace-proxies.html#the-thunk-object-space

-- Devin


From nathan.alexander.rice at gmail.com  Fri Jan 13 15:21:07 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Fri, 13 Jan 2012 09:21:07 -0500
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
 from the inside out)
In-Reply-To: <CABicbJJG60vQnMDM3_6JYuvmWY5OtOdptDdqVP4HC2O-TEiF=Q@mail.gmail.com>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<CABicbJJG60vQnMDM3_6JYuvmWY5OtOdptDdqVP4HC2O-TEiF=Q@mail.gmail.com>
Message-ID: <CAOFbRm+AD4dZjwjrZFw9wCLCsKW=PDshBMjhLnzpDK=gg2BBFA@mail.gmail.com>

On Fri, Jan 13, 2012 at 8:45 AM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
> On Thu, Jan 12, 2012 at 3:45 PM, Nathan Rice
> <nathan.alexander.rice at gmail.com> wrote:
>> I'm interested in fixing both issues. I believe both issues I've had
>> could be solved by having a robust "symbolic object". ?These objects
>> would basically usable like ordinary objects, however upon any
>> attribute access or other form of interaction, the object would
>> basically short circuit the calling function, and return a symbolic
>> object directly to the outer scope. ?The symbolic object would behave
>> like a generator function frozen at the point of attribute access, and
>> upon send()-ing (or whatever method), it would behave exactly as if
>> the values sent had been the ones passed in originally (ideally
>> without consuming the generator).
>
> I find the way you've formalized this a bit "weird". It looks like
> you're suggesting adding laziness to Python.
>
> If that's what you want, maybe you should try PyPy and the thunk object space:

While thunk is neat, it doesn't accomplish precisely what I'm
describing in this instance.  When a function starts to run under
thunk, the computations take place as soon as the function gets to the
object inside its scope.

What I'm after is the ability to basically create functions using only
expressions, in a generative manner.  Lambda does accomplish this, but
in an extremely clunky manner... for example:

X = lambda x: x + 1
Y = lambda y: y * 2
Z = lambda z: z % 3

(or XYZ = lambda x: (((x + 1) * 2) % 3)

If I want to perform a second step after this, I need create another
lambda, because they don't chain/aren't generative.

The thing that Elementwise does that is very appealing is that most
operations are generative, so if you have an ElementwiseProxy object
x, x2 = (((x + 1) * 2) % 3) basically does the same thing as the
above, but you can then do x3 = x2 ** 3, and so on until you are ready
to get your results.  Once you have your results, you can use the same
ElementwiseProxy again with different inputs.  It is just a function
generation technique which has some elegant properties.  Having a
native object type designed to facilitate this would let people do a
lot of interesting things, and make things like Elementwise be a lot
more "consistent".  I bet you could do some really neat things by
subclassing such an object as well.

This is certainly not a common programming paradigm, but I think most
people could reap a lot of benefits from it while being oblivious to
its presence.

Nathan


From eike.welk.lists1 at gmx.de  Fri Jan 13 15:53:35 2012
From: eike.welk.lists1 at gmx.de (Eike Welk)
Date: Fri, 13 Jan 2012 15:53:35 +0100
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
	from the inside out)
In-Reply-To: <CABicbJJG60vQnMDM3_6JYuvmWY5OtOdptDdqVP4HC2O-TEiF=Q@mail.gmail.com>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<CABicbJJG60vQnMDM3_6JYuvmWY5OtOdptDdqVP4HC2O-TEiF=Q@mail.gmail.com>
Message-ID: <12518550.K8fIVbB0Kl@lixie>

On Friday 13.01.2012 08:45:38 Devin Jeanpierre wrote:

> I find the way you've formalized this a bit "weird". It looks like
> you're suggesting adding laziness to Python.

IMHO this is quoting like in Lisp: He stores a bit of code in an un-evaluated 
form. The bit of code is then interpreted somewhere else, in special contexts, 
that he can choose with his hacked version of "isinstance". 

Eike.


From phd at phdru.name  Fri Jan 13 16:10:01 2012
From: phd at phdru.name (Oleg Broytman)
Date: Fri, 13 Jan 2012 19:10:01 +0400
Subject: [Python-ideas] Symbolic expressions
In-Reply-To: <12518550.K8fIVbB0Kl@lixie>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<CABicbJJG60vQnMDM3_6JYuvmWY5OtOdptDdqVP4HC2O-TEiF=Q@mail.gmail.com>
	<12518550.K8fIVbB0Kl@lixie>
Message-ID: <20120113151001.GC26517@iskra.aviel.ru>

On Fri, Jan 13, 2012 at 03:53:35PM +0100, Eike Welk wrote:
> On Friday 13.01.2012 08:45:38 Devin Jeanpierre wrote:
> 
> > I find the way you've formalized this a bit "weird". It looks like
> > you're suggesting adding laziness to Python.
> 
> IMHO this is quoting like in Lisp: He stores a bit of code in an un-evaluated 
> form. The bit of code is then interpreted somewhere else, in special contexts, 
> that he can choose with his hacked version of "isinstance". 

   Symbolic algebra is a widely-deployed technique in Python world. It
is popular, e.g., in object-relational mappers. IMHO Python is powerful
enough in that area, there is no need to change the language.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From jeanpierreda at gmail.com  Fri Jan 13 17:24:31 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Fri, 13 Jan 2012 11:24:31 -0500
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
 from the inside out)
In-Reply-To: <12518550.K8fIVbB0Kl@lixie>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<CABicbJJG60vQnMDM3_6JYuvmWY5OtOdptDdqVP4HC2O-TEiF=Q@mail.gmail.com>
	<12518550.K8fIVbB0Kl@lixie>
Message-ID: <CABicbJLuyy3WiLBFVFNu5=O_UVkpv8Y+DGNtmFhFC6aQrdLNxw@mail.gmail.com>

On Fri, Jan 13, 2012 at 9:53 AM, Eike Welk <eike.welk.lists1 at gmx.de> wrote:
> IMHO this is quoting like in Lisp: He stores a bit of code in an un-evaluated
> form. The bit of code is then interpreted somewhere else, in special contexts,
> that he can choose with his hacked version of "isinstance".
>

I wanted to think he wanted quote, because quote is awesome, but that
doesn't gel with his desired behavior inside functions.

def foo(A):
    return (A + 2) * 5

(define (foo A)
    (* (+ A 2) 5)))

If in Python one calls foo(SymbolicExpr + 3), then it returns a
SymbolicExpr that can be lazily evaluated to compute the expression.
This is fairly general and could work even with mutating statements
and all sorts of nasty things.

If, in Scheme, one calls (foo '(+ X 3)), it is an error. There's no
automatic laziness associated with quoted expressions -- they're just
data. If one called (foo '3) then it would outright eagerly evaluate
the function, returning 25, because '3 == 3.

OTOH, you can define a foo that operates on quoted expressions and
would return '(* (+ (+ X 3) 2) 5), as follows:

(define (foo A)
    `(* (+ ,A 2) 5))

That might be sufficient for his use-case, can't tell. It isn't
specifically what he asked for, though. (See also: XY problem).

-- Devin


From eike.welk.lists1 at gmx.de  Sat Jan 14 04:25:50 2012
From: eike.welk.lists1 at gmx.de (Eike Welk)
Date: Sat, 14 Jan 2012 04:25:50 +0100
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
	from the inside out)
In-Reply-To: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
Message-ID: <1462421.BIX0U07Ti7@lixie>

I think something like your SymbolicObject class should be in Python's 
standard library. Quite a few projects implement similar classes. If there is 
a basic infrastructure for symbolic computation in the standard library, 
symbolic algorithms can be reused much more simple across projects.

The pattern you employ with SymbolicObject, is a nice way to mix code that is 
executed immediately, with code that is executed later. Symbolic algebra 
languages, like Maple and Mathematica, function this way too.


I propose however a slightly different interface:

There should be a special object "quoted" to create instances of 
"SymbolicObject". The objects should be created in the "__getattribute__" 
method. This way the objects know their name, which is quite useful. A 
symbolic object would be created like this:

    X = quoted.X

You could also write:

    const = Constraints(quoted.X * 2 + 1 >= 5, quoted.X % 2 != 0)

The (tree of) symbolic objects should have a method that translate it to an 
abstract syntax tree (ast.AST from the standard library). Subsequent 
algorithms should operate on this AST. This way symbolic algorithms can also 
operate on all possible Python code (SymbolicObject can only create 
expressions).

    my_ast = (quoted.X * 2 + 1).to_ast()

For convenience I think that all symbolic algorithms should also accept 
SymbolicObject, and convert them to ast.AST internally.


The standard library should contain at least these symbolic algorithms:

evalsym(tree, environment, return_type="any")
    Evaluates an expression and returns the result. Works similar to:

    eval(compile(tree, "", "eval"), environment)

    This call would return 7:

        evalsym(quoted.X * 2 + 1, {"X":3})

    However it should do partial evaluation if some symbols are 
    unknown, and return an ast.AST in this case (similar to Maple and
    Mathematica). This expression should be True:

        evalsym(quoted.X + quoted.Y, {"Y":3}) == quoted.X + 3

    The optional argument "return_type" specifies the type of the 
    result. This simplifies algorithms using "eval". Argument 
    "return_type" can be: "any", "ast", or "object".

substitute(tree, substitutions)
    Replaces a symbol with a value (a sub-tree of ast.Ast).
    This expression should be True:

        substitute(quoted.X + quoted.Y, {"Y": quoted.X * 2}) \
        == (quoted.X + quoted.X * 2).to_ast()

    Maybe this algorithm should also be able to match general fragments of
    the tree and replace them.


Eike.


From liu at cs.stonybrook.edu  Sat Jan 14 06:25:19 2012
From: liu at cs.stonybrook.edu (Annie Liu)
Date: Sat, 14 Jan 2012 05:25:19 +0000
Subject: [Python-ideas] quantifications, and tuple patterns
Message-ID: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>

Hi! 
 
I'd like to propose adding quantifications to python.  It'd be very 
helpful to have tuple patterns as well. 
 
I found extensive use of quantifications while learning and teaching 
distributed algorithms in the last few years, and I saw more and more 
places where they can help make programming easier and make programs 
clearer and less error-prone.  They didn't show up this much until I 
started to pay attention, since my students (even the best ones) and 
myself too just coded them away too quickly. 
 
For example, even the simplest "some x in s has pred", when a witness 
for x is not needed, could end up like "{x for x in s if pred}", 
"any(pred for x in s)", "len(...)!=0", "len...>0", len...>=1", ... not 
to mention the many more kinds of loops and tests written by the 
majority of students not used to writing queries/comprehensions.  Even 
more kinds of loops and bigger hassles are involved when a witness for 
x is needed. 
 
For another example, a simple "each x in s has x > y" tends to end up 
like "min(s)>y", which is incorrect, since s could be empty, so one 
has to either add this condition or invent some boundary value for 
min, which is error-prone since it is sensitive to the possible values 
of elements of s and the uses of these values. 
 
For an example distributed algorithm, one may look at Robbert von 
Renesse's "Paxos made moderately complex", at 
http://www.cs.cornell.edu/courses/CS7412/2011sp/paxos.pdf (he has 
newer versions but the pseudo code didn't change).  It has 8 uses of 
quantifications.  His algorithm can be coded almost exactly like that 
in our language DistAlgo (python with small extensions for distributed 
programming) if we have quantifications and tuple patterns.  For 
example (using "!" to indicate a bound variable in tuple patterns), 
 
        while some (!slot_num,p1) in decisions: 
           if some (!slot_num,p2) in proposals has p2 != p1: 
 
is currently written as follows (it happens to be correct here to use 
"for" for "if" but generally requires more code to be correct): 
 
        while {p1 for (s0,p1) in decisions if s0==slot_num}: 
           p1 = {p1 for (s0,p1) in decisions if s0==slot_num}.pop() 
           for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 != p1}: 
 
Both a witness for p1 and a witness for p2 are needed here. (BTW a 
best student working on this, after given advanced warnings and hints, 
wrote a 4-line nested "for"s and "if"s that were actually incorrect)

So we started to think about adding quantifications and tuple patterns 
to DistAlgo too (other extensions are all distributed programming 
features). 
 
In general, "some x in s has pred" should return true/false AND bind x 
to a witness if true is returned.  One might object to the side effect 
of the binding, but it is most natural here for ease of programming 
and clarity of programs, and it helps program efficiency too.  Witness 
in existential quantification is also standard in logic. 
 
Of course, there is the obvious problem of new keywords: "some", 
"each", "has" (perhaps one may use ":" for "has") or some others.  The 
"!"  syntax would be ok for tuple patterns but only if there are no 
"equal cards", i.e., multiple occurrences of a same variable (though 
such occurrences don't seem to be useful in current python semantics 
for "for".  A special symbol for wildcard would be more useful).  So 
we are looking for good syntax. 

Thanks, 
Annie Liu


From cmjohnson.mailinglist at gmail.com  Sat Jan 14 07:27:35 2012
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Fri, 13 Jan 2012 20:27:35 -1000
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
	from the inside out)
In-Reply-To: <1462421.BIX0U07Ti7@lixie>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<1462421.BIX0U07Ti7@lixie>
Message-ID: <0567BC83-2437-4881-8F8D-F6E47DB76B72@gmail.com>

You guys know this can more or less be done in Python today, right?

>>> class Symbolic:
...     def __init__(self, func=lambda x: x):
...         self.func = func
...     
...     def __call__(self, arg):
...         return self.func(arg)
...     
...     def __add__(self, other):
...         func = lambda x: self.func(x) + other
...         return Symbolic(func)
...     
...     __radd__ = __add__
...     
...     def __mul__(self, other):
...         func = lambda x: self.func(x) * other
...         return Symbolic(func)
...     
...     __rmul__ = __mul__
... 
>>> 
>>> X = Symbolic()
>>> Y = (2 * X) + 10
>>> Y(20)
50

I leave further extensions as an exercise for the reader, but it's not especially tricky to add various boolean tests and whatnot.

I don't claim any originality here. I'm fairly sure I read about this idea in blog post somewhere years ago.

From p.f.moore at gmail.com  Sat Jan 14 13:32:05 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 14 Jan 2012 12:32:05 +0000
Subject: [Python-ideas] Fwd:  quantifications, and tuple patterns
In-Reply-To: <CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
Message-ID: <CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>

Sorry, should have replied on list.

On 14 January 2012 05:25, Annie Liu <liu at cs.stonybrook.edu> wrote:
> For example, even the simplest "some x in s has pred", when a witness
> for x is not needed, could end up like "{x for x in s if pred}",
> "any(pred for x in s)", "len(...)!=0", "len...>0", len...>=1", ... not
> to mention the many more kinds of loops and tests written by the
> majority of students not used to writing queries/comprehensions. ?Even
> more kinds of loops and bigger hassles are involved when a witness for
> x is needed.

I'm confused as to why any(pred for x in s) is unacceptable here?

> For another example, a simple "each x in s has x > y" tends to end up
> like "min(s)>y", which is incorrect, since s could be empty, so one
> has to either add this condition or invent some boundary value for
> min, which is error-prone since it is sensitive to the possible values
> of elements of s and the uses of these values.

all(x > y for x in s) seems to me both idiomatic and correct.

I'm not sure quite what you're proposing here - it seems like you're
asking for new syntax to do the same job as any and all, which already
do exactly what's needed. New syntax has to provide something much,
much better if it's likely to be accepted.

Please explain more clearly.
Paul.


From liu at cs.stonybrook.edu  Sat Jan 14 14:57:14 2012
From: liu at cs.stonybrook.edu (Annie Liu)
Date: Sat, 14 Jan 2012 13:57:14 +0000
Subject: [Python-ideas] Fwd:  quantifications, and tuple patterns
In-Reply-To: <CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
Message-ID: <913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>

Hi!

For those two simplest examples, yes, "any" and "all" can be used.
However, there are two general problems.

1.  The order of coding using "any/all" is the opposite order of
thinking in one's head.  In our experience, those kinds of simple
examples are coded much more often using "len" than "any/all".  The
ordering problem gets worse when quantifications are needed in more
complex conditions or are nested.

2.  The much worse problem is when a witness is needed from existential
quantification.  In my last/third example from a distributed algorithm,
this is the desired code (recall that ! indicates a bound variable):

  while some (!slot_num,p1) in decisions: 
     if some (!slot_num,p2) in proposals has p2 != p1: 
        propose(p2) 
     perform(p1) 

"any" is not sufficient to code this easily and clearly.  Such
witnesses are needed in all worklist algorithms.

Thanks,
Annie



From p.f.moore at gmail.com  Sat Jan 14 17:19:29 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 14 Jan 2012 16:19:29 +0000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
Message-ID: <CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>

On 14 January 2012 13:57, Annie Liu <liu at cs.stonybrook.edu> wrote:
> For those two simplest examples, yes, "any" and "all" can be used.
> However, there are two general problems.
>
> 1. ?The order of coding using "any/all" is the opposite order of
> thinking in one's head. ?In our experience, those kinds of simple
> examples are coded much more often using "len" than "any/all". ?The
> ordering problem gets worse when quantifications are needed in more
> complex conditions or are nested.

To be honest, that's a matter of opinion. I find any/all with a
generator expression more readable than your syntax, because I am used
to generator expressions as they are common in Python.

> 2. ?The much worse problem is when a witness is needed from existential
> quantification. ?In my last/third example from a distributed algorithm,
> this is the desired code (recall that ! indicates a bound variable):
>
> ?while some (!slot_num,p1) in decisions:
> ? ? if some (!slot_num,p2) in proposals has p2 != p1:
> ? ? ? ?propose(p2)
> ? ? perform(p1)

Wow, I find that unreadable!. The ! doesn't read as any sort of
assignment to me. Don't forget that in Python, assignments are
statements and don't get embedded in expressions. There is no way
you're going to get new syntax that creates an expression which embeds
a hidden assignment accepted into Python.

I'd offer to re write this in idiomatic Python, but looking at it
closer, I've no idea how to. I don't know what slot_num is meant to
be, at first I thought p1 and p2 were predicates, but then I notice
you are comparing them later.

Can I suggest you write your example out in Python that works today,
and then show how it looks with your proposed syntax alongside? If you
can't find the "best" way of writing it in existing Python, just write
it however works, no need to try to make it compact, or elegant.
There'll be plenty of people here who will show you how to write
idiomatic Python versions of what you post :-)

Paul.


From guido at python.org  Sat Jan 14 20:24:11 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 14 Jan 2012 11:24:11 -0800
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
Message-ID: <CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>

On Sat, Jan 14, 2012 at 8:19 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 14 January 2012 13:57, Annie Liu <liu at cs.stonybrook.edu> wrote:
>

Hi Annie! (*)


>  > For those two simplest examples, yes, "any" and "all" can be used.
> > However, there are two general problems.
> >
> > 1.  The order of coding using "any/all" is the opposite order of
> > thinking in one's head.  In our experience, those kinds of simple
> > examples are coded much more often using "len" than "any/all".  The
> > ordering problem gets worse when quantifications are needed in more
> > complex conditions or are nested.
>
> To be honest, that's a matter of opinion. I find any/all with a
> generator expression more readable than your syntax, because I am used
> to generator expressions as they are common in Python.
>

But Paul, aren't you missing the fact that for the algorithms that Annie
and her students want to write, the "witness" concept is essential? I.e.
they can't just use any(P(x) for x in xs) because if it returns True, they
want to know the x that made P(x) be true. Her ! notation is a (perhaps
unpythonic) attempt at exporting this witness from the quantification.

> 2.  The much worse problem is when a witness is needed from existential
> > quantification.  In my last/third example from a distributed algorithm,
> > this is the desired code (recall that ! indicates a bound variable):
> >
> >  while some (!slot_num,p1) in decisions:
> >     if some (!slot_num,p2) in proposals has p2 != p1:
> >        propose(p2)
> >     perform(p1)
>
> Wow, I find that unreadable!. The ! doesn't read as any sort of
> assignment to me. Don't forget that in Python, assignments are
> statements and don't get embedded in expressions.


I think she's well aware of that, and proposing to change it. :-)

There is no way
> you're going to get new syntax that creates an expression which embeds
> a hidden assignment accepted into Python.
>

That's your opinion. But this is python-ideas. :-)

I'd offer to re write this in idiomatic Python, but looking at it
> closer, I've no idea how to. I don't know what slot_num is meant to
> be, at first I thought p1 and p2 were predicates, but then I notice
> you are comparing them later.
>
> Can I suggest you write your example out in Python that works today,
> and then show how it looks with your proposed syntax alongside? If you
> can't find the "best" way of writing it in existing Python, just write
> it however works, no need to try to make it compact, or elegant.
> There'll be plenty of people here who will show you how to write
> idiomatic Python versions of what you post :-)
>

Actually she gave one in her first post. Here it is again:

       while {p1 for (s0,p1) in decisions if s0==slot_num}:
          p1 = {p1 for (s0,p1) in decisions if s0==slot_num}.pop()
          for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 !=
p1}:

Note that the set {p1 for (s0,p1) in decisions if s0==slot_num} is computed
twice, once to decide whether to stop, and then again to compute the
witness (p1). Obviously this is inefficient, and that's what she's after.
To make this same code more efficient in Python, you'd have to do the
following, which is natural for us programmers (since we're so used to
working around limitations and inefficiencies in the systems we work with)
but unnatural for mathematicians, who count to infinity at the drop of a
hat:

while True:
  temp = {p1 for (s0,p1) in decisions if s0==slot_num}
  if not temp:
    break
  p1 = temp.pop()
  for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 != p1}:
    <whatever>

The 5 tedious lines from "while" through "pop()" would be collapsed into a
single line if you could write

  while some s0, p1 in decisions if s0 == slot_num:
  for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 != p1}:
    <whatever>

TBH I'm not sure what the !slot_num notation is for -- it appears that

  while some (!slot_num, p1) in decisions:

is equivalent in Annie's proposal to

  while some s0, p1 in decisions if s0 == slot_num:

but I'm not sure and it doesn't feel necessary to me.

Also note that SOME and EACH quantifiers were present in ABC (Python's
predecessor: http://homepages.cwi.nl/~steven/abc/qr.html#TESTS); I dropped
them for simplicity, not because I didn't like them. If we wanted to we
could have them back (except for the problems of introducing new keywords).

__________
(*) See
http://neopythonic.blogspot.com/2011/06/depth-and-breadth-of-python.html

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120114/f2121e31/attachment.html>

From nathan.alexander.rice at gmail.com  Sat Jan 14 21:13:36 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Sat, 14 Jan 2012 15:13:36 -0500
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
 from the inside out)
In-Reply-To: <0567BC83-2437-4881-8F8D-F6E47DB76B72@gmail.com>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<1462421.BIX0U07Ti7@lixie>
	<0567BC83-2437-4881-8F8D-F6E47DB76B72@gmail.com>
Message-ID: <CAOFbRmJQnVObBemu-nKGvc-MPHcMS=_8_dKhFQRVTP92HCDAKw@mail.gmail.com>

On Sat, Jan 14, 2012 at 1:27 AM, Carl M. Johnson
<cmjohnson.mailinglist at gmail.com> wrote:
> You guys know this can more or less be done in Python today, right?

I did mention in the first post that I am the author of a library that
does this right? :D

The issue is of consistency.  some functions can break out of this
behavior.  Some "type" functions also want to behave a certain way
regardless of input (which makes sense in most cases but breaks
chaining symbolic expressions).

Additionally, the amount of code required to do that sort of object
proxying is really obnoxious.  Things like operator methods should be
hooked into__getattribute__ on symbolic objects.

There are also a LOT of neat possibilities that come out of symbolic
objects, for instance, function templates, fairly easy currying and
generation of closed form solutions by performing calculations on
functions rather than data.

Nathan


From greg.ewing at canterbury.ac.nz  Sat Jan 14 21:18:37 2012
From: greg.ewing at canterbury.ac.nz (Greg)
Date: Sun, 15 Jan 2012 09:18:37 +1300
Subject: [Python-ideas] Symbolic expressions
In-Reply-To: <20120113151001.GC26517@iskra.aviel.ru>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<CABicbJJG60vQnMDM3_6JYuvmWY5OtOdptDdqVP4HC2O-TEiF=Q@mail.gmail.com>
	<12518550.K8fIVbB0Kl@lixie> <20120113151001.GC26517@iskra.aviel.ru>
Message-ID: <4F11E31D.50403@canterbury.ac.nz>

Oleg Broytman wrote:

> Symbolic algebra is a widely-deployed technique in Python world. It
> is popular, e.g., in object-relational mappers. IMHO Python is powerful
> enough in that area, there is no need to change the language.

The existing techniques for this get a bit awkward at times,
though, for example when you want boolean operations, since
'and' etc. can't be overridden.

My Overloadable Boolean Operators PEP was one attempt to
address this problem, but it hasn't met with a very enthusiastic
reception.

Recently I've been working on another approach using that
I've been calling "deferred expressions" (although I'll probably
change that name because "deferred" seems to have taken on a
different meaning already).

The idea is that you write something like this (sorry about the
ugly syntax, I haven't thought of anything better yet):

    e = <(a + b * c)>

This gives you something that behaves in one respect like a
parameterless lambda -- you can evaluate it, and any free
variables are interpreted in their original environment.

However, its internal structure is effectively an AST that
you can pick apart and process however you want, including
selectively evaluating sub-expressions and building up new
expressions.

Currently they're not generative the way the OP wants, i.e.
you can't do 'e1 + e2' and get a new deferred expression, but
that could be arranged, and would provide a nice way of
building new expressions. The only problem would be attribute
access, since they currently have attributes modelled after
those of the corresponding AST nodes, whereas under a
generative regime, 'e.a' ought to return a new expression node
representing the access of attribute 'a'.

I have about 90% of this implemented. I'll make a release
as soon as I have something fit for public consumption.

-- 
Greg


From p.f.moore at gmail.com  Sat Jan 14 22:38:29 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 14 Jan 2012 21:38:29 +0000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
Message-ID: <CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>

On 14 January 2012 19:24, Guido van Rossum <guido at python.org> wrote:
> But Paul, aren't you missing the fact that for the algorithms that Annie and
> her students want to write, the "witness" concept is essential? I.e. they
> can't just use any(P(x) for x in xs) because if it returns True, they want
> to know the x that made P(x) be true. Her ! notation is a (perhaps
> unpythonic) attempt at exporting this witness from the quantification.

Fair point. I was thinking that common cases worked with any(), and
more complex cases that needed a witness would be sufficiently rare
that the extra verbosity would (a) not be a huge burden, and (b) help
to make the intent clearer.

>> > ?while some (!slot_num,p1) in decisions:
>> > ? ? if some (!slot_num,p2) in proposals has p2 != p1:
>> > ? ? ? ?propose(p2)
>> > ? ? perform(p1)
[...]
>> Can I suggest you write your example out in Python that works today,
>> and then show how it looks with your proposed syntax alongside? If you
>> can't find the "best" way of writing it in existing Python, just write
>> it however works, no need to try to make it compact, or elegant.
>> There'll be plenty of people here who will show you how to write
>> idiomatic Python versions of what you post :-)
>
> Actually she gave one in her first post. Here it is again:

I'm sorry about that! I got confused part-way through the original
post and skimmed from there, and then didn't go back and reread it in
the context of the follow-up, so I missed the example. My mistake.

> ? ? ?? while {p1 for (s0,p1) in decisions if s0==slot_num}:
> ? ? ? ? ? p1 = {p1 for (s0,p1) in decisions if s0==slot_num}.pop()
> ? ? ? ? ? for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 !=
> p1}:
>
> Note that the set {p1 for (s0,p1) in decisions if s0==slot_num} is computed
> twice, once to decide whether to stop, and then again to compute the witness
> (p1). Obviously this is inefficient, and that's what she's after.

Agreed, that's inefficient, and it also violates DRY - I can easily
imagine those two lines getting out of sync after a while...

> To make
> this same code more efficient in Python, you'd have to do the following,
> which is natural for us programmers (since we're so used to working around
> limitations and inefficiencies in the systems we work with) but unnatural
> for mathematicians, who count to infinity at the drop of a hat:

Hmm, I have an aversion to languages (or constructs) based around
theoretical principles. Blame a couple of encounters with Haskell a
few years ago. I lost. :-) But it tends to be the over-compressed
syntax rather than the ideas that I'm particularly allergic to.

> while True:
> ? temp = {p1 for (s0,p1) in decisions if s0==slot_num}
> ? if not temp:
> ??? break
> ? p1 = temp.pop()
>
> ? for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 != p1}:
> ? ? <whatever>

Point taken. On the other hand, if (x := val) was an expression form
of assignment, like C's assignment, you could write

while temp := {p1 for (s0,p1) in decisions if s0==slot_num}:
  p1 = temp.pop()

  for s2 ...

which is to my mind as succinct, and clearer to a non-mathematician,
as your proposal below (and Annie's proposal). It also builds off a
much more commonly requested feature :-)

Actually,

while any(p1 := p for (s,p) in decisions if s == slot_num):
  for s2 ...

works, and is just as short as your proposal or Annie's.

> The 5 tedious lines from "while" through "pop()" would be collapsed into a
> single line if you could write
>
> ? while some s0, p1 in decisions if s0 == slot_num:
>
> ? for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 != p1}:
> ? ? <whatever>
>
> TBH I'm not sure what the !slot_num notation is for -- it appears that
>
> ? while some (!slot_num, p1) in decisions:
>
> is equivalent in Annie's proposal to
>
> ? while some s0, p1 in decisions if s0 == slot_num:
>
> but I'm not sure and it doesn't feel necessary to me.

Yes, I think that's right. It looks like the idea comes from the
concept of unification in logic languages (and the ! notation means
"don't unify this value, but rather treat it as a fixed value that
must match"). Thanks for your analysis, by the way, I think understand
the proposal a lot better now.

So, going back to what Annie was referring to, there seem to be three
key concepts:

Quantifications, which are covered in Python by any() and all()
Capturing a "witness", which can be done using assignment-as-expression
Tuple matching, which you have shown can be handled using tuple
unpacking plus the generator expression if clause, but could probably
gain from a more compact notation.

BTW, as I'm sure everyone knows, you can simulate assignment-as-expression with

def asgn(o, val):
  o.ans = val
  return val

>>> any(asgn(c,x) for x in (1,2,3) if x%2 == 0)
True
>>> c.ans
2

> Also note that SOME and EACH quantifiers were present in ABC (Python's
> predecessor: http://homepages.cwi.nl/~steven/abc/qr.html#TESTS); I dropped
> them for simplicity, not because I didn't like them. If we wanted to we
> could have them back (except for the problems of introducing new keywords).

One day, I really must read up on ABC.

Paul.


From guido at python.org  Sun Jan 15 00:07:49 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 14 Jan 2012 15:07:49 -0800
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
Message-ID: <CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>

On Sat, Jan 14, 2012 at 1:38 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 14 January 2012 19:24, Guido van Rossum <guido at python.org> wrote:
> > But Paul, aren't you missing the fact that for the algorithms that Annie
> and
> > her students want to write, the "witness" concept is essential? I.e. they
> > can't just use any(P(x) for x in xs) because if it returns True, they
> want
> > to know the x that made P(x) be true. Her ! notation is a (perhaps
> > unpythonic) attempt at exporting this witness from the quantification.
>
> Fair point. I was thinking that common cases worked with any(), and
> more complex cases that needed a witness would be sufficiently rare
> that the extra verbosity would (a) not be a huge burden, and (b) help
> to make the intent clearer.
>
> >> >  while some (!slot_num,p1) in decisions:
> >> >     if some (!slot_num,p2) in proposals has p2 != p1:
> >> >        propose(p2)
> >> >     perform(p1)
> [...]
> >> Can I suggest you write your example out in Python that works today,
> >> and then show how it looks with your proposed syntax alongside? If you
> >> can't find the "best" way of writing it in existing Python, just write
> >> it however works, no need to try to make it compact, or elegant.
> >> There'll be plenty of people here who will show you how to write
> >> idiomatic Python versions of what you post :-)
> >
> > Actually she gave one in her first post. Here it is again:
>
> I'm sorry about that! I got confused part-way through the original
> post and skimmed from there, and then didn't go back and reread it in
> the context of the follow-up, so I missed the example. My mistake.
>
> >        while {p1 for (s0,p1) in decisions if s0==slot_num}:
> >           p1 = {p1 for (s0,p1) in decisions if s0==slot_num}.pop()
> >           for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2
> != p1}:
> >
> > Note that the set {p1 for (s0,p1) in decisions if s0==slot_num} is
> computed
> > twice, once to decide whether to stop, and then again to compute the
> witness
> > (p1). Obviously this is inefficient, and that's what she's after.
>
> Agreed, that's inefficient, and it also violates DRY - I can easily
> imagine those two lines getting out of sync after a while...
>

Actually it's worse. neither expression needs to compute the full set --
they only need to iterate until the first match.


> > To make
> > this same code more efficient in Python, you'd have to do the following,
> > which is natural for us programmers (since we're so used to working
> around
> > limitations and inefficiencies in the systems we work with) but unnatural
> > for mathematicians, who count to infinity at the drop of a hat:
>
> Hmm, I have an aversion to languages (or constructs) based around
> theoretical principles. Blame a couple of encounters with Haskell a
> few years ago. I lost. :-)


In my case the jury is still out, but I'm not worried about Haskell ever
overtaking Python. :-)

FWIW, comprehensions also come from these theoretical principles, so it's
not all bad...


> But it tends to be the over-compressed
> syntax rather than the ideas that I'm particularly allergic to.
>
> > while True:
> >   temp = {p1 for (s0,p1) in decisions if s0==slot_num}
> >   if not temp:
> >     break
> >   p1 = temp.pop()
> >
> >   for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 != p1}:
> >     <whatever>
>
> Point taken. On the other hand, if (x := val) was an expression form
> of assignment, like C's assignment, you could write
>
> while temp := {p1 for (s0,p1) in decisions if s0==slot_num}:
>  p1 = temp.pop()
>
>  for s2 ...
>
> which is to my mind as succinct, and clearer to a non-mathematician,
> as your proposal below (and Annie's proposal). It also builds off a
> much more commonly requested feature :-)
>
> Actually,
>
> while any(p1 := p for (s,p) in decisions if s == slot_num):
>  for s2 ...
>
> works, and is just as short as your proposal or Annie's.
>

Yeah, and it also avoids computing the elements of the set beyond the first.


> > The 5 tedious lines from "while" through "pop()" would be collapsed into
> a
> > single line if you could write
> >
> >   while some s0, p1 in decisions if s0 == slot_num:
> >
> >   for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 != p1}:
> >     <whatever>
> >
> > TBH I'm not sure what the !slot_num notation is for -- it appears that
> >
> >   while some (!slot_num, p1) in decisions:
> >
> > is equivalent in Annie's proposal to
> >
> >   while some s0, p1 in decisions if s0 == slot_num:
> >
> > but I'm not sure and it doesn't feel necessary to me.
>
> Yes, I think that's right. It looks like the idea comes from the
> concept of unification in logic languages (and the ! notation means
> "don't unify this value, but rather treat it as a fixed value that
> must match"). Thanks for your analysis, by the way, I think understand
> the proposal a lot better now.
>

And thanks for the link with logic languages, that's an area I know even
less about...


> So, going back to what Annie was referring to, there seem to be three
> key concepts:
>
> Quantifications, which are covered in Python by any() and all()
> Capturing a "witness", which can be done using assignment-as-expression
> Tuple matching, which you have shown can be handled using tuple
> unpacking plus the generator expression if clause, but could probably
> gain from a more compact notation.
>

I'm not sure we need a new construct for tuple matching. Witness capturing
seems the most important missing feature here.


> BTW, as I'm sure everyone knows, you can simulate assignment-as-expression
> with
>
> def asgn(o, val):
>  o.ans = val
>  return val
>
> >>> any(asgn(c,x) for x in (1,2,3) if x%2 == 0)
> True
> >>> c.ans
> 2
>

Eew. :-(


> > Also note that SOME and EACH quantifiers were present in ABC (Python's
> > predecessor: http://homepages.cwi.nl/~steven/abc/qr.html#TESTS); I
> dropped
> > them for simplicity, not because I didn't like them. If we wanted to we
> > could have them back (except for the problems of introducing new
> keywords).
>
> One day, I really must read up on ABC.
>

Just follow the above link and scroll to the top. :-)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120114/8e6c25b4/attachment.html>

From p.f.moore at gmail.com  Sun Jan 15 00:27:48 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 14 Jan 2012 23:27:48 +0000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
Message-ID: <CACac1F-hVB_DHcO+4ejgaq-ZSc5ncXLCBT9g60V_WpqGT02ZYA@mail.gmail.com>

On 14 January 2012 23:07, Guido van Rossum <guido at python.org> wrote:
>> BTW, as I'm sure everyone knows, you can simulate assignment-as-expression
>> with
>>
>> def asgn(o, val):
>> ?o.ans = val
>> ?return val
>>
>> >>> any(asgn(c,x) for x in (1,2,3) if x%2 == 0)
>> True
>> >>> c.ans
>> 2
>
>
> Eew. :-(

Agreed. Assignment-as-expression is nicer. But I have to say I've
never really missed it in Python until this discussion started.

Question. Do you object to assignment-as-expression per se, or merely
as the default way of looking at assignment? Or to put it another way,
would something like my x := val (or maybe x <- val or something else)
be acceptable?

Paul.


From cmjohnson.mailinglist at gmail.com  Sun Jan 15 01:03:10 2012
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Sat, 14 Jan 2012 14:03:10 -1000
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
	from the inside out)
In-Reply-To: <CAOFbRmJQnVObBemu-nKGvc-MPHcMS=_8_dKhFQRVTP92HCDAKw@mail.gmail.com>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<1462421.BIX0U07Ti7@lixie>
	<0567BC83-2437-4881-8F8D-F6E47DB76B72@gmail.com>
	<CAOFbRmJQnVObBemu-nKGvc-MPHcMS=_8_dKhFQRVTP92HCDAKw@mail.gmail.com>
Message-ID: <E8A30A54-4FA3-4122-907A-53D432611F6F@gmail.com>


On Jan 14, 2012, at 10:13 AM, Nathan Rice wrote:

> I did mention in the first post that I am the author of a library that
> does this right? :D

Sorry, my mistake for only skimming the thread. I just kept wondering why none of the later emails mentioned that it was possible?

From steve at pearwood.info  Sun Jan 15 03:01:24 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 15 Jan 2012 13:01:24 +1100
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
Message-ID: <4F123374.5030402@pearwood.info>

Guido van Rossum wrote:
> On Sat, Jan 14, 2012 at 1:38 PM, Paul Moore <p.f.moore at gmail.com> wrote:
[...]
>> So, going back to what Annie was referring to, there seem to be three
>> key concepts:
>>
>> Quantifications, which are covered in Python by any() and all()
>> Capturing a "witness", which can be done using assignment-as-expression
>> Tuple matching, which you have shown can be handled using tuple
>> unpacking plus the generator expression if clause, but could probably
>> gain from a more compact notation.
>>
> 
> I'm not sure we need a new construct for tuple matching. Witness capturing
> seems the most important missing feature here.


If I recall correctly, there have been occasional discussions about changing 
any() and all() to return the item found rather than a flag. Given the need 
for backwards compatibility, I don't think we can or should do this, but a 
hypothetical quant module, or a few new built-ins, could possibly help here. 
I'm not sure that quantifications are quite important enough to justify new 
syntax.

For lack of better names:


def all2(iterable, pred=bool):
     """Like all(pred(obj) for obj in iterable), returning True if it is true,
     otherwise obj, the first witness that it false.
     """
     for obj in iterable:
         if not pred(obj):
             return obj
     return True

def any2(iterable, pred=bool):
     """Like any(pred(x) for x in iterable), returning False if it is false,
     otherwise obj, the first witness that it is true.
     """
     for obj in iterable:
         if pred(obj):
             return obj
     # I look forward to the bike-shedding about returning None
     # vs returning False ;)
     return False


One disadvantage of returning a single value to represent both the success or 
failure of the quantification *and* the witness is that it leaves the caller 
vulnerable to this sort of bug:

py> witness = any2([3, 0, 2, 4], lambda n: n%2==0)  # get first even number
py> if witness:
...     print("first even number is,", witness)
... else:
...     print("everything is odd")
...
everything is odd


I don't have a good solution for this.


-- 
Steven


From ncoghlan at gmail.com  Sun Jan 15 03:27:10 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jan 2012 12:27:10 +1000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CACac1F-hVB_DHcO+4ejgaq-ZSc5ncXLCBT9g60V_WpqGT02ZYA@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<CACac1F-hVB_DHcO+4ejgaq-ZSc5ncXLCBT9g60V_WpqGT02ZYA@mail.gmail.com>
Message-ID: <CADiSq7fXeDfTKxTyFoQNb_++EgYNTtdjW-7BtfH8L7pYBxVpkg@mail.gmail.com>

On Sun, Jan 15, 2012 at 9:27 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> Agreed. Assignment-as-expression is nicer. But I have to say I've
> never really missed it in Python until this discussion started.
>
> Question. Do you object to assignment-as-expression per se, or merely
> as the default way of looking at assignment? Or to put it another way,
> would something like my x := val (or maybe x <- val or something else)
> be acceptable?

Just to make sure I have the example right, I believe this is what the
original example looks like without using comprehensions at all:

    for dec_slot, p1 in decisions:
        if dec_slot == slot_num:
            for prop_slot, p2 in proposals:
                if prop_slot == slot_num and p2 != p1:
                    # Do something!

You can merge the for loops and the if statements, and hence avoid the
naming conflict for the slot variables with a couple of generator
expressions (there's apparently no need to waste memory realising the
set for this example). You can also trivially adopt a mathematics
style ordering for any() and all() by using "True" as the body and
moving the predicate to the comprehension's if clause.

That means, the example can already be written as:

    for p1 in (p for slot, p in decisions if slot == slot_num):
        if any(True for slot, p2 in proposals if slot == slot_num and p2 != p1):
            # Do something!

The tricks are:

1. Use generator expressions in order to get at the values as they're
produced, rather than realising the sets in memory
2. Use a for loop instead of a while loop in order to capture those values
3. Use "True" with any()/all() to get a mathematical style
quantification ordering of expressions

Cheers,
Nick.

P.S. My reply appears in this part of the thread because it started
out as a reference to my previous suite expressions concept [1] that
supports embedded assignment statements in a way that allows the value
saved and the value used as a predicate to differ. As I wrote it up
though, I realised that the specific example given could be translated
fairly neatly into *existing* Python constructs, as shown above.

[1] http://ncoghlan_devs-python-notes.readthedocs.org/en/latest/pep_ideas/suite_expr.html


-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Sun Jan 15 03:36:22 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jan 2012 12:36:22 +1000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <4F123374.5030402@pearwood.info>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info>
Message-ID: <CADiSq7c6nMzK94-6j=7uFUJfdUmC1=bwuQ8x6yOSz2ejg4-Azw@mail.gmail.com>

On Sun, Jan 15, 2012 at 12:01 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> One disadvantage of returning a single value to represent both the success
> or failure of the quantification *and* the witness is that it leaves the
> caller vulnerable to this sort of bug:
>
> py> witness = any2([3, 0, 2, 4], lambda n: n%2==0) ?# get first even number
> py> if witness:
> ... ? ? print("first even number is,", witness)
> ... else:
> ... ? ? print("everything is odd")
> ...
> everything is odd
>
>
> I don't have a good solution for this.

The way around it is to always return a tuple: empty if no answer was
found, a singleton-tuple if it was. That way truth-testing will always
give the right found/not-found answer, but in the found case you can
access the original object.

def any2(iterable, pred=bool):
   """Like any(pred(x) for x in iterable), but returns () if nothing
matches the predicate,
   otherwise (obj,), a singleton-tuple holding the first value that matched it.
   """
   for obj in iterable:
       if pred(obj):
           return (obj,)
   return ()

py> found = any2([3, 0, 2, 4], lambda n: n%2==0)  # get first even number
py> if found:
...     print("first even number is,", found[0])
... else:
...     print("everything is odd")
...
first even number is 0

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From python at mrabarnett.plus.com  Sun Jan 15 03:45:28 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 15 Jan 2012 02:45:28 +0000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <4F123374.5030402@pearwood.info>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info>
Message-ID: <4F123DC8.5020603@mrabarnett.plus.com>

On 15/01/2012 02:01, Steven D'Aprano wrote:
> Guido van Rossum wrote:
>>  On Sat, Jan 14, 2012 at 1:38 PM, Paul Moore<p.f.moore at gmail.com>  wrote:
> [...]
>>>  So, going back to what Annie was referring to, there seem to be three
>>>  key concepts:
>>>
>>>  Quantifications, which are covered in Python by any() and all()
>>>  Capturing a "witness", which can be done using assignment-as-expression
>>>  Tuple matching, which you have shown can be handled using tuple
>>>  unpacking plus the generator expression if clause, but could probably
>>>  gain from a more compact notation.
>>>
>>
>>  I'm not sure we need a new construct for tuple matching. Witness capturing
>>  seems the most important missing feature here.
>
>
> If I recall correctly, there have been occasional discussions about changing
> any() and all() to return the item found rather than a flag. Given the need
> for backwards compatibility, I don't think we can or should do this, but a
> hypothetical quant module, or a few new built-ins, could possibly help here.
> I'm not sure that quantifications are quite important enough to justify new
> syntax.
>
> For lack of better names:
>
>
> def all2(iterable, pred=bool):
>       """Like all(pred(obj) for obj in iterable), returning True if it is true,
>       otherwise obj, the first witness that it false.
>       """
>       for obj in iterable:
>           if not pred(obj):
>               return obj
>       return True
>
> def any2(iterable, pred=bool):
>       """Like any(pred(x) for x in iterable), returning False if it is false,
>       otherwise obj, the first witness that it is true.
>       """
>       for obj in iterable:
>           if pred(obj):
>               return obj
>       # I look forward to the bike-shedding about returning None
>       # vs returning False ;)
>       return False
>
>
> One disadvantage of returning a single value to represent both the success or
> failure of the quantification *and* the witness is that it leaves the caller
> vulnerable to this sort of bug:
>
> py>  witness = any2([3, 0, 2, 4], lambda n: n%2==0)  # get first even number
> py>  if witness:
> ...     print("first even number is,", witness)
> ... else:
> ...     print("everything is odd")
> ...
> everything is odd
>
>
> I don't have a good solution for this.
>
A 'default' parameter perhaps?

 >>> witness = any2([3, 0, 2, 4], lambda n: n % 2 == 0, default=None)
 >>> if witness is not None:
...     print("first even number is", witness)
... else:
...     print("everything is odd")
...
first even number is 0


From nathan.alexander.rice at gmail.com  Sun Jan 15 08:04:11 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Sun, 15 Jan 2012 02:04:11 -0500
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
 from the inside out)
In-Reply-To: <1462421.BIX0U07Ti7@lixie>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<1462421.BIX0U07Ti7@lixie>
Message-ID: <CAOFbRmKZLBrY+iQFGyYXNKqJ9-DBU0AKMufvPGumQWB696TGYg@mail.gmail.com>

> The pattern you employ with SymbolicObject, is a nice way to mix code that is
> executed immediately, with code that is executed later. Symbolic algebra
> languages, like Maple and Mathematica, function this way too.

I find it to be useful quite frequently.  I think making generative
SQL expressions with SQL Alchemy got me hooked.

> I propose however a slightly different interface:
>
> There should be a special object "quoted" to create instances of
> "SymbolicObject". The objects should be created in the "__getattribute__"
> method. This way the objects know their name, which is quite useful. A
> symbolic object would be created like this:
>
> ? ?X = quoted.X
>
> You could also write:
>
> ? ?const = Constraints(quoted.X * 2 + 1 >= 5, quoted.X % 2 != 0)

The factory there seems to add complexity.  The Class(args)
constructor style is less magic, and I can imagine wanting to pass
things to the symbolic object, like a name, default value, a
docstring, etc.

> The standard library should contain at least these symbolic algorithms:
>
> evalsym(tree, environment, return_type="any")
> ? ?Evaluates an expression and returns the result. Works similar to:
>
> ? ?eval(compile(tree, "", "eval"), environment)
>
> ? ?This call would return 7:
>
> ? ? ? ?evalsym(quoted.X * 2 + 1, {"X":3})
>
> ? ?However it should do partial evaluation if some symbols are
> ? ?unknown, and return an ast.AST in this case (similar to Maple and
> ? ?Mathematica). This expression should be True:
>
> ? ? ? ?evalsym(quoted.X + quoted.Y, {"Y":3}) == quoted.X + 3
>
> ? ?The optional argument "return_type" specifies the type of the
> ? ?result. This simplifies algorithms using "eval". Argument
> ? ?"return_type" can be: "any", "ast", or "object".

I think a method call leads to more readable code than a function,
because they grow left to right and are structured like English.
Something along the lines of:

expr = X ** (M*(N+1))
expr.realize({N:2, M:3, X:4})
expr.realize((N,2)).realize((M, 3)).realize((X, 4))

Tasty curry.

As a side note, perhaps PyPy is a good place to prototype something
like this? :)

Nathan


From ncoghlan at gmail.com  Sun Jan 15 08:44:30 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jan 2012 17:44:30 +1000
Subject: [Python-ideas] Special method to delegate PEP 3118 buffer access to
	another object?
Message-ID: <CADiSq7csguW20TvegLb6NyrOxfq-XhBX6_Y_nKLMiZ6aG0_jsA@mail.gmail.com>

I'm still working on fixing CPython's operand precedence bug for pure
sequence types implemented in C [1], and, while trying to test
bytearray, came across the problem that there doesn't appear to be a
way to write a class in pure Python that redirects the PEP 3118 buffer
access slots to another object (e.g. a memoryview instance or a bytes
object).

Did I miss something? And, if not, would it make sense to have the
buffer retrieval operations look for a __buffer__ method in the type
dictionary as a last resort if the relevant C level slots aren't
provided?

Cheers,
Nick.

[1] http://bugs.python.org/issue11477

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From techtonik at gmail.com  Sun Jan 15 09:31:50 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 15 Jan 2012 11:31:50 +0300
Subject: [Python-ideas] Pythonic buffering in Py3 print()
In-Reply-To: <jelij4$3e6$1@dough.gmane.org>
References: <jel8mm$bb0$1@dough.gmane.org>
	<20120112014017.GA14688@cskk.homeip.net>
	<jelij4$3e6$1@dough.gmane.org>
Message-ID: <CAPkN8xJATn9mgR3Pb4tGvTgTrtOpYfgrB32bDULSBSUZR2X4WQ@mail.gmail.com>

Thanks everybody for discussion. It took only 6 days between noticing the
problem [1] and finding the cause [2]. I wish the process  didn't require
so intensive communication, because without holidays I'd probably gave up
on this. =) It's good that I know English.

1. https://bitbucket.org/techtonik/python-pager/changeset/3e3d0700ba5f
2. https://bitbucket.org/techtonik/python-pager/changeset/ffe9da2d5316
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120115/7422433a/attachment.html>

From p.f.moore at gmail.com  Sun Jan 15 09:59:43 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 15 Jan 2012 08:59:43 +0000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CADiSq7fXeDfTKxTyFoQNb_++EgYNTtdjW-7BtfH8L7pYBxVpkg@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<CACac1F-hVB_DHcO+4ejgaq-ZSc5ncXLCBT9g60V_WpqGT02ZYA@mail.gmail.com>
	<CADiSq7fXeDfTKxTyFoQNb_++EgYNTtdjW-7BtfH8L7pYBxVpkg@mail.gmail.com>
Message-ID: <CACac1F_qwmm6j3p9VBP2AR5hq9xmTb4py2H5M45a1ELN3FEDfA@mail.gmail.com>

On 15 January 2012 02:27, Nick Coghlan <ncoghlan at gmail.com> wrote:
> That means, the example can already be written as:
>
> ? ?for p1 in (p for slot, p in decisions if slot == slot_num):
> ? ? ? ?if any(True for slot, p2 in proposals if slot == slot_num and p2 != p1):
> ? ? ? ? ? ?# Do something!

Actually, that may not work - the original requirement was to scan
decisions on each iterations, and pick one item that satisfied the
condition each time. If decisions is changing each iteration, you need
to rescan each time, which your for loop doesn't do.

I think that's why the (otherwise odd) "while there are any, pick one"
construct is used. Of course, for an algorithm like that, I'd probably
tend to choose a different data structure, maybe something like a work
queue.

Paul.


From simon.sapin at kozea.fr  Sun Jan 15 11:54:20 2012
From: simon.sapin at kozea.fr (Simon Sapin)
Date: Sun, 15 Jan 2012 11:54:20 +0100
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <4F123374.5030402@pearwood.info>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info>
Message-ID: <4F12B05C.5090802@kozea.fr>

Le 15/01/2012 03:01, Steven D'Aprano a ?crit :
> def any2(iterable, pred=bool):
>       """Like any(pred(x) for x in iterable), returning False if it is false,
>       otherwise obj, the first witness that it is true.
>       """
>       for obj in iterable:
>           if pred(obj):
>               return obj
>       # I look forward to the bike-shedding about returning None
>       # vs returning False;)
>       return False

Hi,

If the point is to get some value that matches a predicate, how about this?

next(value for value in iterable if predicate(value))

If a "no value" marker is desired instead of StopIteration:

next((value for value in iterable if predicate(value)), None)

But yes, assignment-expression is still required to avoid the while 
True/if/break dance.

Regards,
-- 
Simon Sapin


From solipsis at pitrou.net  Sun Jan 15 16:26:37 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 15 Jan 2012 16:26:37 +0100
Subject: [Python-ideas] Special method to delegate PEP 3118 buffer
 access to another object?
References: <CADiSq7csguW20TvegLb6NyrOxfq-XhBX6_Y_nKLMiZ6aG0_jsA@mail.gmail.com>
Message-ID: <20120115162637.3b13c4b4@pitrou.net>

On Sun, 15 Jan 2012 17:44:30 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> I'm still working on fixing CPython's operand precedence bug for pure
> sequence types implemented in C [1], and, while trying to test
> bytearray, came across the problem that there doesn't appear to be a
> way to write a class in pure Python that redirects the PEP 3118 buffer
> access slots to another object (e.g. a memoryview instance or a bytes
> object).
> 
> Did I miss something? And, if not, would it make sense to have the
> buffer retrieval operations look for a __buffer__ method in the type
> dictionary as a last resort if the relevant C level slots aren't
> provided?

Would make sense indeed.

Regards

Antoine.




From liu at cs.stonybrook.edu  Sun Jan 15 18:01:56 2012
From: liu at cs.stonybrook.edu (Annie Liu)
Date: Sun, 15 Jan 2012 17:01:56 +0000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <4F12B05C.5090802@kozea.fr>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info> <4F12B05C.5090802@kozea.fr>
Message-ID: <913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>

Hi!  Thanks for all the ideas and discussions!

Here is a summary of my thinking, followed by some specific comments.

1. I think we should support proper quantifications, because there are
no conceptual or technical difficulties, and it is win-win-win to do.

   No conceptual difficulties:
   . They are first-order logic, used in people's day-to-day reasoning.
   . They are in one of the first two CS courses, the other being programming.
   . They are used commonly in pseudocode.
   They are very far from requiring mathematicians or infinity. :-)
   It would be wonderful if students taking the first two CS courses
   can actually write quantifications in programs at the drop of a hat!

   No technical difficulties:
   . A straightforward implementation suffices, stopping at the first witness.
   . Alternatives/workarounds have many different kinds of difficulties.
   . They are in SETL, the first language that used sets for programming,
     developed over 40 years ago.
   They don't break any Python rules, as far as I know. :-)
   In fact, "some x in s" is better than "s.pop()" in terms of side effect,
   and it gives the programmer a test and a choice of not changing s.
   (not to mention that computing a comprehension to get s followed by
   a pop is less efficient than doing "some x in s" directly)

   It is win-win-win to do, because they help
   . make programming easier
   . make programs clearer
   . make programs more efficient
   This is a dream situation, considering the usual conflicts among these!

2. OK, I think we can ignore tuple patterns for now (they were why I
used "!" for bound variables, for pattern matching).  They are a much
smaller conceptual and technical help.


 > Don't forget that in Python, assignments are
 > statements and don't get embedded in expressions.

I like this principle dearly, but sometimes breaking it is better. :-)
In fact, the alternative I had to use, s.pop(), is a good example of a
statement that does get used in expressions.

In general, any time we have a statement that also returns a value, it
is the same as an expression that has side effect.  There are plenty
of those in Python. :-)

 > while True:
 >   temp = {p1 for (s0,p1) in decisions if s0==slot_num}
 >   if not temp:
 >     break
 >   p1 = temp.pop()
 >   for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 != p1}:
 >     <whatever>
 > 
 > The 5 tedious lines from "while" through "pop()" would be collapsed
 > into a single line if you could write
 > 
 >   while some s0, p1 in decisions if s0 == slot_num:
 >   for p2 in {p2 for (s0,p2) in proposals if s0==slot_num if p2 != p1}:
 >     <whatever>

Right!(with a little indentation in front of the "for")  Thank you!

In fact, as I mentioned, it happens that using "for" in place of "if"
is ok here (in Robbert's algorithm), but in general, to capture the
original "if" correctly, one would need another temp variable and some
more tedious lines.

 >      """Like all(pred(obj) for obj in iterable), returning True if it is true,
 >      otherwise obj, the first witness that it false.

Thanks for the idea of witness for "all"!  I have actually not needed
to use it, but I can see that the symmetry helps general uses, and
we'd get it for free just like for "any".

 > >    for p1 in (p for slot, p in decisions if slot == slot_num):
 > >        if any(True for slot, p2 in proposals if slot == slot_num and p2 != p1):
 > >            # Do something!
 > 
 > Actually, that may not work - the original requirement was to scan
 > decisions on each iterations, and pick one item that satisfied the
 > condition each time. If decisions is changing each iteration, you need
 > to rescan each time, which your for loop doesn't do.

Right!  Thank you!

 > I think that's why the (otherwise odd) "while there are any, pick one"
 > construct is used. Of course, for an algorithm like that, I'd
 > probably tend to choose a different data structure, maybe something
 > like a work queue.

In fact, "while some x in s" is exactly a best way to write a work
list algorithm, where either a queue or stack could be used (in graph
algorithms, a queue corresponds to breadth-first-search, and a stack
corresponds to depth-first-search), but when the order of picking
elements doesn't matter, "while some x in s" is easier and clearer.

 > Also note that SOME and EACH quantifiers were present in ABC (Python's
 > predecessor: http://homepages.cwi.nl/~steven/abc/qr.html#TESTS); I
 > dropped them for simplicity, not because I didn't like them. If we
 > wanted to we could have them back (except for the problems of
 > introducing new keywords).

I hope new keywords are not the stopping factor.  I remember some
of my programs used "as" a lot (for many nouns starting with a, like
arguments), and I avoided running them with py3 for several years,
until I had to one day, and they were so easy to fix that I regretted
not doing it sooner.

I wonder what the best keywords (and separator) are to use:

   SETL:     forall, exists       e.g., exists x in s | pred
   html:     &forall, &exist 
   latex:    \forall, \exists
   ABC:      each, some           e.g., some x in s has pred
   Ada 2012: for all, for some    e.g., for some x in s => pred

I had used "forall" and "exists", but quite liked "each" and "some"
after I learned them from Lambert Meertens two years ago: besides being
shorter, "each" gives the right grammar (singular) when read out.

But I also feel conflicted, since "all" and "exist" are used for so
long in history, to justify the reversed "A" and "E" in symbols.

BTW, one could use "all" and "any", even shorter and has a reversed
"A":-), but "any" is ambiguous when read out.  For example, we say
"some apple in the bag is red", not "any apple in the bag is red".

For separator, I had used "if" (not "has") because it is consistent
with uses of conditions in general in Python, and with uses of "|" in
SETL.  SETL has:

   comprehension        {ret_exp: x in s | pred}
   quantification       exists x in s | pred
                        forall x in s | pred
   for loop             for x in s | pred loop ... end loop

   and you get "while exists x in s | pred" for free.

   "| pred" can be omitted when "pred" is true.

Interestingly, I noticed that Guido used "if" in his code (after the
"5 tedious lines").

I feel that "has" reads well for single quantification, but not as
well for nested (though "if" sounds a bit weird also, and I'd really
like to use "|" if it doesn't break Python rules :-)).  Compare:

   each x in s has (some y in t has pred)

   each x in s if (some y in t if pred)

   each x in s | (some y in t | pred)

BTW, functions "all" and "any" are much harder to read here:

   all(any(pred for y in t) for x in s)

and it is even harder when you have some real contents for pred, s,
and t, not just the template here.

This reminds me that Guy Steele had a popular lecture making fun of
languages that have features that require people to swing their
eyeballs back and forth on a line to read it. :-)

In summary, again, I think we should support proper quantifications.
BTW, some of the workarounds proposed here are looking like what a
Haskell genius was trying to show me yesterday how to get around. :-)

Thanks,
Annie


From p.f.moore at gmail.com  Sun Jan 15 20:46:38 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 15 Jan 2012 19:46:38 +0000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info> <4F12B05C.5090802@kozea.fr>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
Message-ID: <CACac1F_c03Yz8bbbeqWBDLCGHL3wxFwducTE26u8t5XCia8QuA@mail.gmail.com>

On 15 January 2012 17:01, Annie Liu <liu at cs.stonybrook.edu> wrote:
> ? It is win-win-win to do, because they help
> ? . make programming easier
> ? . make programs clearer
> ? . make programs more efficient
> ? This is a dream situation, considering the usual conflicts among these!

Hmm, while I'm coming to like the concept, I still think that
"clearer" might be a bit strong. For simple uses where all() and any()
work as replacements, those builtins are already pretty clear. For
cases where they won't do, then quantifiers as you propose are
certainly no worse than some of the other proposed alternatives, but
I'm not sure how much better they are, either. But never mind, clarity
is a matter of perspective at best :-)

> ?> I think that's why the (otherwise odd) "while there are any, pick one"
> ?> construct is used. Of course, for an algorithm like that, I'd
> ?> probably tend to choose a different data structure, maybe something
> ?> like a work queue.
>
> In fact, "while some x in s" is exactly a best way to write a work
> list algorithm, where either a queue or stack could be used (in graph
> algorithms, a queue corresponds to breadth-first-search, and a stack
> corresponds to depth-first-search), but when the order of picking
> elements doesn't matter, "while some x in s" is easier and clearer.

Yes, I see your point here - in this case "while some x in s" reads
nicely, and does what I'd expect.

> I feel that "has" reads well for single quantification, but not as
> well for nested (though "if" sounds a bit weird also, and I'd really
> like to use "|" if it doesn't break Python rules :-)). ?Compare:
>
> ? each x in s has (some y in t has pred)
>
> ? each x in s if (some y in t if pred)
>
> ? each x in s | (some y in t | pred)
>
> BTW, functions "all" and "any" are much harder to read here:
>
> ? all(any(pred for y in t) for x in s)
>
> and it is even harder when you have some real contents for pred, s,
> and t, not just the template here.

OK, I'm willing to concede that the all/any formulation isn't as
"clean looking" as the alternatives you propose. But I did understand
it instantly, whereas I found that I couldn't work out the meaning of
your form straight off, but had to "cheat" by looking at the all/any
formulation, and work back from that. How much of that is simply
because I'm more familiar with the conventional form, I'm not sure
(it's certainly a reasonably substantial part of it - and if each/some
become part of the language, they will become familiar too).

But what I want to do is break things down into individual constructs
- that's how my mind works (so sue me! :-)) With the all/any form, it
goes:

   all(...) - function call, with a generator expression as argument
   ... for x in s - generator expression, argument to all
   any(...) - function call again
   pred for y in t - generator expression

So we have 2 fundamental constructs, both used in many contexts
throughout Python, and hence both familiar, combined in simple ways.

For your each/some formulation we have:

   each x in s has ... - a new type of construct, and each expression
   some y in t has pred - another new type of construct, a some expression

each and some expressions feel like they have structure in common with
generator expressions, but they look slightly different. Actually,
isn't it true that "each x in s has pred" is basically "all(x for x in
s if pred)" except that no witness is captured? So syntactically the
differences are:

   - you avoid the repetition of "x for x" (which is a known wart of
generator expressions)
   - you're using has instead of if (it reads better, but otherwise is
a bit of a spurious difference - you did propose if as an alternative)
   - you avoid the parentheses

There are some subtler differences (you use "pred for x..." in the
all/any version, whereas I'm equating each/some forms with "x for x"
variations, for example). But basically this is my real concern - your
proposed constructs are special case syntax that is close to, but
subtly different from, existing forms. That "nearly but not quite the
same" aspect is where I have concerns about readability (and
teachability).

The problem is that each/some can't build on generators, because if
they do you need to explicitly specify the variable to use to capture
the witness. And then if you use a generator expression, you end up
with something like "some VAR GENERATOR", which becomes a monstrosity
like "some x (x for x in ...)". So they have to build on generator
*expressions* which are a subset of general generators. And we get to
special case synax straight off. The advantage of any/all is that they
work on any generator - whether it's a generator expression or some
other form of generator).

> In summary, again, I think we should support proper quantifications.

I'm getting more comfortable with the idea, but I think it needs a bit
more discussion to come up with something that fits in cleanly with
the rest of Python, while still adding the value you want.

Paul.


From jeanpierreda at gmail.com  Mon Jan 16 00:29:39 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 15 Jan 2012 18:29:39 -0500
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info> <4F12B05C.5090802@kozea.fr>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
Message-ID: <CABicbJJ1p=AYmWkGs6NAXCcbqSx1rwAZVNTtueiBz=upxCTNNw@mail.gmail.com>

On Sun, Jan 15, 2012 at 12:01 PM, Annie Liu <liu at cs.stonybrook.edu> wrote:
> For separator, I had used "if" (not "has") because it is consistent
> with uses of conditions in general in Python, and with uses of "|" in
> SETL. ?SETL has:
>
> ? comprehension ? ? ? ?{ret_exp: x in s | pred}
> ? quantification ? ? ? exists x in s | pred
> ? ? ? ? ? ? ? ? ? ? ? ?forall x in s | pred
> ? for loop ? ? ? ? ? ? for x in s | pred loop ... end loop
>
> ? and you get "while exists x in s | pred" for free.

I hadn't been _really_ paying attention, [no active reading] but the
"if" had confused me until I read this -- I didn't know what it meant.
I'm so much more used to "|" and "s.t. (such that)". If reads like
implication, which is something else.

Of course, the same complaint applies to list comprehensions, so I
expect I'll get used to it, but... with list comprehensions it seemed
obvious because it replaced an idiom that literally used the if
statement. Is the same true here?

-- Devin


From ncoghlan at gmail.com  Mon Jan 16 15:15:05 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Jan 2012 00:15:05 +1000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CACac1F_c03Yz8bbbeqWBDLCGHL3wxFwducTE26u8t5XCia8QuA@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info> <4F12B05C.5090802@kozea.fr>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
	<CACac1F_c03Yz8bbbeqWBDLCGHL3wxFwducTE26u8t5XCia8QuA@mail.gmail.com>
Message-ID: <CADiSq7cL0F+n=t4cAKU2JO6XfqsESdCeHyqrj+5oohRyRnNQvA@mail.gmail.com>

On Mon, Jan 16, 2012 at 5:46 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>> In summary, again, I think we should support proper quantifications.
>
> I'm getting more comfortable with the idea, but I think it needs a bit
> more discussion to come up with something that fits in cleanly with
> the rest of Python, while still adding the value you want.

Indeed.

With the clarification that my original "long form" expansion was
incorrect (since it only looped over the outer set once), I'll put
forward a slightly shorter variant of Guido's expansion (as someone
else already noted, next() can be a useful alternative to pop() if you
aren't keeping the original set around)

    _sentinel = object()
    while True:
        p1 = next((p for (s0,p1) in decisions if s0==slot_num), _sentinel)
        if p1 is _sentinel:
            break

        for p2 in {p2 for (s0,p2) in proposals if s0==slot_num and p2 != p1}:
            ...

So, let's consider this proposal:

"some VAR in ITERABLE if COND" would be an expression that:

1. *deliberately* leaks the variable into the surrounding scope
(embedded assignment ahoy)
2. Defines the expression value roughly as follows:

    _itr = (VAR for VAR in ITERABLE if COND) # i.e. it's an implicit
generator expression
    try:
        VAR = next(_itr)
    except StopIteration:
        _expr_result = False
    else:
        _expr_result = True

And, further suppose, that we finally cave on allowing a filter clause
in for statements

Then Annie's original example would look like this:

    while some s0, p1 in decisions if s0==slot_num:
        for s0, p2 in proposals if s0==slot_num and p2 != p1:
            ...

And, since we *know* anything even vaguely resembling an embedded
assignment syntax would almost immediately be adopted for any-and-all
of the various embedded assignment requests we've seen over the years,
here's how it would look for the regex use case:

    if some m in [pattern.search(data)] if m is not None:
        ...

Hmm, still doesn't feel right to me - trying to shove a square peg
(mathematical quantification) into a round hole (Python's syntax). In
particular, the any()/all() way of doing things avoids the double-if
problem when used in an if statement, whereas the version that
captures the variable (and hence moves the predicate to the end) reads
very strangely in that case.

Removing the if clause doesn't help, since you can easily add it back
by using a generator expression as your iterable.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From jeanpierreda at gmail.com  Mon Jan 16 15:47:39 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Mon, 16 Jan 2012 09:47:39 -0500
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CADiSq7cL0F+n=t4cAKU2JO6XfqsESdCeHyqrj+5oohRyRnNQvA@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info> <4F12B05C.5090802@kozea.fr>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
	<CACac1F_c03Yz8bbbeqWBDLCGHL3wxFwducTE26u8t5XCia8QuA@mail.gmail.com>
	<CADiSq7cL0F+n=t4cAKU2JO6XfqsESdCeHyqrj+5oohRyRnNQvA@mail.gmail.com>
Message-ID: <CABicbJJVhxEFE9t9EbSvoUEGH3ig3CUwbO1iYYOhqu4Rd8rr5Q@mail.gmail.com>

On Mon, Jan 16, 2012 at 9:15 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> And, since we *know* anything even vaguely resembling an embedded
> assignment syntax would almost immediately be adopted for any-and-all
> of the various embedded assignment requests we've seen over the years,
> here's how it would look for the regex use case:
>
> ? ?if some m in [pattern.search(data)] if m is not None:
> ? ? ? ?...
>
> Hmm, still doesn't feel right to me - trying to shove a square peg
> (mathematical quantification) into a round hole (Python's syntax). In
> particular, the any()/all() way of doing things avoids the double-if
> problem when used in an if statement, whereas the version that
> captures the variable (and hence moves the predicate to the end) reads
> very strangely in that case.

You can always add a full-on assignment-as-expression, if you're
concerned about that.

    if (m := pat.search(data)) is not None:

Also, the keyword doesn't have to be "if". I've always read the bar as
"such that" or "where".

-- Devin


From p.f.moore at gmail.com  Mon Jan 16 20:29:11 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 16 Jan 2012 19:29:11 +0000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CABicbJJVhxEFE9t9EbSvoUEGH3ig3CUwbO1iYYOhqu4Rd8rr5Q@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info> <4F12B05C.5090802@kozea.fr>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
	<CACac1F_c03Yz8bbbeqWBDLCGHL3wxFwducTE26u8t5XCia8QuA@mail.gmail.com>
	<CADiSq7cL0F+n=t4cAKU2JO6XfqsESdCeHyqrj+5oohRyRnNQvA@mail.gmail.com>
	<CABicbJJVhxEFE9t9EbSvoUEGH3ig3CUwbO1iYYOhqu4Rd8rr5Q@mail.gmail.com>
Message-ID: <CACac1F_SyOybjbt7So+jTBZ5RbcCNGjYBtomYLkTTwO0oT6y3w@mail.gmail.com>

On 16 January 2012 14:47, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
> You can always add a full-on assignment-as-expression, if you're
> concerned about that.
>
> ? ?if (m := pat.search(data)) is not None:
>
> Also, the keyword doesn't have to be "if". I've always read the bar as
> "such that" or "where".

I suggested that a while back in this thread (in the form of any(var
:= val for val in itr if pred)). It didn't get much traction.

Paul.


From jimjjewett at gmail.com  Mon Jan 16 21:06:29 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 16 Jan 2012 15:06:29 -0500
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <4F123374.5030402@pearwood.info>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info>
Message-ID: <CA+OGgf6haZucPnBMGX-yWKyB9Y_byL=sch14XKRETmkTT3OnZg@mail.gmail.com>

On Sat, Jan 14, 2012 at 9:01 PM, Steven D'Aprano <steve at pearwood.info> wrote:

>> On Sat, Jan 14, 2012 at 1:38 PM, Paul Moore <p.f.moore at gmail.com> wrote:

>>> Capturing a (quantification) "witness", which can be done using
>>> assignment-as-expression

> If I recall correctly, there have been occasional discussions about
> changing any() and all() to return the item found rather than a flag.

Even in this thread, there have been multiple posts that would shown
bugs when the witness itself had a boolean evaluation of False.

What if any and all started to return a Witness object that evaluated
to True/False, but made the value available?  I could see an object
like this being useful in other code as well.

    class Witness(object):
        def __bool__(self):  return self.truth
        def __init__(self, truth, value):
            self.truth=truth
            self.value=value

Of course, it might be nicer to inherit from type(True), and it still
doesn't quite solve the laziness problem with for, or the
assign-in-the-right-place problem with while.

-jJ


From tom at rothamel.us  Tue Jan 17 02:57:07 2012
From: tom at rothamel.us (Tom Rothamel)
Date: Mon, 16 Jan 2012 20:57:07 -0500
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
Message-ID: <CANdAuzc6q-uo1imtv752U+gYXReam0eSj97AR02pHu5eh_0wtg@mail.gmail.com>

(Sorry about the formatting - I'm a former student of Annie's, who just
joined the list.)

Jim Jewett wrote:


> Even in this thread, there have been multiple posts that would shown
> bugs when the witness itself had a boolean evaluation of False.
> What if any and all started to return a Witness object that evaluated
> to True/False, but made the value available?  I could see an object
> like this being useful in other code as well.
>     class Witness(object):
>         def __bool__(self):  return self.truth
>         def __init__(self, truth, value):
>             self.truth=truth
>             self.value=value
> Of course, it might be nicer to inherit from type(True), and it still
> doesn't quite solve the laziness problem with for, or the
> assign-in-the-right-place problem with while.


I'm wondering if would make sense to add an "as" clause to the while and if
statements, similar to the as clause in the with statement. It would
essentially retrieve the value field from the witness object described
above.

This would let one write the main loop in a topological sort (a classic
workset algorithm) as:

    while any(v for v in vertices if v.incoming_count == 0) as v1:
        result.append(v)
        for v2 in v1.outgoing:
            v2.incoming_count -= 1

Another use of this could be for database queries that return 0 or 1
results:

    if db.query(page_name=page_name) as page:
        render_template(page)
    else:
        error404()

This is a common pattern that often involves either exception handling or a
sentinel value like None.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120116/e3994cc6/attachment.html>

From python at 2sn.net  Tue Jan 17 05:00:52 2012
From: python at 2sn.net (Alexander Heger)
Date: Mon, 16 Jan 2012 22:00:52 -0600
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CANdAuzc6q-uo1imtv752U+gYXReam0eSj97AR02pHu5eh_0wtg@mail.gmail.com>
References: <CANdAuzc6q-uo1imtv752U+gYXReam0eSj97AR02pHu5eh_0wtg@mail.gmail.com>
Message-ID: <4F14F274.2070908@2sn.net>

Dear Tom,

> I'm wondering if would make sense to add an "as" clause to the while and if
> statements, similar to the as clause in the with statement. It would
> essentially retrieve the value field from the witness object described
> above.
>
> This would let one write the main loop in a topological sort (a classic
> workset algorithm) as:
>
>      while any(v for v in vertices if v.incoming_count == 0) as v1:
>          result.append(v)
>          for v2 in v1.outgoing:
>              v2.incoming_count -= 1

I had the same thought going through the thread, but I think the problem 
here would be that any is a function that returns a Boolean value.  You 
could add a key parameter to return something else, say a tuple of a 
truth value and a sample ( any( ..., sample = True) ) but that could 
break because

1) "any" could be another function in the context,

2) while would need to deal with special return values in this case

A suggestion to replace the "while any" case is

sample <generator expression> as x:
    # do something with x

  ("sample" may not be the best keyword choice) From the discussion so 
far I do not see how to easily avoid having a new keyword

specifically, your example would become

sample v for v in vertices if v.incoming_count == 0 as v1:
     # do things with v1

In case you only want one sample, you could add the break statement

sample db.query(page_name=page_name) as page:
     render_template(page)
     break
else:
     error404()


-Alexander


From ethan at stoneleaf.us  Tue Jan 17 06:39:18 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 16 Jan 2012 21:39:18 -0800
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CACac1F_SyOybjbt7So+jTBZ5RbcCNGjYBtomYLkTTwO0oT6y3w@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>	<4F123374.5030402@pearwood.info>
	<4F12B05C.5090802@kozea.fr>	<913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>	<CACac1F_c03Yz8bbbeqWBDLCGHL3wxFwducTE26u8t5XCia8QuA@mail.gmail.com>	<CADiSq7cL0F+n=t4cAKU2JO6XfqsESdCeHyqrj+5oohRyRnNQvA@mail.gmail.com>	<CABicbJJVhxEFE9t9EbSvoUEGH3ig3CUwbO1iYYOhqu4Rd8rr5Q@mail.gmail.com>
	<CACac1F_SyOybjbt7So+jTBZ5RbcCNGjYBtomYLkTTwO0oT6y3w@mail.gmail.com>
Message-ID: <4F150986.303@stoneleaf.us>

Paul Moore wrote:
> On 16 January 2012 14:47, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
>> You can always add a full-on assignment-as-expression, if you're
>> concerned about that.
>>
>>    if (m := pat.search(data)) is not None:
>>
>> Also, the keyword doesn't have to be "if". I've always read the bar as
>> "such that" or "where".
> 
> I suggested that a while back in this thread (in the form of any(var
> := val for val in itr if pred)). It didn't get much traction

In the matter of := being used for assignment-as-expression... +1!

~Ethan~


From ncoghlan at gmail.com  Tue Jan 17 12:16:39 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Jan 2012 21:16:39 +1000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <4F150986.303@stoneleaf.us>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info> <4F12B05C.5090802@kozea.fr>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
	<CACac1F_c03Yz8bbbeqWBDLCGHL3wxFwducTE26u8t5XCia8QuA@mail.gmail.com>
	<CADiSq7cL0F+n=t4cAKU2JO6XfqsESdCeHyqrj+5oohRyRnNQvA@mail.gmail.com>
	<CABicbJJVhxEFE9t9EbSvoUEGH3ig3CUwbO1iYYOhqu4Rd8rr5Q@mail.gmail.com>
	<CACac1F_SyOybjbt7So+jTBZ5RbcCNGjYBtomYLkTTwO0oT6y3w@mail.gmail.com>
	<4F150986.303@stoneleaf.us>
Message-ID: <CADiSq7ecr5smfd_JUsMDK-f12UarA0Bx3N79-Dijt8krNqZZeQ@mail.gmail.com>

On Tue, Jan 17, 2012 at 3:39 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> Paul Moore wrote:
>>
>> On 16 January 2012 14:47, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
>>>
>>> You can always add a full-on assignment-as-expression, if you're
>>> concerned about that.
>>>
>>> ? if (m := pat.search(data)) is not None:
>>>
>>> Also, the keyword doesn't have to be "if". I've always read the bar as
>>> "such that" or "where".
>>
>>
>> I suggested that a while back in this thread (in the form of any(var
>> := val for val in itr if pred)). It didn't get much traction
>
>
> In the matter of := being used for assignment-as-expression... +1!

Assignments as expressions aren't all that easy to use in the "some"
case that started the thread, and the reason is the same as the reason
iteration variables in comprehensions don't leak into the surrounding
scope any more: generator expressions and comprehensions create their
own scope, separate from that of the containing function.

If Python ever did get assignment expressions, the most likely
candidate would actually be based on the existing use of "as" for
naming (or renaming) things:

    if (pat.search(data) as m) is not None:
        ...

    while (next(p1 for (s0, p1) in decisions if s0==slot_num, None) as
p1) is not None:
        ...

However, assignments as expressions are still a long way from being
shown to ever improve clarity enough to be worth the additional
language complexity.

Getting back specifically to the quantification question, I wonder if
being able to explicitly export an iteration variable from a
comprehension might work:

    while any(s0 == slot_num for (s0, some p1) in decisions):
        ...

Abusing it for embedded assignments is still ugly, but at least
vaguely comprehensible once you're familiar with the meaning of
'some':

    if any(m is not None for some m in [pattern.search(data)]):
        ...


(As far as how that could actually work goes, with the bindings
limited to simple names, as they are for iteration variables, you
could probably just make the relevant mapping available inside the
comprehension for class and module namespaces and make use of the
nonlocal machinery for function namespaces)

I still can't honestly say I like the idea, but I at least dislike it
less than anything else I've seen in the thread so far :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From p.f.moore at gmail.com  Tue Jan 17 14:34:21 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 17 Jan 2012 13:34:21 +0000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CADiSq7ecr5smfd_JUsMDK-f12UarA0Bx3N79-Dijt8krNqZZeQ@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info> <4F12B05C.5090802@kozea.fr>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
	<CACac1F_c03Yz8bbbeqWBDLCGHL3wxFwducTE26u8t5XCia8QuA@mail.gmail.com>
	<CADiSq7cL0F+n=t4cAKU2JO6XfqsESdCeHyqrj+5oohRyRnNQvA@mail.gmail.com>
	<CABicbJJVhxEFE9t9EbSvoUEGH3ig3CUwbO1iYYOhqu4Rd8rr5Q@mail.gmail.com>
	<CACac1F_SyOybjbt7So+jTBZ5RbcCNGjYBtomYLkTTwO0oT6y3w@mail.gmail.com>
	<4F150986.303@stoneleaf.us>
	<CADiSq7ecr5smfd_JUsMDK-f12UarA0Bx3N79-Dijt8krNqZZeQ@mail.gmail.com>
Message-ID: <CACac1F-7A1a4T-V821zQ5JtmwJkYgVUy-vPjvV1Yr=QKSE3DmA@mail.gmail.com>

On 17 January 2012 11:16, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Assignments as expressions aren't all that easy to use in the "some"
> case that started the thread, and the reason is the same as the reason
> iteration variables in comprehensions don't leak into the surrounding
> scope any more: generator expressions and comprehensions create their
> own scope, separate from that of the containing function.

Ah. I tested the idea by setting an attribute of an existing object,
which *does* work (but is ugly). I'd forgotten that scoping comes into
the picture when you switch to assignment.

I'll refrain from proposing "nonlocal x := var"... :-)

Paul.


From ncoghlan at gmail.com  Tue Jan 17 14:39:40 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Jan 2012 23:39:40 +1000
Subject: [Python-ideas] Fwd: quantifications, and tuple patterns
In-Reply-To: <CACac1F-7A1a4T-V821zQ5JtmwJkYgVUy-vPjvV1Yr=QKSE3DmA@mail.gmail.com>
References: <913240E0EEE49B48B1A9D5CFD10658FC3B79F1E6@mail2.cs.stonybrook.edu>
	<CACac1F9rhw23kww0p2BLYA=kqs0heqijEH-1oYydaQGaER-EJg@mail.gmail.com>
	<CACac1F9w0hA=5KnNfa72YjJtkHs8ADa2sytetcTHxV-y6BhH3Q@mail.gmail.com>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F21F@mail2.cs.stonybrook.edu>
	<CACac1F8LQbqxknzYbrTa7LNjr0AxDr7P+6u3e7A5er0dkM5ROA@mail.gmail.com>
	<CAP7+vJJ9iqyEZHH+yqBLKoV1Yn5kmQQt69k1PUq2U552N4vqPA@mail.gmail.com>
	<CACac1F9m9QU=UvOM-fS6aZg4dO-4mL0FyW5R16pT9u18FET5Bg@mail.gmail.com>
	<CAP7+vJLWHPbGKq5PyKzQWKOSUdmCv+g8rOZSTOQOx+4XP3nkmQ@mail.gmail.com>
	<4F123374.5030402@pearwood.info> <4F12B05C.5090802@kozea.fr>
	<913240E0EEE49B48B1A9D5CFD10658FC3B79F43E@mail2.cs.stonybrook.edu>
	<CACac1F_c03Yz8bbbeqWBDLCGHL3wxFwducTE26u8t5XCia8QuA@mail.gmail.com>
	<CADiSq7cL0F+n=t4cAKU2JO6XfqsESdCeHyqrj+5oohRyRnNQvA@mail.gmail.com>
	<CABicbJJVhxEFE9t9EbSvoUEGH3ig3CUwbO1iYYOhqu4Rd8rr5Q@mail.gmail.com>
	<CACac1F_SyOybjbt7So+jTBZ5RbcCNGjYBtomYLkTTwO0oT6y3w@mail.gmail.com>
	<4F150986.303@stoneleaf.us>
	<CADiSq7ecr5smfd_JUsMDK-f12UarA0Bx3N79-Dijt8krNqZZeQ@mail.gmail.com>
	<CACac1F-7A1a4T-V821zQ5JtmwJkYgVUy-vPjvV1Yr=QKSE3DmA@mail.gmail.com>
Message-ID: <CADiSq7cdhH2M=8A=ngrv5hq+jQBx3LtZEO=Bgtase3QCG_MROA@mail.gmail.com>

On Tue, Jan 17, 2012 at 11:34 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> I'll refrain from proposing "nonlocal x := var"... :-)

That idea (or rather the "(var as nonlocal x)" equivalent) was
actually the starting point for my train of thought that led to the
suggestion of just using "some" as a marker for iteration variable
export :)

Cheers,
Nick.


-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From nathan.alexander.rice at gmail.com  Wed Jan 18 14:30:15 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Wed, 18 Jan 2012 08:30:15 -0500
Subject: [Python-ideas] Symbolic expressions (or: partials and closures
 from the inside out)
In-Reply-To: <1462421.BIX0U07Ti7@lixie>
References: <CAOFbRmKxZ1RLs6b_taNAJ-4zVa+LG=5CWK5dwxTdL-TXfr9X2A@mail.gmail.com>
	<1462421.BIX0U07Ti7@lixie>
Message-ID: <CAOFbRmJGReXHWaKg+82LJ5sUo0X7eta_3bvWSf9z=36WgP55Dg@mail.gmail.com>

> There should be a special object "quoted" to create instances of
> "SymbolicObject". The objects should be created in the "__getattribute__"
> method. This way the objects know their name, which is quite useful. A
> symbolic object would be created like this:
>
> ? ?X = quoted.X

I did some additional thinking about how the interface could be
designed, and I think there might be a good middle ground with regard
to your quoted object.  I still feel like creating variables should
be independent, and use the standard class constructor method.  I feel
like a Context object would be a good place to keep variable values,
and the expression evaluation method.

thus, you might have:

context = Context()
expr = SymbolicObject(name="A") + SymbolicObject(name="B") *
SymbolicObject(name="C", default=5)
context.A = 1
context.B = 2
print context.eval(expr)
11
context.C = 3
print context.eval(expr)
7

> The (tree of) symbolic objects should have a method that translate it to an
> abstract syntax tree (ast.AST from the standard library). Subsequent
> algorithms should operate on this AST. This way symbolic algorithms can also
> operate on all possible Python code (SymbolicObject can only create
> expressions).
>
> ? ?my_ast = (quoted.X * 2 + 1).to_ast()
>
> For convenience I think that all symbolic algorithms should also accept
> SymbolicObject, and convert them to ast.AST internally.

I don't think it would be too bad to create an AST representation
inside the symbolic expression while building it.

> ? ?However it should do partial evaluation if some symbols are
> ? ?unknown, and return an ast.AST in this case (similar to Maple and
> ? ?Mathematica). This expression should be True:

Having the evaluate method return an AST with incomplete variable
information would be confusing to me.  I would expect to get a new
symbolic expression with the specified parameters replaced.

>
> substitute(tree, substitutions)
> ? ?Replaces a symbol with a value (a sub-tree of ast.Ast).
> ? ?This expression should be True:
>
> ? ? ? ?substitute(quoted.X + quoted.Y, {"Y": quoted.X * 2}) \
> ? ? ? ?== (quoted.X + quoted.X * 2).to_ast()
>
> ? ?Maybe this algorithm should also be able to match general fragments of
> ? ?the tree and replace them.

If you build up the AST representation while creating the symbolic
expression, replacing a symbolic expression object should have the
effect of updating the tree.

I think having clearly delineated AST and SymbolicExpr representations
might be easier (possibly via converter functions).  The actual
different may be negligible, but it would serve as an explicit way of
declaring what your expectations are with regard to return values,
etc.

As a side note, it appears that between PyPy object spaces and
continulets, it should be possible to create this without any weird
interpreter hacking.  Hopefully I'll have time to put something
together this weekend.

Nathan


From mwm at mired.org  Sat Jan 21 01:36:48 2012
From: mwm at mired.org (Mike Meyer)
Date: Fri, 20 Jan 2012 16:36:48 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
Message-ID: <20120120163648.69288796@mikmeyer-vm-fedora>

Just running another half-baked "easy" idea up to see what others
think of it.

One of the cases where EAFP is clumsy is if the "forgiveness" portion
is actually retrying what failed after fixing things. I just ran into
this in real code, and the resulting discussion suggested this change.

Eiffel's exception-handling mechanism is built assuming that's the
*standard* way to do things. Seems to work ok, but if I had to choose
one of the two, I'll take try/except.

So here's the proposal:

A single new keyword/clause, "retry". It has the same syntax as an
"except" clause, can be used anywhere "except" can be used, and can be
intermingled with them in the same try statement. There's probably a
better syntax, but this is easy to describe.

The behavior change from except is that instead of exiting the "try"
statement when the "retry" clause ends, it restarts the try
clause. In python, this code:

    try:
        # try block
    except:
        # except block
    retry:
        # retry block
    else:
        # else block
    finally:
        # finally block

Would provide the same behavior as this code:

    while True:
        try:
            # try block
        except:
            # except block
        retry:
            # retry block
        else:
            # else block
        finally:
            # finally block
            break

Where except & retry can be repeated, include exceptions to check
against, and intermixed with each other. The break would propagate up
to the else block and any except blocks if there was no finally. If
there's no else, the break from it winds up at the end of the try.

The use case, as mentioned, is avoiding doing EAFP without repeating
the code in the exception handler. I.e., the choices without retry
are things like (LBYL):

    if not isinstance(x, my_class):
       x = fixup(x)
    mangle(x)
    other_stuff(x)

or EAFP:

    try:
        mangle(x)
    except:
        x = fixup(x)
	mangle(x)
    other_stuff(x)

or:

    while True:
        try:
	    mangle(x)
	    break
	except:
	    x = fixup(x)
    other_stuff(x)

with:

    try:
        mangle(x)
    retry:
        e = fixup(x)
    other_stuff(x)


My gut reaction is cool, but not clear it's worth a new keyword. So
maybe there's a tweak to the except clause that would have the same
effect, but I don't see anything obvious.

	<mike


From bruce at leapyear.org  Sat Jan 21 02:40:15 2012
From: bruce at leapyear.org (Bruce Leban)
Date: Fri, 20 Jan 2012 17:40:15 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120120163648.69288796@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
Message-ID: <CAGu0AnurFr_ATD8vaX+deLtp+kJq2m=E=2Tm1eyu1r1D_ojy6Q@mail.gmail.com>

On Fri, Jan 20, 2012 at 4:36 PM, Mike Meyer <mwm at mired.org> wrote:

> A single new keyword/clause, "retry". It has the same syntax as an
> "except" clause, can be used anywhere "except" can be used, and can be
> intermingled with them in the same try statement. There's probably a
> better syntax, but this is easy to describe.
>
> The behavior change from except is that instead of exiting the "try"
> statement when the "retry" clause ends, it restarts the try
> clause. In python, this code:
>
>    try:
>        # try block
>    except:
>        # except block
>    retry:
>        # retry block
>    else:
>        # else block
>    finally:
>        # finally block
>

Can you write this in terms of current Python code? I don't understand
exactly when a retry block would be executed. In Eiffel, retry is a
statement, not a clause. Analogous to that would be:

try:
    # block
except:
    # block
    if condition: retry

The equivalent of this in current Python is

while True:
    try:
        # block
    except:
        # block
        if condition: continue # retry
    break

FWIW, if this turns out to be a good idea, an alternate spelling that would
not require a new keyword is 'try again'.

--- Bruce
Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120120/e067c101/attachment.html>

From ron3200 at gmail.com  Sat Jan 21 03:24:30 2012
From: ron3200 at gmail.com (Ron Adam)
Date: Fri, 20 Jan 2012 20:24:30 -0600
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120120163648.69288796@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
Message-ID: <1327112670.2481.4.camel@Gutsy>

On Fri, 2012-01-20 at 16:36 -0800, Mike Meyer wrote:
> The behavior change from except is that instead of exiting the "try"
> statement when the "retry" clause ends, it restarts the try
> clause. In python, this code:
> 
>     try:
>         # try block
>     except:
>         # except block
>     retry:
>         # retry block
>     else:
>         # else block
>     finally:
>         # finally block
> 
> Would provide the same behavior as this code:
> 
>     while True:
>         try:
>             # try block
>         except:
>             # except block
>         retry:
>             # retry block
>         else:
>             # else block
>         finally:
>             # finally block
>             break

This looks like it would be very tricky to get right from a python
programming stand point.  Remember that python is full of one time
iterators that won't repeat properly on a second try. Also many function
calls mutate the arguments.  So it's possible that a data structure
could be changed on the first try, and give different results on a
second try in an unexpected way.

Cheers,
   Ron




From benjamin at python.org  Sat Jan 21 04:13:46 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 21 Jan 2012 03:13:46 +0000 (UTC)
Subject: [Python-ideas] Retrying EAFP without DRY
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<CAGu0AnurFr_ATD8vaX+deLtp+kJq2m=E=2Tm1eyu1r1D_ojy6Q@mail.gmail.com>
Message-ID: <loom.20120121T041319-555@post.gmane.org>

Bruce Leban <bruce at ...> writes:
> 
> FWIW, if this turns out to be a good idea, an alternate spelling that would
not require a new keyword is 'try again'.

How is that not a new keyword?





From steve at pearwood.info  Sat Jan 21 05:16:36 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 21 Jan 2012 15:16:36 +1100
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <loom.20120121T041319-555@post.gmane.org>
References: <20120120163648.69288796@mikmeyer-vm-fedora>	<CAGu0AnurFr_ATD8vaX+deLtp+kJq2m=E=2Tm1eyu1r1D_ojy6Q@mail.gmail.com>
	<loom.20120121T041319-555@post.gmane.org>
Message-ID: <4F1A3C24.9090508@pearwood.info>

Benjamin Peterson wrote:
> Bruce Leban <bruce at ...> writes:
>> FWIW, if this turns out to be a good idea, an alternate spelling that would
> not require a new keyword is 'try again'.
> 
> How is that not a new keyword?


"again" could be a not-actually-a-keyword keyword like "as" used to be.

[steve at sylar ~]$ python2.3
Python 2.3.7 (#1, Aug 12 2010, 00:17:29)
[GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> as = 20
 >>> import math as m
 >>> m.sin(as)
0.91294525072762767


Not that I support that, but it is possible.


-- 
Steven


From pyideas at rebertia.com  Sat Jan 21 05:20:51 2012
From: pyideas at rebertia.com (Chris Rebert)
Date: Fri, 20 Jan 2012 20:20:51 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120120163648.69288796@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
Message-ID: <CAMZYqRSkd1cz30x1pjpnETGBozxSdiwMS8KJaKuCm32m7bRQ1g@mail.gmail.com>

On Fri, Jan 20, 2012 at 4:36 PM, Mike Meyer <mwm at mired.org> wrote:
> Just running another half-baked "easy" idea up to see what others
> think of it.
>
> One of the cases where EAFP is clumsy is if the "forgiveness" portion
> is actually retrying what failed after fixing things. I just ran into
> this in real code, and the resulting discussion suggested this change.

I would be very interested if you give at least the general gist of
the part of your code in question. Real-world use cases are always
useful data.

> Eiffel's exception-handling mechanism is built assuming that's the
> *standard* way to do things. Seems to work ok, but if I had to choose
> one of the two, I'll take try/except.
>
> So here's the proposal:
>
> A single new keyword/clause, "retry". It has the same syntax as an
> "except" clause, can be used anywhere "except" can be used, and can be
> intermingled with them in the same try statement. There's probably a
> better syntax, but this is easy to describe.
>
> The behavior change from except is that instead of exiting the "try"
> statement when the "retry" clause ends, it restarts the try
> clause. In python, this code:
>
> ? ?try:
> ? ? ? ?# try block
> ? ?except:
> ? ? ? ?# except block
> ? ?retry:
> ? ? ? ?# retry block
> ? ?else:
> ? ? ? ?# else block
> ? ?finally:
> ? ? ? ?# finally block
<snip>
> The use case, as mentioned, is avoiding doing EAFP without repeating
> the code in the exception handler. I.e., the choices without retry
> are things like (LBYL):
<snip>
> or EAFP:
>
> ? ?try:
> ? ? ? ?mangle(x)
> ? ?except:
> ? ? ? ?x = fixup(x)
> ? ? ? ?mangle(x)
> ? ?other_stuff(x)

Right, that has evil code duplication.

> or:
>
> ? ?while True:
> ? ? ? ?try:
> ? ? ? ? ? ?mangle(x)
> ? ? ? ? ? ?break

Personally, I would put the `break` in a separate `else` clause.

> ? ? ? ?except:
> ? ? ? ? ? ?x = fixup(x)
> ? ?other_stuff(x)

Your mention duplication as a gripe, yet there's no duplication in
this version. I take then it that you just dislike this idiom?

> with:
>
> ? ?try:
> ? ? ? ?mangle(x)
> ? ?retry:
> ? ? ? ?e = fixup(x)
> ? ?other_stuff(x)

Do I misunderstand your proposal, or are you missing an "except:
retry" here? If you aren't, then how is infinite-retrying avoided?

I would think that in many (most?) cases, we'd like to cap the number
of reattempts.
Currently, this looks like:

for _ in range(5):
    try:
        buy_off_ebay(item, price)
    except OutbidError:
        price = highest_bid_for(item) * 1.05
        # continue
    else:
        break
# Some fiddling can be had over where to put the `break`
# and whether to include an explicit `continue`.

With what I understand `retry` would be:

# Not even bothering with the capping
try:
    buy_off_ebay(item, price)
except OutbidError:
    retry
retry:
    price = highest_bid_for(item) * 1.05
# I'd have to maintain the count myself?!
# That's easily going to squander the 1 line this saved.

I grant you that this has saved 1 level of indentation, whatever
that's worth (personally, I don't put enough premium on it in this
case to justify new syntax). It's also slightly more explicit (once
you know what `retry` does, of course). And, in more general cases, a
very few lines can be saved.

But when one gets to the more general, complicated cases, it seems to
me that you're better off just writing and making use of your own
try_N_times(make_bid, N) [or whatever your "keep retrying" heuristic
is] function, which argues against adding new syntax for this.

> My gut reaction is cool, but not clear it's worth a new keyword. So
> maybe there's a tweak to the except clause that would have the same
> effect, but I don't see anything obvious.

My view on Python's choice of control structures is that it provides a
near-minimal, simple (so long as you overlook the obscure and
seldom-used `else` clause of `for` & `while`) set of them that you can
then combine to achieve fancier behavior (e.g. while + try + break =
retry). For example, there's no do-while (instead: while True +
if-break); there's no unless/until (instead: while/if + not); there's
no C-style `for` (instead: for+range(), or a custom while); there's no
`redo` (instead: nested while True + if-break/continue); and thus, no
`retry` (instead: nested try within a loop).

This keeps the core language small, aesthetically clean, uniform, and
easy-to-learn. If one understands `if`, `while`, `for`, `try`,
`break`, and `continue` (and `not`, but that's merely a boolean
operator, as opposed to a control structure), then one can easily work
out from scratch what the idioms do, and one learns to recognize them
with just a bit of practice/experience. Also, it's easy to tweak the
idioms when your control flow changes or becomes more complicated,
thus allowing for flexibility.

On the other hand, some clarity/explicitness is definitely lost.
There's a design trade-off here; I don't think Python has tended
towards the side that would add `retry`.

Cheers,
Chris
--
http://rebertia.com


From steve at pearwood.info  Sat Jan 21 08:47:46 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 21 Jan 2012 18:47:46 +1100
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120120163648.69288796@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
Message-ID: <4F1A6DA2.5020807@pearwood.info>

Mike Meyer wrote:

> A single new keyword/clause, "retry". It has the same syntax as an
> "except" clause, can be used anywhere "except" can be used, and can be
> intermingled with them in the same try statement. There's probably a
> better syntax, but this is easy to describe.

> The behavior change from except is that instead of exiting the "try"
> statement when the "retry" clause ends, it restarts the try

This sounds like retry should be a flow-control statement, like continue or 
break, not a block. E.g.:

try:
     something()
except ValueError:
     if condition: retry
     else: raise


"retry" will jump back to the start of the try block -- a limited form of 
GOTO, with all the pros and cons of this.



> clause. In python, this code:
>
>     try:
>         # try block
>     except:
>         # except block
>     retry:
>         # retry block
>     else:
>         # else block
>     finally:
>         # finally block
> 
> Would provide the same behavior as this code:
> 
>     while True:
>         try:
>             # try block
>         except:
>             # except block
>         retry:
>             # retry block
>         else:
>             # else block
>         finally:
>             # finally block
>             break

Um, you're trying to explain the behaviour of retry here, but your Python code 
includes a retry. This is one case where recursion is NOT your friend.



> Where except & retry can be repeated, include exceptions to check
> against, and intermixed with each other. The break would propagate up
> to the else block and any except blocks if there was no finally. If
> there's no else, the break from it winds up at the end of the try.

None of your explanation is clear to me.

Under what circumstances will the retry block be executed? E.g. given:

   try:
       len(None)
   except TypeError:
       print("spam spam spam")
   retry:
       print("a")

what will happen?

How about these?

   #1
   try:
       len(None)
   retry:
       print("a")
   except TypeError:
       print("spam spam spam")

   #2
   try:
       len(None)
   except TypeError:
       print("spam spam spam")
   retry:
       print("a")
   except ValueError:
       print("this never gets called")
   retry:
       print("b")

   #3
   try:
       len(None)
   except TypeError:
       print("spam spam spam")
   retry:
       print("a")
   retry:
       print("b")


> The use case, as mentioned, is avoiding doing EAFP without repeating
> the code in the exception handler. I.e., the choices without retry
> are things like (LBYL):

I think that the idiom of a while or for loop is easy enough to read and write:

for _ in range(5):  # retry five times
     try:
         do_something(x)
     except SpamError:
         x = fix_up(x)
     else:
         break
else:
     raise HamError("tried 5 times, giving up now")


I just wish that break and continue could be written outside of a loop, so you 
can factor out common code:

def do_the_thing(x):
     try:
         do_something(x)
     except SpamError:
         x = fix_up(x)
     else:
         break

def try_repeatedly(n, func):
     for _ in range(n):
         func()
     else:
         raise HamError('tried %d times, giving up now" % n)

try_repeatedly(5, do_the_thing)



-- 
Steven


From pyideas at rebertia.com  Sat Jan 21 09:25:25 2012
From: pyideas at rebertia.com (Chris Rebert)
Date: Sat, 21 Jan 2012 00:25:25 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <4F1A6DA2.5020807@pearwood.info>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
Message-ID: <CAMZYqRQhPS_Rp1DSFDCy5bQUfP1U8WbcwFy2xX0cr65CNwMuOA@mail.gmail.com>

On Fri, Jan 20, 2012 at 11:47 PM, Steven D'Aprano <steve at pearwood.info> wrote:
<snip>
> I think that the idiom of a while or for loop is easy enough to read and
> write:
>
> for _ in range(5): ?# retry five times
> ? ?try:
> ? ? ? ?do_something(x)
> ? ?except SpamError:
> ? ? ? ?x = fix_up(x)
> ? ?else:
> ? ? ? ?break
> else:
> ? ?raise HamError("tried 5 times, giving up now")
>
>
> I just wish that break and continue could be written outside of a loop, so
> you can factor out common code:

You also seem to have some shenanigans going on with `x`.

> def do_the_thing(x):
> ? ?try:
> ? ? ? ?do_something(x)
> ? ?except SpamError:
> ? ? ? ?x = fix_up(x)
> ? ?else:
> ? ? ? ?break
>
> def try_repeatedly(n, func):
> ? ?for _ in range(n):
> ? ? ? ?func()
> ? ?else:
> ? ? ? ?raise HamError('tried %d times, giving up now" % n)
>
> try_repeatedly(5, do_the_thing)

Easily accomplished:

def do_the_thing(x):
    try:
        do_something(x)
    except SpamError:
        fix_up(x)
        return False
    else:
        return True

def try_repeatedly(n, func, arg):
   for _ in range(n):
       if func(arg): break
   else:
       raise HamError('tried %d times, giving up now" % n)

try_repeatedly(5, do_the_thing, y)

Cheers,
Chris


From steve at pearwood.info  Sat Jan 21 09:57:16 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 21 Jan 2012 19:57:16 +1100
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CAMZYqRQhPS_Rp1DSFDCy5bQUfP1U8WbcwFy2xX0cr65CNwMuOA@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>	<4F1A6DA2.5020807@pearwood.info>
	<CAMZYqRQhPS_Rp1DSFDCy5bQUfP1U8WbcwFy2xX0cr65CNwMuOA@mail.gmail.com>
Message-ID: <4F1A7DEC.40503@pearwood.info>

Chris Rebert wrote:
> On Fri, Jan 20, 2012 at 11:47 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> <snip>
>> I just wish that break and continue could be written outside of a loop, so
>> you can factor out common code:
[...]
> Easily accomplished:
> 
> def do_the_thing(x):
>     try:
>         do_something(x)
>     except SpamError:
>         fix_up(x)
>         return False
>     else:
>         return True
> 
> def try_repeatedly(n, func, arg):
>    for _ in range(n):
>        if func(arg): break
>    else:
>        raise HamError('tried %d times, giving up now" % n)
> 
> try_repeatedly(5, do_the_thing, y)

Not so easily accomplished if you need the return result from do_the_thing. 
Naturally you can always return a tuple

(result_I_actually_want, code_to_break_or_continue)

but that's not exactly elegant.




-- 
Steven



From stephen at xemacs.org  Sat Jan 21 10:11:48 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 21 Jan 2012 18:11:48 +0900
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <4F1A6DA2.5020807@pearwood.info>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
Message-ID: <87r4yt5siz.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > Mike Meyer wrote:

 > "retry" will jump back to the start of the try block

This doesn't sound very intuitive to me.

 > -- a limited form of GOTO, with all the pros and cons of this.

All control flow is a limited form of GOTO.  But in this case, retry
can only go to one place, about as unlike GOTO as I can imagine.  It
certainly doesn't have any cons of GOTO that "while" doesn't have!

 > > The use case, as mentioned, is avoiding doing EAFP without repeating
 > > the code in the exception handler. I.e., the choices without retry
 > > are things like (LBYL):
 > 
 > I think that the idiom of a while or for loop is easy enough to read and write:
 > 
 > for _ in range(5):  # retry five times
 >      try:
 >          do_something(x)
 >      except SpamError:
 >          x = fix_up(x)
 >      else:
 >          break
 > else:
 >      raise HamError("tried 5 times, giving up now")

Agreed.  I find this much easier to understand than the "retry" form.

It seems to me that rather than a new keyword "retry", I'd want to use
the existing "continue" here as with other looping constructs.  The
problem is that this would interfere with other uses of continue, not
to mention that it would be inconsistent with break.  While this
thought experiment doesn't prove anything, it makes me wonder if the
idea is really coherent (at least in the context of Python today).

 > I just wish that break and continue could be written outside of a
 > loop, so you can factor out common code:
 > 
 > def do_the_thing(x):
 >      try:
 >          do_something(x)
 >      except SpamError:
 >          x = fix_up(x)
 >      else:
 >          break
 > 
 > def try_repeatedly(n, func):
 >      for _ in range(n):
 >          func()
 >      else:
 >          raise HamError('tried %d times, giving up now" % n)
 > 
 > try_repeatedly(5, do_the_thing)

I really don't like the idea of using a nonlocal construct as part of
ordinary control flow.  I guess I could get used to this particular
idiom, but it's not like the explicit loop is terribly burdensome, to
write or to read.



From cmjohnson.mailinglist at gmail.com  Sat Jan 21 10:14:41 2012
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Fri, 20 Jan 2012 23:14:41 -1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <4F1A7DEC.40503@pearwood.info>
References: <20120120163648.69288796@mikmeyer-vm-fedora>	<4F1A6DA2.5020807@pearwood.info>
	<CAMZYqRQhPS_Rp1DSFDCy5bQUfP1U8WbcwFy2xX0cr65CNwMuOA@mail.gmail.com>
	<4F1A7DEC.40503@pearwood.info>
Message-ID: <56CBC685-6091-49EB-B2C3-865D89C3B4F4@gmail.com>


On Jan 20, 2012, at 10:57 PM, Steven D'Aprano wrote:

> Chris Rebert wrote:
>> On Fri, Jan 20, 2012 at 11:47 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> <snip>
>>> I just wish that break and continue could be written outside of a loop, so
>>> you can factor out common code:
> [...]
>> Easily accomplished:
>> def do_the_thing(x):
>>    try:
>>        do_something(x)
>>    except SpamError:
>>        fix_up(x)
>>        return False
>>    else:
>>        return True
>> def try_repeatedly(n, func, arg):
>>   for _ in range(n):
>>       if func(arg): break
>>   else:
>>       raise HamError('tried %d times, giving up now" % n)
>> try_repeatedly(5, do_the_thing, y)
> 
> Not so easily accomplished if you need the return result from do_the_thing. Naturally you can always return a tuple
> 
> (result_I_actually_want, code_to_break_or_continue)
> 
> but that's not exactly elegant.

I'd make the place I factor out code different:

    def try_repeatedly(times, exception, callback, args, kwargs):
        for _ in range(times):
            try:
                return callback(*args, **kwargs)
            except exception:
                continue
            break
        error_message = "Tried {} times but could not complete callback {}"
        error_message = error_message.format(times, repr(callback))
        raise Exception(error_message)

    try_repeatedly(5, SpamError, do_something, [x])

From cmjohnson.mailinglist at gmail.com  Sat Jan 21 10:17:50 2012
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Fri, 20 Jan 2012 23:17:50 -1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <56CBC685-6091-49EB-B2C3-865D89C3B4F4@gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>	<4F1A6DA2.5020807@pearwood.info>
	<CAMZYqRQhPS_Rp1DSFDCy5bQUfP1U8WbcwFy2xX0cr65CNwMuOA@mail.gmail.com>
	<4F1A7DEC.40503@pearwood.info>
	<56CBC685-6091-49EB-B2C3-865D89C3B4F4@gmail.com>
Message-ID: <86BAF22E-86C8-4DCE-8E6B-BDE0E3EC502A@gmail.com>


On Jan 20, 2012, at 11:14 PM, Carl M. Johnson wrote:

>    def try_repeatedly(times, exception, callback, args, kwargs):
>        for _ in range(times):
>            try:
>                return callback(*args, **kwargs)
>            except exception:
>                continue
>            break
>        error_message = "Tried {} times but could not complete callback {}"
>        error_message = error_message.format(times, repr(callback))
>        raise Exception(error_message)
> 
>    try_repeatedly(5, SpamError, do_something, [x])

Oops, don't need the "break".


From steve at pearwood.info  Sat Jan 21 10:38:55 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 21 Jan 2012 20:38:55 +1100
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <87r4yt5siz.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120120163648.69288796@mikmeyer-vm-fedora>	<4F1A6DA2.5020807@pearwood.info>
	<87r4yt5siz.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4F1A87AF.7060604@pearwood.info>

Stephen J. Turnbull wrote:
> Steven D'Aprano writes:
> 
>  > Mike Meyer wrote:
> 
>  > "retry" will jump back to the start of the try block
> 
> This doesn't sound very intuitive to me.

No, not very. You might not guess what it does, but it's not that hard to 
learn: "retry jumps back to the beginning of the try block".


>  > -- a limited form of GOTO, with all the pros and cons of this.
> 
> All control flow is a limited form of GOTO.  But in this case, retry
> can only go to one place, about as unlike GOTO as I can imagine.  It
> certainly doesn't have any cons of GOTO that "while" doesn't have!

The main con that I can think of is that it isn't clear that you're looping 
until you reach the retry.


try:
    ...
except Spam:
    ...
except Ham:
    ...
else:
    if flag: ...
    else: retry  # Ah, we're in a loop!


I don't hate this, but nor am I going to champion it. If someone else wants to 
champion it, I'd vote +0. It just feels more natural than making retry a block 
clause (Mike's proposal), but I'm not really convinced it's necessary.

[...]
> It seems to me that rather than a new keyword "retry", I'd want to use
> the existing "continue" here as with other looping constructs.  The
> problem is that this would interfere with other uses of continue, not
> to mention that it would be inconsistent with break.  While this
> thought experiment doesn't prove anything, it makes me wonder if the
> idea is really coherent (at least in the context of Python today).

continue can't work, because it introduces ambiguity if you have a try inside 
a loop, or a loop inside a try. We could arbitrarily declare that try-continue 
wins over loop-continue, or visa versa, but whatever decision we make, it will 
leave half the would-be users unhappy.

Besides, there's a neat symmetry: you try something, then you re-try it. 
(Believe it or not, I've only just noticed this.)



-- 
Steven



From p.f.moore at gmail.com  Sat Jan 21 12:46:19 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 21 Jan 2012 11:46:19 +0000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <4F1A6DA2.5020807@pearwood.info>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
Message-ID: <CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>

On 21 January 2012 07:47, Steven D'Aprano <steve at pearwood.info> wrote:
> This sounds like retry should be a flow-control statement, like continue or
> break, not a block. E.g.:
>
> try:
> ? ?something()
> except ValueError:
> ? ?if condition: retry
> ? ?else: raise
>
>
> "retry" will jump back to the start of the try block -- a limited form of
> GOTO, with all the pros and cons of this.

That's the way I would interpret a "retry" statement - and it's
nothing like the OP's proposal as far as I can see (where retry
introduces a suite).

I'd be -1 on a retry keyword that worked any way other than this
(simply because this is "clearly" the most obvious interpretation).
Whether a retry statement is worth having at all, though, is something
I'm not sure of - I'd like to see some real-world use cases first.
I've never encountered the need for it myself. And I can't honestly
imagine why the while True...try...break" idiom would ever not be
sufficient.

Paul.


From ncoghlan at gmail.com  Sat Jan 21 13:06:11 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Jan 2012 22:06:11 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CAGu0AnurFr_ATD8vaX+deLtp+kJq2m=E=2Tm1eyu1r1D_ojy6Q@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<CAGu0AnurFr_ATD8vaX+deLtp+kJq2m=E=2Tm1eyu1r1D_ojy6Q@mail.gmail.com>
Message-ID: <CADiSq7dT0cNbadBvGCA96CeW5CrN4fRJKfYDL66J+7tCgPaEKQ@mail.gmail.com>

On Sat, Jan 21, 2012 at 11:40 AM, Bruce Leban <bruce at leapyear.org> wrote:
> while True:
> ? ? try:
> ? ? ? ? # block
> ? ? except:
> ? ? ? ? # block
> ? ? ? ? if condition: continue # retry
> ? ? break

And, indeed, I'd consider "while True" to be a fairly idiomatic way to
write "keep trying this until it works (and we exit the loop via
break) or we give up (and raise an exception)" without repeating
ourselves. Alternatively, I'd push the entire try/except block into a
closure and call that from the loop until it reported it had worked. A
nested generator could achieve much the same thing.

I don't think there needs to be "one obvious way" to do this, since
it's a rare need and the existing tools (while loops with
break/continue, a loop with an appropriate flag, a loop plus a
closure, a nested generator, etc) all provide ways to implement it
reasonably cleanly.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From guido at python.org  Sat Jan 21 17:42:05 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Jan 2012 08:42:05 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <4F1A3C24.9090508@pearwood.info>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<CAGu0AnurFr_ATD8vaX+deLtp+kJq2m=E=2Tm1eyu1r1D_ojy6Q@mail.gmail.com>
	<loom.20120121T041319-555@post.gmane.org>
	<4F1A3C24.9090508@pearwood.info>
Message-ID: <CAP7+vJ+zkDd2MiceJ5N+bubk7FZq2Vex+7bH7jrETns9y4VpaA@mail.gmail.com>

On Fri, Jan 20, 2012 at 8:16 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> "again" could be a not-actually-a-keyword keyword like "as" used to be.

Let's not do that again. It was a major pain.

-- 
--Guido van Rossum (python.org/~guido)


From sho at eikehein.com  Sat Jan 21 23:23:20 2012
From: sho at eikehein.com (Eike Hein)
Date: Sat, 21 Jan 2012 23:23:20 +0100
Subject: [Python-ideas] Combining test and assignment
Message-ID: <4F1B3AD8.6030402@eikehein.com>


Hello,

this is very spur of the moment, and so I apologize if
I am missing the obvious reasons for why this would be
horribly dumb.

I sometimes find myself annoyed that I cannot access the
result of an expression in a conditional statememt.

Consider:

if spam():
    ... frobulate the return value of spam() ...
else:
    ... value too low to frobulate ...

In order to frobulate spam(), either need to call it
again in the indented block, or preassign:

x = spam()

if x:
     ... frobulate x ...

The first option is usually inefficient, and the second
feels ugly to me because something relevant to a block is
happening outside it and its header.

Instead, I'd love to be able to:

if spam() as x:
    ... frobulate x ...

Further, there are two reasons this seems a particularly
good fit in Python:
- There is a _syntactic_ precedent, or at least great
   similarity to the 'with' statement.
- Because calls in Python return None if they don't ex-
   plicitly return anything else, there is no "can't assign
   void to x" to worry about.

I haven't given much thought to scoping, particularly for
else clauses, yet, at least not enough to make up my mind.

The other question is if this reads good "as English" or
needs to be 'if spam() then as x:', or if that's even an
issue.

Thoughts?


Best regards,
Eike


From jkbbwr at gmail.com  Sun Jan 22 00:30:49 2012
From: jkbbwr at gmail.com (Jakob Bowyer)
Date: Sat, 21 Jan 2012 23:30:49 +0000
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <4F1B3AD8.6030402@eikehein.com>
References: <4F1B3AD8.6030402@eikehein.com>
Message-ID: <CAA+RL7EmJHYX8D5BTrZi4ZmYewT_MuN__4ux=pKf2c+U_8YpKQ@mail.gmail.com>

Say I have a function that is
def test():
    return sometimes_false

if test() as x:
    print "Its not false"
# what happens to x now? is it a NameError or is it None

On Sat, Jan 21, 2012 at 10:23 PM, Eike Hein <sho at eikehein.com> wrote:

>
> Hello,
>
> this is very spur of the moment, and so I apologize if
> I am missing the obvious reasons for why this would be
> horribly dumb.
>
> I sometimes find myself annoyed that I cannot access the
> result of an expression in a conditional statememt.
>
> Consider:
>
> if spam():
>   ... frobulate the return value of spam() ...
> else:
>   ... value too low to frobulate ...
>
> In order to frobulate spam(), either need to call it
> again in the indented block, or preassign:
>
> x = spam()
>
> if x:
>    ... frobulate x ...
>
> The first option is usually inefficient, and the second
> feels ugly to me because something relevant to a block is
> happening outside it and its header.
>
> Instead, I'd love to be able to:
>
> if spam() as x:
>   ... frobulate x ...
>
> Further, there are two reasons this seems a particularly
> good fit in Python:
> - There is a _syntactic_ precedent, or at least great
>  similarity to the 'with' statement.
> - Because calls in Python return None if they don't ex-
>  plicitly return anything else, there is no "can't assign
>  void to x" to worry about.
>
> I haven't given much thought to scoping, particularly for
> else clauses, yet, at least not enough to make up my mind.
>
> The other question is if this reads good "as English" or
> needs to be 'if spam() then as x:', or if that's even an
> issue.
>
> Thoughts?
>
>
> Best regards,
> Eike
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120121/ec096d7a/attachment.html>

From steve at pearwood.info  Sun Jan 22 02:15:42 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 22 Jan 2012 12:15:42 +1100
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CAA+RL7EmJHYX8D5BTrZi4ZmYewT_MuN__4ux=pKf2c+U_8YpKQ@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CAA+RL7EmJHYX8D5BTrZi4ZmYewT_MuN__4ux=pKf2c+U_8YpKQ@mail.gmail.com>
Message-ID: <4F1B633E.9060406@pearwood.info>

Jakob Bowyer wrote:
> Say I have a function that is
> def test():
>     return sometimes_false
> 
> if test() as x:
>     print "Its not false"
> # what happens to x now? is it a NameError or is it None

Neither. It should be syntactic sugar for this:

x = test()
if x:
     print "Its not false"

So x will always be sometimes_false, whatever that happens to be.



-- 
Steven


From ncoghlan at gmail.com  Sun Jan 22 04:48:20 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 22 Jan 2012 13:48:20 +1000
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <4F1B3AD8.6030402@eikehein.com>
References: <4F1B3AD8.6030402@eikehein.com>
Message-ID: <CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>

This suggestion (i.e. embedded assignment in while and if headers) has
been made several times, and always come to a grinding halt on one
simple problem: it isn't expressive enough.

The current idiom is this :

    x = EXPR
    if predicate(x):
        # Do something

Given a value and a predicate, this allows you to execute code
conditionally based on whether or not the predicate is true.

Simple embedded assignment, though, only works when the predicate is
just "bool" - as soon as the condition differs from the value you want
to access, you need to revert to the existing idiom *anyway*. Given
the non-trivial costs of adding new syntax, that gets the idea put
into the "not worth the hassle" bucket.

The obvious escalation of the suggestion is to adopt "(EXPR as NAME)"
as a general purpose embedded assignment expression. However, this
idea has problems of its own:

1. In with statements and exception handlers, 'as' does *not* denote
ordinary assignment. In the former case, the name is bound to the
result of the EXPR.__enter__() call, in the latter to the exception
instance that was actually caught. If assignment expressions were
added, these could become a rather subtle trap (especially since many
__enter__() methods just return self)

2. Generator expressions and the various forms of comprehension create
new scopes to avoid leaking iteration variables, so, if embedded
assignments were to do anything useful there, additional forms like
"(EXPR as nonlocal NAME)" and "(EXPR as global NAME)" would be needed.
(And, in the nonlocal case, would still require that the name already
be defined in the containing scope)

3. Adding assignment expressions would mean having two ways to perform
assignments at the statement level (either using the existing
statement form or using the new expression form)

To date, nobody has been interested enough in the latter idea to put
together a formal PEP and reference implementation for python-dev's
consideration, and the former idea has been informally rejected
several times due to the lack of generality.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From sho at eikehein.com  Sun Jan 22 05:05:48 2012
From: sho at eikehein.com (Eike Hein)
Date: Sun, 22 Jan 2012 05:05:48 +0100
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
Message-ID: <4F1B8B1C.8050106@eikehein.com>

On 1/22/2012 4:48 AM, Nick Coghlan wrote:
> Simple embedded assignment, though, only works when the predicate is
> just "bool" - as soon as the condition differs from the value you want
> to access, you need to revert to the existing idiom *anyway*. Given
> the non-trivial costs of adding new syntax, that gets the idea put
> into the "not worth the hassle" bucket.

Actually, just to be clear, in the use case I had in mind
spam() is returning a numeric value. That satisfies a truth
test and is still useful in other ways (but I assume you
are actually on the same page, hence the quotation marks
around bool).

Another case I come across frequently is with the re module,
where re.match() returns either None or a MatchObject.

if re.match(pattern, string) as m:
     ... make use of the match object ...

Anyway, thank you for a comprehensive reply and recap of
the history.


> To date, nobody has been interested enough in the latter idea to put
> together a formal PEP and reference implementation for python-dev's
> consideration, and the former idea has been informally rejected
> several times due to the lack of generality.

I thought a bit about what Python offers today in terms
of built-in tools to approximate what I want. My first
thought was to try and abuse context managers, but aside
from being horribly ugly, a with statement also can't
prevent its body from being run, it seems.

Another, possibly better way is to modify locals(), i.e.:

if test(expr, 'var'):
    ... do stuff ...

Where the test function returns the expression but also
stores its results in locals()['var'].

I wonder if a function like this (with a better name
obviously) would have a place in the stdlib somewhere,
or is it too much of a limited-use hack?


> Cheers,
> Nick.

-- 
Best regards,
Eike Hein


From pyideas at rebertia.com  Sun Jan 22 05:11:46 2012
From: pyideas at rebertia.com (Chris Rebert)
Date: Sat, 21 Jan 2012 20:11:46 -0800
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <4F1B8B1C.8050106@eikehein.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B8B1C.8050106@eikehein.com>
Message-ID: <CAMZYqRRjj=JXjkOSQNkzg8S6cG_rUJjuOge=p=D6LQKRkcfuyA@mail.gmail.com>

On Sat, Jan 21, 2012 at 8:05 PM, Eike Hein <sho at eikehein.com> wrote:
> On 1/22/2012 4:48 AM, Nick Coghlan wrote:
>> Simple embedded assignment, though, only works when the predicate is
>> just "bool" - as soon as the condition differs from the value you want
>> to access, you need to revert to the existing idiom *anyway*. Given
>> the non-trivial costs of adding new syntax, that gets the idea put
>> into the "not worth the hassle" bucket.
<snip>
> I thought a bit about what Python offers today in terms
> of built-in tools to approximate what I want. My first
> thought was to try and abuse context managers, but aside
> from being horribly ugly, a with statement also can't
> prevent its body from being run, it seems.
>
> Another, possibly better way is to modify locals(), i.e.:
>
> if test(expr, 'var'):
> ? ... do stuff ...
>
> Where the test function returns the expression but also
> stores its results in locals()['var'].

That doesn't actually work (unless you happen to be writing code in
module-level scope, where globals are locals):
http://docs.python.org/library/functions.html#locals :
"Note: The contents of this dictionary should not be modified; changes
may not affect the values of local and free variables used by the
interpreter."

Cheers,
Chris


From sho at eikehein.com  Sun Jan 22 05:18:24 2012
From: sho at eikehein.com (Eike Hein)
Date: Sun, 22 Jan 2012 05:18:24 +0100
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CAMZYqRRjj=JXjkOSQNkzg8S6cG_rUJjuOge=p=D6LQKRkcfuyA@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B8B1C.8050106@eikehein.com>
	<CAMZYqRRjj=JXjkOSQNkzg8S6cG_rUJjuOge=p=D6LQKRkcfuyA@mail.gmail.com>
Message-ID: <4F1B8E10.9050609@eikehein.com>

On 1/22/2012 5:11 AM, Chris Rebert wrote:
> That doesn't actually work (unless you happen to be writing code in
> module-level scope, where globals are locals):

Right, brainfart - the locals() of test aren't the
locals of the callsite obviously :). And nonlocal
won't work due to the lack of an existing binding
in the outer scope.

Can anyone think of another way to do an assignment
and a truth test in the same expression with today's
tools?


> http://docs.python.org/library/functions.html#locals :
> "Note: The contents of this dictionary should not be modified; changes
> may not affect the values of local and free variables used by the
> interpreter."

Yep, I was actually confusing this with vars() in terms
of whether modifying its return value is considered
kosher. Thanks. Sadly that makes the above harder ...


> Cheers,
> Chris

-- 
Best regards,
Eike Hein


From steve at pearwood.info  Sun Jan 22 05:51:13 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 22 Jan 2012 15:51:13 +1100
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
Message-ID: <4F1B95C1.7020402@pearwood.info>

Nick Coghlan wrote:
> This suggestion (i.e. embedded assignment in while and if headers) has
> been made several times, and always come to a grinding halt on one
> simple problem: it isn't expressive enough.
[...]
> The obvious escalation of the suggestion is to adopt "(EXPR as NAME)"
> as a general purpose embedded assignment expression. However, this
> idea has problems of its own:


I dislike assignment as an expression, and am glad that Python doesn't have 
it. Even though the form "expr as name" avoids the common C gotcha, I am -1 on 
adding it to Python.

I believe it is an unnatural way of writing that obscures the code rather than 
simplifies it. (Sometimes, shorter is not better.) For example, when writing 
code, I might start off by thinking:

if spam() != 'ham':
     frobulate(spam())


Then I would apply DRY and factor out the call to spam:

x = spam()
if x != 'ham':
     frobulate(x)


This matches my thought processes and corresponds to how you might naturally 
describe the algorithm in English:

Let x equal spam().
If x != 'ham', then frobulate(x).

The important thing is that you name the thing you care about before using it. 
I think this is a very natural way of writing: first you give the thing you 
care about a name, then you refer to it by name.

Assignment as an expression feels unnatural to me:

if spam() as x != 'ham':
     frobulate(x)

doesn't really correspond to any natural English order. The assignment is 
dropped in the middle of another clause:

If -- let x = spam() -- x != 'ham', then frobulate(x).

It saves a line, but is not a natural way of writing for me. I would not like 
to read code written that way.

Assignment as an expression also lends itself to writing buggy code like this:

while spam() as x and ham() as y:
     frobulate(x)
     glommify(y)

which is wrong, because y won't be defined if x has a true value. The 
alternative would require defeating the short-circuit nature of "and", which 
would be bad.

Perhaps the only thing going for it is that it would allow list comprehensions 
to not repeat themselves:

# instead of this
[frobulate(x) for x in seq if frobulate(x) > 0]
# you could have this
[y for x in seq if frobulate(x) as y > 0]

Although I prefer to use an inner map:

[y for y in map(frobulate, seq) if y > 0]  # use itertools.imap in Python2



-- 
Steven


From python at 2sn.net  Sun Jan 22 06:33:23 2012
From: python at 2sn.net (Alexander Heger)
Date: Sat, 21 Jan 2012 23:33:23 -0600
Subject: [Python-ideas] Combining test and assignment, quantifications,
 and tuple patterns
In-Reply-To: <4F1B95C1.7020402@pearwood.info>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
Message-ID: <4F1B9FA3.10108@2sn.net>

I think one could use the same new construct I proposed for one of the 
recent lengthy discussions.  For your convenience, I copy this below, in 
disregards of quoting style as it provides an overarching reply to this 
thread.

You could use

sample <generator expression> as x:
     # do something with x
     break
else:
     # do something else

here the generator expression would be "spam()"

sample y for y in (spam(),) if cond(y) as x:
(...)

or allow a short form

sample y for spam() if cond(y) as x:
(...)

I really think this expressions could add a lot to Python's expressiveness.

As an important note, in the case below, the difference to just a "for 
loop" or "generator" would be that <generator expression> is 
re-evaluated from scratch every time the loop returns to the sample 
statement.  This is what I think would set it apart besides providing 
extra uses otherwise.

Admittedly, it is a very complicated construct.
Longer than most good English sentences.
(for which those above are not good examples)

-Alexander

-------- Original Message --------
Subject: Re: [Python-ideas] Fwd: quantifications, and tuple patterns
Date: Mon, 16 Jan 2012 22:00:52 -0600
From: Alexander Heger <python at 2sn.net>
To: python-ideas at python.org

Dear Tom,

 > I'm wondering if would make sense to add an "as" clause to the while 
and if
 > statements, similar to the as clause in the with statement. It would
 > essentially retrieve the value field from the witness object described
 > above.
 >
 > This would let one write the main loop in a topological sort (a classic
 > workset algorithm) as:
 >
 >      while any(v for v in vertices if v.incoming_count == 0) as v1:
 >          result.append(v)
 >          for v2 in v1.outgoing:
 >              v2.incoming_count -= 1

I had the same thought going through the thread, but I think the problem
here would be that any is a function that returns a Boolean value.  You
could add a key parameter to return something else, say a tuple of a
truth value and a sample ( any( ..., sample = True) ) but that could
break because

1) "any" could be another function in the context,

2) while would need to deal with special return values in this case

A suggestion to replace the "while any" case is

sample <generator expression> as x:
     # do something with x

   ("sample" may not be the best keyword choice) From the discussion so
far I do not see how to easily avoid having a new keyword

specifically, your example would become

sample v for v in vertices if v.incoming_count == 0 as v1:
      # do things with v1

In case you only want one sample, you could add the break statement

sample db.query(page_name=page_name) as page:
      render_template(page)
      break
else:
      error404()


-Alexander



On 01/21/2012 10:51 PM, Steven D'Aprano wrote:
> Nick Coghlan wrote:
>> This suggestion (i.e. embedded assignment in while and if headers) has
>> been made several times, and always come to a grinding halt on one
>> simple problem: it isn't expressive enough.
> [...]
>> The obvious escalation of the suggestion is to adopt "(EXPR as NAME)"
>> as a general purpose embedded assignment expression. However, this
>> idea has problems of its own:
>
>
> I dislike assignment as an expression, and am glad that Python doesn't have
> it. Even though the form "expr as name" avoids the common C gotcha, I am -1 on
> adding it to Python.
>
> I believe it is an unnatural way of writing that obscures the code rather than
> simplifies it. (Sometimes, shorter is not better.) For example, when writing
> code, I might start off by thinking:
>
> if spam() != 'ham':
>       frobulate(spam())
>
>
> Then I would apply DRY and factor out the call to spam:
>
> x = spam()
> if x != 'ham':
>       frobulate(x)
>
>
> This matches my thought processes and corresponds to how you might naturally
> describe the algorithm in English:
>
> Let x equal spam().
> If x != 'ham', then frobulate(x).
>
> The important thing is that you name the thing you care about before using it.
> I think this is a very natural way of writing: first you give the thing you
> care about a name, then you refer to it by name.
>
> Assignment as an expression feels unnatural to me:
>
> if spam() as x != 'ham':
>       frobulate(x)
>
> doesn't really correspond to any natural English order. The assignment is
> dropped in the middle of another clause:
>
> If -- let x = spam() -- x != 'ham', then frobulate(x).
>
> It saves a line, but is not a natural way of writing for me. I would not like
> to read code written that way.
>
> Assignment as an expression also lends itself to writing buggy code like this:
>
> while spam() as x and ham() as y:
>       frobulate(x)
>       glommify(y)
>
> which is wrong, because y won't be defined if x has a true value. The
> alternative would require defeating the short-circuit nature of "and", which
> would be bad.
>
> Perhaps the only thing going for it is that it would allow list comprehensions
> to not repeat themselves:
>
> # instead of this
> [frobulate(x) for x in seq if frobulate(x)>  0]
> # you could have this
> [y for x in seq if frobulate(x) as y>  0]
>
> Although I prefer to use an inner map:
>
> [y for y in map(frobulate, seq) if y>  0]  # use itertools.imap in Python2
>
>
>


From cmjohnson.mailinglist at gmail.com  Sun Jan 22 07:23:24 2012
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Sat, 21 Jan 2012 20:23:24 -1000
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <4F1B95C1.7020402@pearwood.info>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
Message-ID: <D9D6A259-568D-4B42-A3D4-56FB9755E736@gmail.com>


On Jan 21, 2012, at 6:51 PM, Steven D'Aprano wrote:

> I dislike assignment as an expression, and am glad that Python doesn't have it. Even though the form "expr as name" avoids the common C gotcha, I am -1 on adding it to Python.

I find it interesting that Go-lang follows your intuition and so added the "assignment first" conditional. Here's an example from their docs:

if x := f(); x < y {
	return x
} else if x > z {
	return z
} else {
	return y
}

In Python, we'd just put the x = f() first, but in Go, putting the assignment into the conditional affects the scoping of things. The x goes out of scope once the if is over.

From p.f.moore at gmail.com  Sun Jan 22 13:04:48 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 22 Jan 2012 12:04:48 +0000
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <4F1B95C1.7020402@pearwood.info>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
Message-ID: <CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>

On 22 January 2012 04:51, Steven D'Aprano <steve at pearwood.info> wrote:
> Assignment as an expression feels unnatural to me:
>
> if spam() as x != 'ham':
> ? ?frobulate(x)
>
> doesn't really correspond to any natural English order.

While in general I agree with you, there *is* a natural reading of this:

"if spam() isn't 'ham', frobulate it"

The fact that you have to choose a name is because computer languages
have to be more explicit than natural languages, and can't deal with
the implicit referent involved in the English usage of "it". The need
for assignment comes from the need for a name. You could actually deal
with the if-or-while-expression-assignment case by defining a special
variable __it__ to mean the value of the condition in the containing
if or while expression, but it has all the same lack of generality
issues as any proposal that limits itself to if/while conditions. And
short of explicitly marking the expression somehow, you can't
generalise any further.

Paul.


From ncoghlan at gmail.com  Sun Jan 22 13:37:40 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 22 Jan 2012 22:37:40 +1000
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
	<CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
Message-ID: <CADiSq7dvs_ADu7=SEN78RFizfZLGN8+VHOXm-OtXrdhoNsuG1A@mail.gmail.com>

On Sun, Jan 22, 2012 at 10:04 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 22 January 2012 04:51, Steven D'Aprano <steve at pearwood.info> wrote:
>> Assignment as an expression feels unnatural to me:
>>
>> if spam() as x != 'ham':
>> ? ?frobulate(x)
>>
>> doesn't really correspond to any natural English order.
>
> While in general I agree with you, there *is* a natural reading of this:
>
> "if spam() isn't 'ham', frobulate it"

Yeah, I agree characterising embedded assignments as a more formal
equivalent of natural language pronouns is a useful way to think about
them.

However, one of the other reasons that these proposals tend not to go
anywhere is that they ultimately boil down to some people hating the
idea of having to write something they personally think of as one
operation as a sequence of operations instead. Like multi-line
lambdas, adding embedded assignments doesn't really increase the power
and expressivity of the language as a whole all that much - it just
moves the dividing line between what can be cleanly expressed in a
single statement and what must be expressed as multiple statements a
bit.

Embedded assignments *do* address some instances of the
loop-and-a-half problem, but even there, the more general idiom of
"while True:" + "if exit_cond: break" can handle the situation quite
happily.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From storchaka at gmail.com  Sun Jan 22 19:02:40 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 22 Jan 2012 20:02:40 +0200
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
	<CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
Message-ID: <jfhj01$5n5$1@dough.gmane.org>

22.01.12 14:04, Paul Moore ???????(??):
> While in general I agree with you, there *is* a natural reading of this:
>
> "if spam() isn't 'ham', frobulate it"
>
> The fact that you have to choose a name is because computer languages
> have to be more explicit than natural languages, and can't deal with
> the implicit referent involved in the English usage of "it". The need
> for assignment comes from the need for a name. You could actually deal
> with the if-or-while-expression-assignment case by defining a special
> variable __it__ to mean the value of the condition in the containing
> if or while expression, but it has all the same lack of generality
> issues as any proposal that limits itself to if/while conditions. And
> short of explicitly marking the expression somehow, you can't
> generalise any further.

Kotlin (http://confluence.jetbrains.net/display/Kotlin/Kotlin) has "it":

names filter {it.startsWith("A")} sortby {it} map {it.toUpperCase()} 
foreach {print(it)}



From pyideas at rebertia.com  Sun Jan 22 19:36:05 2012
From: pyideas at rebertia.com (Chris Rebert)
Date: Sun, 22 Jan 2012 10:36:05 -0800
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
	<CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
Message-ID: <CAMZYqRSFyF4MXCEOXgrKRGbjsaPwg3+dkT5SpLBTeGct4OSTYA@mail.gmail.com>

On Sun, Jan 22, 2012 at 4:04 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 22 January 2012 04:51, Steven D'Aprano <steve at pearwood.info> wrote:
>> Assignment as an expression feels unnatural to me:
>>
>> if spam() as x != 'ham':
>> ? ?frobulate(x)
>>
>> doesn't really correspond to any natural English order.
>
> While in general I agree with you, there *is* a natural reading of this:
>
> "if spam() isn't 'ham', frobulate it"
>
> The fact that you have to choose a name is because computer languages
> have to be more explicit than natural languages, and can't deal with
> the implicit referent involved in the English usage of "it". The need
> for assignment comes from the need for a name. You could actually deal
> with the if-or-while-expression-assignment case by defining a special
> variable __it__ to mean the value of the condition in the containing
> if or while expression, but it has all the same lack of generality
> issues as any proposal that limits itself to if/while conditions. And
> short of explicitly marking the expression somehow, you can't
> generalise any further.

This is reminiscent of Perl's $_ variable, which represents the "current topic".
http://www.perl.com/pub/2002/10/30/topic.html

Cheers,
Chris


From mwm at mired.org  Sun Jan 22 22:30:14 2012
From: mwm at mired.org (Mike Meyer)
Date: Sun, 22 Jan 2012 13:30:14 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
Message-ID: <20120122133014.5e920a43@bhuda.mired.org>

On Sat, 21 Jan 2012 11:46:19 +0000
Paul Moore <p.f.moore at gmail.com> wrote:

> On 21 January 2012 07:47, Steven D'Aprano <steve at pearwood.info> wrote:
> > This sounds like retry should be a flow-control statement, like continue or
> > break, not a block. E.g.:
> >
> > try:
> > ? ?something()
> > except ValueError:
> > ? ?if condition: retry
> > ? ?else: raise
> >
> >
> > "retry" will jump back to the start of the try block -- a limited form of
> > GOTO, with all the pros and cons of this.
> 
> That's the way I would interpret a "retry" statement - and it's
> nothing like the OP's proposal as far as I can see (where retry
> introduces a suite).

Yes, it's not much like what I proposed. But it does solve the same
problem, and in a much better way than I proposed.

Because of that, I'm not going to try and fix the error in the OP of
not translating the proposal to proper Python :-(.

> I'd be -1 on a retry keyword that worked any way other than this
> (simply because this is "clearly" the most obvious interpretation).
> Whether a retry statement is worth having at all, though, is something
> I'm not sure of - I'd like to see some real-world use cases first.
> I've never encountered the need for it myself. And I can't honestly
> imagine why the while True...try...break" idiom would ever not be
> sufficient.

I saw a number of request for "real world uses cases". I thought I
covered that in the OP. This ideas was *prompted* by a real world use
case where we wanted to wrap an exception in our own private exception
before handling it. Because the code that would raise the exception
- other exceptions missing the attributes we added to ours - was the
code we want to run after handling them, we're left with LBYL or DRY
or wrapping a loop around the code when it wasn't really a loop.

   <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From pyideas at rebertia.com  Sun Jan 22 22:52:52 2012
From: pyideas at rebertia.com (Chris Rebert)
Date: Sun, 22 Jan 2012 13:52:52 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120122133014.5e920a43@bhuda.mired.org>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
Message-ID: <CAMZYqRSHWpvVu4M=zPkU-zU-CKP12y-82G26VQPpAmRWqDQzFA@mail.gmail.com>

On Sun, Jan 22, 2012 at 1:30 PM, Mike Meyer <mwm at mired.org> wrote:
> On Sat, 21 Jan 2012 11:46:19 +0000
> Paul Moore <p.f.moore at gmail.com> wrote:
>> On 21 January 2012 07:47, Steven D'Aprano <steve at pearwood.info> wrote:
<snip>
>> I'd be -1 on a retry keyword that worked any way other than this
>> (simply because this is "clearly" the most obvious interpretation).
>> Whether a retry statement is worth having at all, though, is something
>> I'm not sure of - I'd like to see some real-world use cases first.
>> I've never encountered the need for it myself. And I can't honestly
>> imagine why the while True...try...break" idiom would ever not be
>> sufficient.
>
> I saw a number of request for "real world uses cases". I thought I
> covered that in the OP. This ideas was *prompted* by a real world use
> case where we wanted to wrap an exception in our own private exception
> before handling it. Because the code that would raise the exception
> - other exceptions missing the attributes we added to ours - was the
> code we want to run after handling them, we're left with LBYL or DRY
> or wrapping a loop around the code when it wasn't really a loop.

The thing is, until this message, you never described any concrete
details about your specific use-case, other than that you had one.
Now you have, but in prose (which is regrettably imprecise) rather
than code; I'm still not entirely sure what your use case looks like.
As best I can parse it, a nested try-except sounds like it would work.

People like use-cases because they can reveal motivations or
subtleties that often turn out to be significant, and they can more
easily critique them than completely abstract/general cases.

Cheers,
Chris


From pyideas at rebertia.com  Sun Jan 22 22:59:47 2012
From: pyideas at rebertia.com (Chris Rebert)
Date: Sun, 22 Jan 2012 13:59:47 -0800
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CADwd9XmGgE7XkfrKT=PVukthHuGmn0d+-A+=1Z1GoK8n0YVzuQ@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
	<CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
	<CAMZYqRSFyF4MXCEOXgrKRGbjsaPwg3+dkT5SpLBTeGct4OSTYA@mail.gmail.com>
	<CADwd9XmGgE7XkfrKT=PVukthHuGmn0d+-A+=1Z1GoK8n0YVzuQ@mail.gmail.com>
Message-ID: <CAMZYqRRUvNQ1w2cO8EMkbzc_C0n7CLHGX+K+=-RSTK_yDeC7bg@mail.gmail.com>

> On Sun, Jan 22, 2012 at 1:36 PM, Chris Rebert <pyideas at rebertia.com> wrote:
>> On Sun, Jan 22, 2012 at 4:04 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>> > On 22 January 2012 04:51, Steven D'Aprano <steve at pearwood.info> wrote:
>> >> Assignment as an expression feels unnatural to me:
>> >>
>> >> if spam() as x != 'ham':
>> >> ? ?frobulate(x)
>> >>
>> >> doesn't really correspond to any natural English order.
>> >
>> > While in general I agree with you, there *is* a natural reading of this:
>> >
>> > "if spam() isn't 'ham', frobulate it"
>> >
>> > The fact that you have to choose a name is because computer languages
>> > have to be more explicit than natural languages, and can't deal with
>> > the implicit referent involved in the English usage of "it". The need
>> > for assignment comes from the need for a name. You could actually deal
>> > with the if-or-while-expression-assignment case by defining a special
>> > variable __it__ to mean the value of the condition in the containing
>> > if or while expression, but it has all the same lack of generality
>> > issues as any proposal that limits itself to if/while conditions. And
>> > short of explicitly marking the expression somehow, you can't
>> > generalise any further.
>>
>> This is reminiscent of Perl's $_ variable, which represents the "current
>> topic".
>> http://www.perl.com/pub/2002/10/30/topic.html

On Sun, Jan 22, 2012 at 1:24 PM, Edward Lesmes <ehlesmes at gmail.com> wrote:
> why not make python do something like
> _ = test()
> if _:
> ? ? ...
> Automatically, so you can do something like
>
> if test():
> ? ? frobulate(_)
>
>
> It seems natural to me, because in prompt,
> python assigns the last result to the
> _ variable

I think "Explicit is better than implicit"[0] pretty much covers why
that sort of thing is generally disfavored.

Cheers,
Chris
--
[0]: http://www.python.org/dev/peps/pep-0020/


From mwm at mired.org  Sun Jan 22 23:24:12 2012
From: mwm at mired.org (Mike Meyer)
Date: Sun, 22 Jan 2012 14:24:12 -0800
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
	<CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
Message-ID: <20120122142412.772fc323@bhuda.mired.org>

On Sun, 22 Jan 2012 12:04:48 +0000
Paul Moore <p.f.moore at gmail.com> wrote:

> On 22 January 2012 04:51, Steven D'Aprano <steve at pearwood.info> wrote:
> > Assignment as an expression feels unnatural to me:
> >
> > if spam() as x != 'ham':
> > ? ?frobulate(x)
> >
> > doesn't really correspond to any natural English order.
> 
> While in general I agree with you, there *is* a natural reading of this:
> 
> "if spam() isn't 'ham', frobulate it"
> 
> The fact that you have to choose a name is because computer languages
> have to be more explicit than natural languages, and can't deal with
> the implicit referent involved in the English usage of "it".

Others have pointed out that they can. My favorite example is Paul
Graham's anaphoric macros, which bind the evaluated expression to "it"
and then call the associated blocks.

Two things of note in his discussion
(http://dunsmor.com/lisp/onlisp/onlisp_18.html). First, he provides
versions for the other conditional execution statements. Second, the
one that is *most* used - he claims even more often than the regular
version - is for while.

      <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From steve at pearwood.info  Sun Jan 22 23:58:26 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 23 Jan 2012 09:58:26 +1100
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
	<CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
Message-ID: <20120122225826.GC4148@ando>

On Sun, Jan 22, 2012 at 12:04:48PM +0000, Paul Moore wrote:
> On 22 January 2012 04:51, Steven D'Aprano <steve at pearwood.info> wrote:
> > Assignment as an expression feels unnatural to me:
> >
> > if spam() as x != 'ham':
> > ? ?frobulate(x)
> >
> > doesn't really correspond to any natural English order.
> 
> While in general I agree with you, there *is* a natural reading of this:
> 
> "if spam() isn't 'ham', frobulate it"

That's not quite the same, because there is an implicit assignment, 
which is ambiguous. Are you frobulating spam(), or 'ham'? Without having 
domain-specific knowledge, guessing is risky. E.g. this is easy to 
interpret:

If the cat is on the mat, feed it.

since we know that cats can eat but mats don't. But this is not:

If the cat is on the mat, put it in the box.

So I stand by my claim: there is no natural English analog of an 
explicit assignment in the middle of the clause.

 
> The fact that you have to choose a name is because computer languages
> have to be more explicit than natural languages, and can't deal with
> the implicit referent involved in the English usage of "it".

Actually they can. Hypertalk had at least two such implicit variables, 
"it" and "the result", with different rules for when each were set. I'm 
sure other languages have done similar.

But really, it's not a great idea. The possibility of ambuiguity and 
confusion is too great. And what happens when you have two or more such 
variables? And in practice, you end up assigning "it" to a named 
variable anyway, otherwise it will be overwritten before you get around 
to using it.

The CPython interactive interpreter already has one implicit variable, _ 
which holds the result of the previous command. I don't think Python the 
language should follow.


-- 
Steven


From mwm at mired.org  Sun Jan 22 23:59:51 2012
From: mwm at mired.org (Mike Meyer)
Date: Sun, 22 Jan 2012 14:59:51 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CAMZYqRSHWpvVu4M=zPkU-zU-CKP12y-82G26VQPpAmRWqDQzFA@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CAMZYqRSHWpvVu4M=zPkU-zU-CKP12y-82G26VQPpAmRWqDQzFA@mail.gmail.com>
Message-ID: <20120122145951.16abc641@bhuda.mired.org>

On Sun, 22 Jan 2012 13:52:52 -0800
Chris Rebert <pyideas at rebertia.com> wrote:
> On Sun, Jan 22, 2012 at 1:30 PM, Mike Meyer <mwm at mired.org> wrote:
> > On Sat, 21 Jan 2012 11:46:19 +0000
> > Paul Moore <p.f.moore at gmail.com> wrote:
> >> On 21 January 2012 07:47, Steven D'Aprano <steve at pearwood.info> wrote:
> <snip>
> >> I'd be -1 on a retry keyword that worked any way other than this
> >> (simply because this is "clearly" the most obvious interpretation).
> >> Whether a retry statement is worth having at all, though, is something
> >> I'm not sure of - I'd like to see some real-world use cases first.
> >> I've never encountered the need for it myself. And I can't honestly
> >> imagine why the while True...try...break" idiom would ever not be
> >> sufficient.
> >
> > I saw a number of request for "real world uses cases". I thought I
> > covered that in the OP. This ideas was *prompted* by a real world use
> > case where we wanted to wrap an exception in our own private exception
> > before handling it. Because the code that would raise the exception
> > - other exceptions missing the attributes we added to ours - was the
> > code we want to run after handling them, we're left with LBYL or DRY
> > or wrapping a loop around the code when it wasn't really a loop.
> 
> The thing is, until this message, you never described any concrete
> details about your specific use-case, other than that you had one.
> Now you have, but in prose (which is regrettably imprecise) rather
> than code; I'm still not entirely sure what your use case looks like.

I can't give you the actual code (without having to deal with lawyers)
because it comes from a client's proprietary product. I could make up
code, but that's no better than the pseudo-code I gave in the OP.

> As best I can parse it, a nested try-except sounds like it would work.

I don't see how. Here's the pseudo-code example from the OP that most
resembles the code we finally committed, doing LBYL instead of EAFP:

    if not isinstance(x, my_class):
       x = fixup(x)
    mangle(x)
    other_stuff(x)

That doesn't show you where the exception would happen though, so here
(also from the OP) is the version that violates DRY:

    try:
        mangle(x)
    except:
        x = fixup(x)
	mangle(x)
    other_stuff(x)

The only thing the prose deception added to the above is that x is
known to be an exception, and we wanted to make sure it had an
appropriate attribute. The fact that it's an exception isn't really
relevant, but that we needed it to have a specific attribute means
those two might be better written as:

    if not isinstance(x, my_class):
       x = my_class(fixup(x))
    x.mangle()
    other_stuff(x)

and

    try:
        x.mangle()
    except:
        x = my_class(fixup(x))
	x.mangle()
    other_stuff(x)

If you've got a nested try/except solution that does EAFP and DRY, I'd
be interested in seeing it. The OP included some loop-based variants
as well.

> People like use-cases because they can reveal motivations or
> subtleties that often turn out to be significant, and they can more
> easily critique them than completely abstract/general cases.

Which is why I provided the best I could under the circumstances and
why I'm one of the first to ask for them when they aren't present.

	<mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From jeanpierreda at gmail.com  Mon Jan 23 00:15:49 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 22 Jan 2012 18:15:49 -0500
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <20120122225826.GC4148@ando>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
	<CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
	<20120122225826.GC4148@ando>
Message-ID: <CABicbJJS=jE_XV2TsJ2mhD9L0TRSGXVhRfY51f5P1k+q5ZbB1A@mail.gmail.com>

On Sun, Jan 22, 2012 at 5:58 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> So I stand by my claim: there is no natural English analog of an
> explicit assignment in the middle of the clause.

There isn't a natural English analogue of explicit assignment at all.
English doesn't have variables or anything like them. It has names,
but these names cannot be redefined -- try telling someone "Bill is my
cat", followed by, "Bill is my dad", and they'll give you a queer
look. And then they'll interpret this as two different Bills, rather
than just one that was "redefined" to mean something else, and if you
refer to Bill again they will ask "which one?"

There are unnatural phrases that work like assignment, and those same
phrases can be used unnaturally to refer to assignment within an "if"
as well. "Let the letter X refer to Y", "now let the letter X refer to
Z", "if foo, which the letter X will now refer to, is a piece of ham,
..." But that's kind of silly.

More generally, I'm not convinced programming has to fit into
"natural" English. It should be expressible, yes, but it doesn't have
to be idiomatic English. "Programming is an unnatural act. " (Perlis)

(Perhaps it must be idiomatic in Dutch?)

-- Devin


From tjreedy at udel.edu  Mon Jan 23 00:46:14 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 22 Jan 2012 18:46:14 -0500
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B95C1.7020402@pearwood.info>
	<CACac1F8wwMNO2BGn93VEt6erc2kieqX8orbzQBNn-6gt+2uPow@mail.gmail.com>
Message-ID: <jfi747$42b$1@dough.gmane.org>

On 1/22/2012 7:04 AM, Paul Moore wrote:
> On 22 January 2012 04:51, Steven D'Aprano<steve at pearwood.info>  wrote:
>> Assignment as an expression feels unnatural to me:
>>
>> if spam() as x != 'ham':
>>     frobulate(x)
>>
>> doesn't really correspond to any natural English order.

The best I can come up with is "if x, which is the result of calling 
'spam', is not equal to 'ham', then frobulate x". But I am not going to 
call that 'natural' speech. More normal would be "Get the result of 
spam(). If it it not 'ham', then frobulate it." As others noted, the 
pronoun 'it' is a substitute for local names. But doing that for more 
than one object or value gets awkward. "Get ham() and spam(). If the 
first is less than the second, multiply the first by the second plus 3." 
"Former' and 'latter' are another pair of standardized pronoun-like 
names. The math device  of temporary names is really handy.

-- 
Terry Jan Reedy



From jimjjewett at gmail.com  Mon Jan 23 02:44:38 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 22 Jan 2012 20:44:38 -0500
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <4F1B8B1C.8050106@eikehein.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B8B1C.8050106@eikehein.com>
Message-ID: <CA+OGgf6C4HTnhn4hjCGE4+dHoRtqyN=A4UNBnWJnyfdxwZXtdQ@mail.gmail.com>

On Sat, Jan 21, 2012 at 11:05 PM, Eike Hein <sho at eikehein.com> wrote:
> On 1/22/2012 4:48 AM, Nick Coghlan wrote:
>> Simple embedded assignment, though, only works when the
>> predicate is just "bool" - as soon as the condition differs from the
>> value you want to access, you need to revert to the existing idiom
>> *anyway*.

> Actually, just to be clear, in the use case I had in mind
> spam() is returning a numeric value. That satisfies a truth
> test and is still useful in other ways (but I assume you
> are actually on the same page, hence the quotation marks
> around bool).

Perhaps.  Does bool(spam()) produce the right answer?  Or did you need
to work with 0, or ignore -1?

> Another, possibly better way is to modify locals(), i.e.:

Well, not locals directly, but the following was inspired by the
quantification thread.

You still need to declare the Var before the test expression, but the
declaration can be just a placeholder.

    x=Var()
    ...
    if Var(expr):

-jJ
-------------- next part --------------
class Var:
    """Var proxies another value; it allows assignment expressions

    Written by JimJJewett, inspired by Paul Moore's
    http://mail.python.org/pipermail/python-ideas/2012-January/013417.html
    
    >>> x=Var()
    >>> if x:
    ...     print("True", x)
    ... else:
    ...     print("False", x)
    False Var(None)
    
    >>> if x(5):
    ...     print("True", x.value)
    ... else:
    ...     print("False", x.value)
    True 5
    
    >>> if x({}):
    ...     print("True", x.value)
    ... else:
    ...     print("False", x.value)
    False {}
    
    """
    def __init__(self, value=None): self(value)
    def __call__(self, value): self.value=value; return self
    def __bool__(self): return bool(self.value)
    def __repr__(self): return "Var(%r)" % (self.value,)
    
def _test():
    import doctest
    doctest.testmod()

if __name__ == "__main__":
    _test()


From ncoghlan at gmail.com  Mon Jan 23 03:18:37 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 23 Jan 2012 12:18:37 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120122133014.5e920a43@bhuda.mired.org>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
Message-ID: <CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>

On Mon, Jan 23, 2012 at 7:30 AM, Mike Meyer <mwm at mired.org> wrote:
> or wrapping a loop around the code when it wasn't really a loop.

You want to do the same thing more than once: that's a loop.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From stephen at xemacs.org  Mon Jan 23 05:34:44 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 23 Jan 2012 13:34:44 +0900
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
Message-ID: <871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>

Nick Coghlan writes:
 > On Mon, Jan 23, 2012 at 7:30 AM, Mike Meyer <mwm at mired.org> wrote:
 > > or wrapping a loop around the code when it wasn't really a loop.
 > 
 > You want to do the same thing more than once: that's a loop.

That's a question of point of view.  If "thing" is thought of as a
"try" (an operation that might fail), yes, he wants to do the same
thing a nondeterministic number of times, and in general, more than
once: it's a loop.  If "thing" is thought of as a "block" (which
isn't a try), then the "things" done with and without exception are
different: it's not a loop, it's a conditional.

Looking at his example "superficially" (I intend no deprecation by
that word, just a point of view in looking at the code), I have some
sympathy for Mike's claim that "it's not a loop and it violates DRY."
I "know what he means" (and I bet you do, too!)  However, when I try
to define that, even informally, I arrive at the paragraph above, and
I end up coming down on the side that you can't consistently claim
that "it's not a loop" *and* claim that "it violates DRY", in some
deeper sense.



From ncoghlan at gmail.com  Mon Jan 23 06:07:01 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 23 Jan 2012 15:07:01 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7dim=KTBm0kKMnUVdQGzbMiACqsyejRQi+=_Y6J8-FnLg@mail.gmail.com>

On Mon, Jan 23, 2012 at 2:34 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Nick Coghlan writes:
> ?> On Mon, Jan 23, 2012 at 7:30 AM, Mike Meyer <mwm at mired.org> wrote:
> ?> > or wrapping a loop around the code when it wasn't really a loop.
> ?>
> ?> You want to do the same thing more than once: that's a loop.
>
> That's a question of point of view. ?If "thing" is thought of as a
> "try" (an operation that might fail), yes, he wants to do the same
> thing a nondeterministic number of times, and in general, more than
> once: it's a loop. ?If "thing" is thought of as a "block" (which
> isn't a try), then the "things" done with and without exception are
> different: it's not a loop, it's a conditional.
>
> Looking at his example "superficially" (I intend no deprecation by
> that word, just a point of view in looking at the code), I have some
> sympathy for Mike's claim that "it's not a loop and it violates DRY."
> I "know what he means" (and I bet you do, too!) ?However, when I try
> to define that, even informally, I arrive at the paragraph above, and
> I end up coming down on the side that you can't consistently claim
> that "it's not a loop" *and* claim that "it violates DRY", in some
> deeper sense.

Exactly - in a very real sense, "retry once" is just a special case of
a more general looping pattern: you execute the body either once or
twice, perhaps with some massaging of state between the two attempts.

One way to write it is:

    for attempt in (1, 2): # This iterable controls your number of attempts
        try:
            # Do whatever
        except ExpectedException:
            x = prevent_expected_exception(x)
            continue  # continue == retry
        break # break == success (just like a search loop)
    else:
        raise Exception("Operation failed (made %d attempts)" % attempt)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From mwm at mired.org  Mon Jan 23 19:26:06 2012
From: mwm at mired.org (Mike Meyer)
Date: Mon, 23 Jan 2012 10:26:06 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CADiSq7dim=KTBm0kKMnUVdQGzbMiACqsyejRQi+=_Y6J8-FnLg@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7dim=KTBm0kKMnUVdQGzbMiACqsyejRQi+=_Y6J8-FnLg@mail.gmail.com>
Message-ID: <20120123102606.3d4f5ff6@mikmeyer-vm-fedora>

On Mon, 23 Jan 2012 15:07:01 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Mon, Jan 23, 2012 at 2:34 PM, Stephen J. Turnbull
> <stephen at xemacs.org> wrote:
> > Nick Coghlan writes:
> > ?> On Mon, Jan 23, 2012 at 7:30 AM, Mike Meyer <mwm at mired.org>
> > wrote: > > or wrapping a loop around the code when it wasn't really
> > a loop. >
> > ?> You want to do the same thing more than once: that's a loop.
> >
> > That's a question of point of view. ?If "thing" is thought of as a
> > "try" (an operation that might fail), yes, he wants to do the same
> > thing a nondeterministic number of times, and in general, more than
> > once: it's a loop. ?If "thing" is thought of as a "block" (which
> > isn't a try), then the "things" done with and without exception are
> > different: it's not a loop, it's a conditional.
> >
> > Looking at his example "superficially" (I intend no deprecation by
> > that word, just a point of view in looking at the code), I have some
> > sympathy for Mike's claim that "it's not a loop and it violates
> > DRY." I "know what he means" (and I bet you do, too!) ?However,
> > when I try to define that, even informally, I arrive at the
> > paragraph above, and I end up coming down on the side that you
> > can't consistently claim that "it's not a loop" *and* claim that
> > "it violates DRY", in some deeper sense.
> 
> Exactly - in a very real sense, "retry once" is just a special case of
> a more general looping pattern: you execute the body either once or
> twice, perhaps with some massaging of state between the two attempts.
> 
> One way to write it is:
> 
>     for attempt in (1, 2): # This iterable controls your number of
> attempts try:

I think that highlights why it's not really a loop. While "retry once"
is a general case of a looping pattern, it's *also* just a special
case of a more general exception handling pattern. Because it fits
under both special cases, it can be wrapped in a loop.

But *any* sequence of instructions can be wrapped in a loop. All you
need is a program counter (python is missing the tools to express it
really cleanly, so I'm pretending):

for pc in range(sys.maxsize):
    switch pc:
        0: first_block
	1: second_block
	2, 3: third_block
        4, 6: fourth_block
	5: fifth_block
	7: sixth_block
	   if not done:
	       pc -= 1
	else: break
    pc += 1


So what of this is a real loop? The third block is in a two-count
loop. The sixth block is in a while not done loop. The fourth block
corresponds to the case here - you want to rerun one bit of code with
another in the middle. You really don't want a general-purpose
loop. So having to write one doesn't express the intent correctly.

Of course, you can write a general-purpose loop this way. And there
are probably cases where such a loop is the intent. Which lowers the
value of a "retry" statement.

	<mike


From ron3200 at gmail.com  Mon Jan 23 20:06:52 2012
From: ron3200 at gmail.com (Ron Adam)
Date: Mon, 23 Jan 2012 13:06:52 -0600
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1327345612.12382.51.camel@Gutsy>

On Mon, 2012-01-23 at 13:34 +0900, Stephen J. Turnbull wrote:
> Nick Coghlan writes:
>  > On Mon, Jan 23, 2012 at 7:30 AM, Mike Meyer <mwm at mired.org> wrote:
>  > > or wrapping a loop around the code when it wasn't really a loop.
>  > 
>  > You want to do the same thing more than once: that's a loop.
> 
> That's a question of point of view.  If "thing" is thought of as a
> "try" (an operation that might fail), yes, he wants to do the same
> thing a nondeterministic number of times, and in general, more than
> once: it's a loop.  If "thing" is thought of as a "block" (which
> isn't a try), then the "things" done with and without exception are
> different: it's not a loop, it's a conditional.
> 
> Looking at his example "superficially" (I intend no deprecation by
> that word, just a point of view in looking at the code), I have some
> sympathy for Mike's claim that "it's not a loop and it violates DRY."
> I "know what he means" (and I bet you do, too!)  However, when I try
> to define that, even informally, I arrive at the paragraph above, and
> I end up coming down on the side that you can't consistently claim
> that "it's not a loop" *and* claim that "it violates DRY", in some
> deeper sense.

You also have to consider the size and scope of the block.  Anything
more than one or two simple instructions will be difficult to achieve in
a clean way.  In the case of only one or to instruction, a try block
works just fine.

For larger scopes, I think it would require some form of "UNDO_BLOCK"
opcode, so that the visited frame states can be restored before a retry
attempt is done.  It would break if any c code functions visits any
frame that is outside the frames the byte code is executing in. I'd also
be concerned about memory usage and speed, because it could lead to some
very inefficient routines.

Cheers,
   Ron





From steve at pearwood.info  Tue Jan 24 01:00:53 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 24 Jan 2012 11:00:53 +1100
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20120124000053.GC6693@ando>

On Mon, Jan 23, 2012 at 01:34:44PM +0900, Stephen J. Turnbull wrote:
> Nick Coghlan writes:
>  > On Mon, Jan 23, 2012 at 7:30 AM, Mike Meyer <mwm at mired.org> wrote:
>  > > or wrapping a loop around the code when it wasn't really a loop.
>  > 
>  > You want to do the same thing more than once: that's a loop.
> 
> That's a question of point of view.  If "thing" is thought of as a
> "try" (an operation that might fail), yes, he wants to do the same
> thing a nondeterministic number of times, and in general, more than
> once: it's a loop.  If "thing" is thought of as a "block" (which
> isn't a try), then the "things" done with and without exception are
> different: it's not a loop, it's a conditional.

I'm afraid I don't understand this. What's a block (that isn't a try) 
here, and how does it differ from a thing, and what makes it a 
conditional?

If all you are saying is that the caller may want to abstract out
"repeat this thing until it succeeds (or fails permanently)" into a
single operation without explicitly writing a for or while loop, that's
what functions are for. map(func, seq) doesn't cease to be a loop just
because you don't write it as an explicit loop.

If you mean something different from this, I have no idea what you mean.

 
> Looking at his example "superficially" (I intend no deprecation by
> that word, just a point of view in looking at the code), I have some
> sympathy for Mike's claim that "it's not a loop and it violates DRY."
> I "know what he means" (and I bet you do, too!)

I don't. To me, "retry this thing repeatedly" is fundamentally a loop. 


> However, when I try
> to define that, even informally, I arrive at the paragraph above, and
> I end up coming down on the side that you can't consistently claim
> that "it's not a loop" *and* claim that "it violates DRY", in some
> deeper sense.

If it violates DRY, then by definition you must be repeating yourself. 
If you repeat yourself, then the obvious way to avoid repeating yourself 
is to place the repeated code inside a loop. Why is this a problem?

What is especially confusing is that the proposed syntax is *defined* as 
looping back to the start of the try block, like a GOTO. If you draw the 
program flowchart of the construct, it loops backwards. Mike even stated 
that his proposal was exactly identical in behaviour to a try inside a 
while True loop. If he wants to argue that saving one indent level is so 
important that it's worth new syntax, that's one thing, but I am 
perplexed at claims that something which is identical to a loop isn't a 
loop.


-- 
Steven



From mwm at mired.org  Tue Jan 24 02:06:33 2012
From: mwm at mired.org (Mike Meyer)
Date: Mon, 23 Jan 2012 17:06:33 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120124000053.GC6693@ando>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
Message-ID: <20120123170633.1bfdbf11@mikmeyer-vm-fedora>

On Tue, 24 Jan 2012 11:00:53 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> I don't. To me, "retry this thing repeatedly" is fundamentally a loop. 

What's being abstracted out isn't "retry this thing repeatedly".  it's
"I want to retry this thing after tweaking things if it fails." In
particular, different ways of failure might require different tweaks
before the retry, second failures would be handled differently from
first failures, etc.

      <mike


From jimjjewett at gmail.com  Tue Jan 24 02:13:29 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 23 Jan 2012 20:13:29 -0500
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120123170633.1bfdbf11@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp> <20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
Message-ID: <CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>

On Mon, Jan 23, 2012 at 8:06 PM, Mike Meyer <mwm at mired.org> wrote:
> On Tue, 24 Jan 2012 11:00:53 +1100
> Steven D'Aprano <steve at pearwood.info> wrote:
>> I don't. To me, "retry this thing repeatedly" is fundamentally a loop.

> What's being abstracted out isn't "retry this thing repeatedly". ?it's
> "I want to retry this thing after tweaking things if it fails." In
> particular, different ways of failure might require different tweaks
> before the retry, second failures would be handled differently from
> first failures, etc.

So it doesn't seem like a loop because you hope to do it only once?

Or because you're thinking of the thing-retried as the candidate loop,
when that is just a constant function, and actual the loop is a loop
over things-to-try-first (a loop over functions, rather than data)?

-jJ


From ncoghlan at gmail.com  Tue Jan 24 02:20:18 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Jan 2012 11:20:18 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120123170633.1bfdbf11@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
Message-ID: <CADiSq7fUbHW31P6NQgPcsJ+=cQUuHruCCo4A1daMzNeb6sQAbw@mail.gmail.com>

On Tue, Jan 24, 2012 at 11:06 AM, Mike Meyer <mwm at mired.org> wrote:
> On Tue, 24 Jan 2012 11:00:53 +1100
> Steven D'Aprano <steve at pearwood.info> wrote:
>> I don't. To me, "retry this thing repeatedly" is fundamentally a loop.
>
> What's being abstracted out isn't "retry this thing repeatedly". ?it's
> "I want to retry this thing after tweaking things if it fails." In
> particular, different ways of failure might require different tweaks
> before the retry, second failures would be handled differently from
> first failures, etc.

But that's just normal loop-and-a-half behaviour, where the first half
of the loop is consistent, but the second half depends on the results
of the first half (including whether or not an exception is thrown).

    while attempts_remaining():
        try:
            # attempt consistent operation
        except ExpectedException:
            # set up for next attempt based on result of current attempt
            # even the list of expected exceptions can be made dynamic!
        else:
            break # success!
    else:
        # All attempts failed!

I'm not sure what it is about having an upper limit of 2 iterations,
or having the loop exit criteria be "didn't throw an exception" rather
than an ordinary conditional expression, that makes you feel like this
construct isn't just an ordinary loop-and-a-half (and best thought
about that way).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From stephen at xemacs.org  Tue Jan 24 04:18:05 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 24 Jan 2012 12:18:05 +0900
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
Message-ID: <87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>

Jim Jewett writes:

 > So it doesn't seem like a loop because you hope to do it only once?

With s/hope/expect/, that hits the nail on the head.

I don't think syntax can express that cleanly, but I can't help
thinking it's of good thing if somebody like Mike tries to find a way.
He might succeed!


From ncoghlan at gmail.com  Tue Jan 24 04:56:38 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Jan 2012 13:56:38 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7cAcaAHDMMW+K3S07BgccusP3Jzzhwiit6AZj5jj0rdMA@mail.gmail.com>

On Tue, Jan 24, 2012 at 1:18 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> I don't think syntax can express that cleanly, but I can't help
> thinking it's of good thing if somebody like Mike tries to find a way.
> He might succeed!

Special casing "looping with at most two iterations" and "looping
where the body is a single try statement" both seem like very poor
ideas.

OK, so some people apparently take issue with having to map "retry" to
"loop", but how does that even come close to justifying making
*everyone* learn a third looping construct?

We can't even get consensus that PEP 315's generalised while loops
(which allow you to write loop-and-a-half constructs without using
break) would be a net win for the language over the existing idiom.

I'll note that under PEP 315, the problem discussed in this thread
could be handled as:

    do ... while retry:  # '...' stands in for the suite below
        retry = False
        try:
            # attempt consistent operation
        except ExpectedException:
            # set up for next attempt based on result of current attempt
            # even the list of expected exceptions can be made dynamic!
            retry = True

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From mwm at mired.org  Tue Jan 24 05:44:27 2012
From: mwm at mired.org (Mike Meyer)
Date: Mon, 23 Jan 2012 20:44:27 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>



"Stephen J. Turnbull" <stephen at xemacs.org> wrote:

>Jim Jewett writes:
>
> > So it doesn't seem like a loop because you hope to do it only once?
>
>With s/hope/expect/, that hits the nail on the head.
>
>I don't think syntax can express that cleanly, but I can't help
>thinking it's of good thing if somebody like Mike tries to find a way.

Right. Doing it more than once is an exceptional case.  And is more exceptional every time through.

That using an LBYL idiom, or simply repeating yourself, means you don't need a loop says to me that it isn't really a loop.  Which is why using loop to write it feels wrong. 

That I translated into a loop was because the only tools Python has for this is a loop.  That no more makes it a loop than map being a function call makes it not a loop. 

I'd say that using retry to write what really is a loop (ie - you can't know a maximum number of iterations at compile time) would constitute abuse. A very attractive abuse, at that. Which certainly counts against out.

>He might succeed!

Or someone else will figure it out after I've correctly described the problem.
-- 
Sent from my Android tablet with K-9 Mail. Please excuse my brevity.


From ncoghlan at gmail.com  Tue Jan 24 06:18:00 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Jan 2012 15:18:00 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
Message-ID: <CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>

On Tue, Jan 24, 2012 at 2:44 PM, Mike Meyer <mwm at mired.org> wrote:
> That using an LBYL idiom, or simply repeating yourself, means you don't need a loop says to me that it isn't really a loop. ?Which is why using loop to write it feels wrong.

A retry is just a specific kind of loop that executes 1 or 2 times (or
perhaps more if you're allowed to trigger the retry more than once).
You don't actually need loops in general, since you can use recursion
or repetition instead, so saying "look, i can rewrite it without the
loop, so it's not really a loop!" doesn't mean all that much in an
objective sense.

The argument that it might be worth having dedicated syntax for a loop
that runs 1 or 2 times is rather unconvincing when we don't even have
dedicated syntax for a loop that runs 1 or more times (see PEP 315).

We've survived this long with two variants of a loop that runs 0 or
more times and using break as appropriate to handle all the other
cases (e.g. while+break provides loop-and-a-half semantics and I've
written for loops with an unconditional break at the end to get "0 or
1" iteration). Making the case that we need another looping construct
is a fairly tall order (even PEP 315 doesn't suggest a completely new
construct - it only proposes a generalisation of the existing while
loops).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From mwm at mired.org  Tue Jan 24 06:47:21 2012
From: mwm at mired.org (Mike Meyer)
Date: Mon, 23 Jan 2012 21:47:21 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
Message-ID: <669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>



Nick Coghlan <ncoghlan at gmail.com> wrote:

>On Tue, Jan 24, 2012 at 2:44 PM, Mike Meyer <mwm at mired.org> wrote:
>> That using an LBYL idiom, or simply repeating yourself, means you
>don't need a loop says to me that it isn't really a loop. ?Which is why
>using loop to write it feels wrong.
>The argument that it might be worth having dedicated syntax for a loop
>that runs 1 or 2 times is rather unconvincing when we don't even have
>dedicated syntax for a loop that runs 1 or more times (see PEP 315).

The argument isn't that we need a new syntax for a small set of loops, it's that the only ways to implement retrying after an exception leave a code smell.
-- 
Sent from my Android tablet with K-9 Mail. Please excuse my brevity.


From ncoghlan at gmail.com  Tue Jan 24 08:00:28 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Jan 2012 17:00:28 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
Message-ID: <CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>

On Tue, Jan 24, 2012 at 3:47 PM, Mike Meyer <mwm at mired.org> wrote:
> The argument isn't that we need a new syntax for a small set of loops, it's that the only ways to implement retrying after an exception leave a code smell.

Uh, saying "retrying is fundamentally a looping operation" is not a
code smell. How do you plan to implement retry if not as a looping
construct under the hood? No matter how many times you assert
otherwise, "go back to this earlier point in the code and resume
execution from there" is pretty much the *definition* of a loop.

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From cmjohnson.mailinglist at gmail.com  Tue Jan 24 08:19:08 2012
From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson)
Date: Mon, 23 Jan 2012 21:19:08 -1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
Message-ID: <8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>


On Jan 23, 2012, at 9:00 PM, Nick Coghlan wrote:

> On Tue, Jan 24, 2012 at 3:47 PM, Mike Meyer <mwm at mired.org> wrote:
>> The argument isn't that we need a new syntax for a small set of loops, it's that the only ways to implement retrying after an exception leave a code smell.
> 
> Uh, saying "retrying is fundamentally a looping operation" is not a
> code smell. 

I do think there's something code smelly about a retry--if it didn't work the first time, why should it work the second time after you give it a whack? Either whacking is good and you should do it in advance or it's bad and you should do something more sophisticated--but I don't see how creating language level support for retrying would remove the smell. Accessing deeply.nested.methods.and_.properties is a code smell too, even though it has language level support from Python. (You could imagine it being otherwise, if Python insisted that each attribute access get its own line, but that wouldn't remove the smell either.)

The whole point of a "smell" is that it's not directly bad, but it's a sign that maybe you were thinking of something wrong at a different level, so it's time to re-architect a little.

From wuwei23 at gmail.com  Tue Jan 24 08:33:16 2012
From: wuwei23 at gmail.com (alex23)
Date: Mon, 23 Jan 2012 23:33:16 -0800 (PST)
Subject: [Python-ideas] Combining test and assignment
In-Reply-To: <4F1B8E10.9050609@eikehein.com>
References: <4F1B3AD8.6030402@eikehein.com>
	<CADiSq7ffj2EwfQKX0PEUsP88+c81aPpkr1BV85zrHvEwPVnXYg@mail.gmail.com>
	<4F1B8B1C.8050106@eikehein.com>
	<CAMZYqRRjj=JXjkOSQNkzg8S6cG_rUJjuOge=p=D6LQKRkcfuyA@mail.gmail.com>
	<4F1B8E10.9050609@eikehein.com>
Message-ID: <4c498c9b-9287-4eb6-ace0-3a761bc93701@pk8g2000pbb.googlegroups.com>

On Jan 22, 2:18?pm, Eike Hein <s... at eikehein.com> wrote:
> Can anyone think of another way to do an assignment
> and a truth test in the same expression with today's
> tools?

I've seen something similar to this used before:

    class Holder(object):
        def __call__(self, value):
            self.value = value
            return self

        def __eq__(self, val):
            return self.value == val

        __req__ = __eq__


    from random import choice
    some_func = lambda: choice(['a','b','c'])

    holder = Holder()
    if holder(some_func()) == 'a':
        print 'A'
    elif holder == 'b':
        print 'B'
    elif holder == 'c':
        print 'C'

As you can see, it still requires a value to be defined before the
conditional, though :)


From ncoghlan at gmail.com  Tue Jan 24 08:34:37 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Jan 2012 17:34:37 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
Message-ID: <CADiSq7eHsQQtVgp6By6GpQafG02N4PoZx3Rb9axs=HEsemOk+w@mail.gmail.com>

On Tue, Jan 24, 2012 at 5:19 PM, Carl M. Johnson
<cmjohnson.mailinglist at gmail.com> wrote:
> I don't see how creating language level support for retrying would remove the smell.

I didn't even think of that aspect - very good point :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From p.f.moore at gmail.com  Tue Jan 24 10:47:38 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 24 Jan 2012 09:47:38 +0000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CACac1F_M6B+H0LmEVCk8UWsRCXUWj0s_nwT+SJDGf=qw-Aed=w@mail.gmail.com>

On 24 January 2012 03:18, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Jim Jewett writes:
>
> ?> So it doesn't seem like a loop because you hope to do it only once?
>
> With s/hope/expect/, that hits the nail on the head.
>
> I don't think syntax can express that cleanly, but I can't help
> thinking it's of good thing if somebody like Mike tries to find a way.
> He might succeed!

If Python was like Haskell or Lisp, where you can define your own
control structures, and this was a proposal to add a new control
structure to the stdlib, I'd be fine with it. It *is* a somewhat
unusual case, and from some people's viewpoint, certainly, the
construct is not really a loop. So having "from exception_handling
import retry" would be plausible.

But Python doesn't have user defined control structures, by design,
and indeed takes a strongly minimalist position on adding control
structures - we don't even have do...while (yet), as Nick has pointed
out. So in that context, I believe that this case is not sufficiently
special, and the "not a loop" viewpoint is not sufficiently universal,
to warrant a language change.

Certainly, I wouldn't argue that this should be part of the language
before do...while (which is more generally useful) gets in.

Paul.


From julien at tayon.net  Tue Jan 24 11:19:08 2012
From: julien at tayon.net (julien tayon)
Date: Tue, 24 Jan 2012 11:19:08 +0100
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
Message-ID: <CAFpLVkz_bUw+7MfZQMOD2m=APfnD-y=iv2j+gR_k8_J2AbAYug@mail.gmail.com>

2012/1/24 Carl M. Johnson <cmjohnson.mailinglist at gmail.com>:
>
> On Jan 23, 2012, at 9:00 PM, Nick Coghlan wrote:
>
>> On Tue, Jan 24, 2012 at 3:47 PM, Mike Meyer <mwm at mired.org> wrote:
>>> The argument isn't that we need a new syntax for a small set of loops, it's that the only ways to implement retrying after an exception leave a code smell.
>>
>> Uh, saying "retrying is fundamentally a looping operation" is not a
>> code smell.
>
> I do think there's something code smelly about a retry--if it didn't work the first time, why should it work the second time after you give it a whack? Either whacking is good and you should do it in advance or it's bad and you should do something more sophisticated--but I don't see how creating language level support for retrying would remove the smell. Accessing deeply.nested.methods.and_.properties is a code smell too, even though it has language level support from Python. (You could imagine it being otherwise, if Python insisted that each attribute access get its own line, but that wouldn't remove the smell either.)

When could you need to  retry a second time in a legitimate way?

Mhhh, because you are trying to acquire a DHCP lease, you made a
DHCPDISCOVER, and you have to retry on the same udp connexion you
opened as long as you have no response, or you dont dont have a
timeout, therefore  you need to count the retry/time elapsed.

Following the metaphor of the DHCP lease, where you need to retry with
the same context than the try, you'd then claim that you may also need
DHCPLEASE handling (a low level auth in DHCP) and retry all the way
down from the DHCPDISCOVER if your IP is not accepted after a DHCPNAK.
And then you discover that these problems of error handling with
keeping context would best be handled in an evenemential fashion
rather than with adding a weired hack in sequential programming.

This case, and others are best treated with twisted or tornado, but do
you really want to import tornado or twisted when you need a rough
finite state automata ?
(I don't have the answer)

-- 
Regards,
Jul


From steve at pearwood.info  Tue Jan 24 12:19:35 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 24 Jan 2012 22:19:35 +1100
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>	<4F1A6DA2.5020807@pearwood.info>	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>	<20120122133014.5e920a43@bhuda.mired.org>	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>	<20120124000053.GC6693@ando>	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
Message-ID: <4F1E93C7.4050703@pearwood.info>

Carl M. Johnson wrote:
> On Jan 23, 2012, at 9:00 PM, Nick Coghlan wrote:
> 
>> On Tue, Jan 24, 2012 at 3:47 PM, Mike Meyer <mwm at mired.org> wrote:
>>> The argument isn't that we need a new syntax for a small set of loops,
>>> it's that the only ways to implement retrying after an exception leave
>>> a code smell.
>> Uh, saying "retrying is fundamentally a looping operation" is not a code
>> smell.
> 
> I do think there's something code smelly about a retry--if it didn't work
> the first time, why should it work the second time after you give it a
> whack? Either whacking is good and you should do it in advance or it's bad
> and you should do something more sophisticated


"Whacking" is not necessarily bad or difficult, and is not necessarily a code 
smell. There are many common situations where "try again" is natural and 
expected. E.g. in response to a busy signal, you should wait a little while 
before retrying:


delay = 5  # Initial delay between attempts in seconds.
for _ in range(MAX_ATTEMPTS):
     try:
         response = urllib2.urlopen(url)
     except urllib2.HTTPError as e:
         if e.code == 503:  # Service Unavailable.
             time.sleep(delay)
             delay *= 2  # Exponential back-off.
         else:
             raise
     else:
         break



This could be written without the for-loop using a hypothetical retry 
statement, but it doesn't really gain us much:


delay = 2  # Initial delay between attempts in seconds.
count = 0
try:
     response = urllib2.urlopen(url)
except urllib2.HTTPError as e:
     if e.code == 503 and count < MAX_ATTEMPTS:  # Service Unavailable
         time.sleep(delay)
         delay *= 2  # Exponential back-off
         count += 1
         retry
     else:
         raise



Although you save one indent level, you don't save any lines of code, and it 
costs you the effort of handling the book-keeping that a for-loop would give 
you for free. I don't count this as a win.



-- 
Steven



From p.f.moore at gmail.com  Tue Jan 24 12:36:29 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 24 Jan 2012 11:36:29 +0000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <4F1E93C7.4050703@pearwood.info>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
	<4F1E93C7.4050703@pearwood.info>
Message-ID: <CACac1F9kK6Zppe8HvfaQSw5pPcm=SPmnssMGgVjdnzOaBnZ3GQ@mail.gmail.com>

On 24 January 2012 11:19, Steven D'Aprano <steve at pearwood.info> wrote:
> "Whacking" is not necessarily bad or difficult, and is not necessarily a
> code smell. There are many common situations where "try again" is natural
> and expected. E.g. in response to a busy signal, you should wait a little
> while before retrying:
[...]
> Although you save one indent level, you don't save any lines of code, and it
> costs you the effort of handling the book-keeping that a for-loop would give
> you for free. I don't count this as a win.

Having read that, I agree that the "retry" doesn't buy you much. But I
do think there's a pattern in there that it would be nice to be able
to abstract out, namely the exponential back-off. The pattern of "try
X, if Y happens, then do Z and wait, then retry" is probably common
enough in certain types of application that it would be nice to
encapsulate it in a library routine. (I haven't needed it myself,
admittedly...) I can't quite see how to do it, though - I thought
about clever uses of with statements, or abstract classes with defined
callback methods you could use, but couldn't find anything obvious.
(The big issue being that "Y" is that a particular exception is raised
in this case, but may be something else in general).

A construct that let end users abstract this type of pattern would
probably be a far bigger win than a retry statement. (And it may be
that it has the benefit of already existing, I just couldn't see it
:-))

Paul.


From ncoghlan at gmail.com  Tue Jan 24 15:06:23 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Jan 2012 00:06:23 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CACac1F9kK6Zppe8HvfaQSw5pPcm=SPmnssMGgVjdnzOaBnZ3GQ@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
	<4F1E93C7.4050703@pearwood.info>
	<CACac1F9kK6Zppe8HvfaQSw5pPcm=SPmnssMGgVjdnzOaBnZ3GQ@mail.gmail.com>
Message-ID: <CADiSq7fzP1E=6rVDGXh5wQN96Lh+ru2ECjGA-RDpsbKFuDEuyA@mail.gmail.com>

On Tue, Jan 24, 2012 at 9:36 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> A construct that let end users abstract this type of pattern would
> probably be a far bigger win than a retry statement. (And it may be
> that it has the benefit of already existing, I just couldn't see it
> :-))

You just need to move the pause inside the iterator:

    def backoff(attempts, first_delay, scale=2):
        delay = first_delay
        for attempt in range(1, attempts+1):
            yield attempt
            time.sleep(delay)
            delay *= 2

    for __ in backoff(MAX_ATTEMPTS, 5):
       try:
           response = urllib2.urlopen(url)
       except urllib2.HTTPError as e:
           if e.code == 503:  # Service Unavailable.
               continue
           raise
       break

You can also design smarter versions where the object yielded is
mutable, making it easy to pass state back into the iterator.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From jkbbwr at gmail.com  Tue Jan 24 16:21:19 2012
From: jkbbwr at gmail.com (Jakob Bowyer)
Date: Tue, 24 Jan 2012 15:21:19 +0000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CADiSq7fzP1E=6rVDGXh5wQN96Lh+ru2ECjGA-RDpsbKFuDEuyA@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
	<4F1E93C7.4050703@pearwood.info>
	<CACac1F9kK6Zppe8HvfaQSw5pPcm=SPmnssMGgVjdnzOaBnZ3GQ@mail.gmail.com>
	<CADiSq7fzP1E=6rVDGXh5wQN96Lh+ru2ECjGA-RDpsbKFuDEuyA@mail.gmail.com>
Message-ID: <D3EE8103-1462-4D66-9708-6104C94E1338@gmail.com>

Would this not be better expressed as a context manager?

with backoff(maxattempts, 5):
    # do stuff

Sent from my iPad

On 24 Jan 2012, at 14:06, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Tue, Jan 24, 2012 at 9:36 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>> A construct that let end users abstract this type of pattern would
>> probably be a far bigger win than a retry statement. (And it may be
>> that it has the benefit of already existing, I just couldn't see it
>> :-))
> 
> You just need to move the pause inside the iterator:
> 
>    def backoff(attempts, first_delay, scale=2):
>        delay = first_delay
>        for attempt in range(1, attempts+1):
>            yield attempt
>            time.sleep(delay)
>            delay *= 2
> 
>    for __ in backoff(MAX_ATTEMPTS, 5):
>       try:
>           response = urllib2.urlopen(url)
>       except urllib2.HTTPError as e:
>           if e.code == 503:  # Service Unavailable.
>               continue
>           raise
>       break
> 
> You can also design smarter versions where the object yielded is
> mutable, making it easy to pass state back into the iterator.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From mwm at mired.org  Tue Jan 24 18:19:50 2012
From: mwm at mired.org (Mike Meyer)
Date: Tue, 24 Jan 2012 09:19:50 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
Message-ID: <20120124091950.435c3519@mikmeyer-vm-fedora>

On Tue, 24 Jan 2012 17:00:28 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Tue, Jan 24, 2012 at 3:47 PM, Mike Meyer <mwm at mired.org> wrote:
> > The argument isn't that we need a new syntax for a small set of
> > loops, it's that the only ways to implement retrying after an
> > exception leave a code smell.
> Uh, saying "retrying is fundamentally a looping operation" is not a
> code smell.

No, the code smell is a loop that just gets run through once. I mean,
I can write any if statement as a while loop, but it's probably a bad
idea and a code smell.

     <mike


From pyideas at rebertia.com  Tue Jan 24 19:01:42 2012
From: pyideas at rebertia.com (Chris Rebert)
Date: Tue, 24 Jan 2012 10:01:42 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <D3EE8103-1462-4D66-9708-6104C94E1338@gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
	<4F1E93C7.4050703@pearwood.info>
	<CACac1F9kK6Zppe8HvfaQSw5pPcm=SPmnssMGgVjdnzOaBnZ3GQ@mail.gmail.com>
	<CADiSq7fzP1E=6rVDGXh5wQN96Lh+ru2ECjGA-RDpsbKFuDEuyA@mail.gmail.com>
	<D3EE8103-1462-4D66-9708-6104C94E1338@gmail.com>
Message-ID: <CAMZYqRQS55-5iyDQhsDOjbm_KTzHGqUfhg+=yABsgQ3CgFaASA@mail.gmail.com>

> On 24 Jan 2012, at 14:06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Tue, Jan 24, 2012 at 9:36 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>> A construct that let end users abstract this type of pattern would
>>> probably be a far bigger win than a retry statement. (And it may be
>>> that it has the benefit of already existing, I just couldn't see it
>>> :-))
>>
>> You just need to move the pause inside the iterator:
>>
>> ? ?def backoff(attempts, first_delay, scale=2):
>> ? ? ? ?delay = first_delay
>> ? ? ? ?for attempt in range(1, attempts+1):
>> ? ? ? ? ? ?yield attempt
>> ? ? ? ? ? ?time.sleep(delay)
>> ? ? ? ? ? ?delay *= 2
>>
>> ? ?for __ in backoff(MAX_ATTEMPTS, 5):
>> ? ? ? try:
>> ? ? ? ? ? response = urllib2.urlopen(url)
>> ? ? ? except urllib2.HTTPError as e:
>> ? ? ? ? ? if e.code == 503: ?# Service Unavailable.
>> ? ? ? ? ? ? ? continue
>> ? ? ? ? ? raise
>> ? ? ? break
On Tue, Jan 24, 2012 at 7:21 AM, Jakob Bowyer <jkbbwr at gmail.com> wrote:
> Would this not be better expressed as a context manager?
>
> with backoff(maxattempts, 5):
> ? ?# do stuff

It can't be. The `with` statement always executes its block exactly
once; the context manager(s) have no say in the matter (unless perhaps
you count raising an exception prior to the block's execution).

Cheers,
Chris


From mwm at mired.org  Tue Jan 24 19:09:34 2012
From: mwm at mired.org (Mike Meyer)
Date: Tue, 24 Jan 2012 10:09:34 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
Message-ID: <20120124100934.063344a7@mikmeyer-vm-fedora>

On Mon, 23 Jan 2012 21:19:08 -1000
"Carl M. Johnson" <cmjohnson.mailinglist at gmail.com> wrote:
> On Jan 23, 2012, at 9:00 PM, Nick Coghlan wrote:
> > On Tue, Jan 24, 2012 at 3:47 PM, Mike Meyer <mwm at mired.org> wrote:
> >> The argument isn't that we need a new syntax for a small set of
> >> loops, it's that the only ways to implement retrying after an
> >> exception leave a code smell.
> > Uh, saying "retrying is fundamentally a looping operation" is not a
> > code smell. 
> I do think there's something code smelly about a retry--if it didn't
> work the first time, why should it work the second time after you
> give it a whack? Either whacking is good and you should do it in
> advance

That's the LBYL way of expressing the code. You know, the one that
manages to express the "loop" without either repeating any code or
using an actual loop. Except applying the fix without checking to see
if it's needed is most likely a bug.

You might want to argue that LBYL isn't a code smell. I've fixed
enough bugs caused by it to disagree, but at this point it's a style
argument.

	<mike


From steve at pearwood.info  Tue Jan 24 19:37:07 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 25 Jan 2012 05:37:07 +1100
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120124091950.435c3519@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>	<4F1A6DA2.5020807@pearwood.info>	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>	<20120122133014.5e920a43@bhuda.mired.org>	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>	<20120124000053.GC6693@ando>	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
Message-ID: <4F1EFA53.50008@pearwood.info>

Mike Meyer wrote:
> On Tue, 24 Jan 2012 17:00:28 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
>> On Tue, Jan 24, 2012 at 3:47 PM, Mike Meyer <mwm at mired.org> wrote:
>>> The argument isn't that we need a new syntax for a small set of
>>> loops, it's that the only ways to implement retrying after an
>>> exception leave a code smell.
>> Uh, saying "retrying is fundamentally a looping operation" is not a
>> code smell.
> 
> No, the code smell is a loop that just gets run through once.

By this reasoning, "for i in range(n)" is a code smell, because n might happen 
to be 1.

You can't know that the loop will run once until you actually try. The point 
of the retry idiom is that it could run twice, thrice, four times, ... up to 
whatever limit you impose (if any!). Even if you expect that 999 times out of 
a thousand it will only run once, you write it in a loop because you want to 
cover the 1 remaining time where it will run multiple times.

That's not a code smell, it is the obvious way to write an operation that may 
need to be retried.


> I mean,
> I can write any if statement as a while loop, but it's probably a bad
> idea and a code smell.

There's no limit to the bizarre and obfuscatory code that a clever enough, or 
foolish enough, coder can write. But putting something that needs to run one 
OR MORE times inside a loop is neither bizarre nor obfuscatory. That's what 
loops are for.



-- 
Steven


From mwm at mired.org  Tue Jan 24 19:45:36 2012
From: mwm at mired.org (Mike Meyer)
Date: Tue, 24 Jan 2012 10:45:36 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <4F1EFA53.50008@pearwood.info>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
Message-ID: <20120124104536.788b1186@mikmeyer-vm-fedora>

On Wed, 25 Jan 2012 05:37:07 +1100
Steven D'Aprano <steve at pearwood.info> wrote:

> Mike Meyer wrote:
> > On Tue, 24 Jan 2012 17:00:28 +1000
> > Nick Coghlan <ncoghlan at gmail.com> wrote:
> > 
> >> On Tue, Jan 24, 2012 at 3:47 PM, Mike Meyer <mwm at mired.org> wrote:
> >>> The argument isn't that we need a new syntax for a small set of
> >>> loops, it's that the only ways to implement retrying after an
> >>> exception leave a code smell.
> >> Uh, saying "retrying is fundamentally a looping operation" is not a
> >> code smell.
> > 
> > No, the code smell is a loop that just gets run through once.
> 
> By this reasoning, "for i in range(n)" is a code smell, because n
> might happen to be 1.

Not quite, because n might also happen to *not* be 1. You could even
run it no times, if n were 0. Nothing wrong with any of that.

> You can't know that the loop will run once until you actually try.

Which is not a code smell. However, if you can tell by reading the
code that it will only run once (or never run), like this one:

     for i in range(1):

Then it's a code smell!

	 <mike


From ethan at stoneleaf.us  Tue Jan 24 20:16:47 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 24 Jan 2012 11:16:47 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120124104536.788b1186@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>	<4F1A6DA2.5020807@pearwood.info>	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>	<20120122133014.5e920a43@bhuda.mired.org>	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>	<20120124000053.GC6693@ando>	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>	<20120124091950.435c3519@mikmeyer-vm-fedora>	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
Message-ID: <4F1F039F.5040406@stoneleaf.us>

Mike Meyer wrote:
> On Wed, 25 Jan 2012 05:37:07 +1100
> Steven D'Aprano <steve at pearwood.info> wrote:
>> You can't know that the loop will run once until you actually try.
> 
> Which is not a code smell. However, if you can tell by reading the
> code that it will only run once (or never run), like this one:
> 
>      for i in range(1):
> 
> Then it's a code smell!

Absolutely.  But your retry may run through twice.  :)

~Ethan~


From ncoghlan at gmail.com  Wed Jan 25 01:04:37 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Jan 2012 10:04:37 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120124104536.788b1186@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
Message-ID: <CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>

On Wed, Jan 25, 2012 at 4:45 AM, Mike Meyer <mwm at mired.org> wrote:
> Which is not a code smell. However, if you can tell by reading the
> code that it will only run once (or never run), like this one:
>
> ? ? for i in range(1):
>
> Then it's a code smell!

I agree specifically in regards to range() with a literal argument,
but it really depends on the iterator. Using break to force a single
iteration can be nicer than calling next() and catching StopIteration.
For example, code like the following makes it easy to special case the
first entry in an iterator:

    walk_iter = os.walk(topdir)
    for dirpath, files, subdirs in walk_iter:
        # Special case the top level directory
        break
    else:
       raise RuntimeError("No dir entry for {!r}".format(topdir))
    for dirpath, files, subdirs in walk_iter:
        # Process the subdirectories

Python has very powerful looping constructs already - we don't need
more just because some folks have been trained to think that break and
continue are evil and haven't considered whether or not these
prejudices inherited from C and C++ are still relevant in Python. Like
early returns, break and continue are potentially dangerous in C and
C++ because having multiple exit points from a scope increases the
chance of leaking memory (or some other resource). By contrast,
garbage collection and context managers mean that making appropriate
use of early returns, break and continue is quite easy and safe in
Python (and often clearer than the alternatives that try to avoid
them).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From mwm at mired.org  Wed Jan 25 01:52:01 2012
From: mwm at mired.org (Mike Meyer)
Date: Tue, 24 Jan 2012 16:52:01 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
	<CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>
Message-ID: <20120124165201.1b91c620@mikmeyer-vm-fedora>

On Wed, 25 Jan 2012 10:04:37 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Jan 25, 2012 at 4:45 AM, Mike Meyer <mwm at mired.org> wrote:
> > Which is not a code smell. However, if you can tell by reading the
> > code that it will only run once (or never run), like this one:
> >
> > ? ? for i in range(1):
> >
> > Then it's a code smell!
> 
> I agree specifically in regards to range() with a literal argument,
> but it really depends on the iterator. Using break to force a single
> iteration can be nicer than calling next() and catching StopIteration.
> For example, code like the following makes it easy to special case the
> first entry in an iterator:
> 
>     walk_iter = os.walk(topdir)
>     for dirpath, files, subdirs in walk_iter:
>         # Special case the top level directory
>         break
>     else:
>        raise RuntimeError("No dir entry for {!r}".format(topdir))
>     for dirpath, files, subdirs in walk_iter:
>         # Process the subdirectories

And we've now gotten to the level of picking apart real examples. I
don't think a loop that never loops is a code smell because I think
break or continue are evil (the opposite, in fact; I'd like a
multi-level break/continue, but that's for another topic). I think
it's a code smell because there's a good chance it can be refactored
into straight-line code that would be better in a number of ways.

For instance, I would write your example like so:

    try:
        walk_iter = os.walk(topdir)
        dirpath, files, subdirs = next(wi)
        # Special case the top level directory
    except StopIteration:
        raise RuntimeError("No dir entry for {!r}".format(topdir))
    # other exception handling here, as appropriate.

    for dirpath, files, subdirs in walk_iter:
        # Process the subdirectories

Exactly what winds up in the try block will vary depending on
circumstances. Putting only the invocation of next in it would
duplicate your code. os.walk isn't documented as returning exceptions,
and ignores problems with listdir, so adding it seems to be a
nop. Handling the special case for the top level directory with this
try seems like a win, because all the exceptions from dealing with the
top level directory get grouped together. It might be useful to handle
exceptions from the subdir processing as well, but the choice is easy
to make with this version.

In any case, it's easier to change this version as needed to deal with
exceptions than the original version.

The other issue may be just me - I expect exiting a loop from the
bottom to be a normal flow path. So the for version reads to me like
raising RuntimeError is normal, not exceptional.

> Python has very powerful looping constructs already - we don't need
> more just because some folks have been trained to think that break and
> continue are evil and haven't considered whether or not these
> prejudices inherited from C and C++ are still relevant in Python.

Likewise, the try statement is very powerful. That break/continue may
or may not be evil is immaterial. The point is to make the try
statement more powerful.

So long as you incorrectly see this as "just another looping
construct", your conclusions will be flawed. Not necessarily wrong,
just flawed. It can also be used to fix LBYL and DRY code smells.

	<mike


From ncoghlan at gmail.com  Wed Jan 25 02:13:18 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Jan 2012 11:13:18 +1000
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120124165201.1b91c620@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
	<CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>
	<20120124165201.1b91c620@mikmeyer-vm-fedora>
Message-ID: <CADiSq7fpS+1uesn_XV_FBKEGvY9V61mRcSgNzXWHnDLOhxkYEg@mail.gmail.com>

On Wed, Jan 25, 2012 at 10:52 AM, Mike Meyer <mwm at mired.org> wrote:
> So long as you incorrectly see this as "just another looping
> construct", your conclusions will be flawed. Not necessarily wrong,
> just flawed. It can also be used to fix LBYL and DRY code smells.

It *is* just another looping construct: "retry" = "go back to the
start of the try block". It's really just proposing a weird way to
spell while+continue.

Exception handling and repeating a section of code are *not* the same
thing - it doesn't make sense to bundle the two into a single
mega-construct.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From mwm at mired.org  Wed Jan 25 02:19:43 2012
From: mwm at mired.org (Mike Meyer)
Date: Tue, 24 Jan 2012 17:19:43 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CADiSq7fpS+1uesn_XV_FBKEGvY9V61mRcSgNzXWHnDLOhxkYEg@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
	<CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>
	<20120124165201.1b91c620@mikmeyer-vm-fedora>
	<CADiSq7fpS+1uesn_XV_FBKEGvY9V61mRcSgNzXWHnDLOhxkYEg@mail.gmail.com>
Message-ID: <20120124171943.57860103@mikmeyer-vm-fedora>

On Wed, 25 Jan 2012 11:13:18 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> Exception handling and repeating a section of code are *not* the same
> thing - it doesn't make sense to bundle the two into a single
> mega-construct.

Exception handling and retrying the code that caused the exception are
part of the same concept. There is value in being able to express that
concept directly with a single construct.

	<mike


From stephen at xemacs.org  Wed Jan 25 02:28:06 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 25 Jan 2012 10:28:06 +0900
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120124165201.1b91c620@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
	<CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>
	<20120124165201.1b91c620@mikmeyer-vm-fedora>
Message-ID: <87ehuo4lll.fsf@uwakimon.sk.tsukuba.ac.jp>

Mike Meyer writes:

 > The point is to make the try statement more powerful.

Not on python-ideas, it isn't.  Here, the point is to make Python more
expressive (including clarity in "expressive").  That may mean making
some statements less expressive (in the extreme, eliminating them
entirely, as in the case of "print").

 > So long as you incorrectly see this as "just another looping
 > construct", your conclusions will be flawed.

That's unfair.  Nobody said "just".  If anything, you were trying in
earlier posts to maintain the opposite extreme ("not a looping
construct").


From vince.vinet at gmail.com  Wed Jan 25 03:53:49 2012
From: vince.vinet at gmail.com (Vince)
Date: Tue, 24 Jan 2012 21:53:49 -0500
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120124171943.57860103@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
	<CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>
	<20120124165201.1b91c620@mikmeyer-vm-fedora>
	<CADiSq7fpS+1uesn_XV_FBKEGvY9V61mRcSgNzXWHnDLOhxkYEg@mail.gmail.com>
	<20120124171943.57860103@mikmeyer-vm-fedora>
Message-ID: <CAJ=7gQLH58WGkNokdOgYC7=LLcJ7Uw4B7KgvN_XCFzve+sdP7w@mail.gmail.com>

On Tue, Jan 24, 2012 at 8:19 PM, Mike Meyer <mwm at mired.org> wrote:

> Exception handling and retrying the code that caused the exception are
> part of the same concept. There is value in being able to express that
> concept directly with a single construct.

It seems to me that combining looping and exception handling in the
same construct makes code harder to read. When you come across any
looping construct, you know immediately that the following block could
occur any number of times, depending on the line where the construct
starts. In the case of try/except/retry, it may come as a surprise that the
block can actually repeat.

Also, you would probably, in most cases, need to add additional logic
before the "retry" to ensure that you don't loop forever if the "fixup"
you apply does not prevent the exception from happening again.
This logic, instead of being tied to the loop itself, now ends up well
hidden in one or many possible exception handlers.


From mwm at mired.org  Wed Jan 25 04:22:30 2012
From: mwm at mired.org (Mike Meyer)
Date: Tue, 24 Jan 2012 19:22:30 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CAJ=7gQLH58WGkNokdOgYC7=LLcJ7Uw4B7KgvN_XCFzve+sdP7w@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
	<CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>
	<20120124165201.1b91c620@mikmeyer-vm-fedora>
	<CADiSq7fpS+1uesn_XV_FBKEGvY9V61mRcSgNzXWHnDLOhxkYEg@mail.gmail.com>
	<20120124171943.57860103@mikmeyer-vm-fedora>
	<CAJ=7gQLH58WGkNokdOgYC7=LLcJ7Uw4B7KgvN_XCFzve+sdP7w@mail.gmail.com>
Message-ID: <20120124192230.65cd5c7f@bhuda.mired.org>

On Tue, 24 Jan 2012 21:53:49 -0500
Vince <vince.vinet at gmail.com> wrote:

> On Tue, Jan 24, 2012 at 8:19 PM, Mike Meyer <mwm at mired.org> wrote:
> > Exception handling and retrying the code that caused the exception are
> > part of the same concept. There is value in being able to express that
> > concept directly with a single construct.
> 
> It seems to me that combining looping and exception handling in the
> same construct makes code harder to read. When you come across any
> looping construct, you know immediately that the following block could
> occur any number of times, depending on the line where the construct
> starts. In the case of try/except/retry, it may come as a surprise that the
> block can actually repeat.

Except the block in the try *doesn't* repeat. That block is only
executed once. It may be *started* multiple times if there are
exceptional conditions, but once it finishes executing, it's
over. That's sort of the point.

      <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From mwm at mired.org  Wed Jan 25 04:26:12 2012
From: mwm at mired.org (Mike Meyer)
Date: Tue, 24 Jan 2012 19:26:12 -0800
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <87ehuo4lll.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
	<CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>
	<20120124165201.1b91c620@mikmeyer-vm-fedora>
	<87ehuo4lll.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20120124192612.50edd619@bhuda.mired.org>

On Wed, 25 Jan 2012 10:28:06 +0900
"Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> Mike Meyer writes:
>  > So long as you incorrectly see this as "just another looping
>  > construct", your conclusions will be flawed.
> That's unfair.  Nobody said "just".  If anything, you were trying in
> earlier posts to maintain the opposite extreme ("not a looping
> construct").

I don't know that nobody said "just". What got said by Nick -
repeatedly - was "We don't need another looping construct." I don't
think it's reasonable to dismiss it without considering how it works
in the other roles for which it's suitable.

>  > The point is to make the try statement more powerful.
> Not on python-ideas, it isn't.  Here, the point is to make Python more
> expressive (including clarity in "expressive").

Didn't we just have a discussion about the ambiguity of English, and
how amorphism isn't pythonic?

The point of the *proposal* is to make the try statement more powerful.

The point of the *list* is to make Python more expressive.

The point of the *thread* is to see if the former does the latter.

Given that intro, I think the conclusion is "not this proposal", with
opinions ranging from "we don't need it at all" to "Might be nice, but
costs to much for the expected use cases."

In thinking about it, I think it's to special purpose. There are at
least four interesting variants of "retry": 1) start the block from
the top; 2) start the block somewhere internally (and where?); 3&4)
Same two, disabling the except handler that ran the retry. It's not
clear there's even a sane way to define #2.

      <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From tjreedy at udel.edu  Wed Jan 25 04:33:19 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 24 Jan 2012 22:33:19 -0500
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <20120124165201.1b91c620@mikmeyer-vm-fedora>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<20120124091950.435c3519@mikmeyer-vm-fedora>
	<4F1EFA53.50008@pearwood.info>
	<20120124104536.788b1186@mikmeyer-vm-fedora>
	<CADiSq7dw46Dtf2L05b0roEyivd3f+2hJ_RdkYff-VysPpHmc4w@mail.gmail.com>
	<20120124165201.1b91c620@mikmeyer-vm-fedora>
Message-ID: <jfnt63$26i$1@dough.gmane.org>

On 1/24/2012 7:52 PM, Mike Meyer wrote:

> For instance, I would write your example like so:
>
>      try:
>          walk_iter = os.walk(topdir)
>          dirpath, files, subdirs = next(wi)
>          # Special case the top level directory
>      except StopIteration:
>          raise RuntimeError("No dir entry for {!r}".format(topdir))
>      # other exception handling here, as appropriate.
>
>      for dirpath, files, subdirs in walk_iter:
>          # Process the subdirectories

I would too, but ...

> Likewise, the try statement is very powerful. That break/continue may
> or may not be evil is immaterial. The point is to make the try
> statement more powerful.

I disagree with turning 'try' into a specialized loop construct by 
adding a version of continue, which is a version of goto. We currently 
have a general overt loop (while), the covert equivalent (tail 
recursion), and a 'specialized' loop (for) that covers the majority of 
loop needs, leaving the generalized loop (while) to cover everything else.

A 'try' statement in itself it a glorified label statement that also 
sets up the context for exception statements. Exception statement are 
conditional statements where the condition is an exception state. A 
marked program position plus a conditional jump is what makes a loop.

-- 
Terry Jan Reedy



From aquavitae69 at gmail.com  Wed Jan 25 07:36:02 2012
From: aquavitae69 at gmail.com (David Townshend)
Date: Wed, 25 Jan 2012 08:36:02 +0200
Subject: [Python-ideas] Allow Profile() to be used as a context manager
Message-ID: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>

I often find it useful to profile small sections to code in a running
application to pinpoint a known bottleneck. The cProfile and profile
modules don't make this easy as they stand, requiring the section of code
to be wrapped into a single statement with can be called as a string using
exec().  This always feels a bit clumsy to me, so my idea is simply to
provide __enter__ and __exit__ methods to cProfile.Profile() and
profile.Profile().  From a quick look at the code it seems that this should
be very easy for cProfile, and slightly less easy for profile (I can't
quite figure out how to deal with the set_cmd() line, or how important it
is - maybe someone can help?).

Any major objections to this?

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120125/8a7530f2/attachment.html>

From anacrolix at gmail.com  Wed Jan 25 08:08:59 2012
From: anacrolix at gmail.com (Matt Joiner)
Date: Wed, 25 Jan 2012 18:08:59 +1100
Subject: [Python-ideas] -m pstats should combine all the profiles given as
	arguments
Message-ID: <CAB4yi1O_XMbu0D8xFm3QWRTiQekS8qPCnj69OGtvamHvPouxxQ@mail.gmail.com>

Frequently when profiling multiple threads, I need to combine several
dump stat files. Currently -m pstats reads the profiling data at only
the first path given. It should merge all the profiling data from all
the paths given.

$ python3.3 -m pstats prof/5506-7f00f*

http://bugs.python.org/issue13839


From ubershmekel at gmail.com  Wed Jan 25 09:19:33 2012
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Wed, 25 Jan 2012 10:19:33 +0200
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
Message-ID: <CANSw7KzsHhGjCqqU9tUUuo-=s0GnPvp1+tXOpZQ9wvkjzmLUgA@mail.gmail.com>

On Wed, Jan 25, 2012 at 8:36 AM, David Townshend <aquavitae69 at gmail.com>wrote:

> [...]
> Any major objections to this?
>
>

 At first this sounded like a good idea but 2 things pop to mind:


   1. How does one get the pstat profiling output from the context manager?
   Will printing out the results be the only option?
   2. Usually when I profile I don't want to touch the code at all so
   eventually if I'll use this context manager it'll be more akin to


    with profile.Profiler():
        main()

Which isn't much of a win over

    profile.run('main()')


Now if you're talking about a reentrant profiler, that's interesting:

    profiler = profile.Profiler()

    while is_working:
        with profiler:
            do_hard_task()

    do_a_hard_but_i_dont_care_now_task()

    with profiler:
        do_a_diff_hard_task()


and then I can call whatever method is there to get the pstat. If this is
the vision then it sounds awesome.


Yuval
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120125/2124b024/attachment.html>

From ncoghlan at gmail.com  Wed Jan 25 09:45:16 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Jan 2012 18:45:16 +1000
Subject: [Python-ideas] Coroutines and PEP 380
In-Reply-To: <CAB4yi1PygqkPbV=H=QD57NL35uWBdPDYbUuxwwCxemGM-Y8kEg@mail.gmail.com>
References: <CADiSq7f8giAGV2+3FvERG+yE4E96NkQ0PsffsjAZ46J8Q8_=cQ@mail.gmail.com>
	<4F15F041.6010607@hotpy.org>
	<20DB36E8-2538-4FE8-9FBF-6B3DA67E3CD6@twistedmatrix.com>
	<4F168FA5.2000503@hotpy.org>
	<7F3B6F9E-A901-4FA5-939E-CDD7B1E6E5B5@twistedmatrix.com>
	<4F188DFD.6080401@canterbury.ac.nz>
	<E2EE2889-BB36-4256-9D57-E4869EB17E1B@twistedmatrix.com>
	<4F1B48D0.3060309@canterbury.ac.nz>
	<CAB4yi1NnJgP4Webc0K+N2XnJF68RLOdRyqJHKw5=DEMxdSOBKw@mail.gmail.com>
	<CAB4yi1PygqkPbV=H=QD57NL35uWBdPDYbUuxwwCxemGM-Y8kEg@mail.gmail.com>
Message-ID: <CADiSq7e1T1WnoAu1U9oug_pqVgAT-jz6p+mYqb7upH1H41P2Mg@mail.gmail.com>

(redirecting to python-ideas - coroutine proposals are nowhere near
mature enough for python-dev)

On Wed, Jan 25, 2012 at 5:35 PM, Matt Joiner <anacrolix at gmail.com> wrote:
> If someone can explain what's stopping real coroutines being into
> Python (3.3), that would be great.

The general issues with that kind of idea:
- the author hasn't offered the code for inclusion and relicensing
under the PSF license (thus we legally aren't allowed to do it)
- complexity
- maintainability
- platform support

In the specific case of coroutines, you have the additional hurdle of
convincing people whether or not they're a good idea at all.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From g.rodola at gmail.com  Wed Jan 25 10:40:16 2012
From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=)
Date: Wed, 25 Jan 2012 10:40:16 +0100
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
Message-ID: <CAFYqXL8=6GMFk7hgZHN3xrF-6Y3PTKjD7wVCm6YLMi3ZMeby5g@mail.gmail.com>

Il 25 gennaio 2012 07:36, David Townshend <aquavitae69 at gmail.com> ha scritto:
> I often find it useful to profile small sections to code in a running
> application to pinpoint a known bottleneck. The cProfile and profile modules
> don't make this easy as they stand, requiring the section of code to be
> wrapped into a single statement with can be called as a string using exec().
> ?This always feels a bit clumsy to me, so my idea?is simply to provide
> __enter__ and __exit__ methods to cProfile.Profile() and profile.Profile().
> ?From a quick look at the code it seems that this should be very easy for
> cProfile, and slightly less easy for profile (I can't quite figure out how
> to deal with the set_cmd() line, or how important it is - maybe someone can
> help?).
>
> Any major objections to this?

Related: http://bugs.python.org/issue9285

--- Giampaolo
http://code.google.com/p/pyftpdlib/
http://code.google.com/p/psutil/
http://code.google.com/p/pysendfile/


From aquavitae69 at gmail.com  Wed Jan 25 10:47:54 2012
From: aquavitae69 at gmail.com (David Townshend)
Date: Wed, 25 Jan 2012 11:47:54 +0200
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <CANSw7KzsHhGjCqqU9tUUuo-=s0GnPvp1+tXOpZQ9wvkjzmLUgA@mail.gmail.com>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
	<CANSw7KzsHhGjCqqU9tUUuo-=s0GnPvp1+tXOpZQ9wvkjzmLUgA@mail.gmail.com>
Message-ID: <CAEgL-feuftbRyCimf=J+8s=PBCBj+S6mZPnpTWyoEw6SRg4paw@mail.gmail.com>

On Wed, Jan 25, 2012 at 10:19 AM, Yuval Greenfield <ubershmekel at gmail.com>wrote:

> On Wed, Jan 25, 2012 at 8:36 AM, David Townshend <aquavitae69 at gmail.com>wrote:
>
>> [...]
>> Any major objections to this?
>>
>>
>
>  At first this sounded like a good idea but 2 things pop to mind:
>
>
>    1. How does one get the pstat profiling output from the context
>    manager? Will printing out the results be the only option?
>
> The same way as currently:

profile = cProfile.Profile()
with profile:
    do_something()

profile.dump_stats(filename)

Perhaps it would be a good idea to provide a convenience function (like
run()), so that:

with cProfile.do(filename):
    do_something()

I don't like calling the function do(), but I can't think of anything
better off-hand, and it illustrates my point.

>
>    1. Usually when I profile I don't want to touch the code at all so
>    eventually if I'll use this context manager it'll be more akin to
>
>
>     with profile.Profiler():
>         main()
>
> Which isn't much of a win over
>
>     profile.run('main()')
>

I agree, but this isn't the use case.


> Now if you're talking about a reentrant profiler, that's interesting:
>
>     profiler = profile.Profiler()
>
>     while is_working:
>         with profiler:
>             do_hard_task()
>
>     do_a_hard_but_i_dont_care_now_task()
>
>     with profiler:
>         do_a_diff_hard_task()
>
>
> and then I can call whatever method is there to get the pstat. If this is
> the vision then it sounds awesome.
>
>
And this is the use case!  I'm not entirely sure how re-entrancy would be
dealt with by the profiler though.  From what I read in the source code it
looks like it would all accumulate, and this is certainly the way I would
expect it to behave, but I'd need to run a few tests to be sure.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120125/75180ebc/attachment.html>

From robert.kern at gmail.com  Wed Jan 25 11:16:58 2012
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 25 Jan 2012 10:16:58 +0000
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
Message-ID: <jfokqq$clv$1@dough.gmane.org>

On 1/25/12 6:36 AM, David Townshend wrote:
> I often find it useful to profile small sections to code in a running
> application to pinpoint a known bottleneck. The cProfile and profile modules
> don't make this easy as they stand, requiring the section of code to be wrapped
> into a single statement with can be called as a string using exec().  This
> always feels a bit clumsy to me, so my idea is simply to provide __enter__ and
> __exit__ methods to cProfile.Profile() and profile.Profile().  From a quick look
> at the code it seems that this should be very easy for cProfile, and slightly
> less easy for profile (I can't quite figure out how to deal with the set_cmd()
> line, or how important it is - maybe someone can help?).
>
> Any major objections to this?

I've done this in my kernprof.py script, which I use as a convenient generic 
profiling script. I subclass cProfile.Profile to add 
enable_by_count()/disable_by_count() methods that allow nesting. 
enable_by_count() increments a counter and only calls the enable() method the 
first time. disable_by_count() decrements the counter and only calls disable() 
when it hits 0 again. __enter__() and __exit__() just call these methods. I also 
add a __call__() method that lets a Profile instance act as a decorator, which I 
actually find somewhat more useful than the context manager.

https://bitbucket.org/robertkern/line_profiler/src/tip/kernprof.py#cl-57

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ncoghlan at gmail.com  Wed Jan 25 11:24:39 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Jan 2012 20:24:39 +1000
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <CAEgL-feuftbRyCimf=J+8s=PBCBj+S6mZPnpTWyoEw6SRg4paw@mail.gmail.com>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
	<CANSw7KzsHhGjCqqU9tUUuo-=s0GnPvp1+tXOpZQ9wvkjzmLUgA@mail.gmail.com>
	<CAEgL-feuftbRyCimf=J+8s=PBCBj+S6mZPnpTWyoEw6SRg4paw@mail.gmail.com>
Message-ID: <CADiSq7c91gMSYybVy=WW2yPTOmvdTO73v9T5kUHZ7efJY3XLXA@mail.gmail.com>

On Wed, Jan 25, 2012 at 7:47 PM, David Townshend <aquavitae69 at gmail.com> wrote:
> The same way as currently:
>
> profile = cProfile.Profile()
> with profile:
> ? ? do_something()
>
> profile.dump_stats(filename)

If __enter__() returns self (as is recommended when you don't have
anything more context specific to return), then the following would
also work:

  with cProfile.Profile() as profile:
      do_something()

  profile.dump_stats(filename)

Assuming the re-entrancy question can be answered adequately, this
sounds like a reasonable idea to me.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From aquavitae69 at gmail.com  Wed Jan 25 11:28:46 2012
From: aquavitae69 at gmail.com (David Townshend)
Date: Wed, 25 Jan 2012 12:28:46 +0200
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <jfokqq$clv$1@dough.gmane.org>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
	<jfokqq$clv$1@dough.gmane.org>
Message-ID: <CAEgL-fc=4+z9fqAwfLjKuOT_8uR8JscsBHufSLuTbe1+DFPP5w@mail.gmail.com>

Would that stop the profiler between calles though?  In Yuval's example,
wouldn't  do_a_hard_but_i_dont_care_now_task be included in the profile
then?

On Wed, Jan 25, 2012 at 12:16 PM, Robert Kern <robert.kern at gmail.com> wrote:

> On 1/25/12 6:36 AM, David Townshend wrote:
>
>> I often find it useful to profile small sections to code in a running
>> application to pinpoint a known bottleneck. The cProfile and profile
>> modules
>> don't make this easy as they stand, requiring the section of code to be
>> wrapped
>> into a single statement with can be called as a string using exec().  This
>> always feels a bit clumsy to me, so my idea is simply to provide
>> __enter__ and
>> __exit__ methods to cProfile.Profile() and profile.Profile().  From a
>> quick look
>> at the code it seems that this should be very easy for cProfile, and
>> slightly
>> less easy for profile (I can't quite figure out how to deal with the
>> set_cmd()
>> line, or how important it is - maybe someone can help?).
>>
>> Any major objections to this?
>>
>
> I've done this in my kernprof.py script, which I use as a convenient
> generic profiling script. I subclass cProfile.Profile to add
> enable_by_count()/disable_by_**count() methods that allow nesting.
> enable_by_count() increments a counter and only calls the enable() method
> the first time. disable_by_count() decrements the counter and only calls
> disable() when it hits 0 again. __enter__() and __exit__() just call these
> methods. I also add a __call__() method that lets a Profile instance act as
> a decorator, which I actually find somewhat more useful than the context
> manager.
>
> https://bitbucket.org/**robertkern/line_profiler/src/**
> tip/kernprof.py#cl-57<https://bitbucket.org/robertkern/line_profiler/src/tip/kernprof.py#cl-57>
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma
>  that is made terrible by our own mad attempt to interpret it as though it
> had
>  an underlying truth."
>  -- Umberto Eco
>
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120125/cac58fac/attachment.html>

From ncoghlan at gmail.com  Wed Jan 25 11:52:45 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Jan 2012 20:52:45 +1000
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <CAEgL-fc=4+z9fqAwfLjKuOT_8uR8JscsBHufSLuTbe1+DFPP5w@mail.gmail.com>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
	<jfokqq$clv$1@dough.gmane.org>
	<CAEgL-fc=4+z9fqAwfLjKuOT_8uR8JscsBHufSLuTbe1+DFPP5w@mail.gmail.com>
Message-ID: <CADiSq7f+xAFnLXjL7R7krHruBUcpQBb=P1BfXC=H8Ak=hDr2rQ@mail.gmail.com>

On Wed, Jan 25, 2012 at 8:28 PM, David Townshend <aquavitae69 at gmail.com> wrote:
> Would that stop the profiler between calles though? ?In Yuval's example,
> wouldn't? do_a_hard_but_i_dont_care_now_task?be included in the profile
> then?

No, because the two profiling blocks aren't overlapping - the count
drops to zero between them so the profiler will be disabled for that
period.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From aquavitae69 at gmail.com  Wed Jan 25 12:03:04 2012
From: aquavitae69 at gmail.com (David Townshend)
Date: Wed, 25 Jan 2012 13:03:04 +0200
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <CADiSq7f+xAFnLXjL7R7krHruBUcpQBb=P1BfXC=H8Ak=hDr2rQ@mail.gmail.com>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
	<jfokqq$clv$1@dough.gmane.org>
	<CAEgL-fc=4+z9fqAwfLjKuOT_8uR8JscsBHufSLuTbe1+DFPP5w@mail.gmail.com>
	<CADiSq7f+xAFnLXjL7R7krHruBUcpQBb=P1BfXC=H8Ak=hDr2rQ@mail.gmail.com>
Message-ID: <CAEgL-fdUsFYCoP6k8jqVkwviSdPJgEE6=7c4PvyN-0+GtzNvvw@mail.gmail.com>

Oh yes, of course. Silly me!

On Wed, Jan 25, 2012 at 12:52 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Wed, Jan 25, 2012 at 8:28 PM, David Townshend <aquavitae69 at gmail.com>
> wrote:
> > Would that stop the profiler between calles though?  In Yuval's example,
> > wouldn't  do_a_hard_but_i_dont_care_now_task be included in the profile
> > then?
>
> No, because the two profiling blocks aren't overlapping - the count
> drops to zero between them so the profiler will be disabled for that
> period.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120125/9c894e83/attachment.html>

From robert.kern at gmail.com  Wed Jan 25 12:37:46 2012
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 25 Jan 2012 11:37:46 +0000
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <CANSw7KzsHhGjCqqU9tUUuo-=s0GnPvp1+tXOpZQ9wvkjzmLUgA@mail.gmail.com>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
	<CANSw7KzsHhGjCqqU9tUUuo-=s0GnPvp1+tXOpZQ9wvkjzmLUgA@mail.gmail.com>
Message-ID: <jfopic$e1g$1@dough.gmane.org>

On 1/25/12 8:19 AM, Yuval Greenfield wrote:
> On Wed, Jan 25, 2012 at 8:36 AM, David Townshend
> <aquavitae69 at gmail.com
> <mailto:aquavitae69 at gmail.com>> wrote:
>
>     [...]
>     Any major objections to this?
>
>
>
>   At first this sounded like a good idea but 2 things pop to mind:
>
>  1. How does one get the pstat profiling output from the context manager? Will
>     printing out the results be the only option?
>  2. Usually when I profile I don't want to touch the code at all so eventually
>     if I'll use this context manager it'll be more akin to
>
>
>      with profile.Profiler():
>          main()
>
> Which isn't much of a win over
>
>      profile.run('main()')
>
>
> Now if you're talking about a reentrant profiler, that's interesting:
>
>      profiler = profile.Profiler()
>
>      while is_working:
>          with profiler:
>              do_hard_task()
>      do_a_hard_but_i_dont_care_now_task()
>
>      with profiler:
>          do_a_diff_hard_task()
>
>
> and then I can call whatever method is there to get the pstat. If this is the
> vision then it sounds awesome.

<shameless-advertisement>

You may want to take a look at my kernprof script mentioned elsewhere in the thread:

   http://packages.python.org/line_profiler/
   http://pypi.python.org/pypi/line_profiler/

Besides adding the context manager to Profile, it is a generic "runner" script 
that will run your script under the profiler and dump the results to a file. In 
the normal use case of profiling the entire script, you don't have to modify 
your code at all:

   [line_profiler]$ kernprof.py -v pystone.py
   Pystone(1.1) time for 50000 passes = 0.918603
   This machine benchmarks at 54430.5 pystones/second
   Wrote profile results to pystone.py.prof
            1100070 function calls in 0.929 seconds

      Ordered by: standard name

      ncalls  tottime  percall  cumtime  percall filename:lineno(function)
           1    0.000    0.000    0.929    0.929 <string>:1(<module>)
       50000    0.140    0.000    0.322    0.000 pystone.py:137(Proc1)
       50000    0.029    0.000    0.029    0.000 pystone.py:153(Proc2)
       50000    0.036    0.000    0.048    0.000 pystone.py:164(Proc3)
       50000    0.015    0.000    0.015    0.000 pystone.py:174(Proc4)
       50000    0.013    0.000    0.013    0.000 pystone.py:181(Proc5)
       50000    0.033    0.000    0.043    0.000 pystone.py:188(Proc6)
      150000    0.034    0.000    0.034    0.000 pystone.py:207(Proc7)
       50000    0.100    0.000    0.126    0.000 pystone.py:212(Proc8)
      150000    0.035    0.000    0.035    0.000 pystone.py:225(Func1)
   ...

But if you do want to use the context manager or the decorator, then there is a 
flag to the script that injects the Profile instance into the builtins. Then you 
can use "@profile" or "with profile:" to only enable the profiler at certain 
points (and multiple points as in your use case above). These are the only 
modifications you need to make. The stats dumping is handled by kernprof.

   [line_profiler]$ grin -A10 '@profile' pystone.py
   pystone.py:
     164 : @profile
     165 + def Proc3(PtrParOut):
     166 +     global IntGlob
     167 +
     168 +     if PtrGlb is not None:
     169 +         PtrParOut = PtrGlb.PtrComp
     170 +     else:
     171 +         IntGlob = 100
     172 +     PtrGlb.IntComp = Proc7(10, IntGlob)
     173 +     return PtrParOut
     174 +
     182 : @profile
     183 + def Proc5():
     184 +     global Char1Glob
     185 +     global BoolGlob
     186 +
     187 +     Char1Glob = 'A'
     188 +     BoolGlob = FALSE
     189 +
     190 + def Proc6(EnumParIn):
     191 +     EnumParOut = EnumParIn
     192 +     if not Func3(EnumParIn):

   [line_profiler]$ kernprof.py -v --builtin pystone.py
   Pystone(1.1) time for 50000 passes = 1.08921
   This machine benchmarks at 45904.8 pystones/second
   Wrote profile results to pystone.py.prof
            350000 function calls in 0.192 seconds

      Ordered by: standard name

      ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100000    0.116    0.000    0.123    0.000 kernprof.py:73(disable_by_count)
       50000    0.042    0.000    0.055    0.000 pystone.py:164(Proc3)
       50000    0.013    0.000    0.013    0.000 pystone.py:182(Proc5)
       50000    0.013    0.000    0.013    0.000 pystone.py:209(Proc7)
      100000    0.007    0.000    0.007    0.000 {method 'disable' of 
'_lsprof.Profiler' objects}

</shameless-advertisement>

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From aquavitae69 at gmail.com  Wed Jan 25 13:18:03 2012
From: aquavitae69 at gmail.com (David Townshend)
Date: Wed, 25 Jan 2012 14:18:03 +0200
Subject: [Python-ideas] Allow Profile() to be used as a context manager
In-Reply-To: <jfopic$e1g$1@dough.gmane.org>
References: <CAEgL-feeuczbSYh96zUULU5961AHthN7N_KHqRqBrdRd3C_wkA@mail.gmail.com>
	<CANSw7KzsHhGjCqqU9tUUuo-=s0GnPvp1+tXOpZQ9wvkjzmLUgA@mail.gmail.com>
	<jfopic$e1g$1@dough.gmane.org>
Message-ID: <CAEgL-fe8qNyi1RmPhs3eW8=Tp_LDJWG+UQuOardwO+EHywB2yg@mail.gmail.com>

shameless-advertisement noted!  It looks like a useful tool, but I had
something rather simpler in mind, and something included in the stdlib.  I
think adding a decorator too is a good idea, but as pointed out there is
already a patch for this.

David

On Wed, Jan 25, 2012 at 1:37 PM, Robert Kern <robert.kern at gmail.com> wrote:

> On 1/25/12 8:19 AM, Yuval Greenfield wrote:
>
>> On Wed, Jan 25, 2012 at 8:36 AM, David Townshend
>> <aquavitae69 at gmail.com
>> <mailto:aquavitae69 at gmail.com>**> wrote:
>>
>>    [...]
>>    Any major objections to this?
>>
>>
>>
>>  At first this sounded like a good idea but 2 things pop to mind:
>>
>>  1. How does one get the pstat profiling output from the context manager?
>> Will
>>
>>    printing out the results be the only option?
>>  2. Usually when I profile I don't want to touch the code at all so
>> eventually
>>
>>    if I'll use this context manager it'll be more akin to
>>
>>
>>     with profile.Profiler():
>>         main()
>>
>> Which isn't much of a win over
>>
>>     profile.run('main()')
>>
>>
>> Now if you're talking about a reentrant profiler, that's interesting:
>>
>>     profiler = profile.Profiler()
>>
>>     while is_working:
>>         with profiler:
>>             do_hard_task()
>>     do_a_hard_but_i_dont_care_now_**task()
>>
>>     with profiler:
>>         do_a_diff_hard_task()
>>
>>
>> and then I can call whatever method is there to get the pstat. If this is
>> the
>> vision then it sounds awesome.
>>
>
> <shameless-advertisement>
>
> You may want to take a look at my kernprof script mentioned elsewhere in
> the thread:
>
>  http://packages.python.org/**line_profiler/<http://packages.python.org/line_profiler/>
>  http://pypi.python.org/pypi/**line_profiler/<http://pypi.python.org/pypi/line_profiler/>
>
> Besides adding the context manager to Profile, it is a generic "runner"
> script that will run your script under the profiler and dump the results to
> a file. In the normal use case of profiling the entire script, you don't
> have to modify your code at all:
>
>  [line_profiler]$ kernprof.py -v pystone.py
>  Pystone(1.1) time for 50000 passes = 0.918603
>  This machine benchmarks at 54430.5 pystones/second
>  Wrote profile results to pystone.py.prof
>           1100070 function calls in 0.929 seconds
>
>     Ordered by: standard name
>
>     ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>          1    0.000    0.000    0.929    0.929 <string>:1(<module>)
>      50000    0.140    0.000    0.322    0.000 pystone.py:137(Proc1)
>      50000    0.029    0.000    0.029    0.000 pystone.py:153(Proc2)
>      50000    0.036    0.000    0.048    0.000 pystone.py:164(Proc3)
>      50000    0.015    0.000    0.015    0.000 pystone.py:174(Proc4)
>      50000    0.013    0.000    0.013    0.000 pystone.py:181(Proc5)
>      50000    0.033    0.000    0.043    0.000 pystone.py:188(Proc6)
>     150000    0.034    0.000    0.034    0.000 pystone.py:207(Proc7)
>      50000    0.100    0.000    0.126    0.000 pystone.py:212(Proc8)
>     150000    0.035    0.000    0.035    0.000 pystone.py:225(Func1)
>  ...
>
> But if you do want to use the context manager or the decorator, then there
> is a flag to the script that injects the Profile instance into the
> builtins. Then you can use "@profile" or "with profile:" to only enable the
> profiler at certain points (and multiple points as in your use case above).
> These are the only modifications you need to make. The stats dumping is
> handled by kernprof.
>
>  [line_profiler]$ grin -A10 '@profile' pystone.py
>  pystone.py:
>    164 : @profile
>    165 + def Proc3(PtrParOut):
>    166 +     global IntGlob
>    167 +
>    168 +     if PtrGlb is not None:
>    169 +         PtrParOut = PtrGlb.PtrComp
>    170 +     else:
>    171 +         IntGlob = 100
>    172 +     PtrGlb.IntComp = Proc7(10, IntGlob)
>    173 +     return PtrParOut
>    174 +
>    182 : @profile
>    183 + def Proc5():
>    184 +     global Char1Glob
>    185 +     global BoolGlob
>    186 +
>    187 +     Char1Glob = 'A'
>    188 +     BoolGlob = FALSE
>    189 +
>    190 + def Proc6(EnumParIn):
>    191 +     EnumParOut = EnumParIn
>    192 +     if not Func3(EnumParIn):
>
>  [line_profiler]$ kernprof.py -v --builtin pystone.py
>  Pystone(1.1) time for 50000 passes = 1.08921
>  This machine benchmarks at 45904.8 pystones/second
>  Wrote profile results to pystone.py.prof
>           350000 function calls in 0.192 seconds
>
>     Ordered by: standard name
>
>     ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>     100000    0.116    0.000    0.123    0.000 kernprof.py:73(disable_by_*
> *count)
>      50000    0.042    0.000    0.055    0.000 pystone.py:164(Proc3)
>      50000    0.013    0.000    0.013    0.000 pystone.py:182(Proc5)
>      50000    0.013    0.000    0.013    0.000 pystone.py:209(Proc7)
>     100000    0.007    0.000    0.007    0.000 {method 'disable' of
> '_lsprof.Profiler' objects}
>
> </shameless-advertisement>
>
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma
>  that is made terrible by our own mad attempt to interpret it as though it
> had
>  an underlying truth."
>  -- Umberto Eco
>
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120125/a5e03245/attachment.html>

From nathan.alexander.rice at gmail.com  Wed Jan 25 15:39:01 2012
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Wed, 25 Jan 2012 09:39:01 -0500
Subject: [Python-ideas] Coroutines and PEP 380
In-Reply-To: <CADiSq7e1T1WnoAu1U9oug_pqVgAT-jz6p+mYqb7upH1H41P2Mg@mail.gmail.com>
References: <CADiSq7f8giAGV2+3FvERG+yE4E96NkQ0PsffsjAZ46J8Q8_=cQ@mail.gmail.com>
	<4F15F041.6010607@hotpy.org>
	<20DB36E8-2538-4FE8-9FBF-6B3DA67E3CD6@twistedmatrix.com>
	<4F168FA5.2000503@hotpy.org>
	<7F3B6F9E-A901-4FA5-939E-CDD7B1E6E5B5@twistedmatrix.com>
	<4F188DFD.6080401@canterbury.ac.nz>
	<E2EE2889-BB36-4256-9D57-E4869EB17E1B@twistedmatrix.com>
	<4F1B48D0.3060309@canterbury.ac.nz>
	<CAB4yi1NnJgP4Webc0K+N2XnJF68RLOdRyqJHKw5=DEMxdSOBKw@mail.gmail.com>
	<CAB4yi1PygqkPbV=H=QD57NL35uWBdPDYbUuxwwCxemGM-Y8kEg@mail.gmail.com>
	<CADiSq7e1T1WnoAu1U9oug_pqVgAT-jz6p+mYqb7upH1H41P2Mg@mail.gmail.com>
Message-ID: <CAOFbRmJz7eDAwA7EzjaujhHqVZyWWe5HJ3WQXAyNAqpJ3UJ0YQ@mail.gmail.com>

On Wed, Jan 25, 2012 at 3:45 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> (redirecting to python-ideas - coroutine proposals are nowhere near
> mature enough for python-dev)
>
> On Wed, Jan 25, 2012 at 5:35 PM, Matt Joiner <anacrolix at gmail.com> wrote:
>> If someone can explain what's stopping real coroutines being into
>> Python (3.3), that would be great.
>
> The general issues with that kind of idea:
> - the author hasn't offered the code for inclusion and relicensing
> under the PSF license (thus we legally aren't allowed to do it)
> - complexity
> - maintainability
> - platform support
>
> In the specific case of coroutines, you have the additional hurdle of
> convincing people whether or not they're a good idea at all.

I think coroutines can be immensely useful.  I am using continulets in
PyPy to implement the flow control for symbolic expression capture,
and it is by far the least complex method I have examined.  I can't
think of any other examples where I've really felt blocked by a
standard stack though.


Nathan


From solipsis at pitrou.net  Wed Jan 25 15:45:24 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 25 Jan 2012 15:45:24 +0100
Subject: [Python-ideas] Coroutines and PEP 380
References: <CADiSq7f8giAGV2+3FvERG+yE4E96NkQ0PsffsjAZ46J8Q8_=cQ@mail.gmail.com>
	<4F15F041.6010607@hotpy.org>
	<20DB36E8-2538-4FE8-9FBF-6B3DA67E3CD6@twistedmatrix.com>
	<4F168FA5.2000503@hotpy.org>
	<7F3B6F9E-A901-4FA5-939E-CDD7B1E6E5B5@twistedmatrix.com>
	<4F188DFD.6080401@canterbury.ac.nz>
	<E2EE2889-BB36-4256-9D57-E4869EB17E1B@twistedmatrix.com>
	<4F1B48D0.3060309@canterbury.ac.nz>
	<CAB4yi1NnJgP4Webc0K+N2XnJF68RLOdRyqJHKw5=DEMxdSOBKw@mail.gmail.com>
	<CAB4yi1PygqkPbV=H=QD57NL35uWBdPDYbUuxwwCxemGM-Y8kEg@mail.gmail.com>
	<CADiSq7e1T1WnoAu1U9oug_pqVgAT-jz6p+mYqb7upH1H41P2Mg@mail.gmail.com>
Message-ID: <20120125154524.74010b64@pitrou.net>

On Wed, 25 Jan 2012 18:45:16 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Jan 25, 2012 at 5:35 PM, Matt Joiner <anacrolix at gmail.com> wrote:
> > If someone can explain what's stopping real coroutines being into
> > Python (3.3), that would be great.
> 
> The general issues with that kind of idea:
> - the author hasn't offered the code for inclusion and relicensing
> under the PSF license (thus we legally aren't allowed to do it)
> - complexity
> - maintainability
> - platform support
> 
> In the specific case of coroutines, you have the additional hurdle of
> convincing people whether or not they're a good idea at all.

Do you mean preemptive coroutines as opposed to ("yield from"-based)
cooperative coroutines?
Because I don't see the point of PEP 380 if not for coroutines
(tree-walking examples are academic).

(I'd also like to point out the following paper which makes an useful
distinction between "task management" and "stack management":
http://www.usenix.org/event/usenix02/adyahowell.html )

Regards

Antoine.




From jimjjewett at gmail.com  Wed Jan 25 16:32:35 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Jan 2012 10:32:35 -0500
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp> <20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CA+OGgf5o1Bc2NWDg1v_jCyWRAm3EVU_cMRgET=JDggny5bcj=g@mail.gmail.com>

On Mon, Jan 23, 2012 at 10:18 PM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Jim Jewett writes:

> ?> So it doesn't seem like a loop because you hope to do it only once?

> With s/hope/expect/, that hits the nail on the head.

> I don't think syntax can express that cleanly, but I can't help
> thinking it's of good thing if somebody like Mike tries to find a way.
> He might succeed!

Ah...

I have often wanted a clean way to indicate "This branch can happen,
but isn't normal."  (Well, besides a comment that might be seen as
condescending if it is *obviously* an edge case.)  The retry proposal
is just specializing that for when the weird branch includes a loop.

Right now, the best I can do is hope that the special case (and
possibly the normal case, if it is repeated) can be factored out, so
that I can write

    if not doit(args):
        if not doit(tweak1(args)):
            if not doit(tweak2(args)):
                raise ReallyCant(args)

or

    if oddcase(args):
        handle_oddcase(args)
    else:
        # Alternatively, make this suite much longer, so that it is
        # "obviously" the main point of the function.
        return _real_function(args)


That said, I've wanted unusual-case annotation more when I thought the
compiler might use the information.  Without compiler support, I'm not
sure how much takeup there would be for the resulting
documentation-only construct.

-jJ


From jimjjewett at gmail.com  Wed Jan 25 16:42:31 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Jan 2012 10:42:31 -0500
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp> <20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
Message-ID: <CA+OGgf7Cak6vChS+tv8htHcXJTQO3yacptLNOKwbeDOBwtMtvA@mail.gmail.com>

On Tue, Jan 24, 2012 at 12:18 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> ... I've written for loops with an unconditional break at the end to
> get "0 or 1" iteration.

Ouch ... That seems even worse than the C do {...} while (0); idiom.

Why didn't you just use an "if"?  (Or at least a try ... except StopIteration)

-jJ


From jimjjewett at gmail.com  Wed Jan 25 17:18:49 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Jan 2012 11:18:49 -0500
Subject: [Python-ideas] Retrying EAFP without DRY
In-Reply-To: <CAMZYqRQS55-5iyDQhsDOjbm_KTzHGqUfhg+=yABsgQ3CgFaASA@mail.gmail.com>
References: <20120120163648.69288796@mikmeyer-vm-fedora>
	<4F1A6DA2.5020807@pearwood.info>
	<CACac1F8CwvHmvfKC-wpWMW9-QOQgUL_2jFtH8vbm_PR4w0hskQ@mail.gmail.com>
	<20120122133014.5e920a43@bhuda.mired.org>
	<CADiSq7f57OLuCOt5WbEnQGB-+akM07i8NSQWuYyXOMk6YtHiJw@mail.gmail.com>
	<871uqr595n.fsf@uwakimon.sk.tsukuba.ac.jp> <20120124000053.GC6693@ando>
	<20120123170633.1bfdbf11@mikmeyer-vm-fedora>
	<CA+OGgf4DLbksJmREmZ3EXVQJbN+eFcpQVah3e97nY6rGrB_Y2w@mail.gmail.com>
	<87obtt4wlu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<ecaf0dd1-d5de-48c9-9c2b-cbacee690fbf@email.android.com>
	<CADiSq7f8v2nS22xKpj8KX5w7V9YgAAg1f2KLZKkCmFK39f6D4g@mail.gmail.com>
	<669fa0fa-beca-4250-90ad-3a073f0d559c@email.android.com>
	<CADiSq7diXgCOSwV_B9fyRHeXA67tdM7w72eF_jKV3mTVZP4Kqw@mail.gmail.com>
	<8F64C8BB-6033-404E-944E-F58E9830C584@gmail.com>
	<4F1E93C7.4050703@pearwood.info>
	<CACac1F9kK6Zppe8HvfaQSw5pPcm=SPmnssMGgVjdnzOaBnZ3GQ@mail.gmail.com>
	<CADiSq7fzP1E=6rVDGXh5wQN96Lh+ru2ECjGA-RDpsbKFuDEuyA@mail.gmail.com>
	<D3EE8103-1462-4D66-9708-6104C94E1338@gmail.com>
	<CAMZYqRQS55-5iyDQhsDOjbm_KTzHGqUfhg+=yABsgQ3CgFaASA@mail.gmail.com>
Message-ID: <CA+OGgf5R21rQQCzZAdtf0gGhtwD1YLHrSK_OMryh9iZWXHbKaQ@mail.gmail.com>

On Tue, Jan 24, 2012 at 1:01 PM, Chris Rebert <pyideas at rebertia.com> wrote:
>> On 24 Jan 2012, at 14:06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> On Tue, Jan 24, 2012 at 9:36 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>>> [ sees value in a RetryStrategy construct of some sort]

>>> You just need to move the pause inside the iterator:
>>>
>>> ? ?def backoff(attempts, first_delay, scale=2):
>>> ? ? ? ?delay = first_delay
>>> ? ? ? ?for attempt in range(1, attempts+1):
>>> ? ? ? ? ? ?yield attempt
>>> ? ? ? ? ? ?time.sleep(delay)
>>> ? ? ? ? ? ?delay *= 2
>>>
>>> ? ?for __ in backoff(MAX_ATTEMPTS, 5):
>>> ? ? ? try:
>>> ? ? ? ? ? response = urllib2.urlopen(url)
>>> ? ? ? except urllib2.HTTPError as e:
>>> ? ? ? ? ? if e.code == 503: ?# Service Unavailable.
>>> ? ? ? ? ? ? ? continue
>>> ? ? ? ? ? raise
>>> ? ? ? break

> On Tue, Jan 24, 2012 at 7:21 AM, Jakob Bowyer <jkbbwr at gmail.com> wrote:
>> Would this not be better expressed as a context manager?
>>
>> with backoff(maxattempts, 5):
>> ? ?# do stuff

> It can't be.

It really should be, though.  What do to on a fixable or temporary
failure is pretty clearly an execution context.

> The `with` statement always executes its block exactly once;
> the context manager(s) have no say in the matter (unless perhaps
> you count raising an exception prior to the block's execution).

    with RetryStrategy() as keepgoing:
        while keepgoing():
            ...

The catch is that keepgoing() is usually either simple enough to
inline (even without the with statement) or sufficiently complicated
that it needs to modify something the rest of the suite sees.  Passing
an object just to hold these changes adds its own code smell.

-jJ


From p.f.moore at gmail.com  Wed Jan 25 19:47:06 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Jan 2012 18:47:06 +0000
Subject: [Python-ideas] Coroutines and PEP 380
In-Reply-To: <20120125154524.74010b64@pitrou.net>
References: <CADiSq7f8giAGV2+3FvERG+yE4E96NkQ0PsffsjAZ46J8Q8_=cQ@mail.gmail.com>
	<4F15F041.6010607@hotpy.org>
	<20DB36E8-2538-4FE8-9FBF-6B3DA67E3CD6@twistedmatrix.com>
	<4F168FA5.2000503@hotpy.org>
	<7F3B6F9E-A901-4FA5-939E-CDD7B1E6E5B5@twistedmatrix.com>
	<4F188DFD.6080401@canterbury.ac.nz>
	<E2EE2889-BB36-4256-9D57-E4869EB17E1B@twistedmatrix.com>
	<4F1B48D0.3060309@canterbury.ac.nz>
	<CAB4yi1NnJgP4Webc0K+N2XnJF68RLOdRyqJHKw5=DEMxdSOBKw@mail.gmail.com>
	<CAB4yi1PygqkPbV=H=QD57NL35uWBdPDYbUuxwwCxemGM-Y8kEg@mail.gmail.com>
	<CADiSq7e1T1WnoAu1U9oug_pqVgAT-jz6p+mYqb7upH1H41P2Mg@mail.gmail.com>
	<20120125154524.74010b64@pitrou.net>
Message-ID: <CACac1F-eZZYn2UGijxXnbbwx-Gg-Nh5nWdF9Y1T6yV5n_fe5kQ@mail.gmail.com>

On 25 January 2012 14:45, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Do you mean preemptive coroutines as opposed to ("yield from"-based)
> cooperative coroutines?

My understanding (in general, and specifically of the various
discussions that have happened on python-ideas) is that we're talking
about co-operative coroutines.

To be clear, what I mean by that is that a coroutine switch is invoked
by an explicit call/statement (which may be "yield from" or some
as-yet unspecified variation - a cocall statement, a call to a
switch() method on a coroutine object, or whatever). The discussions
seem to be more about whether incrementally extending generators (with
send methods, yield from, etc) to turn them into coroutines is better
or worse than starting with a new approach which is closer to
"conventional" approaches from other languages with first-class
coroutines (for example Lua, which uses method calls on coroutine
objects).

I presume that "preemptive coroutines" means that a coroutine switch
can occur at any point. I can't see how that is in any practical sense
different from a thread...

Paul.


From anacrolix at gmail.com  Thu Jan 26 03:08:41 2012
From: anacrolix at gmail.com (Matt Joiner)
Date: Thu, 26 Jan 2012 13:08:41 +1100
Subject: [Python-ideas] Coroutines and PEP 380
In-Reply-To: <CACac1F-eZZYn2UGijxXnbbwx-Gg-Nh5nWdF9Y1T6yV5n_fe5kQ@mail.gmail.com>
References: <CADiSq7f8giAGV2+3FvERG+yE4E96NkQ0PsffsjAZ46J8Q8_=cQ@mail.gmail.com>
	<4F15F041.6010607@hotpy.org>
	<20DB36E8-2538-4FE8-9FBF-6B3DA67E3CD6@twistedmatrix.com>
	<4F168FA5.2000503@hotpy.org>
	<7F3B6F9E-A901-4FA5-939E-CDD7B1E6E5B5@twistedmatrix.com>
	<4F188DFD.6080401@canterbury.ac.nz>
	<E2EE2889-BB36-4256-9D57-E4869EB17E1B@twistedmatrix.com>
	<4F1B48D0.3060309@canterbury.ac.nz>
	<CAB4yi1NnJgP4Webc0K+N2XnJF68RLOdRyqJHKw5=DEMxdSOBKw@mail.gmail.com>
	<CAB4yi1PygqkPbV=H=QD57NL35uWBdPDYbUuxwwCxemGM-Y8kEg@mail.gmail.com>
	<CADiSq7e1T1WnoAu1U9oug_pqVgAT-jz6p+mYqb7upH1H41P2Mg@mail.gmail.com>
	<20120125154524.74010b64@pitrou.net>
	<CACac1F-eZZYn2UGijxXnbbwx-Gg-Nh5nWdF9Y1T6yV5n_fe5kQ@mail.gmail.com>
Message-ID: <CAB4yi1OD4eao7aE=Cn_oAZNrpqjiqwjKrCNzun62kDnD3-KJWw@mail.gmail.com>

On Thu, Jan 26, 2012 at 5:47 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> My understanding (in general, and specifically of the various
> discussions that have happened on python-ideas) is that we're talking
> about co-operative coroutines.
>
> To be clear, what I mean by that is that a coroutine switch is invoked
> by an explicit call/statement (which may be "yield from" or some
> as-yet unspecified variation - a cocall statement, a call to a
> switch() method on a coroutine object, or whatever). The discussions
> seem to be more about whether incrementally extending generators (with
> send methods, yield from, etc) to turn them into coroutines is better
> or worse than starting with a new approach which is closer to
> "conventional" approaches from other languages with first-class
> coroutines (for example Lua, which uses method calls on coroutine
> objects).
>
> I presume that "preemptive coroutines" means that a coroutine switch
> can occur at any point. I can't see how that is in any practical sense
> different from a thread...

I'm not sure that preemptive coroutines as you've described them are
what people are aiming for.

Without better terminology I've taken to calling them implicit and
explicit coroutines:

Implicit coroutines can arbitrary switch without changes in syntax,
calling convention, or traversing back up the stack.

Explicit coroutines must be "driven" (yield from), are called
differently (yield from), and send values back up the stack before
they can switch cleanly (yield from).

Clearly all coroutines share several fantastic properties, they:

* Are stored on the heap,
* Are not preempted,
* They can execute within a single native thread,
* Store state without tying up a native stack for this purpose.

Used for concurrency with an eventing scheduler in Python they give:

* Massive concurrency with minimal system impact (try 20k threads
instead of 20k coroutines),
* Side-step the GIL,
* Ridiculously cheap context switching,
* Remove the need for callbacks and all that rubbish (I saw Guido was
keen on this one),
* Allow existing synchronous code to remain unchanged by modifying the
underlying blocking calls (implicit coroutines only).


From ubershmekel at gmail.com  Thu Jan 26 08:53:02 2012
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Thu, 26 Jan 2012 09:53:02 +0200
Subject: [Python-ideas] Coroutines and PEP 380
In-Reply-To: <CAB4yi1OD4eao7aE=Cn_oAZNrpqjiqwjKrCNzun62kDnD3-KJWw@mail.gmail.com>
References: <CADiSq7f8giAGV2+3FvERG+yE4E96NkQ0PsffsjAZ46J8Q8_=cQ@mail.gmail.com>
	<4F15F041.6010607@hotpy.org>
	<20DB36E8-2538-4FE8-9FBF-6B3DA67E3CD6@twistedmatrix.com>
	<4F168FA5.2000503@hotpy.org>
	<7F3B6F9E-A901-4FA5-939E-CDD7B1E6E5B5@twistedmatrix.com>
	<4F188DFD.6080401@canterbury.ac.nz>
	<E2EE2889-BB36-4256-9D57-E4869EB17E1B@twistedmatrix.com>
	<4F1B48D0.3060309@canterbury.ac.nz>
	<CAB4yi1NnJgP4Webc0K+N2XnJF68RLOdRyqJHKw5=DEMxdSOBKw@mail.gmail.com>
	<CAB4yi1PygqkPbV=H=QD57NL35uWBdPDYbUuxwwCxemGM-Y8kEg@mail.gmail.com>
	<CADiSq7e1T1WnoAu1U9oug_pqVgAT-jz6p+mYqb7upH1H41P2Mg@mail.gmail.com>
	<20120125154524.74010b64@pitrou.net>
	<CACac1F-eZZYn2UGijxXnbbwx-Gg-Nh5nWdF9Y1T6yV5n_fe5kQ@mail.gmail.com>
	<CAB4yi1OD4eao7aE=Cn_oAZNrpqjiqwjKrCNzun62kDnD3-KJWw@mail.gmail.com>
Message-ID: <CANSw7KzYSaCvUA07dRoNvfo=TBVd3_YOuT3iNeFyUta0xrwnZw@mail.gmail.com>

On Thu, Jan 26, 2012 at 4:08 AM, Matt Joiner <anacrolix at gmail.com> wrote:

> Used for concurrency with an eventing scheduler in Python they give:
>
> * Massive concurrency with minimal system impact (try 20k threads
> instead of 20k coroutines),
> * Side-step the GIL,
>
>

How would the eventing scheduler  and coroutines side-step the GIL?


Yuval
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120126/d45aeb61/attachment.html>

From p.f.moore at gmail.com  Thu Jan 26 09:02:21 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 26 Jan 2012 08:02:21 +0000
Subject: [Python-ideas] Coroutines and PEP 380
In-Reply-To: <CAB4yi1OD4eao7aE=Cn_oAZNrpqjiqwjKrCNzun62kDnD3-KJWw@mail.gmail.com>
References: <CADiSq7f8giAGV2+3FvERG+yE4E96NkQ0PsffsjAZ46J8Q8_=cQ@mail.gmail.com>
	<4F15F041.6010607@hotpy.org>
	<20DB36E8-2538-4FE8-9FBF-6B3DA67E3CD6@twistedmatrix.com>
	<4F168FA5.2000503@hotpy.org>
	<7F3B6F9E-A901-4FA5-939E-CDD7B1E6E5B5@twistedmatrix.com>
	<4F188DFD.6080401@canterbury.ac.nz>
	<E2EE2889-BB36-4256-9D57-E4869EB17E1B@twistedmatrix.com>
	<4F1B48D0.3060309@canterbury.ac.nz>
	<CAB4yi1NnJgP4Webc0K+N2XnJF68RLOdRyqJHKw5=DEMxdSOBKw@mail.gmail.com>
	<CAB4yi1PygqkPbV=H=QD57NL35uWBdPDYbUuxwwCxemGM-Y8kEg@mail.gmail.com>
	<CADiSq7e1T1WnoAu1U9oug_pqVgAT-jz6p+mYqb7upH1H41P2Mg@mail.gmail.com>
	<20120125154524.74010b64@pitrou.net>
	<CACac1F-eZZYn2UGijxXnbbwx-Gg-Nh5nWdF9Y1T6yV5n_fe5kQ@mail.gmail.com>
	<CAB4yi1OD4eao7aE=Cn_oAZNrpqjiqwjKrCNzun62kDnD3-KJWw@mail.gmail.com>
Message-ID: <CACac1F-KG5LtR=GarWWotiLn6wki-KK94mC-gTV80Ommssjhuw@mail.gmail.com>

On 26 January 2012 02:08, Matt Joiner <anacrolix at gmail.com> wrote:
> Implicit coroutines can arbitrary switch without changes in syntax,
> calling convention, or traversing back up the stack.
[...]
> * Are not preempted,

That's what confuses me. If an implicit coroutine can arbitrarily
switch without changes in syntax, etc, how is it not pre-empted? How
does the switch happen, if not by pre-emption or by some explicit
request/syntax?

An example of what you mean would probably help.

Paul


From simon.sapin at kozea.fr  Thu Jan 26 09:19:29 2012
From: simon.sapin at kozea.fr (Simon Sapin)
Date: Thu, 26 Jan 2012 09:19:29 +0100
Subject: [Python-ideas] Coroutines and PEP 380
In-Reply-To: <CAB4yi1OD4eao7aE=Cn_oAZNrpqjiqwjKrCNzun62kDnD3-KJWw@mail.gmail.com>
References: <CADiSq7f8giAGV2+3FvERG+yE4E96NkQ0PsffsjAZ46J8Q8_=cQ@mail.gmail.com>
	<4F15F041.6010607@hotpy.org>
	<20DB36E8-2538-4FE8-9FBF-6B3DA67E3CD6@twistedmatrix.com>
	<4F168FA5.2000503@hotpy.org>
	<7F3B6F9E-A901-4FA5-939E-CDD7B1E6E5B5@twistedmatrix.com>
	<4F188DFD.6080401@canterbury.ac.nz>
	<E2EE2889-BB36-4256-9D57-E4869EB17E1B@twistedmatrix.com>
	<4F1B48D0.3060309@canterbury.ac.nz>
	<CAB4yi1NnJgP4Webc0K+N2XnJF68RLOdRyqJHKw5=DEMxdSOBKw@mail.gmail.com>
	<CAB4yi1PygqkPbV=H=QD57NL35uWBdPDYbUuxwwCxemGM-Y8kEg@mail.gmail.com>
	<CADiSq7e1T1WnoAu1U9oug_pqVgAT-jz6p+mYqb7upH1H41P2Mg@mail.gmail.com>
	<20120125154524.74010b64@pitrou.net>
	<CACac1F-eZZYn2UGijxXnbbwx-Gg-Nh5nWdF9Y1T6yV5n_fe5kQ@mail.gmail.com>
	<CAB4yi1OD4eao7aE=Cn_oAZNrpqjiqwjKrCNzun62kDnD3-KJWw@mail.gmail.com>
Message-ID: <4F210C91.303@kozea.fr>

Le 26/01/2012 03:08, Matt Joiner a ?crit :
> Implicit coroutines can arbitrary switch without changes in syntax,
> calling convention, or traversing back up the stack.

Do you mean that greenlets? some_greenlet.switch() looks like a method 
call and does not required a special syntax like "yield from" so it can 
be hidden in some_file.read() like gevent does.

Regards,

-- 
Simon Sapin



From techtonik at gmail.com  Thu Jan 26 15:18:55 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 26 Jan 2012 16:18:55 +0200
Subject: [Python-ideas] Module field in tracker
Message-ID: <CAPkN8xLLeyb4Ttgk1e48FwYk02LBSx8oJgLyhCCjZ+6tL+=5ug@mail.gmail.com>

Is it a good idea to have a list of modules in tracker?
So that you can have a summary of all bugs per component similar to Trac's
http://trac.edgewall.org/milestone/0.12.3
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120126/9d64bdea/attachment.html>

From g.rodola at gmail.com  Thu Jan 26 17:04:40 2012
From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=)
Date: Thu, 26 Jan 2012 17:04:40 +0100
Subject: [Python-ideas] Module field in tracker
In-Reply-To: <CAPkN8xLLeyb4Ttgk1e48FwYk02LBSx8oJgLyhCCjZ+6tL+=5ug@mail.gmail.com>
References: <CAPkN8xLLeyb4Ttgk1e48FwYk02LBSx8oJgLyhCCjZ+6tL+=5ug@mail.gmail.com>
Message-ID: <CAFYqXL8t2sdKRiQJK1CzaXebBXV6NHeCei7SRojUUTpSQPr62g@mail.gmail.com>

Il 26 gennaio 2012 15:18, anatoly techtonik <techtonik at gmail.com> ha scritto:
> Is it a good idea to have a list of modules in tracker?
> So that you can have a summary of all bugs per component similar to Trac's
> http://trac.edgewall.org/milestone/0.12.3
> --
> anatoly t.

+1.
I think there's already an issue opened on the bug tracker's bug
tracker for this. :-)

--- Giampaolo
http://code.google.com/p/pyftpdlib/
http://code.google.com/p/psutil/
http://code.google.com/p/pysendfile/


From techtonik at gmail.com  Thu Jan 26 18:25:40 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 26 Jan 2012 19:25:40 +0200
Subject: [Python-ideas] Dict-like object with property access
Message-ID: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>

I expected to find the answer to this question in FAQ, but because there is
no FAQ I ask it anyway.

How about adding a new standard dict-like container type that allows access
using . (dot) to its members instead of ['index']?
Why? It is convenient to write options.help instead of options['halp'] etc.

Example:
>>> mydict = container(someprop=somevalue)
>>> mydict['someprop']
somevalue
>>> mydict.someprop
somevalue
>>> mydict.otherprop
Exception KeyError ...

I know that it is easy to implement, but wouldn't it be nice to make it
available by default?
A side benefit of having this in stdlib is that newbies will be aware of
the behaviour of derived classes without having to understand the mechanics
of magic methods.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120126/d99d3ae1/attachment.html>

From mwm at mired.org  Thu Jan 26 18:38:15 2012
From: mwm at mired.org (Mike Meyer)
Date: Thu, 26 Jan 2012 09:38:15 -0800
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
Message-ID: <20120126093815.54f8d5b9@mikmeyer-vm-fedora>

On Thu, 26 Jan 2012 19:25:40 +0200
anatoly techtonik <techtonik at gmail.com> wrote:

> I expected to find the answer to this question in FAQ, but because
> there is no FAQ I ask it anyway.

Better to have searched the python-ideas mail list archive.

> How about adding a new standard dict-like container type that allows
> access using . (dot) to its members instead of ['index']?
> Why? It is convenient to write options.help instead of
> options['halp'] etc.

Because it doesn't work in general. There are strings that can be used
as an index, but not as an attribute. There are existing attributes
that you have to avoid, etc.

The only way this really works in practice is if you start with a
fixed set of names you want to access, in which case
collections.namedtuple will probably do the job.

       <mike


From guido at python.org  Thu Jan 26 18:47:00 2012
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jan 2012 09:47:00 -0800
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
Message-ID: <CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>

On Thu, Jan 26, 2012 at 9:25 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> I expected to find the answer to this question in FAQ, but because there is
> no FAQ I ask it anyway.
>
> How about adding a new standard dict-like container type that allows access
> using . (dot) to its members instead of ['index']?
> Why? It is convenient to write options.help instead of options['halp'] etc.
>
> Example:
>>>> mydict = container(someprop=somevalue)
>>>> mydict['someprop']
> somevalue
>>>> mydict.someprop
> somevalue
>>>> mydict.otherprop
> Exception KeyError ...
>
> I know that it is easy to implement, but wouldn't it be nice to make it
> available by default?
> A side benefit of having this in stdlib is that newbies will be aware of the
> behaviour of derived classes without having to understand the mechanics of
> magic methods.

That is pretty much JavaScript's 'object', and I hate this ambiguity.
If your keys are always constants, define a proper options class so
you can say options.help instead of options['help']. You can also
write a generic subclass of dict that works this way, if you really
think you like it so much. But please keep it out of the stdlib. It
leads to confused users, not happy users. An example of the problems
that arise: If d['a'] == d.a, then how come d['clear'] != d.clear ?

-- 
--Guido van Rossum (python.org/~guido)


From tjreedy at udel.edu  Thu Jan 26 22:28:06 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 26 Jan 2012 16:28:06 -0500
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <20120126093815.54f8d5b9@mikmeyer-vm-fedora>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<20120126093815.54f8d5b9@mikmeyer-vm-fedora>
Message-ID: <jfsghe$uob$1@dough.gmane.org>

On 1/26/2012 12:38 PM, Mike Meyer wrote:
> On Thu, 26 Jan 2012 19:25:40 +0200
> anatoly techtonik<techtonik at gmail.com>  wrote:

>> How about adding a new standard dict-like container type that allows
>> access using . (dot) to its members instead of ['index']?

We already have them: any subclass of *object*.

 >>> class Option: pass

 >>> options = Option
 >>> options.help = 'be happy'
 >>> options.help
'be happy'

Or did you mean 'instead of' to mean 'in addition to'?
That is actually possible too.
 >>> options.__dict__['help']
'be happy'
 >>> options_d = options.__dict__ # dict view of options
 >>> options.bye = 'Nice to know you!'
 >>> options_d['bye']
'Nice to know you!'

>> Why? It is convenient to write options.help instead of
>> options['halp'] etc.
>
> Because it doesn't work in general. There are strings that can be used
> as an index, but not as an attribute. There are existing attributes
> that you have to avoid, etc.
>
> The only way this really works in practice is if you start with a
> fixed set of names you want to access, in which case
> collections.namedtuple will probably do the job.

An advantage of collections.namedtuple is that all the pre-existing 
attributes start with '_' so as to avoid name clashes. Another is that 
name subscripts do *not* work, so there is no ambiguity there either.

-- 
Terry Jan Reedy



From techtonik at gmail.com  Mon Jan 30 15:43:06 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 30 Jan 2012 17:43:06 +0300
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
Message-ID: <CAPkN8xKEuY4caST6vv=QZiVE078w0qwh3f4W_ndkXDa-RyfPOg@mail.gmail.com>

On Thu, Jan 26, 2012 at 8:47 PM, Guido van Rossum <guido at python.org> wrote:

> On Thu, Jan 26, 2012 at 9:25 AM, anatoly techtonik <techtonik at gmail.com>
> wrote:
> > I expected to find the answer to this question in FAQ, but because there
> is
> > no FAQ I ask it anyway.
> >
> > How about adding a new standard dict-like container type that allows
> access
> > using . (dot) to its members instead of ['index']?
> > Why? It is convenient to write options.help instead of options['halp']
> etc.
> >
> > Example:
> >>>> mydict = container(someprop=somevalue)
> >>>> mydict['someprop']
> > somevalue
> >>>> mydict.someprop
> > somevalue
> >>>> mydict.otherprop
> > Exception KeyError ...
> >
> > I know that it is easy to implement, but wouldn't it be nice to make it
> > available by default?
> > A side benefit of having this in stdlib is that newbies will be aware of
> the
> > behaviour of derived classes without having to understand the mechanics
> of
> > magic methods.
>
> That is pretty much JavaScript's 'object', and I hate this ambiguity.
> If your keys are always constants, define a proper options class so
> you can say options.help instead of options['help']. You can also
> write a generic subclass of dict that works this way, if you really
> think you like it so much. But please keep it out of the stdlib. It
> leads to confused users, not happy users. An example of the problems
> that arise: If d['a'] == d.a, then how come d['clear'] != d.clear ?


In which case d['clear'] != d.clear can be true?


I've found a MIT licensed library that implements just that:
http://pypi.python.org/pypi/bunch/

Q. Why it is better than subclass of object?
A. Because it is implicit.
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120130/5f35f903/attachment.html>

From simon.sapin at kozea.fr  Mon Jan 30 15:48:15 2012
From: simon.sapin at kozea.fr (Simon Sapin)
Date: Mon, 30 Jan 2012 15:48:15 +0100
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CAPkN8xKEuY4caST6vv=QZiVE078w0qwh3f4W_ndkXDa-RyfPOg@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<CAPkN8xKEuY4caST6vv=QZiVE078w0qwh3f4W_ndkXDa-RyfPOg@mail.gmail.com>
Message-ID: <4F26ADAF.7010907@kozea.fr>

Le 30/01/2012 15:43, anatoly techtonik a ?crit :
> In which case d['clear'] != d.clear can be true?

dict.clear is a method:

 >>> import bunch
 >>> b = bunch.Bunch()
 >>> b['clear'] = 4
 >>> b['clear']
4
 >>> b.clear
<function clear>
 >>> b.clear.__doc__
'D.clear() -> None.  Remove all items from D.'

Regards,
-- 
Simon Sapin


From massimo.dipierro at gmail.com  Mon Jan 30 15:58:46 2012
From: massimo.dipierro at gmail.com (Massimo Di Pierro)
Date: Mon, 30 Jan 2012 08:58:46 -0600
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
Message-ID: <DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>

In web2py we have a class called Storage. (web.py has a similar class too).

works exactly like this except that if you do mydict.someprop and someprop does not exist returns None (which plays the role of JS undefined) instead of raining an exception. Users like this a lot because they can do:

    a = mydict.somevalue or 'somedefault'

which new users find more readable than

   a = mydict.get('somevalue','somedefault')

mydict.__getattr__ is the single most called method in web2py and it does affect performance. It it were supported natively by the language we would benefit from it.

Massimo


On Jan 26, 2012, at 11:47 AM, Guido van Rossum wrote:

> On Thu, Jan 26, 2012 at 9:25 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>> I expected to find the answer to this question in FAQ, but because there is
>> no FAQ I ask it anyway.
>> 
>> How about adding a new standard dict-like container type that allows access
>> using . (dot) to its members instead of ['index']?
>> Why? It is convenient to write options.help instead of options['halp'] etc.
>> 
>> Example:
>>>>> mydict = container(someprop=somevalue)
>>>>> mydict['someprop']
>> somevalue
>>>>> mydict.someprop
>> somevalue
>>>>> mydict.otherprop
>> Exception KeyError ...
>> 
>> I know that it is easy to implement, but wouldn't it be nice to make it
>> available by default?
>> A side benefit of having this in stdlib is that newbies will be aware of the
>> behaviour of derived classes without having to understand the mechanics of
>> magic methods.
> 
> That is pretty much JavaScript's 'object', and I hate this ambiguity.
> If your keys are always constants, define a proper options class so
> you can say options.help instead of options['help']. You can also
> write a generic subclass of dict that works this way, if you really
> think you like it so much. But please keep it out of the stdlib. It
> leads to confused users, not happy users. An example of the problems
> that arise: If d['a'] == d.a, then how come d['clear'] != d.clear ?
> 
> -- 
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From solipsis at pitrou.net  Mon Jan 30 16:06:56 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 30 Jan 2012 16:06:56 +0100
Subject: [Python-ideas] Dict-like object with property access
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
Message-ID: <20120130160656.204dd1d1@pitrou.net>

On Mon, 30 Jan 2012 08:58:46 -0600
Massimo Di Pierro
<massimo.dipierro at gmail.com> wrote:
> In web2py we have a class called Storage. (web.py has a similar class too).
> 
> works exactly like this except that if you do mydict.someprop and someprop does not exist returns None (which plays the role of JS undefined) instead of raining an exception. Users like this a lot because they can do:
> 
>     a = mydict.somevalue or 'somedefault'
> 
> which new users find more readable than
> 
>    a = mydict.get('somevalue','somedefault')
> 
> mydict.__getattr__ is the single most called method in web2py and it does affect performance. It it were supported natively by the language we would benefit from it.

The easy and Pythonic way to benefit from native performance is to
use a dict instead...

Regards

Antoine.




From masklinn at masklinn.net  Mon Jan 30 17:01:31 2012
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 30 Jan 2012 17:01:31 +0100
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
Message-ID: <17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>

On 2012-01-30, at 15:58 , Massimo Di Pierro wrote:
> In web2py we have a class called Storage. (web.py has a similar class too).
> 
> works exactly like this except that if you do mydict.someprop and someprop does not exist returns None (which plays the role of JS undefined) instead of raining an exception. Users like this a lot because they can do:
> 
>    a = mydict.somevalue or 'somedefault'
> 
> which new users find more readable than
> 
>   a = mydict.get('somevalue','somedefault')

And with completely different semantics, as a falsy value (empty string, 0, empty collection, None, etc?) will yield the default for the first case, but not for the second case.

From massimo.dipierro at gmail.com  Mon Jan 30 17:22:26 2012
From: massimo.dipierro at gmail.com (Massimo Di Pierro)
Date: Mon, 30 Jan 2012 10:22:26 -0600
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
Message-ID: <CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>

I do not think the issue is whether the people who use that semantic understand it or not. I can assure you they do and they know when it is appropriate to use it or not. The issue is whether there is any value is making it faster by including it in python or not. Because of the increasing popularity of JS I think new users are starting to expect something like it out of the box.  Anyway, Guido's object about the "clear" method (and other methods) is the most serious problem. One way around could be preventing keys which conflicts with the method names.

Massimo

On Jan 30, 2012, at 10:01 AM, Masklinn wrote:

> On 2012-01-30, at 15:58 , Massimo Di Pierro wrote:
>> In web2py we have a class called Storage. (web.py has a similar class too).
>> 
>> works exactly like this except that if you do mydict.someprop and someprop does not exist returns None (which plays the role of JS undefined) instead of raining an exception. Users like this a lot because they can do:
>> 
>>   a = mydict.somevalue or 'somedefault'
>> 
>> which new users find more readable than
>> 
>>  a = mydict.get('somevalue','somedefault')
> 
> And with completely different semantics, as a falsy value (empty string, 0, empty collection, None, etc?) will yield the default for the first case, but not for the second case.



From arnodel at gmail.com  Mon Jan 30 18:14:11 2012
From: arnodel at gmail.com (Arnaud Delobelle)
Date: Mon, 30 Jan 2012 17:14:11 +0000
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
Message-ID: <CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>

On Jan 30, 2012 4:23 PM, "Massimo Di Pierro" <massimo.dipierro at gmail.com>
wrote:
>
> I do not think the issue is whether the people who use that semantic
understand it or not. I can assure you they do and they know when it is
appropriate to use it or not. The issue is whether there is any value is
making it faster by including it in python or not. Because of the
increasing popularity of JS I think new users are starting to expect
something like it out of the box.

But this design decision in JavaScript is at the heart of many problems
(e.g. simply looping over keys is a pain).  That it is widely used doesn't
make it desirable. My experience with JavaScript is that we should keep
this 'feature' out of Python. If people want it they can implement it very
easily but encouraging them would be wrong.

-- 
Arnaud
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120130/77706752/attachment.html>

From ericsnowcurrently at gmail.com  Mon Jan 30 18:28:01 2012
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Mon, 30 Jan 2012 10:28:01 -0700
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
Message-ID: <CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>

On Mon, Jan 30, 2012 at 10:14 AM, Arnaud Delobelle <arnodel at gmail.com> wrote:
>
> On Jan 30, 2012 4:23 PM, "Massimo Di Pierro" <massimo.dipierro at gmail.com>
> wrote:
>>
>> I do not think the issue is whether the people who use that semantic
>> understand it or not. I can assure you they do and they know when it is
>> appropriate to use it or not. The issue is whether there is any value is
>> making it faster by including it in python or not. Because of the increasing
>> popularity of JS I think new users are starting to expect something like it
>> out of the box.
>
> But this design decision in JavaScript is at the heart of many problems
> (e.g. simply looping over keys is a pain).? That it is widely used doesn't
> make it desirable. My experience with JavaScript is that we should keep this
> 'feature' out of Python. If people want it they can implement it very easily
> but encouraging them would be wrong.

+1


From aquavitae69 at gmail.com  Mon Jan 30 18:50:11 2012
From: aquavitae69 at gmail.com (David Townshend)
Date: Mon, 30 Jan 2012 19:50:11 +0200
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
Message-ID: <CAEgL-fe_Jbw_m2yqSxw+JXtMB9R5g6drqfk25x+ftMhC+_7_eg@mail.gmail.com>

I actually implemented this while ago in a dictionary I created. And a few
weeks ago I went through all my code and removed it because it just wasn't
worth the problems it gave. It's really just sugar for something that's
pretty neat already.
On Jan 30, 2012 7:28 PM, "Eric Snow" <ericsnowcurrently at gmail.com> wrote:

> On Mon, Jan 30, 2012 at 10:14 AM, Arnaud Delobelle <arnodel at gmail.com>
> wrote:
> >
> > On Jan 30, 2012 4:23 PM, "Massimo Di Pierro" <massimo.dipierro at gmail.com
> >
> > wrote:
> >>
> >> I do not think the issue is whether the people who use that semantic
> >> understand it or not. I can assure you they do and they know when it is
> >> appropriate to use it or not. The issue is whether there is any value is
> >> making it faster by including it in python or not. Because of the
> increasing
> >> popularity of JS I think new users are starting to expect something
> like it
> >> out of the box.
> >
> > But this design decision in JavaScript is at the heart of many problems
> > (e.g. simply looping over keys is a pain).  That it is widely used
> doesn't
> > make it desirable. My experience with JavaScript is that we should keep
> this
> > 'feature' out of Python. If people want it they can implement it very
> easily
> > but encouraging them would be wrong.
>
> +1
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120130/25fcb248/attachment.html>

From massimo.dipierro at gmail.com  Mon Jan 30 19:49:54 2012
From: massimo.dipierro at gmail.com (Massimo Di Pierro)
Date: Mon, 30 Jan 2012 12:49:54 -0600
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
Message-ID: <3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>

Trying to make sure I understand where we disagree and perhaps explain my problem better. For me this has very little to do with dictionaries.

STEP 1) Today we can do this:

    class Dummy(object): pass

    d = Dummy()
    d.something = 5
    print d.something

Is anybody calling this un-pythonic?

STEP 2) We can now overload the [] to make the dynamic attributes accessible in an alternative syntax:

    class Dummy(object):
          def __getitem__(self,key): return getattr(self,key)
          def __setitem__(self,key,value): return setattr(self,key,value)
    d = Dummy()
    d.something = 5
    d['something'] = 5
    print d.something
    print d['something']

STEP 3) Is anybody calling this un-pythonic?

We can add redefine __getattr__ so that it never raises an exception:

STEP 4)
 class Dummy(object):
          def __getitem__(self,key): return getattr(self,key)
          def __setitem__(self,key,value): return setattr(self,key,value)
          def __getattr__(self,key):
                  return object.__getattr__(self,key) if hasattr(self,key) else Dummy()

Is this un-pythonic?

I do not think so but I do have a problem with it:

    class Dummy(object):
        def __getitem__(self,key): return getattr(self,key)
        def __setitem__(self,key,value): return setattr(self,key,value)
        def __getattr__(self,key):
            print 'wtf'
            return object.__getattr__(self,key) if hasattr(self,key) else None

    >>> d=Dummy()
    >>> print d.somethingelse
    wtf
    ... 334 wtf times with python2.7, 999 times with python2.5...
    wtf
    None

whatever this does internally, it makes some programs slower then I would like them to be. Why is it calling itself 334 times?

STEP 5) 
We can add methods to make this object behave like a dictionary by redefining d.keys() in terms of d.__dict__.keys() etc.

STEP 6)
we can re-factor it a bit so that actually class Dummy is derived from dict.

Is this the part that people do not like?


I would be happy if Python provided an efficient way to do something like STEP 4 without the problem I mentioned.
Perhaps there is one and I ignore it. I do not necessarily require STEP5 and STEP6.

Use case:

settings = Dummy()
settings.parameter1 = 'a'
settings.parameter2 = 'b'
etc.

if not settings.parameter1: do something ...

Massimo



On Jan 30, 2012, at 11:28 AM, Eric Snow wrote:

> On Mon, Jan 30, 2012 at 10:14 AM, Arnaud Delobelle <arnodel at gmail.com> wrote:
>> 
>> On Jan 30, 2012 4:23 PM, "Massimo Di Pierro" <massimo.dipierro at gmail.com>
>> wrote:
>>> 
>>> I do not think the issue is whether the people who use that semantic
>>> understand it or not. I can assure you they do and they know when it is
>>> appropriate to use it or not. The issue is whether there is any value is
>>> making it faster by including it in python or not. Because of the increasing
>>> popularity of JS I think new users are starting to expect something like it
>>> out of the box.
>> 
>> But this design decision in JavaScript is at the heart of many problems
>> (e.g. simply looping over keys is a pain).  That it is widely used doesn't
>> make it desirable. My experience with JavaScript is that we should keep this
>> 'feature' out of Python. If people want it they can implement it very easily
>> but encouraging them would be wrong.
> 
> +1



From ckaynor at zindagigames.com  Mon Jan 30 20:00:30 2012
From: ckaynor at zindagigames.com (Chris Kaynor)
Date: Mon, 30 Jan 2012 11:00:30 -0800
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
Message-ID: <CALvWhxtXH6Wqg6zevhQ+mmNaP9ogkfD5j2NRM6jUhyDxvAjagw@mail.gmail.com>

On Mon, Jan 30, 2012 at 10:49 AM, Massimo Di Pierro <
massimo.dipierro at gmail.com> wrote:

> STEP 4)
>  class Dummy(object):
>          def __getitem__(self,key): return getattr(self,key)
>          def __setitem__(self,key,value): return setattr(self,key,value)
>          def __getattr__(self,key):
>                  return object.__getattr__(self,key) if hasattr(self,key)
> else Dummy()
>
> Is this un-pythonic?
>
> I do not think so but I do have a problem with it:
>
>    class Dummy(object):
>        def __getitem__(self,key): return getattr(self,key)
>        def __setitem__(self,key,value): return setattr(self,key,value)
>        def __getattr__(self,key):
>            print 'wtf'
>            return object.__getattr__(self,key) if hasattr(self,key) else
> None
>
>    >>> d=Dummy()
>    >>> print d.somethingelse
>    wtf
>    ... 334 wtf times with python2.7, 999 times with python2.5...
>    wtf
>    None
>

getattr calls hasattr, which calls getattr again. You are hitting the
recursion limit before failing the test, due to hasattr failing with a
recursion error, thus returning false. You could refactor this to use
exceptions, such as:

def __getattr__(self, key):
    try:
        return object.__getattr__(self, key)
    except Exception:
        return None


>
> whatever this does internally, it makes some programs slower then I would
> like them to be. Why is it calling itself 334 times?
>
> STEP 5)
> We can add methods to make this object behave like a dictionary by
> redefining d.keys() in terms of d.__dict__.keys() etc.
>
> STEP 6)
> we can re-factor it a bit so that actually class Dummy is derived from
> dict.
>
> Is this the part that people do not like?
>

The general discontent with the idea is step 4. Doing so leads to
abnormalities, such as d['clear'] != d.clear. The only ways to resolve such
abnormalities are to make using any keys/attributes which conflict illegal,
or to use different access methods for the two forms.


>
>
> I would be happy if Python provided an efficient way to do something like
> STEP 4 without the problem I mentioned.
> Perhaps there is one and I ignore it. I do not necessarily require STEP5
> and STEP6.
>
> Use case:
>
> settings = Dummy()
> settings.parameter1 = 'a'
> settings.parameter2 = 'b'
> etc.
>
> if not settings.parameter1: do something ...
>
> Massimo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120130/2ec49f14/attachment.html>

From solipsis at pitrou.net  Mon Jan 30 20:02:26 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 30 Jan 2012 20:02:26 +0100
Subject: [Python-ideas] Dict-like object with property access
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
Message-ID: <20120130200226.0a5ab1d8@pitrou.net>

On Mon, 30 Jan 2012 12:49:54 -0600
Massimo Di Pierro
<massimo.dipierro at gmail.com> wrote:
> Trying to make sure I understand where we disagree and perhaps explain my problem better. For me this has very little to do with dictionaries.
> 
> STEP 1) Today we can do this:
> 
>     class Dummy(object): pass
> 
>     d = Dummy()
>     d.something = 5
>     print d.something
> 
> Is anybody calling this un-pythonic?

Depends what you're doing with it, but having a custom class which
serves as nothing but a plain container is quite contrived in my
opinion.

> STEP 2) We can now overload the [] to make the dynamic attributes accessible in an alternative syntax:
> 
>     class Dummy(object):
>           def __getitem__(self,key): return getattr(self,key)
>           def __setitem__(self,key,value): return setattr(self,key,value)
>     d = Dummy()
>     d.something = 5
>     d['something'] = 5
>     print d.something
>     print d['something']
> 
> STEP 3) Is anybody calling this un-pythonic?

Yes. You don't need both kinds of accesses.

Regards

Antoine.




From massimo.dipierro at gmail.com  Mon Jan 30 20:05:26 2012
From: massimo.dipierro at gmail.com (Massimo Di Pierro)
Date: Mon, 30 Jan 2012 13:05:26 -0600
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
Message-ID: <B8CE17DA-B613-4993-BFE0-FFE09BB1FF69@gmail.com>

Let me add that one can do this:

class Dummy(object):
   def __getitem__(self,key): return getattr(self,key)
   def __setitem__(self,key,value): return setattr(self,key,value)
   def __getattr__(self,key): return self.__dict__.get(key,None)

d=Dummy()
d.something = 5
print d.something
print d.somethingelse

which is more or less efficient and accomplished 4. Not as fast as a dict anyway.

On Jan 30, 2012, at 12:49 PM, Massimo Di Pierro wrote:

> Trying to make sure I understand where we disagree and perhaps explain my problem better. For me this has very little to do with dictionaries.
> 
> STEP 1) Today we can do this:
> 
>    class Dummy(object): pass
> 
>    d = Dummy()
>    d.something = 5
>    print d.something
> 
> Is anybody calling this un-pythonic?
> 
> STEP 2) We can now overload the [] to make the dynamic attributes accessible in an alternative syntax:
> 
>    class Dummy(object):
>          def __getitem__(self,key): return getattr(self,key)
>          def __setitem__(self,key,value): return setattr(self,key,value)
>    d = Dummy()
>    d.something = 5
>    d['something'] = 5
>    print d.something
>    print d['something']
> 
> STEP 3) Is anybody calling this un-pythonic?
> 
> We can add redefine __getattr__ so that it never raises an exception:
> 
> STEP 4)
> class Dummy(object):
>          def __getitem__(self,key): return getattr(self,key)
>          def __setitem__(self,key,value): return setattr(self,key,value)
>          def __getattr__(self,key):
>                  return object.__getattr__(self,key) if hasattr(self,key) else Dummy()
> 
> Is this un-pythonic?
> 
> I do not think so but I do have a problem with it:
> 
>    class Dummy(object):
>        def __getitem__(self,key): return getattr(self,key)
>        def __setitem__(self,key,value): return setattr(self,key,value)
>        def __getattr__(self,key):
>            print 'wtf'
>            return object.__getattr__(self,key) if hasattr(self,key) else None
> 
>>>> d=Dummy()
>>>> print d.somethingelse
>    wtf
>    ... 334 wtf times with python2.7, 999 times with python2.5...
>    wtf
>    None
> 
> whatever this does internally, it makes some programs slower then I would like them to be. Why is it calling itself 334 times?
> 
> STEP 5) 
> We can add methods to make this object behave like a dictionary by redefining d.keys() in terms of d.__dict__.keys() etc.
> 
> STEP 6)
> we can re-factor it a bit so that actually class Dummy is derived from dict.
> 
> Is this the part that people do not like?
> 
> 
> I would be happy if Python provided an efficient way to do something like STEP 4 without the problem I mentioned.
> Perhaps there is one and I ignore it. I do not necessarily require STEP5 and STEP6.
> 
> Use case:
> 
> settings = Dummy()
> settings.parameter1 = 'a'
> settings.parameter2 = 'b'
> etc.
> 
> if not settings.parameter1: do something ...
> 
> Massimo
> 
> 
> 
> On Jan 30, 2012, at 11:28 AM, Eric Snow wrote:
> 
>> On Mon, Jan 30, 2012 at 10:14 AM, Arnaud Delobelle <arnodel at gmail.com> wrote:
>>> 
>>> On Jan 30, 2012 4:23 PM, "Massimo Di Pierro" <massimo.dipierro at gmail.com>
>>> wrote:
>>>> 
>>>> I do not think the issue is whether the people who use that semantic
>>>> understand it or not. I can assure you they do and they know when it is
>>>> appropriate to use it or not. The issue is whether there is any value is
>>>> making it faster by including it in python or not. Because of the increasing
>>>> popularity of JS I think new users are starting to expect something like it
>>>> out of the box.
>>> 
>>> But this design decision in JavaScript is at the heart of many problems
>>> (e.g. simply looping over keys is a pain).  That it is widely used doesn't
>>> make it desirable. My experience with JavaScript is that we should keep this
>>> 'feature' out of Python. If people want it they can implement it very easily
>>> but encouraging them would be wrong.
>> 
>> +1
> 



From phd at phdru.name  Mon Jan 30 20:11:36 2012
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 30 Jan 2012 23:11:36 +0400
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
Message-ID: <20120130191136.GA1555@iskra.aviel.ru>

On Mon, Jan 30, 2012 at 12:49:54PM -0600, Massimo Di Pierro wrote:
> STEP 2) We can now overload the [] to make the dynamic attributes accessible in an alternative syntax:
> 
>     class Dummy(object):
>           def __getitem__(self,key): return getattr(self,key)
>           def __setitem__(self,key,value): return setattr(self,key,value)
>     d = Dummy()
>     d.something = 5
>     d['something'] = 5
>     print d.something
>     print d['something']
> 
> STEP 3) Is anybody calling this un-pythonic?

   I do. The object has two different interfaces for the same values and
that's IMO unpythonic. The Zen says

   "There should be one-- and preferably only one --obvious way to do it."

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From ethan at stoneleaf.us  Mon Jan 30 20:23:35 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 30 Jan 2012 11:23:35 -0800
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <20120130200226.0a5ab1d8@pitrou.net>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<20120130200226.0a5ab1d8@pitrou.net>
Message-ID: <4F26EE37.2040104@stoneleaf.us>

Antoine Pitrou wrote:
> On Mon, 30 Jan 2012 12:49:54 -0600
> Massimo Di Pierro
> <massimo.dipierro at gmail.com> wrote:
>> Trying to make sure I understand where we disagree and perhaps explain my problem better. For me this has very little to do with dictionaries.
> 
>> STEP 2) We can now overload the [] to make the dynamic attributes accessible in an alternative syntax:
>>
>>     class Dummy(object):
>>           def __getitem__(self,key): return getattr(self,key)
>>           def __setitem__(self,key,value): return setattr(self,key,value)
>>     d = Dummy()
>>     d.something = 5
>>     d['something'] = 5
>>     print d.something
>>     print d['something']
>>
>> STEP 3) Is anybody calling this un-pythonic?
> 
> Yes. You don't need both kinds of accesses.

Sure you do -- as soon as 'something' can be passed in via a variable:

def some_func(name):
     print(d.name)  # uh, no
     print(d[name]) # okay, this works


And when you are working with known objects (not passed in names):

def some_other_func():
     flam(d.something)
     sniggle(d.somethingelse)


Okay, *need* might be too strong, as you could get by with [] access -- 
but . access is so much nicer when possible (saves three characters, 
which can make a difference for those of us with wimpy wrists!).

~Ethan~


From masklinn at masklinn.net  Mon Jan 30 20:39:08 2012
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 30 Jan 2012 20:39:08 +0100
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <4F26EE37.2040104@stoneleaf.us>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<20120130200226.0a5ab1d8@pitrou.net>
	<4F26EE37.2040104@stoneleaf.us>
Message-ID: <F86DCEC1-AF9E-4695-93E7-58A2C82E3D65@masklinn.net>

On 2012-01-30, at 20:23 , Ethan Furman wrote:
> Sure you do -- as soon as 'something' can be passed in via a variable:
That's what getattr is for, isn't it?

The issue I have is that, to me, string keys say "arbitrary" and attributes say "enumerated set". Using one for the other? Wreaks that information. I like that Python (and most languages) makes the distinction, and I don't think Javascript and Lua should be emulated on this point.

From massimo.dipierro at gmail.com  Mon Jan 30 20:49:01 2012
From: massimo.dipierro at gmail.com (Massimo Di Pierro)
Date: Mon, 30 Jan 2012 13:49:01 -0600
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <F86DCEC1-AF9E-4695-93E7-58A2C82E3D65@masklinn.net>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<20120130200226.0a5ab1d8@pitrou.net>
	<4F26EE37.2040104@stoneleaf.us>
	<F86DCEC1-AF9E-4695-93E7-58A2C82E3D65@masklinn.net>
Message-ID: <4EC01939-F741-4192-9781-E39387AFA11F@gmail.com>


On Jan 30, 2012, at 1:39 PM, Masklinn wrote:

> On 2012-01-30, at 20:23 , Ethan Furman wrote:
>> Sure you do -- as soon as 'something' can be passed in via a variable:
> That's what getattr is for, isn't it?
> 
> The issue I have is that, to me, string keys say "arbitrary" and attributes say "enumerated set".

I do not understand the distinciton. In fact getattr(x,...) functionally just delegates to getitem(x.__dict__, ...). I do not see why this delegation should not be achieved using the same operator __getitem__.


> Using one for the other? Wreaks that information. I like that Python (and most languages) makes the distinction, and I don't think Javascript and Lua should be emulated on this point.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From ethan at stoneleaf.us  Mon Jan 30 20:25:12 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 30 Jan 2012 11:25:12 -0800
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <20120130191136.GA1555@iskra.aviel.ru>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<20120130191136.GA1555@iskra.aviel.ru>
Message-ID: <4F26EE98.90001@stoneleaf.us>

Oleg Broytman wrote:
> On Mon, Jan 30, 2012 at 12:49:54PM -0600, Massimo Di Pierro wrote:
>> STEP 2) We can now overload the [] to make the dynamic attributes accessible in an alternative syntax:
>>
>>     class Dummy(object):
>>           def __getitem__(self,key): return getattr(self,key)
>>           def __setitem__(self,key,value): return setattr(self,key,value)
>>     d = Dummy()
>>     d.something = 5
>>     d['something'] = 5
>>     print d.something
>>     print d['something']
>>
>> STEP 3) Is anybody calling this un-pythonic?
> 
>    I do. The object has two different interfaces for the same values and
> that's IMO unpythonic. The Zen says
> 
>    "There should be one-- and preferably only one --obvious way to do it."

Indeed.

When you have the attribute name stored in a variable, then the one 
obvious way is [] access.

When you already know the attribute name, the one obvious way is . access.

~Ethan~


From arnodel at gmail.com  Mon Jan 30 20:56:41 2012
From: arnodel at gmail.com (Arnaud Delobelle)
Date: Mon, 30 Jan 2012 19:56:41 +0000
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <4F26EE98.90001@stoneleaf.us>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<20120130191136.GA1555@iskra.aviel.ru>
	<4F26EE98.90001@stoneleaf.us>
Message-ID: <CAJ6cK1bE-=a4G9ctdJnLVhfqPJ=-cS6KMaJQ5+teUW6zxm1FUw@mail.gmail.com>

On 30 January 2012 19:25, Ethan Furman <ethan at stoneleaf.us> wrote:
>> ? I do. The object has two different interfaces for the same values and
>> that's IMO unpythonic. The Zen says
>>
>> ? "There should be one-- and preferably only one --obvious way to do it."
>
>
> Indeed.
>
> When you have the attribute name stored in a variable, then the one obvious
> way is [] access.

If it's an attribute, the obvious way is getattr(obj, attrname)

> When you already know the attribute name, the one obvious way is . access.

Yes.

-- 
Arnaud


From phd at phdru.name  Mon Jan 30 21:06:32 2012
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 31 Jan 2012 00:06:32 +0400
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <4F26EE98.90001@stoneleaf.us>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<20120130191136.GA1555@iskra.aviel.ru>
	<4F26EE98.90001@stoneleaf.us>
Message-ID: <20120130200632.GA4588@iskra.aviel.ru>

On Mon, Jan 30, 2012 at 11:25:12AM -0800, Ethan Furman wrote:
> Oleg Broytman wrote:
> >On Mon, Jan 30, 2012 at 12:49:54PM -0600, Massimo Di Pierro wrote:
> >>STEP 2) We can now overload the [] to make the dynamic attributes accessible in an alternative syntax:
> >>
> >>    class Dummy(object):
> >>          def __getitem__(self,key): return getattr(self,key)
> >>          def __setitem__(self,key,value): return setattr(self,key,value)
> >>    d = Dummy()
> >>    d.something = 5
> >>    d['something'] = 5
> >>    print d.something
> >>    print d['something']
> >>
> >>STEP 3) Is anybody calling this un-pythonic?
> >
> >   I do. The object has two different interfaces for the same values and
> >that's IMO unpythonic. The Zen says
> >
> >   "There should be one-- and preferably only one --obvious way to do it."
> 
> Indeed.
> 
> When you have the attribute name stored in a variable, then the one
> obvious way is [] access.
> 
> When you already know the attribute name, the one obvious way is . access.

   Sure, for attributes. But a dict contents ain't attributes, it's
(key => value) mapping. So for a dict, getitem interface give access to
dict's content, while getattr interface give access to dict's services -
all those .keys(), .items(), .values(), .clear(), .update() et al.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From ethan at stoneleaf.us  Mon Jan 30 21:11:44 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 30 Jan 2012 12:11:44 -0800
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CAJ6cK1bE-=a4G9ctdJnLVhfqPJ=-cS6KMaJQ5+teUW6zxm1FUw@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>	<20120130191136.GA1555@iskra.aviel.ru>	<4F26EE98.90001@stoneleaf.us>
	<CAJ6cK1bE-=a4G9ctdJnLVhfqPJ=-cS6KMaJQ5+teUW6zxm1FUw@mail.gmail.com>
Message-ID: <4F26F980.1040404@stoneleaf.us>

Arnaud Delobelle wrote:
> On 30 January 2012 19:25, Ethan Furman <ethan at stoneleaf.us> wrote:
>>>   I do. The object has two different interfaces for the same values and
>>> that's IMO unpythonic. The Zen says
>>>
>>>   "There should be one-- and preferably only one --obvious way to do it."
>>
>> Indeed.
>>
>> When you have the attribute name stored in a variable, then the one obvious
>> way is [] access.
> 
> If it's an attribute, the obvious way is getattr(obj, attrname)

Fair point.

I guess I'll be happy that Python will let me do it the way that works 
better for me. :)

I think the thing to keep in mind is that this Bunch type object is not 
general purpose, like a dict is:  it is specialized -- only string keys, 
only keys that don't conflict with methods (or no methods, depending on 
the needs), etc., etc.

My custom object that works this way has attributes/methods divided by 
size:  10 or less and it's an attribute; more and it's a method.  Works 
well for me.  (10 because that's the maximum size for a field name in a 
dbf table (version 5 and below, anyway).)

~Ethan~


From ncoghlan at gmail.com  Mon Jan 30 21:56:13 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 31 Jan 2012 06:56:13 +1000
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
Message-ID: <CADiSq7csoJqu+pjApyhJNNnfZLWzcBpJ_QyXhrmdD--BFRX1gA@mail.gmail.com>

On Tue, Jan 31, 2012 at 4:49 AM, Massimo Di Pierro
<massimo.dipierro at gmail.com> wrote:
> Trying to make sure I understand where we disagree and perhaps explain my problem better. For me this has very little to do with dictionaries.

One general challenge with "arbitrary keys as attributes" data stores
in Python is that they can get into messy namespace conflicts, because
you want to allow arbitrary keys, but you also want access to ordinary
methods. collections.namedtuple gets around this by leaving the
attribute namespace largely free for end users and prefixing method
names and class information with underscores.

Ultimately, what ends up happening as far as the standard library goes
is that "explicit is better than implicit" wins. If you don't know
what keys you're going to get, then the appropriate data structure is
a dict, not an object. If you *do* know which keys you're going to
get, then the appropriate data structure is a predefined class
(perhaps generated programmatically once the full set of permitted
attributes is determined - e.g. when creating an appropriate
namedtuple definition from a set of column headers).

*Outside* the standard library, it's "practicality beats purity" that
wins. People *like* being able to type ".attr" instead of "['attr']"
when a field name happens to be a legal identifier. The core problem
is that it's so easy to write a "good enough" version of such a class
for yourself that nobody has ever bothered to standardise on a
particular way of handling this that is suitable for stdlib inclusion
(particularly when there are so many people that object to the idea
*in principle*, regardless of the details of how it is implemented)

Cheers,
Nick.

> STEP 4)
>  class Dummy(object):
>          def __getitem__(self,key): return getattr(self,key)
>          def __setitem__(self,key,value): return setattr(self,key,value)
>          def __getattr__(self,key):
>                  return object.__getattr__(self,key) if hasattr(self,key) else Dummy()

P.S. Tip for fast autovivification in Python:

    from collections import defaultdict

    def autodict():
        return defaultdict(autodict)

    >>> store['a']['b']['c']
    defaultdict(<function autodict at 0x7f5adfe349b0>, {})

If you want this behaviour in instances of a particular class, set the
instance __dict__ attributes to one of those in __new__ instead of
overriding __getattr__.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From tjreedy at udel.edu  Mon Jan 30 22:17:24 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 30 Jan 2012 16:17:24 -0500
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
Message-ID: <jg71d6$32t$1@dough.gmane.org>

On 1/30/2012 1:49 PM, Massimo Di Pierro wrote:
> Trying to make sure I understand where we disagree...
>      class Dummy(object): pass
>
>      d = Dummy()
>      d.something = 5
>      print d.something
>
> Is anybody calling this un-pythonic?

I already suggested that that is the way to make a pure *name* to data 
mapping object. If you want a default data object, add your corrected 
method:
    def __getattr__(self,key): return self.__dict__.get(key,None)
To make Dummy instances key iterable like dicts, add
	def __iter__(self): return iter(self.__dict__)
If you prefer to iterate by name-object pairs:
	def __iter__(self): return iter(self.__dict__.items())
One could define a few other special methods, like __eq__, to tie into 
other syntax.

> STEP 2) We can now overload the [] to make the dynamic attributes accessible in an alternative syntax:
>
>      class Dummy(object):
>            def __getitem__(self,key): return getattr(self,key)
>            def __setitem__(self,key,value): return setattr(self,key,value)

> Is anybody calling this un-pythonic?

Guido already did, insofar as he defines 'pythonic'. Anyway, skip that 
word. This is where many of us 'disagree'. This design redefines Dummy 
as a *string* to data mapping object. You add a second access method 
that makes the first access method only partial. To me, it is a 
conceptually crazy object. And it gets worse when you try to make it a 
dict, with some names now reserved for methods. Dicts, lists, and tuples 
have a clean separation between contents, accessed by subscript, and 
methods to work on contents, accessed as attributes. Many of us consider 
that a virtue and a feature.

-- 
Terry Jan Reedy



From cs at zip.com.au  Mon Jan 30 22:22:10 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 31 Jan 2012 08:22:10 +1100
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
References: <CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
Message-ID: <20120130212210.GA9664@cskk.homeip.net>

On 30Jan2012 10:28, Eric Snow <ericsnowcurrently at gmail.com> wrote:
| On Mon, Jan 30, 2012 at 10:14 AM, Arnaud Delobelle <arnodel at gmail.com> wrote:
| > On Jan 30, 2012 4:23 PM, "Massimo Di Pierro" <massimo.dipierro at gmail.com>
| > wrote:
| >> I do not think the issue is whether the people who use that semantic
| >> understand it or not. I can assure you they do and they know when it is
| >> appropriate to use it or not. The issue is whether there is any value is
| >> making it faster by including it in python or not. Because of the increasing
| >> popularity of JS I think new users are starting to expect something like it
| >> out of the box.
| >
| > But this design decision in JavaScript is at the heart of many problems
| > (e.g. simply looping over keys is a pain).? That it is widely used doesn't
| > make it desirable. My experience with JavaScript is that we should keep this
| > 'feature' out of Python. If people want it they can implement it very easily
| > but encouraging them would be wrong.
| 
| +1

+1

Like a few others, I have implemented this kind of class. The only one
of mine that survives only supports .UPPERCASE attribute->key access and
is quite special purpose. I'm very much against including such a
facility in the language directly.
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Excuse me, do you know what time it is?
-About Noon.
I'd prefer something more exact . . .
-About Noon, Stupid.


From steve at pearwood.info  Tue Jan 31 01:28:53 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 31 Jan 2012 11:28:53 +1100
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CADiSq7csoJqu+pjApyhJNNnfZLWzcBpJ_QyXhrmdD--BFRX1gA@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<CADiSq7csoJqu+pjApyhJNNnfZLWzcBpJ_QyXhrmdD--BFRX1gA@mail.gmail.com>
Message-ID: <4F2735C5.4000307@pearwood.info>

Nick Coghlan wrote:

> *Outside* the standard library, it's "practicality beats purity" that
> wins. People *like* being able to type ".attr" instead of "['attr']"
> when a field name happens to be a legal identifier.

Yes, but people like all sorts of things that aren't good for them, and while 
Python can and does allow people to shoot themselves in the foot, I don't 
think we should be providing a standard foot-shooting class :)

As I see it, x.name is best used for attributes of the object, which typically 
means that (1) in some sense, they are inherently part of the object; (2) they 
come from a relatively small enumerated set of names which is usually fixed; 
and (3) you are very unlikely to be added or deleting them from an object 
(usually a static set rather than dynamic).

x['name'] is for the opposite cases:

(1) the key/value pairs are not inherently part of the object, but in some 
sense are possessions of the object; (2) they come from a potentially 
arbitrarily large set of names which could grow indefinitely large; and (3) 
are usually dynamic rather than static.

In my experience, there are very few use-cases which fall between those two 
extremes and don't see the need for a standard Javascript-like dotted-access 
dict. But I think the first place to put one would be a recipe on (say) 
ActiveState's cookbook, and see what sort of feedback and use it gets.



-- 
Steven


From ncoghlan at gmail.com  Tue Jan 31 01:49:33 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 31 Jan 2012 10:49:33 +1000
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <4F2735C5.4000307@pearwood.info>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<CADiSq7csoJqu+pjApyhJNNnfZLWzcBpJ_QyXhrmdD--BFRX1gA@mail.gmail.com>
	<4F2735C5.4000307@pearwood.info>
Message-ID: <CADiSq7d22aE5diV8fh2DaM5r89WcZMoRD5wx8MtE+S--Uzkf-A@mail.gmail.com>

On Tue, Jan 31, 2012 at 10:28 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Nick Coghlan wrote:
>
>> *Outside* the standard library, it's "practicality beats purity" that
>> wins. People *like* being able to type ".attr" instead of "['attr']"
>> when a field name happens to be a legal identifier.
>
>
> Yes, but people like all sorts of things that aren't good for them, and
> while Python can and does allow people to shoot themselves in the foot, I
> don't think we should be providing a standard foot-shooting class :)

You don't have to persuade *me* of that. I spent a bunch of time years
ago working with Steven Bethard on the "namespaces" proto-PEP and
package, and the idea is logically incoherent enough that it's simply
hard to cover all the use case variants in a single class. You either
end up with a baroque monstrosity that handles everything, or you have
people still rolling their own because the "official" one doesn't
behave exactly the way they want.

About the only variant of the idea that I *could* get behind these
days is a collections.record class factory that was basically a
variant of collections.namedtuple that produced mutable objects
instead of tuples.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From stephen at xemacs.org  Tue Jan 31 02:03:11 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 31 Jan 2012 10:03:11 +0900
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <4EC01939-F741-4192-9781-E39387AFA11F@gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<20120130200226.0a5ab1d8@pitrou.net>
	<4F26EE37.2040104@stoneleaf.us>
	<F86DCEC1-AF9E-4695-93E7-58A2C82E3D65@masklinn.net>
	<4EC01939-F741-4192-9781-E39387AFA11F@gmail.com>
Message-ID: <87mx941y5s.fsf@uwakimon.sk.tsukuba.ac.jp>

Massimo Di Pierro writes:
 > On Jan 30, 2012, at 1:39 PM, Masklinn wrote:

 > > The issue I have is that, to me, string keys say "arbitrary" and attributes say "enumerated set".
 > 
 > I do not understand the distinciton. In fact getattr(x,...) functionally just delegates to getitem(x.__dict__, ...). I do not see why this delegation should not be achieved using the same operator __getitem__.

That's because you're looking at it through the lens of the
implementation.  But you're suggesting changing the language.  The
argument for that cannot depend on the implementation.


From ethan at stoneleaf.us  Tue Jan 31 02:09:51 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 30 Jan 2012 17:09:51 -0800
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CADiSq7d22aE5diV8fh2DaM5r89WcZMoRD5wx8MtE+S--Uzkf-A@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>	<CADiSq7csoJqu+pjApyhJNNnfZLWzcBpJ_QyXhrmdD--BFRX1gA@mail.gmail.com>	<4F2735C5.4000307@pearwood.info>
	<CADiSq7d22aE5diV8fh2DaM5r89WcZMoRD5wx8MtE+S--Uzkf-A@mail.gmail.com>
Message-ID: <4F273F5F.2050406@stoneleaf.us>

Nick Coghlan wrote:
> On Tue, Jan 31, 2012 at 10:28 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> Nick Coghlan wrote:
>>
>>> *Outside* the standard library, it's "practicality beats purity" that
>>> wins. People *like* being able to type ".attr" instead of "['attr']"
>>> when a field name happens to be a legal identifier.
>>
>> Yes, but people like all sorts of things that aren't good for them, and
>> while Python can and does allow people to shoot themselves in the foot, I
>> don't think we should be providing a standard foot-shooting class :)
> 
> You don't have to persuade *me* of that. I spent a bunch of time years
> ago working with Steven Bethard on the "namespaces" proto-PEP and
> package, and the idea is logically incoherent enough that it's simply
> hard to cover all the use case variants in a single class. You either
> end up with a baroque monstrosity that handles everything, or you have
> people still rolling their own because the "official" one doesn't
> behave exactly the way they want.
> 
> About the only variant of the idea that I *could* get behind these
> days is a collections.record class factory that was basically a
> variant of collections.namedtuple that produced mutable objects
> instead of tuples.

I agree that making a general purpose object is not feasible, but I am 
very glad that Python doesn't put giant roadblocks in the way so we can 
make special purpose ones that meet our own needs and tradeoffs.

~Ethan~


From g.brandl at gmx.net  Tue Jan 31 08:10:03 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 31 Jan 2012 08:10:03 +0100
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <4F26EE37.2040104@stoneleaf.us>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<20120130200226.0a5ab1d8@pitrou.net>
	<4F26EE37.2040104@stoneleaf.us>
Message-ID: <jg8448$ioq$1@dough.gmane.org>

Am 30.01.2012 20:23, schrieb Ethan Furman:
> Antoine Pitrou wrote:
>> On Mon, 30 Jan 2012 12:49:54 -0600
>> Massimo Di Pierro
>> <massimo.dipierro at gmail.com> wrote:
>>> Trying to make sure I understand where we disagree and perhaps explain my problem better. For me this has very little to do with dictionaries.
>> 
>>> STEP 2) We can now overload the [] to make the dynamic attributes accessible in an alternative syntax:
>>>
>>>     class Dummy(object):
>>>           def __getitem__(self,key): return getattr(self,key)
>>>           def __setitem__(self,key,value): return setattr(self,key,value)
>>>     d = Dummy()
>>>     d.something = 5
>>>     d['something'] = 5
>>>     print d.something
>>>     print d['something']
>>>
>>> STEP 3) Is anybody calling this un-pythonic?
>> 
>> Yes. You don't need both kinds of accesses.
> 
> Sure you do -- as soon as 'something' can be passed in via a variable:
> 
> def some_func(name):
>      print(d.name)  # uh, no
>      print(d[name]) # okay, this works

If anything, the d.[name] (short for getattr(d, name)) proposal should be
reconsidered.  But IIRC it was concluded when discussed last time that the
syntax is too hard to quickly distinguish from item access, and a better
one couldn't be found.

Georg



From eric at trueblade.com  Tue Jan 31 15:31:50 2012
From: eric at trueblade.com (Eric Smith)
Date: Tue, 31 Jan 2012 09:31:50 -0500 (EST)
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <CADiSq7d22aE5diV8fh2DaM5r89WcZMoRD5wx8MtE+S--Uzkf-A@mail.gmail.com>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<CADiSq7csoJqu+pjApyhJNNnfZLWzcBpJ_QyXhrmdD--BFRX1gA@mail.gmail.com>
	<4F2735C5.4000307@pearwood.info>
	<CADiSq7d22aE5diV8fh2DaM5r89WcZMoRD5wx8MtE+S--Uzkf-A@mail.gmail.com>
Message-ID: <83d6e47f0afc488953529cce99197350.squirrel@mail.trueblade.com>

> About the only variant of the idea that I *could* get behind these
> days is a collections.record class factory that was basically a
> variant of collections.namedtuple that produced mutable objects
> instead of tuples.

There are many of these. My own http://pypi.python.org/pypi/recordtype is
one such. It's primary use cases are:

- mutability
- default values

Eric.



From ericsnowcurrently at gmail.com  Tue Jan 31 19:06:43 2012
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Tue, 31 Jan 2012 11:06:43 -0700
Subject: [Python-ideas] Dict-like object with property access
In-Reply-To: <jg8448$ioq$1@dough.gmane.org>
References: <CAPkN8x+i4X1a_8a_FB8QmUB328uoWQXP6ALQYJq6b8Lb9Q8HrQ@mail.gmail.com>
	<CAP7+vJJuDMva6FNA9xph-DrXckO2oa7xAtD-C3MRQ3YmEBOigw@mail.gmail.com>
	<DBEB52B3-8F25-46FF-8501-CC1472C33494@gmail.com>
	<17E44F92-A978-4E93-9713-7D542E61ED10@masklinn.net>
	<CCACB5E5-6753-404C-9A4A-62489D99FF0A@gmail.com>
	<CAJ6cK1bK-=9AaTsOZQ6CiDo77Ya_ie4kWVFp5MXcxGnTTmYGdg@mail.gmail.com>
	<CALFfu7AArrX7wnUGjaN2HtmxsWqYQ7ktK7xy+atZBn0zKKM8Dg@mail.gmail.com>
	<3DFDD08E-D82B-4706-8DAB-9F06B9E1F403@gmail.com>
	<20120130200226.0a5ab1d8@pitrou.net>
	<4F26EE37.2040104@stoneleaf.us> <jg8448$ioq$1@dough.gmane.org>
Message-ID: <CALFfu7C=pwkwjBxukaY1w7_wVPPQA36efqoCAJTzNFfttuCmGw@mail.gmail.com>

On Tue, Jan 31, 2012 at 12:10 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> If anything, the d.[name] (short for getattr(d, name)) proposal should be
> reconsidered. ?But IIRC it was concluded when discussed last time that the
> syntax is too hard to quickly distinguish from item access, and a better
> one couldn't be found.

The last discussion I recall for that syntax was less than a year
ago[1] and got lost amid other proposals.  The idea did come up (and
floundered) in 2007, though.  It would be worth having another look.
As Raymond put it[2]:

    IIRC, the idea for a __getattr__ syntax was favorably received at first,
    but it then drowned in a sea of syntax bikeshedding which precluded any
    serious discussion of use cases and benefits.

    Also remember that not all dead proposals have to stay dead.  When
    generator expressions were first proposed, the PEP was rejected.
    The same was true for generator exceptions and for pushing data into
    running generators, yey these were ultimately accepted in the form of
    throw() and send().

+1 for reconsidering the d.[name] / d.(name) / d!name syntax.

-eric


[1]  blow-by-blow compilation from March 2011:
 * Raymond brought it up --
http://mail.python.org/pipermail/python-ideas/2011-March/009265.html
 * ...and remained relatively neutral --
http://mail.python.org/pipermail/python-ideas/2011-March/009269.html
 * Greg Ewing approved --
http://mail.python.org/pipermail/python-ideas/2011-March/009271.html
 * Guido was thinking about it --
http://mail.python.org/pipermail/python-ideas/2011-March/009290.html
 * Larry Hastings on the 2007 thread --
http://mail.python.org/pipermail/python-ideas/2011-March/009284.html
 * Raymond in response --
http://mail.python.org/pipermail/python-ideas/2011-March/009286.html
[2]  http://mail.python.org/pipermail/python-ideas/2011-March/009286.html