From joshua.spoerri at gmail.com  Tue May 13 15:52:35 2008
From: joshua.spoerri at gmail.com (Joshua Spoerri)
Date: Tue, 13 May 2008 09:52:35 -0400
Subject: [Python-ideas] decimal by default
Message-ID: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com>

Should decimal be the default for floating period literals?

E.g.
1.2 would actually be decimal.Decimal("1.2")
and float(1.2) would be used to get traditional binary float point.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20080513/c7875383/attachment.html>

From guido at python.org  Tue May 13 16:04:49 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 13 May 2008 07:04:49 -0700
Subject: [Python-ideas] decimal by default
In-Reply-To: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com>
References: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com>
Message-ID: <ca471dc20805130704s14100b50gbcc94fafc6df3f38@mail.gmail.com>

On Tue, May 13, 2008 at 6:52 AM, Joshua Spoerri
<joshua.spoerri at gmail.com> wrote:
> Should decimal be the default for floating period literals?
>
> E.g.
> 1.2 would actually be decimal.Decimal("1.2")
> and float(1.2) would be used to get traditional binary float point.

Not in 3.0, there are too many things that are subtly different.
Perhaps at some point post 3.0 we can invent a mechanism whereby
modules can enable this feature on a per-module basis, and then some
number of revisions later we can change the default.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From santagada at gmail.com  Tue May 13 16:12:21 2008
From: santagada at gmail.com (Leonardo Santagada)
Date: Tue, 13 May 2008 11:12:21 -0300
Subject: [Python-ideas] decimal by default
In-Reply-To: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com>
References: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com>
Message-ID: <AC47AD3E-BBD5-4EC2-AEA0-51886F509350@gmail.com>


On 13/05/2008, at 10:52, Joshua Spoerri wrote:

> Should decimal be the default for floating period literals?

-Inf

:)

> E.g.
> 1.2 would actually be decimal.Decimal("1.2")
> and float(1.2) would be used to get traditional binary float point.

--
Leonardo Santagada








From santagada at gmail.com  Tue May 13 17:03:28 2008
From: santagada at gmail.com (Leonardo Santagada)
Date: Tue, 13 May 2008 12:03:28 -0300
Subject: [Python-ideas] decimal by default
In-Reply-To: <ca471dc20805130704s14100b50gbcc94fafc6df3f38@mail.gmail.com>
References: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com>
	<ca471dc20805130704s14100b50gbcc94fafc6df3f38@mail.gmail.com>
Message-ID: <F87458AD-3AD6-4427-AC0E-5F7334A42FFE@gmail.com>


On 13/05/2008, at 11:04, Guido van Rossum wrote:

> On Tue, May 13, 2008 at 6:52 AM, Joshua Spoerri
> <joshua.spoerri at gmail.com> wrote:
>> Should decimal be the default for floating period literals?
>>
>> E.g.
>> 1.2 would actually be decimal.Decimal("1.2")
>> and float(1.2) would be used to get traditional binary float point.
>
> Not in 3.0, there are too many things that are subtly different.
> Perhaps at some point post 3.0 we can invent a mechanism whereby
> modules can enable this feature on a per-module basis, and then some
> number of revisions later we can change the default.


I would be happier with a d in front of the number following the  
scheme of raw strings, either to mean Decimal or to mean Double, or  
maybe f. This way things would work for both cientists and the rest of  
the users.

--
Leonardo Santagada








From george.sakkis at gmail.com  Tue May 13 20:07:15 2008
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 13 May 2008 14:07:15 -0400
Subject: [Python-ideas] decimal by default
In-Reply-To: <F87458AD-3AD6-4427-AC0E-5F7334A42FFE@gmail.com>
References: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com>
	<ca471dc20805130704s14100b50gbcc94fafc6df3f38@mail.gmail.com>
	<F87458AD-3AD6-4427-AC0E-5F7334A42FFE@gmail.com>
Message-ID: <91ad5bf80805131107o69e377a5m94dd7a9c52a1144@mail.gmail.com>

On Tue, May 13, 2008 at 11:03 AM, Leonardo Santagada
<santagada at gmail.com> wrote:
>
>  On 13/05/2008, at 11:04, Guido van Rossum wrote:
>
>
> > On Tue, May 13, 2008 at 6:52 AM, Joshua Spoerri
> > <joshua.spoerri at gmail.com> wrote:
> >
> > > Should decimal be the default for floating period literals?
> > >
> > > E.g.
> > > 1.2 would actually be decimal.Decimal("1.2")
> > > and float(1.2) would be used to get traditional binary float point.
> > >
> >
> > Not in 3.0, there are too many things that are subtly different.
> > Perhaps at some point post 3.0 we can invent a mechanism whereby
> > modules can enable this feature on a per-module basis, and then some
> > number of revisions later we can change the default.
> >
>
>
>  I would be happier with a d in front of the number following the scheme of
> raw strings, either to mean Decimal or to mean Double, or maybe f. This way
> things would work for both cientists and the rest of the users.

I would prefer "from __future__ import default_decimals", with target
version for being enforced the 6.0 or later; we'll all be carrying
quantum laptops by then ;-)

George


From tjreedy at udel.edu  Thu May 15 08:36:25 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 15 May 2008 02:36:25 -0400
Subject: [Python-ideas] Give generators a name?
Message-ID: <g0gll4$il7$1@ger.gmane.org>

I have 2 related proposals:

1. Give generators a .__name__ attribute that is the same as their curent 
(3.0a5) .gi_code.co_name subattribute.  just as funct.__name__ is 
func.__code__.co_name.

My reason is, I expect, much the same as that for func.__name__.  I am 
using the generator name (for bad-iterator-output messages in a test 
function) and would prefer to get it through a cross-implementation 
'public' interface' rather than a cPython internal  implementation detail 
(which I understand code object to be).  I am otherwise trying to avoid 
using cPython internals.

(Is there any plan to change the gi_* attributes the way the func_* 
attributes were?)

2. Whether or not 1 is adopted, add the name to the representation:
 <gfuncname generator object as..> or <generator object gfuncname at ..>

Conceptually, I see a generator function as an abbreviated version of a 
iterator class, with most of the boilerplate removed, that defines a 
subclass of the generator class.  So I think the subclass name should be 
part of its representation.

Terry Jan Reedy






From guido at python.org  Thu May 15 14:56:22 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 15 May 2008 05:56:22 -0700
Subject: [Python-ideas] Give generators a name?
In-Reply-To: <g0gll4$il7$1@ger.gmane.org>
References: <g0gll4$il7$1@ger.gmane.org>
Message-ID: <ca471dc20805150556o34dfe521rd588a0d2a91dba82@mail.gmail.com>

Sounds fine with me. Have you cooked up a patch yet so we can see how
complex the change is, whether it breaks anything, etc.?

On Wed, May 14, 2008 at 11:36 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> I have 2 related proposals:
>
> 1. Give generators a .__name__ attribute that is the same as their curent
> (3.0a5) .gi_code.co_name subattribute.  just as funct.__name__ is
> func.__code__.co_name.
>
> My reason is, I expect, much the same as that for func.__name__.  I am
> using the generator name (for bad-iterator-output messages in a test
> function) and would prefer to get it through a cross-implementation
> 'public' interface' rather than a cPython internal  implementation detail
> (which I understand code object to be).  I am otherwise trying to avoid
> using cPython internals.
>
> (Is there any plan to change the gi_* attributes the way the func_*
> attributes were?)
>
> 2. Whether or not 1 is adopted, add the name to the representation:
>  <gfuncname generator object as..> or <generator object gfuncname at ..>
>
> Conceptually, I see a generator function as an abbreviated version of a
> iterator class, with most of the boilerplate removed, that defines a
> subclass of the generator class.  So I think the subclass name should be
> part of its representation.
>
> Terry Jan Reedy
>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From g.brandl at gmx.net  Thu May 15 16:28:51 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 15 May 2008 16:28:51 +0200
Subject: [Python-ideas] Give generators a name?
In-Reply-To: <ca471dc20805150556o34dfe521rd588a0d2a91dba82@mail.gmail.com>
References: <g0gll4$il7$1@ger.gmane.org>
	<ca471dc20805150556o34dfe521rd588a0d2a91dba82@mail.gmail.com>
Message-ID: <g0hh4p$gns$1@ger.gmane.org>

A patch suggestion is at <http://bugs.python.org/issue2863>.

Georg

Guido van Rossum schrieb:
> Sounds fine with me. Have you cooked up a patch yet so we can see how
> complex the change is, whether it breaks anything, etc.?
> 
> On Wed, May 14, 2008 at 11:36 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> I have 2 related proposals:
>>
>> 1. Give generators a .__name__ attribute that is the same as their curent
>> (3.0a5) .gi_code.co_name subattribute.  just as funct.__name__ is
>> func.__code__.co_name.
>>
>> My reason is, I expect, much the same as that for func.__name__.  I am
>> using the generator name (for bad-iterator-output messages in a test
>> function) and would prefer to get it through a cross-implementation
>> 'public' interface' rather than a cPython internal  implementation detail
>> (which I understand code object to be).  I am otherwise trying to avoid
>> using cPython internals.
>>
>> (Is there any plan to change the gi_* attributes the way the func_*
>> attributes were?)
>>
>> 2. Whether or not 1 is adopted, add the name to the representation:
>>  <gfuncname generator object as..> or <generator object gfuncname at ..>
>>
>> Conceptually, I see a generator function as an abbreviated version of a
>> iterator class, with most of the boilerplate removed, that defines a
>> subclass of the generator class.  So I think the subclass name should be
>> part of its representation.
>>
>> Terry Jan Reedy
>>
>>
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
> 
> 
> 



From amk at amk.ca  Tue May 13 19:05:48 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 13 May 2008 13:05:48 -0400
Subject: [Python-ideas] Removing some string methods?
Message-ID: <20080513170548.GA10158@amk-desktop.matrixgroup.net>

Has any consideration been given to removing some of the lesser-used
string methods in 3.0?  e.g. .center, .ljust/rjust, .zfill, .swapcase?

--amk


From brett at python.org  Fri May 16 21:29:58 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 16 May 2008 12:29:58 -0700
Subject: [Python-ideas] Removing some string methods?
In-Reply-To: <20080513170548.GA10158@amk-desktop.matrixgroup.net>
References: <20080513170548.GA10158@amk-desktop.matrixgroup.net>
Message-ID: <bbaeab100805161229g25bdeef1r86cba69e6d184277@mail.gmail.com>

On Tue, May 13, 2008 at 10:05 AM, A.M. Kuchling <amk at amk.ca> wrote:
> Has any consideration been given to removing some of the lesser-used
> string methods in 3.0?  e.g. .center, .ljust/rjust, .zfill, .swapcase?
>

Don't remember any specifically coming up. But are you sure those are
really lesser-used? I can remember using ljust, rjust, and zfill on
multiple occasions. Have you tried looking at the stdlib or something
to get usage stats?

-Brett


From python at rcn.com  Fri May 16 21:36:17 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 16 May 2008 12:36:17 -0700
Subject: [Python-ideas] Removing some string methods?
References: <20080513170548.GA10158@amk-desktop.matrixgroup.net>
Message-ID: <00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1>

[AMK]
> Has any consideration been given to removing some of the lesser-used
> string methods in 3.0?  e.g. .center, .ljust/rjust, .zfill, .swapcase?

I don't think we gain anything by taking justification and centering methods away. It just makes life difficult for people like me 
who use those
methods for formatting text.  Also, these methods have nearly zero mental overhead -- they are self-explanatory and have no learning 
curve.

The swapcase() method is more use case challenged and could probably be zapped without anyone caring.

The zfill() method is no longer necessary because rjust() takes an optional fillchar argument:

    '123'.zfill(10) == '123'.rjust(10, '0')


Raymond



From amk at amk.ca  Fri May 16 22:15:14 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 16 May 2008 16:15:14 -0400
Subject: [Python-ideas] Removing some string methods?
In-Reply-To: <00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1>
References: <20080513170548.GA10158@amk-desktop.matrixgroup.net>
	<00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1>
Message-ID: <20080516201514.GA14694@amk-desktop.matrixgroup.net>

On Fri, May 16, 2008 at 12:36:17PM -0700, Raymond Hettinger wrote:
> I don't think we gain anything by taking justification and centering 
> methods away. It just makes life difficult for people like me who use those
> methods for formatting text.  Also, these methods have nearly zero mental 
> overhead -- they are self-explanatory and have no learning curve.

Fair enough.

> The swapcase() method is more use case challenged and could probably be 
> zapped without anyone caring.

I did a Google code search, and outside of Python test suites and the
other implementations, I found exactly two uses of swapcase.

--amk


From brett at python.org  Sat May 17 07:16:55 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 16 May 2008 22:16:55 -0700
Subject: [Python-ideas] Removing some string methods?
In-Reply-To: <00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1>
References: <20080513170548.GA10158@amk-desktop.matrixgroup.net>
	<00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1>
Message-ID: <bbaeab100805162216q743839barfbd965efeb42ec0c@mail.gmail.com>

On Fri, May 16, 2008 at 12:36 PM, Raymond Hettinger <python at rcn.com> wrote:
> [AMK]
>>
>> Has any consideration been given to removing some of the lesser-used
>> string methods in 3.0?  e.g. .center, .ljust/rjust, .zfill, .swapcase?
>
> I don't think we gain anything by taking justification and centering methods
> away. It just makes life difficult for people like me who use those
> methods for formatting text.  Also, these methods have nearly zero mental
> overhead -- they are self-explanatory and have no learning curve.
>
> The swapcase() method is more use case challenged and could probably be
> zapped without anyone caring.
>
> The zfill() method is no longer necessary because rjust() takes an optional
> fillchar argument:
>
>   '123'.zfill(10) == '123'.rjust(10, '0')

Ah cool! I don't think I knew about that because I always knew about
zfill(). Probably would have found it if I zfill() didn't exist.

So for Python 3.1 we can probably safely ditch the two methods (and
probably review the methods on the other types as well).

-Brett


From greg at krypto.org  Sat May 17 07:51:27 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 16 May 2008 22:51:27 -0700
Subject: [Python-ideas] Removing some string methods?
In-Reply-To: <bbaeab100805162216q743839barfbd965efeb42ec0c@mail.gmail.com>
References: <20080513170548.GA10158@amk-desktop.matrixgroup.net>
	<00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1>
	<bbaeab100805162216q743839barfbd965efeb42ec0c@mail.gmail.com>
Message-ID: <52dc1c820805162251y493e25d7s2c1053977dd71a77@mail.gmail.com>

On Fri, May 16, 2008 at 10:16 PM, Brett Cannon <brett at python.org> wrote:
> On Fri, May 16, 2008 at 12:36 PM, Raymond Hettinger <python at rcn.com> wrote:
>> [AMK]
>>>
>>> Has any consideration been given to removing some of the lesser-used
>>> string methods in 3.0?  e.g. .center, .ljust/rjust, .zfill, .swapcase?
>>
>> I don't think we gain anything by taking justification and centering methods
>> away. It just makes life difficult for people like me who use those
>> methods for formatting text.  Also, these methods have nearly zero mental
>> overhead -- they are self-explanatory and have no learning curve.
>>
>> The swapcase() method is more use case challenged and could probably be
>> zapped without anyone caring.
>>
>> The zfill() method is no longer necessary because rjust() takes an optional
>> fillchar argument:
>>
>>   '123'.zfill(10) == '123'.rjust(10, '0')
>
> Ah cool! I don't think I knew about that because I always knew about
> zfill(). Probably would have found it if I zfill() didn't exist.
>
> So for Python 3.1 we can probably safely ditch the two methods (and
> probably review the methods on the other types as well).
>
> -Brett

zfill and rjust are not the same.  zfill treats it as a number and
moves the sign.

>>> e = b'-33'
>>> e.zfill(7)
b'-000033'
>>> e.rjust(7, '0')
b'0000-33'


From python at rcn.com  Mon May 19 05:10:45 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 18 May 2008 20:10:45 -0700
Subject: [Python-ideas] Removing some string methods?
References: <20080513170548.GA10158@amk-desktop.matrixgroup.net><00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1>
	<20080516201514.GA14694@amk-desktop.matrixgroup.net>
Message-ID: <009901c8b95d$ee9a5d10$ac00a8c0@RaymondLaptop1>

From: "A.M. Kuchling" <amk at amk.ca>
> I did a Google code search, and outside of Python test suites and the
> other implementations, I found exactly two uses of swapcase.

Looks like there is a consensus on removing swapcase().

Go ahead a zap it.


Raymond


From bmintern at gmail.com  Thu May 22 09:21:23 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Thu, 22 May 2008 03:21:23 -0400
Subject: [Python-ideas] Add kwargs to built-in function object
Message-ID: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>

I would like to propose to change the built-in function "object" to
have the following syntax:

object(**kwargs)
Return a new featureless object. object is a base for all new style
classes. It has the methods that are common to all instances of new
style classes.

If kwargs is given, the returned object's __dict__ will be kwargs (or
something to that effect).


An example:

a = object(one=1, two=2)
a.one # returns 1
a.two # returns 2


The justification:

On several occasions I have needed a collection of attributes to
represent a single item. When this happens, there are really three
options:

1. Use a tuple. This works well for temporarily re-packaging items in
a loop or for quick-and-dirty code, but it rapidly becomes unreadable
and hard to maintain. It is not long before you forget what order the
attributes are in, and at first glance, it is not clear what kind of
object is being indexed.

2. Use a dict. This is an improvement over tuples on readability, but
they can be a pain to build and overly-cumbersome to access later. I
understand that dicts are used all over the place in Python, but I
still think of them (in the general case) as a map of keys to values
where the dict represents a collection, not an object.

3. Use a class. This requires coming up with a name for the class and
then writing the class (admittedly, this should be easy). Afterwards,
this is the most convenient, readable method for representing the
data, but since it requires non-trivial effort up front, this method
may be avoided until it's truly apparent that it is necessary.


A real-world example:

Let's say that I want to have a map of employee SSNs to employee data.
I am going to be manipulating this information in various ways, but
not really in any ways that justify the use of class methods. At any
rate, let's build this data from a file where each line is

SSN  First  Last  Salary

with the items being whitespace-delimited. The code, then, will be:

employees = {}
for ssn, first, last, salary in (line.split() for line in open(employee_file)):
    employees[ssn] = (ssn, first, last, salary) # tuple method
    employees[ssn] = {"ssn": ssn, "first": first, "last": last,
"salary": salary} # dict method
    employees[ssn] = Employee(ssn, first, last, salary) # assumes
class with proper constructor

# first name of employee with SSN
employees[SSN][1] # tuple method -- quite unreadable
employees[SSN]["first"] # dict method -- more readable but sub-optimal
employees[SSN].first # class method -- readable and simple


Now, because of the advantages the class method offers in terms of
readability, I have written a convenience class that makes using it
easier:

class Record:
   """
   A class intended to provide a simple interface for creating objects
   with named fields. That is, instead of returning a tuple and indexing
   it or writing a unique class, you can simply do something like:
   a = Record(x=1, y=2)
   a.x # 1
   a.y # 2
   a   # Record(x=1, y=2)
   """
   def __init__ (self, **kwargs):
       self.__dict__.update(kwargs)

   def __repr__ (self):
       return "Record(%s)" \
              % ", ".join("%s=%s" \
                          % field for field in self.__dict__.iteritems())

Now, the line in the for loop above becomes

    employees[ssn] = Record(ssn=ssn, first=first, last=last, salary=salary)

and I have completely avoided the need to define a unique class. Note
that as the number of fields increases, this becomes easier to use
inline than the dict method, at the same time that it avoids the
upfront costs of having to build a new class for every distinct type
of object in the program.

It turns out that other people find such a method useful as well.
According to Catherine Devlin on the centralOH Python list, it is
recipe 4.18 from the Python Cookbook (2nd ed.). Several others from
the mailing list stated that they had created similar solutions
themselves.

Thus, my suggestion is to simply build such functionality directly
into the language. While scanning the built-in functions page (I don't
use any, all, or enumerate nearly enough), I noticed the object()
function, and since its purpose is to return a featureless object, it
seems to fit the bill quite well. Adding my suggested functionality
should break nothing (since the kwargs would be optional) and would
allow people to stop baking their own solutions to this common
problem, while getting the speed bonus of a C implementation (if it
helps).

Thus, the code above becomes

for...
    employees[ssn] = object(ssn=ssn, first=first, last=last, salary=salary)

employees[SSN].first


Does anyone else think this might be a good idea for Python 2.6/3K?

Thanks,
Brandon


From idadesub at users.sourceforge.net  Thu May 22 09:49:04 2008
From: idadesub at users.sourceforge.net (Erick Tryzelaar)
Date: Thu, 22 May 2008 00:49:04 -0700
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
Message-ID: <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com>

On Thu, May 22, 2008 at 12:21 AM, Brandon Mintern <bmintern at gmail.com> wrote:
> I would like to propose to change the built-in function "object" to
> have the following syntax:
>
> object(**kwargs)
> Return a new featureless object. object is a base for all new style
> classes. It has the methods that are common to all instances of new
> style classes.
>
> If kwargs is given, the returned object's __dict__ will be kwargs (or
> something to that effect).

python 2.6 and 3.0 has something similar to this with collections.namedtuple:

http://docs.python.org/dev/3.0/library/collections.html#collections.namedtuple

The interface is a bit more verbose though:

>>> Point = namedtuple('Point', 'x y')
>>> p = Point(11, y=22)     # instantiate with positional or keyword arguments
>>> p[0] + p[1]             # indexable like the plain tuple (11, 22)
33
>>> x, y = p                # unpack like a regular tuple
>>> x, y
(11, 22)
>>> p.x + p.y               # fields also accessible by name
33
>>> p                       # readable __repr__ with a name=value style
Point(x=11, y=22)

I like the syntax of using arguments to object though. Maybe there's a
discussion behind namedtuple why they went that way instead of this
one?


From adlaiff6 at gmail.com  Thu May 22 09:57:45 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Thu, 22 May 2008 03:57:45 -0400 (EDT)
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com>
Message-ID: <alpine.DEB.1.00.0805220355440.10785@lappy.toscanini-resnet.stonybrook.edu>

On Thu, 22 May 2008, Erick Tryzelaar wrote:
> python 2.6 and 3.0 has something similar to this with collections.namedtuple:
>
> http://docs.python.org/dev/3.0/library/collections.html#collections.namedtuple
>
> The interface is a bit more verbose though:
>
> >>> Point = namedtuple('Point', 'x y')
> >>> p = Point(11, y=22)     # instantiate with positional or keyword arguments
> >>> p[0] + p[1]             # indexable like the plain tuple (11, 22)
> 33
> >>> x, y = p                # unpack like a regular tuple
> >>> x, y
> (11, 22)
> >>> p.x + p.y               # fields also accessible by name
> 33
> >>> p                       # readable __repr__ with a name=value style
> Point(x=11, y=22)
>
> I like the syntax of using arguments to object though. Maybe there's a
> discussion behind namedtuple why they went that way instead of this
> one?

Just like lambdas were gotten rid of in favor of named inner functions
(in part) for readability, it sounds like readability would be a
strong argument for using namedtuple rather than some kind of an
anonymous object.

-- 
Cheers,
Leif


From bmintern at gmail.com  Thu May 22 10:41:23 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Thu, 22 May 2008 04:41:23 -0400
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <alpine.DEB.1.00.0805220355440.10785@lappy.toscanini-resnet.stonybrook.edu>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com>
	<alpine.DEB.1.00.0805220355440.10785@lappy.toscanini-resnet.stonybrook.edu>
Message-ID: <4c0fccce0805220141q4e4d278bmdf5b905778d7937a@mail.gmail.com>

> Just like lambdas were gotten rid of in favor of named inner functions
> (in part) for readability, it sounds like readability would be a
> strong argument for using namedtuple rather than some kind of an
> anonymous object.
>
> --
> Cheers,
> Leif

My only argument would be that classic usage of lambda had three main
purposes: (1) to create a function that is assigned to a variable, in
which case using def makes more sense, (2) to pass to functions like
map and filter, where the same functionality can be achieved with list
comprehensions, and (3) to pass to other functions. Getting rid of
lambda helps to eliminate the fallacy that is purpose 1, with list
comprehensions purpose 2 is less necessary, and purpose 3 is the only
real casualty. Even when you really do want to pass a lambda, it's not
very hard to say:

def f (x): return x*x

and then simply pass the f. Thus getting rid of lambda helps alleviate
a few issues while not hurting too badly.

With an anonymous object, however, I feel the case is a bit different.
You are not really using it to create an anonymous class, or even a
named tuple. You are instead using it to create an instance of object
which already has some fields defined. The same thing could be
accomplished with:

class anonobject:
    pass
a = anonobject()
a.one = 1
a.two = 2

But there is really no easier way that I can think of to do this on
the fly except to have already built a Record class as I previously
indicated. As long as this is the case, many users will be
implementing their own Record class with all the inefficiencies and
errors that come along with that. For example, my first iteration
looked like:

class Fields:
   """
   A class intended to provide a simple interface for creating objects
   with named fields. That is, instead of returning a tuple and indexing
   it or writing a unique class, you can simply do something like:
   a = Fields(x=1, y=2)
   a.x # 1
   a.y # 2
   """
   def __init__ (self, **kwargs):
       for name, value in kwargs.iteritems():
           self.__dict__[name] = value

Not horrible, but it still failed to utilize dict.update(), and the
original name was much less intuitive. Someone reading my code would
likely have to check out the Fields source to see what exactly I'm
doing.

Come to think of it, the change to object that I am suggesting could
even place the kwargs in __slots__ so that the instance it returns is
similar to that currently returned by object(), in that it is not
possible to add new fields.

All in all, though, I feel like such a change would actually *improve*
code readability in general, as we would be less likely to use a tuple
or a dict to hold what is truly an object. As soon as it becomes
apparent that our object is being used in more complicated ways, it
should be easy to write our own class with the same fields as our
original call to object and drop it in, adding a reasonable
constructor, methods, etc.


From aahz at pythoncraft.com  Thu May 22 14:18:19 2008
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 22 May 2008 05:18:19 -0700
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <alpine.DEB.1.00.0805220355440.10785@lappy.toscanini-resnet.stonybrook.edu>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com>
	<alpine.DEB.1.00.0805220355440.10785@lappy.toscanini-resnet.stonybrook.edu>
Message-ID: <20080522121819.GB9262@panix.com>

On Thu, May 22, 2008, Leif Walsh wrote:
>
> Just like lambdas were gotten rid of in favor of named inner functions
> (in part) for readability, it sounds like readability would be a
> strong argument for using namedtuple rather than some kind of an
> anonymous object.

Um, what?  lambda is still here...
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Need a book?  Use your library!


From aahz at pythoncraft.com  Thu May 22 14:22:04 2008
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 22 May 2008 05:22:04 -0700
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
Message-ID: <20080522122204.GC9262@panix.com>

On Thu, May 22, 2008, Brandon Mintern wrote:
>
> I would like to propose to change the built-in function "object" to
> have the following syntax:
> 
> object(**kwargs)
> Return a new featureless object. object is a base for all new style
> classes. It has the methods that are common to all instances of new
> style classes.

Enh.  It's easy enough to write

    class Obj:
        def __init__(self, **kwargs):
            self.__dict__.update(kwargs)

It's not clear to me that your functionality is desired frequently enough
*in that specific form* to warrant changing object().  Note particularly
the emphasis; over the years, I've seen various small variations in how
people want kwargs processed and it's not at all clear to me that
codifying one specific form into the language would be helpful.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Need a book?  Use your library!


From taleinat at gmail.com  Thu May 22 14:28:52 2008
From: taleinat at gmail.com (Tal Einat)
Date: Thu, 22 May 2008 15:28:52 +0300
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <4c0fccce0805220141q4e4d278bmdf5b905778d7937a@mail.gmail.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com>
	<alpine.DEB.1.00.0805220355440.10785@lappy.toscanini-resnet.stonybrook.edu>
	<4c0fccce0805220141q4e4d278bmdf5b905778d7937a@mail.gmail.com>
Message-ID: <7afdee2f0805220528jf5dd560q76a2fc335bd7f5c6@mail.gmail.com>

Brandon Mintern wrote:
> The same thing could be accomplished with:
>
> class anonobject:
>    pass
> a = anonobject()
> a.one = 1
> a.two = 2
>
> But there is really no easier way that I can think of to do this on
> the fly except to have already built a Record class as I previously
> indicated.

There is an "easier" way, for some definitions of "easy":
a=type('anonobject', (object,), {})() # a is an instance of class anonobject
a.one = 1
a.two = 2

Or if you want to keep it really short:
a=type('anonobject', (object,), dict(one=1, two=2)) # a is a class
named anonobject

I'll grant that this looks ugly, the intent of the code is far from
obvious. But if you don't do this too often and place appropriate
comments this should be fine. If you do find yourself doing this
often, I would say writing an appropriate class is the best solution;
it is certainly what I would have done in your example with the
employees.

- Tal


From ironfroggy at socialserve.com  Thu May 22 15:43:31 2008
From: ironfroggy at socialserve.com (Calvin Spealman)
Date: Thu, 22 May 2008 09:43:31 -0400
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
Message-ID: <E14B16CD-F154-4CD0-B9AB-85F877A2E905@socialserve.com>

Start using those and suddenly you are going to have lots of little  
anonymous objects around and you'll start to want to add  
functionality to them and otherwise enforce things about them. Now  
you have a higher barrier than when you would have been creating it  
from scratch.

and remember, object is not a function. It is a type. It is the base  
type of all types and so this has implications beyond what you  
suggest it for. If object(a=10).a == 10, then when custom classes'  
__init__ methods upcall, they'll start to fall back on letting kwargs  
fall up to object.__init__ to set initialization attributes. Now we  
gain this rule of thumb that keyword arguments are probably going to  
end up as attributes, because if we don't 'catch' them and they reach  
object.__init__, they will be. This isn't a behavior the language  
should encourage, because it should be left up to the developer.  
Currently, object.__init__ specifically takes any arguments and does  
nothing with any of them. This is to allow upcall in your __init__  
gracefully, because if the upclass is one of your bases or the final  
base, object, you can still pass your parameters along safely.  
Implying some behavior when it reaches object.__init__, especially  
(in most cases) after you've initialized your object, would probably  
just trounce all over your object and piss you off.

No thanks.

On May 22, 2008, at 3:21 AM, Brandon Mintern wrote:

> I would like to propose to change the built-in function "object" to
> have the following syntax:
>
> object(**kwargs)
> Return a new featureless object. object is a base for all new style
> classes. It has the methods that are common to all instances of new
> style classes.
>
> If kwargs is given, the returned object's __dict__ will be kwargs (or
> something to that effect).
>
>
> An example:
>
> a = object(one=1, two=2)
> a.one # returns 1
> a.two # returns 2
>
>
> The justification:
>
> On several occasions I have needed a collection of attributes to
> represent a single item. When this happens, there are really three
> options:
>
> 1. Use a tuple. This works well for temporarily re-packaging items in
> a loop or for quick-and-dirty code, but it rapidly becomes unreadable
> and hard to maintain. It is not long before you forget what order the
> attributes are in, and at first glance, it is not clear what kind of
> object is being indexed.
>
> 2. Use a dict. This is an improvement over tuples on readability, but
> they can be a pain to build and overly-cumbersome to access later. I
> understand that dicts are used all over the place in Python, but I
> still think of them (in the general case) as a map of keys to values
> where the dict represents a collection, not an object.
>
> 3. Use a class. This requires coming up with a name for the class and
> then writing the class (admittedly, this should be easy). Afterwards,
> this is the most convenient, readable method for representing the
> data, but since it requires non-trivial effort up front, this method
> may be avoided until it's truly apparent that it is necessary.
>
>
> A real-world example:
>
> Let's say that I want to have a map of employee SSNs to employee data.
> I am going to be manipulating this information in various ways, but
> not really in any ways that justify the use of class methods. At any
> rate, let's build this data from a file where each line is
>
> SSN  First  Last  Salary
>
> with the items being whitespace-delimited. The code, then, will be:
>
> employees = {}
> for ssn, first, last, salary in (line.split() for line in open 
> (employee_file)):
>     employees[ssn] = (ssn, first, last, salary) # tuple method
>     employees[ssn] = {"ssn": ssn, "first": first, "last": last,
> "salary": salary} # dict method
>     employees[ssn] = Employee(ssn, first, last, salary) # assumes
> class with proper constructor
>
> # first name of employee with SSN
> employees[SSN][1] # tuple method -- quite unreadable
> employees[SSN]["first"] # dict method -- more readable but sub-optimal
> employees[SSN].first # class method -- readable and simple
>
>
> Now, because of the advantages the class method offers in terms of
> readability, I have written a convenience class that makes using it
> easier:
>
> class Record:
>    """
>    A class intended to provide a simple interface for creating objects
>    with named fields. That is, instead of returning a tuple and  
> indexing
>    it or writing a unique class, you can simply do something like:
>    a = Record(x=1, y=2)
>    a.x # 1
>    a.y # 2
>    a   # Record(x=1, y=2)
>    """
>    def __init__ (self, **kwargs):
>        self.__dict__.update(kwargs)
>
>    def __repr__ (self):
>        return "Record(%s)" \
>               % ", ".join("%s=%s" \
>                           % field for field in  
> self.__dict__.iteritems())
>
> Now, the line in the for loop above becomes
>
>     employees[ssn] = Record(ssn=ssn, first=first, last=last,  
> salary=salary)
>
> and I have completely avoided the need to define a unique class. Note
> that as the number of fields increases, this becomes easier to use
> inline than the dict method, at the same time that it avoids the
> upfront costs of having to build a new class for every distinct type
> of object in the program.
>
> It turns out that other people find such a method useful as well.
> According to Catherine Devlin on the centralOH Python list, it is
> recipe 4.18 from the Python Cookbook (2nd ed.). Several others from
> the mailing list stated that they had created similar solutions
> themselves.
>
> Thus, my suggestion is to simply build such functionality directly
> into the language. While scanning the built-in functions page (I don't
> use any, all, or enumerate nearly enough), I noticed the object()
> function, and since its purpose is to return a featureless object, it
> seems to fit the bill quite well. Adding my suggested functionality
> should break nothing (since the kwargs would be optional) and would
> allow people to stop baking their own solutions to this common
> problem, while getting the speed bonus of a C implementation (if it
> helps).
>
> Thus, the code above becomes
>
> for...
>     employees[ssn] = object(ssn=ssn, first=first, last=last,  
> salary=salary)
>
> employees[SSN].first
>
>
> Does anyone else think this might be a good idea for Python 2.6/3K?
>
> Thanks,
> Brandon
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From bruce at leapyear.org  Thu May 22 17:36:26 2008
From: bruce at leapyear.org (Bruce Leban)
Date: Thu, 22 May 2008 08:36:26 -0700
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
Message-ID: <cf5b87740805220836v6c2b579doc506800e4cae8f3@mail.gmail.com>

On Thu, May 22, 2008 at 12:21 AM, Brandon Mintern <bmintern at gmail.com>
wrote:

> I would like to propose to change the built-in function "object" to
> have the following syntax:
>
> object(**kwargs)
> Return a new featureless object. object is a base for all new style
> classes. It has the methods that are common to all instances of new
> style classes.
>
>
I agree this is a useful pattern. One thing I don't like about it is that:
   object(x=1, y=2)
looks very much like
   object(x=1, y=2, z=3)
and I probably won't discover I've been given the wrong one until it's too
late.

I think it can better addressed by implementing NamedList and NamedDict:

NamedList(typename, fieldnames[, optionalfields[, verbose]])

Returns a new list subclass named typename. The new subclass is used to
create list-like objects that have fields accessible by attribute lookup as
well as supporting other list operations. Instances of a NamedList may be
created using a mixture of positional and keyword arguments. If
optionalfields is not true, then the NamedList must always contain at least
as many elements as the number of fields. If the NamedDict contains fewer
elements than the number of fields, missing fields return None when accessed
by attribute (a.third) and raise IndexError when accessed by index (a[3]).

NamedDict(typename, fieldnames[, optionalfields[, verbose]])

Returns a new dict subclass named typename. The new subclass is used to
create dict-like objects that have fields accessible by attribute lookup as
well as supporting other dict operations. Instances of a NamedDict may be
created using keyword arguments only. If optionalfields is not true, then
the NamedDict must have a value for every field. If a NamedDict does not
contain a field, accessing it returns None when accessed by attribute (a.x)
and raises KeyError when accessed using by key (a['x']).

--- Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20080522/885f9cba/attachment.html>

From guido at python.org  Thu May 22 19:32:14 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 22 May 2008 10:32:14 -0700
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <20080522122204.GC9262@panix.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<20080522122204.GC9262@panix.com>
Message-ID: <ca471dc20805221032y6d9478e0r722283d16fd74383@mail.gmail.com>

On Thu, May 22, 2008 at 5:22 AM, Aahz <aahz at pythoncraft.com> wrote:
> On Thu, May 22, 2008, Brandon Mintern wrote:
>>
>> I would like to propose to change the built-in function "object" to
>> have the following syntax:
>>
>> object(**kwargs)
>> Return a new featureless object. object is a base for all new style
>> classes. It has the methods that are common to all instances of new
>> style classes.
>
> Enh.  It's easy enough to write
>
>    class Obj:
>        def __init__(self, **kwargs):
>            self.__dict__.update(kwargs)
>
> It's not clear to me that your functionality is desired frequently enough
> *in that specific form* to warrant changing object().  Note particularly
> the emphasis; over the years, I've seen various small variations in how
> people want kwargs processed and it's not at all clear to me that
> codifying one specific form into the language would be helpful.

This is the answer that came to my mind.

Also note that not all classes give their instances a __dict__, and
object (being the simplest object possible) does not -- so
object(foo=1) wold not have a place where to store the 'foo': 1
mapping.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From adlaiff6 at gmail.com  Thu May 22 20:34:27 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Thu, 22 May 2008 14:34:27 -0400 (EDT)
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <20080522121819.GB9262@panix.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com>
	<alpine.DEB.1.00.0805220355440.10785@lappy.toscanini-resnet.stonybrook.edu>
	<20080522121819.GB9262@panix.com>
Message-ID: <alpine.DEB.1.00.0805221433480.10785@lappy.toscanini-resnet.stonybrook.edu>

On Thu, 22 May 2008, Aahz wrote:
> Um, what?  lambda is still here...

They are leaving in python 3.0 (and maybe 2.6, I forget), my mistake.

-- 
Cheers,
Leif


From adlaiff6 at gmail.com  Thu May 22 20:47:01 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Thu, 22 May 2008 14:47:01 -0400 (EDT)
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <cf5b87740805220836v6c2b579doc506800e4cae8f3@mail.gmail.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<cf5b87740805220836v6c2b579doc506800e4cae8f3@mail.gmail.com>
Message-ID: <alpine.DEB.1.00.0805221438370.10785@lappy.toscanini-resnet.stonybrook.edu>

On Thu, 22 May 2008, Bruce Leban wrote:
> I think it can better addressed by implementing NamedList and NamedDict:

Oh no...

> NamedList(typename, fieldnames[, optionalfields[, verbose]])
>
> Returns a new list subclass named typename. The new subclass is used to
> create list-like objects that have fields accessible by attribute lookup as
> well as supporting other list operations. Instances of a NamedList may be
> created using a mixture of positional and keyword arguments. If
> optionalfields is not true, then the NamedList must always contain at least
> as many elements as the number of fields. If the NamedDict contains fewer
> elements than the number of fields, missing fields return None when accessed
> by attribute (a.third) and raise IndexError when accessed by index (a[3]).

Is this just a variable-length version of NamedTuple?  It doesn't seem
to offer very much over NamedTuple to me, but I'm open to convincing.

> NamedDict(typename, fieldnames[, optionalfields[, verbose]])
>
> Returns a new dict subclass named typename. The new subclass is used to
> create dict-like objects that have fields accessible by attribute lookup as
> well as supporting other dict operations. Instances of a NamedDict may be
> created using keyword arguments only. If optionalfields is not true, then
> the NamedDict must have a value for every field. If a NamedDict does not
> contain a field, accessing it returns None when accessed by attribute (a.x)
> and raises KeyError when accessed using by key (a['x']).

This one, though, doesn't sound any different from a struct-style
class, except that it's easier (by ".__dict__") to access the class as
a dictionary.  Hell, here you go:

>>> class NamedDict(object):
>>>   def __getitem__(self, key):
>>>     return self.__dict__[key]

-- 
Cheers,
Leif


From george.sakkis at gmail.com  Thu May 22 21:04:10 2008
From: george.sakkis at gmail.com (George Sakkis)
Date: Thu, 22 May 2008 15:04:10 -0400
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <alpine.DEB.1.00.0805221433480.10785@lappy.toscanini-resnet.stonybrook.edu>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com>
	<alpine.DEB.1.00.0805220355440.10785@lappy.toscanini-resnet.stonybrook.edu>
	<20080522121819.GB9262@panix.com>
	<alpine.DEB.1.00.0805221433480.10785@lappy.toscanini-resnet.stonybrook.edu>
Message-ID: <91ad5bf80805221204x43ba4046g33236c24a23f967c@mail.gmail.com>

On Thu, May 22, 2008 at 2:34 PM, Leif Walsh <adlaiff6 at gmail.com> wrote:

> On Thu, 22 May 2008, Aahz wrote:
>> Um, what?  lambda is still here...
>
> They are leaving in python 3.0 (and maybe 2.6, I forget), my mistake.

Wrong again, they are not leaving at all.

George


From bruce at leapyear.org  Thu May 22 21:14:34 2008
From: bruce at leapyear.org (Bruce Leban)
Date: Thu, 22 May 2008 12:14:34 -0700
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <alpine.DEB.1.00.0805221438370.10785@lappy.toscanini-resnet.stonybrook.edu>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<cf5b87740805220836v6c2b579doc506800e4cae8f3@mail.gmail.com>
	<alpine.DEB.1.00.0805221438370.10785@lappy.toscanini-resnet.stonybrook.edu>
Message-ID: <cf5b87740805221214mfb24f8cu5e29d8d0846325f6@mail.gmail.com>

On Thu, May 22, 2008 at 11:47 AM, Leif Walsh <adlaiff6 at gmail.com> wrote:

> On Thu, 22 May 2008, Bruce Leban wrote:
> > I think it can better addressed by implementing NamedList and NamedDict:
>
> Oh no...
>
> > NamedList(typename, fieldnames[, optionalfields[, verbose]])
> >
> > Returns a new list subclass named typename. The new subclass is used to
> > create list-like objects that have fields accessible by attribute lookup
> as
> > well as supporting other list operations. Instances of a NamedList may be
> > created using a mixture of positional and keyword arguments. If
> > optionalfields is not true, then the NamedList must always contain at
> least
> > as many elements as the number of fields. If the NamedDict contains fewer
> > elements than the number of fields, missing fields return None when
> accessed
> > by attribute (a.third) and raise IndexError when accessed by index
> (a[3]).
>
> Is this just a variable-length version of NamedTuple?  It doesn't seem
> to offer very much over NamedTuple to me, but I'm open to convincing.
>

Tuples are immutable. Lists are mutable. The reason I suggest making it
variable length is so that it supports the full list semantics and you could
for example create a NamedList object with no values and then add values to
it (ditto for NamedDict).


>
> > NamedDict(typename, fieldnames[, optionalfields[, verbose]])
> >
> > Returns a new dict subclass named typename. The new subclass is used to
> > create dict-like objects that have fields accessible by attribute lookup
> as
> > well as supporting other dict operations. Instances of a NamedDict may be
> > created using keyword arguments only. If optionalfields is not true, then
> > the NamedDict must have a value for every field. If a NamedDict does not
> > contain a field, accessing it returns None when accessed by attribute
> (a.x)
> > and raises KeyError when accessed using by key (a['x']).
>
> This one, though, doesn't sound any different from a struct-style
> class, except that it's easier (by ".__dict__") to access the class as
> a dictionary.  Hell, here you go:
>
> >>> class NamedDict(object):
> >>>   def __getitem__(self, key):
> >>>     return self.__dict__[key]
>

Well, I think the "Named" aspect is important. For example, repr(x) tells
you the name. I can tell if two objects are the same type.

Also the NamedDict supports attribute access to the specific fields you've
declared and no others. So it's not a freewheeling object that you can add
any attribute to it. It's just a lightweight way to create an object backed
by a dict (just as NamedTuple is).


> --
> Cheers,
> Leif
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20080522/365d5073/attachment.html>

From adlaiff6 at gmail.com  Thu May 22 21:30:47 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Thu, 22 May 2008 15:30:47 -0400 (EDT)
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <cf5b87740805221214mfb24f8cu5e29d8d0846325f6@mail.gmail.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<cf5b87740805220836v6c2b579doc506800e4cae8f3@mail.gmail.com>
	<alpine.DEB.1.00.0805221438370.10785@lappy.toscanini-resnet.stonybrook.edu>
	<cf5b87740805221214mfb24f8cu5e29d8d0846325f6@mail.gmail.com>
Message-ID: <alpine.DEB.1.00.0805221521390.10785@lappy.toscanini-resnet.stonybrook.edu>

On Thu, 22 May 2008, Bruce Leban wrote:
> Tuples are immutable. Lists are mutable. The reason I suggest making it
> variable length is so that it supports the full list semantics and you could
> for example create a NamedList object with no values and then add values to
> it (ditto for NamedDict).

I might be wrong here, but I think one of the reasons NamedTuple was
decided upon (its immutability, specifically) was a performance issue.
It's a lot easier to optimize a struct when you know exactly what its
fields are.  I think making a mutable one of these is really not
offering all that much advantage to anyone in a large number of cases.

> Well, I think the "Named" aspect is important. For example, repr(x) tells
> you the name. I can tell if two objects are the same type.

Well, right.  That's the point of a struct-style class.  You name it
whatever you would like.

> Also the NamedDict supports attribute access to the specific fields you've
> declared and no others. So it's not a freewheeling object that you can add
> any attribute to it. It's just a lightweight way to create an object backed
> by a dict (just as NamedTuple is).

I'm confused.  I thought you said you wanted mutability.  Now you can
only have the fieldnames you specify in the beginning?  This is not
really how a dict usually operates.

That said, you can mess with __slots__ all you want in your free time,
but I think we're straying pretty far from "one obvious way" here.

-- 
Cheers,
Leif


From bmintern at gmail.com  Thu May 22 22:01:49 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Thu, 22 May 2008 16:01:49 -0400
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <E14B16CD-F154-4CD0-B9AB-85F877A2E905@socialserve.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<E14B16CD-F154-4CD0-B9AB-85F877A2E905@socialserve.com>
Message-ID: <4c0fccce0805221301g57bd9b8ai27554530319a6484@mail.gmail.com>

A lot to address here.

On Thu, May 22, 2008 at 9:43 AM, Calvin Spealman
<ironfroggy at socialserve.com> wrote:
> Start using those and suddenly you are going to have lots of little
> anonymous objects around and you'll start to want to add functionality to
> them and otherwise enforce things about them. Now you have a higher barrier
> than when you would have been creating it from scratch.

That wasn't my intended use case, though. I wouldn't be using an
anonymous object in more than one place to create the same type of
object. In that case, actually defining a class or using NamedTuple
(or some equivalent) clearly wins out. This would be a similar
situation to that of lambda. You would not use lambda in several
different places to define the same function; as soon as you start to
write it a second time, you'd be much better to def the function and
use it in both places.

My main use case is in packing related information together for
convenience and readability when passing it around to several
different functions. In other words, the exact case where someone
might be tempted to use a tuple (in spite of poor readability later)
or a dict (in spite of a cumbersome access pattern). If you're using
it for a case like this and you later decide that you should expand
the anonymous object to a full-blown class, you would only have to
change it in one place. Moreover, it would be much easier than
changing a tuple or dict to a class. In the anonymous object
situation, you only have to change the one instantiation; as long as
you create a class with the same field names, your other accesses are
still valid. In the tuple or dict cases, however, you have to change
the code everywhere that you are using the object. Thus, I feel that
having an anonymous object available actually lowers the barrier to
making smart changes to the code.

> and remember, object is not a function. It is a type. It is the base type of
> all types and so this has implications beyond what you suggest it for. If
> object(a=10).a == 10, then when custom classes' __init__ methods upcall,
> they'll start to fall back on letting kwargs fall up to object.__init__ to
> set initialization attributes. Now we gain this rule of thumb that keyword
> arguments are probably going to end up as attributes, because if we don't
> 'catch' them and they reach object.__init__, they will be. This isn't a
> behavior the language should encourage, because it should be left up to the
> developer. Currently, object.__init__ specifically takes any arguments and
> does nothing with any of them. This is to allow upcall in your __init__
> gracefully, because if the upclass is one of your bases or the final base,
> object, you can still pass your parameters along safely. Implying some
> behavior when it reaches object.__init__, especially (in most cases) after
> you've initialized your object, would probably just trounce all over your
> object and piss you off.

This is a very good argument for why object() should not be used for
this purpose; you have completely changed my mind. I still think there
would be value in having a built-in function for declaring such
objects. Perhaps a function (constructor) named struct(**kwargs) would
be appropriate?

On Thu, May 22, 2008 at 8:22 AM, Aahz <aahz at pythoncraft.com> wrote:
> Enh.  It's easy enough to write
>
>    class Obj:
>        def __init__(self, **kwargs):
>            self.__dict__.update(kwargs)

Yes, it is easy enough, once you have worked with Python a lot.
Personally, I didn't know much at all about __dict__ or **kwargs when
I first started, and looking back at my old code now, there are
several places where I have used tuples or dicts and the code is hard
to manage. Having a struct type to hold related information which does
not justify a full class would have been very nice.

> It's not clear to me that your functionality is desired frequently enough
> *in that specific form* to warrant changing object().  Note particularly
> the emphasis; over the years, I've seen various small variations in how
> people want kwargs processed and it's not at all clear to me that
> codifying one specific form into the language would be helpful.
> No thanks.

This is another good argument for why object() should not be used for
this purpose. I still think that adding a new type "struct" would have
several benefits.

Others wrote:
> NamedList, NamedDict, NamedTuple...

I agree that these are very useful functions to have when what you
really want to do is just subclass the indicated type. For the use
case I have in mind, however, they have much more functionality (and
take up more space?) than necessary, and specifically having to decide
on and declare a name that will never be used constitutes wasted
effort.

I will be sending out a new reply with my updated proposal, using the
name "struct" instead of multi-purposing object, as that is clearly a
bad idea.

Thanks for all the constructive criticism and feedback,
Brandon


From bmintern at gmail.com  Thu May 22 23:42:28 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Thu, 22 May 2008 17:42:28 -0400
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs
	to built-in function object)
Message-ID: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>

This is a proposal to add a new built-in named struct:

struct(**kwargs)
    Return a struct object which has the attributes given in kwargs.

The name is really unimportant, and I'm open to other ideas, but I
feel that many people have a good idea of what a struct is, and
accessing an object returned from struct would look identical to the
access of a struct in C, so it seems appropriate to me.


The rationale:

It is often helpful to package related information together in order
to make passing the information to various functions more convenient.
The easiest ways (currently) to package such information is to put it
into a tuple (the easiest) or a dict.

Putting the information into a tuple may be easy initially, but it has
high costs later on as it becomes hard to remember what order the
information is in, and to someone reading the code, the intent of the
code is far from clear.

Putting the information in a dict is a bit harder than a tuple, but
since it is more readable later on than a tuple, this method is often
used. Still, the access pattern is more cumbersome than it could be;
foo["bar"] is more cumbersome than, say, foo.bar. This is especially
the case if you have a dict or list of foos, where you then have to
use foos[i]["bar"].

Both tuple and dict solutions suffer down the line when the
information gets to be complicated enough to warrant a class of its
own. It involves changing both the spot in the code where the
information is created (to use the new class constructor), as well as
changing every single field access in the code (changing every foo[0]
or foo["bar"] to foo.bar).

An alternative is to use NamedTuple, NamedDict, NamedList, or to
create your own class. As long as these are more complicated to use
than a tuple or a dict, however, they are not likely to be used for
this purpose. Another problem is that all of these methods require you
to go to the trouble of thinking of a name for your class, and if you
later decide to add more information to your packaged object, you have
to make two changes (in the list of attributes / constructor and in
the place where you instantiate your object).

Enter struct. Using struct is intended to be just as easy as using a
dict (actually, easier when the number of fields is more than two or
three), and not much harder than using a tuple. To declare a struct
foo with attribute bar, you simply use:

foo = struct(bar="barvalue")

Access becomes very easy and readable:

foo.bar

Adding new fields is as easy as changing the initial instantiation, in
one place:

foo = struct(bar="barvalue", baz="bazvalue")

Later on down the line, when you decide that you are doing more with
foo than a struct should be doing, you can easily define a class Foo
which inherits from struct, and since accesses to foo already look
like foo.bar, you only have one spot in the code to change:

foo = Foo(bar="barvalue", baz="bazvalue")

and the rest "just works" with no changes.


The implementation:

class struct (object):

    def __init__ (self, **kwargs):
        self.__dict__.update(kwargs)

    def __repr__ (self):
        """
        Using self.__class__.__name__ allows classes to inherit from
        struct and automatically have a nice __repr__ method.
        """
        return "%s(%s)" % (self.__class__.__name__,
                           ", ".join("%s=%s" % (attr, repr(val))
                                     for attr, val
                                     in self.__dict__.iteritems()),)
                                                # or .items()
                                                # in Python 3K

    def __str__ (self):
        return self.__repr__()

    def __eq__ (self, other):
        """
        Implements comparison operation mirroring that of a C struct.
        """
        return self.__dict__ == other.__dict__

    def __ne__ (self, other):
        """
        See note for __eq__.
        """
        return not self.__eq__(other)

    def __setattr__ (self, name, value):
        """
        I think it makes the most sense for a struct to have immutable
        fields. As soon as you start to add more fields, you should be
        using something other than a struct.
        """
        if name in self.__dict__:
            self.__dict__[name] = value
        else:
            raise(AttributeError("'struct' object has no attribute '%s'" \
                                 % (name,)))

    def __len__ (self):
        """
        I'm not sure that it's really necessary to include this, but I
        could see where it might be helpful in some instances.
        """
        return len(self.__dict__)

    def __iter__ (self):
        """
        See note for __len__
        """
        return self.__dict__.itervalues()
                       # or .values() in Python 3K


Sample usage:

>>> a = struct(one=1, two=2)
>>> a
struct(two=2, one=1)
>>> a.one
1
>>> a.two
2
>>> a.three
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'struct' object has no attribute 'three'
>>> a.one = "one"
>>> a.one
'one'
>>> a.three = 3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "struct.py", line 39, in __setattr__
    % (name,)))
AttributeError: 'struct' object has no attribute 'three'
>>> b = struct(one=1, two=2)
>>> b
struct(two=2, one=1)
>>> a == b
False
>>> a.one = 1
>>> a == b
True
>>> len(a)
2
>>> print ", ".join(str(v) for v in a)
2, 1
>>> 1 in a
True
>>> "one" in a
False


Ideas or feedback, anyone?


From bmintern at gmail.com  Thu May 22 23:55:35 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Thu, 22 May 2008 17:55:35 -0400
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
	kwargs to built-in function object)
In-Reply-To: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
Message-ID: <4c0fccce0805221455r5a6204aj6400ed5dd0b5b867@mail.gmail.com>

On Thu, May 22, 2008 at 5:42 PM, Brandon Mintern <bmintern at gmail.com> wrote:
> This is a proposal to add a new built-in named struct:

One thing I forgot to mention... this is mainly intended to be used
where it makes sense to quickly build an anonymous object. It would be
considered bad practice to have more than one place in the code which
creates a struct having the same fields; in that case a NamedTuple (or
some equivalent) would be more appropriate.

In other words, this would be used in a similar situation to that of
lambda. You would not use lambda in several different places to define
the same function; as soon as you start to write it a second time,
you'd be much better to def the function and use it in both places. In
the same way, creating the same struct in two places is a good
indication that a NamedTuple or an actual class (possibly inheriting
from struct and using a non-kwargs constructor) is more appropriate.


From brett at python.org  Fri May 23 00:10:02 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 22 May 2008 15:10:02 -0700
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
	kwargs to built-in function object)
In-Reply-To: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
Message-ID: <bbaeab100805221510s7f8e9aadib88f4ae33899ba37@mail.gmail.com>

On Thu, May 22, 2008 at 2:42 PM, Brandon Mintern <bmintern at gmail.com> wrote:
> This is a proposal to add a new built-in named struct:
>
> struct(**kwargs)
>    Return a struct object which has the attributes given in kwargs.
>
> The name is really unimportant, and I'm open to other ideas, but I
> feel that many people have a good idea of what a struct is, and
> accessing an object returned from struct would look identical to the
> access of a struct in C, so it seems appropriate to me.
>
>
> The rationale:
>
> It is often helpful to package related information together in order
> to make passing the information to various functions more convenient.
> The easiest ways (currently) to package such information is to put it
> into a tuple (the easiest) or a dict.
>
> Putting the information into a tuple may be easy initially, but it has
> high costs later on as it becomes hard to remember what order the
> information is in, and to someone reading the code, the intent of the
> code is far from clear.
>
> Putting the information in a dict is a bit harder than a tuple, but
> since it is more readable later on than a tuple, this method is often
> used. Still, the access pattern is more cumbersome than it could be;
> foo["bar"] is more cumbersome than, say, foo.bar. This is especially
> the case if you have a dict or list of foos, where you then have to
> use foos[i]["bar"].
>

So you save three characters? I don't call that cumbersome.

> Both tuple and dict solutions suffer down the line when the
> information gets to be complicated enough to warrant a class of its
> own. It involves changing both the spot in the code where the
> information is created (to use the new class constructor), as well as
> changing every single field access in the code (changing every foo[0]
> or foo["bar"] to foo.bar).
>
> An alternative is to use NamedTuple, NamedDict, NamedList, or to
> create your own class. As long as these are more complicated to use
> than a tuple or a dict, however, they are not likely to be used for
> this purpose. Another problem is that all of these methods require you
> to go to the trouble of thinking of a name for your class, and if you
> later decide to add more information to your packaged object, you have
> to make two changes (in the list of attributes / constructor and in
> the place where you instantiate your object).
>

Thinking of a name for your class is not difficult, especially if you
keep it private to the module, class, function, etc.

This does not strike me as useful enough to have as a built-in. It
would be better placed in the stdlib.

-Brett


From bmintern at gmail.com  Fri May 23 00:24:19 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Thu, 22 May 2008 18:24:19 -0400
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
	kwargs to built-in function object)
In-Reply-To: <bbaeab100805221510s7f8e9aadib88f4ae33899ba37@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<bbaeab100805221510s7f8e9aadib88f4ae33899ba37@mail.gmail.com>
Message-ID: <4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com>

On Thu, May 22, 2008 at 6:10 PM, Brett Cannon <brett at python.org> wrote:
> So you save three characters? I don't call that cumbersome.

If foo["bar"] is not cumbersome, it is at least less elegant and the
intent is less clear than foo.bar. Moreover, as I stated in the next
paragraph, it does become cumbersome down the line when you decide
that you should have used a class after all, and now you have to
change all of those foo["bar"] lines to foo.bar. Note that simple
search-and-replace wouldn't help if you are passing foo to various
functions.

> Thinking of a name for your class is not difficult, especially if you
> keep it private to the module, class, function, etc.

It may not be difficult, but when the name is unnecessary, simply
needing to declare it seems silly.

> This does not strike me as useful enough to have as a built-in. It
> would be better placed in the stdlib.

I would be happy with it at least becoming part of collections or some
other module, but then I wonder how many new-ish Python programmers
would persist in using a tuple or a dict instead of a more elegant
struct solution for lack of knowing about it. At least if it was in
Python somewhere, though, searching for "python struct" would be more
likely to return what the programmer is looking for.

Ouch... it seems that struct is already the name of a module. If
enough people like my idea, perhaps that module could be renamed to
"cstruct". Then again, if my idea did become a part of collections
(rather than a built-in), collections.struct and the struct module
would be able to co-exist, albeit somewhat confusingly.

Brandon


From bmintern at gmail.com  Fri May 23 00:53:31 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Thu, 22 May 2008 18:53:31 -0400
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
	kwargs to built-in function object)
In-Reply-To: <3b5110850805221525k6237c647u38d65a78a14fedf3@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<3b5110850805221525k6237c647u38d65a78a14fedf3@mail.gmail.com>
Message-ID: <4c0fccce0805221553s77d7d047r995a6222d39bd2ba@mail.gmail.com>

On Thu, May 22, 2008 at 6:25 PM, Matthew Russell
<matt.horizon5 at gmail.com> wrote:
> How about:
>   collections.attrs(**kwargs)
> or
>   collections.record(**kwargs)

I've used the name "Record" in the past, but when I was trying to
figure out if Python had anything like a C struct, I searched
specifically for "python struct". I'm not sure how many others do the
same, but perhaps using a name like "structure" would be better than
"record"?

(On a side note, see what I mean about the work involved in naming a class? :-)

> I think I prefer the idea of  this being a factory that returns an anonymous
> object as opposed to a class:
>
> def  record(**kwargs):
>     class Record(object):
>           def __init__(self, **kw):
>               for (k, v) in kwargs.items():
>                   setattr(self, k, v)
>     return Record()

I'm guessing that the last line was meant to be "return Record(**kwargs)"

> This is mainly because a class you later substitude for the
> result of record(**kwargs) may later end up wanting to take kwargs as
> optional values which you might not want as instance variables, or even
> posistional arguements.
>
> x = MyThing(x=1,y=2,  default_spam=False)
>
>
> So you could easilty do:
>
> x = MyThing(default_spam=False, **record_obj.__dict__)
>
> Matt

I can see where you're coming from, but this change would mean that
you would no longer be able to inherit from record (or structure, or
whatever). Also, if you're making that many changes to the class, it
wouldn't be too hard to rewrite the initial instantiation to not use
positional parameters with or without kwargs (after rewriting the
constructor, which you would clearly be doing, anyways). Another
caveat is that each call to record() will create an object of a
different class. Thus, if you have the following:

employees = {}
for ssn, fn, ln in employee_data_list:
    employees[ssn] = record(ssn=ssn, first=fn, last=ln)

Then employees[ssn1].__class__ != employees[ssn2].__class__. This
seems like it could be an issue at some point.

Brandon


From brett at python.org  Fri May 23 01:00:11 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 22 May 2008 16:00:11 -0700
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
	kwargs to built-in function object)
In-Reply-To: <4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<bbaeab100805221510s7f8e9aadib88f4ae33899ba37@mail.gmail.com>
	<4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com>
Message-ID: <bbaeab100805221600u60ecaf1cv49aad793203efc73@mail.gmail.com>

On Thu, May 22, 2008 at 3:24 PM, Brandon Mintern <bmintern at gmail.com> wrote:
> On Thu, May 22, 2008 at 6:10 PM, Brett Cannon <brett at python.org> wrote:
>> So you save three characters? I don't call that cumbersome.
>
> If foo["bar"] is not cumbersome, it is at least less elegant and the
> intent is less clear than foo.bar. Moreover, as I stated in the next
> paragraph, it does become cumbersome down the line when you decide
> that you should have used a class after all, and now you have to
> change all of those foo["bar"] lines to foo.bar. Note that simple
> search-and-replace wouldn't help if you are passing foo to various
> functions.
>

But you are suggesting that people think far enough ahead to think
that a sequence or mapping will be cumbersome and thus something with
attribute access should be used instead.

>> Thinking of a name for your class is not difficult, especially if you
>> keep it private to the module, class, function, etc.
>
> It may not be difficult, but when the name is unnecessary, simply
> needing to declare it seems silly.
>

Well, we almost ditched lambda and were going to require people to
define a simple function to replace lambda functions, so not everyone
thinks it is silly.

>> This does not strike me as useful enough to have as a built-in. It
>> would be better placed in the stdlib.
>
> I would be happy with it at least becoming part of collections or some
> other module, but then I wonder how many new-ish Python programmers
> would persist in using a tuple or a dict instead of a more elegant
> struct solution for lack of knowing about it. At least if it was in
> Python somewhere, though, searching for "python struct" would be more
> likely to return what the programmer is looking for.
>

Sticking something in built-ins so that it is easier for newbies to
find it is not a good argument. Things only go into builtins if they
are frequently used and warrant skipping an import statement.

> Ouch... it seems that struct is already the name of a module. If
> enough people like my idea, perhaps that module could be renamed to
> "cstruct". Then again, if my idea did become a part of collections
> (rather than a built-in), collections.struct and the struct module
> would be able to co-exist, albeit somewhat confusingly.
>

I don't agree with that worry. re.compile() exists and people don't
worry about conflicting with the built-in function. An import
statement makes it clear what object 'struct' maps to in the
namespace.

-Brett


From greg.ewing at canterbury.ac.nz  Fri May 23 01:03:00 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 23 May 2008 11:03:00 +1200
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
 kwargs	to built-in function object)
In-Reply-To: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
Message-ID: <4835FBA4.8020206@canterbury.ac.nz>

Brandon Mintern wrote:
> This is a proposal to add a new built-in named struct:
> 
> struct(**kwargs)
>     Return a struct object which has the attributes given in kwargs.

I think I'd prefer 'record', to avoid any potential
confusion with the struct module, which does something
quite different.

Also, my Pascal background makes the term 'record'
seem more high-level and therefore Pythonic to me.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri May 23 01:07:54 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 23 May 2008 11:07:54 +1200
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
 kwargs to built-in function object)
In-Reply-To: <bbaeab100805221600u60ecaf1cv49aad793203efc73@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<bbaeab100805221510s7f8e9aadib88f4ae33899ba37@mail.gmail.com>
	<4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com>
	<bbaeab100805221600u60ecaf1cv49aad793203efc73@mail.gmail.com>
Message-ID: <4835FCCA.7020001@canterbury.ac.nz>

Brett Cannon wrote:

> I don't agree with that worry. re.compile() exists and people don't
> worry about conflicting with the built-in function.

But that's just a matter of two functions with the same
name in different places. It's not a case of something
being a function or type in one place and a module in
another. That would be more confusing, I think.

-- 
Greg


From adlaiff6 at gmail.com  Fri May 23 01:15:08 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Thu, 22 May 2008 19:15:08 -0400 (EDT)
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
 kwargs to built-in function object)
In-Reply-To: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
Message-ID: <alpine.DEB.1.00.0805221913001.10785@lappy.toscanini-resnet.stonybrook.edu>

On Thu, 22 May 2008, Brandon Mintern wrote:
> The implementation:
>
> class struct (object):
>
>     <things here>
>
>     def __setattr__ (self, name, value):
>         """
>         I think it makes the most sense for a struct to have immutable
>         fields. As soon as you start to add more fields, you should be
>         using something other than a struct.
>         """
>         if name in self.__dict__:
>             self.__dict__[name] = value
>         else:
>             raise(AttributeError("'struct' object has no attribute '%s'" \
>                                  % (name,)))

I think it makes the most sense, if this construct is adopted, to use
__slots__ to control mutability.  Someone more well-versed in the
python object model should determine if this is actually a good idea.

-- 
Cheers,
Leif


From bmintern at gmail.com  Fri May 23 01:17:16 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Thu, 22 May 2008 19:17:16 -0400
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
	kwargs to built-in function object)
In-Reply-To: <bbaeab100805221600u60ecaf1cv49aad793203efc73@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<bbaeab100805221510s7f8e9aadib88f4ae33899ba37@mail.gmail.com>
	<4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com>
	<bbaeab100805221600u60ecaf1cv49aad793203efc73@mail.gmail.com>
Message-ID: <4c0fccce0805221617v37ec425akfc8505b9d6ffbd7f@mail.gmail.com>

On Thu, May 22, 2008 at 7:00 PM, Brett Cannon <brett at python.org> wrote:
>> If foo["bar"] is not cumbersome, it is at least less elegant and the
>> intent is less clear than foo.bar. Moreover, as I stated in the next
>> paragraph, it does become cumbersome down the line when you decide
>> that you should have used a class after all, and now you have to
>> change all of those foo["bar"] lines to foo.bar. Note that simple
>> search-and-replace wouldn't help if you are passing foo to various
>> functions.
>>
>
> But you are suggesting that people think far enough ahead to think
> that a sequence or mapping will be cumbersome and thus something with
> attribute access should be used instead.

Perhaps a lot of people wouldn't, unless the tutorial reflected the
idea of using struct for anonymous, structured data. When I started
out, I initially settled on tuples because it was the easiest to build
up front, and I didn't see any better alternatives. After the
frustration of maintaining tuple-based structures, I began to use
dicts instead, but I didn't like the syntax all that much, and it
looks weird (in my opinion) to have code that declares and indexes a
dictionary when what you really want is something like a C struct.

I guess what I'm suggesting is that if a clear solution is presented
that fits the use case well, I would expect a nontrivial number of
people to use it. They would not be required to notice that it is then
easy to convert to a real class, because good practice is "enforced"
by the use of struct.

>>> Thinking of a name for your class is not difficult, especially if you
>>> keep it private to the module, class, function, etc.
>>
>> It may not be difficult, but when the name is unnecessary, simply
>> needing to declare it seems silly.
>>
>
> Well, we almost ditched lambda and were going to require people to
> define a simple function to replace lambda functions, so not everyone
> thinks it is silly.

I'm sorry to keep hammering on this issue, but anonymous functions
already have standard names from Math: f, g, h, fn, func, etc. In
other words,

def f (x, y): return x+y
f

is not much harder than

lambda x,y: x+y

(except for the fact that it cannot be used as an expression, which is
why I personally still like lambda).

With class names, there is no such convention. Therefore, something like

c = NamedTuple("C", "one two")
a = c(1, 2)

seems a bit strange. Why "c" and "C"? Why anything? Why should I have
to include "C" as an argument only to never use it? I think that

a = struct(one=1, two=2)

clearly beats out the NamedTuple solution if all you really need is a struct.

>>> This does not strike me as useful enough to have as a built-in. It
>>> would be better placed in the stdlib.
>>
>> I would be happy with it at least becoming part of collections or some
>> other module, but then I wonder how many new-ish Python programmers
>> would persist in using a tuple or a dict instead of a more elegant
>> struct solution for lack of knowing about it. At least if it was in
>> Python somewhere, though, searching for "python struct" would be more
>> likely to return what the programmer is looking for.
>>
>
> Sticking something in built-ins so that it is easier for newbies to
> find it is not a good argument. Things only go into builtins if they
> are frequently used and warrant skipping an import statement.

Fair enough. It would make sense to initially put it into collections,
make sure people know about it, and then see how often it's used.

>> Ouch... it seems that struct is already the name of a module. If
>> enough people like my idea, perhaps that module could be renamed to
>> "cstruct". Then again, if my idea did become a part of collections
>> (rather than a built-in), collections.struct and the struct module
>> would be able to co-exist, albeit somewhat confusingly.
>>
>
> I don't agree with that worry. re.compile() exists and people don't
> worry about conflicting with the built-in function. An import
> statement makes it clear what object 'struct' maps to in the
> namespace.

Good, because I much-prefer using the name "struct" since it maps so
closely to a C struct.

Brandon


From bmintern at gmail.com  Fri May 23 01:22:24 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Thu, 22 May 2008 19:22:24 -0400
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
	kwargs to built-in function object)
In-Reply-To: <alpine.DEB.1.00.0805221913001.10785@lappy.toscanini-resnet.stonybrook.edu>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<alpine.DEB.1.00.0805221913001.10785@lappy.toscanini-resnet.stonybrook.edu>
Message-ID: <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com>

On Thu, May 22, 2008 at 7:15 PM, Leif Walsh <adlaiff6 at gmail.com> wrote:
> On Thu, 22 May 2008, Brandon Mintern wrote:
>> The implementation:
>>
>> class struct (object):
>>
>>     <things here>
>>
>>     def __setattr__ (self, name, value):
>>         """
>>         I think it makes the most sense for a struct to have immutable
>>         fields. As soon as you start to add more fields, you should be
>>         using something other than a struct.
>>         """
>>         if name in self.__dict__:
>>             self.__dict__[name] = value
>>         else:
>>             raise(AttributeError("'struct' object has no attribute '%s'" \
>>                                  % (name,)))
>
> I think it makes the most sense, if this construct is adopted, to use
> __slots__ to control mutability.  Someone more well-versed in the
> python object model should determine if this is actually a good idea.
>
> --
> Cheers,
> Leif

Whoops... I should have said "An implementation." I intended the
implementation to be a specification for behavior and not the
one-and-only implementation. Personally, I'm not especially familiar
with using __slots__ in anything but standard usage, so I found it
easier to show the code using __dict__. Certainly if __slots__ can be
used, it avoids the need to write __setattr__ and to explicitly raise
the AttributeError exception (which I already see is wrong because it
doesn't support inheritance).


From adlaiff6 at gmail.com  Fri May 23 01:49:05 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Thu, 22 May 2008 19:49:05 -0400 (EDT)
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
 kwargs to built-in function object)
In-Reply-To: <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<alpine.DEB.1.00.0805221913001.10785@lappy.toscanini-resnet.stonybrook.edu>
	<4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com>
Message-ID: <alpine.DEB.1.00.0805221947590.10785@lappy.toscanini-resnet.stonybrook.edu>

On Thu, 22 May 2008, Brandon Mintern wrote:
> > I think it makes the most sense, if this construct is adopted, to use
> > __slots__ to control mutability.  Someone more well-versed in the
> > python object model should determine if this is actually a good idea.
> >
> > --
> > Cheers,
> > Leif
>
> Whoops... I should have said "An implementation." I intended the
> implementation to be a specification for behavior and not the
> one-and-only implementation. Personally, I'm not especially familiar
> with using __slots__ in anything but standard usage, so I found it
> easier to show the code using __dict__. Certainly if __slots__ can be
> used, it avoids the need to write __setattr__ and to explicitly raise
> the AttributeError exception (which I already see is wrong because it
> doesn't support inheritance).

Don't take my word as gospel.  I've only used __slots__ once, and it
was at the suggest of another programmer in a kind of "I think this
does what we want" suggestion, and we are still not entirely sure if
it's doing what we think it's doing.

-- 
Cheers,
Leif


From greg.ewing at canterbury.ac.nz  Fri May 23 00:29:59 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 23 May 2008 10:29:59 +1200
Subject: [Python-ideas] Add kwargs to built-in function object
In-Reply-To: <E14B16CD-F154-4CD0-B9AB-85F877A2E905@socialserve.com>
References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com>
	<E14B16CD-F154-4CD0-B9AB-85F877A2E905@socialserve.com>
Message-ID: <4835F3E7.7090207@canterbury.ac.nz>

Calvin Spealman wrote:

> and remember, object is not a function. It is a type. It is the base  
> type of all types and so this has implications beyond what you  suggest 
> it for.

Another serious consequence is that supporting this would
require the object type to have a __dict__, which would
force *all* types, including the built-in ones, to have
the overhead of a __dict__ as well.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri May 23 03:59:36 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 23 May 2008 13:59:36 +1200
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
 kwargs to built-in function object)
In-Reply-To: <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<alpine.DEB.1.00.0805221913001.10785@lappy.toscanini-resnet.stonybrook.edu>
	<4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com>
Message-ID: <48362508.3080706@canterbury.ac.nz>

On Thu, May 22, 2008 at 7:15 PM, Leif Walsh <adlaiff6 at gmail.com> wrote:
> I think it makes the most sense, if this construct is adopted, to use
> __slots__ to control mutability.

That wouldn't work, because the slots that a class has
are fixed when the class is defined. Changing __slots__
subsequent to that doesn't affect anything.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri May 23 04:02:10 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 23 May 2008 14:02:10 +1200
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
 kwargs to built-in function object)
In-Reply-To: <alpine.DEB.1.00.0805221947590.10785@lappy.toscanini-resnet.stonybrook.edu>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<alpine.DEB.1.00.0805221913001.10785@lappy.toscanini-resnet.stonybrook.edu>
	<4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com>
	<alpine.DEB.1.00.0805221947590.10785@lappy.toscanini-resnet.stonybrook.edu>
Message-ID: <483625A2.1000206@canterbury.ac.nz>

Leif Walsh wrote:
> I've only used __slots__ once, and it
> was at the suggest of another programmer in a kind of "I think this
> does what we want" suggestion, and we are still not entirely sure if
> it's doing what we think it's doing.

Another thing to keep in mind about __slots__ is that
its primary purpose is to save memory. It currently
has the side effect of preventing other attributes
from being added, but it may not remain that way
forever.

-- 
Greg




From george.sakkis at gmail.com  Fri May 23 07:23:54 2008
From: george.sakkis at gmail.com (George Sakkis)
Date: Fri, 23 May 2008 01:23:54 -0400
Subject: [Python-ideas] Queue suggestions
Message-ID: <91ad5bf80805222223q32a92cb8sfb0ebbca92db5b99@mail.gmail.com>

I'd like to propose a signature simplification and a new method for
the Queue.Queue class:

1) Drop the optional `block` argument from put() and get(), since all
the meaningful combinations of  (block, timeout) are equivalent to
passing block = (timeout is None or timeout>0). IOW, instead of
passing block=False pass timeout=0 (or negative). Obviously this is to
be considered for 3.x only.

2) Add a new `rotate` method as the atomic equivalent of
put_nowait(get_nowait()). Currently I use the following subclass but
it would be nice to have it in the base Queue:

class RQueue(Queue):

    def rotate(self, n=1):
        '''Rotate this queue n steps to the left (if it is not empty).

        Rotating one step is equivalent to an atomic
q.put_nowait(q.get_nowait()).
        '''
        if n < 0:
            raise ValueError('n must be non-negative')
        self.mutex.acquire()
        try:
            if not self._empty():
                self._rotate(n)
        finally:
            self.mutex.release()

    def __init__(self, maxsize=0):
        Queue.__init__(self, maxsize)
        if hasattr(self.queue, 'rotate'):  # deque has rotate() since v2.5
            def _rotate(n):
                # negative n for left rotate
                self.queue.rotate(-n)
        else:
            def _rotate(n):
                put,get = self._put, self._get
                for i in xrange(n):
                    put(get())
        self._rotate = _rotate


What do you think ?

George


From greg at krypto.org  Fri May 23 08:56:58 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 22 May 2008 23:56:58 -0700
Subject: [Python-ideas] Queue suggestions
In-Reply-To: <91ad5bf80805222223q32a92cb8sfb0ebbca92db5b99@mail.gmail.com>
References: <91ad5bf80805222223q32a92cb8sfb0ebbca92db5b99@mail.gmail.com>
Message-ID: <52dc1c820805222356j607a3511n868950da0c56df30@mail.gmail.com>

Do you have a compelling use case for Queue.rotate?

On Thu, May 22, 2008 at 10:23 PM, George Sakkis <george.sakkis at gmail.com> wrote:
> I'd like to propose a signature simplification and a new method for
> the Queue.Queue class:
>
> 1) Drop the optional `block` argument from put() and get(), since all
> the meaningful combinations of  (block, timeout) are equivalent to
> passing block = (timeout is None or timeout>0). IOW, instead of
> passing block=False pass timeout=0 (or negative). Obviously this is to
> be considered for 3.x only.
>
> 2) Add a new `rotate` method as the atomic equivalent of
> put_nowait(get_nowait()). Currently I use the following subclass but
> it would be nice to have it in the base Queue:
>
> class RQueue(Queue):
>
>    def rotate(self, n=1):
>        '''Rotate this queue n steps to the left (if it is not empty).
>
>        Rotating one step is equivalent to an atomic
> q.put_nowait(q.get_nowait()).
>        '''
>        if n < 0:
>            raise ValueError('n must be non-negative')
>        self.mutex.acquire()
>        try:
>            if not self._empty():
>                self._rotate(n)
>        finally:
>            self.mutex.release()
>
>    def __init__(self, maxsize=0):
>        Queue.__init__(self, maxsize)
>        if hasattr(self.queue, 'rotate'):  # deque has rotate() since v2.5
>            def _rotate(n):
>                # negative n for left rotate
>                self.queue.rotate(-n)
>        else:
>            def _rotate(n):
>                put,get = self._put, self._get
>                for i in xrange(n):
>                    put(get())
>        self._rotate = _rotate
>
>
> What do you think ?
>
> George
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From george.sakkis at gmail.com  Fri May 23 08:51:30 2008
From: george.sakkis at gmail.com (George Sakkis)
Date: Fri, 23 May 2008 02:51:30 -0400
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
	kwargs to built-in function object)
In-Reply-To: <48362508.3080706@canterbury.ac.nz>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<alpine.DEB.1.00.0805221913001.10785@lappy.toscanini-resnet.stonybrook.edu>
	<4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com>
	<48362508.3080706@canterbury.ac.nz>
Message-ID: <91ad5bf80805222351p419d3c9cl30225a0860528101@mail.gmail.com>

On Thu, May 22, 2008 at 9:59 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> On Thu, May 22, 2008 at 7:15 PM, Leif Walsh <adlaiff6 at gmail.com> wrote:
>>
>> I think it makes the most sense, if this construct is adopted, to use
>> __slots__ to control mutability.
>
> That wouldn't work, because the slots that a class has
> are fixed when the class is defined. Changing __slots__
> subsequent to that doesn't affect anything.

Unless you generate the class on the fly with a class factory, like
namedtuple. Actually I prefer a metaclass factory, to avoid the
repetition of the class name:

        >>> class Foo(object):
        ...           __metaclass__ = RecordType('x y z is_on',
field_defaults={'is_on':False}, default=0.0)
        >>> a = Foo()
        >>> a
        Foo(x=0.0, y=0.0, z=0.0, is_on=False)
        >>> a.__slots__
        ('x', 'y', 'z', 'is_on')
        >>> a.__dict__
        Traceback (most recent call last):
        AttributeError: 'Foo' object has no attribute '__dict__'


George


From carl at carlsensei.com  Fri May 23 09:15:06 2008
From: carl at carlsensei.com (Carl Johnson)
Date: Thu, 22 May 2008 21:15:06 -1000
Subject: [Python-ideas] Function to unnest for-statements
Message-ID: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>

This is based off of http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations 
  . It's also discussed at http://reddit.com/info/5ywrs/comments/ and  
someone with a lot of spare time can read Knuth's fascile on the  
subject at http://www-cs-faculty.stanford.edu/~knuth/fasc2a.ps.gz .

OK, suppose you have a situation where you need to loop through all  
combinations of 3 lists. The simple solution is to write three nested  
for-loops:

for x in xs:
    for y in ys:
        for z in zs:
            #Do something

Of course, this is obvious O(n^3), which is bad, but sometimes you  
don't have a choice. However, what if (to use an example I ran into),  
you're making up a truth table. If you know you have three variables,  
you might write:

for a in [True, False]:
    for b in [True, False]:
        for c in [True, False]:
            print a, b, c

Which yields

True True True
True True False
True False True
True False False
False True True
False True False
False False True
False False False

But if you don't know how many variables you'll have, you're stuck  
writing a complicated function instead of using a nice, simple for-loop.

So, going back to the first example, wouldn't it be nicer to write:

for x, y, z in combine(xs, ys, zs):
    #Do something

If nothing else, this has the advantage that if you don't have to nest  
the for-loops, things don't end up being so deeply indented on the  
screen.

Similarly, a general truth table maker can be written:

ts_and_fs = [[True, False]] * num_vars
for variables in combine(*ts_and_fs):
    print variables

So, I think a "combine" function (or some other, better name) would be  
a good thing to have, at least in the itertools, if not as a built-in  
function. How to implement it? Well, probably it should be done in C  
for efficiency, but of the versions done http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations 
  the one that I like best is

def combine(*sequences):
    #Test to guard against infinite recursion
    if sequences:
        def _inner(*seqs):
            if len(seqs) == 1:
                for i in seqs[0]: yield (i, )
            else:
                for rest in combine(*seqs[:-1]):
                    for i in seqs[-1]:
                        yield rest + (i, )
        return _inner(*sequences)
    else:
        #Empty generator that yields StopIteration
        def _inner():
            return
            yield
        return _inner()

However, I'm also convinced that there is a more efficient way to do  
this.

So, what do you think? Is this a common enough need that it should be  
built into itertools? The main namespace? Or should we leave it out,  
since adding it would encourage people writing O(n^x) algorithms? If  
nothing else, list members can enjoy rewriting this function for fun.

-- Carl Johnson


From ntoronto at cs.byu.edu  Fri May 23 10:19:29 2008
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 23 May 2008 02:19:29 -0600
Subject: [Python-ideas] Function to unnest for-statements
In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
Message-ID: <48367E11.90203@cs.byu.edu>

Carl Johnson wrote:
> So, going back to the first example, wouldn't it be nicer to write:
> 
> for x, y, z in combine(xs, ys, zs):
>    #Do something
> 
> If nothing else, this has the advantage that if you don't have to nest 
> the for-loops, things don't end up being so deeply indented on the screen.

FWIW, I find myself doing this often with numpy:

     for index in numpy.ndindex(arr.shape):
         # do something using arr's indexes, whatever they are

(Broadcasting is better, but sometimes this is unavoidable.)

The point is there's a general use-case Carl left out: what if you don't 
know how deeply you'll need to nest? Our Functional friends know the 
answer to this, but IIRC the Python design philosophy doesn't actively 
encourage recursive solutions.

I don't think the use-cases are common enough to warrant a new builtin 
(though that's just my $0.000001 opinion - it may be that *everyone* 
will want to flatten their nested loops), but something like "combine" 
would fit very well into itertools.

Neil



From adlaiff6 at gmail.com  Fri May 23 10:26:29 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Fri, 23 May 2008 04:26:29 -0400 (EDT)
Subject: [Python-ideas] Function to unnest for-statements
In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
Message-ID: <alpine.DEB.1.00.0805230347460.10785@lappy.toscanini-resnet.stonybrook.edu>

On Thu, 22 May 2008, Carl Johnson wrote:
> So, going back to the first example, wouldn't it be nicer to write:
>
> for x, y, z in combine(xs, ys, zs):
>     #Do something
>
> If nothing else, this has the advantage that if you don't have to nest
> the for-loops, things don't end up being so deeply indented on the
> screen.

I think extra indentation helps, because it makes it clearer how the
loops nest, but I suppose you could do this if you really wanted;

>>> xs = ys = zs = [True, False]
>>> for a, b, c in [(x, y, z) for x in xs for y in ys for z in zs]:
...     print a, b, c
...
True True True
True True False
True False True
True False False
False True True
False True False
False False True
False False False
>>>

> def combine(*sequences):
>     #Test to guard against infinite recursion
>     if sequences:
>         def _inner(*seqs):
>             if len(seqs) == 1:
>                 for i in seqs[0]: yield (i, )
>             else:
>                 for rest in combine(*seqs[:-1]):
>                     for i in seqs[-1]:
>                         yield rest + (i, )
>         return _inner(*sequences)
>     else:
>         #Empty generator that yields StopIteration
>         def _inner():
>             return
>             yield
>         return _inner()
>
> However, I'm also convinced that there is a more efficient way to do
> this.

I've worked this out (I'm sure I did this in intro CS once before...),
although there is probably some optimization trick I'm missing, and
I'm probably doing some silly double-allocation thing or something.

>>> def combine(lists):
...     if len(lists) is 0:
...             return [[]]
...     first = combine(lists[0:-1])
...     total = []
...     for item in lists[-1]:
...             total += [[item] + subset for subset in first]
...     return total
...
>>> for set in combine([[True, False]] * 4):
...     print set
...
[True, True, True, True]
[True, True, True, False]
[True, True, False, True]
[True, True, False, False]
[True, False, True, True]
[True, False, True, False]
[True, False, False, True]
[True, False, False, False]
[False, True, True, True]
[False, True, True, False]
[False, True, False, True]
[False, True, False, False]
[False, False, True, True]
[False, False, True, False]
[False, False, False, True]
[False, False, False, False]
>>>

> So, what do you think? Is this a common enough need that it should be
> built into itertools? The main namespace? Or should we leave it out,
> since adding it would encourage people writing O(n^x) algorithms? If
> nothing else, list members can enjoy rewriting this function for fun.

I really, really don't think this is worth putting in any distributed
package, as it's a pretty simple thing to code up if you're careful.
That said, it _was_ a lot of fun rewriting it.  Thanks.  :D

-- 
Cheers,
Leif


From adlaiff6 at gmail.com  Fri May 23 10:33:05 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Fri, 23 May 2008 04:33:05 -0400 (EDT)
Subject: [Python-ideas] Function to unnest for-statements
In-Reply-To: <alpine.DEB.1.00.0805230347460.10785@lappy.toscanini-resnet.stonybrook.edu>
References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
	<alpine.DEB.1.00.0805230347460.10785@lappy.toscanini-resnet.stonybrook.edu>
Message-ID: <alpine.DEB.1.00.0805230430320.10785@lappy.toscanini-resnet.stonybrook.edu>

On Fri, 23 May 2008, Leif Walsh wrote:
> I've worked this out (I'm sure I did this in intro CS once before...),
> although there is probably some optimization trick I'm missing, and
> I'm probably doing some silly double-allocation thing or something.
>
> >>> def combine(lists):
> ...     if len(lists) is 0:
> ...             return [[]]
> ...     first = combine(lists[0:-1])
> ...     total = []
> ...     for item in lists[-1]:
> ...             total += [[item] + subset for subset in first]
> ...     return total

Oops, that should probably be:

...     for subset in first:
...             total += [subset + [item] for item in lists[-1]]

to maintain order.  My excuse is that it's past my bedtime.

-- 
Cheers,
Leif


From bmintern at gmail.com  Fri May 23 10:38:55 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Fri, 23 May 2008 04:38:55 -0400
Subject: [Python-ideas] Function to unnest for-statements
In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
Message-ID: <4c0fccce0805230138y6ca53705lcb2dc4398aac750d@mail.gmail.com>

On Fri, May 23, 2008 at 3:15 AM, Carl Johnson <carl at carlsensei.com> wrote:
> This is based off of
> http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations .
> It's also discussed at http://reddit.com/info/5ywrs/comments/ and someone
> with a lot of spare time can read Knuth's fascile on the subject at
> http://www-cs-faculty.stanford.edu/~knuth/fasc2a.ps.gz .
>
> OK, suppose you have a situation where you need to loop through all
> combinations of 3 lists. The simple solution is to write three nested
> for-loops:
>
> for x in xs:
>   for y in ys:
>       for z in zs:
>           #Do something
>
> Of course, this is obvious O(n^3), which is bad, but sometimes you don't
> have a choice. However, what if (to use an example I ran into), you're
> making up a truth table. If you know you have three variables, you might
> write:
>
> for a in [True, False]:
>   for b in [True, False]:
>       for c in [True, False]:
>           print a, b, c
>
<snip>
>
> But if you don't know how many variables you'll have, you're stuck writing a
> complicated function instead of using a nice, simple for-loop.
>
> So, going back to the first example, wouldn't it be nicer to write:
>
> for x, y, z in combine(xs, ys, zs):
>   #Do something
>
<snip>
>
> So, I think a "combine" function (or some other, better name) would be a
> good thing to have, at least in the itertools, if not as a built-in
> function. How to implement it? Well, probably it should be done in C for
> efficiency, but of the versions done
> http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations the
> one that I like best is
>
<snip>
> However, I'm also convinced that there is a more efficient way to do this.
>
> So, what do you think? Is this a common enough need that it should be built
> into itertools? The main namespace? Or should we leave it out, since adding
> it would encourage people writing O(n^x) algorithms? If nothing else, list
> members can enjoy rewriting this function for fun.
>
> -- Carl Johnson

I really like this function, and I could have used it about a year
ago. I actually like the name "combinations" better than "combine".
IMO, "combinations" implies the intended meaning more directly, while
"combine" could be combining the lists in some other way.

Whatever the name, I think it would fit quite nicely into itertools. I
don't think efficiency should be considered as a reason to leave it
out, because as you said, there are simply cases (any time you need to
enumerate all possible values) where you can't get around it. Besides,
it's not too difficult to create quadratic or cubic lists using list
comprehensions... the syntax is actually designed to support it.

As for a stab at a good implementation:

def combinations (*args):
   def combiner (seqs):
       try:
           seq0 = seqs.next()
       except StopIteration:
           yield ()
       else:
           for prefix in combiner(iter(seqs)):
               for x in seq0:
                   yield prefix + (x,)
   return combiner(reversed(args))

I tried to avoid unpacking and repacking lists, I returned the
combinations as tuples, and I tried to use generators wherever
possible. I didn't actually do any performance tests, though.

Brandon


From bmintern at gmail.com  Fri May 23 10:51:05 2008
From: bmintern at gmail.com (Brandon Mintern)
Date: Fri, 23 May 2008 04:51:05 -0400
Subject: [Python-ideas] Function to unnest for-statements
In-Reply-To: <4c0fccce0805230138y6ca53705lcb2dc4398aac750d@mail.gmail.com>
References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
	<4c0fccce0805230138y6ca53705lcb2dc4398aac750d@mail.gmail.com>
Message-ID: <4c0fccce0805230151k13807b93mcb5cab084502f45f@mail.gmail.com>

On Fri, May 23, 2008 at 4:38 AM, Brandon Mintern <bmintern at gmail.com> wrote:
> def combinations (*args):
>   def combiner (seqs):
>       try:
>           seq0 = seqs.next()
>       except StopIteration:
>           yield ()
>       else:
>           for prefix in combiner(iter(seqs)):
>               for x in seq0:
>                   yield prefix + (x,)
>   return combiner(reversed(args))

I was incorrectly thinking that combiner would be called more than
once on seqs, and thus I used iter(seqs) to avoid making a call on an
exhausted iterator. It turns out that's unnecessary since there is
only one recursive call per call to combiner, so the result is the
somewhat improved:

def combinations (*args):
    def combiner (seqs):
        try:
            seq0 = seqs.next()
        except StopIteration:
            yield ()
        else:
            for prefix in combiner(seqs):
                for x in seq0:
                    yield prefix + (x,)
    return combiner(reversed(args))

I think this is a solution I can be proud of, in terms of correctness
(iterating in the proper order), readability, and efficiency.

Brandon


From g.brandl at gmx.net  Fri May 23 11:47:32 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 23 May 2008 11:47:32 +0200
Subject: [Python-ideas] Function to unnest for-statements
In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
Message-ID: <g163rp$6me$1@ger.gmane.org>

Carl Johnson schrieb:

> for a in [True, False]:
>     for b in [True, False]:
>         for c in [True, False]:
>             print a, b, c
> 
> Which yields
> 
> True True True
> True True False
> True False True
> True False False
> False True True
> False True False
> False False True
> False False False
> 
> But if you don't know how many variables you'll have, you're stuck  
> writing a complicated function instead of using a nice, simple for-loop.
> 
> So, going back to the first example, wouldn't it be nicer to write:
> 
> for x, y, z in combine(xs, ys, zs):
>     #Do something

> So, what do you think? Is this a common enough need that it should be  
> built into itertools?

Presumably, since it has been added to itertools in 2.6 under the name
product(). (Maybe Raymond borrowed the time machine?) :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From g.brandl at gmx.net  Fri May 23 11:49:45 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 23 May 2008 11:49:45 +0200
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
 kwargs to built-in function object)
In-Reply-To: <4835FBA4.8020206@canterbury.ac.nz>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<4835FBA4.8020206@canterbury.ac.nz>
Message-ID: <g163vu$6me$2@ger.gmane.org>

Greg Ewing schrieb:
> Brandon Mintern wrote:
>> This is a proposal to add a new built-in named struct:
>> 
>> struct(**kwargs)
>>     Return a struct object which has the attributes given in kwargs.
> 
> I think I'd prefer 'record', to avoid any potential
> confusion with the struct module, which does something
> quite different.
> 
> Also, my Pascal background makes the term 'record'
> seem more high-level and therefore Pythonic to me.

In the past, people have also suggested 'namespace' for the same
concept.

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From steven.bethard at gmail.com  Sat May 24 22:40:11 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sat, 24 May 2008 14:40:11 -0600
Subject: [Python-ideas] Proposal to add new built-in struct (was: Add
	kwargs to built-in function object)
In-Reply-To: <91ad5bf80805222351p419d3c9cl30225a0860528101@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<alpine.DEB.1.00.0805221913001.10785@lappy.toscanini-resnet.stonybrook.edu>
	<4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com>
	<48362508.3080706@canterbury.ac.nz>
	<91ad5bf80805222351p419d3c9cl30225a0860528101@mail.gmail.com>
Message-ID: <d11dcfba0805241340m693872e2ke8c441186be34aca@mail.gmail.com>

On Fri, May 23, 2008 at 12:51 AM, George Sakkis <george.sakkis at gmail.com> wrote:
> On Thu, May 22, 2008 at 9:59 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
>> On Thu, May 22, 2008 at 7:15 PM, Leif Walsh <adlaiff6 at gmail.com> wrote:
>>>
>>> I think it makes the most sense, if this construct is adopted, to use
>>> __slots__ to control mutability.
>>
>> That wouldn't work, because the slots that a class has
>> are fixed when the class is defined. Changing __slots__
>> subsequent to that doesn't affect anything.
>
> Unless you generate the class on the fly with a class factory, like
> namedtuple. Actually I prefer a metaclass factory, to avoid the
> repetition of the class name:
>
>        >>> class Foo(object):
>        ...           __metaclass__ = RecordType('x y z is_on',
> field_defaults={'is_on':False}, default=0.0)
>        >>> a = Foo()
>        >>> a
>        Foo(x=0.0, y=0.0, z=0.0, is_on=False)
>        >>> a.__slots__
>        ('x', 'y', 'z', 'is_on')
>        >>> a.__dict__
>        Traceback (most recent call last):
>        AttributeError: 'Foo' object has no attribute '__dict__'

The recipe here does roughly that (though without the defaulting extras):

    http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/502237

Note however that use of __slots__ is still controversial. (Because
it's really only appropriate if you plan to make many of these objects
and you want to keep memory consumption down.)

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
 --- Bucky Katt, Get Fuzzy


From greg.ewing at canterbury.ac.nz  Sun May 25 03:19:05 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 25 May 2008 13:19:05 +1200
Subject: [Python-ideas] Proposal to add new built-in struct
In-Reply-To: <d11dcfba0805241340m693872e2ke8c441186be34aca@mail.gmail.com>
References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com>
	<alpine.DEB.1.00.0805221913001.10785@lappy.toscanini-resnet.stonybrook.edu>
	<4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com>
	<48362508.3080706@canterbury.ac.nz>
	<91ad5bf80805222351p419d3c9cl30225a0860528101@mail.gmail.com>
	<d11dcfba0805241340m693872e2ke8c441186be34aca@mail.gmail.com>
Message-ID: <4838BE89.2020006@canterbury.ac.nz>

Steven Bethard wrote:

> Note however that use of __slots__ is still controversial. (Because
> it's really only appropriate if you plan to make many of these objects
> and you want to keep memory consumption down.)

Also, if you were going to go down the route of creating
classes on the fly, you would want a factory function
to create a class suitable for the purpose at hand,
and then make instances of that class. Otherwise, you
would be creating a new class for every *instance* of
your struct, which would be extremely wasteful (type
objects are *big*!)

And if you're doing that, you might as well just have
some built-in or library class that you can subclass.

-- 
Greg


From arnodel at googlemail.com  Sun May 25 10:01:39 2008
From: arnodel at googlemail.com (Arnaud Delobelle)
Date: Sun, 25 May 2008 09:01:39 +0100
Subject: [Python-ideas] Function to unnest for-statements
In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
Message-ID: <970E1C53-2F9A-4DC7-AF4A-2C12805F00CA@gmail.com>


On 23 May 2008, at 08:15, Carl Johnson wrote:

> This is based off of http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations 
>  . It's also discussed at http://reddit.com/info/5ywrs/comments/ and  
> someone with a lot of spare time can read Knuth's fascile on the  
> subject at http://www-cs-faculty.stanford.edu/~knuth/fasc2a.ps.gz .
>
> OK, suppose you have a situation where you need to loop through all  
> combinations of 3 lists. The simple solution is to write three  
> nested for-loops:
>
> for x in xs:
>   for y in ys:
>       for z in zs:
>           #Do something
>
> Of course, this is obvious O(n^3), which is bad, but sometimes you  
> don't have a choice. However, what if (to use an example I ran  
> into), you're making up a truth table. If you know you have three  
> variables, you might write:
>
> for a in [True, False]:
>   for b in [True, False]:
>       for c in [True, False]:
>           print a, b, c
>
> Which yields
>
> True True True
> True True False
> True False True
> True False False
> False True True
> False True False
> False False True
> False False False
>
> But if you don't know how many variables you'll have, you're stuck  
> writing a complicated function instead of using a nice, simple for- 
> loop.
>
> So, going back to the first example, wouldn't it be nicer to write:
>
> for x, y, z in combine(xs, ys, zs):
>   #Do something
>
> If nothing else, this has the advantage that if you don't have to  
> nest the for-loops, things don't end up being so deeply indented on  
> the screen.
>
> Similarly, a general truth table maker can be written:
>
> ts_and_fs = [[True, False]] * num_vars
> for variables in combine(*ts_and_fs):
>   print variables
>
> So, I think a "combine" function (or some other, better name) would  
> be a good thing to have, at least in the itertools, if not as a  
> built-in function. How to implement it? Well, probably it should be  
> done in C for efficiency, but of the versions done http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations 
>  the one that I like best is
>
> def combine(*sequences):
>   #Test to guard against infinite recursion
>   if sequences:
>       def _inner(*seqs):
>           if len(seqs) == 1:
>               for i in seqs[0]: yield (i, )
>           else:
>               for rest in combine(*seqs[:-1]):
>                   for i in seqs[-1]:
>                       yield rest + (i, )
>       return _inner(*sequences)
>   else:
>       #Empty generator that yields StopIteration
>       def _inner():
>           return
>           yield
>       return _inner()
>
> However, I'm also convinced that there is a more efficient way to do  
> this'


I don't understand why you define those _inner() generator functions.   
Anyway, here's a version that doesn't use recursion:

def product(*sequences):
    if not sequences: return
    i, imax = 0, len(sequences)-1
    val = [None]*(imax+1)
    iters = [iter(seq) for seq in sequences]
    while i >= 0:
        for val[i] in iters[i]:
            if i == imax:
                yield tuple(val)
            else:
                i += 1
                break
        else:
            iters[i] = iter(sequences[i])
            i -= 1




>
> So, what do you think? Is this a common enough need that it should  
> be built into itertools? The main namespace? Or should we leave it  
> out, since adding it would encourage people writing O(n^x)  
> algorithms? If nothing else, list members can enjoy rewriting this  
> function for fun.

It is already in itertools:

Python 3.0a4+ (py3k:62388, Apr 19 2008, 15:34:00)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> from itertools import product
 >>> for v in product(*[[True, False]]*3):
...     print(v)
...
(True, True, True)
(True, True, False)
(True, False, True)
(True, False, False)
(False, True, True)
(False, True, False)
(False, False, True)
(False, False, False)
 >>>

-- 
Arnaud



From george.sakkis at gmail.com  Sun May 25 14:11:39 2008
From: george.sakkis at gmail.com (George Sakkis)
Date: Sun, 25 May 2008 08:11:39 -0400
Subject: [Python-ideas] Function to unnest for-statements
In-Reply-To: <970E1C53-2F9A-4DC7-AF4A-2C12805F00CA@gmail.com>
References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com>
	<970E1C53-2F9A-4DC7-AF4A-2C12805F00CA@gmail.com>
Message-ID: <91ad5bf80805250511n306c0d5ap4a4831bd891eda2f@mail.gmail.com>

On Sun, May 25, 2008 at 4:01 AM, Arnaud Delobelle
<arnodel at googlemail.com> wrote:
>
> On 23 May 2008, at 08:15, Carl Johnson wrote:
>
>> So, what do you think? Is this a common enough need that it should be
>> built into itertools? The main namespace? Or should we leave it out, since
>> adding it would encourage people writing O(n^x) algorithms? If nothing else,
>> list members can enjoy rewriting this function for fun.
>
> It is already in itertools:
>
> Python 3.0a4+ (py3k:62388, Apr 19 2008, 15:34:00)
> [GCC 4.0.1 (Apple Inc. build 5465)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from itertools import product
>>>> for v in product(*[[True, False]]*3):
> ...     print(v)
> ...
> (True, True, True)
> (True, True, False)
> (True, False, True)
> (True, False, False)
> (False, True, True)
> (False, True, False)
> (False, False, True)
> (False, False, False)

Or a bit more readable:

>>> for v in product([True, False], repeat=3):
...         print(v)

George