From joshua.spoerri at gmail.com Tue May 13 15:52:35 2008 From: joshua.spoerri at gmail.com (Joshua Spoerri) Date: Tue, 13 May 2008 09:52:35 -0400 Subject: [Python-ideas] decimal by default Message-ID: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com> Should decimal be the default for floating period literals? E.g. 1.2 would actually be decimal.Decimal("1.2") and float(1.2) would be used to get traditional binary float point. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue May 13 16:04:49 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 13 May 2008 07:04:49 -0700 Subject: [Python-ideas] decimal by default In-Reply-To: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com> References: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com> Message-ID: On Tue, May 13, 2008 at 6:52 AM, Joshua Spoerri wrote: > Should decimal be the default for floating period literals? > > E.g. > 1.2 would actually be decimal.Decimal("1.2") > and float(1.2) would be used to get traditional binary float point. Not in 3.0, there are too many things that are subtly different. Perhaps at some point post 3.0 we can invent a mechanism whereby modules can enable this feature on a per-module basis, and then some number of revisions later we can change the default. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From santagada at gmail.com Tue May 13 16:12:21 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Tue, 13 May 2008 11:12:21 -0300 Subject: [Python-ideas] decimal by default In-Reply-To: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com> References: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com> Message-ID: On 13/05/2008, at 10:52, Joshua Spoerri wrote: > Should decimal be the default for floating period literals? -Inf :) > E.g. > 1.2 would actually be decimal.Decimal("1.2") > and float(1.2) would be used to get traditional binary float point. -- Leonardo Santagada From santagada at gmail.com Tue May 13 17:03:28 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Tue, 13 May 2008 12:03:28 -0300 Subject: [Python-ideas] decimal by default In-Reply-To: References: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com> Message-ID: On 13/05/2008, at 11:04, Guido van Rossum wrote: > On Tue, May 13, 2008 at 6:52 AM, Joshua Spoerri > wrote: >> Should decimal be the default for floating period literals? >> >> E.g. >> 1.2 would actually be decimal.Decimal("1.2") >> and float(1.2) would be used to get traditional binary float point. > > Not in 3.0, there are too many things that are subtly different. > Perhaps at some point post 3.0 we can invent a mechanism whereby > modules can enable this feature on a per-module basis, and then some > number of revisions later we can change the default. I would be happier with a d in front of the number following the scheme of raw strings, either to mean Decimal or to mean Double, or maybe f. This way things would work for both cientists and the rest of the users. -- Leonardo Santagada From george.sakkis at gmail.com Tue May 13 20:07:15 2008 From: george.sakkis at gmail.com (George Sakkis) Date: Tue, 13 May 2008 14:07:15 -0400 Subject: [Python-ideas] decimal by default In-Reply-To: References: <9a55c53f0805130652p5a44dce2qc25f64bd0982e059@mail.gmail.com> Message-ID: <91ad5bf80805131107o69e377a5m94dd7a9c52a1144@mail.gmail.com> On Tue, May 13, 2008 at 11:03 AM, Leonardo Santagada wrote: > > On 13/05/2008, at 11:04, Guido van Rossum wrote: > > > > On Tue, May 13, 2008 at 6:52 AM, Joshua Spoerri > > wrote: > > > > > Should decimal be the default for floating period literals? > > > > > > E.g. > > > 1.2 would actually be decimal.Decimal("1.2") > > > and float(1.2) would be used to get traditional binary float point. > > > > > > > Not in 3.0, there are too many things that are subtly different. > > Perhaps at some point post 3.0 we can invent a mechanism whereby > > modules can enable this feature on a per-module basis, and then some > > number of revisions later we can change the default. > > > > > I would be happier with a d in front of the number following the scheme of > raw strings, either to mean Decimal or to mean Double, or maybe f. This way > things would work for both cientists and the rest of the users. I would prefer "from __future__ import default_decimals", with target version for being enforced the 6.0 or later; we'll all be carrying quantum laptops by then ;-) George From tjreedy at udel.edu Thu May 15 08:36:25 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 15 May 2008 02:36:25 -0400 Subject: [Python-ideas] Give generators a name? Message-ID: I have 2 related proposals: 1. Give generators a .__name__ attribute that is the same as their curent (3.0a5) .gi_code.co_name subattribute. just as funct.__name__ is func.__code__.co_name. My reason is, I expect, much the same as that for func.__name__. I am using the generator name (for bad-iterator-output messages in a test function) and would prefer to get it through a cross-implementation 'public' interface' rather than a cPython internal implementation detail (which I understand code object to be). I am otherwise trying to avoid using cPython internals. (Is there any plan to change the gi_* attributes the way the func_* attributes were?) 2. Whether or not 1 is adopted, add the name to the representation: or Conceptually, I see a generator function as an abbreviated version of a iterator class, with most of the boilerplate removed, that defines a subclass of the generator class. So I think the subclass name should be part of its representation. Terry Jan Reedy From guido at python.org Thu May 15 14:56:22 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 15 May 2008 05:56:22 -0700 Subject: [Python-ideas] Give generators a name? In-Reply-To: References: Message-ID: Sounds fine with me. Have you cooked up a patch yet so we can see how complex the change is, whether it breaks anything, etc.? On Wed, May 14, 2008 at 11:36 PM, Terry Reedy wrote: > I have 2 related proposals: > > 1. Give generators a .__name__ attribute that is the same as their curent > (3.0a5) .gi_code.co_name subattribute. just as funct.__name__ is > func.__code__.co_name. > > My reason is, I expect, much the same as that for func.__name__. I am > using the generator name (for bad-iterator-output messages in a test > function) and would prefer to get it through a cross-implementation > 'public' interface' rather than a cPython internal implementation detail > (which I understand code object to be). I am otherwise trying to avoid > using cPython internals. > > (Is there any plan to change the gi_* attributes the way the func_* > attributes were?) > > 2. Whether or not 1 is adopted, add the name to the representation: > or > > Conceptually, I see a generator function as an abbreviated version of a > iterator class, with most of the boilerplate removed, that defines a > subclass of the generator class. So I think the subclass name should be > part of its representation. > > Terry Jan Reedy > > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Thu May 15 16:28:51 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 15 May 2008 16:28:51 +0200 Subject: [Python-ideas] Give generators a name? In-Reply-To: References: Message-ID: A patch suggestion is at . Georg Guido van Rossum schrieb: > Sounds fine with me. Have you cooked up a patch yet so we can see how > complex the change is, whether it breaks anything, etc.? > > On Wed, May 14, 2008 at 11:36 PM, Terry Reedy wrote: >> I have 2 related proposals: >> >> 1. Give generators a .__name__ attribute that is the same as their curent >> (3.0a5) .gi_code.co_name subattribute. just as funct.__name__ is >> func.__code__.co_name. >> >> My reason is, I expect, much the same as that for func.__name__. I am >> using the generator name (for bad-iterator-output messages in a test >> function) and would prefer to get it through a cross-implementation >> 'public' interface' rather than a cPython internal implementation detail >> (which I understand code object to be). I am otherwise trying to avoid >> using cPython internals. >> >> (Is there any plan to change the gi_* attributes the way the func_* >> attributes were?) >> >> 2. Whether or not 1 is adopted, add the name to the representation: >> or >> >> Conceptually, I see a generator function as an abbreviated version of a >> iterator class, with most of the boilerplate removed, that defines a >> subclass of the generator class. So I think the subclass name should be >> part of its representation. >> >> Terry Jan Reedy >> >> >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > > > From amk at amk.ca Tue May 13 19:05:48 2008 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 13 May 2008 13:05:48 -0400 Subject: [Python-ideas] Removing some string methods? Message-ID: <20080513170548.GA10158@amk-desktop.matrixgroup.net> Has any consideration been given to removing some of the lesser-used string methods in 3.0? e.g. .center, .ljust/rjust, .zfill, .swapcase? --amk From brett at python.org Fri May 16 21:29:58 2008 From: brett at python.org (Brett Cannon) Date: Fri, 16 May 2008 12:29:58 -0700 Subject: [Python-ideas] Removing some string methods? In-Reply-To: <20080513170548.GA10158@amk-desktop.matrixgroup.net> References: <20080513170548.GA10158@amk-desktop.matrixgroup.net> Message-ID: On Tue, May 13, 2008 at 10:05 AM, A.M. Kuchling wrote: > Has any consideration been given to removing some of the lesser-used > string methods in 3.0? e.g. .center, .ljust/rjust, .zfill, .swapcase? > Don't remember any specifically coming up. But are you sure those are really lesser-used? I can remember using ljust, rjust, and zfill on multiple occasions. Have you tried looking at the stdlib or something to get usage stats? -Brett From python at rcn.com Fri May 16 21:36:17 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 16 May 2008 12:36:17 -0700 Subject: [Python-ideas] Removing some string methods? References: <20080513170548.GA10158@amk-desktop.matrixgroup.net> Message-ID: <00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1> [AMK] > Has any consideration been given to removing some of the lesser-used > string methods in 3.0? e.g. .center, .ljust/rjust, .zfill, .swapcase? I don't think we gain anything by taking justification and centering methods away. It just makes life difficult for people like me who use those methods for formatting text. Also, these methods have nearly zero mental overhead -- they are self-explanatory and have no learning curve. The swapcase() method is more use case challenged and could probably be zapped without anyone caring. The zfill() method is no longer necessary because rjust() takes an optional fillchar argument: '123'.zfill(10) == '123'.rjust(10, '0') Raymond From amk at amk.ca Fri May 16 22:15:14 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 16 May 2008 16:15:14 -0400 Subject: [Python-ideas] Removing some string methods? In-Reply-To: <00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1> References: <20080513170548.GA10158@amk-desktop.matrixgroup.net> <00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1> Message-ID: <20080516201514.GA14694@amk-desktop.matrixgroup.net> On Fri, May 16, 2008 at 12:36:17PM -0700, Raymond Hettinger wrote: > I don't think we gain anything by taking justification and centering > methods away. It just makes life difficult for people like me who use those > methods for formatting text. Also, these methods have nearly zero mental > overhead -- they are self-explanatory and have no learning curve. Fair enough. > The swapcase() method is more use case challenged and could probably be > zapped without anyone caring. I did a Google code search, and outside of Python test suites and the other implementations, I found exactly two uses of swapcase. --amk From brett at python.org Sat May 17 07:16:55 2008 From: brett at python.org (Brett Cannon) Date: Fri, 16 May 2008 22:16:55 -0700 Subject: [Python-ideas] Removing some string methods? In-Reply-To: <00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1> References: <20080513170548.GA10158@amk-desktop.matrixgroup.net> <00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1> Message-ID: On Fri, May 16, 2008 at 12:36 PM, Raymond Hettinger wrote: > [AMK] >> >> Has any consideration been given to removing some of the lesser-used >> string methods in 3.0? e.g. .center, .ljust/rjust, .zfill, .swapcase? > > I don't think we gain anything by taking justification and centering methods > away. It just makes life difficult for people like me who use those > methods for formatting text. Also, these methods have nearly zero mental > overhead -- they are self-explanatory and have no learning curve. > > The swapcase() method is more use case challenged and could probably be > zapped without anyone caring. > > The zfill() method is no longer necessary because rjust() takes an optional > fillchar argument: > > '123'.zfill(10) == '123'.rjust(10, '0') Ah cool! I don't think I knew about that because I always knew about zfill(). Probably would have found it if I zfill() didn't exist. So for Python 3.1 we can probably safely ditch the two methods (and probably review the methods on the other types as well). -Brett From greg at krypto.org Sat May 17 07:51:27 2008 From: greg at krypto.org (Gregory P. Smith) Date: Fri, 16 May 2008 22:51:27 -0700 Subject: [Python-ideas] Removing some string methods? In-Reply-To: References: <20080513170548.GA10158@amk-desktop.matrixgroup.net> <00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1> Message-ID: <52dc1c820805162251y493e25d7s2c1053977dd71a77@mail.gmail.com> On Fri, May 16, 2008 at 10:16 PM, Brett Cannon wrote: > On Fri, May 16, 2008 at 12:36 PM, Raymond Hettinger wrote: >> [AMK] >>> >>> Has any consideration been given to removing some of the lesser-used >>> string methods in 3.0? e.g. .center, .ljust/rjust, .zfill, .swapcase? >> >> I don't think we gain anything by taking justification and centering methods >> away. It just makes life difficult for people like me who use those >> methods for formatting text. Also, these methods have nearly zero mental >> overhead -- they are self-explanatory and have no learning curve. >> >> The swapcase() method is more use case challenged and could probably be >> zapped without anyone caring. >> >> The zfill() method is no longer necessary because rjust() takes an optional >> fillchar argument: >> >> '123'.zfill(10) == '123'.rjust(10, '0') > > Ah cool! I don't think I knew about that because I always knew about > zfill(). Probably would have found it if I zfill() didn't exist. > > So for Python 3.1 we can probably safely ditch the two methods (and > probably review the methods on the other types as well). > > -Brett zfill and rjust are not the same. zfill treats it as a number and moves the sign. >>> e = b'-33' >>> e.zfill(7) b'-000033' >>> e.rjust(7, '0') b'0000-33' From python at rcn.com Mon May 19 05:10:45 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 18 May 2008 20:10:45 -0700 Subject: [Python-ideas] Removing some string methods? References: <20080513170548.GA10158@amk-desktop.matrixgroup.net><00fb01c8b78c$1cc11720$ae00a8c0@RaymondLaptop1> <20080516201514.GA14694@amk-desktop.matrixgroup.net> Message-ID: <009901c8b95d$ee9a5d10$ac00a8c0@RaymondLaptop1> From: "A.M. Kuchling" > I did a Google code search, and outside of Python test suites and the > other implementations, I found exactly two uses of swapcase. Looks like there is a consensus on removing swapcase(). Go ahead a zap it. Raymond From bmintern at gmail.com Thu May 22 09:21:23 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Thu, 22 May 2008 03:21:23 -0400 Subject: [Python-ideas] Add kwargs to built-in function object Message-ID: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> I would like to propose to change the built-in function "object" to have the following syntax: object(**kwargs) Return a new featureless object. object is a base for all new style classes. It has the methods that are common to all instances of new style classes. If kwargs is given, the returned object's __dict__ will be kwargs (or something to that effect). An example: a = object(one=1, two=2) a.one # returns 1 a.two # returns 2 The justification: On several occasions I have needed a collection of attributes to represent a single item. When this happens, there are really three options: 1. Use a tuple. This works well for temporarily re-packaging items in a loop or for quick-and-dirty code, but it rapidly becomes unreadable and hard to maintain. It is not long before you forget what order the attributes are in, and at first glance, it is not clear what kind of object is being indexed. 2. Use a dict. This is an improvement over tuples on readability, but they can be a pain to build and overly-cumbersome to access later. I understand that dicts are used all over the place in Python, but I still think of them (in the general case) as a map of keys to values where the dict represents a collection, not an object. 3. Use a class. This requires coming up with a name for the class and then writing the class (admittedly, this should be easy). Afterwards, this is the most convenient, readable method for representing the data, but since it requires non-trivial effort up front, this method may be avoided until it's truly apparent that it is necessary. A real-world example: Let's say that I want to have a map of employee SSNs to employee data. I am going to be manipulating this information in various ways, but not really in any ways that justify the use of class methods. At any rate, let's build this data from a file where each line is SSN First Last Salary with the items being whitespace-delimited. The code, then, will be: employees = {} for ssn, first, last, salary in (line.split() for line in open(employee_file)): employees[ssn] = (ssn, first, last, salary) # tuple method employees[ssn] = {"ssn": ssn, "first": first, "last": last, "salary": salary} # dict method employees[ssn] = Employee(ssn, first, last, salary) # assumes class with proper constructor # first name of employee with SSN employees[SSN][1] # tuple method -- quite unreadable employees[SSN]["first"] # dict method -- more readable but sub-optimal employees[SSN].first # class method -- readable and simple Now, because of the advantages the class method offers in terms of readability, I have written a convenience class that makes using it easier: class Record: """ A class intended to provide a simple interface for creating objects with named fields. That is, instead of returning a tuple and indexing it or writing a unique class, you can simply do something like: a = Record(x=1, y=2) a.x # 1 a.y # 2 a # Record(x=1, y=2) """ def __init__ (self, **kwargs): self.__dict__.update(kwargs) def __repr__ (self): return "Record(%s)" \ % ", ".join("%s=%s" \ % field for field in self.__dict__.iteritems()) Now, the line in the for loop above becomes employees[ssn] = Record(ssn=ssn, first=first, last=last, salary=salary) and I have completely avoided the need to define a unique class. Note that as the number of fields increases, this becomes easier to use inline than the dict method, at the same time that it avoids the upfront costs of having to build a new class for every distinct type of object in the program. It turns out that other people find such a method useful as well. According to Catherine Devlin on the centralOH Python list, it is recipe 4.18 from the Python Cookbook (2nd ed.). Several others from the mailing list stated that they had created similar solutions themselves. Thus, my suggestion is to simply build such functionality directly into the language. While scanning the built-in functions page (I don't use any, all, or enumerate nearly enough), I noticed the object() function, and since its purpose is to return a featureless object, it seems to fit the bill quite well. Adding my suggested functionality should break nothing (since the kwargs would be optional) and would allow people to stop baking their own solutions to this common problem, while getting the speed bonus of a C implementation (if it helps). Thus, the code above becomes for... employees[ssn] = object(ssn=ssn, first=first, last=last, salary=salary) employees[SSN].first Does anyone else think this might be a good idea for Python 2.6/3K? Thanks, Brandon From idadesub at users.sourceforge.net Thu May 22 09:49:04 2008 From: idadesub at users.sourceforge.net (Erick Tryzelaar) Date: Thu, 22 May 2008 00:49:04 -0700 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> Message-ID: <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com> On Thu, May 22, 2008 at 12:21 AM, Brandon Mintern wrote: > I would like to propose to change the built-in function "object" to > have the following syntax: > > object(**kwargs) > Return a new featureless object. object is a base for all new style > classes. It has the methods that are common to all instances of new > style classes. > > If kwargs is given, the returned object's __dict__ will be kwargs (or > something to that effect). python 2.6 and 3.0 has something similar to this with collections.namedtuple: http://docs.python.org/dev/3.0/library/collections.html#collections.namedtuple The interface is a bit more verbose though: >>> Point = namedtuple('Point', 'x y') >>> p = Point(11, y=22) # instantiate with positional or keyword arguments >>> p[0] + p[1] # indexable like the plain tuple (11, 22) 33 >>> x, y = p # unpack like a regular tuple >>> x, y (11, 22) >>> p.x + p.y # fields also accessible by name 33 >>> p # readable __repr__ with a name=value style Point(x=11, y=22) I like the syntax of using arguments to object though. Maybe there's a discussion behind namedtuple why they went that way instead of this one? From adlaiff6 at gmail.com Thu May 22 09:57:45 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 22 May 2008 03:57:45 -0400 (EDT) Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com> References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com> Message-ID: On Thu, 22 May 2008, Erick Tryzelaar wrote: > python 2.6 and 3.0 has something similar to this with collections.namedtuple: > > http://docs.python.org/dev/3.0/library/collections.html#collections.namedtuple > > The interface is a bit more verbose though: > > >>> Point = namedtuple('Point', 'x y') > >>> p = Point(11, y=22) # instantiate with positional or keyword arguments > >>> p[0] + p[1] # indexable like the plain tuple (11, 22) > 33 > >>> x, y = p # unpack like a regular tuple > >>> x, y > (11, 22) > >>> p.x + p.y # fields also accessible by name > 33 > >>> p # readable __repr__ with a name=value style > Point(x=11, y=22) > > I like the syntax of using arguments to object though. Maybe there's a > discussion behind namedtuple why they went that way instead of this > one? Just like lambdas were gotten rid of in favor of named inner functions (in part) for readability, it sounds like readability would be a strong argument for using namedtuple rather than some kind of an anonymous object. -- Cheers, Leif From bmintern at gmail.com Thu May 22 10:41:23 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Thu, 22 May 2008 04:41:23 -0400 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com> Message-ID: <4c0fccce0805220141q4e4d278bmdf5b905778d7937a@mail.gmail.com> > Just like lambdas were gotten rid of in favor of named inner functions > (in part) for readability, it sounds like readability would be a > strong argument for using namedtuple rather than some kind of an > anonymous object. > > -- > Cheers, > Leif My only argument would be that classic usage of lambda had three main purposes: (1) to create a function that is assigned to a variable, in which case using def makes more sense, (2) to pass to functions like map and filter, where the same functionality can be achieved with list comprehensions, and (3) to pass to other functions. Getting rid of lambda helps to eliminate the fallacy that is purpose 1, with list comprehensions purpose 2 is less necessary, and purpose 3 is the only real casualty. Even when you really do want to pass a lambda, it's not very hard to say: def f (x): return x*x and then simply pass the f. Thus getting rid of lambda helps alleviate a few issues while not hurting too badly. With an anonymous object, however, I feel the case is a bit different. You are not really using it to create an anonymous class, or even a named tuple. You are instead using it to create an instance of object which already has some fields defined. The same thing could be accomplished with: class anonobject: pass a = anonobject() a.one = 1 a.two = 2 But there is really no easier way that I can think of to do this on the fly except to have already built a Record class as I previously indicated. As long as this is the case, many users will be implementing their own Record class with all the inefficiencies and errors that come along with that. For example, my first iteration looked like: class Fields: """ A class intended to provide a simple interface for creating objects with named fields. That is, instead of returning a tuple and indexing it or writing a unique class, you can simply do something like: a = Fields(x=1, y=2) a.x # 1 a.y # 2 """ def __init__ (self, **kwargs): for name, value in kwargs.iteritems(): self.__dict__[name] = value Not horrible, but it still failed to utilize dict.update(), and the original name was much less intuitive. Someone reading my code would likely have to check out the Fields source to see what exactly I'm doing. Come to think of it, the change to object that I am suggesting could even place the kwargs in __slots__ so that the instance it returns is similar to that currently returned by object(), in that it is not possible to add new fields. All in all, though, I feel like such a change would actually *improve* code readability in general, as we would be less likely to use a tuple or a dict to hold what is truly an object. As soon as it becomes apparent that our object is being used in more complicated ways, it should be easy to write our own class with the same fields as our original call to object and drop it in, adding a reasonable constructor, methods, etc. From aahz at pythoncraft.com Thu May 22 14:18:19 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 22 May 2008 05:18:19 -0700 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com> Message-ID: <20080522121819.GB9262@panix.com> On Thu, May 22, 2008, Leif Walsh wrote: > > Just like lambdas were gotten rid of in favor of named inner functions > (in part) for readability, it sounds like readability would be a > strong argument for using namedtuple rather than some kind of an > anonymous object. Um, what? lambda is still here... -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From aahz at pythoncraft.com Thu May 22 14:22:04 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 22 May 2008 05:22:04 -0700 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> Message-ID: <20080522122204.GC9262@panix.com> On Thu, May 22, 2008, Brandon Mintern wrote: > > I would like to propose to change the built-in function "object" to > have the following syntax: > > object(**kwargs) > Return a new featureless object. object is a base for all new style > classes. It has the methods that are common to all instances of new > style classes. Enh. It's easy enough to write class Obj: def __init__(self, **kwargs): self.__dict__.update(kwargs) It's not clear to me that your functionality is desired frequently enough *in that specific form* to warrant changing object(). Note particularly the emphasis; over the years, I've seen various small variations in how people want kwargs processed and it's not at all clear to me that codifying one specific form into the language would be helpful. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From taleinat at gmail.com Thu May 22 14:28:52 2008 From: taleinat at gmail.com (Tal Einat) Date: Thu, 22 May 2008 15:28:52 +0300 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: <4c0fccce0805220141q4e4d278bmdf5b905778d7937a@mail.gmail.com> References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com> <4c0fccce0805220141q4e4d278bmdf5b905778d7937a@mail.gmail.com> Message-ID: <7afdee2f0805220528jf5dd560q76a2fc335bd7f5c6@mail.gmail.com> Brandon Mintern wrote: > The same thing could be accomplished with: > > class anonobject: > pass > a = anonobject() > a.one = 1 > a.two = 2 > > But there is really no easier way that I can think of to do this on > the fly except to have already built a Record class as I previously > indicated. There is an "easier" way, for some definitions of "easy": a=type('anonobject', (object,), {})() # a is an instance of class anonobject a.one = 1 a.two = 2 Or if you want to keep it really short: a=type('anonobject', (object,), dict(one=1, two=2)) # a is a class named anonobject I'll grant that this looks ugly, the intent of the code is far from obvious. But if you don't do this too often and place appropriate comments this should be fine. If you do find yourself doing this often, I would say writing an appropriate class is the best solution; it is certainly what I would have done in your example with the employees. - Tal From ironfroggy at socialserve.com Thu May 22 15:43:31 2008 From: ironfroggy at socialserve.com (Calvin Spealman) Date: Thu, 22 May 2008 09:43:31 -0400 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> Message-ID: Start using those and suddenly you are going to have lots of little anonymous objects around and you'll start to want to add functionality to them and otherwise enforce things about them. Now you have a higher barrier than when you would have been creating it from scratch. and remember, object is not a function. It is a type. It is the base type of all types and so this has implications beyond what you suggest it for. If object(a=10).a == 10, then when custom classes' __init__ methods upcall, they'll start to fall back on letting kwargs fall up to object.__init__ to set initialization attributes. Now we gain this rule of thumb that keyword arguments are probably going to end up as attributes, because if we don't 'catch' them and they reach object.__init__, they will be. This isn't a behavior the language should encourage, because it should be left up to the developer. Currently, object.__init__ specifically takes any arguments and does nothing with any of them. This is to allow upcall in your __init__ gracefully, because if the upclass is one of your bases or the final base, object, you can still pass your parameters along safely. Implying some behavior when it reaches object.__init__, especially (in most cases) after you've initialized your object, would probably just trounce all over your object and piss you off. No thanks. On May 22, 2008, at 3:21 AM, Brandon Mintern wrote: > I would like to propose to change the built-in function "object" to > have the following syntax: > > object(**kwargs) > Return a new featureless object. object is a base for all new style > classes. It has the methods that are common to all instances of new > style classes. > > If kwargs is given, the returned object's __dict__ will be kwargs (or > something to that effect). > > > An example: > > a = object(one=1, two=2) > a.one # returns 1 > a.two # returns 2 > > > The justification: > > On several occasions I have needed a collection of attributes to > represent a single item. When this happens, there are really three > options: > > 1. Use a tuple. This works well for temporarily re-packaging items in > a loop or for quick-and-dirty code, but it rapidly becomes unreadable > and hard to maintain. It is not long before you forget what order the > attributes are in, and at first glance, it is not clear what kind of > object is being indexed. > > 2. Use a dict. This is an improvement over tuples on readability, but > they can be a pain to build and overly-cumbersome to access later. I > understand that dicts are used all over the place in Python, but I > still think of them (in the general case) as a map of keys to values > where the dict represents a collection, not an object. > > 3. Use a class. This requires coming up with a name for the class and > then writing the class (admittedly, this should be easy). Afterwards, > this is the most convenient, readable method for representing the > data, but since it requires non-trivial effort up front, this method > may be avoided until it's truly apparent that it is necessary. > > > A real-world example: > > Let's say that I want to have a map of employee SSNs to employee data. > I am going to be manipulating this information in various ways, but > not really in any ways that justify the use of class methods. At any > rate, let's build this data from a file where each line is > > SSN First Last Salary > > with the items being whitespace-delimited. The code, then, will be: > > employees = {} > for ssn, first, last, salary in (line.split() for line in open > (employee_file)): > employees[ssn] = (ssn, first, last, salary) # tuple method > employees[ssn] = {"ssn": ssn, "first": first, "last": last, > "salary": salary} # dict method > employees[ssn] = Employee(ssn, first, last, salary) # assumes > class with proper constructor > > # first name of employee with SSN > employees[SSN][1] # tuple method -- quite unreadable > employees[SSN]["first"] # dict method -- more readable but sub-optimal > employees[SSN].first # class method -- readable and simple > > > Now, because of the advantages the class method offers in terms of > readability, I have written a convenience class that makes using it > easier: > > class Record: > """ > A class intended to provide a simple interface for creating objects > with named fields. That is, instead of returning a tuple and > indexing > it or writing a unique class, you can simply do something like: > a = Record(x=1, y=2) > a.x # 1 > a.y # 2 > a # Record(x=1, y=2) > """ > def __init__ (self, **kwargs): > self.__dict__.update(kwargs) > > def __repr__ (self): > return "Record(%s)" \ > % ", ".join("%s=%s" \ > % field for field in > self.__dict__.iteritems()) > > Now, the line in the for loop above becomes > > employees[ssn] = Record(ssn=ssn, first=first, last=last, > salary=salary) > > and I have completely avoided the need to define a unique class. Note > that as the number of fields increases, this becomes easier to use > inline than the dict method, at the same time that it avoids the > upfront costs of having to build a new class for every distinct type > of object in the program. > > It turns out that other people find such a method useful as well. > According to Catherine Devlin on the centralOH Python list, it is > recipe 4.18 from the Python Cookbook (2nd ed.). Several others from > the mailing list stated that they had created similar solutions > themselves. > > Thus, my suggestion is to simply build such functionality directly > into the language. While scanning the built-in functions page (I don't > use any, all, or enumerate nearly enough), I noticed the object() > function, and since its purpose is to return a featureless object, it > seems to fit the bill quite well. Adding my suggested functionality > should break nothing (since the kwargs would be optional) and would > allow people to stop baking their own solutions to this common > problem, while getting the speed bonus of a C implementation (if it > helps). > > Thus, the code above becomes > > for... > employees[ssn] = object(ssn=ssn, first=first, last=last, > salary=salary) > > employees[SSN].first > > > Does anyone else think this might be a good idea for Python 2.6/3K? > > Thanks, > Brandon > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From bruce at leapyear.org Thu May 22 17:36:26 2008 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 22 May 2008 08:36:26 -0700 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> Message-ID: On Thu, May 22, 2008 at 12:21 AM, Brandon Mintern wrote: > I would like to propose to change the built-in function "object" to > have the following syntax: > > object(**kwargs) > Return a new featureless object. object is a base for all new style > classes. It has the methods that are common to all instances of new > style classes. > > I agree this is a useful pattern. One thing I don't like about it is that: object(x=1, y=2) looks very much like object(x=1, y=2, z=3) and I probably won't discover I've been given the wrong one until it's too late. I think it can better addressed by implementing NamedList and NamedDict: NamedList(typename, fieldnames[, optionalfields[, verbose]]) Returns a new list subclass named typename. The new subclass is used to create list-like objects that have fields accessible by attribute lookup as well as supporting other list operations. Instances of a NamedList may be created using a mixture of positional and keyword arguments. If optionalfields is not true, then the NamedList must always contain at least as many elements as the number of fields. If the NamedDict contains fewer elements than the number of fields, missing fields return None when accessed by attribute (a.third) and raise IndexError when accessed by index (a[3]). NamedDict(typename, fieldnames[, optionalfields[, verbose]]) Returns a new dict subclass named typename. The new subclass is used to create dict-like objects that have fields accessible by attribute lookup as well as supporting other dict operations. Instances of a NamedDict may be created using keyword arguments only. If optionalfields is not true, then the NamedDict must have a value for every field. If a NamedDict does not contain a field, accessing it returns None when accessed by attribute (a.x) and raises KeyError when accessed using by key (a['x']). --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu May 22 19:32:14 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 22 May 2008 10:32:14 -0700 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: <20080522122204.GC9262@panix.com> References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> <20080522122204.GC9262@panix.com> Message-ID: On Thu, May 22, 2008 at 5:22 AM, Aahz wrote: > On Thu, May 22, 2008, Brandon Mintern wrote: >> >> I would like to propose to change the built-in function "object" to >> have the following syntax: >> >> object(**kwargs) >> Return a new featureless object. object is a base for all new style >> classes. It has the methods that are common to all instances of new >> style classes. > > Enh. It's easy enough to write > > class Obj: > def __init__(self, **kwargs): > self.__dict__.update(kwargs) > > It's not clear to me that your functionality is desired frequently enough > *in that specific form* to warrant changing object(). Note particularly > the emphasis; over the years, I've seen various small variations in how > people want kwargs processed and it's not at all clear to me that > codifying one specific form into the language would be helpful. This is the answer that came to my mind. Also note that not all classes give their instances a __dict__, and object (being the simplest object possible) does not -- so object(foo=1) wold not have a place where to store the 'foo': 1 mapping. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From adlaiff6 at gmail.com Thu May 22 20:34:27 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 22 May 2008 14:34:27 -0400 (EDT) Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: <20080522121819.GB9262@panix.com> References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com> <20080522121819.GB9262@panix.com> Message-ID: On Thu, 22 May 2008, Aahz wrote: > Um, what? lambda is still here... They are leaving in python 3.0 (and maybe 2.6, I forget), my mistake. -- Cheers, Leif From adlaiff6 at gmail.com Thu May 22 20:47:01 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 22 May 2008 14:47:01 -0400 (EDT) Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> Message-ID: On Thu, 22 May 2008, Bruce Leban wrote: > I think it can better addressed by implementing NamedList and NamedDict: Oh no... > NamedList(typename, fieldnames[, optionalfields[, verbose]]) > > Returns a new list subclass named typename. The new subclass is used to > create list-like objects that have fields accessible by attribute lookup as > well as supporting other list operations. Instances of a NamedList may be > created using a mixture of positional and keyword arguments. If > optionalfields is not true, then the NamedList must always contain at least > as many elements as the number of fields. If the NamedDict contains fewer > elements than the number of fields, missing fields return None when accessed > by attribute (a.third) and raise IndexError when accessed by index (a[3]). Is this just a variable-length version of NamedTuple? It doesn't seem to offer very much over NamedTuple to me, but I'm open to convincing. > NamedDict(typename, fieldnames[, optionalfields[, verbose]]) > > Returns a new dict subclass named typename. The new subclass is used to > create dict-like objects that have fields accessible by attribute lookup as > well as supporting other dict operations. Instances of a NamedDict may be > created using keyword arguments only. If optionalfields is not true, then > the NamedDict must have a value for every field. If a NamedDict does not > contain a field, accessing it returns None when accessed by attribute (a.x) > and raises KeyError when accessed using by key (a['x']). This one, though, doesn't sound any different from a struct-style class, except that it's easier (by ".__dict__") to access the class as a dictionary. Hell, here you go: >>> class NamedDict(object): >>> def __getitem__(self, key): >>> return self.__dict__[key] -- Cheers, Leif From george.sakkis at gmail.com Thu May 22 21:04:10 2008 From: george.sakkis at gmail.com (George Sakkis) Date: Thu, 22 May 2008 15:04:10 -0400 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> <1ef034530805220049sa915f66v423430481fe3d512@mail.gmail.com> <20080522121819.GB9262@panix.com> Message-ID: <91ad5bf80805221204x43ba4046g33236c24a23f967c@mail.gmail.com> On Thu, May 22, 2008 at 2:34 PM, Leif Walsh wrote: > On Thu, 22 May 2008, Aahz wrote: >> Um, what? lambda is still here... > > They are leaving in python 3.0 (and maybe 2.6, I forget), my mistake. Wrong again, they are not leaving at all. George From bruce at leapyear.org Thu May 22 21:14:34 2008 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 22 May 2008 12:14:34 -0700 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> Message-ID: On Thu, May 22, 2008 at 11:47 AM, Leif Walsh wrote: > On Thu, 22 May 2008, Bruce Leban wrote: > > I think it can better addressed by implementing NamedList and NamedDict: > > Oh no... > > > NamedList(typename, fieldnames[, optionalfields[, verbose]]) > > > > Returns a new list subclass named typename. The new subclass is used to > > create list-like objects that have fields accessible by attribute lookup > as > > well as supporting other list operations. Instances of a NamedList may be > > created using a mixture of positional and keyword arguments. If > > optionalfields is not true, then the NamedList must always contain at > least > > as many elements as the number of fields. If the NamedDict contains fewer > > elements than the number of fields, missing fields return None when > accessed > > by attribute (a.third) and raise IndexError when accessed by index > (a[3]). > > Is this just a variable-length version of NamedTuple? It doesn't seem > to offer very much over NamedTuple to me, but I'm open to convincing. > Tuples are immutable. Lists are mutable. The reason I suggest making it variable length is so that it supports the full list semantics and you could for example create a NamedList object with no values and then add values to it (ditto for NamedDict). > > > NamedDict(typename, fieldnames[, optionalfields[, verbose]]) > > > > Returns a new dict subclass named typename. The new subclass is used to > > create dict-like objects that have fields accessible by attribute lookup > as > > well as supporting other dict operations. Instances of a NamedDict may be > > created using keyword arguments only. If optionalfields is not true, then > > the NamedDict must have a value for every field. If a NamedDict does not > > contain a field, accessing it returns None when accessed by attribute > (a.x) > > and raises KeyError when accessed using by key (a['x']). > > This one, though, doesn't sound any different from a struct-style > class, except that it's easier (by ".__dict__") to access the class as > a dictionary. Hell, here you go: > > >>> class NamedDict(object): > >>> def __getitem__(self, key): > >>> return self.__dict__[key] > Well, I think the "Named" aspect is important. For example, repr(x) tells you the name. I can tell if two objects are the same type. Also the NamedDict supports attribute access to the specific fields you've declared and no others. So it's not a freewheeling object that you can add any attribute to it. It's just a lightweight way to create an object backed by a dict (just as NamedTuple is). > -- > Cheers, > Leif > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adlaiff6 at gmail.com Thu May 22 21:30:47 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 22 May 2008 15:30:47 -0400 (EDT) Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> Message-ID: On Thu, 22 May 2008, Bruce Leban wrote: > Tuples are immutable. Lists are mutable. The reason I suggest making it > variable length is so that it supports the full list semantics and you could > for example create a NamedList object with no values and then add values to > it (ditto for NamedDict). I might be wrong here, but I think one of the reasons NamedTuple was decided upon (its immutability, specifically) was a performance issue. It's a lot easier to optimize a struct when you know exactly what its fields are. I think making a mutable one of these is really not offering all that much advantage to anyone in a large number of cases. > Well, I think the "Named" aspect is important. For example, repr(x) tells > you the name. I can tell if two objects are the same type. Well, right. That's the point of a struct-style class. You name it whatever you would like. > Also the NamedDict supports attribute access to the specific fields you've > declared and no others. So it's not a freewheeling object that you can add > any attribute to it. It's just a lightweight way to create an object backed > by a dict (just as NamedTuple is). I'm confused. I thought you said you wanted mutability. Now you can only have the fieldnames you specify in the beginning? This is not really how a dict usually operates. That said, you can mess with __slots__ all you want in your free time, but I think we're straying pretty far from "one obvious way" here. -- Cheers, Leif From bmintern at gmail.com Thu May 22 22:01:49 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Thu, 22 May 2008 16:01:49 -0400 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> Message-ID: <4c0fccce0805221301g57bd9b8ai27554530319a6484@mail.gmail.com> A lot to address here. On Thu, May 22, 2008 at 9:43 AM, Calvin Spealman wrote: > Start using those and suddenly you are going to have lots of little > anonymous objects around and you'll start to want to add functionality to > them and otherwise enforce things about them. Now you have a higher barrier > than when you would have been creating it from scratch. That wasn't my intended use case, though. I wouldn't be using an anonymous object in more than one place to create the same type of object. In that case, actually defining a class or using NamedTuple (or some equivalent) clearly wins out. This would be a similar situation to that of lambda. You would not use lambda in several different places to define the same function; as soon as you start to write it a second time, you'd be much better to def the function and use it in both places. My main use case is in packing related information together for convenience and readability when passing it around to several different functions. In other words, the exact case where someone might be tempted to use a tuple (in spite of poor readability later) or a dict (in spite of a cumbersome access pattern). If you're using it for a case like this and you later decide that you should expand the anonymous object to a full-blown class, you would only have to change it in one place. Moreover, it would be much easier than changing a tuple or dict to a class. In the anonymous object situation, you only have to change the one instantiation; as long as you create a class with the same field names, your other accesses are still valid. In the tuple or dict cases, however, you have to change the code everywhere that you are using the object. Thus, I feel that having an anonymous object available actually lowers the barrier to making smart changes to the code. > and remember, object is not a function. It is a type. It is the base type of > all types and so this has implications beyond what you suggest it for. If > object(a=10).a == 10, then when custom classes' __init__ methods upcall, > they'll start to fall back on letting kwargs fall up to object.__init__ to > set initialization attributes. Now we gain this rule of thumb that keyword > arguments are probably going to end up as attributes, because if we don't > 'catch' them and they reach object.__init__, they will be. This isn't a > behavior the language should encourage, because it should be left up to the > developer. Currently, object.__init__ specifically takes any arguments and > does nothing with any of them. This is to allow upcall in your __init__ > gracefully, because if the upclass is one of your bases or the final base, > object, you can still pass your parameters along safely. Implying some > behavior when it reaches object.__init__, especially (in most cases) after > you've initialized your object, would probably just trounce all over your > object and piss you off. This is a very good argument for why object() should not be used for this purpose; you have completely changed my mind. I still think there would be value in having a built-in function for declaring such objects. Perhaps a function (constructor) named struct(**kwargs) would be appropriate? On Thu, May 22, 2008 at 8:22 AM, Aahz wrote: > Enh. It's easy enough to write > > class Obj: > def __init__(self, **kwargs): > self.__dict__.update(kwargs) Yes, it is easy enough, once you have worked with Python a lot. Personally, I didn't know much at all about __dict__ or **kwargs when I first started, and looking back at my old code now, there are several places where I have used tuples or dicts and the code is hard to manage. Having a struct type to hold related information which does not justify a full class would have been very nice. > It's not clear to me that your functionality is desired frequently enough > *in that specific form* to warrant changing object(). Note particularly > the emphasis; over the years, I've seen various small variations in how > people want kwargs processed and it's not at all clear to me that > codifying one specific form into the language would be helpful. > No thanks. This is another good argument for why object() should not be used for this purpose. I still think that adding a new type "struct" would have several benefits. Others wrote: > NamedList, NamedDict, NamedTuple... I agree that these are very useful functions to have when what you really want to do is just subclass the indicated type. For the use case I have in mind, however, they have much more functionality (and take up more space?) than necessary, and specifically having to decide on and declare a name that will never be used constitutes wasted effort. I will be sending out a new reply with my updated proposal, using the name "struct" instead of multi-purposing object, as that is clearly a bad idea. Thanks for all the constructive criticism and feedback, Brandon From bmintern at gmail.com Thu May 22 23:42:28 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Thu, 22 May 2008 17:42:28 -0400 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) Message-ID: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> This is a proposal to add a new built-in named struct: struct(**kwargs) Return a struct object which has the attributes given in kwargs. The name is really unimportant, and I'm open to other ideas, but I feel that many people have a good idea of what a struct is, and accessing an object returned from struct would look identical to the access of a struct in C, so it seems appropriate to me. The rationale: It is often helpful to package related information together in order to make passing the information to various functions more convenient. The easiest ways (currently) to package such information is to put it into a tuple (the easiest) or a dict. Putting the information into a tuple may be easy initially, but it has high costs later on as it becomes hard to remember what order the information is in, and to someone reading the code, the intent of the code is far from clear. Putting the information in a dict is a bit harder than a tuple, but since it is more readable later on than a tuple, this method is often used. Still, the access pattern is more cumbersome than it could be; foo["bar"] is more cumbersome than, say, foo.bar. This is especially the case if you have a dict or list of foos, where you then have to use foos[i]["bar"]. Both tuple and dict solutions suffer down the line when the information gets to be complicated enough to warrant a class of its own. It involves changing both the spot in the code where the information is created (to use the new class constructor), as well as changing every single field access in the code (changing every foo[0] or foo["bar"] to foo.bar). An alternative is to use NamedTuple, NamedDict, NamedList, or to create your own class. As long as these are more complicated to use than a tuple or a dict, however, they are not likely to be used for this purpose. Another problem is that all of these methods require you to go to the trouble of thinking of a name for your class, and if you later decide to add more information to your packaged object, you have to make two changes (in the list of attributes / constructor and in the place where you instantiate your object). Enter struct. Using struct is intended to be just as easy as using a dict (actually, easier when the number of fields is more than two or three), and not much harder than using a tuple. To declare a struct foo with attribute bar, you simply use: foo = struct(bar="barvalue") Access becomes very easy and readable: foo.bar Adding new fields is as easy as changing the initial instantiation, in one place: foo = struct(bar="barvalue", baz="bazvalue") Later on down the line, when you decide that you are doing more with foo than a struct should be doing, you can easily define a class Foo which inherits from struct, and since accesses to foo already look like foo.bar, you only have one spot in the code to change: foo = Foo(bar="barvalue", baz="bazvalue") and the rest "just works" with no changes. The implementation: class struct (object): def __init__ (self, **kwargs): self.__dict__.update(kwargs) def __repr__ (self): """ Using self.__class__.__name__ allows classes to inherit from struct and automatically have a nice __repr__ method. """ return "%s(%s)" % (self.__class__.__name__, ", ".join("%s=%s" % (attr, repr(val)) for attr, val in self.__dict__.iteritems()),) # or .items() # in Python 3K def __str__ (self): return self.__repr__() def __eq__ (self, other): """ Implements comparison operation mirroring that of a C struct. """ return self.__dict__ == other.__dict__ def __ne__ (self, other): """ See note for __eq__. """ return not self.__eq__(other) def __setattr__ (self, name, value): """ I think it makes the most sense for a struct to have immutable fields. As soon as you start to add more fields, you should be using something other than a struct. """ if name in self.__dict__: self.__dict__[name] = value else: raise(AttributeError("'struct' object has no attribute '%s'" \ % (name,))) def __len__ (self): """ I'm not sure that it's really necessary to include this, but I could see where it might be helpful in some instances. """ return len(self.__dict__) def __iter__ (self): """ See note for __len__ """ return self.__dict__.itervalues() # or .values() in Python 3K Sample usage: >>> a = struct(one=1, two=2) >>> a struct(two=2, one=1) >>> a.one 1 >>> a.two 2 >>> a.three Traceback (most recent call last): File "", line 1, in AttributeError: 'struct' object has no attribute 'three' >>> a.one = "one" >>> a.one 'one' >>> a.three = 3 Traceback (most recent call last): File "", line 1, in File "struct.py", line 39, in __setattr__ % (name,))) AttributeError: 'struct' object has no attribute 'three' >>> b = struct(one=1, two=2) >>> b struct(two=2, one=1) >>> a == b False >>> a.one = 1 >>> a == b True >>> len(a) 2 >>> print ", ".join(str(v) for v in a) 2, 1 >>> 1 in a True >>> "one" in a False Ideas or feedback, anyone? From bmintern at gmail.com Thu May 22 23:55:35 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Thu, 22 May 2008 17:55:35 -0400 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> Message-ID: <4c0fccce0805221455r5a6204aj6400ed5dd0b5b867@mail.gmail.com> On Thu, May 22, 2008 at 5:42 PM, Brandon Mintern wrote: > This is a proposal to add a new built-in named struct: One thing I forgot to mention... this is mainly intended to be used where it makes sense to quickly build an anonymous object. It would be considered bad practice to have more than one place in the code which creates a struct having the same fields; in that case a NamedTuple (or some equivalent) would be more appropriate. In other words, this would be used in a similar situation to that of lambda. You would not use lambda in several different places to define the same function; as soon as you start to write it a second time, you'd be much better to def the function and use it in both places. In the same way, creating the same struct in two places is a good indication that a NamedTuple or an actual class (possibly inheriting from struct and using a non-kwargs constructor) is more appropriate. From brett at python.org Fri May 23 00:10:02 2008 From: brett at python.org (Brett Cannon) Date: Thu, 22 May 2008 15:10:02 -0700 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> Message-ID: On Thu, May 22, 2008 at 2:42 PM, Brandon Mintern wrote: > This is a proposal to add a new built-in named struct: > > struct(**kwargs) > Return a struct object which has the attributes given in kwargs. > > The name is really unimportant, and I'm open to other ideas, but I > feel that many people have a good idea of what a struct is, and > accessing an object returned from struct would look identical to the > access of a struct in C, so it seems appropriate to me. > > > The rationale: > > It is often helpful to package related information together in order > to make passing the information to various functions more convenient. > The easiest ways (currently) to package such information is to put it > into a tuple (the easiest) or a dict. > > Putting the information into a tuple may be easy initially, but it has > high costs later on as it becomes hard to remember what order the > information is in, and to someone reading the code, the intent of the > code is far from clear. > > Putting the information in a dict is a bit harder than a tuple, but > since it is more readable later on than a tuple, this method is often > used. Still, the access pattern is more cumbersome than it could be; > foo["bar"] is more cumbersome than, say, foo.bar. This is especially > the case if you have a dict or list of foos, where you then have to > use foos[i]["bar"]. > So you save three characters? I don't call that cumbersome. > Both tuple and dict solutions suffer down the line when the > information gets to be complicated enough to warrant a class of its > own. It involves changing both the spot in the code where the > information is created (to use the new class constructor), as well as > changing every single field access in the code (changing every foo[0] > or foo["bar"] to foo.bar). > > An alternative is to use NamedTuple, NamedDict, NamedList, or to > create your own class. As long as these are more complicated to use > than a tuple or a dict, however, they are not likely to be used for > this purpose. Another problem is that all of these methods require you > to go to the trouble of thinking of a name for your class, and if you > later decide to add more information to your packaged object, you have > to make two changes (in the list of attributes / constructor and in > the place where you instantiate your object). > Thinking of a name for your class is not difficult, especially if you keep it private to the module, class, function, etc. This does not strike me as useful enough to have as a built-in. It would be better placed in the stdlib. -Brett From bmintern at gmail.com Fri May 23 00:24:19 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Thu, 22 May 2008 18:24:19 -0400 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> Message-ID: <4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com> On Thu, May 22, 2008 at 6:10 PM, Brett Cannon wrote: > So you save three characters? I don't call that cumbersome. If foo["bar"] is not cumbersome, it is at least less elegant and the intent is less clear than foo.bar. Moreover, as I stated in the next paragraph, it does become cumbersome down the line when you decide that you should have used a class after all, and now you have to change all of those foo["bar"] lines to foo.bar. Note that simple search-and-replace wouldn't help if you are passing foo to various functions. > Thinking of a name for your class is not difficult, especially if you > keep it private to the module, class, function, etc. It may not be difficult, but when the name is unnecessary, simply needing to declare it seems silly. > This does not strike me as useful enough to have as a built-in. It > would be better placed in the stdlib. I would be happy with it at least becoming part of collections or some other module, but then I wonder how many new-ish Python programmers would persist in using a tuple or a dict instead of a more elegant struct solution for lack of knowing about it. At least if it was in Python somewhere, though, searching for "python struct" would be more likely to return what the programmer is looking for. Ouch... it seems that struct is already the name of a module. If enough people like my idea, perhaps that module could be renamed to "cstruct". Then again, if my idea did become a part of collections (rather than a built-in), collections.struct and the struct module would be able to co-exist, albeit somewhat confusingly. Brandon From bmintern at gmail.com Fri May 23 00:53:31 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Thu, 22 May 2008 18:53:31 -0400 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <3b5110850805221525k6237c647u38d65a78a14fedf3@mail.gmail.com> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <3b5110850805221525k6237c647u38d65a78a14fedf3@mail.gmail.com> Message-ID: <4c0fccce0805221553s77d7d047r995a6222d39bd2ba@mail.gmail.com> On Thu, May 22, 2008 at 6:25 PM, Matthew Russell wrote: > How about: > collections.attrs(**kwargs) > or > collections.record(**kwargs) I've used the name "Record" in the past, but when I was trying to figure out if Python had anything like a C struct, I searched specifically for "python struct". I'm not sure how many others do the same, but perhaps using a name like "structure" would be better than "record"? (On a side note, see what I mean about the work involved in naming a class? :-) > I think I prefer the idea of this being a factory that returns an anonymous > object as opposed to a class: > > def record(**kwargs): > class Record(object): > def __init__(self, **kw): > for (k, v) in kwargs.items(): > setattr(self, k, v) > return Record() I'm guessing that the last line was meant to be "return Record(**kwargs)" > This is mainly because a class you later substitude for the > result of record(**kwargs) may later end up wanting to take kwargs as > optional values which you might not want as instance variables, or even > posistional arguements. > > x = MyThing(x=1,y=2, default_spam=False) > > > So you could easilty do: > > x = MyThing(default_spam=False, **record_obj.__dict__) > > Matt I can see where you're coming from, but this change would mean that you would no longer be able to inherit from record (or structure, or whatever). Also, if you're making that many changes to the class, it wouldn't be too hard to rewrite the initial instantiation to not use positional parameters with or without kwargs (after rewriting the constructor, which you would clearly be doing, anyways). Another caveat is that each call to record() will create an object of a different class. Thus, if you have the following: employees = {} for ssn, fn, ln in employee_data_list: employees[ssn] = record(ssn=ssn, first=fn, last=ln) Then employees[ssn1].__class__ != employees[ssn2].__class__. This seems like it could be an issue at some point. Brandon From brett at python.org Fri May 23 01:00:11 2008 From: brett at python.org (Brett Cannon) Date: Thu, 22 May 2008 16:00:11 -0700 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com> Message-ID: On Thu, May 22, 2008 at 3:24 PM, Brandon Mintern wrote: > On Thu, May 22, 2008 at 6:10 PM, Brett Cannon wrote: >> So you save three characters? I don't call that cumbersome. > > If foo["bar"] is not cumbersome, it is at least less elegant and the > intent is less clear than foo.bar. Moreover, as I stated in the next > paragraph, it does become cumbersome down the line when you decide > that you should have used a class after all, and now you have to > change all of those foo["bar"] lines to foo.bar. Note that simple > search-and-replace wouldn't help if you are passing foo to various > functions. > But you are suggesting that people think far enough ahead to think that a sequence or mapping will be cumbersome and thus something with attribute access should be used instead. >> Thinking of a name for your class is not difficult, especially if you >> keep it private to the module, class, function, etc. > > It may not be difficult, but when the name is unnecessary, simply > needing to declare it seems silly. > Well, we almost ditched lambda and were going to require people to define a simple function to replace lambda functions, so not everyone thinks it is silly. >> This does not strike me as useful enough to have as a built-in. It >> would be better placed in the stdlib. > > I would be happy with it at least becoming part of collections or some > other module, but then I wonder how many new-ish Python programmers > would persist in using a tuple or a dict instead of a more elegant > struct solution for lack of knowing about it. At least if it was in > Python somewhere, though, searching for "python struct" would be more > likely to return what the programmer is looking for. > Sticking something in built-ins so that it is easier for newbies to find it is not a good argument. Things only go into builtins if they are frequently used and warrant skipping an import statement. > Ouch... it seems that struct is already the name of a module. If > enough people like my idea, perhaps that module could be renamed to > "cstruct". Then again, if my idea did become a part of collections > (rather than a built-in), collections.struct and the struct module > would be able to co-exist, albeit somewhat confusingly. > I don't agree with that worry. re.compile() exists and people don't worry about conflicting with the built-in function. An import statement makes it clear what object 'struct' maps to in the namespace. -Brett From greg.ewing at canterbury.ac.nz Fri May 23 01:03:00 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 23 May 2008 11:03:00 +1200 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> Message-ID: <4835FBA4.8020206@canterbury.ac.nz> Brandon Mintern wrote: > This is a proposal to add a new built-in named struct: > > struct(**kwargs) > Return a struct object which has the attributes given in kwargs. I think I'd prefer 'record', to avoid any potential confusion with the struct module, which does something quite different. Also, my Pascal background makes the term 'record' seem more high-level and therefore Pythonic to me. -- Greg From greg.ewing at canterbury.ac.nz Fri May 23 01:07:54 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 23 May 2008 11:07:54 +1200 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com> Message-ID: <4835FCCA.7020001@canterbury.ac.nz> Brett Cannon wrote: > I don't agree with that worry. re.compile() exists and people don't > worry about conflicting with the built-in function. But that's just a matter of two functions with the same name in different places. It's not a case of something being a function or type in one place and a module in another. That would be more confusing, I think. -- Greg From adlaiff6 at gmail.com Fri May 23 01:15:08 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 22 May 2008 19:15:08 -0400 (EDT) Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> Message-ID: On Thu, 22 May 2008, Brandon Mintern wrote: > The implementation: > > class struct (object): > > > > def __setattr__ (self, name, value): > """ > I think it makes the most sense for a struct to have immutable > fields. As soon as you start to add more fields, you should be > using something other than a struct. > """ > if name in self.__dict__: > self.__dict__[name] = value > else: > raise(AttributeError("'struct' object has no attribute '%s'" \ > % (name,))) I think it makes the most sense, if this construct is adopted, to use __slots__ to control mutability. Someone more well-versed in the python object model should determine if this is actually a good idea. -- Cheers, Leif From bmintern at gmail.com Fri May 23 01:17:16 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Thu, 22 May 2008 19:17:16 -0400 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4c0fccce0805221524t7a86886fu35229cdda0c4b168@mail.gmail.com> Message-ID: <4c0fccce0805221617v37ec425akfc8505b9d6ffbd7f@mail.gmail.com> On Thu, May 22, 2008 at 7:00 PM, Brett Cannon wrote: >> If foo["bar"] is not cumbersome, it is at least less elegant and the >> intent is less clear than foo.bar. Moreover, as I stated in the next >> paragraph, it does become cumbersome down the line when you decide >> that you should have used a class after all, and now you have to >> change all of those foo["bar"] lines to foo.bar. Note that simple >> search-and-replace wouldn't help if you are passing foo to various >> functions. >> > > But you are suggesting that people think far enough ahead to think > that a sequence or mapping will be cumbersome and thus something with > attribute access should be used instead. Perhaps a lot of people wouldn't, unless the tutorial reflected the idea of using struct for anonymous, structured data. When I started out, I initially settled on tuples because it was the easiest to build up front, and I didn't see any better alternatives. After the frustration of maintaining tuple-based structures, I began to use dicts instead, but I didn't like the syntax all that much, and it looks weird (in my opinion) to have code that declares and indexes a dictionary when what you really want is something like a C struct. I guess what I'm suggesting is that if a clear solution is presented that fits the use case well, I would expect a nontrivial number of people to use it. They would not be required to notice that it is then easy to convert to a real class, because good practice is "enforced" by the use of struct. >>> Thinking of a name for your class is not difficult, especially if you >>> keep it private to the module, class, function, etc. >> >> It may not be difficult, but when the name is unnecessary, simply >> needing to declare it seems silly. >> > > Well, we almost ditched lambda and were going to require people to > define a simple function to replace lambda functions, so not everyone > thinks it is silly. I'm sorry to keep hammering on this issue, but anonymous functions already have standard names from Math: f, g, h, fn, func, etc. In other words, def f (x, y): return x+y f is not much harder than lambda x,y: x+y (except for the fact that it cannot be used as an expression, which is why I personally still like lambda). With class names, there is no such convention. Therefore, something like c = NamedTuple("C", "one two") a = c(1, 2) seems a bit strange. Why "c" and "C"? Why anything? Why should I have to include "C" as an argument only to never use it? I think that a = struct(one=1, two=2) clearly beats out the NamedTuple solution if all you really need is a struct. >>> This does not strike me as useful enough to have as a built-in. It >>> would be better placed in the stdlib. >> >> I would be happy with it at least becoming part of collections or some >> other module, but then I wonder how many new-ish Python programmers >> would persist in using a tuple or a dict instead of a more elegant >> struct solution for lack of knowing about it. At least if it was in >> Python somewhere, though, searching for "python struct" would be more >> likely to return what the programmer is looking for. >> > > Sticking something in built-ins so that it is easier for newbies to > find it is not a good argument. Things only go into builtins if they > are frequently used and warrant skipping an import statement. Fair enough. It would make sense to initially put it into collections, make sure people know about it, and then see how often it's used. >> Ouch... it seems that struct is already the name of a module. If >> enough people like my idea, perhaps that module could be renamed to >> "cstruct". Then again, if my idea did become a part of collections >> (rather than a built-in), collections.struct and the struct module >> would be able to co-exist, albeit somewhat confusingly. >> > > I don't agree with that worry. re.compile() exists and people don't > worry about conflicting with the built-in function. An import > statement makes it clear what object 'struct' maps to in the > namespace. Good, because I much-prefer using the name "struct" since it maps so closely to a C struct. Brandon From bmintern at gmail.com Fri May 23 01:22:24 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Thu, 22 May 2008 19:22:24 -0400 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> Message-ID: <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com> On Thu, May 22, 2008 at 7:15 PM, Leif Walsh wrote: > On Thu, 22 May 2008, Brandon Mintern wrote: >> The implementation: >> >> class struct (object): >> >> >> >> def __setattr__ (self, name, value): >> """ >> I think it makes the most sense for a struct to have immutable >> fields. As soon as you start to add more fields, you should be >> using something other than a struct. >> """ >> if name in self.__dict__: >> self.__dict__[name] = value >> else: >> raise(AttributeError("'struct' object has no attribute '%s'" \ >> % (name,))) > > I think it makes the most sense, if this construct is adopted, to use > __slots__ to control mutability. Someone more well-versed in the > python object model should determine if this is actually a good idea. > > -- > Cheers, > Leif Whoops... I should have said "An implementation." I intended the implementation to be a specification for behavior and not the one-and-only implementation. Personally, I'm not especially familiar with using __slots__ in anything but standard usage, so I found it easier to show the code using __dict__. Certainly if __slots__ can be used, it avoids the need to write __setattr__ and to explicitly raise the AttributeError exception (which I already see is wrong because it doesn't support inheritance). From adlaiff6 at gmail.com Fri May 23 01:49:05 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 22 May 2008 19:49:05 -0400 (EDT) Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com> Message-ID: On Thu, 22 May 2008, Brandon Mintern wrote: > > I think it makes the most sense, if this construct is adopted, to use > > __slots__ to control mutability. Someone more well-versed in the > > python object model should determine if this is actually a good idea. > > > > -- > > Cheers, > > Leif > > Whoops... I should have said "An implementation." I intended the > implementation to be a specification for behavior and not the > one-and-only implementation. Personally, I'm not especially familiar > with using __slots__ in anything but standard usage, so I found it > easier to show the code using __dict__. Certainly if __slots__ can be > used, it avoids the need to write __setattr__ and to explicitly raise > the AttributeError exception (which I already see is wrong because it > doesn't support inheritance). Don't take my word as gospel. I've only used __slots__ once, and it was at the suggest of another programmer in a kind of "I think this does what we want" suggestion, and we are still not entirely sure if it's doing what we think it's doing. -- Cheers, Leif From greg.ewing at canterbury.ac.nz Fri May 23 00:29:59 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 23 May 2008 10:29:59 +1200 Subject: [Python-ideas] Add kwargs to built-in function object In-Reply-To: References: <4c0fccce0805220021v1f64e515q9f227650e7954ebc@mail.gmail.com> Message-ID: <4835F3E7.7090207@canterbury.ac.nz> Calvin Spealman wrote: > and remember, object is not a function. It is a type. It is the base > type of all types and so this has implications beyond what you suggest > it for. Another serious consequence is that supporting this would require the object type to have a __dict__, which would force *all* types, including the built-in ones, to have the overhead of a __dict__ as well. -- Greg From greg.ewing at canterbury.ac.nz Fri May 23 03:59:36 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 23 May 2008 13:59:36 +1200 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com> Message-ID: <48362508.3080706@canterbury.ac.nz> On Thu, May 22, 2008 at 7:15 PM, Leif Walsh wrote: > I think it makes the most sense, if this construct is adopted, to use > __slots__ to control mutability. That wouldn't work, because the slots that a class has are fixed when the class is defined. Changing __slots__ subsequent to that doesn't affect anything. -- Greg From greg.ewing at canterbury.ac.nz Fri May 23 04:02:10 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 23 May 2008 14:02:10 +1200 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com> Message-ID: <483625A2.1000206@canterbury.ac.nz> Leif Walsh wrote: > I've only used __slots__ once, and it > was at the suggest of another programmer in a kind of "I think this > does what we want" suggestion, and we are still not entirely sure if > it's doing what we think it's doing. Another thing to keep in mind about __slots__ is that its primary purpose is to save memory. It currently has the side effect of preventing other attributes from being added, but it may not remain that way forever. -- Greg From george.sakkis at gmail.com Fri May 23 07:23:54 2008 From: george.sakkis at gmail.com (George Sakkis) Date: Fri, 23 May 2008 01:23:54 -0400 Subject: [Python-ideas] Queue suggestions Message-ID: <91ad5bf80805222223q32a92cb8sfb0ebbca92db5b99@mail.gmail.com> I'd like to propose a signature simplification and a new method for the Queue.Queue class: 1) Drop the optional `block` argument from put() and get(), since all the meaningful combinations of (block, timeout) are equivalent to passing block = (timeout is None or timeout>0). IOW, instead of passing block=False pass timeout=0 (or negative). Obviously this is to be considered for 3.x only. 2) Add a new `rotate` method as the atomic equivalent of put_nowait(get_nowait()). Currently I use the following subclass but it would be nice to have it in the base Queue: class RQueue(Queue): def rotate(self, n=1): '''Rotate this queue n steps to the left (if it is not empty). Rotating one step is equivalent to an atomic q.put_nowait(q.get_nowait()). ''' if n < 0: raise ValueError('n must be non-negative') self.mutex.acquire() try: if not self._empty(): self._rotate(n) finally: self.mutex.release() def __init__(self, maxsize=0): Queue.__init__(self, maxsize) if hasattr(self.queue, 'rotate'): # deque has rotate() since v2.5 def _rotate(n): # negative n for left rotate self.queue.rotate(-n) else: def _rotate(n): put,get = self._put, self._get for i in xrange(n): put(get()) self._rotate = _rotate What do you think ? George From greg at krypto.org Fri May 23 08:56:58 2008 From: greg at krypto.org (Gregory P. Smith) Date: Thu, 22 May 2008 23:56:58 -0700 Subject: [Python-ideas] Queue suggestions In-Reply-To: <91ad5bf80805222223q32a92cb8sfb0ebbca92db5b99@mail.gmail.com> References: <91ad5bf80805222223q32a92cb8sfb0ebbca92db5b99@mail.gmail.com> Message-ID: <52dc1c820805222356j607a3511n868950da0c56df30@mail.gmail.com> Do you have a compelling use case for Queue.rotate? On Thu, May 22, 2008 at 10:23 PM, George Sakkis wrote: > I'd like to propose a signature simplification and a new method for > the Queue.Queue class: > > 1) Drop the optional `block` argument from put() and get(), since all > the meaningful combinations of (block, timeout) are equivalent to > passing block = (timeout is None or timeout>0). IOW, instead of > passing block=False pass timeout=0 (or negative). Obviously this is to > be considered for 3.x only. > > 2) Add a new `rotate` method as the atomic equivalent of > put_nowait(get_nowait()). Currently I use the following subclass but > it would be nice to have it in the base Queue: > > class RQueue(Queue): > > def rotate(self, n=1): > '''Rotate this queue n steps to the left (if it is not empty). > > Rotating one step is equivalent to an atomic > q.put_nowait(q.get_nowait()). > ''' > if n < 0: > raise ValueError('n must be non-negative') > self.mutex.acquire() > try: > if not self._empty(): > self._rotate(n) > finally: > self.mutex.release() > > def __init__(self, maxsize=0): > Queue.__init__(self, maxsize) > if hasattr(self.queue, 'rotate'): # deque has rotate() since v2.5 > def _rotate(n): > # negative n for left rotate > self.queue.rotate(-n) > else: > def _rotate(n): > put,get = self._put, self._get > for i in xrange(n): > put(get()) > self._rotate = _rotate > > > What do you think ? > > George > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From george.sakkis at gmail.com Fri May 23 08:51:30 2008 From: george.sakkis at gmail.com (George Sakkis) Date: Fri, 23 May 2008 02:51:30 -0400 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <48362508.3080706@canterbury.ac.nz> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com> <48362508.3080706@canterbury.ac.nz> Message-ID: <91ad5bf80805222351p419d3c9cl30225a0860528101@mail.gmail.com> On Thu, May 22, 2008 at 9:59 PM, Greg Ewing wrote: > On Thu, May 22, 2008 at 7:15 PM, Leif Walsh wrote: >> >> I think it makes the most sense, if this construct is adopted, to use >> __slots__ to control mutability. > > That wouldn't work, because the slots that a class has > are fixed when the class is defined. Changing __slots__ > subsequent to that doesn't affect anything. Unless you generate the class on the fly with a class factory, like namedtuple. Actually I prefer a metaclass factory, to avoid the repetition of the class name: >>> class Foo(object): ... __metaclass__ = RecordType('x y z is_on', field_defaults={'is_on':False}, default=0.0) >>> a = Foo() >>> a Foo(x=0.0, y=0.0, z=0.0, is_on=False) >>> a.__slots__ ('x', 'y', 'z', 'is_on') >>> a.__dict__ Traceback (most recent call last): AttributeError: 'Foo' object has no attribute '__dict__' George From carl at carlsensei.com Fri May 23 09:15:06 2008 From: carl at carlsensei.com (Carl Johnson) Date: Thu, 22 May 2008 21:15:06 -1000 Subject: [Python-ideas] Function to unnest for-statements Message-ID: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> This is based off of http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations . It's also discussed at http://reddit.com/info/5ywrs/comments/ and someone with a lot of spare time can read Knuth's fascile on the subject at http://www-cs-faculty.stanford.edu/~knuth/fasc2a.ps.gz . OK, suppose you have a situation where you need to loop through all combinations of 3 lists. The simple solution is to write three nested for-loops: for x in xs: for y in ys: for z in zs: #Do something Of course, this is obvious O(n^3), which is bad, but sometimes you don't have a choice. However, what if (to use an example I ran into), you're making up a truth table. If you know you have three variables, you might write: for a in [True, False]: for b in [True, False]: for c in [True, False]: print a, b, c Which yields True True True True True False True False True True False False False True True False True False False False True False False False But if you don't know how many variables you'll have, you're stuck writing a complicated function instead of using a nice, simple for-loop. So, going back to the first example, wouldn't it be nicer to write: for x, y, z in combine(xs, ys, zs): #Do something If nothing else, this has the advantage that if you don't have to nest the for-loops, things don't end up being so deeply indented on the screen. Similarly, a general truth table maker can be written: ts_and_fs = [[True, False]] * num_vars for variables in combine(*ts_and_fs): print variables So, I think a "combine" function (or some other, better name) would be a good thing to have, at least in the itertools, if not as a built-in function. How to implement it? Well, probably it should be done in C for efficiency, but of the versions done http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations the one that I like best is def combine(*sequences): #Test to guard against infinite recursion if sequences: def _inner(*seqs): if len(seqs) == 1: for i in seqs[0]: yield (i, ) else: for rest in combine(*seqs[:-1]): for i in seqs[-1]: yield rest + (i, ) return _inner(*sequences) else: #Empty generator that yields StopIteration def _inner(): return yield return _inner() However, I'm also convinced that there is a more efficient way to do this. So, what do you think? Is this a common enough need that it should be built into itertools? The main namespace? Or should we leave it out, since adding it would encourage people writing O(n^x) algorithms? If nothing else, list members can enjoy rewriting this function for fun. -- Carl Johnson From ntoronto at cs.byu.edu Fri May 23 10:19:29 2008 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Fri, 23 May 2008 02:19:29 -0600 Subject: [Python-ideas] Function to unnest for-statements In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> Message-ID: <48367E11.90203@cs.byu.edu> Carl Johnson wrote: > So, going back to the first example, wouldn't it be nicer to write: > > for x, y, z in combine(xs, ys, zs): > #Do something > > If nothing else, this has the advantage that if you don't have to nest > the for-loops, things don't end up being so deeply indented on the screen. FWIW, I find myself doing this often with numpy: for index in numpy.ndindex(arr.shape): # do something using arr's indexes, whatever they are (Broadcasting is better, but sometimes this is unavoidable.) The point is there's a general use-case Carl left out: what if you don't know how deeply you'll need to nest? Our Functional friends know the answer to this, but IIRC the Python design philosophy doesn't actively encourage recursive solutions. I don't think the use-cases are common enough to warrant a new builtin (though that's just my $0.000001 opinion - it may be that *everyone* will want to flatten their nested loops), but something like "combine" would fit very well into itertools. Neil From adlaiff6 at gmail.com Fri May 23 10:26:29 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Fri, 23 May 2008 04:26:29 -0400 (EDT) Subject: [Python-ideas] Function to unnest for-statements In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> Message-ID: On Thu, 22 May 2008, Carl Johnson wrote: > So, going back to the first example, wouldn't it be nicer to write: > > for x, y, z in combine(xs, ys, zs): > #Do something > > If nothing else, this has the advantage that if you don't have to nest > the for-loops, things don't end up being so deeply indented on the > screen. I think extra indentation helps, because it makes it clearer how the loops nest, but I suppose you could do this if you really wanted; >>> xs = ys = zs = [True, False] >>> for a, b, c in [(x, y, z) for x in xs for y in ys for z in zs]: ... print a, b, c ... True True True True True False True False True True False False False True True False True False False False True False False False >>> > def combine(*sequences): > #Test to guard against infinite recursion > if sequences: > def _inner(*seqs): > if len(seqs) == 1: > for i in seqs[0]: yield (i, ) > else: > for rest in combine(*seqs[:-1]): > for i in seqs[-1]: > yield rest + (i, ) > return _inner(*sequences) > else: > #Empty generator that yields StopIteration > def _inner(): > return > yield > return _inner() > > However, I'm also convinced that there is a more efficient way to do > this. I've worked this out (I'm sure I did this in intro CS once before...), although there is probably some optimization trick I'm missing, and I'm probably doing some silly double-allocation thing or something. >>> def combine(lists): ... if len(lists) is 0: ... return [[]] ... first = combine(lists[0:-1]) ... total = [] ... for item in lists[-1]: ... total += [[item] + subset for subset in first] ... return total ... >>> for set in combine([[True, False]] * 4): ... print set ... [True, True, True, True] [True, True, True, False] [True, True, False, True] [True, True, False, False] [True, False, True, True] [True, False, True, False] [True, False, False, True] [True, False, False, False] [False, True, True, True] [False, True, True, False] [False, True, False, True] [False, True, False, False] [False, False, True, True] [False, False, True, False] [False, False, False, True] [False, False, False, False] >>> > So, what do you think? Is this a common enough need that it should be > built into itertools? The main namespace? Or should we leave it out, > since adding it would encourage people writing O(n^x) algorithms? If > nothing else, list members can enjoy rewriting this function for fun. I really, really don't think this is worth putting in any distributed package, as it's a pretty simple thing to code up if you're careful. That said, it _was_ a lot of fun rewriting it. Thanks. :D -- Cheers, Leif From adlaiff6 at gmail.com Fri May 23 10:33:05 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Fri, 23 May 2008 04:33:05 -0400 (EDT) Subject: [Python-ideas] Function to unnest for-statements In-Reply-To: References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> Message-ID: On Fri, 23 May 2008, Leif Walsh wrote: > I've worked this out (I'm sure I did this in intro CS once before...), > although there is probably some optimization trick I'm missing, and > I'm probably doing some silly double-allocation thing or something. > > >>> def combine(lists): > ... if len(lists) is 0: > ... return [[]] > ... first = combine(lists[0:-1]) > ... total = [] > ... for item in lists[-1]: > ... total += [[item] + subset for subset in first] > ... return total Oops, that should probably be: ... for subset in first: ... total += [subset + [item] for item in lists[-1]] to maintain order. My excuse is that it's past my bedtime. -- Cheers, Leif From bmintern at gmail.com Fri May 23 10:38:55 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Fri, 23 May 2008 04:38:55 -0400 Subject: [Python-ideas] Function to unnest for-statements In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> Message-ID: <4c0fccce0805230138y6ca53705lcb2dc4398aac750d@mail.gmail.com> On Fri, May 23, 2008 at 3:15 AM, Carl Johnson wrote: > This is based off of > http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations . > It's also discussed at http://reddit.com/info/5ywrs/comments/ and someone > with a lot of spare time can read Knuth's fascile on the subject at > http://www-cs-faculty.stanford.edu/~knuth/fasc2a.ps.gz . > > OK, suppose you have a situation where you need to loop through all > combinations of 3 lists. The simple solution is to write three nested > for-loops: > > for x in xs: > for y in ys: > for z in zs: > #Do something > > Of course, this is obvious O(n^3), which is bad, but sometimes you don't > have a choice. However, what if (to use an example I ran into), you're > making up a truth table. If you know you have three variables, you might > write: > > for a in [True, False]: > for b in [True, False]: > for c in [True, False]: > print a, b, c > > > But if you don't know how many variables you'll have, you're stuck writing a > complicated function instead of using a nice, simple for-loop. > > So, going back to the first example, wouldn't it be nicer to write: > > for x, y, z in combine(xs, ys, zs): > #Do something > > > So, I think a "combine" function (or some other, better name) would be a > good thing to have, at least in the itertools, if not as a built-in > function. How to implement it? Well, probably it should be done in C for > efficiency, but of the versions done > http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations the > one that I like best is > > However, I'm also convinced that there is a more efficient way to do this. > > So, what do you think? Is this a common enough need that it should be built > into itertools? The main namespace? Or should we leave it out, since adding > it would encourage people writing O(n^x) algorithms? If nothing else, list > members can enjoy rewriting this function for fun. > > -- Carl Johnson I really like this function, and I could have used it about a year ago. I actually like the name "combinations" better than "combine". IMO, "combinations" implies the intended meaning more directly, while "combine" could be combining the lists in some other way. Whatever the name, I think it would fit quite nicely into itertools. I don't think efficiency should be considered as a reason to leave it out, because as you said, there are simply cases (any time you need to enumerate all possible values) where you can't get around it. Besides, it's not too difficult to create quadratic or cubic lists using list comprehensions... the syntax is actually designed to support it. As for a stab at a good implementation: def combinations (*args): def combiner (seqs): try: seq0 = seqs.next() except StopIteration: yield () else: for prefix in combiner(iter(seqs)): for x in seq0: yield prefix + (x,) return combiner(reversed(args)) I tried to avoid unpacking and repacking lists, I returned the combinations as tuples, and I tried to use generators wherever possible. I didn't actually do any performance tests, though. Brandon From bmintern at gmail.com Fri May 23 10:51:05 2008 From: bmintern at gmail.com (Brandon Mintern) Date: Fri, 23 May 2008 04:51:05 -0400 Subject: [Python-ideas] Function to unnest for-statements In-Reply-To: <4c0fccce0805230138y6ca53705lcb2dc4398aac750d@mail.gmail.com> References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> <4c0fccce0805230138y6ca53705lcb2dc4398aac750d@mail.gmail.com> Message-ID: <4c0fccce0805230151k13807b93mcb5cab084502f45f@mail.gmail.com> On Fri, May 23, 2008 at 4:38 AM, Brandon Mintern wrote: > def combinations (*args): > def combiner (seqs): > try: > seq0 = seqs.next() > except StopIteration: > yield () > else: > for prefix in combiner(iter(seqs)): > for x in seq0: > yield prefix + (x,) > return combiner(reversed(args)) I was incorrectly thinking that combiner would be called more than once on seqs, and thus I used iter(seqs) to avoid making a call on an exhausted iterator. It turns out that's unnecessary since there is only one recursive call per call to combiner, so the result is the somewhat improved: def combinations (*args): def combiner (seqs): try: seq0 = seqs.next() except StopIteration: yield () else: for prefix in combiner(seqs): for x in seq0: yield prefix + (x,) return combiner(reversed(args)) I think this is a solution I can be proud of, in terms of correctness (iterating in the proper order), readability, and efficiency. Brandon From g.brandl at gmx.net Fri May 23 11:47:32 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 23 May 2008 11:47:32 +0200 Subject: [Python-ideas] Function to unnest for-statements In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> Message-ID: Carl Johnson schrieb: > for a in [True, False]: > for b in [True, False]: > for c in [True, False]: > print a, b, c > > Which yields > > True True True > True True False > True False True > True False False > False True True > False True False > False False True > False False False > > But if you don't know how many variables you'll have, you're stuck > writing a complicated function instead of using a nice, simple for-loop. > > So, going back to the first example, wouldn't it be nicer to write: > > for x, y, z in combine(xs, ys, zs): > #Do something > So, what do you think? Is this a common enough need that it should be > built into itertools? Presumably, since it has been added to itertools in 2.6 under the name product(). (Maybe Raymond borrowed the time machine?) :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri May 23 11:49:45 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 23 May 2008 11:49:45 +0200 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <4835FBA4.8020206@canterbury.ac.nz> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4835FBA4.8020206@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Brandon Mintern wrote: >> This is a proposal to add a new built-in named struct: >> >> struct(**kwargs) >> Return a struct object which has the attributes given in kwargs. > > I think I'd prefer 'record', to avoid any potential > confusion with the struct module, which does something > quite different. > > Also, my Pascal background makes the term 'record' > seem more high-level and therefore Pythonic to me. In the past, people have also suggested 'namespace' for the same concept. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From steven.bethard at gmail.com Sat May 24 22:40:11 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 24 May 2008 14:40:11 -0600 Subject: [Python-ideas] Proposal to add new built-in struct (was: Add kwargs to built-in function object) In-Reply-To: <91ad5bf80805222351p419d3c9cl30225a0860528101@mail.gmail.com> References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com> <48362508.3080706@canterbury.ac.nz> <91ad5bf80805222351p419d3c9cl30225a0860528101@mail.gmail.com> Message-ID: On Fri, May 23, 2008 at 12:51 AM, George Sakkis wrote: > On Thu, May 22, 2008 at 9:59 PM, Greg Ewing wrote: > >> On Thu, May 22, 2008 at 7:15 PM, Leif Walsh wrote: >>> >>> I think it makes the most sense, if this construct is adopted, to use >>> __slots__ to control mutability. >> >> That wouldn't work, because the slots that a class has >> are fixed when the class is defined. Changing __slots__ >> subsequent to that doesn't affect anything. > > Unless you generate the class on the fly with a class factory, like > namedtuple. Actually I prefer a metaclass factory, to avoid the > repetition of the class name: > > >>> class Foo(object): > ... __metaclass__ = RecordType('x y z is_on', > field_defaults={'is_on':False}, default=0.0) > >>> a = Foo() > >>> a > Foo(x=0.0, y=0.0, z=0.0, is_on=False) > >>> a.__slots__ > ('x', 'y', 'z', 'is_on') > >>> a.__dict__ > Traceback (most recent call last): > AttributeError: 'Foo' object has no attribute '__dict__' The recipe here does roughly that (though without the defaulting extras): http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/502237 Note however that use of __slots__ is still controversial. (Because it's really only appropriate if you plan to make many of these objects and you want to keep memory consumption down.) Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From greg.ewing at canterbury.ac.nz Sun May 25 03:19:05 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 25 May 2008 13:19:05 +1200 Subject: [Python-ideas] Proposal to add new built-in struct In-Reply-To: References: <4c0fccce0805221442w5a4c40a4ma0097b42f86558af@mail.gmail.com> <4c0fccce0805221622oc18cdd8o98112991c7aab8ee@mail.gmail.com> <48362508.3080706@canterbury.ac.nz> <91ad5bf80805222351p419d3c9cl30225a0860528101@mail.gmail.com> Message-ID: <4838BE89.2020006@canterbury.ac.nz> Steven Bethard wrote: > Note however that use of __slots__ is still controversial. (Because > it's really only appropriate if you plan to make many of these objects > and you want to keep memory consumption down.) Also, if you were going to go down the route of creating classes on the fly, you would want a factory function to create a class suitable for the purpose at hand, and then make instances of that class. Otherwise, you would be creating a new class for every *instance* of your struct, which would be extremely wasteful (type objects are *big*!) And if you're doing that, you might as well just have some built-in or library class that you can subclass. -- Greg From arnodel at googlemail.com Sun May 25 10:01:39 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 25 May 2008 09:01:39 +0100 Subject: [Python-ideas] Function to unnest for-statements In-Reply-To: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> Message-ID: <970E1C53-2F9A-4DC7-AF4A-2C12805F00CA@gmail.com> On 23 May 2008, at 08:15, Carl Johnson wrote: > This is based off of http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations > . It's also discussed at http://reddit.com/info/5ywrs/comments/ and > someone with a lot of spare time can read Knuth's fascile on the > subject at http://www-cs-faculty.stanford.edu/~knuth/fasc2a.ps.gz . > > OK, suppose you have a situation where you need to loop through all > combinations of 3 lists. The simple solution is to write three > nested for-loops: > > for x in xs: > for y in ys: > for z in zs: > #Do something > > Of course, this is obvious O(n^3), which is bad, but sometimes you > don't have a choice. However, what if (to use an example I ran > into), you're making up a truth table. If you know you have three > variables, you might write: > > for a in [True, False]: > for b in [True, False]: > for c in [True, False]: > print a, b, c > > Which yields > > True True True > True True False > True False True > True False False > False True True > False True False > False False True > False False False > > But if you don't know how many variables you'll have, you're stuck > writing a complicated function instead of using a nice, simple for- > loop. > > So, going back to the first example, wouldn't it be nicer to write: > > for x, y, z in combine(xs, ys, zs): > #Do something > > If nothing else, this has the advantage that if you don't have to > nest the for-loops, things don't end up being so deeply indented on > the screen. > > Similarly, a general truth table maker can be written: > > ts_and_fs = [[True, False]] * num_vars > for variables in combine(*ts_and_fs): > print variables > > So, I think a "combine" function (or some other, better name) would > be a good thing to have, at least in the itertools, if not as a > built-in function. How to implement it? Well, probably it should be > done in C for efficiency, but of the versions done http://boredzo.org/blog/archives/2007-10-22/generating-all-combinations > the one that I like best is > > def combine(*sequences): > #Test to guard against infinite recursion > if sequences: > def _inner(*seqs): > if len(seqs) == 1: > for i in seqs[0]: yield (i, ) > else: > for rest in combine(*seqs[:-1]): > for i in seqs[-1]: > yield rest + (i, ) > return _inner(*sequences) > else: > #Empty generator that yields StopIteration > def _inner(): > return > yield > return _inner() > > However, I'm also convinced that there is a more efficient way to do > this' I don't understand why you define those _inner() generator functions. Anyway, here's a version that doesn't use recursion: def product(*sequences): if not sequences: return i, imax = 0, len(sequences)-1 val = [None]*(imax+1) iters = [iter(seq) for seq in sequences] while i >= 0: for val[i] in iters[i]: if i == imax: yield tuple(val) else: i += 1 break else: iters[i] = iter(sequences[i]) i -= 1 > > So, what do you think? Is this a common enough need that it should > be built into itertools? The main namespace? Or should we leave it > out, since adding it would encourage people writing O(n^x) > algorithms? If nothing else, list members can enjoy rewriting this > function for fun. It is already in itertools: Python 3.0a4+ (py3k:62388, Apr 19 2008, 15:34:00) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from itertools import product >>> for v in product(*[[True, False]]*3): ... print(v) ... (True, True, True) (True, True, False) (True, False, True) (True, False, False) (False, True, True) (False, True, False) (False, False, True) (False, False, False) >>> -- Arnaud From george.sakkis at gmail.com Sun May 25 14:11:39 2008 From: george.sakkis at gmail.com (George Sakkis) Date: Sun, 25 May 2008 08:11:39 -0400 Subject: [Python-ideas] Function to unnest for-statements In-Reply-To: <970E1C53-2F9A-4DC7-AF4A-2C12805F00CA@gmail.com> References: <020C483D-7FC8-4064-9CA5-67FC9DD73794@carlsensei.com> <970E1C53-2F9A-4DC7-AF4A-2C12805F00CA@gmail.com> Message-ID: <91ad5bf80805250511n306c0d5ap4a4831bd891eda2f@mail.gmail.com> On Sun, May 25, 2008 at 4:01 AM, Arnaud Delobelle wrote: > > On 23 May 2008, at 08:15, Carl Johnson wrote: > >> So, what do you think? Is this a common enough need that it should be >> built into itertools? The main namespace? Or should we leave it out, since >> adding it would encourage people writing O(n^x) algorithms? If nothing else, >> list members can enjoy rewriting this function for fun. > > It is already in itertools: > > Python 3.0a4+ (py3k:62388, Apr 19 2008, 15:34:00) > [GCC 4.0.1 (Apple Inc. build 5465)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> from itertools import product >>>> for v in product(*[[True, False]]*3): > ... print(v) > ... > (True, True, True) > (True, True, False) > (True, False, True) > (True, False, False) > (False, True, True) > (False, True, False) > (False, False, True) > (False, False, False) Or a bit more readable: >>> for v in product([True, False], repeat=3): ... print(v) George