I proposed an (admittedly more controversial) version of this a few months back at the py3k list and the reaction was unexpectedly (IMO) negative or indifferent, so I'm wondering if things have changed a bit since. The proposal is to make the the builtin iter() return an object with an API that consists of (most) functions currently at itertools. In addition to saving one "from itertools import chain,islice,..." line in every other module I write these days, an extra bonus of the OO interface is that islice can be replaced with slice syntax and chain with '+' (and/or perhaps the "pipe" character '|'). As a (deliberately involved) example, consider this: # A composite iterator over two files specified as follows: # - each yielded line is right stripped. # - the first 3 lines of the first file are yielded. # - the first line of the second file is skipped and its next 4 lines are yielded # - empty lines (after the right stripping) are filtered out. # - the remaining lines are enumerated. f1,f2 = [iter(open(f)).map(str.rstrip) for f in 'foo.txt','bar.txt'] for i,line in (f1[:3] + f2[1:5]).filter(None).enumerate(): print i,line The equivalent itertools version is left as an exercise to the reader. This is actually backwards compatible and could even go in 2.x if accepted, but I'm focusing on py3K here. Comments ? George PS: FYI, a proof of concept implementation is posted as a recipe at: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/498272
George Sakkis wrote:
+0 on your proposal I just don't see itertools being used often enough to justify your change, but I can see the utility for those instances where it is used heavily. +1 on adding your Iter class (or something similar) to itertools Less controversial and just as succinct/convenient as your proposal (Iter() vs iter()), save another line for the requisite import. - Chris Rebert
On 4/13/07, Chris Rebert <cvrebert@gmail.com> wrote:
In some sense it's a chicken-and-egg problem. My guess is that one reason itertools are not used as much as they could/should is that they are "hidden away" in a module, which makes one think it twice before importing it (let alone newbies that don't even know its existence). As a single data point, I'm a big fan itertools and still I'm often lazy to import it to use, say izip() only once; I just go with zip() instead. George
"George Sakkis" <george.sakkis@gmail.com> wrote in message news:91ad5bf80704130849y5117321ev475acf240b4f086f@mail.gmail.com... | In some sense it's a chicken-and-egg problem. My guess is that one | reason itertools are not used as much as they could/should is that | they are "hidden away" in a module, which makes one think it twice | before importing it (let alone newbies that don't even know its | existence). As a single data point, I'm a big fan itertools and still | I'm often lazy to import it to use, say izip() only once; I just go | with zip() instead. I personally think there are too many builtins. So I would like some pushed to modules, which means more import statements. Oh, dear. If you have trouble writing 'from itertools import izip' or 'import itertools as it', then I guess it is hard to promote a module. Nonetheless, I think perhaps you should write your own based on iter and itertools. And put it up on PyPI if it works at least for you. Terry Jan Reedy
"George Sakkis" <gsakkis@rutgers.edu> wrote:
Map was already going to be removed because it can be replaced by... [f(x) for x in y] Filter was already going to be removed because it can be replaced by... [x for x in y if f(x)] I can't remember if anything was going to happen to zip or any of the other functional programming functions. - Josiah
I think it's a great idea. In a language where everything is meant to be object-oriented and easy-to-use, it's funny that iterators (which are becoming increasingly popular) have such a bumpy interface. We wouldn't want to do "from numtools import nadd,nmul ; nadd(1,2)". The claim that iterators aren't "being used often enough to justify your change" is very disturbing considering that in future pythons most default functions will return iterators if possible (such as key()/items()/values() of dict). The WILL be used more than files, and it would be foolish to force the user to do "from filetools import seek, tell, readlines" The specifics of your proposal may need a little improvement here and there, but the idea itself, IMHO, is very good and pythonic. +1 On 4/13/07, George Sakkis <george.sakkis@gmail.com> wrote:
Erez Sh. wrote:
The claim that iterators aren't "being used often enough to justify your change" is very disturbing
The claim wasn't that iterators are used infrequently, but the functions in the itertools module. The vast majority of the time, the only thing people do with iterators is iterate over them.
This is wrong -- they won't return iterators, they'll return *views* that can be indexed and otherwise used as sequences or mappings. So this has nothing to do with the proposal at hand. -- Greg
On 4/13/07, Jason Orendorff <jason.orendorff@gmail.com> wrote:
Yeah, that's pretty awful. ;-) Maybe a more reasonable example:: # skip the first line for line in iter(fileobj)[1:]: .... where currently you'd write:: # skip the first line fileobj.next() for line in fileobj: ... I'm floating around -0.5 on this one. The itertools functions I use most are chain(), izip(), and count(). None of these are particularly natural as a method of a single iterator object. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
George Sakkis wrote:
The proposal is to make the the builtin iter() return an object with an API that consists of (most) functions currently at itertools.
The problem with this kind of thing is that it becomes an arbitrary choice what is included as a method. Anything not included in that choice is left out in the cold and has to be applied as a function anyway. If there were a certain set of iterator algebra functions that were *very* frequently used, there could be an argument for making methods of them. But I think you're overestimating how much the itertools functions are used. Some people may make heavy use of them, but they're not used much in general. If you happen to be a heavy user, there's nothing stopping you from creating your own version of iter() that returns an object with all the methods you want. Let's keep the standard iterator objects clean and simple. -- Greg
George Sakkis wrote:
+0 on your proposal I just don't see itertools being used often enough to justify your change, but I can see the utility for those instances where it is used heavily. +1 on adding your Iter class (or something similar) to itertools Less controversial and just as succinct/convenient as your proposal (Iter() vs iter()), save another line for the requisite import. - Chris Rebert
On 4/13/07, Chris Rebert <cvrebert@gmail.com> wrote:
In some sense it's a chicken-and-egg problem. My guess is that one reason itertools are not used as much as they could/should is that they are "hidden away" in a module, which makes one think it twice before importing it (let alone newbies that don't even know its existence). As a single data point, I'm a big fan itertools and still I'm often lazy to import it to use, say izip() only once; I just go with zip() instead. George
"George Sakkis" <george.sakkis@gmail.com> wrote in message news:91ad5bf80704130849y5117321ev475acf240b4f086f@mail.gmail.com... | In some sense it's a chicken-and-egg problem. My guess is that one | reason itertools are not used as much as they could/should is that | they are "hidden away" in a module, which makes one think it twice | before importing it (let alone newbies that don't even know its | existence). As a single data point, I'm a big fan itertools and still | I'm often lazy to import it to use, say izip() only once; I just go | with zip() instead. I personally think there are too many builtins. So I would like some pushed to modules, which means more import statements. Oh, dear. If you have trouble writing 'from itertools import izip' or 'import itertools as it', then I guess it is hard to promote a module. Nonetheless, I think perhaps you should write your own based on iter and itertools. And put it up on PyPI if it works at least for you. Terry Jan Reedy
"George Sakkis" <gsakkis@rutgers.edu> wrote:
Map was already going to be removed because it can be replaced by... [f(x) for x in y] Filter was already going to be removed because it can be replaced by... [x for x in y if f(x)] I can't remember if anything was going to happen to zip or any of the other functional programming functions. - Josiah
I think it's a great idea. In a language where everything is meant to be object-oriented and easy-to-use, it's funny that iterators (which are becoming increasingly popular) have such a bumpy interface. We wouldn't want to do "from numtools import nadd,nmul ; nadd(1,2)". The claim that iterators aren't "being used often enough to justify your change" is very disturbing considering that in future pythons most default functions will return iterators if possible (such as key()/items()/values() of dict). The WILL be used more than files, and it would be foolish to force the user to do "from filetools import seek, tell, readlines" The specifics of your proposal may need a little improvement here and there, but the idea itself, IMHO, is very good and pythonic. +1 On 4/13/07, George Sakkis <george.sakkis@gmail.com> wrote:
Erez Sh. wrote:
The claim that iterators aren't "being used often enough to justify your change" is very disturbing
The claim wasn't that iterators are used infrequently, but the functions in the itertools module. The vast majority of the time, the only thing people do with iterators is iterate over them.
This is wrong -- they won't return iterators, they'll return *views* that can be indexed and otherwise used as sequences or mappings. So this has nothing to do with the proposal at hand. -- Greg
On 4/13/07, Jason Orendorff <jason.orendorff@gmail.com> wrote:
Yeah, that's pretty awful. ;-) Maybe a more reasonable example:: # skip the first line for line in iter(fileobj)[1:]: .... where currently you'd write:: # skip the first line fileobj.next() for line in fileobj: ... I'm floating around -0.5 on this one. The itertools functions I use most are chain(), izip(), and count(). None of these are particularly natural as a method of a single iterator object. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
George Sakkis wrote:
The proposal is to make the the builtin iter() return an object with an API that consists of (most) functions currently at itertools.
The problem with this kind of thing is that it becomes an arbitrary choice what is included as a method. Anything not included in that choice is left out in the cold and has to be applied as a function anyway. If there were a certain set of iterator algebra functions that were *very* frequently used, there could be an argument for making methods of them. But I think you're overestimating how much the itertools functions are used. Some people may make heavy use of them, but they're not used much in general. If you happen to be a heavy user, there's nothing stopping you from creating your own version of iter() that returns an object with all the methods you want. Let's keep the standard iterator objects clean and simple. -- Greg
participants (10)
-
Brett Cannon -
Chris Rebert -
Erez Sh. -
George Sakkis -
George Sakkis -
Greg Ewing -
Jason Orendorff -
Josiah Carlson -
Steven Bethard -
Terry Reedy