Mailman 3 copyable iterators, named loops, continue a expression per indentation and a few functional things - Python-ideas

copyable iterators, named loops, continue a expression per indentation and a few functional things

Mathias Panzenböck

24 Dec 2006 24 Dec '06

1:20 a.m.

Copyable iterators ------------------ There are a few ways how you could implement this. However, it only makes sense for iterators, not for generators! Way 1: Copyable iterators have a copy-method:

...

...
...
L = [1,2,3,4] it1 = iter(L) it1.next() 1 it2 = it1.copy() it2.next() 2 it1.next() 2

Way 2: Copyable iterators will be copied when you call it's __iter__-method:

...

...
...
L = [1,2,3,4] it1 = iter(L) it1.next() 1 it2 = iter(it1) it2.next() 2 it1.next() 2

In way 1 you could check for copy ability per hasattr(it,"copy"). Named loops ----------- With named loops you could break or continue other than only the inner most loop. I'm not sure about a syntax, though. for x in X as loop_a: for y in Y as loop_b: if cond1(): break loop_a elif cond2(): continue loop_a elif cond3(): break # breaks loop_b elif cond4(): continue # continues loop_b Auto-curryfication for operators -------------------------------- I like haskells feature for curryfication of operators. I think this would be nice to have in python. You could write instead of 'operator.add' the much shorter '(+)'. Or even '(2+)' works.

...

...
...
map((2+),[1,2,3,4]) [3,4,5,6] filter((>=2),[1,2,3,4]) [2,3,4]

__add__-method for functions ---------------------------- In haskell you can write 'f . g . h', which would be the same like '(\a -> f(g(h(a))))' (in python: 'lambda a: f(g(a))'). In python it would also be nice to have such a functionality. It could be implemented with a __add__-method. Here is this functionality simulated with functors:

...

...
...
class FunctionBase(object): def __add__(self,other): return lambda *args,**kwargs: self(other(*args,**kwargs))

...

...
...
class F(FunctionBase): def __call__(self,s): return s+" foo"

...

...
...
class G(FunctionBase): def __call__(self,s): return s+" bar"

...

...
...
f = F() g = G() (f + g)("x") 'x bar foo' (g + f)("x") 'x foo bar'

What do you think? panzi

Show replies by date

Florian Weimer

24 Dec 24 Dec

1:59 a.m.

New subject: copyable iterators, named loops, continue a expression per indentation and a few functional things

* Mathias Panzenböck:

...

...
...
...
f = F() g = G() (f + g)("x") 'x bar foo' (g + f)("x") 'x foo bar'

I would expect that (f + g)(x) == f(x) + g(x). Using "+" for a non-commutative operation doesn't seem right.

Guido van Rossum

9:14 a.m.

New subject: copyable iterators, named loops, continue a expression per indentation and a few functional things

On 12/23/06, Florian Weimer wrote:

...

Using "+" for a non-commutative operation doesn't seem right.

Perhaps you've never used string concatenation in Python? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Calvin Spealman

11 a.m.

New subject: copyable iterators, named loops, continue a expression per indentation and a few functional things

On, Copyable Iterators: I would like the option of copyable iterators, and I think a copy method is better than just iter(), because it can be common to be slightly unsure if you have an iteratable or an iterator (hasattr(x, '__iter__') vs hasattr(x, 'next')). Also, it might be more work but I would love to see generators that can do this. On, Named Loops: Although I appriciate what problems this idea aims to solve, I think it would lead to things far too close to spagetti code to be worth the trouble. On, Auto-curryfication for operators: This looks a little strange in Python, but its certainly clear enough in intention that it could be pretty valuable. Likewise, what if we expand this a bit to supporting __r*__ methods on generators expressions so we can do the following? indexes_from_one = 1 + (i for (i, j) in enumerate(L)) Which reads well as "One plug i for each ...", etc. On, __add__-method for functions: Although this is not a terrible idea, I can not be convinced that its the thing most people would expect when seeing it in code. In other words, when anyone new to this sees the code (f + g)(x, y) are they going to expect f(g(x,y)) or f(x)+g(y), as some sort of an extra-argument or parameter spillover catch? No, most wouldn't expect something terribly wrong, other than getting f and g backwards in the translated f(g(x,y)), but the point is that it might not be intuitive enough not to be used incorrectly a lot. On 12/23/06, Mathias Panzenböck wrote:

...

Copyable iterators ------------------

There are a few ways how you could implement this. However, it only makes sense for iterators, not for generators!

Way 1: Copyable iterators have a copy-method:

...
...
...
L = [1,2,3,4] it1 = iter(L) it1.next() 1 it2 = it1.copy() it2.next() 2 it1.next() 2

Way 2: Copyable iterators will be copied when you call it's __iter__-method:

...
...
...
L = [1,2,3,4] it1 = iter(L) it1.next() 1 it2 = iter(it1) it2.next() 2 it1.next() 2

In way 1 you could check for copy ability per hasattr(it,"copy").

Named loops -----------

With named loops you could break or continue other than only the inner most loop. I'm not sure about a syntax, though.

for x in X as loop_a: for y in Y as loop_b: if cond1(): break loop_a

elif cond2(): continue loop_a

elif cond3(): break # breaks loop_b

elif cond4(): continue # continues loop_b

Auto-curryfication for operators --------------------------------

I like haskells feature for curryfication of operators. I think this would be nice to have in python.

You could write instead of 'operator.add' the much shorter '(+)'. Or even '(2+)' works.

...
...
...
map((2+),[1,2,3,4]) [3,4,5,6] filter((>=2),[1,2,3,4]) [2,3,4]

__add__-method for functions ----------------------------

In haskell you can write 'f . g . h', which would be the same like '(\a -> f(g(h(a))))' (in python: 'lambda a: f(g(a))'). In python it would also be nice to have such a functionality. It could be implemented with a __add__-method.

Here is this functionality simulated with functors:

...
...
...
class FunctionBase(object): def __add__(self,other): return lambda *args,**kwargs: self(other(*args,**kwargs))

...
...
...
class F(FunctionBase): def __call__(self,s): return s+" foo"

...
...
...
class G(FunctionBase): def __call__(self,s): return s+" bar"

...
...
...
f = F() g = G() (f + g)("x") 'x bar foo' (g + f)("x") 'x foo bar'

What do you think?

panzi _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/

Josiah Carlson

1:14 p.m.

New subject: copyable iterators, named loops, continue a expression per indentation and a few functional things

Mathias Panzenböck wrote:

...

Copyable iterators ------------------

There are a few ways how you could implement this. However, it only makes sense for iterators, not for generators!

-1 on the entire proposal. All iterators (and generators) are already copyable. It's called list(). - Josiah

Talin

3:32 p.m.

Josiah Carlson wrote:

...

Mathias Panzenböck wrote:

...
Copyable iterators ------------------

There are a few ways how you could implement this. However, it only makes sense for iterators, not for generators!

-1 on the entire proposal. All iterators (and generators) are already copyable. It's called list().

Actually, what he wants to be able to do is to 'tee' an iterator. However, we already have a 'tee' function in itertools that does just that. (And it works on generators too!) -- Talin

Calvin Spealman

25 Dec 25 Dec

12:20 a.m.

New subject: copyable iterators, named loops, continue a expression per indentation and a few functional things

On 12/24/06, Talin wrote:

...

Josiah Carlson wrote:

...
Mathias Panzenböck wrote:

...
Copyable iterators ------------------

There are a few ways how you could implement this. However, it only makes sense for iterators, not for generators!

-1 on the entire proposal. All iterators (and generators) are already copyable. It's called list().

Actually, what he wants to be able to do is to 'tee' an iterator. However, we already have a 'tee' function in itertools that does just that. (And it works on generators too!)

-- Talin

If I am understanding the proposal correctly, which I think I am, itertools.tee is not the same as what this is proposing. itertools.tee simply iterates the original and caches the results to be given when iterating over the two tees created for it. This is not the same as copying an iterator. Copying an iterator would create a new iterator of the same state as the original. For example, with generators the copy would be a new generator object with the same state as the original and iterating over the copy would execute the same code as the first. def g(): i = 0 while True: print i yield i i = i + 1 If you "copy" this with itertools.tee, you only see i printed once for each iteration of the generator, not each iteration of the tees. With a real copying of iterators, if I copied this, iterating over the copies would always produce an output of that copies value of i. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/

Talin

2:07 a.m.

Calvin Spealman wrote:

...

On 12/24/06, Talin wrote:

...
...
Mathias Panzenböck wrote:

...
Copyable iterators ------------------

There are a few ways how you could implement this. However, it only makes sense for iterators, not for generators! -1 on the entire proposal. All iterators (and generators) are already copyable. It's called list(). Actually, what he wants to be able to do is to 'tee' an iterator. However, we already have a 'tee' function in itertools that does just

Josiah Carlson wrote: that. (And it works on generators too!)

-- Talin

If I am understanding the proposal correctly, which I think I am, itertools.tee is not the same as what this is proposing. itertools.tee simply iterates the original and caches the results to be given when iterating over the two tees created for it. This is not the same as copying an iterator. Copying an iterator would create a new iterator of the same state as the original. For example, with generators the copy would be a new generator object with the same state as the original and iterating over the copy would execute the same code as the first.

def g(): i = 0 while True: print i yield i i = i + 1

If you "copy" this with itertools.tee, you only see i printed once for each iteration of the generator, not each iteration of the tees. With a real copying of iterators, if I copied this, iterating over the copies would always produce an output of that copies value of i.

The OP already stated that the proposal is not meant to apply to generators, and rightly so, because there's no way to copy all of the mutable state that a generator might access. However, I think that the idea of copying 'iterators but not generators' is too restrictive, since externally you might not be able to tell the difference. That's why I suggested tee, since it works on both. No, you don't get the same side effects, but that's not what the OP was asking for. -- Talin

Brett Cannon

4:18 a.m.

New subject: copyable iterators, named loops, continue a expression per indentation and a few functional things

On 12/24/06, Talin wrote:

...

Calvin Spealman wrote:

...
On 12/24/06, Talin wrote:

...
...
Mathias Panzenböck wrote:

...
Copyable iterators ------------------

There are a few ways how you could implement this. However, it only makes sense for iterators, not for generators! -1 on the entire proposal. All iterators (and generators) are already copyable. It's called list(). Actually, what he wants to be able to do is to 'tee' an iterator. However, we already have a 'tee' function in itertools that does just

Josiah Carlson wrote: that. (And it works on generators too!)

-- Talin

If I am understanding the proposal correctly, which I think I am, itertools.tee is not the same as what this is proposing. itertools.tee simply iterates the original and caches the results to be given when iterating over the two tees created for it. This is not the same as copying an iterator. Copying an iterator would create a new iterator of the same state as the original. For example, with generators the copy would be a new generator object with the same state as the original and iterating over the copy would execute the same code as the first.

def g(): i = 0 while True: print i yield i i = i + 1

If you "copy" this with itertools.tee, you only see i printed once for each iteration of the generator, not each iteration of the tees. With a real copying of iterators, if I copied this, iterating over the copies would always produce an output of that copies value of i.

The OP already stated that the proposal is not meant to apply to generators, and rightly so, because there's no way to copy all of the mutable state that a generator might access.

However, I think that the idea of copying 'iterators but not generators' is too restrictive, since externally you might not be able to tell the difference. That's why I suggested tee, since it works on both. No, you don't get the same side effects, but that's not what the OP was asking for.

Ditto from me. Generators are just a type of iterator technically, so the OP really wanted to say "non-generator iterators". But that is a nasty restriction. -Brett

Eduardo "EdCrypt" O. Padoan

7:16 a.m.

New subject: copyable iterators, named loops, continue a expression per indentation and a few functional things

...

Named loops

This is an useful idea. But lets explore it a little. First, it is not exactly a name, a name is a reference to an object, it is more like a lable.

...

-----------

With named loops you could break or continue other than only the inner most loop. I'm not sure about a syntax, though.

That is the useful part, and I will show why I think your syntax is not the best for it.

...

for x in X as loop_a:

The 'as' keyword (not a keyword untill 2.6) is used to give a name to an object. Here, the lable 'loop_a', I think, should not be a name to any object. Second, it gives the impression that with '... X as loop_a' you are giving X the name 'loop_a'. It reads that way and it is how 'import ... as ...', 'with ... as ...' works and, in Py3k, how 'except ... as ...' will work. Your syntax is an exception to the 'OBJ as NAME' pattern, and could be misleading. The problem is, I couldnt think yet of any better syntax. Besides, the loop label is very usefull, and could be used in while too if someone find a better syntax. -- EduardoOPadoan (eopadoan->altavix::com) Bookmarks: http://del.icio.us/edcrypt Blog: http://edcrypt.blogspot.com Jabber: edcrypt at jabber dot org ICQ: 161480283 GTalk: eduardo dot padoan at gmail dot com MSN: eopadoan at altavix dot com

Mathias Panzenböck

8:12 a.m.

Eduardo "EdCrypt" O. Padoan schrieb:

...

...
Named loops

This is an useful idea. But lets explore it a little. First, it is not exactly a name, a name is a reference to an object, it is more like a lable.

...
-----------

With named loops you could break or continue other than only the inner most loop. I'm not sure about a syntax, though.

That is the useful part, and I will show why I think your syntax is not the best for it.

...
for x in X as loop_a:

The 'as' keyword (not a keyword untill 2.6) is used to give a name to an object. Here, the lable 'loop_a', I think, should not be a name to any object.

Second, it gives the impression that with '... X as loop_a' you are giving X the name 'loop_a'. It reads that way and it is how 'import ... as ...', 'with ... as ...' works and, in Py3k, how 'except ... as ...' will work. Your syntax is an exception to the 'OBJ as NAME' pattern, and could be misleading. The problem is, I couldnt think yet of any better syntax.

Besides, the loop label is very usefull, and could be used in while too if someone find a better syntax.

Indeed, that's a problem. Also that a name should always be a object. Maybe: break # breaks inner most loop break . # the same break .. # breaks parent break ... # breaks grandparent but this is ugly, too. maybe: raise StopLoop(level) I don't know. Well this is a good way to break the outer most loop without any new syntax: try: while cond1(): while cond2(): if cond3(): raise MyLoopStopper() except MyLoopStopper: pass But how to make something like this for continue?

6332

Age (days ago)

6334

Last active (days ago)

List overview

Download

10 comments

8 participants

participants (8)

Brett Cannon
Calvin Spealman
Eduardo "EdCrypt" O. Padoan
Florian Weimer
Guido van Rossum
Josiah Carlson
Mathias Panzenböck
Talin

copyable iterators, named loops, continue a expression per indentation and a few functional things

Mathias Panzenböck

Florian Weimer

Guido van Rossum

Calvin Spealman

Josiah Carlson

Talin

Calvin Spealman

Talin

Brett Cannon

Eduardo "EdCrypt" O. Padoan

Mathias Panzenböck

tags

participants (8)