Mailman 3 Re: [Python-ideas] Adding "Typed" collections/iterators to Python - Python-ideas

Re: [Python-ideas] Adding "Typed" collections/iterators to Python

older
context managers used to allocate...

Nathan Rice

19 Dec 2011 19 Dec '11

7:52 a.m.

...
L2 = [X(e) for e in L1]>> L3 = [Y(e) for e in L2]>> vs>> L2 = X(L1) # assuming X has been updated to work in both vector/scalar>> L3 = Y(L2) # context...>>> L = ['a', 'bc', ['ada', 'a']]>> What is len(L)? 3 or [1, 2, 2] or [1, 2, [3, 1]]?>>>> L2 = [Z(Y(X(e))) for e in L1]>> vs>> L2 = Z(Y(X(L1)))>>>> L2 = [e.X().Y().Z() for e in L1]>> vs>> L2 = L1.X().Y().Z() # assuming vectorized versions of member methods>> #are folded into the collection via the mixin.>>> What is L.count('a')? 1 or [1, 0, 1] or [1, 0, [2, 1]]?

A fair concern; if the vectorized version of the child method were given the same name as the child method, I agree that this could result in ambiguity. There are multiple ways that member methods could be made available on the collection, including a proxy attribute, renaming, etc. ...

...

...
I find using operator& functools _far_ clearer in intent than using lambda,

_and it works right now_, which was the point I was trying to make here.

I find using list comprehensions and generator expressions even more clearer.

In general I think comprehensions are superior to map because they are more intuitive. Based on my experience reading code in the community I think that is well supported. Sadly, even though I have issues with the way they are used in many cases, lambdas are superior to a lot of other options because they are simple and cover many use cases. When used inline with simple expressions they provide a lot of mileage.

Show replies by thread

Serhiy Storchaka

19 Dec 19 Dec

11:12 a.m.

New subject: Adding "Typed" collections/iterators to Python

19.12.11 15:52, Nathan Rice написав(ла):

...

...
...
L2 = [X(e) for e in L1]>> L3 = [Y(e) for e in L2]>> vs>> L2 = X(L1) # assuming X has been updated to work in both vector/scalar>> L3 = Y(L2) # context...>>> L = ['a', 'bc', ['ada', 'a']]>> What is len(L)? 3 or [1, 2, 2] or [1, 2, [3, 1]]?>>>> L2 = [Z(Y(X(e))) for e in L1]>> vs>> L2 = Z(Y(X(L1)))>>>> L2 = [e.X().Y().Z() for e in L1]>> vs>> L2 = L1.X().Y().Z() # assuming vectorized versions of member methods>> #are folded into the collection via the mixin.>>> What is L.count('a')? 1 or [1, 0, 1] or [1, 0, [2, 1]]?

A fair concern; if the vectorized version of the child method were given the same name as the child method, I agree that this could result in ambiguity.

There are multiple ways that member methods could be made available on the collection, including a proxy attribute, renaming, etc.

len is not method, it is function (although it uses method __len__). There are many functions applicable to list and to its elements (in particular when members of list are lists).

Nathan Rice

11:47 a.m.

New subject: Adding "Typed" collections/iterators to Python

...

...
...
...
L2 = [X(e) for e in L1]>> L3 = [Y(e) for e in L2]>> vs>> L2 = X(L1) # assuming X has been updated to work in both vector/scalar>> L3 = Y(L2) # context...>>> L = ['a', 'bc', ['ada', 'a']]>> What is len(L)? 3 or [1, 2, 2] or [1, 2, [3, 1]]?>>>> L2 = [Z(Y(X(e))) for e in L1]>> vs>> L2 = Z(Y(X(L1)))>>>> L2 = [e.X().Y().Z() for e in L1]>> vs>> L2 = L1.X().Y().Z() # assuming vectorized versions of member methods>> #are folded into the collection via the mixin.>>> What is L.count('a')? 1 or [1, 0, 1] or [1, 0, [2, 1]]?

A fair concern; if the vectorized version of the child method were given the same name as the child method, I agree that this could result in ambiguity.

There are multiple ways that member methods could be made available on the collection, including a proxy attribute, renaming, etc.

len is not method, it is function (although it uses method __len__). There are many functions applicable to list and to its elements (in particular when members of list are lists).

I am aware of this. I was talking about the "collection inside a collection" example after that. Additionally, now I realize my initial reading of your example was too hasty. You violate the contract in the mixed list example with strings and lists, and that should rightfully not provide vectorized methods; providing type decorators for very abstract notions such as "iterable" is a bad idea imho. Of course, [[1, 2], [3, 4], [5, 6]] could serve as an alternate example of why you can't directly map child methods with the same names.

Ethan Furman

12:01 p.m.

New subject: Adding "Typed" collections/iterators to Python

Nathan Rice wrote:

...

[[1, 2], [3, 4], [5, 6]] could serve as an alternate example of why you can't directly map child methods with the same names.

If you have to change the names, doesn't that negate the ability of "a reliable method to make functions operate on both scalar and vector values"? ~Ethan~

Nathan Rice

1:26 p.m.

New subject: Adding "Typed" collections/iterators to Python

On Mon, Dec 19, 2011 at 1:01 PM, Ethan Furman wrote:

...

Nathan Rice wrote:

...
[[1, 2], [3, 4], [5, 6]] could serve as an alternate example of why you can't directly map child methods with the same names.

If you have to change the names, doesn't that negate the ability of "a reliable method to make functions operate on both scalar and vector values"?

To clarify... Functions could reliably support both vector and scalar context, by checking that a type contract exists. Because of the name clash between parent and elementwise member methods when dealing with collections of collections (and some other classes), if the type contract provided "broadcast" versions of child methods, they would have to be provided with under an alias, perhaps X -> elementwise_X. I do not think people should be able to be oblivious of when they are calling collection and member methods, that encourages sloppy programming. I like the broadcast feature because it makes it easier to write code in a clear left to right narrative style, and that seems very pythonic to me compared with an "inside-out" style.

Ethan Furman

2:09 p.m.

New subject: Adding "Typed" collections/iterators to Python

Nathan Rice wrote:

...

On Mon, Dec 19, 2011 at 1:01 PM, Ethan Furman wrote:

...
Nathan Rice wrote:

...
[[1, 2], [3, 4], [5, 6]] could serve as an alternate example of why you can't directly map child methods with the same names.

If you have to change the names, doesn't that negate the ability of "a reliable method to make functions operate on both scalar and vector values"?

To clarify...

Functions could reliably support both vector and scalar context, by checking that a type contract exists.

That sounds like type checking, which is not Pythonic. Duck-typing, on the other hand, *is*. def capitalize_me(some_obj): some_obj = some_obj.capitalize() --> capitalize_me("a simple string") "A simple string" --> capitalize_me(["a", "list", "of", "strings"]) ["A", "List", "Of", "Strings"]

...

Because of the name clash between parent and elementwise member methods when dealing with collections of collections (and some other classes), if the type contract provided "broadcast" versions of child methods, they would have to be provided with under an alias, perhaps X -> elementwise_X. I do not think people should be able to be oblivious of when they are calling collection and member methods, that encourages sloppy programming. I like the broadcast feature because it makes it easier to write code in a clear left to right narrative style, and that seems very pythonic to me compared with an "inside-out" style.

Duck-typing is neither oblivious, nor sloppy -- at least, not inherently. Is my sample code above not what you had in mind? ~Ethan~

Joshua Landau

2:37 p.m.

New subject: Adding "Typed" collections/iterators to Python

On 19 December 2011 19:26, Nathan Rice wrote:

...

On Mon, Dec 19, 2011 at 1:01 PM, Ethan Furman wrote:

...
Nathan Rice wrote:

...
[[1, 2], [3, 4], [5, 6]] could serve as an alternate example of why you can't directly map child methods with the same names.

If you have to change the names, doesn't that negate the ability of "a reliable method to make functions operate on both scalar and vector values"?

To clarify...

Functions could reliably support both vector and scalar context, by checking that a type contract exists.

Because of the name clash between parent and elementwise member methods when dealing with collections of collections (and some other classes), if the type contract provided "broadcast" versions of child methods, they would have to be provided with under an alias, perhaps X -> elementwise_X. I do not think people should be able to be oblivious of when they are calling collection and member methods, that encourages sloppy programming. I like the broadcast feature because it makes it easier to write code in a clear left to right narrative style, and that seems very pythonic to me compared with an "inside-out" style. _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

...or we could just extend Pep 225http://www.python.org/dev/peps/pep-0225/ (deferred) with "~." so we have "['a', 'b', 'c']~.upper()" [syntax debatable]. You don't get the type-checking but that seemed more of a problem to me, as I'm not a real duck. Your example: my_string.split("\n").capitalize().join_items("\n") to "\n".join(my_string.split("\n")~.capitalize()) And personally I like "string.join(lst)" over "lst.join_with(string)"... -- Joshua, pointlessly interjecting once again

Nathan Rice

4:28 p.m.

New subject: Adding "Typed" collections/iterators to Python

...

...
Functions could reliably support both vector and scalar context, by>> checking that a type contract exists.>>> That sounds like type checking, which is not Pythonic. Duck-typing, on the> other hand, *is*. While you'll hear no argument from me on duck typing, in this instance that collides head-on with having clear names for elementwise versions of member methods. I personally would feel better erring on the side of clearer names at the cost of some kind of *very cursory* examination of what you are being passed. Exactly what form that examination might take is an open question that deserves further thought.

...

def capitalize_me(some_obj):> some_obj = some_obj.capitalize()>> --> capitalize_me("a simple string")> "A simple string">> --> capitalize_me(["a", "list", "of", "strings"])> ["A", "List", "Of", "Strings"] Though I don't want to comment on the definition of capitalize_me, in terms of function behavior across the string and list, that is pretty much what I am talking about.

...

...
Because of the name clash between parent and elementwise member>> methods when dealing with collections of collections (and some other>> classes), if the type contract provided "broadcast" versions of child>> methods, they would have to be provided with under an alias, perhaps X>> -> elementwise_X. I do not think people should be able to be>> oblivious of when they are calling collection and member methods, that>> encourages sloppy programming. I like the broadcast feature because>> it makes it easier to write code in a clear left to right narrative>> style, and that seems very pythonic to me compared with an>> "inside-out" style.>>> Duck-typing is neither oblivious, nor sloppy -- at least, not inherently.> Is my sample code above not what you had in mind? I don't think duck typing is sloppy, with the caveat that the thing you expect to behave like a duck is some kind of animal, reasonably crafted animatronic robot, or operated puppet :). I feel like when you move from a single object to a collection that is basically like having a second function, which has been integrated in with the first function for the purposes of API simplicity. Then, it becomes an outer function that has the job of returning the correct inner function and executing it with the given input.

--- On Mon, Dec 19, 2011 at 3:37 PM, Joshua Landau wrote:

...

...or we could just extend Pep 225 (deferred) with "~." so we have "['a', 'b', 'c']~.upper()" [syntax debatable]. You don't get the type-checking but that seemed more of a problem to me, as I'm not a real duck.

That does run into some of the same problems w.r.t duck typing, and solutions to problems with overloaded operators are hard to google for. If the method calls were aliased, google searches would be easier and less experienced users wouldn't have to worry about wrestling with operator overloading. That being said, I think elementwise operators are a great idea in a numeric/scientific context. There are of course both pros and cons to having type declarations. I know that people do not like to be limited by a lack of foresight in their predecessors; I have run into unreasonable interface/type specifications and annoyances with private/protected variables in Java and I would never want to see that infiltrate Python. I think what I am trying to sell here is more akin to metadata than static types. People should be able to basically keep doing what they are already doing, but have a reasonable mechanism to provide additional object metadata in a standard way that is easily accessible by downstream consumers. That gives you a lot of the upsides of type declarations while staying Pythonic. Maybe a better option than having a "type contract" base for homogeneous collections is to give objects a metadata namespace with a loose schema, and declare it there? As long as you could provide the metadata at object creation time you would still get all the same benefits, and it would avoid potentially overworking the class/mixin concept.

...

Your example: my_string.split("\n").capitalize().join_items("\n") to "\n".join(my_string.split("\n")~.capitalize())

And personally I like "string.join(lst)" over "lst.join_with(string)"...

Of course, everyone has their own preferences; having a very small set of internally consistent ways to do something rather than just one way is good for this reason. Nathan

Joshua Landau

7:05 p.m.

New subject: Adding "Typed" collections/iterators to Python

On 19 December 2011 22:28, Nathan Rice wrote:

...

On Mon, Dec 19, 2011 at 3:37 PM, Joshua Landau wrote:

...
...or we could just extend Pep 225 (deferred) with "~." so we have "['a', 'b', 'c']~.upper()" [syntax debatable]. You don't get the type-checking but that seemed more of a problem to me, as I'm not a real duck.

That does run into some of the same problems w.r.t duck typing, and solutions to problems with overloaded operators are hard to google for. If the method calls were aliased, google searches would be easier and less experienced users wouldn't have to worry about wrestling with operator overloading.

The thing is, "listofstrings.method()" is less obviously elementwise than "listofstrings~.method()", *especially* if "~" becomes the default elementwise character. So I have to disagree. Additionally, "listofstrings~.method()" is easy to paste into an interpreter, and should quickly yield enlightenment. I don't see that as a valid argument.

...

That being said, I think elementwise operators are a great idea in a numeric/scientific context.

But not always in the same case as your broadcasting idea? The elementwise operator is far more general, so I'd say it has a more general use-case in response. Only a fraction of my code is of the "numeric/scientific" type, and I'd lap up elementwise operators.

...

There are of course both pros and cons to having type declarations. I know that people do not like to be limited by a lack of foresight in their predecessors; I have run into unreasonable interface/type specifications and annoyances with private/protected variables in Java and I would never want to see that infiltrate Python. I think what I am trying to sell here is more akin to metadata than static types. People should be able to basically keep doing what they are already doing, but have a reasonable mechanism to provide additional object metadata in a standard way that is easily accessible by downstream consumers. That gives you a lot of the upsides of type declarations while staying Pythonic.

The thing is... what? This isn't a criticism. I'm genuinely lost as to what you mean. I lost you on the "metadata" part. In what way is this metadata, and in turn not typing? Can you do: foo = "a b".split() foo[1] = 1 If you can, then this whole idea should really be generalised more, preferably to the elementwise operator level, as it's got no real type constraints. If you can't, then how is it not typing? Again, this is a genuine question. I'm home-taught, so please don't expect me to know the lingo :P. I actually love the idea of typing when done well. I've just started Haskell (quite a shock to my Python'd brain) and I've thoroughly enjoyed its type system. I just think that it has no place in python. It only works in Haskell because Haskell is built on it, and types can't go a-changin' (nor can values, for that matter). In python, this sort of system will mean that typed code/arrays/iterators will severely grind with untyped code/values/arrays/iterators. Someone defines a 'typed collection or iterator', to paraphrase the title. Someone else makes a goose-that-looks-like-a-duck, but doesn't have a beak. With an untyped system - such as the current method or an elementwise operator - that's fine as you just want it swim, but with your typed container something has to go wrong! Either it crashed, which is anti-duck-typing, or it falls back down, in which case you end up reverting everything to the old method anyway. Haskell gets around this by having both types and 'things that implement things' and so could do this because it knows what everything implements. But in python we don't know, and won't until all our dusty code is rewritten. I apologise for using a language I've just started to learn as a reference, but that's the best reference I've had. *I really should check this, but it's late and I'm tired. If I've said the same thing twice or rambled on about hexagonal motorised paintings for too long, blame my cat...* Maybe a better option than having a "type contract" base for

...

homogeneous collections is to give objects a metadata namespace with a loose schema, and declare it there? As long as you could provide the metadata at object creation time you would still get all the same benefits, and it would avoid potentially overworking the class/mixin concept.

...
Your example: my_string.split("\n").capitalize().join_items("\n") to "\n".join(my_string.split("\n")~.capitalize())

And personally I like "string.join(lst)" over "lst.join_with(string)"...

Of course, everyone has their own preferences; having a very small set of internally consistent ways to do something rather than just one way is good for this reason.

Nathan _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

Before anyone gets too many ideas about what is and isn't:

...

...
...
str.capitalize("ABCDEFGH") 'Abcdefgh'

Remember that these are predominantly lowercasing actions :P

4508

Age (days ago)

4509

Last active (days ago)

List overview

Download

8 comments

4 participants

participants (4)

Ethan Furman
Joshua Landau
Nathan Rice
Serhiy Storchaka

Re: [Python-ideas] Adding "Typed" collections/iterators to Python

tags

participants (4)