Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.

[Joshua Landau]

On 21 December 2011 15:07, Nathan Rice <nathan.alexander.rice at gmail.com>wrote:

> On Tue, Dec 20, 2011 at 8:37 PM, Joshua Landau
> <joshua.landau.ws at gmail.com> wrote:
> > On 21 December 2011 00:24, Nathan Rice <nathan.alexander.rice at gmail.com>
> > wrote:
> >> efoo_res = ((efoo2.capitalize() + " little indian").split("
> >> ").apply(reversed) * 2).apply("_".join) # note that you could do
> >> reversed(...) instead, I just like to read left to right
> >> efoo_res.parent.parent.parent # same as ((efoo2.capitalize() + "
> >> little indian").split(" ") in case you need to debug something and
> >> want to look at intermediate values
> >
> > How is any of this better than the elementwise operators ("~")? People
> > should be able to expect len(x) to always return a number or raise an
> error.
> > I know it's not part of the spec, but a lot breaks without these
> guarantees.
> > When "str(x)" isn't a string, all the formatting code breaks*. And when
> the
> > other suggestion ("~str(x)" or "str~(x)" or something similar) has all
> the
> > benifits and none of the drawbacks, why should I use this?
>
> len() will always return a number or raise an error, just like the
> type functions (bool/int/etc) return that type or raise an error.  The
> interpreter guarantees that for you.


The point wasn't that either way was better, but that with this
implementation you get neither choice ("len(x)" vs "len~(x)") or
reliability.

The reliability point works like this: You want to elementwise a few
functions, that before you were doing on a single item.
BEFORE:
item = foreignfunc1(item)
item = foreignfunc2(item)
item = foreignfunc3(item)

NOW (your method):
item = ElementwiseProxy(item)
item = foreignfunc1(item)
item = foreignfunc2(item)
item = foreignfunc3(item)
item = list(item)

NOW (the PEP):
item = foreignfunc1~(item)
item = foreignfunc2~(item)
item = foreignfunc3~(item)

NOW (map): # or imap to do it lazily
item = map(foreignfunc1, (item))
item = map(foreignfunc2, (item))
item = map(foreignfunc3, (item))

The PEP is short and quaint, and makes lots of things shorter than map
("[1,2,3]~+[5,6,7]"), so you can see the advantages. That's why I like the
PEP as "syntatic sugar". The mapping method works fine.

*You might think your method works. *But what if foreignfunc is "str"? *And
you can't blacklist functions.* You can't say everything works bar A, B and
C. What if you get: "lambda x:str(x)"? You can't blacklist that. That makes
the ElementwiseProxy version buggy and prone to unstable operation. If it's
consistent, fine. But it's not.


This has a couple of advantages over element-wise operators:
>

Couple == 2 :P


> 1. Because everything is handled in terms of generator chains, all
> operations on an ElementwiseProxy are evaluated lazily.  With
> element-wise operator overloading you would need to perform each
> operation immediately.
>

I agree this can be a preferred advantage.But you still have to choose one.
"~" could be lazy, or it could be eager. But in both implementations you
have to choose. That said, you have map and imap, and so you could have
ElemetwiseProxy and eElementwiseProxy (eager), and you can have "~" and
"i~". Remember that the syntax I'm using is just for PEP consistency. Some
more creative people can find a syntax that works.


> 2. As a result of #1, you can "undo" operations you perform on an
> ElementwiseProxy with the parent property.
>

Use case? If this is actually a wanted feature, "parent" could be made a
general property of iterators.
(x for x in foo).parent == foo
I think that's a separate proposal that isn't intrinsic to this idea.

3. This still works if the person who created the class you're working
> with doesn't add support for element-wise operators.  Sure, you could
> monkey patch their code, but that can lead to other problems down the
> line.
>

Example? I'm struggling to think of a case where you could do this only
with one.


> 4. There isn't an obvious/intuitive character for element-wise
> versions of operators, and fewer symbols is better than more IMHO
> (see: Perl).  Also, if you use the ElementwiseProxyMixin, you can
> sprinkle element-wise stuff in neatly just by using "variable.each"
> where you would use "~" in your examples.


Obvious/intuitive is meaningless for something so potentially common. "**"
is far more obscure ("*" is obscure enough). Why is a double star the
exponent character? Because it works. "~" would be easy to learn and
consistent, once a standard was agreed. And again, I'm only using that
syntax because it's the one in the PEP. We're not going to get to Perl, I
hope, with this addition.

Explicit > Implicit. If someone passes you a "variable.each", things go
haywire (see: first section of reply). "~" is explicit. It doesn't have
these problems. I also think explicit looks nicer. 'Cause it is, y'know :P


> > Upon this implementation I take back my comment on the whole typing
> thing.
> > Your title just really confused me.
> >
> > * You can't just make functions non-elementwise unless called through
> > ".apply" either:
> > def str(x): return x.__str__()
>
> str is one of the special cases (along with repr, unicode, int, float,
> long, bool, etc).  These can't ever be elementwise in CPython (I don't
> know that this holds in other interpreters).
>

I'll hammer my point in again.

"foo~(x)" is elementwise.
"typeofx(foo(ElementwiseProxy(x)))" *may* be elementwise, depending on
whether foo is a special case, or uses a special case, or is an obscure bug
if it uses both a special case and a non special case.

"x~.foo" is an elementwise "getattr" on a list of things.
"ElementwiseProxy(x).foo" is elementwise or a *ramble ramble* *exceptions*
*special cases*.

---------------------------------------------------

To wrap this up, one of your points was style and I disagree, two were
implementation agnostic and thus not advantages to yours, and one was a
plausible but rare use-case where I feel I need an example. Additionally,
your implementation technique has several seemingly unavoidable yet
inexplicable behaviours.

The idea is that you use ElementwiseProxy(x) or preferably x.each,
> work with your data in an element-wise capacity as needed, then list()
> or iter() back out.  I think it should be a deliberate change of
> context.  Interleaving the proxy (say via x.each) in mostly scalar
> expressions works, but wasn't the use case I designed it for, and in
> those circumstances it doesn't really offer anything above the map
> function.
>
> Nathan
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20111221/e387b0ba/attachment.html>