[Python-ideas] Vectorization [was Re: Add list.join() please]

[David Mertz]

On Sun, Feb 3, 2019 at 1:38 PM Adrien Ricocotam <ricocotam at gmail.com> wrote:

> I honestly don’t understand what you don’t like the @ syntax.
>

Can you show any single example that would work with the @ syntax that
would not work in almost exactly the same way without it? I have not seen
any yet, and none seem obvious.  Adding new syntax for its own sake is
definitely to be avoided when possible (even though technically the
operator exists, so it wouldn't be actual new syntax).


> My idea is using functions that takes on argument : an object of the type
> of the vector. That’s actually how map works.
>

I do not understand this.  Spell my simple example using @ notation.  I.e.

my_vec @ replace {something? here for 'foo' with 'bar'}



> What I understood from your previous message is that there’s ambiguity
> when using magic functions on whether it’s applied to each element of the
> vector or the vector itself. That was the first thing I saw.
>

I decided there really isn't.  I think that any function applied to the
vector should operate on the sequence as a whole.  E.g. what length does it
have? Cast it to a different kind of sequence. Print it out. Serialize it.
Etc.

The things that are vectorized should always be methods of the vector
instead.  And ALMOST every method should in fact be a vectorized
operation.  In most cases, those will be a "pass through" to the methods of
the items inside of the vector. We won't write every possible method in the
Vector class.

My toy so far only works with methods that the items actually have.  In the
examples, string methods.  But actually, I should add one method like this:

my_vec.apply(lambda x: x*2)


That is, we might want to vectorize custom functions also.  Maybe in that
example we should name the function 'double' for clarity: '
my_vec.apply(double)'.  I do think that just a few methods need to be
custom programmed because they correspond to magic methods of the items
rather than regular names (or not even directly to magic methods, but more
machinery).  So:

my_vec.list()  #-> cast each item to a list

my_vec.tuple() #-> cast each item to a tuple

my_vec.set()   #-> cast each item to a set


Maybe that's doing too much though.  We could always do that with map() or
comprehensions; it's not clear it's a common enough use case.

Functions that could be used are then the same we can use in map. But I do
> agree it’s not easy to have functions with parameters. That’s why I used
> functools.partial
>

I really did not understand how that was meant to work.  But it was a whole
lot of lines to accomplish something very small either way.


> On Sun 3 Feb 2019 at 19:23, David Mertz <mertz at gnosis.cx> wrote:
>
>> On Sun, Feb 3, 2019 at 3:54 AM Adrien Ricocotam <ricocotam at gmail.com>
>> wrote:
>>
>>> I think all the issues you have right now would go of using another
>>> operation. I proposed the @ notation that is clear and different from
>>> everything else,
>>>
>> plus the operator is called "matmul" so it completely makes sense. The
>>> the examples would be :
>>>
>>
>>
>>> >>> l = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()
>>> >>> v = Vector(l)
>>> >>> len(v)
>>> 12
>>> >>> v @ len
>>> <Vector of [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3]>
>>>
>>
>> I cannot really see how using the @ operator helps anything here.  If
>> this were a language that isn't Python (or conceivably some future version
>> of Python, but that doesn't feel likely or desirable to me), I could
>> imagine @ as an operator to vectorize any arbitrary sequence (or
>> iterator).  But given that we've already made the sequence into a Vector,
>> there's no need for extra syntax to say it should act in a vectorized way.
>>
>> Moreover, your syntax is awkward for methods with arguments.  How would I
>> spell:
>>
>> v.replace('foo', 'bar')
>>
>>
>> In the @ syntax? I actually made an error on my first pass where simply
>> naming a method was calling it.  I thought about keeping it for a moment,
>> but that really only allows zero argument calls.
>>
>> I think the principled thing to do here is add the minimal number of
>> methods to Vector itself, and have everything else pass through as
>> vectorized calls.  Most of that minimal number are "magic method":
>> __len__(), __contains__(), __str__(), __repr__(), __iter__(),
>> __reversed__().  I might have forgotten a couple.  All of those should not
>> be called directly, normally, but act as magic for operators or built-in
>> functions.
>>
>> I think I should then create regular methods of the same name that
>> perform the vectorized version.  So we would have:
>>
>> len(v)   # -> 12
>>
>> v.len()  # -> <Vector of [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3]>
>>
>> list(v)  # -> ["Jan", "Feb", "Mar", "Apr", "May", "Jul" ...]
>> v.list() # -> <Vector of [["J", "a", "n"], ["F", "e", "b"] ... >
>>
>>
>> I can't implement every single constructor that users might
>> conceivably want, of course, but I can do it for the basic types in
>> builtins and common standard library.  E.g. I might do:
>>
>> v.deque() # -> <Vector of [deque(["J", "a", "n"]), deque(["F", "e",
>> "b"]) ... >
>>
>>
>> But I certainly won't manually add:
>>
>> v.custom_linked_list()  # From my_inhouse_module.py
>>
>>
>> Hmm... maybe even I could look at names of maybe-constructors in the
>> current namespace and try them.  That starts to feel too magic.  Falling
>> back to this feels better:
>>
>> map(custom_linked_list, v)  # From my_inhouse_module.py
>>
>>
>>
>> --
>> Keeping medicines from the bloodstreams of the sick; food
>> from the bellies of the hungry; books from the hands of the
>> uneducated; technology from the underdeveloped; and putting
>> advocates of freedom in prisons.  Intellectual property is
>> to the 21st century what the slave trade was to the 16th.
>>
>

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20190203/5ee4bfde/attachment.html>