Dear all,
The matrix multiplication operator @ is going to be introduced in Python
3.5 and I am thinking about the following idea:
The semantics of matrix multiplication is the composition of the
corresponding linear transformations.
A linear transformation is a particular example of a more general concept -
functions.
The latter are frequently composed with ("wrap") each other. For example:
plot(real(sqrt(data)))
However, it is not very readable in case of many wrapping layers.
Therefore, it could be useful to employ
the matrix multiplication operator @ for indication of function
composition. This could be done by such (simplified) decorator:
class composable:
def __init__(self, func):
self.func = func
def __call__(self, arg):
return self.func(arg)
def __matmul__(self, other):
def composition(*args, **kwargs):
return self.func(other(*args, **kwargs))
return composable(composition)
I think using such decorator with functions that are going to be deeply
wrapped
could improve readability.
You could compare (note that only the outermost function should be
decorated):
plot(sorted(sqrt(real(data_array)))) vs. (plot @ sorted @ sqrt @ real)
(data_array)
I think the latter is more readable, also compare
def sunique(lst):
return sorted(list(set(lst)))
vs.
sunique = sorted @ list @ set
Apart from readability, there are following pros of the proposed decorator:
1. Similar semantics as for matrix multiplication.
2. Same symbol for composition as for decorators.
3. The symbol @ resembles mathematical notation for function composition: ∘
I think it could be a good idea to add such a decorator to the stdlib
functools module.
A big blocker to making certain sweeping changes to CPython (e.g.
ref-counting) is compatibility with the vast body of C extension
modules out there that use the C-API. While there are certainly
drastic long-term solutions to that problem, there is one thing we can
do in the short-term that would at least get the ball rolling. We can
put a big red note at the top of every page of the C-API docs that
encourages folks to either use CFFI or Cython.
Thoughts?
-eric
Greetings,
I am attempting to use pathlib to recursively glob and/or find files. File
permissions and groups are all over the place due to poor management of the
filesystem which is out of my control.
The problem occurs when I lack both permissions and group membership to a
directory that rglob attempts to descend into. Rglob throws a KeyError and
then a PermissionError and finally stops entirely. I see no way to recover
gracefully from this and continue globbing. Is this the expected behavior
in this case?
The behavior that I want is for rglob to skip directories that I don't have
permissions on and to generate the list of everything that it saw/had
permissions on. The all or nothing nature isn't going to get me very far in
this particular case because I'm almost guaranteed to have bad permissions
on some directory or another on every run.
More specifics: Python: 3.4.1 (and 3.4.3) compiled from source for linux
Filesystem I am globbing on: automounted nfs share
How to reproduce:
mkdir /tmp/path_test && cd /tmp/path_test && mkdir dir1 dir2 dir2/dir3
&& touch dir1/file1 dir1/file2 dir2/file1 dir2/file2 dir2/dir3/file1
su
chmod 700 dir2/dir3/
chown root:root dir2/dir3/
exit
python 3.4.1
from pathlib import Path
p = Path('/tmp/path_test')
for x in p.rglob('*') : print(x)
> On Thu, May 7, 2015 at 7:09 PM, Jo?o Santos <jmcs(a)jsantos.eu> wrote:
> > On Wed, 6 May 2015 at 16:51 Steven D'Aprano <steve(a)pearwood.info> wrote:
> >>
> >>
> >> I think that there are some questions that would need to be answered.
> >> For instance, given some composition:
> >>
> >> f = math.sin @ (lambda x: x**2)
> >>
> >> what would f.__name__ return? What about str(f)?
> >
> >
> > Lambdas return '<lambda>' so maybe something like '<composed>'?
> > Then str(f) would be '<function <composed> at 0xffffffffffff>'.
>
> Would be nice to use "<sin @ <lambda>>", incorporating both names, but
> that could get unwieldy once you compose a bunch of functions.
Maybe it would be better to have '<function <composed> at 0xffffffffffff>'
for str(f) but 'sin @ <lambda>' for repr (f). So that one can have more
info and it would be closer to idealistic obj == eval (repr (obj)).
Dear all,
The matrix multiplication operator @ is going to be introduced in Python
3.5 and I am thinking about the following idea:
The semantics of matrix multiplication is the composition of the
corresponding linear transformations.
A linear transformation is a particular example of a more general concept -
functions.
The latter are frequently composed with ("wrap") each other. For example:
plot(real(sqrt(data)))
However, it is not very readable in case of many wrapping layers.
Therefore, it could be useful to employ
the matrix multiplication operator @ for indication of function
composition. This could be done by such (simplified) decorator:
class composable:
def __init__(self, func):
self.func = func
def __call__(self, arg):
return self.func(arg)
def __matmul__(self, other):
def composition(*args, **kwargs):
return self.func(other(*args, **kwargs))
return composable(composition)
I think using such decorator with functions that are going to be deeply
wrapped
could improve readability.
You could compare (note that only the outermost function should be
decorated):
plot(sorted(sqrt(real(data_array)))) vs. (plot @ sorted @ sqrt @ real)
(data_array)
I think the latter is more readable, also compare
def sunique(lst):
return sorted(list(set(lst)))
vs.
sunique = sorted @ list @ set
Apart from readability, there are following pros of the proposed decorator:
1. Similar semantics as for matrix multiplication.
2. Same symbol for composition as for decorators.
3. The symbol @ resembles mathematical notation for function composition: ∘
I think it could be a good idea to add such a decorator to the stdlib
functools module.
I should clarify why I would like to have the possibility to easily compose
functions.
I am a physicist (not a real programmer), and in my code I often compose
functions.
To do this I need to write something like
def new_func(x):
return f(g(h(x)))
This means I see f(g(h())) quite often and I would prefer to see f @ g @ h
instead.
> > Compared to asynchronous code, I would say function composition is
> > trivial. Anyone who can learn the correspondence
> >
> > (a @ b)(arg) <=> a(b(arg))
> >
> > can deal with it.
>
>
> Personally, I can certainly "deal" with it, but it'll never come naturally
> to me. As soon as I see code like this I have to mentally pick it apart
and
> rewrite it in the more familiar form before I understand what's going on.
>
> Maybe if I needed this frequently I'd learn to fly with it, but I just
> don't see the need that often. I see things like f().g() much more often
> than f(g()).
>
> --
> --Guido van Rossum (python.org/~guido)
> Apologies for the split replies; is everyone else seeing this
> as three separate threads spawned
> from two copies of the original mail, or is this just Yahoo sucking again?
Probably that is my fault. I have sent first a message via google-group,
but then received a message from python-ideas(a)python.org that my message
has not been delivered and sent the second directly. Sorry for that. This
is my first post here.
> And notice that the author of this current proposal
> thinks we should add the same thing to Python.
> Doesn't that make you worry that maybe compose
> belongs to the wrong universe?
I would like to clarify that I don't want to add all Haskell to Python, on
the contrary, I wanted to propose a small subset of tools that could be
useful. Your position as I understand is that it is not easy: either you
get many complex tools, or you get useless tools.
Still, I think one could try to find some kind of compromise.
Dear Chris,
> > My original idea was to make the composable functions auto-curried
(similar
> > to proposed here
> > http://code.activestate.com/recipes/52902-function-composition/ as
pointed
> > out by Steve) so that
> >
> > my_fun = square @ add(1)
> > my_fun(x)
> >
> > evaluates to
> >
> > square(add(1,x))
> It also requires that
> the right hand function be
> composable, unlike in your earlier
> example.
This is true. One can only use single argument "normal" functions. Multiple
argument ones should be made "composable".
>
> ChrisA
>
Dear Steve,
Thank you for the feedback and for the links!
I think that both (f(a)g).__name__ and str(f@g) should be
f.__name__ + ' @ ' + g.__name__
and
str(f) + ' @ ' +str(g)
Concerning the performance, I think that it could be poor, and I don't know
yet how to improve this.
> > Dear all,
> >
> > The matrix multiplication operator @ is going to be introduced in Python
> > 3.5 and I am thinking about the following idea:
> >
> > The semantics of matrix multiplication is the composition of the
> > corresponding linear transformations.
> > A linear transformation is a particular example of a more general
concept -
> > functions.
> > The latter are frequently composed with ("wrap") each other. For
example:
> >
> > plot(real(sqrt(data)))
> >
> > However, it is not very readable in case of many wrapping layers.
> > Therefore, it could be useful to employ
> > the matrix multiplication operator @ for indication of function
> > composition. This could be done by such (simplified) decorator:
>
> I like the idea of @ as a function compose operator.
>
> There have been many requests and attempts at support for function
> composition:
>
> http://code.activestate.com/recipes/574458-composable-functions/
>
> http://code.activestate.com/recipes/52902-function-composition/
>
>
http://code.activestate.com/recipes/528929-dynamic-function-composition-dec…
>
> http://blog.o1iver.net/2011/08/09/python-function-composition.html
>
> https://mail.python.org/pipermail/python-dev/2009-August/091161.html
>
>
http://stackoverflow.com/questions/2281693/is-it-a-good-idea-to-have-a-synt…
>
>
> The last one is notable, as it floundered in part on the lack of a good
> operator. I think @ makes a good operator for function composition.
>
>
> I think that there are some questions that would need to be answered.
> For instance, given some composition:
>
> f = math.sin @ (lambda x: x**2)
>
> what would f.__name__ return? What about str(f)?
>
>
> Do the composed functions:
>
> (spam @ eggs @ cheese)(x)
>
> perform acceptibly compared to the traditional syntax?
>
> spam(eggs(cheese(x))
>
>
>
> --
> Steve
Hi all,
I noticed a typo in my first email (had written__aenter__ instead of
__enter__). I fixed the typo below.
-- Koos
On 5.5.2015 16:55, Koos Zevenhoven wrote:
>
> Relation to PEP492:
>
> This of course still needs more thinking, but I wanted to post it here
> now in case there is desire to prepare for something like this already
> in PEP492. It is not completely clear if/how this would need to affect
> PEP492, but some things come to mind. For example, this could
> potentially remove the need for __aenter__, __aiter__, etc. or even
> "async for" and "async with". If __enter__ is defined as "async def",
> then a with statement would do an "await" on it, and the context
> manager would have __async__.framework (or whatever it would be
> called) available, for determining what behavior is appropriate.
>
