Python-ideas May 2015

python-ideas@python.org

83 participants
57 discussions

by Lesego Moloko

Dear all I am python novice and I am experiencing some problems with one of my programs that I converted from Matlab to Python. Is this an appropriate platform to ask such questions? I will be awaiting your answer before posting details of my problem, Thank you Regards Lesego Sent from Lesego's iPhone

5 years

(no subject)

by Ivan Levkivskyi

Dear all, The matrix multiplication operator @ is going to be introduced in Python 3.5 and I am thinking about the following idea: The semantics of matrix multiplication is the composition of the corresponding linear transformations. A linear transformation is a particular example of a more general concept - functions. The latter are frequently composed with ("wrap") each other. For example: plot(real(sqrt(data))) However, it is not very readable in case of many wrapping layers. Therefore, it could be useful to employ the matrix multiplication operator @ for indication of function composition. This could be done by such (simplified) decorator: class composable: def __init__(self, func): self.func = func def __call__(self, arg): return self.func(arg) def __matmul__(self, other): def composition(*args, **kwargs): return self.func(other(*args, **kwargs)) return composable(composition) I think using such decorator with functions that are going to be deeply wrapped could improve readability. You could compare (note that only the outermost function should be decorated): plot(sorted(sqrt(real(data_array)))) vs. (plot @ sorted @ sqrt @ real) (data_array) I think the latter is more readable, also compare def sunique(lst): return sorted(list(set(lst))) vs. sunique = sorted @ list @ set Apart from readability, there are following pros of the proposed decorator: 1. Similar semantics as for matrix multiplication. 2. Same symbol for composition as for decorators. 3. The symbol @ resembles mathematical notation for function composition: ∘ I think it could be a good idea to add such a decorator to the stdlib functools module.

5 years

discouraging direct use of the C-API

by Eric Snow

A big blocker to making certain sweeping changes to CPython (e.g. ref-counting) is compatibility with the vast body of C extension modules out there that use the C-API. While there are certainly drastic long-term solutions to that problem, there is one thing we can do in the short-term that would at least get the ball rolling. We can put a big red note at the top of every page of the C-API docs that encourages folks to either use CFFI or Cython. Thoughts? -eric

5 years

Handling lack of permissions/groups with pathlib's rglob

by Frank Woodall

Greetings, I am attempting to use pathlib to recursively glob and/or find files. File permissions and groups are all over the place due to poor management of the filesystem which is out of my control. The problem occurs when I lack both permissions and group membership to a directory that rglob attempts to descend into. Rglob throws a KeyError and then a PermissionError and finally stops entirely. I see no way to recover gracefully from this and continue globbing. Is this the expected behavior in this case? The behavior that I want is for rglob to skip directories that I don't have permissions on and to generate the list of everything that it saw/had permissions on. The all or nothing nature isn't going to get me very far in this particular case because I'm almost guaranteed to have bad permissions on some directory or another on every run. More specifics: Python: 3.4.1 (and 3.4.3) compiled from source for linux Filesystem I am globbing on: automounted nfs share How to reproduce: mkdir /tmp/path_test && cd /tmp/path_test && mkdir dir1 dir2 dir2/dir3 && touch dir1/file1 dir1/file2 dir2/file1 dir2/file2 dir2/dir3/file1 su chmod 700 dir2/dir3/ chown root:root dir2/dir3/ exit python 3.4.1 from pathlib import Path p = Path('/tmp/path_test') for x in p.rglob('*') : print(x)

5 years

Re: [Python-ideas] Function composition (was no subject)

by Ivan Levkivskyi

> On Thu, May 7, 2015 at 7:09 PM, Jo?o Santos <jmcs(a)jsantos.eu> wrote: > > On Wed, 6 May 2015 at 16:51 Steven D'Aprano <steve(a)pearwood.info> wrote: > >> > >> > >> I think that there are some questions that would need to be answered. > >> For instance, given some composition: > >> > >> f = math.sin @ (lambda x: x**2) > >> > >> what would f.__name__ return? What about str(f)? > > > > > > Lambdas return '<lambda>' so maybe something like '<composed>'? > > Then str(f) would be '<function <composed> at 0xffffffffffff>'. > > Would be nice to use "<sin @ <lambda>>", incorporating both names, but > that could get unwieldy once you compose a bunch of functions. Maybe it would be better to have '<function <composed> at 0xffffffffffff>' for str(f) but 'sin @ <lambda>' for repr (f). So that one can have more info and it would be closer to idealistic obj == eval (repr (obj)).

5 years

Add 'composable' decorator to functools (with @ matrix multiplication)

by Ivan Levkivskyi

5 years

Re: [Python-ideas] (no subject)

by Ivan Levkivskyi

I should clarify why I would like to have the possibility to easily compose functions. I am a physicist (not a real programmer), and in my code I often compose functions. To do this I need to write something like def new_func(x): return f(g(h(x))) This means I see f(g(h())) quite often and I would prefer to see f @ g @ h instead. > > Compared to asynchronous code, I would say function composition is > > trivial. Anyone who can learn the correspondence > > > > (a @ b)(arg) <=> a(b(arg)) > > > > can deal with it. > > > Personally, I can certainly "deal" with it, but it'll never come naturally > to me. As soon as I see code like this I have to mentally pick it apart and > rewrite it in the more familiar form before I understand what's going on. > > Maybe if I needed this frequently I'd learn to fly with it, but I just > don't see the need that often. I see things like f().g() much more often > than f(g()). > > -- > --Guido van Rossum (python.org/~guido)

5 years

Re: [Python-ideas] (no subject)

by Ivan Levkivskyi

> Apologies for the split replies; is everyone else seeing this > as three separate threads spawned > from two copies of the original mail, or is this just Yahoo sucking again? Probably that is my fault. I have sent first a message via google-group, but then received a message from python-ideas(a)python.org that my message has not been delivered and sent the second directly. Sorry for that. This is my first post here. > And notice that the author of this current proposal > thinks we should add the same thing to Python. > Doesn't that make you worry that maybe compose > belongs to the wrong universe? I would like to clarify that I don't want to add all Haskell to Python, on the contrary, I wanted to propose a small subset of tools that could be useful. Your position as I understand is that it is not easy: either you get many complex tools, or you get useless tools. Still, I think one could try to find some kind of compromise.

5 years

Re: [Python-ideas] Add 'composable' decorator to functools

by Ivan Levkivskyi

Dear Chris, > > My original idea was to make the composable functions auto-curried (similar > > to proposed here > > http://code.activestate.com/recipes/52902-function-composition/ as pointed > > out by Steve) so that > > > > my_fun = square @ add(1) > > my_fun(x) > > > > evaluates to > > > > square(add(1,x)) > It also requires that > the right hand function be > composable, unlike in your earlier > example. This is true. One can only use single argument "normal" functions. Multiple argument ones should be made "composable". > > ChrisA >

5 years

Re: [Python-ideas] Function composition

by Ivan Levkivskyi

Dear Steve, Thank you for the feedback and for the links! I think that both (f(a)g).__name__ and str(f@g) should be f.__name__ + ' @ ' + g.__name__ and str(f) + ' @ ' +str(g) Concerning the performance, I think that it could be poor, and I don't know yet how to improve this. > > Dear all, > > > > The matrix multiplication operator @ is going to be introduced in Python > > 3.5 and I am thinking about the following idea: > > > > The semantics of matrix multiplication is the composition of the > > corresponding linear transformations. > > A linear transformation is a particular example of a more general concept - > > functions. > > The latter are frequently composed with ("wrap") each other. For example: > > > > plot(real(sqrt(data))) > > > > However, it is not very readable in case of many wrapping layers. > > Therefore, it could be useful to employ > > the matrix multiplication operator @ for indication of function > > composition. This could be done by such (simplified) decorator: > > I like the idea of @ as a function compose operator. > > There have been many requests and attempts at support for function > composition: > > http://code.activestate.com/recipes/574458-composable-functions/ > > http://code.activestate.com/recipes/52902-function-composition/ > > http://code.activestate.com/recipes/528929-dynamic-function-composition-dec… > > http://blog.o1iver.net/2011/08/09/python-function-composition.html > > https://mail.python.org/pipermail/python-dev/2009-August/091161.html > > http://stackoverflow.com/questions/2281693/is-it-a-good-idea-to-have-a-synt… > > > The last one is notable, as it floundered in part on the lack of a good > operator. I think @ makes a good operator for function composition. > > > I think that there are some questions that would need to be answered. > For instance, given some composition: > > f = math.sin @ (lambda x: x**2) > > what would f.__name__ return? What about str(f)? > > > Do the composed functions: > > (spam @ eggs @ cheese)(x) > > perform acceptibly compared to the traditional syntax? > > spam(eggs(cheese(x)) > > > > -- > Steve

5 years

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas May 2015