Mailman 3 May 2019 - Python-ideas

Re: [Python-ideas] Operator as first class citizens -- like in scala -- or yet another new operator?
by Chris Angelico 23 May '19

23 May '19

On Wed, May 22, 2019 at 10:03 PM Yanghao Hua <yanghao.py(a)gmail.com> wrote: > > To be first-class citizens, operators would have to be able to be > > passed to functions - for instance: > > > > def frob(x, y, oper): > > return x oper y > > assert frob(10, 20, +) == 30 > > assert frob(10, 20, *) == 200 > > > > The nearest Python currently has to this is the "operator" module, in > > which you'll find function versions of the operators: > > > > def frob(x, y, oper): > > return oper(x, y) > > assert frob(10, 20, operator.add) == 30 > > assert frob(10, 20, operator.mul) == 200 > > > > Is this what you're talking about? > > Yes this is exactly what I am talking about .. and this is exactly > what scala does. > In scala, you can do "a + b", as well as "a.+(b)", where "+" is just a > function of object a. > > You know people keep building domain specific languages for specific > problems, and scala seems currently the only one that can truely > enable people to build DSL elegantly. I actually do not know at all > how to do this in CPython as I just started playing with Python > internals a few days ago, but having an arrow operator would solve my > immediate problem in terms of using Python to build a DSL for hardware > design (elegantly!). > > Does this make sense? Yes, it does make sense. Forgive my lack of Scala knowledge, but is it possible for 'b' in your example to be the one that handles the addition? Specific case: class Seven: def __add__(self, other): return other + 7 def __radd__(self, other): return 7 + other seven = Seven() print(seven + 1) print(2 + seven) This works in Python even though 2.+(seven) wouldn't know how to handle this object. So it returns NotImplemented, and Python says, oh okay, maybe the other object knows how to do this. (That's what __radd__ is for.) This is why Python's operators are all defined by the language, with multiple levels of protocol. ChrisA

6 8

Operator as first class citizens -- like in scala -- or yet another new operator?
by Yanghao Hua 23 May '19

23 May '19

> Perhaps you might be able to do what you want using an import hook. I have done some experiments with introducing some new operators that way: https://github.com/aroberge/experimental/blob/master/experimental/transform… Thanks for the link, it is a very interesting module. I have to confess there is one more level of intention I have not talked about ... it is about analyzing the python VM code and translate it to C++/SystemC/Verilog or even doing direct synthesis to hardware basic building blocks (think about AND/OR gates, Flip-Flops etc.). So basically we need a way to tell the difference of a "<-" arrow assign from the normal assign. So having source code modification is definitely an interesting idea, but difficult for this second step to proceed. As one of the first step of checking if a module/function is synthesize-able is to make sure it does NOT just call arbitrary functions. (but of course I can try to decorate the function in certain ways ... but then you have to force this policy to all users for their logic block definitions ... I really wanted an almost one-to-one mapping between a python function to a hardware block ... without even a single decorator ...). I will give it a deeper try to see how far I can go though with this library (it has a few hundred lines of code but supporting arrow operators in python is really just below 100 lines of code ..)

1 0

Operator as first class citizens -- like in scala -- or yet another new operator?
by Yanghao Hua 23 May '19

23 May '19

> If you are proposing to add "<-" and "->" to the fundamental Python > grammar, then it could work as PEP465 does. But if you are proposing > that classes can define new lexical operators, it would have to work > very very differently. You started with "operators as first class > citizens," but I don't know what you mean by that. It is my bad to mix this two topics together ... I am not strongly proposing for now to have python to support arbitrary operator definitions, I can live with the arrow propose if it makes sense. I see also even the term "first class citizen" makes a confusion here ... by which python do have certain extent of support for the existing operators. What I should have emphasized more is the dynamic nature of creating new operators. Which scala seems does a better job in this area for now. > When Python compiles a file, it does not read the imported files at > all. That doesn't happen until run time. So how will Python know what > "<-" means in circuit.py? It's never seen this token before, so it > can't know what special method it should map onto. It can't even know > what the token is. What if it encounters "x ~++<-+ y"? What are the > tokens in that source? I see your point now. Scala does this by simply looking up a method on the object named as "<-" e.g. x <- y is equivalent to "x.<-(y)". If "x ~++<-+ y" is encountered scala will interpret it to "x.~++<-+(y)" most likely (haven't tried), they had also a precedence rule regarding user defined operators. So if we want to change python to support arbitrary operators, the operator itself probably need to be loaded by a VM instruction, and then finding that operator on the object being operated. This sounds like a lot of overhead with probably even a string comparison needed? I think in the short term this is going to be difficult to justify and I have not looked into the performance implications. But anyway in this case the importing seems not necessary, as the operator should have *NOT* being implemented in hdl.py, but is part of the object used in circuit.py.

1 0

Fwd: Operator as first class citizens -- like in scala -- or yet another new operator?
by Yanghao Hua 23 May '19

23 May '19

On Wed, May 22, 2019 at 2:54 PM Rhodri James <rhodri(a)kynesim.co.uk> wrote: > > On 22/05/2019 13:29, Yanghao Hua wrote: > > Problem is python do not allow you to define new operators in the > > language itself, except those pre-defined you can modify their > > behavior. Even in case, if python would have been able to allow me to > > redefine the behavior of "=", e.g. by checking if the left hand side > > has a method (e.g. __assign__) to override the default behavior of the > > equal sign, I would be 100% happy already :) I have looked into all > > the existing operators that python supports which looks like an > > "assignment", none has been fallen in love with me. The example in > > Chisel w/ Scala is: they defined quite a few operators: ":=" for > > signal assignment, "<>" for bulk signal connections etc. And you can > > keep inventing new operators with which your design can look really > > elegant. > > A note of caution: your design can look really elegant *as long as you > know what your new operators mean*. If you don't, it's looks like > incomprehensible noise. I have had far too many bad experiences with > "clever" C++ with redefined operators to assume that "elegant" and > "obvious" go hand in hand. > > If you really must prat around with redefining assignment, make the > relevant attribute a property. Fully acknowledge it. I am from hardware design background so I do assume I might know what it means by elegant in hardware design. I do agree too that overloading the equal sign is probably asking for more problem than the problem it solves. That is the reason why I would like to see if this arrow method I proposed makes sense, or actually Python developers would love to make it possible for users to define their own operators (consider in math, how many operators would mathematicians love to see, and make it possible to write beautiful math formula directly in Python, like the matrix operations in PEP465. Just another example from the TLA+ formal language, if Python would allow defining new operators, you don't even need to write a parser for TLA+, you can write TLA+ using the mathematical operators you like directly in Python instead. https://github.com/tlaplus/Examples/blob/master/specifications/Missionaries… IsSafe(S) == \/ S \subseteq Cannibals \/ Cardinality(S \cap Cannibals) =< Cardinality(S \cap Missionaries)

1 0

DB-API support for sets?
by Skip Montanaro 22 May '19

22 May '19

The DB-API doesn't support sets directly, so you wind up having to manually expand them: >>> curs.execute("select count(*) from sometable where somecol in ?", ({4126,11638},)) Traceback (most recent call last): File "<stdin>", line 1, in <module> sqlite3.OperationalError: near "?": syntax error >>> curs.execute("select count(*) from sometable where somecol in (?)", ({4126,11638},)) Traceback (most recent call last): File "<stdin>", line 1, in <module> sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type. >>> curs.execute("select count(*) from sometable where somecol in (11638, 4126)") <sqlite3.Cursor object at 0x7f8ef5f6c570> In the example above, I'm able to skip input validation because the element values in the set are ints, but programmers being the lazy sort that they are, when such values are strings there's probably also the tendency to skip argument escaping. A quick check on bugs.python.org as well as Google yielded no obvious earlier discussion. Is it simply that underlying database drivers are unable to support this or was DB-API 2.0 frozen (no pun intended) before sets were available or in wide use in Python? Set support and the SQL "in" operator would seem to be handy, both as a convenience and as a barrier to SQL injection errors. Skip

5 4

Passing positional arguments as keyword arguments (to provide function arguments out of order)
by Robert Vanden Eynde 16 May '19

16 May '19

Currently if one wants to provide positional arguments after keyword arguments, it's not possible, one must begin with positional arguments [1] or use keyword arguments [2] : ``` def f(x, *, long_name='foo'): return ... f(2, long_name='bar') # [1] f(long_name='bar', x=2) # [2] ``` The problem is that it's not always possible to do so because some functions cannot have keyword arguments (because they call a C function) like math.hypot : ``` import math math.hypot(y=5, x=2) # TypeError ``` A solution is to use partial. ``` from functools import partial partial(f, long_name='bar')(2) ``` But that begins to be "functional programming style" and that doesn't solve the case where one would like to change the order of (I created funcoperators.elipartial for that reason) : ``` from funcoperators import elipartial elipartial(math.hypot, ..., 5)(2) ``` Of course one could create a tuple and a dict they call f(*a, **k) but cannot be used inside an expression. ``` k = dict(long_name='bar') a = (2, ) f(*a, **k) ``` So what would be a good new syntax for that or workaround I didn't think of ? ``` f(**dict(long_name='bar'), *(2, )) f(long_name='bar', 0=2) math.hypot(0=5, 1=2) f(long_name='bar', args[0]=2) math.hypot(args[0]=5, args[1]=2) ... ``` I don't know if such an idea has been asked before (I don't know how to formulate it). robertvandeneynde.be

6 10

A plea for people to please remember to assume good faith
by Steven D'Aprano 16 May '19

16 May '19

In another thread, a couple of days ago, there were a few ill-tempered[1] accusations thrown around about "rudeness" and "attitude". Please remember that email is notorious for communicating tone of voice very badly. Especially if people overcompensate by trying hard to be extra polite, which makes the tone come across as formal and patronizing. Face to face: "Dude, you need to read the docs." (Yes, I do!) Email: "You need to read the docs." (Really?) Hyper-polite email: "Perhaps you should consider reading the docs." (Oh, I do, do I? What makes you so sure of that you patronizing git?) I know I have a bad habit of over-compensating. Sorry. It is hard to get tone right without slipping too far one way or the other, and phrases which sound "just right" in your head, and the first five times you re-read the email before hitting Send, can sound harsher than intended an hour later. And remember that the writer's attitude is an internal mental state, and judging it from email is not even close to easy or accurate. I mean, sure, you can probably guess the attitude of the author of abusive emails: "Dear Bob, you're a poopy-head filled with poop and your head is poop, you poop." but generally it isn't safe to try to guess the writer's attitude. It is especially not welcoming, considerate, friendly etc etc etc to assume bad faith based on a medium as infamous for leading to over-reactions and misunderstandings as email. So please: - try to assume the author of emails is writing in good faith - especially if the email is trying to express a friendly tone with smileys or winks to soften the tone (give them credit for the attempt even if you think it wasn't enough) - but even without them, it may not always be wise to assume that just because a sentence comes across as cold or harsh that it was intended that way - assume small slights are inadvertent or accidental, not attacks - in mild or borderline cases, don't assume that your judgement of tone is accurate - even if it is, try not to escalate the situation - try to respond to what they say rather than how they say it - and don't make the mistake of thinking that others aren't judging your tone exactly the same way you are judging theirs. They might be just too polite to call you out on it. Even if we *correctly* judge something was intended as a rebuke or snark, so what? Does it matter? We shouldn't be so hyper-sensitive that any and every "insult" requires retaliation. Remember the Englightenment values of *dignity culture* where small slights are beneath our dignity to respond: "Dignity does not depend upon reputation but exists as unalienable rights that do not depend on what other people think of one's bravery. Having a thick skin and shrugging off slights become virtues because they help maintain social peace." https://reason.com/2015/09/11/victimhood-culture-in-america-beyond-dig/ And for the record, I completely acknowledge that I do not always live up to these ideals. [1] And here I am, judging tone and attitude from emails. I know. -- Steven

2 1

easier lazy logging in stdlib ?
by mak apuf 16 May '19

16 May '19

Hello all, Let's consider logging.debug(expensive_function()) There are a few possibilities to run an expensive function when logging only if the log is effectively output : - eager formatting + calling by using fstrings or .format(expensive_func()) - formatting being done lazily by separating the log string from the arguments (by using % syntax which is dated maybe) - creating a Lazy object that calls expensive_func when calling its __str__ method, but that it a bit cumbersome because you have to provide a lazy class for each, that only work for string values and the cost of creating Lazy is eager. - checking logging.isEnabledFor(lvl) explicitely (but then the level is displayed twice, in enabled_for and in logging.LEVEL) Maybe we can integrate a form of lazy evaluation for simple logging. Different options are : a- just evaluate when an argument is callable (not bw compatible, what if this is unexpected and has side effects) b- provide a LazyFormatter that does the preceding for any "%s" or "%d" arg when used c- provide a callable as part of the logging arguments and provide a "%*s or %*d" format string instead of %s and %d (by example) for calling the function before embedding it d- check if a parameter has a special method such as __lazy__ at render time e- provide a Lazy class in logging module to wrap lazy logging idiom: class Lazy : def __init__(self, callable, *args, **kwargs) : self.callable = callable self.args=args self.kwargs=kwargs def __lazy__(self) : return self.callable(*args, **kwargs) and checking if an argument is of Lazy type (again the drawback is that this eagerly instanciate this class) I'd prefer b,c or d+e any thoughts? -- makapuf

3 2

(meta) Mailman cleartext passwords
by Tom Hale 14 May '19

14 May '19

When I signed up, I got an email with subject: Welcome to the "Python-ideas" mailing list It included the text: ====================================================================== You must know your password to change your options (including changing the password, itself) or to unsubscribe without confirmation. It is: my-cleartext-password ====================================================================== This has bugged me about Mailman in the past... luckily I used a low-tier password for this (I know, I should use a unique password). But some users will use a password which is valuable to them. Should this "feature" be turned off for new subscribers? -- Tom Hale

2 1

(meta) Mailman list searching
by Tom Hale 14 May '19

14 May '19

How do I search this list's archives? I can't see any search functionality at: https://mail.python.org/pipermail/python-ideas/ Using the other archive recommended recently: https://code.activestate.com/search/lists/?sa=Search&list=python-ideas#q=ma… Gives me: The Google Web Search API is no longer available. Please migrate to the Google Custom Search API (https://developers.google.com/custom-search/) -- Tom Hale

2 2