[Python-ideas] Operator as first class citizens -- like in scala -- or yet another new operator?

[Ned Batchelder]

On 5/22/19 8:56 AM, Yanghao Hua wrote:
>>> And this is something I have in mind for a Python DSL for HDL:
>>>
>>> def combinational_or_sequential_logic(in: Signal, out: Signal):
>>>       local_signal = Signal()
>>>       local_signal <- in << 10 # read as: local_signal <- (in << 10)
>>>       out <- local_signal + 5 # read as out <- (local_signal + 5)
>>>       # new_signal <- 3 will raise exception, as <- does not create new object.
>>>
>>> And the arrow operators could also be used for bulk connections of a
>>> chain of hardware modules like this:
>>> module_instance1 -> instance2 -> instance3->... to naturally form a
>>> hardware pipeline ... this looks very elegant.
>>
>> I think the largest problem with this idea has to do with where the new
>> operator would be defined, and then where would it be used.  At first
>> blush, it seems like you'd want to define the operator in one file
>> (let's call it hdl.py), then import that into another file (circuit.py),
>> which could use the new operator.
>>
>> But Python compiles circuit.py without reading hdl.py at all. It merely
>> compiles "import hdl" to some bytecode to import hdl.py. So how would
>> the operator's definition be used during the compilation of circuit.py?
>> The compiler has no idea that new operators have been defined.
> The scala implementation purely relies on the object to provide the
> operator definition, e.g. "a b c" would be interpreted as "a.b(c)".
> With the current implementation of the arrow operators I have, "<-"
> and "->" are part of of the basic python operators pretty much like
> "+" and "-". Where it relies on the object to provide the special
> method "__arrow__". so, this operation could be defined in a separate
> file with a base class e.g. Signal or Module, where other python
> module has to import it to use it or inherit it.
>
> In this case, essentially when python compiler reads circuit.py, it
> reads hdl.py too as it is imported in circuit.py. This is exactly how
> __matmult__ special method works too in PEP465.

If you are proposing to add "<-" and "->" to the fundamental Python 
grammar, then it could work as PEP465 does. But if you are proposing 
that classes can define new lexical operators, it would have to work 
very very differently.  You started with "operators as first class 
citizens," but I don't know what you mean by that.

Usually "first class citizens" means that a thing can be used as a 
value, for example, passed as an argument to a function.  I don't think 
that is what you mean.

Earlier you said:

> With Python, if people already had the thought to allow user to define
> arbitrary operators, I would like to wait and contribute to that
> effort. Because I think this will make Python the language of choice
> for almost all future domain specific language designs.
So I think what you mean by "first-class citizens" is that ability: to 
define new operator tokens in Python itself. Unfortunately, this would 
require untenable changes to how Python operates.

When Python compiles a file, it does not read the imported files at 
all.  That doesn't happen until run time.  So how will Python know what 
"<-" means in circuit.py?  It's never seen this token before, so it 
can't know what special method it should map onto. It can't even know 
what the token is.  What if it encounters "x ~++<-+ y"?  What are the 
tokens in that source?

--Ned.