On Thu, May 23, 2019 at 4:49 AM Ricky Teachey <ricky@teachey.org> wrote:

Can I offer an alternative suggestion?

How about these arrow operators, which despite being 3 characters, look pretty nice, and are not currently valid python:

a <== b <== c a ==> b ==> c

Perhaps __larrow__ and __rarrow__ for the dunders.

And they would not be the only 3 character operators in python:

//=, **=, >>=, <<=

I wouldn't mind one more char to type ... as long as it looks good. My preference order is: = (almost impossible), := (taken already ...), <-/<= (conflicts existing code and breaks bad coding styles). Looks like in the short term there is not really many choice left. I like it, definitely better than signal.next ... and still looks like verilog <= assignment. Is this (<== and ==>) something can be made into CPython?

On Fri, May 24, 2019 at 1:59 AM Steven D'Aprano <steve@pearwood.info> wrote:

On Thu, May 23, 2019 at 06:23:31PM +0200, Yanghao Hua wrote:

...

Is this (<== and ==>) something can be made into CPython?

If it goes into CPython, eventually every other Python needs to do the same.

Of course it *could* be put into Python, but you haven't given sufficient justification for why it *should* be put into Python.

Why does your DSL need a seperate assignment operator? How is it different from regular assignment?

Could you use __setitem__ instead?

Instead of this:

variable <== thing # magic happens here

can you write this?

obj.variable = thing # magic happens here

If your DSL is so different from Python that it needs new operators, perhaps you should write an interpreter for your language and run that seperately.

code = """ any syntax you like <- thing """ result = interpret(code)

Well, if python is not envisioned to be able to represent almost everything elegantly maybe I should indeed walk away from this idea. (This is sad for me ... I have been saying and believing python is envisioned that way for more than 20 years now). Your argument could be applied on PEP465 as well, where the only justification is to make matrix operation prettier. I have explained the problem of use descriptors in previous replies, where you cannot have a local signal, e.g. obj.signal = thing # works, but local_signal = thing # doesn't work.

On Fri, May 24, 2019 at 10:30 AM Paul Moore <p.f.moore@gmail.com> wrote:

Personally, my understanding is that Python is not designed to make writing DSLs in Python easy (I'm a little sad about this fact myself, but I accept that trying to make a language good at everything is counter-productive). Features like the ability to call functions without parentheses, user-defined operators, user-defined syntax in general, are directly intended to make writing DSLs easier, and have been requested and rejected many times in the past.

I am struggling with this myself as well ... but Python is arguably one of the best language to represent ideas. When I wrote "if python is going to support scala like operator constructions" I meant more to ask "If someone out there already had this plan and working on it", rather than advocating it myself too strong (I would really love it though, but not the part that calls a function without parentheses).

I believe Python is OK (I may even go as far as saying "pretty good") for defining simple DSLs, but it's not a core goal of the language, and you have to make your DSL fit with Python's syntax, not the other way round.

Python is great! I am in no way saying anything bad about python only because it is missing one feature. But I do think it can still be better and better. I guess in the past there is just not enough interests from hardware people to use Python to design hardware. We had MyHDL, cocotb, etc. as well as my own Python-Verilog co-simulator ... because SystemVerilog really makes me sick ... for a language that has more than 250+ reserved keywords you can imagine it is even worse than C++ and no two vendor could ever build a compiler for system verilog behaves exactly the same (I don't even know if the spec itself can be self-consistent ... just too many possibilities to go wrong) ... that's why I build the Python-Verilog co-simulator where I can write much more powerful testbenches just in a few lines of Python.

That's not to say that features useful for DSLs aren't ever going to be accepted, but they will need justification independent of their value in DSLs. The matrix @ operator that you quote wasn't accepted because "it reads well in a maths context", the proponents did a *lot* of research to demonstrate that the operator simplified existing code, and existing language features couldn't deliver the benefits. PEP 465 is well worth a read to give an idea of the sorts of arguments needed to successfully introduce a new operator to Python.

Sure and will work on it. I am not sure if the argument can come as strong as in PEP 465, as numpy community (which has almost the entire machine-learning world) is considerably larger than python hardware design community ... but I will try to see if I can convince some of you guys.

On Fri, May 24, 2019 at 11:47 AM Steven D'Aprano <steve@pearwood.info> wrote:

PEP 465 had a concrete purpose for its new operator, and although no built-in object supported the @ operator, we understood why Numpy wanted it. But I do not understand why you want an arrow operator or what you will do with it.

OK, I see your point and will work on a more comprehensive example with some investigations how it can simplify existing libraries outside there.

What should built-in types do with this new dunder?

"Hello world!" <== "aardvark"

23.5 <== 1.25

[1, 2, 3] <== 4

It will generate error: unsupported operand types(s) for: 'int' and 'int'. (or list and int etc.)

Have you considered the __ilshift__ method?

signal <<= 5

I explained already but here again: you will need to have "this_signal <<= (that_signal << 4) + something, and next line you should be able to write this_signal <<= 4 bit, where all arithmetic operation still have to be kept. Otherwise it makes signal doesn't look like a value ... it needs to be looking like a integer but with a different assignment behavior. In hardware design domain a signal is just a value but with its own assignment behavior involving delta-cycles (a virtual time step increase of the value zero ... I know it is a bit confusing to SW developers). So the intention is to keep all arithmetic ops like a int, but only changing how the assignment is done.

Yes. You should study PEP 465. It is one of the best PEPs in Python's history. If you want your proposal to succeed, you should think "How can I write this to be more like PEP 465?"

I am writing just to see if I missed something significantly in this field ... will come back with a comprehensive PEP.

I am completely aware that we can't hook into assignment except with attribute or item assignment. But I still do not understand why you want to hook into assignment.

What is going to be hooked into the assignment is the pending update of the value to the signal, which only happens until a delta cycle is passed, such that you can have a deterministic behavior when using software to mimic actual hardware behaviors.

Perhaps I have missed something when reading your earlier posts, but I don't understand what you will do with this new operator. Perhaps you can show up your intended method?

x <== y

def __arrow__(self, other): # what goes here?

For example: def __larrow__(self, other): self.next_value = other.value And when increasing virtual time (e.g. a delta cycle): for sig in all_signals: sig.current_value = self.next_value

I am not sure if I understand what you mean by "feeding 5 into this signal". But if it means what I think it means, then you can give signal a method:

signal.feed(5)

which is explicit and readable. Or you could use <<= as I mentioned above.

The intention is to make the signal looks like an integer, and yet to change the assignment behavior. As I explained above that all normal arithmetic ops still need to be kept.

You also said:

"[arrow operator] looks like kind of the non-blocking assignment operators in hardware description languages"

What is "non-blocking assignment"?

non-blocking assignment is a terminology in HDL design, where the assignment is not actually happening until a later time. In simple words if the "virtual-time" is not increasing, then the assignment is not happening. To allow all hardware threads seeing the same consistent value at that particular virtual time. I have the prototype working, let me implement a much more elaborated example, and also look into how existing modules like MyHDL could benefit out of it, and give you guys a written PEP.

From this good discussion so far (Thanks everyone for your candid responses!), I understood I have not missed anything significantly, Python community has no intention to make Python supporting operator-on-demand-definition like in scala, so the only possible way forward is (for me): build the "<== and ==>" operator and write a PEP to justify its usefulness. The general intention is to make HDL design in python looks and feels at least as good as in verilog/vhdl, so once I have my library complete I will explain in depth how it looks side by side for Python vs Verilog, with and without the arrow operators.

On Fri, May 24, 2019 at 5:45 PM Terry Reedy <tjreedy@udel.edu> wrote:

What I understand is that you are doing discrete-time hardware simulation and that you need a operator that will schedule future assigments to int-like objects. Have you considered using '@' to do that? int @ int-expression is currently invalid, so defining it will not interfere with other int operations. What am I not understanding?

I am not sure if I understood this. The intention is to e.g. assign a signal with a value in a nature way, e.g. signal <== 5, are you saying to replace <== with @? I guess that should work (signal @ 5), not really intuitive though. I really really would like either a equal sign or something like an arrow ... I know I am a bit picky ...

On Sat, May 25, 2019 at 8:28 PM Terry Reedy <tjreedy@udel.edu> wrote:

On 5/24/2019 4:25 PM, Yanghao Hua wrote:

...

On Fri, May 24, 2019 at 5:45 PM Terry Reedy <tjreedy@udel.edu> wrote:

...

What I understand is that you are doing discrete-time hardware simulation and that you need a operator that will schedule future assigments to int-like objects. Have you considered using '@' to do that? int @ int-expression is currently invalid, so defining it will not interfere with other int operations. What am I not understanding?

I am not sure if I understood this. The intention is to e.g. assign a signal with a value in a nature way, e.g. signal <== 5, are you saying to replace <== with @?

That is what i meant, but reading

...

I really really would like either a equal sign

suggests '@=' as a better alternative. (I am not sure if the implementation would be equally easy or hard.) How does 'signal @= 5' look?

Either is pragmatic in that these exist since a few versions ago, and cannot interfere with existing integer expressions, rather than in the far very hypothetical future.

...

not really intuitive though

'@' means 'at' and you want to make the assignment 'at the next time mark' (or whatever you call it). This is more intuitive to me than seeing '@' as 'matrix-multiply' because 'matrix' contains 'at'.

When we added @, it was known and intended that it could serve other then unknown uses.

@= has all the same issues like <<= or >>=, in that you are basically sacrificing a well known number operation and force it to mean something completely different. In previous examples I have shown that HDLs are basically handling numbers, so <<=, >>=, |=, &=, +=, -=, %=, *=, <<, >> all could be, and should be used directly to signals. I admit this (@=) is a much rarer case, but why do we want to exclude the possibility for a matrix of signals to multiply another matrix of signals and assign the result to another matrix of signals? how does this look like? X @= (X @ Y), where @= means signal assignment, and X @= Y, does it mean signal assignment of Y to X, or does it mean X = X @ Y? This simply causes a lot of confusions. I want a solution that is impossible to cause any confusions. As @ and @= is built in python, any one could be using it in all possible ways that is beyond our imagine ... unless I have a way to make sure user cannot use @= as matrix multiplication (which obviously I cannot), otherwise I am very much de-motivated to use it to mean HDL signal assigns ...

On Sun, May 26, 2019 at 6:05 AM Terry Reedy <tjreedy@udel.edu> wrote:

On 5/25/2019 3:09 PM, Yanghao Hua wrote:

...

@= has all the same issues like <<= or >>=,

No, it does not

...

in that you are basically sacrificing a well known number operation

because @= is not a number operation at all.

Yes you are right. @ is not a number operation, it is number-collection operation. What is preventing the same operation on signal-collections?

...

I admit this (@=) is a much rarer case,

It is a different case.

Really not much different for me as you can use it to operate on matrix (which can be either a matrix of number or matrix of signals).

...

but why do we want to exclude the possibility for a matrix of signals to multiply another matrix of signals and assign the result to another matrix of signals?

We do not. <int subclass instance> @= int would be implemented by the __imatmul__ method of the int subclass. matrix @= matrix is implemented by the __imatmul__ method of the matrix class. This is similar to 1 + 2 and [1] + [2] being implemented by the __add__ methods of int and list respectively.

I really don't understand the argument here. And let's apply the same argument to PEP465 why not matrix multiply override <<= instead? For me not using @= is exactly the same reason for not using <<= and others.

how does

...

this look like? X @= (X @ Y), where @= means signal assignment, and X @= Y, does it mean signal assignment of Y to X, or does it mean X = X @ Y? This simply causes a lot of confusions.

Why don't people more often get confused by a + b? Partly because they use longer-that-one-char names that suggest the class. Partly because they know what a function is doing, perhaps from a name like set_signals. Party because they read the definitions of names. Conventionally in math, scalars values are lower case and matrices are upper case. So x*y and X * Y are not confused.

I think people don't get confused by a + b because a + b does mean a + b and does not mean a * b and it has nothing to do with how you name the operands.

On Fri, May 24, 2019 at 12:29 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

Yanghao Hua wrote:

...

I have explained the problem of use descriptors in previous replies, where you cannot have a local signal, e.g. obj.signal = thing # works, but local_signal = thing # doesn't work.

Maybe you could do something like:

local = Signals() local.signal1 = ... local.signal2 = ...

In structure design ... and especially when you design a hardware that is meant to be automatically converted into verilog or even logic gates, I personally would really want to have a one-to-one relationship of the python-objects vs the actual hardware structures. The granularity is at signal/bit level. This is why I really think giving a special assignment in python which users could override is really helpful, rather than having to have this kind of special case: if you do "self.abc = thing" descriptor mechanism is invoked, but the very next line if you do "abc = thing" ... nothing will happen. This special case can be completely removed and having a much better conceptual consistency if the "<==" assignment operator always behaves the same, doesn't matter if it is "self.abc <== thing" or "abc <== thing".

Another idea if you really want to be able to do `foo = 5` and have it behave the way you want: Create a custom dictionary type to hold locals() (and perhaps globals() if needed). Unless I'm wrong, that dict type can pretty much do whatever you want, including overriding assignment behavior. Then just run the code using exec(), passing the custom hdl_locals(). You could package up a custom python interpreter for hardware programming which simply execs() the python code using this customized assignment behavior provided by hdl_locals(). Such a customized namespace is a very pythonic approach, and if I understand correctly, most regular non hdl python would probably be able to run.

You can do things like this with exec(): class SafeDict(dict): curated_keys = {"a", "b", "c"} def __setitem__(self, k, v): if k not i self.curated_keys: raise Exception(f"{k!r} key not allowed") super().__setitem__(k, v) locals_dict = SafeDict() globals_dict = SafeDict() exec("d=1", locals_dict, globals_dict) # Exception: 'd' key not allowed You can do all sorts of things using that technique, including the way assignment to variables is handled. Beyond that, I declare my competence in this matter to have been hereby exceeded, since I really don't follow or understand the HDL paradigm you are working in. Maybe others might be able to help.

There is probably some existing python API you can hijack to make custom locals() and globals() work everywhere. Perhaps pdb and inspect.stack are good places to start; maybe there’s a PDB API to break on every new stack frame and maybe you can use inspect to do the proper assignment overrides. Python is a very layered language. Sent from my iPhone

On May 25, 2019, at 4:53 AM, Yanghao Hua <yanghao.py@gmail.com> wrote:

...

On Fri, May 24, 2019 at 10:34 PM Ricky Teachey <ricky@teachey.org> wrote:

You can do things like this with exec():

class SafeDict(dict): curated_keys = {"a", "b", "c"} def __setitem__(self, k, v): if k not i self.curated_keys: raise Exception(f"{k!r} key not allowed") super().__setitem__(k, v)

locals_dict = SafeDict() globals_dict = SafeDict()

exec("d=1", locals_dict, globals_dict) # Exception: 'd' key not allowed

You can do all sorts of things using that technique, including the way assignment to variables is handled.

I see your point now. In this case user will need to write HDLs probably in below fashion:

class my_module: def __init__(self, input, output, ...): ... def process(self): exec("signal = 5", locals, globals) # compare this with: signal <== 5

I fully agree this can work, but also it doesn't seem to be any better looking than signal.next = 5. I think there are two important point here, first is to make HDL design feels as natural as possible and as pythonic as possible as well as leaving very little room for mistakes, second is to make HDL design at least as easy and as intuitive as in traditional HDLs. And when I see this can actually be achieved easily with less than 100 lines of CPython code changes I am tempted to give it a try (I am so in love with python ...).

The other thing I was thinking about is PEP572 assignment expression (a := b), if it could be modified to allow user override (e.g. via __assign__()), and allow it to be used without bracket like "y := f(x)" if __assign__ is present in y. Then this is even aligned with Chisel's assignment operation and I'd be completely happy with it. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 24 May 2019, at 22:11, Yanghao Hua <yanghao.py@gmail.com> wrote:

...

On Fri, May 24, 2019 at 3:27 PM Ricky Teachey <ricky@teachey.org> wrote: This seems like a hurdle you're going to have trouble passing... especially given that all the functionality that is required can be provided using existing descriptor behavior. You will need to pretty concretely demonstrate why the special handling of signals in assignment (no matter which operator is the operator of choice) is something the language at large really needs, and why descriptors aren't sufficient.

Just a quick example, suppose you have a class A and class B representing two circuit blocks, where in class C you want instantiate A() and B() and connecting them together. Please do let me know if you could have a more reasonable way of representation to make it working. class A: def __init__(self, output): self.output = output def process(self): self.output = 5 # !!! this does not work for descriptors passed in # self.c_self.signal = 5 # this might work, but what the heck really?!

class C: signal = Signal() def __init__(self): a = A(output=self.signal) # a = A(output=self) # it is only possible for signal to work if you pass C's self into a ... b = B(input=self.signal) # !!! This does not work !!!

Instead, a much more natural way of doing it is: class A: def __init__(self, output): self.output = output def process(self): self.output <== 5 # this always works!

class C: def __init__(self): signal = Signal() a = A(output=signal) b = B(input=signal) # this feels much better, isn't it?

process() in A could look like: self.send(output=5) To me that looks OK, and scales nicely with multiple outputs: self.send(a=5, b=3) send() is implemented simply as def send(self, **kwargs): for k, v in kwargs.items(): signal = self.signals[k] signal.c_self.output = v Or something. I'm not sure about the details since I don't understand the example you give. It's pretty abstract and vague. Just a simple example of how to use A with a print() and the expected output of said print would help? / Anders

I don't really understand HDL/Verilog, but I've worked with people who do. In fact, I even wrote a pre-processor that transformed the same DSL to Python, C++, and Verilog. In my mind, the HDL use case is FAR too narrow and specialized to warrant a new arrow operator, let an entirely new parser and semantics around arbitrary operators. There are several existing dunders that could plausibly be repurposed already (<<, <<=, <=, etc). Those might look sightly different than the verilog operators, but that's a very small price. In fact, just using attributes and assignment is an incredibly low bar too, and allows whatever overriding you wish. I just don't buy the idea that such a DSL can only be useful if it spells 'abc <== message' and useless if it spelled the same thing as 'abc <<= message'. On Fri, May 24, 2019, 9:06 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

On Fri, May 24, 2019 at 12:29 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

...

Yanghao Hua wrote:

...

I have explained the problem of use descriptors in previous replies, where you cannot have a local signal, e.g. obj.signal = thing # works, but local_signal = thing # doesn't work.

Maybe you could do something like:

local = Signals() local.signal1 = ... local.signal2 = ...

In structure design ... and especially when you design a hardware that is meant to be automatically converted into verilog or even logic gates, I personally would really want to have a one-to-one relationship of the python-objects vs the actual hardware structures. The granularity is at signal/bit level. This is why I really think giving a special assignment in python which users could override is really helpful, rather than having to have this kind of special case: if you do "self.abc = thing" descriptor mechanism is invoked, but the very next line if you do "abc = thing" ... nothing will happen. This special case can be completely removed and having a much better conceptual consistency if the "<==" assignment operator always behaves the same, doesn't matter if it is "self.abc <== thing" or "abc <== thing". _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 26 May 2019, at 08:07, Yanghao Hua <yanghao.py@gmail.com> wrote:

On Sun, May 26, 2019 at 12:04 AM David Mertz <mertz@gnosis.cx> wrote:

...

I don't really understand HDL/Verilog, but I've worked with people who do. In fact, I even wrote a pre-processor that transformed the same DSL to Python, C++, and Verilog.

In my mind, the HDL use case is FAR too narrow and specialized to warrant a new arrow operator, let an entirely new parser and semantics around arbitrary operators. There are several existing dunders that could plausibly be repurposed already (<<, <<=, <=, etc). Those might look sightly different than the verilog operators, but that's a very small price. In fact, just using attributes and assignment is an incredibly low bar too, and allows whatever overriding you wish.

Well, depends on how we define narrow ... you are writing probably this email on a HDL designed machine ... and the entire world is powered by HDL designed silicons. that is not small for me at all.

I think you are confusing the number of people that use HDL with the amount of product created. Also I was under the impression that HDL tools exist already that are considered usable and yet do not need python. What is the problem that you aim to solve with a python HDL tool? If the syntax of the HDL is so important I do not understand why you do not write a parser for the HDL and build the run-time model in python. Then run the model - no new syntax required. For any non-trivia hardware I'm finding it hard to believe that python will run fast enough to be useful. What is it that I'm missing?

...

I just don't buy the idea that such a DSL can only be useful if it spells 'abc <== message' and useless if it spelled the same thing as 'abc <<= message'.

So you don't find this is confusing?

signal <<= 5 # does it mean left shifting signal or assigning 5 to signal??

I really do think it is confusing.

Isn't that a matter of familiarity with HDL? Many people replying are not familiar with HDL. Barry

On 26 May 2019, at 17:04, Yanghao Hua <yanghao.py@gmail.com> wrote:

On Sun, May 26, 2019 at 11:27 AM Barry Scott <barry@barrys-emacs.org> wrote:

...

I think you are confusing the number of people that use HDL with the amount of product created.

I don't see how I did that but if you intercepted that way I must have done that somehow.

You said this: "Well, depends on how we define narrow ... you are writing probably this email on a HDL designed machine ... and the entire world is powered by HDL designed silicons. that is not small for me at all." Which I take to mean that because there are billions of chips in the world there are billions of python users. And the change you want is justified by the billions of python users.

...

Also I was under the impression that HDL tools exist already that are considered usable and yet do not need python.

What is the problem that you aim to solve with a python HDL tool?

I think the question should be which part of existing HDLs does not need to be fixed? The answer would be easier: the assignment syntax is pretty elegant. I would recommend to take a look at Chisel, all the motivations for creating Chisel is pretty much the same reason I would create a python equivalent and I had a prototype shows in some area it could even be better.

There is a reason that there are 1,000s of computer languages in the world. Not all computer languages are able to solve all problems. If Chisel or Scala is closer to what you want why are you not using them for your HDL tools?

And what is the problem python solves that C doesn't solve? And what is the problem C solves that assembly doesn't solve? One common answer to all of them would be: fewer chars for bigger ideas.

What has that got to do with justifying a change to the python language?

...

If the syntax of the HDL is so important I do not understand why you do not write a parser for the HDL and build the run-time model in python. Then run the model - no new syntax required.

Many many people and company did it already ... I am just exploring a different possibility.

And it seems that you have hit a major problem with the HDL syntax not mapping to python. Maybe python is not the right choice?

...

For any non-trivia hardware I'm finding it hard to believe that python will run fast enough to be useful. What is it that I'm missing?

We are all Python users and we start to worry about running fast? really? ;-) I thought we all understood development time matters (if not even more ...)

Packages like numpy are not written in python for a reason. A pure python numpy would be very slow. So yes we worry about run time speed. Barry

On Sun, May 26, 2019 at 8:00 PM David Mertz <mertz@gnosis.cx> wrote:

On Sun, May 26, 2019, 1:12 PM Barry Scott <barry@barrys-emacs.org> wrote:

...

You said this: "Well, depends on how we define narrow ... you are writing probablythis email on a HDL designed machine ... and the entire world is powered by HDL designed silicons. that is not small for me at all."

Which I take to mean that because there are billions of chips in the world there are billions of python users. And the change you want is justified by the billions of python users.

I think the analogy is more like saying there are billions of Verilog users. Chips designed using this software are probably more numerous than are human beings, and nearly all humans use one or more such chips. This seems like a poor line of reasoning though. There are really only maybe tens of thousands of people in the world who use Verilog, and that is a nice target.

Thank you. It is definitely more than tens of thousands. I count only the 20 big names and it is already at least tens of thousands. All in all it should be a few millions. We should argue really from technical point of view, in the spirit of python, if this is worth or not rather than saying HDL is just too small a problem to address.

Pursuing the same indirect reasoning, we could also claim there are billions of users of PSS/E, something I had not heard of until a web search a few minutes ago. Apparently this is software used in design of electrical distribution systems, required to make all of those ICs operate at all. Or software used for building the construction equipment needed to construct the power lines and factories where chips are made. Or...

You know, Python and C are very close friends and operates almost like natively together and you have things like Cython to ease the creation of C-Python interface. If we look at the problem of building any complicated piece of hardware, it almost certainly done in this way: (1) C algo model -> (2) Virtual Prototype in SystemC (qemu-like software model but with proper partitioning of sw/hw, hw is still kind of a C model) -> (3) Verilog RTL model (Yes, RTL is actually just a model too, which is easier to be translated to actual hardware implementation). At each of the levels, it is a end-to-end complete system, and yet in each level, there are a lot of things duplicated, re-developed and eventually extremely difficult to have consistency. The big picture I want to achieve is to replace those (1)(2)(3) with Python + C -> Python + C -> Python + C. Where the interface between each layer can be easily unified through adapters and 99% of the knowledge only need to be developed once (if it is developed in C there will be a python interface).

In any case, as I said, modifying the whole Python language to allow use of exactly the same symbols as Verilog users, seems foolish to me.

Agree, that would be foolish. I would just ask for any way that overloads assign but not messing up existing operators (think of HDL signals as integers).

That said, '<=' is already perfectly well available for non-blocking assignment. '=' cannot be overriden, but it occurs to me that a custom class could perfectly well use '==' for blocking assignment just by customizing a class. Yes, that would require 'foo.eq(bar)' for the old meaning of equality (or something similar), but it seems like assignment is more common than equality checks in Verilog.

Not really, HDLs uses <= as less or equal than, as well as the equality check VERY VERY often. The sad thing is when I went through the entire python supported operators they can all be used with signals ... including the matrix operator @/@=.

On Mon, May 27, 2019 at 5:25 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

On Sun, May 26, 2019 at 7:11 PM Barry Scott <barry@barrys-emacs.org> wrote:

...

You said this: "Well, depends on how we define narrow ... you are writing probably this email on a HDL designed machine ... and the entire world is powered by HDL designed silicons. that is not small for me at all."

Which I take to mean that because there are billions of chips in the world there are billions of python users. And the change you want is justified by the billions of python users.

No offense but it is a bit too optimistic too think there are billions of python users (who actually write python code). And it is a bit too pessimistic to think there are billions of chips in the world ... the chips I personally worked on most likely exceeded the billion mark. So put this in perspective there are hundreds of billions of chips being produced, this is the problem space of *secondary* impact. the *primary* impact could be a few million (my estimation) HDL developers, and a dozen million HDL verification engineers. It is definitely smaller than the entire python developer community because probably all of the hardware/silicon engineers write a few lines of python. I just want to say this is *NOT* a small problem at all.

SiFive can now develop RISC-V CPUs which is faster and more efficient than the respective ARM versions is probably the best testimony that a better HDL matters.

Irrelevant though. I've ridden in a car - does that make me a petrochemical engineer?

...

Packages like numpy are not written in python for a reason. A pure python numpy would be very slow. So yes we worry about run time speed.

numpy is not Python! numpy is C!

(Fortran, for what it's worth)

Python does not have numpy! I am a bit surprised that we python community is even denying python is slow? You bench mark it in all possible ways python is still slow, if not the slowest. That's a known fact and we always do python prototyping and optimize the critical path/parts into C. Python is not build for speed. (don't bring in pypy for now please ... ) What we love about python is its development speed. The same applies to the world of hardware design and verification.

Actually, I just code in Python. And when I ask if a dictionary contains a particular key, I don't care that this is implemented in C; all I care about is whether the dictionary contains that key. Am I writing C code? Nope. I'm writing Python code. And PyPy is actually quite relevant, because it runs normal Python code. You don't have to "optimize ... into C". You just write Python, and then run it using a Python interpreter. This discussion may need to move to python-list rather than -ideas, as this isn't really progressing towards any sort of language improvement. ChrisA

On Mon, May 27, 2019 at 6:05 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

Doesn't matter how it ends up, I urge the python community do give it a second thought. (Don't you guys think it is odd that Python can overrides almost every operation but not for assignment ... is assignment really worthy being a special case?!)

Yes. It IS a special case, because assignment is not handled by the object being assigned to. When you assign to an attribute or subscript of an object, the *parent* object determines how the assignment is done, not the one being replaced. Look at languages where the being-replaced object gets to redefine assignment (C++ and PHP come to mind, and there may be others). MANY MANY parts of your code become harder to comprehend. It's not the only thing special enough to be non-overridable. In Python, you have several fundamentals that are absolutely guaranteed: a = b a is b b if a else c a or b # a and b There is no way that the values of a, b, or c can change the meanings of these expressions. (The truthiness of 'a' will define which of two options is chosen, but you cannot redefine the operator itself.) This is a Good Thing. ChrisA

On 2019-05-26 21:34, Yanghao Hua wrote:

On Sun, May 26, 2019 at 10:25 PM Chris Angelico <rosuav@gmail.com> wrote:

...

On Mon, May 27, 2019 at 6:05 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

...

Doesn't matter how it ends up, I urge the python community do give it a second thought. (Don't you guys think it is odd that Python can overrides almost every operation but not for assignment ... is assignment really worthy being a special case?!)

Yes. It IS a special case, because assignment is not handled by the object being assigned to. When you assign to an attribute or subscript of an object, the *parent* object determines how the assignment is done, not the one being replaced. Look at languages where the being-replaced object gets to redefine assignment (C++ and PHP come to mind, and there may be others). MANY MANY parts of your code become harder to comprehend. It's not the only thing special enough to be non-overridable. In Python, you have several fundamentals that are absolutely guaranteed:

a = b a is b b if a else c a or b # a and b

There is no way that the values of a, b, or c can change the meanings of these expressions. (The truthiness of 'a' will define which of two options is chosen, but you cannot redefine the operator itself.)

This is a Good Thing.

Absolutely right and good. But if you just had := or <== mean "assignment behavior is handover to the object if __assign__ is defined", then you can remove the whole descriptors thing (or rather, it can do all a descriptor can and better, and actually := or <== is saying, hey look, I am a descriptor ... ) . ;-)

I am not saying "=" should be overloaded, I am saying there should be an assignment that can be overloaded, which completes the picture in python.

It seems to me that this is a little like "variables" in tkinter (IntVar, StringVar, etc.). You create and then set and get the contents/value, but in tkinter there's always a clear distinction between the variable itself and its value. For example, you don't say "var + 1", but "var.get() + 1". However, in the examples that you've given, the variables on the RHS are being used in calculations in the same way that the value would be used, e.g. "var + 1" (basically "var.get() + 1"), which means that those operations need to be defined in the classes, but one operation that cannot be defined as an operator is setting the value. It's not possible to write "var.set(var.get() + 1)" as, say, "var <== var + 1". I'm not sure what I think about that, but given that you're willing to define a whole load of other operators... Hmm... Not sure.

On 26/05/2019 17:04, Yanghao Hua wrote:

On Sun, May 26, 2019 at 11:27 AM Barry Scott <barry@barrys-emacs.org> wrote:

...

I think you are confusing the number of people that use HDL with the amount of product created.

I don't see how I did that but if you intercepted that way I must have done that somehow.

...

Also I was under the impression that HDL tools exist already that are considered usable and yet do not need python.

What is the problem that you aim to solve with a python HDL tool?

I think the question should be which part of existing HDLs does not need to be fixed? The answer would be easier: the assignment syntax is pretty elegant. I would recommend to take a look at Chisel, all the motivations for creating Chisel is pretty much the same reason I would create a python equivalent and I had a prototype shows in some area it could even be better.

You do realise that repeatedly avoiding the question "what problem do you think you are solving?" does not convincingly make the case that there is a problem to be solved, don't you? -- Rhodri James *-* Kynesim Ltd

On Wed, May 29, 2019 at 6:30 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

...

You do realise that repeatedly avoiding the question "what problem do you think you are solving?" does not convincingly make the case that there is a problem to be solved, don't you?

It might be that I am not good enough yet to present it in a way for some of you to better comprehend it. But I do see positive feedbacks too. I plan to stop this discussion here and make a toe-to-toe comparison how it looks like without assignment overloading, how it looks like with assignment overloading, and how it compares with other languages like verilog and Scala/Chisel, and in the meantime to look into some corner cases to make sure the proposed solution can cover all situations.

To repeat what the problem do I think I am solving? A variable, that behaves like an integer (e.g. all normal integer ops should just work), but has a different assignment behavior, such that it can be used to develop equally good hardware descriptions.

So what you want is for this code to behave very very weirdly: a = thing() a = other_thing() a = another_thing() Because if one of the things redefines assignment, the subsequent ones are going to do something different. I recommend sticking with something that's deliberately and consciously namespaced. Descriptors give you a convenient way to do this with dot notation, and a custom object with a __setitem__ method lets you use bracket notation. stuff = thing_collection() stuff.a = other_thing() stuff.a = another_thing() This is much clearer, because you can see a distinct difference between initializing the thing and making the thing do stuff. Alternatively, using a different operator works: a = thing() a += other_thing() a += another_thing() because augmented assignment IS handled by the target object. (I don't recommend abusing <= for this, as people and linters will get very surprised by that. But that would also work.) Redefining assignment is a minefield in the languages that support it. Please can you seek a better way to do this? I'm done posting now. If you're not listening, I'm not going to keep talking. ChrisA

On 5/28/2019 4:29 PM, Yanghao Hua wrote:

To repeat what the problem do I think I am solving? A variable, that behaves like an integer (e.g. all normal integer ops should just work), but has a different assignment behavior, such that it can be used to develop equally good hardware descriptions.

This is the part that you're not explaining: what does "a different assignment behavior" mean? We all understand what Python means by assignment (name binding), but we don't understand what you would like it to be instead. It seems that you want these two statements to work differently: x = something() # bind a name for the first time # ("create a variable", if you will) x = 4 # do something different when x already exists # and is of some special type Is that true? What is the "something different"? I've got to be honest with you: I don't see Python changing in this regard. Eric

On Tue, May 28, 2019 at 10:46 PM Ricky Teachey <ricky@teachey.org> wrote:

I don't want to pile on, but: you have also not given any actual reason why overloading the = operator using descriptors:

stuff.a = 8

...is not sufficient, other than a general "well I don't like that". Come on, guy. If you can't give a real actual reason why that won't work for HDL programming, you can be sure as heck nobody is ever going to listen to your idea of changing the way the language behaves.

Have you even attempted to write a descriptor that handles the assignment of signals the way you want and tried it out for a bit?

Yes I have, and none of them working in a way that can easily reflect the hierarchical nature of hardware design. Suppose you have class A and class B, and a third class C want to use class A and B, the very natural way that all hardware engineer would construct this is like below: class C: def __init__(self, in, out, ...): # suppose A has one input and two output, B has two input and one output # so A and B can be chained together sig0 = signal() sig1 = signal() a = A(in, sig0, sig1) b = B(sig0, sig1, out) Descriptor does not work as a local variable, it does not even work if you instantiate it in the __init__() function, it has to be declared as a class attribute for it to work. And when you passing it around, it will not work, unless you pass class C's self around ... which is really not pleasant to read (I had this example already in previous postings). If anyone could teach me how to do hardware hierarchical design as elegant as in verilog or Chisel, I'd be happy to use it. What I see is, myhdl is using signal.next = something, cocotb is only for verification so ... not really relevant. In short, I haven't found any evidence out there that this can be done naturally (e.g. as in verilog or Chisel). You can argue that why do I even want to define verilog/chisel's way is the natural way, well, this is I believe the common understanding in hardware design community, and this is the reason why new HDL like Chisel choose Scala to implement exactly the same thing. But anyone who could propose a even simpler syntax that can describe the same thing with less chars I'd definitely be happy to take it.

On Tue, May 28, 2019 at 05:45:54PM -0400, Ricky Teachey wrote:

As things stand right now, it seems like you don't understand what we are saying about using descriptors to accomplish this. However it is certainly possible I simultaneously don't understand what you are really trying to do in the code you wrote (I definitely don't claim to).

Descriptors are, I think, a total red herring. Yanghao Hua wants to customise the behaviour of assignment. I believe that he wants to emulate the behaviour of some hardware description languages, where the equals sign = doesn't mean assignment (if I have understood correctly, which I may not have), but Something Else. Yanghao Hua can write a method to perform that Something Else, but he wants to use the standard variable assignment syntax: x = expression but there is no hook to customise the behaviour, variable assignment is built-into the language. But *attribute assignment* is different: obj.x = expression may call either the descriptor protocol, or simply call type(obj).__setattr__, which gives us the opportunity to hook into attribute assignment. Likewise for item assignment: obj[x] = expression which calls type(obj).__setitem__. So Yanghao Hua has lots of options: 1. Choose a different operator. The obvious choice is << if his object is not already using it. 2. Use a method: x.set(expression) 3. Use a pre-processor, like Coconut or "Like, Python!". He may need to write his own pre-processor, but that may not be very difficult if all it does is convert "x = ..." into a method call. 4. Write an interpreter that does what he likes: code = """ x = expression """ result = interpret(code) 5. It may be possible to play games with the interpreter by using metaclasses, replacing modules with custom objects, etc. But I doubt that will change the behaviour of assignment inside the module itself. 6. Check out MacroPy and see whether its macros will help. 7. Use a custom interpreter. There is at least one superset of Python in common use: Cython. 8. The most obvious solution is to just use an instance and attribute: obj.x = expression will work fine, but he just doesn't like the look of it. Aesthetics are an important part of language design, but it is only one factor out of many. All languages make compromises, and sometimes compromising on aesthetics is one of them. In Python, the compromise is that we can't hook into plain old assignment; but what we gain is certainty (we always know what x=... does) and performance. On the flip side, we *can* customise attribute assignment obj.x=... but what we lose is certainty and performance: obj.x = ... is slower, because it has to look up the __setattr__ method, or call the descriptor protocol, and then call custom code; and we can't know what it will do. ``obj.x = 1`` can call arbitrary code, so it could do literally anything. Extending that ability to hook assignment to plain old variables is unlikely. It would need a PEP. Adding a new operator like <== is less unlikely, but it too would need a PEP, and the big question that everyone will ask is "why not just use << instead?" The bottom line is, Python is not Lisp or Forth, where the very syntax itself can be customised. Regular assignment counts as syntax: it's unlikely to change, without a very good reason. -- Steven

Steven D'Aprano wrote:

Yanghao Hua wants to customise the behaviour of assignment. I believe that he wants to emulate the behaviour of some hardware description languages, where the equals sign = doesn't mean assignment (if I have understood correctly, which I may not have), but Something Else.

Maybe I can help a bit here. I don't have a deep knowledge of HDLs. but from what I gather, they typically have two *different* assignment-like operations. One of them is like assignment in a functional language, where you're defining a signal as a function of other existing signals. This corresponds to combinatorial logic in hardware. Python's existing assigment is fine for this, I think. The other represents updating a signal when some event occurs, such as a clock transition. In hardware terms, it corresponds to changing the state of a flip-flop. The OP wants a new operator and associated dunder method to represent this operation. He doesn't want to override normal assignment. -- Greg

On 5/30/19 4:09 AM, Yanghao Hua wrote:

On Thu, May 30, 2019 at 12:50 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

...

Steven D'Aprano wrote:

...

Yanghao Hua wants to customise the behaviour of assignment. I believe that he wants to emulate the behaviour of some hardware description languages, where the equals sign = doesn't mean assignment (if I have understood correctly, which I may not have), but Something Else. Maybe I can help a bit here. I don't have a deep knowledge of HDLs. but from what I gather, they typically have two *different* assignment-like operations. One of them is like assignment in a functional language, where you're defining a signal as a function of other existing signals. This corresponds to combinatorial logic in hardware. Python's existing assigment is fine for this, I think.

The other represents updating a signal when some event occurs, such as a clock transition. In hardware terms, it corresponds to changing the state of a flip-flop. The OP wants a new operator and associated dunder method to represent this operation. He doesn't want to override normal assignment. That's basically right. And three small addition/correction: 1. Actually combinational logics can be modeled with both blocking and non-blocking logic. So one of them is enough already. 2. python's assignment is having very different semantics, so even for the verilog blocking assignment, which need to make the current "thread/process" to wait until the assignment finishes, is still not possible to model with python's assignment operator. 3. the signal change is really modeled on gate level, which are AND/OR/XOR/MUX gates. There are optimized flip-flop entities too but basically you can also build FFs from the primary gates. And the change of the signal, is actually just change the input of the FFs, not really the FF states itself. FF state change is modeled and inferred with a sensitivity list (a list of signals which triggers a thread/process to run) that has a clock.

As someone who does hardware design with HDLs, I would say that to think that it would be easy to recreate the abilities of them in a 'conventional' programming language doesn't really understand at least one of the languages, as there is a SIGNIFICANT difference in there fundamental mode of operation. The one key thing here is that with non-blocking assignment, the order they are written is totally not important. I can write: x<= y; y <= z; or y <= z; x <= y; ( <= is the non-blocking assignment operator in verilog), and in both instances get exactly the same result, namely that x gets the value of y BEFORE either of those statements occurred. Basically, all the non-blocking assignments happen in parallel through the whole design, and when that assignment happens is controlled by the context of where that assignment is written (it might be conditioned on a clock edge, of by an if statement to only happen when some other combination of signals occur) There ARE programs that will take a HDL program and convert it into a conventional programming language, and other programs to take a design written in a conventional programming language and convert it to HDL, and these mostly work as long as you started off remembering their restrictions when you started writing the original program. To expect that you could get anything similar by just defining a few operators to deal with thing in a special manner seems unlikely. I think that I personally would have a hard time understanding a program written where many of the assignments behave sequentially like is normal in a conventional programming language but many other behave concurrently as in HDL. I would think it would be much clearer if you wanted to simulate the action of hardware, to first have 'definition' stage where you defined the interconnection of the signals within a system (and here you would naturally be dealing with methods and attributes of some object that represents the hardware system), and then after defining that, using some method to 'run' that hardware. Thus having the above be written as hardware.x = hardware.y hardware.y = hardware.x and then later: hardware,run() where the attribute assignment recorded the interconnections and the run did them. I think this is all possible currently in Python where hardware is an object of an appropriatly written class. -- Richard Damon

On Tue, May 28, 2019 at 10:40 PM Eric V. Smith <eric@trueblade.com> wrote:

On 5/28/2019 4:29 PM, Yanghao Hua wrote:

...

To repeat what the problem do I think I am solving? A variable, that behaves like an integer (e.g. all normal integer ops should just work), but has a different assignment behavior, such that it can be used to develop equally good hardware descriptions.

This is the part that you're not explaining: what does "a different assignment behavior" mean? We all understand what Python means by assignment (name binding), but we don't understand what you would like it to be instead.

a different assignment behavior in HDL is your assignment does not take effect until a delta cycle of zero virtual time has passed. (did you really looked at the previous postings? :)

It seems that you want these two statements to work differently:

x = something() # bind a name for the first time # ("create a variable", if you will) x = 4 # do something different when x already exists # and is of some special type

Is that true? What is the "something different"?

x = 4 should be something like x <== 4 or x := 4 (the latter has been taken by the assignment expressions though ...). Such that variable initialization (=) and utilization (<==) can be differentiated.

I've got to be honest with you: I don't see Python changing in this regard.

Sad to see that's the case from you when it seems you barely even read all of the postings in this thread.

On Wed, May 29, 2019 at 3:15 AM Eric V. Smith <eric@trueblade.com> wrote:

Yes, I’ve read every one of the emails in this thread, many of them multiple times.

Python does not know what “a delta cycle of zero virtual time has passed” means, so there’s no way of implementing this feature. If indeed you’re serious about this feature, you would do yourself a favor by explaining it in terms of Python, not in terms of HDL.

Python does not need to know this ... just hand it over to end user who knows how to implement such a thing. Python need to provide the mechanism.

...

...

It seems that you want these two statements to work differently:

x = something() # bind a name for the first time # ("create a variable", if you will) x = 4 # do something different when x already exists # and is of some special type

Is that true? What is the "something different"?

x = 4 should be something like x <== 4 or x := 4 (the latter has been taken by the assignment expressions though ...). Such that variable initialization (=) and utilization (<==) can be differentiated.

I take this as an answer of “yes” to my question of the two assignments being different. If that’s so, then I’m positive this feature will not be implemented.

I respect your view however I cannot agree with it.

...

...

I've got to be honest with you: I don't see Python changing in this regard.

Sad to see that's the case from you when it seems you barely even read all of the postings in this thread.

It’s true that I’ve tried and failed to understand what behavior you’re suggesting be added to Python. Good luck going forward, but I’m dropping t out of the conversation.

Thanks for trying and all the feedbacks.

On Fri, May 31, 2019 at 3:48 PM Rhodri James <rhodri@kynesim.co.uk> wrote:

It really doesn't. If the end user is going to implement the logic of this anyway, implementing signal linkage as a method call or class all of its own is not a significant extra burden.

There are very constructive discussions in terms how this could be handled and which turns out not really elegant. Your justification could be used to reject features like descriptors, decorators, meta classes, and you can use that reasoning even just to say "let's all go java". This is not the pythonic for me at all. the burden, is not only on the savings of a few chars, it is on how easy/difficult it can be interpreted and understood.

I'm pretty much done with this conversation too. You have repeatedly been asked what problem you are trying to solve and repeatedly respond by restating your solution, which appears to be to impose HDL sematics onto a non-HDL language. That's never going to be a good idea.

Please let's be constructive and be specific, this kind of conclusion without any reasoning behind it not only makes it weak, this is not the healthy type of discussion at all.

On Tue, Jun 4, 2019 at 10:11 AM Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Yanghao Hua writes:

...

On Fri, May 31, 2019 at 3:48 PM Rhodri James <rhodri@kynesim.co.uk> wrote:

...

...

It really doesn't. If the end user is going to implement the logic of this anyway, implementing signal linkage as a method call or class all of its own is not a significant extra burden.

...

There are very constructive discussions in terms how this could be handled and which turns out not really elegant. Your justification could be used to reject features like descriptors, decorators, meta classes,

And has been so used. The question is always "does the use case justify increasing complexity for a couple hundred maintainers and a few hundred million readers?"

That's a valid question. and let's address the items one by one. Complexity & hundreds of maintainers. It is nothing complex and it is almost a direct copy of the implementation of @= operator, all logic behind <== are the same of @=. With some refactoring @= and <== implementation might even be able to share 99% of the code, so I think it is not complex at all, and there is not much additional overhead for maintenance. hundred million readers. I can only guess you mean python developers here? Just like @=, <== is not enforced on anyone, but can be used by anyone who wants to. As it is today I can choose to give @= a complete different meaning if I chose to, other uses may chose (and most importantly, being able to) give <== the meaning they see fit when they want to preserve all number operators and matrix operators. It is merely a new possibility for end user with very limited effort. I am glad that this question comes, which means at least the python community is no longer denying there is a problem (a small one by the way), and it can be improved in terms of allowing end user to write more readable code in this specific domain.

As far as I can tell, here we have (a) *one* person *strongly* in favor of adding just one new operator token, who justifies it based on *personal* preference over style of expression for *one* specific DSL, and (b) a few supporting a generic facility for adding new operators.

The former (a) is obviously insufficient, considering past practice in Python. One person can probably use MacroPy or a preprocessor. If the DSL becomes popular, as with "@", first applied for matrix multiplication, it might become justification for a new operator token, but the popularity has to come before more than a tiny fraction of committers (if any) will sign on. The other possibility would be to present a complete implementation of a toy language using existing Python facilities, and then show the alternative with the new operator. Preferably at the same time showing how the preprocessor/ MacroPy strategy falls short.

Will do.

The latter (b) has been discussed and rejected a number of times, on the grounds that unfamiliar symbols are *harder* to read than functions, and that Python is not in the DSL-producing industry. Some features are clearly more descriptive than procedural in nature (eg, comprehensions), but they are justified on the ground that they increase readability when expressing algorithms.

Understood.

...

Please let's be constructive and be specific, this kind of conclusion without any reasoning behind it

In Python practice, the conclusion of "reject" needs no reasoning, unfortunately. Python adds new features in response to a demonstrated need that overcomes a general conservatism, a conservatism that extends far past backward compatibility to a certain amount of forward compatibility as well.

So you should take your own advice. We have seen you express your *preference* for a new operator numerous times, but there is no *visible* logic behind it. Show us the best you can do with actual working hardware description classes (say a NAND gate and the composition into a half-adder, or perhaps you need more to demonstrate the relevant difficulties -- I know little about hardware, quite a bit about how the Python enhancement process works). Then show us the syntax you want to use to express the same thing.

I will come up with a complete working implementation of HDL-in-Python with <== to show the differences, how it enhances readability, and how it compares without it.

It's possible that people with wizard-level Python skills can find a way around the difficulties you describe with descriptors and other features that have been suggested in this thread. If not, your case for a new operator token would be much stronger (though still insufficient IMO -- fortunately for you, few committers listen to me about language features ;-).

I'd like to take any recommendations that can achieve the same logical integrity and readability.

Without such a test, or a few tens of thousands of hardware designers lobbying for a decade or so (which is what happened with the numerical community, although this time around I bet Python will be substantially more forward-leaning, say 5 years and 5000 hardware designers), it's your taste against Python's conservatism. I'll take long odds that conservatism wins.

Thanks for the advice, I will (I was) working on it and will show you the ideas behind it. I knew it is going to be difficult and probably I have failed to use some cool features and I think a completely working system will help to demonstrate the ideas better. Stay tuned.

On Fri, May 31, 2019 at 02:48:24PM +0100, Rhodri James wrote:

On 29/05/2019 08:31, Yanghao Hua wrote:

...

Python does not need to know this ... just hand it over to end user who knows how to implement such a thing. Python need to provide the mechanism.

It really doesn't. If the end user is going to implement the logic of this anyway, implementing signal linkage as a method call or class all of its own is not a significant extra burden.

That's not really fair: syntax matters, and for people working in a domain where certain syntax is expected, asking them to use something different is a significant cognitive burden. We don't ask people doing arithmetic to write code like this: x.mul(y.sub(1)) # a * (y - 1) There have been at least two times that Python has added syntax to the language to allow a third-party library to write more-idiomatic code in their domain. Both extended slicing seq[a:b:c] and the matrix- multiplication operator were added for numpy. So *in principle* we certainly could add a new arrow operator for Yanghao Hua so his libraries and code will be more expressive and idiomatic in his domain. But *in practice*, the hard truth is this: - Yanghao Hua is one developer interested in HDL in Python; - numpy was, and still is, one of the most important "killer apps" responsible for Python's success. Adding syntax for the benefit of numpy helps millions of users; adding syntax for HDL-like helps... how many people? Twenty? A hundred? Millions of users will have to learn the syntax. Unless they get some benefit, why consider this? (But having said that... I reckon that if we had left and right arrow operators, <== and ==>, I have some DSLs where they would work for me too. Maybe.)

I'm pretty much done with this conversation too. You have repeatedly been asked what problem you are trying to solve and repeatedly respond by restating your solution, which appears to be to impose HDL sematics onto a non-HDL language. That's never going to be a good idea.

That's totally unfair to Yanghao Hua on two levels. (1) He has never asked for the interpreter to support the semantics he wants. He's not asking for Python to understand and implement HDL semantics in the language: he can do that himself, in the class. (2) He has explained the problem he is trying to solve: he wants to write a DSL using syntax which doesn't look like crap from the perspective of people in that domain. He just wants an operator that looks kinda like assignment that calls a dunder. He can't use assignment because the semantics are baked into the language (and there's no chance of that changing). He can't use existing operators because they're already in use. This is not an unreasonable request. Python has bowed to similar requests at least twice before, and honestly, it's not like a <== operator would be weirder than other operators in use in mainstream languages. We're not talking APL or J here. On the other hand... its not clear that, as small as this request is, the language would be better. There's lots of other domains where Python syntax is sub-optimal: - We don't have good syntax for writing XML, or calling SQL. We just have to use strings. - No nice syntax for writing Prolog-style deductive code; - or concatenative DSLs; - or natural language DSLs like Hypertalk or Inform. Python can't be all things to all people. -- Steven

Correct me if I am wrong: Yanghao would like an elegant way to build graphs. Simply using << to declare the connections in the graph [1] is not an option because << is already needed for legitimate left-shift operation.  The problem is not assignment; rather, Yanghao's HDL requires more operators than Python has in total (or at least the remaining operators look terrible). Here is a half baked idea: Instead of solving the problem of operator-needed-for-use-case-X, can we solve the bigger problem of adding user-defined operators? "User-defined operators" are a limited set of operators, with no preconceived definition, and can be defined by writing the appropriate dunder method.  User-defined operators match /[!@$%^-+=&*:<>?/|]{2,4}/ and necessarily exclude all the existing python-defined operator combinations.  Maybe 50K operators is easier to add to Python than a few pre-defined operators? Example:

class A:      def assign(self, other):          # whatever this means

To add a user defined operator to a class, you add it as a method, exactly the name of the operator:

setattr(A, "<==", A.assign)

I admit this is a bit ugly, but some decorator can make this nicer:

class A:      @operator("<==")      def assign(self, other):          # whatever this means

Maybe the biggest problem would be with linting; not being able to distinguish between a typo and a user-defined operator until runtime, when no implementation is found. A ++= B  # is this typo? Abusive use of operators will likely happen; but the codebases that do will not be popular because of the learning curve. Instead, experiments will be done, some will succeed, and the best operators for each domain will become popular.  Then the python devs can pick the winners to include in the core library, if any. [1] Here is toy example that will build a graph using << and >>: https://github.com/klahnakoski/graph-builder/blob/master/graph_builder.py On 2019-06-04 08:28, Steven D'Aprano wrote:

On Fri, May 31, 2019 at 02:48:24PM +0100, Rhodri James wrote:

...

...

Python does not need to know this ... just hand it over to end user who knows how to implement such a thing. Python need to provide the mechanism. It really doesn't. If the end user is going to implement the logic of

On 29/05/2019 08:31, Yanghao Hua wrote: this anyway, implementing signal linkage as a method call or class all of its own is not a significant extra burden. That's not really fair: syntax matters, and for people working in a domain where certain syntax is expected, asking them to use something different is a significant cognitive burden. We don't ask people doing arithmetic to write code like this:

x.mul(y.sub(1)) # a * (y - 1)

There have been at least two times that Python has added syntax to the language to allow a third-party library to write more-idiomatic code in their domain. Both extended slicing seq[a:b:c] and the matrix- multiplication operator were added for numpy.

So *in principle* we certainly could add a new arrow operator for Yanghao Hua so his libraries and code will be more expressive and idiomatic in his domain.

But *in practice*, the hard truth is this:

- Yanghao Hua is one developer interested in HDL in Python;

- numpy was, and still is, one of the most important "killer apps" responsible for Python's success.

Adding syntax for the benefit of numpy helps millions of users; adding syntax for HDL-like helps... how many people? Twenty? A hundred?

Millions of users will have to learn the syntax. Unless they get some benefit, why consider this?

(But having said that... I reckon that if we had left and right arrow operators, <== and ==>, I have some DSLs where they would work for me too. Maybe.)

...

I'm pretty much done with this conversation too. You have repeatedly been asked what problem you are trying to solve and repeatedly respond by restating your solution, which appears to be to impose HDL sematics onto a non-HDL language. That's never going to be a good idea. That's totally unfair to Yanghao Hua on two levels.

(1) He has never asked for the interpreter to support the semantics he wants. He's not asking for Python to understand and implement HDL semantics in the language: he can do that himself, in the class.

(2) He has explained the problem he is trying to solve: he wants to write a DSL using syntax which doesn't look like crap from the perspective of people in that domain.

He just wants an operator that looks kinda like assignment that calls a dunder. He can't use assignment because the semantics are baked into the language (and there's no chance of that changing). He can't use existing operators because they're already in use.

This is not an unreasonable request. Python has bowed to similar requests at least twice before, and honestly, it's not like a <== operator would be weirder than other operators in use in mainstream languages. We're not talking APL or J here.

On the other hand... its not clear that, as small as this request is, the language would be better. There's lots of other domains where Python syntax is sub-optimal:

- We don't have good syntax for writing XML, or calling SQL. We just have to use strings.

- No nice syntax for writing Prolog-style deductive code;

- or concatenative DSLs;

- or natural language DSLs like Hypertalk or Inform.

Python can't be all things to all people.

Stephen J. Turnbull wrote:

Note that signal matrices will almost certainly be a completely different type from signals, so as far as the compiler is concerned there's no conflict between "@=" for signal injection and "@=" for signal matrix multiplication.

Except that if @= represents signal injection for individual signals, it could plausibly represent elementwise signal injection for matrices of signals, conflicting with in-place matrix multiplication of signal matrices. My solution to this would be not to use @ for matrix multiplication of signals at all, and represent it some other way. But it seems that Yanghao doesn't even want to compromise that far.

This means that draconian discipline (such as restriction to one compact and intuitive DSL!) is required or readability will be dramatically adversely impacted.

Perhaps the interpreter should be made so that it would only run code containing user-defined operators if it is appropriately signed. To obtain a signing key, a user would have to take a training course administered by the PSF, and pass written and practical tests showing that they are able to use user-defined operators responsibly.

Hard to say. First, you have the problems of precedence and associativity.

It seems that Scala does this based on the first character of the operator, e.g. all operators beginning with "+" have the same precedence and associativity as the built-in "+" operator. This neatly avoids the need for declaring new operators, but it could be restrictive. E.g. the proposed "<==" operator would get the same precedence and associativity as comparison operators, which might not be what you want. To make matters worse, if it were treated as a true Python-style comparison operator, then x <== a < b and c < d would be interpreted as (x <== a) and (a < b) and (c < d) which is definitely not what we want here! Of course, we could avoid this problem by choosing a different operator symbol. By the first-character rule, "::=" would have the same precedence as ":=", which would probably give what we want. However, we might want to have a special rule for assignment-like operators, such as if an operator ends with "=" and doesn't start with "<", ">" or "=" then it's treated like an in-place operator. -- Greg

Kyle Lahnakoski wrote:

Here is a half baked idea:

...

class A: def assign(self, other): # whatever this means

...

setattr(A, "<==", A.assign)

Some things that would need to be addressed to fully bake this idea: * What determines the precedence of these new operators? * How to distinguish left and right operator methods? -- Greg

On Wed, May 29, 2019 at 3:45 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

Yanghao Hua wrote:

...

a different assignment behavior in HDL is your assignment does not take effect until a delta cycle of zero virtual time has passed. (did you really looked at the previous postings? :)

You need to understand that most of the people reading this are not familiar with the workings and semantics of HDLs, so that phrases like "delta cycle of zero virtual time" are meaningless word salad. We need you to tell us what this new kind of assignment will do in *Python* terms.

It should do nothing in python terms, but rather let the user decide what to do. This is for representing ideas for a specific domain, if I explain in HDL terms, people complain python people cannot make sense out of it, if I explain in Python terms, people complain what problem are you trying to solve? So ... the short answer is this is the last thing python lacks of flexibility and would be great if python community are interested to support it.

...

x = 4 should be something like x <== 4 or x := 4 (the latter has been taken by the assignment expressions though ...). Such that variable initialization (=) and utilization (<==) can be differentiated.

Since you seem to be willing to use a different syntax for this kind of assignment, my suggestion is something like

x = Signal() ... x.next = 4

If I understand correctly what you want to use this for, that would be both suggestive of the semantics and entirely doable with existing Python features.

Yes sure this is doable, and this is exactly how I am doing it with python to interface verilog, e.g. signals are always accessed through module instances, and this is perfectly fine for verification. And there is another dozen different way to do this. But this is just not as intuitive as x <== 4 I am afraid. And for me this just kills all my motivation to use python to design hardware. If I use python to do something and I have to type more chars it doesn't make sense for me.

On Wed, May 29, 2019 at 9:46 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

Yanghao Hua wrote:

...

If I use python to do something and I have to type more chars it doesn't make sense for me.

If you shorten it to "n" you get

x.n = 4

which is exactly the same number of characters as your "<==" proposal:

x <== 4

Getting a bit more creative, you could use the little-known "._=" operator that already exists in Python. :-)

x ._= 4

Indeed this is clever and less chars, but not as readable as x.next = 4 nor x <== 4, right? and I think one thing python is very good at is about readability. Sorry about my paranoia ... I get the idea that the python community are very cautious and not willing to just add a new operator which I fully understood (and I understood it is difficult for the existing python community to see the value), for me I believe it is worth doing and I will make a PEP even though it is going nowhere, as well as a reference implementation to show how it can simplify things in a dozen of cases: easier descriptor like feature where the object can be instantiated anywhere, constructing pipe-lined operations like drawing, separation of instantiation and use, and of course a complete HDL design (as well as a simulator) in python with this syntax. And I think I will settle on the "<==" and "==>" operator, it is really intuitive and I liked it. I just hope this two will not be taken for other purposes in python in the future ... :) Next time someone proposed for using this two operator please you guys help to reject it ;-)

On Wed, May 29, 2019 at 2:57 PM Ricky Teachey <ricky@teachey.org> wrote:

For the understanding of all, it would help tremendously for you to implement a WORKING TOY example showing exactly the behavior you want in real (not theoretical) python, maybe something like this as a starting point:

class SignalBehavior: """"A descriptor that behaves like a HDL signal""" def __init__(self): # actual values for each inst stored here inst_dict = dict() def __get__(self,inst,owner): if inst is None: return self # getting of a signal behavior here # most basic version: return self.inst_dict[inst] def __set__(self,inst,value): # assignment of a signal behavior here # most basic version: self.inst_dict[inst] = value

class HDL: """Every new member of this namespace has signal behavior""" def __setattr__(self, attr, value): # note: if attr existed already, # SignalBehavior.__set__ was called

# add attr as signal behavior setattr(type(self), attr, SignalBehavior()) # SignalBehavior.__set__ will now be called setattr(self, attr, value)

And then demo some simple operations like this:

hdlns = HDL() hdlns.x = 1. # .x is a new descriptor hdlns.y = 2. # .y is a new descriptor # .z will also be a new descriptor hdlns.z = hdlns.x + hdlns.y

~~~ all hdlns members above behave like signals ~~~

Problem remains that you cannot pass hdlns.x/y around, x and y are really the things you want to pass around. You can pass hdlns in this case, but the receiving object has to figure out which signal (x or y) to use. And yes, I realized a comprehensive working example will help, give me some days ... to prepare it all.

On Wed, May 29, 2019 at 3:47 PM Ricky Teachey <ricky@teachey.org> wrote:

I look forward to seeing the working example. Hopefully it's clear already, but: I don't think anybody is yet claiming the descriptor approach is the "correct" or "best" answer for you. But it will help a lot to demo what you want, and it also allows you to use your first choice operator, which is =.

Will keep python-ideas and all of you posted.

...

Problem remains that you cannot pass hdlns.x/y around, x and y are really the things you want to pass around. You can pass hdlns in this case, but the receiving object has to figure out which signal (x or y) to use.

If you can pass x around, you can certainly pass hdlns.x around, or something shorter if you prefer-- ns.x, perhaps.

Take my previous example of class C which uses class A and B, let's integrate this ns namespace: class A: def __init__(self, in, o1, o2): self.in = in self.o1 = o1 ; self.o2 = o2 def process(self): # is self.in, self.o1 and self.o2 are still descriptor here to be used? class C: def __init__(self, in, out): ns.in = 0 ns.o1 = 0 ns.o2 = 0 # how should it be passed to A()? a = A(ns.in, ns.o1, ns.o2) # ?? doesn't work ... a = A(ns) # ok this works, but how does class A's instance a figures out is it ns.in/o1/o2 need to be used? a = A(ns, ["in", "o1", "o2"]) # maybe this will do the trick? you see, it starts to melt down ... of course it can work ... just doesn't feel right does it?

The problem of: "the receiving object has to figure out which signal (x or y) to use" seems easily addressed by creating a Signal class that knows how to return the Right Thing™ when doing math:

from numbers import Real # or whatever number abc is appropriate class Signal(Real): # override all mathematical operations

And modify the __set__ method of the SignalBehavior descriptor so it stores a Signal:

class SignalBehavior: ... def __set__(self,inst,value): self.inst_dict[inst] = value if isinstance(value,Signal) else Signal(value)

Now any math operation you care to do can result in whatever you wish it to be:

...

...

...

ns = HDL() ns.x = 1 ns.y = ns.x / 2 ns.z = 8 / ns.y

I am not sure if I understood this part, the problem is how a submodule uses this signals, it is pretty clear the top module can use descriptors and hook up the assignment behavior.

Another approach is to abandon the idea of HDL namespace parent class, and just create a HDL (or hdl) decorator: @hdl # decorator adds a custom __setattr__ class C: ... The decorator would modify the __setatrr__ method of the class so that members of the class have signal behavior. You could even write the decorator so that you specify which attributes have signal behavior, and the remainders don't: @hdl(list('xyz')) class C: def __init__(self, **kwargs): for k,v in kwargs.items(): setattr(self, k, v) c = C(x=1, a=2) # x is a signal, a is not

On Wed, May 29, 2019 at 4:43 PM Ricky Teachey <ricky@teachey.org> wrote:

Seems like this would do it (note: `in` can't actually be a variable):

class A(HDL): # A is now an HDL namespace - all members have signal behavior def __init__(self, in, o1, o2): self.in = in # descriptor self.o1 = o1 ; self.o2 = o2 # descriptors def process(self): # is self.in, self.o1 and self.o2 are still descriptor here to be used? # ^ yes, all of those are SignalBehavior descriptors if A is subclass of HDL ^

class C: # depending on behavior desired, C could be HDL namespace, too def __init__(self, in, out): # these work, but probably are not needed: ns.in = 0 ns.o1 = 0 ns.o2 = 0 a = A( ns.in , ns.o1 , ns.o2 ) # ...you can just pass the values to A directly as long as # A (or HDL) knows what to do with arbitrary in and out values: a = A(in, out, out)

The plus side of this approach is that you do not even need to create/declare signals, and you still can if you insist doing so. And you actually can enforce all of HDL-type of classes to have only signal-type object attribute, this is actually another good feature of it. I start to like it. To be completely honest (sorry again for my paranoia), this only works e.g. for class A(), if you do self.signal, you still cannot have local signals e.g. in __init__() you can not have local_signal without self, so is in process(). This means, for a middle-sized class implementation, you will start to worry about name space collisions pretty quickly among all the class methods. Information is no longer perfectly localized but spreading across all over your class (and its parents ... which probably in a different file ...). thoughts?

On Wed, May 29, 2019 at 9:34 PM Ricky Teachey <ricky@teachey.org> wrote:

Seems to me that namespace collisions are a possibility no matter what namespace you are working in, right? If you are in the global (module) namespace, and use up a, b, and c, that doesn't seem any different to me than using up ns.a, ns.b, and ns.c.

I am talking about each function definition, which when using this approach, effectively has to have its own namespace. e.g. at each of the function start: def process(self): ns = HDL() ns.x = 3 ns.y = 4 def another_process(self): ns = HDL() ns.x = 5 ns.y = 6 Compare with this: def process(self): x = Signal(width=32, value=0) y = Signal(width=16, value=0) def another_process(self): x = Signal(w=32,v=0) y = Signal(w=8,v=0xFF) Where should the signal attributes go for the namespace case? hmmm ... or maybe one namespace for each and every signal like this: def process(self): x = HDL(w=32,v=0) x.signal = 5 # only one signal or one type of signal allowed for each name space! y = HDL(w=8,v=0xFF) y.signal = 6 # ... You see, eventually this is not better than just arbitrary object and use obj.next = 5 ... I really think the separation of declaration and use ... is sometimes important (you don't have to be forced to use it, but you'd better be able to use it).

In fact, you could say it expands your namespace. You can have multiple HDL namespaces and interact between them without any problems:

# these will be descriptors that store Signal instances ns1.x = 1 ns2.x = 2 # you can combine them into another namespace ns3.x = ns1.x+ns2.x # or into an existing namespace ns1.y = ns1.x+ns2.x # or put them into a data structure x = [ns1.x, ns2.x, ns3.x] # and pull them back out again: ns4.a, ns4.b, ns4.c = x

Behavior is preserved in all cases.

see above.

but that doesn't really work. I know in my previous message I said the problem was the aesthetics, but on further thought there are more problems with it.

I thought about those problems myself, but are they not solved by doing all assigning of cuntion results within some "HDL namespace"? We define that namespace, and do all operations to variables using that ns: ns.y = function(ns.x) The question of what function(ns.x) returns goes away-- the ns.x object knows what to when function is called on it, and ns.y knows what to do with the return value of any function that makes semantic sense to be assigned to it. Other than the problem of "I have to remember to assign function results into an HDL namespace, and not a regular variable" (which I agree is not fantastic, but it certainly isn't a slam-dunk justification to add new operators), where is the problem?

On Thu, May 30, 2019 at 11:00:41AM +1200, Greg Ewing wrote:

Steven D'Aprano wrote:

...

The obvious solution to customising assignment is to use a dotted target:

obj.x = value

Another problem with this is that we don't want to customise *all* assignments. Sometimes we just want a regular Python assignment. (See my previous post about two different kinds of assignment.)

I think that talking about this in terms of "customising assignment" is misleading and is causing a lot of confusion.

You might be right, but then the first first post in this thread talked about it: I realize there is no way to overload the behavior of the assignment operator in python ... where signal = 5 will always make signal to be 5, instead of feeding 5 into this signal. (the second being a *bad thing*) so I'm pretty sure that using a new assignment-like operator instead of = is the OP's second choice. -- Steven

On Thu, May 30, 2019 at 7:07 AM Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Steven D'Aprano writes:

...

You might be right, but then the first post in this thread talked about it:

I realize there is no way to overload the behavior of the assignment operator in python

... where signal = 5 will always make signal to be 5, instead of feeding 5 into this signal.

I've seen no hint that "signal arithmetic" is a thing, so as far as I can see Chris Angelico's "+=" (analogous to "feeding characters into the end of a string") wins. (The OP simply ignored the argument, and replied "but I want '<==='".) Or you can make the algebraists happy with "*=". If you need "+=" for signal addition in an arithmetical sense, maybe "*=" is still available at the sacrifice of the possibility of saying "signal *= 5" to mean "repeat last signal 5 times", and some degree of "WTF multiplication? oh yeah ...".

signal = Signal() signal = Signal(args) # args might preload stuff into a buffer # or configure the Signal in some way signal += value # signal changes state to value, read # "signal gets next <value>".

Unless Yanghao can come up with a reason not to use this such as (a) he needs that operator for a more natural meaning, or (b) that it confuses other software (eg, linters, as Chris admits "<=" would do), or (c) that he needs polymorphic variables such that having the semantics of the operator differ across classes in the union would confuse humans, I think this moots the whole "new operator" proposal, and we can move on to bikeshedding the choice of operator symbol.

Digital signals are really just integers, you can have arbitrary length of bits, e.g. 1 bit as bool, 8 bit as byte, 32 bit as word and hundreds of bits (we have bus width that is 1024 bits in some system). So yes all the arithmetic operations, almost all of them can be used for simulation, and majority of them is even directly synthesizable into actual hardware implementation, that's the reason why I would even like to keep the matrix operator as is rather than using it to mean signal assignments, because I might very well want to do matrix operations on signals as they are really just a special type of integers. (a) Yes I really needs all the arithmetic operators for their natural meaning (b) Yes it confuses almost all existing python libraries that related to arithmetic (hopefully <== does not confuse too much for linters) (c) Yes, signals should be a different kind of integer, but only on assignment, so introduce a new one really makes sense for me.

On Wed, May 29, 2019 at 2:57 PM Ricky Teachey <ricky@teachey.org> wrote:

class HDL: """Every new member of this namespace has signal behavior""" def __setattr__(self, attr, value): # note: if attr existed already, # SignalBehavior.__set__ was called

# add attr as signal behavior setattr(type(self), attr, SignalBehavior()) # SignalBehavior.__set__ will now be called setattr(self, attr, value)

This doesn't seem to work, the first time you setattr on hdlns.x, it will recursively doing so due to setattr(self, attr, value). I do understand you need override __setattr__() to provide an on demand signal creation, and you do need to call setattr() again to actually activate the descriptor, so stack eventually overflows. With a bit modification it can work, e.g. the last setattr(self, attr, value) should explicitly invoke SignalBehavior().__set__() instead.

Perhaps you should take a look at Dlang. Using opDispatch and a with statement and compile time code generation, you may be able to accomplish what you’re trying to do entirely in D.

On May 24, 2019, at 4:05 AM, Yanghao Hua <yanghao.py@gmail.com> wrote:

...

On Fri, May 24, 2019 at 1:59 AM Steven D'Aprano <steve@pearwood.info> wrote:

...

On Thu, May 23, 2019 at 06:23:31PM +0200, Yanghao Hua wrote:

Is this (<== and ==>) something can be made into CPython?

If it goes into CPython, eventually every other Python needs to do the same.

Of course it *could* be put into Python, but you haven't given sufficient justification for why it *should* be put into Python.

Why does your DSL need a seperate assignment operator? How is it different from regular assignment?

Could you use __setitem__ instead?

Instead of this:

variable <== thing # magic happens here

can you write this?

obj.variable = thing # magic happens here

If your DSL is so different from Python that it needs new operators, perhaps you should write an interpreter for your language and run that seperately.

code = """ any syntax you like <- thing """ result = interpret(code)

Well, if python is not envisioned to be able to represent almost everything elegantly maybe I should indeed walk away from this idea. (This is sad for me ... I have been saying and believing python is envisioned that way for more than 20 years now). Your argument could be applied on PEP465 as well, where the only justification is to make matrix operation prettier. I have explained the problem of use descriptors in previous replies, where you cannot have a local signal, e.g. obj.signal = thing # works, but local_signal = thing # doesn't work. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, May 23, 2019 at 4:49 AM Ricky Teachey <ricky@teachey.org> wrote:

Can I offer an alternative suggestion?

How about these arrow operators, which despite being 3 characters, look pretty nice, and are not currently valid python:

a <== b <== c a ==> b ==> c

Perhaps __larrow__ and __rarrow__ for the dunders.

And they would not be the only 3 character operators in python:

//=, **=, >>=, <<=

I wouldn't mind one more char to type ... as long as it looks good. My preference order is: = (almost impossible), := (taken already ...), <-/<= (conflicts existing code and breaks bad coding styles). Looks like in the short term there is not really many choice left. I like it, definitely better than signal.next ... and still looks like verilog <= assignment. Is this (<== and ==>) something can be made into CPython?

On Fri, May 24, 2019 at 1:59 AM Steven D'Aprano <steve@pearwood.info> wrote:

On Thu, May 23, 2019 at 06:23:31PM +0200, Yanghao Hua wrote:

...

Is this (<== and ==>) something can be made into CPython?

If it goes into CPython, eventually every other Python needs to do the same.

Of course it *could* be put into Python, but you haven't given sufficient justification for why it *should* be put into Python.

Why does your DSL need a seperate assignment operator? How is it different from regular assignment?

Could you use __setitem__ instead?

Instead of this:

variable <== thing # magic happens here

can you write this?

obj.variable = thing # magic happens here

If your DSL is so different from Python that it needs new operators, perhaps you should write an interpreter for your language and run that seperately.

code = """ any syntax you like <- thing """ result = interpret(code)

Well, if python is not envisioned to be able to represent almost everything elegantly maybe I should indeed walk away from this idea. (This is sad for me ... I have been saying and believing python is envisioned that way for more than 20 years now). Your argument could be applied on PEP465 as well, where the only justification is to make matrix operation prettier. I have explained the problem of use descriptors in previous replies, where you cannot have a local signal, e.g. obj.signal = thing # works, but local_signal = thing # doesn't work.

On Fri, May 24, 2019 at 10:30 AM Paul Moore <p.f.moore@gmail.com> wrote:

Personally, my understanding is that Python is not designed to make writing DSLs in Python easy (I'm a little sad about this fact myself, but I accept that trying to make a language good at everything is counter-productive). Features like the ability to call functions without parentheses, user-defined operators, user-defined syntax in general, are directly intended to make writing DSLs easier, and have been requested and rejected many times in the past.

I am struggling with this myself as well ... but Python is arguably one of the best language to represent ideas. When I wrote "if python is going to support scala like operator constructions" I meant more to ask "If someone out there already had this plan and working on it", rather than advocating it myself too strong (I would really love it though, but not the part that calls a function without parentheses).

I believe Python is OK (I may even go as far as saying "pretty good") for defining simple DSLs, but it's not a core goal of the language, and you have to make your DSL fit with Python's syntax, not the other way round.

Python is great! I am in no way saying anything bad about python only because it is missing one feature. But I do think it can still be better and better. I guess in the past there is just not enough interests from hardware people to use Python to design hardware. We had MyHDL, cocotb, etc. as well as my own Python-Verilog co-simulator ... because SystemVerilog really makes me sick ... for a language that has more than 250+ reserved keywords you can imagine it is even worse than C++ and no two vendor could ever build a compiler for system verilog behaves exactly the same (I don't even know if the spec itself can be self-consistent ... just too many possibilities to go wrong) ... that's why I build the Python-Verilog co-simulator where I can write much more powerful testbenches just in a few lines of Python.

That's not to say that features useful for DSLs aren't ever going to be accepted, but they will need justification independent of their value in DSLs. The matrix @ operator that you quote wasn't accepted because "it reads well in a maths context", the proponents did a *lot* of research to demonstrate that the operator simplified existing code, and existing language features couldn't deliver the benefits. PEP 465 is well worth a read to give an idea of the sorts of arguments needed to successfully introduce a new operator to Python.

Sure and will work on it. I am not sure if the argument can come as strong as in PEP 465, as numpy community (which has almost the entire machine-learning world) is considerably larger than python hardware design community ... but I will try to see if I can convince some of you guys.

On Fri, May 24, 2019 at 11:47 AM Steven D'Aprano <steve@pearwood.info> wrote:

PEP 465 had a concrete purpose for its new operator, and although no built-in object supported the @ operator, we understood why Numpy wanted it. But I do not understand why you want an arrow operator or what you will do with it.

OK, I see your point and will work on a more comprehensive example with some investigations how it can simplify existing libraries outside there.

What should built-in types do with this new dunder?

"Hello world!" <== "aardvark"

23.5 <== 1.25

[1, 2, 3] <== 4

It will generate error: unsupported operand types(s) for: 'int' and 'int'. (or list and int etc.)

Have you considered the __ilshift__ method?

signal <<= 5

I explained already but here again: you will need to have "this_signal <<= (that_signal << 4) + something, and next line you should be able to write this_signal <<= 4 bit, where all arithmetic operation still have to be kept. Otherwise it makes signal doesn't look like a value ... it needs to be looking like a integer but with a different assignment behavior. In hardware design domain a signal is just a value but with its own assignment behavior involving delta-cycles (a virtual time step increase of the value zero ... I know it is a bit confusing to SW developers). So the intention is to keep all arithmetic ops like a int, but only changing how the assignment is done.

Yes. You should study PEP 465. It is one of the best PEPs in Python's history. If you want your proposal to succeed, you should think "How can I write this to be more like PEP 465?"

I am writing just to see if I missed something significantly in this field ... will come back with a comprehensive PEP.

I am completely aware that we can't hook into assignment except with attribute or item assignment. But I still do not understand why you want to hook into assignment.

What is going to be hooked into the assignment is the pending update of the value to the signal, which only happens until a delta cycle is passed, such that you can have a deterministic behavior when using software to mimic actual hardware behaviors.

Perhaps I have missed something when reading your earlier posts, but I don't understand what you will do with this new operator. Perhaps you can show up your intended method?

x <== y

def __arrow__(self, other): # what goes here?

For example: def __larrow__(self, other): self.next_value = other.value And when increasing virtual time (e.g. a delta cycle): for sig in all_signals: sig.current_value = self.next_value

I am not sure if I understand what you mean by "feeding 5 into this signal". But if it means what I think it means, then you can give signal a method:

signal.feed(5)

which is explicit and readable. Or you could use <<= as I mentioned above.

The intention is to make the signal looks like an integer, and yet to change the assignment behavior. As I explained above that all normal arithmetic ops still need to be kept.

You also said:

"[arrow operator] looks like kind of the non-blocking assignment operators in hardware description languages"

What is "non-blocking assignment"?

non-blocking assignment is a terminology in HDL design, where the assignment is not actually happening until a later time. In simple words if the "virtual-time" is not increasing, then the assignment is not happening. To allow all hardware threads seeing the same consistent value at that particular virtual time. I have the prototype working, let me implement a much more elaborated example, and also look into how existing modules like MyHDL could benefit out of it, and give you guys a written PEP.

From this good discussion so far (Thanks everyone for your candid responses!), I understood I have not missed anything significantly, Python community has no intention to make Python supporting operator-on-demand-definition like in scala, so the only possible way forward is (for me): build the "<== and ==>" operator and write a PEP to justify its usefulness. The general intention is to make HDL design in python looks and feels at least as good as in verilog/vhdl, so once I have my library complete I will explain in depth how it looks side by side for Python vs Verilog, with and without the arrow operators.

On Fri, May 24, 2019 at 5:45 PM Terry Reedy <tjreedy@udel.edu> wrote:

What I understand is that you are doing discrete-time hardware simulation and that you need a operator that will schedule future assigments to int-like objects. Have you considered using '@' to do that? int @ int-expression is currently invalid, so defining it will not interfere with other int operations. What am I not understanding?

I am not sure if I understood this. The intention is to e.g. assign a signal with a value in a nature way, e.g. signal <== 5, are you saying to replace <== with @? I guess that should work (signal @ 5), not really intuitive though. I really really would like either a equal sign or something like an arrow ... I know I am a bit picky ...

On Sat, May 25, 2019 at 8:28 PM Terry Reedy <tjreedy@udel.edu> wrote:

On 5/24/2019 4:25 PM, Yanghao Hua wrote:

...

On Fri, May 24, 2019 at 5:45 PM Terry Reedy <tjreedy@udel.edu> wrote:

...

What I understand is that you are doing discrete-time hardware simulation and that you need a operator that will schedule future assigments to int-like objects. Have you considered using '@' to do that? int @ int-expression is currently invalid, so defining it will not interfere with other int operations. What am I not understanding?

I am not sure if I understood this. The intention is to e.g. assign a signal with a value in a nature way, e.g. signal <== 5, are you saying to replace <== with @?

That is what i meant, but reading

...

I really really would like either a equal sign

suggests '@=' as a better alternative. (I am not sure if the implementation would be equally easy or hard.) How does 'signal @= 5' look?

Either is pragmatic in that these exist since a few versions ago, and cannot interfere with existing integer expressions, rather than in the far very hypothetical future.

...

not really intuitive though

'@' means 'at' and you want to make the assignment 'at the next time mark' (or whatever you call it). This is more intuitive to me than seeing '@' as 'matrix-multiply' because 'matrix' contains 'at'.

When we added @, it was known and intended that it could serve other then unknown uses.

@= has all the same issues like <<= or >>=, in that you are basically sacrificing a well known number operation and force it to mean something completely different. In previous examples I have shown that HDLs are basically handling numbers, so <<=, >>=, |=, &=, +=, -=, %=, *=, <<, >> all could be, and should be used directly to signals. I admit this (@=) is a much rarer case, but why do we want to exclude the possibility for a matrix of signals to multiply another matrix of signals and assign the result to another matrix of signals? how does this look like? X @= (X @ Y), where @= means signal assignment, and X @= Y, does it mean signal assignment of Y to X, or does it mean X = X @ Y? This simply causes a lot of confusions. I want a solution that is impossible to cause any confusions. As @ and @= is built in python, any one could be using it in all possible ways that is beyond our imagine ... unless I have a way to make sure user cannot use @= as matrix multiplication (which obviously I cannot), otherwise I am very much de-motivated to use it to mean HDL signal assigns ...

On Sun, May 26, 2019 at 6:05 AM Terry Reedy <tjreedy@udel.edu> wrote:

On 5/25/2019 3:09 PM, Yanghao Hua wrote:

...

@= has all the same issues like <<= or >>=,

No, it does not

...

in that you are basically sacrificing a well known number operation

because @= is not a number operation at all.

Yes you are right. @ is not a number operation, it is number-collection operation. What is preventing the same operation on signal-collections?

...

I admit this (@=) is a much rarer case,

It is a different case.

Really not much different for me as you can use it to operate on matrix (which can be either a matrix of number or matrix of signals).

...

but why do we want to exclude the possibility for a matrix of signals to multiply another matrix of signals and assign the result to another matrix of signals?

We do not. <int subclass instance> @= int would be implemented by the __imatmul__ method of the int subclass. matrix @= matrix is implemented by the __imatmul__ method of the matrix class. This is similar to 1 + 2 and [1] + [2] being implemented by the __add__ methods of int and list respectively.

I really don't understand the argument here. And let's apply the same argument to PEP465 why not matrix multiply override <<= instead? For me not using @= is exactly the same reason for not using <<= and others.

how does

...

this look like? X @= (X @ Y), where @= means signal assignment, and X @= Y, does it mean signal assignment of Y to X, or does it mean X = X @ Y? This simply causes a lot of confusions.

Why don't people more often get confused by a + b? Partly because they use longer-that-one-char names that suggest the class. Partly because they know what a function is doing, perhaps from a name like set_signals. Party because they read the definitions of names. Conventionally in math, scalars values are lower case and matrices are upper case. So x*y and X * Y are not confused.

I think people don't get confused by a + b because a + b does mean a + b and does not mean a * b and it has nothing to do with how you name the operands.

On Fri, May 24, 2019 at 12:29 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

Yanghao Hua wrote:

...

I have explained the problem of use descriptors in previous replies, where you cannot have a local signal, e.g. obj.signal = thing # works, but local_signal = thing # doesn't work.

Maybe you could do something like:

local = Signals() local.signal1 = ... local.signal2 = ...

In structure design ... and especially when you design a hardware that is meant to be automatically converted into verilog or even logic gates, I personally would really want to have a one-to-one relationship of the python-objects vs the actual hardware structures. The granularity is at signal/bit level. This is why I really think giving a special assignment in python which users could override is really helpful, rather than having to have this kind of special case: if you do "self.abc = thing" descriptor mechanism is invoked, but the very next line if you do "abc = thing" ... nothing will happen. This special case can be completely removed and having a much better conceptual consistency if the "<==" assignment operator always behaves the same, doesn't matter if it is "self.abc <== thing" or "abc <== thing".

Another idea if you really want to be able to do `foo = 5` and have it behave the way you want: Create a custom dictionary type to hold locals() (and perhaps globals() if needed). Unless I'm wrong, that dict type can pretty much do whatever you want, including overriding assignment behavior. Then just run the code using exec(), passing the custom hdl_locals(). You could package up a custom python interpreter for hardware programming which simply execs() the python code using this customized assignment behavior provided by hdl_locals(). Such a customized namespace is a very pythonic approach, and if I understand correctly, most regular non hdl python would probably be able to run.

You can do things like this with exec(): class SafeDict(dict): curated_keys = {"a", "b", "c"} def __setitem__(self, k, v): if k not i self.curated_keys: raise Exception(f"{k!r} key not allowed") super().__setitem__(k, v) locals_dict = SafeDict() globals_dict = SafeDict() exec("d=1", locals_dict, globals_dict) # Exception: 'd' key not allowed You can do all sorts of things using that technique, including the way assignment to variables is handled. Beyond that, I declare my competence in this matter to have been hereby exceeded, since I really don't follow or understand the HDL paradigm you are working in. Maybe others might be able to help.

There is probably some existing python API you can hijack to make custom locals() and globals() work everywhere. Perhaps pdb and inspect.stack are good places to start; maybe there’s a PDB API to break on every new stack frame and maybe you can use inspect to do the proper assignment overrides. Python is a very layered language. Sent from my iPhone

On May 25, 2019, at 4:53 AM, Yanghao Hua <yanghao.py@gmail.com> wrote:

...

On Fri, May 24, 2019 at 10:34 PM Ricky Teachey <ricky@teachey.org> wrote:

You can do things like this with exec():

class SafeDict(dict): curated_keys = {"a", "b", "c"} def __setitem__(self, k, v): if k not i self.curated_keys: raise Exception(f"{k!r} key not allowed") super().__setitem__(k, v)

locals_dict = SafeDict() globals_dict = SafeDict()

exec("d=1", locals_dict, globals_dict) # Exception: 'd' key not allowed

You can do all sorts of things using that technique, including the way assignment to variables is handled.

I see your point now. In this case user will need to write HDLs probably in below fashion:

class my_module: def __init__(self, input, output, ...): ... def process(self): exec("signal = 5", locals, globals) # compare this with: signal <== 5

I fully agree this can work, but also it doesn't seem to be any better looking than signal.next = 5. I think there are two important point here, first is to make HDL design feels as natural as possible and as pythonic as possible as well as leaving very little room for mistakes, second is to make HDL design at least as easy and as intuitive as in traditional HDLs. And when I see this can actually be achieved easily with less than 100 lines of CPython code changes I am tempted to give it a try (I am so in love with python ...).

The other thing I was thinking about is PEP572 assignment expression (a := b), if it could be modified to allow user override (e.g. via __assign__()), and allow it to be used without bracket like "y := f(x)" if __assign__ is present in y. Then this is even aligned with Chisel's assignment operation and I'd be completely happy with it. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 24 May 2019, at 22:11, Yanghao Hua <yanghao.py@gmail.com> wrote:

...

On Fri, May 24, 2019 at 3:27 PM Ricky Teachey <ricky@teachey.org> wrote: This seems like a hurdle you're going to have trouble passing... especially given that all the functionality that is required can be provided using existing descriptor behavior. You will need to pretty concretely demonstrate why the special handling of signals in assignment (no matter which operator is the operator of choice) is something the language at large really needs, and why descriptors aren't sufficient.

Just a quick example, suppose you have a class A and class B representing two circuit blocks, where in class C you want instantiate A() and B() and connecting them together. Please do let me know if you could have a more reasonable way of representation to make it working. class A: def __init__(self, output): self.output = output def process(self): self.output = 5 # !!! this does not work for descriptors passed in # self.c_self.signal = 5 # this might work, but what the heck really?!

class C: signal = Signal() def __init__(self): a = A(output=self.signal) # a = A(output=self) # it is only possible for signal to work if you pass C's self into a ... b = B(input=self.signal) # !!! This does not work !!!

Instead, a much more natural way of doing it is: class A: def __init__(self, output): self.output = output def process(self): self.output <== 5 # this always works!

class C: def __init__(self): signal = Signal() a = A(output=signal) b = B(input=signal) # this feels much better, isn't it?

process() in A could look like: self.send(output=5) To me that looks OK, and scales nicely with multiple outputs: self.send(a=5, b=3) send() is implemented simply as def send(self, **kwargs): for k, v in kwargs.items(): signal = self.signals[k] signal.c_self.output = v Or something. I'm not sure about the details since I don't understand the example you give. It's pretty abstract and vague. Just a simple example of how to use A with a print() and the expected output of said print would help? / Anders

I don't really understand HDL/Verilog, but I've worked with people who do. In fact, I even wrote a pre-processor that transformed the same DSL to Python, C++, and Verilog. In my mind, the HDL use case is FAR too narrow and specialized to warrant a new arrow operator, let an entirely new parser and semantics around arbitrary operators. There are several existing dunders that could plausibly be repurposed already (<<, <<=, <=, etc). Those might look sightly different than the verilog operators, but that's a very small price. In fact, just using attributes and assignment is an incredibly low bar too, and allows whatever overriding you wish. I just don't buy the idea that such a DSL can only be useful if it spells 'abc <== message' and useless if it spelled the same thing as 'abc <<= message'. On Fri, May 24, 2019, 9:06 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

On Fri, May 24, 2019 at 12:29 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

...

Yanghao Hua wrote:

...

I have explained the problem of use descriptors in previous replies, where you cannot have a local signal, e.g. obj.signal = thing # works, but local_signal = thing # doesn't work.

Maybe you could do something like:

local = Signals() local.signal1 = ... local.signal2 = ...

In structure design ... and especially when you design a hardware that is meant to be automatically converted into verilog or even logic gates, I personally would really want to have a one-to-one relationship of the python-objects vs the actual hardware structures. The granularity is at signal/bit level. This is why I really think giving a special assignment in python which users could override is really helpful, rather than having to have this kind of special case: if you do "self.abc = thing" descriptor mechanism is invoked, but the very next line if you do "abc = thing" ... nothing will happen. This special case can be completely removed and having a much better conceptual consistency if the "<==" assignment operator always behaves the same, doesn't matter if it is "self.abc <== thing" or "abc <== thing". _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 26 May 2019, at 08:07, Yanghao Hua <yanghao.py@gmail.com> wrote:

On Sun, May 26, 2019 at 12:04 AM David Mertz <mertz@gnosis.cx> wrote:

...

I don't really understand HDL/Verilog, but I've worked with people who do. In fact, I even wrote a pre-processor that transformed the same DSL to Python, C++, and Verilog.

In my mind, the HDL use case is FAR too narrow and specialized to warrant a new arrow operator, let an entirely new parser and semantics around arbitrary operators. There are several existing dunders that could plausibly be repurposed already (<<, <<=, <=, etc). Those might look sightly different than the verilog operators, but that's a very small price. In fact, just using attributes and assignment is an incredibly low bar too, and allows whatever overriding you wish.

Well, depends on how we define narrow ... you are writing probably this email on a HDL designed machine ... and the entire world is powered by HDL designed silicons. that is not small for me at all.

I think you are confusing the number of people that use HDL with the amount of product created. Also I was under the impression that HDL tools exist already that are considered usable and yet do not need python. What is the problem that you aim to solve with a python HDL tool? If the syntax of the HDL is so important I do not understand why you do not write a parser for the HDL and build the run-time model in python. Then run the model - no new syntax required. For any non-trivia hardware I'm finding it hard to believe that python will run fast enough to be useful. What is it that I'm missing?

...

I just don't buy the idea that such a DSL can only be useful if it spells 'abc <== message' and useless if it spelled the same thing as 'abc <<= message'.

So you don't find this is confusing?

signal <<= 5 # does it mean left shifting signal or assigning 5 to signal??

I really do think it is confusing.

Isn't that a matter of familiarity with HDL? Many people replying are not familiar with HDL. Barry

On 26 May 2019, at 17:04, Yanghao Hua <yanghao.py@gmail.com> wrote:

On Sun, May 26, 2019 at 11:27 AM Barry Scott <barry@barrys-emacs.org> wrote:

...

I think you are confusing the number of people that use HDL with the amount of product created.

I don't see how I did that but if you intercepted that way I must have done that somehow.

You said this: "Well, depends on how we define narrow ... you are writing probably this email on a HDL designed machine ... and the entire world is powered by HDL designed silicons. that is not small for me at all." Which I take to mean that because there are billions of chips in the world there are billions of python users. And the change you want is justified by the billions of python users.

...

Also I was under the impression that HDL tools exist already that are considered usable and yet do not need python.

What is the problem that you aim to solve with a python HDL tool?

I think the question should be which part of existing HDLs does not need to be fixed? The answer would be easier: the assignment syntax is pretty elegant. I would recommend to take a look at Chisel, all the motivations for creating Chisel is pretty much the same reason I would create a python equivalent and I had a prototype shows in some area it could even be better.

There is a reason that there are 1,000s of computer languages in the world. Not all computer languages are able to solve all problems. If Chisel or Scala is closer to what you want why are you not using them for your HDL tools?

And what is the problem python solves that C doesn't solve? And what is the problem C solves that assembly doesn't solve? One common answer to all of them would be: fewer chars for bigger ideas.

What has that got to do with justifying a change to the python language?

...

If the syntax of the HDL is so important I do not understand why you do not write a parser for the HDL and build the run-time model in python. Then run the model - no new syntax required.

Many many people and company did it already ... I am just exploring a different possibility.

And it seems that you have hit a major problem with the HDL syntax not mapping to python. Maybe python is not the right choice?

...

For any non-trivia hardware I'm finding it hard to believe that python will run fast enough to be useful. What is it that I'm missing?

We are all Python users and we start to worry about running fast? really? ;-) I thought we all understood development time matters (if not even more ...)

Packages like numpy are not written in python for a reason. A pure python numpy would be very slow. So yes we worry about run time speed. Barry

On Sun, May 26, 2019 at 8:00 PM David Mertz <mertz@gnosis.cx> wrote:

On Sun, May 26, 2019, 1:12 PM Barry Scott <barry@barrys-emacs.org> wrote:

...

You said this: "Well, depends on how we define narrow ... you are writing probablythis email on a HDL designed machine ... and the entire world is powered by HDL designed silicons. that is not small for me at all."

Which I take to mean that because there are billions of chips in the world there are billions of python users. And the change you want is justified by the billions of python users.

I think the analogy is more like saying there are billions of Verilog users. Chips designed using this software are probably more numerous than are human beings, and nearly all humans use one or more such chips. This seems like a poor line of reasoning though. There are really only maybe tens of thousands of people in the world who use Verilog, and that is a nice target.

Thank you. It is definitely more than tens of thousands. I count only the 20 big names and it is already at least tens of thousands. All in all it should be a few millions. We should argue really from technical point of view, in the spirit of python, if this is worth or not rather than saying HDL is just too small a problem to address.

Pursuing the same indirect reasoning, we could also claim there are billions of users of PSS/E, something I had not heard of until a web search a few minutes ago. Apparently this is software used in design of electrical distribution systems, required to make all of those ICs operate at all. Or software used for building the construction equipment needed to construct the power lines and factories where chips are made. Or...

You know, Python and C are very close friends and operates almost like natively together and you have things like Cython to ease the creation of C-Python interface. If we look at the problem of building any complicated piece of hardware, it almost certainly done in this way: (1) C algo model -> (2) Virtual Prototype in SystemC (qemu-like software model but with proper partitioning of sw/hw, hw is still kind of a C model) -> (3) Verilog RTL model (Yes, RTL is actually just a model too, which is easier to be translated to actual hardware implementation). At each of the levels, it is a end-to-end complete system, and yet in each level, there are a lot of things duplicated, re-developed and eventually extremely difficult to have consistency. The big picture I want to achieve is to replace those (1)(2)(3) with Python + C -> Python + C -> Python + C. Where the interface between each layer can be easily unified through adapters and 99% of the knowledge only need to be developed once (if it is developed in C there will be a python interface).

In any case, as I said, modifying the whole Python language to allow use of exactly the same symbols as Verilog users, seems foolish to me.

Agree, that would be foolish. I would just ask for any way that overloads assign but not messing up existing operators (think of HDL signals as integers).

That said, '<=' is already perfectly well available for non-blocking assignment. '=' cannot be overriden, but it occurs to me that a custom class could perfectly well use '==' for blocking assignment just by customizing a class. Yes, that would require 'foo.eq(bar)' for the old meaning of equality (or something similar), but it seems like assignment is more common than equality checks in Verilog.

Not really, HDLs uses <= as less or equal than, as well as the equality check VERY VERY often. The sad thing is when I went through the entire python supported operators they can all be used with signals ... including the matrix operator @/@=.

On Mon, May 27, 2019 at 5:25 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

On Sun, May 26, 2019 at 7:11 PM Barry Scott <barry@barrys-emacs.org> wrote:

...

You said this: "Well, depends on how we define narrow ... you are writing probably this email on a HDL designed machine ... and the entire world is powered by HDL designed silicons. that is not small for me at all."

Which I take to mean that because there are billions of chips in the world there are billions of python users. And the change you want is justified by the billions of python users.

No offense but it is a bit too optimistic too think there are billions of python users (who actually write python code). And it is a bit too pessimistic to think there are billions of chips in the world ... the chips I personally worked on most likely exceeded the billion mark. So put this in perspective there are hundreds of billions of chips being produced, this is the problem space of *secondary* impact. the *primary* impact could be a few million (my estimation) HDL developers, and a dozen million HDL verification engineers. It is definitely smaller than the entire python developer community because probably all of the hardware/silicon engineers write a few lines of python. I just want to say this is *NOT* a small problem at all.

SiFive can now develop RISC-V CPUs which is faster and more efficient than the respective ARM versions is probably the best testimony that a better HDL matters.

Irrelevant though. I've ridden in a car - does that make me a petrochemical engineer?

...

Packages like numpy are not written in python for a reason. A pure python numpy would be very slow. So yes we worry about run time speed.

numpy is not Python! numpy is C!

(Fortran, for what it's worth)

Python does not have numpy! I am a bit surprised that we python community is even denying python is slow? You bench mark it in all possible ways python is still slow, if not the slowest. That's a known fact and we always do python prototyping and optimize the critical path/parts into C. Python is not build for speed. (don't bring in pypy for now please ... ) What we love about python is its development speed. The same applies to the world of hardware design and verification.

Actually, I just code in Python. And when I ask if a dictionary contains a particular key, I don't care that this is implemented in C; all I care about is whether the dictionary contains that key. Am I writing C code? Nope. I'm writing Python code. And PyPy is actually quite relevant, because it runs normal Python code. You don't have to "optimize ... into C". You just write Python, and then run it using a Python interpreter. This discussion may need to move to python-list rather than -ideas, as this isn't really progressing towards any sort of language improvement. ChrisA

On Mon, May 27, 2019 at 6:05 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

Doesn't matter how it ends up, I urge the python community do give it a second thought. (Don't you guys think it is odd that Python can overrides almost every operation but not for assignment ... is assignment really worthy being a special case?!)

Yes. It IS a special case, because assignment is not handled by the object being assigned to. When you assign to an attribute or subscript of an object, the *parent* object determines how the assignment is done, not the one being replaced. Look at languages where the being-replaced object gets to redefine assignment (C++ and PHP come to mind, and there may be others). MANY MANY parts of your code become harder to comprehend. It's not the only thing special enough to be non-overridable. In Python, you have several fundamentals that are absolutely guaranteed: a = b a is b b if a else c a or b # a and b There is no way that the values of a, b, or c can change the meanings of these expressions. (The truthiness of 'a' will define which of two options is chosen, but you cannot redefine the operator itself.) This is a Good Thing. ChrisA

On 2019-05-26 21:34, Yanghao Hua wrote:

On Sun, May 26, 2019 at 10:25 PM Chris Angelico <rosuav@gmail.com> wrote:

...

On Mon, May 27, 2019 at 6:05 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

...

Doesn't matter how it ends up, I urge the python community do give it a second thought. (Don't you guys think it is odd that Python can overrides almost every operation but not for assignment ... is assignment really worthy being a special case?!)

Yes. It IS a special case, because assignment is not handled by the object being assigned to. When you assign to an attribute or subscript of an object, the *parent* object determines how the assignment is done, not the one being replaced. Look at languages where the being-replaced object gets to redefine assignment (C++ and PHP come to mind, and there may be others). MANY MANY parts of your code become harder to comprehend. It's not the only thing special enough to be non-overridable. In Python, you have several fundamentals that are absolutely guaranteed:

a = b a is b b if a else c a or b # a and b

There is no way that the values of a, b, or c can change the meanings of these expressions. (The truthiness of 'a' will define which of two options is chosen, but you cannot redefine the operator itself.)

This is a Good Thing.

Absolutely right and good. But if you just had := or <== mean "assignment behavior is handover to the object if __assign__ is defined", then you can remove the whole descriptors thing (or rather, it can do all a descriptor can and better, and actually := or <== is saying, hey look, I am a descriptor ... ) . ;-)

I am not saying "=" should be overloaded, I am saying there should be an assignment that can be overloaded, which completes the picture in python.

It seems to me that this is a little like "variables" in tkinter (IntVar, StringVar, etc.). You create and then set and get the contents/value, but in tkinter there's always a clear distinction between the variable itself and its value. For example, you don't say "var + 1", but "var.get() + 1". However, in the examples that you've given, the variables on the RHS are being used in calculations in the same way that the value would be used, e.g. "var + 1" (basically "var.get() + 1"), which means that those operations need to be defined in the classes, but one operation that cannot be defined as an operator is setting the value. It's not possible to write "var.set(var.get() + 1)" as, say, "var <== var + 1". I'm not sure what I think about that, but given that you're willing to define a whole load of other operators... Hmm... Not sure.

On 26/05/2019 17:04, Yanghao Hua wrote:

On Sun, May 26, 2019 at 11:27 AM Barry Scott <barry@barrys-emacs.org> wrote:

...

I think you are confusing the number of people that use HDL with the amount of product created.

I don't see how I did that but if you intercepted that way I must have done that somehow.

...

Also I was under the impression that HDL tools exist already that are considered usable and yet do not need python.

What is the problem that you aim to solve with a python HDL tool?

I think the question should be which part of existing HDLs does not need to be fixed? The answer would be easier: the assignment syntax is pretty elegant. I would recommend to take a look at Chisel, all the motivations for creating Chisel is pretty much the same reason I would create a python equivalent and I had a prototype shows in some area it could even be better.

You do realise that repeatedly avoiding the question "what problem do you think you are solving?" does not convincingly make the case that there is a problem to be solved, don't you? -- Rhodri James *-* Kynesim Ltd

On Wed, May 29, 2019 at 6:30 AM Yanghao Hua <yanghao.py@gmail.com> wrote:

...

You do realise that repeatedly avoiding the question "what problem do you think you are solving?" does not convincingly make the case that there is a problem to be solved, don't you?

It might be that I am not good enough yet to present it in a way for some of you to better comprehend it. But I do see positive feedbacks too. I plan to stop this discussion here and make a toe-to-toe comparison how it looks like without assignment overloading, how it looks like with assignment overloading, and how it compares with other languages like verilog and Scala/Chisel, and in the meantime to look into some corner cases to make sure the proposed solution can cover all situations.

To repeat what the problem do I think I am solving? A variable, that behaves like an integer (e.g. all normal integer ops should just work), but has a different assignment behavior, such that it can be used to develop equally good hardware descriptions.

So what you want is for this code to behave very very weirdly: a = thing() a = other_thing() a = another_thing() Because if one of the things redefines assignment, the subsequent ones are going to do something different. I recommend sticking with something that's deliberately and consciously namespaced. Descriptors give you a convenient way to do this with dot notation, and a custom object with a __setitem__ method lets you use bracket notation. stuff = thing_collection() stuff.a = other_thing() stuff.a = another_thing() This is much clearer, because you can see a distinct difference between initializing the thing and making the thing do stuff. Alternatively, using a different operator works: a = thing() a += other_thing() a += another_thing() because augmented assignment IS handled by the target object. (I don't recommend abusing <= for this, as people and linters will get very surprised by that. But that would also work.) Redefining assignment is a minefield in the languages that support it. Please can you seek a better way to do this? I'm done posting now. If you're not listening, I'm not going to keep talking. ChrisA

On 5/28/2019 4:29 PM, Yanghao Hua wrote:

To repeat what the problem do I think I am solving? A variable, that behaves like an integer (e.g. all normal integer ops should just work), but has a different assignment behavior, such that it can be used to develop equally good hardware descriptions.

This is the part that you're not explaining: what does "a different assignment behavior" mean? We all understand what Python means by assignment (name binding), but we don't understand what you would like it to be instead. It seems that you want these two statements to work differently: x = something() # bind a name for the first time # ("create a variable", if you will) x = 4 # do something different when x already exists # and is of some special type Is that true? What is the "something different"? I've got to be honest with you: I don't see Python changing in this regard. Eric

On Tue, May 28, 2019 at 10:46 PM Ricky Teachey <ricky@teachey.org> wrote:

I don't want to pile on, but: you have also not given any actual reason why overloading the = operator using descriptors:

stuff.a = 8

...is not sufficient, other than a general "well I don't like that". Come on, guy. If you can't give a real actual reason why that won't work for HDL programming, you can be sure as heck nobody is ever going to listen to your idea of changing the way the language behaves.

Have you even attempted to write a descriptor that handles the assignment of signals the way you want and tried it out for a bit?

Yes I have, and none of them working in a way that can easily reflect the hierarchical nature of hardware design. Suppose you have class A and class B, and a third class C want to use class A and B, the very natural way that all hardware engineer would construct this is like below: class C: def __init__(self, in, out, ...): # suppose A has one input and two output, B has two input and one output # so A and B can be chained together sig0 = signal() sig1 = signal() a = A(in, sig0, sig1) b = B(sig0, sig1, out) Descriptor does not work as a local variable, it does not even work if you instantiate it in the __init__() function, it has to be declared as a class attribute for it to work. And when you passing it around, it will not work, unless you pass class C's self around ... which is really not pleasant to read (I had this example already in previous postings). If anyone could teach me how to do hardware hierarchical design as elegant as in verilog or Chisel, I'd be happy to use it. What I see is, myhdl is using signal.next = something, cocotb is only for verification so ... not really relevant. In short, I haven't found any evidence out there that this can be done naturally (e.g. as in verilog or Chisel). You can argue that why do I even want to define verilog/chisel's way is the natural way, well, this is I believe the common understanding in hardware design community, and this is the reason why new HDL like Chisel choose Scala to implement exactly the same thing. But anyone who could propose a even simpler syntax that can describe the same thing with less chars I'd definitely be happy to take it.

On Tue, May 28, 2019 at 05:45:54PM -0400, Ricky Teachey wrote:

As things stand right now, it seems like you don't understand what we are saying about using descriptors to accomplish this. However it is certainly possible I simultaneously don't understand what you are really trying to do in the code you wrote (I definitely don't claim to).

Descriptors are, I think, a total red herring. Yanghao Hua wants to customise the behaviour of assignment. I believe that he wants to emulate the behaviour of some hardware description languages, where the equals sign = doesn't mean assignment (if I have understood correctly, which I may not have), but Something Else. Yanghao Hua can write a method to perform that Something Else, but he wants to use the standard variable assignment syntax: x = expression but there is no hook to customise the behaviour, variable assignment is built-into the language. But *attribute assignment* is different: obj.x = expression may call either the descriptor protocol, or simply call type(obj).__setattr__, which gives us the opportunity to hook into attribute assignment. Likewise for item assignment: obj[x] = expression which calls type(obj).__setitem__. So Yanghao Hua has lots of options: 1. Choose a different operator. The obvious choice is << if his object is not already using it. 2. Use a method: x.set(expression) 3. Use a pre-processor, like Coconut or "Like, Python!". He may need to write his own pre-processor, but that may not be very difficult if all it does is convert "x = ..." into a method call. 4. Write an interpreter that does what he likes: code = """ x = expression """ result = interpret(code) 5. It may be possible to play games with the interpreter by using metaclasses, replacing modules with custom objects, etc. But I doubt that will change the behaviour of assignment inside the module itself. 6. Check out MacroPy and see whether its macros will help. 7. Use a custom interpreter. There is at least one superset of Python in common use: Cython. 8. The most obvious solution is to just use an instance and attribute: obj.x = expression will work fine, but he just doesn't like the look of it. Aesthetics are an important part of language design, but it is only one factor out of many. All languages make compromises, and sometimes compromising on aesthetics is one of them. In Python, the compromise is that we can't hook into plain old assignment; but what we gain is certainty (we always know what x=... does) and performance. On the flip side, we *can* customise attribute assignment obj.x=... but what we lose is certainty and performance: obj.x = ... is slower, because it has to look up the __setattr__ method, or call the descriptor protocol, and then call custom code; and we can't know what it will do. ``obj.x = 1`` can call arbitrary code, so it could do literally anything. Extending that ability to hook assignment to plain old variables is unlikely. It would need a PEP. Adding a new operator like <== is less unlikely, but it too would need a PEP, and the big question that everyone will ask is "why not just use << instead?" The bottom line is, Python is not Lisp or Forth, where the very syntax itself can be customised. Regular assignment counts as syntax: it's unlikely to change, without a very good reason. -- Steven

Steven D'Aprano wrote:

Yanghao Hua wants to customise the behaviour of assignment. I believe that he wants to emulate the behaviour of some hardware description languages, where the equals sign = doesn't mean assignment (if I have understood correctly, which I may not have), but Something Else.

Maybe I can help a bit here. I don't have a deep knowledge of HDLs. but from what I gather, they typically have two *different* assignment-like operations. One of them is like assignment in a functional language, where you're defining a signal as a function of other existing signals. This corresponds to combinatorial logic in hardware. Python's existing assigment is fine for this, I think. The other represents updating a signal when some event occurs, such as a clock transition. In hardware terms, it corresponds to changing the state of a flip-flop. The OP wants a new operator and associated dunder method to represent this operation. He doesn't want to override normal assignment. -- Greg

On 5/30/19 4:09 AM, Yanghao Hua wrote:

On Thu, May 30, 2019 at 12:50 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

...

Steven D'Aprano wrote:

...

Yanghao Hua wants to customise the behaviour of assignment. I believe that he wants to emulate the behaviour of some hardware description languages, where the equals sign = doesn't mean assignment (if I have understood correctly, which I may not have), but Something Else. Maybe I can help a bit here. I don't have a deep knowledge of HDLs. but from what I gather, they typically have two *different* assignment-like operations. One of them is like assignment in a functional language, where you're defining a signal as a function of other existing signals. This corresponds to combinatorial logic in hardware. Python's existing assigment is fine for this, I think.

The other represents updating a signal when some event occurs, such as a clock transition. In hardware terms, it corresponds to changing the state of a flip-flop. The OP wants a new operator and associated dunder method to represent this operation. He doesn't want to override normal assignment. That's basically right. And three small addition/correction: 1. Actually combinational logics can be modeled with both blocking and non-blocking logic. So one of them is enough already. 2. python's assignment is having very different semantics, so even for the verilog blocking assignment, which need to make the current "thread/process" to wait until the assignment finishes, is still not possible to model with python's assignment operator. 3. the signal change is really modeled on gate level, which are AND/OR/XOR/MUX gates. There are optimized flip-flop entities too but basically you can also build FFs from the primary gates. And the change of the signal, is actually just change the input of the FFs, not really the FF states itself. FF state change is modeled and inferred with a sensitivity list (a list of signals which triggers a thread/process to run) that has a clock.

As someone who does hardware design with HDLs, I would say that to think that it would be easy to recreate the abilities of them in a 'conventional' programming language doesn't really understand at least one of the languages, as there is a SIGNIFICANT difference in there fundamental mode of operation. The one key thing here is that with non-blocking assignment, the order they are written is totally not important. I can write: x<= y; y <= z; or y <= z; x <= y; ( <= is the non-blocking assignment operator in verilog), and in both instances get exactly the same result, namely that x gets the value of y BEFORE either of those statements occurred. Basically, all the non-blocking assignments happen in parallel through the whole design, and when that assignment happens is controlled by the context of where that assignment is written (it might be conditioned on a clock edge, of by an if statement to only happen when some other combination of signals occur) There ARE programs that will take a HDL program and convert it into a conventional programming language, and other programs to take a design written in a conventional programming language and convert it to HDL, and these mostly work as long as you started off remembering their restrictions when you started writing the original program. To expect that you could get anything similar by just defining a few operators to deal with thing in a special manner seems unlikely. I think that I personally would have a hard time understanding a program written where many of the assignments behave sequentially like is normal in a conventional programming language but many other behave concurrently as in HDL. I would think it would be much clearer if you wanted to simulate the action of hardware, to first have 'definition' stage where you defined the interconnection of the signals within a system (and here you would naturally be dealing with methods and attributes of some object that represents the hardware system), and then after defining that, using some method to 'run' that hardware. Thus having the above be written as hardware.x = hardware.y hardware.y = hardware.x and then later: hardware,run() where the attribute assignment recorded the interconnections and the run did them. I think this is all possible currently in Python where hardware is an object of an appropriatly written class. -- Richard Damon

On Tue, May 28, 2019 at 10:40 PM Eric V. Smith <eric@trueblade.com> wrote:

On 5/28/2019 4:29 PM, Yanghao Hua wrote:

...

To repeat what the problem do I think I am solving? A variable, that behaves like an integer (e.g. all normal integer ops should just work), but has a different assignment behavior, such that it can be used to develop equally good hardware descriptions.

This is the part that you're not explaining: what does "a different assignment behavior" mean? We all understand what Python means by assignment (name binding), but we don't understand what you would like it to be instead.

a different assignment behavior in HDL is your assignment does not take effect until a delta cycle of zero virtual time has passed. (did you really looked at the previous postings? :)

It seems that you want these two statements to work differently:

x = something() # bind a name for the first time # ("create a variable", if you will) x = 4 # do something different when x already exists # and is of some special type

Is that true? What is the "something different"?

x = 4 should be something like x <== 4 or x := 4 (the latter has been taken by the assignment expressions though ...). Such that variable initialization (=) and utilization (<==) can be differentiated.

I've got to be honest with you: I don't see Python changing in this regard.

Sad to see that's the case from you when it seems you barely even read all of the postings in this thread.

On Wed, May 29, 2019 at 3:15 AM Eric V. Smith <eric@trueblade.com> wrote:

Yes, I’ve read every one of the emails in this thread, many of them multiple times.

Python does not know what “a delta cycle of zero virtual time has passed” means, so there’s no way of implementing this feature. If indeed you’re serious about this feature, you would do yourself a favor by explaining it in terms of Python, not in terms of HDL.

Python does not need to know this ... just hand it over to end user who knows how to implement such a thing. Python need to provide the mechanism.

...

...

It seems that you want these two statements to work differently:

x = something() # bind a name for the first time # ("create a variable", if you will) x = 4 # do something different when x already exists # and is of some special type

Is that true? What is the "something different"?

x = 4 should be something like x <== 4 or x := 4 (the latter has been taken by the assignment expressions though ...). Such that variable initialization (=) and utilization (<==) can be differentiated.

I take this as an answer of “yes” to my question of the two assignments being different. If that’s so, then I’m positive this feature will not be implemented.

I respect your view however I cannot agree with it.

...

...

I've got to be honest with you: I don't see Python changing in this regard.

Sad to see that's the case from you when it seems you barely even read all of the postings in this thread.

It’s true that I’ve tried and failed to understand what behavior you’re suggesting be added to Python. Good luck going forward, but I’m dropping t out of the conversation.

Thanks for trying and all the feedbacks.

On Fri, May 31, 2019 at 3:48 PM Rhodri James <rhodri@kynesim.co.uk> wrote:

It really doesn't. If the end user is going to implement the logic of this anyway, implementing signal linkage as a method call or class all of its own is not a significant extra burden.

There are very constructive discussions in terms how this could be handled and which turns out not really elegant. Your justification could be used to reject features like descriptors, decorators, meta classes, and you can use that reasoning even just to say "let's all go java". This is not the pythonic for me at all. the burden, is not only on the savings of a few chars, it is on how easy/difficult it can be interpreted and understood.

I'm pretty much done with this conversation too. You have repeatedly been asked what problem you are trying to solve and repeatedly respond by restating your solution, which appears to be to impose HDL sematics onto a non-HDL language. That's never going to be a good idea.

Please let's be constructive and be specific, this kind of conclusion without any reasoning behind it not only makes it weak, this is not the healthy type of discussion at all.

On Tue, Jun 4, 2019 at 10:11 AM Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Yanghao Hua writes:

...

On Fri, May 31, 2019 at 3:48 PM Rhodri James <rhodri@kynesim.co.uk> wrote:

...

...

It really doesn't. If the end user is going to implement the logic of this anyway, implementing signal linkage as a method call or class all of its own is not a significant extra burden.

...

There are very constructive discussions in terms how this could be handled and which turns out not really elegant. Your justification could be used to reject features like descriptors, decorators, meta classes,

And has been so used. The question is always "does the use case justify increasing complexity for a couple hundred maintainers and a few hundred million readers?"

That's a valid question. and let's address the items one by one. Complexity & hundreds of maintainers. It is nothing complex and it is almost a direct copy of the implementation of @= operator, all logic behind <== are the same of @=. With some refactoring @= and <== implementation might even be able to share 99% of the code, so I think it is not complex at all, and there is not much additional overhead for maintenance. hundred million readers. I can only guess you mean python developers here? Just like @=, <== is not enforced on anyone, but can be used by anyone who wants to. As it is today I can choose to give @= a complete different meaning if I chose to, other uses may chose (and most importantly, being able to) give <== the meaning they see fit when they want to preserve all number operators and matrix operators. It is merely a new possibility for end user with very limited effort. I am glad that this question comes, which means at least the python community is no longer denying there is a problem (a small one by the way), and it can be improved in terms of allowing end user to write more readable code in this specific domain.

As far as I can tell, here we have (a) *one* person *strongly* in favor of adding just one new operator token, who justifies it based on *personal* preference over style of expression for *one* specific DSL, and (b) a few supporting a generic facility for adding new operators.

The former (a) is obviously insufficient, considering past practice in Python. One person can probably use MacroPy or a preprocessor. If the DSL becomes popular, as with "@", first applied for matrix multiplication, it might become justification for a new operator token, but the popularity has to come before more than a tiny fraction of committers (if any) will sign on. The other possibility would be to present a complete implementation of a toy language using existing Python facilities, and then show the alternative with the new operator. Preferably at the same time showing how the preprocessor/ MacroPy strategy falls short.

Will do.

The latter (b) has been discussed and rejected a number of times, on the grounds that unfamiliar symbols are *harder* to read than functions, and that Python is not in the DSL-producing industry. Some features are clearly more descriptive than procedural in nature (eg, comprehensions), but they are justified on the ground that they increase readability when expressing algorithms.

Understood.

...

Please let's be constructive and be specific, this kind of conclusion without any reasoning behind it

In Python practice, the conclusion of "reject" needs no reasoning, unfortunately. Python adds new features in response to a demonstrated need that overcomes a general conservatism, a conservatism that extends far past backward compatibility to a certain amount of forward compatibility as well.

So you should take your own advice. We have seen you express your *preference* for a new operator numerous times, but there is no *visible* logic behind it. Show us the best you can do with actual working hardware description classes (say a NAND gate and the composition into a half-adder, or perhaps you need more to demonstrate the relevant difficulties -- I know little about hardware, quite a bit about how the Python enhancement process works). Then show us the syntax you want to use to express the same thing.

I will come up with a complete working implementation of HDL-in-Python with <== to show the differences, how it enhances readability, and how it compares without it.

It's possible that people with wizard-level Python skills can find a way around the difficulties you describe with descriptors and other features that have been suggested in this thread. If not, your case for a new operator token would be much stronger (though still insufficient IMO -- fortunately for you, few committers listen to me about language features ;-).

I'd like to take any recommendations that can achieve the same logical integrity and readability.

Without such a test, or a few tens of thousands of hardware designers lobbying for a decade or so (which is what happened with the numerical community, although this time around I bet Python will be substantially more forward-leaning, say 5 years and 5000 hardware designers), it's your taste against Python's conservatism. I'll take long odds that conservatism wins.

Thanks for the advice, I will (I was) working on it and will show you the ideas behind it. I knew it is going to be difficult and probably I have failed to use some cool features and I think a completely working system will help to demonstrate the ideas better. Stay tuned.

On Fri, May 31, 2019 at 02:48:24PM +0100, Rhodri James wrote:

On 29/05/2019 08:31, Yanghao Hua wrote:

...

Python does not need to know this ... just hand it over to end user who knows how to implement such a thing. Python need to provide the mechanism.

It really doesn't. If the end user is going to implement the logic of this anyway, implementing signal linkage as a method call or class all of its own is not a significant extra burden.

That's not really fair: syntax matters, and for people working in a domain where certain syntax is expected, asking them to use something different is a significant cognitive burden. We don't ask people doing arithmetic to write code like this: x.mul(y.sub(1)) # a * (y - 1) There have been at least two times that Python has added syntax to the language to allow a third-party library to write more-idiomatic code in their domain. Both extended slicing seq[a:b:c] and the matrix- multiplication operator were added for numpy. So *in principle* we certainly could add a new arrow operator for Yanghao Hua so his libraries and code will be more expressive and idiomatic in his domain. But *in practice*, the hard truth is this: - Yanghao Hua is one developer interested in HDL in Python; - numpy was, and still is, one of the most important "killer apps" responsible for Python's success. Adding syntax for the benefit of numpy helps millions of users; adding syntax for HDL-like helps... how many people? Twenty? A hundred? Millions of users will have to learn the syntax. Unless they get some benefit, why consider this? (But having said that... I reckon that if we had left and right arrow operators, <== and ==>, I have some DSLs where they would work for me too. Maybe.)

I'm pretty much done with this conversation too. You have repeatedly been asked what problem you are trying to solve and repeatedly respond by restating your solution, which appears to be to impose HDL sematics onto a non-HDL language. That's never going to be a good idea.

That's totally unfair to Yanghao Hua on two levels. (1) He has never asked for the interpreter to support the semantics he wants. He's not asking for Python to understand and implement HDL semantics in the language: he can do that himself, in the class. (2) He has explained the problem he is trying to solve: he wants to write a DSL using syntax which doesn't look like crap from the perspective of people in that domain. He just wants an operator that looks kinda like assignment that calls a dunder. He can't use assignment because the semantics are baked into the language (and there's no chance of that changing). He can't use existing operators because they're already in use. This is not an unreasonable request. Python has bowed to similar requests at least twice before, and honestly, it's not like a <== operator would be weirder than other operators in use in mainstream languages. We're not talking APL or J here. On the other hand... its not clear that, as small as this request is, the language would be better. There's lots of other domains where Python syntax is sub-optimal: - We don't have good syntax for writing XML, or calling SQL. We just have to use strings. - No nice syntax for writing Prolog-style deductive code; - or concatenative DSLs; - or natural language DSLs like Hypertalk or Inform. Python can't be all things to all people. -- Steven

Correct me if I am wrong: Yanghao would like an elegant way to build graphs. Simply using << to declare the connections in the graph [1] is not an option because << is already needed for legitimate left-shift operation.  The problem is not assignment; rather, Yanghao's HDL requires more operators than Python has in total (or at least the remaining operators look terrible). Here is a half baked idea: Instead of solving the problem of operator-needed-for-use-case-X, can we solve the bigger problem of adding user-defined operators? "User-defined operators" are a limited set of operators, with no preconceived definition, and can be defined by writing the appropriate dunder method.  User-defined operators match /[!@$%^-+=&*:<>?/|]{2,4}/ and necessarily exclude all the existing python-defined operator combinations.  Maybe 50K operators is easier to add to Python than a few pre-defined operators? Example:

class A:      def assign(self, other):          # whatever this means

To add a user defined operator to a class, you add it as a method, exactly the name of the operator:

setattr(A, "<==", A.assign)

I admit this is a bit ugly, but some decorator can make this nicer:

class A:      @operator("<==")      def assign(self, other):          # whatever this means

Maybe the biggest problem would be with linting; not being able to distinguish between a typo and a user-defined operator until runtime, when no implementation is found. A ++= B  # is this typo? Abusive use of operators will likely happen; but the codebases that do will not be popular because of the learning curve. Instead, experiments will be done, some will succeed, and the best operators for each domain will become popular.  Then the python devs can pick the winners to include in the core library, if any. [1] Here is toy example that will build a graph using << and >>: https://github.com/klahnakoski/graph-builder/blob/master/graph_builder.py On 2019-06-04 08:28, Steven D'Aprano wrote:

On Fri, May 31, 2019 at 02:48:24PM +0100, Rhodri James wrote:

...

...

Python does not need to know this ... just hand it over to end user who knows how to implement such a thing. Python need to provide the mechanism. It really doesn't. If the end user is going to implement the logic of

On 29/05/2019 08:31, Yanghao Hua wrote: this anyway, implementing signal linkage as a method call or class all of its own is not a significant extra burden. That's not really fair: syntax matters, and for people working in a domain where certain syntax is expected, asking them to use something different is a significant cognitive burden. We don't ask people doing arithmetic to write code like this:

x.mul(y.sub(1)) # a * (y - 1)

There have been at least two times that Python has added syntax to the language to allow a third-party library to write more-idiomatic code in their domain. Both extended slicing seq[a:b:c] and the matrix- multiplication operator were added for numpy.

So *in principle* we certainly could add a new arrow operator for Yanghao Hua so his libraries and code will be more expressive and idiomatic in his domain.

But *in practice*, the hard truth is this:

- Yanghao Hua is one developer interested in HDL in Python;

- numpy was, and still is, one of the most important "killer apps" responsible for Python's success.

Adding syntax for the benefit of numpy helps millions of users; adding syntax for HDL-like helps... how many people? Twenty? A hundred?

Millions of users will have to learn the syntax. Unless they get some benefit, why consider this?

(But having said that... I reckon that if we had left and right arrow operators, <== and ==>, I have some DSLs where they would work for me too. Maybe.)

...

I'm pretty much done with this conversation too. You have repeatedly been asked what problem you are trying to solve and repeatedly respond by restating your solution, which appears to be to impose HDL sematics onto a non-HDL language. That's never going to be a good idea. That's totally unfair to Yanghao Hua on two levels.

(1) He has never asked for the interpreter to support the semantics he wants. He's not asking for Python to understand and implement HDL semantics in the language: he can do that himself, in the class.

(2) He has explained the problem he is trying to solve: he wants to write a DSL using syntax which doesn't look like crap from the perspective of people in that domain.

He just wants an operator that looks kinda like assignment that calls a dunder. He can't use assignment because the semantics are baked into the language (and there's no chance of that changing). He can't use existing operators because they're already in use.

This is not an unreasonable request. Python has bowed to similar requests at least twice before, and honestly, it's not like a <== operator would be weirder than other operators in use in mainstream languages. We're not talking APL or J here.

On the other hand... its not clear that, as small as this request is, the language would be better. There's lots of other domains where Python syntax is sub-optimal:

- We don't have good syntax for writing XML, or calling SQL. We just have to use strings.

- No nice syntax for writing Prolog-style deductive code;

- or concatenative DSLs;

- or natural language DSLs like Hypertalk or Inform.

Python can't be all things to all people.