On Thu, Sep 21, 2017 at 4:07 AM, ex vito <ex.vitorino@gmail.com> wrote:
On 2017-09-20, at 22:33, Bryan Murdock <bmurdock@gmail.com> wrote:
Welcome to the club! :)
Thanks!
Now I have a crazy idea. I'd like that Python code that my Verilog calls out to to use twisted. OK, actually I've already done that, my Verilog calls out to my that TCP server that I wrote. I had to spawn a thread and run the server in that thread so that the Verilog could continue to do stuff in parallel with the server. It's working great.
Can you clarify what "your Verilog" is? Python code running a Verilog simulation, maybe? Something else? <snip/> I do not fully understand this: if "Verilog calls out to TCP server" means it establishes a TCP connection (does it?) what does "Verilog calls out to a TCP client" mean? Does it call a function/method with async Twisted code? How does it handle the fact that the result is async?
Sorry, I was worried that this would be too hard to explain and/or I may be giving too many details and confusing the issue. Let me try again. It's probably simplest to think of the whole simulation as a TCP proxy. It looks like this: some TCP client process <--> Python TCP Server <--> Verilog <--> Python TCP Client <--> some TCP server The Verilog, Python TCP server, and Python TCP client are all running in the same process (some TCP client and some TCP server can be a web browser and web server or whatever). The Verilog code starts up, calls a python function to start the Python TCP Server as a thread, calls another python function to start the Python TCP Client as a thread, and then essentially just loops asking (polling) the Python TCP Server if it has any data, if so, it calls a function to send that data to the Python TCP Client. It does the same in reverse to flow data from client to server. I originally wrote the Python TCP Client and Server as low-level socket code, handling only one connection at a time and not automatically reconnecting if a connection was lost. All the downsides of using multiple processes instead of threads that you listed are exactly the reasons I used threads. Verilog has no support for spawning subprocesses or doing interprocess communication. It can't spawn threads either, to be fair though[1]. I have to do "real programming" things like that by calling python functions. Threads just seemed easier at the time because I can just use a Queue to get data to/from the server and client threads from/to the Verilog. And it all worked great. Then, when I went to add support for multiple connections and auto-reconnects I discovered twisted. I rewrote the Python TCP server and client with twisted but now data doesn't flow. I assume that's because the client and server are making thread-unsafe calls to the reactor. Is that true? (By the way, I can run the client and server standalone as separate processes, without verilog involved, and they work fine that way. I have also replaced the twisted Python TCP Client with my original socket-based TCP client and everything works fine. It's just when I try to use twisted for both client and server that it fails).
In general you are able to run concurrent (not parallel) code for TCP clients and servers within the same thread - after all, that's the key idea behind async programming with Twisted, with the reactor driving concurrent handling of multiple connections and events.
Right, I think the solution is going to have to be: have verilog spawn one thread that runs the Python TCP Server and Client. Or start them both as subprocesses and figure out how to communicate with them, I guess. Bryan 1. Aside from it's "green threads" that it uses to model concurrent digital logic