Winter Madness - Passing Python objects as Strings

[Hendrik van Rooyen]

 "Gabriel Genellina" <ga..y2 at yahoo.com.ar> wrote:

>Ah... I had the same impression as Mr. Reedy, that you were directly  
>reading from a socket and processing right there, so you *had* to use  
>strings for everything.

not "had to" - "chose to" - to keep the most used path as short as I could.

>
>But if you already have a queue, you may put other objects there (instead  
>of "canning" them). Testing the object type with isinstance(msg, str) is  
>pretty fast, and if you bind locally those names I'd say the overhead is  
>negligible.

Maybe you are right and I am pre optimising - but the heart of this 
box really is that silly loop and the processor really is not fast at all.
I felt it was right to keep the processing of the stuff coming out of 
the queue standard with the split(','), as the brainless way seemed
to be the best - anything else I could think of just added overhead.
I thought of putting the string in a list, with the record type being the 
first item, and the string the second, with a queue replacing the string
for the state change record.  I basically rejected this as it would have
added extra processing both at the entry and the exit of the critical
queue, for every record.

I admit that I did not think of testing the type with isinstance, but even 
if the overhead is minimal, it does add extra cycles to the innermost loop,
for every one of the thousands of times that nothing of importance is detected.
This is what I was trying to avoid, as it is important to get as much 
performance out of the box as I can (given that I am using Python to
get the job done fast, because that is also important). So it is a kind of
juggling with priorities - "make it fast" would imply do not use python,
but "get it done quickly" implies using python, and it looks to me that
if I am careful and think more like an assembler programmer, the
real time performance will be adequate.  And I do not want to do 
anything that could conceivably compromise that.  Even if it means
jumping through a funny hoop like I am doing now, and inventing
a weird way to pass an object.

"adequate" here is up to now quite good - if I set up a client on the 
LAN and I reflect the inputs back to the outputs, then to my human 
senses there is no difference in the way the output relays chatter and 
bounce when I play with a wire on the inputs, to what I would expect
of a locally hard wired setup.

So to a large extent I think that I am doing the job as fast as it is 
possible - the comma delimited input string is a fact, decided on
between myself and the customer a long time ago, and it comes over
a socket, so it has to be a string.  The least I can do with it is nothing
before I put it on the critical queue. Then when it comes out of the 
queue, I have to break it up into its constituent parts, and I would think
that split is the canonical way of doing that. Then there follows some
jiggery pokery to get the outputs out and the inputs in, that involves some
ctypes stuff to address the real hardware, and then the input results
have to be sent back to the client(s), over and over.
That is basically what the box does, until another connection is made
(from a control room) and the local "master" client is pre empted and 
the outputs from the new "master" must be obeyed, and the results
reflected back to the new connection too.  It is a fairly simple state
machine, and it has to be as fast as possible, as twice its loop time
plus the network round trip time defines its responsiveness.

I would really appreciate it if someone can dream up a faster way
of getting round this basic loop.  (even if it involves other equally
weird constructs as "canning")

- Hendrik