2.6, 3.0, and truly independent intepreters

[Andy O'Meara]

> > And in the case of hundreds of megs of data
>
> ... and I would be surprised at someone that would embed hundreds of
> megs of data into an object such that it had to be serialized... seems
> like the proper design is to point at the data, or a subset of it, in a
> big buffer.  Then data transfers would just transfer the offset/length
> and the reference to the buffer.
>
> > and/or thousands of data structure instances,
>
> ... and this is another surprise!  You have thousands of objects (data
> structure instances) to move from one thread to another?
>

I think we miscommunicated there--I'm actually agreeing with you.  I
was trying to make the same point you were: that intricate and/or
large structures are meant to be passed around by a top-level pointer,
not using and serialization/messaging.  This is what I've been trying
to explain to others here; that IPC and shared memory unfortunately
aren't viable options, leaving app threads (rather than child
processes) as the solution.


> Of course, I know that data get large, but typical multimedia streams
> are large, binary blobs.  I was under the impression that processing
> them usually proceeds along the lines of keeping offsets into the blobs,
> and interpreting, etc.  Editing is usually done by making a copy of a
> blob, transforming it or a subset in some manner during the copy
> process, resulting in a new, possibly different-sized blob.


Your instincts are right.  I'd only add on that when you're talking
about data structures associated with an intricate video format, the
complexity and depth of the data structures is insane -- the LAST
thing you want to burn cycles on is serializing and unserializing that
stuff (so IPC is out)--again, we're already on the same page here.

I think at one point you made the comment that shared memory is a
solution to handle large data sets between a child process and the
parent.  Although this is certainty true in principle, it doesn't hold
up in practice since complex data structures often contain 3rd party
and OS API objects that have their own allocators.  For example, in
video encoding, there's TONS of objects that comprise memory-resident
video from all kinds of APIs, so the idea of having them allocated
from shared/mapped memory block isn't even possible. Again, I only
raise this to offer evidence that doing real-world work in a child
process is a deal breaker--a shared address space is just way too much
to give up.


Andy