[Tim]
But I wonder why it isn't done with a new framing opcode instead (say, FRAME followed by 8-byte count). I suppose that would be like the "prefetch" idea, except that framing opcodes would be mandatory (instead of optional) in proto 4. Why I initially like that:
- Uniform decoding loop ("the next thing" _always_ starts with an opcode).
But it's not actually uniform. A frame isn't a normal opcode, it's a large section of bytes that contains potentially many opcodes.
The framing layer is really below the opcode layer, it makes also sense to implement it like that.
That makes sense to me.
(I also tried to implement Serhiy's PREFETCH idea, but it didn't bring any actual simplification)
But it has a different kind of advantage: PREFETCH was optional. As Guido said, it's annoying to bloat the size of small pickles (which may, although individually small, occur in great numbers) by 8 bytes each. There's really no point to framing small chunks of data, right? Which leads to another idea: after the PROTO opcode, there is, or is not, an optional PREFETCH opcde with an 8-byte argument. If the PREFETCH opcode exists, then it gives the number of bytes up to and including the pickle's STOP opcode. So there's exactly 0 or 1 PREFETCH opcodes per pickle. Is there an advantage to spraying multiple 8-byte "frame counts" throughout a pickle stream? 8 bytes is surely enough to specify the size of any single pickle for half a generation ;-) to come.
When slinging 8-byte counts, _some_ sanity-checking seems like a good idea ;-)
I don't know. It's not much worse (for denial of service opportunities) than a 4-byte count, which already exists in earlier protocols.
I'm not thinking of DOS at all, just general sanity as data objects get larger & larger. Pickles have almost no internal checks now. But I've seen my share of corrupted pickles! About the only thing that catches them early is hitting a byte that isn't a legitimate pickle opcode. That _used_ to be a much stronger check than it is now, because the 8-bit opcode space was sparsely populated at first. But, over time, more and more opcodes get added, so the chance of mistaking a garbage byte for a legit opcode has increased correspondingly. A PREFETCH opcode with a "bytes until STOP" makes for a decent bad ;-) sanity check too ;-)