On 26/03/07, Josiah Carlson
But really, transferring little bits of data back and forth isn't what is of my concern in terms of speed. My real concern is transferring nontrivial blocks of data; I usually benchmark blocks of sizes: 1k, 4k, 16k, 64k, 256k, 1M, 4M, 16M, and 64M. Those are usually pretty good to discover the "sweet spot" for a particular implementation, and also allow a person to discover whether or not their system can be used for nontrivial processor loads.
The "20,000 fetches/sec" was just for retreving a "small" object (an integer), so it only really reflects the server overhead. (Sending integer objects directly between processes is maybe 6 times faster.) Fetching string objects of particular sizes from a shared dict gives the following results on the same computer: string size fetches/sec throughput ----------- ----------- ---------- 1 kb 15,000 15 Mb/s 4 kb 13,000 52 Mb/s 16 kb 8,500 130 Mb/s 64 kb 1,800 110 Mb/s 256 kb 196 49 Mb/s 1 Mb 50 50 Mb/s 4 Mb 13 52 Mb/s 16 Mb 3.2 51 Mb/s 64 Mb 0.84 54 Mb/s