On Sat, 31 Dec 2022 23:45:54 -0800 Bill Ross <bross_phobrain@sonic.net> wrote:
How best to write a 1D ndarray as a block of doubles, for reading in java as double[] or a stream of double?
Maybe the performance of simple looping over doubles in python.write() and java.read() is fine, but maybe there are representational diffs? Maybe there's a better solution for the use case?
Java is known to be big-endian ... but your CPU is probably little-endian. Numpy has the tools to represent an array of double BE.
Use case: I get the ndarray from keras, and it represents a 2D distance matrix. I want to find the top-50 matches for each item, per row and column. I'm looking at moving the top-50 task to java for its superior parallel threading. (Java doesn't fork processes with a copy of the array, which is ~5% of memory; rather one gets 1 process with e.g. 1475% CPU.)
What about numba or cython then ? Happy new year Jerome