[Numpy-discussion] Efficient square distance computation
heng at cantab.net
Tue Oct 8 05:56:52 EDT 2013
On 08/10/13 09:49, Matthew Brett wrote:
> On Tue, Oct 8, 2013 at 1:06 AM, Ke Sun<sunk.cs at gmail.com> wrote:
>> >Dear all,
>> >I have written the following function to compute the square distances of a large
>> >matrix (each sample a row). It compute row by row and print the overall progress.
>> >The progress output is important and I didn't use matrix multiplication.
>> >I give as input a 70,000x800 matrix. The output should be a 70,000x70,000
>> >matrix. The program runs really slow (16 hours for 1/3 progress). And it eats
>> >36G memory (fortunately I have enough).
> That is very slow.
> As a matter of interest - why didn't you use matrix multiplication?
> On a machine I had access to it took about 20 minutes.
Out of interest, how did you do this with matrix multiplication?
More information about the NumPy-Discussion