Hi,<div><br></div><div>I am developing a Python wrapper of the NASA CDF C library in Cython. I have got a working version of it now, but it is slower than the counterpart in IDL. For loading the same file, mine takes about 400 ms, whereas the IDL version takes about 290 ms. </div>
<div><br></div><div>The main overhead in my code is caused by a for-loop of element-by-element copying. Here is the relevant code in cython:</div><div>#-------------------------------- code -----------------------------------------------------</div>
<div><div>    #-- double</div><div>        realData = numpy.zeros(lenData, np_dtype)</div><div><br></div><div>        dblEntry = &lt;double *&gt;malloc(lenData * sizeof(double))</div><div>        status = CDFlib(</div><div>
                       SELECT_, zVAR_RECCOUNT_, numRecs,</div><div>                       NULL_)</div><div>        status = CDFlib(</div><div>                       GET_, zVAR_HYPERDATA_, dblEntry,</div><div>                       NULL_)</div>
<div>       <font color="#cc0000"> for ii in range(lenData):</font></div><div><font color="#cc0000">            realData[ii] = dblEntry[ii]</font></div><div>        realData.shape = np_shape</div><div>        free(dblEntry)</div>
</div><div>#--------------------------------- end of code -------------------------------------------</div><div>The time-consuming for-loop is highlighted in red. If I change <font color="#cc0000">range(lenData)</font> to <font color="#cc0000">range(lenData/2)</font>, the cpu time will be down from 400 ms to 230 ms for the case I mentioned above. Because the element-by-element copying for-loop seems pretty naive to me, I am wondering if there is a better way to copy data from the C array, dblEntry, to the numpy array, realData. </div>
<div><br></div><div>I tried the numpy C API <span style="background-color:rgb(255,255,255);font-family:monospace;font-size:16px;font-weight:bold">PyArray_NewFromDescr</span> with flag <span style="background-color:rgb(255,255,255);font-family:monospace;font-size:16px;font-weight:bold;line-height:20px;text-align:justify">NPY_ENSURECOPY</span>, but didn&#39;t get any luck. On the one hand, the flag above didn&#39;t seem to work as I expected, because I got memory deallocation failure error messages when I quitted ipython, where I tested my code, which I don&#39;t get if I use the naive for-loop. On the other hand, I can&#39;t figure out how to use <span style="background-color:rgb(255,255,255);font-family:monospace;font-size:16px;font-weight:bold">PyArray_NewFromDescr</span> correctly because the loaded data I got were not correct. Anyway, here is how I used <span style="background-color:rgb(255,255,255);font-family:monospace;font-size:16px;font-weight:bold">PyArray_NewFromDescr</span>:</div>
<div>#----------------------------------------- code ------------------------------------------</div><div>        cdef np.npy_intp dims[1]</div><div>        dims[0] = lenData</div><div><div>        realData = PyArray_NewFromDescr(numpy.ndarray, numpy.dtype(np_dtype),</div>
<div>                                        1, dims, NULL, &lt;void *&gt;dblEntry,</div><div>                                        NPY_CARRAY|NPY_ENSURECOPY, None)</div></div><div>        free(dblEntry)</div><div>#-------------------------------------- end of code --------------------------------------</div>
<div>BTW,  it can be compiled successfully by cython, in case you are wondering if the code had all the necessary pieces,</div><div><br></div><div>Thank you very much for reading. :-)</div><div><br></div><div>Cheers,</div>
<div>Jianbao</div><div><br></div><div><br></div><div><br></div>