<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>Hello,</div><div>Yeah, I read data from a file say at each node and each time step, but when i try to use Marshal approach i get gibberish but when i use simple iter i get correct values. i have been trying the approach used in example in the previous post and that example makes sense but it doesnt make sense when i use it in my case. I am right now assigning it to a variable, i am now thinking of exploring the possibility of saving data to a dot net array maybe using System.Array and saving data to it but not sure if that even make sense. <br><br>Sent from my iPhone</div><div><br>On 28 Oct 2014, at 12:50 am, Bradley Friedman <<a href="mailto:brad@fie.us">brad@fie.us</a>> wrote:<br><br></div><blockquote type="cite"><div><meta http-equiv="Content-Type" content="text/html charset=utf-8"><div class="">You indicate that you are reading from a file. The thread you reference was about copying data in memory. I’d think matters of buffering and read-ahead caches would be far more relevant than anything else. Am I missing something?</div><br class=""><div><blockquote type="cite" class=""><div class="">On Oct 26, 2014, at 5:18 AM, Nikhil Garg <<a href="mailto:nikhilgarg.gju@gmail.com" class="">nikhilgarg.gju@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><br clear="all" class=""><div class="">Hi,</div><div class=""><br class=""></div><div class="">I have looked at the following thread to copy a c# array to numpy list using python dot net.</div><div class=""><br class=""></div><div class=""><a href="https://mail.python.org/pipermail/pythondotnet/2014-May/001525.html" class="">https://mail.python.org/pipermail/pythondotnet/2014-May/001525.html</a><br class=""></div><div class=""><br class=""></div><div class="">When I try the methods mentioned in the complete thread I manage to get the array from simple iteration method or </div><div class="">np.fromiter, but I have not been able to make GCHandle or Marshal.Copy work for my case.</div><div class=""><br class=""></div><div class="">To explain about my issue, I use a computer code written in c# which produces output in a custom build format. I am able to access the data from the file system using python dot net, but it seems that it take almost half an hour to get the data from the file.</div><div class=""><br class=""></div><div class="">For example my file has an array of size 5018x73x400, if i use <b class="">fromiter, </b>i provide the last bit of array to copy to python due to the way file stores output.</div><div class=""><br class=""></div><div class=""><div class="">for j in xrange(num_nodes-1):</div><div class=""><span class="" style="white-space:pre"> </span>for i in xrange(nsteps):</div><div class=""><span class="" style="white-space:pre"> </span>temp = file_in.ReadItemTimeStep(j+1,i).Data</div><div class=""><span class="" style="white-space:pre"> </span>temp_1 = np.fromiter(temp, float)</div><div class=""><span class="" style="white-space:pre"> </span>dest[j, i] = temp_1</div></div><div class=""><br class=""></div><div class="">Other way of doing same task would be to use a for loop but tht would be really slow.</div><div class=""><br class=""></div><div class="">for k in xrange(num_nodes-1):</div><div class=""> for k in xrange(nsteps):</div><div class=""> for i in xrange(nvals):</div><div class=""> temp[k,j,i] = file_in.ReadItemTimeStep(k+1, j).Data[i]</div><div class=""><br class=""></div><div class="">I am trying to find a faster way to extract the data from the file. I was wondering if there is a possibility to use something like GChandle or Marshal.Copy. I have also attached same output from Marshal.Copy</div><div class=""><br class=""></div><div class=""><div class="">Marshal.Copy(temp, 0, IntPtr.__overloads__[int](temp2.__array_interface__['data'][0]), len(temp))</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><div class=""> 0.00000000e+000, 0.00000000e+000, 0.00000000e+000,</div><div class=""> 0.00000000e+000, 3.13551399e-242, 4.20993645e-161,</div><div class=""> 8.71245319e-119, 8.38370432e-089, 2.60762781e-075,</div><div class=""> 1.92616374e-072, 2.89006184e-072, 3.06007265e-082,</div><div class=""> 3.81879776e-315, 0.00000000e+000, 0.00000000e+000,</div><div class=""> 0.00000000e+000, 1.03959661e-297, 7.22038448e-196,</div><div class=""> 3.05237954e-130, 2.58469346e-093, 2.29495911e-070,</div><div class=""> 3.74354820e-062, 1.74288060e-061, 2.13172553e-067,</div><div class=""> 2.74305386e-077, 0.00000000e+000, 0.00000000e+000,</div><div class=""> 0.00000000e+000, 0.00000000e+000, 1.02484756e-240,</div><div class=""> 1.70193770e-154, 1.30625320e-108, 1.16793762e-075,</div><div class=""> 2.96889340e-061, 7.84687103e-058, 1.16879604e-057,</div><div class=""> 5.56256478e-068, 4.06437857e-315, 0.00000000e+000,</div><div class=""> 0.00000000e+000, 0.00000000e+000, 1.11705239e-296,</div><div class=""> 1.46978156e-194, 1.98034372e-128, 3.79615170e-091,</div><div class=""> 5.88936509e-068, 1.85305929e-059, 2.16186001e-058,</div></div><div class=""><br class=""></div><div class="">Whereas the actual values stored in the array are </div><div class=""><div class=""> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,</div><div class=""> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,</div><div class=""> 0.00000000e+00, 0.00000000e+00, 5.66641200e-37,</div><div class=""> 1.24961600e-30, 5.34846800e-25, 1.72519600e-20,</div><div class=""> 1.49402200e-17, 3.27685300e-15, 4.10687500e-13,</div><div class=""> 1.87978200e-11, 2.01397900e-10, 8.83561100e-10,</div><div class=""> 2.14397600e-09, 2.02600300e-09, 2.47599800e-09,</div><div class=""> 2.15987100e-09, 4.82694100e-10, 1.18759300e-10,</div><div class=""> 3.32024100e-11, 0.00000000e+00, 0.00000000e+00,</div><div class=""> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,</div><div class=""> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,</div><div class=""> 1.45288600e-37, 7.37245900e-31, 7.59282500e-25,</div><div class=""> 5.85463400e-20, 1.20634700e-16, 3.06414300e-14,</div><div class=""> 5.16041300e-12, 2.87562800e-10, 3.62841500e-09,</div><div class=""> 1.63540600e-08, 4.06988300e-08, 3.97764200e-08,</div><div class=""> 4.88768500e-08, 4.07385000e-08, 8.78868600e-09,</div><div class=""><br class=""></div></div><div class="">I would appreciate any help i can get regarding this matter.</div><div class=""><br class=""></div><div class="">Nikhil</div>
</div>
_________________________________________________<br class=""><a href="http://Python.NET" class="">Python.NET</a> mailing list - <a href="mailto:PythonDotNet@python.org" class="">PythonDotNet@python.org</a><br class=""><a href="https://mail.python.org/mailman/listinfo/pythondotnet" class="">https://mail.python.org/mailman/listinfo/pythondotnet</a></div></blockquote></div><br class=""></div></blockquote><blockquote type="cite"><div><span>_________________________________________________</span><br><span><a href="http://Python.NET">Python.NET</a> mailing list - <a href="mailto:PythonDotNet@python.org">PythonDotNet@python.org</a></span><br><span><a href="https://mail.python.org/mailman/listinfo/pythondotnet">https://mail.python.org/mailman/listinfo/pythondotnet</a></span></div></blockquote></body></html>