Hello, Yeah, I read data from a file say at each node and each time step, but when i try to use Marshal approach i get gibberish but when i use simple iter i get correct values. i have been trying the approach used in example in the previous post and that example makes sense but it doesnt make sense when i use it in my case. I am right now assigning it to a variable, i am now thinking of exploring the possibility of saving data to a dot net array maybe using System.Array and saving data to it but not sure if that even make sense. Sent from my iPhone
On 28 Oct 2014, at 12:50 am, Bradley Friedman
wrote: You indicate that you are reading from a file. The thread you reference was about copying data in memory. I’d think matters of buffering and read-ahead caches would be far more relevant than anything else. Am I missing something?
On Oct 26, 2014, at 5:18 AM, Nikhil Garg
wrote: Hi,
I have looked at the following thread to copy a c# array to numpy list using python dot net.
https://mail.python.org/pipermail/pythondotnet/2014-May/001525.html
When I try the methods mentioned in the complete thread I manage to get the array from simple iteration method or np.fromiter, but I have not been able to make GCHandle or Marshal.Copy work for my case.
To explain about my issue, I use a computer code written in c# which produces output in a custom build format. I am able to access the data from the file system using python dot net, but it seems that it take almost half an hour to get the data from the file.
For example my file has an array of size 5018x73x400, if i use fromiter, i provide the last bit of array to copy to python due to the way file stores output.
for j in xrange(num_nodes-1): for i in xrange(nsteps): temp = file_in.ReadItemTimeStep(j+1,i).Data temp_1 = np.fromiter(temp, float) dest[j, i] = temp_1
Other way of doing same task would be to use a for loop but tht would be really slow.
for k in xrange(num_nodes-1): for k in xrange(nsteps): for i in xrange(nvals): temp[k,j,i] = file_in.ReadItemTimeStep(k+1, j).Data[i]
I am trying to find a faster way to extract the data from the file. I was wondering if there is a possibility to use something like GChandle or Marshal.Copy. I have also attached same output from Marshal.Copy
Marshal.Copy(temp, 0, IntPtr.__overloads__[int](temp2.__array_interface__['data'][0]), len(temp))
0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 3.13551399e-242, 4.20993645e-161, 8.71245319e-119, 8.38370432e-089, 2.60762781e-075, 1.92616374e-072, 2.89006184e-072, 3.06007265e-082, 3.81879776e-315, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 1.03959661e-297, 7.22038448e-196, 3.05237954e-130, 2.58469346e-093, 2.29495911e-070, 3.74354820e-062, 1.74288060e-061, 2.13172553e-067, 2.74305386e-077, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 1.02484756e-240, 1.70193770e-154, 1.30625320e-108, 1.16793762e-075, 2.96889340e-061, 7.84687103e-058, 1.16879604e-057, 5.56256478e-068, 4.06437857e-315, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 1.11705239e-296, 1.46978156e-194, 1.98034372e-128, 3.79615170e-091, 5.88936509e-068, 1.85305929e-059, 2.16186001e-058,
Whereas the actual values stored in the array are 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 5.66641200e-37, 1.24961600e-30, 5.34846800e-25, 1.72519600e-20, 1.49402200e-17, 3.27685300e-15, 4.10687500e-13, 1.87978200e-11, 2.01397900e-10, 8.83561100e-10, 2.14397600e-09, 2.02600300e-09, 2.47599800e-09, 2.15987100e-09, 4.82694100e-10, 1.18759300e-10, 3.32024100e-11, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.45288600e-37, 7.37245900e-31, 7.59282500e-25, 5.85463400e-20, 1.20634700e-16, 3.06414300e-14, 5.16041300e-12, 2.87562800e-10, 3.62841500e-09, 1.63540600e-08, 4.06988300e-08, 3.97764200e-08, 4.88768500e-08, 4.07385000e-08, 8.78868600e-09,
I would appreciate any help i can get regarding this matter.
Nikhil _________________________________________________ Python.NET mailing list - PythonDotNet@python.org https://mail.python.org/mailman/listinfo/pythondotnet
_________________________________________________ Python.NET mailing list - PythonDotNet@python.org https://mail.python.org/mailman/listinfo/pythondotnet