why Segmentation fault (core dumped)?
Dear all, Previously I am able to run a script on our server but now it gave me a Segmentation fault (core dumped) error. try I tried the script with same type of netcdf file but with much smaller file size and it works. So I think the error is related with memory stuff. I guess it's because our system administrator make some change somewhere and that cause my problem? the file size that cause the error to appear is 2.6G (in the script I read this file with NetCDF4 to numpy array and make some manipulation), the small one without error is only 48M. I tried to use "limit" command to list the following, then I change stacksize to unlimited and the problem still occurs. ychao@obelix2 - ...CRU_NEW - 12 >limit cputime unlimited filesize unlimited datasize unlimited stacksize 10240 kbytes coredumpsize 0 kbytes memoryuse unlimited vmemoryuse unlimited descriptors 1024 memorylocked 64 kbytes maxproc 1024 would anybody be able to give me a short explanation or direct me to some webpage which can help to understand the problem? thanks et cheers, Chao -- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************
On 5/26/12 5:51 AM, Chao YUE wrote:
Dear all,
Previously I am able to run a script on our server but now it gave me a Segmentation fault (core dumped) error. try I tried the script with same type of netcdf file but with much smaller file size and it works. So I think the error is related with memory stuff. I guess it's because our system administrator make some change somewhere and that cause my problem? the file size that cause the error to appear is 2.6G (in the script I read this file with NetCDF4 to numpy array and make some manipulation), the small one without error is only 48M.
Chao: Without seeing your script, there's not much I can say. I suggest opening an issue at netcdf4-python.google.com, including your script as an attachment. You'll probably have to post the data file somewhere (dropbox perhaps?) so I can run the script that triggers the segfault. -Jeff
Dear Jeff, Thanks a lot for your reply. I think it might be related with the memory management on our sever. But anyway, as you suggested, I open an issue on netcdf4-python.google.co <http://netcdf4-python.google.com>m. you can find the data and script on ftp://ftp.cea.fr/incoming/y2k01/chaoyue/. The cal_cmi_big.py gave the core dumped error, in which I tried to operate with big files (~2G). The cal_cmi_small.py works fine, which handles data only ~25M (a subset of big files). I used a small function (which use NetCDF4) in the script to read and write nc data with NetCDF4. You can also find this function in ncfunc.py. I tested all the script and data before I upload on our ftp. thanks again for your help, cheers, Chao 2012/5/26 Jeff Whitaker <jswhit@fastmail.fm>
On 5/26/12 5:51 AM, Chao YUE wrote:
Dear all,
Previously I am able to run a script on our server but now it gave me a Segmentation fault (core dumped) error. try I tried the script with same type of netcdf file but with much smaller file size and it works. So I think the error is related with memory stuff. I guess it's because our system administrator make some change somewhere and that cause my problem? the file size that cause the error to appear is 2.6G (in the script I read this file with NetCDF4 to numpy array and make some manipulation), the small one without error is only 48M.
Chao: Without seeing your script, there's not much I can say. I suggest opening an issue at netcdf4-python.google.com, including your script as an attachment. You'll probably have to post the data file somewhere (dropbox perhaps?) so I can run the script that triggers the segfault.
-Jeff
-- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************
Just one more sentence. We are using version 0.9.7 on our server. when I tried to use: f=nc.Dataset('file.nc') to open the file (either big or small) within ipython, it works fine. Then when I tried to do %run cal_cmi_big.py, core dumped error. but %run cal_cmi_small.py works fine. The cal_cmi_big.py did work several days ago, I still have the file generated by this script in my directory. cheers, Chao 2012/5/27 Chao YUE <chaoyuejoy@gmail.com>
Dear Jeff,
Thanks a lot for your reply. I think it might be related with the memory management on our sever. But anyway, as you suggested, I open an issue on netcdf4-python.google.co <http://netcdf4-python.google.com>m. you can find the data and script on ftp://ftp.cea.fr/incoming/y2k01/chaoyue/. The cal_cmi_big.py gave the core dumped error, in which I tried to operate with big files (~2G). The cal_cmi_small.py works fine, which handles data only ~25M (a subset of big files). I used a small function (which use NetCDF4) in the script to read and write nc data with NetCDF4. You can also find this function in ncfunc.py. I tested all the script and data before I upload on our ftp.
thanks again for your help,
cheers,
Chao
2012/5/26 Jeff Whitaker <jswhit@fastmail.fm>
On 5/26/12 5:51 AM, Chao YUE wrote:
Dear all,
Previously I am able to run a script on our server but now it gave me a Segmentation fault (core dumped) error. try I tried the script with same type of netcdf file but with much smaller file size and it works. So I think the error is related with memory stuff. I guess it's because our system administrator make some change somewhere and that cause my problem? the file size that cause the error to appear is 2.6G (in the script I read this file with NetCDF4 to numpy array and make some manipulation), the small one without error is only 48M.
Chao: Without seeing your script, there's not much I can say. I suggest opening an issue at netcdf4-python.google.com, including your script as an attachment. You'll probably have to post the data file somewhere (dropbox perhaps?) so I can run the script that triggers the segfault.
-Jeff
--
*********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16
************************************************************************************
-- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************
participants (2)
-
Chao YUE -
Jeff Whitaker