From thomas.robitaille at gmail.com Wed Jul 6 11:03:54 2022 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Wed, 6 Jul 2022 16:03:54 +0100 Subject: [AstroPy] Nightly wheels for the astropy core package Message-ID: Hi everyone, As part of the Astropy project, we provide nightly wheel builds of the development version of astropy which can be used e.g. in continuous integration for other packages to test against the developer version of astropy without having to waste computing time building it from source. Until now, these wheels were hosted on the pkgs.dev.azure.com server, but as we have now migrated the building of wheels away from Azure (to GitHub actions), the URL where the developer wheels are stored has now changed to: https://pypi.anaconda.org/astropy/simple To install the latest nightly wheel of astropy, you would now need to do: pip install -U -i https://pypi.anaconda.org/astropy/simple astropy --pre Let me know if you run into any issues! Cheers, Tom -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivvv68 at gmail.com Mon Jul 11 06:58:59 2022 From: ivvv68 at gmail.com (Ivan Valtchanov) Date: Mon, 11 Jul 2022 12:58:59 +0200 Subject: [AstroPy] Changing table column from float64 to float32 Message-ID: Hi all, I have been trying to change a FITS BinTableHDU column from float64 to float32. I tried three obvious paths to do so and they all failed. Here is a very simple code to check. #################### import numpy as np from astropy.io import fits # create a simple table with float64 and float32 c1 = fits.Column(name='a', array=np.array([1, 2],dtype=np.double),format='D') c2 = fits.Column(name='b', array=np.array([3, 4],dtype=np.single),format='E') table_hdu = fits.BinTableHDU.from_columns([c1, c2]) # # check columns types for j in table_hdu.columns: print (j.name,j.dtype) #a float64 #b float32 #================= # first attempt to change float64 to float32 # table_hdu.data['a'] = table_hdu.data['a'].astype(np.single) # for j in table_hdu.columns: print (j.name,j.dtype) # Nope #a float64 #b float32 #================= # second attempt, also obvious #table_hdu.data['a'].dtype = np.single # # Value error: To change to a dtype of a different size, the array must be C-contiguous # not sure what this means... #================== # third attempt, list comprehension and creating a numpy array of type float32 xx = np.array([float(v) for v in table_hdu.data['a']],dtype=np.single) print (xx.dtype) # float32 # Good, it's float32 table_hdu.data['a'] = xx # for j in table_hdu.columns: print (j.name,j.dtype) # Nope, still a is float64 #a float64 #b float32 ############ Is there something I'm missing? Using python 3.10, astropy 5.1, numpy 1.22.4 Thanks, Ivan -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbaer25 at gmail.com Mon Jul 11 09:03:42 2022 From: rbaer25 at gmail.com (Rudolf Baer) Date: Mon, 11 Jul 2022 15:03:42 +0200 Subject: [AstroPy] Summing an array Message-ID: I have not been able to sum an array; the array is created as follows rpflux = np.array(rpflux, dtype=float) rpflux=np.zeros(len(rptime),dtype='float64') rpflux=np.genfromtxt(path,delimiter=',',dtype="float64",autostrip=True,skip_header=137,skip_footer=0, usecols=(19)) print(rpflux.astype('float64')) [3140. nan 2770. 3080. 3170. 3050. 3060. 3100. 2990. 3120. 3060. nan 3100. 3070. 3020. nan 3200. 3150. 3350. 2990. 2910. 2920. 3030. 3000. 2800. 2980. 2860. 2840. 2610. 2650. 2610. 2650. 2490. 2490. 2540. 2510. 2470. nan nan 35. nan 3099. nan ] print (np.sum(rpflux)) and print (np.sum([rpflux])) result in 'numpy.ndarray' object is not callable' I have tried several alternatives, none of them work. Any suggestions would be appreciated. Rudolf Baer Mac Os 10.15.6, python 3.7 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ejensen1 at swarthmore.edu Mon Jul 11 10:03:40 2022 From: ejensen1 at swarthmore.edu (Eric Jensen) Date: Mon, 11 Jul 2022 10:03:40 -0400 Subject: [AstroPy] Summing an array In-Reply-To: References: Message-ID: <797C3379-14C1-4F16-ACCE-96F3F309C18C@swarthmore.edu> Hi Rudolf, Can you try restarting your Python interpreter, and stripping your code down to a minimal example that shows the problem in the shortest number of lines? The first line of your code as written below references an array that doesn?t exist yet (?rpflux?) on the right-hand side of the expression, so that would give an error right away. So I suspect that you may have run several expressions before that and already have some defined variables, making it harder to tell what is happening here. Starting with the third line (genfromtxt) would make sense (since you?re overwriting the array anyway there), as long as you define ?path? before that. Giving a set of lines that, by themselves, show the problem when run straight through will make it easiest to troubleshoot. Strip it down to a minimal example, and we can go from there. Also, note that if you have any NaNs in your array, then np.sum will return NaN. If you want those to be treated as zeroes, use np.nansum instead. Eric > On Jul 11, 2022, at 9:03 AM, Rudolf Baer wrote: > > I have not been able to sum an array; the array is created as follows > > rpflux = np.array(rpflux, dtype=float) > rpflux=np.zeros(len(rptime),dtype='float64') > rpflux=np.genfromtxt(path,delimiter=',',dtype="float64",autostrip=True,skip_header=137,skip_footer=0, usecols=(19)) > print(rpflux.astype('float64')) > [3140. nan 2770. 3080. 3170. 3050. 3060. > 3100. 2990. 3120. 3060. nan 3100. 3070. > 3020. nan 3200. 3150. 3350. 2990. 2910. > 2920. 3030. 3000. 2800. 2980. 2860. 2840. > 2610. 2650. 2610. 2650. 2490. 2490. 2540. > 2510. 2470. nan nan 35. nan 3099. > nan ] > > print (np.sum(rpflux)) > and > print (np.sum([rpflux])) > result in > 'numpy.ndarray' object is not callable' > > I have tried several alternatives, none of them work. Any suggestions would be appreciated. > Rudolf Baer > > Mac Os 10.15.6, python 3.7 > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From dhomeie at gwdg.de Mon Jul 11 10:10:33 2022 From: dhomeie at gwdg.de (Homeier, Derek) Date: Mon, 11 Jul 2022 14:10:33 +0000 Subject: [AstroPy] Summing an array In-Reply-To: References: Message-ID: <2FB9E34F-86C0-41A0-9DE9-E5C13CE06E8C@gwdg.de> On 11 Jul 2022, at 3:03 pm, Rudolf Baer wrote: > > I have not been able to sum an array; the array is created as follows > This is really a pure numpy issue, but it looks like something in your numpy installation is broken or overwritten in the `np` namespace. > rpflux = np.array(rpflux, dtype=float) > rpflux=np.zeros(len(rptime),dtype='float64') The above is redundant btw. as np.genfromtxt will create, populate and return the array on the fly (and simply replace your previously created objects). > rpflux=np.genfromtxt(path,delimiter=',',dtype="float64",autostrip=True,skip_header=137,skip_footer=0, usecols=(19)) > print(rpflux.astype('float64')) > [3140. nan 2770. 3080. 3170. 3050. 3060. > 3100. 2990. 3120. 3060. nan 3100. 3070. > 3020. nan 3200. 3150. 3350. 2990. 2910. > 2920. 3030. 3000. 2800. 2980. 2860. 2840. > 2610. 2650. 2610. 2650. 2490. 2490. 2540. > 2510. 2470. nan nan 35. nan 3099. > nan ] > > > print (np.sum(rpflux)) > and > print (np.sum([rpflux])) > result in > 'numpy.ndarray' object is not callable' > Could you post some details on your Numpy installation as well? print(np.version.version) print(np.sum.__class__) print(np.sum.__name__) print(np.sum.__module__) print(rpflux.sum()) On another note, you?ll probably want to use `np.nansum()` on those particular data. HTH Derek From rbaer25 at gmail.com Mon Jul 11 11:39:10 2022 From: rbaer25 at gmail.com (Rudolf Baer) Date: Mon, 11 Jul 2022 17:39:10 +0200 Subject: [AstroPy] Summing an array In-Reply-To: <797C3379-14C1-4F16-ACCE-96F3F309C18C@swarthmore.edu> References: <797C3379-14C1-4F16-ACCE-96F3F309C18C@swarthmore.edu> Message-ID: Hi Eric thanks for the quick reply. The following works: rpflux=[3140., 2770. , 3080. , 3170. , 3050., 3060.,] print(sum(rpflux)) 18270.0 The rpflux which I read with rpflux=np.genfromtxt(path,delimiter=',',dtype="float64",autostrip=True,skip_header=137,skip_footer=0, usecols=(19)) comes from a csv-file which reads in part 3140 2770 3080 3170 3050 3060 3100 the whole file is printed out above as [3140. nan 2770. 3080. 3170. 3050. 3060. 3100. 2990. 3120. 3060. nan 3100. 3070. 3020. nan 3200. 3150. 3350. 2990. 2910. 2920. 3030. 3000. 2800. 2980. 2860. 2840. 2610. 2650. 2610. 2650. 2490. 2490. 2540. 2510. 2470. nan nan 35. nan 3099. nan ] it does contain nan s, but np.nansum still produces 'numpy.ndarray' object is not callable. What do I do wrong? With kind regards Rudolf PS: I can provide you with an extract of csv- file On Mon, Jul 11, 2022 at 4:04 PM Eric Jensen wrote: > Hi Rudolf, > > Can you try restarting your Python interpreter, and stripping your code > down to a minimal example that shows the problem in the shortest number of > lines? The first line of your code as written below references an array > that doesn?t exist yet (?rpflux?) on the right-hand side of the expression, > so that would give an error right away. So I suspect that you may have run > several expressions before that and already have some defined variables, > making it harder to tell what is happening here. Starting with the third > line (genfromtxt) would make sense (since you?re overwriting the array > anyway there), as long as you define ?path? before that. Giving a set of > lines that, by themselves, show the problem when run straight through will > make it easiest to troubleshoot. > > Strip it down to a minimal example, and we can go from there. > > Also, note that if you have any NaNs in your array, then np.sum will > return NaN. If you want those to be treated as zeroes, use np.nansum > instead. > > Eric > > > On Jul 11, 2022, at 9:03 AM, Rudolf Baer wrote: > > I have not been able to sum an array; the array is created as follows > > rpflux = np.array(rpflux, dtype=float) > rpflux=np.zeros(len(rptime),dtype='float64') > rpflux=np.genfromtxt(path,delimiter=',',dtype="float64",autostrip=True,skip_header=137,skip_footer=0, > usecols=(19)) > print(rpflux.astype('float64')) > > [3140. nan 2770. 3080. 3170. 3050. 3060. > 3100. 2990. 3120. 3060. nan 3100. 3070. > 3020. nan 3200. 3150. 3350. 2990. 2910. > 2920. 3030. 3000. 2800. 2980. 2860. 2840. > 2610. 2650. 2610. 2650. 2490. 2490. 2540. > 2510. 2470. nan nan 35. nan 3099. > nan ] > > > print (np.sum(rpflux)) > and > print (np.sum([rpflux])) > result in > 'numpy.ndarray' object is not callable' > > I have tried several alternatives, none of them work. Any suggestions > would be appreciated. > Rudolf Baer > > Mac Os 10.15.6, python 3.7 > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbaer25 at gmail.com Mon Jul 11 11:43:38 2022 From: rbaer25 at gmail.com (Rudolf Baer) Date: Mon, 11 Jul 2022 17:43:38 +0200 Subject: [AstroPy] Summing an array In-Reply-To: References: <797C3379-14C1-4F16-ACCE-96F3F309C18C@swarthmore.edu> Message-ID: print(np.version.version) 1.18.5 class 'numpy.ndarray 1 print(np.version.version) 2 print(np.sum.__class__)----> 3 print(np.sum.__name__) 4 print(np.sum.__module__) AttributeError: 'numpy.ndarray' object has no attribute '__name__' 2 print(np.sum.__class__) 3 #print(np.sum.__name__)----> 4 print(np.sum.__module__) AttributeError: 'numpy.ndarray' object has no attribute '__module__' On Mon, Jul 11, 2022 at 5:39 PM Rudolf Baer wrote: > Hi Eric > thanks for the quick reply. The following works: > > rpflux=[3140., 2770. , 3080. , 3170. , 3050., 3060.,] > print(sum(rpflux)) > 18270.0 > > The rpflux which I read with > rpflux=np.genfromtxt(path,delimiter=',',dtype="float64",autostrip=True,skip_header=137,skip_footer=0, > usecols=(19)) > comes from a csv-file which reads in part > 3140 > > 2770 > 3080 > 3170 > 3050 > 3060 > 3100 the whole file is printed out above as > > [3140. nan 2770. 3080. 3170. 3050. 3060. > 3100. 2990. 3120. 3060. nan 3100. 3070. > 3020. nan 3200. 3150. 3350. 2990. 2910. > 2920. 3030. 3000. 2800. 2980. 2860. 2840. > 2610. 2650. 2610. 2650. 2490. 2490. 2540. > 2510. 2470. nan nan 35. nan 3099. > nan ] > > it does contain nan s, but np.nansum still produces 'numpy.ndarray' object is not callable. What do I do wrong? > > With kind regards > > Rudolf > > > PS: I can provide you with an extract of csv- file > > > > > > On Mon, Jul 11, 2022 at 4:04 PM Eric Jensen > wrote: > >> Hi Rudolf, >> >> Can you try restarting your Python interpreter, and stripping your code >> down to a minimal example that shows the problem in the shortest number of >> lines? The first line of your code as written below references an array >> that doesn?t exist yet (?rpflux?) on the right-hand side of the expression, >> so that would give an error right away. So I suspect that you may have run >> several expressions before that and already have some defined variables, >> making it harder to tell what is happening here. Starting with the third >> line (genfromtxt) would make sense (since you?re overwriting the array >> anyway there), as long as you define ?path? before that. Giving a set of >> lines that, by themselves, show the problem when run straight through will >> make it easiest to troubleshoot. >> >> Strip it down to a minimal example, and we can go from there. >> >> Also, note that if you have any NaNs in your array, then np.sum will >> return NaN. If you want those to be treated as zeroes, use np.nansum >> instead. >> >> Eric >> >> >> On Jul 11, 2022, at 9:03 AM, Rudolf Baer wrote: >> >> I have not been able to sum an array; the array is created as follows >> >> rpflux = np.array(rpflux, dtype=float) >> rpflux=np.zeros(len(rptime),dtype='float64') >> rpflux=np.genfromtxt(path,delimiter=',',dtype="float64",autostrip=True,skip_header=137,skip_footer=0, >> usecols=(19)) >> print(rpflux.astype('float64')) >> >> [3140. nan 2770. 3080. 3170. 3050. 3060. >> 3100. 2990. 3120. 3060. nan 3100. 3070. >> 3020. nan 3200. 3150. 3350. 2990. 2910. >> 2920. 3030. 3000. 2800. 2980. 2860. 2840. >> 2610. 2650. 2610. 2650. 2490. 2490. 2540. >> 2510. 2470. nan nan 35. nan 3099. >> nan ] >> >> >> print (np.sum(rpflux)) >> and >> print (np.sum([rpflux])) >> result in >> 'numpy.ndarray' object is not callable' >> >> I have tried several alternatives, none of them work. Any suggestions >> would be appreciated. >> Rudolf Baer >> >> Mac Os 10.15.6, python 3.7 >> _______________________________________________ >> AstroPy mailing list >> AstroPy at python.org >> https://mail.python.org/mailman/listinfo/astropy >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at python.org >> https://mail.python.org/mailman/listinfo/astropy >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dhomeie at gwdg.de Mon Jul 11 12:09:28 2022 From: dhomeie at gwdg.de (Homeier, Derek) Date: Mon, 11 Jul 2022 16:09:28 +0000 Subject: [AstroPy] Summing an array In-Reply-To: References: <797C3379-14C1-4F16-ACCE-96F3F309C18C@swarthmore.edu> Message-ID: <61AB8479-23BF-4049-B170-BEF694242E58@gwdg.de> Hi Rudolf, print(np.version.version) 1.18.5 That?s kind of old (Python 3.7 is supported by up to 1.21.6), but the interface really has not changed notably since then. class 'numpy.ndarray 1 print(np.version.version) 2 print(np.sum.__class__) ----> 3 print(np.sum.__name__) 4 print(np.sum.__module__) AttributeError: 'numpy.ndarray' object has no attribute '__name__' 2 print(np.sum.__class__) 3 #print(np.sum.__name__) ----> 4 print(np.sum.__module__) AttributeError: 'numpy.ndarray' object has no attribute ?__module__' Really looks like a polluted namespace (meaning some methods like `np.sum`, `np.nansum` have been overwritten by some other operation). As Eric suggested, could you provide a standalone example as a script including in particular everything from the import numpy as np statement in a fresh Python session onwards (I am assuming you are seeing the same error with rpflux = np.array([3140., 2770. , 3080. , 3170. , 3050., 3060.]) np.sum(rpflux) so no need to include your actual data file in that case). Cheers, Derek -------------- next part -------------- An HTML attachment was scrubbed... URL: From pweilbacher at aip.de Mon Jul 11 12:17:03 2022 From: pweilbacher at aip.de (Peter Weilbacher) Date: Mon, 11 Jul 2022 18:17:03 +0200 (CEST) Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: References: Message-ID: Hi Ivan, I have no particular insight into this, but it seems to me that since you created the Column objects with a specific format, assigning any data to it will cause it to be casted to match that. (Otherwise the underlying memory buffer does not match.) Instead, I guess you could create a new column with the desired type and format and recreate the table using that: c1_single = fits.Column(name='a', array=table_hdu.data['a'].astype(np.single), format='E') table_hdu2 = fits.BinTableHDU.from_columns(c1_single + table_hdu.columns[1:]) (It appears that you can even remove the .astype() from this and it still works.) Peter. On Mon, 11 Jul 2022, Ivan Valtchanov wrote: > Hi all, > I have been trying to change a FITS BinTableHDU column from float64 to > float32. I tried three obvious paths to do so and they all failed. Here is > a very simple code to check. > > #################### > import numpy as np > from astropy.io import fits > > # create a simple table with float64 and float32 > c1 = fits.Column(name='a', array=np.array([1, > 2],dtype=np.double),format='D') > c2 = fits.Column(name='b', array=np.array([3, > 4],dtype=np.single),format='E') > table_hdu = fits.BinTableHDU.from_columns([c1, c2]) > # > # check columns types > for j in table_hdu.columns: > print (j.name,j.dtype) > #a float64 > #b float32 > #================= > # first attempt to change float64 to float32 > # > table_hdu.data['a'] = table_hdu.data['a'].astype(np.single) > # > for j in table_hdu.columns: > print (j.name,j.dtype) > # Nope > #a float64 > #b float32 > #================= > # second attempt, also obvious > #table_hdu.data['a'].dtype = np.single > # > # Value error: To change to a dtype of a different size, the array must be > C-contiguous > # not sure what this means... > #================== > # third attempt, list comprehension and creating a numpy array of type > float32 > xx = np.array([float(v) for v in table_hdu.data['a']],dtype=np.single) > print (xx.dtype) > # float32 > # Good, it's float32 > table_hdu.data['a'] = xx > # > for j in table_hdu.columns: > print (j.name,j.dtype) > # Nope, still a is float64 > #a float64 > #b float32 > ############ > > Is there something I'm missing? > > Using python 3.10, astropy 5.1, numpy 1.22.4 > > Thanks, > Ivan > -- Dr. Peter M. Weilbacher https://www.aip.de/Members/pweilbacher Phone +49 331 74 99-667 encryption key ID 7D6B4AA0 ------------------------------------------------------------------------ Leibniz-Institut f?r Astrophysik Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam Vorstand: Prof. Dr. Matthias Steinmetz, Wolfram Rosenbach Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: 26 742-00/7026 From brewer at astro.umass.edu Mon Jul 11 17:14:06 2022 From: brewer at astro.umass.edu (Michael Brewer) Date: Mon, 11 Jul 2022 17:14:06 -0400 Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: References: Message-ID: <3fb3e116-d600-5f2e-8709-2f18a4d19bde@astro.umass.edu> Ivan, Please try this: table_hdu.replace_column('a', table_hdu.data['a'].astype(np.single)) Michael On 7/11/22 12:17 PM, Peter Weilbacher wrote: > Hi Ivan, > > I have no particular insight into this, but it seems to me that since > you created the Column objects with a specific format, assigning any > data to it will cause it to be casted to match that. (Otherwise the > underlying memory buffer does not match.) > > Instead, I guess you could create a new column with the desired type and > format and recreate the table using that: > > c1_single = fits.Column(name='a', array=table_hdu.data['a'].astype(np.single), format='E') > table_hdu2 = fits.BinTableHDU.from_columns(c1_single + table_hdu.columns[1:]) > > (It appears that you can even remove the .astype() from this and it > still works.) > > Peter. > > On Mon, 11 Jul 2022, Ivan Valtchanov wrote: > >> Hi all, >> I have been trying to change a FITS BinTableHDU column from float64 to >> float32. I tried three obvious paths to do so and they all failed. Here is >> a very simple code to check. >> >> #################### >> import numpy as np >> from astropy.io import fits >> >> # create a simple table with float64 and float32 >> c1 = fits.Column(name='a', array=np.array([1, >> 2],dtype=np.double),format='D') >> c2 = fits.Column(name='b', array=np.array([3, >> 4],dtype=np.single),format='E') >> table_hdu = fits.BinTableHDU.from_columns([c1, c2]) >> # >> # check columns types >> for j in table_hdu.columns: >> print (j.name,j.dtype) >> #a float64 >> #b float32 >> #================= >> # first attempt to change float64 to float32 >> # >> table_hdu.data['a'] = table_hdu.data['a'].astype(np.single) >> # >> for j in table_hdu.columns: >> print (j.name,j.dtype) >> # Nope >> #a float64 >> #b float32 >> #================= >> # second attempt, also obvious >> #table_hdu.data['a'].dtype = np.single >> # >> # Value error: To change to a dtype of a different size, the array must be >> C-contiguous >> # not sure what this means... >> #================== >> # third attempt, list comprehension and creating a numpy array of type >> float32 >> xx = np.array([float(v) for v in table_hdu.data['a']],dtype=np.single) >> print (xx.dtype) >> # float32 >> # Good, it's float32 >> table_hdu.data['a'] = xx >> # >> for j in table_hdu.columns: >> print (j.name,j.dtype) >> # Nope, still a is float64 >> #a float64 >> #b float32 >> ############ >> >> Is there something I'm missing? >> >> Using python 3.10, astropy 5.1, numpy 1.22.4 >> >> Thanks, >> Ivan >> From dhomeie at gwdg.de Mon Jul 11 20:13:21 2022 From: dhomeie at gwdg.de (Homeier, Derek) Date: Tue, 12 Jul 2022 00:13:21 +0000 Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: <3fb3e116-d600-5f2e-8709-2f18a4d19bde@astro.umass.edu> References: <3fb3e116-d600-5f2e-8709-2f18a4d19bde@astro.umass.edu> Message-ID: On 11 Jul 2022, at 11:14 pm, Michael Brewer > wrote: Please try this: table_hdu.replace_column('a', table_hdu.data['a'].astype(np.single)) Michael replace_column is a [astropy.table.]Table method; this won?t work on a FITS BinTable ? I think the most direct way is indeed to recreate the HDU from the appropriately cast Columns as outlined by Peter. Note that the HDU also contains the corresponding header information characterising the data section formats, so direct manipulation of the data parts is limited for a reason. On 7/11/22 12:17 PM, Peter Weilbacher wrote: Hi Ivan, I have no particular insight into this, but it seems to me that since you created the Column objects with a specific format, assigning any data to it will cause it to be casted to match that. (Otherwise the underlying memory buffer does not match.) Instead, I guess you could create a new column with the desired type and format and recreate the table using that: c1_single = fits.Column(name='a', array=table_hdu.data['a'].astype(np.single), format='E') table_hdu2 = fits.BinTableHDU.from_columns(c1_single + table_hdu.columns[1:]) (It appears that you can even remove the .astype() from this and it still works.) Yes, the `format=?E?` specifier will already recast to the respective FITS (= Fortran) format. Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbaer25 at gmail.com Tue Jul 12 04:13:34 2022 From: rbaer25 at gmail.com (Rudolf Baer) Date: Tue, 12 Jul 2022 10:13:34 +0200 Subject: [AstroPy] Summing an array In-Reply-To: <61AB8479-23BF-4049-B170-BEF694242E58@gwdg.de> References: <797C3379-14C1-4F16-ACCE-96F3F309C18C@swarthmore.edu> <61AB8479-23BF-4049-B170-BEF694242E58@gwdg.de> Message-ID: Hi Derek In a new jupyter notebook it works !!! Thank you v e r y much Do you think I should update numpy? if yes to version 1.21.6? with kind regards Rudolf import numpy as np import matplotlib.pyplot as plt print(np.version.version) print(np.sum.__class__) print(np.sum.__name__) print(np.sum.__module__) object=str('361') path='Gaia/''BASS'+object+'_107.6252849 59.13899216.csv' rpflux= np.genfromtxt(path,delimiter=',',dtype="float64",autostrip=True,skip_header=137,skip_footer=0, usecols=(19)) print(rpflux) print('sum', np.nansum(rpflux)) 1.18.5 sum numpy [3140. nan 2770. 3080. 3170. 3050. 3060. 3100. 2990. 3120. 3060. 9670. 3100. 3070. 3020. nan 3200. 3150. 3350. 2990. 2910. 2920. 3030. 3000. 2800. 2980. 2860. 2840. 2610. 2650. 2610. 2650. 2490. 2490. 2540. 2510. 2470.] sum 108450.0 On Mon, Jul 11, 2022 at 6:10 PM Homeier, Derek wrote: > Hi Rudolf, > > > print(np.version.version) 1.18.5 > > > That?s kind of old (Python 3.7 is supported by up to 1.21.6), but the > interface really has not changed > notably since then. > > class 'numpy.ndarray > > 1 print(np.version.version) 2 print(np.sum.__class__)----> 3 print(np.sum.__name__) 4 print(np.sum.__module__) > AttributeError: 'numpy.ndarray' object has no attribute '__name__' > > > 2 print(np.sum.__class__) 3 #print(np.sum.__name__)----> 4 print(np.sum.__module__) > AttributeError: 'numpy.ndarray' object has no attribute ?__module__' > > > Really looks like a polluted namespace (meaning some methods like > `np.sum`, `np.nansum` have been > overwritten by some other operation). > > As Eric suggested, could you provide a standalone example as a script > including in particular everything from the > > import numpy as np > > statement in a fresh Python session onwards (I am assuming you are seeing > the same error with > > rpflux = np.array([3140., 2770. , 3080. , 3170. , 3050., > 3060.]) > np.sum(rpflux) > > so no need to include your actual data file in that case). > > Cheers, > Derek > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at sconseil.fr Tue Jul 12 04:21:08 2022 From: simon at sconseil.fr (Simon Conseil) Date: Tue, 12 Jul 2022 10:21:08 +0200 Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: References: <3fb3e116-d600-5f2e-8709-2f18a4d19bde@astro.umass.edu> Message-ID: <07c8c80a6eadab40bebf5f4e113b888fd267a701.camel@sconseil.fr> Indeed the BinTableHDU object doesn't have a method to easily replace a column (which is defined as float64 by the format='D' attribute in your example, so data assigned to the column will always be recast to float64). One option would be to use table_hdu.columns.del_col and table_hdu.columns.add_col but this isn't the most practical.? In [3]: a = table_hdu.data['a'].astype('float32') In [4]: cols = table_hdu.columns.del_col('a') In [6]: cols = table_hdu.columns.add_col(fits.Column(name='a', array=a, format='E')) In [8]: table_hdu = fits.BinTableHDU.from_columns(cols) In [9]: table_hdu.columns Out[9]: ColDefs( name = 'b'; format = 'E' name = 'a'; format = 'E' ) The easiest here would be to use astropy.table.Table: In [3]: t = Table.read(table_hdu) In [4]: t['a'] = t['a'].astype('float32') In [5]: table_hdu = fits.table_to_hdu(t) In [6]: table_hdu.columns Out[6]: ColDefs( name = 'a'; format = 'E' name = 'b'; format = 'E' ) Simon On Tue, 2022-07-12 at 00:13 +0000, Homeier, Derek wrote: > On 11 Jul 2022, at 11:14 pm, Michael Brewer > wrote: > > > > Please try this: > > > > table_hdu.replace_column('a', > > table_hdu.data['a'].astype(np.single)) > > > > Michael > > > replace_column is a [astropy.table.]Table method; this won?t work on > a FITS BinTable ? > I think the most direct way is indeed to recreate the HDU from the > appropriately cast > Columns as outlined by Peter. Note that the HDU also contains the > corresponding header > information characterising the data section formats, so direct > manipulation of the data > parts is limited for a reason. > > > > On 7/11/22 12:17 PM, Peter Weilbacher wrote: > > > Hi Ivan, > > > > > > I have no particular insight into this, but it seems to me that > > > since > > > you created the Column objects with a specific format, assigning > > > any > > > data to it will cause it to be casted to match that. (Otherwise > > > the > > > underlying memory buffer does not match.) > > > > > > Instead, I guess you could create a new column with the desired > > > type and > > > format and recreate the table using that: > > > > > > c1_single = fits.Column(name='a', > > > array=table_hdu.data['a'].astype(np.single), format='E') > > > table_hdu2 = fits.BinTableHDU.from_columns(c1_single + > > > table_hdu.columns[1:]) > > > > > > (It appears that you can even remove the .astype() from this and > > > it > > > still works.) > > Yes, the `format=?E?` specifier will already recast to the respective > FITS (= Fortran) format. > > Cheers > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy From ivvv68 at gmail.com Tue Jul 12 06:11:52 2022 From: ivvv68 at gmail.com (Ivan Valtchanov) Date: Tue, 12 Jul 2022 12:11:52 +0200 Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: <07c8c80a6eadab40bebf5f4e113b888fd267a701.camel@sconseil.fr> References: <3fb3e116-d600-5f2e-8709-2f18a4d19bde@astro.umass.edu> <07c8c80a6eadab40bebf5f4e113b888fd267a701.camel@sconseil.fr> Message-ID: Thanks all, Yes, I managed to do the change of float64 to float32 columns using astropy.table.Table. It's funny how people keep float64 columns in huge catalogues for something that's perfectly well stored with float32, e.g. RA, Dec, LII, BII, MJD, etc... It's true that RAM is cheap nowadays but this laziness puzzles me a lot (/rant_over). Cheers, Ivan On Tue, 12 Jul 2022 at 10:21, Simon Conseil wrote: > Indeed the BinTableHDU object doesn't have a method to easily replace a > column (which is defined as float64 by the format='D' attribute in your > example, so data assigned to the column will always be recast to > float64). > > One option would be to use table_hdu.columns.del_col and > table_hdu.columns.add_col but this isn't the most practical. > > In [3]: a = table_hdu.data['a'].astype('float32') > > In [4]: cols = table_hdu.columns.del_col('a') > > In [6]: cols = table_hdu.columns.add_col(fits.Column(name='a', array=a, > format='E')) > > In [8]: table_hdu = fits.BinTableHDU.from_columns(cols) > > In [9]: table_hdu.columns > Out[9]: > ColDefs( > name = 'b'; format = 'E' > name = 'a'; format = 'E' > ) > > The easiest here would be to use astropy.table.Table: > > In [3]: t = Table.read(table_hdu) > > In [4]: t['a'] = t['a'].astype('float32') > > In [5]: table_hdu = fits.table_to_hdu(t) > > In [6]: table_hdu.columns > Out[6]: > ColDefs( > name = 'a'; format = 'E' > name = 'b'; format = 'E' > ) > > > Simon > > On Tue, 2022-07-12 at 00:13 +0000, Homeier, Derek wrote: > > On 11 Jul 2022, at 11:14 pm, Michael Brewer > > wrote: > > > > > > Please try this: > > > > > > table_hdu.replace_column('a', > > > table_hdu.data['a'].astype(np.single)) > > > > > > Michael > > > > > replace_column is a [astropy.table.]Table method; this won?t work on > > a FITS BinTable ? > > I think the most direct way is indeed to recreate the HDU from the > > appropriately cast > > Columns as outlined by Peter. Note that the HDU also contains the > > corresponding header > > information characterising the data section formats, so direct > > manipulation of the data > > parts is limited for a reason. > > > > > > On 7/11/22 12:17 PM, Peter Weilbacher wrote: > > > > Hi Ivan, > > > > > > > > I have no particular insight into this, but it seems to me that > > > > since > > > > you created the Column objects with a specific format, assigning > > > > any > > > > data to it will cause it to be casted to match that. (Otherwise > > > > the > > > > underlying memory buffer does not match.) > > > > > > > > Instead, I guess you could create a new column with the desired > > > > type and > > > > format and recreate the table using that: > > > > > > > > c1_single = fits.Column(name='a', > > > > array=table_hdu.data['a'].astype(np.single), format='E') > > > > table_hdu2 = fits.BinTableHDU.from_columns(c1_single + > > > > table_hdu.columns[1:]) > > > > > > > > (It appears that you can even remove the .astype() from this and > > > > it > > > > still works.) > > > > Yes, the `format=?E?` specifier will already recast to the respective > > FITS (= Fortran) format. > > > > Cheers > > _______________________________________________ > > AstroPy mailing list > > AstroPy at python.org > > https://mail.python.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dhomeie at gwdg.de Tue Jul 12 09:42:58 2022 From: dhomeie at gwdg.de (Homeier, Derek) Date: Tue, 12 Jul 2022 13:42:58 +0000 Subject: [AstroPy] Summing an array In-Reply-To: References: <797C3379-14C1-4F16-ACCE-96F3F309C18C@swarthmore.edu> <61AB8479-23BF-4049-B170-BEF694242E58@gwdg.de> Message-ID: <1CD56A34-A6CE-4D10-91AA-BDE884BF206D@gwdg.de> Hi Rudolf, > > In a new jupyter notebook it works !!! > Thank you v e r y much > Do you think I should update numpy? if yes to version 1.21.6? > glad to hear that! Generally I think it?s a good idea to keep up to date with the package versions for bug fixes and better performance. If possible I would actually recommend to upgrade your Python installation to 3.8 or even better, 3.10, since those support the newest releases of both Numpy and Astropy, and the next major Astropy release will probably drop 3.8 as well. Cheers, Derek From bjweiner at gmail.com Tue Jul 12 12:52:34 2022 From: bjweiner at gmail.com (Benjamin Weiner) Date: Tue, 12 Jul 2022 09:52:34 -0700 Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: References: Message-ID: Ivan Valtchanov wrote: > Yes, I managed to do the change of float64 to float32 columns using > astropy.table.Table. > It's funny how people keep float64 columns in huge catalogues for something > that's perfectly well stored with float32, e.g. RA, Dec, LII, BII, MJD, > etc... It's true that RAM is cheap nowadays but this laziness puzzles me a > lot (/rant_over). Hi, Glad you found a solution. For quantities that have a large value but where small differences may be important, using double precision is often warranted. Single precision has a precision of 1 in 2^23 or about 6.9 dex in log. This translates to of order 0.1 arcsecond in RA/Dec, or 9 minutes of time on an MJD of 50000. Propagating truncation errors through further code or processing can magnify these effects, which is one reason that catalog makers carry around the extra digits and internal routines/constants are often written in double where explicit typing is needed. Cheers, Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.m.bray at gmail.com Tue Jul 12 13:53:58 2022 From: erik.m.bray at gmail.com (E. Madison Bray) Date: Tue, 12 Jul 2022 19:53:58 +0200 Subject: [AstroPy] Summing an array In-Reply-To: <1CD56A34-A6CE-4D10-91AA-BDE884BF206D@gwdg.de> References: <797C3379-14C1-4F16-ACCE-96F3F309C18C@swarthmore.edu> <61AB8479-23BF-4049-B170-BEF694242E58@gwdg.de> <1CD56A34-A6CE-4D10-91AA-BDE884BF206D@gwdg.de> Message-ID: On Tue, Jul 12, 2022 at 3:43 PM Homeier, Derek wrote: > > Hi Rudolf, > > > > In a new jupyter notebook it works !!! > > Thank you v e r y much > > Do you think I should update numpy? if yes to version 1.21.6? > > > glad to hear that! > Generally I think it?s a good idea to keep up to date with the package versions > for bug fixes and better performance. > If possible I would actually recommend to upgrade your Python installation to 3.8 > or even better, 3.10, since those support the newest releases of both Numpy and > Astropy, and the next major Astropy release will probably drop 3.8 as well. > > Cheers, > Derek Nevertheless, np.sum has been np.sum since like the first version of NumPy, so it should have worked. Somehow, somewhere in Rudolph's notebook, they wrote, perhaps as a typo, np.sum = . This is why I would suggest specifically if you are working in a Jupyter notebook and having a mysterious problem like this to always link, if possible, to the *entire* notebook. Or if that's not possible (for research privacy reasons), start a new notebook and see if you can reproduce the problem in there. Personally I've soured a bit on the use of notebooks for research fiddling. I think they're very good for presentation of working code and data, and to some extent as lab notebooks for more experienced users, but I think the capability for out-of-order execution and re-running errors in previously run code hands too many foot-guns to novices, compared to a normal, linear, REPL. From parejkoj at uw.edu Tue Jul 12 15:59:43 2022 From: parejkoj at uw.edu (John K. Parejko) Date: Tue, 12 Jul 2022 12:59:43 -0700 Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: References: <3fb3e116-d600-5f2e-8709-2f18a4d19bde@astro.umass.edu> <07c8c80a6eadab40bebf5f4e113b888fd267a701.camel@sconseil.fr> Message-ID: For the record, float32 is only around 10mas precision for ra/dec degrees. For many modern datasets, that?s comparable with the nominal centroiding accuracy (e.g. LSST expects ~10mas repeatability for bright sources). 64bit floats are absolutely the correct data type for sky coordinates these days. John > On 12Jul 2022, at 03:11, Ivan Valtchanov wrote: > > Thanks all, > > Yes, I managed to do the change of float64 to float32 columns using astropy.table.Table. > > It's funny how people keep float64 columns in huge catalogues for something that's perfectly well stored with float32, e.g. RA, Dec, LII, BII, MJD, etc... It's true that RAM is cheap nowadays but this laziness puzzles me a lot (/rant_over). > > Cheers, > Ivan > > > On Tue, 12 Jul 2022 at 10:21, Simon Conseil wrote: > Indeed the BinTableHDU object doesn't have a method to easily replace a > column (which is defined as float64 by the format='D' attribute in your > example, so data assigned to the column will always be recast to > float64). > > One option would be to use table_hdu.columns.del_col and > table_hdu.columns.add_col but this isn't the most practical. > > In [3]: a = table_hdu.data['a'].astype('float32') > > In [4]: cols = table_hdu.columns.del_col('a') > > In [6]: cols = table_hdu.columns.add_col(fits.Column(name='a', array=a, > format='E')) > > In [8]: table_hdu = fits.BinTableHDU.from_columns(cols) > > In [9]: table_hdu.columns > Out[9]: > ColDefs( > name = 'b'; format = 'E' > name = 'a'; format = 'E' > ) > > The easiest here would be to use astropy.table.Table: > > In [3]: t = Table.read(table_hdu) > > In [4]: t['a'] = t['a'].astype('float32') > > In [5]: table_hdu = fits.table_to_hdu(t) > > In [6]: table_hdu.columns > Out[6]: > ColDefs( > name = 'a'; format = 'E' > name = 'b'; format = 'E' > ) > > > Simon > > On Tue, 2022-07-12 at 00:13 +0000, Homeier, Derek wrote: > > On 11 Jul 2022, at 11:14 pm, Michael Brewer > > wrote: > > > > > > Please try this: > > > > > > table_hdu.replace_column('a', > > > table_hdu.data['a'].astype(np.single)) > > > > > > Michael > > > > > replace_column is a [astropy.table.]Table method; this won?t work on > > a FITS BinTable ? > > I think the most direct way is indeed to recreate the HDU from the > > appropriately cast > > Columns as outlined by Peter. Note that the HDU also contains the > > corresponding header > > information characterising the data section formats, so direct > > manipulation of the data > > parts is limited for a reason. > > > > > > On 7/11/22 12:17 PM, Peter Weilbacher wrote: > > > > Hi Ivan, > > > > > > > > I have no particular insight into this, but it seems to me that > > > > since > > > > you created the Column objects with a specific format, assigning > > > > any > > > > data to it will cause it to be casted to match that. (Otherwise > > > > the > > > > underlying memory buffer does not match.) > > > > > > > > Instead, I guess you could create a new column with the desired > > > > type and > > > > format and recreate the table using that: > > > > > > > > c1_single = fits.Column(name='a', > > > > array=table_hdu.data['a'].astype(np.single), format='E') > > > > table_hdu2 = fits.BinTableHDU.from_columns(c1_single + > > > > table_hdu.columns[1:]) > > > > > > > > (It appears that you can even remove the .astype() from this and > > > > it > > > > still works.) > > > > Yes, the `format=?E?` specifier will already recast to the respective > > FITS (= Fortran) format. > > > > Cheers > > _______________________________________________ > > AstroPy mailing list > > AstroPy at python.org > > https://mail.python.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy -- ************************* John Parejko parejkoj at uw.edu Rubin Observatory, DIRAC Fellow http://staff.washington.edu/parejkoj/ Department of Physics and Astronomy University of Washington Seattle, WA ************************** From mcbeth at broggs.org Tue Jul 12 20:36:03 2022 From: mcbeth at broggs.org (Jeffrey Brent McBeth) Date: Tue, 12 Jul 2022 20:36:03 -0400 Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: References: Message-ID: On Tue, Jul 12, 2022 at 09:52:34AM -0700, Benjamin Weiner wrote: > For quantities that have a large value but where small differences may be > important, using double precision is often warranted.? Single precision > has a precision of 1 in 2^23 or about 6.9 dex in log. This translates to > of order 0.1 arcsecond in RA/Dec, or 9 minutes?of time on an MJD of 50000. > Propagating truncation errors through further code or processing can > magnify these effects, which is one reason that catalog makers carry > around the extra digits and internal routines/constants are often written > in double where explicit typing is needed. Time was going to be my point out. Unless I'm mistaken, astropy keeps time as two float64 and tries to keep the units in the first and fraction in the second. When we have to export to file, for our needs, even with MJD we store in float128, as float64 leavs us around 0.5?s, which is too much jitter for some activities involving fine propagation. The space and computational slowdown is worth it for our needs (even if we don't get the full 113 bits (or whatever) Jeffrey McBeth -- "The man who does not read good books has no advantage over the man who cannot read them." -- not Mark Twain, maybe a southen librarian in 1910 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: not available URL: From ivvv68 at gmail.com Wed Jul 13 03:42:36 2022 From: ivvv68 at gmail.com (Ivan Valtchanov) Date: Wed, 13 Jul 2022 09:42:36 +0200 Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: References: Message-ID: Hi all, Don't get me wrong. I am not advocating to use float32 in all cases of coordinates, times etc. I am working with a particular catalogue where I don't really see the need to use float64 for some of the columns. Of course it's up to the catalogue builders to decide. I've suffered some extreme cases of teams deciding to use double precision for everything float and as a result processing that data needed at least 64 GB RAM, no way to process it on a normal laptop. Cheers, Ivan On Wed, 13 Jul 2022 at 02:36, Jeffrey Brent McBeth wrote: > On Tue, Jul 12, 2022 at 09:52:34AM -0700, Benjamin Weiner wrote: > > For quantities that have a large value but where small differences > may be > > important, using double precision is often warranted. Single > precision > > has a precision of 1 in 2^23 or about 6.9 dex in log. This translates > to > > of order 0.1 arcsecond in RA/Dec, or 9 minutes of time on an MJD of > 50000. > > Propagating truncation errors through further code or processing can > > magnify these effects, which is one reason that catalog makers carry > > around the extra digits and internal routines/constants are often > written > > in double where explicit typing is needed. > > Time was going to be my point out. Unless I'm mistaken, astropy keeps > time as two float64 and tries to keep the units in the first and fraction > in > the second. When we have to export to file, for our needs, even with MJD > we store in float128, as float64 leavs us around 0.5?s, which is too > much jitter for some activities involving fine propagation. The space > and computational slowdown is worth it for our needs (even if we don't get > the full > 113 bits (or whatever) > > Jeffrey McBeth > > > -- > "The man who does not read good books has no advantage over > the man who cannot read them." > -- not Mark Twain, maybe a southen librarian in 1910 > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pdzwig at summaventures.com Wed Jul 13 03:50:07 2022 From: pdzwig at summaventures.com (Peter Dzwig) Date: Wed, 13 Jul 2022 08:50:07 +0100 Subject: [AstroPy] Summing an array In-Reply-To: References: <797C3379-14C1-4F16-ACCE-96F3F309C18C@swarthmore.edu> <61AB8479-23BF-4049-B170-BEF694242E58@gwdg.de> <1CD56A34-A6CE-4D10-91AA-BDE884BF206D@gwdg.de> Message-ID: <41505845-b60c-a259-405b-b9387c7c07dc@summaventures.com> On 12/07/2022 18:53, E. Madison Bray wrote: > > Personally I've soured a bit on the use of notebooks for research > fiddling. I think they're very good for presentation of working code > and data, and to some extent as lab notebooks for more experienced > users, but I think the capability for out-of-order execution and > re-running errors in previously run code hands too many foot-guns to > novices, compared to a normal, linear, REPL. > I agree about the potential for oddities creeping in ... and the only way to work with them is to re-initialise the kernel and clear down state fairly frequently. The integration of so many packages for displaying data etc makes them (fairly) straightforward to work with if you want to (e.g) get a quick look at that spectrum you just removed background from or whatever. To those of us who tried to run research programs using punched-cards.. :-) -- Dr. Peter Dzwig From juliocampagnolo at gmail.com Thu Jul 14 21:36:05 2022 From: juliocampagnolo at gmail.com (Julio Campagnolo) Date: Thu, 14 Jul 2022 22:36:05 -0300 Subject: [AstroPy] Help with Quantity in model fitting Message-ID: Hello everyone, Currently, I'm implementing a model for fitting using astropy and support for quantities is an important security factor. This model is a trigonometric function to compute Stokes parameters and has as input an array of retarder positions in angle quantity, dimensionless output, and dimensionless Stokes parameters. The model I implemented is: class QuarterWaveModel(Fittable1DModel): """Polarimetry z(psi) model for quarter wavelength retarder. Z= Q*cos(2psi)**2 + U*sin(2psi)*cos(2psi) - V*sin(2psi) psi in degrees. """ q = Parameter(default=0) u = Parameter(default=0) v = Parameter(default=0) zero = Parameter(default=0, unit=units.degree, fixed=True) @staticmethod def evaluate(x, q, u, v, zero): psi = np.radians(x+zero) psi2 = 2*psi r = q*(np.cos(psi2)**2) + u*np.sin(psi2)*np.cos(psi2) - v*np.sin(psi2) return r @property def input_units(self): return {self.inputs[0]: self.zero.unit} def _parameter_units_for_data_units(self, inputs_unit, outputs_unit): return {'q': outputs_unit[self.outputs[0]], 'u': outputs_unit[self.outputs[0]], 'v': outputs_unit[self.outputs[0]]} The model evaluates fine, but raise units error on fitting: q = 0.0130 u = -0.0021 v = 0.03044 zero = 60*units.degree pos = np.arange(0, 360.5, 22.5)*units.degree zi = QuarterWaveModel(q, u, v, zero)(pos) fitter = fitting.LevMarLSQFitter() m_fit = fitter(QuarterWaveModel(), pos, zi) # raises error 2113 if (not self.input_units_allow_dimensionless[input_name] and 2114 input_unit is not dimensionless_unscaled and 2115 input_unit is not None): 2116 if np.any(inputs[i] != 0):-> 2117 raise UnitsError("{0}: Units of input '{1}', (dimensionless), could not be " 2118 "converted to required input units of " 2119 "{2} ({3})".format(name, self.inputs[i], input_unit, 2120 input_unit.physical_type)) 2121 return inputs UnitsError: QuarterWaveModel: Units of input 'x', (dimensionless), could not be converted to required input units of deg (angle) What am I doing wrong? Julio -------------- next part -------------- An HTML attachment was scrubbed... URL: From jzuhone at gmail.com Wed Jul 20 11:38:59 2022 From: jzuhone at gmail.com (John Zuhone) Date: Wed, 20 Jul 2022 10:38:59 -0500 Subject: [AstroPy] Changing table column from float64 to float32 In-Reply-To: References: <3fb3e116-d600-5f2e-8709-2f18a4d19bde@astro.umass.edu> <07c8c80a6eadab40bebf5f4e113b888fd267a701.camel@sconseil.fr> Message-ID: > > 64bit floats are absolutely the correct data type for sky coordinates > these days. > For what it?s worth, this is not true for X-ray event data, which is stored in FITS binary tables and is frequently 32-bit. The PSF of Chandra, which has the highest angular resolution of any currently operational X-ray telescope, has a FWHM of 0.5 arcsecond. On Jul 12, 2022 at 3:59:43 PM, John K. Parejko wrote: > For the record, float32 is only around 10mas precision for ra/dec degrees. > For many modern datasets, that?s comparable with the nominal centroiding > accuracy (e.g. LSST expects ~10mas repeatability for bright sources). 64bit > floats are absolutely the correct data type for sky coordinates these days. > > John > > On 12Jul 2022, at 03:11, Ivan Valtchanov wrote: > > > Thanks all, > > > Yes, I managed to do the change of float64 to float32 columns using > astropy.table.Table. > > > It's funny how people keep float64 columns in huge catalogues for > something that's perfectly well stored with float32, e.g. RA, Dec, LII, > BII, MJD, etc... It's true that RAM is cheap nowadays but this laziness > puzzles me a lot (/rant_over). > > > Cheers, > > Ivan > > > > On Tue, 12 Jul 2022 at 10:21, Simon Conseil wrote: > > Indeed the BinTableHDU object doesn't have a method to easily replace a > > column (which is defined as float64 by the format='D' attribute in your > > example, so data assigned to the column will always be recast to > > float64). > > > One option would be to use table_hdu.columns.del_col and > > table_hdu.columns.add_col but this isn't the most practical. > > > In [3]: a = table_hdu.data['a'].astype('float32') > > > In [4]: cols = table_hdu.columns.del_col('a') > > > In [6]: cols = table_hdu.columns.add_col(fits.Column(name='a', array=a, > > format='E')) > > > In [8]: table_hdu = fits.BinTableHDU.from_columns(cols) > > > In [9]: table_hdu.columns > > Out[9]: > > ColDefs( > > name = 'b'; format = 'E' > > name = 'a'; format = 'E' > > ) > > > The easiest here would be to use astropy.table.Table: > > > In [3]: t = Table.read(table_hdu) > > > In [4]: t['a'] = t['a'].astype('float32') > > > In [5]: table_hdu = fits.table_to_hdu(t) > > > In [6]: table_hdu.columns > > Out[6]: > > ColDefs( > > name = 'a'; format = 'E' > > name = 'b'; format = 'E' > > ) > > > > Simon > > > On Tue, 2022-07-12 at 00:13 +0000, Homeier, Derek wrote: > > > On 11 Jul 2022, at 11:14 pm, Michael Brewer > > > wrote: > > > > > > > > Please try this: > > > > > > > > table_hdu.replace_column('a', > > > > table_hdu.data['a'].astype(np.single)) > > > > > > > > Michael > > > > > > > replace_column is a [astropy.table.]Table method; this won?t work on > > > a FITS BinTable ? > > > I think the most direct way is indeed to recreate the HDU from the > > > appropriately cast > > > Columns as outlined by Peter. Note that the HDU also contains the > > > corresponding header > > > information characterising the data section formats, so direct > > > manipulation of the data > > > parts is limited for a reason. > > > > > > > > On 7/11/22 12:17 PM, Peter Weilbacher wrote: > > > > > Hi Ivan, > > > > > > > > > > I have no particular insight into this, but it seems to me that > > > > > since > > > > > you created the Column objects with a specific format, assigning > > > > > any > > > > > data to it will cause it to be casted to match that. (Otherwise > > > > > the > > > > > underlying memory buffer does not match.) > > > > > > > > > > Instead, I guess you could create a new column with the desired > > > > > type and > > > > > format and recreate the table using that: > > > > > > > > > > c1_single = fits.Column(name='a', > > > > > array=table_hdu.data['a'].astype(np.single), format='E') > > > > > table_hdu2 = fits.BinTableHDU.from_columns(c1_single + > > > > > table_hdu.columns[1:]) > > > > > > > > > > (It appears that you can even remove the .astype() from this and > > > > > it > > > > > still works.) > > > > > > Yes, the `format=?E?` specifier will already recast to the respective > > > FITS (= Fortran) format. > > > > > > Cheers > > > _______________________________________________ > > > AstroPy mailing list > > > AstroPy at python.org > > > https://mail.python.org/mailman/listinfo/astropy > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at python.org > > https://mail.python.org/mailman/listinfo/astropy > > _______________________________________________ > > AstroPy mailing list > > AstroPy at python.org > > https://mail.python.org/mailman/listinfo/astropy > > > -- > ************************* > John Parejko > parejkoj at uw.edu > Rubin Observatory, DIRAC Fellow > http://staff.washington.edu/parejkoj/ > Department of Physics and Astronomy > University of Washington > Seattle, WA > ************************** > > > > > > > > > > > > > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: