New normalization option/kwarg for fft normalization
Hi all, on the NumPy mailing list (and PR) it came up that CuPy would like to the normalization in FFTs in the opposite way as the defaults. This is for technical reasons on the GPU, but we need to add a new `norm="something"` mode for it. And CuPy, numpy, and scipy should all be happy with the choice. The current favorits are: * "forward" ("backward" could be added to be identical to default) * "reversed" * "inverse" Currently, we scale the *inverse* transformation, instead of the direct transformation. A nice property of the `norm` kwarg is that round- tripping is defined if used always the same: x ~= ifft(fft(x, norm=...), norm=...) (assuming the lengths are good). So that disabling normalization entirely loses the property. If I read it correctly, "forward" is currently the crowd-favorite, and we will probably go ahead with something in a few days. But alternative proposals or opinions on which to choose are very welcome! Cheers, Sebastian
I am not sure if I understand it correctly but why do they have to be about the virtual direction of the transform. This might not be the canonical way but reads quite nice to my subjective opinion. fft(x, norm="by_N") fft(x, norm="by_sqrt_N") fft(x, norm=None) # Can even be "by_1" if OCD kicks in ifft(x, norm="by_n") # just to place examples with small n ifft(x, norm="by_sqrt_n") ifft(x, norm=None) Obviously, I'm assuming that these options are not typed every two minutes during the day. But then probably you have bigger problems than this. Unrelated : This naming always ignites questions from people I'm teaching or assisting with FFTs. "norm" is already a well-defined concept applicable to any vector-concept. "normalize" is also a well-defined concept specific to FFTs. Calling it "norm" would not have been my option. Saving 5 characters doesn't justify the confusion in my opinion. But anyways just wanted to mention in case any deprecation is on the horizon. But of course the ship has sailed long ago. On Sat, Jun 27, 2020 at 4:20 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
Hi all,
on the NumPy mailing list (and PR) it came up that CuPy would like to the normalization in FFTs in the opposite way as the defaults. This is for technical reasons on the GPU, but we need to add a new `norm="something"` mode for it. And CuPy, numpy, and scipy should all be happy with the choice.
The current favorits are:
* "forward" ("backward" could be added to be identical to default) * "reversed" * "inverse"
Currently, we scale the *inverse* transformation, instead of the direct transformation. A nice property of the `norm` kwarg is that round- tripping is defined if used always the same:
x ~= ifft(fft(x, norm=...), norm=...)
(assuming the lengths are good). So that disabling normalization entirely loses the property.
If I read it correctly, "forward" is currently the crowd-favorite, and we will probably go ahead with something in a few days. But alternative proposals or opinions on which to choose are very welcome!
Cheers,
Sebastian _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Sat, 2020-06-27 at 17:47 +0200, Ilhan Polat wrote:
I am not sure if I understand it correctly but why do they have to be about the virtual direction of the transform. This might not be the canonical way but reads quite nice to my subjective opinion.
I confess the direction notion confused me at first, it requires the step of realizing that this is about the pair of fft/ifft. OTOH, I am not if custom norm(alize) is actually used mainly in a context where you do both transforms. On small argument against the "by_n" approach is that both `"ortho"` and `None` use the directional notion already implicitly. Since below you would need the `ifft(x, norm="by_1")` to achieve the desired effect here as None will normalize the `ifft` by default. So using `by_n` or similar, works well, but may make a call without `norm` slightly more confusing (you may expect no normalization at all on the `ifft`)? Not sure its a big argument, though... - Sebastian
fft(x, norm="by_N") fft(x, norm="by_sqrt_N") fft(x, norm=None) # Can even be "by_1" if OCD kicks in
ifft(x, norm="by_n") # just to place examples with small n ifft(x, norm="by_sqrt_n") ifft(x, norm=None)
Obviously, I'm assuming that these options are not typed every twominutes during the day. But then probably you have bigger problems than this.
Unrelated : This naming always ignites questions from people I'm teaching or assisting with FFTs. "norm" is already a well-defined concept applicable to any vector-concept. "normalize" is also a well-defined concept specific to FFTs. Calling it "norm" would not have been my option. Saving 5 characters doesn't justify the confusion in my opinion. But anyways just wanted to mention in case any deprecation is on the horizon. But of course the ship has sailed long ago.
Indeed, can't say I disagree, but likely not worth the the confusion of switching/having two options. - Sebastian
On Sat, Jun 27, 2020 at 4:20 PM Sebastian Berg < sebastian@sipsolutions.net> wrote:
Hi all,
on the NumPy mailing list (and PR) it came up that CuPy would like to the normalization in FFTs in the opposite way as the defaults. This is for technical reasons on the GPU, but we need to add a new `norm="something"` mode for it. And CuPy, numpy, and scipy should all be happy with the choice.
The current favorits are:
* "forward" ("backward" could be added to be identical to default) * "reversed" * "inverse"
Currently, we scale the *inverse* transformation, instead of the direct transformation. A nice property of the `norm` kwarg is that round- tripping is defined if used always the same:
x ~= ifft(fft(x, norm=...), norm=...)
(assuming the lengths are good). So that disabling normalization entirely loses the property.
If I read it correctly, "forward" is currently the crowd-favorite, and we will probably go ahead with something in a few days. But alternative proposals or opinions on which to choose are very welcome!
Cheers,
Sebastian _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On small argument against the "by_n" approach is that both `"ortho"` and `None` use the directional notion already implicitly. Since below you would need the `ifft(x, norm="by_1")` to achieve the desired effect here as None will normalize the `ifft` by default.
One option would be to change the default None to an alias "inverse" to make it clearer. This could be done without a deprecation cycle. So using `by_n` or similar, works well, but may make a call without
`norm` slightly more confusing (you may expect no normalization at all on the `ifft`)? Not sure its a big argument, though...
To me the least ambiguous naming are: 1. Related to the transforms themselves, as these have the most clearly understood usage and naming for most people in most FFT/IFFT contexts that I've seen; and 2. Can be passed to both functions the same way to achieve round-trip equality (i.e., np.allclose(ifft(fft(x,norm=norm),norm=norm),x)). People who want to do something weird like no normalization in either direction are not the ones that I think should be prioritized here, as they are the exception and not the rule (far rarer I'd expect). So I'd prefer that the naming reflect the generally desirable round-trip equality. This makes it easiest for people working with FFT and IFFT pairs (or even using FFT and thinking about Parseval's relation of the resulting frequency-domain data) to understand what's going on. For example this is a pretty clear docstring to me (as is NumPy's current one <https://numpy.org/doc/stable/reference/routines.fft.html#normalization>): norm : {'forward', 'inverse', 'ortho', None} The normalization to apply during the FFT or IFFT, each of which will ensure round-trip numerical equivalence when the same value is passed to both forward and inverse transform functions: - "inverse" (default; same as None) will apply full normalization only during the IFFT (1/N); this follows the standard definition of the discrete Fourier transform and its inverse. - "forward" will apply full normalization only during the FFT (1/N). - "ortho" will apply half the normalization during the FFT and the IFFT (1/sqrt(N)), thereby ortho-normalizing the bases. One could argue that "ortho" could equivalently be "split" or "half" or something, but it already seems clear enough about what it will do (and even why it's potentially useful). My 2c, Eric
I am a bit suspicious of the need for round-trip-equality for the majority of the users. For one, I hardly ever used them as a pair for the same object. So this idempotent requirement seems a bit of an occupational hazard to me. All users that will perform a one-way transform and be done with it, are now going to be thinking about a direction that they have nothing to do with. I don't see why IFFT comes into play in the docstring of FFT. These are exclusively separate transformations. On Mon, Jun 29, 2020 at 1:34 AM Eric Larson <larson.eric.d@gmail.com> wrote:
On small argument against the "by_n" approach is that both `"ortho"` and `None` use the directional notion already implicitly. Since below you would need the `ifft(x, norm="by_1")` to achieve the desired effect here as None will normalize the `ifft` by default.
One option would be to change the default None to an alias "inverse" to make it clearer. This could be done without a deprecation cycle.
So using `by_n` or similar, works well, but may make a call without
`norm` slightly more confusing (you may expect no normalization at all on the `ifft`)? Not sure its a big argument, though...
To me the least ambiguous naming are:
1. Related to the transforms themselves, as these have the most clearly understood usage and naming for most people in most FFT/IFFT contexts that I've seen; and 2. Can be passed to both functions the same way to achieve round-trip equality (i.e., np.allclose(ifft(fft(x,norm=norm),norm=norm),x)).
People who want to do something weird like no normalization in either direction are not the ones that I think should be prioritized here, as they are the exception and not the rule (far rarer I'd expect). So I'd prefer that the naming reflect the generally desirable round-trip equality. This makes it easiest for people working with FFT and IFFT pairs (or even using FFT and thinking about Parseval's relation of the resulting frequency-domain data) to understand what's going on. For example this is a pretty clear docstring to me (as is NumPy's current one <https://numpy.org/doc/stable/reference/routines.fft.html#normalization>):
norm : {'forward', 'inverse', 'ortho', None} The normalization to apply during the FFT or IFFT, each of which will ensure round-trip numerical equivalence when the same value is passed to both forward and inverse transform functions:
- "inverse" (default; same as None) will apply full normalization only during the IFFT (1/N); this follows the standard definition of the discrete Fourier transform and its inverse. - "forward" will apply full normalization only during the FFT (1/N). - "ortho" will apply half the normalization during the FFT and the IFFT (1/sqrt(N)), thereby ortho-normalizing the bases.
One could argue that "ortho" could equivalently be "split" or "half" or something, but it already seems clear enough about what it will do (and even why it's potentially useful).
My 2c, Eric _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
As someone who uses FFTs a lot for filtering or applying operators in the spectral domain, I find the consistency of the FFT/IFFT pairs extremely important. I also have to say that I am not a big fan of the names for the normalisations that are currently discussed but I also don‘t have a better idea. Therefore, mentioning in the docstring how these normalisations affect the round trip is a good idea from my point of view. Hanno
On 29. Jun 2020, at 04:51, Ilhan Polat <ilhanpolat@gmail.com> wrote:
I am a bit suspicious of the need for round-trip-equality for the majority of the users. For one, I hardly ever used them as a pair for the same object. So this idempotent requirement seems a bit of an occupational hazard to me. All users that will perform a one-way transform and be done with it, are now going to be thinking about a direction that they have nothing to do with. I don't see why IFFT comes into play in the docstring of FFT. These are exclusively separate transformations.
On Mon, Jun 29, 2020 at 1:34 AM Eric Larson <larson.eric.d@gmail.com> wrote:
On small argument against the "by_n" approach is that both `"ortho"` and `None` use the directional notion already implicitly. Since below you would need the `ifft(x, norm="by_1")` to achieve the desired effect here as None will normalize the `ifft` by default.
One option would be to change the default None to an alias "inverse" to make it clearer. This could be done without a deprecation cycle.
So using `by_n` or similar, works well, but may make a call without `norm` slightly more confusing (you may expect no normalization at all on the `ifft`)? Not sure its a big argument, though...
To me the least ambiguous naming are:
1. Related to the transforms themselves, as these have the most clearly understood usage and naming for most people in most FFT/IFFT contexts that I've seen; and 2. Can be passed to both functions the same way to achieve round-trip equality (i.e., np.allclose(ifft(fft(x,norm=norm),norm=norm),x)).
People who want to do something weird like no normalization in either direction are not the ones that I think should be prioritized here, as they are the exception and not the rule (far rarer I'd expect). So I'd prefer that the naming reflect the generally desirable round-trip equality. This makes it easiest for people working with FFT and IFFT pairs (or even using FFT and thinking about Parseval's relation of the resulting frequency-domain data) to understand what's going on. For example this is a pretty clear docstring to me (as is NumPy's current one): norm : {'forward', 'inverse', 'ortho', None} The normalization to apply during the FFT or IFFT, each of which will ensure round-trip numerical equivalence when the same value is passed to both forward and inverse transform functions:
- "inverse" (default; same as None) will apply full normalization only during the IFFT (1/N); this follows the standard definition of the discrete Fourier transform and its inverse. - "forward" will apply full normalization only during the FFT (1/N). - "ortho" will apply half the normalization during the FFT and the IFFT (1/sqrt(N)), thereby ortho-normalizing the bases. One could argue that "ortho" could equivalently be "split" or "half" or something, but it already seems clear enough about what it will do (and even why it's potentially useful).
My 2c, Eric _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Ilhan, So are you suggesting that rather than using the same "backward" (or None), "ortho", or "forward" argument in fft and ifft you would specify something similar to the following?: # normalize by 1/N on the inverse transform y = fft(x, norm="1") # or norm=None to preserve backwards-compatibility x_roundtrip = ifft(y, norm="N") # or norm=None to preserving backwards-compatibility # normalize by 1/N on the forward transform y = fft(x, norm="N") x_roundtrip = ifft(y, norm="1") # normalize by 1/sqrt(N) on each y = fft(x, norm="ortho") x_roundtrip = ifft(y, norm="ortho") Neal Becker suggested something like the above in the NumPy thread, but with norm="full" to indicate normalization by 1/N I like the "backward"/"ortho"/"forward" approach where the same kwarg is used in both directions a little better, but agree that either approach has its merits. It seems that we can maintain backwards compatibility either way. On Sun, Jun 28, 2020 at 10:50 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I am a bit suspicious of the need for round-trip-equality for the majority of the users. For one, I hardly ever used them as a pair for the same object. So this idempotent requirement seems a bit of an occupational hazard to me. All users that will perform a one-way transform and be done with it, are now going to be thinking about a direction that they have nothing to do with. I don't see why IFFT comes into play in the docstring of FFT. These are exclusively separate transformations.
On Mon, Jun 29, 2020 at 1:34 AM Eric Larson <larson.eric.d@gmail.com> wrote:
On small argument against the "by_n" approach is that both `"ortho"` and `None` use the directional notion already implicitly. Since below you would need the `ifft(x, norm="by_1")` to achieve the desired effect here as None will normalize the `ifft` by default.
One option would be to change the default None to an alias "inverse" to make it clearer. This could be done without a deprecation cycle.
So using `by_n` or similar, works well, but may make a call without
`norm` slightly more confusing (you may expect no normalization at all on the `ifft`)? Not sure its a big argument, though...
To me the least ambiguous naming are:
1. Related to the transforms themselves, as these have the most clearly understood usage and naming for most people in most FFT/IFFT contexts that I've seen; and 2. Can be passed to both functions the same way to achieve round-trip equality (i.e., np.allclose(ifft(fft(x,norm=norm),norm=norm),x)).
People who want to do something weird like no normalization in either direction are not the ones that I think should be prioritized here, as they are the exception and not the rule (far rarer I'd expect). So I'd prefer that the naming reflect the generally desirable round-trip equality. This makes it easiest for people working with FFT and IFFT pairs (or even using FFT and thinking about Parseval's relation of the resulting frequency-domain data) to understand what's going on. For example this is a pretty clear docstring to me (as is NumPy's current one <https://numpy.org/doc/stable/reference/routines.fft.html#normalization> ):
norm : {'forward', 'inverse', 'ortho', None} The normalization to apply during the FFT or IFFT, each of which will ensure round-trip numerical equivalence when the same value is passed to both forward and inverse transform functions:
- "inverse" (default; same as None) will apply full normalization only during the IFFT (1/N); this follows the standard definition of the discrete Fourier transform and its inverse. - "forward" will apply full normalization only during the FFT (1/N). - "ortho" will apply half the normalization during the FFT and the IFFT (1/sqrt(N)), thereby ortho-normalizing the bases.
One could argue that "ortho" could equivalently be "split" or "half" or something, but it already seems clear enough about what it will do (and even why it's potentially useful).
My 2c, Eric _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
To be honest I changed my mind at least 3 times while typing this. On one hand this direction stuff doesn't make any sense to me. On the other hand it makes sense to many people so maybe I should shut up about it. I view this as a single command namely FFT. I transform my sequence and I might enjoy some normalization such that certain properties match with the original signal I've started with or I don't care about normalization because I work with the relative quantities. Fortunately I have a keyword "norm" to control this. It's value is called "forward" and another value is "inverse". Probably I can't justify this part. On Mon, Jun 29, 2020, 21:46 Gregory Lee <grlee77@gmail.com> wrote:
Ilhan,
So are you suggesting that rather than using the same "backward" (or None), "ortho", or "forward" argument in fft and ifft you would specify something similar to the following?:
# normalize by 1/N on the inverse transform y = fft(x, norm="1") # or norm=None to preserve backwards-compatibility x_roundtrip = ifft(y, norm="N") # or norm=None to preserving backwards-compatibility
# normalize by 1/N on the forward transform y = fft(x, norm="N") x_roundtrip = ifft(y, norm="1")
# normalize by 1/sqrt(N) on each y = fft(x, norm="ortho") x_roundtrip = ifft(y, norm="ortho")
Neal Becker suggested something like the above in the NumPy thread, but with norm="full" to indicate normalization by 1/N
I like the "backward"/"ortho"/"forward" approach where the same kwarg is used in both directions a little better, but agree that either approach has its merits. It seems that we can maintain backwards compatibility either way.
On Sun, Jun 28, 2020 at 10:50 PM Ilhan Polat <ilhanpolat@gmail.com> wrote:
I am a bit suspicious of the need for round-trip-equality for the majority of the users. For one, I hardly ever used them as a pair for the same object. So this idempotent requirement seems a bit of an occupational hazard to me. All users that will perform a one-way transform and be done with it, are now going to be thinking about a direction that they have nothing to do with. I don't see why IFFT comes into play in the docstring of FFT. These are exclusively separate transformations.
On Mon, Jun 29, 2020 at 1:34 AM Eric Larson <larson.eric.d@gmail.com> wrote:
On small argument against the "by_n" approach is that both `"ortho"` and `None` use the directional notion already implicitly. Since below you would need the `ifft(x, norm="by_1")` to achieve the desired effect here as None will normalize the `ifft` by default.
One option would be to change the default None to an alias "inverse" to make it clearer. This could be done without a deprecation cycle.
So using `by_n` or similar, works well, but may make a call without
`norm` slightly more confusing (you may expect no normalization at all on the `ifft`)? Not sure its a big argument, though...
To me the least ambiguous naming are:
1. Related to the transforms themselves, as these have the most clearly understood usage and naming for most people in most FFT/IFFT contexts that I've seen; and 2. Can be passed to both functions the same way to achieve round-trip equality (i.e., np.allclose(ifft(fft(x,norm=norm),norm=norm),x)).
People who want to do something weird like no normalization in either direction are not the ones that I think should be prioritized here, as they are the exception and not the rule (far rarer I'd expect). So I'd prefer that the naming reflect the generally desirable round-trip equality. This makes it easiest for people working with FFT and IFFT pairs (or even using FFT and thinking about Parseval's relation of the resulting frequency-domain data) to understand what's going on. For example this is a pretty clear docstring to me (as is NumPy's current one <https://numpy.org/doc/stable/reference/routines.fft.html#normalization> ):
norm : {'forward', 'inverse', 'ortho', None} The normalization to apply during the FFT or IFFT, each of which will ensure round-trip numerical equivalence when the same value is passed to both forward and inverse transform functions:
- "inverse" (default; same as None) will apply full normalization only during the IFFT (1/N); this follows the standard definition of the discrete Fourier transform and its inverse. - "forward" will apply full normalization only during the FFT (1/N). - "ortho" will apply half the normalization during the FFT and the IFFT (1/sqrt(N)), thereby ortho-normalizing the bases.
One could argue that "ortho" could equivalently be "split" or "half" or something, but it already seems clear enough about what it will do (and even why it's potentially useful).
My 2c, Eric _______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
participants (5)
-
Eric Larson -
Gregory Lee -
Hanno Klemm -
Ilhan Polat -
Sebastian Berg