instantiating interp1d() is pathologically slow for me, why?
I've just posted this question in SO as well, happy with an answer either place: https://stackoverflow.com/q/49427533/3904031 The following takes over a minute for a few thousand points, whereas it seems it should be taking milliseconds. np.__version__ '1.13.0' scipy.__version__ '0.17.0' https://i.stack.imgur.com/3C69J.png import time import numpy as np import matplotlib.pyplot as plt from scipy.interpolate import interp1d times = [] for n in np.logspace(1, 3.5, 6).astype(int): x = np.arange(n, dtype=float) y = np.vstack((np.cos(x), np.sin(x))) start = time.clock() bob = interp1d(x, y, kind='quadratic', assume_sorted=True) times.append((n, time.clock() - start)) n, tim = zip(*times) plt.figure() plt.plot(n, tim) plt.xscale('log') plt.yscale('log') plt.show()
Why not profile it using the cprofile module in Python? (Look for it in the docs.) Then you can see exactly where the bottleneck is. You can view the output directly or use either snakeviz or cprofilev from pypi to view results interactively. (Search Google for "snakeviz" or "ymichael cprofilev".) On Thu, Mar 22, 2018, 4:46 AM David Mikolas <david.mikolas1@gmail.com> wrote:
I've just posted this question in SO as well, happy with an answer either place:
https://stackoverflow.com/q/49427533/3904031
The following takes over a minute for a few thousand points, whereas it seems it should be taking milliseconds.
np.__version__ '1.13.0' scipy.__version__ '0.17.0'
https://i.stack.imgur.com/3C69J.png
import time import numpy as np import matplotlib.pyplot as plt from scipy.interpolate import interp1d
times = [] for n in np.logspace(1, 3.5, 6).astype(int): x = np.arange(n, dtype=float) y = np.vstack((np.cos(x), np.sin(x))) start = time.clock() bob = interp1d(x, y, kind='quadratic', assume_sorted=True) times.append((n, time.clock() - start))
n, tim = zip(*times)
plt.figure() plt.plot(n, tim) plt.xscale('log') plt.yscale('log') plt.show()
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
Mark thank you for the suggestion. I'd forgotten that I posted this here. It turns out, much to my surprise, that even though interp1d has been in SciPy for many years, it was refactored and this was included in v0.19, leading to literally a half-dozen orders of magnitude speed up for say 10,000 points. So all is well. See this SO answer: https://stackoverflow.com/a/49428804/3904031 On Fri, Mar 23, 2018 at 11:28 AM, Mark Alexander Mikofski < mikofski@berkeley.edu> wrote:
Why not profile it using the cprofile module in Python? (Look for it in the docs.) Then you can see exactly where the bottleneck is. You can view the output directly or use either snakeviz or cprofilev from pypi to view results interactively. (Search Google for "snakeviz" or "ymichael cprofilev".)
On Thu, Mar 22, 2018, 4:46 AM David Mikolas <david.mikolas1@gmail.com> wrote:
I've just posted this question in SO as well, happy with an answer either place:
https://stackoverflow.com/q/49427533/3904031
The following takes over a minute for a few thousand points, whereas it seems it should be taking milliseconds.
np.__version__ '1.13.0' scipy.__version__ '0.17.0'
https://i.stack.imgur.com/3C69J.png
import time import numpy as np import matplotlib.pyplot as plt from scipy.interpolate import interp1d
times = [] for n in np.logspace(1, 3.5, 6).astype(int): x = np.arange(n, dtype=float) y = np.vstack((np.cos(x), np.sin(x))) start = time.clock() bob = interp1d(x, y, kind='quadratic', assume_sorted=True) times.append((n, time.clock() - start))
n, tim = zip(*times)
plt.figure() plt.plot(n, tim) plt.xscale('log') plt.yscale('log') plt.show()
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
participants (2)
-
David Mikolas -
Mark Alexander Mikofski