Hi Everyone, I am using numpy on pypy to train a deep neural network. For my workload numpy on pypy is taking twice the time to train as numpy on Cpython. I am using Numpy via cpyext. I read in the documentation, "Performance-wise, the speed is mostly the same as CPython's NumPy (it is the same code); the exception is that interactions between the Python side and NumPy objects are mediated through the slower cpyext layer (which hurts a few benchmarks that do a lot of element-by-element array accesses, for example)." Is there any way in which I can profile my application to see how much additional overhead cypext layer is adding or is it the numpy via pypy which is slowing down the things. I have tried vmprof, but I couldn't figure out from it how much time cpyext layer is taking. Any help will be highly appreciated. Regards Yash
Hi Yash Is your software open source? I'm happy to check it out for you I think the c-level profiling for vmprof is relatively new, you would need to use pypy nightly in order to get that level of insight. Additionally, we're working on cpyext improvements *right now* stay tuned. If there is a good case for speeding up numpy, we can get it a lot faster than it is right now and seek some funding for that. Neural networks might be one of those! Best regards, Maciej Fijalkowski On Fri, Mar 3, 2017 at 2:31 AM, Singh, Yashwardhan <yashwardhan.singh@intel.com> wrote:
Hi Everyone,
I am using numpy on pypy to train a deep neural network. For my workload numpy on pypy is taking twice the time to train as numpy on Cpython. I am using Numpy via cpyext.
I read in the documentation, "Performance-wise, the speed is mostly the same as CPython's NumPy (it is the same code); the exception is that interactions between the Python side and NumPy objects are mediated through the slower cpyext layer (which hurts a few benchmarks that do a lot of element-by-element array accesses, for example)." Is there any way in which I can profile my application to see how much additional overhead cypext layer is adding or is it the numpy via pypy which is slowing down the things. I have tried vmprof, but I couldn't figure out from it how much time cpyext layer is taking.
Any help will be highly appreciated.
Regards Yash
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Hi Maciej, I have applied for clearance to publicly upload the code. I will upload it once I get the permission. Regards Yash -----Original Message----- From: Maciej Fijalkowski [mailto:fijall@gmail.com] Sent: Friday, March 3, 2017 4:41 AM To: Singh, Yashwardhan <yashwardhan.singh@intel.com> Cc: pypy-dev@python.org Subject: Re: [pypy-dev] Numpy on PyPy : cpyext Hi Yash Is your software open source? I'm happy to check it out for you I think the c-level profiling for vmprof is relatively new, you would need to use pypy nightly in order to get that level of insight. Additionally, we're working on cpyext improvements *right now* stay tuned. If there is a good case for speeding up numpy, we can get it a lot faster than it is right now and seek some funding for that. Neural networks might be one of those! Best regards, Maciej Fijalkowski On Fri, Mar 3, 2017 at 2:31 AM, Singh, Yashwardhan <yashwardhan.singh@intel.com> wrote:
Hi Everyone,
I am using numpy on pypy to train a deep neural network. For my workload numpy on pypy is taking twice the time to train as numpy on Cpython. I am using Numpy via cpyext.
I read in the documentation, "Performance-wise, the speed is mostly the same as CPython's NumPy (it is the same code); the exception is that interactions between the Python side and NumPy objects are mediated through the slower cpyext layer (which hurts a few benchmarks that do a lot of element-by-element array accesses, for example)." Is there any way in which I can profile my application to see how much additional overhead cypext layer is adding or is it the numpy via pypy which is slowing down the things. I have tried vmprof, but I couldn't figure out from it how much time cpyext layer is taking.
Any help will be highly appreciated.
Regards Yash
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
participants (2)
-
Maciej Fijalkowski
-
Singh, Yashwardhan