
2017-07-21 4:52 GMT+02:00 Nick Coghlan <ncoghlan@gmail.com>:
On 21 July 2017 at 12:44, Nick Coghlan <ncoghlan@gmail.com> wrote:
We can separately measure the cost of unmarshalling the code object:
$ python3 -m perf timeit -s "import typing; from marshal import loads; from importlib.util import cache_from_source; cache = cache_from_source(typing.__file__); data = open(cache, 'rb').read()[12:]" "loads(data)" ..................... Mean +- std dev: 286 us +- 4 us
Slight adjustment here, as the cost of locating the cached bytecode and reading it from disk should really be accounted for in each iteration:
$ python3 -m perf timeit -s "import typing; from marshal import loads; from importlib.util import cache_from_source" "cache = cache_from_source(typing.__spec__.origin); data = open(cache, 'rb').read()[12:]; loads(data)" ..................... Mean +- std dev: 337 us +- 8 us
That will have a bigger impact when loading from spinning disk or a network drive, but it's fairly negligible when loading from a local SSD or an already primed filesystem cache.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Thanks for your tests, Nick. It's quite evident that the marshal code cannot improve the situation, so I regret from my proposal. I took a look at the typing module, and there are some small things that can be optimized, but it'll not change the overall situation unfortunately. Code execution can be improved. :) However, it requires a massive amount of time experimenting... Bests, Cesare <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Mail priva di virus. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>