Re: [Python-Dev] [Core-mentorship] C version of weakrefset.py.
Hello,
1)I might be totally wrong here, but even if we go with stripping
annotation(in .pyc)...still the "lag" that comes from ABCs needs to be
addressed.
2) I been reading for past few days about your fat-optimizer project and
the corresponding TO-DO list, if you think that PEP-0511 should be
improved, I can work on that.
3)Also I was seeing to existing repos which implement some-kind of
optimizaton, like
numpy,snake-oil....
Finally I wanted to decide upon a roadmap, so that I could put more
specific efforts.(fat optimizer?)
Thank You
Regards,
Bhavishya
On Wed, Jun 21, 2017 at 8:05 PM, Victor Stinner
2017-06-21 15:21 GMT+02:00 INADA Naoki
: ABC slowdown Python startup only 2ms. But importing typing module take 11ms. While typing is not imported from site.py, many new Python application will import it. It may take over 100ms for applications or libraries heavily depending on ABCs.
When typing is not used in the application, only used for static checks, you can try to "strip" annotations to avoid any overhead on the application startup. It's not only a matter of "import typing", it's also the cost of instanciating types like "List[int]" (or even more complex ones).
I discussed with Jukka Lehtosalo at Pycon US about stripping completely annotations. He told me that my PEP 511 may be a good solution to keep annotation in the .py code, but strip them for "production code", in the cached .pyc files:
"PEP 511 -- API for code transformers" https://www.python.org/dev/peps/pep-0511/
This PEP is somehow controversal. Some people fear that it would allow people to hack the Python language to write their own incompatible variant of Python. I don't think that my PEP adds anything new, it's already possible to do that, importlib made it even more easy. I used my FAT Python optimizer project to sell this PEP. Since FAT Python is also controversal (it hasn't been proved to be actually faster), the PEP didn't go far at my last attempt.
Note: Is core-menthorship the best place for such performance discussion? :-) Maybe we should open a thread on python-dev@ or speed@ mailing list.
Victor
Also I saw your conversation with "Brett Cannon" on lazy-loading some
modules at startup, and also doing so using ModuleProxy(also several
implementations that exist like importlib's Lazyloader
https://docs.python.org/3/library/importlib.html?highlight=lazy%20import#imp...,
PEAK http://peak.telecommunity.com/DevCenter/Importing ).
So your suggestion's on this too?
Thank You
regards,
Bhavishya
On Thu, Jun 22, 2017 at 7:24 PM, Bhavishya
Hello, 1)I might be totally wrong here, but even if we go with stripping annotation(in .pyc)...still the "lag" that comes from ABCs needs to be addressed.
2) I been reading for past few days about your fat-optimizer project and the corresponding TO-DO list, if you think that PEP-0511 should be improved, I can work on that.
3)Also I was seeing to existing repos which implement some-kind of optimizaton, like numpy,snake-oil....
Finally I wanted to decide upon a roadmap, so that I could put more specific efforts.(fat optimizer?)
Thank You Regards, Bhavishya
On Wed, Jun 21, 2017 at 8:05 PM, Victor Stinner
wrote: 2017-06-21 15:21 GMT+02:00 INADA Naoki
: ABC slowdown Python startup only 2ms. But importing typing module take 11ms. While typing is not imported from site.py, many new Python application will import it. It may take over 100ms for applications or libraries heavily depending on ABCs.
When typing is not used in the application, only used for static checks, you can try to "strip" annotations to avoid any overhead on the application startup. It's not only a matter of "import typing", it's also the cost of instanciating types like "List[int]" (or even more complex ones).
I discussed with Jukka Lehtosalo at Pycon US about stripping completely annotations. He told me that my PEP 511 may be a good solution to keep annotation in the .py code, but strip them for "production code", in the cached .pyc files:
"PEP 511 -- API for code transformers" https://www.python.org/dev/peps/pep-0511/
This PEP is somehow controversal. Some people fear that it would allow people to hack the Python language to write their own incompatible variant of Python. I don't think that my PEP adds anything new, it's already possible to do that, importlib made it even more easy. I used my FAT Python optimizer project to sell this PEP. Since FAT Python is also controversal (it hasn't been proved to be actually faster), the PEP didn't go far at my last attempt.
Note: Is core-menthorship the best place for such performance discussion? :-) Maybe we should open a thread on python-dev@ or speed@ mailing list.
Victor
On Thu, 22 Jun 2017 at 16:26 Bhavishya
Also I saw your conversation with "Brett Cannon" on lazy-loading some modules at startup, and also doing so using ModuleProxy(also several implementations that exist like importlib's Lazyloader https://docs.python.org/3/library/importlib.html?highlight=lazy%20import#imp..., PEAK http://peak.telecommunity.com/DevCenter/Importing ). So your suggestion's on this too?
As stated earlier, if you wish to discuss seeing if lazy loading will help in Python's startup performance then please start a thread over at speed@python.org and we can discuss it there. -Brett
Thank You
regards, Bhavishya
On Thu, Jun 22, 2017 at 7:24 PM, Bhavishya
wrote: Hello, 1)I might be totally wrong here, but even if we go with stripping annotation(in .pyc)...still the "lag" that comes from ABCs needs to be addressed.
2) I been reading for past few days about your fat-optimizer project and the corresponding TO-DO list, if you think that PEP-0511 should be improved, I can work on that.
3)Also I was seeing to existing repos which implement some-kind of optimizaton, like numpy,snake-oil....
Finally I wanted to decide upon a roadmap, so that I could put more specific efforts.(fat optimizer?)
Thank You Regards, Bhavishya
On Wed, Jun 21, 2017 at 8:05 PM, Victor Stinner
wrote:
2017-06-21 15:21 GMT+02:00 INADA Naoki
: ABC slowdown Python startup only 2ms. But importing typing module take 11ms. While typing is not imported from site.py, many new Python application will import it. It may take over 100ms for applications or libraries heavily depending on ABCs.
When typing is not used in the application, only used for static checks, you can try to "strip" annotations to avoid any overhead on the application startup. It's not only a matter of "import typing", it's also the cost of instanciating types like "List[int]" (or even more complex ones).
I discussed with Jukka Lehtosalo at Pycon US about stripping completely annotations. He told me that my PEP 511 may be a good solution to keep annotation in the .py code, but strip them for "production code", in the cached .pyc files:
"PEP 511 -- API for code transformers" https://www.python.org/dev/peps/pep-0511/
This PEP is somehow controversal. Some people fear that it would allow people to hack the Python language to write their own incompatible variant of Python. I don't think that my PEP adds anything new, it's already possible to do that, importlib made it even more easy. I used my FAT Python optimizer project to sell this PEP. Since FAT Python is also controversal (it hasn't been proved to be actually faster), the PEP didn't go far at my last attempt.
Note: Is core-menthorship the best place for such performance discussion? :-) Maybe we should open a thread on python-dev@ or speed@ mailing list.
Victor
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org
participants (2)
-
Bhavishya
-
Brett Cannon