Hi, I'm having some trouble making the lazy_import "implicit", I tried following approaches:
1)overriding "__import__ " using 'builtins.__import__ = lazy_import.lazy_import_module' but this can't take more than 2 args whereas "__import__" takes upto 5 args, so now I have to make changes to "_bootstrap.py"(which I shouldn't probably make) and __init__.py to accomadate everything new.
2)Using "meta_path" and "path_hooks" to register custom "Loader" and "finder" but somehow it's breaking during 'make' ...probably due to 'finder'(unable to import stuff like '.org*')...Working on this.
3)The Whitelist option(that Antoine suggested) ,somewhat hardcode, i.e. to identify modules and in place of "import" add the lazy_import_module function.
On Sun, Jun 25, 2017 at 12:52 PM, Bhavishya firstname.lastname@example.org wrote:
Also, the "fatoptimizer" project's compile fails due to missing "ma_version_tag" (and works on changing it to "ma_version").
On Sun, Jun 25, 2017 at 12:32 PM, Bhavishya email@example.com wrote:
Hello, I have added the "lazy_import" https://github.com/bhavishyagopesh/gsoc_2017/blob/master/python_startup_time/lazy_loader.py function but still working on adding it implicitly(To ensure that at startup it is actually used.)
Thanks Antoine for the suggestion
On Sat, Jun 24, 2017 at 9:30 PM, firstname.lastname@example.org wrote:
Send Speed mailing list submissions to email@example.com
To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/speed or, via email, send a message with subject or body 'help' to firstname.lastname@example.org
You can reach the person managing the list at email@example.com
When replying, please edit your Subject line so it is more specific than "Re: Contents of Speed digest..."
- Re: Lazy-loading to decrease python_startup time (Brett Cannon)
- Re: Lazy-loading to decrease python_startup time (Antoine Pitrou)
Message: 1 Date: Fri, 23 Jun 2017 23:03:57 +0000 From: Brett Cannon firstname.lastname@example.org To: Bhavishya email@example.com, firstname.lastname@example.org, Ramya Meruva email@example.com, Victor Stinner firstname.lastname@example.org Subject: Re: [Speed] Lazy-loading to decrease python_startup time Message-ID: <CAP1=2W41_xTDt7gkuUWnmYA=+Bz9Ox-XT-pAjtXkDGg4TDrY5Q@mail.gm ail.com> Content-Type: text/plain; charset="utf-8"
On Fri, 23 Jun 2017 at 12:17 Bhavishya email@example.com wrote:
As suggested, I'd like to discuss if lazy-loading is an option for improving python-startup time.And if could be done inside the scope of
It's a possibility and it could be done in the scope of a GSoC project easily. Basically what would be needed is an importlib.util.lazy_import() function which does mostly what is outlined in https://docs.python.org/3/library/importlib.html#approximati ng-importlib-import-module but where the proper lazy loader is set on the spec object as an extra step. Then every module that is used during startup would use importlib.util.lazy_import() for importing instead of the normal import statement. What this would do is help guarantee that all modules that are identified as part of startup never import a module needlessly as the lazy loader would simply postpone the load until necessary. This would also allow for pulling out local imports that are currently done in modules that are a part of startup and make them global so they are more visible.
But I have no idea if this will actually speed things up. :) At worst it would slow things down ever so slightly due to the extra overhead of lazy loading for things that are known to be needed already during startup. At best, though, is we accidentally discover modules that are being imported needlessly at startup as well as not having to hide imports in functions for performance. This fact that it may not be worth it is why I haven't bothered to try it out yet.