Hi, I'm having some trouble making the lazy_import "implicit", I tried following approaches:
1)overriding "__import__ " using 'builtins.__import__ = lazy_import.lazy_import_module' but this can't take more than 2 args whereas "__import__" takes upto 5 args, so now I have to make changes to "_bootstrap.py"(which I shouldn't probably make) and __init__.py to accomadate everything new.
2)Using "meta_path" and "path_hooks" to register custom "Loader" and "finder" but somehow it's breaking during 'make' ...probably due to 'finder'(unable to import stuff like '.org*')...Working on this.
3)The Whitelist option(that Antoine suggested) ,somewhat hardcode, i.e. to identify modules and in place of "import" add the lazy_import_module function.
Any suggestion?
Regards, Bhavishya
On Sun, Jun 25, 2017 at 12:52 PM, Bhavishya <bhavishyagopesh@gmail.com> wrote:
Also, the "fatoptimizer" project's compile fails due to missing "ma_version_tag" (and works on changing it to "ma_version").
Regards, Bhavishya
On Sun, Jun 25, 2017 at 12:32 PM, Bhavishya <bhavishyagopesh@gmail.com> wrote:
Hello, I have added the "lazy_import" <https://github.com/bhavishyagopesh/gsoc_2017/blob/master/python_startup_time/lazy_loader.py> function but still working on adding it implicitly(To ensure that at startup it is actually used.)
Thanks Antoine for the suggestion
Regards, Bhavishya
On Sat, Jun 24, 2017 at 9:30 PM, <speed-request@python.org> wrote:
Send Speed mailing list submissions to speed@python.org
To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/speed or, via email, send a message with subject or body 'help' to speed-request@python.org
You can reach the person managing the list at speed-owner@python.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Speed digest..."
Today's Topics:
- Re: Lazy-loading to decrease python_startup time (Brett Cannon)
- Re: Lazy-loading to decrease python_startup time (Antoine Pitrou)
Message: 1 Date: Fri, 23 Jun 2017 23:03:57 +0000 From: Brett Cannon <brett@python.org> To: Bhavishya <bhavishyagopesh@gmail.com>, speed@python.org, Ramya Meruva <meruvaramya116@gmail.com>, Victor Stinner <victor.stinner@gmail.com> Subject: Re: [Speed] Lazy-loading to decrease python_startup time Message-ID: <CAP1=2W41_xTDt7gkuUWnmYA=+Bz9Ox-XT-pAjtXkDGg4TDrY5Q@mail.gm ail.com> Content-Type: text/plain; charset="utf-8"
On Fri, 23 Jun 2017 at 12:17 Bhavishya <bhavishyagopesh@gmail.com> wrote:
As suggested, I'd like to discuss if lazy-loading is an option for improving python-startup time.And if could be done inside the scope of a GSoc project.
It's a possibility and it could be done in the scope of a GSoC project easily. Basically what would be needed is an importlib.util.lazy_import() function which does mostly what is outlined in https://docs.python.org/3/library/importlib.html#approximati ng-importlib-import-module but where the proper lazy loader is set on the spec object as an extra step. Then every module that is used during startup would use importlib.util.lazy_import() for importing instead of the normal import statement. What this would do is help guarantee that all modules that are identified as part of startup never import a module needlessly as the lazy loader would simply postpone the load until necessary. This would also allow for pulling out local imports that are currently done in modules that are a part of startup and make them global so they are more visible.
But I have no idea if this will actually speed things up. :) At worst it would slow things down ever so slightly due to the extra overhead of lazy loading for things that are known to be needed already during startup. At best, though, is we accidentally discover modules that are being imported needlessly at startup as well as not having to hide imports in functions for performance. This fact that it may not be worth it is why I haven't bothered to try it out yet.