Also, the "fatoptimizer" project's compile fails due to missing "ma_version_tag" (and works on changing it to "ma_version").
Regards,
BhavishyaOn Sun, Jun 25, 2017 at 12:32 PM, Bhavishya <bhavishyagopesh@gmail.com> wrote:BhavishyaRegards,Thanks Antoine for the suggestionHello,I have added the "lazy_import" function but still working on adding it implicitly(To ensure that at startup it is actually used.)On Sat, Jun 24, 2017 at 9:30 PM, <speed-request@python.org> wrote:Send Speed mailing list submissions to
speed@python.org
To subscribe or unsubscribe via the World Wide Web, visit
https://mail.python.org/mailman/listinfo/speed
or, via email, send a message with subject or body 'help' to
speed-request@python.org
You can reach the person managing the list at
speed-owner@python.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Speed digest..."
Today's Topics:
1. Re: Lazy-loading to decrease python_startup time (Brett Cannon)
2. Re: Lazy-loading to decrease python_startup time (Antoine Pitrou)
------------------------------------------------------------ ----------
Message: 1
Date: Fri, 23 Jun 2017 23:03:57 +0000
From: Brett Cannon <brett@python.org>
To: Bhavishya <bhavishyagopesh@gmail.com>, speed@python.org, Ramya
Meruva <meruvaramya116@gmail.com>, Victor Stinner
<victor.stinner@gmail.com>
Subject: Re: [Speed] Lazy-loading to decrease python_startup time
Message-ID:
<CAP1=2W41_xTDt7gkuUWnmYA=+Bz9Ox-XT-pAjtXkDGg4TDrY5Q@mail.gm >ail.com
Content-Type: text/plain; charset="utf-8"
On Fri, 23 Jun 2017 at 12:17 Bhavishya <bhavishyagopesh@gmail.com> wrote:
> As suggested, I'd like to discuss if lazy-loading is an option for
> improving python-startup time.And if could be done inside the scope of a
> GSoc project.
>
It's a possibility and it could be done in the scope of a GSoC project
easily. Basically what would be needed is an importlib.util.lazy_import()
function which does mostly what is outlined in
https://docs.python.org/3/library/importlib.html#approximati ng-importlib-import-module
but
where the proper lazy loader is set on the spec object as an extra step.
Then every module that is used during startup would use
importlib.util.lazy_import() for importing instead of the normal import
statement. What this would do is help guarantee that all modules that are
identified as part of startup never import a module needlessly as the lazy
loader would simply postpone the load until necessary. This would also
allow for pulling out local imports that are currently done in modules that
are a part of startup and make them global so they are more visible.
But I have no idea if this will actually speed things up. :) At worst it
would slow things down ever so slightly due to the extra overhead of lazy
loading for things that are known to be needed already during startup. At
best, though, is we accidentally discover modules that are being imported
needlessly at startup as well as not having to hide imports in functions
for performance. This fact that it may not be worth it is why I haven't
bothered to try it out yet.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/speed/attachments/20170623 /ed7416e2/attachment-0001.html >
------------------------------
Message: 2
Date: Sat, 24 Jun 2017 10:59:49 +0200
From: Antoine Pitrou <solipsis@pitrou.net>
To: speed@python.org
Subject: Re: [Speed] Lazy-loading to decrease python_startup time
Message-ID: <20170624105949.62590bbf@fsol>
Content-Type: text/plain; charset=US-ASCII
On Fri, 23 Jun 2017 23:03:57 +0000
Brett Cannon <brett@python.org> wrote:
> On Fri, 23 Jun 2017 at 12:17 Bhavishya <bhavishyagopesh@gmail.com> wrote:
>
> > As suggested, I'd like to discuss if lazy-loading is an option for
> > improving python-startup time.And if could be done inside the scope of a
> > GSoc project.
> >
>
> It's a possibility and it could be done in the scope of a GSoC project
> easily. Basically what would be needed is an importlib.util.lazy_import()
> function which does mostly what is outlined in
> https://docs.python.org/3/library/importlib.html#approximati ng-importlib-import-module
> but
> where the proper lazy loader is set on the spec object as an extra step.
> Then every module that is used during startup would use
> importlib.util.lazy_import() for importing instead of the normal import
> statement.
My experience is that:
- you want lazy imports to be implicit, i.e. they should work using the
"import" statement and not any special syntax or function invocation
- you want a blacklist and/or whitelist mechanism to restrict lazy
imports to a particular set of modules and packages, because some
modules may not play well with lazy importing (e.g. anything that
registers plugins, specializations -- think singledispatch --, etc.)
For example, I may want to register the "tornado", "asyncio" and "numpy"
namespaces / packages for lazy importing, but not the "numba" namespace
as it uses registration mechanisms quite heavily.
(and the stdlib could be part of the default lazy import whitelist)
Regards
Antoine.
------------------------------
Subject: Digest Footer
_______________________________________________
Speed mailing list
Speed@python.org
https://mail.python.org/mailman/listinfo/speed
------------------------------
End of Speed Digest, Vol 36, Issue 1
************************************