[Python-Dev] this is what happens if you freeze all the modules required for startup

[Brett Cannon]

On Fri Apr 18 2014 at 5:03:33 PM, Ezio Melotti <ezio.melotti at gmail.com>
wrote:

> Hi,
>
> On Thu, Apr 17, 2014 at 9:09 PM, Brett Cannon <bcannon at gmail.com> wrote:
> >
> >
> > On Thu Apr 17 2014 at 1:34:23 PM, Jurko Gospodnetić
> > <jurko.gospodnetic at pke.hr> wrote:
> >>
> >>    Hi.
> >>
> >> On 14.4.2014. 23:51, Brett Cannon wrote:
> >> > Now the question is whether the maintenance cost of having to rebuild
> >> > Python for a select number of stdlib modules is enough to warrant
> >> > putting in the effort to make this work.
> >>
> >>    I would really love to have better startup times in production, but I
> >> would also really hate to lose the ability to hack around in stdlib
> >> sources during development just to get better startup performance.
> >>
> >>    In general, what I really like about using Python for software
> >> development is the ability to open any stdlib file and easily go poking
> >> around using stuff like 'import pdb;pdb.set_trace()' or simple print
> >> statements. Researching mysterious behaviour is generally much much
> >> MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
> >> into a stdlib Python module than if it takes you down into the bowels of
> >> some C module (think zipimport.c *grin*). Not to mention the effect that
> >> being able to quickly resolve a mystery by hacking on some Python
> >> internals leaves you feeling very satisfied, while having to entrench
> >> yourself in those internals for a long time just to find out you've made
> >> something foolish on your end leaves you feeling exhausted at best.
> >
> >
> > Freezing modules does not affect the ability to use gdb. And as long as
> you
> > set the appropriate __file__ values then tracebacks will contain even the
> > file line and location.
> >
>
> Will the tracebacks only contain the line number or the line of code too?
>

Yes.


>
> I've seen tracebacks involving importlib,_bootstrap that didn't
> include the code, and I'm wondering if we will get something similar
> for all the other modules you are freezing:
>
> Traceback (most recent call last):
>   File "/tmp/foo.py", line 7, in <module>
>     import email
>   File "<frozen importlib._bootstrap>", line 1561, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1519, in
> _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 1473, in _find_module
>   File "<frozen importlib._bootstrap>", line 1308, in find_module
>   File "<frozen importlib._bootstrap>", line 1284, in _get_loader
>   File "<frozen importlib._bootstrap>", line 1273, in _path_importer_cache
>   File "<frozen importlib._bootstrap>", line 1254, in _path_hooks
> TypeError: 'NoneType' object is not iterable
>
> Best Regards,
> Ezio Melotti
>

That's because the frozen importer doesn't define get_source(). But since
we have the source in this instance the __loader__ can be updated to be
SourceFileLoader so that get_source() is available:
http://bugs.python.org/issue21335 .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140423/1e06d016/attachment.html>