[Python-Dev] Reading Python source file

[Ryan Gonzalez]

Well, not quite the same thing, but https://github.com/kirbyfan64/pfbuild/blob/master/pfbuild embeds the compressed version of 16k LOC. Would it be affected negatively in any way be this?

Since all the data is on one line, I'd think the old (current) parser would end up reading in the whole line anyway.

On November 18, 2015 9:48:41 AM CST, Guido van Rossum <guido at python.org> wrote:
>On Wed, Nov 18, 2015 at 4:15 AM, Hrvoje Niksic <hrvoje.niksic at avl.com>
>wrote:
>> On 11/18/2015 03:31 AM, Nick Coghlan wrote:
>>>
>>> That behaviour is then inherited at the command line by both the -m
>>> switch and the support for executing directories and zip archives.
>>> When we consider that the "-c" switch also executes an in-memory
>>> string, direct script execution is currently the odd one out in
>*not*
>>> reading the entire source file into memory first, so Serhiy's
>proposed
>>> simplification of the implementation makes sense to me.
>>
>>
>> Reading the whole script in memory will incur an overhead when
>executing
>> scripts that contain (potentially large) data embedded after the end
>of
>> script source.
>>
>> The technique of reading data from sys.argv[0] is probably obsolete
>now that
>> Python supports executing zipped archives, but it is popular in shell
>> scripting and might still be used for self-extracting scripts that
>must
>> support older Python versions. This feature doesn't affect imports
>and -c
>> which are not expected to contain non-Python data.
>
>That trick doesn't work unless the data looks like Python comments or
>data (e.g. a docstring). Python has always insisted on being able to
>parse until EOF. The only extreme case would be a small script
>followed by e.g. 4 GB of comments (where the old parser would indeed
>be more efficient). But unless you can point me to an occurrence of
>this in the wild I'm going to speculate that you just made this up
>based on the shell analogy (which isn't perfect).
>
>-- 
>--Guido van Rossum (python.org/~guido)
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>https://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe:
>https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20151118/f6bbaf1a/attachment.html>