[Python-Dev] Reading Python source file

[Guido van Rossum]

On Wed, Nov 18, 2015 at 4:15 AM, Hrvoje Niksic <hrvoje.niksic at avl.com> wrote:
> On 11/18/2015 03:31 AM, Nick Coghlan wrote:
>>
>> That behaviour is then inherited at the command line by both the -m
>> switch and the support for executing directories and zip archives.
>> When we consider that the "-c" switch also executes an in-memory
>> string, direct script execution is currently the odd one out in *not*
>> reading the entire source file into memory first, so Serhiy's proposed
>> simplification of the implementation makes sense to me.
>
>
> Reading the whole script in memory will incur an overhead when executing
> scripts that contain (potentially large) data embedded after the end of
> script source.
>
> The technique of reading data from sys.argv[0] is probably obsolete now that
> Python supports executing zipped archives, but it is popular in shell
> scripting and might still be used for self-extracting scripts that must
> support older Python versions. This feature doesn't affect imports and -c
> which are not expected to contain non-Python data.

That trick doesn't work unless the data looks like Python comments or
data (e.g. a docstring). Python has always insisted on being able to
parse until EOF. The only extreme case would be a small script
followed by e.g. 4 GB of comments (where the old parser would indeed
be more efficient). But unless you can point me to an occurrence of
this in the wild I'm going to speculate that you just made this up
based on the shell analogy (which isn't perfect).

-- 
--Guido van Rossum (python.org/~guido)