[Python-ideas] Custom string prefixes

[Eric Snow]

On Thu, May 30, 2013 at 4:42 AM, Haoyi Li <haoyi.sg at gmail.com> wrote:
>> The language needs to fit in our brains and you could argue
> that we've already pushed past that threshold
>
> I would argue that "prefix just calls the registered function at import
> time" fits with my brain better then "20 special cases in the lexer which
> result in magic things happening", but maybe that's just me.

The existing prefixes are not going away, but they are easy to look up
in the docs.  So that brain-space is not going way.  Additionally,
under this proposal those prefixes could change meaning at run-time.
Furthermore, you could encounter prefixes that are not in the docs and
you would have to know how to figure out what they are supposed to do.
 Right now there is zero ambiguity about what the prefixes do.

So this proposal would simply be adding one more thing that has to fit
inside your brain without adding any power to the language.  On top of
that I'm not seeing why this would be very widely used meaning people
would probably not recognize what's going on at first.  This is rough
on the non-experts, and adds more complexity to the language where we
already have a perfectly good way of doing the same thing (explicit
function calls).

>
>> One concern I'd have
> is in how you would look up what some arbitrary prefix is supposed to
> do
>
> In theory PyCharm/Vim/Emacs would be able to just jump to the definition of
> the prefix, instantly giving you the docstring along with the source code of
> what it does, so I don't think this is a real problem. Tool support will
> catch up, as always, and you could always grep "@str_prefix\ndef blah" or
> something similar.

So you would have to use some editor with up-to-date support and do it
at run-time.  No web viewers.  No distro-installed editors.  This has
a particularly smell to it.

>
>> Furthermore, it sounds like the intent is to apply the prefix at
> run-time.
>
> Is there any other time in Python? There's only import-time and
> execution-time to choose from.

1. compile-time (during import with no up-to-date .pyc file)
2. run-time
2a. import time (execution of the module)
2b. call-time (execution of a function body)

.pyc files allow us to skip this part after the first time.  Currently
string prefixes are handled at compile time.  This proposal moves it
to run-time.  Furthermore, only the raw string literals would still
actually be literals and able to be interned/compiled away/stored in a
functions constants.  But now there would be extra byte codes used in
the compiled code to process the raw literals.  All for the sake of
hypothetical use cases that can already be handled by explicit
function calls.

>
>> That means that the function that gets applied depends on
> what's in some registry at a given moment.  So source with these
> custom string prefixes will be even more ambiguous when read.
>
> I don't think the second line follows from the first. You could say the same
> of PEP302 import hooks, which overload a fundamental operation (import xxx)
> to do whatever the hell you want, depending on what's in your sys.meta_path
> at any given moment. Heck, you could say the same of any function call
> my_func(...), that it depends on what my_func has been defined as at a given
> moment! Remember functions can be (and often are!) rebound multiple times,
> but people seem to get by just fine.

Consider that the import system is a giant black box for most people
and even gives the experts fits.  Comparing this proposal to it really
says something.  Furthermore, existing dynamic-language-induced
challenges do not justify adding more.

>
>> In that case, why not just call the
> function at the module scope, bind the result to a name there, and use
> that name wherever you like.  It's effectively a constant, right?
>
> The main reason I'd give is that moving a whole bunch of things into module
> scope spaghettifies your program. If I'm making a bunch of SQL queries in a
> bunch of functions, I expect the SQL to be in the functions where I'm using
> them (nobody else cares about them, right?) rather than at module scope.
> Moving stuff to module scope for performance adds another layer of
> unnecessary indirection when i look at my_function and wonder what
> my_function_sql_query and my_function_xml_template is meant to do, for every
> single function.

Then put them in a class with the related methods.  However, my whole
moving-things-around recommendation is orthogonal to this proposal.
Handling a prefix during a function call would be equivalent (or even
less performant) to explicitly calling the handler.

Bottom line: I'm still not seeing the utility of this feature over
plain function calls.

-eric

p.s. Do you think this feature would ever be used in the standard
library if it made it into the language?