From: "Guido van Rossum"

...

I love this suggestion. It's the sort of thing you can't do in C++ ;-) I suspect the arguments against will run to efficiency and complexity, since you need to compile the backquoted expressions (in some context).

Actually, I had planned a secret feature that skips matching nested {...} inside ${...}, so that you could write a magic dict whose keys were eval()'ed in the caller's context. The %(...) parser does this (skipping nested (...)) because someone wanted to do that.

Ooh, magic and secrets! Maybe a little too magical for me to understand easily. Is the stuff between ${...} allowed to be any valid expression? harry-potter's-got-nothing-on-you-ly y'rs, dave

See http://tothink.com/python/embedpp

How come you never submitted this PEP to the PEPmeister? I can't comment on what I don't know. It certainly comes closest to the original ABC feature. (The main problem with `...` is that many people can't distinguish between ` and ', as user testing has shown.) --Guido van Rossum (home page: http://www.python.org/~guido/)

On Thu, Jun 20, 2002 at 01:30:47PM -0400, Guido van Rossum wrote:

...

See http://tothink.com/python/embedpp

How come you never submitted this PEP to the PEPmeister? I can't comment on what I don't know. It certainly comes closest to the original ABC feature. (The main problem with `...` is that many people can't distinguish between ` and ', as user testing has shown.)

I guess I got a bit discouraged by the response on python-list back then. Now I know better :-) Oren

On Thu, 20 Jun 2002, Oren Tirosh wrote:

See http://tothink.com/python/embedpp

Hi Oren, Your proposal brings up some valid concerns with PEP 215: 1. run-time vs. compile-time parsing 2. how to decide what's an expression 3. balanced quoting instead of $ PEP 215 actually agrees with you on point #1. That is, the intent (though poorly explained) was that the interpolated strings would be turned into bytecode by the compiler. That is why the PEP insists on having the interpolated expressions in the literal itself -- they can be taken apart at compile time. However, i don't necessarily agree with PEP 215. (I mentioned this once before, but it might not hurt to reiterate that i didn't write the PEP because i desperately wanted string interpolation. I wrote it because i wanted to try to get one local optimum written down in a PEP, so there would be something for discussion.) Using compile-time parsing, as in PEP 215, has the advantage that it avoids any possible security problems; but it also eliminates the possibility of using this for internationalization. I see this as the key tension in the string interpolation issue (aside from all the syntax stuff -- which is naturally controversial). -- ?!ng "Computers are useless. They can only give you answers." -- Pablo Picasso

Guido van Rossum wrote:

...

Using compile-time parsing, as in PEP 215, has the advantage that it avoids any possible security problems;

It is also the only way to properly support nested scopes. It would be confusing and inconsistent if you can use a variable from a nested scope in an expression but not in a "string display" (which I think is a cute name for strings with embedded expressions). ... Yes, I believe that Barry's main purpose is i18n. But I think i18n should not be approached in a cavalier way. If you need i18n of your application, you have to be very disciplined anyway. I think collecting the variable available for interpolation in a dict and passing them explicitly to an interpolation function is the way to go here.

I think that what I hear you saying is that interpolation should ideally be done at a compile time for simple uses and at runtime for i18n. The compile-time version should have the ability to do full expressions (array indexes and self.members at the very least) and will have access to nested scopes. The runtime version should only work with dictionaries. I think you also said that they should both use named parameters instead of positional parameters. And presumably just for simplicity they would use similar syntax although one would be triggered at compile time and one at runtime. If "%" survives, it would be used for positional parameters, instead of named parameters. Is that your current thinking on the matter? I think we are making progress if we're coming to understand that the two different problem domains (simple scripts versus i18n) have different needs and that there is probably no one solution that fits both. Paul Prescod

pinard@iro.umontreal.ca:

I really, really think that with enough and proper care, Python could be set so internationalisation of Python scripts is just unobtrusive routine. There should not be one way to write Python when one does not internationalise, and another different way to use it when one internationalises.

As long as you have a Turing-complete programming language available for constructing strings, there will always be ways to write code that defies any straightforward means of internationalisation. Or in other words, if internationalisation is a goal, you'll always have to keep it in mind when coding, one way or another. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

[Paul]

I think that what I hear you saying is that interpolation should ideally be done at a compile time for simple uses and at runtime for i18n. The compile-time version should have the ability to do full expressions (array indexes and self.members at the very least) and will have access to nested scopes. The runtime version should only work with dictionaries.

Yes.

I think you also said that they should both use named parameters instead of positional parameters. And presumably just for simplicity they would use similar syntax although one would be triggered at compile time and one at runtime.

Yes.

If "%" survives, it would be used for positional parameters, instead of named parameters.

Yes (in Python 3). I can also see the viewpoint that the printf syntax should be abandoned entirely (in Python 3), in favor of a different (and probably more verbose) way to spell things like "%6.3f" or "%04x". Although there may be application areas (like producing output from numeric programs) where the formatting options are very convenient. In that case Python 3 could retain the positional % syntax but drop the by-name syntax. I'm undecided on this.

Is that your current thinking on the matter?

Yes. But based on a lot of feedback (e.g. Alex's anecdote) I'm inclined to let the matter rest rather than rush to add a new language feature.

I think we are making progress if we're coming to understand that the two different problem domains (simple scripts versus i18n) have different needs and that there is probably no one solution that fits both.

OTOH, there's François's position: [François]

The mantra I repeated all along had two key points:

1) internationalisation will only be successful if designed to be unobtrusive, otherwise average maintainers and implementors will resist it.

2) programmer duties and translation duties are to be kept separate, so these activities could be done asynchronously from one another.[1]

I really, really think that with enough and proper care, Python could be set so internationalisation of Python scripts is just unobtrusive routine. There should not be one way to write Python when one does not internationalise, and another different way to use it when one internationalises. The full power and facilities of Python should be available at all times, unrelated to internationalisation intents. Non-English people should not have to pay a penalty, or if they do, the penalty should be minimised As Much As Possible.

However, he fails to suggest even a glimpse of a solution that satisfies his requirements, so I'm intended to write him off as the crank he usually is. ;-)

Our BDFL, Guido, should favour internationalisation as a principle in the evolution for the language, that is, more than a random negligible feature. I sincerely hope he will do. For many people, internationalisation issues cannot be separated out that simply, or otherwise dismissed. We should rather learn to collaborate at properly addressing and solving them at each evolutionary step, so Python really remains a language for everybody.

To the contrary, I think most users don't care about writing code that can be switched easily from one language to the next. They only care about being able to write code that prints text in their own language (and perhaps about being able to use words in their own language as identifiers). --Guido van Rossum (home page: http://www.python.org/~guido/)

On Thursday 20 June 2002 06:48 pm, Ka-Ping Yee wrote:

On Thu, 20 Jun 2002, Oren Tirosh wrote:

...

See http://tothink.com/python/embedpp

Hi Oren,

Your proposal brings up some valid concerns with PEP 215:

1. run-time vs. compile-time parsing 2. how to decide what's an expression 3. balanced quoting instead of $

I like Oren's PEP as a replacement for PEP 292. But there is one major problem with his notation. I would change the "`" character to something more readable. I tried examples with "@", "$", "%", "!", and "?". My preference was "?", "@", or "$". (The choice should consider the easy of typing on international keyboards.) The "?" seems like a good choice because the replacement expresssion will answer the question of what will appear in the string at that location. Here is Oren's example using the "?" to quote the expression. print e"X=?x?, Y=?calc_y(x)?." The following example is provided for contrast. It has a larger text to variable substitution ratio. p = e"""A new character prefix "e" is defined for strings. This prefix precedes the 'u' and 'r' prefixes, if present. Capital 'E' is also acceptable. Within an e-string any ?expressions? enclosed in backquotes are evaluated, converted to strings using the equivalent of the ?str()? function and embedded in-place into the In the larger body of text the "?" is clearly visible. I'm not so sure I like the "?" in the smaller example. It may be because the "?" looks too much like letters that can appear in a variable name. The "@" stands out a bit better than "?". This is probably because there are more pixels turned on and the character is fatter. print e"X=@x@, Y=@calc_y(x)@." p = e"""A new character prefix "e" is defined for strings. This prefix precedes the 'u' and 'r' prefixes, if present. Capital 'E' is also acceptable. Within an e-string any @expressions@ enclosed in backquotes are evaluated, converted to strings using the equivalent of the @str()@ function and embedded in-place into the e-string.""" The function of the "$" would be recognizable to people migrating from other languages, but it would be used as a balanced quote, rather than as a starting character in a variable that will be substituted. (Is this character easy to type on non-US keyboards? I thought the "$" was one of the character that is replaced on European keyboards.) If The "@" is available on international keyboards then I think it would be a better choice. print e"X=$x$, Y=$calc_y(x)$." p = e"""A new character prefix "e" is defined for strings. This prefix precedes the 'u' and 'r' prefixes, if present. Capital 'E' is also acceptable. Within an e-string any $expressions$ enclosed in backquotes are evaluated, converted to strings using the equivalent of the $str()$ function and embedded in-place into the e-string."""

Oren Tirosh wrote:

... No need for double backslash. No need for a special string prefix either because \( currently has no meaning.

I like this idea but note that \( does have a current meaning:

...

...

"\(" '\\(' "\(" =="\\(" 1

I think this is weird but it is inherited from C... So it would take time to phase this in. First we have to warn about \( and then give people time to find instances of it and change them to \\(. Then we could introduce a new meaning for it. Paul Prescod

[Paul Prescod]

I like this idea but note that \( does have a current meaning:

...

...

...

"\(" '\\(' "\(" =="\\(" 1

I think this is weird but it is inherited from C...

C89 doesn't define the effect. C99 specifically forbids this treatment, and requires a diagnostic if \( appears. Guido did this originally to make it easier to write Emacsish regexps; the later raw strings were a better solution to that problem, although 99.7% of Python newbies seem to believe that raw strings are an idiot's attempt to make it easier to embed Windows file path literals (newbies -- gotta love 'em <wink>).

[Oren Tirosh]

On Thu, Jun 20, 2002 at 03:48:52PM -0700, Ka-Ping Yee wrote:

...

Using compile-time parsing, as in PEP 215, has the advantage that it avoids any possible security problems; but it also eliminates the possibility of using this for internationalization.

Compile-time parsing may eliminate the possibility of using the same mechanism for internationalization, but not the possibility of using the same syntax.

Parsing must be done at some time. Maybe the solution lies into finding some way so Python could lazily delay the "compilation" of the string to after its translation (at run-time), when it is known beforehand that a given string is internationalised. The `.pyc' would contain byte-code and data slot for driving the laziness. The translation and compilation should occur only once for a particular string, of course, as the internationalised string may appears within a loop, or within a function which gets called often. In threaded contexts, if we allow for spurious re-compilations once in a long while, and with a simple bit of care, locks could be fully avoided.[1] The good in the above approach is that people would write Python about the same way irrelevant to the fact internationalisation is in the picture or not, and would not have to suffer the complexities of "hand" optimisation of string interpolation in internationalised context. It would simple for _everybody_, on the road meant to make internationalisation a breeze. For Python to know at initial compile time if a string is going to be internationalised of not, it has to be modified, but a positive side of this effort is that internationalisation becomes part of the language design. A possible way towards this (suggested a long while ago) could be to use, beside `eru', some `t' prefix letter asking for translation. Two problems are still to be solved, however. First, going from `_("TEXT")' to `t"TEXT"', the translation function (`_' here) and textual domain should have proper defaults, while offering a way to override them for bigger applications needing finer control or tuning. A simple solution might lie, here, into inventing some special module attribute to that purpose. Second, some applications accept switching national language at run-time. So a mechanism is needed to invalidate lazily-compiled strings when such a switch occurs. An avenue would be to use the national language string code as the "done" flag in the lazy compilation process, allowing recompilation to occur on the fly, as needed. -------------------- [1] Temporarily switching locale-related environment variables in threaded contexts may yield pretty surprising results, this is well-known already. It only stresses, in my opinion, that the design has been frozen without having all the vision it would have taken. Many internationalisation devices implement half-hearted solutions for half-thought problems. I'm not at all asserting that it is possible to foresee everything in advance. Yet, we could be more productive by _not_ slavishly sticking to actual "standards". -- François Pinard http://www.iro.umontreal.ca/~pinard

...

...

...

...

"PP" == Paul Prescod writes:

PP> I think that there are enough benefits for each form (compile PP> time with expressions, runtime without) that we should expect PP> any final solution to support both. Maybe you guys should PP> merge your PEPs! Only two of them are official PEPs currently <294 winks to Oren>. -Barry