[Python-ideas] Draft PEP on string interpolation

[Eric V. Smith]

On 8/27/2015 2:13 AM, Ron Adam wrote:
> On 08/26/2015 09:41 PM, Eric V. Smith wrote:
>> On 8/26/2015 9:08 PM, Ron Adam wrote:
>>> >If you store a string instead of the value, then mutating the object
>>> >won't effect the i-string.  Also you don't get held references to
>>> >objects that may be more expensive than a string.
>>> >
>>> >I think these points need to be in the PEP.
>> Well, it's Nick's PEP, so you'll have to convince him.
>>
>> Here I'll talk about my ideas on i-strings, which I've been implementing
>> on that bitbucket repo I've posted. Although I believe they're
>> consistent with where Nick is taking PEP 501.
>>
>> As I've said before, it's not possible for an i-string to convert all of
>> its expressions to text when the i-string is first constructed. The
>> entire point of delaying the interpolation until some point after the
>> object is constructed is that you don't know how the string conversion
>> is going to be done.
> 
>> Take this i-string:
>> i'value: {value}'
>>
>> How would you convert value to a string before you know how it's being
>> converted, or even if it's being converted to a string?  What if you use
>> a conversion function that converts the i-string to a list, containing
>> the values of the expressions? Or maybe your converter is going to call
>> repr() on each expression. If you convert to a string first, you've
>> destroyed information that the converter needs.
>>
>> n = 10
>> s = 'text'
>> x = i'{n}:{s}'
>>
>> to_list(x)  -> [10, ':', 'text']
>> to_repr(x)  -> '10:"text"'
>>
>> And this doesn't even take into account the format_specs or conversions,
>> which only have meaning to the conversion function.
> 
> Sure it does, you can access the format spec and apply it manually to
> each item or not.  Is there another choice?

You're not reading what I'm writing. Using your proposal of immediately
converting to strings, how would you write the version of "to_list"
whose output I show above?

> Depending on how you want to make the values and specs visible.
> 
>    def to_repr(istr):
>        return ''.join(repr(item.format(spec)) for item, spec
>                   in istr.items())
> 
> 
> I think an actual repr of an i-string may look like this...
> 
>     repr(x)         #->  i'{10}:{"text"}'
> 
> Or maybe...  "i'{10}:{\"text\"}'", so it can be used with eval.

Again, that's not what I'm talking about. How would you write the
"to_repr" function whose output I show above?

> What concerns me is how much memory it could take to keep object
> references arround.  Considder a logging situation that logs thousands
> of items.  Each i-string could contains references to several objects.
> And possibly each of those objects contains references to more objects
> of which memory would have been released hours ago if it weren't for the
> i-strings.  Oops.. my computer is now disc caching so bad it will take
> days to finish the process it is logging.   Meanwhile, it can't process
> any new input.
> 
> If one of the use cases is logging, then this is a realistic possibility.

Logging is already passed the object references. This is how logging is
called today:

logging.info('the values are %d and %f', an_int, get_a_float())

As you can see, it's passed a string and some objects. That's what an
i-string is! But with a nicer syntax and a more flexible way to convert
objects to strings.

If logging were instead passed an i-string:
logging.info(i'the values are {an_int} and {get_a_float()}')

and if logging were changed so that where it currently builds a string
using "msg = str(msg), msg = msg % self.args" [1],  it instead said:

if (isinstance(msg, types.InterpolationTemplate)):
   msg = str(msg)
else:
   msg = str(msg) % self.args

then there would be zero change in the memory usage of the logging
module [2].

Anyway, that's my last input on the subject. You can either follow the
code in my bitbucket repo and show how you'd implement its use cases
with your approach, or we can just wait for Nick to update the PEP.

Eric.

[1]: https://hg.python.org/cpython/file/tip/Lib/logging/__init__.py#l328
[2]: Sadly, it's not quite so simple since logging has a pluggable
setLogRecordFactory architecture. But the point on memory usage stands.

> I do recognize the added flexibility that keeping the references offers,
> but I'm not sure it's needed.
> 
> Cheers,
>    Ron