Pre PEP: Python Literals (was custom strings before)
Thank you Guido, Chris, Matt and Richard for your feedback to my last email. Here is an updated version called "Template Literals". I am looking for a core developer who can sponsor this PEP. Please speak up if you want to help me. Regards, Thomas Güttler Source and updates: Pre-PEP 9999 <https://github.com/guettli/peps/blob/master/pep-9999.rst> PEP: 9999 Title: Template Literals Author: Thomas Güttler <info at thomas-guettler.de> Sponsor: TODO Status: Draft Type: Standards Track Content-Type: text/x-rst </dev/peps/pep-0012> Created: 08-Jun-2021 Python-Version: TODO Post-History: 08-Jun-2021 ------------------------------ Contents - Abstract <#abstract> - Motivation <#motivation> - Rationale <#rationale> - Specification <#specification> - Security Implications <#security-implications> - Reference Implementation <#reference-implementation> - Alternative Ideas <#alternative-ideas> - Rejected Ideas <#rejected-ideas> - Open Issues <#open-issues> - References <#references> - Copyright <#copyright> Abstract <#id6> This PEP adds Template Literals to Python. To avoid code injection like XSS or SQL-injection Template Literals can help you to write save Python code. Template Literals provide an easy way to access the local and global variables (like f-strings), so that passing a dictionary to the Template is not necessary. Motivation <#id7> In the context of web development Python can do more than providing REST APIs via http. With the trend to Server-Side-Rendering, we face a fundamental question: How to create HTML with Python? If you use the FrOW pattern (HTML fragments over the wire) [1] <#frow>, then you will be writing small methods returning small HTML fragments. As a developer I want to pass escaped data into template literals to be as simple as possible. Rationale <#id8> Imagine you want to create a small HTML fragment in Python, and return it as HTTP-Response: HttpResponse(f''' <h1>Hi {name}</h1> Your messages: {messages}''') The problem in above example is, that no escaping gets done. In above example "name" and "messages" should be treated differently. The variable "name" should get escaped. For example if the name is "Mary & Bob", the result should be "Mary & Bob". The variable "messages" contains HTML which is already escaped. It should not be escaped again. Most frameworks have a way to do this conditional escaping. For example Django uses conditional_escape() <https://docs.djangoproject.com/en/3.2/ref/utils/#django.utils.html.condition...> [2] <#id2> With the help of conditional_escape() the above problem could be solved like this: HttpResponse(f''' <h1>Hi {conditional_escape(name)}</h1> Your messages: {conditional_escape(messages)}''') This solution has two drawbacks: 1. It is too verbose. Typing "conditional_escape(...)" again and again is cumbersome. 2. If a conditional_escape() gets forgotten Cross-site scripting attacks could be possible, since malicious users could inject HTML. Specification <#id9> Template Literals use backticks (like JavaScript Template Literals <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l...> [3] <#id4>) Example: name = 'Mary & Bob' messages = `<ul><li>message1</li><li>message2</li></ul>` return HttpResponse(` <h1>Hi {name}</h1> Your messages: {messages} Today: {datetime.date.today()}`) Expressions within curly braces get handled liked in f-strings (PEP-498). The Template Literal creates an instance of the new class types.TemplateLiteral. types.TemplateLiteral has two attributes: - template: The raw string inside the backticks. - tokens: A list of tuples: (value, is_literal). For above example this would mean: template = ''' <h1>Hi {name}</h1> Your messages: {messages} Today: {datetime.date.today()}''' tokens = [ ('\n <h1>Hi ', True), ('Mary & Bob', False), ('</h1>\n Your messages: ', True), (<TemplateLiteral "<ul><li>message1</li><li>message2</li></ul>">, False), ('\n\n Today: ', True), (<datetime.date(2021, 6, 9)>, False) ] It is outside this PEP how a consumer of TemplateLiteral handles this data structure. For example the Django web framework could transform a TemplateLiteral to a SafeString like this: def template_literal_to_safestring(template_literal): return mark_safe( ''.join( [ conditional_escape(value) if not is_literal else value for (value, is_literal) in template_literal.tokens ] ) ) This PEP is not related or constraint to the Django framework. It is even not related to HTML. It can be used for any kind of templating. Security Implications <#id10> Template Literals can execute arbitrary code (like f-strings). Template Literals get created by Python developers, not by users. If you want to make templates available for users (for example if you develop a CMS), then please use a different solution. Reference Implementation <#id11> TODO Alternative Ideas <#id12> Instead of backticks for example t'...' could be used. Rejected Ideas <#id13> TODO Open Issues <#id14> TODO References <#id15> [1] <#id1> FrOW, "HTML Fragments Over the Wire". Frameworks like Unpoly, Hotwire or htmx. [2] <#id3> https://docs.djangoproject.com/en/3.2/ref/utils/#django.utils.html.condition... [3] <#id5> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l... Copyright <#id16> This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
I am concerned that we have too many string formatting methods: % formatting .format() f-strings string templates (are there others I've missed?). And now we have another proposed one. It's all getting a bit much. And IMO a turnoff for people learning Python (which one should I use, and why?). Best wishes Rob Cliffe On 10/06/2021 07:30, Thomas Güttler wrote:
Thank you Guido, Chris, Matt and Richard for your feedback to my last email.
Here is an updated version called "Template Literals".
I am looking for a core developer who can sponsor this PEP.
Please speak up if you want to help me.
Regards, Thomas Güttler
Source and updates: Pre-PEP 9999 <https://github.com/guettli/peps/blob/master/pep-9999.rst>
PEP: 9999 Title: Template Literals Author: Thomas Güttler <info at thomas-guettler.de <http://thomas-guettler.de>> Sponsor: TODO Status: Draft Type: Standards Track Content-Type: text/x-rst </dev/peps/pep-0012> Created: 08-Jun-2021 Python-Version: TODO Post-History: 08-Jun-2021
------------------------------------------------------------------------
Contents
* Abstract <#abstract> * Motivation <#motivation> * Rationale <#rationale> * Specification <#specification> * Security Implications <#security-implications> * Reference Implementation <#reference-implementation> * Alternative Ideas <#alternative-ideas> * Rejected Ideas <#rejected-ideas> * Open Issues <#open-issues> * References <#references> * Copyright <#copyright>
Abstract <#id6>
This PEP adds Template Literals to Python.
To avoid code injection like XSS or SQL-injection Template Literals can help you to write save Python code.
Template Literals provide an easy way to access the local and global variables (like f-strings), so that passing a dictionary to the Template is not necessary.
Motivation <#id7>
In the context of web development Python can do more than providing REST APIs via http. With the trend to Server-Side-Rendering, we face a fundamental question:
How to create HTML with Python?
If you use the FrOW pattern (HTML fragments over the wire) [1] <#frow>, then you will be writing small methods returning small HTML fragments.
As a developer I want to pass escaped data into template literals to be as simple as possible.
Rationale <#id8>
Imagine you want to create a small HTML fragment in Python, and return it as HTTP-Response:
HttpResponse(f''' <h1>Hi {name}</h1> Your messages: {messages}''')
The problem in above example is, that no escaping gets done.
In above example "name" and "messages" should be treated differently.
The variable "name" should get escaped. For example if the name is "Mary & Bob", the result should be "Mary & Bob".
The variable "messages" contains HTML which is already escaped. It should not be escaped again.
Most frameworks have a way to do this conditional escaping.
For example Django uses conditional_escape() <https://docs.djangoproject.com/en/3.2/ref/utils/#django.utils.html.condition...> [2] <#id2>
With the help of conditional_escape() the above problem could be solved like this:
HttpResponse(f''' <h1>Hi {conditional_escape(name)}</h1> Your messages: {conditional_escape(messages)}''')
This solution has two drawbacks:
1. It is too verbose. Typing "conditional_escape(...)" again and again is cumbersome. 2. If a conditional_escape() gets forgotten Cross-site scripting attacks could be possible, since malicious users could inject HTML.
Specification <#id9>
Template Literals use backticks (like JavaScript Template Literals <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l...> [3] <#id4>)
Example:
name = 'Mary & Bob' messages = `<ul><li>message1</li><li>message2</li></ul>` return HttpResponse(` <h1>Hi {name}</h1> Your messages: {messages}
Today: {datetime.date.today()}`)
Expressions within curly braces get handled liked in f-strings (PEP-498).
The Template Literal creates an instance of the new class types.TemplateLiteral.
types.TemplateLiteral has two attributes:
* template: The raw string inside the backticks. * tokens: A list of tuples: (value, is_literal).
For above example this would mean:
template = ''' <h1>Hi {name}</h1> Your messages: {messages}
Today: {datetime.date.today()}'''
tokens = [ ('\n <h1>Hi ', True), ('Mary & Bob', False), ('</h1>\n Your messages: ', True), (<TemplateLiteral "<ul><li>message1</li><li>message2</li></ul>">, False), ('\n\n Today: ', True), (<datetime.date(2021, 6, 9)>, False) ]
It is outside this PEP how a consumer of TemplateLiteral handles this data structure.
For example the Django web framework could transform a TemplateLiteral to a SafeString like this:
def template_literal_to_safestring(template_literal): return mark_safe( ''.join( [ conditional_escape(value) if not is_literal else value for (value, is_literal) in template_literal.tokens ] ) )
This PEP is not related or constraint to the Django framework. It is even not related to HTML. It can be used for any kind of templating.
Security Implications <#id10>
Template Literals can execute arbitrary code (like f-strings).
Template Literals get created by Python developers, not by users. If you want to make templates available for users (for example if you develop a CMS), then please use a different solution.
Reference Implementation <#id11>
TODO
Alternative Ideas <#id12>
Instead of backticks for example t'...' could be used.
Rejected Ideas <#id13>
TODO
Open Issues <#id14>
TODO
References <#id15>
[1] <#id1> FrOW, "HTML Fragments Over the Wire". Frameworks like Unpoly, Hotwire or htmx.
[2] <#id3> https://docs.djangoproject.com/en/3.2/ref/utils/#django.utils.html.condition... <https://docs.djangoproject.com/en/3.2/ref/utils/#django.utils.html.condition...>
[3] <#id5> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l... <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l...>
Copyright <#id16>
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LYAC7J... Code of Conduct: http://python.org/psf/codeofconduct/
Am Do., 10. Juni 2021 um 14:42 Uhr schrieb Rob Cliffe via Python-ideas < python-ideas@python.org>:
I am concerned that we have too many string formatting methods: % formatting .format() f-strings string templates (are there others I've missed?). And now we have another proposed one. It's all getting a bit much. And IMO a turnoff for people learning Python (which one should I use, and why?). Best wishes Rob Cliffe
I can feel with you. There are some new things in Python which make it harder to learn. I personally don't see a need for the walrus operator. But on the other hand you can't compare Template Literals with normal string handling. This really helps developers to avoid cross-site-scripting attacks by enabling a secure escaping of all strings which are not explicitly marked as safe. Template Literals only make sense if you want to escape values like in HTML, XML or SQL templates. Regards, Thomas
Well, I share Rob's concern but I do see the point of this:
Template Literals only make sense if you want to escape values like in HTML, XML or SQL templates.
Maybe they should have a name to better reflect this intended use case? Safe Templates, Escaped Strings, I don't know, I'm terrible at naming things but I do think this would be a nice modification. Best, [image: --] *Felipe* Bidu *(Felipe Rodrigues)* [image: https://]*https://felipevr.com* <https://felipevr.com> On Thu, Jun 10, 2021 at 10:18 AM Thomas Güttler <info@thomas-guettler.de> wrote:
Am Do., 10. Juni 2021 um 14:42 Uhr schrieb Rob Cliffe via Python-ideas < python-ideas@python.org>:
I am concerned that we have too many string formatting methods: % formatting .format() f-strings string templates (are there others I've missed?). And now we have another proposed one. It's all getting a bit much. And IMO a turnoff for people learning Python (which one should I use, and why?). Best wishes Rob Cliffe
I can feel with you. There are some new things in Python which make it harder to learn. I personally don't see a need for the walrus operator.
But on the other hand you can't compare Template Literals with normal string handling.
This really helps developers to avoid cross-site-scripting attacks by enabling a secure escaping of all strings which are not explicitly marked as safe.
Template Literals only make sense if you want to escape values like in HTML, XML or SQL templates.
Regards, Thomas
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EYO6IZ... Code of Conduct: http://python.org/psf/codeofconduct/
Well, I share Rob's concern but I do see the point of this:
Template Literals only make sense if you want to escape values like in HTML, XML or SQL templates.
Maybe they should have a name to better reflect this intended use case? Safe Templates, Escaped Strings, I don't know, I'm terrible at naming things but I do think this would be a nice modification.
Thank you Felipe for looking into this. Yes, you are right. A better
Am Do., 10. Juni 2021 um 15:48 Uhr schrieb Felipe Rodrigues < felipe@felipevr.com>: heading might make sense. I just copied the name (Template Literals) from JavaScript. I am not good at naming either. Maybe someone else has an idea how to name this? Regards, Thomas
Thomas Güttler writes:
This really helps developers to avoid cross-site-scripting attacks by enabling a secure escaping of all strings which are not explicitly marked as safe.
Frameworks can already do this by unconditionally applying a function like conditional_escape to all evaluated template variables. (If that's too drastic for your taste, there could be a pragma %conditional_escape_everything to turn it on.) Why don't they? If it's not "they just didn't think of it", and there's a real reason, why doesn't that reason apply to your template literals? Note that str has no "safe" attribute, and attributes defined by a framework are not known to Python. You need to explain how you can Python-evaluate an expression to a str as your template literal does, and still preserve the "safe" mark that I presume is an attribute of a class defined by the framework. I guess the problem of accessing the framework's attribute can be solved by delegating that to the __format__ method of the framework type, and maybe preserving it can be handled by having that __format__ method return a subclass of str. But this reintroduces a strong possibility of programmer error, because any function that constructs and returns a new str will strip the "safe" mark. This happens *before* the __format__ method can be invoked -- str's __format__ does not check for a safe mark -- so it's a real problem. This might dramatically reduce the utility of these template literals because it's simply not safe to allow the full range of expressions that f-strings allow. (That could be a YAGNI, but you need to explain and if possible document that.) Also, this means that frameworks can no longer just inherit from str: they need to reimplement literally every method that returns str, or prohibit its use in templates. Note that 'is_literal' is not the same as "safe". Based on the example, this is intentional: is_literal simply means that this isn't the value of an expression, simplifying implementation of the internal function that evaluates the template string to a TemplateLiteral. But this means that the function joining a TemplateLiteral needs to consider both is_literal (which is safe but apparently unmarked) and the 'safe' attribute. This seems more complicated than it needs to be. TemplateLiteral is not a good name for that type. The backtick construct is a literal (except it's really not ;-), the TemplateLiteral is a constructed value. TemplateValue or TemplateTokenSequence or something like that might be a better name. In any case it's a little confusing that both the syntax and the value are called "literal". It's not impossible to understand, but at least for me I have to think "is this the syntax or is this an object?" every time I see it.
There may well be use cases for this, but one thing struck me. From the PEP: "Template Literals provide an easy way to access the local and global variables (like f-strings), so that passing a dictionary to the Template is not necessary." This seems to be crossing the line between "data" and "code" -- that's a line that can get pretty fuzzy in Python, but it's still a distinction I find helpful to think about. f-strings provide an easy way to build strings with stuff stored in code: local variables. This makes them very helpful for things like building Exception messages and the like, or showing the results of computation. But for the most part, populating a template is likely to be done from data, rather than code -- results of a database query, or what have you. So this kind of template building is usually very well suited to passing a dictionary around, rather than using the local namespace. Even if you are using Python objects to model your data (e.g. dataclasses and the like) -- you still have: a) one object to pass in to your template builder and/or b) an easy way to make a dict out of the object to pass into a template. I could be missing something, but I just don't see the benefits of having f-string like access to local variables in this context. -CHB On Thu, Jun 10, 2021 at 8:56 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Thomas Güttler writes:
This really helps developers to avoid cross-site-scripting attacks by enabling a secure escaping of all strings which are not explicitly marked as safe.
Frameworks can already do this by unconditionally applying a function like conditional_escape to all evaluated template variables. (If that's too drastic for your taste, there could be a pragma %conditional_escape_everything to turn it on.) Why don't they? If it's not "they just didn't think of it", and there's a real reason, why doesn't that reason apply to your template literals?
Note that str has no "safe" attribute, and attributes defined by a framework are not known to Python. You need to explain how you can Python-evaluate an expression to a str as your template literal does, and still preserve the "safe" mark that I presume is an attribute of a class defined by the framework.
I guess the problem of accessing the framework's attribute can be solved by delegating that to the __format__ method of the framework type, and maybe preserving it can be handled by having that __format__ method return a subclass of str.
But this reintroduces a strong possibility of programmer error, because any function that constructs and returns a new str will strip the "safe" mark. This happens *before* the __format__ method can be invoked -- str's __format__ does not check for a safe mark -- so it's a real problem. This might dramatically reduce the utility of these template literals because it's simply not safe to allow the full range of expressions that f-strings allow. (That could be a YAGNI, but you need to explain and if possible document that.) Also, this means that frameworks can no longer just inherit from str: they need to reimplement literally every method that returns str, or prohibit its use in templates.
Note that 'is_literal' is not the same as "safe". Based on the example, this is intentional: is_literal simply means that this isn't the value of an expression, simplifying implementation of the internal function that evaluates the template string to a TemplateLiteral. But this means that the function joining a TemplateLiteral needs to consider both is_literal (which is safe but apparently unmarked) and the 'safe' attribute. This seems more complicated than it needs to be.
TemplateLiteral is not a good name for that type. The backtick construct is a literal (except it's really not ;-), the TemplateLiteral is a constructed value. TemplateValue or TemplateTokenSequence or something like that might be a better name. In any case it's a little confusing that both the syntax and the value are called "literal". It's not impossible to understand, but at least for me I have to think "is this the syntax or is this an object?" every time I see it.
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WHOPY2... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Am Fr., 11. Juni 2021 um 00:10 Uhr schrieb Christopher Barker < pythonchb@gmail.com>:
There may well be use cases for this, but one thing struck me. From the PEP:
"Template Literals provide an easy way to access the local and global variables (like f-strings), so that passing a dictionary to the Template is not necessary."
This seems to be crossing the line between "data" and "code" -- that's a line that can get pretty fuzzy in Python, but it's still a distinction I find helpful to think about.
f-strings provide an easy way to build strings with stuff stored in code: local variables. This makes them very helpful for things like building Exception messages and the like, or showing the results of computation.
But for the most part, populating a template is likely to be done from data, rather than code -- results of a database query, or what have you. So this kind of template building is usually very well suited to passing a dictionary around, rather than using the local namespace.
If you don't create HTML with Python daily, then you might not feel the pain. If you create many HTML strings daily, then you will be typing `foo=foo, bar=bar` (to pass the variables into the template) over and over again. I would reduce cognitive load if you could avoid reading/writing `foo=foo, bar=bar`. My goal is to have a super reduced syntax, so that developers need only a few characters to create save (properly escaped) html with Python. In the past people created whole HTML pages. There it made sense to use a template language. But if you use FrOW, then you will create many small methods returning small fragments, and then this small extra work of passing in variables gets .... (please insert your favorite negative adjective). Back your point "data vs code". The new class types.TemplateLiteral is pure data. It is up to the consumer to process the data. Of course I am biased, since my main concern is creating HTML. But I guess there are several other use-cases where a TemplateLiteral could be used.
Even if you are using Python objects to model your data (e.g. dataclasses and the like) -- you still have: a) one object to pass in to your template builder and/or b) an easy way to make a dict out of the object to pass into a template.
I could be missing something, but I just don't see the benefits of having f-string like access to local variables in this context.
-CHB
- Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Fri, Jun 11, 2021 at 2:37 AM Thomas Güttler <info@thomas-guettler.de> wrote:
If you don't create HTML with Python daily, then you might not feel the pain.
If you create many HTML strings daily, then you will be typing `foo=foo,
bar=bar` (to pass the variables into the template) over and over again. My goal is to have a super reduced syntax, so that developers need only a few characters to create save (properly escaped) html with Python.
This is exactly why I oppose this. I, like many, many other developers, simply do not do that. Like basically not ever. In the very rare cases I do, f-strings are more than enough. This is trying to turn Python into PHP, a language dedicated to HTML processing. We simply do not need dedicated syntax that is ONLY relevant to making HTML strings, saving a small number of characters for that one single task. ... and no, there isn't any other non-fanciful usage. It is specifically harmful to performance in SQL queries, so is an anti-pattern. Maybe, conceivably, once in a blue moon, someone will come across some other domain where the new syntax could be useful (LaTeX templating?), but far, far, less than to make it worth changing the language and imposing the burden on 10s or 100s of millions of people who use Python. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
Am Do., 10. Juni 2021 um 17:56 Uhr schrieb Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp>:
Thomas Güttler writes:
This really helps developers to avoid cross-site-scripting attacks by enabling a secure escaping of all strings which are not explicitly marked as safe.
Frameworks can already do this by unconditionally applying a function like conditional_escape to all evaluated template variables. (If that's too drastic for your taste, there could be a pragma %conditional_escape_everything to turn it on.) Why don't they? If it's not "they just didn't think of it", and there's a real reason, why doesn't that reason apply to your template literals?
I don't understand what you mean with "pragma %conditional_escape_everything". Could you please elaborate?
Note that str has no "safe" attribute, and attributes defined by a framework are not known to Python. You need to explain how you can Python-evaluate an expression to a str as your template literal does, and still preserve the "safe" mark that I presume is an attribute of a class defined by the framework.
That "safe" attribute is outside the scope of the PEP. At least in Django there is a way to handle this. I guess the problem of accessing the framework's attribute can be
solved by delegating that to the __format__ method of the framework type, and maybe preserving it can be handled by having that __format__ method return a subclass of str.
But this reintroduces a strong possibility of programmer error, because any function that constructs and returns a new str will strip the "safe" mark. This happens *before* the __format__ method can be invoked -- str's __format__ does not check for a safe mark -- so it's a real problem.
In Django this is solved via mark_safe(), conditional_escape() and fomat_html().
This might dramatically reduce the utility of these template literals because it's simply not safe to allow the full range of expressions that f-strings allow. (That could be a YAGNI, but you need to explain and if possible document that.) Also, this means that frameworks can no longer just inherit from str: they need to reimplement literally every method that returns str, or prohibit its use in templates.
Note that 'is_literal' is not the same as "safe". Based on the example, this is intentional: is_literal simply means that this isn't the value of an expression, simplifying implementation of the internal function that evaluates the template string to a TemplateLiteral. But this means that the function joining a TemplateLiteral needs to consider both is_literal (which is safe but apparently unmarked) and the 'safe' attribute. This seems more complicated than it needs to be.
conditional_escape() escapes everything which is not marked "safe". The body of the template is considered safe. Please provide an example how to simplify `template_literal_to_safestring()` of the draft: https://github.com/guettli/peps/blob/master/pep-9999.rst#specification
TemplateLiteral is not a good name for that type. The backtick construct is a literal (except it's really not ;-), the TemplateLiteral is a constructed value. TemplateValue or TemplateTokenSequence or something like that might be a better name. In any case it's a little confusing that both the syntax and the value are called "literal". It's not impossible to understand, but at least for me I have to think "is this the syntax or is this an object?" every time I see it.
Thank you for the idea for alternative names. I added them to the PEP. I am open, and think that finding the right name is important. I guess the average user won't notice the word "literal" at all. He/she will see: `<h1>Hello {name}</h1>` is the way to create a HTML fragment and he/she will do so. Thank you for your feedback, Thomas
Thomas Güttler writes:
I don't understand what you mean with "pragma %conditional_escape_everything". Could you please elaborate?
"Pragma" just means it's a way to switch on conditional_escape for all template variable accesses for the rest of the file. The main point is that Django already can conditional_escape all the template variable accesses if it wants to, but it doesn't. It can provide a way for the template author to conditional_escape all the template variable accesses if the author wants to, but apparently it doesn't do that either. I have to wonder why not.
That "safe" attribute is outside the scope of the PEP.
But it's not outside the scope of the PEP, because by using conditional_escape in the template_literal_to_safestring() example, you used the assumption that it's accessible. But that's only true if you pass "safe strings" straight through to the tokens in the TemplateLiteral. If you don't, conditional_escape in your example would escape a safe string. I don't see how you can guarantee that it won't if you allow full f-string syntax.
In Django this is solved via mark_safe(), conditional_escape() and fomat_html().
This isn't an explanation though. The question is if you are implementing a web framework that depends on returning something that is not a str in the tokens, how do you do that and still retain the power of f-string format expression syntax? Or are you proposing that a template literal be restricted to plain variables -- no expressions and maybe no format specs that involve constructing strs -- in the {}?
Please provide an example how to simplify `template_literal_to_safestring()` of the draft: https://github.com/guettli/peps/blob/master/pep-9999.rst#specification
That's not my problem. I do not want this PEP, but if you want to push it to completion, I want to be sure that it works well for the proposed applications. For the reasons above, so far I don't see how it can be a big win for the web templating application. I'm not clear on how it's a win for the logging application, either. Note that as proposed in your PEP, template literal syntax would be forbidden by Google for the same reasons that f-strings are: (1) the logging implementation can theoretically collect the patterns from .template as queryable strs, but that seems to be just wasted effort because they are just strs and not usable as TemplateLiterals, and (2) it always gets rendered (to TemplateLiteral), which is also wasted effort if the logging function has no logs to send it to. PEP 501 is intended to address both issues. Note that in this example (a slight extension of your example): while (theres_work_to_do_matey): if load > max_load: logging.warn(f'Load is too high: {load}') the f-string works because it is evaluated when 'warn' is called. The logging problem that PEP 501 addresses is this: load_warning = f'Load is too high: {load}' while (theres_work_to_do_matey): if load > max_load: logging.warn(load_warning) Here the f-string does *not* work because it's evaluated at the binding to 'load_warning', not at 'warn' time. Template literal syntax as in your PEP also fails, for the same reason. This works: load_warning = 'Load is too high: {load}' while (theres_work_to_do_matey): if load > max_load: logging.warn(load_warning.format(load=load)) but for some reason ;-) people hate that, and Google recommends load_warning = 'Load is too high: {load}' while (theres_work_to_do_matey): if load > max_load: logging.warn(load_warning, load=load) if the logging package supports it, which is still kinda ugly. But a PEP 501 i-string "just works" nicely: load_warning = i'Load is too high: {load}' while (theres_work_to_do_matey): if load > max_load: logging.warn(load_warning) (This assumes a future version of logging.warn that calls str() on the first argument if it is an InterpolationTemplate.)
He/she will see: `<h1>Hello {name}</h1>` is the way to create a HTML fragment and he/she will do so.
That's not interesting, though. The interesting question is what is an author of a consumer of TemplateLiteral going to do. I'll guarantee you she/he will see this fragment = ''.join([x for x, _ in template_literal]) somewhere. I hope she/he doesn't copy it! The problem here is that frameworks like Django, or even templating languages like Jinja, are quite heavy. People looking for more lightweight approaches are likely to grab on to template literals and implement a stripped-down "vocabulary" directly as Python functions and methods. If this is going to be implemented, I want this to work naturally for them, and not be more restricted (especially not if the restrictions are "don't do this or you'll be sorry" conventions like Google's Logging Style Guide!) than f-strings are. I'm NOT saying it won't "just work" in any of the applications suggested: I haven't done, and am not going to do, the necessary work to show it doesn't. I'm "just asking questions". But if I didn't ask them, eventually somebody will, and as PEP proponent you probably need to answer them. If you don't, I believe the risk that the PEP will be rejected increases. Steve
Stephen J. Turnbull wrote:
But a PEP 501 i-string "just works" nicely: load_warning = i'Load is too high: {load}' while (theres_work_to_do_matey): if load > max_load: logging.warn(load_warning) (This assumes a future version of logging.warn that calls str() on the first argument if it is an InterpolationTemplate.)
A "Why this rather than PEP 501's interpolation templates?" is the main thing I was looking for in the PEP and I didn't find it. If the proposal is just a variant on PEP 501 with the syntax changed from i"template" to \`template\` and the template type name changed from InterpolationTemplate to TemplateLiteral, it doesn't need to be a new PEP, I can just explicitly reject those spelling options in PEP 501. (The reasons for that PEP's deferral unfortunately still hold, though - eager rendering, runtime value interpolation, and dedicated templating libraries together cover enough cases that the motivation for introducing the semantic complexity of yet another templating option gets weakened dramatically). If the differences between the proposals run deeper than that, then the proposed new PEP needs to spell them out.
Hi Nick and all other Python ideas friends, yes, you are right. There is not much difference between PEP-501 or my proposal. One argument why I would like to prefer backticks: Some IDEs detect that you want to use a f-string automatically: You type: name = 'Peter' print('Hello {name... and the IDE automatically adds the missing "f" in front of the string: name = 'Peter' print(f'Hello {name... This is a handy feature (of PyCharm), which would not work reliably if there are two different prefixes. ------- You mentioned these things: eager rendering: I think deferred rendering would increase the complexity a lot. And I think it is not needed. runtime value interpolation: It is up to the receiver of types.InterpolationTemplate to handle the data structure. dedicated templating libraries: One temp after the other. I think HTML and SQL libraries would adapt as soon as the foundation is available. I would be happy if PEP-501 would come true. Regards, Thomas Güttler Am So., 27. Juni 2021 um 06:45 Uhr schrieb Nick Coghlan <ncoghlan@gmail.com
:
Stephen J. Turnbull wrote:
But a PEP 501 i-string "just works" nicely: load_warning = i'Load is too high: {load}' while (theres_work_to_do_matey): if load > max_load: logging.warn(load_warning) (This assumes a future version of logging.warn that calls str() on the first argument if it is an InterpolationTemplate.)
A "Why this rather than PEP 501's interpolation templates?" is the main thing I was looking for in the PEP and I didn't find it.
If the proposal is just a variant on PEP 501 with the syntax changed from i"template" to \`template\` and the template type name changed from InterpolationTemplate to TemplateLiteral, it doesn't need to be a new PEP, I can just explicitly reject those spelling options in PEP 501. (The reasons for that PEP's deferral unfortunately still hold, though - eager rendering, runtime value interpolation, and dedicated templating libraries together cover enough cases that the motivation for introducing the semantic complexity of yet another templating option gets weakened dramatically).
If the differences between the proposals run deeper than that, then the proposed new PEP needs to spell them out. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XSGJO3... Code of Conduct: http://python.org/psf/codeofconduct/
On Fri, 2 Jul 2021, 5:12 pm Thomas Güttler, <info@thomas-guettler.de> wrote:
Hi Nick and all other Python ideas friends,
yes, you are right. There is not much difference between PEP-501 or my proposal.
One argument why I would like to prefer backticks:
Some IDEs detect that you want to use a f-string automatically:
You type:
name = 'Peter' print('Hello {name...
and the IDE automatically adds the missing "f" in front of the string:
name = 'Peter' print(f'Hello {name...
This is a handy feature (of PyCharm), which would not work reliably if there are two different prefixes.
-------
You mentioned these things:
eager rendering: I think deferred rendering would increase the complexity a lot. And I think it is not needed.
Eager rendering is f-strings. Any templating proposal necessarily involves a delayed rendering step, when the template is combined with the interpolated values. runtime value interpolation: It is up to the receiver of
types.InterpolationTemplate to handle the data structure.
I really meant runtime template parsing here (i.e. str.format). dedicated templating libraries: One temp after the other. I think HTML and
SQL libraries would adapt as soon as the foundation is available.
The existence of i-strings likely wouldn't change the syntax of jinja2 templates, Django templates, SQL Alchemy, pandas, etc. I would be happy if PEP-501 would come true.
So would I, but I still don't have a compelling answer to the "but it's yet another subtly different way to do it" objection. Cheers, Nick.
Am Fr., 2. Juli 2021 um 12:06 Uhr schrieb Nick Coghlan <ncoghlan@gmail.com>:
On Fri, 2 Jul 2021, 5:12 pm Thomas Güttler, <info@thomas-guettler.de> wrote:
Hi Nick and all other Python ideas friends,
yes, you are right. There is not much difference between PEP-501 or my proposal.
One argument why I would like to prefer backticks:
Some IDEs detect that you want to use a f-string automatically:
You type:
name = 'Peter' print('Hello {name...
and the IDE automatically adds the missing "f" in front of the string:
name = 'Peter' print(f'Hello {name...
This is a handy feature (of PyCharm), which would not work reliably if there are two different prefixes.
-------
You mentioned these things:
eager rendering: I think deferred rendering would increase the complexity a lot. And I think it is not needed.
Eager rendering is f-strings. Any templating proposal necessarily involves a delayed rendering step, when the template is combined with the interpolated values.
runtime value interpolation: It is up to the receiver of
types.InterpolationTemplate to handle the data structure.
I really meant runtime template parsing here (i.e. str.format).
dedicated templating libraries: One temp after the other. I think HTML and
SQL libraries would adapt as soon as the foundation is available.
The existence of i-strings likely wouldn't change the syntax of jinja2 templates, Django templates, SQL Alchemy, pandas, etc.
I would be happy if PEP-501 would come true.
So would I, but I still don't have a compelling answer to the "but it's yet another subtly different way to do it" objection.
Today I read the replies to this thread again. Of course there where some "-1" replies, but overall there was positive feedback. Today I played with template literals from Javascript: [image: image.png] I like it. Nick, would you be open to adapting to the JS syntax for PEP-501? I propose `...{var} ...` This means backticks, but without the dollar sign. I would make it like in JS: The string in backticks can span several lines. So what is the next step now? Regards, Thomas
JavaScript's tagged template literals do provide nice ergonomics. These should work well with some variation on PEP 501's types.InterpolationTemplate as the backend implementation. Some thoughts on what that should look like: * I also increasingly prefer the idea of using backticks to define such templates, as opposed to further overloading standard quoting. More below. * We should separate the static elements of the template from evaluated expressions. This implies that a tagged template function would have a signature like def html(template, *exprs). To provide the same user interface as PEP 501, one could write a tag template function "i" that is an identity function - thus demonstrating a basic equivalence of these proposals. * Literal static strings from the templates are available in both raw and cooked forms (Unicode escapes applied). So there's no need to add a "r" prefix for raw. * Expressions are evaluated immediately using normal scoping and rendered later (taking into account escaping rules, et). Such rendering can be into any Python object. So using some "html" tag function would return a DOM (let's call it HtmlElement in this example) which can be further rendered eventually into returned text in the scenario of serving HTML. * Greater likelihood of syntax support. Being able to specify html`<p>Some expr: {x * y}</p>` probably simplifies being able to provide an IDE extension that is aware that the html tag function was imported from some popular library, and can therefore provide assistance with HTML tags or any other aspects of the templating language used. * Backticks are multiline and they nest in the context of expressions. This means there's no need to invent new templating syntax, such as Jinja's {% for ... %} ... {% endfor %} to provide iteration over a sequence. Just use Python in the evaluated expressions. So with reference to the example in the Jinja doc https://jinja.palletsprojects.com/en/3.0.x/templates/#synopsis, we could have functions like the following that can be further composed to create larger DOM objects: def ulistify(id: str, linkage: dict[str, str]) -> HtmlElement: # uses a dict to represent the linkage in this example, so slightly different than the Jinja example return html` <ul id="{id}">{ html`<li><a href="{href}">{caption}</a></li>` for href, caption in linkage.items() }</ul>` It's possible to write something similar using f-string syntax now, but it involves careful tracking of which quotes are being used to support nesting. The interaction of backticks and expression syntax keeps it simple in comparison. An example library in the JavaScript ecosystem that builds out from the JS tagged template literal support is Lit (https://lit.dev/, note that I haven't used this library.) Lit provides some nice functionality by composing HTML DOM out of DOM fragments; and doing this in a lazy fashion (virtual DOM, as ReactJS demonstrated; I am not aware of a fast virtual DOM library in Python as a C Extension, but it would seem like a straightforward thing to write). Anyway, Lit and other similar libraries in JS could help understand the scope of the available innovation if we were to add such functionality. Format specifiers and ! conversions should presumably still be available in expressions and thus available from the template literal object to be used as part of any rendering. However, I'm leaving this open for the moment as to exactly what this looks like - especially if I overlooked any parsing issues for such support! Lastly, it's worth noting that types.InterpolationTemplate could look quite similar to https://262.ecma-international.org/6.0/#sec-tagged-templates if it's a constant object - a template literal in other words - which a tagged function is then applied to with the evaluated expressions. - Jim On Mon, Jul 5, 2021 at 1:10 AM Thomas Güttler <info@thomas-guettler.de> wrote:
Am Fr., 2. Juli 2021 um 12:06 Uhr schrieb Nick Coghlan <ncoghlan@gmail.com
:
On Fri, 2 Jul 2021, 5:12 pm Thomas Güttler, <info@thomas-guettler.de> wrote:
Hi Nick and all other Python ideas friends,
yes, you are right. There is not much difference between PEP-501 or my proposal.
One argument why I would like to prefer backticks:
Some IDEs detect that you want to use a f-string automatically:
You type:
name = 'Peter' print('Hello {name...
and the IDE automatically adds the missing "f" in front of the string:
name = 'Peter' print(f'Hello {name...
This is a handy feature (of PyCharm), which would not work reliably if there are two different prefixes.
-------
You mentioned these things:
eager rendering: I think deferred rendering would increase the complexity a lot. And I think it is not needed.
Eager rendering is f-strings. Any templating proposal necessarily involves a delayed rendering step, when the template is combined with the interpolated values.
runtime value interpolation: It is up to the receiver of
types.InterpolationTemplate to handle the data structure.
I really meant runtime template parsing here (i.e. str.format).
dedicated templating libraries: One temp after the other. I think HTML
and SQL libraries would adapt as soon as the foundation is available.
The existence of i-strings likely wouldn't change the syntax of jinja2 templates, Django templates, SQL Alchemy, pandas, etc.
I would be happy if PEP-501 would come true.
So would I, but I still don't have a compelling answer to the "but it's yet another subtly different way to do it" objection.
Today I read the replies to this thread again.
Of course there where some "-1" replies, but overall there was positive feedback.
Today I played with template literals from Javascript:
[image: image.png]
I like it.
Nick, would you be open to adapting to the JS syntax for PEP-501?
I propose `...{var} ...`
This means backticks, but without the dollar sign.
I would make it like in JS: The string in backticks can span several lines.
So what is the next step now?
Regards, Thomas
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NRJ7NB... Code of Conduct: http://python.org/psf/codeofconduct/
On 5 Jul 2021, at 08:07, Thomas Güttler <info@thomas-guettler.de> wrote:
This means backticks, but without the dollar sign.
In bash the backtick was so often a problem that $(cmd) was added. Having removes the grit-on-Tim's-screen backtick in python 3 I would not like to see it return with its issue of being confused with single-quote. Barry
On Mon, Jul 5, 2021, 12:56 PM Barry Scott <barry@barrys-emacs.org> wrote:
On 5 Jul 2021, at 08:07, Thomas Güttler <info@thomas-guettler.de> wrote:
This means backticks, but without the dollar sign.
In bash the backtick was so often a problem that $(cmd) was added.
Having removes the grit-on-Tim's-screen backtick in python 3 I would not like to see it return with its issue of being confused with single-quote.
One mitigation is that the backtick should always require a tag as a prefix. So seeing something like elem = html`<p>Some item: {value}</p>` is hopefully fairly obvious what's going on - it's not just going to be mixed up with a single quote. Uses like log(f`foo`) should be hopefully discouraged, in the same way that we don't use l (that's the lower-case letter L if you're not reading this email with the numeric codepoints) as a variable, but we are happy enough to write something like limit = 42 - it's clear in the context. However, I think the Bash example is exactly illustrative of the opposite. So in reviewing at least this one FAQ on the topic http://mywiki.wooledge.org/BashFAQ/082, it reminds me of why I don't use backticks in Bash - it's the lack of nesting support when compared to $(...). Ironically, this nesting is exactly what backticks can help here on - an unused character as of Python 3, commonly used for building some type of string/identifier in a variety of languages, that we can give some nice semantics that allows for simple nesting when used in conjunction with braces delimiting expressions. Such braces of course always give us a new nesting, similar to the statement in the wiki above that "$() forces an entirely new context for quoting, so that everything within the command substitution is protected and can be treated as though it were on its own, with no special concern over quoting and escaping." That this usage of backticks has worked quite well for JavaScript provides some useful confirmation. I also expect that no one will confuse this with Bash usage, given that's going to be in Python code - except perhaps in a readily written sh function (so something like sh`...`, which returns a list of stdout or something like that and makes use of shlex, etc). I will leave what horrors that could actually look like to the reader :) although in limited form, it could be quite useful. - Jim
On Tue, Jul 6, 2021 at 6:10 AM Jim Baker <jim.baker@python.org> wrote:
On Mon, Jul 5, 2021, 12:56 PM Barry Scott <barry@barrys-emacs.org> wrote:
On 5 Jul 2021, at 08:07, Thomas Güttler <info@thomas-guettler.de> wrote:
This means backticks, but without the dollar sign.
In bash the backtick was so often a problem that $(cmd) was added.
Having removes the grit-on-Tim's-screen backtick in python 3 I would not like to see it return with its issue of being confused with single-quote.
One mitigation is that the backtick should always require a tag as a prefix. So seeing something like
elem = html`<p>Some item: {value}</p>`
is hopefully fairly obvious what's going on - it's not just going to be mixed up with a single quote. Uses like log(f`foo`) should be hopefully discouraged, in the same way that we don't use l (that's the lower-case letter L if you're not reading this email with the numeric codepoints) as a variable, but we are happy enough to write something like limit = 42 - it's clear in the context.
Question: what's the advantage of this magic syntax over something much simpler: elem = html(i'<p>Some item: {value}</p>') That avoids the backtick problem because it actually IS an apostrophe. Or a double quote. Or triple quotes, whatever you want to use. It's the exact format already used for other special string literal types, including f-strings. ChrisA
On Mon, Jul 5, 2021 at 4:12 PM Jim Baker <jim.baker@python.org> wrote:
in the same way that we don't use l (that's the lower-case letter L if you're not reading this email with the numeric codepoints) as a variable
Speaking of grit on one's screen, I first thought that was a lowercase "i" because I had actual light-colored grit on my screen in precisely the exact place to make the lowercase L look like a lowercase i until I wiped it off with my finger. Which goes to show that we should not underestimate the grit-on-screen concern. It's real, and it's a huge -1 against backticks for any purpose. I'm +0 on the overall concept here, but please say no to backticks.
FWIW, we could make f-strings properly nest too, like you are proposing for backticks. It's just that we'd have to change the lexer. But it would not be any harder than would be for backticks (since it would be the same algorithm), nor would it be backward incompatible. So this is not an argument for backticks. Separately, should there be a way to *delay* evaluation of the templated expressions (like we explored in our private little prototype last year)? -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Mon, Jul 5, 2021, 2:40 PM Guido van Rossum <guido@python.org> wrote:
FWIW, we could make f-strings properly nest too, like you are proposing for backticks. It's just that we'd have to change the lexer. But it would not be any harder than would be for backticks (since it would be the same algorithm), nor would it be backward incompatible. So this is not an argument for backticks.
Good point. At some point, I was probably thinking of backticks without a tag, since JS supports this for their f-string like scenario. but if we always require a tag - so long as it's not a prefix already in use (b, f, r, fr, hopefully not forgetting as I type this email in a parking lot...) - then it can be disambiguated using standard quotes.
Separately, should there be a way to *delay* evaluation of the templated expressions (like we explored in our private little prototype last year)?
I think so, but probably with an explicit marker on *each* deferred expression. I'm in favor of Julia's expression quote, which generally needs to be enclosed in parentheses, but possibly not needed in expression braces (at the risk of looking like a standalone format spec). https://docs.julialang.org/en/v1/manual/metaprogramming/ So this would like x = 42 d = deferred_tag"Some expr: {:(x*2)}" All that is happening here is that this being wrapped in a lambda, which captures any scope lexically as usual. Then per that experiment you mentioned, it's possible to use that scope using fairly standard - or at least portable to other Python implementations - metaprogramming, including the deferred evaluation of the lambda. (No frame walking required!) Other syntax could work for deferring. Maybe backticks, they could be available? 😉 - Jim
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On 6/07/21 9:56 am, Jim Baker wrote:
d = deferred_tag"Some expr: {:(x*2)}"
All that is happening here is that this being wrapped in a lambda, which captures any scope lexically as usual.
Is there reason to think this will be a common enough requirement to justify having an abbreviated syntax for a parameterless lambda that's only available in template expressions? -- Greg
On Tue, Jul 6, 2021 at 9:39 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 6/07/21 9:56 am, Jim Baker wrote:
d = deferred_tag"Some expr: {:(x*2)}"
All that is happening here is that this being wrapped in a lambda, which captures any scope lexically as usual.
Is there reason to think this will be a common enough requirement to justify having an abbreviated syntax for a parameterless lambda that's only available in template expressions?
If it's just being wrapped in a lambda function, probably nothing, but I think that that would be *very* surprising behaviour. People will expect that the expressions' values will be collected at the point you hit the interpolated string, not later when it gets used. ChrisA
On Mon, Jul 5, 2021 at 5:05 PM Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Jul 6, 2021 at 9:39 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 6/07/21 9:56 am, Jim Baker wrote:
d = deferred_tag"Some expr: {:(x*2)}"
All that is happening here is that this being wrapped in a lambda,
which
captures any scope lexically as usual.
Is there reason to think this will be a common enough requirement to justify having an abbreviated syntax for a parameterless lambda that's only available in template expressions?
If it's just being wrapped in a lambda function, probably nothing, but I think that that would be *very* surprising behaviour. People will expect that the expressions' values will be collected at the point you hit the interpolated string, not later when it gets used.
It would be surprising indeed if this was the *default* behavior, that's why you have to specially mark it using {:...} in Jim's proposal. IIUC the idea is that the entire template becomes effectively a lambda. It's useful if the evaluation is relatively expensive and may never be needed, e.g. for logging at a level that is off in production. We can debate whether it's better to mark individual substitutions with something like {:...} or whether we should mark the template as a whole (obviously the marking must be understandable by the parser). -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Tue, Jul 6, 2021 at 10:44 AM Guido van Rossum <guido@python.org> wrote:
On Mon, Jul 5, 2021 at 5:05 PM Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Jul 6, 2021 at 9:39 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 6/07/21 9:56 am, Jim Baker wrote:
d = deferred_tag"Some expr: {:(x*2)}"
All that is happening here is that this being wrapped in a lambda, which captures any scope lexically as usual.
Is there reason to think this will be a common enough requirement to justify having an abbreviated syntax for a parameterless lambda that's only available in template expressions?
If it's just being wrapped in a lambda function, probably nothing, but I think that that would be *very* surprising behaviour. People will expect that the expressions' values will be collected at the point you hit the interpolated string, not later when it gets used.
It would be surprising indeed if this was the *default* behavior, that's why you have to specially mark it using {:...} in Jim's proposal.
IIUC the idea is that the entire template becomes effectively a lambda. It's useful if the evaluation is relatively expensive and may never be needed, e.g. for logging at a level that is off in production. We can debate whether it's better to mark individual substitutions with something like {:...} or whether we should mark the template as a whole (obviously the marking must be understandable by the parser).
Ahh, I see what you mean. It'll still be surprising at times (cf the bizarre issues that can happen with mutable objects and Chrome's JS developer tools), but could be beneficial, yeah. ChrisA
On Tue, 6 Jul 2021, 7:56 am Jim Baker, <jim.baker@python.org> wrote:
On Mon, Jul 5, 2021, 2:40 PM Guido van Rossum <guido@python.org> wrote:
FWIW, we could make f-strings properly nest too, like you are proposing for backticks. It's just that we'd have to change the lexer. But it would not be any harder than would be for backticks (since it would be the same algorithm), nor would it be backward incompatible. So this is not an argument for backticks.
Good point. At some point, I was probably thinking of backticks without a tag, since JS supports this for their f-string like scenario. but if we always require a tag - so long as it's not a prefix already in use (b, f, r, fr, hopefully not forgetting as I type this email in a parking lot...) - then it can be disambiguated using standard quotes.
There's a deferred PEP proposing a resolution to the f-string nesting limitations: https://www.python.org/dev/peps/pep-0536/#motivation
Separately, should there be a way to *delay* evaluation of the templated expressions (like we explored in our private little prototype last year)?
I think so, but probably with an explicit marker on *each* deferred expression. I'm in favor of Julia's expression quote, which generally needs to be enclosed in parentheses, but possibly not needed in expression braces (at the risk of looking like a standalone format spec). https://docs.julialang.org/en/v1/manual/metaprogramming/
So this would like x = 42 d = deferred_tag"Some expr: {:(x*2)}"
All that is happening here is that this being wrapped in a lambda, which captures any scope lexically as usual. Then per that experiment you mentioned, it's possible to use that scope using fairly standard - or at least portable to other Python implementations - metaprogramming, including the deferred evaluation of the lambda. (No frame walking required!)
Other syntax could work for deferring.
It reminds me of the old simple implicit lambda proposal: https://www.python.org/dev/peps/pep-0312/ The old proposal has more ambiguities to avoid now due to type hinting syntax but a parentheses-required version could work: "(:call_result)" Cheers, Nick.
On Thu, Jul 8, 2021 at 6:25 PM Nick Coghlan <ncoghlan@gmail.com> wrote:
On Tue, 6 Jul 2021, 7:56 am Jim Baker, <jim.baker@python.org> wrote:
On Mon, Jul 5, 2021, 2:40 PM Guido van Rossum <guido@python.org> wrote:
FWIW, we could make f-strings properly nest too, like you are proposing for backticks. It's just that we'd have to change the lexer. But it would not be any harder than would be for backticks (since it would be the same algorithm), nor would it be backward incompatible. So this is not an argument for backticks.
Good point. At some point, I was probably thinking of backticks without a tag, since JS supports this for their f-string like scenario. but if we always require a tag - so long as it's not a prefix already in use (b, f, r, fr, hopefully not forgetting as I type this email in a parking lot...) - then it can be disambiguated using standard quotes.
There's a deferred PEP proposing a resolution to the f-string nesting limitations: https://www.python.org/dev/peps/pep-0536/#motivation
Thanks for pointing that PEP out - this looks quite reasonable for both f-strings and for the tagged templates proposed here. I believe some limitations on nesting expressions in f-strings with quote changing was introduced in a relatively recent fix in 3.8 or 3.9, but I need to find the specifics.
Separately, should there be a way to *delay* evaluation of the templated expressions (like we explored in our private little prototype last year)?
I think so, but probably with an explicit marker on *each* deferred expression. I'm in favor of Julia's expression quote, which generally needs to be enclosed in parentheses, but possibly not needed in expression braces (at the risk of looking like a standalone format spec). https://docs.julialang.org/en/v1/manual/metaprogramming/
So this would like x = 42 d = deferred_tag"Some expr: {:(x*2)}"
All that is happening here is that this being wrapped in a lambda, which captures any scope lexically as usual. Then per that experiment you mentioned, it's possible to use that scope using fairly standard - or at least portable to other Python implementations - metaprogramming, including the deferred evaluation of the lambda. (No frame walking required!)
Other syntax could work for deferring.
It reminds me of the old simple implicit lambda proposal: https://www. python.org/dev/peps/pep-0312/
The old proposal has more ambiguities to avoid now due to type hinting syntax but a parentheses-required version could work: "(:call_result)"
PEP 312 - suitably modified - could be quite useful for both deferring evaluation in tagged templates and function calls (or other places where we might want to put in a lambda). Ideally we can use minimum parens as well, so it's tag"{:x+1}" f(:x+1, :y+2) (but I haven't looked at ambiguities here). A further idea that might make the approach in PEP 312 more powerful than simply being an implicit lambda is if this was like Julia's quoted expressions https://docs.julialang.org/en/v1/manual/metaprogramming/#Quoting and we recorded the *text body of the expression*. (Such recording would presumably be done in any template scheme, but let's make this explicit now.) So let's call this implicit lambda with recorded text body a *quoted expression*. What's nice about doing such quoted expressions is that the lambda also captures any lexical scope. So if I have an expression tag"{:x*y}", the variables x and y are referenced appropriately. This means it would be possible to do such things as rewrite complex expressions - eg an index query on a Pandas data frame - while avoiding the use of dynamic scope. I wrote a small piece of code to show how this can be exercised. The text of the expression is simply an attribute on the lambda named "body", but we might have a new type defined (eg types.QuotedExpression). ``` from functools import update_wrapper from textwrap import dedent from types import FunctionType def rewrite(f, new_body): # Create a temporary outer function with arguments for the free variables of # the original lambda wrapping an expression. This approach allows the new # inner function, which has a new body, to compile with its dependency on # free vars. # # When called, this new inner function code object can continue to access # these variables from the cell vars in the closure - even without this # outer function, which we discard. # # This rewriting is generalizable to arbitrary functions, although the # syntax we are exploring is only for expressions. # # A similar idea - for monkeypatching an inner function - is explored in # https://stackoverflow.com/a/27550237 by Martijn Pieters' detailed answer. # # Related ideas include https://en.wikipedia.org/wiki/Lambda_lifting and # lambda dropping where the free vars are lifted up/dropped from the # function parameters. This requires the (usual :) extra level of # indirection by another function. scoped = dedent(f""" def outer({", ".join(f.__code__.co_freevars)}): def inner(): return {new_body} """) capture = {} exec(scoped, f.__globals__, capture) # NOTE: outer is co_consts[0], inner is co_consts[1] - this may not be # guaranteed by the compiler, so iterate over if necessary. inner_code = capture["outer"].__code__.co_consts[1] new_f = FunctionType( inner_code, f.__globals__, closure=f.__closure__) update_wrapper(new_f, f) new_f.body = new_body return new_f def test_rewrite(): x = 2 y = 3 # x, y and free vars in scopeit, and its own nested lambdas, assigned # to f and g below. We will also introduce an additional free var z # in the parameter of scopeit. def scopeit(z): f = lambda: x * y + z f.body = "x * y + z" print(f"{f()=}") # We can do an arbitrary manipulation on the above expression, so # long as we use variables that are in the original scope (or symtab). # Note that adding new variables will look up globally (may or may # not exist, if not a NameError is raised). print(f.__code__.co_freevars) g = rewrite(f, "-(x * y + z)") print(f"{g()=}") scopeit(5) if __name__ == "__main__": test_rewrite() ``` Similar ideas were explored in https://github.com/jimbaker/fl-string-pep/issues - Jim On Thu, Jul 8, 2021 at 6:25 PM Nick Coghlan <ncoghlan@gmail.com> wrote:
On Tue, 6 Jul 2021, 7:56 am Jim Baker, <jim.baker@python.org> wrote:
On Mon, Jul 5, 2021, 2:40 PM Guido van Rossum <guido@python.org> wrote:
FWIW, we could make f-strings properly nest too, like you are proposing for backticks. It's just that we'd have to change the lexer. But it would not be any harder than would be for backticks (since it would be the same algorithm), nor would it be backward incompatible. So this is not an argument for backticks.
Good point. At some point, I was probably thinking of backticks without a tag, since JS supports this for their f-string like scenario. but if we always require a tag - so long as it's not a prefix already in use (b, f, r, fr, hopefully not forgetting as I type this email in a parking lot...) - then it can be disambiguated using standard quotes.
There's a deferred PEP proposing a resolution to the f-string nesting limitations: https://www.python.org/dev/peps/pep-0536/#motivation
Separately, should there be a way to *delay* evaluation of the templated expressions (like we explored in our private little prototype last year)?
I think so, but probably with an explicit marker on *each* deferred expression. I'm in favor of Julia's expression quote, which generally needs to be enclosed in parentheses, but possibly not needed in expression braces (at the risk of looking like a standalone format spec). https://docs.julialang.org/en/v1/manual/metaprogramming/
So this would like x = 42 d = deferred_tag"Some expr: {:(x*2)}"
All that is happening here is that this being wrapped in a lambda, which captures any scope lexically as usual. Then per that experiment you mentioned, it's possible to use that scope using fairly standard - or at least portable to other Python implementations - metaprogramming, including the deferred evaluation of the lambda. (No frame walking required!)
Other syntax could work for deferring.
It reminds me of the old simple implicit lambda proposal: https://www.python.org/dev/peps/pep-0312/
The old proposal has more ambiguities to avoid now due to type hinting syntax but a parentheses-required version could work: "(:call_result)"
Cheers, Nick.
On 6/07/21 8:39 am, Guido van Rossum wrote:
FWIW, we could make f-strings properly nest too... So this is not an argument for backticks.
An argument might be that single and double quotes currently do not nest and never have done, so having them start to nest but only in the context of f-strings could be confusing. Whereas backticks are currently unused, and they used to nest back when they were used, so giving them a new nesting usage will be unsurprising to those with long enough memories. -- Greg
On 02/07/2021 08:12, Thomas Güttler wrote:
Hi Nick and all other Python ideas friends,
yes, you are right. There is not much difference between PEP-501 or my proposal.
One argument why I would like to prefer backticks:
Some IDEs detect that you want to use a f-string automatically:
You type:
name = 'Peter' print('Hello {name...
and the IDE automatically adds the missing "f" in front of the string:
name = 'Peter' print(f'Hello {name...
This is a handy feature (of PyCharm), which would not work reliably if there are two different prefixes.
Grrr! And I thought I was an idiot for frequently omitting the _"f" in f-strings_. (Say the underlined bit repeatedly to yourself ... but only as long as you won't be overheard.)
Strong -1 As others noted in prior discussion, even if this existed, it works be an anti-pattern for SQL. So basically, it's just baking in an HTML-only template language into the language syntax. Python already had excellent HTML templating in libraries. The fact Django has a function with a long name just suggests importing it with a shorter name. Python has many uses having nothing to do with web pages. This could make sense for PHP (does anyone use that still?). It's a poor for for Python. On Thu, Jun 10, 2021, 2:31 AM Thomas Güttler <info@thomas-guettler.de> wrote:
Thank you Guido, Chris, Matt and Richard for your feedback to my last email.
Here is an updated version called "Template Literals".
I am looking for a core developer who can sponsor this PEP.
Please speak up if you want to help me.
Regards, Thomas Güttler
Source and updates: Pre-PEP 9999 <https://github.com/guettli/peps/blob/master/pep-9999.rst>
PEP: 9999 Title: Template Literals Author: Thomas Güttler <info at thomas-guettler.de> Sponsor: TODO Status: Draft Type: Standards Track Content-Type: text/x-rst <http:///dev/peps/pep-0012> Created: 08-Jun-2021 Python-Version: TODO Post-History: 08-Jun-2021 ------------------------------
Contents
- Abstract <#m_-138379983175526697_abstract> - Motivation <#m_-138379983175526697_motivation> - Rationale <#m_-138379983175526697_rationale> - Specification <#m_-138379983175526697_specification> - Security Implications <#m_-138379983175526697_security-implications> - Reference Implementation <#m_-138379983175526697_reference-implementation> - Alternative Ideas <#m_-138379983175526697_alternative-ideas> - Rejected Ideas <#m_-138379983175526697_rejected-ideas> - Open Issues <#m_-138379983175526697_open-issues> - References <#m_-138379983175526697_references> - Copyright <#m_-138379983175526697_copyright>
Abstract <#m_-138379983175526697_id6>
This PEP adds Template Literals to Python.
To avoid code injection like XSS or SQL-injection Template Literals can help you to write save Python code.
Template Literals provide an easy way to access the local and global variables (like f-strings), so that passing a dictionary to the Template is not necessary. Motivation <#m_-138379983175526697_id7>
In the context of web development Python can do more than providing REST APIs via http. With the trend to Server-Side-Rendering, we face a fundamental question:
How to create HTML with Python?
If you use the FrOW pattern (HTML fragments over the wire) [1] <#m_-138379983175526697_frow>, then you will be writing small methods returning small HTML fragments.
As a developer I want to pass escaped data into template literals to be as simple as possible. Rationale <#m_-138379983175526697_id8>
Imagine you want to create a small HTML fragment in Python, and return it as HTTP-Response:
HttpResponse(f''' <h1>Hi {name}</h1> Your messages: {messages}''')
The problem in above example is, that no escaping gets done.
In above example "name" and "messages" should be treated differently.
The variable "name" should get escaped. For example if the name is "Mary & Bob", the result should be "Mary & Bob".
The variable "messages" contains HTML which is already escaped. It should not be escaped again.
Most frameworks have a way to do this conditional escaping.
For example Django uses conditional_escape() <https://docs.djangoproject.com/en/3.2/ref/utils/#django.utils.html.condition...> [2] <#m_-138379983175526697_id2>
With the help of conditional_escape() the above problem could be solved like this:
HttpResponse(f''' <h1>Hi {conditional_escape(name)}</h1> Your messages: {conditional_escape(messages)}''')
This solution has two drawbacks:
1. It is too verbose. Typing "conditional_escape(...)" again and again is cumbersome. 2. If a conditional_escape() gets forgotten Cross-site scripting attacks could be possible, since malicious users could inject HTML.
Specification <#m_-138379983175526697_id9>
Template Literals use backticks (like JavaScript Template Literals <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l...> [3] <#m_-138379983175526697_id4>)
Example:
name = 'Mary & Bob' messages = `<ul><li>message1</li><li>message2</li></ul>` return HttpResponse(` <h1>Hi {name}</h1> Your messages: {messages}
Today: {datetime.date.today()}`)
Expressions within curly braces get handled liked in f-strings (PEP-498).
The Template Literal creates an instance of the new class types.TemplateLiteral.
types.TemplateLiteral has two attributes:
- template: The raw string inside the backticks. - tokens: A list of tuples: (value, is_literal).
For above example this would mean:
template = ''' <h1>Hi {name}</h1> Your messages: {messages}
Today: {datetime.date.today()}'''
tokens = [ ('\n <h1>Hi ', True), ('Mary & Bob', False), ('</h1>\n Your messages: ', True), (<TemplateLiteral "<ul><li>message1</li><li>message2</li></ul>">, False), ('\n\n Today: ', True), (<datetime.date(2021, 6, 9)>, False) ]
It is outside this PEP how a consumer of TemplateLiteral handles this data structure.
For example the Django web framework could transform a TemplateLiteral to a SafeString like this:
def template_literal_to_safestring(template_literal): return mark_safe( ''.join( [ conditional_escape(value) if not is_literal else value for (value, is_literal) in template_literal.tokens ] ) )
This PEP is not related or constraint to the Django framework. It is even not related to HTML. It can be used for any kind of templating. Security Implications <#m_-138379983175526697_id10>
Template Literals can execute arbitrary code (like f-strings).
Template Literals get created by Python developers, not by users. If you want to make templates available for users (for example if you develop a CMS), then please use a different solution. Reference Implementation <#m_-138379983175526697_id11>
TODO Alternative Ideas <#m_-138379983175526697_id12>
Instead of backticks for example t'...' could be used. Rejected Ideas <#m_-138379983175526697_id13>
TODO Open Issues <#m_-138379983175526697_id14>
TODO References <#m_-138379983175526697_id15> [1] <#m_-138379983175526697_id1> FrOW, "HTML Fragments Over the Wire". Frameworks like Unpoly, Hotwire or htmx. [2] <#m_-138379983175526697_id3> https://docs.djangoproject.com/en/3.2/ref/utils/#django.utils.html.condition... [3] <#m_-138379983175526697_id5> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l... Copyright <#m_-138379983175526697_id16>
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LYAC7J... Code of Conduct: http://python.org/psf/codeofconduct/
Am Do., 10. Juni 2021 um 15:33 Uhr schrieb David Mertz <mertz@gnosis.cx>:
Strong -1
As others noted in prior discussion, even if this existed, it works be an anti-pattern for SQL. So basically, it's just baking in an HTML-only template language into the language syntax.
Python already had excellent HTML templating in libraries. The fact Django has a function with a long name just suggests importing it with a shorter name.
Python has many uses having nothing to do with web pages. This could make sense for PHP (does anyone use that still?). It's a poor for for Python.
PHP is very productive. The majority of sites are written in PHP. The only thing which PHP has not: Respect. It does not get respect since it is too easy. Python is getting pushed into the backend during the last years, while web frontend is mostly done in Javascript. I think this is the wrong direction. Together with tools like htmx (fragments over the wire), I would rather choose PHP than Javascript. PHP is great for creating HTML. I know that the easy solution does not get much respect, but maybe you understand that making it easy to create HTML would be a major benefit for Python. Regards, Thomas
On Thu, Jun 10, 2021 at 11:50 PM Thomas Güttler <info@thomas-guettler.de> wrote:
Am Do., 10. Juni 2021 um 15:33 Uhr schrieb David Mertz <mertz@gnosis.cx>:
Strong -1
As others noted in prior discussion, even if this existed, it works be an anti-pattern for SQL. So basically, it's just baking in an HTML-only template language into the language syntax.
Python already had excellent HTML templating in libraries. The fact Django has a function with a long name just suggests importing it with a shorter name.
Python has many uses having nothing to do with web pages. This could make sense for PHP (does anyone use that still?). It's a poor for for Python.
PHP is very productive. The majority of sites are written in PHP.
The only thing which PHP has not: Respect.
It does not get respect since it is too easy.
Too easy to do things badly, that's why PHP lacks respect. It's not about how easy it is to quickly whip up a web site; it's about how easy it is to quickly whip up a really really bad web site, and how the design of the language encourages practices that tend towards that.
Python is getting pushed into the backend during the last years, while web frontend is mostly done in Javascript.
I think this is the wrong direction.
Together with tools like htmx (fragments over the wire), I would rather choose PHP than Javascript.
What's the advantage of htmx? When I want to build a good interactive web site, my general pattern is a back end with a well-defined API, and a front end in JavaScript that makes use of this API. That API is usually going to be based on either a RESTful (or roughly REST-like) JSON transactional system, or something like websockets, again carrying JSON payloads. HTML is the realm of the display, not the back end.
PHP is great for creating HTML. I know that the easy solution does not get much respect, but maybe you understand that making it easy to create HTML would be a major benefit for Python.
I actually like the idea, but this justification seems *extremely* weak. ChrisA
Am Do., 10. Juni 2021 um 16:04 Uhr schrieb Chris Angelico <rosuav@gmail.com
:
On Thu, Jun 10, 2021 at 11:50 PM Thomas Güttler <info@thomas-guettler.de> wrote:
What's the advantage of htmx? When I want to build a good interactive web site, my general pattern is a back end with a well-defined API, and a front end in JavaScript that makes use of this API. That API is usually going to be based on either a RESTful (or roughly REST-like) JSON transactional system, or something like websockets, again carrying JSON payloads. HTML is the realm of the display, not the back end.
It depends on the use-case. If you want to create a "real" application like gmail, then it is ok to let the user wait 3 seconds until the page is loaded. But if you want your page to load fast, then the current best practice is server-side-rendering. I personally was a big fan of the "backend for frontend" solutions like GraphQL. But my mind changed. I prefer SQL+ORM and server side rendering today. HTMX gives you a way to create fast loading pages which are interactive. If you write many small methods returning small html snippets, then a f-string like solution with conditional_escape() support would be super cool. I held a talk about htmx at DjangoCon EU, here are the slides: https://docs.google.com/presentation/d/1Gx1UGVAgD2ALLOucsIm9myF5mDflbP06-M6_... But this is getting off-topic. Regards, Thomas Güttler
On Thu, Jun 10, 2021 at 4:07 PM Chris Angelico <rosuav@gmail.com> wrote:
What's the advantage of htmx? When I want to build a good interactive web site, my general pattern is a back end with a well-defined API, and a front end in JavaScript that makes use of this API. That API is usually going to be based on either a RESTful (or roughly REST-like) JSON transactional system, or something like websockets, again carrying JSON payloads. HTML is the realm of the display, not the back end.
I use unpoly but the deal is the same: backend HTML rendering lets us leverage all sorts of meta programing, ie. add a field to a form and you don't have to change your presentation layer: the Form instance will automatically render a form field with all the necessary validation, ie. in case of a username field which must be unique in the database. -- ∞
On Thu, Jun 10, 2021 at 2:35 PM David Mertz <mertz@gnosis.cx> wrote:
Strong -1
As others noted in prior discussion, even if this existed, it works be an anti-pattern for SQL. So basically, it's just baking in an HTML-only template language into the language syntax.
The discussion I could find on the mailing list talked about some issues around implementation, but I didn't see it being accepted as an anti-pattern.
Something I don't understand is whether there is anything about this proposed feature that can't be accomplished with a simple function. IIUC, the proposal turns this: foo = "spam & eggs" `Here, have some {foo}.` ...into something like this (I am making up a more readable repr): TemplateLiteral( template = " Here, have some {foo}.", tokens = (("Here, have some ", True), ("spam & eggs", False)), ) What is it about this task that cannot be accomplished with a class and function? Skeleton might look like: @dataclass class TemplateLiteral: template: str tokens: Sequence(Tuple(str, bool)) = field(init = False) def templify(s: str) -> TemplateLiteral: ... And use it like this:
templify("Here, have some {foo}.") TemplateLiteral(template = " Here, have some {foo}.", tokens = (("Here, have some ", True), ("spam & eggs", False)))
What is it about this task that requires it being handled at the language level...? --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler On Thu, Jun 10, 2021 at 9:36 AM David Mertz <mertz@gnosis.cx> wrote:
Strong -1
As others noted in prior discussion, even if this existed, it works be an anti-pattern for SQL. So basically, it's just baking in an HTML-only template language into the language syntax.
Python already had excellent HTML templating in libraries. The fact Django has a function with a long name just suggests importing it with a shorter name.
Python has many uses having nothing to do with web pages. This could make sense for PHP (does anyone use that still?). It's a poor for for Python.
On Thu, Jun 10, 2021, 2:31 AM Thomas Güttler <info@thomas-guettler.de> wrote:
Thank you Guido, Chris, Matt and Richard for your feedback to my last email.
Here is an updated version called "Template Literals".
I am looking for a core developer who can sponsor this PEP.
Please speak up if you want to help me.
Regards, Thomas Güttler
Source and updates: Pre-PEP 9999 <https://github.com/guettli/peps/blob/master/pep-9999.rst>
PEP: 9999 Title: Template Literals Author: Thomas Güttler <info at thomas-guettler.de> Sponsor: TODO Status: Draft Type: Standards Track Content-Type: text/x-rst <http:///dev/peps/pep-0012> Created: 08-Jun-2021 Python-Version: TODO Post-History: 08-Jun-2021 ------------------------------
Contents
- Abstract <#m_2818986526677889823_m_-138379983175526697_abstract> - Motivation <#m_2818986526677889823_m_-138379983175526697_motivation> - Rationale <#m_2818986526677889823_m_-138379983175526697_rationale> - Specification <#m_2818986526677889823_m_-138379983175526697_specification> - Security Implications <#m_2818986526677889823_m_-138379983175526697_security-implications> - Reference Implementation <#m_2818986526677889823_m_-138379983175526697_reference-implementation> - Alternative Ideas <#m_2818986526677889823_m_-138379983175526697_alternative-ideas> - Rejected Ideas <#m_2818986526677889823_m_-138379983175526697_rejected-ideas> - Open Issues <#m_2818986526677889823_m_-138379983175526697_open-issues> - References <#m_2818986526677889823_m_-138379983175526697_references> - Copyright <#m_2818986526677889823_m_-138379983175526697_copyright>
Abstract <#m_2818986526677889823_m_-138379983175526697_id6>
This PEP adds Template Literals to Python.
To avoid code injection like XSS or SQL-injection Template Literals can help you to write save Python code.
Template Literals provide an easy way to access the local and global variables (like f-strings), so that passing a dictionary to the Template is not necessary. Motivation <#m_2818986526677889823_m_-138379983175526697_id7>
In the context of web development Python can do more than providing REST APIs via http. With the trend to Server-Side-Rendering, we face a fundamental question:
How to create HTML with Python?
If you use the FrOW pattern (HTML fragments over the wire) [1] <#m_2818986526677889823_m_-138379983175526697_frow>, then you will be writing small methods returning small HTML fragments.
As a developer I want to pass escaped data into template literals to be as simple as possible. Rationale <#m_2818986526677889823_m_-138379983175526697_id8>
Imagine you want to create a small HTML fragment in Python, and return it as HTTP-Response:
HttpResponse(f''' <h1>Hi {name}</h1> Your messages: {messages}''')
The problem in above example is, that no escaping gets done.
In above example "name" and "messages" should be treated differently.
The variable "name" should get escaped. For example if the name is "Mary & Bob", the result should be "Mary & Bob".
The variable "messages" contains HTML which is already escaped. It should not be escaped again.
Most frameworks have a way to do this conditional escaping.
For example Django uses conditional_escape() <https://docs.djangoproject.com/en/3.2/ref/utils/#django.utils.html.condition...> [2] <#m_2818986526677889823_m_-138379983175526697_id2>
With the help of conditional_escape() the above problem could be solved like this:
HttpResponse(f''' <h1>Hi {conditional_escape(name)}</h1> Your messages: {conditional_escape(messages)}''')
This solution has two drawbacks:
1. It is too verbose. Typing "conditional_escape(...)" again and again is cumbersome. 2. If a conditional_escape() gets forgotten Cross-site scripting attacks could be possible, since malicious users could inject HTML.
Specification <#m_2818986526677889823_m_-138379983175526697_id9>
Template Literals use backticks (like JavaScript Template Literals <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l...> [3] <#m_2818986526677889823_m_-138379983175526697_id4>)
Example:
name = 'Mary & Bob' messages = `<ul><li>message1</li><li>message2</li></ul>` return HttpResponse(` <h1>Hi {name}</h1> Your messages: {messages}
Today: {datetime.date.today()}`)
Expressions within curly braces get handled liked in f-strings (PEP-498).
The Template Literal creates an instance of the new class types.TemplateLiteral.
types.TemplateLiteral has two attributes:
- template: The raw string inside the backticks. - tokens: A list of tuples: (value, is_literal).
For above example this would mean:
template = ''' <h1>Hi {name}</h1> Your messages: {messages}
Today: {datetime.date.today()}'''
tokens = [ ('\n <h1>Hi ', True), ('Mary & Bob', False), ('</h1>\n Your messages: ', True), (<TemplateLiteral "<ul><li>message1</li><li>message2</li></ul>">, False), ('\n\n Today: ', True), (<datetime.date(2021, 6, 9)>, False) ]
It is outside this PEP how a consumer of TemplateLiteral handles this data structure.
For example the Django web framework could transform a TemplateLiteral to a SafeString like this:
def template_literal_to_safestring(template_literal): return mark_safe( ''.join( [ conditional_escape(value) if not is_literal else value for (value, is_literal) in template_literal.tokens ] ) )
This PEP is not related or constraint to the Django framework. It is even not related to HTML. It can be used for any kind of templating. Security Implications <#m_2818986526677889823_m_-138379983175526697_id10>
Template Literals can execute arbitrary code (like f-strings).
Template Literals get created by Python developers, not by users. If you want to make templates available for users (for example if you develop a CMS), then please use a different solution. Reference Implementation <#m_2818986526677889823_m_-138379983175526697_id11>
TODO Alternative Ideas <#m_2818986526677889823_m_-138379983175526697_id12>
Instead of backticks for example t'...' could be used. Rejected Ideas <#m_2818986526677889823_m_-138379983175526697_id13>
TODO Open Issues <#m_2818986526677889823_m_-138379983175526697_id14>
TODO References <#m_2818986526677889823_m_-138379983175526697_id15> [1] <#m_2818986526677889823_m_-138379983175526697_id1> FrOW, "HTML Fragments Over the Wire". Frameworks like Unpoly, Hotwire or htmx. [2] <#m_2818986526677889823_m_-138379983175526697_id3> https://docs.djangoproject.com/en/3.2/ref/utils/#django.utils.html.condition... [3] <#m_2818986526677889823_m_-138379983175526697_id5> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l... Copyright <#m_2818986526677889823_m_-138379983175526697_id16>
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LYAC7J... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/6MMGTZ... Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Jun 10, 2021 at 11:58 PM Ricky Teachey <ricky@teachey.org> wrote:
Something I don't understand is whether there is anything about this proposed feature that can't be accomplished with a simple function.
IIUC, the proposal turns this:
foo = "spam & eggs" `Here, have some {foo}.`
...into something like this (I am making up a more readable repr):
TemplateLiteral( template = " Here, have some {foo}.", tokens = (("Here, have some ", True), ("spam & eggs", False)), )
What is it about this task that cannot be accomplished with a class and function? Skeleton might look like:
@dataclass class TemplateLiteral: template: str tokens: Sequence(Tuple(str, bool)) = field(init = False)
def templify(s: str) -> TemplateLiteral: ...
And use it like this:
templify("Here, have some {foo}.") TemplateLiteral(template = " Here, have some {foo}.", tokens = (("Here, have some ", True), ("spam & eggs", False)))
What is it about this task that requires it being handled at the language level...?
The same thing that stops it for f-strings: there's no way to fetch variables from context other than either passing all your locals or repeating all the names multiple times. This proposal is basically for a way to take an f-string-like construct and, instead of calling format() on each of the values and joining them together into a string, you do something else with it. Or from a language perspective, you package it all up and hand it to a custom function. So it's basically an f-string minus the final step - which is why PEP 501 described f-strings in terms of interpolated strings. ChrisA
On Thu, Jun 10, 2021 at 7:10 AM Chris Angelico <rosuav@gmail.com> wrote:
This proposal is basically for a way to take an f-string-like construct and, instead of calling format() on each of the values and joining them together into a string, you do something else with it. Or from a language perspective, you package it all up and hand it to a custom function.
So it's basically an f-string minus the final step - which is why PEP 501 described f-strings in terms of interpolated strings.
Unevaluated f-strings is a nice way to think about this functionality. Another use-case that comes to mind is logging. The Google Python style guide says not to use f-strings, for example, because it wants to be able to collect the unexpanded pattern strings and not waste time rendering unlogged messages: https://google.github.io/styleguide/pyguide.html#3101-logging
Am Fr., 11. Juni 2021 um 03:17 Uhr schrieb Stephan Hoyer <shoyer@gmail.com>:
On Thu, Jun 10, 2021 at 7:10 AM Chris Angelico <rosuav@gmail.com> wrote:
This proposal is basically for a way to take an f-string-like construct and, instead of calling format() on each of the values and joining them together into a string, you do something else with it. Or from a language perspective, you package it all up and hand it to a custom function.
So it's basically an f-string minus the final step - which is why PEP 501 described f-strings in terms of interpolated strings.
Unevaluated f-strings is a nice way to think about this functionality.
Another use-case that comes to mind is logging. The Google Python style guide says not to use f-strings, for example, because it wants to be able to collect the unexpanded pattern strings and not waste time rendering unlogged messages: https://google.github.io/styleguide/pyguide.html#3101-logging
Thank you Stephan for this feedback. I added Logging to the draft: https://github.com/guettli/peps/blob/master/pep-9999.rst#logging Regards, Thomas
Thomas Güttler writes:
Am Fr., 11. Juni 2021 um 03:17 Uhr schrieb Stephan Hoyer <shoyer@gmail.com>:
Unevaluated f-strings is a nice way to think about this functionality.
But they're not "unevaluated" in a lot of important ways. A better term might be "pre-assembled". :-)
Another use-case that comes to mind is logging. The Google Python style guide says not to use f-strings, for example, because it wants to be able to collect the unexpanded pattern strings and not waste time rendering unlogged messages: https://google.github.io/styleguide/pyguide.html#3101-logging
Thank you Stephan for this feedback. I added Logging to the draft: https://github.com/guettli/peps/blob/master/pep-9999.rst#logging
If this does solve the logging problem, it would be a killer app (but that still might not be enough, given that PEP 501 couldn't clear the bar and logging was a leading application for that PEP). Perhaps you should focus on that rather than the HTML escaping problem. Steve
Am Fr., 11. Juni 2021 um 11:10 Uhr schrieb Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp>:
Thomas Güttler writes:
Am Fr., 11. Juni 2021 um 03:17 Uhr schrieb Stephan Hoyer < shoyer@gmail.com>:
Unevaluated f-strings is a nice way to think about this functionality.
But they're not "unevaluated" in a lot of important ways. A better term might be "pre-assembled". :-)
Another use-case that comes to mind is logging. The Google Python style guide says not to use f-strings, for example, because it wants to be able to collect the unexpanded pattern strings and not waste time rendering unlogged messages: https://google.github.io/styleguide/pyguide.html#3101-logging
Thank you Stephan for this feedback. I added Logging to the draft: https://github.com/guettli/peps/blob/master/pep-9999.rst#logging
If this does solve the logging problem, it would be a killer app (but that still might not be enough, given that PEP 501 couldn't clear the bar and logging was a leading application for that PEP). Perhaps you should focus on that rather than the HTML escaping problem.
Hi Stephen, I can't solve a problem which I don't know. Logging plays no big role in my development tasks any more. I use checks, metrics and sentry. In the past logging was important for me, but this changed. But I am sure that the proposal can be used to improve logging. Regards, Thomas
I think this idea is promising but instead of doing it by adding new syntax and a totally different object, why not attach a __templates__ dunder member to every string but only OPTIONALLY populate it when a string is formatted? For every regular string it would just be None:
"".__template__
But if you create a string using an f string, you can provide a directive, as part of the format specification mini language, to store it:
s = f"{foo!t}"
...or a directive at the front of the string to store all the f string arguments:
s = ft"{foo}"
This would save the values marked for storage as a tuple of Template arguments in the __templates__ member for later. It may also be desirable for the Template object to store the name foo and the position in the string the foo object was formatted. So:
foo = "spam" s = f"{foo!t}"
Now instead of being None, s.__template__ would store an object recording the name foo, what was passed to foo, and the position:
s.__templates___ ( Template (name="foo", value="spam", position=(0,4)), )
You could do the same with positional f string arguments, but using an integer as the name. An html or log function would later use, or discard, the template information as it sees fit. On Fri, Jun 11, 2021, 6:23 AM Thomas Güttler <info@thomas-guettler.de> wrote:
Am Fr., 11. Juni 2021 um 11:10 Uhr schrieb Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp>:
Thomas Güttler writes:
Am Fr., 11. Juni 2021 um 03:17 Uhr schrieb Stephan Hoyer < shoyer@gmail.com>:
Unevaluated f-strings is a nice way to think about this functionality.
But they're not "unevaluated" in a lot of important ways. A better term might be "pre-assembled". :-)
Another use-case that comes to mind is logging. The Google Python style guide says not to use f-strings, for example, because it wants to be able to collect the unexpanded pattern strings and not waste time rendering unlogged messages: https://google.github.io/styleguide/pyguide.html#3101-logging
Thank you Stephan for this feedback. I added Logging to the draft: https://github.com/guettli/peps/blob/master/pep-9999.rst#logging
If this does solve the logging problem, it would be a killer app (but that still might not be enough, given that PEP 501 couldn't clear the bar and logging was a leading application for that PEP). Perhaps you should focus on that rather than the HTML escaping problem.
Hi Stephen,
I can't solve a problem which I don't know. Logging plays no big role in my development tasks any more. I use checks, metrics and sentry. In the past logging was important for me, but this changed.
But I am sure that the proposal can be used to improve logging.
Regards, Thomas
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YEYP2P... Code of Conduct: http://python.org/psf/codeofconduct/
Am Fr., 11. Juni 2021 um 14:51 Uhr schrieb Ricky Teachey <ricky@teachey.org
:
I think this idea is promising but instead of doing it by adding new syntax and a totally different object, why not attach a __templates__ dunder member to every string but only OPTIONALLY populate it when a string is formatted?
I am very happy that you think this is promising. Changing the interface of every string in Python feels much too big for me. I don't dare to think about it. I have concerns that it could have negative impacts on parts which I am not aware of.
For every regular string it would just be None:
"".__template__
But if you create a string using an f string, you can provide a directive, as part of the format specification mini language, to store it:
s = f"{foo!t}"
...or a directive at the front of the string to store all the f string arguments:
s = ft"{foo}"
This would save the values marked for storage as a tuple of Template arguments in the __templates__ member for later. It may also be desirable for the Template object to store the name foo and the position in the string the foo object was formatted.
The "t" prefix makes sense. I would reduce it from ft'...' to t'...'. But it looks good. This is in the "alternatives": https://github.com/guettli/peps/blob/master/pep-9999.rst#alternative-ideas I am happy with backticks and/or t'...'. Backticks look a bit "cleaner" to my eyes, but I am fine with the "t" prefix, too.
On Fri, Jun 11, 2021 at 10:12 AM Thomas Güttler <info@thomas-guettler.de> wrote:
Am Fr., 11. Juni 2021 um 14:51 Uhr schrieb Ricky Teachey < ricky@teachey.org>:
I think this idea is promising but instead of doing it by adding new syntax and a totally different object, why not attach a __templates__ dunder member to every string but only OPTIONALLY populate it when a string is formatted?
I am very happy that you think this is promising.
Changing the interface of every string in Python feels much too big for me. I don't dare to think about it. I have concerns that it could have negative impacts on parts which I am not aware of.
I'm not a python internals expert, but i am imagining it wouldn't be a very significant performance hog for anyone not interested in the feature... seems like all that would be added is the equivalent of a __slot__ to the str class object, and a reference to None inside it. so the memory footprint of every string object would not have to grow. the class just grows a new dunder. the templated string objects themselves would have to have a bigger memory footprint of course. but the user/developer is choosing to do that by using templated strings. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On Thu, Jun 10, 2021 at 2:58 PM Ricky Teachey <ricky@teachey.org> wrote:
Something I don't understand is whether there is anything about this proposed feature that can't be accomplished with a simple function...
.... And use it like this:
templify("Here, have some {foo}.") TemplateLiteral(template = " Here, have some {foo}.", tokens = (("Here, have some ", True), ("spam & eggs", False)))
What is it about this task that requires it being handled at the language level...?
AIUI, that is hard to do without language support, because the function templify can't easily access the value of 'foo' because the binding only exists in the caller's scope. So for a simple function to be used, you'd have to pass in the placeholder values explicitly as separate arguments, which somewhat defeats the point. Steve
On Thu, Jun 10, 2021 at 10:12 AM Stestagg <stestagg@gmail.com> wrote:
On Thu, Jun 10, 2021 at 2:58 PM Ricky Teachey <ricky@teachey.org> wrote:
Something I don't understand is whether there is anything about this proposed feature that can't be accomplished with a simple function...
.... And use it like this:
templify("Here, have some {foo}.") TemplateLiteral(template = " Here, have some {foo}.", tokens = (("Here, have some ", True), ("spam & eggs", False)))
What is it about this task that requires it being handled at the language level...?
AIUI, that is hard to do without language support, because the function templify can't easily access the value of 'foo' because the binding only exists in the caller's scope.
So for a simple function to be used, you'd have to pass in the placeholder values explicitly as separate arguments, which somewhat defeats the point.
Steve
Got it now. Should have been able to figure that out on my own thanks for explaining! Makes me wonder if stepping back and thinking, instead, about a way to automatically (optionally?) put the current calling context dictionary(ies) somewhere that a called function can get at it/them, so it can do what it wants with the context. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On 10/06/2021 15:46, Ricky Teachey wrote:
Makes me wonder if stepping back and thinking, instead, about a way to automatically (optionally?) put the current calling context dictionary(ies) somewhere that a called function can get at it/them, so it can do what it wants with the context.
I think that would violate the expectation that a function call means the same thing in different contexts, hence f-strings work by supplying values at the "call site" not variables. I know you will find exceptions (like import context), but they are not as all-consuming as snapping the entire namespace. If you really want to, there's sys._getframe. Jeff
Supporting JavaScript-style template literals, specifically tagged templates, could be generally useful. If we look at https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_l..., it's a really simple idea: The tag function is called with the following arguments: the strings from the template (the raw string is also available), plus any evaluated expressions referenced by that string. This idea can be equivalently expressed in a number of ways in Python. So this might be: ``` from some_html_templating_library import html_template_function as h num = 42 constraint = "A number > 40" html_text = h"<b>{num}</b> complies with the constraint <i>{constraint}</i>" ``` which results in `html_text` being set to `h(template, 42, "A number > 40")`, where `template` is some object that contains at the very least the original raw string as well as the non-expression strings. Notably such tagged template literals - so just like JavaScript tagged templates - are not restricted to HTML or even to returning strings, but instead could be used for a variety of additional purposes, that I won't work out this moment: * Logging strings which don't have to format their expressions - so similar to current logging when not using f-strings, but with the easy interface of f-strings, including desired formatting of expressions in simple logger scenarios * Internationalization of strings * SQL queries with sanitization of expressions (to avoid "Bobby Tables" injection) * Object literals - something like D"10.2" instead of the somewhat longer D("10.2"), assuming for the latter `from decimal import Decimal as D` (this is a pretty weak argument on the face of it, but I can see the possibility of being more restrictive in the use of monkey patching namespaces with such literals to avoid unnecessary function call overhead...) * Code (or AST) construction, following the quasiquote idea from Lisp * Latex and other minilanguages (is Latex properly a minilanguage? ;) ) For some of these use cases, ideally we don't use `sys._getframe` if this were to be implemented - assuming f-string style expressions, it's perfectly feasible to use a lambda to capture any variables as desired using lexical scoping instead of the dynamic scoping of `sys._getframe` AND we need deferred evaluation of the expression. Some shorthand for the lambda keyword could be nice however. I should mention that when I looked at some similar ideas last summer ( https://github.com/jimbaker/fl-string-pep/, useful for some experiments on lexical scoping), in part with Guido, I was thinking that an implicit lambda should be used for all such expressions in tagged template literals. But my feeling now is that this would not be so Pythonic - we explicitly mark such usages now, and we should continue to do so. Just need to find the right character sequence for such shorthand! On Sat, Jun 12, 2021 at 5:51 AM Jeff Allen <ja.py@farowl.co.uk> wrote:
On 10/06/2021 15:46, Ricky Teachey wrote:
Makes me wonder if stepping back and thinking, instead, about a way to automatically (optionally?) put the current calling context dictionary(ies) somewhere that a called function can get at it/them, so it can do what it wants with the context.
I think that would violate the expectation that a function call means the same thing in different contexts, hence f-strings work by supplying values at the "call site" not variables. I know you will find exceptions (like import context), but they are not as all-consuming as snapping the entire namespace. If you really want to, there's sys._getframe.
Jeff _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DUEB3H... Code of Conduct: http://python.org/psf/codeofconduct/
On Mon, Jun 14, 2021 at 8:58 AM Jim Baker <jim.baker@python.org> wrote:
* Logging strings which don't have to format their expressions - so similar to current logging when not using f-strings, but with the easy interface of f-strings, including desired formatting of expressions in simple logger scenarios
The only benefit would be the string formatting; the expressions have to be evaluated still. Is that worth it?
* Internationalization of strings
Of dubious value, since the strings have to be externalized. MAYBE useful but would need something complicated where the original form is a code construct, but the i18n library can get the source code and use it as a lookup key to find the translated form. Seems pretty messy.
* SQL queries with sanitization of expressions (to avoid "Bobby Tables" injection)
You shouldn't be sanitizing your expressions, you should be parameterizing your query. That doesn't mean this is of *no* value, but it does make it look a lot less like a string literal.
* Object literals - something like D"10.2" instead of the somewhat longer D("10.2"), assuming for the latter `from decimal import Decimal as D` (this is a pretty weak argument on the face of it, but I can see the possibility of being more restrictive in the use of monkey patching namespaces with such literals to avoid unnecessary function call overhead...)
Of value only if it gets an extremely compact syntax. Other use-cases could be better served by a syntax like html(i".....") but this one only works if it's that kind of notation.
* Code (or AST) construction, following the quasiquote idea from Lisp
Not sure how this would work, but given that I don't use Lisp enough to be familiar with the use-cases, I won't argue this one.
* Latex and other minilanguages (is Latex properly a minilanguage? ;) )
Explain? I'd love to have some sort of generic interpolation construct, but the use-cases all seem really weak... ChrisA
On 6/13/2021 7:24 PM, Chris Angelico wrote:
On Mon, Jun 14, 2021 at 8:58 AM Jim Baker <jim.baker@python.org> wrote:
* Logging strings which don't have to format their expressions - so similar to current logging when not using f-strings, but with the easy interface of f-strings, including desired formatting of expressions in simple logger scenarios
The only benefit would be the string formatting; the expressions have to be evaluated still. Is that worth it?
Personally, I always just log with f-strings and skip the delayed formatting step. I suspect (but have run no benchmarks) that it's faster than delayed formatting, since you skip the whole "parse the string and look for %'s" step. In my case, most of my logging calls actually get logged, anyway.
* Internationalization of strings Of dubious value, since the strings have to be externalized. MAYBE useful but would need something complicated where the original form is a code construct, but the i18n library can get the source code and use it as a lookup key to find the translated form. Seems pretty messy.
I don't think this is of dubious value. It might require some tooling support.
* SQL queries with sanitization of expressions (to avoid "Bobby Tables" injection) You shouldn't be sanitizing your expressions, you should be parameterizing your query. That doesn't mean this is of *no* value, but it does make it look a lot less like a string literal.
Agreed.
* Object literals - something like D"10.2" instead of the somewhat longer D("10.2"), assuming for the latter `from decimal import Decimal as D` (this is a pretty weak argument on the face of it, but I can see the possibility of being more restrictive in the use of monkey patching namespaces with such literals to avoid unnecessary function call overhead...)
Of value only if it gets an extremely compact syntax. Other use-cases could be better served by a syntax like html(i".....") but this one only works if it's that kind of notation.
* Code (or AST) construction, following the quasiquote idea from Lisp Not sure how this would work, but given that I don't use Lisp enough to be familiar with the use-cases, I won't argue this one.
* Latex and other minilanguages (is Latex properly a minilanguage? ;) ) Explain?
I'd love to have some sort of generic interpolation construct, but the use-cases all seem really weak...
Ditto. I do plan to turn my attention to it, however. Eric
Am Mo., 14. Juni 2021 um 01:25 Uhr schrieb Chris Angelico <rosuav@gmail.com
:
I'd love to have some sort of generic interpolation construct, but the use-cases all seem really weak...
It depends on your daily work. If you create many small methods returning small HTML fragments, then it really helps if you can avoid some extra typing. But maybe it is not about typing. Maybe it is about reading. Passing variables via `foo=foo, bar=bar` contains no value. This means your eyes need to parse information which has little value. I would like to avoid that. f-strings are great. Unfortunately they do too much. Regards, Thomas
On Thu, Jun 10, 2021 at 7:31 AM Thomas Güttler <info@thomas-guettler.de> wrote:
Thank you Guido, Chris, Matt and Richard for your feedback to my last email.
Here is an updated version called "Template Literals".
I much prefer: Alternative Ideas Instead of backticks for example t'...' could be used. instead of a new syntax being introduced. It would be great to consider a variation of this, where `t` is a normal function in scope that takes the proposed class attributes as arguments, and simply returns a new object. Something similar to: def html(raw_str, parts): escaped = [] for part, is_substitute in parts: if is_substitute: escaped.append(html_escape(part)) else: escaped.append(part) return MarkupSafe("".join(escaped)) With the idea that: name = 'Bob <the rock>' fragment = html'Hello: <b>{name}</b>' results in fragment being: MarkupSafe("Hello: <b>Bob <the rock></b>") ---- The equivalent for SQL could return a `Statement` object with appropriate placeholders used. If you don't like this approach, it would be nice to include it in the alternate/rejected sections. Thanks Steve
On Thu, Jun 10, 2021 at 8:34 AM Thomas Güttler <info@thomas-guettler.de> wrote:
This solution has two drawbacks:
1. It is too verbose. Typing "conditional_escape(...)" again and again is cumbersome.
from django import conditional_espace as esc
f''' <h1>Hi {esc(name)}</h1> Your messages: {esc(messages)} '''
1. If a conditional_escape() gets forgotten Cross-site scripting attacks could be possible, since malicious users could inject HTML.
This is specific to Django and other frameworks out there which accept
anything as user input by default, that's an anti-pattern which OWASP recommends against because obviously it opens a wide range of attack vectors, absolutely no security audit would ever validate the default validation of a CharField or a TextField. Another problem I see with this proposal is how do you actually use safe HTML in variables? msgs = [f'<li>{msg}</li>' for msg in messages] f''' <h1>Hi {name}</h1> Your messages: <ul>{msgs}</ul> ''' Will output: <h1>Hi Your name</h1> Your messages: <ul><li>Your message</li></ul> Instead of what we would want in this situation: <h1>Hi Your name</h1> Your messages: <ul><li>Your message</li></ul> Otherwise good idea, it's an issue we have, even though the first immediate fix needed is Django's default input validation which is just openbar.
My bad, I had missed nesting template literals support in the PEP, it makes it interesting then!
1. It is too verbose. Typing "conditional_escape(...)" again and again is cumbersome.
from django import conditional_espace as esc f'<h1>Hi {esc(name)}</h1>'
It's too bad that ␛ U+251B isn't a valid Python identifier. Then we could do it in one character :-) But as I keep saying, confusing new syntax to save a couple characters in calling an existing function is not a good thing. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
Am Mo., 14. Juni 2021 um 11:12 Uhr schrieb J. Pic <jpic@yourlabs.org>:
On Thu, Jun 10, 2021 at 8:34 AM Thomas Güttler <info@thomas-guettler.de> wrote:
This solution has two drawbacks:
1. It is too verbose. Typing "conditional_escape(...)" again and again is cumbersome.
from django import conditional_espace as esc
f''' <h1>Hi {esc(name)}</h1> Your messages: {esc(messages)} '''
1. If a conditional_escape() gets forgotten Cross-site scripting attacks could be possible, since malicious users could inject HTML.
This is specific to Django and other frameworks out there which accept
anything as user input by default, that's an anti-pattern which OWASP recommends against because obviously it opens a wide range of attack vectors, absolutely no security audit would ever validate the default validation of a CharField or a TextField.
You are right. Validating user input is an important topic. But it is a different topic, which should be discussed at a different time.
Another problem I see with this proposal is how do you actually use safe HTML in variables?
msgs = [f'<li>{msg}</li>' for msg in messages] f''' <h1>Hi {name}</h1> Your messages: <ul>{msgs}</ul> '''
Will output:
<h1>Hi Your name</h1> Your messages: <ul><li>Your message</li></ul>
Instead of what we would want in this situation:
<h1>Hi Your name</h1> Your messages: <ul><li>Your message</li></ul>
Otherwise good idea, it's an issue we have, even though the first immediate fix needed is Django's default input validation which is just openbar.
Thank you for your feedback James. The "magic" is done in conditional_escape(): https://github.com/django/django/blob/824981b2dc61a76a59d0e470bed6e61626a44c... I updated the PEP so that it contains a hyperlink to the github repo. Regards, Thomas
participants (19)
-
Barry Scott -
Chris Angelico -
Christopher Barker -
David Mertz -
Eric V. Smith -
Felipe Rodrigues -
Greg Ewing -
Guido van Rossum -
J. Pic -
Jeff Allen -
Jim Baker -
Jonathan Goble -
Nick Coghlan -
Ricky Teachey -
Rob Cliffe -
Stephan Hoyer -
Stephen J. Turnbull -
Stestagg -
Thomas Güttler