setup.cfg - to interpolate or not to interpolate
In Setuptools #889 (https://github.com/pypa/setuptools/issues/889) and Python 20754 (http://bugs.python.org/issue20754), setuptools and distutils are proposing a backward-incompatible change - to remove ConfigParser interpolation when parsing setup.cfg (and maybe other config) files. I’d previously thought that it was an unintended regression from Python 2 to introduce interpolation, but on deeper investigation, interpolation has been present all along. While it’s conceivable that no projects are relying on the interpolation behavior, I’d be surprised if none have at least worked around the interpolation behavior by escaping percent signs. So I’m left with the sinking feeling there’s no easy transition from the status quo to the desired state (no interpolation). I’d like feedback, in either ticket, about whether this change is worth the trouble and if so if there are any ways to mitigate the risks of introducing such a change. If someone can assist on this effort, that would also be greatly appreciated. I’ll likely abandon the effort and close the tickets as won’t fix unless we can identify someone to champion the effort.
On 28 July 2017 at 01:32, Jason R. Coombs <jaraco@jaraco.com> wrote:
I’d like feedback, in either ticket, about whether this change is worth the trouble and if so if there are any ways to mitigate the risks of introducing such a change.
The one way I can see of doing this with a graceful transition period would be to implement it like this: 1. Switch to raw parsing with no interpolation as the default 2. Check the result for any field values containing either "%(<text>)s" or "%%" 3. If you find any, print a suitable deprecation warning and parse it again with interpolation enabled (I'm assuming the current interpolation support is the mod-formatting style one supported by default in Py2, rather than the braces based variant now available in Py3) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 28 July 2017 at 13:30, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 28 July 2017 at 01:32, Jason R. Coombs <jaraco@jaraco.com> wrote:
I’d like feedback, in either ticket, about whether this change is worth the trouble and if so if there are any ways to mitigate the risks of introducing such a change.
The one way I can see of doing this with a graceful transition period would be to implement it like this:
1. Switch to raw parsing with no interpolation as the default 2. Check the result for any field values containing either "%(<text>)s" or "%%" 3. If you find any, print a suitable deprecation warning and parse it again with interpolation enabled
Clarifying to add: I think this is a worthwhile change, as it helps to ensure that the static config files actually *are* static, and hence less dependent on being read specifically with Python's configparser library. That's a boon for interoperability, even if it means that folks that genuinely want interpolation will need to switch to a "setup.cfg.in" style approach where they use the templating tool of their choice to read in the input file, substitute values, and then write out a completely static setup.cfg file. Cheers, NIck. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 28 July 2017 at 15:34, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 28 July 2017 at 13:30, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 28 July 2017 at 01:32, Jason R. Coombs <jaraco@jaraco.com> wrote:
I’d like feedback, in either ticket, about whether this change is worth the trouble and if so if there are any ways to mitigate the risks of introducing such a change.
The one way I can see of doing this with a graceful transition period would be to implement it like this:
1. Switch to raw parsing with no interpolation as the default 2. Check the result for any field values containing either "%(<text>)s" or "%%" 3. If you find any, print a suitable deprecation warning and parse it again with interpolation enabled
Clarifying to add: I think this is a worthwhile change, as it helps to ensure that the static config files actually *are* static, and hence less dependent on being read specifically with Python's configparser library. That's a boon for interoperability, even if it means that folks that genuinely want interpolation will need to switch to a "setup.cfg.in" style approach where they use the templating tool of their choice to read in the input file, substitute values, and then write out a completely static setup.cfg file.
Yeah, I'm going to disagree here. While its true that using interpolation binds the config file semantic interpretation to Python, its *already* bound that way due to the idiosyncratic continuations style and nested sections. Doing a .in -> actual-foo is a great pattern for build systems building expensive artifacts with complex dependency chains. Not problems have, and copying their solutions without the problem is just unnecessary, and unhelpful, complexity. +1 from me for retaining interpolation. -Rob
On 28 July 2017 at 15:39, Robert Collins <robertc@robertcollins.net> wrote:
On 28 July 2017 at 15:34, Nick Coghlan <ncoghlan@gmail.com> wrote:
Clarifying to add: I think this is a worthwhile change, as it helps to ensure that the static config files actually *are* static, and hence less dependent on being read specifically with Python's configparser library. That's a boon for interoperability, even if it means that folks that genuinely want interpolation will need to switch to a "setup.cfg.in" style approach where they use the templating tool of their choice to read in the input file, substitute values, and then write out a completely static setup.cfg file.
Yeah, I'm going to disagree here. While its true that using interpolation binds the config file semantic interpretation to Python, its *already* bound that way due to the idiosyncratic continuations style and nested sections.
Doing a .in -> actual-foo is a great pattern for build systems building expensive artifacts with complex dependency chains. Not problems have, and copying their solutions without the problem is just unnecessary, and unhelpful, complexity.
+1 from me for retaining interpolation.
Fair point, and reading the original bug report in https://bugs.python.org/issue20754, it seems the main problems were that: - it wasn't documented that interpolation is enabled when parsing setup.cfg - the Python 2.7 implementation currently uses ConfigParser rather than SafeConfigParser, so escaping with `%%` didn't work in 2.x That would make the following the lowest impact resolution that still fixes the original reported problem: - update the ConfigParser (3.x) and SafeConfigParser (2.x) docs to explicitly say that "%%" works as an escape - update setuptools and distutils to use SafeConfigParser in 2.7 - update the setuptools and distutils docs to say that interpolation is enabled when reading setup.cfg and "%%" works as an escape (except on older versions of Python 2.x with either distutils or an older version of setuptools). So I'll retract my initial response as being ill-informed (since I didn't click through and read the original bug report). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan <ncoghlan@gmail.com> writes:
That would make the following the lowest impact resolution that still fixes the original reported problem:
- update the ConfigParser (3.x) and SafeConfigParser (2.x) docs to explicitly say that "%%" works as an escape - update setuptools and distutils to use SafeConfigParser in 2.7 - update the setuptools and distutils docs to say that interpolation is enabled when reading setup.cfg and "%%" works as an escape (except on older versions of Python 2.x with either distutils or an older version of setuptools).
So I'll retract my initial response as being ill-informed (since I didn't click through and read the original bug report).
I'm glad to see this result, thank you for taking all that into account. -- \ “Repent before ring 31 moves.” —Damon Anton Permezel, hanoi.el | `\ _Tower of Hanoi_ source code, Unix doomsday version, 1985 | _o__) | Ben Finney
participants (4)
-
Ben Finney -
Jason R. Coombs -
Nick Coghlan -
Robert Collins