ConfigParser - parsing options with no value
Hi all, I am currently using the ConfigParser module to parse MySQL configuration files. Just plain config files, nothing fancy except... options without values. There is a number of options here that does not require a value, they are basically just turning on a feature. They are also common in the standard template files for the server. Options that are for mysqld can have a value even though it is not required and the option file parser will not complain, but for some of the client tools, values may not be given or there will be a error. Looking at the tests of ConfigParser, I see that it is a deliberate design decision to not allow options without values (or I am misunderstanding something). Why? Being able to parse these files without having to write a new module would be very valuable. Best wishes, Mats Kindahl -- Mats Kindahl Senior Software Engineer Database Technology Group Sun Microsystems
On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> wrote:
Hi all,
I am currently using the ConfigParser module to parse MySQL configuration files. Just plain config files, nothing fancy except... options without values.
There is a number of options here that does not require a value, they are basically just turning on a feature. They are also common in the standard template files for the server. Options that are for mysqld can have a value even though it is not required and the option file parser will not complain, but for some of the client tools, values may not be given or there will be a error.
Looking at the tests of ConfigParser, I see that it is a deliberate design decision to not allow options without values (or I am misunderstanding something). Why?
Who knows. =) Module is old enough it's quite possible no one remembers why. Maybe the original creator thought it would help catch errors when people accidentally left off a value. -Brett
Brett Cannon wrote:
On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> wrote:
Hi all,
I am currently using the ConfigParser module to parse MySQL configuration files. Just plain config files, nothing fancy except... options without values.
There is a number of options here that does not require a value, they are basically just turning on a feature. They are also common in the standard template files for the server. Options that are for mysqld can have a value even though it is not required and the option file parser will not complain, but for some of the client tools, values may not be given or there will be a error.
Looking at the tests of ConfigParser, I see that it is a deliberate design decision to not allow options without values (or I am misunderstanding something). Why?
Who knows. =) Module is old enough it's quite possible no one remembers why. Maybe the original creator thought it would help catch errors when people accidentally left off a value.
Hmmm.... If one should allow value-less options, I see two ways to handle that: - For options lacking values, set the value to the empty string, or - For options lacking values, set the value to None. I have a patch for the first case, but I really think that the second approach is better. That would make it possible to distinguish between options that were given the empty string value and that were not given a value at all. Where should I post such patches and what is people opinion of these to alternatives? Another consideration is how to allow this extension. It would be possible to add flags to allow new and old behaviour, but I don't know how much value people attribute to be able to get an exception if there is no value for the option. I am also thinking that some option files have extensions to be able to include files (MySQL option parsing package has), so it might be a good idea to add callbacks to handle unrecognized lines, allowing the callback to inject text into the parsing stream to, for example, support include files or even a macro language. Just my few cents, Mats Kindahl -- Mats Kindahl Senior Software Engineer Database Technology Group Sun Microsystems
On Thu, 17 Sep 2009 at 22:28, Mats Kindahl wrote:
Where should I post such patches and what is people opinion of these to alternatives?
The bug tracker (bugs.python.org).
Another consideration is how to allow this extension. It would be possible to add flags to allow new and old behaviour, but I don't know how much value people attribute to be able to get an exception if there is no value for the option.
Python's philosophy is to remain backward compatible, so this new behavior should be controlled by an option of some sort whose default value is the old behavior.
I am also thinking that some option files have extensions to be able to include files (MySQL option parsing package has), so it might be a good idea to add callbacks to handle unrecognized lines, allowing the callback to inject text into the parsing stream to, for example, support include files or even a macro language.
That should be a separate feature request/tracker issue/patch. --David
R. David Murray wrote:
On Thu, 17 Sep 2009 at 22:28, Mats Kindahl wrote:
Where should I post such patches and what is people opinion of these to alternatives?
The bug tracker (bugs.python.org).
Another consideration is how to allow this extension. It would be possible to add flags to allow new and old behaviour, but I don't know how much value people attribute to be able to get an exception if there is no value for the option.
Python's philosophy is to remain backward compatible, so this new behavior should be controlled by an option of some sort whose default value is the old behavior.
OK. This suggestion for improvement is now registered as Issue 7005. It contain changes to ConfigParser.py as well as an extended test case. I elected to add a parameter allow_no_value to __init__ and use the default False. Also, options without values are registered with the value None, so that they can be distinguished from options that have the empty string as value.
I am also thinking that some option files have extensions to be able to include files (MySQL option parsing package has), so it might be a good idea to add callbacks to handle unrecognized lines, allowing the callback to inject text into the parsing stream to, for example, support include files or even a macro language.
That should be a separate feature request/tracker issue/patch.
It's on my todo list. Best wishes, Mats Kindahl
--David
-- Mats Kindahl Senior Software Engineer Database Technology Group Sun Microsystems
Brett Cannon schrieb:
On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> wrote:
Hi all,
I am currently using the ConfigParser module to parse MySQL configuration files. Just plain config files, nothing fancy except... options without values.
There is a number of options here that does not require a value, they are basically just turning on a feature. They are also common in the standard template files for the server. Options that are for mysqld can have a value even though it is not required and the option file parser will not complain, but for some of the client tools, values may not be given or there will be a error.
Looking at the tests of ConfigParser, I see that it is a deliberate design decision to not allow options without values (or I am misunderstanding something). Why?
Who knows. =) Module is old enough it's quite possible no one remembers why.
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
Georg Brandl wrote:
Brett Cannon schrieb:
On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> wrote:
Hi all,
I am currently using the ConfigParser module to parse MySQL configuration files. Just plain config files, nothing fancy except... options without values.
There is a number of options here that does not require a value, they are basically just turning on a feature. They are also common in the standard template files for the server. Options that are for mysqld can have a value even though it is not required and the option file parser will not complain, but for some of the client tools, values may not be given or there will be a error.
Looking at the tests of ConfigParser, I see that it is a deliberate design decision to not allow options without values (or I am misunderstanding something). Why?
Who knows. =) Module is old enough it's quite possible no one remembers why.
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
Georg
I happen to know a good one we could add. ;-) Michael -- http://www.ironpythoninaction.com/
On Sep 17, 2009, at 7:34 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Georg Brandl wrote:
Brett Cannon schrieb:
On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> wrote:
Hi all,
I am currently using the ConfigParser module to parse MySQL configuration files. Just plain config files, nothing fancy except... options without values.
There is a number of options here that does not require a value, they are basically just turning on a feature. They are also common in the standard template files for the server. Options that are for mysqld can have a value even though it is not required and the option file parser will not complain, but for some of the client tools, values may not be given or there will be a error.
Looking at the tests of ConfigParser, I see that it is a deliberate design decision to not allow options without values (or I am misunderstanding something). Why?
Who knows. =) Module is old enough it's quite possible no one remembers why.
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
Georg
I happen to know a good one we could add. ;-)
Michael
+1 I eagerly await a PEP
Sorry to top-post, sent from phone-like-device. Actually Guido has shot down the inclusion of ConfigObj on the past, so it doesn't seem worth a PEP. He doesn't like that sections inherit from dict (although they are functionally key-value mappings) as it gives them an over-wide API. This is fair enough. He also fundamentally disliked the idea of nested sections, doesn't see the need and suggested people should be using XML for this kind of use case! It is an often requested feature, so this I thought was odd. Anyway, so be it. Michael -- http://www.ironpythoninaction.com On 18 Sep 2009, at 00:59, Jesse Noller <jnoller@gmail.com> wrote:
On Sep 17, 2009, at 7:34 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Georg Brandl wrote:
Brett Cannon schrieb:
On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> wrote:
Hi all,
I am currently using the ConfigParser module to parse MySQL configuration files. Just plain config files, nothing fancy except... options without values.
There is a number of options here that does not require a value, they are basically just turning on a feature. They are also common in the standard template files for the server. Options that are for mysqld can have a value even though it is not required and the option file parser will not complain, but for some of the client tools, values may not be given or there will be a error.
Looking at the tests of ConfigParser, I see that it is a deliberate design decision to not allow options without values (or I am misunderstanding something). Why?
Who knows. =) Module is old enough it's quite possible no one remembers why.
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
Georg
I happen to know a good one we could add. ;-)
Michael
+1 I eagerly await a PEP
I was being ever slightly tongue in cheek, but I for one would like to see bits of configobj get into configparser, having used the latter several times until I discovered the magical world of json/YAML On Sep 17, 2009, at 8:09 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Sorry to top-post, sent from phone-like-device.
Actually Guido has shot down the inclusion of ConfigObj on the past, so it doesn't seem worth a PEP.
He doesn't like that sections inherit from dict (although they are functionally key-value mappings) as it gives them an over-wide API. This is fair enough.
He also fundamentally disliked the idea of nested sections, doesn't see the need and suggested people should be using XML for this kind of use case! It is an often requested feature, so this I thought was odd. Anyway, so be it.
Michael
-- http://www.ironpythoninaction.com
On 18 Sep 2009, at 00:59, Jesse Noller <jnoller@gmail.com> wrote:
On Sep 17, 2009, at 7:34 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Georg Brandl wrote:
Brett Cannon schrieb:
On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> wrote:
Hi all,
I am currently using the ConfigParser module to parse MySQL configuration files. Just plain config files, nothing fancy except... options without values.
There is a number of options here that does not require a value, they are basically just turning on a feature. They are also common in the standard template files for the server. Options that are for mysqld can have a value even though it is not required and the option file parser will not complain, but for some of the client tools, values may not be given or there will be a error.
Looking at the tests of ConfigParser, I see that it is a deliberate design decision to not allow options without values (or I am misunderstanding something). Why?
Who knows. =) Module is old enough it's quite possible no one remembers why.
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
Georg
I happen to know a good one we could add. ;-)
Michael
+1 I eagerly await a PEP
Yeah - I just never miss an opportunity to promote ConfigObj. :-) The nice thing about it is that the basic API is *so* much nicer than ConfigParser. Parts of that would be easy to get on and other parts would require major surgery and result on a hybrid there's-now-two- ways-to-do-it API. ConfigObj is bizarrely widely used so I'm happy with it as a third party module. Fact of the day though, the first version (about five years ago now) was only written because I didn't know ConfigParser existed... Michael -- http://www.ironpythoninaction.com On 18 Sep 2009, at 01:20, Jesse Noller <jnoller@gmail.com> wrote:
I was being ever slightly tongue in cheek, but I for one would like to see bits of configobj get into configparser, having used the latter several times until I discovered the magical world of json/YAML
On Sep 17, 2009, at 8:09 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Sorry to top-post, sent from phone-like-device.
Actually Guido has shot down the inclusion of ConfigObj on the past, so it doesn't seem worth a PEP.
He doesn't like that sections inherit from dict (although they are functionally key-value mappings) as it gives them an over-wide API. This is fair enough.
He also fundamentally disliked the idea of nested sections, doesn't see the need and suggested people should be using XML for this kind of use case! It is an often requested feature, so this I thought was odd. Anyway, so be it.
Michael
-- http://www.ironpythoninaction.com
On 18 Sep 2009, at 00:59, Jesse Noller <jnoller@gmail.com> wrote:
On Sep 17, 2009, at 7:34 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Georg Brandl wrote:
Brett Cannon schrieb:
On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> wrote:
> Hi all, > > I am currently using the ConfigParser module to parse MySQL > configuration files. > Just plain config files, nothing fancy except... options > without values. > > There is a number of options here that does not require a > value, they are > basically just turning on a feature. They are also common in > the standard > template files for the server. Options that are for mysqld can > have a value even > though it is not required and the option file parser will not > complain, but for > some of the client tools, values may not be given or there > will be a error. > > Looking at the tests of ConfigParser, I see that it is a > deliberate design > decision to not allow options without values (or I am > misunderstanding > something). Why? > > Who knows. =) Module is old enough it's quite possible no one remembers why.
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
Georg
I happen to know a good one we could add. ;-)
Michael
+1 I eagerly await a PEP
And YAML can suck it... -- http://www.ironpythoninaction.com On 18 Sep 2009, at 01:20, Jesse Noller <jnoller@gmail.com> wrote:
I was being ever slightly tongue in cheek, but I for one would like to see bits of configobj get into configparser, having used the latter several times until I discovered the magical world of json/YAML
On Sep 17, 2009, at 8:09 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Sorry to top-post, sent from phone-like-device.
Actually Guido has shot down the inclusion of ConfigObj on the past, so it doesn't seem worth a PEP.
He doesn't like that sections inherit from dict (although they are functionally key-value mappings) as it gives them an over-wide API. This is fair enough.
He also fundamentally disliked the idea of nested sections, doesn't see the need and suggested people should be using XML for this kind of use case! It is an often requested feature, so this I thought was odd. Anyway, so be it.
Michael
-- http://www.ironpythoninaction.com
On 18 Sep 2009, at 00:59, Jesse Noller <jnoller@gmail.com> wrote:
On Sep 17, 2009, at 7:34 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Georg Brandl wrote:
Brett Cannon schrieb:
On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> wrote:
> Hi all, > > I am currently using the ConfigParser module to parse MySQL > configuration files. > Just plain config files, nothing fancy except... options > without values. > > There is a number of options here that does not require a > value, they are > basically just turning on a feature. They are also common in > the standard > template files for the server. Options that are for mysqld can > have a value even > though it is not required and the option file parser will not > complain, but for > some of the client tools, values may not be given or there > will be a error. > > Looking at the tests of ConfigParser, I see that it is a > deliberate design > decision to not allow options without values (or I am > misunderstanding > something). Why? > > Who knows. =) Module is old enough it's quite possible no one remembers why.
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
Georg
I happen to know a good one we could add. ;-)
Michael
+1 I eagerly await a PEP
Go back to your Visual studio and XML there beard boy! On Sep 17, 2009, at 8:30 PM, Michael Foord <michael@voidspace.org.uk> wrote:
And YAML can suck it...
-- http://www.ironpythoninaction.com
On 18 Sep 2009, at 01:20, Jesse Noller <jnoller@gmail.com> wrote:
I was being ever slightly tongue in cheek, but I for one would like to see bits of configobj get into configparser, having used the latter several times until I discovered the magical world of json/ YAML
On Sep 17, 2009, at 8:09 PM, Michael Foord <michael@voidspace.org.uk> wrote:
Sorry to top-post, sent from phone-like-device.
Actually Guido has shot down the inclusion of ConfigObj on the past, so it doesn't seem worth a PEP.
He doesn't like that sections inherit from dict (although they are functionally key-value mappings) as it gives them an over-wide API. This is fair enough.
He also fundamentally disliked the idea of nested sections, doesn't see the need and suggested people should be using XML for this kind of use case! It is an often requested feature, so this I thought was odd. Anyway, so be it.
Michael
-- http://www.ironpythoninaction.com
On 18 Sep 2009, at 00:59, Jesse Noller <jnoller@gmail.com> wrote:
On Sep 17, 2009, at 7:34 PM, Michael Foord <michael@voidspace.org.uk
wrote:
Georg Brandl wrote:
Brett Cannon schrieb:
> On Thu, Sep 17, 2009 at 09:53, Mats Kindahl <mats@sun.com> > wrote: > >> Hi all, >> >> I am currently using the ConfigParser module to parse MySQL >> configuration files. >> Just plain config files, nothing fancy except... options >> without values. >> >> There is a number of options here that does not require a >> value, they are >> basically just turning on a feature. They are also common in >> the standard >> template files for the server. Options that are for mysqld >> can have a value even >> though it is not required and the option file parser will not >> complain, but for >> some of the client tools, values may not be given or there >> will be a error. >> >> Looking at the tests of ConfigParser, I see that it is a >> deliberate design >> decision to not allow options without values (or I am >> misunderstanding >> something). Why? >> >> > Who knows. =) Module is old enough it's quite possible no one > remembers why. >
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
Georg
I happen to know a good one we could add. ;-)
Michael
+1 I eagerly await a PEP
Le jeudi 17 septembre 2009 à 23:58 +0200, Georg Brandl a écrit :
Who knows. =) Module is old enough it's quite possible no one remembers why.
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
If so, can it have a .network attribute?
Adding a network attribute is easy. Plus I should have added that the core parser in ConfigObj (using and abusing regexes) is pretty hairy and I try to avoid touching it. :-) The rest of the code is basically fine (working on a new release now ~25% perf improvement thanks to Christian Heimes) although the module is waay too long. Doesn't take much maintenance though. Will add the network attribute as an undocumented new feature in honour of Antoine... Michael -- http://www.ironpythoninaction.com On 18 Sep 2009, at 01:11, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le jeudi 17 septembre 2009 à 23:58 +0200, Georg Brandl a écrit :
Who knows. =) Module is old enough it's quite possible no one remembers why.
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
If so, can it have a .network attribute?
_______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
Georg Brandl <g.brandl@...> writes:
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
Way back in the mists of time, a ConfigParserShootout was mooted and a wiki page dedicated to looking at alternative implementations - a whole bunch of candidates were nominated (disclosure: including one that I wrote, but also Michael's ConfigObj) but then it never went anywhere. Here's the link to the Wiki page: http://wiki.python.org/moin/ConfigParserShootout Perhaps we should have a vote to select the one or two to include ;-) Regards, Vinay Sajip
Vinay Sajip wrote:
Georg Brandl <g.brandl@...> writes:
Good point. There's also an awful shortage of config parsers in the standard library; we should add one or two.
Way back in the mists of time, a ConfigParserShootout was mooted and a wiki page dedicated to looking at alternative implementations - a whole bunch of candidates were nominated (disclosure: including one that I wrote, but also Michael's ConfigObj) but then it never went anywhere. Here's the link to the Wiki page:
http://wiki.python.org/moin/ConfigParserShootout
Perhaps we should have a vote to select the one or two to include ;-)
It did go somewhere - it was discussed on Python-dev and Guido didn't see the need to replace ConfigParser (and specifically didn't see the need for nested sections that was at least in part the motivation for the shootout). Michael
Regards,
Vinay Sajip
_______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog
Michael Foord <michael@...> writes:
It did go somewhere - it was discussed on Python-dev and Guido didn't see the need to replace ConfigParser (and specifically didn't see the need for nested sections that was at least in part the motivation for the shootout).
Oh - must've missed that. Perhaps the Wiki page should be updated. Thanks, Michael. Regards, Vinay Sajip
I'll see if I can dig out the discussion and update the page. It might also be worth adding feature requests or doc examples for ConfigParser to cover the more common use cases / features that other config file modules provide that you don't get out of the box with ConfigParser. Michael -- http://www.ironpythoninaction.com On 19 Sep 2009, at 08:54, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Michael Foord <michael@...> writes:
It did go somewhere - it was discussed on Python-dev and Guido didn't see the need to replace ConfigParser (and specifically didn't see the need for nested sections that was at least in part the motivation for the shootout).
Oh - must've missed that. Perhaps the Wiki page should be updated.
Thanks, Michael.
Regards,
Vinay Sajip
_______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
Michael Foord <michael@...> writes:
I'll see if I can dig out the discussion and update the page.
Good idea.
It might also be worth adding feature requests or doc examples for ConfigParser to cover the more common use cases / features that other config file modules provide that you don't get out of the box with ConfigParser.
There's already a ConfigParser examples page http://wiki.python.org/moin/ConfigParserExamples which has only some basic stuff - but perhaps any additional examples/recipes should be added there. Regards, Vinay Sajip
Vinay Sajip wrote:
Michael Foord <michael@...> writes:
I'll see if I can dig out the discussion and update the page.
Good idea.
It might also be worth adding feature requests or doc examples for ConfigParser to cover the more common use cases / features that other config file modules provide that you don't get out of the box with ConfigParser.
There's already a ConfigParser examples page
http://wiki.python.org/moin/ConfigParserExamples
which has only some basic stuff - but perhaps any additional examples/recipes should be added there.
For common use cases that *can* be achieved with simple recipes it would be nice to have them in the documentation. Anyway - I'll put this in my 'list' of things to do. Michael
Regards,
Vinay Sajip
_______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog
Mats Kindahl wrote:
Hi all,
I am currently using the ConfigParser module to parse MySQL configuration files. Just plain config files, nothing fancy except... options without values.
There is a number of options here that does not require a value, they are basically just turning on a feature. They are also common in the standard template files for the server. Options that are for mysqld can have a value even though it is not required and the option file parser will not complain, but for some of the client tools, values may not be given or there will be a error.
Looking at the tests of ConfigParser, I see that it is a deliberate design decision to not allow options without values (or I am misunderstanding something). Why?
Probably to provide better error reporting and avoid casual user errors.
Being able to parse these files without having to write a new module would be very valuable.
Please submit a feature request for this, ideally with patch to the Python tracker. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 17 2009)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
participants (10)
-
Antoine Pitrou -
Brett Cannon -
Georg Brandl -
Jesse Noller -
M.-A. Lemburg -
Mats Kindahl -
Michael -
Michael Foord -
R. David Murray -
Vinay Sajip