ConfigParser saves the data in a not-predefined order. This is because it keeps, internally, the information in dictionaries. I opened a patch in SF 1399309 that orders the info to be saved in the file through the ConfigParser write() method. This patch does not let the user to specify the order, but allows to predict the order in which it'll be saved. There's another patch, the 1371075, that allows to specify the order through an "ordered dict". What do we want ConfigParser to do? Regards, . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/
I think it's moot unless you also preserve comments. Ideally would be
something that prserved everything (ordering, blank lines, comments
etc.) from how it was read in. Modifying a value should keep its
position. Adding a value should add it to the end of the section it's
in (unless there are cases where ordering is important to the
semantics -- are there?).
--Guido
On 1/7/06, Facundo Batista
ConfigParser saves the data in a not-predefined order. This is because it keeps, internally, the information in dictionaries.
I opened a patch in SF 1399309 that orders the info to be saved in the file through the ConfigParser write() method.
This patch does not let the user to specify the order, but allows to predict the order in which it'll be saved.
There's another patch, the 1371075, that allows to specify the order through an "ordered dict".
What do we want ConfigParser to do?
Regards,
. Facundo
Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
2006/1/7, Guido van Rossum
I think it's moot unless you also preserve comments. Ideally would be something that prserved everything (ordering, blank lines, comments etc.) from how it was read in. Modifying a value should keep its position. Adding a value should add it to the end of the section it's in (unless there are cases where ordering is important to the semantics -- are there?).
Not in my case. We can rewrite ConfigParser to store everything and write it back in the exact order it took it, with new values at the end of each section (or where the user inserted it), but it'll took a big rewrite of the module: right now it's using dictionaries and should use lists to achieve that behaviour. What I wanted to add to the module was predicatibility: a very needed feature when you're writing test cases (and that's where I got bite). . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/
On Monday 09 January 2006 12:08, Facundo Batista wrote:
What I wanted to add to the module was predicatibility: a very needed feature when you're writing test cases (and that's where I got bite).
In that case, would sorting the keys within each section be sufficient when writing it back out? I had a class to do the really-careful editing bit like Guido described once, but lost it in a disk crash several years ago. I really wish I'd gotten that into revision control somewhere, but it's too late now. I've not needed that badly enough to re-write it. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
2006/1/9, Fred L. Drake, Jr.
On Monday 09 January 2006 12:08, Facundo Batista wrote:
What I wanted to add to the module was predicatibility: a very needed feature when you're writing test cases (and that's where I got bite).
In that case, would sorting the keys within each section be sufficient when writing it back out?
Yes, because giving a set of sections and values inside them, you know in advance how they'll finish in the file.
I had a class to do the really-careful editing bit like Guido described once, but lost it in a disk crash several years ago. I really wish I'd gotten that into revision control somewhere, but it's too late now. I've not needed that badly enough to re-write it.
Somebody (me, for example) can write it if we all decide that that's the desired behaviour. We just need to take a side. Me, I just need the predicatibility that sort() gives. . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/
On Mon, 2006-01-09 at 14:08 -0300, Facundo Batista wrote:
We can rewrite ConfigParser to store everything and write it back in the exact order it took it, with new values at the end of each section (or where the user inserted it), but it'll took a big rewrite of the module: right now it's using dictionaries and should use lists to achieve that behaviour.
What I wanted to add to the module was predicatibility: a very needed feature when you're writing test cases (and that's where I got bite).
It's also important when you do crazy things like keep your configuration files in a revision control system. That's something that I do as much as possible. In general it works great for some thing, and pretty poorly for others, but when it works it makes keeping systems in sync or backed up much easier. -Barry
2006/1/10, Barry Warsaw
It's also important when you do crazy things like keep your configuration files in a revision control system. That's something that
Good point, Barry. So, unless somebody screams a blatant "no", I'll apply my patch to gain predictability and reject the other one because, if further work is made, it should be done towards a controlled position structure (a.k.a. use lists instead of dictionaries). Thanks. . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/
[Guido]
I think it's moot unless you also preserve comments. Ideally would be something that prserved everything (ordering, blank lines, comments etc.) from how it was read in. Modifying a value should keep its position. Adding a value should add it to the end of the section it's in (unless there are cases where ordering is important to the semantics -- are there?).
[Facundo Batista]
We can rewrite ConfigParser to store everything and write it back in the exact order it took it, with new values at the end of each section (or where the user inserted it), but it'll took a big rewrite of the module: right now it's using dictionaries and should use lists to achieve that behaviour.
It's not at all that complicated. Simply running through the original file when writing gets you the original order and comments. This is simple to do (a simple example is in the SpamBayes source code in the OptionsClass.py module). Remember that there has been a lot of discussion about how ConfigParser should work in the past; for example (ignoring c.l.p): http://mail.python.org/pipermail/python-dev/2004-October/049454.html http://mail.python.org/pipermail/python-dev/2004-October/049527.html http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1483518 =Tony.Meyer
2006/1/11, Tony Meyer
Remember that there has been a lot of discussion about how ConfigParser should work in the past; for example (ignoring c.l.p):
http://mail.python.org/pipermail/python-dev/2004-October/049454.html http://mail.python.org/pipermail/python-dev/2004-October/049527.html http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1483518
My god. Read it all (which took me near a thermo flask (or whatever you call to an insulated bottle to keep liquids cold or hot) of "mate"!). I see two paths here: - Good one - Bad one No, sorry, seriously: - Rewrite ConfigParser entirely. - Apply my patch. The first option is the better, though it's the one which will take more work. Someone needs to make a PEP. Michael Chermside, in the first mail of the first link that you sent, was working in a solution, but don't know what is done. In this case, both patchs I talked before should be rejected. The second one is the quickest. Both paths solves my problem. I think we should go with the second option in the short term, and with the first one in the long term. What do you think? . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/
I see two paths here:
- Rewrite ConfigParser entirely. - Apply my patch.
Allowing 'surgical' editing of configuration files, as has been proposed many times both here and c.l.p would not require ConfigParser to be entirely rewritten (just more extensive modification of the write() method).
Someone needs to make a PEP. Michael Chermside, in the first mail of the first link that you sent, was working in a solution, but don't know what is done.
I believe that this always trails off without anything being achieved. IMO what would be best would be a pronouncement (or something of similar weight) about the direction that should be taken wrt to configuration file parsing support in the stdlib. * The status quo. * The status quo, but output files in a fixed order (but unrelated to the read order); i.e. http://python.org/sf/1399309 * 'Surgical' editing of files; i.e. preserve comments, whitespace, and ordering, appending new options/sections as needed. * A more complex configuration system (perhaps like others that exist in the wild). I'm happy to either submit or help with a patch for the third option (and maybe the fourth) if need be. I haven't personally found the lack of a more complex system in the stdlib a problem (although I suppose that the various more complex systems I have worked with could have been simplified if there was one). I do believe that failing to preserve comments and ordering is a problem. =Tony.Meyer
[Tony Meyer]
Allowing 'surgical' editing of configuration files, as has been proposed many times both here and c.l.p would not require ConfigParser to be entirely rewritten (just more extensive modification of the write() method).
After writing the summary of this thread, I figured I might as well prove this by submitting a patch, so did so: [ 1410680 ] Add 'surgical editing' to ConfigParser http://python.org/sf/1410680 So there are now at least three open ConfigParser patches, all addressing similar things, although not in the same way, and all three could actually be applied. My patch (which includes new unittests and a marked-up doc patch) runs through a file and updates it to match the in-memory configuration object. Comments, blank lines, and order are preserved. By default (this can be turned off with a keyword arg) new sections/options are added to the file as well (new options in existing sections are added at the end of the last instance of that section, new sections are added at the end of the file). These new options are added in alphabetical (well, list.sort()) order, so ordering is consistent, and if an empty file is 'updated' one ends up with the output of write() but sorted. This is essentially a tidied-up version of the SpamBayes code I wrote a few years back. If anyone is still interested in adding this, feel free to take a look at the tracker :) =Tony.Meyer
I think it's moot unless you also preserve comments. Ideally would be something that prserved everything (ordering, blank lines, comments etc.) from how it was read in. Modifying a value should keep its position. Adding a value should add it to the end of the section it's in (unless there are cases where ordering is important to the semantics -- are there?).
I wrote some code to do just that sometime back: http://www.cs.wisc.edu/~param/software/cfgparse/ It would probably need some cleanup and restructuring to be used, but it may be a good starting point if configparser is to be rewritten. -param
participants (6)
-
Barry Warsaw -
Facundo Batista -
Fred L. Drake, Jr. -
Guido van Rossum -
Paramjit Oberoi -
Tony Meyer