[zc.buildout] running in safe mode
Hello I know it is a bad practice for a recipe to return some paths that contains important data in the install() method, because zc.buildout might remove them. Nevertheless, it happens from time to time that a developer lose some content because of a misconfiguration, or a zealous recipe. That is his responsability, and backups are done for that. But I think we can improve zc.buildout a bit for that: what about introducing a safe-mode where the developer get prompted everytime zc.buildout.rmtree is about to call shutil.rmtree ? The option could be set in [buildout] like this: [buildout] ... safe-mode = true ... and challenge the user when a tree is about to be delete. (it might be overkill for single files, and they are most of the time unimportant configuration files) This is a small change, and it would avoid running a backup everytime the .cfg are changed. because it happens all day long when you are developing. Regards Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/
On Oct 1, 2008, at 6:34 AM, Tarek Ziadé wrote:
Hello
I know it is a bad practice for a recipe to return some paths that contains important data in the install() method, because zc.buildout might remove them.
Nevertheless, it happens from time to time that a developer lose some content because of a misconfiguration, or a zealous recipe. That is his responsability, and backups are done for that.
I don't think backups are the right approach. It's a mistake to have recipes manage precious data. If you really really really think that's a good idea, then the recipe should at least manage uninstall and move precious data aside, rather than remove it. I don't think it is really the user's problem is a recipe misbehaves by allowing precious data to be removed.
But I think we can improve zc.buildout a bit for that:
what about introducing a safe-mode where the developer get prompted everytime zc.buildout.rmtree is about to call shutil.rmtree ?
The option could be set in [buildout] like this:
[buildout] ... safe-mode = true ...
and challenge the user when a tree is about to be delete. (it might be overkill for single files, and they are most of the time unimportant configuration files)
This is a small change, and it would avoid running a backup everytime the .cfg are changed. because it happens all day long when you are developing.
I suspect this would be so annoying to use that no one would use it. I think it's probably easier to fix the broken recipes. I also think calling this "safe" mode is misleading. I could live with an option named something like "prompt-before-removing-files-or-directories". A better option might be something like "move-aside-on-uninstall", which would move files or directories aside rather than deleting them. I still think it would be better to just fix the broken recipes. Jim -- Jim Fulton Zope Corporation
Jim Fulton wrote:
I know it is a bad practice for a recipe to return some paths that contains important data in the install() method, because zc.buildout might remove them.
Nevertheless, it happens from time to time that a developer lose some content because of a misconfiguration, or a zealous recipe. That is his responsability, and backups are done for that.
I don't think backups are the right approach. It's a mistake to have recipes manage precious data. If you really really really think that's a good idea, then the recipe should at least manage uninstall and move precious data aside, rather than remove it.
I don't think it is really the user's problem is a recipe misbehaves by allowing precious data to be removed.
I'll note fassembler uses a file abstraction layer so that its recipes are safe by default: https://svn.openplans.org/svn/fassembler/trunk/fassembler/filemaker.py I think buildout would be a lot more humane if it took the same approach. -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
On Oct 2, 2008, at 6:15 PM, Ian Bicking wrote:
Jim Fulton wrote:
I know it is a bad practice for a recipe to return some paths that contains important data in the install() method, because zc.buildout might remove them.
Nevertheless, it happens from time to time that a developer lose some content because of a misconfiguration, or a zealous recipe. That is his responsability, and backups are done for that. I don't think backups are the right approach. It's a mistake to have recipes manage precious data. If you really really really think that's a good idea, then the recipe should at least manage uninstall and move precious data aside, rather than remove it. I don't think it is really the user's problem is a recipe misbehaves by allowing precious data to be removed.
I'll note fassembler uses a file abstraction layer so that its recipes are safe by default: https://svn.openplans.org/svn/fassembler/trunk/fassembler/filemaker.py
I think buildout would be a lot more humane if it took the same approach.
I'd be interested to know what you mean by this, but I'm not willing to read that source to find out. Can you be a little more specific? Jim -- Jim Fulton Zope Corporation
Jim Fulton wrote:
On Oct 2, 2008, at 6:15 PM, Ian Bicking wrote:
Jim Fulton wrote:
I know it is a bad practice for a recipe to return some paths that contains important data in the install() method, because zc.buildout might remove them.
Nevertheless, it happens from time to time that a developer lose some content because of a misconfiguration, or a zealous recipe. That is his responsability, and backups are done for that. I don't think backups are the right approach. It's a mistake to have recipes manage precious data. If you really really really think that's a good idea, then the recipe should at least manage uninstall and move precious data aside, rather than remove it. I don't think it is really the user's problem is a recipe misbehaves by allowing precious data to be removed.
I'll note fassembler uses a file abstraction layer so that its recipes are safe by default: https://svn.openplans.org/svn/fassembler/trunk/fassembler/filemaker.py
I think buildout would be a lot more humane if it took the same approach.
I'd be interested to know what you mean by this, but I'm not willing to read that source to find out.
Can you be a little more specific?
Instead of using open(), etc, to write files, there's an instance of Maker which holds some of the settings (--interactive, --simulate, a base directory). Then you do all your file operations like: maker.ensure_file('path/to/file.txt', content) If that file exists with different content then the user gets asked about what to do. It also logs all the writing, shows diffs, can make backups, etc. You can force overwriting, but that's a keyword argument that defaults to False, so only if you actually have good reason to overwrite files (without asking) then that's fine, but you will start developing the easy way, which is to be safe about this stuff. -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
On Fri, Oct 3, 2008 at 12:37 AM, Ian Bicking <ianb@colorstudy.com> wrote:
Can you be a little more specific?
Instead of using open(), etc, to write files, there's an instance of Maker which holds some of the settings (--interactive, --simulate, a base directory). Then you do all your file operations like:
maker.ensure_file('path/to/file.txt', content)
If that file exists with different content then the user gets asked about what to do.
That is basically what I wanted to introduce
On Oct 2, 2008, at 6:37 PM, Ian Bicking wrote:
Jim Fulton wrote:
On Oct 2, 2008, at 6:15 PM, Ian Bicking wrote:
Jim Fulton wrote:
I know it is a bad practice for a recipe to return some paths that contains important data in the install() method, because zc.buildout might remove them.
Nevertheless, it happens from time to time that a developer lose some content because of a misconfiguration, or a zealous recipe. That is his responsability, and backups are done for that. I don't think backups are the right approach. It's a mistake to have recipes manage precious data. If you really really really think that's a good idea, then the recipe should at least manage uninstall and move precious data aside, rather than remove it. I don't think it is really the user's problem is a recipe misbehaves by allowing precious data to be removed.
I'll note fassembler uses a file abstraction layer so that its recipes are safe by default: https://svn.openplans.org/svn/fassembler/trunk/fassembler/filemaker.py
I think buildout would be a lot more humane if it took the same approach. I'd be interested to know what you mean by this, but I'm not willing to read that source to find out. Can you be a little more specific?
Instead of using open(), etc, to write files, there's an instance of Maker which holds some of the settings (--interactive, --simulate, a base directory). Then you do all your file operations like:
maker.ensure_file('path/to/file.txt', content)
If that file exists with different content then the user gets asked about what to do. It also logs all the writing, shows diffs, can make backups, etc. You can force overwriting, but that's a keyword argument that defaults to False, so only if you actually have good reason to overwrite files (without asking) then that's fine, but you will start developing the easy way, which is to be safe about this stuff.
In a system in which most data is managed automatically, asking the user before doing anything that might remove or overwrite data is, in my experience, counterproductive. It's like a security system that constantly asks for permission do do things, training users to hit an "OK" button very quickly. In a previous version of buildout, it worked the way you and Tarek suggest. It asked users before performing any action that caused a part to be uninstalled. This was extremely annoying. I finally just started piping the output of the yes command into it. Again, I can live with people adding an option that causes buildout to prompt before removing files or directories (or maybe just uninstalling parts that would cause it to remove files or directories). I know that I wouldn't use the option myself. Jim -- Jim Fulton Zope Corporation
On Fri, Oct 3, 2008 at 2:57 PM, Jim Fulton <jim@zope.com> wrote:
In a system in which most data is managed automatically, asking the user before doing anything that might remove or overwrite data is, in my experience, counterproductive. It's like a security system that constantly asks for permission do do things, training users to hit an "OK" button very quickly.
In a previous version of buildout, it worked the way you and Tarek suggest. It asked users before performing any action that caused a part to be uninstalled. This was extremely annoying. I finally just started piping the output of the yes command into it.
Again, I can live with people adding an option that causes buildout to prompt before removing files or directories (or maybe just uninstalling parts that would cause it to remove files or directories). I know that I wouldn't use the option myself.
Ok. To reduce the noise, I think we should work with a list of folders or files we would like to 'protect', with a glob-style pattern, relative to the buildout folder, or absolute. For instance if I work with a zope application I would probably do something like that: [buildout] ... prompt-before-delete = var ... Or maybe: [buildout] ... prompt-before-delete = var/filestorage/*.fs ... Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/
Tarek Ziadé wrote:
[buildout] ... prompt-before-delete = var ...
Or maybe:
[buildout] ... prompt-before-delete = var/filestorage/*.fs
I like this idea :-) Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Jim Fulton wrote:
Instead of using open(), etc, to write files, there's an instance of Maker which holds some of the settings (--interactive, --simulate, a base directory). Then you do all your file operations like:
maker.ensure_file('path/to/file.txt', content)
If that file exists with different content then the user gets asked about what to do. It also logs all the writing, shows diffs, can make backups, etc. You can force overwriting, but that's a keyword argument that defaults to False, so only if you actually have good reason to overwrite files (without asking) then that's fine, but you will start developing the easy way, which is to be safe about this stuff.
In a system in which most data is managed automatically, asking the user before doing anything that might remove or overwrite data is, in my experience, counterproductive. It's like a security system that constantly asks for permission do do things, training users to hit an "OK" button very quickly.
In a previous version of buildout, it worked the way you and Tarek suggest. It asked users before performing any action that caused a part to be uninstalled. This was extremely annoying. I finally just started piping the output of the yes command into it.
Again, I can live with people adding an option that causes buildout to prompt before removing files or directories (or maybe just uninstalling parts that would cause it to remove files or directories). I know that I wouldn't use the option myself.
You can also treat that nuisance like a bug and resolve the problem. I think fassembler has mostly done that, so that only really interesting problems are noted. We also switched to a system inspired by gentoo ports, where these queries are deferred until the end of the build. Yes, it can be annoying, but it doesn't have to be annoying. I'd rather start at a slightly annoying situation and try to decrease that problem, than start with a periodically infuriating situation. -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
On Fri, Oct 3, 2008 at 12:09 AM, Jim Fulton <jim@zope.com> wrote:
A better option might be something like "move-aside-on-uninstall", which would move files or directories aside rather than deleting them.
ok why not,
I still think it would be better to just fix the broken recipes.
I agree it is the solution, but we tend not to be able to control all the recipes, and zc.buildout just calls out rmtree in some cases, wich is rather violent imho. ++ Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/
participants (4)
-
Chris Withers -
Ian Bicking -
Jim Fulton -
Tarek Ziadé