[Python-ideas] `to_file()` method for strings

Chris Angelico rosuav at gmail.com
Wed Mar 23 23:34:12 EDT 2016


On Thu, Mar 24, 2016 at 2:10 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> And then there are file media which lie to you, and return from a fsync
> before actually syncing, because that makes their benchmarks look good.
> I've seen file corruption on USB sticks that do this, including some
> otherwise "respectable" brands. Worst case I ever saw was a USB stick
> that (fortunately) had a blinking light to show when it was writing. It
> continuing writing for *eight minutes* (I timed it) after returning from
> fsync and the OS had unmounted the stick and it was nominally safe to
> remove.
>
> I'm told that some hard drives will do the same, although I've never
> knowingly seen it.

Yes, but not as much of it; usually with hard drives, it's a small
amount of on-board cache, which tends to improve overall performance
by batching writes. That can be disabled by the paranoid (which you
should be). More commonly, SSDs do the same bare-faced lying that you
noted with USB sticks.

There was a time when I took the paranoid approach of eschewing SSDs
even on my client computers because they're not trustworthy. I'm
beginning to shift toward an even more paranoid approach of not
trusting _any_ local storage - all my important data gets pushed to a
remote server anyway (maybe GitHub, maybe something on the LAN,
whatever), so it won't matter if a power failure corrupts stuff. But
for a server with a database on it, you kinda need to solve this
problem three ways at once, rather than depending on just one.

ChrisA


More information about the Python-ideas mailing list