Python and file locking - NFS or MySQL?

Jeremy Jones zanesdad at bellsouth.net
Fri Sep 2 08:00:18 EDT 2005


Christopher DeMarco wrote:

>Hi all...
>
>...I've got a Python script running on a bunch of boxen sharing some
>common NFS-exported space.  I need (not want :) to lock files for
>writing, and I need (not want :) to do it safely (i.e. atomically).
>I'm doing this in Linux; NFS4 is available.  As I understand it, my
>options are:
>
>1.  Python's fcntl() is an interface to the fcntl(2) system call,
>which is claimed to work "mostly" over NFS v >= 3.
>  
>
I would go with this one, but test the crap out of it.  This *should* 
work just fine for you on NFS, but again, test the crap out of it.  
Write a script that spawns slightly beyond the number of processes (by 
spawning either threads or processes) you expect to actually occur and 
mercilessly lock, update, unlock the file while checking for the results 
to be consistent with what you think they ought to be.

>2.  open(2) is atomic on a local FS, I've read discussions that imply
>that link(2) is atomic over NFS (is it?), so I can link from local
>lockfile to remote target atomically.  I don't grok this; open(2) +
>link(2) + stat(2) == 3 calls on my fingers.  HTH is this supposed to
>work?
>
>3.  Atomically update a MySQL database indicating that the file is
>locked - MySQL has atomic transactions now, right?  And how good is
>the Python MySQL API - IIRC Perl didn't have atomic transactions last
>year; will this work in contemporary Python?
>
>I've got a while (several weeks) to chew this problem over (my current
>implementation is ``assert("Poof!  File locked")'').
>
>What are my options for safely locking files via NFS?  I don't want to
>get involved with NLM as my impression is it's being buggy and
>unwieldy.  Thanks in advance!
>
>
>I was present at an undersea, unexplained mass sponge migration.
>  
>
HTH,

JMJ



More information about the Python-list mailing list