Locking around

Nikolaus Rath Nikolaus at rath.org
Wed Aug 6 20:33:11 CEST 2008

Tobiah <toby at tobiah.org> writes:
> On Mon, 04 Aug 2008 15:30:51 +0200, Nikolaus Rath wrote:
>> Hello,
>> I need to synchronize the access to a couple of hundred-thousand
>> files[1]. It seems to me that creating one lock object for each of the
>> files is a waste of resources, but I cannot use a global lock for all
>> of them either (since the locked operations go over the network, this
>> would make the whole application essentially single-threaded even
>> though most operations act on different files).
> Do you think you could use an SQL database on the network to
> handle the locking?

Yeah, I could. It wouldn't even have to be over the network (I'm
synchronizing access from within the same program). But I think that
is even more resource-wasteful than my original idea.

> Hey, maybe the files themselves should go into blobs.

Nope, not possible.  They're on Amazon S3.



 »It is not worth an intelligent man's time to be in the majority.
  By definition, there are already enough people to do that.«
                                                         -J.H. Hardy

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C

More information about the Python-list mailing list