Stale locks are not removed

[yuriiz]

Lock file should be removed after lock is released. But it is not.

[mwilliamson]

Thanks for the report. Since I find file-locking is easy to get wrong, locket intentionally does a minimal amount of work to implement file locks. I'd have to check to see what the correct behaviour is to remove a lock file without introducing race conditions, but I'm afraid I don't have the time right now. If you'd like to investigate yourself, and submit a pull request, then that'd be great!

[mwilliamson]

I'm not sure this is possible to implement since you I don't know how you'd atomically release the lock and delete the file. But presumably other programs do this successfully, so how do they implement this?

Assume that you unlock the file and then delete it. I think there's a possible race condition:

• Process 1 creates and locks the file at the path blah.lock. Let's call this file A.
• Process 1 releases the lock.
• Process 2 locks file A.
• Process 1 unlinks file A from blah.lock. Since fcntl locks are based on files, not paths, process 1 can delete the file while process 2 still holds a lock.
• Process 3 creates and locks a new file at the path blah.lock (since file A no longer exists at that path).
• Process 2 and 3 both hold a lock and think they have exclusive access.

What if we delete the file then unlock it?

• Process 1 creates and locks the file at the path blah.lock. Let's call this file A.
• Process 2 attempts to lock file A, and blocks until it acquires the lock.
• Process 1 unlinks file A from blah.lock, then releases the lock.
• Process 2 acquires the lock on file A.
• Process 3 creates and locks a new file at the path blah.lock (since file A no longer exists at that path).
• Process 2 and 3 both hold a lock and think they have exclusive access.

Come to think of it, the second race condition happens regardless of whether you unlink then unlock, or unlock then unlink. The problem is really that you want to lock on a path, not a file, but the conversion of a path to a file doesn't happen atomically with the locking.

I'm not particularly familiar with the implementation of locking (neither fcntl nor msvcrt), so if anybody knows whether the above is actually true, let me know!

[mwilliamson]

Thinking a bit more, I suspect delete then unlock works if you stat the path and stat the fd, and make sure they're actually the same inode just after acquiring the fcntl lock. Not certain if this is true on Windows/msvcrt though.

[yuriiz]

I think you can use os.open(O_CREAT | O_EXCL) to make sure you do not open
file created by another process that will be deleted later.

[mwilliamson]

But presumably using O_EXCL causes an exception if the file exists? The advantage of using fcntl (and not O_EXCL) is that you can implement a blocking lock function.

[cchriste]

Great desc of potential race conditions, @mwilliamson
I stopped here looking for a solution to one of these races in some other code (thank google). Seems there is no silver bullet and keeping one file around is definitely a more stable solution if you can swing it.
Turns out the exclusive open solution is better for my problem of the moment, so thanks also @yuriiz !