git/list[1] front-page[2] threads[3] people[4] search[5] about
 

RE: Non-robust lock files in containers can lead to repo corruption

From
Randall S. Becker <rsbecker@nexbridge.com>
Date
Aug 12, 2019, 13:48 UTC
Message-ID
<003901d55114$9428df40$bc7a9dc0$@nexbridge.com>
In-Reply-To
<qimq0i$49gn$1@blaine.gmane.org>
On August 10, 2019 12:06 PM, Gregory Szorc wrote:
Show 29 quoted lines
> I tracked down a source of Git corrupting repositories to lock file design not
> being robust when containers / PID namespaces are present.
> 
> In my case, the corruption stemmed from premature release of the `git gc`
> lock in the gc.pid file. But since the lock file code for that file is in gc.c, there
> could be other lock files in Git affected by the same design limitation as well.
> 
> The lock design of gc.pid stores the current hostname and PID of the locking
> process in the file. If another process comes along and its hostname matches
> the stored hostname, it checks to see if the listed PID exists. If the PID is
> missing, it assumes the lock is stale and releases the lock.
> 
> A limitation with this approach is it isn't robust in the presence of containers
> / PID namespaces. In containers, it is common for the hostname to match
> the container host's hostname. Or the hostname will be static string. In
> Kubernetes, all containers within a pod share the same hostname. Containers
> (almost always) run in separate PID namespaces, so PIDs from outside the
> container aren't visible to the container itself.
> This means that if e.g. 2 `git gc` processes are running with the same
> hostname in separate containers / PID namespaces, Git could prematurely
> release the lock file because it thinks the "other" PID is dead and repo
> corruption could ensue due to the 2 `git gc` processes racing with each other.
> 
> The on-disk format of lock files obviously needs to be backwards compatible
> with older clients. One backwards compatible solution is to append
> something to the hostname to disambiguate containers / PID namespaces.
> Mercurial appends the current PID namespace identifier to the hostname [1]
> and my experience is that this is sufficient to mitigate the issue. It is possible
> more robust solutions are achievable.
While I like the idea personally, many platforms, including NonStop (TNS/E),  do not support pid namespaces. In particular setns(2) may not be implemented. Please make sure that any changes detect this condition properly and omit the use of namespaces.

Regards, Randall

Previous: Gregory SzorcNext: Junio C Hamano
Message 2 of 5 in “Non-robust lock files in containers can lead to repo corruption”
  1. Gregory SzorcAug 10, 2019
  2. Randall S. BeckerAug 12, 2019
  3. Junio C HamanoAug 12, 2019
  4. Jeff KingAug 12, 2019
  5. Junio C HamanoAug 13, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.