git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Protecting old temporary objects being reused from concurrent "git gc"?

From
Matt McCutchen <matt@mattmccutchen.net>
Date
Nov 15, 2016, 14:13 UTC
Message-ID
<1479219194.2406.73.camel@mattmccutchen.net>

The Braid subproject management tool stores the subproject content in the main tree and is able to switch to a different upstream revision of a subproject by doing the equivalent of "git read-tree -m" on the superproject tree and the two upstream trees.  The tricky part is preparing temporary trees with the upstream content moved to the path configured for the superproject.  The usual method is "git read-tree --prefix", but using what index file?  Braid currently uses the user's actual worktree, which can leave a mess if it gets interrupted:

https://github.com/cristibalan/braid/blob/7d81da6e86e24de62a74f3ab8d880666cb343b04/lib/braid/commands/update.rb#L98
I want to change this to something that won't leave an inconsistent
state if interrupted.  I've written code for this kind of thing before
that sets GIT_INDEX_FILE and uses a temporary index file and "git
write-tree".  But I realized that if "git gc" runs concurrently, the
generated tree could be deleted before it is used and the tool would
fail.  If I had a need to run "git commit-tree", it seems like I might
even end up with a commit object with a broken reference to a tree.
 "git gc" normally doesn't delete objects that were created in the last
2 weeks, but if an identical tree was added to the object database more
than 2 weeks ago by another operation and is unreferenced, it could be
reused without updating its mtime and it could still get deleted.

Is there a recommended way to avoid this kind of problem in add-on tools?  (I searched the Git documentation and the web for information about races with "git gc" and didn't find anything useful.)  If not, it seems to be a significant design flaw in "git gc", even if the problem is extremely rare in practice.  I wonder if some of the built-in commands may have the same problem, though I haven't tried to test them.  If this is confirmed to be a known problem affecting built-in commands, then at least I won't feel bad about introducing the same problem into add-on tools. :/

Thanks, Matt

Next: Jeff King
Message 1 of 13 in “Protecting old temporary objects being reused from concurrent "git gc"?”
  1. Matt McCutchenNov 15, 2016
  2. Jeff KingNov 15, 2016
  3. Matt McCutchenNov 15, 2016
  4. Jeff KingNov 15, 2016
  5. git-gc.txt: expand discussion of races with other processesMatt McCutchen, Nov 15, 2016
  6. Matt McCutchenNov 15, 2016
  7. Junio C HamanoNov 15, 2016
  8. Jeff KingNov 16, 2016
  9. Junio C HamanoNov 16, 2016
  10. Junio C HamanoNov 16, 2016
  11. Jeff KingNov 17, 2016
  12. Re* Protecting old temporary objects being reused from concurrent "git gc"?Junio C Hamano, Nov 17, 2016
  13. Jeff KingNov 17, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.