git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 6/7] rev-list: let traversal die when --missing is not in use

From
Taylor Blau <me@ttaylorr.com>
Date
Apr 5, 2019, 03:37 UTC
Message-ID
<a3a80b4b2a988eb65d85a5acd54c584d047073c7.1554435033.git.me@ttaylorr.com>
In-Reply-To
<cover.1554435033.git.me@ttaylorr.com>
From: Jeff King <peff@peff.net>

Commit 7c0fe330d5 (rev-list: handle missing tree objects properly, 2018-10-05) taught the traversal machinery used by git-rev-list to ignore missing trees, so that rev-list could handle them itself.

However, it does so only by checking via oid_object_info_extended() that the object exists at all. This can miss several classes of errors that were previously detected by rev-list:

  - type mismatches (e.g., we expected a tree but got a blob)
  - failure to read the object data (e.g., due to bitrot on disk)

This is especially important because we use "rev-list --objects" as our connectivity check to admit new objects to the repository, and it will now miss these cases (though the bitrot one is less important here, because we'd typically have just hashed and stored the object).

There are a few options to fix this:
 1. we could check these properties in rev-list when we do the existence
    check. This is probably too expensive in practice (perhaps even for
    a type check, but definitely for checking the whole content again,
    which implies loading each object into memory twice).
 2. teach the traversal machinery to differentiate between a missing
    object, and one that could not be loaded as expected. This probably
    wouldn't be too hard to detect type mismatches, but detecting bitrot
    versus a truly missing object would require deep changes to the
    object-loading code.
 3. have the traversal machinery communicate the failure to the caller,
    so that it can decide how to proceed without re-evaluting the object
    itself.

Of those, I think (3) is probably the best path forward. However, this patch does none of them. In the name of expediently fixing the regression to a normal "rev-list --objects" that we use for connectivity checks, this simply restores the pre-7c0fe330d5 behavior of having the traversal die as soon as it fails to load a tree (when --missing is set to MA_ERROR, which is the default).

Note that we can't get rid of the object-existence check in finish_object(), because this also handles blobs (which are not otherwise checked at all by the traversal code).

Signed-off-by: Jeff King <peff@peff.net>
---
 builtin/rev-list.c                     | 4 +++-
 t/t6102-rev-list-unexpected-objects.sh | 4 ++--
 2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 425a5774db..9f31837d30 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -379,7 +379,6 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 	repo_init_revisions(the_repository, &revs, prefix);
 	revs.abbrev = DEFAULT_ABBREV;
 	revs.commit_format = CMIT_FMT_UNSPECIFIED;
-	revs.do_not_die_on_missing_tree = 1;
 
 	/*
 	 * Scan the argument list before invoking setup_revisions(), so that we
@@ -409,6 +408,9 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 		}
 	}
 
+	if (arg_missing_action)
+		revs.do_not_die_on_missing_tree = 1;
+
 	argc = setup_revisions(argc, argv, &revs, &s_r_opt);
 
 	memset(&info, 0, sizeof(info));
diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh
index 30976385a8..c8d4b31f8f 100755
--- a/t/t6102-rev-list-unexpected-objects.sh
+++ b/t/t6102-rev-list-unexpected-objects.sh
@@ -29,7 +29,7 @@ test_expect_success 'setup unexpected non-tree entry' '
 	broken_tree="$(git hash-object -w --literally -t tree broken-tree)"
 '
 
-test_expect_failure 'traverse unexpected non-tree entry (lone)' '
+test_expect_success 'traverse unexpected non-tree entry (lone)' '
 	test_must_fail git rev-list --objects $broken_tree
 '
 
@@ -64,7 +64,7 @@ test_expect_success 'setup unexpected non-tree root' '
 		broken-commit)"
 '
 
-test_expect_failure 'traverse unexpected non-tree root (lone)' '
+test_expect_success 'traverse unexpected non-tree root (lone)' '
 	test_must_fail git rev-list --objects $broken_commit
 '
 
-- 
2.21.0.203.g358da99528
Previous: Taylor BlauNext: Jeff King
Message 28 of 41 in “harden unexpected object types checks”
  1. 0/7 harden unexpected object types checksTaylor Blau, Apr 5, 2019
  2. 1/7 t: move 'hex2oct' into test-lib-functions.shTaylor Blau, Apr 5, 2019
  3. 2/7 t: introduce tests for unexpected object typesTaylor Blau, Apr 5, 2019
  4. SZEDER GáborApr 5, 2019
  5. Jeff KingApr 5, 2019
  6. SZEDER GáborApr 5, 2019
  7. Jeff KingApr 5, 2019
  8. Ævar Arnfjörð BjarmasonApr 7, 2019
  9. Taylor BlauApr 9, 2019
  10. Ævar Arnfjörð BjarmasonApr 9, 2019
  11. Taylor BlauApr 10, 2019
  12. Junio C HamanoApr 8, 2019
  13. Eric SunshineApr 5, 2019
  14. Jeff KingApr 5, 2019
  15. Taylor BlauApr 6, 2019
  16. Junio C HamanoApr 8, 2019
  17. Taylor BlauApr 9, 2019
  18. Eric SunshineApr 9, 2019
  19. Taylor BlauApr 9, 2019
  20. Eric SunshineApr 9, 2019
  21. Taylor BlauApr 10, 2019
  22. Taylor BlauApr 6, 2019
  23. Jeff KingApr 5, 2019
  24. Taylor BlauApr 6, 2019
  25. 3/7 list-objects.c: handle unexpected non-blob entriesTaylor Blau, Apr 5, 2019
  26. 4/7 list-objects.c: handle unexpected non-tree entriesTaylor Blau, Apr 5, 2019
  27. 5/7 get_commit_tree(): return NULL for broken treeTaylor Blau, Apr 5, 2019
  28. 6/7 rev-list: let traversal die when --missing is not in useTaylor Blau, Apr 5, 2019
  29. Jeff KingApr 5, 2019
  30. Taylor BlauApr 6, 2019
  31. Jeff KingApr 7, 2019
  32. Taylor BlauApr 9, 2019
  33. 7/7 rev-list: detect broken root treesTaylor Blau, Apr 5, 2019
  34. 0/7 harden unexpected object types checksTaylor Blau, Apr 10, 2019
  35. 1/7 t: move 'hex2oct' into test-lib-functions.shTaylor Blau, Apr 10, 2019
  36. 2/7 t: introduce tests for unexpected object typesTaylor Blau, Apr 10, 2019
  37. 3/7 list-objects.c: handle unexpected non-blob entriesTaylor Blau, Apr 10, 2019
  38. 4/7 list-objects.c: handle unexpected non-tree entriesTaylor Blau, Apr 10, 2019
  39. 5/7 get_commit_tree(): return NULL for broken treeTaylor Blau, Apr 10, 2019
  40. 6/7 rev-list: let traversal die when --missing is not in useTaylor Blau, Apr 10, 2019
  41. 7/7 rev-list: detect broken root treesTaylor Blau, Apr 10, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.