git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 5/6] fsck: parse loose object paths directly

From
Jeff King <peff@peff.net>
Date
Jan 13, 2017, 17:59 UTC
Message-ID
<20170113175944.tdbfqx3e4xhris7m@sigill.intra.peff.net>
In-Reply-To
<20170113175258.e66taigy4wpokohk@sigill.intra.peff.net>

When we iterate over the list of loose objects to check, we get the actual path of each object. But we then throw it away and pass just the sha1 to fsck_sha1(), which will do a fresh lookup. Usually it would find the same object, but it may not if an object exists both as a loose and a packed object. We may end up checking the packed object twice, and never look at the loose one.

In practice this isn't too terrible, because if fsck doesn't complain, it means you have at least one good copy. But since the point of fsck is to look for corruption, we should be thorough.

The new read_loose_object() interface can help us get the data from disk, and then we replace parse_object() with parse_object_buffer(). As a bonus, our error messages now mention the path to a corrupted object, which should make it easier to track down errors when they do happen.

Signed-off-by: Jeff King <peff@peff.net>
---
 builtin/fsck.c  | 46 +++++++++++++++++++++++++++++++++-------------
 t/t1450-fsck.sh | 16 ++++++++++++++++
 2 files changed, 49 insertions(+), 13 deletions(-)
diff --git a/builtin/fsck.c b/builtin/fsck.c
index f01b81eeb..4b91ee95e 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -362,18 +362,6 @@ static int fsck_obj(struct object *obj)
 	return 0;
 }
 
-static int fsck_sha1(const unsigned char *sha1)
-{
-	struct object *obj = parse_object(sha1);
-	if (!obj) {
-		errors_found |= ERROR_OBJECT;
-		return error("%s: object corrupt or missing",
-			     sha1_to_hex(sha1));
-	}
-	obj->flags |= HAS_OBJ;
-	return fsck_obj(obj);
-}
-
 static int fsck_obj_buffer(const unsigned char *sha1, enum object_type type,
 			   unsigned long size, void *buffer, int *eaten)
 {
@@ -488,9 +476,41 @@ static void get_default_heads(void)
 	}
 }
 
+static struct object *parse_loose_object(const unsigned char *sha1,
+					 const char *path)
+{
+	struct object *obj;
+	void *contents;
+	enum object_type type;
+	unsigned long size;
+	int eaten;
+
+	if (read_loose_object(path, sha1, &type, &size, &contents) < 0)
+		return NULL;
+
+	if (!contents && type != OBJ_BLOB)
+		die("BUG: read_loose_object streamed a non-blob");
+
+	obj = parse_object_buffer(sha1, type, size, contents, &eaten);
+
+	if (!eaten)
+		free(contents);
+	return obj;
+}
+
 static int fsck_loose(const unsigned char *sha1, const char *path, void *data)
 {
-	if (fsck_sha1(sha1))
+	struct object *obj = parse_loose_object(sha1, path);
+
+	if (!obj) {
+		errors_found |= ERROR_OBJECT;
+		error("%s: object corrupt or missing: %s",
+		      sha1_to_hex(sha1), path);
+		return 0; /* keep checking other objects */
+	}
+
+	obj->flags = HAS_OBJ;
+	if (fsck_obj(obj))
 		errors_found |= ERROR_OBJECT;
 	return 0;
 }
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index c39d42120..455c186fe 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -581,4 +581,20 @@ test_expect_success 'fsck errors in packed objects' '
 	! grep corrupt out
 '
 
+test_expect_success 'fsck finds problems in duplicate loose objects' '
+	rm -rf broken-duplicate &&
+	git init broken-duplicate &&
+	(
+		cd broken-duplicate &&
+		test_commit duplicate &&
+		# no "-d" here, so we end up with duplicates
+		git repack &&
+		# now corrupt the loose copy
+		file=$(sha1_file "$(git rev-parse HEAD)") &&
+		rm "$file" &&
+		echo broken >"$file" &&
+		test_must_fail git fsck
+	)
+'
+
 test_done
-- 
2.11.0.629.g10075098c
Previous: Jeff KingNext: Ævar Arnfjörð Bjarmason
Message 10 of 39 in “"git fsck" not detecting garbage at the end of blob object files...”
  1. John SzakmeisterJan 7, 2017
  2. Dennis KaarsemakerJan 7, 2017
  3. Jeff KingJan 8, 2017
  4. John SzakmeisterJan 13, 2017
  5. 0/6 loose-object fsck fixes/tighteningJeff King, Jan 13, 2017
  6. 1/6 t1450: refactor loose-object removalJeff King, Jan 13, 2017
  7. 2/6 sha1_file: fix error message for alternate objectsJeff King, Jan 13, 2017
  8. 3/6 t1450: test fsck of packed objectsJeff King, Jan 13, 2017
  9. 4/6 sha1_file: add read_loose_object() functionJeff King, Jan 13, 2017
  10. 5/6 fsck: parse loose object paths directlyJeff King, Jan 13, 2017
  11. Infinite loop regression in git-fsck in v2.12.0Ævar Arnfjörð Bjarmason, Oct 30, 2018
  12. Jeff KingOct 30, 2018
  13. Junio C HamanoOct 30, 2018
  14. Jeff KingOct 30, 2018
  15. Jeff KingOct 30, 2018
  16. 1/3 t1450: check large blob in trailing-garbage testJeff King, Oct 30, 2018
  17. 2/3 check_stream_sha1(): handle input underflowJeff King, Oct 30, 2018
  18. Junio C HamanoOct 31, 2018
  19. Jeff KingOct 31, 2018
  20. Junio C HamanoOct 31, 2018
  21. Jeff KingOct 31, 2018
  22. Jeff KingOct 31, 2018
  23. Junio C HamanoOct 31, 2018
  24. 3/3 cat-file: handle streaming failures consistentlyJeff King, Oct 30, 2018
  25. 0/3 Add a GIT_TEST_FSCK test modeÆvar Arnfjörð Bjarmason, Oct 31, 2018
  26. 1/3 tests: add a "env-bool" helper to test-toolÆvar Arnfjörð Bjarmason, Oct 31, 2018
  27. 2/3 tests: mark those tests where "git fsck" fails at the endÆvar Arnfjörð Bjarmason, Oct 31, 2018
  28. Junio C HamanoNov 1, 2018
  29. 3/3 tests: add a special test setup that runs "git fsck" before exitingÆvar Arnfjörð Bjarmason, Oct 31, 2018
  30. Torsten BögershausenOct 31, 2018
  31. Junio C HamanoOct 31, 2018
  32. Jeff KingOct 31, 2018
  33. Eric SunshineOct 31, 2018
  34. Jeff KingOct 31, 2018
  35. Ævar Arnfjörð BjarmasonOct 30, 2018
  36. Jeff KingOct 30, 2018
  37. 6/6 fsck: detect trailing garbage in all object typesJeff King, Jan 13, 2017
  38. John SzakmeisterJan 19, 2017
  39. John SzakmeisterJan 13, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.