git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 4/4] midx: report checksum mismatches during 'verify'

From
Taylor Blau <me@ttaylorr.com>
Date
Jun 23, 2021, 18:39 UTC
Message-ID
<94e9de44e3b52513c5ab48aecd74f809dc34cbe3.1624473543.git.me@ttaylorr.com>
In-Reply-To
<cover.1624473543.git.me@ttaylorr.com>

'git multi-pack-index verify' inspects the data in an existing MIDX for correctness by checking that the recorded object offsets are correct, and so on.

But it does not check that the file's trailing checksum matches the data that it records. So, if an on-disk corruption happened to occur in the final few bytes (and all other data was recorded correctly), we would:

  - get a clean result from 'git multi-pack-index verify', but
  - be unable to reuse the existing MIDX when writing a new one (since
    we now check for checksum mismatches before reusing a MIDX)

Teach the 'verify' sub-command to recognize corruption in the checksum by calling midx_checksum_valid().

Suggested-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 midx.c                      | 3 +++
 t/t5319-multi-pack-index.sh | 5 +++++
 2 files changed, 8 insertions(+)
diff --git a/midx.c b/midx.c
index a12cbbf928..9a35b0255d 100644
--- a/midx.c
+++ b/midx.c
@@ -1228,6 +1228,9 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag
 		return result;
 	}
 
+	if (!midx_checksum_valid(m))
+		midx_report(_("incorrect checksum"));
+
 	if (flags & MIDX_PROGRESS)
 		progress = start_delayed_progress(_("Looking for referenced packfiles"),
 					  m->num_packs);
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index d582f370c4..7609f1ea64 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -418,6 +418,11 @@ test_expect_success 'corrupt MIDX is not reused' '
 	git multi-pack-index verify
 '
 
+test_expect_success 'verify incorrect checksum' '
+	pos=$(($(wc -c <$objdir/pack/multi-pack-index) - 1)) &&
+	corrupt_midx_and_verify $pos "\377" $objdir "incorrect checksum"
+'
+
 test_expect_success 'repack progress off for redirected stderr' '
 	GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir repack 2>err &&
 	test_line_count = 0 err
-- 
2.31.1.163.ga65ce7f831
Previous: Jeff KingNext: Bagas Sanjaya
Message 8 of 16 in “midx: verify MIDX checksum before reusing”
  1. 0/4 midx: verify MIDX checksum before reusingTaylor Blau, Jun 23, 2021
  2. 1/4 csum-file: introduce checksum_valid()Taylor Blau, Jun 23, 2021
  3. Jeff KingJun 24, 2021
  4. 2/4 commit-graph: rewrite to use checksum_valid()Taylor Blau, Jun 23, 2021
  5. Jeff KingJun 24, 2021
  6. 3/4 midx: don't reuse corrupt MIDXs when writingTaylor Blau, Jun 23, 2021
  7. Jeff KingJun 24, 2021
  8. 4/4 midx: report checksum mismatches during 'verify'Taylor Blau, Jun 23, 2021
  9. Bagas SanjayaJun 24, 2021
  10. Jeff KingJun 24, 2021
  11. SZEDER GáborNov 10, 2021
  12. Jeff KingNov 11, 2021
  13. Taylor BlauNov 16, 2021
  14. t5319: corrupt more bytes of the midx checksumJeff King, Nov 16, 2021
  15. Taylor BlauNov 16, 2021
  16. Derrick StoleeNov 16, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.