git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[JGIT PATCH 7/9] Validate the pack's footer checksum matches that in the index

From
Shawn O. Pearce <spearce@spearce.org>
Date
Feb 12, 2009, 02:36 UTC
Message-ID
<1234406219-19547-8-git-send-email-spearce@spearce.org>
In-Reply-To
<1234406219-19547-7-git-send-email-spearce@spearce.org>

If the SHA-1 checksum of the entire pack content does not match the checksum stored in the corresponding *.idx file we cannot use the index, as the offsets recorded with it are most likely incorrect.

This situation can arise if another application repacks the pack with different compression settings, but puts the same set of objects into it. The offsets are going to be different, but the resulting pack file and index file names will be the same. If we had previously read the old index into memory, but the pack isn't a match anymore, we can't safely access this pack.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../src/org/spearce/jgit/lib/PackFile.java         |   11 ++++++++++-
 .../src/org/spearce/jgit/lib/PackIndex.java        |    3 +++
 .../src/org/spearce/jgit/lib/PackIndexV1.java      |    3 +++
 .../src/org/spearce/jgit/lib/PackIndexV2.java      |    3 +++
 4 files changed, 19 insertions(+), 1 deletions(-)
diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/PackFile.java b/org.spearce.jgit/src/org/spearce/jgit/lib/PackFile.java
index d34df03..1ee0ad7 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/PackFile.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/PackFile.java
@@ -42,6 +42,7 @@
 import java.io.File;
 import java.io.IOException;
 import java.io.OutputStream;
+import java.util.Arrays;
 import java.util.Collections;
 import java.util.Comparator;
 import java.util.Iterator;
@@ -265,6 +266,7 @@ boolean supportsFastCopyRawData() throws IOException {
 	}
 
 	private void onOpenPack() throws IOException {
+		final PackIndex idx = idx();
 		final WindowCursor curs = new WindowCursor();
 		long position = 0;
 		final byte[] sig = new byte[Constants.PACK_SIGNATURE.length];
@@ -287,11 +289,18 @@ private void onOpenPack() throws IOException {
 
 		pack.readFully(position, intbuf, curs);
 		final long packCnt = NB.decodeUInt32(intbuf, 0);
-		final long idxCnt = idx().getObjectCount();
+		final long idxCnt = idx.getObjectCount();
 		if (idxCnt != packCnt)
 			throw new IOException("Pack index"
 					+ " object count mismatch; expected " + packCnt
 					+ " found " + idxCnt + ": " + pack.getName());
+		
+		final byte[] csumbuf = new byte[20];
+		pack.readFully(pack.length() - 20, csumbuf, curs);
+		if (!Arrays.equals(csumbuf, idx.packChecksum))
+			throw new IOException("Pack index mismatch; pack SHA-1 is "
+					+ ObjectId.fromRaw(csumbuf).name() + ", index expects "
+					+ ObjectId.fromRaw(idx.packChecksum).name());
 	}
 
 	private PackedObjectLoader reader(final WindowCursor curs,
diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndex.java b/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndex.java
index 13acbd1..473ce1e 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndex.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndex.java
@@ -112,6 +112,9 @@ private static boolean isTOC(final byte[] h) {
 		return true;
 	}
 
+	/** Footer checksum applied on the bottom of the pack file. */
+	protected byte[] packChecksum;
+
 	/**
 	 * Determine if an object is contained within the pack file.
 	 * 
diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndexV1.java b/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndexV1.java
index 90b5f6f..fdaa094 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndexV1.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndexV1.java
@@ -79,6 +79,9 @@ PackIndexV1(final InputStream fd, final byte[] hdr)
 			}
 		}
 		objectCnt = idxHeader[255];
+
+		packChecksum = new byte[20];
+		NB.readFully(fd, packChecksum, 0, packChecksum.length);
 	}
 
 	long getObjectCount() {
diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndexV2.java b/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndexV2.java
index 48a5206..eb56ed9 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndexV2.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/PackIndexV2.java
@@ -142,6 +142,9 @@ PackIndexV2(final InputStream fd) throws IOException {
 		} else {
 			offset64 = NO_BYTES;
 		}
+
+		packChecksum = new byte[20];
+		NB.readFully(fd, packChecksum, 0, packChecksum.length);
 	}
 
 	@Override
-- 
1.6.2.rc0.204.gf6b427
Previous: Shawn O. PearceNext: Shawn O. Pearce
Message 8 of 10 in “Misc. pack code cleanups”
  1. 0/9 Misc. pack code cleanupsShawn O. Pearce, Feb 12, 2009
  2. 1/9 Remove the Repository parameter from PackFile's constructorShawn O. Pearce, Feb 12, 2009
  3. 2/9 Remove dead stats code from WindowCacheShawn O. Pearce, Feb 12, 2009
  4. 3/9 Document why WindowFile's hash is *31Shawn O. Pearce, Feb 12, 2009
  5. 4/9 Allow PackFile to lazily load its PackIndex on first demandShawn O. Pearce, Feb 12, 2009
  6. 5/9 Arrange pack files in recency order to improve quick hitsShawn O. Pearce, Feb 12, 2009
  7. 6/9 Rename readPackHeader() to be onOpenPack()Shawn O. Pearce, Feb 12, 2009
  8. 7/9 Validate the pack's footer checksum matches that in the indexShawn O. Pearce, Feb 12, 2009
  9. 8/9 Remove yet another legacy StGit utility functionShawn O. Pearce, Feb 12, 2009
  10. 9/9 Make pack.indexversion config option default to version 2Shawn O. Pearce, Feb 12, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.