git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH/WIP 6/7] fast-import: workaround data corruption

From
Dmitry Ivankov <divanorama@gmail.com>
Date
Jul 28, 2011, 04:46 UTC
Message-ID
<1311828370-30477-7-git-send-email-divanorama@gmail.com>
In-Reply-To
<1311828370-30477-1-git-send-email-divanorama@gmail.com>

fast-import keeps track of some delta-base for tree objects. When it is time to compute the delta, base object is constructed from in-memory tree representation using children's delta bases sha1. But these can be unrelated due to several bugs, and it leads to object with wrong sha1 being delta-written to the packfile.

We have the base sha1 and what we think it's data is. Verify sha1 and if it doesn't match, report it to stderr and don't use delta for this tree.

We could also die() here when bugs are fixed. Or we can see if the data we've got is from our pack file and so still try to use it as a base.

Signed-off-by: Dmitry Ivankov <divanorama@gmail.com>
---
 fast-import.c |   14 +++++++++++++-
 1 files changed, 13 insertions(+), 1 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index 9f0d2fe..14a2a63 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1469,7 +1469,8 @@ static void drop_old(struct tree_entry *root)
 static void store_tree(struct tree_entry *root)
 {
 	struct tree_content *t = root->tree;
-	struct last_object lo = { STRBUF_INIT, 0, 0, /* no_swap */ 1 };
+	struct strbuf empty = STRBUF_INIT;
+	struct last_object lo = { empty, 0, 0, /* no_swap */ 1 };
 	struct object_entry *le;
 	unsigned int i;
 
@@ -1486,10 +1487,21 @@ static void store_tree(struct tree_entry *root)
 
 	le = find_object(root->versions[0].sha1);
 	if (S_ISDIR(root->versions[0].mode) && le && le->pack_id == pack_id) {
+		unsigned char sh[20];
 		mktree(t, 0, &old_tree);
 		lo.data = old_tree;
 		lo.offset = le->idx.offset;
 		lo.depth = t->delta_depth;
+
+		prepare_object_hash(OBJ_TREE, &old_tree, NULL, NULL, sh);
+		if (hashcmp(sh, root->versions[0].sha1)) {
+			fprintf(stderr, "internal sha1 delta base mismatch,"
+					" won't use delta for that tree\n");
+			lo.data = empty;
+			lo.offset = 0;
+			lo.depth = 0;
+		}
+
 	}
 
 	mktree(t, 1, &new_tree);
-- 
1.7.3.4
Previous: Dmitry IvankovNext: Jonathan Nieder
Message 11 of 13 in “was: long fast-import errors out "failed to apply delta"”
  1. 0/7 was: long fast-import errors out "failed to apply delta"Dmitry Ivankov, Jul 28, 2011
  2. 1/7 fast-import: extract object preparation functionDmitry Ivankov, Jul 28, 2011
  3. 2/7 fast-import: be saner with temporary treesDmitry Ivankov, Jul 28, 2011
  4. Jonathan NiederJul 28, 2011
  5. 3/7 fast-import: fix a data corruption in parse_lsDmitry Ivankov, Jul 28, 2011
  6. Jonathan NiederJul 28, 2011
  7. 4/7 fast-import: fix data corruption in store_treeDmitry Ivankov, Jul 28, 2011
  8. Jonathan NiederJul 28, 2011
  9. Dmitry IvankovJul 28, 2011
  10. 5/7 fast-import: extract tree_content reading functionDmitry Ivankov, Jul 28, 2011
  11. 6/7 fast-import: workaround data corruptionDmitry Ivankov, Jul 28, 2011
  12. Jonathan NiederJul 28, 2011
  13. 7/7 fast-import: fix data corruption in load_treeDmitry Ivankov, Jul 28, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.