git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 11/12] pack-objects: add checks for duplicate objects

From
Christian Couder <christian.couder@gmail.com>
Date
Dec 18, 2019, 11:25 UTC
Message-ID
<20191218112547.4974-12-chriscool@tuxfamily.org>
In-Reply-To
<20191218112547.4974-1-chriscool@tuxfamily.org>
From: Jeff King <peff@peff.net>

Additional checks are added in have_duplicate_entry() and obj_is_packed() to avoid duplicate objects in the reuse bitmap. It was probably buggy to not have such a check before.

Git as a client would never both asks for a tag by sha1 and specify "include-tag", but libgit2 will, so a libgit2 client cloning from a Git server would trigger the bug.

If a client both asks for a tag by sha1 and specifies "include-tag", we may end up including the tag in the reuse bitmap (due to the first thing), and then later adding it to the packlist (due to the second). This results in duplicate objects in the pack, which git chokes on. We should notice that we are already including it when doing the include-tag portion, and avoid adding it to the packlist.

The simplest place to fix this is right in add_ref_tag(), where we could avoid peeling the tag at all if we know that we are already including it. However, this pushes the check instead into have_duplicate_entry(). This fixes not only this case, but also means that we cannot have any similar problems lurking in other code.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 builtin/pack-objects.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index c80c1fac94..b1998202fb 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1127,6 +1127,10 @@ static int have_duplicate_entry(const struct object_id *oid,
 {
 	struct object_entry *entry;
 
+	if (reuse_packfile_bitmap &&
+	    bitmap_walk_contains(bitmap_git, reuse_packfile_bitmap, oid))
+		return 1;
+
 	entry = packlist_find(&to_pack, oid);
 	if (!entry)
 		return 0;
@@ -2681,7 +2685,9 @@ static void ll_find_deltas(struct object_entry **list, unsigned list_size,
 
 static int obj_is_packed(const struct object_id *oid)
 {
-	return !!packlist_find(&to_pack, oid);
+	return packlist_find(&to_pack, oid) ||
+		(reuse_packfile_bitmap &&
+		 bitmap_walk_contains(bitmap_git, reuse_packfile_bitmap, oid));
 }
 
 static void add_tag_chain(const struct object_id *oid)
-- 
2.24.1.498.g561400140f
Previous: Christian Couder
Message 13 of 13 in “Rewrite packfile reuse code”
  1. 00/12 Rewrite packfile reuse codeChristian Couder, Dec 18, 2019
  2. 01/12 builtin/pack-objects: report reused packfile objectsChristian Couder, Dec 18, 2019
  3. 03/12 ewah/bitmap: introduce bitmap_word_alloc()Christian Couder, Dec 18, 2019
  4. 02/12 packfile: expose get_delta_base()Christian Couder, Dec 18, 2019
  5. 04/12 pack-bitmap: introduce bitmap_walk_contains()Christian Couder, Dec 18, 2019
  6. 05/12 pack-bitmap: uninteresting oid can be outside bitmapped packfileChristian Couder, Dec 18, 2019
  7. 06/12 pack-bitmap: simplify bitmap_has_oid_in_uninteresting()Christian Couder, Dec 18, 2019
  8. 09/12 builtin/pack-objects: introduce obj_is_packed()Christian Couder, Dec 18, 2019
  9. 07/12 csum-file: introduce hashfile_total()Christian Couder, Dec 18, 2019
  10. 08/12 pack-objects: introduce pack.allowPackReuseChristian Couder, Dec 18, 2019
  11. 10/12 pack-objects: improve partial packfile reuseChristian Couder, Dec 18, 2019
  12. 12/12 pack-bitmap: don't rely on bitmap_git->reuse_objectsChristian Couder, Dec 18, 2019
  13. 11/12 pack-objects: add checks for duplicate objectsChristian Couder, Dec 18, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.