git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 3/4] reject duplicates when indexing a packfile we created

From
Jeff King <peff@peff.net>
Date
Aug 21, 2013, 20:53 UTC
Message-ID
<20130821205332.GC28165@sigill.intra.peff.net>
In-Reply-To
<20130821204955.GA28025@sigill.intra.peff.net>

The pack index-writing code is capable of detecting and rejecting duplicate entries. This can be used with index-pack to prevent buggy foreign packs from entering the repository. However, we can also be careful about the packs we generate, by double-checking during the index write that we do not have any duplicate objects. This should never happen, but it serves as an additional check that we are not generating such packfiles.

Signed-off-by: Jeff King <peff@peff.net>
---
This turns on rejection for everywhere _except_ a separately-run
index-pack. If we decide to turn it on there, too, it would make sense
to scrap this patch and just put it in reset_pack_idx_opts().
 builtin/pack-objects.c | 1 +
 bulk-checkin.c         | 1 +
 fast-import.c          | 1 +
 3 files changed, 3 insertions(+)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f069462..8da2a66 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2511,6 +2511,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	read_replace_refs = 0;
 
 	reset_pack_idx_option(&pack_idx_opts);
+	pack_idx_opts.duplicates = WRITE_IDX_DUPLICATES_REJECT;
 	git_config(git_pack_config, NULL);
 	if (!pack_compression_seen && core_compression_seen)
 		pack_compression_level = core_compression_level;
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 6b0b6d4..bad191f 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -174,6 +174,7 @@ static void prepare_to_stream(struct bulk_checkin_state *state,
 
 	state->f = create_tmp_packfile(&state->pack_tmp_name);
 	reset_pack_idx_option(&state->pack_idx_opts);
+	state->pack_idx_opts.duplicates = WRITE_IDX_DUPLICATES_REJECT;
 
 	/* Pretend we are going to write only one object */
 	state->offset = write_pack_header(state->f, 1);
diff --git a/fast-import.c b/fast-import.c
index 23f625f..b7beab0 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -3360,6 +3360,7 @@ int main(int argc, char **argv)
 
 	setup_git_directory();
 	reset_pack_idx_option(&pack_idx_opts);
+	pack_idx_opts.duplicates = WRITE_IDX_DUPLICATES_REJECT;
 	git_config(git_pack_config, NULL);
 	if (!pack_compression_seen && core_compression_seen)
 		pack_compression_level = core_compression_level;
-- 
1.8.4.rc2.28.g6bb5f3f
Previous: Nicolas PitreNext: Jeff King
Message 32 of 37 in “duplicate objects in packfile”
  1. Jeff KingAug 14, 2013
  2. Junio C HamanoAug 14, 2013
  3. Junio C HamanoAug 14, 2013
  4. Nicolas PitreAug 14, 2013
  5. Jeff KingAug 16, 2013
  6. 0/4 duplicate objects in packfilesJeff King, Aug 21, 2013
  7. 1/4 sha1-lookup: handle duplicate keys with GIT_USE_LOOKUPJeff King, Aug 21, 2013
  8. 2/4 index-pack: optionally reject packs with duplicate objectsJeff King, Aug 21, 2013
  9. Duy NguyenAug 22, 2013
  10. Jeff KingAug 22, 2013
  11. 0/6 duplicate objects and delta cycles, oh my!Jeff King, Aug 22, 2013
  12. 1/6 test-sha1: add a binary output modeJeff King, Aug 22, 2013
  13. 2/6 sha1-lookup: handle duplicate keys with GIT_USE_LOOKUPJeff King, Aug 22, 2013
  14. Junio C HamanoAug 23, 2013
  15. Jeff KingAug 23, 2013
  16. Nicolas PitreAug 23, 2013
  17. Jeff KingAug 23, 2013
  18. 0/6 duplicate objects and delta cyclesJeff King, Aug 24, 2013
  19. 1/6 test-sha1: add a binary output modeJeff King, Aug 24, 2013
  20. 2/6 sha1-lookup: handle duplicate keys with GIT_USE_LOOKUPJeff King, Aug 24, 2013
  21. 3/6 add tests for indexing packs with delta cyclesJeff King, Aug 24, 2013
  22. 4/6 test index-pack on packs with recoverable delta cyclesJeff King, Aug 24, 2013
  23. 5/6 index-pack: optionally reject packs with duplicate objectsJeff King, Aug 24, 2013
  24. 6/6 default pack.indexDuplicates to falseJeff King, Aug 24, 2013
  25. Johannes SixtAug 23, 2013
  26. Jeff KingAug 23, 2013
  27. 3/6 add tests for indexing packs with delta cyclesJeff King, Aug 22, 2013
  28. 4/6 test index-pack on packs with recoverable delta cyclesJeff King, Aug 22, 2013
  29. 5/6 index-pack: optionally reject packs with duplicate objectsJeff King, Aug 22, 2013
  30. 6/6 default pack.indexDuplicates to falseJeff King, Aug 22, 2013
  31. Nicolas PitreAug 22, 2013
  32. 3/4 reject duplicates when indexing a packfile we createdJeff King, Aug 21, 2013
  33. 4/4 index-pack: optionally skip duplicate packfile entriesJeff King, Aug 21, 2013
  34. Junio C HamanoAug 21, 2013
  35. Jeff KingAug 22, 2013
  36. Junio C HamanoAug 21, 2013
  37. Nicolas PitreAug 14, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.