git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 05/13] midx: check both pack and index names for containment

From
Jeff King <peff@peff.net>
Date
Apr 5, 2019, 18:06 UTC
Message-ID
<20190405180604.GE32243@sigill.intra.peff.net>
In-Reply-To
<20190405180306.GA21113@sigill.intra.peff.net>

A midx file (and the struct we parse from it) contains a list of all of the covered packfiles, mentioned by their ".idx" names (e.g., "pack-1234.idx", etc). And thus calls to midx_contains_pack() expect callers to provide the idx name.

This works for most of the calls, but the one in open_packed_git_1() tries to feed a packed_git->pack_name, which is the ".pack" name, meaning we'll never find a match (even if the pack is covered by the midx).

We can fix this by converting the ".pack" to ".idx" in the caller. However, that requires allocating a new string. Instead, let's make midx_contains_pack() a bit friendlier, and allow it take _either_ the .pack or .idx variant.

All cleverness in the matching code is credited to René. Bugs are mine.

There's no test here, because while this does fix _a_ bug, it's masked by another bug in that same caller. That will be covered (with a test) in the next patch.

Helped-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Jeff King <peff@peff.net>
---
I was tempted to suggest that the midx struct just store the base name
without ".idx" at all, but having callers pass that is no less tricky
than passing ".idx" (they still have to allocate a new string).
 midx.c | 36 ++++++++++++++++++++++++++++++++++--
 midx.h |  2 +-
 2 files changed, 35 insertions(+), 3 deletions(-)
diff --git a/midx.c b/midx.c
index 8a505fd423..0ceca1938f 100644
--- a/midx.c
+++ b/midx.c
@@ -307,7 +307,39 @@ int fill_midx_entry(const struct object_id *oid, struct pack_entry *e, struct mu
 	return nth_midxed_pack_entry(m, e, pos);
 }
 
-int midx_contains_pack(struct multi_pack_index *m, const char *idx_name)
+/* Match "foo.idx" against either "foo.pack" _or_ "foo.idx". */
+static int cmp_idx_or_pack_name(const char *idx_or_pack_name,
+				const char *idx_name)
+{
+	/* Skip past any initial matching prefix. */
+	while (*idx_name && *idx_name == *idx_or_pack_name) {
+		idx_name++;
+		idx_or_pack_name++;
+	}
+
+	/*
+	 * If we didn't match completely, we may have matched "pack-1234." and
+	 * be left with "idx" and "pack" respectively, which is also OK. We do
+	 * not have to check for "idx" and "idx", because that would have been
+	 * a complete match (and in that case these strcmps will be false, but
+	 * we'll correctly return 0 from the final strcmp() below.
+	 *
+	 * Technically this matches "fooidx" and "foopack", but we'd never have
+	 * such names in the first place.
+	 */
+	if (!strcmp(idx_name, "idx") && !strcmp(idx_or_pack_name, "pack"))
+		return 0;
+
+	/*
+	 * This not only checks for a complete match, but also orders based on
+	 * the first non-identical character, which means our ordering will
+	 * match a raw strcmp(). That makes it OK to use this to binary search
+	 * a naively-sorted list.
+	 */
+	return strcmp(idx_or_pack_name, idx_name);
+}
+
+int midx_contains_pack(struct multi_pack_index *m, const char *idx_or_pack_name)
 {
 	uint32_t first = 0, last = m->num_packs;
 
@@ -317,7 +349,7 @@ int midx_contains_pack(struct multi_pack_index *m, const char *idx_name)
 		int cmp;
 
 		current = m->pack_names[mid];
-		cmp = strcmp(idx_name, current);
+		cmp = cmp_idx_or_pack_name(idx_or_pack_name, current);
 		if (!cmp)
 			return 1;
 		if (cmp > 0) {
diff --git a/midx.h b/midx.h
index 774f652530..26dd042d63 100644
--- a/midx.h
+++ b/midx.h
@@ -43,7 +43,7 @@ struct object_id *nth_midxed_object_oid(struct object_id *oid,
 					struct multi_pack_index *m,
 					uint32_t n);
 int fill_midx_entry(const struct object_id *oid, struct pack_entry *e, struct multi_pack_index *m);
-int midx_contains_pack(struct multi_pack_index *m, const char *idx_name);
+int midx_contains_pack(struct multi_pack_index *m, const char *idx_or_pack_name);
 int prepare_multi_pack_index_one(struct repository *r, const char *object_dir, int local);
 
 int write_midx_file(const char *object_dir);
-- 
2.21.0.729.g7d31bf3764
Previous: Jeff KingNext: René Scharfe
Message 37 of 48 in “a rabbit hole of update-server-info fixes”
  1. 0/12 a rabbit hole of update-server-info fixesJeff King, Apr 4, 2019
  2. 01/12 t5319: fix bogus cat-file argumentJeff King, Apr 4, 2019
  3. Ramsay JonesApr 5, 2019
  4. Jeff KingApr 5, 2019
  5. Jeff KingApr 5, 2019
  6. 02/12 t5319: drop useless --buffer from cat-fileJeff King, Apr 4, 2019
  7. 03/12 packfile: factor out .pack to .idx name conversionJeff King, Apr 4, 2019
  8. 04/12 packfile: check midx coverage with .idx rather than .packJeff King, Apr 4, 2019
  9. René ScharfeApr 5, 2019
  10. Jeff KingApr 5, 2019
  11. Jeff KingApr 5, 2019
  12. SZEDER GáborApr 5, 2019
  13. Jeff KingApr 5, 2019
  14. 05/12 http: simplify parsing of remote objects/info/packsJeff King, Apr 4, 2019
  15. René ScharfeApr 5, 2019
  16. Jeff KingApr 5, 2019
  17. René ScharfeApr 5, 2019
  18. 06/12 server-info: fix blind pointer arithmeticJeff King, Apr 4, 2019
  19. 07/12 server-info: simplify cleanup in parse_pack_def()Jeff King, Apr 4, 2019
  20. 08/12 server-info: use strbuf to read old info/packs fileJeff King, Apr 4, 2019
  21. 09/12 server-info: drop nr_alloc struct memberJeff King, Apr 4, 2019
  22. 10/12 packfile.h: drop extern from function declarationsJeff King, Apr 4, 2019
  23. 11/12 server-info: drop objdirlen pointer arithmeticJeff King, Apr 4, 2019
  24. 12/12 update_info_refs(): drop unused force parameterJeff King, Apr 4, 2019
  25. Junio C HamanoApr 5, 2019
  26. 0/13 a rabbit hole of update-server-info (and midx!) fixesJeff King, Apr 5, 2019
  27. 01/13 packfile.h: drop extern from function declarationsJeff King, Apr 5, 2019
  28. Ramsay JonesApr 5, 2019
  29. Jeff KingApr 5, 2019
  30. Junio C HamanoApr 8, 2019
  31. Jeff KingApr 8, 2019
  32. Junio C HamanoApr 9, 2019
  33. Jeff KingApr 9, 2019
  34. 02/13 pack-revindex: open index if necessaryJeff King, Apr 5, 2019
  35. 03/13 t5319: fix bogus cat-file argumentJeff King, Apr 5, 2019
  36. 04/13 t5319: drop useless --buffer from cat-fileJeff King, Apr 5, 2019
  37. 05/13 midx: check both pack and index names for containmentJeff King, Apr 5, 2019
  38. René ScharfeApr 5, 2019
  39. 06/13 packfile: fix pack basename computationJeff King, Apr 5, 2019
  40. 07/13 http: simplify parsing of remote objects/info/packsJeff King, Apr 5, 2019
  41. 08/13 server-info: fix blind pointer arithmeticJeff King, Apr 5, 2019
  42. 09/13 server-info: simplify cleanup in parse_pack_def()Jeff King, Apr 5, 2019
  43. Jeff KingApr 5, 2019
  44. 10/13 server-info: use strbuf to read old info/packs fileJeff King, Apr 5, 2019
  45. 11/13 server-info: drop nr_alloc struct memberJeff King, Apr 5, 2019
  46. 12/13 server-info: drop objdirlen pointer arithmeticJeff King, Apr 5, 2019
  47. 13/13 update_info_refs(): drop unused force parameterJeff King, Apr 5, 2019
  48. Jeff KingApr 5, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.