git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 2/8] midx-write.c: reduce argument count for `get_sorted_entries()`

From
Taylor Blau <me@ttaylorr.com>
Date
May 29, 2024, 22:55 UTC
Message-ID
<9d422efe6e8eea13f48d3f52e4c69c94f8e72114.1717023301.git.me@ttaylorr.com>
In-Reply-To
<cover.1717023301.git.me@ttaylorr.com>

The function `midx-write.c::get_sorted_entries()` is responsible for constructing the array of OIDs from a given list of packs which will comprise the MIDX being written.

The singular call-site for this function looks something like:
    ctx.entries = get_sorted_entries(ctx.m, ctx.info, ctx.nr,
                                     &ctx.entries_nr,
                                     ctx.preferred_pack_idx);

This function has five formal arguments, all of which are members of the shared `struct write_midx_context` used to track various pieces of information about the MIDX being written.

The function `get_sorted_entries()` dates back to fe1ed56f5e4 (midx: sort and deduplicate objects from packfiles, 2018-07-12), which came shortly after 396f257018a (multi-pack-index: read packfile list, 2018-07-12). The latter patch introduced the `pack_list` structure, which was a precursor to the structure we now know as `write_midx_context` (c.f. 577dc49696a (midx: rename pack_info to write_midx_context, 2021-02-18)).

At the time, `get_sorted_entries()` likely could have used the pack_list structure introduced earlier in 396f257018a, but understandably did not since the structure only contained three fields (only two of which were relevant to `get_sorted_entries()`) at the time.

Simplify the declaration of this function by taking a single pointer to the whole `struct write_midx_context` instead of various members within it. Since this function is now computing the entire result (populating both `ctx->entries`, and `ctx->entries_nr`), rename it to something that doesn't start with "get_" to make clear that this function has a side-effect.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 midx-write.c | 45 +++++++++++++++++++--------------------------
 1 file changed, 19 insertions(+), 26 deletions(-)
diff --git a/midx-write.c b/midx-write.c
index 591b79bcbf..15965ceb70 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -299,21 +299,16 @@ static void midx_fanout_add_pack_fanout(struct midx_fanout *fanout,
  * Copy only the de-duplicated entries (selected by most-recent modified time
  * of a packfile containing the object).
  */
-static struct pack_midx_entry *get_sorted_entries(struct multi_pack_index *m,
-						  struct pack_info *info,
-						  uint32_t nr_packs,
-						  size_t *nr_objects,
-						  int preferred_pack)
+static void compute_sorted_entries(struct write_midx_context *ctx)
 {
 	uint32_t cur_fanout, cur_pack, cur_object;
 	size_t alloc_objects, total_objects = 0;
 	struct midx_fanout fanout = { 0 };
-	struct pack_midx_entry *deduplicated_entries = NULL;
-	uint32_t start_pack = m ? m->num_packs : 0;
+	uint32_t start_pack = ctx->m ? ctx->m->num_packs : 0;
 
-	for (cur_pack = start_pack; cur_pack < nr_packs; cur_pack++)
+	for (cur_pack = start_pack; cur_pack < ctx->nr; cur_pack++)
 		total_objects = st_add(total_objects,
-				       info[cur_pack].p->num_objects);
+				       ctx->info[cur_pack].p->num_objects);
 
 	/*
 	 * As we de-duplicate by fanout value, we expect the fanout
@@ -323,26 +318,26 @@ static struct pack_midx_entry *get_sorted_entries(struct multi_pack_index *m,
 	alloc_objects = fanout.alloc = total_objects > 3200 ? total_objects / 200 : 16;
 
 	ALLOC_ARRAY(fanout.entries, fanout.alloc);
-	ALLOC_ARRAY(deduplicated_entries, alloc_objects);
-	*nr_objects = 0;
+	ALLOC_ARRAY(ctx->entries, alloc_objects);
+	ctx->entries_nr = 0;
 
 	for (cur_fanout = 0; cur_fanout < 256; cur_fanout++) {
 		fanout.nr = 0;
 
-		if (m)
-			midx_fanout_add_midx_fanout(&fanout, m, cur_fanout,
-						    preferred_pack);
+		if (ctx->m)
+			midx_fanout_add_midx_fanout(&fanout, ctx->m, cur_fanout,
+						    ctx->preferred_pack_idx);
 
-		for (cur_pack = start_pack; cur_pack < nr_packs; cur_pack++) {
-			int preferred = cur_pack == preferred_pack;
+		for (cur_pack = start_pack; cur_pack < ctx->nr; cur_pack++) {
+			int preferred = cur_pack == ctx->preferred_pack_idx;
 			midx_fanout_add_pack_fanout(&fanout,
-						    info, cur_pack,
+						    ctx->info, cur_pack,
 						    preferred, cur_fanout);
 		}
 
-		if (-1 < preferred_pack && preferred_pack < start_pack)
-			midx_fanout_add_pack_fanout(&fanout, info,
-						    preferred_pack, 1,
+		if (-1 < ctx->preferred_pack_idx && ctx->preferred_pack_idx < start_pack)
+			midx_fanout_add_pack_fanout(&fanout, ctx->info,
+						    ctx->preferred_pack_idx, 1,
 						    cur_fanout);
 
 		midx_fanout_sort(&fanout);
@@ -356,17 +351,16 @@ static struct pack_midx_entry *get_sorted_entries(struct multi_pack_index *m,
 						&fanout.entries[cur_object].oid))
 				continue;
 
-			ALLOC_GROW(deduplicated_entries, st_add(*nr_objects, 1),
+			ALLOC_GROW(ctx->entries, st_add(ctx->entries_nr, 1),
 				   alloc_objects);
-			memcpy(&deduplicated_entries[*nr_objects],
+			memcpy(&ctx->entries[ctx->entries_nr],
 			       &fanout.entries[cur_object],
 			       sizeof(struct pack_midx_entry));
-			(*nr_objects)++;
+			ctx->entries_nr++;
 		}
 	}
 
 	free(fanout.entries);
-	return deduplicated_entries;
 }
 
 static int write_midx_pack_names(struct hashfile *f, void *data)
@@ -1060,8 +1054,7 @@ static int write_midx_internal(const char *object_dir,
 		}
 	}
 
-	ctx.entries = get_sorted_entries(ctx.m, ctx.info, ctx.nr, &ctx.entries_nr,
-					 ctx.preferred_pack_idx);
+	compute_sorted_entries(&ctx);
 
 	ctx.large_offsets_needed = 0;
 	for (i = 0; i < ctx.entries_nr; i++) {
-- 
2.45.1.321.gbcadaf92783
Previous: Taylor BlauNext: Taylor Blau
Message 22 of 34 in “midx-write: miscellaneous clean-ups for incremental MIDXs”
  1. 0/8 midx-write: miscellaneous clean-ups for incremental MIDXsTaylor Blau, May 23, 2024
  2. 1/8 midx-write.c: tolerate `--preferred-pack` without bitmapsTaylor Blau, May 23, 2024
  3. Patrick SteinhardtMay 29, 2024
  4. Taylor BlauMay 29, 2024
  5. 2/8 midx-write.c: reduce argument count for `get_sorted_entries()`Taylor Blau, May 23, 2024
  6. Patrick SteinhardtMay 29, 2024
  7. Taylor BlauMay 29, 2024
  8. 3/8 midx-write.c: pass `start_pack` to `get_sorted_entries()`Taylor Blau, May 23, 2024
  9. Patrick SteinhardtMay 29, 2024
  10. Taylor BlauMay 29, 2024
  11. 4/8 midx-write.c: extract `should_include_pack()`Taylor Blau, May 23, 2024
  12. Patrick SteinhardtMay 29, 2024
  13. Taylor BlauMay 29, 2024
  14. 5/8 midx-write.c: extract `fill_packs_from_midx()`Taylor Blau, May 23, 2024
  15. 6/8 midx-write.c: support reading an existing MIDX with `packs_to_include`Taylor Blau, May 23, 2024
  16. Patrick SteinhardtMay 29, 2024
  17. Taylor BlauMay 29, 2024
  18. 7/8 midx: replace `get_midx_rev_filename()` with a generic helperTaylor Blau, May 23, 2024
  19. 8/8 pack-bitmap.c: reimplement `midx_bitmap_filename()` with helperTaylor Blau, May 23, 2024
  20. 0/8 midx-write: miscellaneous clean-ups for incremental MIDXsTaylor Blau, May 29, 2024
  21. 1/8 midx-write.c: tolerate `--preferred-pack` without bitmapsTaylor Blau, May 29, 2024
  22. 2/8 midx-write.c: reduce argument count for `get_sorted_entries()`Taylor Blau, May 29, 2024
  23. 3/8 midx-write.c: pass `start_pack` to `compute_sorted_entries()`Taylor Blau, May 29, 2024
  24. Jeff KingMay 30, 2024
  25. 4/8 midx-write.c: extract `should_include_pack()`Taylor Blau, May 29, 2024
  26. 5/8 midx-write.c: extract `fill_packs_from_midx()`Taylor Blau, May 29, 2024
  27. 6/8 midx-write.c: support reading an existing MIDX with `packs_to_include`Taylor Blau, May 29, 2024
  28. Jeff KingMay 30, 2024
  29. 7/8 midx: replace `get_midx_rev_filename()` with a generic helperTaylor Blau, May 29, 2024
  30. Jeff KingMay 30, 2024
  31. 8/8 pack-bitmap.c: reimplement `midx_bitmap_filename()` with helperTaylor Blau, May 29, 2024
  32. Jeff KingMay 30, 2024
  33. Taylor BlauMay 30, 2024
  34. Patrick SteinhardtMay 31, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.