git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 04/11] midx: simplify computation of pack name lengths

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Jun 10, 2019, 23:35 UTC
Message-ID
<dec7f384eea97511224ae5a77a82ed9a4330872c.1560209720.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.261.git.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>

Before writing the multi-pack-index, we compute the length of the pack-index names concatenated together. This forms the data in the pack name chunk, and we precompute it to compute chunk offsets. The value is also modified to fit alignment needs.

Previously, this computation was coupled with adding packs from the existing multi-pack-index and the remaining packs in the object dir not already covered by the multi-pack-index.

In anticipation of this becoming more complicated with the 'expire' subcommand, simplify the computation by centralizing it to a single loop before writing the file.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 midx.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/midx.c b/midx.c
index 3b7da1a360..62404620ad 100644
--- a/midx.c
+++ b/midx.c
@@ -433,7 +433,6 @@ struct pack_list {
 	uint32_t nr;
 	uint32_t alloc_list;
 	uint32_t alloc_names;
-	size_t pack_name_concat_len;
 	struct multi_pack_index *m;
 };
 
@@ -468,7 +467,6 @@ static void add_pack_to_midx(const char *full_path, size_t full_path_len,
 		}
 
 		packs->names[packs->nr] = xstrdup(file_name);
-		packs->pack_name_concat_len += strlen(file_name) + 1;
 		packs->nr++;
 	}
 }
@@ -812,6 +810,7 @@ int write_midx_file(const char *object_dir)
 	uint32_t nr_entries, num_large_offsets = 0;
 	struct pack_midx_entry *entries = NULL;
 	int large_offsets_needed = 0;
+	int pack_name_concat_len = 0;
 
 	midx_name = get_midx_filename(object_dir);
 	if (safe_create_leading_directories(midx_name)) {
@@ -827,7 +826,6 @@ int write_midx_file(const char *object_dir)
 	packs.alloc_names = packs.alloc_list;
 	packs.list = NULL;
 	packs.names = NULL;
-	packs.pack_name_concat_len = 0;
 	ALLOC_ARRAY(packs.list, packs.alloc_list);
 	ALLOC_ARRAY(packs.names, packs.alloc_names);
 
@@ -838,7 +836,6 @@ int write_midx_file(const char *object_dir)
 
 			packs.list[packs.nr] = NULL;
 			packs.names[packs.nr] = xstrdup(packs.m->pack_names[i]);
-			packs.pack_name_concat_len += strlen(packs.names[packs.nr]) + 1;
 			packs.nr++;
 		}
 	}
@@ -848,10 +845,6 @@ int write_midx_file(const char *object_dir)
 	if (packs.m && packs.nr == packs.m->num_packs)
 		goto cleanup;
 
-	if (packs.pack_name_concat_len % MIDX_CHUNK_ALIGNMENT)
-		packs.pack_name_concat_len += MIDX_CHUNK_ALIGNMENT -
-					      (packs.pack_name_concat_len % MIDX_CHUNK_ALIGNMENT);
-
 	ALLOC_ARRAY(pack_perm, packs.nr);
 	sort_packs_by_name(packs.names, packs.nr, pack_perm);
 
@@ -864,6 +857,13 @@ int write_midx_file(const char *object_dir)
 			large_offsets_needed = 1;
 	}
 
+	for (i = 0; i < packs.nr; i++)
+		pack_name_concat_len += strlen(packs.names[i]) + 1;
+
+	if (pack_name_concat_len % MIDX_CHUNK_ALIGNMENT)
+		pack_name_concat_len += MIDX_CHUNK_ALIGNMENT -
+					(pack_name_concat_len % MIDX_CHUNK_ALIGNMENT);
+
 	hold_lock_file_for_update(&lk, midx_name, LOCK_DIE_ON_ERROR);
 	f = hashfd(lk.tempfile->fd, lk.tempfile->filename.buf);
 	FREE_AND_NULL(midx_name);
@@ -881,7 +881,7 @@ int write_midx_file(const char *object_dir)
 
 	cur_chunk++;
 	chunk_ids[cur_chunk] = MIDX_CHUNKID_OIDFANOUT;
-	chunk_offsets[cur_chunk] = chunk_offsets[cur_chunk - 1] + packs.pack_name_concat_len;
+	chunk_offsets[cur_chunk] = chunk_offsets[cur_chunk - 1] + pack_name_concat_len;
 
 	cur_chunk++;
 	chunk_ids[cur_chunk] = MIDX_CHUNKID_OIDLOOKUP;
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 9 of 29 in “Create 'expire' and 'repack' verbs for git-multi-pack-index”
  1. 00/11 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee via GitGitGadget, Jun 10, 2019
  2. 01/11 repack: refactor pack deletion for future useDerrick Stolee via GitGitGadget, Jun 10, 2019
  3. 02/11 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee via GitGitGadget, Jun 10, 2019
  4. 03/11 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee via GitGitGadget, Jun 10, 2019
  5. 05/11 midx: refactor permutation logic and pack sortingDerrick Stolee via GitGitGadget, Jun 10, 2019
  6. 10/11 midx: add test that 'expire' respects .keep filesDerrick Stolee via GitGitGadget, Jun 10, 2019
  7. 11/11 t5319-multi-pack-index.sh: test batch size zeroDerrick Stolee via GitGitGadget, Jun 10, 2019
  8. 08/11 midx: implement midx_repack()Derrick Stolee via GitGitGadget, Jun 10, 2019
  9. 04/11 midx: simplify computation of pack name lengthsDerrick Stolee via GitGitGadget, Jun 10, 2019
  10. 07/11 multi-pack-index: prepare 'repack' subcommandDerrick Stolee via GitGitGadget, Jun 10, 2019
  11. 09/11 multi-pack-index: test expire while adding packsDerrick Stolee via GitGitGadget, Jun 10, 2019
  12. 06/11 multi-pack-index: implement 'expire' subcommandDerrick Stolee via GitGitGadget, Jun 10, 2019
  13. Junio C HamanoJun 11, 2019
  14. t5319: don't trip over a user name with whitespaceJohannes Sixt, Jun 30, 2019
  15. Eric SunshineJun 30, 2019
  16. Johannes SixtJun 30, 2019
  17. Jeff KingJun 30, 2019
  18. Johannes SixtJul 1, 2019
  19. Jeff KingJul 1, 2019
  20. SZEDER GáborJul 1, 2019
  21. Derrick StoleeJul 1, 2019
  22. Johannes SchindelinJul 1, 2019
  23. Derrick StoleeJul 1, 2019
  24. Johannes SixtJul 1, 2019
  25. Derrick StoleeJul 1, 2019
  26. Jeff KingJul 1, 2019
  27. SZEDER GáborJul 1, 2019
  28. Andreas SchwabJul 1, 2019
  29. Junio C HamanoJul 1, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.