git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: ds/midx-with-less-memory

From
Jeff King <peff@peff.net>
Date
Jul 26, 2022, 22:05 UTC
Message-ID
<YuBlD3o5bYMg8f6i@coredump.intra.peff.net>
In-Reply-To
<xmqqh738bq21.fsf@gitster.g>
On Fri, Jul 22, 2022 at 06:01:10PM -0700, Junio C Hamano wrote:
Show 12 quoted lines
> * ds/midx-with-less-memory (2022-07-19) 3 commits
>   (merged to 'next' on 2022-07-20 at 250d257c3e)
>  + midx: reduce memory pressure while writing bitmaps
>  + midx: extract bitmap write setup
>  + pack-bitmap-write: use const for hashes
> 
>  The codepath to write multi-pack index has been taught to release a
>  large chunk of memory that holds an array of objects in the packs,
>  as soon as it is done with the array, to reduce memory consumption.
> 
>  Will merge to 'master'.
>  source: <pull.1292.v2.git.1658244366.gitgitgadget@gmail.com>

This topic looks very nice, but I think we might want this additional cleanup on top:

-- >8 --
Subject: [PATCH] write_midx_bitmap(): drop unused refs_snapshot parameter

The refactoring in 90b2bb710d (midx: extract bitmap write setup, 2022-07-19) hoisted our call to find_commits_for_midx_bitmap() into the caller, which means we no longer need to see the refs_snapshot at all.

Signed-off-by: Jeff King <peff@peff.net>
---
 midx.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/midx.c b/midx.c
index 772ab7d294..4e956cacb7 100644
--- a/midx.c
+++ b/midx.c
@@ -1059,7 +1059,6 @@ static int write_midx_bitmap(const char *midx_name,
 			     struct commit **commits,
 			     uint32_t commits_nr,
 			     uint32_t *pack_order,
-			     const char *refs_snapshot,
 			     unsigned flags)
 {
 	int ret, i;
@@ -1462,7 +1461,7 @@ static int write_midx_internal(const char *object_dir,
 
 		if (write_midx_bitmap(midx_name.buf, midx_hash, &pdata,
 				      commits, commits_nr, ctx.pack_order,
-				      refs_snapshot, flags) < 0) {
+				      flags) < 0) {
 			error(_("could not write multi-pack bitmap"));
 			result = 1;
 			goto cleanup;
-- 
2.37.1.641.g73229dff8a
Previous: Derrick StoleeNext: Derrick Stolee
Message 6 of 9 in “What's cooking in git.git (Jul 2022, #07; Fri, 22)”
  1. Junio C HamanoJul 23, 2022
  2. en/merge-restore-to-pristine (Was: Re: What's cooking in git.git (Jul 2022, #07; Fri, 22))Elijah Newren, Jul 23, 2022
  3. mt/doc-config (Was: Re: What's cooking in git.git (Jul 2022, #07; Fri, 22))Matheus Tavares, Jul 23, 2022
  4. tb/commit-graph-genv2-upgrade-fixTaylor Blau, Jul 25, 2022
  5. Derrick StoleeJul 26, 2022
  6. Jeff KingJul 26, 2022
  7. Derrick StoleeJul 27, 2022
  8. Junio C HamanoJul 27, 2022
  9. Jeff KingJul 27, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.