git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: ds/midx-with-less-memory

From
Derrick Stolee <derrickstolee@github.com>
Date
Jul 27, 2022, 11:29 UTC
Message-ID
<1b2f7486-faeb-c71e-76c4-1afce2a1519d@github.com>
In-Reply-To
<YuBlD3o5bYMg8f6i@coredump.intra.peff.net>
On 7/26/22 6:05 PM, Jeff King wrote:
Show 24 quoted lines
> On Fri, Jul 22, 2022 at 06:01:10PM -0700, Junio C Hamano wrote:
> 
>> * ds/midx-with-less-memory (2022-07-19) 3 commits
>>   (merged to 'next' on 2022-07-20 at 250d257c3e)
>>  + midx: reduce memory pressure while writing bitmaps
>>  + midx: extract bitmap write setup
>>  + pack-bitmap-write: use const for hashes
>>
>>  The codepath to write multi-pack index has been taught to release a
>>  large chunk of memory that holds an array of objects in the packs,
>>  as soon as it is done with the array, to reduce memory consumption.
>>
>>  Will merge to 'master'.
>>  source: <pull.1292.v2.git.1658244366.gitgitgadget@gmail.com>
> 
> This topic looks very nice, but I think we might want this additional
> cleanup on top:
> 
> -- >8 --
> Subject: [PATCH] write_midx_bitmap(): drop unused refs_snapshot parameter
> 
> The refactoring in 90b2bb710d (midx: extract bitmap write setup,
> 2022-07-19) hoisted our call to find_commits_for_midx_bitmap() into the
> caller, which means we no longer need to see the refs_snapshot at all.

Good find! I'm happy to see the unused parameter work is coming along ;)

Show 17 quoted lines
> @@ -1059,7 +1059,6 @@ static int write_midx_bitmap(const char *midx_name,
>  			     struct commit **commits,
>  			     uint32_t commits_nr,
>  			     uint32_t *pack_order,
> -			     const char *refs_snapshot,
>  			     unsigned flags)
>  {
>  	int ret, i;
> @@ -1462,7 +1461,7 @@ static int write_midx_internal(const char *object_dir,
>  
>  		if (write_midx_bitmap(midx_name.buf, midx_hash, &pdata,
>  				      commits, commits_nr, ctx.pack_order,
> -				      refs_snapshot, flags) < 0) {
> +				      flags) < 0) {
>  			error(_("could not write multi-pack bitmap"));
>  			result = 1;
>  			goto cleanup;
Thanks for the patch!
-Stolee
Previous: Jeff KingNext: Junio C Hamano
Message 7 of 9 in “What's cooking in git.git (Jul 2022, #07; Fri, 22)”
  1. Junio C HamanoJul 23, 2022
  2. en/merge-restore-to-pristine (Was: Re: What's cooking in git.git (Jul 2022, #07; Fri, 22))Elijah Newren, Jul 23, 2022
  3. mt/doc-config (Was: Re: What's cooking in git.git (Jul 2022, #07; Fri, 22))Matheus Tavares, Jul 23, 2022
  4. tb/commit-graph-genv2-upgrade-fixTaylor Blau, Jul 25, 2022
  5. Derrick StoleeJul 26, 2022
  6. Jeff KingJul 26, 2022
  7. Derrick StoleeJul 27, 2022
  8. Junio C HamanoJul 27, 2022
  9. Jeff KingJul 27, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.