git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 6/8] midx-write.c: support reading an existing MIDX with `packs_to_include`

From
Patrick Steinhardt <ps@pks.im>
Date
May 29, 2024, 07:48 UTC
Message-ID
<ZlbdxJmIBvgV_VIF@tanuki>
In-Reply-To
<61268114c6562ba882210fd94b3f336efcb5c486.1716482279.git.me@ttaylorr.com>
On Thu, May 23, 2024 at 12:38:19PM -0400, Taylor Blau wrote:
Show 6 quoted lines
> Avoid unconditionally copying all packs from an existing MIDX into a new
> MIDX by checking that packs added via `fill_packs_from_midx()` don't
> appear in the `to_include` set, if one was provided.
> 
> Do so by calling `should_include_pack()` from both `add_pack_to_midx()`
> and `fill_packs_from_midx()`.

This is missing an explanation why exactly we want that. Is the current behaviour a bug? Is it a preparation for a future change? Is this change expected to modify any existing behaviour?

Reading through the patch we now unconditionally load the existing MIDX when writing a new one, but I'm not exactly sure what the effect of that is going to be.

[snip]
Show 34 quoted lines
> diff --git a/midx-write.c b/midx-write.c
> index 9712ac044f..36ac4ab65b 100644
> --- a/midx-write.c
> +++ b/midx-write.c
> @@ -101,27 +101,13 @@ struct write_midx_context {
>  };
>  
>  static int should_include_pack(const struct write_midx_context *ctx,
> -			       const char *file_name)
> +			       const char *file_name,
> +			       int exclude_from_midx)
>  {
> -	/*
> -	 * Note that at most one of ctx->m and ctx->to_include are set,
> -	 * so we are testing midx_contains_pack() and
> -	 * string_list_has_string() independently (guarded by the
> -	 * appropriate NULL checks).
> -	 *
> -	 * We could support passing to_include while reusing an existing
> -	 * MIDX, but don't currently since the reuse process drags
> -	 * forward all packs from an existing MIDX (without checking
> -	 * whether or not they appear in the to_include list).
> -	 *
> -	 * If we added support for that, these next two conditional
> -	 * should be performed independently (likely checking
> -	 * to_include before the existing MIDX).
> -	 */
> -	if (ctx->m && midx_contains_pack(ctx->m, file_name))
> +	if (exclude_from_midx && ctx->m && midx_contains_pack(ctx->m, file_name))
>  		return 0;
> -	else if (ctx->to_include &&
> -		 !string_list_has_string(ctx->to_include, file_name))
> +	if (ctx->to_include && !string_list_has_string(ctx->to_include,
> +						       file_name))

The second branch is a no-op change, right? The only change was that you converted from `else if` to `if`. I'd propose to either keep this as-is, or to do this change in the preceding patch already that introduces this function.

Patrick
Previous: Taylor BlauNext: Taylor Blau
Message 16 of 34 in “midx-write: miscellaneous clean-ups for incremental MIDXs”
  1. 0/8 midx-write: miscellaneous clean-ups for incremental MIDXsTaylor Blau, May 23, 2024
  2. 1/8 midx-write.c: tolerate `--preferred-pack` without bitmapsTaylor Blau, May 23, 2024
  3. Patrick SteinhardtMay 29, 2024
  4. Taylor BlauMay 29, 2024
  5. 2/8 midx-write.c: reduce argument count for `get_sorted_entries()`Taylor Blau, May 23, 2024
  6. Patrick SteinhardtMay 29, 2024
  7. Taylor BlauMay 29, 2024
  8. 3/8 midx-write.c: pass `start_pack` to `get_sorted_entries()`Taylor Blau, May 23, 2024
  9. Patrick SteinhardtMay 29, 2024
  10. Taylor BlauMay 29, 2024
  11. 4/8 midx-write.c: extract `should_include_pack()`Taylor Blau, May 23, 2024
  12. Patrick SteinhardtMay 29, 2024
  13. Taylor BlauMay 29, 2024
  14. 5/8 midx-write.c: extract `fill_packs_from_midx()`Taylor Blau, May 23, 2024
  15. 6/8 midx-write.c: support reading an existing MIDX with `packs_to_include`Taylor Blau, May 23, 2024
  16. Patrick SteinhardtMay 29, 2024
  17. Taylor BlauMay 29, 2024
  18. 7/8 midx: replace `get_midx_rev_filename()` with a generic helperTaylor Blau, May 23, 2024
  19. 8/8 pack-bitmap.c: reimplement `midx_bitmap_filename()` with helperTaylor Blau, May 23, 2024
  20. 0/8 midx-write: miscellaneous clean-ups for incremental MIDXsTaylor Blau, May 29, 2024
  21. 1/8 midx-write.c: tolerate `--preferred-pack` without bitmapsTaylor Blau, May 29, 2024
  22. 2/8 midx-write.c: reduce argument count for `get_sorted_entries()`Taylor Blau, May 29, 2024
  23. 3/8 midx-write.c: pass `start_pack` to `compute_sorted_entries()`Taylor Blau, May 29, 2024
  24. Jeff KingMay 30, 2024
  25. 4/8 midx-write.c: extract `should_include_pack()`Taylor Blau, May 29, 2024
  26. 5/8 midx-write.c: extract `fill_packs_from_midx()`Taylor Blau, May 29, 2024
  27. 6/8 midx-write.c: support reading an existing MIDX with `packs_to_include`Taylor Blau, May 29, 2024
  28. Jeff KingMay 30, 2024
  29. 7/8 midx: replace `get_midx_rev_filename()` with a generic helperTaylor Blau, May 29, 2024
  30. Jeff KingMay 30, 2024
  31. 8/8 pack-bitmap.c: reimplement `midx_bitmap_filename()` with helperTaylor Blau, May 29, 2024
  32. Jeff KingMay 30, 2024
  33. Taylor BlauMay 30, 2024
  34. Patrick SteinhardtMay 31, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.