git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 3/8] midx-write.c: pass `start_pack` to `compute_sorted_entries()`

From
Jeff King <peff@peff.net>
Date
May 30, 2024, 06:59 UTC
Message-ID
<20240530065943.GA1949834@coredump.intra.peff.net>
In-Reply-To
<e81296f8ccf8b47758d0ec4fbb04f0f2b356b490.1717023301.git.me@ttaylorr.com>
On Wed, May 29, 2024 at 06:55:28PM -0400, Taylor Blau wrote:
Show 13 quoted lines
> The function `compute_sorted_entries()` is broadly responsible for
> building an array of the objects to be written into a MIDX based on the
> provided list of packs.
> 
> If we have loaded an existing MIDX, however, we may not use all of its
> packs, despite loading them into the ctx->info array.
> 
> The existing implementation simply skips past the first
> ctx->m->num_packs (if ctx->m is non-NULL, indicating that we loaded an
> existing MIDX). This is because we read objects in packs from an
> existing MIDX via the MIDX itself, rather than from the pack-level
> fanout to guarantee a de-duplicated result (see: a40498a1265 (midx: use
> existing midx when writing new one, 2018-07-12)).

Hmm, if the last patch had not removed the separate array/nr pair for packs, then you could just pass (array + start_pack, nr - start_pack). :)

But I think it is probably reasonable to keep the notion of "here are all the packs" and "you are skipping some of them" clear through the call stack anyway. Especially if the notion of start_pack gets more complicated:

> Future changes (outside the scope of this patch series) to the MIDX code
> will require us to skip *at most* that number[^1].
So this patch seems fine to me.
-Peff
Previous: Taylor BlauNext: Taylor Blau
Message 24 of 34 in “midx-write: miscellaneous clean-ups for incremental MIDXs”
  1. 0/8 midx-write: miscellaneous clean-ups for incremental MIDXsTaylor Blau, May 23, 2024
  2. 1/8 midx-write.c: tolerate `--preferred-pack` without bitmapsTaylor Blau, May 23, 2024
  3. Patrick SteinhardtMay 29, 2024
  4. Taylor BlauMay 29, 2024
  5. 2/8 midx-write.c: reduce argument count for `get_sorted_entries()`Taylor Blau, May 23, 2024
  6. Patrick SteinhardtMay 29, 2024
  7. Taylor BlauMay 29, 2024
  8. 3/8 midx-write.c: pass `start_pack` to `get_sorted_entries()`Taylor Blau, May 23, 2024
  9. Patrick SteinhardtMay 29, 2024
  10. Taylor BlauMay 29, 2024
  11. 4/8 midx-write.c: extract `should_include_pack()`Taylor Blau, May 23, 2024
  12. Patrick SteinhardtMay 29, 2024
  13. Taylor BlauMay 29, 2024
  14. 5/8 midx-write.c: extract `fill_packs_from_midx()`Taylor Blau, May 23, 2024
  15. 6/8 midx-write.c: support reading an existing MIDX with `packs_to_include`Taylor Blau, May 23, 2024
  16. Patrick SteinhardtMay 29, 2024
  17. Taylor BlauMay 29, 2024
  18. 7/8 midx: replace `get_midx_rev_filename()` with a generic helperTaylor Blau, May 23, 2024
  19. 8/8 pack-bitmap.c: reimplement `midx_bitmap_filename()` with helperTaylor Blau, May 23, 2024
  20. 0/8 midx-write: miscellaneous clean-ups for incremental MIDXsTaylor Blau, May 29, 2024
  21. 1/8 midx-write.c: tolerate `--preferred-pack` without bitmapsTaylor Blau, May 29, 2024
  22. 2/8 midx-write.c: reduce argument count for `get_sorted_entries()`Taylor Blau, May 29, 2024
  23. 3/8 midx-write.c: pass `start_pack` to `compute_sorted_entries()`Taylor Blau, May 29, 2024
  24. Jeff KingMay 30, 2024
  25. 4/8 midx-write.c: extract `should_include_pack()`Taylor Blau, May 29, 2024
  26. 5/8 midx-write.c: extract `fill_packs_from_midx()`Taylor Blau, May 29, 2024
  27. 6/8 midx-write.c: support reading an existing MIDX with `packs_to_include`Taylor Blau, May 29, 2024
  28. Jeff KingMay 30, 2024
  29. 7/8 midx: replace `get_midx_rev_filename()` with a generic helperTaylor Blau, May 29, 2024
  30. Jeff KingMay 30, 2024
  31. 8/8 pack-bitmap.c: reimplement `midx_bitmap_filename()` with helperTaylor Blau, May 29, 2024
  32. Jeff KingMay 30, 2024
  33. Taylor BlauMay 30, 2024
  34. Patrick SteinhardtMay 31, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.