git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/9] some more chunk-file bounds-checks fixes

From
Jeff King <peff@peff.net>
Date
Nov 9, 2023, 07:03 UTC
Message-ID
<20231109070310.GA2697602@coredump.intra.peff.net>
This is a follow-up to the series from:
  https://lore.kernel.org/git/20231009205544.GA3281950@coredump.intra.peff.net/

which was merged to master as jk/chunk-bounds. There were a few issues left open by that series and its review:

  1. the midx code didn't check fanout ordering
  2. whether we needed to sprinkle some more st_mult() on it
  3. improving some of the error messages (translations, some
     consistency, maybe more details)
  4. possible refactoring with a pair_chunk_expect() API (Taylor posted
     a series in that direction, which is currently in limbo)

The patches here fix the remaining correctness issues (points 1 and 2, along with a few other small issues I found). There's some improvement on 3, but I stopped short of adding lots more details. Partially because the series was already getting big, and partially because going further may depend on what we do with 4.

With regards to the pair_chunk_expect() thing, this series is incompatible (textually, but not conceptually) with what Taylor posted earlier, just because I'm moving some checks into the chunk-reader callbacks. Because it's fixing user-visible bugs, I think we'd want to do this first, and then (possibly) rebase Taylor's series on top. But I also think some of the things I noticed around overflow (especially patches 1 and 6) may inform how we'd want the pair_chunk_expect() API to look.

This is a continuation of the jk/chunk-bounds topic, which is new in the v2.43 cycle. But it should be OK to leave this until after the release. Nothing here is fixing a regression in the 2.43 release candidates; it's just a few bits that were incomplete. That said, I did try to float the correctness bits to the first two patches just in case. ;)

  [1/9]: commit-graph: handle overflow in chunk_size checks
  [2/9]: midx: check consistency of fanout table
  [3/9]: commit-graph: drop redundant call to "lite" verification
  [4/9]: commit-graph: clarify missing-chunk error messages
  [5/9]: commit-graph: abort as soon as we see a bogus chunk
  [6/9]: commit-graph: use fanout value for graph size
  [7/9]: commit-graph: check order while reading fanout chunk
  [8/9]: commit-graph: drop verify_commit_graph_lite()
  [9/9]: commit-graph: mark chunk error messages for translation
 commit-graph.c              | 94 +++++++++++++------------------------
 midx.c                      | 20 ++++----
 t/t5318-commit-graph.sh     | 16 ++++---
 t/t5319-multi-pack-index.sh | 14 ++++++
 4 files changed, 67 insertions(+), 77 deletions(-)
-Peff
Next: Jeff King
Message 1 of 39 in “some more chunk-file bounds-checks fixes”
  1. 0/9 some more chunk-file bounds-checks fixesJeff King, Nov 9, 2023
  2. 1/9 commit-graph: handle overflow in chunk_size checksJeff King, Nov 9, 2023
  3. Taylor BlauNov 9, 2023
  4. Jeff KingNov 9, 2023
  5. 2/9 midx: check consistency of fanout tableJeff King, Nov 9, 2023
  6. 3/9 commit-graph: drop redundant call to "lite" verificationJeff King, Nov 9, 2023
  7. 4/9 commit-graph: clarify missing-chunk error messagesJeff King, Nov 9, 2023
  8. 5/9 commit-graph: abort as soon as we see a bogus chunkJeff King, Nov 9, 2023
  9. Taylor BlauNov 9, 2023
  10. 6/9 commit-graph: use fanout value for graph sizeJeff King, Nov 9, 2023
  11. Taylor BlauNov 9, 2023
  12. Jeff KingNov 9, 2023
  13. Taylor BlauNov 9, 2023
  14. Jeff KingNov 10, 2023
  15. 7/9 commit-graph: check order while reading fanout chunkJeff King, Nov 9, 2023
  16. 8/9 commit-graph: drop verify_commit_graph_lite()Jeff King, Nov 9, 2023
  17. 9/9 commit-graph: mark chunk error messages for translationJeff King, Nov 9, 2023
  18. Taylor BlauNov 9, 2023
  19. 0/7 chunk-format: introduce `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  20. 1/7 chunk-format: introduce `pair_chunk_expect()` helperTaylor Blau, Nov 9, 2023
  21. Junio C HamanoNov 10, 2023
  22. Taylor BlauNov 10, 2023
  23. Jeff KingNov 10, 2023
  24. Junio C HamanoNov 10, 2023
  25. Junio C HamanoNov 10, 2023
  26. Jeff KingNov 10, 2023
  27. Jeff KingNov 10, 2023
  28. Jeff KingNov 10, 2023
  29. Linus ArverJan 15, 2024
  30. Linus ArverJan 15, 2024
  31. Jeff KingJan 16, 2024
  32. Linus ArverJan 18, 2024
  33. 2/7 commit-graph: read `OIDL` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  34. Jeff KingNov 10, 2023
  35. 3/7 commit-graph: read `CDAT` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  36. 4/7 commit-graph: read `GDAT` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  37. 5/7 commit-graph: read `BIDX` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  38. 6/7 midx: read `OIDL` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  39. 7/7 midx: read `OOFF` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.