git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 1/8] reftable/stack: do not overwrite errors when compacting

From
Patrick Steinhardt <ps@pks.im>
Date
Feb 15, 2024, 07:43 UTC
Message-ID
<Zc3AsEu0FYzU6tMX@tanuki>
In-Reply-To
<CAOw_e7b72HVQob_hiV0gtRhGWsb=rz40WL=oaV63t7oOmEA-mw@mail.gmail.com>
On Wed, Feb 14, 2024 at 04:12:54PM +0100, Han-Wen Nienhuys wrote:
> Good catch!
> 
> Sorry for messing this up.
Nothing to be sorry about, bugs happen to all of us.
Show 13 quoted lines
> > In the worst case,
> > this can lead to a compacted stack that is missing records.
> 
> Yeah, that would be an insidious corruption. Have you considered
> writing a test to reproduce this (and thus verify that the fix really
> fixes the problem?)
> 
> I think it wouldn't be too difficult: you could create a custom
> blocksource wrapper that returns I/O error on the Nth read, and then
> create a reftable with two ref blocks (could just be 2 records if you
> use a small blocksize and a large refname) and two log blocks.  Merge
> that with an empty table, and see if the compacted result is what you
> got in. Loop over N to get coverage for all error paths.

Ah, that's a feasible way to write such a test indeed. I may come back to it in the future and will add it to our backlog. Thanks!

Patrick
Previous: Han-Wen NienhuysNext: Patrick Steinhardt
Message 26 of 36 in “reftable: fixes and optimizations (pt.2)”
  1. 0/7 reftable: fixes and optimizations (pt.2)Patrick Steinhardt, Dec 20, 2023
  2. 1/7 reftable/stack: do not overwrite errors when compactingPatrick Steinhardt, Dec 20, 2023
  3. 2/7 reftable/writer: fix index corruption when writing multiple indicesPatrick Steinhardt, Dec 20, 2023
  4. 3/7 reftable/record: constify some parts of the interfacePatrick Steinhardt, Dec 20, 2023
  5. 4/7 reftable/record: store "val1" hashes as static arraysPatrick Steinhardt, Dec 20, 2023
  6. Patrick SteinhardtDec 20, 2023
  7. 5/7 reftable/record: store "val2" hashes as static arraysPatrick Steinhardt, Dec 20, 2023
  8. 6/7 reftable/merged: really reuse buffers to compute record keysPatrick Steinhardt, Dec 20, 2023
  9. 7/7 reftable/merged: transfer ownership of records when iteratingPatrick Steinhardt, Dec 20, 2023
  10. Junio C HamanoDec 20, 2023
  11. 0/8 reftable: fixes and optimizations (pt.2)Patrick Steinhardt, Dec 28, 2023
  12. 1/8 reftable/stack: do not overwrite errors when compactingPatrick Steinhardt, Dec 28, 2023
  13. 2/8 reftable/stack: do not auto-compact twice in `reftable_stack_add()`Patrick Steinhardt, Dec 28, 2023
  14. 3/8 reftable/writer: fix index corruption when writing multiple indicesPatrick Steinhardt, Dec 28, 2023
  15. 4/8 reftable/record: constify some parts of the interfacePatrick Steinhardt, Dec 28, 2023
  16. 5/8 reftable/record: store "val1" hashes as static arraysPatrick Steinhardt, Dec 28, 2023
  17. Junio C HamanoDec 28, 2023
  18. 6/8 reftable/record: store "val2" hashes as static arraysPatrick Steinhardt, Dec 28, 2023
  19. 7/8 reftable/merged: really reuse buffers to compute record keysPatrick Steinhardt, Dec 28, 2023
  20. Junio C HamanoDec 28, 2023
  21. 8/8 reftable/merged: transfer ownership of records when iteratingPatrick Steinhardt, Dec 28, 2023
  22. Junio C HamanoDec 28, 2023
  23. 0/8 reftable: fixes and optimizations (pt.2)Patrick Steinhardt, Jan 3, 2024
  24. 1/8 reftable/stack: do not overwrite errors when compactingPatrick Steinhardt, Jan 3, 2024
  25. Han-Wen NienhuysFeb 14, 2024
  26. Patrick SteinhardtFeb 15, 2024
  27. 2/8 reftable/stack: do not auto-compact twice in `reftable_stack_add()`Patrick Steinhardt, Jan 3, 2024
  28. 3/8 reftable/writer: fix index corruption when writing multiple indicesPatrick Steinhardt, Jan 3, 2024
  29. 4/8 reftable/record: constify some parts of the interfacePatrick Steinhardt, Jan 3, 2024
  30. 5/8 reftable/record: store "val1" hashes as static arraysPatrick Steinhardt, Jan 3, 2024
  31. Karthik NayakFeb 5, 2024
  32. Patrick SteinhardtFeb 6, 2024
  33. 6/8 reftable/record: store "val2" hashes as static arraysPatrick Steinhardt, Jan 3, 2024
  34. 7/8 reftable/merged: really reuse buffers to compute record keysPatrick Steinhardt, Jan 3, 2024
  35. 8/8 reftable/merged: transfer ownership of records when iteratingPatrick Steinhardt, Jan 3, 2024
  36. Karthik NayakFeb 5, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.