git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 7/8] reftable: add code to facilitate consistency checks

From
Patrick Steinhardt <ps@pks.im>
Date
Sep 25, 2025, 06:14 UTC
Message-ID
<aNTdwzUMlubjcppb@pks.im>
In-Reply-To
<CAOLa=ZQ641MncC9ACm9jfjx0WtQ+nK2shtyucQOxd08LDXDzAw@mail.gmail.com>
On Wed, Sep 24, 2025 at 11:40:31AM -0700, Karthik Nayak wrote:
Show 7 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> > On Thu, Sep 18, 2025 at 10:11:48AM +0200, Karthik Nayak wrote:
> >> diff --git a/reftable/fsck.c b/reftable/fsck.c
> >> new file mode 100644
> >> index 0000000000..785e4b43e8
> >> --- /dev/null
> >> +++ b/reftable/fsck.c
[snip]
Show 52 quoted lines
> >> +	struct reftable_fsck_info info;
> >> +	struct dirent *d = NULL;
> >> +	uint64_t min, max;
> >> +	int err = 0;
> >> +
> >> +	if (!dir)
> >> +		return 0;
> >> +
> >> +	while ((d = readdir(dir))) {
> >> +		if (!strcmp(d->d_name, "tables.list"))
> >> +			continue;
> >> +
> >> +		if ((d->d_name[0] == '.' &&
> >> +		     (d->d_name[1] == '\0' ||
> >> +		      (d->d_name[1] == '.' && d->d_name[2] == '\0'))))
> >> +			continue;
> >> +
> >> +		if (d->d_type == DT_REG) {
> >> +			if (!valid_table_name(d->d_name, &min, &max)) {
> >> +				info.error = REFTABLE_FSCK_ERROR_TABLE_NAME;
> >> +				info.msg = "file with invalid table name";
> >> +				info.path = d->d_name;
> >> +
> >> +				err |= report_fn(&info, cb_data);
> >> +			}
> >
> > One problem with this is that this is racy with concurrent writers. We
> > don't recognize the "tables.list.lock" file, and neither do we recognize
> > "0x*-0x*.{ref,log}.temp.XXXXXX"-style files.
> >
> > Would it be a better approach be to instead go through table names as
> > loaded by the stack? The reftable code already knows to prune unknown
> > files anyway, so I don't think we should scan for any other files.
> >
> 
> I actually had a more structured code here, where the idea was:
> 
> - For each stack
>   - Run stack level checks
>   - For each table in stack
>     - Run table level checks
>     - For each block in table
>       - Run block level checks
>       - For each ref / log
>         - Run ref / log level checks
> 
> But we move some of my tests to be runtime checks, leaving this as the
> only check remaining. We could still do the first level of what I
> mentioned above. The only reason I didn't was because we wanted to check
> all files in the stack dir. But I think this is much better, having
> unknown files in the reftable directory doesn't affect the repository in
> any way. So I would argue perhaps that we shouldn't even care about it.

Yeah, agreed. As long as we don't know about any edge cases where this does or did create problems I agree.

Patrick
Previous: Karthik NayakNext: Karthik Nayak
Message 38 of 64 in “refs/reftable: add fsck checks”
  1. 0/5 refs/reftable: add fsck checksKarthik Nayak, Aug 19, 2025
  2. 1/5 fsck: order 'fsck_msg_type' alphabeticallyKarthik Nayak, Aug 19, 2025
  3. 2/5 refs/reftable: add fsck check for checking the table nameKarthik Nayak, Aug 19, 2025
  4. shejialuoAug 26, 2025
  5. Karthik NayakSep 1, 2025
  6. shejialuoSep 3, 2025
  7. 3/5 refs/reftable: add fsck check for number of tablesKarthik Nayak, Aug 19, 2025
  8. shejialuoAug 26, 2025
  9. Karthik NayakSep 1, 2025
  10. shejialuoAug 26, 2025
  11. Karthik NayakSep 1, 2025
  12. 4/5 refs/reftable: add fsck check for trailing newlineKarthik Nayak, Aug 19, 2025
  13. 5/5 refs/reftable: add fsck check for incorrect update indexKarthik Nayak, Aug 19, 2025
  14. shejialuoAug 26, 2025
  15. Karthik NayakSep 1, 2025
  16. 0/8 refs/reftable: add consistency checksKarthik Nayak, Sep 18, 2025
  17. 1/8 refs: remove unused headersKarthik Nayak, Sep 18, 2025
  18. 2/8 refs: move consistency check msg to generic layerKarthik Nayak, Sep 18, 2025
  19. 3/8 reftable: check for trailing newline in 'tables.list'Karthik Nayak, Sep 18, 2025
  20. Junio C HamanoSep 18, 2025
  21. Karthik NayakSep 23, 2025
  22. Patrick SteinhardtSep 24, 2025
  23. Karthik NayakSep 24, 2025
  24. Kristoffer HaugsbakkSep 24, 2025
  25. Karthik NayakSep 24, 2025
  26. 5/8 Documentation/fsck-msgids: remove duplicate msg idKarthik Nayak, Sep 18, 2025
  27. 4/8 reftable: ensure tables in a stack use sequential update indicesKarthik Nayak, Sep 18, 2025
  28. Patrick SteinhardtSep 24, 2025
  29. Karthik NayakSep 24, 2025
  30. Junio C HamanoSep 24, 2025
  31. Karthik NayakSep 24, 2025
  32. Patrick SteinhardtSep 25, 2025
  33. Junio C HamanoSep 25, 2025
  34. 6/8 fsck: order 'fsck_msg_type' alphabeticallyKarthik Nayak, Sep 18, 2025
  35. 7/8 reftable: add code to facilitate consistency checksKarthik Nayak, Sep 18, 2025
  36. Patrick SteinhardtSep 24, 2025
  37. Karthik NayakSep 24, 2025
  38. Patrick SteinhardtSep 25, 2025
  39. 8/8 refs/reftable: add fsck check for checking the table nameKarthik Nayak, Sep 18, 2025
  40. Patrick SteinhardtSep 24, 2025
  41. Karthik NayakSep 24, 2025
  42. 0/7 refs/reftable: add consistency checksKarthik Nayak, Oct 6, 2025
  43. 1/7 refs: remove unused headersKarthik Nayak, Oct 6, 2025
  44. 2/7 refs: move consistency check msg to generic layerKarthik Nayak, Oct 6, 2025
  45. 3/7 reftable: check for trailing newline in 'tables.list'Karthik Nayak, Oct 6, 2025
  46. 4/7 Documentation/fsck-msgids: remove duplicate msg idKarthik Nayak, Oct 6, 2025
  47. 5/7 fsck: order 'fsck_msg_type' alphabeticallyKarthik Nayak, Oct 6, 2025
  48. 6/7 reftable: add code to facilitate consistency checksKarthik Nayak, Oct 6, 2025
  49. 7/7 refs/reftable: add fsck check for checking the table nameKarthik Nayak, Oct 6, 2025
  50. Jeff KingOct 7, 2025
  51. Karthik NayakOct 7, 2025
  52. Junio C HamanoOct 6, 2025
  53. Karthik NayakOct 7, 2025
  54. Junio C HamanoOct 7, 2025
  55. 0/7 refs/reftable: add consistency checksKarthik Nayak, Oct 7, 2025
  56. 1/7 refs: remove unused headersKarthik Nayak, Oct 7, 2025
  57. 2/7 refs: move consistency check msg to generic layerKarthik Nayak, Oct 7, 2025
  58. 3/7 reftable: check for trailing newline in 'tables.list'Karthik Nayak, Oct 7, 2025
  59. 4/7 Documentation/fsck-msgids: remove duplicate msg idKarthik Nayak, Oct 7, 2025
  60. 5/7 fsck: order 'fsck_msg_type' alphabeticallyKarthik Nayak, Oct 7, 2025
  61. 6/7 reftable: add code to facilitate consistency checksKarthik Nayak, Oct 7, 2025
  62. 7/7 refs/reftable: add fsck check for checking the table nameKarthik Nayak, Oct 7, 2025
  63. Patrick SteinhardtOct 7, 2025
  64. Junio C HamanoOct 7, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.