git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 3/4] ref: add symbolic ref content check for files backend

From
shejialuo <shejialuo@gmail.com>
Date
Aug 27, 2024, 16:08 UTC
Message-ID
<Zs3558scHssaG_XS@ArchLinux>
In-Reply-To
<Zs348uXMBdCuwF-2@ArchLinux>

We have already introduced the checks for regular refs. There is no need to check the consistency of the target which the symbolic ref points to. Instead, we just check the content of the symbolic ref itself.

In order to check the content of the symbolic ref, create a function "files_fsck_symref_target". It will first check whether the "pointee" is under the "refs/" directory and then we will check the "pointee" itself.

There is no specification about the content of the symbolic ref. Although we do write "ref: %s\n" to create a symbolic ref by using "git-symbolic-ref(1)" command. However, this is not mandatory. We still accept symbolic refs with null trailing garbage. Put it more specific, the following are correct:

1. "ref: refs/heads/master   "
2. "ref: refs/heads/master   \n  \n"
3. "ref: refs/heads/master\n\n"

But we do not allow any non-null trailing garbage. The following are bad symbolic contents which will be reported as fsck error by "git-fsck(1)".

1. "ref: refs/heads/master garbage\n"
2. "ref: refs/heads/master \n\n\n garbage  "

In order to provide above checks, we will use "strrchr" to check whether we have newline in the ref content. Then we will check the name of the "pointee" is correct by using "check_refname_format". If the function fails, we need to trim the "pointee" to see whether the null-garbage causes the function fails. If so, we need to report that there is null-garbage in the symref content. Otherwise, we should report the user the "pointee" is bad.

Mentored-by: Patrick Steinhardt <ps@pks.im>
Mentored-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: shejialuo <shejialuo@gmail.com>
---
 Documentation/fsck-msgids.txt |  3 ++
 fsck.h                        |  1 +
 refs/files-backend.c          | 77 +++++++++++++++++++++++++++++++++++
 t/t0602-reffiles-fsck.sh      | 54 ++++++++++++++++++++++++
 4 files changed, 135 insertions(+)
diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt
index fc074fc571..85fd058c81 100644
--- a/Documentation/fsck-msgids.txt
+++ b/Documentation/fsck-msgids.txt
@@ -28,6 +28,9 @@
 `badRefName`::
 	(ERROR) A ref has an invalid format.
 
+`badSymrefPointee`::
+	(ERROR) The pointee of a symref is bad.
+
 `badTagName`::
 	(INFO) A tag has an invalid format.
 
diff --git a/fsck.h b/fsck.h
index b85072df57..cbe837f84c 100644
--- a/fsck.h
+++ b/fsck.h
@@ -34,6 +34,7 @@ enum fsck_msg_type {
 	FUNC(BAD_REF_CONTENT, ERROR) \
 	FUNC(BAD_REF_FILETYPE, ERROR) \
 	FUNC(BAD_REF_NAME, ERROR) \
+	FUNC(BAD_SYMREF_POINTEE, ERROR) \
 	FUNC(BAD_TIMEZONE, ERROR) \
 	FUNC(BAD_TREE, ERROR) \
 	FUNC(BAD_TREE_SHA1, ERROR) \
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 69c00073eb..382c73fcf7 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3434,11 +3434,81 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,
 				  const char *refs_check_dir,
 				  struct dir_iterator *iter);
 
+/*
+ * Check the symref "pointee_name" and "pointee_path". The caller should
+ * make sure that "pointee_path" is absolute. For symbolic ref, "pointee_name"
+ * would be the content after "refs:".
+ */
+static int files_fsck_symref_target(struct fsck_options *o,
+				    struct fsck_ref_report *report,
+				    const char *refname,
+				    struct strbuf *pointee_name,
+				    struct strbuf *pointee_path)
+{
+	const char *newline_pos = NULL;
+	const char *p = NULL;
+	struct stat st;
+	int ret = 0;
+
+	if (!skip_prefix(pointee_name->buf, "refs/", &p)) {
+
+		ret = fsck_report_ref(o, report,
+				      FSCK_MSG_BAD_SYMREF_POINTEE,
+				      "points to ref outside the refs directory");
+		goto out;
+	}
+
+	newline_pos = strrchr(p, '\n');
+	if (!newline_pos || *(newline_pos + 1)) {
+		ret = fsck_report_ref(o, report,
+				      FSCK_MSG_REF_MISSING_NEWLINE,
+				      "missing newline");
+	}
+
+	if (check_refname_format(pointee_name->buf, 0)) {
+		/*
+		 * When containing null-garbage, "check_refname_format" will
+		 * fail, we should trim the "pointee" to check again.
+		 */
+		strbuf_rtrim(pointee_name);
+		if (!check_refname_format(pointee_name->buf, 0)) {
+			ret = fsck_report_ref(o, report,
+					      FSCK_MSG_TRAILING_REF_CONTENT,
+					      "trailing null-garbage");
+			goto out;
+		}
+
+		ret = fsck_report_ref(o, report,
+				      FSCK_MSG_BAD_SYMREF_POINTEE,
+				      "points to refname with invalid format");
+	}
+
+	/*
+	 * Missing target should not be treated as any error worthy event and
+	 * not even warn. It is a common case that a symbolic ref points to a
+	 * ref that does not exist yet. If the target ref does not exist, just
+	 * skip the check for the file type.
+	 */
+	if (lstat(pointee_path->buf, &st) < 0)
+		goto out;
+
+	if (!S_ISREG(st.st_mode) && !S_ISLNK(st.st_mode)) {
+		ret = fsck_report_ref(o, report,
+				      FSCK_MSG_BAD_SYMREF_POINTEE,
+				      "points to an invalid file type");
+		goto out;
+	}
+
+out:
+	return ret;
+}
+
 static int files_fsck_refs_content(struct ref_store *ref_store,
 				   struct fsck_options *o,
 				   const char *refs_check_dir,
 				   struct dir_iterator *iter)
 {
+	struct strbuf pointee_path = STRBUF_INIT;
 	struct strbuf ref_content = STRBUF_INIT;
 	struct strbuf referent = STRBUF_INIT;
 	struct strbuf refname = STRBUF_INIT;
@@ -3482,6 +3552,12 @@ static int files_fsck_refs_content(struct ref_store *ref_store,
 						      "trailing garbage in ref");
 				goto cleanup;
 			}
+		} else {
+			strbuf_addf(&pointee_path, "%s/%s",
+				    ref_store->gitdir, referent.buf);
+			ret = files_fsck_symref_target(o, &report, refname.buf,
+						       &referent,
+						       &pointee_path);
 		}
 		goto cleanup;
 	}
@@ -3490,6 +3566,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,
 	strbuf_release(&refname);
 	strbuf_release(&ref_content);
 	strbuf_release(&referent);
+	strbuf_release(&pointee_path);
 	return ret;
 }
 
diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh
index 7c1910d784..69280795ca 100755
--- a/t/t0602-reffiles-fsck.sh
+++ b/t/t0602-reffiles-fsck.sh
@@ -176,4 +176,58 @@ test_expect_success 'regular ref content should be checked' '
 	test_cmp expect err
 '
 
+test_expect_success 'symbolic ref content should be checked' '
+	test_when_finished "rm -rf repo" &&
+	git init repo &&
+	branch_dir_prefix=.git/refs/heads &&
+	tag_dir_prefix=.git/refs/tags &&
+	cd repo &&
+	git commit --allow-empty -m initial &&
+	git checkout -b branch-1 &&
+	git tag tag-1 &&
+	git checkout -b a/b/branch-2 &&
+
+	printf "ref: refs/heads/branch" > $branch_dir_prefix/branch-1-no-newline &&
+	git refs verify 2>err &&
+	cat >expect <<-EOF &&
+	warning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline
+	EOF
+	rm $branch_dir_prefix/branch-1-no-newline &&
+	test_cmp expect err &&
+
+	printf "ref: refs/heads/branch     " > $branch_dir_prefix/a/b/branch-trailing &&
+	git refs verify 2>err &&
+	cat >expect <<-EOF &&
+	warning: refs/heads/a/b/branch-trailing: refMissingNewline: missing newline
+	warning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage
+	EOF
+	rm $branch_dir_prefix/a/b/branch-trailing &&
+	test_cmp expect err &&
+
+	printf "ref: refs/heads/branch\n\n" > $branch_dir_prefix/a/b/branch-trailing &&
+	git refs verify 2>err &&
+	cat >expect <<-EOF &&
+	warning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage
+	EOF
+	rm $branch_dir_prefix/a/b/branch-trailing &&
+	test_cmp expect err &&
+
+	printf "ref: refs/heads/branch \n\n " > $branch_dir_prefix/a/b/branch-trailing &&
+	git refs verify 2>err &&
+	cat >expect <<-EOF &&
+	warning: refs/heads/a/b/branch-trailing: refMissingNewline: missing newline
+	warning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage
+	EOF
+	rm $branch_dir_prefix/a/b/branch-trailing &&
+	test_cmp expect err &&
+
+	printf "ref: refs/heads/.branch\n" > $branch_dir_prefix/branch-2-bad &&
+	test_must_fail git refs verify 2>err &&
+	cat >expect <<-EOF &&
+	error: refs/heads/branch-2-bad: badSymrefPointee: points to refname with invalid format
+	EOF
+	rm $branch_dir_prefix/branch-2-bad &&
+	test_cmp expect err
+'
+
 test_done
-- 
2.46.0
Previous: Junio C HamanoNext: Junio C Hamano
Message 40 of 209 in “[RFC] Implement ref content consistency check”
  1. shejialuoAug 13, 2024
  2. karthik nayakAug 15, 2024
  3. shejialuoAug 15, 2024
  4. Patrick SteinhardtAug 16, 2024
  5. Junio C HamanoAug 16, 2024
  6. 0/4 add ref content check for files backendshejialuo, Aug 18, 2024
  7. 1/4 fsck: introduce "FSCK_REF_REPORT_DEFAULT" macroshejialuo, Aug 18, 2024
  8. Junio C HamanoAug 20, 2024
  9. shejialuoAug 21, 2024
  10. 2/4 ref: add regular ref content check for files backendshejialuo, Aug 18, 2024
  11. Junio C HamanoAug 20, 2024
  12. shejialuoAug 21, 2024
  13. Patrick SteinhardtAug 22, 2024
  14. Junio C HamanoAug 22, 2024
  15. Junio C HamanoAug 22, 2024
  16. Patrick SteinhardtAug 23, 2024
  17. shejialuoAug 23, 2024
  18. Patrick SteinhardtAug 22, 2024
  19. shejialuoAug 22, 2024
  20. 3/4 ref: add symbolic ref content check for files backendshejialuo, Aug 18, 2024
  21. Patrick SteinhardtAug 22, 2024
  22. shejialuoAug 22, 2024
  23. Patrick SteinhardtAug 23, 2024
  24. shejialuoAug 23, 2024
  25. 4/4 ref: add symlink ref consistency check for files backendshejialuo, Aug 18, 2024
  26. 0/4 add ref content check for files backendshejialuo, Aug 27, 2024
  27. 1/4 ref: initialize "fsck_ref_report" with zeroshejialuo, Aug 27, 2024
  28. Junio C HamanoAug 27, 2024
  29. 2/4 ref: add regular ref content check for files backendshejialuo, Aug 27, 2024
  30. shejialuoAug 27, 2024
  31. Junio C HamanoAug 27, 2024
  32. Patrick SteinhardtAug 28, 2024
  33. Junio C HamanoAug 28, 2024
  34. Patrick SteinhardtAug 29, 2024
  35. shejialuoAug 28, 2024
  36. Junio C HamanoAug 28, 2024
  37. Patrick SteinhardtAug 28, 2024
  38. shejialuoAug 28, 2024
  39. Junio C HamanoAug 28, 2024
  40. 3/4 ref: add symbolic ref content check for files backendshejialuo, Aug 27, 2024
  41. Junio C HamanoAug 27, 2024
  42. shejialuoAug 28, 2024
  43. Patrick SteinhardtAug 28, 2024
  44. shejialuoAug 28, 2024
  45. Junio C HamanoAug 28, 2024
  46. Patrick SteinhardtAug 29, 2024
  47. 4/4 ref: add symlink ref check for files backendshejialuo, Aug 27, 2024
  48. SQUASH??? remove unused parametersJunio C Hamano, Aug 28, 2024
  49. Junio C HamanoAug 28, 2024
  50. Jeff KingAug 29, 2024
  51. Junio C HamanoAug 29, 2024
  52. Patrick SteinhardtAug 29, 2024
  53. Junio C HamanoAug 29, 2024
  54. Jeff KingAug 29, 2024
  55. shejialuoAug 29, 2024
  56. Junio C HamanoAug 29, 2024
  57. 8/6 CodingGuidelines: also mention MAYBE_UNUSEDJunio C Hamano, Aug 29, 2024
  58. Jeff KingAug 29, 2024
  59. Junio C HamanoAug 29, 2024
  60. CodingGuidelines: also mention MAYBE_UNUSEDJunio C Hamano, Aug 29, 2024
  61. 9/6 git-compat-util: guard definition of MAYBE_UNUSED with __GNUC__Junio C Hamano, Aug 29, 2024
  62. Jeff KingAug 29, 2024
  63. Junio C HamanoAug 29, 2024
  64. Jeff KingAug 29, 2024
  65. 0/4 add ref content check for files backendshejialuo, Sep 3, 2024
  66. 1/4 ref: initialize "fsck_ref_report" with zeroshejialuo, Sep 3, 2024
  67. 2/4 ref: add regular ref content check for files backendshejialuo, Sep 3, 2024
  68. Patrick SteinhardtSep 9, 2024
  69. shejialuoSep 10, 2024
  70. karthik nayakSep 10, 2024
  71. shejialuoSep 13, 2024
  72. 3/4 ref: add symref content check for files backendshejialuo, Sep 3, 2024
  73. Patrick SteinhardtSep 9, 2024
  74. shejialuoSep 10, 2024
  75. karthik nayakSep 10, 2024
  76. shejialuoSep 12, 2024
  77. 4/4 ref: add symlink ref content check for files backendshejialuo, Sep 3, 2024
  78. Patrick SteinhardtSep 9, 2024
  79. shejialuoSep 10, 2024
  80. 0/5 add ref content check for files backendshejialuo, Sep 13, 2024
  81. 1/5 ref: initialize "fsck_ref_report" with zeroshejialuo, Sep 13, 2024
  82. Junio C HamanoSep 18, 2024
  83. 2/5 ref: port git-fsck(1) regular refs check for files backendshejialuo, Sep 13, 2024
  84. Junio C HamanoSep 18, 2024
  85. shejialuoSep 22, 2024
  86. 3/5 ref: add more strict checks for regular refsshejialuo, Sep 13, 2024
  87. Junio C HamanoSep 18, 2024
  88. shejialuoSep 22, 2024
  89. Junio C HamanoSep 22, 2024
  90. 4/5 ref: add symref content check for files backendshejialuo, Sep 13, 2024
  91. Junio C HamanoSep 18, 2024
  92. shejialuoSep 22, 2024
  93. Junio C HamanoSep 22, 2024
  94. 5/5 ref: add symlink ref content check for files backendshejialuo, Sep 13, 2024
  95. Junio C HamanoSep 18, 2024
  96. Junio C HamanoSep 18, 2024
  97. 0/9 add ref content check for files backendshejialuo, Sep 29, 2024
  98. 1/9 ref: initialize "fsck_ref_report" with zeroshejialuo, Sep 29, 2024
  99. Karthik NayakOct 8, 2024
  100. 2/9 builtin/refs: support multiple worktrees check for refs.shejialuo, Sep 29, 2024
  101. Patrick SteinhardtOct 7, 2024
  102. shejialuoOct 7, 2024
  103. Patrick SteinhardtOct 7, 2024
  104. shejialuoOct 7, 2024
  105. 3/9 ref: port git-fsck(1) regular refs check for files backendshejialuo, Sep 29, 2024
  106. Patrick SteinhardtOct 7, 2024
  107. shejialuoOct 7, 2024
  108. Patrick SteinhardtOct 7, 2024
  109. shejialuoOct 7, 2024
  110. Karthik NayakOct 8, 2024
  111. shejialuoOct 8, 2024
  112. Junio C HamanoOct 8, 2024
  113. Patrick SteinhardtOct 9, 2024
  114. shejialuoOct 9, 2024
  115. Patrick SteinhardtOct 10, 2024
  116. Junio C HamanoOct 10, 2024
  117. shejialuoOct 9, 2024
  118. 4/9 ref: add more strict checks for regular refsshejialuo, Sep 29, 2024
  119. Patrick SteinhardtOct 7, 2024
  120. shejialuoOct 7, 2024
  121. Patrick SteinhardtOct 7, 2024
  122. shejialuoOct 7, 2024
  123. 5/9 ref: add basic symref content check for files backendshejialuo, Sep 29, 2024
  124. Karthik NayakOct 8, 2024
  125. shejialuoOct 8, 2024
  126. 6/9 ref: add escape check for the referent of symrefshejialuo, Sep 29, 2024
  127. Patrick SteinhardtOct 7, 2024
  128. shejialuoOct 7, 2024
  129. Patrick SteinhardtOct 7, 2024
  130. 7/9 ref: enhance escape situation for worktreesshejialuo, Sep 29, 2024
  131. Patrick SteinhardtOct 7, 2024
  132. shejialuoOct 7, 2024
  133. 8/9 t0602: add ref content checks for worktreesshejialuo, Sep 29, 2024
  134. Patrick SteinhardtOct 7, 2024
  135. shejialuoOct 7, 2024
  136. 9/9 ref: add symlink ref content check for files backendshejialuo, Sep 29, 2024
  137. Patrick SteinhardtOct 7, 2024
  138. shejialuoOct 7, 2024
  139. Junio C HamanoSep 30, 2024
  140. shejialuoOct 1, 2024
  141. shejialuoOct 7, 2024
  142. 0/9 add ref content check for files backendshejialuo, Oct 21, 2024
  143. 1/9 ref: initialize "fsck_ref_report" with zeroshejialuo, Oct 21, 2024
  144. 2/9 ref: check the full refname instead of basenameshejialuo, Oct 21, 2024
  145. karthik nayakOct 21, 2024
  146. shejialuoOct 22, 2024
  147. Patrick SteinhardtNov 5, 2024
  148. shejialuoNov 6, 2024
  149. 3/9 ref: initialize target name outside of check functionsshejialuo, Oct 21, 2024
  150. karthik nayakOct 21, 2024
  151. Patrick SteinhardtNov 5, 2024
  152. shejialuoNov 6, 2024
  153. Patrick SteinhardtNov 6, 2024
  154. 4/9 ref: support multiple worktrees check for refsshejialuo, Oct 21, 2024
  155. karthik nayakOct 21, 2024
  156. shejialuoOct 22, 2024
  157. Patrick SteinhardtNov 5, 2024
  158. shejialuoNov 5, 2024
  159. Patrick SteinhardtNov 6, 2024
  160. shejialuoNov 6, 2024
  161. 5/9 ref: port git-fsck(1) regular refs check for files backendshejialuo, Oct 21, 2024
  162. Patrick SteinhardtNov 5, 2024
  163. 6/9 ref: add more strict checks for regular refsshejialuo, Oct 21, 2024
  164. 7/9 ref: add basic symref content check for files backendshejialuo, Oct 21, 2024
  165. 8/9 ref: check whether the target of the symref is a refshejialuo, Oct 21, 2024
  166. 9/9 ref: add symlink ref content check for files backendshejialuo, Oct 21, 2024
  167. Taylor BlauOct 21, 2024
  168. shejialuoOct 22, 2024
  169. Taylor BlauOct 21, 2024
  170. 0/9 add ref content check for files backendshejialuo, Nov 10, 2024
  171. 1/9 ref: initialize "fsck_ref_report" with zeroshejialuo, Nov 10, 2024
  172. 2/9 ref: check the full refname instead of basenameshejialuo, Nov 10, 2024
  173. 3/9 ref: initialize ref name outside of check functionsshejialuo, Nov 10, 2024
  174. 4/9 ref: support multiple worktrees check for refsshejialuo, Nov 10, 2024
  175. 5/9 ref: port git-fsck(1) regular refs check for files backendshejialuo, Nov 10, 2024
  176. Patrick SteinhardtNov 13, 2024
  177. shejialuoNov 14, 2024
  178. 6/9 ref: add more strict checks for regular refsshejialuo, Nov 10, 2024
  179. 7/9 ref: add basic symref content check for files backendshejialuo, Nov 10, 2024
  180. 8/9 ref: check whether the target of the symref is a refshejialuo, Nov 10, 2024
  181. 9/9 ref: add symlink ref content check for files backendshejialuo, Nov 10, 2024
  182. Patrick SteinhardtNov 13, 2024
  183. shejialuoNov 14, 2024
  184. Patrick SteinhardtNov 13, 2024
  185. 0/9 add ref content check for files backendshejialuo, Nov 14, 2024
  186. 1/9 ref: initialize "fsck_ref_report" with zeroshejialuo, Nov 14, 2024
  187. 2/9 ref: check the full refname instead of basenameshejialuo, Nov 14, 2024
  188. 3/9 ref: initialize ref name outside of check functionsshejialuo, Nov 14, 2024
  189. 4/9 ref: support multiple worktrees check for refsshejialuo, Nov 14, 2024
  190. 5/9 ref: port git-fsck(1) regular refs check for files backendshejialuo, Nov 14, 2024
  191. Patrick SteinhardtNov 15, 2024
  192. shejialuoNov 15, 2024
  193. 6/9 ref: add more strict checks for regular refsshejialuo, Nov 14, 2024
  194. 7/9 ref: add basic symref content check for files backendshejialuo, Nov 14, 2024
  195. 8/9 ref: check whether the target of the symref is a refshejialuo, Nov 14, 2024
  196. 9/9 ref: add symlink ref content check for files backendshejialuo, Nov 14, 2024
  197. shejialuoNov 15, 2024
  198. 0/9 add ref content check for files backendshejialuo, Nov 20, 2024
  199. 1/9 ref: initialize "fsck_ref_report" with zeroshejialuo, Nov 20, 2024
  200. 2/9 ref: check the full refname instead of basenameshejialuo, Nov 20, 2024
  201. 3/9 ref: initialize ref name outside of check functionsshejialuo, Nov 20, 2024
  202. 4/9 ref: support multiple worktrees check for refsshejialuo, Nov 20, 2024
  203. 5/9 ref: port git-fsck(1) regular refs check for files backendshejialuo, Nov 20, 2024
  204. 6/9 ref: add more strict checks for regular refsshejialuo, Nov 20, 2024
  205. 7/9 ref: add basic symref content check for files backendshejialuo, Nov 20, 2024
  206. 8/9 ref: check whether the target of the symref is a refshejialuo, Nov 20, 2024
  207. 9/9 ref: add symlink ref content check for files backendshejialuo, Nov 20, 2024
  208. Patrick SteinhardtNov 20, 2024
  209. Junio C HamanoNov 20, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.