{"thread":{"id":"62743","subject":"[PATCH 00/10] add more ref consistency checks","startedAt":"2025-01-05T13:47:05Z","lastAt":"2025-02-28T05:02:13Z","messageCount":168,"participants":["shejialuo","Junio C Hamano","Karthik Nayak","Patrick Steinhardt","Eric Sunshine"],"isPatch":true,"patchVersion":1,"patchTotal":10},"messages":[{"id":"509929","messageId":"Z3qNUizvHJLgMx1y@ArchLinux","threadId":"62743","inReplyTo":null,"subject":"[PATCH 00/10] add more ref consistency checks","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:46:58Z","receivedAt":"2025-01-05T13:47:05Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi all:\n\nThis patch mainly does the following three things:\n\n1. Add some extra checks which I have ignored in the previous patches\n for files-backend in\n2. Add ref checks for packed-backend.\n 1. Check whether the type of \"packed-refs\" is correct.\n 2. Check whether the syntax of \"packed-refs\" is correct by using the\n rules from \"packed-backend.c::create_snapshot\" and\n \"packed-backend.c::next_record\".\n 3. Check whether the pointed object exists and whether the\n \"packed-refs\" file is sorted.\n3. Call \"git refs verify\" for \"git-fsck(1)\".\n\nAlthough I am not mentored by Patrick and Karthik in this patch. I'd\nlike to add \"Mentored-by\" filed for them due to the reason that I\ncontinue my GSoC work.\n\nThanks,\nJialuo\n\nshejialuo (10):\n files-backend: add object check for regular ref\n builtin/refs.h: get worktrees without reading head info\n packed-backend: check whether the \"packed-refs\" is regular\n packed-backend: add \"packed-refs\" header consistency check\n packed-backend: check whether the refname contains NULL binaries\n packed-backend: add \"packed-refs\" entry consistency check\n packed-backend: create \"fsck_packed_ref_entry\" to store parsing info\n packed-backend: add check for object consistency\n packed-backend: check whether the \"packed-refs\" is sorted\n builtin/fsck: add `git refs verify` child process\n\n Documentation/fsck-msgids.txt | 22 ++\n builtin/fsck.c | 28 +++\n builtin/refs.c | 2 +-\n fsck.h | 8 +\n refs/files-backend.c | 54 ++++-\n refs/packed-backend.c | 413 +++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 209 +++++++++++++++++\n worktree.c | 5 +\n worktree.h | 6 +\n 9 files changed, 723 insertions(+), 24 deletions(-)\n\n-- \n2.47.1\n\n"},{"id":"509930","messageId":"Z3qN1T3lJoj82ckl@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 01/10] files-backend: add object check for regular ref","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:49:09Z","receivedAt":"2025-01-05T13:49:15Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Although we use \"parse_loose_ref_content\" to check whether the object id\nis correct, we never parse it into the \"struct object\" structure thus we\nignore checking whether there is a real object existing in the repo and\nwhether the object type is correct.\n\nUse \"parse_object\" to parse the oid for the regular ref content. If the\nobject does not exist, report the error to the user by reusing the fsck\nmessage \"BAD_REF_CONTENT\".\n\nThen, we need to check the type of the object. Just like \"git-fsck(1)\",\nwe only report \"not a commit\" error when the ref is a branch. Last,\nupdate the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 50 ++++++++++++++++++++++++++++++++--------\n t/t0602-reffiles-fsck.sh | 30 ++++++++++++++++++++++++\n 2 files changed, 70 insertions(+), 10 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 64f51f0da9..0a4912c009 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -20,6 +20,7 @@\n #include \"../lockfile.h\"\n #include \"../object.h\"\n #include \"../object-file.h\"\n+#include \"../packfile.h\"\n #include \"../path.h\"\n #include \"../dir.h\"\n #include \"../chdir-notify.h\"\n@@ -3589,6 +3590,34 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \treturn ret;\n }\n \n+static int files_fsck_refs_oid(struct fsck_options *o,\n+\t\t\t struct ref_store *ref_store,\n+\t\t\t struct fsck_ref_report report,\n+\t\t\t const char *target_name,\n+\t\t\t struct object_id *oid)\n+{\n+\tstruct object *obj;\n+\tint ret = 0;\n+\n+\tif (is_promisor_object(ref_store->repo, oid))\n+\t\treturn 0;\n+\n+\tobj = parse_object(ref_store->repo, oid);\n+\tif (!obj) {\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"points to non-existing object %s\",\n+\t\t\t\t oid_to_hex(oid));\n+\t} else if (obj->type != OBJ_COMMIT && is_branch(target_name)) {\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"points to non-commit object %s\",\n+\t\t\t\t oid_to_hex(oid));\n+\t}\n+\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *target_name,\n@@ -3654,18 +3683,19 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t}\n \n \tif (!(type & REF_ISSYMREF)) {\n+\t\tret |= files_fsck_refs_oid(o, ref_store, report, target_name, &oid);\n+\n \t\tif (!*trailing) {\n-\t\t\tret = fsck_report_ref(o, &report,\n-\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n-\t\t\t\t\t \"misses LF at the end\");\n-\t\t\tgoto cleanup;\n-\t\t}\n-\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n-\t\t\tret = fsck_report_ref(o, &report,\n-\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n-\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n-\t\t\tgoto cleanup;\n+\t\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t \"misses LF at the end\");\n+\t\t} else if (*trailing != '\\n' || *(trailing + 1)) {\n+\t\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n \t\t}\n+\n+\t\tgoto cleanup;\n \t} else {\n \t\tret = files_fsck_symref_target(o, &report, &referent, 0);\n \t\tgoto cleanup;\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex d4a08b823b..75f234a94a 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -161,8 +161,10 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n \tcd repo &&\n \ttest_commit default &&\n+\tgit branch branch-1 &&\n \tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n \tgit refs verify 2>err &&\n@@ -198,6 +200,28 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \trm $branch_dir_prefix/branch-no-newline &&\n \ttest_cmp expect err &&\n \n+\tfor non_existing_oid in \"$(test_oid 001)\" \"$(test_oid 002)\"\n+\tdo\n+\t\tprintf \"%s\\n\" $non_existing_oid >$branch_dir_prefix/invalid-commit &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/invalid-commit: badRefContent: points to non-existing object $non_existing_oid\n+\t\tEOF\n+\t\trm $branch_dir_prefix/invalid-commit &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor tree_oid in \"$(git rev-parse main^{tree})\" \"$(git rev-parse branch-1^{tree})\"\n+\tdo\n+\t\tprintf \"%s\\n\" $tree_oid >$branch_dir_prefix/branch-tree &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-tree: badRefContent: points to non-commit object $tree_oid\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-tree &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n \tfor trailing_content in \" garbage\" \" more garbage\"\n \tdo\n \t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n@@ -244,15 +268,21 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tbad_content_1=$(git rev-parse main)x &&\n \tbad_content_2=xfsazqfxcadas &&\n \tbad_content_3=Xfsazqfxcadas &&\n+\tnon_existing_oid=$(test_oid 001) &&\n+\ttree_oid=$(git rev-parse main^{tree}) &&\n \tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n \tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n \tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n \tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n \tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\tprintf \"%s\\n\" $non_existing_oid >$branch_dir_prefix/branch-non-existing-oid &&\n+\tprintf \"%s\\n\" $tree_oid >$branch_dir_prefix/branch-tree &&\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\terror: refs/heads/branch-non-existing-oid: badRefContent: points to non-existing object $non_existing_oid\n+\terror: refs/heads/branch-tree: badRefContent: points to non-commit object $tree_oid\n \terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n \terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n \twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-- \n2.47.1\n\n"},{"id":"509931","messageId":"Z3qN30z1NCXa3AX-@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 02/10] builtin/refs.h: get worktrees without reading head info","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:49:19Z","receivedAt":"2025-01-05T13:49:24Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\nand \"next_record\" which would check the correctness of the content of\nthe \"packed-ref\" file. When anything is bad, the program will die.\n\nIt may seem that we have nothing relevant to above feature, because we\nare going to read and parse the raw \"packed-ref\" file without creating\nthe snapshot and using the ref iterator to check the consistency.\n\nHowever, when using \"get_worktrees\" in \"builtin/refs\", we will parse the\nhead information. If the referent of the \"HEAD\" is inside the\n\"packed-ref\", we will call \"create_snapshot\" and \"next_record\" functions\nto parse the \"packed-ref\" to get the head information. And if there are\nsomething wrong, the program will die.\n\nAlthough this behavior has no harm for the program, it will\nshort-circuit the program. When the users execute \"git refs verify\" or\n\"git fsck\", we don't want to simply die the program but rather show the\nwarnings or errors as many as possible to info the users. So, we should\navoiding reading the head info.\n\nFortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\nworktrees, 2023-12-29), we have introduced a function\n\"get_worktrees_internal\" which allows us to get worktrees without\nreading head info.\n\nCreate a new exposed function \"get_worktrees_without_reading_head\", then\nreplace the \"get_worktrees\" in \"builtin/refs\" with the new created\nfunction.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 2 +-\n worktree.c | 5 +++++\n worktree.h | 6 ++++++\n 3 files changed, 12 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex a29f195834..55ff5dae11 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tworktrees = get_worktrees();\n+\tworktrees = get_worktrees_without_reading_head();\n \tfor (size_t i = 0; worktrees[i]; i++)\n \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n \t\t\t\t &fsck_refs_options, worktrees[i]);\ndiff --git a/worktree.c b/worktree.c\nindex af68b24f9d..74cb463e51 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -174,6 +174,11 @@ struct worktree **get_worktrees(void)\n \treturn get_worktrees_internal(0);\n }\n \n+struct worktree **get_worktrees_without_reading_head(void)\n+{\n+\treturn get_worktrees_internal(1);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\ndiff --git a/worktree.h b/worktree.h\nindex 38145df80f..1ba4a161a0 100644\n--- a/worktree.h\n+++ b/worktree.h\n@@ -30,6 +30,12 @@ struct worktree {\n */\n struct worktree **get_worktrees(void);\n \n+/*\n+ * Like `get_worktrees`, but does not read HEAD. This is useful when checking\n+ * the consistency, as reading HEAD may not be necessary.\n+ */\n+struct worktree **get_worktrees_without_reading_head(void);\n+\n /*\n * Returns 1 if linked worktrees exist, 0 otherwise.\n */\n-- \n2.47.1\n\n"},{"id":"509932","messageId":"Z3qN6C2IpQTdVn_S@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 03/10] packed-backend: check whether the \"packed-refs\" is regular","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:49:28Z","receivedAt":"2025-01-05T13:49:33Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\nconsistency and correctness of \"packed-refs\" file, they never check the\nfiletype of the \"packed-refs\". The user should always use \"git\npacked-refs\" command to create the raw regular \"packed-refs\" file, so we\nneed to explicitly check this in \"git refs verify\".\n\nUse \"lstat\" to check the file mode. If we cannot check the file status,\nthis is OK because there is a chance that there is no \"packed-refs\" in\nthe repo.\n\nReuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\nthe user if \"packed-refs\" is not a regular file.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 33 +++++++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 20 ++++++++++++++++++++\n 2 files changed, 49 insertions(+), 4 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 3406f1e71d..d9eb2f8b71 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -4,6 +4,7 @@\n #include \"../config.h\"\n #include \"../dir.h\"\n #include \"../gettext.h\"\n+#include \"../fsck.h\"\n #include \"../hash.h\"\n #include \"../hex.h\"\n #include \"../refs.h\"\n@@ -1747,15 +1748,39 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n-static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED,\n+static int packed_fsck(struct ref_store *ref_store,\n+\t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n+\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n+\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct stat st;\n+\tint ret = 0;\n \n \tif (!is_main_worktree(wt))\n-\t\treturn 0;\n+\t\tgoto cleanup;\n \n-\treturn 0;\n+\t/*\n+\t * If the packed-refs file doesn't exist, there's nothing to\n+\t * check.\n+\t */\n+\tif (lstat(refs->path, &st) < 0)\n+\t\tgoto cleanup;\n+\n+\tif (o->verbose)\n+\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n+\n+\tif (!S_ISREG(st.st_mode)) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs\";\n+\n+\t\tret = fsck_report_ref(o, &report, FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t \"not a regular file\");\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\treturn ret;\n }\n \n struct ref_storage_be refs_be_packed = {\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 75f234a94a..307f94a3ca 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -626,4 +626,24 @@ test_expect_success 'ref content checks should work with worktrees' '\n \ttest_cmp expect err\n '\n \n+test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit branch branch-3 &&\n+\tgit pack-refs --all &&\n+\n+\tmv .git/packed-refs .git/packed-refs-back &&\n+\tln -sf packed-refs-bak .git/packed-refs &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: packed-refs: badRefFiletype: not a regular file\n+\tEOF\n+\trm .git/packed-refs &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.47.1\n\n"},{"id":"509933","messageId":"Z3qN8U2VbZBnUSWj@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 04/10] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:49:37Z","receivedAt":"2025-01-05T13:49:43Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c::create_snapshot\", if there is a header (the line\nwhich starts with '#'), we will check whether the line starts with \"#\npack-refs with:\". As we are going to implement the header consistency\ncheck, we should port this check into \"packed_fsck\".\n\nHowever, the above check is not enough, this is because \"git pack-refs\"\nwill always write \"PACKED_REFS_HEADER\" which is a constant string to the\n\"packed-refs\" file. So, we should check the following things for the\nheader.\n\n1. If the header does not exist, we may report an error to the user\n because it should exist, but we do allow no header in \"packed-refs\"\n file. So, create a new fsck message \"packedRefMissingHeader(INFO)\" to\n warn the user and also keep compatibility.\n2. If the header content does not start with \"# packed-ref with:\", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n3. If the header content is not the same as the constant string\n \"PACKED_REFS_HEADER\", ideally, we should report an error to the user.\n However, we allow other contents as long as the header content starts\n with \"# packed-ref with:\". To keep compatibility, create a new fsck\n message \"unknownPackedRefHeader(INFO)\" to warn about this. We may\n tighten this rule in the future.\n\nIn order to achieve above checks, read the \"packed-refs\" file via\n\"strbuf_read_file\". Like what \"create_snapshot\" and other functions do,\nwe could split the line by finding the next newline in the buf. If we\ncannot find a newline, this is an error.\n\nSo, create a function \"packed_fsck_ref_next_line\" to find the next\nnewline and if there is no such newline, use\n\"packedRefEntryNotTerminated(INFO)\" to report an error to the user.\n\nThen, parse the first line to apply the above three checks. Update the\ntest to excise the code.\n\nHowever, when adding the new test for a bad header, the program will\nstill die in the \"create_snapshot\" method. This is because we have\nchecked the files-backend firstly and we use \"parse_object\" to check\nwhether the object exists and whether the type is correct. This function\nwill eventually call \"create_snapshot\" and \"next_record\" method, if\nthere is something wrong with packed-backend, the program just dies.\n\nIt's bad to just die the program because we want to report the problems\nas many as possible. We should avoid checking object and its type when\npacked-backend is broken. So, we should first check the consistency of\nthe packed-backend then for files-backend.\n\nAdd a new flag \"safe_object_check\" in \"fsck_options\", when there is\nanything wrong with the parsing process, set this flag to 0 to avoid\nchecking objects in the later checks.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 16 ++++++\n fsck.h | 6 ++\n refs/files-backend.c | 6 +-\n refs/packed-backend.c | 105 ++++++++++++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 44 ++++++++++++++\n 5 files changed, 174 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex b14bc44ca4..34375a3143 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -16,6 +16,10 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefHeader`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid\n+\theader.\n+\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n@@ -176,6 +180,13 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`packedRefEntryNotTerminated`::\n+\t(ERROR) The \"packed-refs\" file contains an entry that is\n+\tnot terminated by a newline.\n+\n+`packedRefMissingHeader`::\n+\t(INFO) The \"packed-refs\" file does not contain the header.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\n@@ -208,6 +219,11 @@\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \n+`unknownPackedRefHeader`::\n+\t(INFO) The \"packed-refs\" header starts with \"# pack-refs with:\"\n+\tbut the remaining content is not the same as what `git pack-refs`\n+\twould write.\n+\n `unknownType`::\n \t(ERROR) Found an unknown object type.\n \ndiff --git a/fsck.h b/fsck.h\nindex a44c231a5f..026ad1d537 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n@@ -53,6 +54,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE, ERROR) \\\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\n@@ -90,6 +92,8 @@ enum fsck_msg_type {\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n \tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n+\tFUNC(UNKNOWN_PACKED_REF_HEADER, INFO) \\\n+\tFUNC(PACKED_REF_MISSING_HEADER, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \n@@ -163,6 +167,7 @@ struct fsck_options {\n \tfsck_error error_func;\n \tunsigned strict;\n \tunsigned verbose;\n+\tint safe_object_check;\n \tenum fsck_msg_type *msg_type;\n \tstruct oidset skip_oids;\n \tstruct oidset gitmodules_found;\n@@ -198,6 +203,7 @@ struct fsck_options {\n }\n #define FSCK_REFS_OPTIONS_DEFAULT { \\\n \t.error_func = fsck_refs_error_function, \\\n+\t.safe_object_check = 1, \\\n }\n \n /* descend in all linked child objects\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0a4912c009..66eae36184 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3599,7 +3599,7 @@ static int files_fsck_refs_oid(struct fsck_options *o,\n \tstruct object *obj;\n \tint ret = 0;\n \n-\tif (is_promisor_object(ref_store->repo, oid))\n+\tif (!o->safe_object_check || is_promisor_object(ref_store->repo, oid))\n \t\treturn 0;\n \n \tobj = parse_object(ref_store->repo, oid);\n@@ -3819,8 +3819,8 @@ static int files_fsck(struct ref_store *ref_store,\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"fsck\");\n \n-\treturn files_fsck_refs(ref_store, o, wt) |\n-\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o, wt);\n+\treturn refs->packed_ref_store->be->fsck(refs->packed_ref_store, o, wt) |\n+\t files_fsck_refs(ref_store, o, wt);\n }\n \n struct ref_storage_be refs_be_files = {\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex d9eb2f8b71..3b11abe5f8 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1748,12 +1748,100 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+static int packed_fsck_ref_next_line(struct fsck_options *o,\n+\t\t\t\t int line_number, const char *start,\n+\t\t\t\t const char *eof, const char **eol)\n+{\n+\tint ret = 0;\n+\n+\t*eol = memchr(start, '\\n', eof - start);\n+\tif (!*eol) {\n+\t\tstruct strbuf packed_entry = STRBUF_INIT;\n+\t\tstruct fsck_ref_report report = { 0 };\n+\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n+\t\t\t\t \"'%.*s' is not terminated with a newline\",\n+\t\t\t\t (int)(eof - start), start);\n+\n+\t\t/*\n+\t\t * There is no newline but we still want to parse it to the end of\n+\t\t * the buffer.\n+\t\t */\n+\t\t*eol = eof;\n+\t\tstrbuf_release(&packed_entry);\n+\t}\n+\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_header(struct fsck_options *o, const char *start, const char *eol)\n+{\n+\tconst char *err_fmt = NULL;\n+\tint fsck_msg_id = -1;\n+\n+\tif (!starts_with(start, \"# pack-refs with:\")) {\n+\t\terr_fmt = \"'%.*s' does not start with '# pack-refs with:'\";\n+\t\tfsck_msg_id = FSCK_MSG_BAD_PACKED_REF_HEADER;\n+\t} else if (strncmp(start, PACKED_REFS_HEADER, strlen(PACKED_REFS_HEADER))) {\n+\t\terr_fmt = \"'%.*s' is not the official packed-refs header\";\n+\t\tfsck_msg_id = FSCK_MSG_UNKNOWN_PACKED_REF_HEADER;\n+\t}\n+\n+\tif (err_fmt && fsck_msg_id >= 0) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs.header\";\n+\n+\t\treturn fsck_report_ref(o, &report, fsck_msg_id, err_fmt,\n+\t\t\t\t (int)(eol - start), start);\n+\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tint line_number = 1;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\tif (*start == '#') {\n+\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t} else {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs\";\n+\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_MISSING_HEADER,\n+\t\t\t\t \"missing header line\");\n+\t}\n+\n+\t/*\n+\t * If there is anything wrong during the parsing of the \"packed-refs\"\n+\t * file, we should not check the object of the refs.\n+\t */\n+\tif (ret)\n+\t\to->safe_object_check = 0;\n+\n+\n+\treturn ret;\n+}\n+\n static int packed_fsck(struct ref_store *ref_store,\n \t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tstruct stat st;\n \tint ret = 0;\n \n@@ -1779,7 +1867,24 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (strbuf_read_file(&packed_ref_content, refs->path, 0) < 0) {\n+\t\t/*\n+\t\t * Although we have checked that the file exists, there is a possibility\n+\t\t * that it has been removed between the lstat() and the read attempt by\n+\t\t * another process. In that case, we should not report an error.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tret = error_errno(\"could not read %s\", refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\n cleanup:\n+\tstrbuf_release(&packed_ref_content);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 307f94a3ca..6c729e749a 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -646,4 +646,48 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n \ttest_cmp expect err\n '\n \n+test_expect_success 'packed-refs header should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"$(git rev-parse main) refs/heads/main\\n\" >.git/packed-refs &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: packed-refs: packedRefMissingHeader: missing header line\n+\tEOF\n+\trm .git/packed-refs &&\n+\ttest_cmp expect err &&\n+\n+\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n+\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n+\t\t\t \"# pack-refs with a: peeled fully-peeled\"\n+\tdo\n+\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with:'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor unknown_header in \"# pack-refs with: peeled fully-peeled sorted garbage\" \\\n+\t\t\t \"# pack-refs with: peeled\" \\\n+\t\t\t \"# pack-refs with: peeled peeled-fully sort\"\n+\tdo\n+\t\tprintf \"%s\\n\" \"$unknown_header\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: packed-refs.header: unknownPackedRefHeader: '\\''$unknown_header'\\'' is not the official packed-refs header\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_done\n-- \n2.47.1\n\n"},{"id":"509934","messageId":"Z3qN_8-HKdspwcDb@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 05/10] packed-backend: check whether the refname contains NULL binaries","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:49:51Z","receivedAt":"2025-01-05T13:49:56Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already implemented the header consistency check for the raw\n\"packed-refs\" file. Before we implement the consistency check for each\nref entry, let's analysis [1] which reports that \"git fsck\" cannot\ndetect some binary zeros.\n\n\"packed-backend.c::next_record\" will use \"check_refname_format\" to check\nthe consistency of the refname. If it is not OK, the program will die.\nSo, we already have the code path and we must miss out something.\n\nWe use the following code to get the refname:\n\n strbuf_add(&iter->refname_buf, p, eol - p);\n iter->base.refname = iter->refname_buf.buf\n\nIn the above code, `p` is the start pointer of the refname and `eol` is\nthe next newline pointer. We calculate the length of the refname by\nsubtracting the two pointers. Then we add the memory range between `p`\nand `eol` to get the refname.\n\nHowever, if there are some NULL binaries in the memory range between `p`\nand `eol`, we will see the refname as a valid ref name as long as the\nmemory range between `p` and the first occurred NULL binary is valid.\n\nIn order to catch above corruption, create a new function\n\"refname_contains_null\" by checking whether the \"refname.len\" equals to\nthe length of the raw string pointer \"refname.buf\". If not equal, there\nmust be some NULL binaries in the refname.\n\nUse this function in \"next_record\" function to die the program if\n\"refname_contains_null\" returns true.\n\n[1] https://lore.kernel.org/git/6cfee0e4-3285-4f18-91ff-d097da9de737@rd10.de/\n\nReported-by: R. Diez \nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 20 ++++++++++++++++++++\n 1 file changed, 20 insertions(+)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 3b11abe5f8..f6142a4402 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -493,6 +493,23 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n \t\t\t\t last_line, eof - last_line);\n }\n \n+/*\n+ * When parsing the \"packed-refs\" file, we will parse it line by line.\n+ * Because we know the start pointer of the refname and the next\n+ * newline pointer, we could calculate the length of the refname by\n+ * subtracting the two pointers. However, there is a corner case where\n+ * the refname contains corrupted embedded NULL binaries. And\n+ * `check_refname_format()` will not catch this when the truncated\n+ * refname is still a valid refname. To prevent this, we need to check\n+ * whether the refname contains the NULL binaries.\n+ */\n+static int refname_contains_null(struct strbuf refname)\n+{\n+\tif (refname.len != strlen(refname.buf))\n+\t\treturn 1;\n+\treturn 0;\n+}\n+\n #define SMALL_FILE_SIZE (32*1024)\n \n /*\n@@ -894,6 +911,9 @@ static int next_record(struct packed_ref_iterator *iter)\n \tstrbuf_add(&iter->refname_buf, p, eol - p);\n \titer->base.refname = iter->refname_buf.buf;\n \n+\tif (refname_contains_null(iter->refname_buf))\n+\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n+\n \tif (check_refname_format(iter->base.refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!refname_is_safe(iter->base.refname))\n \t\t\tdie(\"packed refname is dangerous: %s\",\n-- \n2.47.1\n\n"},{"id":"509935","messageId":"Z3qOB2_zrDHOh-Gx@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 06/10] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:49:59Z","receivedAt":"2025-01-05T13:50:05Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will parse the ref entry to check the\nconsistency. This function has already checked the following things:\n\n1. Parse the main line of the ref entry, if the oid is not correct. It\n will die the program. And then it will check whether the next\n character of the oid is space. Then it will check whether the refname\n is correct.\n2. If the next line starts with '^', it will continue to parse the oid\n of the peeled oid content and check whether the last character is\n '\\n'.\n\nWe can iterate each line by using the \"packed_fsck_ref_unterminated_line\"\nfunction. Then, create a new fsck message \"badPackedRefEntry(ERROR)\" to\nreport to the user when something is wrong.\n\nCreate two new functions \"packed_fsck_ref_main_line\" and\n\"packed_fsck_ref_peeled_line\" for case 1 and case 2 respectively. Last,\nupdate the unit test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 105 +++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 40 +++++++++++++\n 4 files changed, 148 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 34375a3143..2a7ec7592e 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -16,6 +16,9 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefEntry`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid entry.\n+\n `badPackedRefHeader`::\n \t(ERROR) The \"packed-refs\" file contains an invalid\n \theader.\ndiff --git a/fsck.h b/fsck.h\nindex 026ad1d537..4fca304b72 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\n \tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex f6142a4402..6e521a9f87 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1822,7 +1822,96 @@ static int packed_fsck_ref_header(struct fsck_options *o, const char *start, con\n \treturn 0;\n }\n \n+static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store, int line_number,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf peeled_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id peeled;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&peeled_entry, \"packed-refs line %d\", line_number);\n+\treport.path = peeled_entry.buf;\n+\n+\tstart++;\n+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid peeled oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p != eol) {\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n+\t\t\t\t (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&peeled_entry);\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_main_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store, int line_number,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct strbuf refname = STRBUF_INIT;\n+\tstruct object_id oid;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n+\treport.path = packed_entry.buf;\n+\n+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p == eol || !isspace(*p)) {\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n+\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tp++;\n+\tstrbuf_add(&refname, p, eol - p);\n+\tif (refname_contains_null(refname)) {\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"refname '%s' contains NULL binaries\",\n+\t\t\t\t refname.buf);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (check_refname_format(refname.buf, 0)) {\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n+\t\t\t\t \"has bad refname '%s'\", refname.buf);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n \tint line_number = 1;\n@@ -1844,6 +1933,20 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t \"missing header line\");\n \t}\n \n+\twhile (start < eof) {\n+\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\tret |= packed_fsck_ref_main_line(o, ref_store, line_number, start, eol);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t\tif (start < eof && *start == '^') {\n+\t\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, line_number,\n+\t\t\t\t\t\t\t start, eol);\n+\t\t\tstart = eol + 1;\n+\t\t\tline_number++;\n+\t\t}\n+\t}\n+\n \t/*\n \t * If there is anything wrong during the parsing of the \"packed-refs\"\n \t * file, we should not check the object of the refs.\n@@ -1900,7 +2003,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 6c729e749a..7e8b329425 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -690,4 +690,44 @@ test_expect_success 'packed-refs header should be checked' '\n \tdone\n '\n \n+test_expect_success 'packed-refs content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\tgit tag -a annotated-tag-2 -m tag-2 &&\n+\n+\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n+\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n+\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n+\n+\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\tprintf \"%s\\n\" \"$short_oid refs/heads/branch-1\" >>.git/packed-refs &&\n+\tprintf \"%sx\\n\" \"$branch_1_oid\" >>.git/packed-refs &&\n+\tprintf \"%s refs/heads/bad-branch\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n+\tprintf \"%s refs/heads/branch.\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n+\tprintf \"%s refs/tags/annotated-tag-3\\n\" \"$tag_1_oid\" >>.git/packed-refs &&\n+\tprintf \"^%s\\n\" \"$short_oid\" >>.git/packed-refs &&\n+\tprintf \"%s refs/tags/annotated-tag-4.\\n\" \"$tag_2_oid\" >>.git/packed-refs &&\n+\tprintf \"^%s garbage\\n\" \"$tag_2_peeled_oid\" >>.git/packed-refs &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: packed-refs line 2: badPackedRefEntry: '\\''$short_oid refs/heads/branch-1'\\'' has invalid oid\n+\terror: packed-refs line 3: badPackedRefEntry: has no space after oid '\\''$branch_1_oid'\\'' but with '\\''x'\\''\n+\terror: packed-refs line 4: badRefName: has bad refname '\\'' refs/heads/bad-branch'\\''\n+\terror: packed-refs line 5: badRefName: has bad refname '\\''refs/heads/branch.'\\''\n+\terror: packed-refs line 7: badPackedRefEntry: '\\''$short_oid'\\'' has invalid peeled oid\n+\terror: packed-refs line 8: badRefName: has bad refname '\\''refs/tags/annotated-tag-4.'\\''\n+\terror: packed-refs line 9: badPackedRefEntry: has trailing garbage after peeled oid '\\'' garbage'\\''\n+\tEOF\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.47.1\n\n"},{"id":"509936","messageId":"Z3qOEvyeoc7vOW73@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 07/10] packed-backend: create \"fsck_packed_ref_entry\" to store parsing info","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:50:10Z","receivedAt":"2025-01-05T13:50:16Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already check whether the oid hash is correct by using\n`parse_oid_hex_algop`. However, we doesn't check whether the object\nexists. It may seem that we could do this when we are parsing the raw\n\"packed-refs\" file. But this is impossible. Let's analysis why.\n\nWe will use \"parse_object\" function to get the \"struct object\". However,\nthis function will eventually call the \"create_snapshot\" and\n\"next_record\" function in \"packed-backend.c\". If there is anything\nwrong, it will die the program. And we don't want to die the program\nduring the check.\n\nSo, we should store the information in the parsing process. And if there\nis nothing wrong in the parsing process, we could continue to check\nthings. So, create \"fsck_packed_ref_entry\" to do this.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 56 ++++++++++++++++++++++++++++++++++---------\n 1 file changed, 45 insertions(+), 11 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 6e521a9f87..7386e6bfce 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1768,6 +1768,29 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+struct fsck_packed_ref_entry {\n+\tint line_number;\n+\n+\tint has_peeled;\n+\tstruct object_id oid;\n+\tstruct object_id peeled;\n+};\n+\n+static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(int line_number)\n+{\n+\tstruct fsck_packed_ref_entry *entry = xcalloc(1, sizeof(*entry));\n+\tentry->line_number = line_number;\n+\tentry->has_peeled = 0;\n+\treturn entry;\n+}\n+\n+static void free_fsck_packed_ref_entries(struct fsck_packed_ref_entry **entries, int nr)\n+{\n+\tfor (int i = 0; i < nr; i++)\n+\t\tfree(entries[i]);\n+\tfree(entries);\n+}\n+\n static int packed_fsck_ref_next_line(struct fsck_options *o,\n \t\t\t\t int line_number, const char *start,\n \t\t\t\t const char *eof, const char **eol)\n@@ -1823,20 +1846,20 @@ static int packed_fsck_ref_header(struct fsck_options *o, const char *start, con\n }\n \n static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n-\t\t\t\t struct ref_store *ref_store, int line_number,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t struct fsck_packed_ref_entry *entry,\n \t\t\t\t const char *start, const char *eol)\n {\n \tstruct strbuf peeled_entry = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n-\tstruct object_id peeled;\n \tconst char *p;\n \tint ret = 0;\n \n-\tstrbuf_addf(&peeled_entry, \"packed-refs line %d\", line_number);\n+\tstrbuf_addf(&peeled_entry, \"packed-refs line %d\", entry->line_number + 1);\n \treport.path = peeled_entry.buf;\n \n \tstart++;\n-\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n+\tif (parse_oid_hex_algop(start, &entry->peeled, &p, ref_store->repo->hash_algo)) {\n \t\tret |= fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n \t\t\t\t \"'%.*s' has invalid peeled oid\",\n@@ -1858,20 +1881,20 @@ static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n }\n \n static int packed_fsck_ref_main_line(struct fsck_options *o,\n-\t\t\t\t struct ref_store *ref_store, int line_number,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t struct fsck_packed_ref_entry *entry,\n \t\t\t\t const char *start, const char *eol)\n {\n \tstruct strbuf packed_entry = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n \tstruct strbuf refname = STRBUF_INIT;\n-\tstruct object_id oid;\n \tconst char *p;\n \tint ret = 0;\n \n-\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n+\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", entry->line_number);\n \treport.path = packed_entry.buf;\n \n-\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n+\tif (parse_oid_hex_algop(start, &entry->oid, &p, ref_store->repo->hash_algo)) {\n \t\tret |= fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n \t\t\t\t \"'%.*s' has invalid oid\",\n@@ -1883,7 +1906,7 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n \t\tret |= fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n \t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n-\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n+\t\t\t\t oid_to_hex(&entry->oid), (int)(eol - p), p);\n \t\tgoto cleanup;\n \t}\n \n@@ -1914,7 +1937,10 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n+\tstruct fsck_packed_ref_entry **entries;\n+\tint entry_alloc = 20;\n \tint line_number = 1;\n+\tint entry_nr = 0;\n \tconst char *eol;\n \tint ret = 0;\n \n@@ -1933,14 +1959,21 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t \"missing header line\");\n \t}\n \n+\tALLOC_ARRAY(entries, entry_alloc);\n \twhile (start < eof) {\n+\t\tstruct fsck_packed_ref_entry *entry\n+\t\t\t= create_fsck_packed_ref_entry(line_number);\n+\t\tALLOC_GROW(entries, entry_nr + 1, entry_alloc);\n+\t\tentries[entry_nr++] = entry;\n+\n \t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n-\t\tret |= packed_fsck_ref_main_line(o, ref_store, line_number, start, eol);\n+\t\tret |= packed_fsck_ref_main_line(o, ref_store, entry, start, eol);\n \t\tstart = eol + 1;\n \t\tline_number++;\n \t\tif (start < eof && *start == '^') {\n+\t\t\tentry->has_peeled = 1;\n \t\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n-\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, line_number,\n+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, entry,\n \t\t\t\t\t\t\t start, eol);\n \t\t\tstart = eol + 1;\n \t\t\tline_number++;\n@@ -1955,6 +1988,7 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\to->safe_object_check = 0;\n \n \n+\tfree_fsck_packed_ref_entries(entries, entry_nr);\n \treturn ret;\n }\n \n-- \n2.47.1\n\n"},{"id":"509937","messageId":"Z3qOGzfncHlnZOGY@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 08/10] packed-backend: add check for object consistency","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:50:19Z","receivedAt":"2025-01-05T13:50:24Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"If there is nothing wrong when parsing the raw file \"packed-refs\", we\ncould then iterate the \"entries\" to check the object consistency. There\nare two kinds of ref entry: one is the normal and another is peeled. For\nboth situations, we need to use \"parse_object\" function to parse the\nobject id to get the object. If the object does not exist, we will\nreport an error to the user.\n\nCreate a new function \"packed_fsck_ref_oid\" to do above then update the\nunit test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 50 +++++++++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 35 ++++++++++++++++++++++++++++\n 2 files changed, 84 insertions(+), 1 deletion(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 7386e6bfce..d83ce2838f 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -13,6 +13,7 @@\n #include \"../iterator.h\"\n #include \"../lockfile.h\"\n #include \"../chdir-notify.h\"\n+#include \"../packfile.h\"\n #include \"../statinfo.h\"\n #include \"../worktree.h\"\n #include \"../wrapper.h\"\n@@ -1933,6 +1934,52 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n \treturn ret;\n }\n \n+static int packed_fsck_ref_oid(struct fsck_options *o, struct ref_store *ref_store,\n+\t\t\t struct fsck_packed_ref_entry **entries, int nr)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object *obj;\n+\tint ret = 0;\n+\n+\tfor (int i = 0; i < nr; i++) {\n+\t\tstruct fsck_packed_ref_entry *entry = entries[i];\n+\n+\t\tstrbuf_release(&packed_entry);\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", entry->line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tif (is_promisor_object(ref_store->repo, &entry->oid))\n+\t\t\tcontinue;\n+\n+\t\tobj = parse_object(ref_store->repo, &entry->oid);\n+\t\tif (!obj) {\n+\t\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t\t \"'%s' is not a valid object\",\n+\t\t\t\t\t oid_to_hex(&entry->oid));\n+\t\t}\n+\t\tif (entry->has_peeled) {\n+\t\t\tstrbuf_reset(&packed_entry);\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\",\n+\t\t\t\t entry->line_number + 1);\n+\t\t\treport.path = packed_entry.buf;\n+\n+\t\t\tobj = parse_object(ref_store->repo, &entry->peeled);\n+\t\t\tif (!obj) {\n+\t\t\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t\t\t \"'%s' is not a valid object\",\n+\t\t\t\t\t\t oid_to_hex(&entry->peeled));\n+\t\t\t}\n+\t\t}\n+\n+\t}\n+\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n@@ -1986,7 +2033,8 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t */\n \tif (ret)\n \t\to->safe_object_check = 0;\n-\n+\telse\n+\t\tret |= packed_fsck_ref_oid(o, ref_store, entries, entry_nr);\n \n \tfree_fsck_packed_ref_entries(entries, entry_nr);\n \treturn ret;\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 7e8b329425..faa7c80356 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -730,4 +730,39 @@ test_expect_success 'packed-refs content should be checked' '\n \ttest_cmp expect err\n '\n \n+test_expect_success 'packed-refs objects should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\n+\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\n+\tfor non_existing_oid in \"$(test_oid 001)\" \"$(test_oid 002)\"\n+\tdo\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\t\tprintf \"%s refs/heads/foo\\n\" \"$non_existing_oid\" >>.git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: badPackedRefEntry: '\\''$non_existing_oid'\\'' is not a valid object\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor non_existing_oid in \"$(test_oid 001)\" \"$(test_oid 002)\"\n+\tdo\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\t\tprintf \"%s refs/tags/foo\\n\" \"$tag_1_oid\" >>.git/packed-refs &&\n+\t\tprintf \"^$non_existing_oid\\n\" >>.git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 3: badPackedRefEntry: '\\''$non_existing_oid'\\'' is not a valid object\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_done\n-- \n2.47.1\n\n"},{"id":"509938","messageId":"Z3qOJ_ixuoE4yTut@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 09/10] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:50:31Z","receivedAt":"2025-01-05T13:50:39Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We will always try to sort the \"packed-refs\" increasingly by comparing\nthe refname. So, we should add checks to verify whether the \"packed-refs\"\nis sorted.\n\nIt may seem that we could add a new \"struct strbuf refname\" into the\n\"struct fsck_packed_ref_entry\" and during the parsing process, we could\nstore the refname into the entry and then we could compare later.\nHowever, this is not a good design due to the following reasons:\n\n1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n in the \"packed-refs\" file.\n2. The most important is that we cannot reuse the existing compare\n functions which cause repetition.\n\nSo, instead of storing the \"struct strbuf\", let's use the existing\nstructure \"struct snaphost_record\". And thus we could use the existing\nfunction \"cmp_packed_ref_records\".\n\nHowever, this function need an extra parameter for \"struct snaphost\".\nExtract the common part into a new function \"cmp_packed_ref_records\" to\nreuse this function to compare.\n\nThen, create a new function \"packed_fsck_ref_sorted\" to use the new fsck\nmessage \"packedRefUnsorted(ERROR)\" to report to the user.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 ++\n fsck.h | 1 +\n refs/packed-backend.c | 78 ++++++++++++++++++++++++++++++-----\n t/t0602-reffiles-fsck.sh | 40 ++++++++++++++++++\n 4 files changed, 111 insertions(+), 11 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 2a7ec7592e..7a11d35c5e 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -190,6 +190,9 @@\n `packedRefMissingHeader`::\n \t(INFO) The \"packed-refs\" file does not contain the header.\n \n+`packedRefUnsorted`::\n+\t(ERROR) The \"packed-refs\" file is not sorted.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex 4fca304b72..1be7402eb9 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -56,6 +56,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n \tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n+\tFUNC(PACKED_REF_UNSORTED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex d83ce2838f..df65fec5a5 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -300,14 +300,8 @@ struct snapshot_record {\n \tsize_t len;\n };\n \n-static int cmp_packed_ref_records(const void *v1, const void *v2,\n-\t\t\t\t void *cb_data)\n+static int cmp_packed_refname(const char *r1, const char *r2)\n {\n-\tconst struct snapshot *snapshot = cb_data;\n-\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n-\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n-\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n-\n \twhile (1) {\n \t\tif (*r1 == '\\n')\n \t\t\treturn *r2 == '\\n' ? 0 : -1;\n@@ -322,6 +316,17 @@ static int cmp_packed_ref_records(const void *v1, const void *v2,\n \t}\n }\n \n+static int cmp_packed_ref_records(const void *v1, const void *v2,\n+\t\t\t\t void *cb_data)\n+{\n+\tconst struct snapshot *snapshot = cb_data;\n+\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n+\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n+\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n+\n+\treturn cmp_packed_refname(r1, r2);\n+}\n+\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n@@ -1775,13 +1780,17 @@ struct fsck_packed_ref_entry {\n \tint has_peeled;\n \tstruct object_id oid;\n \tstruct object_id peeled;\n+\n+\tstruct snapshot_record record;\n };\n \n-static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(int line_number)\n+static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(int line_number,\n+\t\t\t\t\t\t\t\t const char *start)\n {\n \tstruct fsck_packed_ref_entry *entry = xcalloc(1, sizeof(*entry));\n \tentry->line_number = line_number;\n \tentry->has_peeled = 0;\n+\tentry->record.start = start;\n \treturn entry;\n }\n \n@@ -1980,6 +1989,50 @@ static int packed_fsck_ref_oid(struct fsck_options *o, struct ref_store *ref_sto\n \treturn ret;\n }\n \n+static int packed_fsck_ref_sorted(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t struct fsck_packed_ref_entry **entries,\n+\t\t\t\t int nr)\n+{\n+\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct strbuf refname1 = STRBUF_INIT;\n+\tstruct strbuf refname2 = STRBUF_INIT;\n+\tint ret = 0;\n+\n+\tfor (int i = 1; i < nr; i++) {\n+\t\tconst char *r1 = entries[i - 1]->record.start + hexsz + 1;\n+\t\tconst char *r2 = entries[i]->record.start + hexsz + 1;\n+\n+\t\tif (cmp_packed_refname(r1, r2) >= 0) {\n+\t\t\tconst char *err_fmt =\n+\t\t\t\t\"refname '%s' is not less than next refname '%s'\";\n+\t\t\tconst char *eol;\n+\t\t\teol = memchr(entries[i - 1]->record.start, '\\n',\n+\t\t\t\t entries[i - 1]->record.len);\n+\t\t\tstrbuf_add(&refname1, r1, eol - r1);\n+\t\t\teol = memchr(entries[i]->record.start, '\\n',\n+\t\t\t\t entries[i]->record.len);\n+\t\t\tstrbuf_add(&refname2, r2, eol - r2);\n+\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\",\n+\t\t\t\t entries[i - 1]->line_number);\n+\t\t\treport.path = packed_entry.buf;\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_PACKED_REF_UNSORTED,\n+\t\t\t\t\t err_fmt, refname1.buf, refname2.buf);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname1);\n+\tstrbuf_release(&refname2);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n@@ -2009,7 +2062,7 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \tALLOC_ARRAY(entries, entry_alloc);\n \twhile (start < eof) {\n \t\tstruct fsck_packed_ref_entry *entry\n-\t\t\t= create_fsck_packed_ref_entry(line_number);\n+\t\t\t= create_fsck_packed_ref_entry(line_number, start);\n \t\tALLOC_GROW(entries, entry_nr + 1, entry_alloc);\n \t\tentries[entry_nr++] = entry;\n \n@@ -2025,16 +2078,19 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\tstart = eol + 1;\n \t\t\tline_number++;\n \t\t}\n+\t\tentry->record.len = start - entry->record.start;\n \t}\n \n \t/*\n \t * If there is anything wrong during the parsing of the \"packed-refs\"\n \t * file, we should not check the object of the refs.\n \t */\n-\tif (ret)\n+\tif (ret) {\n \t\to->safe_object_check = 0;\n-\telse\n+\t} else {\n \t\tret |= packed_fsck_ref_oid(o, ref_store, entries, entry_nr);\n+\t\tret |= packed_fsck_ref_sorted(o, ref_store, entries, entry_nr);\n+\t}\n \n \tfree_fsck_packed_ref_entries(entries, entry_nr);\n \treturn ret;\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex faa7c80356..800a19e4e6 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -765,4 +765,44 @@ test_expect_success 'packed-refs objects should be checked' '\n \tdone\n '\n \n+test_expect_success 'packed-ref sorted should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\n+\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\n+\trefname1=\"refs/heads/main\" &&\n+\trefname2=\"refs/heads/foo\" &&\n+\trefname3=\"refs/tags/foo\" &&\n+\n+\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n+\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname1'\\'' is not less than next refname '\\''$refname2'\\''\n+\tEOF\n+\trm .git/packed-refs &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\tprintf \"%s %s\\n\" \"$tag_1_oid\" \"$refname3\" >>.git/packed-refs &&\n+\tprintf \"^%s\\n\" \"$tag_1_peeled_oid\" >>.git/packed-refs &&\n+\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname2\" >>.git/packed-refs &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname3'\\'' is not less than next refname '\\''$refname2'\\''\n+\tEOF\n+\trm .git/packed-refs &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.47.1\n\n"},{"id":"509939","messageId":"Z3qOM5M1ioZ0Px4T@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH 10/10] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-05T13:50:43Z","receivedAt":"2025-01-05T13:50:50Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"At now, we have already implemented the ref consistency checks for both\n\"files-backend\" and \"packed-backend\". Although we would check some\nredundant things, it won't cause trouble. So, let's integrate it into\nthe \"git-fsck(1)\" command to get feedback from the users. And also by\ncalling \"git refs verify\" in \"git-fsck(1)\", we make sure that the new\nadded checks don't break.\n\nIntroduce a new function \"fsck_refs\" that initializes and runs a child\nprocess to execute the \"git refs verify\" command. In order to provide\nthe user interface create a progress which makes the total task be 1.\nIt's hard to know how many loose refs we will check now. We might\nimprove this later.\n\nAnd we run this function in the first execution sequence of\n\"git-fsck(1)\" because we don't want the existing code of \"git-fsck(1)\"\nwhich implicitly checks the consistency of refs to die the program.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/fsck.c | 28 ++++++++++++++++++++++++++++\n 1 file changed, 28 insertions(+)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 0196c54eb6..a10e52b601 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -902,6 +902,32 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n \treturn res;\n }\n \n+static void fsck_refs(void)\n+{\n+\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n+\tstruct progress *progress = NULL;\n+\n+\tif (show_progress)\n+\t\tprogress = start_progress(_(\"Checking ref database\"), 1);\n+\n+\tif (verbose)\n+\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n+\n+\tchild_process_init(&refs_verify);\n+\trefs_verify.git_cmd = 1;\n+\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n+\tif (verbose)\n+\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n+\tif (check_strict)\n+\t\tstrvec_push(&refs_verify.args, \"--strict\");\n+\n+\tif (run_command(&refs_verify))\n+\t\terrors_found |= ERROR_REFS;\n+\n+\tdisplay_progress(progress, 1);\n+\tstop_progress(&progress);\n+}\n+\n static char const * const fsck_usage[] = {\n \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n@@ -967,6 +993,8 @@ int cmd_fsck(int argc,\n \tgit_config(git_fsck_config, &fsck_obj_options);\n \tprepare_repo_settings(the_repository);\n \n+\tfsck_refs();\n+\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n \t\tfor_each_packed_object(the_repository,\n-- \n2.47.1\n\n"},{"id":"510021","messageId":"xmqqv7urwpu1.fsf@gitster.g","threadId":"62743","inReplyTo":"Z3qOM5M1ioZ0Px4T@ArchLinux","subject":"Re: [PATCH 10/10] builtin/fsck: add `git refs verify` child process","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-06T22:16:22Z","receivedAt":"2025-01-06T22:16:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> builtin/fsck.c | 28 ++++++++++++++++++++++++++++\n> 1 file changed, 28 insertions(+)\n>\n> diff --git a/builtin/fsck.c b/builtin/fsck.c\n> index 0196c54eb6..a10e52b601 100644\n> --- a/builtin/fsck.c\n> +++ b/builtin/fsck.c\n> @@ -902,6 +902,32 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n> \treturn res;\n> }\n> \n> +static void fsck_refs(void)\n> +{\n> +\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n> +\tstruct progress *progress = NULL;\n> +\n> +\tif (show_progress)\n> +\t\tprogress = start_progress(_(\"Checking ref database\"), 1);\n\nThis had an obvious semantic conflicts with a topic in flight.\n\nI've resolved it in the latest integration after pushing out the\n2.48-rc2 this morning, so there is no need to resend, but please\nremember that it would be a possibility to rebase on top of an\nupdated 'master' *IF* the other topic graduates to 'master' a lot\nearlier than this topic hits 'next' (IOW, until that happens there\nis no need to rebase).\n\nThanks.\n"},{"id":"510063","messageId":"Z30XXCel6Fd7Thp9@ArchLinux","threadId":"62743","inReplyTo":"xmqqv7urwpu1.fsf@gitster.g","subject":"Re: [PATCH 10/10] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-07T12:00:28Z","receivedAt":"2025-01-07T11:59:26Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Jan 06, 2025 at 02:16:22PM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > builtin/fsck.c | 28 ++++++++++++++++++++++++++++\n> > 1 file changed, 28 insertions(+)\n> >\n> > diff --git a/builtin/fsck.c b/builtin/fsck.c\n> > index 0196c54eb6..a10e52b601 100644\n> > --- a/builtin/fsck.c\n> > +++ b/builtin/fsck.c\n> > @@ -902,6 +902,32 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n> > \treturn res;\n> > }\n> > \n> > +static void fsck_refs(void)\n> > +{\n> > +\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n> > +\tstruct progress *progress = NULL;\n> > +\n> > +\tif (show_progress)\n> > +\t\tprogress = start_progress(_(\"Checking ref database\"), 1);\n> \n> This had an obvious semantic conflicts with a topic in flight.\n> \n> I've resolved it in the latest integration after pushing out the\n> 2.48-rc2 this morning, so there is no need to resend, but please\n> remember that it would be a possibility to rebase on top of an\n> updated 'master' *IF* the other topic graduates to 'master' a lot\n> earlier than this topic hits 'next' (IOW, until that happens there\n> is no need to rebase).\n> \n\nThanks for the careful notification. I'll watch this.\n\n> Thanks.\n\nThanks.\n"},{"id":"510081","messageId":"CAOLa=ZRG_==uXF8RaTjOUzV932bg8xxEx8HfgqDLWQ1OMzd+3w@mail.gmail.com","threadId":"62743","inReplyTo":"Z3qN1T3lJoj82ckl@ArchLinux","subject":"Re: [PATCH 01/10] files-backend: add object check for regular ref","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-01-07T14:17:43Z","receivedAt":"2025-01-07T14:17:46Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> Although we use \"parse_loose_ref_content\" to check whether the object id\n> is correct, we never parse it into the \"struct object\" structure thus we\n> ignore checking whether there is a real object existing in the repo and\n> whether the object type is correct.\n>\n> Use \"parse_object\" to parse the oid for the regular ref content. If the\n> object does not exist, report the error to the user by reusing the fsck\n> message \"BAD_REF_CONTENT\".\n>\n> Then, we need to check the type of the object. Just like \"git-fsck(1)\",\n> we only report \"not a commit\" error when the ref is a branch. Last,\n> update the test to exercise the code.\n\nI found this a bit confusing at first, the code does clear up the\nconfusion. Perhaps we can say something like:\n\n Branches that do not point to a commit type are explicitly called out,\n similar to 'git-fsck(1)'.\n\n>\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> refs/files-backend.c | 50 ++++++++++++++++++++++++++++++++--------\n> t/t0602-reffiles-fsck.sh | 30 ++++++++++++++++++++++++\n> 2 files changed, 70 insertions(+), 10 deletions(-)\n>\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 64f51f0da9..0a4912c009 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -20,6 +20,7 @@\n> #include \"../lockfile.h\"\n> #include \"../object.h\"\n> #include \"../object-file.h\"\n> +#include \"../packfile.h\"\n> #include \"../path.h\"\n> #include \"../dir.h\"\n> #include \"../chdir-notify.h\"\n> @@ -3589,6 +3590,34 @@ static int files_fsck_symref_target(struct fsck_options *o,\n> \treturn ret;\n> }\n>\n> +static int files_fsck_refs_oid(struct fsck_options *o,\n> +\t\t\t struct ref_store *ref_store,\n> +\t\t\t struct fsck_ref_report report,\n> +\t\t\t const char *target_name,\n> +\t\t\t struct object_id *oid)\n> +{\n> +\tstruct object *obj;\n> +\tint ret = 0;\n> +\n> +\tif (is_promisor_object(ref_store->repo, oid))\n> +\t\treturn 0;\n> +\n> +\tobj = parse_object(ref_store->repo, oid);\n> +\tif (!obj) {\n> +\t\tret |= fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> +\t\t\t\t \"points to non-existing object %s\",\n> +\t\t\t\t oid_to_hex(oid));\n\nNit: The two conditionals here are mutually exclusive. So we don't have\nto do `ret |=`, no? We don't even need `ret` here, we could simply do a\n`return fsck_report_ref(...)`.\n\n> +\t} else if (obj->type != OBJ_COMMIT && is_branch(target_name)) {\n> +\t\tret |= fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> +\t\t\t\t \"points to non-commit object %s\",\n> +\t\t\t\t oid_to_hex(oid));\n> +\t}\n\nSince this is a single lined if/else, we can skip the braces here.\n\n> +\treturn ret;\n> +}\n> +\n> static int files_fsck_refs_content(struct ref_store *ref_store,\n> \t\t\t\t struct fsck_options *o,\n> \t\t\t\t const char *target_name,\n> @@ -3654,18 +3683,19 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> \t}\n>\n> \tif (!(type & REF_ISSYMREF)) {\n> +\t\tret |= files_fsck_refs_oid(o, ref_store, report, target_name, &oid);\n> +\n> \t\tif (!*trailing) {\n> -\t\t\tret = fsck_report_ref(o, &report,\n> -\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> -\t\t\t\t\t \"misses LF at the end\");\n> -\t\t\tgoto cleanup;\n> -\t\t}\n> -\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n> -\t\t\tret = fsck_report_ref(o, &report,\n> -\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> -\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n> -\t\t\tgoto cleanup;\n> +\t\t\tret |= fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> +\t\t\t\t\t \"misses LF at the end\");\n> +\t\t} else if (*trailing != '\\n' || *(trailing + 1)) {\n> +\t\t\tret |= fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> +\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n> \t\t}\n> +\n> +\t\tgoto cleanup;\n> \t} else {\n> \t\tret = files_fsck_symref_target(o, &report, &referent, 0);\n> \t\tgoto cleanup;\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index d4a08b823b..75f234a94a 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -161,8 +161,10 @@ test_expect_success 'regular ref content should be checked (individual)' '\n> \ttest_when_finished \"rm -rf repo\" &&\n> \tgit init repo &&\n> \tbranch_dir_prefix=.git/refs/heads &&\n> +\ttag_dir_prefix=.git/refs/tags &&\n> \tcd repo &&\n> \ttest_commit default &&\n> +\tgit branch branch-1 &&\n> \tmkdir -p \"$branch_dir_prefix/a/b\" &&\n>\n> \tgit refs verify 2>err &&\n> @@ -198,6 +200,28 @@ test_expect_success 'regular ref content should be checked (individual)' '\n> \trm $branch_dir_prefix/branch-no-newline &&\n> \ttest_cmp expect err &&\n>\n> +\tfor non_existing_oid in \"$(test_oid 001)\" \"$(test_oid 002)\"\n> +\tdo\n> +\t\tprintf \"%s\\n\" $non_existing_oid >$branch_dir_prefix/invalid-commit &&\n> +\t\ttest_must_fail git refs verify 2>err &&\n> +\t\tcat >expect <<-EOF &&\n> +\t\terror: refs/heads/invalid-commit: badRefContent: points to non-existing object $non_existing_oid\n> +\t\tEOF\n> +\t\trm $branch_dir_prefix/invalid-commit &&\n> +\t\ttest_cmp expect err || return 1\n> +\tdone &&\n> +\n> +\tfor tree_oid in \"$(git rev-parse main^{tree})\" \"$(git rev-parse branch-1^{tree})\"\n> +\tdo\n> +\t\tprintf \"%s\\n\" $tree_oid >$branch_dir_prefix/branch-tree &&\n> +\t\ttest_must_fail git refs verify 2>err &&\n> +\t\tcat >expect <<-EOF &&\n> +\t\terror: refs/heads/branch-tree: badRefContent: points to non-commit object $tree_oid\n\nReading this error here, I think it would be nicer to say\n'badRefContent: branch points to ....' so we know that the specified ref\nis a branch.\n\n> +\t\tEOF\n> +\t\trm $branch_dir_prefix/branch-tree &&\n> +\t\ttest_cmp expect err || return 1\n> +\tdone &&\n> +\n> \tfor trailing_content in \" garbage\" \" more garbage\"\n> \tdo\n> \t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n> @@ -244,15 +268,21 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n> \tbad_content_1=$(git rev-parse main)x &&\n> \tbad_content_2=xfsazqfxcadas &&\n> \tbad_content_3=Xfsazqfxcadas &&\n> +\tnon_existing_oid=$(test_oid 001) &&\n> +\ttree_oid=$(git rev-parse main^{tree}) &&\n> \tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n> \tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n> \tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n> \tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n> \tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n> +\tprintf \"%s\\n\" $non_existing_oid >$branch_dir_prefix/branch-non-existing-oid &&\n> +\tprintf \"%s\\n\" $tree_oid >$branch_dir_prefix/branch-tree &&\n>\n> \ttest_must_fail git refs verify 2>err &&\n> \tcat >expect <<-EOF &&\n> \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n> +\terror: refs/heads/branch-non-existing-oid: badRefContent: points to non-existing object $non_existing_oid\n> +\terror: refs/heads/branch-tree: badRefContent: points to non-commit object $tree_oid\n> \terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n> \terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n> \twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n> --\n> 2.47.1\n"},{"id":"510082","messageId":"CAOLa=ZQ6J9GLQjJihKxbDwH6SmHbmVq4sHrKh0ZtMqyEt3hsiw@mail.gmail.com","threadId":"62743","inReplyTo":"Z3qN30z1NCXa3AX-@ArchLinux","subject":"Re: [PATCH 02/10] builtin/refs.h: get worktrees without reading head info","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-01-07T14:57:08Z","receivedAt":"2025-01-07T14:57:10Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\n> and \"next_record\" which would check the correctness of the content of\n> the \"packed-ref\" file. When anything is bad, the program will die.\n\nSo you're saying, `create_snapshot()` and `next_record()` exit the\nprogram on any error. Okay that seems to be valid.\n\n> It may seem that we have nothing relevant to above feature, because we\n> are going to read and parse the raw \"packed-ref\" file without creating\n> the snapshot and using the ref iterator to check the consistency.\n>\n> However, when using \"get_worktrees\" in \"builtin/refs\", we will parse the\n> head information. If the referent of the \"HEAD\" is inside the\n> \"packed-ref\", we will call \"create_snapshot\" and \"next_record\" functions\n> to parse the \"packed-ref\" to get the head information. And if there are\n> something wrong, the program will die.\n>\n> Although this behavior has no harm for the program, it will\n> short-circuit the program. When the users execute \"git refs verify\" or\n> \"git fsck\", we don't want to simply die the program but rather show the\n> warnings or errors as many as possible to info the users. So, we should\n> avoiding reading the head info.\n>\n\nThis is a bit tricky here. If the information for the `HEAD` ref is\nincorrect in the packed-refs, git would exit early. Which is what we're\ntrying to avoid in this patch, by using the `get_worktrees_internal()`\nfunction.\n\nHowever, I would question if this is the right approach. Shouldn't\n`get_worktree()` failing indicate that the repository is invalid? In\nthat case does it really make sense to allow the user to even run `git\nrefs verify`? Isn't the prerequisite for running the `git-refs(1)`\ncommand a valid repository?\n\nGenerally, I'd agree that we try to obtain all errors so that the user\ncan get a full picture. But exposing internal worktree functions so we\ntreat invalid repos as valid repos so we can do that, seems a bit of a\nstretch.\n\n> Fortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\n> worktrees, 2023-12-29), we have introduced a function\n> \"get_worktrees_internal\" which allows us to get worktrees without\n> reading head info.\n>\n> Create a new exposed function \"get_worktrees_without_reading_head\", then\n> replace the \"get_worktrees\" in \"builtin/refs\" with the new created\n> function.\n>\n\n[snip]\n"},{"id":"510089","messageId":"xmqqwmf6vcxa.fsf@gitster.g","threadId":"62743","inReplyTo":"Z30XXCel6Fd7Thp9@ArchLinux","subject":"Re: [PATCH 10/10] builtin/fsck: add `git refs verify` child process","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-07T15:52:49Z","receivedAt":"2025-01-07T15:52:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n>> I've resolved it in the latest integration after pushing out the\n>> 2.48-rc2 this morning, so there is no need to resend, but please\n>> remember that it would be a possibility to rebase on top of an\n>> updated 'master' *IF* the other topic graduates to 'master' a lot\n>> earlier than this topic hits 'next' (IOW, until that happens there\n>> is no need to rebase).\n>> \n>\n> Thanks for the careful notification. I'll watch this.\n\nFor future reference and to help those who may be reading from the\nsidelines, it is a good practice to see how your topic interacts\nwith other things in flight by making a trial merge to 'next' and to\n'seen'. It would give you an opportunity to learn about what other\npeople are actively doing in the project.\n\nThanks.\n"},{"id":"510094","messageId":"Z31Xj6sZk1th2mRQ@ArchLinux","threadId":"62743","inReplyTo":"CAOLa=ZQ6J9GLQjJihKxbDwH6SmHbmVq4sHrKh0ZtMqyEt3hsiw@mail.gmail.com","subject":"Re: [PATCH 02/10] builtin/refs.h: get worktrees without reading head info","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-07T16:34:23Z","receivedAt":"2025-01-07T16:33:24Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Jan 07, 2025 at 06:57:08AM -0800, Karthik Nayak wrote:\n> shejialuo writes:\n> \n> > In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\n> > and \"next_record\" which would check the correctness of the content of\n> > the \"packed-ref\" file. When anything is bad, the program will die.\n> \n> So you're saying, `create_snapshot()` and `next_record()` exit the\n> program on any error. Okay that seems to be valid.\n> \n> > It may seem that we have nothing relevant to above feature, because we\n> > are going to read and parse the raw \"packed-ref\" file without creating\n> > the snapshot and using the ref iterator to check the consistency.\n> >\n> > However, when using \"get_worktrees\" in \"builtin/refs\", we will parse the\n> > head information. If the referent of the \"HEAD\" is inside the\n> > \"packed-ref\", we will call \"create_snapshot\" and \"next_record\" functions\n> > to parse the \"packed-ref\" to get the head information. And if there are\n> > something wrong, the program will die.\n> >\n> > Although this behavior has no harm for the program, it will\n> > short-circuit the program. When the users execute \"git refs verify\" or\n> > \"git fsck\", we don't want to simply die the program but rather show the\n> > warnings or errors as many as possible to info the users. So, we should\n> > avoiding reading the head info.\n> >\n> \n> This is a bit tricky here. If the information for the `HEAD` ref is\n> incorrect in the packed-refs, git would exit early. Which is what we're\n> trying to avoid in this patch, by using the `get_worktrees_internal()`\n> function.\n> \n\nI think my commit message may confuse you here. The information of the\n\"HEAD\" ref will never be stored in the \"packed-refs\", but if we need to\nread the head information, we need to parse the \"packed-refs\" via\n\"create_snapshot\" method. Even though the corresponding referent is\ncorrect (and even if it is not correct, it won't let the program die),\n\"create_snapshot\" will call \"verify_buffer_safe\" to check whether there\nis a newline in the last line of the file. If not, it will die.\n\nHowever, this is a bad thing. For example, if the HEAD points to\n\"refs/heads/main\", now we need to use the code path from packed-backend,\nwe have to call \"create_snapshot\", the program will die. And we cannot\ntell the user the other faults.\n\n```packed-refs\n refs/heads/main\\n\n \\n\n refs/heads/a\n```\n\nSo, the motivation here is that we should not read HEAD at all when we\nare doing consistency checking to make the code totally independent of\nthe \"create_snapshot\" and \"next_record\".\n\n> However, I would question if this is the right approach. Shouldn't\n> `get_worktree()` failing indicate that the repository is invalid? In\n> that case does it really make sense to allow the user to even run `git\n> refs verify`? Isn't the prerequisite for running the `git-refs(1)`\n> command a valid repository?\n> \n\nAs I have talked about above, even though the referent of \"HEAD\" is\ngood, \"get_worktree()\" will still fail because of some fatal errors in\n\"packed-refs\" file. I don't think that the repository is invalid in this\nsituation.\n\nPut it further more, in what situations, the users want to execute \"git\nrefs verify\" or \"git-fsck\". From my intuitive thinking, the users will\nexecute these check commands when something fails. They want to know\nwhy. So we should execute these commands when the repository is invalid\nto tell the user what may be wrong. And this is the value of these two\ncommands.\n\nThanks,\nJialuo\n"},{"id":"510095","messageId":"CAOLa=ZQ-cRJeWjP-_6N2v4GS5P7oYVUyb9_tbY26W7MAJfJ6ZQ@mail.gmail.com","threadId":"62743","inReplyTo":"Z3qN6C2IpQTdVn_S@ArchLinux","subject":"Re: [PATCH 03/10] packed-backend: check whether the \"packed-refs\" is regular","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-01-07T16:33:56Z","receivedAt":"2025-01-07T16:33:57Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\n> consistency and correctness of \"packed-refs\" file, they never check the\n> filetype of the \"packed-refs\". The user should always use \"git\n> packed-refs\" command to create the raw regular \"packed-refs\" file, so we\n> need to explicitly check this in \"git refs verify\".\n>\n> Use \"lstat\" to check the file mode. If we cannot check the file status,\n> this is OK because there is a chance that there is no \"packed-refs\" in\n> the repo.\n>\n> Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n> the user if \"packed-refs\" is not a regular file.\n>\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> refs/packed-backend.c | 33 +++++++++++++++++++++++++++++----\n> t/t0602-reffiles-fsck.sh | 20 ++++++++++++++++++++\n> 2 files changed, 49 insertions(+), 4 deletions(-)\n>\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 3406f1e71d..d9eb2f8b71 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -4,6 +4,7 @@\n> #include \"../config.h\"\n> #include \"../dir.h\"\n> #include \"../gettext.h\"\n> +#include \"../fsck.h\"\n> #include \"../hash.h\"\n> #include \"../hex.h\"\n> #include \"../refs.h\"\n> @@ -1747,15 +1748,39 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> \treturn empty_ref_iterator_begin();\n> }\n>\n> -static int packed_fsck(struct ref_store *ref_store UNUSED,\n> -\t\t struct fsck_options *o UNUSED,\n> +static int packed_fsck(struct ref_store *ref_store,\n> +\t\t struct fsck_options *o,\n> \t\t struct worktree *wt)\n> {\n> +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> +\tstruct stat st;\n> +\tint ret = 0;\n>\n> \tif (!is_main_worktree(wt))\n> -\t\treturn 0;\n> +\t\tgoto cleanup;\n>\n> -\treturn 0;\n> +\t/*\n> +\t * If the packed-refs file doesn't exist, there's nothing to\n> +\t * check.\n> +\t */\n> +\tif (lstat(refs->path, &st) < 0)\n> +\t\tgoto cleanup;\n\nSince `lstat` return '-1' for all errors, we should check that the\n`errno == ENOENT`.\n\n> +\tif (o->verbose)\n> +\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n> +\n> +\tif (!S_ISREG(st.st_mode)) {\n> +\t\tstruct fsck_ref_report report = { 0 };\n> +\t\treport.path = \"packed-refs\";\n> +\n> +\t\tret = fsck_report_ref(o, &report, FSCK_MSG_BAD_REF_FILETYPE,\n> +\t\t\t\t \"not a regular file\");\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +cleanup:\n> +\treturn ret;\n> }\n>\n> struct ref_storage_be refs_be_packed = {\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 75f234a94a..307f94a3ca 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -626,4 +626,24 @@ test_expect_success 'ref content checks should work with worktrees' '\n> \ttest_cmp expect err\n> '\n>\n> +test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\tcd repo &&\n\nThis should be in a subshell, so that at the end we can actually remove\nthe repo. This seems to be applicable to most of the other tests in this\nfile too. Perhaps, we should clean it up as a precursor commit to this\nseries?\n\n> +\ttest_commit default &&\n> +\tgit branch branch-1 &&\n> +\tgit branch branch-2 &&\n> +\tgit branch branch-3 &&\n> +\tgit pack-refs --all &&\n> +\n> +\tmv .git/packed-refs .git/packed-refs-back &&\n> +\tln -sf packed-refs-bak .git/packed-refs &&\n\nThis should be `ln -sf .git/packed-refs-back .git/packed-refs` no?\n\n> +\ttest_must_fail git refs verify 2>err &&\n> +\tcat >expect <<-EOF &&\n> +\terror: packed-refs: badRefFiletype: not a regular file\n> +\tEOF\n> +\trm .git/packed-refs &&\n> +\ttest_cmp expect err\n> +'\n> +\n> test_done\n> --\n> 2.47.1\n"},{"id":"510138","messageId":"Z33MtrbgN1kxsyGM@ArchLinux","threadId":"62743","inReplyTo":"Z3qN8U2VbZBnUSWj@ArchLinux","subject":"Re: [PATCH 04/10] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-08T00:54:14Z","receivedAt":"2025-01-08T00:53:12Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Sun, Jan 05, 2025 at 09:49:37PM +0800, shejialuo wrote:\n\n[snip]\n\n> However, when adding the new test for a bad header, the program will\n> still die in the \"create_snapshot\" method. This is because we have\n> checked the files-backend firstly and we use \"parse_object\" to check\n> whether the object exists and whether the type is correct. This function\n> will eventually call \"create_snapshot\" and \"next_record\" method, if\n> there is something wrong with packed-backend, the program just dies.\n> \n> It's bad to just die the program because we want to report the problems\n> as many as possible. We should avoid checking object and its type when\n> packed-backend is broken. So, we should first check the consistency of\n> the packed-backend then for files-backend.\n> \n> Add a new flag \"safe_object_check\" in \"fsck_options\", when there is\n> anything wrong with the parsing process, set this flag to 0 to avoid\n> checking objects in the later checks.\n> \n\nHere, I made a mistake. The most simplest way is to call the\n\"disable_replace_refs\" function in \"builtin/refs\". So, there is a lot of\ncode and commit message needs to be fixed in the version 2. I have just\nrealized about this.\n\nSo, tell the reviewers in advance about this.\n\nThanks,\nJialuo\n"},{"id":"510150","messageId":"CAOLa=ZS461s=GxjQ_ifO_FCbDen9WeP6Gogz7nx=zx-jMkiipg@mail.gmail.com","threadId":"62743","inReplyTo":"Z31Xj6sZk1th2mRQ@ArchLinux","subject":"Re: [PATCH 02/10] builtin/refs.h: get worktrees without reading head info","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-01-08T08:40:01Z","receivedAt":"2025-01-08T08:40:03Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> On Tue, Jan 07, 2025 at 06:57:08AM -0800, Karthik Nayak wrote:\n>> shejialuo writes:\n>>\n>> > In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\n>> > and \"next_record\" which would check the correctness of the content of\n>> > the \"packed-ref\" file. When anything is bad, the program will die.\n>>\n>> So you're saying, `create_snapshot()` and `next_record()` exit the\n>> program on any error. Okay that seems to be valid.\n>>\n>> > It may seem that we have nothing relevant to above feature, because we\n>> > are going to read and parse the raw \"packed-ref\" file without creating\n>> > the snapshot and using the ref iterator to check the consistency.\n>> >\n>> > However, when using \"get_worktrees\" in \"builtin/refs\", we will parse the\n>> > head information. If the referent of the \"HEAD\" is inside the\n>> > \"packed-ref\", we will call \"create_snapshot\" and \"next_record\" functions\n>> > to parse the \"packed-ref\" to get the head information. And if there are\n>> > something wrong, the program will die.\n>> >\n>> > Although this behavior has no harm for the program, it will\n>> > short-circuit the program. When the users execute \"git refs verify\" or\n>> > \"git fsck\", we don't want to simply die the program but rather show the\n>> > warnings or errors as many as possible to info the users. So, we should\n>> > avoiding reading the head info.\n>> >\n>>\n>> This is a bit tricky here. If the information for the `HEAD` ref is\n>> incorrect in the packed-refs, git would exit early. Which is what we're\n>> trying to avoid in this patch, by using the `get_worktrees_internal()`\n>> function.\n>>\n>\n> I think my commit message may confuse you here. The information of the\n> \"HEAD\" ref will never be stored in the \"packed-refs\", but if we need to\n> read the head information, we need to parse the \"packed-refs\" via\n> \"create_snapshot\" method. Even though the corresponding referent is\n> correct (and even if it is not correct, it won't let the program die),\n> \"create_snapshot\" will call \"verify_buffer_safe\" to check whether there\n> is a newline in the last line of the file. If not, it will die.\n>\n> However, this is a bad thing. For example, if the HEAD points to\n> \"refs/heads/main\", now we need to use the code path from packed-backend,\n> we have to call \"create_snapshot\", the program will die. And we cannot\n> tell the user the other faults.\n>\n> ```packed-refs\n> refs/heads/main\\n\n> \\n\n> refs/heads/a\n> ```\n>\n> So, the motivation here is that we should not read HEAD at all when we\n> are doing consistency checking to make the code totally independent of\n> the \"create_snapshot\" and \"next_record\".\n>\n\nThanks for clarifying. I understand better the point now.\n\n>> However, I would question if this is the right approach. Shouldn't\n>> `get_worktree()` failing indicate that the repository is invalid? In\n>> that case does it really make sense to allow the user to even run `git\n>> refs verify`? Isn't the prerequisite for running the `git-refs(1)`\n>> command a valid repository?\n>>\n>\n> As I have talked about above, even though the referent of \"HEAD\" is\n> good, \"get_worktree()\" will still fail because of some fatal errors in\n> \"packed-refs\" file. I don't think that the repository is invalid in this\n> situation.\n>\n> Put it further more, in what situations, the users want to execute \"git\n> refs verify\" or \"git-fsck\". From my intuitive thinking, the users will\n> execute these check commands when something fails. They want to know\n> why. So we should execute these commands when the repository is invalid\n> to tell the user what may be wrong. And this is the value of these two\n> commands.\n>\n\nI agree with your inference here, we should try and figure out as much\nas we can and report it, so clients can make informed decisions on how\nto fix their refdb/repo. Thanks for explaining.\n\n>\n> Thanks,\n> Jialuo\n"},{"id":"510700","messageId":"Z4kQP5PBlgjXYNhs@pks.im","threadId":"62743","inReplyTo":"Z3qN1T3lJoj82ckl@ArchLinux","subject":"Re: [PATCH 01/10] files-backend: add object check for regular ref","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-16T13:57:25Z","receivedAt":"2025-01-16T13:57:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 05, 2025 at 09:49:09PM +0800, shejialuo wrote:\n> Although we use \"parse_loose_ref_content\" to check whether the object id\n> is correct, we never parse it into the \"struct object\" structure thus we\n> ignore checking whether there is a real object existing in the repo and\n> whether the object type is correct.\n> \n> Use \"parse_object\" to parse the oid for the regular ref content. If the\n> object does not exist, report the error to the user by reusing the fsck\n> message \"BAD_REF_CONTENT\".\n> \n> Then, we need to check the type of the object. Just like \"git-fsck(1)\",\n> we only report \"not a commit\" error when the ref is a branch. Last,\n> update the test to exercise the code.\n\nI wonder whether it wouldn't make more sense to put this into a generic\npart of `git refs verify`. This isn't a check for whether the format of\nthe files backend is correct, but rather a check whether the refdb is\nsane. As such, it also applies do the reftable backend.\n\nSo should we maybe extend `git refs verify` so that it also knows to\nperform generic checks that apply independent of the backend in use?\n\nPatrick\n"},{"id":"510701","messageId":"Z4kQStWArejI2Zk4@pks.im","threadId":"62743","inReplyTo":"Z3qN30z1NCXa3AX-@ArchLinux","subject":"Re: [PATCH 02/10] builtin/refs.h: get worktrees without reading head info","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-16T13:57:30Z","receivedAt":"2025-01-16T13:57:34Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 05, 2025 at 09:49:19PM +0800, shejialuo wrote:\n\nThe commit subject is a bit funny with \"builtin/refs.h:\". You probably\nwanted to say \"builtin/refs:\".\n\nPatrick\n"},{"id":"510702","messageId":"Z4kQTVLfScW0SgyG@pks.im","threadId":"62743","inReplyTo":"Z3qN6C2IpQTdVn_S@ArchLinux","subject":"Re: [PATCH 03/10] packed-backend: check whether the \"packed-refs\" is regular","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-16T13:57:33Z","receivedAt":"2025-01-16T13:57:37Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 05, 2025 at 09:49:28PM +0800, shejialuo wrote:\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 3406f1e71d..d9eb2f8b71 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -4,6 +4,7 @@\n> #include \"../config.h\"\n> #include \"../dir.h\"\n> #include \"../gettext.h\"\n> +#include \"../fsck.h\"\n\nLet's keep the alphabetic ordering here.\n\nOther than that I have nothing to add on top of what Karthik mentioned\nalready.\n\nPatrick\n"},{"id":"510703","messageId":"Z4kQUb7og2Ce1iCo@pks.im","threadId":"62743","inReplyTo":"Z3qN8U2VbZBnUSWj@ArchLinux","subject":"Re: [PATCH 04/10] packed-backend: add \"packed-refs\" header consistency check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-16T13:57:37Z","receivedAt":"2025-01-16T13:57:41Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 05, 2025 at 09:49:37PM +0800, shejialuo wrote:\n> Add a new flag \"safe_object_check\" in \"fsck_options\", when there is\n> anything wrong with the parsing process, set this flag to 0 to avoid\n> checking objects in the later checks.\n\nOkay, I understand the motivation: a corrupted refdb may be completely\nbogus, so checking its objects may not be sensible.\n\nFor one of the preceding commits I made the suggestion to split out the\nobject checks into a generic part instead, as they aren't specific to\nthe backend. With such a scheme we could adapt the logic to first do the\nbackend-specific checks for the format, and only in case the backend\nlooks sane to us we'd execute those generic checks for that specific\nbackend. That'd allow us to get rid of the \"safe object check\" flag.\n\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index d9eb2f8b71..3b11abe5f8 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1748,12 +1748,100 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> \treturn empty_ref_iterator_begin();\n> }\n> \n> +static int packed_fsck_ref_next_line(struct fsck_options *o,\n> +\t\t\t\t int line_number, const char *start,\n> +\t\t\t\t const char *eof, const char **eol)\n> +{\n> +\tint ret = 0;\n> +\n> +\t*eol = memchr(start, '\\n', eof - start);\n> +\tif (!*eol) {\n> +\t\tstruct strbuf packed_entry = STRBUF_INIT;\n> +\t\tstruct fsck_ref_report report = { 0 };\n> +\n> +\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n> +\t\treport.path = packed_entry.buf;\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n> +\t\t\t\t \"'%.*s' is not terminated with a newline\",\n> +\t\t\t\t (int)(eof - start), start);\n> +\n> +\t\t/*\n> +\t\t * There is no newline but we still want to parse it to the end of\n> +\t\t * the buffer.\n> +\t\t */\n> +\t\t*eol = eof;\n\nI don't quite understand. We've figured out that there isn't a newline,\nso wouldn't that mean that we _are_ at the end of the buffer already?\n\n> +\t\tstrbuf_release(&packed_entry);\n> +\t}\n> +\n> +\treturn ret;\n> +}\n> +\n> +static int packed_fsck_ref_header(struct fsck_options *o, const char *start, const char *eol)\n> +{\n> +\tconst char *err_fmt = NULL;\n> +\tint fsck_msg_id = -1;\n> +\n> +\tif (!starts_with(start, \"# pack-refs with:\")) {\n> +\t\terr_fmt = \"'%.*s' does not start with '# pack-refs with:'\";\n> +\t\tfsck_msg_id = FSCK_MSG_BAD_PACKED_REF_HEADER;\n> +\t} else if (strncmp(start, PACKED_REFS_HEADER, strlen(PACKED_REFS_HEADER))) {\n> +\t\terr_fmt = \"'%.*s' is not the official packed-refs header\";\n\nI wouldn't say \"official\", because it could totally be that whatever is\nofficial changes in the future, e.g. when a new format is introduced.\nUnlikely to happen, but saying \"unknown packed-refs header\" might be a\nbit more future proof.\n\n> +\t\tfsck_msg_id = FSCK_MSG_UNKNOWN_PACKED_REF_HEADER;\n> +\t}\n> +\n> +\tif (err_fmt && fsck_msg_id >= 0) {\n> +\t\tstruct fsck_ref_report report = { 0 };\n> +\t\treport.path = \"packed-refs.header\";\n> +\n> +\t\treturn fsck_report_ref(o, &report, fsck_msg_id, err_fmt,\n> +\t\t\t\t (int)(eol - start), start);\n> +\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +static int packed_fsck_ref_content(struct fsck_options *o,\n> +\t\t\t\t const char *start, const char *eof)\n> +{\n> +\tint line_number = 1;\n> +\tconst char *eol;\n> +\tint ret = 0;\n> +\n> +\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n> +\tif (*start == '#') {\n> +\t\tret |= packed_fsck_ref_header(o, start, eol);\n> +\n> +\t\tstart = eol + 1;\n> +\t\tline_number++;\n\nThe header can only appear at the beginning of the file, can't it? But\nwe accept it in every line here. We should likely verify that it's\nactually a header and not a line at some random place.\n\n> +\t} else {\n> +\t\tstruct fsck_ref_report report = { 0 };\n> +\t\treport.path = \"packed-refs\";\n> +\n> +\t\tret |= fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_PACKED_REF_MISSING_HEADER,\n> +\t\t\t\t \"missing header line\");\n> +\t}\n> +\n> +\t/*\n> +\t * If there is anything wrong during the parsing of the \"packed-refs\"\n> +\t * file, we should not check the object of the refs.\n> +\t */\n> +\tif (ret)\n> +\t\to->safe_object_check = 0;\n> +\n> +\n> +\treturn ret;\n> +}\n> +\n> static int packed_fsck(struct ref_store *ref_store,\n> \t\t struct fsck_options *o,\n> \t\t struct worktree *wt)\n> {\n> \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> +\tstruct strbuf packed_ref_content = STRBUF_INIT;\n> \tstruct stat st;\n> \tint ret = 0;\n> \n> @@ -1779,7 +1867,24 @@ static int packed_fsck(struct ref_store *ref_store,\n> \t\tgoto cleanup;\n> \t}\n> \n> +\tif (strbuf_read_file(&packed_ref_content, refs->path, 0) < 0) {\n> +\t\t/*\n> +\t\t * Although we have checked that the file exists, there is a possibility\n> +\t\t * that it has been removed between the lstat() and the read attempt by\n> +\t\t * another process. In that case, we should not report an error.\n> +\t\t */\n> +\t\tif (errno == ENOENT)\n> +\t\t\tgoto cleanup;\n\nUnlikely, but good to guard us against that condition regardless. It's\nstill not entirely race-free though because the file could meanwhile\nhave changed into a symlink, and we wouldn't notice now. We could fix\nthat by using open(O_NOFOLLOW), fstat the returne file descriptor and\nthen use `strbuf_read()` to slurp in the file.\n\n> +\t\tret = error_errno(\"could not read %s\", refs->path);\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n> +\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n> +\n> cleanup:\n> +\tstrbuf_release(&packed_ref_content);\n> \treturn ret;\n> }\n> \n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 307f94a3ca..6c729e749a 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -646,4 +646,48 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> \ttest_cmp expect err\n> '\n> \n> +test_expect_success 'packed-refs header should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\tcd repo &&\n\nThe same comment applies here as on a preceding test: cd should be\nexecuted in a subshell.\n\nPatrick\n"},{"id":"510704","messageId":"Z4kQVHw7Uio-Pzo5@pks.im","threadId":"62743","inReplyTo":"Z3qN_8-HKdspwcDb@ArchLinux","subject":"Re: [PATCH 05/10] packed-backend: check whether the refname contains NULL binaries","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-16T13:57:40Z","receivedAt":"2025-01-16T13:57:43Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 05, 2025 at 09:49:51PM +0800, shejialuo wrote:\n> We have already implemented the header consistency check for the raw\n> \"packed-refs\" file. Before we implement the consistency check for each\n> ref entry, let's analysis [1] which reports that \"git fsck\" cannot\n> detect some binary zeros.\n> \n> \"packed-backend.c::next_record\" will use \"check_refname_format\" to check\n> the consistency of the refname. If it is not OK, the program will die.\n> So, we already have the code path and we must miss out something.\n> \n> We use the following code to get the refname:\n> \n> strbuf_add(&iter->refname_buf, p, eol - p);\n> iter->base.refname = iter->refname_buf.buf\n> \n> In the above code, `p` is the start pointer of the refname and `eol` is\n> the next newline pointer. We calculate the length of the refname by\n> subtracting the two pointers. Then we add the memory range between `p`\n> and `eol` to get the refname.\n> \n> However, if there are some NULL binaries in the memory range between `p`\n\nYou probably mean NUL characters, not NULL binaries?\n\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 3b11abe5f8..f6142a4402 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -493,6 +493,23 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n> \t\t\t\t last_line, eof - last_line);\n> }\n> \n> +/*\n> + * When parsing the \"packed-refs\" file, we will parse it line by line.\n> + * Because we know the start pointer of the refname and the next\n> + * newline pointer, we could calculate the length of the refname by\n> + * subtracting the two pointers. However, there is a corner case where\n> + * the refname contains corrupted embedded NULL binaries. And\n> + * `check_refname_format()` will not catch this when the truncated\n> + * refname is still a valid refname. To prevent this, we need to check\n> + * whether the refname contains the NULL binaries.\n> + */\n> +static int refname_contains_null(struct strbuf refname)\n> +{\n> +\tif (refname.len != strlen(refname.buf))\n> +\t\treturn 1;\n> +\treturn 0;\n> +}\n> +\n> #define SMALL_FILE_SIZE (32*1024)\n> \n> /*\n> @@ -894,6 +911,9 @@ static int next_record(struct packed_ref_iterator *iter)\n> \tstrbuf_add(&iter->refname_buf, p, eol - p);\n> \titer->base.refname = iter->refname_buf.buf;\n> \n> +\tif (refname_contains_null(iter->refname_buf))\n\nWe can replace this with `memchr(iter->refname_buf.buf, '\\0',\niter->refname_buf.len)`, which should be more efficient than using\nstrlen(3p).\n\n> +\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n> +\n\nI was a bit surprised to find that we modify the way that we read refs\nfrom the packed-refs file instead of adapting the fsck code. But I think\nthis check is sensible.\n\nPatrick\n"},{"id":"510705","messageId":"Z4kQV4Nve632rJ3s@pks.im","threadId":"62743","inReplyTo":"Z3qOB2_zrDHOh-Gx@ArchLinux","subject":"Re: [PATCH 06/10] packed-backend: add \"packed-refs\" entry consistency check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-16T13:57:43Z","receivedAt":"2025-01-16T13:57:46Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 05, 2025 at 09:49:59PM +0800, shejialuo wrote:\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index f6142a4402..6e521a9f87 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1822,7 +1822,96 @@ static int packed_fsck_ref_header(struct fsck_options *o, const char *start, con\n> \treturn 0;\n> }\n> \n> +static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n> +\t\t\t\t struct ref_store *ref_store, int line_number,\n> +\t\t\t\t const char *start, const char *eol)\n> +{\n> +\tstruct strbuf peeled_entry = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = { 0 };\n> +\tstruct object_id peeled;\n> +\tconst char *p;\n> +\tint ret = 0;\n> +\n> +\tstrbuf_addf(&peeled_entry, \"packed-refs line %d\", line_number);\n> +\treport.path = peeled_entry.buf;\n> +\n> +\tstart++;\n> +\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n> +\t\tret |= fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n> +\t\t\t\t \"'%.*s' has invalid peeled oid\",\n> +\t\t\t\t (int)(eol - start), start);\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +\tif (p != eol) {\n> +\t\tret |= fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n> +\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n> +\t\t\t\t (int)(eol - p), p);\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +cleanup:\n> +\tstrbuf_release(&peeled_entry);\n> +\treturn ret;\n> +}\n> +\n> +static int packed_fsck_ref_main_line(struct fsck_options *o,\n> +\t\t\t\t struct ref_store *ref_store, int line_number,\n> +\t\t\t\t const char *start, const char *eol)\n> +{\n> +\tstruct strbuf packed_entry = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = { 0 };\n> +\tstruct strbuf refname = STRBUF_INIT;\n\nIt feels quite inefficient to create a separate buffer for every\ninvocation of this function, as there can be many million refs in a\nrepo. Might be something to avoid by passing in a scratch buffer.\n\nPatrick\n"},{"id":"510706","messageId":"Z4kQWgYXePBDthsl@pks.im","threadId":"62743","inReplyTo":"Z3qOEvyeoc7vOW73@ArchLinux","subject":"Re: [PATCH 07/10] packed-backend: create \"fsck_packed_ref_entry\" to store parsing info","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-16T13:57:46Z","receivedAt":"2025-01-16T13:57:49Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 05, 2025 at 09:50:10PM +0800, shejialuo wrote:\n> We have already check whether the oid hash is correct by using\n> `parse_oid_hex_algop`. However, we doesn't check whether the object\n> exists. It may seem that we could do this when we are parsing the raw\n> \"packed-refs\" file. But this is impossible. Let's analysis why.\n> \n> We will use \"parse_object\" function to get the \"struct object\". However,\n> this function will eventually call the \"create_snapshot\" and\n> \"next_record\" function in \"packed-backend.c\". If there is anything\n> wrong, it will die the program. And we don't want to die the program\n> during the check.\n> \n> So, we should store the information in the parsing process. And if there\n> is nothing wrong in the parsing process, we could continue to check\n> things. So, create \"fsck_packed_ref_entry\" to do this.\n\nThis step can be avoided if we made the check generic.\n\nPatrick\n"},{"id":"510707","messageId":"Z4kQXeLBfpNP7HX_@pks.im","threadId":"62743","inReplyTo":"Z3qOGzfncHlnZOGY@ArchLinux","subject":"Re: [PATCH 08/10] packed-backend: add check for object consistency","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-16T13:57:49Z","receivedAt":"2025-01-16T13:57:53Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 05, 2025 at 09:50:19PM +0800, shejialuo wrote:\n> If there is nothing wrong when parsing the raw file \"packed-refs\", we\n> could then iterate the \"entries\" to check the object consistency. There\n> are two kinds of ref entry: one is the normal and another is peeled. For\n> both situations, we need to use \"parse_object\" function to parse the\n> object id to get the object. If the object does not exist, we will\n> report an error to the user.\n> \n> Create a new function \"packed_fsck_ref_oid\" to do above then update the\n> unit test to exercise the code.\n\nThis one, as well.\n\nPatrick\n"},{"id":"510708","messageId":"Z4kQYfT73geFdMNJ@pks.im","threadId":"62743","inReplyTo":"Z3qOJ_ixuoE4yTut@ArchLinux","subject":"Re: [PATCH 09/10] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-16T13:57:53Z","receivedAt":"2025-01-16T13:57:56Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 05, 2025 at 09:50:31PM +0800, shejialuo wrote:\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index d83ce2838f..df65fec5a5 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1980,6 +1989,50 @@ static int packed_fsck_ref_oid(struct fsck_options *o, struct ref_store *ref_sto\n> \treturn ret;\n> }\n> \n> +static int packed_fsck_ref_sorted(struct fsck_options *o,\n> +\t\t\t\t struct ref_store *ref_store,\n> +\t\t\t\t struct fsck_packed_ref_entry **entries,\n> +\t\t\t\t int nr)\n> +{\n> +\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n> +\tstruct strbuf packed_entry = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = { 0 };\n> +\tstruct strbuf refname1 = STRBUF_INIT;\n> +\tstruct strbuf refname2 = STRBUF_INIT;\n> +\tint ret = 0;\n> +\n> +\tfor (int i = 1; i < nr; i++) {\n> +\t\tconst char *r1 = entries[i - 1]->record.start + hexsz + 1;\n> +\t\tconst char *r2 = entries[i]->record.start + hexsz + 1;\n> +\n> +\t\tif (cmp_packed_refname(r1, r2) >= 0) {\n\nMakes sense. It has been a source of bugs a couple years ago, and it can\nsilently make you receive wrong results, so this is quite a sensible\ncheck to have.\n\nPatrick\n"},{"id":"510808","messageId":"Z4pdwiBvDlyC9TZW@ArchLinux","threadId":"62743","inReplyTo":"Z4kQP5PBlgjXYNhs@pks.im","subject":"Re: [PATCH 01/10] files-backend: add object check for regular ref","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-17T13:40:18Z","receivedAt":"2025-01-17T13:39:05Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 16, 2025 at 02:57:25PM +0100, Patrick Steinhardt wrote:\n> On Sun, Jan 05, 2025 at 09:49:09PM +0800, shejialuo wrote:\n> > Although we use \"parse_loose_ref_content\" to check whether the object id\n> > is correct, we never parse it into the \"struct object\" structure thus we\n> > ignore checking whether there is a real object existing in the repo and\n> > whether the object type is correct.\n> > \n> > Use \"parse_object\" to parse the oid for the regular ref content. If the\n> > object does not exist, report the error to the user by reusing the fsck\n> > message \"BAD_REF_CONTENT\".\n> > \n> > Then, we need to check the type of the object. Just like \"git-fsck(1)\",\n> > we only report \"not a commit\" error when the ref is a branch. Last,\n> > update the test to exercise the code.\n> \n> I wonder whether it wouldn't make more sense to put this into a generic\n> part of `git refs verify`. This isn't a check for whether the format of\n> the files backend is correct, but rather a check whether the refdb is\n> sane. As such, it also applies do the reftable backend.\n> \n> So should we maybe extend `git refs verify` so that it also knows to\n> perform generic checks that apply independent of the backend in use?\n> \n\nI somehow understand your meaning here and I think what your meaning\nhere is that we could use internal ref method to parse the oid after we\ncheck the format of the ref files. Thus, we could totally make these two\ndifferent kinds of checks separately.\n\nHowever, if we have already parsed the raw ref files, we could reuse the\nparsed hex and then use \"parse_object\" to get the object id to check.\nThis is the main reason why I add this check now.\n\nAnd I agree with your thinking here. Actually, we may put this into\nobject check part. Because in \"git-fsck(1)\", we parse the refdb to know\nwhether an object is dangling or not.\n\nI will postpone these checks in the later patches. Really thanks here\nfor this wonderful suggestion.\n\nThanks,\nJialuo\n"},{"id":"510809","messageId":"Z4pijwANZWAP2XKH@ArchLinux","threadId":"62743","inReplyTo":"CAOLa=ZQ-cRJeWjP-_6N2v4GS5P7oYVUyb9_tbY26W7MAJfJ6ZQ@mail.gmail.com","subject":"Re: [PATCH 03/10] packed-backend: check whether the \"packed-refs\" is regular","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-17T14:00:47Z","receivedAt":"2025-01-17T13:59:32Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Jan 07, 2025 at 08:33:56AM -0800, Karthik Nayak wrote:\n> shejialuo writes:\n\n[snip]\n\n> > -static int packed_fsck(struct ref_store *ref_store UNUSED,\n> > -\t\t struct fsck_options *o UNUSED,\n> > +static int packed_fsck(struct ref_store *ref_store,\n> > +\t\t struct fsck_options *o,\n> > \t\t struct worktree *wt)\n> > {\n> > +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> > +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> > +\tstruct stat st;\n> > +\tint ret = 0;\n> >\n> > \tif (!is_main_worktree(wt))\n> > -\t\treturn 0;\n> > +\t\tgoto cleanup;\n> >\n> > -\treturn 0;\n> > +\t/*\n> > +\t * If the packed-refs file doesn't exist, there's nothing to\n> > +\t * check.\n> > +\t */\n> > +\tif (lstat(refs->path, &st) < 0)\n> > +\t\tgoto cleanup;\n> \n> Since `lstat` return '-1' for all errors, we should check that the\n> `errno == ENOENT`.\n> \n\nI agree here, if the reason is not \"errno == ENOENT\", we should report\nan error to the user.\n\n[snip]\n\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -626,4 +626,24 @@ test_expect_success 'ref content checks should work with worktrees' '\n> > \ttest_cmp expect err\n> > '\n> >\n> > +test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\tcd repo &&\n> \n> This should be in a subshell, so that at the end we can actually remove\n> the repo. This seems to be applicable to most of the other tests in this\n> file too. Perhaps, we should clean it up as a precursor commit to this\n> series?\n\nI have searched the usage of \"test_when_finished\", and I don't know why\nwe need to use subshell. Could you please explain this further here.\n\n> \n> > +\ttest_commit default &&\n> > +\tgit branch branch-1 &&\n> > +\tgit branch branch-2 &&\n> > +\tgit branch branch-3 &&\n> > +\tgit pack-refs --all &&\n> > +\n> > +\tmv .git/packed-refs .git/packed-refs-back &&\n> > +\tln -sf packed-refs-bak .git/packed-refs &&\n> \n> This should be `ln -sf .git/packed-refs-back .git/packed-refs` no?\n> \n\nNo. This should not be `ln -sf .git/packed-refs-back .git/packed-refs`.\nThis is because it is a relative symlink. And the file\n\".git/packed-refs-back\" and \".git/packed-refs\" are in the same\ndirectory. So, from the perspective of \".git/packed-refs\", it should be\nthe \"packed-refs-back\".\n\nThanks,\nJialuo\n"},{"id":"510810","messageId":"Z4pnyhF2V2ykuHlg@ArchLinux","threadId":"62743","inReplyTo":"Z4kQUb7og2Ce1iCo@pks.im","subject":"Re: [PATCH 04/10] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-17T14:23:06Z","receivedAt":"2025-01-17T14:21:51Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 16, 2025 at 02:57:37PM +0100, Patrick Steinhardt wrote:\n> On Sun, Jan 05, 2025 at 09:49:37PM +0800, shejialuo wrote:\n> > Add a new flag \"safe_object_check\" in \"fsck_options\", when there is\n> > anything wrong with the parsing process, set this flag to 0 to avoid\n> > checking objects in the later checks.\n> \n> Okay, I understand the motivation: a corrupted refdb may be completely\n> bogus, so checking its objects may not be sensible.\n> \n> For one of the preceding commits I made the suggestion to split out the\n> object checks into a generic part instead, as they aren't specific to\n> the backend. With such a scheme we could adapt the logic to first do the\n> backend-specific checks for the format, and only in case the backend\n> looks sane to us we'd execute those generic checks for that specific\n> backend. That'd allow us to get rid of the \"safe object check\" flag.\n> \n\nYes, I agree with you here. And I won't touch this topic in the next\nversion. Let me make this patch concentrate on the \"packed-ref\" format.\n\n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index d9eb2f8b71..3b11abe5f8 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -1748,12 +1748,100 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> > \treturn empty_ref_iterator_begin();\n> > }\n> > \n> > +static int packed_fsck_ref_next_line(struct fsck_options *o,\n> > +\t\t\t\t int line_number, const char *start,\n> > +\t\t\t\t const char *eof, const char **eol)\n> > +{\n> > +\tint ret = 0;\n> > +\n> > +\t*eol = memchr(start, '\\n', eof - start);\n> > +\tif (!*eol) {\n> > +\t\tstruct strbuf packed_entry = STRBUF_INIT;\n> > +\t\tstruct fsck_ref_report report = { 0 };\n> > +\n> > +\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n> > +\t\treport.path = packed_entry.buf;\n> > +\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n> > +\t\t\t\t \"'%.*s' is not terminated with a newline\",\n> > +\t\t\t\t (int)(eof - start), start);\n> > +\n> > +\t\t/*\n> > +\t\t * There is no newline but we still want to parse it to the end of\n> > +\t\t * the buffer.\n> > +\t\t */\n> > +\t\t*eol = eof;\n> \n> I don't quite understand. We've figured out that there isn't a newline,\n> so wouldn't that mean that we _are_ at the end of the buffer already?\n> \n\nIn the \"packed-refs\" file, the last line should end with a newline. If\nnot, this is a fatal error. The motivation why I do this is that for\neach line, we could pass the \"line_start\" and \"eol\" to check. But if\nthere is no newline, the \"eol\" will be NULL. So, I change it to \"eof\" to\nmake sure that we could follow the same logic when \"eol\" is not NULL.\n\nI guess I should not handle this in this function which may cause\nconfusion here. I will improve this in the next version.\n\n> > +\t\tstrbuf_release(&packed_entry);\n> > +\t}\n> > +\n> > +\treturn ret;\n> > +}\n> > +\n> > +static int packed_fsck_ref_header(struct fsck_options *o, const char *start, const char *eol)\n> > +{\n> > +\tconst char *err_fmt = NULL;\n> > +\tint fsck_msg_id = -1;\n> > +\n> > +\tif (!starts_with(start, \"# pack-refs with:\")) {\n> > +\t\terr_fmt = \"'%.*s' does not start with '# pack-refs with:'\";\n> > +\t\tfsck_msg_id = FSCK_MSG_BAD_PACKED_REF_HEADER;\n> > +\t} else if (strncmp(start, PACKED_REFS_HEADER, strlen(PACKED_REFS_HEADER))) {\n> > +\t\terr_fmt = \"'%.*s' is not the official packed-refs header\";\n> \n> I wouldn't say \"official\", because it could totally be that whatever is\n> official changes in the future, e.g. when a new format is introduced.\n> Unlikely to happen, but saying \"unknown packed-refs header\" might be a\n> bit more future proof.\n> \n\nI will improve this in the next version.\n\n> > +\t\tfsck_msg_id = FSCK_MSG_UNKNOWN_PACKED_REF_HEADER;\n> > +\t}\n> > +\n> > +\tif (err_fmt && fsck_msg_id >= 0) {\n> > +\t\tstruct fsck_ref_report report = { 0 };\n> > +\t\treport.path = \"packed-refs.header\";\n> > +\n> > +\t\treturn fsck_report_ref(o, &report, fsck_msg_id, err_fmt,\n> > +\t\t\t\t (int)(eol - start), start);\n> > +\n> > +\t}\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +static int packed_fsck_ref_content(struct fsck_options *o,\n> > +\t\t\t\t const char *start, const char *eof)\n> > +{\n> > +\tint line_number = 1;\n> > +\tconst char *eol;\n> > +\tint ret = 0;\n> > +\n> > +\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n> > +\tif (*start == '#') {\n> > +\t\tret |= packed_fsck_ref_header(o, start, eol);\n> > +\n> > +\t\tstart = eol + 1;\n> > +\t\tline_number++;\n> \n> The header can only appear at the beginning of the file, can't it? But\n> we accept it in every line here. We should likely verify that it's\n> actually a header and not a line at some random place.\n> \n\nYes. But we don't accept it in every line. Because in here, we are\ngetting the first line \"start\" and \"eol\" by using\n\"packed_fsck_ref_next_line\". Only it starts with \"#\", we will check the\nheader consistency.\n\n> > +\t} else {\n> > +\t\tstruct fsck_ref_report report = { 0 };\n> > +\t\treport.path = \"packed-refs\";\n> > +\n> > +\t\tret |= fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_PACKED_REF_MISSING_HEADER,\n> > +\t\t\t\t \"missing header line\");\n> > +\t}\n> > +\n> > +\t/*\n> > +\t * If there is anything wrong during the parsing of the \"packed-refs\"\n> > +\t * file, we should not check the object of the refs.\n> > +\t */\n> > +\tif (ret)\n> > +\t\to->safe_object_check = 0;\n> > +\n> > +\n> > +\treturn ret;\n> > +}\n> > +\n> > static int packed_fsck(struct ref_store *ref_store,\n> > \t\t struct fsck_options *o,\n> > \t\t struct worktree *wt)\n> > {\n> > \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> > \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> > +\tstruct strbuf packed_ref_content = STRBUF_INIT;\n> > \tstruct stat st;\n> > \tint ret = 0;\n> > \n> > @@ -1779,7 +1867,24 @@ static int packed_fsck(struct ref_store *ref_store,\n> > \t\tgoto cleanup;\n> > \t}\n> > \n> > +\tif (strbuf_read_file(&packed_ref_content, refs->path, 0) < 0) {\n> > +\t\t/*\n> > +\t\t * Although we have checked that the file exists, there is a possibility\n> > +\t\t * that it has been removed between the lstat() and the read attempt by\n> > +\t\t * another process. In that case, we should not report an error.\n> > +\t\t */\n> > +\t\tif (errno == ENOENT)\n> > +\t\t\tgoto cleanup;\n> \n> Unlikely, but good to guard us against that condition regardless. It's\n> still not entirely race-free though because the file could meanwhile\n> have changed into a symlink, and we wouldn't notice now. We could fix\n> that by using open(O_NOFOLLOW), fstat the returne file descriptor and\n> then use `strbuf_read()` to slurp in the file.\n> \n\nWould this be too complicated for us to avoid race condition and we will\nintroduce a lot of code to handle above logic. Because there is a\npossibility that when finishing reading the file content to the memory,\nthe file could be changed into a symlink and we cannot notice. So, I\nwanna say we can't avoid race condition totally. It would be good if we\navoid race, but what I am concern about here is that we would make the\nlogic too complicated. So, could we make it unchanged?\n\n"},{"id":"510811","messageId":"Z4pqVRsCg3KfjJf-@ArchLinux","threadId":"62743","inReplyTo":"Z4kQVHw7Uio-Pzo5@pks.im","subject":"Re: [PATCH 05/10] packed-backend: check whether the refname contains NULL binaries","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-17T14:33:57Z","receivedAt":"2025-01-17T14:32:42Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 16, 2025 at 02:57:40PM +0100, Patrick Steinhardt wrote:\n> On Sun, Jan 05, 2025 at 09:49:51PM +0800, shejialuo wrote:\n> > We have already implemented the header consistency check for the raw\n> > \"packed-refs\" file. Before we implement the consistency check for each\n> > ref entry, let's analysis [1] which reports that \"git fsck\" cannot\n> > detect some binary zeros.\n> > \n> > \"packed-backend.c::next_record\" will use \"check_refname_format\" to check\n> > the consistency of the refname. If it is not OK, the program will die.\n> > So, we already have the code path and we must miss out something.\n> > \n> > We use the following code to get the refname:\n> > \n> > strbuf_add(&iter->refname_buf, p, eol - p);\n> > iter->base.refname = iter->refname_buf.buf\n> > \n> > In the above code, `p` is the start pointer of the refname and `eol` is\n> > the next newline pointer. We calculate the length of the refname by\n> > subtracting the two pointers. Then we add the memory range between `p`\n> > and `eol` to get the refname.\n> > \n> > However, if there are some NULL binaries in the memory range between `p`\n> \n> You probably mean NUL characters, not NULL binaries?\n> \n\nYes, I will improve this in the next version.\n\n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index 3b11abe5f8..f6142a4402 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -493,6 +493,23 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n> > \t\t\t\t last_line, eof - last_line);\n> > }\n> > \n> > +/*\n> > + * When parsing the \"packed-refs\" file, we will parse it line by line.\n> > + * Because we know the start pointer of the refname and the next\n> > + * newline pointer, we could calculate the length of the refname by\n> > + * subtracting the two pointers. However, there is a corner case where\n> > + * the refname contains corrupted embedded NULL binaries. And\n> > + * `check_refname_format()` will not catch this when the truncated\n> > + * refname is still a valid refname. To prevent this, we need to check\n> > + * whether the refname contains the NULL binaries.\n> > + */\n> > +static int refname_contains_null(struct strbuf refname)\n> > +{\n> > +\tif (refname.len != strlen(refname.buf))\n> > +\t\treturn 1;\n> > +\treturn 0;\n> > +}\n> > +\n> > #define SMALL_FILE_SIZE (32*1024)\n> > \n> > /*\n> > @@ -894,6 +911,9 @@ static int next_record(struct packed_ref_iterator *iter)\n> > \tstrbuf_add(&iter->refname_buf, p, eol - p);\n> > \titer->base.refname = iter->refname_buf.buf;\n> > \n> > +\tif (refname_contains_null(iter->refname_buf))\n> \n> We can replace this with `memchr(iter->refname_buf.buf, '\\0',\n> iter->refname_buf.len)`, which should be more efficient than using\n> strlen(3p).\n\nThanks for the suggestion. Will improve this in the next version.\n\n> \n> > +\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n> > +\n> \n> I was a bit surprised to find that we modify the way that we read refs\n> from the packed-refs file instead of adapting the fsck code. But I think\n> this check is sensible.\n\nActually, I am also surprised here. And this thing is extremely\ninteresting. When I implement all the fsck code, I find I still cannot\ndetect the error reported in [1] which is the motivation why we want to\nadd checks for ref explicitly.\n\nAnd I dive into the code to fix this problem. The reason why I put here\nis that we are going to implement the checks like what \"next_record\"\ndoes.\n\n[1] https://lore.kernel.org/git/6cfee0e4-3285-4f18-91ff-d097da9de737@rd10.de/\n\nThanks,\nJialuo\n"},{"id":"510812","messageId":"Z4pqv2JMNgIgFgG0@ArchLinux","threadId":"62743","inReplyTo":"Z4kQV4Nve632rJ3s@pks.im","subject":"Re: [PATCH 06/10] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-17T14:35:43Z","receivedAt":"2025-01-17T14:34:28Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 16, 2025 at 02:57:43PM +0100, Patrick Steinhardt wrote:\n> On Sun, Jan 05, 2025 at 09:49:59PM +0800, shejialuo wrote:\n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index f6142a4402..6e521a9f87 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -1822,7 +1822,96 @@ static int packed_fsck_ref_header(struct fsck_options *o, const char *start, con\n> > \treturn 0;\n> > }\n> > \n> > +static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n> > +\t\t\t\t struct ref_store *ref_store, int line_number,\n> > +\t\t\t\t const char *start, const char *eol)\n> > +{\n> > +\tstruct strbuf peeled_entry = STRBUF_INIT;\n> > +\tstruct fsck_ref_report report = { 0 };\n> > +\tstruct object_id peeled;\n> > +\tconst char *p;\n> > +\tint ret = 0;\n> > +\n> > +\tstrbuf_addf(&peeled_entry, \"packed-refs line %d\", line_number);\n> > +\treport.path = peeled_entry.buf;\n> > +\n> > +\tstart++;\n> > +\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n> > +\t\tret |= fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n> > +\t\t\t\t \"'%.*s' has invalid peeled oid\",\n> > +\t\t\t\t (int)(eol - start), start);\n> > +\t\tgoto cleanup;\n> > +\t}\n> > +\n> > +\tif (p != eol) {\n> > +\t\tret |= fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n> > +\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n> > +\t\t\t\t (int)(eol - p), p);\n> > +\t\tgoto cleanup;\n> > +\t}\n> > +\n> > +cleanup:\n> > +\tstrbuf_release(&peeled_entry);\n> > +\treturn ret;\n> > +}\n> > +\n> > +static int packed_fsck_ref_main_line(struct fsck_options *o,\n> > +\t\t\t\t struct ref_store *ref_store, int line_number,\n> > +\t\t\t\t const char *start, const char *eol)\n> > +{\n> > +\tstruct strbuf packed_entry = STRBUF_INIT;\n> > +\tstruct fsck_ref_report report = { 0 };\n> > +\tstruct strbuf refname = STRBUF_INIT;\n> \n> It feels quite inefficient to create a separate buffer for every\n> invocation of this function, as there can be many million refs in a\n> repo. Might be something to avoid by passing in a scratch buffer.\n> \n\nI see. I will improve this in the next version.\n\nThanks,\nJialuo\n"},{"id":"510839","messageId":"CAPig+cRsAPp1APNJ7W337UNtunETr+Lnn-RcGrAXEFUhN1APyA@mail.gmail.com","threadId":"62743","inReplyTo":"Z4pijwANZWAP2XKH@ArchLinux","subject":"Re: [PATCH 03/10] packed-backend: check whether the \"packed-refs\" is regular","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2025-01-17T22:01:21Z","receivedAt":"2025-01-17T22:01:33Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Jan 17, 2025 at 8:59 AM shejialuo wrote:\n> On Tue, Jan 07, 2025 at 08:33:56AM -0800, Karthik Nayak wrote:\n> > shejialuo writes:\n> > > +test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> > > + test_when_finished \"rm -rf repo\" &&\n> > > + git init repo &&\n> > > + cd repo &&\n> >\n> > This should be in a subshell, so that at the end we can actually remove\n> > the repo. This seems to be applicable to most of the other tests in this\n> > file too. Perhaps, we should clean it up as a precursor commit to this\n> > series?\n>\n> I have searched the usage of \"test_when_finished\", and I don't know why\n> we need to use subshell. Could you please explain this further here.\n\nKarthik may have been thinking about operating systems, such as\nMicrosoft Windows, which won't allow a directory to be deleted if that\ndirectory is in use. In this case, because the test cd's into \"repo\"\nand never cd's elsewhere, the directory is still in use when\ntest_when_finished() tries to delete \"repo\".\n\nHowever, there is an even more important reason to use a subshell, and\nthat is because a subshell ensures that the current working directory\nis effectively restored to the path which was current before the cd\ncommand. This is important since it guarantees that subsequent tests\nwill be run in the correct directory even if the preceding test bombed\nout part way through. For example:\n\n test_expect_success 'foo' '\n git init repo &&\n cd repo &&\n ...some more commands... &&\n cd ..\n '\n\nIf one of the commands in \"...some more commands...\" fails, then the\n`cd ..` will never be reached, and the current working directory will\nremain \"repo\" rather than reverting to the path prior to the cd\ncommand. Thus, any tests which follow this one in the script will end\nup running in the wrong directory. The proper way to protect against\nthis is:\n\n test_expect_success 'foo' '\n git init repo &&\n (\n cd repo &&\n ...some more commands...\n )\n '\n\nExiting the subshell will correctly restore the current working\ndirectory to the original path _regardless_ of whether the test\nsucceeds or fails somewhere in \"...some more commands...\". Using a\nsubshell also means that you don't have to manually restore the\nworking directory via `cd ..` or similar.\n"},{"id":"510863","messageId":"Z4sakhF9O7q8dib5@ArchLinux","threadId":"62743","inReplyTo":"CAPig+cRsAPp1APNJ7W337UNtunETr+Lnn-RcGrAXEFUhN1APyA@mail.gmail.com","subject":"Re: [PATCH 03/10] packed-backend: check whether the \"packed-refs\" is regular","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-18T03:05:54Z","receivedAt":"2025-01-18T03:04:38Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Jan 17, 2025 at 05:01:21PM -0500, Eric Sunshine wrote:\n> On Fri, Jan 17, 2025 at 8:59 AM shejialuo wrote:\n> > On Tue, Jan 07, 2025 at 08:33:56AM -0800, Karthik Nayak wrote:\n> > > shejialuo writes:\n> > > > +test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> > > > + test_when_finished \"rm -rf repo\" &&\n> > > > + git init repo &&\n> > > > + cd repo &&\n> > >\n> > > This should be in a subshell, so that at the end we can actually remove\n> > > the repo. This seems to be applicable to most of the other tests in this\n> > > file too. Perhaps, we should clean it up as a precursor commit to this\n> > > series?\n> >\n> > I have searched the usage of \"test_when_finished\", and I don't know why\n> > we need to use subshell. Could you please explain this further here.\n> \n> Karthik may have been thinking about operating systems, such as\n> Microsoft Windows, which won't allow a directory to be deleted if that\n> directory is in use. In this case, because the test cd's into \"repo\"\n> and never cd's elsewhere, the directory is still in use when\n> test_when_finished() tries to delete \"repo\".\n> \n> However, there is an even more important reason to use a subshell, and\n> that is because a subshell ensures that the current working directory\n> is effectively restored to the path which was current before the cd\n> command. This is important since it guarantees that subsequent tests\n> will be run in the correct directory even if the preceding test bombed\n> out part way through. For example:\n> \n> test_expect_success 'foo' '\n> git init repo &&\n> cd repo &&\n> ...some more commands... &&\n> cd ..\n> '\n> \n> If one of the commands in \"...some more commands...\" fails, then the\n> `cd ..` will never be reached, and the current working directory will\n> remain \"repo\" rather than reverting to the path prior to the cd\n> command. Thus, any tests which follow this one in the script will end\n> up running in the wrong directory. The proper way to protect against\n> this is:\n> \n> test_expect_success 'foo' '\n> git init repo &&\n> (\n> cd repo &&\n> ...some more commands...\n> )\n> '\n> \n> Exiting the subshell will correctly restore the current working\n> directory to the original path _regardless_ of whether the test\n> succeeds or fails somewhere in \"...some more commands...\". Using a\n> subshell also means that you don't have to manually restore the\n> working directory via `cd ..` or similar.\n\nThanks for above detailed explanation. I somehow understand why there\nwould be so many \"repo/repo/repo\" when I execute the test. I have\nthought that `test_expect_success` command will make the environment of\neach test totally independent. I will improve this in the next version.\n\n\nThanks,\nJialuo\n"},{"id":"510887","messageId":"CAOLa=ZSwdk_Vz_8ZMp1+ptjaeEFMWxEwAa8_U1j67fp2PPupSg@mail.gmail.com","threadId":"62743","inReplyTo":"CAPig+cRsAPp1APNJ7W337UNtunETr+Lnn-RcGrAXEFUhN1APyA@mail.gmail.com","subject":"Re: [PATCH 03/10] packed-backend: check whether the \"packed-refs\" is regular","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-01-19T08:03:39Z","receivedAt":"2025-01-19T08:03:40Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Eric Sunshine writes:\n\n> On Fri, Jan 17, 2025 at 8:59 AM shejialuo wrote:\n>> On Tue, Jan 07, 2025 at 08:33:56AM -0800, Karthik Nayak wrote:\n>> > shejialuo writes:\n>> > > +test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n>> > > + test_when_finished \"rm -rf repo\" &&\n>> > > + git init repo &&\n>> > > + cd repo &&\n>> >\n>> > This should be in a subshell, so that at the end we can actually remove\n>> > the repo. This seems to be applicable to most of the other tests in this\n>> > file too. Perhaps, we should clean it up as a precursor commit to this\n>> > series?\n>>\n>> I have searched the usage of \"test_when_finished\", and I don't know why\n>> we need to use subshell. Could you please explain this further here.\n>\n> Karthik may have been thinking about operating systems, such as\n> Microsoft Windows, which won't allow a directory to be deleted if that\n> directory is in use. In this case, because the test cd's into \"repo\"\n> and never cd's elsewhere, the directory is still in use when\n> test_when_finished() tries to delete \"repo\".\n>\n\nI didn't know this either. I was mostly talking about what you mentioned\nbelow.\n\n> However, there is an even more important reason to use a subshell, and\n> that is because a subshell ensures that the current working directory\n> is effectively restored to the path which was current before the cd\n> command. This is important since it guarantees that subsequent tests\n> will be run in the correct directory even if the preceding test bombed\n> out part way through. For example:\n>\n> test_expect_success 'foo' '\n> git init repo &&\n> cd repo &&\n> ...some more commands... &&\n> cd ..\n> '\n>\n> If one of the commands in \"...some more commands...\" fails, then the\n> `cd ..` will never be reached, and the current working directory will\n> remain \"repo\" rather than reverting to the path prior to the cd\n> command. Thus, any tests which follow this one in the script will end\n> up running in the wrong directory. The proper way to protect against\n> this is:\n>\n> test_expect_success 'foo' '\n> git init repo &&\n> (\n> cd repo &&\n> ...some more commands...\n> )\n> '\n>\n> Exiting the subshell will correctly restore the current working\n> directory to the original path _regardless_ of whether the test\n> succeeds or fails somewhere in \"...some more commands...\". Using a\n> subshell also means that you don't have to manually restore the\n> working directory via `cd ..` or similar.\n\nThis is was a super nice explanation compared to my single sentence.\nThanks!\n"},{"id":"511144","messageId":"Z5NGdXJtUngG478V@pks.im","threadId":"62743","inReplyTo":"Z4pnyhF2V2ykuHlg@ArchLinux","subject":"Re: [PATCH 04/10] packed-backend: add \"packed-refs\" header consistency check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-24T07:51:17Z","receivedAt":"2025-01-24T07:51:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Jan 17, 2025 at 10:23:06PM +0800, shejialuo wrote:\n> On Thu, Jan 16, 2025 at 02:57:37PM +0100, Patrick Steinhardt wrote:\n> > On Sun, Jan 05, 2025 at 09:49:37PM +0800, shejialuo wrote:\n> > > @@ -1779,7 +1867,24 @@ static int packed_fsck(struct ref_store *ref_store,\n> > > \t\tgoto cleanup;\n> > > \t}\n> > > \n> > > +\tif (strbuf_read_file(&packed_ref_content, refs->path, 0) < 0) {\n> > > +\t\t/*\n> > > +\t\t * Although we have checked that the file exists, there is a possibility\n> > > +\t\t * that it has been removed between the lstat() and the read attempt by\n> > > +\t\t * another process. In that case, we should not report an error.\n> > > +\t\t */\n> > > +\t\tif (errno == ENOENT)\n> > > +\t\t\tgoto cleanup;\n> > \n> > Unlikely, but good to guard us against that condition regardless. It's\n> > still not entirely race-free though because the file could meanwhile\n> > have changed into a symlink, and we wouldn't notice now. We could fix\n> > that by using open(O_NOFOLLOW), fstat the returne file descriptor and\n> > then use `strbuf_read()` to slurp in the file.\n> > \n> \n> Would this be too complicated for us to avoid race condition and we will\n> introduce a lot of code to handle above logic. Because there is a\n> possibility that when finishing reading the file content to the memory,\n> the file could be changed into a symlink and we cannot notice. So, I\n> wanna say we can't avoid race condition totally. It would be good if we\n> avoid race, but what I am concern about here is that we would make the\n> logic too complicated. So, could we make it unchanged?\n\nIt would ultimately only be two additional function calls, so I don't\nthink it's going to add a ton of complexity. Whether things are changing\n_after_ we have opened and read the file is a different issue, and I\nagree that we shouldn't have to care about that case. What we're after\nis whether things are correct when running consistency checks, it's\nalways a possibility that e.g. the packed-refs file gets rewritten while\nwe do it.\n\nPatrick\n"},{"id":"511145","messageId":"Z5NHHEw1-5Qe7agN@pks.im","threadId":"62743","inReplyTo":"Z4pdwiBvDlyC9TZW@ArchLinux","subject":"Re: [PATCH 01/10] files-backend: add object check for regular ref","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-01-24T07:54:04Z","receivedAt":"2025-01-24T07:54:08Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Jan 17, 2025 at 09:40:18PM +0800, shejialuo wrote:\n> On Thu, Jan 16, 2025 at 02:57:25PM +0100, Patrick Steinhardt wrote:\n> > On Sun, Jan 05, 2025 at 09:49:09PM +0800, shejialuo wrote:\n> > > Although we use \"parse_loose_ref_content\" to check whether the object id\n> > > is correct, we never parse it into the \"struct object\" structure thus we\n> > > ignore checking whether there is a real object existing in the repo and\n> > > whether the object type is correct.\n> > > \n> > > Use \"parse_object\" to parse the oid for the regular ref content. If the\n> > > object does not exist, report the error to the user by reusing the fsck\n> > > message \"BAD_REF_CONTENT\".\n> > > \n> > > Then, we need to check the type of the object. Just like \"git-fsck(1)\",\n> > > we only report \"not a commit\" error when the ref is a branch. Last,\n> > > update the test to exercise the code.\n> > \n> > I wonder whether it wouldn't make more sense to put this into a generic\n> > part of `git refs verify`. This isn't a check for whether the format of\n> > the files backend is correct, but rather a check whether the refdb is\n> > sane. As such, it also applies do the reftable backend.\n> > \n> > So should we maybe extend `git refs verify` so that it also knows to\n> > perform generic checks that apply independent of the backend in use?\n> > \n> \n> I somehow understand your meaning here and I think what your meaning\n> here is that we could use internal ref method to parse the oid after we\n> check the format of the ref files. Thus, we could totally make these two\n> different kinds of checks separately.\n> \n> However, if we have already parsed the raw ref files, we could reuse the\n> parsed hex and then use \"parse_object\" to get the object id to check.\n> This is the main reason why I add this check now.\n> \n> And I agree with your thinking here. Actually, we may put this into\n> object check part. Because in \"git-fsck(1)\", we parse the refdb to know\n> whether an object is dangling or not.\n> \n> I will postpone these checks in the later patches. Really thanks here\n> for this wonderful suggestion.\n\nYeah. I'm thinking ahead a bit in this context and want to avoid that we\neventually have to reimplement the same set of checks for every single\nref backend that we have. So separating the backend-generic bits from\nthe non-generic ones is what I'm after.\n\nPatrick\n"},{"id":"511453","messageId":"Z5r6ZnLH3Ee8IQnN@ArchLinux","threadId":"62743","inReplyTo":"Z3qNUizvHJLgMx1y@ArchLinux","subject":"[PATCH v2 0/8] add more ref consistency checks","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-30T04:04:54Z","receivedAt":"2025-01-30T04:03:24Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis version handles the following things:\n\n1. Remove code which checks the object where refs point to suggested by\n Patrick.\n2. Use subshell for the shell script to fix the problem to make sure the\n current working directory consistent.\n3. Optimize to avoid allocating too much memory.\n\nThis version is rebased to the latest master due to semantic conflict. I\ndon't provide range-diff here the mumber of commit is reduced. However,\nit won't bring too much burdern for the reviewer due to small change.\n\nThanks,\nJialuo\n\nshejialuo (8):\n t0602: use subshell to ensure working directory unchanged\n builtin/refs: get worktrees without reading head info\n packed-backend: check whether the \"packed-refs\" is regular\n packed-backend: add \"packed-refs\" header consistency check\n packed-backend: check whether the refname contains NUL characters\n packed-backend: add \"packed-refs\" entry consistency check\n packed-backend: check whether the \"packed-refs\" is sorted\n builtin/fsck: add `git refs verify` child process\n\n Documentation/fsck-msgids.txt | 22 +\n builtin/fsck.c | 30 +\n builtin/refs.c | 2 +-\n fsck.h | 6 +\n refs/packed-backend.c | 343 +++++++++-\n t/t0602-reffiles-fsck.sh | 1107 +++++++++++++++++++--------------\n worktree.c | 5 +\n worktree.h | 6 +\n 8 files changed, 1040 insertions(+), 481 deletions(-)\n\n-- \n2.48.1\n\n"},{"id":"511454","messageId":"Z5r64p7ZiCoETGnU@ArchLinux","threadId":"62743","inReplyTo":"Z5r6ZnLH3Ee8IQnN@ArchLinux","subject":"[PATCH v2 1/8] t0602: use subshell to ensure working directory unchanged","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-30T04:06:58Z","receivedAt":"2025-01-30T04:05:29Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"For every test, we would execute the command \"cd repo\" in the first but\nwe never execute the command \"cd ..\" to restore the working directory.\nHowever, it's either not a good idea use above way. Because if any test\nfails between \"cd repo\" and \"cd ..\", the \"cd ..\" will never be reached.\nAnd we cannot correctly restore the working directory.\n\nLet's use subshell to ensure that the current working directory could be\nrestored to the correct path.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n t/t0602-reffiles-fsck.sh | 967 ++++++++++++++++++++-------------------\n 1 file changed, 494 insertions(+), 473 deletions(-)\n\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex d4a08b823b..cf7a202d0d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -14,222 +14,229 @@ test_expect_success 'ref name should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b default-branch &&\n-\tgit tag default-tag &&\n-\tgit tag multi_hierarchy/default-tag &&\n-\n-\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n-\trm $branch_dir_prefix/@ &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n-\tgit refs verify 2>err &&\n-\trm $tag_dir_prefix/tag-1.lock &&\n-\ttest_must_be_empty err &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/.lock: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t(\n+\t\tcd repo &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b default-branch &&\n+\t\tgit tag default-tag &&\n+\t\tgit tag multi_hierarchy/default-tag &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\t\trm $branch_dir_prefix/@ &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $tag_dir_prefix/tag-1.lock &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \t\ttest_must_fail git refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\terror: refs/tags/.lock: badRefName: invalid refname format\n \t\tEOF\n-\t\trm -r \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $tag_dir_prefix/.lock &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=warn refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n-\ttest_must_be_empty err\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b branch-1 &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=warn refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm $branch_dir_prefix/.branch-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n '\n \n test_expect_success 'ref name check should work for multiple worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\n-\tcd repo &&\n-\ttest_commit initial &&\n-\tgit checkout -b branch-1 &&\n-\ttest_commit second &&\n-\tgit checkout -b branch-2 &&\n-\ttest_commit third &&\n-\tgit checkout -b branch-3 &&\n-\tgit worktree add ./worktree-1 branch-1 &&\n-\tgit worktree add ./worktree-2 branch-2 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n-\t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n \t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n-\n-\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n-\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err &&\n-\n-\tfor worktree in \"worktree-1\" \"worktree-2\"\n-\tdo\n+\t\tcd repo &&\n+\t\ttest_commit initial &&\n+\t\tgit checkout -b branch-1 &&\n+\t\ttest_commit second &&\n+\t\tgit checkout -b branch-2 &&\n+\t\ttest_commit third &&\n+\t\tgit checkout -b branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-1 &&\n+\t\tgit worktree add ./worktree-2 branch-2 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n \t\t(\n-\t\t\tcd $worktree &&\n-\t\t\ttest_must_fail git refs verify 2>err &&\n-\t\t\tcat >expect <<-EOF &&\n-\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\t\t\tEOF\n-\t\t\tsort err >sorted_err &&\n-\t\t\ttest_cmp expect sorted_err || return 1\n-\t\t)\n-\tdone\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\n+\t\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\t\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err &&\n+\n+\t\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\t\tdo\n+\t\t\t(\n+\t\t\t\tcd $worktree &&\n+\t\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\t\tcat >expect <<-EOF &&\n+\t\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\t\tEOF\n+\t\t\t\tsort err >sorted_err &&\n+\t\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t\t)\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tfor trailing_content in \" garbage\" \" more garbage\"\n-\tdo\n-\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-garbage &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n+\t\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n-\t'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\t'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n \n-\t garbage'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err\n+\t\t garbage'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -237,99 +244,103 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tbad_content_1=$(git rev-parse main)x &&\n-\tbad_content_2=xfsazqfxcadas &&\n-\tbad_content_3=Xfsazqfxcadas &&\n-\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n-\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n-\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n-\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n-\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n-\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tbad_content_1=$(git rev-parse main)x &&\n+\t\tbad_content_2=xfsazqfxcadas &&\n+\t\tbad_content_3=Xfsazqfxcadas &&\n+\t\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\t\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\t\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\t\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\t\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-complicated &&\n-\ttest_cmp expect err\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (aggregate)' '\n@@ -337,32 +348,34 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n-\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'the target of the textual symref should be checked' '\n@@ -370,28 +383,30 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n-\t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n-\n-\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n-\t\tgit refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked' '\n@@ -399,201 +414,207 @@ test_expect_success SYMLINKS 'symlink symref content should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n-\tEOF\n-\trm $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_cmp expect err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-good &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\t\tEOF\n+\t\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tmain_worktree_refdir_prefix=.git/refs/heads &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\n-\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tfor bad_referent_name in \".tag\" \"branch \"\n-\tdo\n-\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tmain_worktree_refdir_prefix=.git/refs/heads &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $worktree1_refdir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor bad_referent_name in \".tag\" \"branch \"\n+\t\tdo\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-garbage &&\n-\ttest_cmp expect err\n+\t\trm $worktree1_refdir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_done\n-- \n2.48.1\n\n"},{"id":"511455","messageId":"Z5r679AyETgMO5Ge@ArchLinux","threadId":"62743","inReplyTo":"Z5r6ZnLH3Ee8IQnN@ArchLinux","subject":"[PATCH v2 2/8] builtin/refs: get worktrees without reading head info","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-30T04:07:11Z","receivedAt":"2025-01-30T04:05:41Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\nand \"next_record\" which would check the correctness of the content of\nthe \"packed-ref\" file. When anything is bad, the program will die.\n\nIt may seem that we have nothing relevant to above feature, because we\nare going to read and parse the raw \"packed-ref\" file without creating\nthe snapshot and using the ref iterator to check the consistency.\n\nHowever, when using \"get_worktrees\" in \"builtin/refs\", we would parse\nthe \"HEAD\" information. If the referent of the \"HEAD\" is inside the\n\"packed-ref\", we will call \"create_snapshot\" function to parse the\n\"packed-ref\" to get the information. No matter whether the entry of\n\"HEAD\" in \"packed-ref\" is correct, \"create_snapshot\" would call\n\"verify_buffer_safe\" to check whether there is a newline in the last\nline of the file. If not, the program will die.\n\nAlthough this behavior has no harm for the program, it will\nshort-circuit the program. When the users execute \"git refs verify\" or\n\"git fsck\", we don't want to simply die the program but rather show the\nwarnings or errors as many as possible to info the users. So, we should\navoid reading the head info.\n\nFortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\nworktrees, 2023-12-29), we have introduced a function\n\"get_worktrees_internal\" which allows us to get worktrees without\nreading head info.\n\nCreate a new exposed function \"get_worktrees_without_reading_head\", then\nreplace the \"get_worktrees\" in \"builtin/refs\" with the new created\nfunction.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 2 +-\n worktree.c | 5 +++++\n worktree.h | 6 ++++++\n 3 files changed, 12 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex a29f195834..55ff5dae11 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tworktrees = get_worktrees();\n+\tworktrees = get_worktrees_without_reading_head();\n \tfor (size_t i = 0; worktrees[i]; i++)\n \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n \t\t\t\t &fsck_refs_options, worktrees[i]);\ndiff --git a/worktree.c b/worktree.c\nindex 248bbb39d4..89b7d86cef 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -175,6 +175,11 @@ struct worktree **get_worktrees(void)\n \treturn get_worktrees_internal(0);\n }\n \n+struct worktree **get_worktrees_without_reading_head(void)\n+{\n+\treturn get_worktrees_internal(1);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\ndiff --git a/worktree.h b/worktree.h\nindex 38145df80f..1ba4a161a0 100644\n--- a/worktree.h\n+++ b/worktree.h\n@@ -30,6 +30,12 @@ struct worktree {\n */\n struct worktree **get_worktrees(void);\n \n+/*\n+ * Like `get_worktrees`, but does not read HEAD. This is useful when checking\n+ * the consistency, as reading HEAD may not be necessary.\n+ */\n+struct worktree **get_worktrees_without_reading_head(void);\n+\n /*\n * Returns 1 if linked worktrees exist, 0 otherwise.\n */\n-- \n2.48.1\n\n"},{"id":"511456","messageId":"Z5r6-52eBgT4TUYG@ArchLinux","threadId":"62743","inReplyTo":"Z5r6ZnLH3Ee8IQnN@ArchLinux","subject":"[PATCH v2 3/8] packed-backend: check whether the \"packed-refs\" is regular","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-30T04:07:23Z","receivedAt":"2025-01-30T04:05:52Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\nconsistency and correctness of \"packed-refs\" file, they never check the\nfiletype of the \"packed-refs\". The user should always use \"git\npacked-refs\" command to create the raw regular \"packed-refs\" file, so we\nneed to explicitly check this in \"git refs verify\".\n\nWe could use the following two ways to check whether the \"packed-refs\"\nis regular:\n\n1. We could use \"lstat\" system call to check the file mode.\n2. We could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file\n If the returned fd value is less than 0, we could check whether the\n \"errno\" is \"ELOOP\" to report an error to the user.\n\nIt might seems that the method one is much easier than method two.\nHowever, method one has a significant drawback. When we have checked the\nfile mode using \"lstat\", we will need to read the file content, there is\na possibility that when finishing reading the file content to the\nmemory, the file could be changed into a symlink and we cannot notice.\n\nWith method two, we could get the \"fd\" firstly. Even if the file is\nchanged into a symlink, we could still operate the \"fd\" in the memory\nwhich is consistent across the checking which avoids race condition.\n\nReuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\nthe user if \"packed-refs\" is not a regular file.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 39 +++++++++++++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 22 ++++++++++++++++++++++\n 2 files changed, 57 insertions(+), 4 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a7b6f74b6e..6401cecd5f 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -4,6 +4,7 @@\n #include \"../git-compat-util.h\"\n #include \"../config.h\"\n #include \"../dir.h\"\n+#include \"../fsck.h\"\n #include \"../gettext.h\"\n #include \"../hash.h\"\n #include \"../hex.h\"\n@@ -1748,15 +1749,45 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n-static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED,\n+static int packed_fsck(struct ref_store *ref_store,\n+\t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n+\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n+\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tint ret = 0;\n+\tint fd;\n \n \tif (!is_main_worktree(wt))\n-\t\treturn 0;\n+\t\tgoto cleanup;\n \n-\treturn 0;\n+\tif (o->verbose)\n+\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n+\n+\tfd = open_nofollow(refs->path, O_RDONLY);\n+\tif (fd < 0) {\n+\t\t/*\n+\t\t * If the packed-refs file doesn't exist, there's nothing\n+\t\t * to check.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tif (errno == ELOOP) {\n+\t\t\tstruct fsck_ref_report report = { 0 };\n+\t\t\treport.path = \"packed-refs\";\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t\t \"not a regular file\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tret = error_errno(_(\"unable to open %s\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\treturn ret;\n }\n \n struct ref_storage_be refs_be_packed = {\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex cf7a202d0d..42c8d4ca1e 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -617,4 +617,26 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t)\n '\n \n+test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit pack-refs --all &&\n+\n+\t\tmv .git/packed-refs .git/packed-refs-back &&\n+\t\tln -sf packed-refs-bak .git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs: badRefFiletype: not a regular file\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"511457","messageId":"Z5r7BuEJvjwQ9f4G@ArchLinux","threadId":"62743","inReplyTo":"Z5r6ZnLH3Ee8IQnN@ArchLinux","subject":"[PATCH v2 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-30T04:07:34Z","receivedAt":"2025-01-30T04:06:03Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c::create_snapshot\", if there is a header (the line\nwhich starts with '#'), we will check whether the line starts with \"#\npack-refs with:\". As we are going to implement the header consistency\ncheck, we should port this check into \"packed_fsck\".\n\nHowever, the above check is not enough, this is because \"git pack-refs\"\nwill always write \"PACKED_REFS_HEADER\" which is a constant string to the\n\"packed-refs\" file. So, we should check the following things for the\nheader.\n\n1. If the header does not exist, we may report an error to the user\n because it should exist, but we do allow no header in \"packed-refs\"\n file. So, create a new fsck message \"packedRefMissingHeader(INFO)\" to\n warn the user and also keep compatibility.\n2. If the header content does not start with \"# packed-ref with:\", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n3. If the header content is not the same as the constant string\n \"PACKED_REFS_HEADER\", ideally, we should report an error to the user.\n However, we allow other contents as long as the header content starts\n with \"# packed-ref with:\". To keep compatibility, create a new fsck\n message \"unknownPackedRefHeader(INFO)\" to warn about this. We may\n tighten this rule in the future.\n\nIn order to achieve above checks, read the \"packed-refs\" file via\n\"strbuf_read\". Like what \"create_snapshot\" and other functions do, we\ncould split the line by finding the next newline in the buffer. When we\ncannot find a newline, we could report an error.\n\nSo, create a function \"packed_fsck_ref_next_line\" to find the next\nnewline and if there is no such newline, use\n\"packedRefEntryNotTerminated(ERROR)\" to report an error to the user.\n\nThen, parse the first line to apply the above three checks. Update the\ntest to excise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 16 +++++++\n fsck.h | 4 ++\n refs/packed-backend.c | 89 +++++++++++++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 46 ++++++++++++++++++\n 4 files changed, 155 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex b14bc44ca4..34375a3143 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -16,6 +16,10 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefHeader`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid\n+\theader.\n+\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n@@ -176,6 +180,13 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`packedRefEntryNotTerminated`::\n+\t(ERROR) The \"packed-refs\" file contains an entry that is\n+\tnot terminated by a newline.\n+\n+`packedRefMissingHeader`::\n+\t(INFO) The \"packed-refs\" file does not contain the header.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\n@@ -208,6 +219,11 @@\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \n+`unknownPackedRefHeader`::\n+\t(INFO) The \"packed-refs\" header starts with \"# pack-refs with:\"\n+\tbut the remaining content is not the same as what `git pack-refs`\n+\twould write.\n+\n `unknownType`::\n \t(ERROR) Found an unknown object type.\n \ndiff --git a/fsck.h b/fsck.h\nindex a44c231a5f..3107a0093d 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n@@ -53,6 +54,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE, ERROR) \\\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\n@@ -90,6 +92,8 @@ enum fsck_msg_type {\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n \tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n+\tFUNC(UNKNOWN_PACKED_REF_HEADER, INFO) \\\n+\tFUNC(PACKED_REF_MISSING_HEADER, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 6401cecd5f..883189f3a1 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1749,12 +1749,92 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+static int packed_fsck_ref_next_line(struct fsck_options *o,\n+\t\t\t\t struct strbuf *packed_entry, const char *start,\n+\t\t\t\t const char *eof, const char **eol)\n+{\n+\tint ret = 0;\n+\n+\t*eol = memchr(start, '\\n', eof - start);\n+\tif (!*eol) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\n+\t\treport.path = packed_entry->buf;\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n+\t\t\t\t \"'%.*s' is not terminated with a newline\",\n+\t\t\t\t (int)(eof - start), start);\n+\n+\t\t/*\n+\t\t * There is no newline but we still want to parse it to the end of\n+\t\t * the buffer.\n+\t\t */\n+\t\t*eol = eof;\n+\t}\n+\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_header(struct fsck_options *o, const char *start, const char *eol)\n+{\n+\tconst char *err_fmt = NULL;\n+\tint fsck_msg_id = -1;\n+\n+\tif (!starts_with(start, \"# pack-refs with:\")) {\n+\t\terr_fmt = \"'%.*s' does not start with '# pack-refs with:'\";\n+\t\tfsck_msg_id = FSCK_MSG_BAD_PACKED_REF_HEADER;\n+\t} else if (strncmp(start, PACKED_REFS_HEADER, strlen(PACKED_REFS_HEADER))) {\n+\t\terr_fmt = \"'%.*s' is an unknown packed-refs header\";\n+\t\tfsck_msg_id = FSCK_MSG_UNKNOWN_PACKED_REF_HEADER;\n+\t}\n+\n+\tif (err_fmt && fsck_msg_id >= 0) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs.header\";\n+\n+\t\treturn fsck_report_ref(o, &report, fsck_msg_id, err_fmt,\n+\t\t\t\t (int)(eol - start), start);\n+\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tint line_number = 1;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n+\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n+\tif (*start == '#') {\n+\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t} else {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs\";\n+\n+\t\tret |= fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_MISSING_HEADER,\n+\t\t\t\t \"missing header line\");\n+\t}\n+\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n static int packed_fsck(struct ref_store *ref_store,\n \t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tint ret = 0;\n \tint fd;\n \n@@ -1786,7 +1866,16 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n+\t\tret = error_errno(_(\"unable to read %s\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\n cleanup:\n+\tstrbuf_release(&packed_ref_content);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 42c8d4ca1e..a7b46b6cb9 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -639,4 +639,50 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-refs header should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tprintf \"$(git rev-parse main) refs/heads/main\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: packed-refs: packedRefMissingHeader: missing header line\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n+\t\t\t\t\"# pack-refs with traits: peeled fully-peeled sorted \" \\\n+\t\t\t\t\"# pack-refs with a: peeled fully-peeled\"\n+\t\tdo\n+\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with:'\\''\n+\t\t\tEOF\n+\t\t\trm .git/packed-refs &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor unknown_header in \"# pack-refs with: peeled fully-peeled sorted garbage\" \\\n+\t\t\t\t\"# pack-refs with: peeled\" \\\n+\t\t\t\t\"# pack-refs with: peeled peeled-fully sort\"\n+\t\tdo\n+\t\t\tprintf \"%s\\n\" \"$unknown_header\" >.git/packed-refs &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: packed-refs.header: unknownPackedRefHeader: '\\''$unknown_header'\\'' is an unknown packed-refs header\n+\t\t\tEOF\n+\t\t\trm .git/packed-refs &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"511458","messageId":"Z5r7EkDwEsxuLJzn@ArchLinux","threadId":"62743","inReplyTo":"Z5r6ZnLH3Ee8IQnN@ArchLinux","subject":"[PATCH v2 5/8] packed-backend: check whether the refname contains NUL characters","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-30T04:07:46Z","receivedAt":"2025-01-30T04:06:16Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already implemented the header consistency check for the raw\n\"packed-refs\" file. Before we implement the consistency check for each\nref entry, let's analysis [1] which reports that \"git fsck\" cannot\ndetect some NUL characters.\n\n\"packed-backend.c::next_record\" will use \"check_refname_format\" to check\nthe consistency of the refname. If it is not OK, the program will die.\nSo, we already have the code path and we must miss out something.\n\nWe use the following code to get the refname:\n\n strbuf_add(&iter->refname_buf, p, eol - p);\n iter->base.refname = iter->refname_buf.buf\n\nIn the above code, `p` is the start pointer of the refname and `eol` is\nthe next newline pointer. We calculate the length of the refname by\nsubtracting the two pointers. Then we add the memory range between `p`\nand `eol` to get the refname.\n\nHowever, if there are some NUL characters in the memory range between `p`\nand `eol`, we will see the refname as a valid ref name as long as the\nmemory range between `p` and first occurred NUL character is valid.\n\nIn order to catch above corruption, create a new function\n\"refname_contains_nul\" by searching the first NUL character. If it is\nnot at the end of the string, there must be some NUL characters in the\nrefname.\n\nUse this function in \"next_record\" function to die the program if\n\"refname_contains_nul\" returns true.\n\n[1] https://lore.kernel.org/git/6cfee0e4-3285-4f18-91ff-d097da9de737@rd10.de/\n\nReported-by: R. Diez \nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 19 +++++++++++++++++++\n 1 file changed, 19 insertions(+)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 883189f3a1..870c8e7aaa 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -494,6 +494,22 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n \t\t\t\t last_line, eof - last_line);\n }\n \n+/*\n+ * When parsing the \"packed-refs\" file, we will parse it line by line.\n+ * Because we know the start pointer of the refname and the next\n+ * newline pointer, we could calculate the length of the refname by\n+ * subtracting the two pointers. However, there is a corner case where\n+ * the refname contains corrupted embedded NUL characters. And\n+ * `check_refname_format()` will not catch this when the truncated\n+ * refname is still a valid refname. To prevent this, we need to check\n+ * whether the refname contains the NUL characters.\n+ */\n+static int refname_contains_nul(struct strbuf *refname)\n+{\n+\tconst char *pos = memchr(refname->buf, '\\0', refname->len + 1);\n+\treturn pos < refname->buf + refname->len;\n+}\n+\n #define SMALL_FILE_SIZE (32*1024)\n \n /*\n@@ -895,6 +911,9 @@ static int next_record(struct packed_ref_iterator *iter)\n \tstrbuf_add(&iter->refname_buf, p, eol - p);\n \titer->base.refname = iter->refname_buf.buf;\n \n+\tif (refname_contains_nul(&iter->refname_buf))\n+\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n+\n \tif (check_refname_format(iter->base.refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!refname_is_safe(iter->base.refname))\n \t\t\tdie(\"packed refname is dangerous: %s\",\n-- \n2.48.1\n\n"},{"id":"511459","messageId":"Z5r7Hlk_VS0jYU74@ArchLinux","threadId":"62743","inReplyTo":"Z5r6ZnLH3Ee8IQnN@ArchLinux","subject":"[PATCH v2 6/8] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-30T04:07:58Z","receivedAt":"2025-01-30T04:06:27Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will parse the ref entry to check the\nconsistency. This function has already checked the following things:\n\n1. Parse the main line of the ref entry, if the oid is not correct. It\n will die the program. And then it will check whether the next\n character of the oid is space. Then it will check whether the refname\n is correct.\n2. If the next line starts with '^', it will continue to parse the oid\n of the peeled oid content and check whether the last character is\n '\\n'.\n\nWe can iterate each line by using the \"packed_fsck_ref_next_line\"\nfunction. Then, create a new fsck message \"badPackedRefEntry(ERROR)\" to\nreport to the user when something is wrong.\n\nCreate two new functions \"packed_fsck_ref_main_line\" and\n\"packed_fsck_ref_peeled_line\" for case 1 and case 2 respectively. Last,\nupdate the unit test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 ++\n fsck.h | 1 +\n refs/packed-backend.c | 98 ++++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 42 +++++++++++++++\n 4 files changed, 143 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 34375a3143..2a7ec7592e 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -16,6 +16,9 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefEntry`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid entry.\n+\n `badPackedRefHeader`::\n \t(ERROR) The \"packed-refs\" file contains an invalid\n \theader.\ndiff --git a/fsck.h b/fsck.h\nindex 3107a0093d..40126242a4 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\n \tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 870c8e7aaa..271c740728 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1819,10 +1819,86 @@ static int packed_fsck_ref_header(struct fsck_options *o, const char *start, con\n \treturn 0;\n }\n \n+static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t struct strbuf *packed_entry,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id peeled;\n+\tconst char *p;\n+\n+\treport.path = packed_entry->buf;\n+\n+\tstart++;\n+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid peeled oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t}\n+\n+\tif (p != eol) {\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n+\t\t\t\t (int)(eol - p), p);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_main_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t struct strbuf *packed_entry,\n+\t\t\t\t struct strbuf *refname,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id oid;\n+\tconst char *p;\n+\n+\treport.path = packed_entry->buf;\n+\n+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t}\n+\n+\tif (p == eol || !isspace(*p)) {\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n+\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n+\t}\n+\n+\tp++;\n+\tstrbuf_reset(refname);\n+\tstrbuf_add(refname, p, eol - p);\n+\tif (refname_contains_nul(refname)) {\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"refname '%s' contains NULL binaries\",\n+\t\t\t\t refname->buf);\n+\t}\n+\n+\tif (check_refname_format(refname->buf, 0)) {\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n+\t\t\t\t \"has bad refname '%s'\", refname->buf);\n+\t}\n+\n+\treturn 0;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n \tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct strbuf refname = STRBUF_INIT;\n \tint line_number = 1;\n \tconst char *eol;\n \tint ret = 0;\n@@ -1843,6 +1919,26 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t \"missing header line\");\n \t}\n \n+\twhile (start < eof) {\n+\t\tstrbuf_reset(&packed_entry);\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n+\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n+\t\tret |= packed_fsck_ref_main_line(o, ref_store, &packed_entry, &refname, start, eol);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t\tif (start < eof && *start == '^') {\n+\t\t\tstrbuf_reset(&packed_entry);\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n+\t\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, &packed_entry,\n+\t\t\t\t\t\t\t start, eol);\n+\t\t\tstart = eol + 1;\n+\t\t\tline_number++;\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname);\n \tstrbuf_release(&packed_entry);\n \treturn ret;\n }\n@@ -1890,7 +1986,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex a7b46b6cb9..e4b4a58684 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -685,4 +685,46 @@ test_expect_success 'packed-refs header should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-refs content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tgit tag -a annotated-tag-2 -m tag-2 &&\n+\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n+\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n+\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\t\tprintf \"%s\\n\" \"$short_oid refs/heads/branch-1\" >>.git/packed-refs &&\n+\t\tprintf \"%sx\\n\" \"$branch_1_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s refs/heads/bad-branch\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s refs/heads/branch.\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s refs/tags/annotated-tag-3\\n\" \"$tag_1_oid\" >>.git/packed-refs &&\n+\t\tprintf \"^%s\\n\" \"$short_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s refs/tags/annotated-tag-4.\\n\" \"$tag_2_oid\" >>.git/packed-refs &&\n+\t\tprintf \"^%s garbage\\n\" \"$tag_2_peeled_oid\" >>.git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: badPackedRefEntry: '\\''$short_oid refs/heads/branch-1'\\'' has invalid oid\n+\t\terror: packed-refs line 3: badPackedRefEntry: has no space after oid '\\''$branch_1_oid'\\'' but with '\\''x'\\''\n+\t\terror: packed-refs line 4: badRefName: has bad refname '\\'' refs/heads/bad-branch'\\''\n+\t\terror: packed-refs line 5: badRefName: has bad refname '\\''refs/heads/branch.'\\''\n+\t\terror: packed-refs line 7: badPackedRefEntry: '\\''$short_oid'\\'' has invalid peeled oid\n+\t\terror: packed-refs line 8: badRefName: has bad refname '\\''refs/tags/annotated-tag-4.'\\''\n+\t\terror: packed-refs line 9: badPackedRefEntry: has trailing garbage after peeled oid '\\'' garbage'\\''\n+\t\tEOF\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"511460","messageId":"Z5r7KvL1bvSO4UQY@ArchLinux","threadId":"62743","inReplyTo":"Z5r6ZnLH3Ee8IQnN@ArchLinux","subject":"[PATCH v2 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-30T04:08:10Z","receivedAt":"2025-01-30T04:06:40Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We will always try to sort the \"packed-refs\" increasingly by comparing\nthe refname. So, we should add checks to verify whether the \"packed-refs\"\nis sorted.\n\nWe already have code to parse the content. Let's create a new structure\n\"fsck_packed_ref_entry\" to store the state during the parsing process\nfor every entry. It may seem that we could just add a new \"struct strbuf\nrefname\" into the \"struct fsck_packed_ref_entry\" and during the parsing\nprocess, we could store the refname into this structure and we could\ncompare later. However, this is not a good design due to the following\nreasons:\n\n1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n in the \"packed-refs\" file.\n2. The most important thing is that we cannot reuse the existing compare\n functions which cause repetition.\n\nSo, instead of storing the \"struct strbuf\", let's use the existing\nstructure \"struct snaphost_record\". And thus we could use the existing\nfunction \"cmp_packed_ref_records\".\n\nHowever, this function need an extra parameter for \"struct snaphost\".\nExtract the common part into a new function \"cmp_packed_ref_records\" to\nreuse this function to compare.\n\nThen, create a new function \"packed_fsck_ref_sorted\" to use the new fsck\nmessage \"packedRefUnsorted(ERROR)\" to report to the user.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 100 +++++++++++++++++++++++++++++++---\n t/t0602-reffiles-fsck.sh | 38 +++++++++++++\n 4 files changed, 135 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 2a7ec7592e..7a11d35c5e 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -190,6 +190,9 @@\n `packedRefMissingHeader`::\n \t(INFO) The \"packed-refs\" file does not contain the header.\n \n+`packedRefUnsorted`::\n+\t(ERROR) The \"packed-refs\" file is not sorted.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex 40126242a4..0d3d1045ae 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -56,6 +56,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n \tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n+\tFUNC(PACKED_REF_UNSORTED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 271c740728..b250f987b2 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -300,14 +300,9 @@ struct snapshot_record {\n \tsize_t len;\n };\n \n-static int cmp_packed_ref_records(const void *v1, const void *v2,\n-\t\t\t\t void *cb_data)\n-{\n-\tconst struct snapshot *snapshot = cb_data;\n-\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n-\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n-\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n \n+static int cmp_packed_refname(const char *r1, const char *r2)\n+{\n \twhile (1) {\n \t\tif (*r1 == '\\n')\n \t\t\treturn *r2 == '\\n' ? 0 : -1;\n@@ -322,6 +317,17 @@ static int cmp_packed_ref_records(const void *v1, const void *v2,\n \t}\n }\n \n+static int cmp_packed_ref_records(const void *v1, const void *v2,\n+\t\t\t\t void *cb_data)\n+{\n+\tconst struct snapshot *snapshot = cb_data;\n+\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n+\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n+\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n+\n+\treturn cmp_packed_refname(r1, r2);\n+}\n+\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n@@ -1768,6 +1774,28 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+struct fsck_packed_ref_entry {\n+\tint line_number;\n+\n+\tstruct snapshot_record record;\n+};\n+\n+static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(int line_number,\n+\t\t\t\t\t\t\t\t const char *start)\n+{\n+\tstruct fsck_packed_ref_entry *entry = xcalloc(1, sizeof(*entry));\n+\tentry->line_number = line_number;\n+\tentry->record.start = start;\n+\treturn entry;\n+}\n+\n+static void free_fsck_packed_ref_entries(struct fsck_packed_ref_entry **entries, int nr)\n+{\n+\tfor (int i = 0; i < nr; i++)\n+\t\tfree(entries[i]);\n+\tfree(entries);\n+}\n+\n static int packed_fsck_ref_next_line(struct fsck_options *o,\n \t\t\t\t struct strbuf *packed_entry, const char *start,\n \t\t\t\t const char *eof, const char **eol)\n@@ -1893,13 +1921,60 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n \treturn 0;\n }\n \n+static int packed_fsck_ref_sorted(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t struct fsck_packed_ref_entry **entries,\n+\t\t\t\t int nr)\n+{\n+\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct strbuf refname1 = STRBUF_INIT;\n+\tstruct strbuf refname2 = STRBUF_INIT;\n+\tint ret = 0;\n+\n+\tfor (int i = 1; i < nr; i++) {\n+\t\tconst char *r1 = entries[i - 1]->record.start + hexsz + 1;\n+\t\tconst char *r2 = entries[i]->record.start + hexsz + 1;\n+\n+\t\tif (cmp_packed_refname(r1, r2) >= 0) {\n+\t\t\tconst char *err_fmt =\n+\t\t\t\t\"refname '%s' is not less than next refname '%s'\";\n+\t\t\tconst char *eol;\n+\t\t\teol = memchr(entries[i - 1]->record.start, '\\n',\n+\t\t\t\t entries[i - 1]->record.len);\n+\t\t\tstrbuf_add(&refname1, r1, eol - r1);\n+\t\t\teol = memchr(entries[i]->record.start, '\\n',\n+\t\t\t\t entries[i]->record.len);\n+\t\t\tstrbuf_add(&refname2, r2, eol - r2);\n+\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\",\n+\t\t\t\t entries[i - 1]->line_number);\n+\t\t\treport.path = packed_entry.buf;\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_PACKED_REF_UNSORTED,\n+\t\t\t\t\t err_fmt, refname1.buf, refname2.buf);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname1);\n+\tstrbuf_release(&refname2);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n \tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_packed_ref_entry **entries;\n \tstruct strbuf refname = STRBUF_INIT;\n+\tint entry_alloc = 20;\n \tint line_number = 1;\n+\tint entry_nr = 0;\n \tconst char *eol;\n \tint ret = 0;\n \n@@ -1919,7 +1994,13 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t \"missing header line\");\n \t}\n \n+\tALLOC_ARRAY(entries, entry_alloc);\n \twhile (start < eof) {\n+\t\tstruct fsck_packed_ref_entry *entry\n+\t\t\t= create_fsck_packed_ref_entry(line_number, start);\n+\n+\t\tALLOC_GROW(entries, entry_nr + 1, entry_alloc);\n+\t\tentries[entry_nr++] = entry;\n \t\tstrbuf_reset(&packed_entry);\n \t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n \t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n@@ -1935,11 +2016,16 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\tstart = eol + 1;\n \t\t\tline_number++;\n \t\t}\n+\t\tentry->record.len = start - entry->record.start;\n \t}\n \n+\tif (!ret)\n+\t\tret |= packed_fsck_ref_sorted(o, ref_store, entries, entry_nr);\n+\n \tstrbuf_release(&packed_entry);\n \tstrbuf_release(&refname);\n \tstrbuf_release(&packed_entry);\n+\tfree_fsck_packed_ref_entries(entries, entry_nr);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex e4b4a58684..9d802d71a9 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -727,4 +727,42 @@ test_expect_success 'packed-refs content should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-ref sorted should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname1'\\'' is not less than next refname '\\''$refname2'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$tag_1_oid\" \"$refname3\" >>.git/packed-refs &&\n+\t\tprintf \"^%s\\n\" \"$tag_1_peeled_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname2\" >>.git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname3'\\'' is not less than next refname '\\''$refname2'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"511461","messageId":"Z5r7NnzvirWEljwV@ArchLinux","threadId":"62743","inReplyTo":"Z5r6ZnLH3Ee8IQnN@ArchLinux","subject":"[PATCH v2 8/8] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-30T04:08:22Z","receivedAt":"2025-01-30T04:06:51Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"At now, we have already implemented the ref consistency checks for both\n\"files-backend\" and \"packed-backend\". Although we would check some\nredundant things, it won't cause trouble. So, let's integrate it into\nthe \"git-fsck(1)\" command to get feedback from the users. And also by\ncalling \"git refs verify\" in \"git-fsck(1)\", we make sure that the new\nadded checks don't break.\n\nIntroduce a new function \"fsck_refs\" that initializes and runs a child\nprocess to execute the \"git refs verify\" command. In order to provide\nthe user interface create a progress which makes the total task be 1.\nIt's hard to know how many loose refs we will check now. We might\nimprove this later.\n\nAnd we run this function in the first execution sequence of\n\"git-fsck(1)\" because we don't want the existing code of \"git-fsck(1)\"\nwhich implicitly checks the consistency of refs to die the program.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/fsck.c | 30 ++++++++++++++++++++++++++++++\n 1 file changed, 30 insertions(+)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 7a4dcb0716..9a8613d07f 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -905,6 +905,34 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n \treturn res;\n }\n \n+static void fsck_refs(struct repository *r)\n+{\n+\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n+\tstruct progress *progress = NULL;\n+\tuint64_t progress_num = 1;\n+\n+\tif (show_progress)\n+\t\tprogress = start_progress(r, _(\"Checking ref database\"),\n+\t\t\t\t\t progress_num);\n+\n+\tif (verbose)\n+\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n+\n+\tchild_process_init(&refs_verify);\n+\trefs_verify.git_cmd = 1;\n+\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n+\tif (verbose)\n+\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n+\tif (check_strict)\n+\t\tstrvec_push(&refs_verify.args, \"--strict\");\n+\n+\tif (run_command(&refs_verify))\n+\t\terrors_found |= ERROR_REFS;\n+\n+\tdisplay_progress(progress, 1);\n+\tstop_progress(&progress);\n+}\n+\n static char const * const fsck_usage[] = {\n \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n@@ -970,6 +998,8 @@ int cmd_fsck(int argc,\n \tgit_config(git_fsck_config, &fsck_obj_options);\n \tprepare_repo_settings(the_repository);\n \n+\tfsck_refs(the_repository);\n+\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n \t\tfor_each_packed_object(the_repository,\n-- \n2.48.1\n\n"},{"id":"511513","messageId":"xmqqcyg4fas5.fsf@gitster.g","threadId":"62743","inReplyTo":"Z5r64p7ZiCoETGnU@ArchLinux","subject":"Re: [PATCH v2 1/8] t0602: use subshell to ensure working directory unchanged","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-30T17:53:46Z","receivedAt":"2025-01-30T17:53:50Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> For every test, we would execute the command \"cd repo\" in the first but\n> we never execute the command \"cd ..\" to restore the working directory.\n> However, it's either not a good idea use above way. Because if any test\n> fails between \"cd repo\" and \"cd ..\", the \"cd ..\" will never be reached.\n> And we cannot correctly restore the working directory.\n>\n> Let's use subshell to ensure that the current working directory could be\n> restored to the correct path.\n>\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> t/t0602-reffiles-fsck.sh | 967 ++++++++++++++++++++-------------------\n> 1 file changed, 494 insertions(+), 473 deletions(-)\n\nNote for bystanders who may be interested in helping to ensure\ncorrectness of this step.\n\nThe patch meant for the machines we see here is unreadable for\nhumans [*], but the result of applying it and then running\n\n $ git show -wW t/\n\ngives me a very clear \"from here to there, the entire thing now has\na pair of () around it\" pattern. If you look at the clean-up step\neach test piece defines with test_when_finished at the front, and\ncomparing it with the directory name the test repository \"git init\"\nin each test piece creates and \"cd\" goes into, it is fairly easy to\nsee that the patch is doing the right thing without doing anything\nunwanted.\n\nAll the here-doc in the test are now indented one level deeper, but\nyou can check that they use \"<<-EOF\" to be oblivious to the leading\ntabs, making this conversion a safe one.\n\nOne thing that is hard to validate by code inspection alone is\n\n - This change will change the commit timestamps of the commits\n created by \"test_commit\" helper function, now that they are run\n in subshells to get their internal clock reset in each test\n piece.\n\nBut if the tests rely on the exact commit object names, running the\nresulting script just once would be sufficient to notice.\n\nOverall, very nicely done.\n\nQueued. Thanks.\n\n\n[Footnote]\n\n * No, I do not mean to say that you should spend time trying to\n make the message readable by humans in a case like this. A patch\n that can be mechanically processed and leave the byte sequence\n you intended to give the recipients is exactly what we want.\n"},{"id":"511514","messageId":"xmqq5xlwfa9u.fsf@gitster.g","threadId":"62743","inReplyTo":"Z5r679AyETgMO5Ge@ArchLinux","subject":"Re: [PATCH v2 2/8] builtin/refs: get worktrees without reading head info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-30T18:04:45Z","receivedAt":"2025-01-30T18:04:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> Although this behavior has no harm for the program, it will\n> short-circuit the program. When the users execute \"git refs verify\" or\n> \"git fsck\", we don't want to simply die the program but rather show the\n> warnings or errors as many as possible to info the users.\n\n\"info\" is not a verb; \"inform\"?\n\nI can understand what you want to say with \"show the warnings or\nerrors as many as possible\", but giving errors on the same issue\nmany times is not what you meant---rather, you want the checker to\nkeep going and discover errors in many _other things_, after it\nfinds a single error in \"HEAD\".\n\n\t..., we do want to diagnose a broken \"HEAD\", but we want to\n\tnotice as many breakages on other refs as we can instead of\n\tdying after finding the first breakage. Dying on a broken\n\t\"HEAD\" done by get_worktrees() goes against this goal.\n\nor something, perhaps. Such a rewrite makes the sentence \"Although\n... short-circuit the program.\" unnecessary.\n\n> So, we should\n> avoid reading the head info.\n\nWith one reservation. We still want to diagnose a broken \"HEAD\", so\nI'd probably strike this sentence out, and add a statement that says\nwe still check the contents of \"HEAD\" elsewhere as a substitute at\nthe end of the proposed commit log message, if I were writing it,\nafter explaining the use of get_worktrees_without_reading_head()\nyou did in the following two paragraphs (both of which read well).\n\nThanks.\n"},{"id":"511515","messageId":"xmqqplk4duuk.fsf@gitster.g","threadId":"62743","inReplyTo":"Z5r6-52eBgT4TUYG@ArchLinux","subject":"Re: [PATCH v2 3/8] packed-backend: check whether the \"packed-refs\" is regular","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-30T18:23:15Z","receivedAt":"2025-01-30T18:23:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> It might seems that the method one is much easier than method two.\n> However, method one has a significant drawback. When we have checked the\n> file mode using \"lstat\", we will need to read the file content, there is\n> a possibility that when finishing reading the file content to the\n> memory, the file could be changed into a symlink and we cannot notice.\n\nTo me, the above sounds like saying:\n\n The user can run 'git refs verify' and it may declare that refs\n are all good, and then somebody else can come in and turn the\n packed-refs file into a bad one, but the user will not notice\n the mischeif until the check is run the next time.\n\nIt is just the time that somebody else comes in becomes a bit\nearlier than the time the 'git refs verify' command finishes, and\nthere is no fundamental difference.\n\n> With method two, we could get the \"fd\" firstly. Even if the file is\n> changed into a symlink, we could still operate the \"fd\" in the memory\n> which is consistent across the checking which avoids race condition.\n\nThe end result is the same with the lstat(2) approach, isn't it,\nthough?. 'git refs verify' may say \"I opened the file without\nfollowing symlink and checked the contents, which turned out to be\nperfectly fine\". But because that somebody else came in just after\nthe command did nofollow-open and swapped the packed-refs file, the\nrepository has a packed-refs file that is not a regular file after\nthe command returns success. So I am not sure if I am following\nyour argument to favor the latter over the former. What am I\nmissing?\n\nAs long as both approaches are equally portable, I do not think it\nmatters which one we pick from correctness point of view, and we can\npick the one that is easier to use to implement the feature.\n\nOn a platform without O_NOFOLLOW, open_nofollow() falls back to the\nlstat and open, so your \"open_nofollow() is better than lstat() and\nopen()\" argument does not portably work, though.\n\n> Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n> the user if \"packed-refs\" is not a regular file.\n\nGood. Say \"regular file\" on the commit title, too, and it would be\nperfect.\n\n> -static int packed_fsck(struct ref_store *ref_store UNUSED,\n> -\t\t struct fsck_options *o UNUSED,\n> +static int packed_fsck(struct ref_store *ref_store,\n> +\t\t struct fsck_options *o,\n> \t\t struct worktree *wt)\n> {\n> +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> +\tint ret = 0;\n> +\tint fd;\n> \n> \tif (!is_main_worktree(wt))\n> -\t\treturn 0;\n> +\t\tgoto cleanup;\n> \n> -\treturn 0;\n> +\tif (o->verbose)\n> +\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n> +\n> +\tfd = open_nofollow(refs->path, O_RDONLY);\n> +\tif (fd < 0) {\n> +\t\t/*\n> +\t\t * If the packed-refs file doesn't exist, there's nothing\n> +\t\t * to check.\n> +\t\t */\n> +\t\tif (errno == ENOENT)\n> +\t\t\tgoto cleanup;\n> +\n> +\t\tif (errno == ELOOP) {\n> +\t\t\tstruct fsck_ref_report report = { 0 };\n> +\t\t\treport.path = \"packed-refs\";\n> +\t\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n> +\t\t\t\t\t \"not a regular file\");\n> +\t\t\tgoto cleanup;\n> +\t\t}\n> +\n> +\t\tret = error_errno(_(\"unable to open %s\"), refs->path);\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +cleanup:\n> +\treturn ret;\n> }\n\nLooking good.\n\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index cf7a202d0d..42c8d4ca1e 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -617,4 +617,26 @@ test_expect_success 'ref content checks should work with worktrees' '\n> \t)\n> '\n> \n> +test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\ttest_commit default &&\n> +\t\tgit branch branch-1 &&\n> +\t\tgit branch branch-2 &&\n> +\t\tgit branch branch-3 &&\n> +\t\tgit pack-refs --all &&\n> +\n> +\t\tmv .git/packed-refs .git/packed-refs-back &&\n> +\t\tln -sf packed-refs-bak .git/packed-refs &&\n> +\t\ttest_must_fail git refs verify 2>err &&\n> +\t\tcat >expect <<-EOF &&\n> +\t\terror: packed-refs: badRefFiletype: not a regular file\n> +\t\tEOF\n> +\t\trm .git/packed-refs &&\n> +\t\ttest_cmp expect err\n> +\t)\n> +'\n> +\n> test_done\n\nOK. I notice that the previous step did not have any new test\nassociated with it. Perhaps we can corrupt \"HEAD\" *and* replace\npacked-refs file with a symbolic link (or do some other damage\nto the refs) and make sure both breakages are reported?\n\nIt does not have to be done in this step, and certainly not as a\npart of this single test this step adds, but we'd want it tested\nsomewhere.\n\nThanks.\n"},{"id":"511516","messageId":"xmqq1pwkdt7r.fsf@gitster.g","threadId":"62743","inReplyTo":"Z5r7BuEJvjwQ9f4G@ArchLinux","subject":"Re: [PATCH v2 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-30T18:58:32Z","receivedAt":"2025-01-30T18:58:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> In \"packed-backend.c::create_snapshot\", if there is a header (the line\n> which starts with '#'), we will check whether the line starts with \"#\n> pack-refs with:\". As we are going to implement the header consistency\n> check, we should port this check into \"packed_fsck\".\n>\n> However, the above check is not enough, this is because \"git pack-refs\"\n> will always write \"PACKED_REFS_HEADER\" which is a constant string to the\n> \"packed-refs\" file. So, we should check the following things for the\n> header.\n\nI haven't done history digging in this area for a while, but we\nshould make sure we are not flagging a file that was written in\nancient version of Git whose repository is still supported.\n\n> 1. If the header does not exist, we may report an error to the user\n> because it should exist, but we do allow no header in \"packed-refs\"\n> file. So, create a new fsck message \"packedRefMissingHeader(INFO)\" to\n> warn the user and also keep compatibility.\n\nAre we sure \"it should exist\"? I think the header did not exist\nbefore \"Git v1.5.0\". I didn't check with other reimplementations of\nGit (like jgit or libgit2), but as long as our reading side of the\nruntime allows a packed-refs file without the header without\ncomplaint, I do not think it is a good idea to treat it as a\nreport-worthy event from \"git fsck\".\n\n> 2. If the header content does not start with \"# packed-ref with:\", we\n> should report an error just like what \"create_snapshot\" does. So,\n> create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n\nThis I can agree with. If the first line begins with \"#\" but not\nwith that string (with a trailing SP), that is a sign that it may\nnot even be a valid packed-refs file, which is a report-worthy\nevent.\n\n> 3. If the header content is not the same as the constant string\n> \"PACKED_REFS_HEADER\", ideally, we should report an error to the user.\n\nNO. THAT IS NOT IDEAL AT ALL.\n\nThe header was written like this:\n\n /* perhaps other traits later as well */\n fprintf(cbdata.refs_file, \"# pack-refs with: peeled \\n\");\n\nin the older versions of Git before it was made into a separate\npreprocessor macro and lost the comment (the above excerpt is from\n\"git show v1.5.0:builtin-pack-refs.c\").\n\nNotice \"other traits later\" in the comment?\n\nThe thing is _designed_ to be extensible. In fact, these days we\nsupport a few more traits\n\n static const char PACKED_REFS_HEADER[] =\n \"# pack-refs with: peeled fully-peeled sorted \\n\";\n\n(an excerpt from the current refs/packed-backend.c).\n\nReporting an error when you see something written by an older\nversion of Git is far from ideal.\n\n> However, we allow other contents as long as the header content starts\n> with \"# packed-ref with:\". To keep compatibility, create a new fsck\n> message \"unknownPackedRefHeader(INFO)\" to warn about this. We may\n> tighten this rule in the future.\n\nWhatever we do, what we do with an unknown trait should be in line\nwith what the runtime does. If the runtime failed (we do not, but\nthis is to illustrate the principle [*]) on a packed-refs file\nwithout \"sorted\" trait, noticing that \"sorted\" is not there and\nflagging as an error is a good thing to do. But if the runtime\ngracefully degrades and sorts the list of refs read from such a\npacked-refs file before continuing, then a packed-refs file that\nlack \"sorted\" trait is not a report-worthy event.\n\nI do not offhand recall if we introduced the concept of mandatory vs\noptional traits in the packed-refs part of the system (like we have\nin the index extension subsystem, where a version of Git that\nencounters an unknown *and* mandatory index extension must refuse to\ntouch the repository), but if there is a mandatory trait declared in\nthe header that our version of Git does not understand, it is a\nreport-worthy event that must be flagged with \"git refs verify\".\n\n> +static int packed_fsck_ref_header(struct fsck_options *o, const char *start, const char *eol)\n> +{\n> +\tconst char *err_fmt = NULL;\n> +\tint fsck_msg_id = -1;\n> +\n> +\tif (!starts_with(start, \"# pack-refs with:\")) {\n> +\t\terr_fmt = \"'%.*s' does not start with '# pack-refs with:'\";\n> +\t\tfsck_msg_id = FSCK_MSG_BAD_PACKED_REF_HEADER;\n> +\t} else if (strncmp(start, PACKED_REFS_HEADER, strlen(PACKED_REFS_HEADER))) {\n> +\t\terr_fmt = \"'%.*s' is an unknown packed-refs header\";\n> +\t\tfsck_msg_id = FSCK_MSG_UNKNOWN_PACKED_REF_HEADER;\n> +\t}\n\nAs I outlined above, this is totally unacceptable. \n\nInspecting the header is good, but if this code claims to be a\nchecker, it should do at least what the runtime does, i.e. parse the\nheader to tell what traits the packed-file declares, not just\nassuming that it is a fixed string. And error on unknown trait(s)\nif they are mandatory (if such a concept is implemented in the\nruntime reading side). Informing on an unknown and optional\ntrait(s) I can live with, but personally I wouldn't recommend it.\n\nIn other words, report loudly if it is an error, but otherwise stay\nsilent if we know we tolerate it well. \n\n> +static int packed_fsck_ref_content(struct fsck_options *o,\n> +\t\t\t\t const char *start, const char *eof)\n> +{\n> +\tstruct strbuf packed_entry = STRBUF_INIT;\n> +\tint line_number = 1;\n\nWe limit ourselves with about 1 billion refs in the packed-refs\nfile, which may be plenty, but I do not quite understand the use of\nthis variable. There is no loop inside this so ...\n\n> +\tconst char *eol;\n> +\tint ret = 0;\n> +\n> +\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n\n... this is always line #1, and then\n\n> +\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n> +\tif (*start == '#') {\n> +\t\tret |= packed_fsck_ref_header(o, start, eol);\n> +\n> +\t\tstart = eol + 1;\n> +\t\tline_number++;\n\n... it may be incremented, but upon returning from the funcition, it\nis lost.\n\nPerhaps you wanted to make it a function-scope static, but then you\nare allowed to read one single packed-refs file during the life of\nyour process before you exit, which I am not sure is what you want?\n\n> +\t} else {\n> +\t\tstruct fsck_ref_report report = { 0 };\n> +\t\treport.path = \"packed-refs\";\n> +\n> +\t\tret |= fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_PACKED_REF_MISSING_HEADER,\n> +\t\t\t\t \"missing header line\");\n> +\t}\n> +\n> +\tstrbuf_release(&packed_entry);\n> +\treturn ret;\n> +}\n\nI'll stop here for now.\n\nThanks.\n\n"},{"id":"511517","messageId":"xmqqwmecceh1.fsf@gitster.g","threadId":"62743","inReplyTo":"Z5r7KvL1bvSO4UQY@ArchLinux","subject":"Re: [PATCH v2 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-30T19:02:18Z","receivedAt":"2025-01-30T19:02:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> We will always try to sort the \"packed-refs\" increasingly by comparing\n> the refname. So, we should add checks to verify whether the \"packed-refs\"\n> is sorted.\n\nDo this _ONLY_ when the packed-refs file has a header that declares\n\"sorted\" trait. Insisting on a packed-refs file that does not would\nmean you are stricter than the runtime contract allows.\n\n> +struct fsck_packed_ref_entry {\n> +\tint line_number;\n> +\n> +\tstruct snapshot_record record;\n> +};\n\nNot a huge deal, as 1 billion is still plenty of a large number, but\nthe same comment on the line-number applies here. We might want to\nconsistently use ulong for line numbers of files we read from.\n"},{"id":"511518","messageId":"xmqqsep0ceec.fsf@gitster.g","threadId":"62743","inReplyTo":"Z5r7NnzvirWEljwV@ArchLinux","subject":"Re: [PATCH v2 8/8] builtin/fsck: add `git refs verify` child process","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-30T19:03:55Z","receivedAt":"2025-01-30T19:03:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> +static void fsck_refs(struct repository *r)\n> +{\n> +\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n> +\tstruct progress *progress = NULL;\n> +\tuint64_t progress_num = 1;\n> +\n> +\tif (show_progress)\n> +\t\tprogress = start_progress(r, _(\"Checking ref database\"),\n> +\t\t\t\t\t progress_num);\n\nI do not see why we need an extra variable progress_num here. Just\npassing a literal constant 1 should be sufficient. The called\nfunction has function prototype to help the compiler promite it to\nthe appropritate type.\n\nThanks.\n"},{"id":"511556","messageId":"Z5zQMCRETczMQMxj@ArchLinux","threadId":"62743","inReplyTo":"xmqq5xlwfa9u.fsf@gitster.g","subject":"Re: [PATCH v2 2/8] builtin/refs: get worktrees without reading head info","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-31T13:29:20Z","receivedAt":"2025-01-31T13:27:47Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 30, 2025 at 10:04:45AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > Although this behavior has no harm for the program, it will\n> > short-circuit the program. When the users execute \"git refs verify\" or\n> > \"git fsck\", we don't want to simply die the program but rather show the\n> > warnings or errors as many as possible to info the users.\n> \n> \"info\" is not a verb; \"inform\"?\n> \n\nLet me improve this in the next version.\n\n> I can understand what you want to say with \"show the warnings or\n> errors as many as possible\", but giving errors on the same issue\n> many times is not what you meant\n\nYes, this is correct.\n\n> ---rather, you want the checker to\n> keep going and discover errors in many _other things_, after it\n> finds a single error in \"HEAD\".\n> \n\nI think this is a misunderstanding. Let me explain more to you.\n\n1. If the content of the \"HEAD\" is not correct, we won't detect the\ncurrent directory as valid git repository.\n2. If the referent of the \"HEAD\" is not in the \"packed-refs\", the\nreferent must be a loose ref or don't exist. In this situation, because\nwe will never touch the packed backend.\n3. If the referent of the \"HEAD\" is in the \"packed-refs\", it will call\n\"create_snapshot\" to create the snapshot. In this function, it would\ncall \"verify_buffer_safe\" to check the following things:\n 1. Check the correctness of th header.\n 2. Check via \"verify_buffer_safe\" method\n So, even the referent entry is not correct in the \"packed-refs\", the\n program won't die. But the above two cases will let the program die.\n\nI want to say we cannot find any error in \"HEAD\" at now as above\ndescribed. From my perspective, we should retain the paragraph:\n\n> Although this behavior has no harm for the program...\n\nBut we should change the statement\n\n> we don't want to simply die the program but rather show the\n> warnings or errors as many as possible to info the users. So, we should\n> avoid reading the head info.\n\nto\n\n We should avoid reading the head information, which may execute the\n read operation in packed backend with stricter checks to die the\n program. Instead, we should continue to check other parts of the\n \"packed-refs\" file completely.\n\n> With one reservation. We still want to diagnose a broken \"HEAD\", so\n> I'd probably strike this sentence out, and add a statement that says\n> we still check the contents of \"HEAD\" elsewhere as a substitute at\n> the end of the proposed commit log message, if I were writing it,\n> after explaining the use of get_worktrees_without_reading_head()\n> you did in the following two paragraphs (both of which read well).\n\nI want to say that we cannot check the content of the \"HEAD\" itself. If\nthe content of \"HEAD\" is not correct, we cannot detect the current\ndirectory as a valid git repository. So, there is no need to say \"we\nwill check the contents of 'HEAD' else where\".\n\nI think the misunderstanding is that you think that if the \"HEAD\" is not\ncorrect, the program will die but actually it is not.\n\nThanks,\nJialuo\n"},{"id":"511558","messageId":"Z5zWE1M4u3NrROI-@ArchLinux","threadId":"62743","inReplyTo":"xmqqplk4duuk.fsf@gitster.g","subject":"Re: [PATCH v2 3/8] packed-backend: check whether the \"packed-refs\" is regular","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-31T13:54:27Z","receivedAt":"2025-01-31T13:52:54Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 30, 2025 at 10:23:15AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > It might seems that the method one is much easier than method two.\n> > However, method one has a significant drawback. When we have checked the\n> > file mode using \"lstat\", we will need to read the file content, there is\n> > a possibility that when finishing reading the file content to the\n> > memory, the file could be changed into a symlink and we cannot notice.\n> \n> To me, the above sounds like saying:\n> \n> The user can run 'git refs verify' and it may declare that refs\n> are all good, and then somebody else can come in and turn the\n> packed-refs file into a bad one, but the user will not notice\n> the mischeif until the check is run the next time.\n> \n\nYes, it is.\n\n> It is just the time that somebody else comes in becomes a bit\n> earlier than the time the 'git refs verify' command finishes, and\n> there is no fundamental difference.\n> \n> > With method two, we could get the \"fd\" firstly. Even if the file is\n> > changed into a symlink, we could still operate the \"fd\" in the memory\n> > which is consistent across the checking which avoids race condition.\n> \n> The end result is the same with the lstat(2) approach, isn't it,\n> though?. 'git refs verify' may say \"I opened the file without\n> following symlink and checked the contents, which turned out to be\n> perfectly fine\". But because that somebody else came in just after\n> the command did nofollow-open and swapped the packed-refs file, the\n> repository has a packed-refs file that is not a regular file after\n> the command returns success. So I am not sure if I am following\n> your argument to favor the latter over the former. What am I\n> missing?\n> \n\nLet me give you some background. In the version 1, I used the following\nway:\n\n```c\nlstat(...)\nif (!IS_REG(...))\n report_error(...);\nstrbuf_read(...)\n```\n\nPatrick has told me that there is a possibility that between the `IS_REG`\nand `strbuf_read`, the \"packed-refs\" could be converted into a symlink.\nSo, my idea is that we could use `open_nofollow`, when we have got the\nfile descriptor, no matter what happens to `packed-refs` file (deleted or\nchanged into a symlink), we could operate the file descriptor and read\nits content.\n\nHowever, on a platform with O_NOFOLLOW, this situation will also happen.\nSo, I think we may just use \"open_nofollow\" now and don't talk about the\nmethod one at all to avoid confusing readers.\n\n> As long as both approaches are equally portable, I do not think it\n> matters which one we pick from correctness point of view, and we can\n> pick the one that is easier to use to implement the feature.\n> \n> On a platform without O_NOFOLLOW, open_nofollow() falls back to the\n> lstat and open, so your \"open_nofollow() is better than lstat() and\n> open()\" argument does not portably work, though.\n> \n\nYes, actually in my first implementation, I didn't notice this. But the\nCI told me that and I finally chose \"open_nofollow\".\n\n> > Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n> > the user if \"packed-refs\" is not a regular file.\n> \n> Good. Say \"regular file\" on the commit title, too, and it would be\n> perfect.\n> \n\nLet me improve this in the next version.\n\n> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> > index cf7a202d0d..42c8d4ca1e 100755\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -617,4 +617,26 @@ test_expect_success 'ref content checks should work with worktrees' '\n> > \t)\n> > '\n> > \n> > +test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\t(\n> > +\t\tcd repo &&\n> > +\t\ttest_commit default &&\n> > +\t\tgit branch branch-1 &&\n> > +\t\tgit branch branch-2 &&\n> > +\t\tgit branch branch-3 &&\n> > +\t\tgit pack-refs --all &&\n> > +\n> > +\t\tmv .git/packed-refs .git/packed-refs-back &&\n> > +\t\tln -sf packed-refs-bak .git/packed-refs &&\n> > +\t\ttest_must_fail git refs verify 2>err &&\n> > +\t\tcat >expect <<-EOF &&\n> > +\t\terror: packed-refs: badRefFiletype: not a regular file\n> > +\t\tEOF\n> > +\t\trm .git/packed-refs &&\n> > +\t\ttest_cmp expect err\n> > +\t)\n> > +'\n> > +\n> > test_done\n> \n> OK. I notice that the previous step did not have any new test\n> associated with it. Perhaps we can corrupt \"HEAD\" *and* replace\n> packed-refs file with a symbolic link (or do some other damage\n> to the refs) and make sure both breakages are reported?\n> \n\nAs I have said in the previous comment, we cannot detect the error if\n\"HEAD\" itself is corrupted. However, we will check the referent in the\nlater. So, we don't need to do this.\n\n> It does not have to be done in this step, and certainly not as a\n> part of this single test this step adds, but we'd want it tested\n> somewhere.\n> \n\nIf we need to check the referent of the \"HEAD\" in the \"packed-refs\". We\ncould do this in the later test. I could cover this in [PATCH 6/8].\n\nThanks,\nJialuo\n"},{"id":"511559","messageId":"Z5zc_QAqYP-Dg4-K@ArchLinux","threadId":"62743","inReplyTo":"xmqq1pwkdt7r.fsf@gitster.g","subject":"Re: [PATCH v2 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-31T14:23:57Z","receivedAt":"2025-01-31T14:22:24Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 30, 2025 at 10:58:32AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > In \"packed-backend.c::create_snapshot\", if there is a header (the line\n> > which starts with '#'), we will check whether the line starts with \"#\n> > pack-refs with:\". As we are going to implement the header consistency\n> > check, we should port this check into \"packed_fsck\".\n> >\n> > However, the above check is not enough, this is because \"git pack-refs\"\n> > will always write \"PACKED_REFS_HEADER\" which is a constant string to the\n> > \"packed-refs\" file. So, we should check the following things for the\n> > header.\n> \n> I haven't done history digging in this area for a while, but we\n> should make sure we are not flagging a file that was written in\n> ancient version of Git whose repository is still supported.\n> \n\nUnderstood.\n\n> > 1. If the header does not exist, we may report an error to the user\n> > because it should exist, but we do allow no header in \"packed-refs\"\n> > file. So, create a new fsck message \"packedRefMissingHeader(INFO)\" to\n> > warn the user and also keep compatibility.\n> \n> Are we sure \"it should exist\"? I think the header did not exist\n> before \"Git v1.5.0\". I didn't check with other reimplementations of\n> Git (like jgit or libgit2), but as long as our reading side of the\n> runtime allows a packed-refs file without the header without\n> complaint, I do not think it is a good idea to treat it as a\n> report-worthy event from \"git fsck\".\n> \n\nOK, let me improve this in the next version.\n\n> > 2. If the header content does not start with \"# packed-ref with:\", we\n> > should report an error just like what \"create_snapshot\" does. So,\n> > create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n> \n> This I can agree with. If the first line begins with \"#\" but not\n> with that string (with a trailing SP), that is a sign that it may\n> not even be a valid packed-refs file, which is a report-worthy\n> event.\n> \n> > 3. If the header content is not the same as the constant string\n> > \"PACKED_REFS_HEADER\", ideally, we should report an error to the user.\n> \n> NO. THAT IS NOT IDEAL AT ALL.\n> \n> The header was written like this:\n> \n> /* perhaps other traits later as well */\n> fprintf(cbdata.refs_file, \"# pack-refs with: peeled \\n\");\n> \n> in the older versions of Git before it was made into a separate\n> preprocessor macro and lost the comment (the above excerpt is from\n> \"git show v1.5.0:builtin-pack-refs.c\").\n> \n> Notice \"other traits later\" in the comment?\n> \n> The thing is _designed_ to be extensible. In fact, these days we\n> support a few more traits\n> \n> static const char PACKED_REFS_HEADER[] =\n> \"# pack-refs with: peeled fully-peeled sorted \\n\";\n> \n> (an excerpt from the current refs/packed-backend.c).\n> \n> Reporting an error when you see something written by an older\n> version of Git is far from ideal.\n> \n\nUnderstood, I think we should be consistency with the runtime check.\n\n> > However, we allow other contents as long as the header content starts\n> > with \"# packed-ref with:\". To keep compatibility, create a new fsck\n> > message \"unknownPackedRefHeader(INFO)\" to warn about this. We may\n> > tighten this rule in the future.\n> \n> Whatever we do, what we do with an unknown trait should be in line\n> with what the runtime does. If the runtime failed (we do not, but\n> this is to illustrate the principle [*]) on a packed-refs file\n> without \"sorted\" trait, noticing that \"sorted\" is not there and\n> flagging as an error is a good thing to do. But if the runtime\n> gracefully degrades and sorts the list of refs read from such a\n> packed-refs file before continuing, then a packed-refs file that\n> lack \"sorted\" trait is not a report-worthy event.\n> \n\nActually, the runtime won't complain about this. I agree with you here.\n\n> I do not offhand recall if we introduced the concept of mandatory vs\n> optional traits in the packed-refs part of the system (like we have\n> in the index extension subsystem, where a version of Git that\n> encounters an unknown *and* mandatory index extension must refuse to\n> touch the repository), but if there is a mandatory trait declared in\n> the header that our version of Git does not understand, it is a\n> report-worthy event that must be flagged with \"git refs verify\".\n> \n\nI don't think any trait in \"packed-refs\" is mandatory. Because I have\ndone some experiments before implementing the code. We should only check\ncase 2 here.\n\n> > +static int packed_fsck_ref_header(struct fsck_options *o, const char *start, const char *eol)\n> > +{\n> > +\tconst char *err_fmt = NULL;\n> > +\tint fsck_msg_id = -1;\n> > +\n> > +\tif (!starts_with(start, \"# pack-refs with:\")) {\n> > +\t\terr_fmt = \"'%.*s' does not start with '# pack-refs with:'\";\n> > +\t\tfsck_msg_id = FSCK_MSG_BAD_PACKED_REF_HEADER;\n> > +\t} else if (strncmp(start, PACKED_REFS_HEADER, strlen(PACKED_REFS_HEADER))) {\n> > +\t\terr_fmt = \"'%.*s' is an unknown packed-refs header\";\n> > +\t\tfsck_msg_id = FSCK_MSG_UNKNOWN_PACKED_REF_HEADER;\n> > +\t}\n> \n> As I outlined above, this is totally unacceptable. \n> \n> Inspecting the header is good, but if this code claims to be a\n> checker, it should do at least what the runtime does, i.e. parse the\n> header to tell what traits the packed-file declares, not just\n> assuming that it is a fixed string. And error on unknown trait(s)\n> if they are mandatory (if such a concept is implemented in the\n> runtime reading side). Informing on an unknown and optional\n> trait(s) I can live with, but personally I wouldn't recommend it.\n> \n\nGot it, I don't want to report unknown trait(s) either.\n\n> In other words, report loudly if it is an error, but otherwise stay\n> silent if we know we tolerate it well. \n> \n\nThanks for this suggestion.\n\n> > +static int packed_fsck_ref_content(struct fsck_options *o,\n> > +\t\t\t\t const char *start, const char *eof)\n> > +{\n> > +\tstruct strbuf packed_entry = STRBUF_INIT;\n> > +\tint line_number = 1;\n> \n> We limit ourselves with about 1 billion refs in the packed-refs\n> file, which may be plenty,\n\nLet me change this to `size_t`. This would be better.\n\n> but I do not quite understand the use of\n> this variable. There is no loop inside this so ...\n> \n\nThe reason why I define this variable is that I am going to use loop to\ncheck each entry in the next patch.\n\n> > +\tconst char *eol;\n> > +\tint ret = 0;\n> > +\n> > +\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n> \n> ... this is always line #1, and then\n> \n> > +\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n> > +\tif (*start == '#') {\n> > +\t\tret |= packed_fsck_ref_header(o, start, eol);\n> > +\n> > +\t\tstart = eol + 1;\n> > +\t\tline_number++;\n> \n> ... it may be incremented, but upon returning from the funcition, it\n> is lost.\n> \n> Perhaps you wanted to make it a function-scope static, but then you\n> are allowed to read one single packed-refs file during the life of\n> your process before you exit, which I am not sure is what you want?\n> \n\nActually, what I want is use this variable for looping the each ref\nentry in the \"packed-refs\" file.\n\n> > +\t} else {\n> > +\t\tstruct fsck_ref_report report = { 0 };\n> > +\t\treport.path = \"packed-refs\";\n> > +\n> > +\t\tret |= fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_PACKED_REF_MISSING_HEADER,\n> > +\t\t\t\t \"missing header line\");\n> > +\t}\n> > +\n> > +\tstrbuf_release(&packed_entry);\n> > +\treturn ret;\n> > +}\n\nThanks,\nJialuo\n"},{"id":"511562","messageId":"Z5zfx0E2neO5MNKs@ArchLinux","threadId":"62743","inReplyTo":"xmqqwmecceh1.fsf@gitster.g","subject":"Re: [PATCH v2 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-31T14:35:51Z","receivedAt":"2025-01-31T14:34:17Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 30, 2025 at 11:02:18AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > We will always try to sort the \"packed-refs\" increasingly by comparing\n> > the refname. So, we should add checks to verify whether the \"packed-refs\"\n> > is sorted.\n> \n> Do this _ONLY_ when the packed-refs file has a header that declares\n> \"sorted\" trait. Insisting on a packed-refs file that does not would\n> mean you are stricter than the runtime contract allows.\n> \n\nFrom my perspective, we should check whether it is sorted when the\nheader has a \"sorted\" trait. Actually, in the runtime, when calling\n`create_snapshot` method, the following would happen:\n\n1. If there is no \"sorted\" trait, it will sort the \"packed-refs\".\n2. If there is, it won't sort the \"packed-refs\".\n\nSo, we DO allow refs unsorted.\n\nActually, I have used `git show v1.5.0:builtin-pack-refs.c`, in this\nversion, it does not sort the ref. However, I quite don't understand the\ncomment from Patrick in the version one about this patch:\n\n> Makes sense. It has been a source of bugs a couple years ago, and it can\n> silently make you receive wrong results, so this is quite a sensible\n> check to have.\n\nPatrick, could you please help to explain this. I don't know whether we\nneed to check whether \"packed-refs\" is sorted always. It seems that we\ntruly allow refs unsorted. We need to know whether we should tighten\nthis?\n\n> > +struct fsck_packed_ref_entry {\n> > +\tint line_number;\n> > +\n> > +\tstruct snapshot_record record;\n> > +};\n> \n> Not a huge deal, as 1 billion is still plenty of a large number, but\n> the same comment on the line-number applies here. We might want to\n> consistently use ulong for line numbers of files we read from.\n\nYes, let me improve this.\n\nThanks,\nJialuo\n"},{"id":"511564","messageId":"Z5zgIp7k0Z9kscmt@ArchLinux","threadId":"62743","inReplyTo":"xmqqsep0ceec.fsf@gitster.g","subject":"Re: [PATCH v2 8/8] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-01-31T14:37:22Z","receivedAt":"2025-01-31T14:35:50Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 30, 2025 at 11:03:55AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > +static void fsck_refs(struct repository *r)\n> > +{\n> > +\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n> > +\tstruct progress *progress = NULL;\n> > +\tuint64_t progress_num = 1;\n> > +\n> > +\tif (show_progress)\n> > +\t\tprogress = start_progress(r, _(\"Checking ref database\"),\n> > +\t\t\t\t\t progress_num);\n> \n> I do not see why we need an extra variable progress_num here. Just\n> passing a literal constant 1 should be sufficient. The called\n> function has function prototype to help the compiler promite it to\n> the appropritate type.\n\nYou are correct, let me improve this in the next version.\n\nThanks,\nJialuo\n"},{"id":"511575","messageId":"xmqqplk39cwl.fsf@gitster.g","threadId":"62743","inReplyTo":"Z5zQMCRETczMQMxj@ArchLinux","subject":"Re: [PATCH v2 2/8] builtin/refs: get worktrees without reading head info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-31T16:16:42Z","receivedAt":"2025-01-31T16:16:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> I want to say that we cannot check the content of the \"HEAD\" itself. If\n> the content of \"HEAD\" is not correct, we cannot detect the current\n> directory as a valid git repository. So, there is no need to say \"we\n> will check the contents of 'HEAD' else where\".\n\nInstead you should say \"we detected your HEAD is broken\" somewhere\nin the documentation for this, and then the end-user should get a\nmessage to telling them about the broken HEAD in such a case,\nthough.\n\n"},{"id":"511576","messageId":"xmqqikpv9cq3.fsf@gitster.g","threadId":"62743","inReplyTo":"Z5zWE1M4u3NrROI-@ArchLinux","subject":"Re: [PATCH v2 3/8] packed-backend: check whether the \"packed-refs\" is regular","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-31T16:20:36Z","receivedAt":"2025-01-31T16:20:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> However, on a platform with O_NOFOLLOW, this situation will also happen.\n> So, I think we may just use \"open_nofollow\" now and don't talk about the\n> method one at all to avoid confusing readers.\n\nExactly. That is what you see below ;-)\n\n>> As long as both approaches are equally portable, I do not think it\n>> matters which one we pick from correctness point of view, and we can\n>> pick the one that is easier to use to implement the feature.\n>> \n>> On a platform without O_NOFOLLOW, open_nofollow() falls back to the\n>> lstat and open, so your \"open_nofollow() is better than lstat() and\n>> open()\" argument does not portably work, though.\n>> ...\n>> OK. I notice that the previous step did not have any new test\n>> associated with it. Perhaps we can corrupt \"HEAD\" *and* replace\n>> packed-refs file with a symbolic link (or do some other damage\n>> to the refs) and make sure both breakages are reported?\n>\n> As I have said in the previous comment, we cannot detect the error if\n> \"HEAD\" itself is corrupted. However, we will check the referent in the\n> later. So, we don't need to do this.\n\nI still think you absolutely need to diagnose and tell the user\nabout the broken HEAD. With your \"don't check HEAD because a\nrepository with a broken HEAD is not a repository\", a check run in\nsuch a place may find everything else in the repository perfectly\nfine, but because the user wanted \"git refs verify\" to tell them\nabout breakages, you would want to somehow tell them about it.\nEither it is missing, malformed, whatever.\n"},{"id":"511577","messageId":"xmqqed0j9clh.fsf@gitster.g","threadId":"62743","inReplyTo":"Z5zfx0E2neO5MNKs@ArchLinux","subject":"Re: [PATCH v2 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-01-31T16:23:22Z","receivedAt":"2025-01-31T16:23:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> On Thu, Jan 30, 2025 at 11:02:18AM -0800, Junio C Hamano wrote:\n>> shejialuo writes:\n>> \n>> > We will always try to sort the \"packed-refs\" increasingly by comparing\n>> > the refname. So, we should add checks to verify whether the \"packed-refs\"\n>> > is sorted.\n>> \n>> Do this _ONLY_ when the packed-refs file has a header that declares\n>> \"sorted\" trait. Insisting on a packed-refs file that does not would\n>> mean you are stricter than the runtime contract allows.\n>> \n>\n> From my perspective, we should check whether it is sorted when the\n> header has a \"sorted\" trait.\n\nSo the three-lines you wrote is not accurate, then. That is why I\nsaid that \"should add checks\" should not be unconditional---we\nshould not check if the file contents is sorted when \"sorted\" trait\nis not declared.\n"},{"id":"511628","messageId":"Z53triurDqskbRaA@ArchLinux","threadId":"62743","inReplyTo":"xmqqikpv9cq3.fsf@gitster.g","subject":"Re: [PATCH v2 3/8] packed-backend: check whether the \"packed-refs\" is regular","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-01T09:47:26Z","receivedAt":"2025-02-01T09:45:51Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Jan 31, 2025 at 08:20:36AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> >\n> > As I have said in the previous comment, we cannot detect the error if\n> > \"HEAD\" itself is corrupted. However, we will check the referent in the\n> > later. So, we don't need to do this.\n> \n> I still think you absolutely need to diagnose and tell the user\n> about the broken HEAD. With your \"don't check HEAD because a\n> repository with a broken HEAD is not a repository\", a check run in\n> such a place may find everything else in the repository perfectly\n> fine, but because the user wanted \"git refs verify\" to tell them\n> about breakages, you would want to somehow tell them about it.\n> Either it is missing, malformed, whatever.\n\nYes, that's absolutely correct. However, I don't want to do this in\nthis series. Actually, there is no check for root ref. I will add checks\nfor root refs later.\n\nThanks,\nJialuo\n"},{"id":"511629","messageId":"Z53uT8BnepXEjv8v@ArchLinux","threadId":"62743","inReplyTo":"xmqqed0j9clh.fsf@gitster.g","subject":"Re: [PATCH v2 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-01T09:50:07Z","receivedAt":"2025-02-01T09:48:32Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Jan 31, 2025 at 08:23:22AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > On Thu, Jan 30, 2025 at 11:02:18AM -0800, Junio C Hamano wrote:\n> >> shejialuo writes:\n> >> \n> >> > We will always try to sort the \"packed-refs\" increasingly by comparing\n> >> > the refname. So, we should add checks to verify whether the \"packed-refs\"\n> >> > is sorted.\n> >> \n> >> Do this _ONLY_ when the packed-refs file has a header that declares\n> >> \"sorted\" trait. Insisting on a packed-refs file that does not would\n> >> mean you are stricter than the runtime contract allows.\n> >> \n> >\n> > From my perspective, we should check whether it is sorted when the\n> > header has a \"sorted\" trait.\n> \n> So the three-lines you wrote is not accurate, then. That is why I\n> said that \"should add checks\" should not be unconditional---we\n> should not check if the file contents is sorted when \"sorted\" trait\n> is not declared.\n\nI have made confusion here. Sorry. Let me improve this in the next\nversion.\n\nThanks,\nJialuo\n\n"},{"id":"511714","messageId":"Z6CA7ZIBuY_YBV1Y@pks.im","threadId":"62743","inReplyTo":"Z5r6-52eBgT4TUYG@ArchLinux","subject":"Re: [PATCH v2 3/8] packed-backend: check whether the \"packed-refs\" is regular","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-03T08:40:13Z","receivedAt":"2025-02-03T08:40:23Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Jan 30, 2025 at 12:07:23PM +0800, shejialuo wrote:\n> Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\n> consistency and correctness of \"packed-refs\" file, they never check the\n> filetype of the \"packed-refs\". The user should always use \"git\n> packed-refs\" command to create the raw regular \"packed-refs\" file, so we\n\nIt's `git pack-refs`, not `git packed-refs`.\n\nOtherwise I'm not going to comment on the rest of the commit, as Junio\nhas already sufficiently discussed it with you, and I very much agree\nwith his assessment that we don't need to discuss whether or not to use\n`open_nofollow()` in this depth.\n\nPatrick\n"},{"id":"511715","messageId":"Z6CA9qCLwMnOsKxR@pks.im","threadId":"62743","inReplyTo":"Z5r7EkDwEsxuLJzn@ArchLinux","subject":"Re: [PATCH v2 5/8] packed-backend: check whether the refname contains NUL characters","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-03T08:40:22Z","receivedAt":"2025-02-03T08:40:25Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Jan 30, 2025 at 12:07:46PM +0800, shejialuo wrote:\n> We have already implemented the header consistency check for the raw\n> \"packed-refs\" file. Before we implement the consistency check for each\n> ref entry, let's analysis [1] which reports that \"git fsck\" cannot\n> detect some NUL characters.\n\nThis paragraph doesn't quite parse. I think it can simply be left out,\nas the remainder of the commit message already explains in more than\nenough detail what you're doing.\n\n> \"packed-backend.c::next_record\" will use \"check_refname_format\" to check\n> the consistency of the refname. If it is not OK, the program will die.\n> So, we already have the code path and we must miss out something.\n> \n> We use the following code to get the refname:\n> \n> strbuf_add(&iter->refname_buf, p, eol - p);\n> iter->base.refname = iter->refname_buf.buf\n> \n> In the above code, `p` is the start pointer of the refname and `eol` is\n> the next newline pointer. We calculate the length of the refname by\n> subtracting the two pointers. Then we add the memory range between `p`\n> and `eol` to get the refname.\n> \n> However, if there are some NUL characters in the memory range between `p`\n> and `eol`, we will see the refname as a valid ref name as long as the\n> memory range between `p` and first occurred NUL character is valid.\n> \n> In order to catch above corruption, create a new function\n> \"refname_contains_nul\" by searching the first NUL character. If it is\n> not at the end of the string, there must be some NUL characters in the\n> refname.\n> \n> Use this function in \"next_record\" function to die the program if\n> \"refname_contains_nul\" returns true.\n\nYeah, makes sense to me. NUL bytes are invalid, and nothing good can\ncome out of it.\n\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 883189f3a1..870c8e7aaa 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -494,6 +494,22 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n> \t\t\t\t last_line, eof - last_line);\n> }\n> \n> +/*\n> + * When parsing the \"packed-refs\" file, we will parse it line by line.\n> + * Because we know the start pointer of the refname and the next\n> + * newline pointer, we could calculate the length of the refname by\n> + * subtracting the two pointers. However, there is a corner case where\n> + * the refname contains corrupted embedded NUL characters. And\n> + * `check_refname_format()` will not catch this when the truncated\n> + * refname is still a valid refname. To prevent this, we need to check\n> + * whether the refname contains the NUL characters.\n> + */\n> +static int refname_contains_nul(struct strbuf *refname)\n> +{\n> +\tconst char *pos = memchr(refname->buf, '\\0', refname->len + 1);\n> +\treturn pos < refname->buf + refname->len;\n> +}\n\nThis can be simplified to:\n\n return !!memchr(refname->buf, '\\0', refname->len);\n\nIdeally, we'd be amending `check_refname_format()` to do the checking\nfor us. But we can't without a wider refactoring because that function\ngets a C string, and C strings are naturally terminadet by NUL\ncharacters.\n\nI think that adding a new function for this is a bit over the top\nthough, as the check is unlikely to be useful in a lot of places and the\nlogic is rather trivial. So I'd just inline the check into\n`next_record()`.\n\nPatrick\n"},{"id":"511716","messageId":"Z6CA-aj7cgAxLTUI@pks.im","threadId":"62743","inReplyTo":"Z5r7Hlk_VS0jYU74@ArchLinux","subject":"Re: [PATCH v2 6/8] packed-backend: add \"packed-refs\" entry consistency check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-03T08:40:25Z","receivedAt":"2025-02-03T08:40:29Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Jan 30, 2025 at 12:07:58PM +0800, shejialuo wrote:\n> \"packed-backend.c::next_record\" will parse the ref entry to check the\n> consistency. This function has already checked the following things:\n> \n> 1. Parse the main line of the ref entry, if the oid is not correct. It\n> will die the program. And then it will check whether the next\n> character of the oid is space. Then it will check whether the refname\n> is correct.\n> 2. If the next line starts with '^', it will continue to parse the oid\n> of the peeled oid content and check whether the last character is\n> '\\n'.\n> \n> We can iterate each line by using the \"packed_fsck_ref_next_line\"\n> function. Then, create a new fsck message \"badPackedRefEntry(ERROR)\" to\n> report to the user when something is wrong.\n> \n> Create two new functions \"packed_fsck_ref_main_line\" and\n> \"packed_fsck_ref_peeled_line\" for case 1 and case 2 respectively. Last,\n> update the unit test to exercise the code.\n\nI think this message is going into too much detail about _how_ you are\ndoing things compared to _what_ you are doing and what the intent is.\n\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 870c8e7aaa..271c740728 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1819,10 +1819,86 @@ static int packed_fsck_ref_header(struct fsck_options *o, const char *start, con\n> \treturn 0;\n> }\n> \n> +static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n> +\t\t\t\t struct ref_store *ref_store,\n> +\t\t\t\t struct strbuf *packed_entry,\n> +\t\t\t\t const char *start, const char *eol)\n> +{\n> +\tstruct fsck_ref_report report = { 0 };\n> +\tstruct object_id peeled;\n> +\tconst char *p;\n> +\n> +\treport.path = packed_entry->buf;\n> +\n> +\tstart++;\n\nIt's a bit weird that we increment `start` here, as it is very intimate\nwith how the caller calls us. Might be easier to reason about when the\ncaller did this for us.\n\n> +\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n> +\t\treturn fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n> +\t\t\t\t \"'%.*s' has invalid peeled oid\",\n> +\t\t\t\t (int)(eol - start), start);\n> +\t}\n\nAll the braces around those single-line return statements can go away.\n\n> @@ -1843,6 +1919,26 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n> \t\t\t\t \"missing header line\");\n> \t}\n> \n> +\twhile (start < eof) {\n> +\t\tstrbuf_reset(&packed_entry);\n> +\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n> +\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n> +\t\tret |= packed_fsck_ref_main_line(o, ref_store, &packed_entry, &refname, start, eol);\n\nDon't we have to stop in case `next_line()` returns an error?\n\nPatrick\n"},{"id":"511717","messageId":"Z6CA_aKlWHLwHghA@pks.im","threadId":"62743","inReplyTo":"Z5r7KvL1bvSO4UQY@ArchLinux","subject":"Re: [PATCH v2 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-03T08:40:29Z","receivedAt":"2025-02-03T08:40:32Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Jan 30, 2025 at 12:08:10PM +0800, shejialuo wrote:\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 271c740728..b250f987b2 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1768,6 +1774,28 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> \treturn empty_ref_iterator_begin();\n> }\n> \n> +struct fsck_packed_ref_entry {\n> +\tint line_number;\n\nThis should rather be a `size_t`, or at least `unsigned`.\n\n> +\n> +\tstruct snapshot_record record;\n> +};\n> +\n> +static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(int line_number,\n> +\t\t\t\t\t\t\t\t const char *start)\n> +{\n> +\tstruct fsck_packed_ref_entry *entry = xcalloc(1, sizeof(*entry));\n> +\tentry->line_number = line_number;\n> +\tentry->record.start = start;\n> +\treturn entry;\n> +}\n> +\n> +static void free_fsck_packed_ref_entries(struct fsck_packed_ref_entry **entries, int nr)\n> +{\n> +\tfor (int i = 0; i < nr; i++)\n\nLet's use `size_t` for both `i` and `nr`.\n\n> +\t\tfree(entries[i]);\n> +\tfree(entries);\n> +}\n> +\n> static int packed_fsck_ref_next_line(struct fsck_options *o,\n> \t\t\t\t struct strbuf *packed_entry, const char *start,\n> \t\t\t\t const char *eof, const char **eol)\n> @@ -1893,13 +1921,60 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n> \treturn 0;\n> }\n> \n> +static int packed_fsck_ref_sorted(struct fsck_options *o,\n> +\t\t\t\t struct ref_store *ref_store,\n> +\t\t\t\t struct fsck_packed_ref_entry **entries,\n> +\t\t\t\t int nr)\n> +{\n> +\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n> +\tstruct strbuf packed_entry = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = { 0 };\n> +\tstruct strbuf refname1 = STRBUF_INIT;\n> +\tstruct strbuf refname2 = STRBUF_INIT;\n> +\tint ret = 0;\n> +\n> +\tfor (int i = 1; i < nr; i++) {\n\nHere, as well.\n\nPatrick\n"},{"id":"511718","messageId":"Z6CBBn6EW2_MXKOK@pks.im","threadId":"62743","inReplyTo":"Z5zfx0E2neO5MNKs@ArchLinux","subject":"Re: [PATCH v2 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-03T08:40:38Z","receivedAt":"2025-02-03T08:40:42Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Jan 31, 2025 at 10:35:51PM +0800, shejialuo wrote:\n> On Thu, Jan 30, 2025 at 11:02:18AM -0800, Junio C Hamano wrote:\n> > shejialuo writes:\n> > Makes sense. It has been a source of bugs a couple years ago, and it can\n> > silently make you receive wrong results, so this is quite a sensible\n> > check to have.\n> \n> Patrick, could you please help to explain this. I don't know whether we\n> need to check whether \"packed-refs\" is sorted always. It seems that we\n> truly allow refs unsorted. We need to know whether we should tighten\n> this?\n\nThe context here is that packed-refs sometimes claim that they are\nsorted, but indeed they aren't. There are two sources for this that I've\nseen in the wild:\n\n - An invalid comparison function. I think I remember that libgit2 at\n one point sorted them incorrectly, but not a 100% sure anymore where\n I've seen this.\n\n - A user manually edits the packed-refs file, but isn't aware of the\n sorting.\n\nSo we should assert that a packed-refs file is correctly sorted, but\nonly when the header claims that it should be sorted.\n\nPatrick\n"},{"id":"511719","messageId":"Z6CBC5kyvIhBuLk6@pks.im","threadId":"62743","inReplyTo":"Z5r7NnzvirWEljwV@ArchLinux","subject":"Re: [PATCH v2 8/8] builtin/fsck: add `git refs verify` child process","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-03T08:40:43Z","receivedAt":"2025-02-03T08:40:46Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Jan 30, 2025 at 12:08:22PM +0800, shejialuo wrote:\n> diff --git a/builtin/fsck.c b/builtin/fsck.c\n> index 7a4dcb0716..9a8613d07f 100644\n> --- a/builtin/fsck.c\n> +++ b/builtin/fsck.c\n> @@ -905,6 +905,34 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n> \treturn res;\n> }\n> \n> +static void fsck_refs(struct repository *r)\n> +{\n> +\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n> +\tstruct progress *progress = NULL;\n> +\tuint64_t progress_num = 1;\n> +\n> +\tif (show_progress)\n> +\t\tprogress = start_progress(r, _(\"Checking ref database\"),\n> +\t\t\t\t\t progress_num);\n\nHm. I don't really think that this progress meter adds anything right\nnow. It only shows either 0 or 1, so it basically only tells you when\nyou're done. And that is something that the user can tell without a\nprogress meter.\n\n> +\n> +\tif (verbose)\n> +\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n> +\n> +\tchild_process_init(&refs_verify);\n> +\trefs_verify.git_cmd = 1;\n> +\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n> +\tif (verbose)\n> +\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n> +\tif (check_strict)\n> +\t\tstrvec_push(&refs_verify.args, \"--strict\");\n> +\n> +\tif (run_command(&refs_verify))\n> +\t\terrors_found |= ERROR_REFS;\n> +\n> +\tdisplay_progress(progress, 1);\n> +\tstop_progress(&progress);\n> +}\n> +\n> static char const * const fsck_usage[] = {\n> \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n> \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n> @@ -970,6 +998,8 @@ int cmd_fsck(int argc,\n> \tgit_config(git_fsck_config, &fsck_obj_options);\n> \tprepare_repo_settings(the_repository);\n> \n> +\tfsck_refs(the_repository);\n\nI think there needs to be a way to disable this. How about we add an\noption `--[no-]references` to do so? I was briefly wondering whether we\nalso want to have `--only-references`, but if a user wants to do that\nthey can simply execute `git refs verify` directly.\n\nPatrick\n"},{"id":"511758","messageId":"xmqqseou239w.fsf@gitster.g","threadId":"62743","inReplyTo":"Z53triurDqskbRaA@ArchLinux","subject":"Re: [PATCH v2 3/8] packed-backend: check whether the \"packed-refs\" is regular","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-03T20:15:39Z","receivedAt":"2025-02-03T20:15:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> On Fri, Jan 31, 2025 at 08:20:36AM -0800, Junio C Hamano wrote:\n>> shejialuo writes:\n>> \n>> >\n>> > As I have said in the previous comment, we cannot detect the error if\n>> > \"HEAD\" itself is corrupted. However, we will check the referent in the\n>> > later. So, we don't need to do this.\n>> \n>> I still think you absolutely need to diagnose and tell the user\n>> about the broken HEAD. With your \"don't check HEAD because a\n>> repository with a broken HEAD is not a repository\", a check run in\n>> such a place may find everything else in the repository perfectly\n>> fine, but because the user wanted \"git refs verify\" to tell them\n>> about breakages, you would want to somehow tell them about it.\n>> Either it is missing, malformed, whatever.\n>\n> Yes, that's absolutely correct. However, I don't want to do this in\n> this series. Actually, there is no check for root ref. I will add checks\n> for root refs later.\n\nAnother thing I just thought of is that what is your plans for\nrepository discovery when HEAD is iffy. In the working tree of our\nproject, you go to a subdirectory, say \"t/\", and then corrupt the\nHEAD, would \"git refs verify\" still recognise that ../.git/ is the\n\"repository\" the user is interested in, but it has a broken HEAD?\n\nsetup.c:is_git_directory() would say \"no\", so I am not sure the\ndiscovery would work without changing that, and I am not sure if it\nis worth doing (i.e. when the user knows the repository's HEAD is\nbroken, it is OK to disable discovery and force them to say\nGIT_DIR=/this/directory).\n\n"},{"id":"511783","messageId":"Z6GQTLxLVtTG6FkY@ArchLinux","threadId":"62743","inReplyTo":"xmqqseou239w.fsf@gitster.g","subject":"Re: [PATCH v2 3/8] packed-backend: check whether the \"packed-refs\" is regular","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-04T03:58:04Z","receivedAt":"2025-02-04T03:56:27Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Feb 03, 2025 at 12:15:39PM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > On Fri, Jan 31, 2025 at 08:20:36AM -0800, Junio C Hamano wrote:\n> >> shejialuo writes:\n> >> \n> >> >\n> >> > As I have said in the previous comment, we cannot detect the error if\n> >> > \"HEAD\" itself is corrupted. However, we will check the referent in the\n> >> > later. So, we don't need to do this.\n> >> \n> >> I still think you absolutely need to diagnose and tell the user\n> >> about the broken HEAD. With your \"don't check HEAD because a\n> >> repository with a broken HEAD is not a repository\", a check run in\n> >> such a place may find everything else in the repository perfectly\n> >> fine, but because the user wanted \"git refs verify\" to tell them\n> >> about breakages, you would want to somehow tell them about it.\n> >> Either it is missing, malformed, whatever.\n> >\n> > Yes, that's absolutely correct. However, I don't want to do this in\n> > this series. Actually, there is no check for root ref. I will add checks\n> > for root refs later.\n> \n> Another thing I just thought of is that what is your plans for\n> repository discovery when HEAD is iffy. In the working tree of our\n> project, you go to a subdirectory, say \"t/\", and then corrupt the\n> HEAD, would \"git refs verify\" still recognise that ../.git/ is the\n> \"repository\" the user is interested in, but it has a broken HEAD?\n> \n> setup.c:is_git_directory() would say \"no\", so I am not sure the\n> discovery would work without changing that, and I am not sure if it\n> is worth doing (i.e. when the user knows the repository's HEAD is\n> broken, it is OK to disable discovery and force them to say\n> GIT_DIR=/this/directory).\n\nI have to say I am not so familiar with the \"setup.c\" code. Thanks for\nthe direction here, I will dive into to figure out a solution.\n\nThanks,\nJialuo\n"},{"id":"511791","messageId":"Z6GXUisyfjvp1Dpn@ArchLinux","threadId":"62743","inReplyTo":"Z6CA-aj7cgAxLTUI@pks.im","subject":"Re: [PATCH v2 6/8] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-04T04:28:02Z","receivedAt":"2025-02-04T04:26:25Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Feb 03, 2025 at 09:40:25AM +0100, Patrick Steinhardt wrote:\n> On Thu, Jan 30, 2025 at 12:07:58PM +0800, shejialuo wrote:\n> > \"packed-backend.c::next_record\" will parse the ref entry to check the\n> > consistency. This function has already checked the following things:\n> > \n> > 1. Parse the main line of the ref entry, if the oid is not correct. It\n> > will die the program. And then it will check whether the next\n> > character of the oid is space. Then it will check whether the refname\n> > is correct.\n> > 2. If the next line starts with '^', it will continue to parse the oid\n> > of the peeled oid content and check whether the last character is\n> > '\\n'.\n> > \n> > We can iterate each line by using the \"packed_fsck_ref_next_line\"\n> > function. Then, create a new fsck message \"badPackedRefEntry(ERROR)\" to\n> > report to the user when something is wrong.\n> > \n> > Create two new functions \"packed_fsck_ref_main_line\" and\n> > \"packed_fsck_ref_peeled_line\" for case 1 and case 2 respectively. Last,\n> > update the unit test to exercise the code.\n> \n> I think this message is going into too much detail about _how_ you are\n> doing things compared to _what_ you are doing and what the intent is.\n> \n\nI think I have caused some confusion here. The reason why I mention what\n\"next_record\" does is that I want to port these two checks. Let me\nimprove this in the next version. I will highlight more about the\nmotivation.\n\n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index 870c8e7aaa..271c740728 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -1819,10 +1819,86 @@ static int packed_fsck_ref_header(struct fsck_options *o, const char *start, con\n> > \treturn 0;\n> > }\n> > \n> > +static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n> > +\t\t\t\t struct ref_store *ref_store,\n> > +\t\t\t\t struct strbuf *packed_entry,\n> > +\t\t\t\t const char *start, const char *eol)\n> > +{\n> > +\tstruct fsck_ref_report report = { 0 };\n> > +\tstruct object_id peeled;\n> > +\tconst char *p;\n> > +\n> > +\treport.path = packed_entry->buf;\n> > +\n> > +\tstart++;\n> \n> It's a bit weird that we increment `start` here, as it is very intimate\n> with how the caller calls us. Might be easier to reason about when the\n> caller did this for us.\n> \n\nFor each ref entry, we have two pointers, one is the `start` which is\nused to indicate the start of the line and `eol` is the end of the line.\n\nLet's see how we call this function:\n\n\t\tif (start < eof && *start == '^') {\n\t\t\tstrbuf_reset(&packed_entry);\n\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n\t\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, &packed_entry,\n\t\t\t\t\t\t\t start, eol);\n\t\t\tstart = eol + 1;\n\t\t\tline_number++;\n\t\t}\n\nThe reason why we do this is that we need to skip the '^' character. I\ndon't do this in the `if` statement. This is because I want to make the\nsemantics of the `start` variable unchanged.\n\nI would add a comment here to explain why we need to execute \"start++\".\n\n> > +\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n> > +\t\treturn fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n> > +\t\t\t\t \"'%.*s' has invalid peeled oid\",\n> > +\t\t\t\t (int)(eol - start), start);\n> > +\t}\n> \n> All the braces around those single-line return statements can go away.\n> \n\nI see. So, I have misunderstanding here. I have thought that we should\nadd braces because we have split this single statement into multiple\nlines. Let me update this in the next version.\n\n> > @@ -1843,6 +1919,26 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n> > \t\t\t\t \"missing header line\");\n> > \t}\n> > \n> > +\twhile (start < eof) {\n> > +\t\tstrbuf_reset(&packed_entry);\n> > +\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n> > +\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n> > +\t\tret |= packed_fsck_ref_main_line(o, ref_store, &packed_entry, &refname, start, eol);\n> \n> Don't we have to stop in case `next_line()` returns an error?\n> \n\nNo, we don't have to stop. We will continue to check the last ref entry,\nthis is intentional, we still need to check the last ref entry even\nthough there is no newline. I don't think we should ignore this part.\n\nThanks,\nJialuo\n"},{"id":"511792","messageId":"Z6GmUSgkZF1rWQgP@ArchLinux","threadId":"62743","inReplyTo":"Z6CBC5kyvIhBuLk6@pks.im","subject":"Re: [PATCH v2 8/8] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-04T05:32:01Z","receivedAt":"2025-02-04T05:30:23Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Feb 03, 2025 at 09:40:43AM +0100, Patrick Steinhardt wrote:\n> On Thu, Jan 30, 2025 at 12:08:22PM +0800, shejialuo wrote:\n> > diff --git a/builtin/fsck.c b/builtin/fsck.c\n> > index 7a4dcb0716..9a8613d07f 100644\n> > --- a/builtin/fsck.c\n> > +++ b/builtin/fsck.c\n> > @@ -905,6 +905,34 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n> > \treturn res;\n> > }\n> > \n> > +static void fsck_refs(struct repository *r)\n> > +{\n> > +\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n> > +\tstruct progress *progress = NULL;\n> > +\tuint64_t progress_num = 1;\n> > +\n> > +\tif (show_progress)\n> > +\t\tprogress = start_progress(r, _(\"Checking ref database\"),\n> > +\t\t\t\t\t progress_num);\n> \n> Hm. I don't really think that this progress meter adds anything right\n> now. It only shows either 0 or 1, so it basically only tells you when\n> you're done. And that is something that the user can tell without a\n> progress meter.\n> \n\nYou are correct in the functionality part. Actually, my very initial\nimplementation is what you have said. I simply used the following way to\nindicate the user that we are going to check ref database.\n\n fprintf_ln(stderr, _(\"Checking ref database\"));\n\nHowever, it will break a test in \"t/t1050-large.sh::fsck large blobs\". I\ncite the shell script below:\n\n\ttest_expect_success 'fsck large blobs' '\n\t\tgit fsck 2>err &&\n\t\ttest_must_be_empty err\n\t'\n\n> > +\n> > +\tif (verbose)\n> > +\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n> > +\n\nThat's the reason why we need to use `verbose` to control the behavior\nhere. Put it futhermore, We either use `process` or `verbose` to print\nthe message to the user. This is a pattern widely used in \"git-fsck(1)\".\nFor example \"builtin/fsck.c::fsck_object_dir\", we have the following\ncode:\n\n\tif (verbose)\n\t\tfprintf_ln(stderr, _(\"Checking object directory\"));\n\n\tif (show_progress)\n\t\tprogress = start_progress(the_repository,\n\t\t\t\t\t _(\"Checking object directories\"), 256);\n\nSo, that's why I use progress here. We need this to print the\ninformation to the user. I have also tried to print to the stdout like\nthe following\n\n\tfprintf_ln(stdout, _(\"Checking ref database\"));\n\nIt will also break the test.\n\n> > +\tchild_process_init(&refs_verify);\n> > +\trefs_verify.git_cmd = 1;\n> > +\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n> > +\tif (verbose)\n> > +\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n> > +\tif (check_strict)\n> > +\t\tstrvec_push(&refs_verify.args, \"--strict\");\n> > +\n> > +\tif (run_command(&refs_verify))\n> > +\t\terrors_found |= ERROR_REFS;\n> > +\n> > +\tdisplay_progress(progress, 1);\n> > +\tstop_progress(&progress);\n> > +}\n> > +\n> > static char const * const fsck_usage[] = {\n> > \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n> > \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n> > @@ -970,6 +998,8 @@ int cmd_fsck(int argc,\n> > \tgit_config(git_fsck_config, &fsck_obj_options);\n> > \tprepare_repo_settings(the_repository);\n> > \n> > +\tfsck_refs(the_repository);\n> \n> I think there needs to be a way to disable this. How about we add an\n> option `--[no-]references` to do so? I was briefly wondering whether we\n> also want to have `--only-references`, but if a user wants to do that\n> they can simply execute `git refs verify` directly.\n> \n\nGood idea, let me improve this in the next version.\n\nThanks,\nJialuo\n\n> Patrick\n"},{"id":"511865","messageId":"Z6M4zHAhA4jZBdrj@ArchLinux","threadId":"62743","inReplyTo":"Z6CA9qCLwMnOsKxR@pks.im","subject":"Re: [PATCH v2 5/8] packed-backend: check whether the refname contains NUL characters","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-05T10:09:16Z","receivedAt":"2025-02-05T10:07:37Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Feb 03, 2025 at 09:40:22AM +0100, Patrick Steinhardt wrote:\n> On Thu, Jan 30, 2025 at 12:07:46PM +0800, shejialuo wrote:\n> > We have already implemented the header consistency check for the raw\n> > \"packed-refs\" file. Before we implement the consistency check for each\n> > ref entry, let's analysis [1] which reports that \"git fsck\" cannot\n> > detect some NUL characters.\n> \n> This paragraph doesn't quite parse. I think it can simply be left out,\n> as the remainder of the commit message already explains in more than\n> enough detail what you're doing.\n> \n\nLet me improve this in the next version.\n\n> > \"packed-backend.c::next_record\" will use \"check_refname_format\" to check\n> > the consistency of the refname. If it is not OK, the program will die.\n> > So, we already have the code path and we must miss out something.\n> > \n> > We use the following code to get the refname:\n> > \n> > strbuf_add(&iter->refname_buf, p, eol - p);\n> > iter->base.refname = iter->refname_buf.buf\n> > \n> > In the above code, `p` is the start pointer of the refname and `eol` is\n> > the next newline pointer. We calculate the length of the refname by\n> > subtracting the two pointers. Then we add the memory range between `p`\n> > and `eol` to get the refname.\n> > \n> > However, if there are some NUL characters in the memory range between `p`\n> > and `eol`, we will see the refname as a valid ref name as long as the\n> > memory range between `p` and first occurred NUL character is valid.\n> > \n> > In order to catch above corruption, create a new function\n> > \"refname_contains_nul\" by searching the first NUL character. If it is\n> > not at the end of the string, there must be some NUL characters in the\n> > refname.\n> > \n> > Use this function in \"next_record\" function to die the program if\n> > \"refname_contains_nul\" returns true.\n> \n> Yeah, makes sense to me. NUL bytes are invalid, and nothing good can\n> come out of it.\n> \n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index 883189f3a1..870c8e7aaa 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -494,6 +494,22 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n> > \t\t\t\t last_line, eof - last_line);\n> > }\n> > \n> > +/*\n> > + * When parsing the \"packed-refs\" file, we will parse it line by line.\n> > + * Because we know the start pointer of the refname and the next\n> > + * newline pointer, we could calculate the length of the refname by\n> > + * subtracting the two pointers. However, there is a corner case where\n> > + * the refname contains corrupted embedded NUL characters. And\n> > + * `check_refname_format()` will not catch this when the truncated\n> > + * refname is still a valid refname. To prevent this, we need to check\n> > + * whether the refname contains the NUL characters.\n> > + */\n> > +static int refname_contains_nul(struct strbuf *refname)\n> > +{\n> > +\tconst char *pos = memchr(refname->buf, '\\0', refname->len + 1);\n> > +\treturn pos < refname->buf + refname->len;\n> > +}\n> \n> This can be simplified to:\n> \n> return !!memchr(refname->buf, '\\0', refname->len);\n> \n\nThis is very nice.\n\n> Ideally, we'd be amending `check_refname_format()` to do the checking\n> for us. But we can't without a wider refactoring because that function\n> gets a C string, and C strings are naturally terminadet by NUL\n> characters.\n> \n\nYes, we cannot. Actually, this is a corner case. NUL character is so\nspecial.\n\n> I think that adding a new function for this is a bit over the top\n> though, as the check is unlikely to be useful in a lot of places and the\n> logic is rather trivial. So I'd just inline the check into\n> `next_record()`.\n> \n\nThe reason why I extract this logic into a separate function is that we\nwill reuse this logic for later packed backend consistency checking. We\nnearly use the same way to parse the raw \"packed-ref\" files. So, I don't\nwant to repeat here.\n\nI will improve the commit message to add the motivation why we need to\nuse a function instead of using it in the inline way.\n\nThanks,\nJialuo\n\n> Patrick\n"},{"id":"511927","messageId":"Z6RPJI10-2QkwyqH@ArchLinux","threadId":"62743","inReplyTo":"Z5r6ZnLH3Ee8IQnN@ArchLinux","subject":"[PATCH v3 0/8] add more ref consistency checks","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-06T05:56:52Z","receivedAt":"2025-02-06T05:55:13Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis new version handles the following problem:\n\n1. [PACTH v3 2/8]: enhance the commit message.\n2. [PACTH v3 3/8]: delete some paragraph in the commit message to make\n it more clear.\n3. [PATCH v3 4/8]: remove unneeded checks for header and related tests\n and update the commit message.\n4. [PATCH v3 5/8]: enhance the code suggested by Patrick.\n5. [PATCH v3 6/8]: enhance the commit message and add a comment to\n explain why we need to execute `start++` for peeled line.\n6. [PATCH v3 7/8]: parse the header to get whether there is a \"sorted\"\n trait. If so, we need to check whether it is sorted and update the\n test to exercise.\n7. [PATCH v3 8/8]: use 1 literal instead of creating a new variable. And\n add options \"--[no-]references\" to allow the user disable checking\n the ref database. Then, update the related documentation and commit\n message.\n\nThanks,\nJialuo\n\nshejialuo (8):\n t0602: use subshell to ensure working directory unchanged\n builtin/refs: get worktrees without reading head information\n packed-backend: check whether the \"packed-refs\" is regular file\n packed-backend: add \"packed-refs\" header consistency check\n packed-backend: check whether the refname contains NUL characters\n packed-backend: add \"packed-refs\" entry consistency check\n packed-backend: check whether the \"packed-refs\" is sorted\n builtin/fsck: add `git refs verify` child process\n\n Documentation/fsck-msgids.txt | 14 +\n Documentation/git-fsck.txt | 6 +-\n builtin/fsck.c | 33 +-\n builtin/refs.c | 2 +-\n fsck.h | 4 +\n refs/packed-backend.c | 338 +++++++++-\n t/t0602-reffiles-fsck.sh | 1111 +++++++++++++++++++--------------\n worktree.c | 5 +\n worktree.h | 6 +\n 9 files changed, 1036 insertions(+), 483 deletions(-)\n\nRange-diff against v2:\n1: 20889b7b18 = 1: 20889b7b18 t0602: use subshell to ensure working directory unchanged\n2: 97688c8700 ! 2: 9d7780e953 builtin/refs: get worktrees without reading head info\n @@ Metadata\n Author: shejialuo \n \n ## Commit message ##\n - builtin/refs: get worktrees without reading head info\n + builtin/refs: get worktrees without reading head information\n \n In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\n and \"next_record\" which would check the correctness of the content of\n @@ Commit message\n \n Although this behavior has no harm for the program, it will\n short-circuit the program. When the users execute \"git refs verify\" or\n - \"git fsck\", we don't want to simply die the program but rather show the\n - warnings or errors as many as possible to info the users. So, we should\n - avoid reading the head info.\n + \"git fsck\", we should avoid reading the head information, which may\n + execute the read operation in packed backend with stricter checks to die\n + the program. Instead, we should continue to check other parts of the\n + \"packed-refs\" file completely.\n \n Fortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\n worktrees, 2023-12-29), we have introduced a function\n \"get_worktrees_internal\" which allows us to get worktrees without\n - reading head info.\n + reading head information.\n \n Create a new exposed function \"get_worktrees_without_reading_head\", then\n replace the \"get_worktrees\" in \"builtin/refs\" with the new created\n3: 122ad3be02 ! 3: 44d26f6440 packed-backend: check whether the \"packed-refs\" is regular\n @@ Metadata\n Author: shejialuo \n \n ## Commit message ##\n - packed-backend: check whether the \"packed-refs\" is regular\n + packed-backend: check whether the \"packed-refs\" is regular file\n \n Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\n consistency and correctness of \"packed-refs\" file, they never check the\n filetype of the \"packed-refs\". The user should always use \"git\n - packed-refs\" command to create the raw regular \"packed-refs\" file, so we\n + pack-refs\" command to create the raw regular \"packed-refs\" file, so we\n need to explicitly check this in \"git refs verify\".\n \n - We could use the following two ways to check whether the \"packed-refs\"\n - is regular:\n -\n - 1. We could use \"lstat\" system call to check the file mode.\n - 2. We could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file\n - If the returned fd value is less than 0, we could check whether the\n - \"errno\" is \"ELOOP\" to report an error to the user.\n -\n - It might seems that the method one is much easier than method two.\n - However, method one has a significant drawback. When we have checked the\n - file mode using \"lstat\", we will need to read the file content, there is\n - a possibility that when finishing reading the file content to the\n - memory, the file could be changed into a symlink and we cannot notice.\n -\n - With method two, we could get the \"fd\" firstly. Even if the file is\n - changed into a symlink, we could still operate the \"fd\" in the memory\n - which is consistent across the checking which avoids race condition.\n + We could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file.\n + If the returned \"fd\" value is less than 0, we could check whether the\n + \"errno\" is \"ELOOP\" to report an error to the user.\n \n Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n the user if \"packed-refs\" is not a regular file.\n4: c3d32993c5 ! 4: a9ab7af16a packed-backend: add \"packed-refs\" header consistency check\n @@ Commit message\n pack-refs with:\". As we are going to implement the header consistency\n check, we should port this check into \"packed_fsck\".\n \n - However, the above check is not enough, this is because \"git pack-refs\"\n - will always write \"PACKED_REFS_HEADER\" which is a constant string to the\n - \"packed-refs\" file. So, we should check the following things for the\n - header.\n + However, we need to consider other situations and discuss whether we\n + need to add checks.\n \n - 1. If the header does not exist, we may report an error to the user\n - because it should exist, but we do allow no header in \"packed-refs\"\n - file. So, create a new fsck message \"packedRefMissingHeader(INFO)\" to\n - warn the user and also keep compatibility.\n + 1. If the header does not exist, we should not report an error to the\n + user. This is because in older Git version, we never write header in\n + the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n + in runtime.\n 2. If the header content does not start with \"# packed-ref with:\", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n 3. If the header content is not the same as the constant string\n - \"PACKED_REFS_HEADER\", ideally, we should report an error to the user.\n - However, we allow other contents as long as the header content starts\n - with \"# packed-ref with:\". To keep compatibility, create a new fsck\n - message \"unknownPackedRefHeader(INFO)\" to warn about this. We may\n - tighten this rule in the future.\n + \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n + intentionally. So, there is no need to report.\n \n - In order to achieve above checks, read the \"packed-refs\" file via\n - \"strbuf_read\". Like what \"create_snapshot\" and other functions do, we\n - could split the line by finding the next newline in the buffer. When we\n - cannot find a newline, we could report an error.\n + As we have analyzed, we only need to check the case 2 in the above. In\n + order to do this, read the \"packed-refs\" file via \"strbuf_read\". Like\n + what \"create_snapshot\" and other functions do, we could split the line\n + by finding the next newline in the buffer. When we cannot find a\n + newline, we could report an error.\n \n So, create a function \"packed_fsck_ref_next_line\" to find the next\n newline and if there is no such newline, use\n \"packedRefEntryNotTerminated(ERROR)\" to report an error to the user.\n \n - Then, parse the first line to apply the above three checks. Update the\n - test to excise the code.\n + Then, parse the first line to apply the checks. Update the test to\n + exercise the code.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n @@ Documentation/fsck-msgids.txt\n +`packedRefEntryNotTerminated`::\n +\t(ERROR) The \"packed-refs\" file contains an entry that is\n +\tnot terminated by a newline.\n -+\n -+`packedRefMissingHeader`::\n -+\t(INFO) The \"packed-refs\" file does not contain the header.\n +\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\n -@@\n - `treeNotSorted`::\n - \t(ERROR) A tree is not properly sorted.\n - \n -+`unknownPackedRefHeader`::\n -+\t(INFO) The \"packed-refs\" header starts with \"# pack-refs with:\"\n -+\tbut the remaining content is not the same as what `git pack-refs`\n -+\twould write.\n -+\n - `unknownType`::\n - \t(ERROR) Found an unknown object type.\n - \n \n ## fsck.h ##\n @@ fsck.h: enum fsck_msg_type {\n @@ fsck.h: enum fsck_msg_type {\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\n -@@ fsck.h: enum fsck_msg_type {\n - \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n - \tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n - \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n -+\tFUNC(UNKNOWN_PACKED_REF_HEADER, INFO) \\\n -+\tFUNC(PACKED_REF_MISSING_HEADER, INFO) \\\n - \t/* ignored (elevated when requested) */ \\\n - \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n - \n \n ## refs/packed-backend.c ##\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\treturn ret;\n +}\n +\n -+static int packed_fsck_ref_header(struct fsck_options *o, const char *start, const char *eol)\n ++static int packed_fsck_ref_header(struct fsck_options *o,\n ++\t\t\t\t const char *start, const char *eol)\n +{\n -+\tconst char *err_fmt = NULL;\n -+\tint fsck_msg_id = -1;\n -+\n +\tif (!starts_with(start, \"# pack-refs with:\")) {\n -+\t\terr_fmt = \"'%.*s' does not start with '# pack-refs with:'\";\n -+\t\tfsck_msg_id = FSCK_MSG_BAD_PACKED_REF_HEADER;\n -+\t} else if (strncmp(start, PACKED_REFS_HEADER, strlen(PACKED_REFS_HEADER))) {\n -+\t\terr_fmt = \"'%.*s' is an unknown packed-refs header\";\n -+\t\tfsck_msg_id = FSCK_MSG_UNKNOWN_PACKED_REF_HEADER;\n -+\t}\n -+\n -+\tif (err_fmt && fsck_msg_id >= 0) {\n +\t\tstruct fsck_ref_report report = { 0 };\n +\t\treport.path = \"packed-refs.header\";\n +\n -+\t\treturn fsck_report_ref(o, &report, fsck_msg_id, err_fmt,\n ++\t\treturn fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n ++\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n +\t\t\t\t (int)(eol - start), start);\n -+\n +\t}\n +\n +\treturn 0;\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\t\t\t\t const char *start, const char *eof)\n +{\n +\tstruct strbuf packed_entry = STRBUF_INIT;\n -+\tint line_number = 1;\n ++\tunsigned long line_number = 1;\n +\tconst char *eol;\n +\tint ret = 0;\n +\n -+\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n ++\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n +\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n +\tif (*start == '#') {\n +\t\tret |= packed_fsck_ref_header(o, start, eol);\n +\n +\t\tstart = eol + 1;\n +\t\tline_number++;\n -+\t} else {\n -+\t\tstruct fsck_ref_report report = { 0 };\n -+\t\treport.path = \"packed-refs\";\n -+\n -+\t\tret |= fsck_report_ref(o, &report,\n -+\t\t\t\t FSCK_MSG_PACKED_REF_MISSING_HEADER,\n -+\t\t\t\t \"missing header line\");\n +\t}\n +\n +\tstrbuf_release(&packed_entry);\n @@ t/t0602-reffiles-fsck.sh: test_expect_success SYMLINKS 'the filetype of packed-r\n +\t\tgit refs verify 2>err &&\n +\t\ttest_must_be_empty err &&\n +\n -+\t\tprintf \"$(git rev-parse main) refs/heads/main\\n\" >.git/packed-refs &&\n -+\t\tgit refs verify 2>err &&\n -+\t\tcat >expect <<-EOF &&\n -+\t\twarning: packed-refs: packedRefMissingHeader: missing header line\n -+\t\tEOF\n -+\t\trm .git/packed-refs &&\n -+\t\ttest_cmp expect err &&\n -+\n +\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n -+\t\t\t\t\"# pack-refs with traits: peeled fully-peeled sorted \" \\\n -+\t\t\t\t\"# pack-refs with a: peeled fully-peeled\"\n ++\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n ++\t\t\t\t \"# pack-refs with a: peeled fully-peeled\"\n +\t\tdo\n +\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n +\t\t\ttest_must_fail git refs verify 2>err &&\n @@ t/t0602-reffiles-fsck.sh: test_expect_success SYMLINKS 'the filetype of packed-r\n +\t\t\tEOF\n +\t\t\trm .git/packed-refs &&\n +\t\t\ttest_cmp expect err || return 1\n -+\t\tdone &&\n -+\n -+\t\tfor unknown_header in \"# pack-refs with: peeled fully-peeled sorted garbage\" \\\n -+\t\t\t\t\"# pack-refs with: peeled\" \\\n -+\t\t\t\t\"# pack-refs with: peeled peeled-fully sort\"\n -+\t\tdo\n -+\t\t\tprintf \"%s\\n\" \"$unknown_header\" >.git/packed-refs &&\n -+\t\t\tgit refs verify 2>err &&\n -+\t\t\tcat >expect <<-EOF &&\n -+\t\t\twarning: packed-refs.header: unknownPackedRefHeader: '\\''$unknown_header'\\'' is an unknown packed-refs header\n -+\t\t\tEOF\n -+\t\t\trm .git/packed-refs &&\n -+\t\t\ttest_cmp expect err || return 1\n +\t\tdone\n +\t)\n +'\n5: c545a61107 ! 5: 9b075434a1 packed-backend: check whether the refname contains NUL characters\n @@ Metadata\n ## Commit message ##\n packed-backend: check whether the refname contains NUL characters\n \n - We have already implemented the header consistency check for the raw\n - \"packed-refs\" file. Before we implement the consistency check for each\n - ref entry, let's analysis [1] which reports that \"git fsck\" cannot\n - detect some NUL characters.\n -\n \"packed-backend.c::next_record\" will use \"check_refname_format\" to check\n the consistency of the refname. If it is not OK, the program will die.\n - So, we already have the code path and we must miss out something.\n + However, it is reported in [1], we cannot catch some corruption. But we\n + already have the code path and we must miss out something.\n \n We use the following code to get the refname:\n \n @@ refs/packed-backend.c: static void verify_buffer_safe(struct snapshot *snapshot)\n + */\n +static int refname_contains_nul(struct strbuf *refname)\n +{\n -+\tconst char *pos = memchr(refname->buf, '\\0', refname->len + 1);\n -+\treturn pos < refname->buf + refname->len;\n ++\treturn !!memchr(refname->buf, '\\0', refname->len);\n +}\n +\n #define SMALL_FILE_SIZE (32*1024)\n6: a480e2bf49 ! 6: a976508319 packed-backend: add \"packed-refs\" entry consistency check\n @@ Commit message\n \"packed-backend.c::next_record\" will parse the ref entry to check the\n consistency. This function has already checked the following things:\n \n - 1. Parse the main line of the ref entry, if the oid is not correct. It\n - will die the program. And then it will check whether the next\n - character of the oid is space. Then it will check whether the refname\n - is correct.\n - 2. If the next line starts with '^', it will continue to parse the oid\n - of the peeled oid content and check whether the last character is\n - '\\n'.\n + 1. Parse the main line of the ref entry to inspect whether the oid is\n + not correct. Then, check whether the next character is oid. Then\n + check the refname.\n + 2. If the next line starts with '^', it would continue to parse the\n + peeled oid and check whether the last character is '\\n'.\n \n - We can iterate each line by using the \"packed_fsck_ref_next_line\"\n - function. Then, create a new fsck message \"badPackedRefEntry(ERROR)\" to\n - report to the user when something is wrong.\n -\n - Create two new functions \"packed_fsck_ref_main_line\" and\n - \"packed_fsck_ref_peeled_line\" for case 1 and case 2 respectively. Last,\n - update the unit test to exercise the code.\n + As we decide to implement the ref consistency check for \"packed-refs\",\n + let's port these two checks and update the test to exercise the code.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n @@ fsck.h: enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \n ## refs/packed-backend.c ##\n -@@ refs/packed-backend.c: static int packed_fsck_ref_header(struct fsck_options *o, const char *start, con\n +@@ refs/packed-backend.c: static int packed_fsck_ref_header(struct fsck_options *o,\n \treturn 0;\n }\n \n @@ refs/packed-backend.c: static int packed_fsck_ref_header(struct fsck_options *o,\n +\n +\treport.path = packed_entry->buf;\n +\n ++\t/*\n ++\t * Skip the '^' and parse the peeled oid.\n ++\t */\n +\tstart++;\n -+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n ++\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo))\n +\t\treturn fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n +\t\t\t\t \"'%.*s' has invalid peeled oid\",\n +\t\t\t\t (int)(eol - start), start);\n -+\t}\n +\n -+\tif (p != eol) {\n ++\tif (p != eol)\n +\t\treturn fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n +\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n +\t\t\t\t (int)(eol - p), p);\n -+\t}\n +\n +\treturn 0;\n +}\n @@ refs/packed-backend.c: static int packed_fsck_ref_header(struct fsck_options *o,\n +\n +\treport.path = packed_entry->buf;\n +\n -+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n ++\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo))\n +\t\treturn fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n +\t\t\t\t \"'%.*s' has invalid oid\",\n +\t\t\t\t (int)(eol - start), start);\n -+\t}\n +\n -+\tif (p == eol || !isspace(*p)) {\n ++\tif (p == eol || !isspace(*p))\n +\t\treturn fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n +\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n +\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n -+\t}\n +\n +\tp++;\n +\tstrbuf_reset(refname);\n +\tstrbuf_add(refname, p, eol - p);\n -+\tif (refname_contains_nul(refname)) {\n ++\tif (refname_contains_nul(refname))\n +\t\treturn fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n +\t\t\t\t \"refname '%s' contains NULL binaries\",\n +\t\t\t\t refname->buf);\n -+\t}\n +\n -+\tif (check_refname_format(refname->buf, 0)) {\n ++\tif (check_refname_format(refname->buf, 0))\n +\t\treturn fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n +\t\t\t\t \"has bad refname '%s'\", refname->buf);\n -+\t}\n +\n +\treturn 0;\n +}\n @@ refs/packed-backend.c: static int packed_fsck_ref_header(struct fsck_options *o,\n {\n \tstruct strbuf packed_entry = STRBUF_INIT;\n +\tstruct strbuf refname = STRBUF_INIT;\n - \tint line_number = 1;\n + \tunsigned long line_number = 1;\n \tconst char *eol;\n \tint ret = 0;\n @@ refs/packed-backend.c: static int packed_fsck_ref_content(struct fsck_options *o,\n - \t\t\t\t \"missing header line\");\n + \t\tline_number++;\n \t}\n \n +\twhile (start < eof) {\n +\t\tstrbuf_reset(&packed_entry);\n -+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n ++\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n +\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n +\t\tret |= packed_fsck_ref_main_line(o, ref_store, &packed_entry, &refname, start, eol);\n +\t\tstart = eol + 1;\n +\t\tline_number++;\n +\t\tif (start < eof && *start == '^') {\n +\t\t\tstrbuf_reset(&packed_entry);\n -+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n ++\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n +\t\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n +\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, &packed_entry,\n +\t\t\t\t\t\t\t start, eol);\n7: 199965dfb7 ! 7: 707e3e2151 packed-backend: check whether the \"packed-refs\" is sorted\n @@ Metadata\n ## Commit message ##\n packed-backend: check whether the \"packed-refs\" is sorted\n \n - We will always try to sort the \"packed-refs\" increasingly by comparing\n - the refname. So, we should add checks to verify whether the \"packed-refs\"\n - is sorted.\n + When there is a \"sorted\" trait in the header of the \"packed-refs\" file,\n + it means that each entry is sorted increasingly by comparing the\n + refname. We should add checks to verify whether the \"packed-refs\" is\n + sorted in this case.\n \n - We already have code to parse the content. Let's create a new structure\n - \"fsck_packed_ref_entry\" to store the state during the parsing process\n - for every entry. It may seem that we could just add a new \"struct strbuf\n - refname\" into the \"struct fsck_packed_ref_entry\" and during the parsing\n - process, we could store the refname into this structure and we could\n - compare later. However, this is not a good design due to the following\n - reasons:\n + Update the \"packed_fsck_ref_header\" to know whether there is a \"sorted\"\n + trail in the header. Then, create a new structure \"fsck_packed_ref_entry\"\n + to store the state during the parsing process for every entry. It may\n + seem that we could just add a new \"struct strbuf refname\" into the\n + \"struct fsck_packed_ref_entry\" and during the parsing process, we could\n + store the refname into this structure and thus we could compare later.\n + However, this is not a good design due to the following reasons:\n \n 1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n @@ Commit message\n \n ## Documentation/fsck-msgids.txt ##\n @@\n - `packedRefMissingHeader`::\n - \t(INFO) The \"packed-refs\" file does not contain the header.\n + \t(ERROR) The \"packed-refs\" file contains an entry that is\n + \tnot terminated by a newline.\n \n +`packedRefUnsorted`::\n +\t(ERROR) The \"packed-refs\" file is not sorted.\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n }\n \n +struct fsck_packed_ref_entry {\n -+\tint line_number;\n ++\tunsigned long line_number;\n +\n +\tstruct snapshot_record record;\n +};\n +\n -+static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(int line_number,\n ++static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(unsigned long line_number,\n +\t\t\t\t\t\t\t\t const char *start)\n +{\n +\tstruct fsck_packed_ref_entry *entry = xcalloc(1, sizeof(*entry));\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\treturn entry;\n +}\n +\n -+static void free_fsck_packed_ref_entries(struct fsck_packed_ref_entry **entries, int nr)\n ++static void free_fsck_packed_ref_entries(struct fsck_packed_ref_entry **entries, size_t nr)\n +{\n -+\tfor (int i = 0; i < nr; i++)\n ++\tfor (size_t i = 0; i < nr; i++)\n +\t\tfree(entries[i]);\n +\tfree(entries);\n +}\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n static int packed_fsck_ref_next_line(struct fsck_options *o,\n \t\t\t\t struct strbuf *packed_entry, const char *start,\n \t\t\t\t const char *eof, const char **eol)\n +@@ refs/packed-backend.c: static int packed_fsck_ref_next_line(struct fsck_options *o,\n + }\n + \n + static int packed_fsck_ref_header(struct fsck_options *o,\n +-\t\t\t\t const char *start, const char *eol)\n ++\t\t\t\t const char *start, const char *eol,\n ++\t\t\t\t unsigned int *sorted)\n + {\n +-\tif (!starts_with(start, \"# pack-refs with:\")) {\n ++\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n ++\tchar *tmp_line;\n ++\tint ret = 0;\n ++\tchar *p;\n ++\n ++\ttmp_line = xmemdupz(start, eol - start);\n ++\tif (!skip_prefix(tmp_line, \"# pack-refs with:\", (const char **)&p)) {\n + \t\tstruct fsck_ref_report report = { 0 };\n + \t\treport.path = \"packed-refs.header\";\n + \n +-\t\treturn fsck_report_ref(o, &report,\n +-\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n +-\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n +-\t\t\t\t (int)(eol - start), start);\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n ++\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n ++\t\t\t\t (int)(eol - start), start);\n ++\t\tgoto cleanup;\n + \t}\n + \n +-\treturn 0;\n ++\tstring_list_split_in_place(&traits, p, \" \", -1);\n ++\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n ++\n ++cleanup:\n ++\tfree(tmp_line);\n ++\tstring_list_clear(&traits, 0);\n ++\treturn ret;\n + }\n + \n + static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n @@ refs/packed-backend.c: static int packed_fsck_ref_main_line(struct fsck_options *o,\n \treturn 0;\n }\n @@ refs/packed-backend.c: static int packed_fsck_ref_main_line(struct fsck_options\n +static int packed_fsck_ref_sorted(struct fsck_options *o,\n +\t\t\t\t struct ref_store *ref_store,\n +\t\t\t\t struct fsck_packed_ref_entry **entries,\n -+\t\t\t\t int nr)\n ++\t\t\t\t size_t nr)\n +{\n +\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n +\tstruct strbuf packed_entry = STRBUF_INIT;\n @@ refs/packed-backend.c: static int packed_fsck_ref_main_line(struct fsck_options\n +\tstruct strbuf refname2 = STRBUF_INIT;\n +\tint ret = 0;\n +\n -+\tfor (int i = 1; i < nr; i++) {\n ++\tfor (size_t i = 1; i < nr; i++) {\n +\t\tconst char *r1 = entries[i - 1]->record.start + hexsz + 1;\n +\t\tconst char *r2 = entries[i]->record.start + hexsz + 1;\n +\n @@ refs/packed-backend.c: static int packed_fsck_ref_main_line(struct fsck_options\n +\t\t\t\t entries[i]->record.len);\n +\t\t\tstrbuf_add(&refname2, r2, eol - r2);\n +\n -+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\",\n ++\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\",\n +\t\t\t\t entries[i - 1]->line_number);\n +\t\t\treport.path = packed_entry.buf;\n +\t\t\tret = fsck_report_ref(o, &report,\n @@ refs/packed-backend.c: static int packed_fsck_ref_main_line(struct fsck_options\n \tstruct strbuf packed_entry = STRBUF_INIT;\n +\tstruct fsck_packed_ref_entry **entries;\n \tstruct strbuf refname = STRBUF_INIT;\n -+\tint entry_alloc = 20;\n - \tint line_number = 1;\n -+\tint entry_nr = 0;\n + \tunsigned long line_number = 1;\n ++\tunsigned int sorted = 0;\n ++\tsize_t entry_alloc = 20;\n ++\tsize_t entry_nr = 0;\n \tconst char *eol;\n \tint ret = 0;\n \n -@@ refs/packed-backend.c: static int packed_fsck_ref_content(struct fsck_options *o,\n - \t\t\t\t \"missing header line\");\n + \tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n + \tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n + \tif (*start == '#') {\n +-\t\tret |= packed_fsck_ref_header(o, start, eol);\n ++\t\tret |= packed_fsck_ref_header(o, start, eol, &sorted);\n + \n + \t\tstart = eol + 1;\n + \t\tline_number++;\n \t}\n \n +\tALLOC_ARRAY(entries, entry_alloc);\n @@ refs/packed-backend.c: static int packed_fsck_ref_content(struct fsck_options *o\n +\t\tALLOC_GROW(entries, entry_nr + 1, entry_alloc);\n +\t\tentries[entry_nr++] = entry;\n \t\tstrbuf_reset(&packed_entry);\n - \t\tstrbuf_addf(&packed_entry, \"packed-refs line %d\", line_number);\n + \t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n \t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n @@ refs/packed-backend.c: static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\tstart = eol + 1;\n @@ refs/packed-backend.c: static int packed_fsck_ref_content(struct fsck_options *o\n +\t\tentry->record.len = start - entry->record.start;\n \t}\n \n -+\tif (!ret)\n ++\tif (!ret && sorted)\n +\t\tret |= packed_fsck_ref_sorted(o, ref_store, entries, entry_nr);\n +\n \tstrbuf_release(&packed_entry);\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs content should be che\n \t)\n '\n \n -+test_expect_success 'packed-ref sorted should be checked' '\n ++test_expect_success 'packed-ref with sorted trait should be checked' '\n +\ttest_when_finished \"rm -rf repo\" &&\n +\tgit init repo &&\n +\t(\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs content should be che\n +\t\tEOF\n +\t\trm .git/packed-refs &&\n +\t\ttest_cmp expect err &&\n ++\n +\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n +\t\tprintf \"%s %s\\n\" \"$tag_1_oid\" \"$refname3\" >>.git/packed-refs &&\n +\t\tprintf \"^%s\\n\" \"$tag_1_peeled_oid\" >>.git/packed-refs &&\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs content should be che\n +\t\ttest_cmp expect err\n +\t)\n +'\n ++\n ++test_expect_success 'packed-ref without sorted trait should not be checked' '\n ++\ttest_when_finished \"rm -rf repo\" &&\n ++\tgit init repo &&\n ++\t(\n ++\t\tcd repo &&\n ++\t\ttest_commit default &&\n ++\t\tgit branch branch-1 &&\n ++\t\tgit branch branch-2 &&\n ++\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n ++\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n ++\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n ++\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n ++\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n ++\t\trefname1=\"refs/heads/main\" &&\n ++\t\trefname2=\"refs/heads/foo\" &&\n ++\t\trefname3=\"refs/tags/foo\" &&\n ++\t\tprintf \"# pack-refs with: peeled fully-peeled \\n\" >.git/packed-refs &&\n ++\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n ++\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n ++\t\tgit refs verify 2>err &&\n ++\t\ttest_must_be_empty err\n ++\t)\n ++'\n +\n test_done\n8: 81a2164c04 ! 8: 4f2170aa7c builtin/fsck: add `git refs verify` child process\n @@ Commit message\n It's hard to know how many loose refs we will check now. We might\n improve this later.\n \n - And we run this function in the first execution sequence of\n - \"git-fsck(1)\" because we don't want the existing code of \"git-fsck(1)\"\n - which implicitly checks the consistency of refs to die the program.\n + Then, introduce the option to allow the user to disable checking ref\n + database consistency. Put this function in the very first execution\n + sequence of \"git-fsck(1)\" due to that we don't want the existing code of\n + \"git-fsck(1)\" which would implicitly check the consistency of refs to\n + die the program.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n Signed-off-by: shejialuo \n \n + ## Documentation/git-fsck.txt ##\n +@@ Documentation/git-fsck.txt: SYNOPSIS\n + 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n + \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n + \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n +-\t [--[no-]name-objects] [...]\n ++\t [--[no-]name-objects] [--[no-]references] [...]\n + \n + DESCRIPTION\n + -----------\n +@@ Documentation/git-fsck.txt: care about this output and want to speed it up further.\n + \tprogress status even if the standard error stream is not\n + \tdirected to a terminal.\n + \n ++--[no-]references::\n ++\tControl whether to check the references database consistency\n ++\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n ++\n + CONFIGURATION\n + -------------\n + \n +\n ## builtin/fsck.c ##\n +@@ builtin/fsck.c: static int verbose;\n + static int show_progress = -1;\n + static int show_dangling = 1;\n + static int name_objects;\n ++static int check_references = 1;\n + #define ERROR_OBJECT 01\n + #define ERROR_REACHABLE 02\n + #define ERROR_PACK 04\n @@ builtin/fsck.c: static int check_pack_rev_indexes(struct repository *r, int show_progress)\n \treturn res;\n }\n @@ builtin/fsck.c: static int check_pack_rev_indexes(struct repository *r, int show\n +{\n +\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n +\tstruct progress *progress = NULL;\n -+\tuint64_t progress_num = 1;\n +\n +\tif (show_progress)\n -+\t\tprogress = start_progress(r, _(\"Checking ref database\"),\n -+\t\t\t\t\t progress_num);\n ++\t\tprogress = start_progress(r, _(\"Checking ref database\"), 1);\n +\n +\tif (verbose)\n +\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n @@ builtin/fsck.c: static int check_pack_rev_indexes(struct repository *r, int show\n static char const * const fsck_usage[] = {\n \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n + \t \" [--[no-]dangling] [--[no-]progress] [--connectivity-only]\\n\"\n +-\t \" [--[no-]name-objects] [...]\"),\n ++\t \" [--[no-]name-objects] [--[no-]references] [...]\"),\n + \tNULL\n + };\n + \n +@@ builtin/fsck.c: static struct option fsck_opts[] = {\n + \t\t\t\tN_(\"write dangling objects in .git/lost-found\")),\n + \tOPT_BOOL(0, \"progress\", &show_progress, N_(\"show progress\")),\n + \tOPT_BOOL(0, \"name-objects\", &name_objects, N_(\"show verbose names for reachable objects\")),\n ++\tOPT_BOOL(0, \"references\", &check_references, N_(\"check reference database consistency\")),\n + \tOPT_END(),\n + };\n + \n @@ builtin/fsck.c: int cmd_fsck(int argc,\n \tgit_config(git_fsck_config, &fsck_obj_options);\n \tprepare_repo_settings(the_repository);\n \n -+\tfsck_refs(the_repository);\n ++\tif (check_references)\n ++\t\tfsck_refs(the_repository);\n +\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n-- \n2.48.1\n\n"},{"id":"511928","messageId":"Z6RPg9i_nhPiSvAx@ArchLinux","threadId":"62743","inReplyTo":"Z6RPJI10-2QkwyqH@ArchLinux","subject":"[PATCH v3 1/8] t0602: use subshell to ensure working directory unchanged","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-06T05:58:27Z","receivedAt":"2025-02-06T05:56:48Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"For every test, we would execute the command \"cd repo\" in the first but\nwe never execute the command \"cd ..\" to restore the working directory.\nHowever, it's either not a good idea use above way. Because if any test\nfails between \"cd repo\" and \"cd ..\", the \"cd ..\" will never be reached.\nAnd we cannot correctly restore the working directory.\n\nLet's use subshell to ensure that the current working directory could be\nrestored to the correct path.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n t/t0602-reffiles-fsck.sh | 967 ++++++++++++++++++++-------------------\n 1 file changed, 494 insertions(+), 473 deletions(-)\n\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex d4a08b823b..cf7a202d0d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -14,222 +14,229 @@ test_expect_success 'ref name should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b default-branch &&\n-\tgit tag default-tag &&\n-\tgit tag multi_hierarchy/default-tag &&\n-\n-\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n-\trm $branch_dir_prefix/@ &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n-\tgit refs verify 2>err &&\n-\trm $tag_dir_prefix/tag-1.lock &&\n-\ttest_must_be_empty err &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/.lock: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t(\n+\t\tcd repo &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b default-branch &&\n+\t\tgit tag default-tag &&\n+\t\tgit tag multi_hierarchy/default-tag &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\t\trm $branch_dir_prefix/@ &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $tag_dir_prefix/tag-1.lock &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \t\ttest_must_fail git refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\terror: refs/tags/.lock: badRefName: invalid refname format\n \t\tEOF\n-\t\trm -r \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $tag_dir_prefix/.lock &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=warn refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n-\ttest_must_be_empty err\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b branch-1 &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=warn refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm $branch_dir_prefix/.branch-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n '\n \n test_expect_success 'ref name check should work for multiple worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\n-\tcd repo &&\n-\ttest_commit initial &&\n-\tgit checkout -b branch-1 &&\n-\ttest_commit second &&\n-\tgit checkout -b branch-2 &&\n-\ttest_commit third &&\n-\tgit checkout -b branch-3 &&\n-\tgit worktree add ./worktree-1 branch-1 &&\n-\tgit worktree add ./worktree-2 branch-2 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n-\t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n \t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n-\n-\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n-\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err &&\n-\n-\tfor worktree in \"worktree-1\" \"worktree-2\"\n-\tdo\n+\t\tcd repo &&\n+\t\ttest_commit initial &&\n+\t\tgit checkout -b branch-1 &&\n+\t\ttest_commit second &&\n+\t\tgit checkout -b branch-2 &&\n+\t\ttest_commit third &&\n+\t\tgit checkout -b branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-1 &&\n+\t\tgit worktree add ./worktree-2 branch-2 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n \t\t(\n-\t\t\tcd $worktree &&\n-\t\t\ttest_must_fail git refs verify 2>err &&\n-\t\t\tcat >expect <<-EOF &&\n-\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\t\t\tEOF\n-\t\t\tsort err >sorted_err &&\n-\t\t\ttest_cmp expect sorted_err || return 1\n-\t\t)\n-\tdone\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\n+\t\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\t\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err &&\n+\n+\t\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\t\tdo\n+\t\t\t(\n+\t\t\t\tcd $worktree &&\n+\t\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\t\tcat >expect <<-EOF &&\n+\t\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\t\tEOF\n+\t\t\t\tsort err >sorted_err &&\n+\t\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t\t)\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tfor trailing_content in \" garbage\" \" more garbage\"\n-\tdo\n-\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-garbage &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n+\t\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n-\t'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\t'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n \n-\t garbage'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err\n+\t\t garbage'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -237,99 +244,103 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tbad_content_1=$(git rev-parse main)x &&\n-\tbad_content_2=xfsazqfxcadas &&\n-\tbad_content_3=Xfsazqfxcadas &&\n-\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n-\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n-\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n-\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n-\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n-\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tbad_content_1=$(git rev-parse main)x &&\n+\t\tbad_content_2=xfsazqfxcadas &&\n+\t\tbad_content_3=Xfsazqfxcadas &&\n+\t\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\t\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\t\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\t\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\t\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-complicated &&\n-\ttest_cmp expect err\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (aggregate)' '\n@@ -337,32 +348,34 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n-\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'the target of the textual symref should be checked' '\n@@ -370,28 +383,30 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n-\t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n-\n-\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n-\t\tgit refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked' '\n@@ -399,201 +414,207 @@ test_expect_success SYMLINKS 'symlink symref content should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n-\tEOF\n-\trm $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_cmp expect err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-good &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\t\tEOF\n+\t\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tmain_worktree_refdir_prefix=.git/refs/heads &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\n-\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tfor bad_referent_name in \".tag\" \"branch \"\n-\tdo\n-\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tmain_worktree_refdir_prefix=.git/refs/heads &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $worktree1_refdir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor bad_referent_name in \".tag\" \"branch \"\n+\t\tdo\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-garbage &&\n-\ttest_cmp expect err\n+\t\trm $worktree1_refdir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_done\n-- \n2.48.1\n\n"},{"id":"511929","messageId":"Z6RPkTOZOvBiLlaV@ArchLinux","threadId":"62743","inReplyTo":"Z6RPJI10-2QkwyqH@ArchLinux","subject":"[PATCH v3 2/8] builtin/refs: get worktrees without reading head information","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-06T05:58:41Z","receivedAt":"2025-02-06T05:57:01Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\nand \"next_record\" which would check the correctness of the content of\nthe \"packed-ref\" file. When anything is bad, the program will die.\n\nIt may seem that we have nothing relevant to above feature, because we\nare going to read and parse the raw \"packed-ref\" file without creating\nthe snapshot and using the ref iterator to check the consistency.\n\nHowever, when using \"get_worktrees\" in \"builtin/refs\", we would parse\nthe \"HEAD\" information. If the referent of the \"HEAD\" is inside the\n\"packed-ref\", we will call \"create_snapshot\" function to parse the\n\"packed-ref\" to get the information. No matter whether the entry of\n\"HEAD\" in \"packed-ref\" is correct, \"create_snapshot\" would call\n\"verify_buffer_safe\" to check whether there is a newline in the last\nline of the file. If not, the program will die.\n\nAlthough this behavior has no harm for the program, it will\nshort-circuit the program. When the users execute \"git refs verify\" or\n\"git fsck\", we should avoid reading the head information, which may\nexecute the read operation in packed backend with stricter checks to die\nthe program. Instead, we should continue to check other parts of the\n\"packed-refs\" file completely.\n\nFortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\nworktrees, 2023-12-29), we have introduced a function\n\"get_worktrees_internal\" which allows us to get worktrees without\nreading head information.\n\nCreate a new exposed function \"get_worktrees_without_reading_head\", then\nreplace the \"get_worktrees\" in \"builtin/refs\" with the new created\nfunction.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 2 +-\n worktree.c | 5 +++++\n worktree.h | 6 ++++++\n 3 files changed, 12 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex a29f195834..55ff5dae11 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tworktrees = get_worktrees();\n+\tworktrees = get_worktrees_without_reading_head();\n \tfor (size_t i = 0; worktrees[i]; i++)\n \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n \t\t\t\t &fsck_refs_options, worktrees[i]);\ndiff --git a/worktree.c b/worktree.c\nindex 248bbb39d4..89b7d86cef 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -175,6 +175,11 @@ struct worktree **get_worktrees(void)\n \treturn get_worktrees_internal(0);\n }\n \n+struct worktree **get_worktrees_without_reading_head(void)\n+{\n+\treturn get_worktrees_internal(1);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\ndiff --git a/worktree.h b/worktree.h\nindex 38145df80f..1ba4a161a0 100644\n--- a/worktree.h\n+++ b/worktree.h\n@@ -30,6 +30,12 @@ struct worktree {\n */\n struct worktree **get_worktrees(void);\n \n+/*\n+ * Like `get_worktrees`, but does not read HEAD. This is useful when checking\n+ * the consistency, as reading HEAD may not be necessary.\n+ */\n+struct worktree **get_worktrees_without_reading_head(void);\n+\n /*\n * Returns 1 if linked worktrees exist, 0 otherwise.\n */\n-- \n2.48.1\n\n"},{"id":"511930","messageId":"Z6RPnX3ff5ub7ojM@ArchLinux","threadId":"62743","inReplyTo":"Z6RPJI10-2QkwyqH@ArchLinux","subject":"[PATCH v3 3/8] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-06T05:58:53Z","receivedAt":"2025-02-06T05:57:12Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\nconsistency and correctness of \"packed-refs\" file, they never check the\nfiletype of the \"packed-refs\". The user should always use \"git\npack-refs\" command to create the raw regular \"packed-refs\" file, so we\nneed to explicitly check this in \"git refs verify\".\n\nWe could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file.\nIf the returned \"fd\" value is less than 0, we could check whether the\n\"errno\" is \"ELOOP\" to report an error to the user.\n\nReuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\nthe user if \"packed-refs\" is not a regular file.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 39 +++++++++++++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 22 ++++++++++++++++++++++\n 2 files changed, 57 insertions(+), 4 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a7b6f74b6e..6401cecd5f 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -4,6 +4,7 @@\n #include \"../git-compat-util.h\"\n #include \"../config.h\"\n #include \"../dir.h\"\n+#include \"../fsck.h\"\n #include \"../gettext.h\"\n #include \"../hash.h\"\n #include \"../hex.h\"\n@@ -1748,15 +1749,45 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n-static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED,\n+static int packed_fsck(struct ref_store *ref_store,\n+\t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n+\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n+\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tint ret = 0;\n+\tint fd;\n \n \tif (!is_main_worktree(wt))\n-\t\treturn 0;\n+\t\tgoto cleanup;\n \n-\treturn 0;\n+\tif (o->verbose)\n+\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n+\n+\tfd = open_nofollow(refs->path, O_RDONLY);\n+\tif (fd < 0) {\n+\t\t/*\n+\t\t * If the packed-refs file doesn't exist, there's nothing\n+\t\t * to check.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tif (errno == ELOOP) {\n+\t\t\tstruct fsck_ref_report report = { 0 };\n+\t\t\treport.path = \"packed-refs\";\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t\t \"not a regular file\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tret = error_errno(_(\"unable to open %s\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\treturn ret;\n }\n \n struct ref_storage_be refs_be_packed = {\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex cf7a202d0d..42c8d4ca1e 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -617,4 +617,26 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t)\n '\n \n+test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit pack-refs --all &&\n+\n+\t\tmv .git/packed-refs .git/packed-refs-back &&\n+\t\tln -sf packed-refs-bak .git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs: badRefFiletype: not a regular file\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"511931","messageId":"Z6RPqE64ScsjzUg7@ArchLinux","threadId":"62743","inReplyTo":"Z6RPJI10-2QkwyqH@ArchLinux","subject":"[PATCH v3 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-06T05:59:04Z","receivedAt":"2025-02-06T05:57:25Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c::create_snapshot\", if there is a header (the line\nwhich starts with '#'), we will check whether the line starts with \"#\npack-refs with:\". As we are going to implement the header consistency\ncheck, we should port this check into \"packed_fsck\".\n\nHowever, we need to consider other situations and discuss whether we\nneed to add checks.\n\n1. If the header does not exist, we should not report an error to the\n user. This is because in older Git version, we never write header in\n the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n in runtime.\n2. If the header content does not start with \"# packed-ref with:\", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n3. If the header content is not the same as the constant string\n \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n intentionally. So, there is no need to report.\n\nAs we have analyzed, we only need to check the case 2 in the above. In\norder to do this, read the \"packed-refs\" file via \"strbuf_read\". Like\nwhat \"create_snapshot\" and other functions do, we could split the line\nby finding the next newline in the buffer. When we cannot find a\nnewline, we could report an error.\n\nSo, create a function \"packed_fsck_ref_next_line\" to find the next\nnewline and if there is no such newline, use\n\"packedRefEntryNotTerminated(ERROR)\" to report an error to the user.\n\nThen, parse the first line to apply the checks. Update the test to\nexercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 8 ++++\n fsck.h | 2 +\n refs/packed-backend.c | 73 +++++++++++++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 25 ++++++++++++\n 4 files changed, 108 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex b14bc44ca4..11906f90fd 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -16,6 +16,10 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefHeader`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid\n+\theader.\n+\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n@@ -176,6 +180,10 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`packedRefEntryNotTerminated`::\n+\t(ERROR) The \"packed-refs\" file contains an entry that is\n+\tnot terminated by a newline.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex a44c231a5f..67e3c97bc0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n@@ -53,6 +54,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE, ERROR) \\\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 6401cecd5f..683cfe78dc 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1749,12 +1749,76 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+static int packed_fsck_ref_next_line(struct fsck_options *o,\n+\t\t\t\t struct strbuf *packed_entry, const char *start,\n+\t\t\t\t const char *eof, const char **eol)\n+{\n+\tint ret = 0;\n+\n+\t*eol = memchr(start, '\\n', eof - start);\n+\tif (!*eol) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\n+\t\treport.path = packed_entry->buf;\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n+\t\t\t\t \"'%.*s' is not terminated with a newline\",\n+\t\t\t\t (int)(eof - start), start);\n+\n+\t\t/*\n+\t\t * There is no newline but we still want to parse it to the end of\n+\t\t * the buffer.\n+\t\t */\n+\t\t*eol = eof;\n+\t}\n+\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_header(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tif (!starts_with(start, \"# pack-refs with:\")) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs.header\";\n+\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n+\t\t\t\t (int)(eol - start), start);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tunsigned long line_number = 1;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n+\tif (*start == '#') {\n+\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n static int packed_fsck(struct ref_store *ref_store,\n \t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tint ret = 0;\n \tint fd;\n \n@@ -1786,7 +1850,16 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n+\t\tret = error_errno(_(\"unable to read %s\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\n cleanup:\n+\tstrbuf_release(&packed_ref_content);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 42c8d4ca1e..da321f16c6 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -639,4 +639,29 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-refs header should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with a: peeled fully-peeled\"\n+\t\tdo\n+\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with:'\\''\n+\t\t\tEOF\n+\t\t\trm .git/packed-refs &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"511932","messageId":"Z6RPtC5KIvYnqtrh@ArchLinux","threadId":"62743","inReplyTo":"Z6RPJI10-2QkwyqH@ArchLinux","subject":"[PATCH v3 5/8] packed-backend: check whether the refname contains NUL characters","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-06T05:59:16Z","receivedAt":"2025-02-06T05:57:36Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will use \"check_refname_format\" to check\nthe consistency of the refname. If it is not OK, the program will die.\nHowever, it is reported in [1], we cannot catch some corruption. But we\nalready have the code path and we must miss out something.\n\nWe use the following code to get the refname:\n\n strbuf_add(&iter->refname_buf, p, eol - p);\n iter->base.refname = iter->refname_buf.buf\n\nIn the above code, `p` is the start pointer of the refname and `eol` is\nthe next newline pointer. We calculate the length of the refname by\nsubtracting the two pointers. Then we add the memory range between `p`\nand `eol` to get the refname.\n\nHowever, if there are some NUL characters in the memory range between `p`\nand `eol`, we will see the refname as a valid ref name as long as the\nmemory range between `p` and first occurred NUL character is valid.\n\nIn order to catch above corruption, create a new function\n\"refname_contains_nul\" by searching the first NUL character. If it is\nnot at the end of the string, there must be some NUL characters in the\nrefname.\n\nUse this function in \"next_record\" function to die the program if\n\"refname_contains_nul\" returns true.\n\n[1] https://lore.kernel.org/git/6cfee0e4-3285-4f18-91ff-d097da9de737@rd10.de/\n\nReported-by: R. Diez \nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 683cfe78dc..c8bb93bb18 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -494,6 +494,21 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n \t\t\t\t last_line, eof - last_line);\n }\n \n+/*\n+ * When parsing the \"packed-refs\" file, we will parse it line by line.\n+ * Because we know the start pointer of the refname and the next\n+ * newline pointer, we could calculate the length of the refname by\n+ * subtracting the two pointers. However, there is a corner case where\n+ * the refname contains corrupted embedded NUL characters. And\n+ * `check_refname_format()` will not catch this when the truncated\n+ * refname is still a valid refname. To prevent this, we need to check\n+ * whether the refname contains the NUL characters.\n+ */\n+static int refname_contains_nul(struct strbuf *refname)\n+{\n+\treturn !!memchr(refname->buf, '\\0', refname->len);\n+}\n+\n #define SMALL_FILE_SIZE (32*1024)\n \n /*\n@@ -895,6 +910,9 @@ static int next_record(struct packed_ref_iterator *iter)\n \tstrbuf_add(&iter->refname_buf, p, eol - p);\n \titer->base.refname = iter->refname_buf.buf;\n \n+\tif (refname_contains_nul(&iter->refname_buf))\n+\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n+\n \tif (check_refname_format(iter->base.refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!refname_is_safe(iter->base.refname))\n \t\t\tdie(\"packed refname is dangerous: %s\",\n-- \n2.48.1\n\n"},{"id":"511933","messageId":"Z6RPzIGD-fSwIEPV@ArchLinux","threadId":"62743","inReplyTo":"Z6RPJI10-2QkwyqH@ArchLinux","subject":"[PATCH v3 6/8] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-06T05:59:40Z","receivedAt":"2025-02-06T05:58:01Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will parse the ref entry to check the\nconsistency. This function has already checked the following things:\n\n1. Parse the main line of the ref entry to inspect whether the oid is\n not correct. Then, check whether the next character is oid. Then\n check the refname.\n2. If the next line starts with '^', it would continue to parse the\n peeled oid and check whether the last character is '\\n'.\n\nAs we decide to implement the ref consistency check for \"packed-refs\",\nlet's port these two checks and update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 ++\n fsck.h | 1 +\n refs/packed-backend.c | 95 ++++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 42 ++++++++++++++++\n 4 files changed, 140 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 11906f90fd..02a7bf0503 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -16,6 +16,9 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefEntry`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid entry.\n+\n `badPackedRefHeader`::\n \t(ERROR) The \"packed-refs\" file contains an invalid\n \theader.\ndiff --git a/fsck.h b/fsck.h\nindex 67e3c97bc0..14d70f6653 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\n \tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex c8bb93bb18..658f6bc7da 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1809,10 +1809,83 @@ static int packed_fsck_ref_header(struct fsck_options *o,\n \treturn 0;\n }\n \n+static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t struct strbuf *packed_entry,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id peeled;\n+\tconst char *p;\n+\n+\treport.path = packed_entry->buf;\n+\n+\t/*\n+\t * Skip the '^' and parse the peeled oid.\n+\t */\n+\tstart++;\n+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo))\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid peeled oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\n+\tif (p != eol)\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n+\t\t\t\t (int)(eol - p), p);\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_main_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t struct strbuf *packed_entry,\n+\t\t\t\t struct strbuf *refname,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id oid;\n+\tconst char *p;\n+\n+\treport.path = packed_entry->buf;\n+\n+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo))\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\n+\tif (p == eol || !isspace(*p))\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n+\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n+\n+\tp++;\n+\tstrbuf_reset(refname);\n+\tstrbuf_add(refname, p, eol - p);\n+\tif (refname_contains_nul(refname))\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"refname '%s' contains NULL binaries\",\n+\t\t\t\t refname->buf);\n+\n+\tif (check_refname_format(refname->buf, 0))\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n+\t\t\t\t \"has bad refname '%s'\", refname->buf);\n+\n+\treturn 0;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n \tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct strbuf refname = STRBUF_INIT;\n \tunsigned long line_number = 1;\n \tconst char *eol;\n \tint ret = 0;\n@@ -1826,6 +1899,26 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\tline_number++;\n \t}\n \n+\twhile (start < eof) {\n+\t\tstrbuf_reset(&packed_entry);\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n+\t\tret |= packed_fsck_ref_main_line(o, ref_store, &packed_entry, &refname, start, eol);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t\tif (start < eof && *start == '^') {\n+\t\t\tstrbuf_reset(&packed_entry);\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, &packed_entry,\n+\t\t\t\t\t\t\t start, eol);\n+\t\t\tstart = eol + 1;\n+\t\t\tline_number++;\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname);\n \tstrbuf_release(&packed_entry);\n \treturn ret;\n }\n@@ -1873,7 +1966,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex da321f16c6..3ab6b5bba5 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -664,4 +664,46 @@ test_expect_success 'packed-refs header should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-refs content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tgit tag -a annotated-tag-2 -m tag-2 &&\n+\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n+\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n+\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\t\tprintf \"%s\\n\" \"$short_oid refs/heads/branch-1\" >>.git/packed-refs &&\n+\t\tprintf \"%sx\\n\" \"$branch_1_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s refs/heads/bad-branch\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s refs/heads/branch.\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s refs/tags/annotated-tag-3\\n\" \"$tag_1_oid\" >>.git/packed-refs &&\n+\t\tprintf \"^%s\\n\" \"$short_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s refs/tags/annotated-tag-4.\\n\" \"$tag_2_oid\" >>.git/packed-refs &&\n+\t\tprintf \"^%s garbage\\n\" \"$tag_2_peeled_oid\" >>.git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: badPackedRefEntry: '\\''$short_oid refs/heads/branch-1'\\'' has invalid oid\n+\t\terror: packed-refs line 3: badPackedRefEntry: has no space after oid '\\''$branch_1_oid'\\'' but with '\\''x'\\''\n+\t\terror: packed-refs line 4: badRefName: has bad refname '\\'' refs/heads/bad-branch'\\''\n+\t\terror: packed-refs line 5: badRefName: has bad refname '\\''refs/heads/branch.'\\''\n+\t\terror: packed-refs line 7: badPackedRefEntry: '\\''$short_oid'\\'' has invalid peeled oid\n+\t\terror: packed-refs line 8: badRefName: has bad refname '\\''refs/tags/annotated-tag-4.'\\''\n+\t\terror: packed-refs line 9: badPackedRefEntry: has trailing garbage after peeled oid '\\'' garbage'\\''\n+\t\tEOF\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"511934","messageId":"Z6RP2_wL1gjsWpkR@ArchLinux","threadId":"62743","inReplyTo":"Z6RPJI10-2QkwyqH@ArchLinux","subject":"[PATCH v3 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-06T05:59:55Z","receivedAt":"2025-02-06T05:58:14Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"When there is a \"sorted\" trait in the header of the \"packed-refs\" file,\nit means that each entry is sorted increasingly by comparing the\nrefname. We should add checks to verify whether the \"packed-refs\" is\nsorted in this case.\n\nUpdate the \"packed_fsck_ref_header\" to know whether there is a \"sorted\"\ntrail in the header. Then, create a new structure \"fsck_packed_ref_entry\"\nto store the state during the parsing process for every entry. It may\nseem that we could just add a new \"struct strbuf refname\" into the\n\"struct fsck_packed_ref_entry\" and during the parsing process, we could\nstore the refname into this structure and thus we could compare later.\nHowever, this is not a good design due to the following reasons:\n\n1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n in the \"packed-refs\" file.\n2. The most important thing is that we cannot reuse the existing compare\n functions which cause repetition.\n\nSo, instead of storing the \"struct strbuf\", let's use the existing\nstructure \"struct snaphost_record\". And thus we could use the existing\nfunction \"cmp_packed_ref_records\".\n\nHowever, this function need an extra parameter for \"struct snaphost\".\nExtract the common part into a new function \"cmp_packed_ref_records\" to\nreuse this function to compare.\n\nThen, create a new function \"packed_fsck_ref_sorted\" to use the new fsck\nmessage \"packedRefUnsorted(ERROR)\" to report to the user.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 131 ++++++++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 63 ++++++++++++++++\n 4 files changed, 183 insertions(+), 15 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 02a7bf0503..9601fff228 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -187,6 +187,9 @@\n \t(ERROR) The \"packed-refs\" file contains an entry that is\n \tnot terminated by a newline.\n \n+`packedRefUnsorted`::\n+\t(ERROR) The \"packed-refs\" file is not sorted.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex 14d70f6653..19f3cb2773 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -56,6 +56,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n \tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n+\tFUNC(PACKED_REF_UNSORTED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 658f6bc7da..0fbdc5c3fa 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -300,14 +300,9 @@ struct snapshot_record {\n \tsize_t len;\n };\n \n-static int cmp_packed_ref_records(const void *v1, const void *v2,\n-\t\t\t\t void *cb_data)\n-{\n-\tconst struct snapshot *snapshot = cb_data;\n-\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n-\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n-\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n \n+static int cmp_packed_refname(const char *r1, const char *r2)\n+{\n \twhile (1) {\n \t\tif (*r1 == '\\n')\n \t\t\treturn *r2 == '\\n' ? 0 : -1;\n@@ -322,6 +317,17 @@ static int cmp_packed_ref_records(const void *v1, const void *v2,\n \t}\n }\n \n+static int cmp_packed_ref_records(const void *v1, const void *v2,\n+\t\t\t\t void *cb_data)\n+{\n+\tconst struct snapshot *snapshot = cb_data;\n+\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n+\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n+\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n+\n+\treturn cmp_packed_refname(r1, r2);\n+}\n+\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n@@ -1767,6 +1773,28 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+struct fsck_packed_ref_entry {\n+\tunsigned long line_number;\n+\n+\tstruct snapshot_record record;\n+};\n+\n+static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(unsigned long line_number,\n+\t\t\t\t\t\t\t\t const char *start)\n+{\n+\tstruct fsck_packed_ref_entry *entry = xcalloc(1, sizeof(*entry));\n+\tentry->line_number = line_number;\n+\tentry->record.start = start;\n+\treturn entry;\n+}\n+\n+static void free_fsck_packed_ref_entries(struct fsck_packed_ref_entry **entries, size_t nr)\n+{\n+\tfor (size_t i = 0; i < nr; i++)\n+\t\tfree(entries[i]);\n+\tfree(entries);\n+}\n+\n static int packed_fsck_ref_next_line(struct fsck_options *o,\n \t\t\t\t struct strbuf *packed_entry, const char *start,\n \t\t\t\t const char *eof, const char **eol)\n@@ -1794,19 +1822,33 @@ static int packed_fsck_ref_next_line(struct fsck_options *o,\n }\n \n static int packed_fsck_ref_header(struct fsck_options *o,\n-\t\t\t\t const char *start, const char *eol)\n+\t\t\t\t const char *start, const char *eol,\n+\t\t\t\t unsigned int *sorted)\n {\n-\tif (!starts_with(start, \"# pack-refs with:\")) {\n+\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n+\tchar *tmp_line;\n+\tint ret = 0;\n+\tchar *p;\n+\n+\ttmp_line = xmemdupz(start, eol - start);\n+\tif (!skip_prefix(tmp_line, \"# pack-refs with:\", (const char **)&p)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \t\treport.path = \"packed-refs.header\";\n \n-\t\treturn fsck_report_ref(o, &report,\n-\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n-\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n-\t\t\t\t (int)(eol - start), start);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n \t}\n \n-\treturn 0;\n+\tstring_list_split_in_place(&traits, p, \" \", -1);\n+\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n+\n+cleanup:\n+\tfree(tmp_line);\n+\tstring_list_clear(&traits, 0);\n+\treturn ret;\n }\n \n static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n@@ -1880,26 +1922,80 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n \treturn 0;\n }\n \n+static int packed_fsck_ref_sorted(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t struct fsck_packed_ref_entry **entries,\n+\t\t\t\t size_t nr)\n+{\n+\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct strbuf refname1 = STRBUF_INIT;\n+\tstruct strbuf refname2 = STRBUF_INIT;\n+\tint ret = 0;\n+\n+\tfor (size_t i = 1; i < nr; i++) {\n+\t\tconst char *r1 = entries[i - 1]->record.start + hexsz + 1;\n+\t\tconst char *r2 = entries[i]->record.start + hexsz + 1;\n+\n+\t\tif (cmp_packed_refname(r1, r2) >= 0) {\n+\t\t\tconst char *err_fmt =\n+\t\t\t\t\"refname '%s' is not less than next refname '%s'\";\n+\t\t\tconst char *eol;\n+\t\t\teol = memchr(entries[i - 1]->record.start, '\\n',\n+\t\t\t\t entries[i - 1]->record.len);\n+\t\t\tstrbuf_add(&refname1, r1, eol - r1);\n+\t\t\teol = memchr(entries[i]->record.start, '\\n',\n+\t\t\t\t entries[i]->record.len);\n+\t\t\tstrbuf_add(&refname2, r2, eol - r2);\n+\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\",\n+\t\t\t\t entries[i - 1]->line_number);\n+\t\t\treport.path = packed_entry.buf;\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_PACKED_REF_UNSORTED,\n+\t\t\t\t\t err_fmt, refname1.buf, refname2.buf);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname1);\n+\tstrbuf_release(&refname2);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n \tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_packed_ref_entry **entries;\n \tstruct strbuf refname = STRBUF_INIT;\n \tunsigned long line_number = 1;\n+\tunsigned int sorted = 0;\n+\tsize_t entry_alloc = 20;\n+\tsize_t entry_nr = 0;\n \tconst char *eol;\n \tint ret = 0;\n \n \tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n \tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n \tif (*start == '#') {\n-\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\t\tret |= packed_fsck_ref_header(o, start, eol, &sorted);\n \n \t\tstart = eol + 1;\n \t\tline_number++;\n \t}\n \n+\tALLOC_ARRAY(entries, entry_alloc);\n \twhile (start < eof) {\n+\t\tstruct fsck_packed_ref_entry *entry\n+\t\t\t= create_fsck_packed_ref_entry(line_number, start);\n+\n+\t\tALLOC_GROW(entries, entry_nr + 1, entry_alloc);\n+\t\tentries[entry_nr++] = entry;\n \t\tstrbuf_reset(&packed_entry);\n \t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n \t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n@@ -1915,11 +2011,16 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\tstart = eol + 1;\n \t\t\tline_number++;\n \t\t}\n+\t\tentry->record.len = start - entry->record.start;\n \t}\n \n+\tif (!ret && sorted)\n+\t\tret |= packed_fsck_ref_sorted(o, ref_store, entries, entry_nr);\n+\n \tstrbuf_release(&packed_entry);\n \tstrbuf_release(&refname);\n \tstrbuf_release(&packed_entry);\n+\tfree_fsck_packed_ref_entries(entries, entry_nr);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 3ab6b5bba5..adcb5c1bda 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -706,4 +706,67 @@ test_expect_success 'packed-refs content should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-ref with sorted trait should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname1'\\'' is not less than next refname '\\''$refname2'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$tag_1_oid\" \"$refname3\" >>.git/packed-refs &&\n+\t\tprintf \"^%s\\n\" \"$tag_1_peeled_oid\" >>.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname2\" >>.git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname3'\\'' is not less than next refname '\\''$refname2'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n+test_expect_success 'packed-ref without sorted trait should not be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\t\tprintf \"# pack-refs with: peeled fully-peeled \\n\" >.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n+\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"511935","messageId":"Z6RP50d7eRsKRCG6@ArchLinux","threadId":"62743","inReplyTo":"Z6RPJI10-2QkwyqH@ArchLinux","subject":"[PATCH v3 8/8] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-06T06:00:07Z","receivedAt":"2025-02-06T05:58:27Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"At now, we have already implemented the ref consistency checks for both\n\"files-backend\" and \"packed-backend\". Although we would check some\nredundant things, it won't cause trouble. So, let's integrate it into\nthe \"git-fsck(1)\" command to get feedback from the users. And also by\ncalling \"git refs verify\" in \"git-fsck(1)\", we make sure that the new\nadded checks don't break.\n\nIntroduce a new function \"fsck_refs\" that initializes and runs a child\nprocess to execute the \"git refs verify\" command. In order to provide\nthe user interface create a progress which makes the total task be 1.\nIt's hard to know how many loose refs we will check now. We might\nimprove this later.\n\nThen, introduce the option to allow the user to disable checking ref\ndatabase consistency. Put this function in the very first execution\nsequence of \"git-fsck(1)\" due to that we don't want the existing code of\n\"git-fsck(1)\" which would implicitly check the consistency of refs to\ndie the program.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/git-fsck.txt | 6 +++++-\n builtin/fsck.c | 33 ++++++++++++++++++++++++++++++++-\n 2 files changed, 37 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt\nindex 5b82e4605c..9bd433028f 100644\n--- a/Documentation/git-fsck.txt\n+++ b/Documentation/git-fsck.txt\n@@ -12,7 +12,7 @@ SYNOPSIS\n 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n-\t [--[no-]name-objects] [...]\n+\t [--[no-]name-objects] [--[no-]references] [...]\n \n DESCRIPTION\n -----------\n@@ -104,6 +104,10 @@ care about this output and want to speed it up further.\n \tprogress status even if the standard error stream is not\n \tdirected to a terminal.\n \n+--[no-]references::\n+\tControl whether to check the references database consistency\n+\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n+\n CONFIGURATION\n -------------\n \ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 7a4dcb0716..f4f395cfbd 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -50,6 +50,7 @@ static int verbose;\n static int show_progress = -1;\n static int show_dangling = 1;\n static int name_objects;\n+static int check_references = 1;\n #define ERROR_OBJECT 01\n #define ERROR_REACHABLE 02\n #define ERROR_PACK 04\n@@ -905,11 +906,37 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n \treturn res;\n }\n \n+static void fsck_refs(struct repository *r)\n+{\n+\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n+\tstruct progress *progress = NULL;\n+\n+\tif (show_progress)\n+\t\tprogress = start_progress(r, _(\"Checking ref database\"), 1);\n+\n+\tif (verbose)\n+\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n+\n+\tchild_process_init(&refs_verify);\n+\trefs_verify.git_cmd = 1;\n+\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n+\tif (verbose)\n+\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n+\tif (check_strict)\n+\t\tstrvec_push(&refs_verify.args, \"--strict\");\n+\n+\tif (run_command(&refs_verify))\n+\t\terrors_found |= ERROR_REFS;\n+\n+\tdisplay_progress(progress, 1);\n+\tstop_progress(&progress);\n+}\n+\n static char const * const fsck_usage[] = {\n \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n \t \" [--[no-]dangling] [--[no-]progress] [--connectivity-only]\\n\"\n-\t \" [--[no-]name-objects] [...]\"),\n+\t \" [--[no-]name-objects] [--[no-]references] [...]\"),\n \tNULL\n };\n \n@@ -928,6 +955,7 @@ static struct option fsck_opts[] = {\n \t\t\t\tN_(\"write dangling objects in .git/lost-found\")),\n \tOPT_BOOL(0, \"progress\", &show_progress, N_(\"show progress\")),\n \tOPT_BOOL(0, \"name-objects\", &name_objects, N_(\"show verbose names for reachable objects\")),\n+\tOPT_BOOL(0, \"references\", &check_references, N_(\"check reference database consistency\")),\n \tOPT_END(),\n };\n \n@@ -970,6 +998,9 @@ int cmd_fsck(int argc,\n \tgit_config(git_fsck_config, &fsck_obj_options);\n \tprepare_repo_settings(the_repository);\n \n+\tif (check_references)\n+\t\tfsck_refs(the_repository);\n+\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n \t\tfor_each_packed_object(the_repository,\n-- \n2.48.1\n\n"},{"id":"512302","messageId":"Z6xwW0N7GG4NhCzZ@pks.im","threadId":"62743","inReplyTo":"Z6RPqE64ScsjzUg7@ArchLinux","subject":"Re: [PATCH v3 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-12T09:56:43Z","receivedAt":"2025-02-12T09:56:52Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Feb 06, 2025 at 01:59:04PM +0800, shejialuo wrote:\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 6401cecd5f..683cfe78dc 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1749,12 +1749,76 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> +static int packed_fsck_ref_header(struct fsck_options *o,\n> +\t\t\t\t const char *start, const char *eol)\n> +{\n> +\tif (!starts_with(start, \"# pack-refs with:\")) {\n> +\t\tstruct fsck_ref_report report = { 0 };\n> +\t\treport.path = \"packed-refs.header\";\n> +\n> +\t\treturn fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n> +\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n> +\t\t\t\t (int)(eol - start), start);\n> +\t}\n> +\n> +\treturn 0;\n> +}\n\nOkay. We still complain about bad headers, but only if there is a line\nstarting with \"#\" and only if the prefix doesn't match. This addresses\nJunio's comment that packfiles don't have to have a header, and that\nthey may contain capabilities that we don't understand.\n\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 42c8d4ca1e..da321f16c6 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -639,4 +639,29 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> \t)\n> '\n> \n> +test_expect_success 'packed-refs header should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\ttest_commit default &&\n> +\n> +\t\tgit refs verify 2>err &&\n> +\t\ttest_must_be_empty err &&\n> +\n> +\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n> +\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n> +\t\t\t\t \"# pack-refs with a: peeled fully-peeled\"\n\nInstead of verifying thrice that we complain about bad header prefixes,\nshould we maybe replace two of these with instances where we check a\npacked-refs file _without_ a header and one with capabilities that we\ndon't understand?\n\nPatrick\n"},{"id":"512303","messageId":"Z6xwYlYFzbn3ft8u@pks.im","threadId":"62743","inReplyTo":"Z6RPzIGD-fSwIEPV@ArchLinux","subject":"Re: [PATCH v3 6/8] packed-backend: add \"packed-refs\" entry consistency check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-12T09:56:50Z","receivedAt":"2025-02-12T09:56:53Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Feb 06, 2025 at 01:59:40PM +0800, shejialuo wrote:\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index c8bb93bb18..658f6bc7da 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1826,6 +1899,26 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n> \t\tline_number++;\n> \t}\n> \n> +\twhile (start < eof) {\n> +\t\tstrbuf_reset(&packed_entry);\n> +\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n\nInstead of greedily computing the name of the line, can we pass in the\nline number? The motivation is that in a well-formatted packed-refs file\nwe won't ever need this string at all, so it's wasteful to proactively\ncompute it for every single line.\n\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index da321f16c6..3ab6b5bba5 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -664,4 +664,46 @@ test_expect_success 'packed-refs header should be checked' '\n> \t)\n> '\n> \n> +test_expect_success 'packed-refs content should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\ttest_commit default &&\n> +\t\tgit branch branch-1 &&\n> +\t\tgit branch branch-2 &&\n> +\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n> +\t\tgit tag -a annotated-tag-2 -m tag-2 &&\n> +\n> +\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n> +\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n> +\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n> +\t\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n> +\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n> +\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n> +\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n> +\n> +\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n> +\t\tprintf \"%s\\n\" \"$short_oid refs/heads/branch-1\" >>.git/packed-refs &&\n> +\t\tprintf \"%sx\\n\" \"$branch_1_oid\" >>.git/packed-refs &&\n> +\t\tprintf \"%s refs/heads/bad-branch\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n> +\t\tprintf \"%s refs/heads/branch.\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n> +\t\tprintf \"%s refs/tags/annotated-tag-3\\n\" \"$tag_1_oid\" >>.git/packed-refs &&\n> +\t\tprintf \"^%s\\n\" \"$short_oid\" >>.git/packed-refs &&\n> +\t\tprintf \"%s refs/tags/annotated-tag-4.\\n\" \"$tag_2_oid\" >>.git/packed-refs &&\n> +\t\tprintf \"^%s garbage\\n\" \"$tag_2_peeled_oid\" >>.git/packed-refs &&\n\nThis can be simplified using HERE docs.\n\n cat >.git/packed-refs <<-EOF\n # pack-refs with: peeled fully-peeled sorted \n $short_oid refs/heads/branch-1\n ${branch_1_oid}x\n $branch_2_oid refs/heads/bad-branch\n $branch_2_oid refs/heads/branch.\n $tag_1_oid refs/tags/annotated-tag-3\n ^$short_oid\\n\n $tag_2_oid refs/tags/annotated-tag-4.\n ^$tag_2_peeled_oid garbage\n EOF\n\nPatrick\n"},{"id":"512304","messageId":"Z6xwZaYLfmWUVNNR@pks.im","threadId":"62743","inReplyTo":"Z6RP50d7eRsKRCG6@ArchLinux","subject":"Re: [PATCH v3 8/8] builtin/fsck: add `git refs verify` child process","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-12T09:56:53Z","receivedAt":"2025-02-12T09:56:57Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Feb 06, 2025 at 02:00:07PM +0800, shejialuo wrote:\n> diff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt\n> index 5b82e4605c..9bd433028f 100644\n> --- a/Documentation/git-fsck.txt\n> +++ b/Documentation/git-fsck.txt\n> @@ -12,7 +12,7 @@ SYNOPSIS\n> 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n> \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n> \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n> -\t [--[no-]name-objects] [...]\n> +\t [--[no-]name-objects] [--[no-]references] [...]\n> \n> DESCRIPTION\n> -----------\n> @@ -104,6 +104,10 @@ care about this output and want to speed it up further.\n> \tprogress status even if the standard error stream is not\n> \tdirected to a terminal.\n> \n> +--[no-]references::\n> +\tControl whether to check the references database consistency\n> +\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n\nI think we should note the default, which is to check them.\n\nIt would also be nice to have a couple of tests to verify that the flag\ndoes what it is intended to do.\n\nPatrick\n"},{"id":"512305","messageId":"Z6xwaMIUx_x6QVrU@pks.im","threadId":"62743","inReplyTo":"Z6RP2_wL1gjsWpkR@ArchLinux","subject":"Re: [PATCH v3 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-12T09:56:56Z","receivedAt":"2025-02-12T09:57:00Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Feb 06, 2025 at 01:59:55PM +0800, shejialuo wrote:\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 658f6bc7da..0fbdc5c3fa 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1767,6 +1773,28 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> \treturn empty_ref_iterator_begin();\n> }\n> \n> +struct fsck_packed_ref_entry {\n> +\tunsigned long line_number;\n> +\n> +\tstruct snapshot_record record;\n> +};\n> +\n> +static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(unsigned long line_number,\n> +\t\t\t\t\t\t\t\t const char *start)\n> +{\n> +\tstruct fsck_packed_ref_entry *entry = xcalloc(1, sizeof(*entry));\n> +\tentry->line_number = line_number;\n> +\tentry->record.start = start;\n> +\treturn entry;\n> +}\n> +\n> +static void free_fsck_packed_ref_entries(struct fsck_packed_ref_entry **entries, size_t nr)\n> +{\n> +\tfor (size_t i = 0; i < nr; i++)\n> +\t\tfree(entries[i]);\n> +\tfree(entries);\n> +}\n> +\n> static int packed_fsck_ref_next_line(struct fsck_options *o,\n> \t\t\t\t struct strbuf *packed_entry, const char *start,\n> \t\t\t\t const char *eof, const char **eol)\n> @@ -1794,19 +1822,33 @@ static int packed_fsck_ref_next_line(struct fsck_options *o,\n> }\n> \n> static int packed_fsck_ref_header(struct fsck_options *o,\n> -\t\t\t\t const char *start, const char *eol)\n> +\t\t\t\t const char *start, const char *eol,\n> +\t\t\t\t unsigned int *sorted)\n> {\n> -\tif (!starts_with(start, \"# pack-refs with:\")) {\n> +\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n> +\tchar *tmp_line;\n> +\tint ret = 0;\n> +\tchar *p;\n> +\n> +\ttmp_line = xmemdupz(start, eol - start);\n> +\tif (!skip_prefix(tmp_line, \"# pack-refs with:\", (const char **)&p)) {\n> \t\tstruct fsck_ref_report report = { 0 };\n> \t\treport.path = \"packed-refs.header\";\n> \n> -\t\treturn fsck_report_ref(o, &report,\n> -\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n> -\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n> -\t\t\t\t (int)(eol - start), start);\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n> +\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n> +\t\t\t\t (int)(eol - start), start);\n> +\t\tgoto cleanup;\n> \t}\n> \n> -\treturn 0;\n> +\tstring_list_split_in_place(&traits, p, \" \", -1);\n> +\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n\nI think we call them capabilities, not traits.\n\n[snip]\n> static int packed_fsck_ref_content(struct fsck_options *o,\n> \t\t\t\t struct ref_store *ref_store,\n> \t\t\t\t const char *start, const char *eof)\n> {\n> \tstruct strbuf packed_entry = STRBUF_INIT;\n> +\tstruct fsck_packed_ref_entry **entries;\n> \tstruct strbuf refname = STRBUF_INIT;\n> \tunsigned long line_number = 1;\n> +\tunsigned int sorted = 0;\n> +\tsize_t entry_alloc = 20;\n> +\tsize_t entry_nr = 0;\n> \tconst char *eol;\n> \tint ret = 0;\n> \n> \tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n> \tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n> \tif (*start == '#') {\n> -\t\tret |= packed_fsck_ref_header(o, start, eol);\n> +\t\tret |= packed_fsck_ref_header(o, start, eol, &sorted);\n> \n> \t\tstart = eol + 1;\n> \t\tline_number++;\n> \t}\n> \n> +\tALLOC_ARRAY(entries, entry_alloc);\n> \twhile (start < eof) {\n> +\t\tstruct fsck_packed_ref_entry *entry\n> +\t\t\t= create_fsck_packed_ref_entry(line_number, start);\n\nInstead of slurping in all entries and allocating them in an array, can\nwe instead remember the last one and just compare that the last record\nis smaller than the current record?\n\n> @@ -1915,11 +2011,16 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n> \t\t\tstart = eol + 1;\n> \t\t\tline_number++;\n> \t\t}\n> +\t\tentry->record.len = start - entry->record.start;\n> \t}\n> \n> +\tif (!ret && sorted)\n> +\t\tret |= packed_fsck_ref_sorted(o, ref_store, entries, entry_nr);\n\nOkay, we now conditionally check whether the refs are sorted based on\nwhether or not we found the \"sorted\" capability.\n\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 3ab6b5bba5..adcb5c1bda 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -706,4 +706,67 @@ test_expect_success 'packed-refs content should be checked' '\n> \t)\n> '\n> \n> +test_expect_success 'packed-ref with sorted trait should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\ttest_commit default &&\n> +\t\tgit branch branch-1 &&\n> +\t\tgit branch branch-2 &&\n> +\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n> +\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n> +\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n> +\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n> +\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n> +\t\trefname1=\"refs/heads/main\" &&\n> +\t\trefname2=\"refs/heads/foo\" &&\n> +\t\trefname3=\"refs/tags/foo\" &&\n> +\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n> +\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n> +\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n\nSame comment here as in the previous patch, this can be simplified with\nHERE docs.\n\n> +\t\ttest_must_fail git refs verify 2>err &&\n> +\t\tcat >expect <<-EOF &&\n> +\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname1'\\'' is not less than next refname '\\''$refname2'\\''\n> +\t\tEOF\n> +\t\trm .git/packed-refs &&\n> +\t\ttest_cmp expect err &&\n> +\n> +\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n> +\t\tprintf \"%s %s\\n\" \"$tag_1_oid\" \"$refname3\" >>.git/packed-refs &&\n> +\t\tprintf \"^%s\\n\" \"$tag_1_peeled_oid\" >>.git/packed-refs &&\n> +\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname2\" >>.git/packed-refs &&\n> +\t\ttest_must_fail git refs verify 2>err &&\n> +\t\tcat >expect <<-EOF &&\n> +\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname3'\\'' is not less than next refname '\\''$refname2'\\''\n> +\t\tEOF\n> +\t\trm .git/packed-refs &&\n> +\t\ttest_cmp expect err\n> +\t)\n> +'\n> +\n> +test_expect_success 'packed-ref without sorted trait should not be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\ttest_commit default &&\n> +\t\tgit branch branch-1 &&\n> +\t\tgit branch branch-2 &&\n> +\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n> +\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n> +\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n> +\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n> +\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n> +\t\trefname1=\"refs/heads/main\" &&\n> +\t\trefname2=\"refs/heads/foo\" &&\n> +\t\trefname3=\"refs/tags/foo\" &&\n> +\t\tprintf \"# pack-refs with: peeled fully-peeled \\n\" >.git/packed-refs &&\n> +\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n> +\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n\nAnd here.\n\nPatrick\n"},{"id":"512306","messageId":"Z6x0AtbmCY4XJsGj@ArchLinux","threadId":"62743","inReplyTo":"Z6xwW0N7GG4NhCzZ@pks.im","subject":"Re: [PATCH v3 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-12T10:12:18Z","receivedAt":"2025-02-12T10:10:26Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Feb 12, 2025 at 10:56:43AM +0100, Patrick Steinhardt wrote:\n\n[snip]\n\n> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> > index 42c8d4ca1e..da321f16c6 100755\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -639,4 +639,29 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> > \t)\n> > '\n> > \n> > +test_expect_success 'packed-refs header should be checked' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\t(\n> > +\t\tcd repo &&\n> > +\t\ttest_commit default &&\n> > +\n> > +\t\tgit refs verify 2>err &&\n> > +\t\ttest_must_be_empty err &&\n> > +\n> > +\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n> > +\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n> > +\t\t\t\t \"# pack-refs with a: peeled fully-peeled\"\n> \n> Instead of verifying thrice that we complain about bad header prefixes,\n> should we maybe replace two of these with instances where we check a\n> packed-refs file _without_ a header and one with capabilities that we\n> don't understand?\n> \n\nI think we could add some tests to verify that we won't complain about\nabove two cases where packed-refs file without a header and one with\ncapabilities that we don't understand.\n\n> Patrick\n"},{"id":"512307","messageId":"Z6x1ZIsgYaLh71D2@ArchLinux","threadId":"62743","inReplyTo":"Z6xwYlYFzbn3ft8u@pks.im","subject":"Re: [PATCH v3 6/8] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-12T10:18:12Z","receivedAt":"2025-02-12T10:16:20Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Feb 12, 2025 at 10:56:50AM +0100, Patrick Steinhardt wrote:\n> On Thu, Feb 06, 2025 at 01:59:40PM +0800, shejialuo wrote:\n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index c8bb93bb18..658f6bc7da 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -1826,6 +1899,26 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n> > \t\tline_number++;\n> > \t}\n> > \n> > +\twhile (start < eof) {\n> > +\t\tstrbuf_reset(&packed_entry);\n> > +\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n> \n> Instead of greedily computing the name of the line, can we pass in the\n> line number? The motivation is that in a well-formatted packed-refs file\n> we won't ever need this string at all, so it's wasteful to proactively\n> compute it for every single line.\n> \n\nI agree with you here. And I already have idea to do this. Let me\nimprove this in the next version.\n\n> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> > index da321f16c6..3ab6b5bba5 100755\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -664,4 +664,46 @@ test_expect_success 'packed-refs header should be checked' '\n> > \t)\n> > '\n> > \n> > +test_expect_success 'packed-refs content should be checked' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\t(\n> > +\t\tcd repo &&\n> > +\t\ttest_commit default &&\n> > +\t\tgit branch branch-1 &&\n> > +\t\tgit branch branch-2 &&\n> > +\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n> > +\t\tgit tag -a annotated-tag-2 -m tag-2 &&\n> > +\n> > +\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n> > +\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n> > +\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n> > +\t\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n> > +\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n> > +\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n> > +\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n> > +\n> > +\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n> > +\t\tprintf \"%s\\n\" \"$short_oid refs/heads/branch-1\" >>.git/packed-refs &&\n> > +\t\tprintf \"%sx\\n\" \"$branch_1_oid\" >>.git/packed-refs &&\n> > +\t\tprintf \"%s refs/heads/bad-branch\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n> > +\t\tprintf \"%s refs/heads/branch.\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n> > +\t\tprintf \"%s refs/tags/annotated-tag-3\\n\" \"$tag_1_oid\" >>.git/packed-refs &&\n> > +\t\tprintf \"^%s\\n\" \"$short_oid\" >>.git/packed-refs &&\n> > +\t\tprintf \"%s refs/tags/annotated-tag-4.\\n\" \"$tag_2_oid\" >>.git/packed-refs &&\n> > +\t\tprintf \"^%s garbage\\n\" \"$tag_2_peeled_oid\" >>.git/packed-refs &&\n> \n> This can be simplified using HERE docs.\n> \n> cat >.git/packed-refs <<-EOF\n> # pack-refs with: peeled fully-peeled sorted \n> $short_oid refs/heads/branch-1\n> ${branch_1_oid}x\n> $branch_2_oid refs/heads/bad-branch\n> $branch_2_oid refs/heads/branch.\n> $tag_1_oid refs/tags/annotated-tag-3\n> ^$short_oid\\n\n> $tag_2_oid refs/tags/annotated-tag-4.\n> ^$tag_2_peeled_oid garbage\n> EOF\n> \n\nThanks for the suggestion, I will improve this in the next version.\n\n> Patrick\n"},{"id":"512308","messageId":"Z6x10U4HUwMYKUDh@ArchLinux","threadId":"62743","inReplyTo":"Z6xwaMIUx_x6QVrU@pks.im","subject":"Re: [PATCH v3 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-12T10:20:01Z","receivedAt":"2025-02-12T10:18:09Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Feb 12, 2025 at 10:56:56AM +0100, Patrick Steinhardt wrote:\n> On Thu, Feb 06, 2025 at 01:59:55PM +0800, shejialuo wrote:\n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index 658f6bc7da..0fbdc5c3fa 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -1767,6 +1773,28 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> > \treturn empty_ref_iterator_begin();\n> > }\n> > \n> > +struct fsck_packed_ref_entry {\n> > +\tunsigned long line_number;\n> > +\n> > +\tstruct snapshot_record record;\n> > +};\n> > +\n> > +static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(unsigned long line_number,\n> > +\t\t\t\t\t\t\t\t const char *start)\n> > +{\n> > +\tstruct fsck_packed_ref_entry *entry = xcalloc(1, sizeof(*entry));\n> > +\tentry->line_number = line_number;\n> > +\tentry->record.start = start;\n> > +\treturn entry;\n> > +}\n> > +\n> > +static void free_fsck_packed_ref_entries(struct fsck_packed_ref_entry **entries, size_t nr)\n> > +{\n> > +\tfor (size_t i = 0; i < nr; i++)\n> > +\t\tfree(entries[i]);\n> > +\tfree(entries);\n> > +}\n> > +\n> > static int packed_fsck_ref_next_line(struct fsck_options *o,\n> > \t\t\t\t struct strbuf *packed_entry, const char *start,\n> > \t\t\t\t const char *eof, const char **eol)\n> > @@ -1794,19 +1822,33 @@ static int packed_fsck_ref_next_line(struct fsck_options *o,\n> > }\n> > \n> > static int packed_fsck_ref_header(struct fsck_options *o,\n> > -\t\t\t\t const char *start, const char *eol)\n> > +\t\t\t\t const char *start, const char *eol,\n> > +\t\t\t\t unsigned int *sorted)\n> > {\n> > -\tif (!starts_with(start, \"# pack-refs with:\")) {\n> > +\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n> > +\tchar *tmp_line;\n> > +\tint ret = 0;\n> > +\tchar *p;\n> > +\n> > +\ttmp_line = xmemdupz(start, eol - start);\n> > +\tif (!skip_prefix(tmp_line, \"# pack-refs with:\", (const char **)&p)) {\n> > \t\tstruct fsck_ref_report report = { 0 };\n> > \t\treport.path = \"packed-refs.header\";\n> > \n> > -\t\treturn fsck_report_ref(o, &report,\n> > -\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n> > -\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n> > -\t\t\t\t (int)(eol - start), start);\n> > +\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n> > +\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n> > +\t\t\t\t (int)(eol - start), start);\n> > +\t\tgoto cleanup;\n> > \t}\n> > \n> > -\treturn 0;\n> > +\tstring_list_split_in_place(&traits, p, \" \", -1);\n> > +\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n> \n> I think we call them capabilities, not traits.\n> \n\nYes, capabilities will be more semantic. But the original code in\n\"packed-backend.c\" uses \"traits\". Let us follow the original style to\nmake sure consistency.\n\n> [snip]\n> > static int packed_fsck_ref_content(struct fsck_options *o,\n> > \t\t\t\t struct ref_store *ref_store,\n> > \t\t\t\t const char *start, const char *eof)\n> > {\n> > \tstruct strbuf packed_entry = STRBUF_INIT;\n> > +\tstruct fsck_packed_ref_entry **entries;\n> > \tstruct strbuf refname = STRBUF_INIT;\n> > \tunsigned long line_number = 1;\n> > +\tunsigned int sorted = 0;\n> > +\tsize_t entry_alloc = 20;\n> > +\tsize_t entry_nr = 0;\n> > \tconst char *eol;\n> > \tint ret = 0;\n> > \n> > \tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n> > \tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n> > \tif (*start == '#') {\n> > -\t\tret |= packed_fsck_ref_header(o, start, eol);\n> > +\t\tret |= packed_fsck_ref_header(o, start, eol, &sorted);\n> > \n> > \t\tstart = eol + 1;\n> > \t\tline_number++;\n> > \t}\n> > \n> > +\tALLOC_ARRAY(entries, entry_alloc);\n> > \twhile (start < eof) {\n> > +\t\tstruct fsck_packed_ref_entry *entry\n> > +\t\t\t= create_fsck_packed_ref_entry(line_number, start);\n> \n> Instead of slurping in all entries and allocating them in an array, can\n> we instead remember the last one and just compare that the last record\n> is smaller than the current record?\n> \n> > @@ -1915,11 +2011,16 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n> > \t\t\tstart = eol + 1;\n> > \t\t\tline_number++;\n> > \t\t}\n> > +\t\tentry->record.len = start - entry->record.start;\n> > \t}\n> > \n> > +\tif (!ret && sorted)\n> > +\t\tret |= packed_fsck_ref_sorted(o, ref_store, entries, entry_nr);\n> \n> Okay, we now conditionally check whether the refs are sorted based on\n> whether or not we found the \"sorted\" capability.\n> \n> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> > index 3ab6b5bba5..adcb5c1bda 100755\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -706,4 +706,67 @@ test_expect_success 'packed-refs content should be checked' '\n> > \t)\n> > '\n> > \n> > +test_expect_success 'packed-ref with sorted trait should be checked' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\t(\n> > +\t\tcd repo &&\n> > +\t\ttest_commit default &&\n> > +\t\tgit branch branch-1 &&\n> > +\t\tgit branch branch-2 &&\n> > +\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n> > +\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n> > +\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n> > +\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n> > +\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n> > +\t\trefname1=\"refs/heads/main\" &&\n> > +\t\trefname2=\"refs/heads/foo\" &&\n> > +\t\trefname3=\"refs/tags/foo\" &&\n> > +\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n> > +\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n> > +\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n> \n> Same comment here as in the previous patch, this can be simplified with\n> HERE docs.\n> \n> > +\t\ttest_must_fail git refs verify 2>err &&\n> > +\t\tcat >expect <<-EOF &&\n> > +\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname1'\\'' is not less than next refname '\\''$refname2'\\''\n> > +\t\tEOF\n> > +\t\trm .git/packed-refs &&\n> > +\t\ttest_cmp expect err &&\n> > +\n> > +\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n> > +\t\tprintf \"%s %s\\n\" \"$tag_1_oid\" \"$refname3\" >>.git/packed-refs &&\n> > +\t\tprintf \"^%s\\n\" \"$tag_1_peeled_oid\" >>.git/packed-refs &&\n> > +\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname2\" >>.git/packed-refs &&\n> > +\t\ttest_must_fail git refs verify 2>err &&\n> > +\t\tcat >expect <<-EOF &&\n> > +\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname3'\\'' is not less than next refname '\\''$refname2'\\''\n> > +\t\tEOF\n> > +\t\trm .git/packed-refs &&\n> > +\t\ttest_cmp expect err\n> > +\t)\n> > +'\n> > +\n> > +test_expect_success 'packed-ref without sorted trait should not be checked' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\t(\n> > +\t\tcd repo &&\n> > +\t\ttest_commit default &&\n> > +\t\tgit branch branch-1 &&\n> > +\t\tgit branch branch-2 &&\n> > +\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n> > +\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n> > +\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n> > +\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n> > +\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n> > +\t\trefname1=\"refs/heads/main\" &&\n> > +\t\trefname2=\"refs/heads/foo\" &&\n> > +\t\trefname3=\"refs/tags/foo\" &&\n> > +\t\tprintf \"# pack-refs with: peeled fully-peeled \\n\" >.git/packed-refs &&\n> > +\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n> > +\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n> \n> And here.\n> \n\nThanks, I will improve this in the next version.\n\n> Patrick\n"},{"id":"512309","messageId":"Z6x2EN3ZVikLh4ne@ArchLinux","threadId":"62743","inReplyTo":"Z6xwZaYLfmWUVNNR@pks.im","subject":"Re: [PATCH v3 8/8] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-12T10:21:04Z","receivedAt":"2025-02-12T10:19:12Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Feb 12, 2025 at 10:56:53AM +0100, Patrick Steinhardt wrote:\n> On Thu, Feb 06, 2025 at 02:00:07PM +0800, shejialuo wrote:\n> > diff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt\n> > index 5b82e4605c..9bd433028f 100644\n> > --- a/Documentation/git-fsck.txt\n> > +++ b/Documentation/git-fsck.txt\n> > @@ -12,7 +12,7 @@ SYNOPSIS\n> > 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n> > \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n> > \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n> > -\t [--[no-]name-objects] [...]\n> > +\t [--[no-]name-objects] [--[no-]references] [...]\n> > \n> > DESCRIPTION\n> > -----------\n> > @@ -104,6 +104,10 @@ care about this output and want to speed it up further.\n> > \tprogress status even if the standard error stream is not\n> > \tdirected to a terminal.\n> > \n> > +--[no-]references::\n> > +\tControl whether to check the references database consistency\n> > +\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n> \n> I think we should note the default, which is to check them.\n> \n\nOK, let me improve the documentation in the next version.\n\n> It would also be nice to have a couple of tests to verify that the flag\n> does what it is intended to do.\n> \n\nGood idea, we could test via trailing contents to do this. Let me\nimprove this.\n\n> Patrick\n"},{"id":"512310","messageId":"Z6x7DwERuCKET77c@pks.im","threadId":"62743","inReplyTo":"Z6x10U4HUwMYKUDh@ArchLinux","subject":"Re: [PATCH v3 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-12T10:42:23Z","receivedAt":"2025-02-12T10:42:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Feb 12, 2025 at 06:20:01PM +0800, shejialuo wrote:\n> On Wed, Feb 12, 2025 at 10:56:56AM +0100, Patrick Steinhardt wrote:\n> > On Thu, Feb 06, 2025 at 01:59:55PM +0800, shejialuo wrote:\n> > > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > > index 658f6bc7da..0fbdc5c3fa 100644\n> > > --- a/refs/packed-backend.c\n> > > +++ b/refs/packed-backend.c\n> > > -\treturn 0;\n> > > +\tstring_list_split_in_place(&traits, p, \" \", -1);\n> > > +\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n> > \n> > I think we call them capabilities, not traits.\n> > \n> \n> Yes, capabilities will be more semantic. But the original code in\n> \"packed-backend.c\" uses \"traits\". Let us follow the original style to\n> make sure consistency.\n\nInteresting, TIL. But yeah, in that case we should continue to call them\ntraits.\n\nPatrick\n"},{"id":"512311","messageId":"Z6x-ccfRlVJXkMMh@ArchLinux","threadId":"62743","inReplyTo":"Z6xwaMIUx_x6QVrU@pks.im","subject":"Re: [PATCH v3 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-12T10:56:49Z","receivedAt":"2025-02-12T10:54:58Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Feb 12, 2025 at 10:56:56AM +0100, Patrick Steinhardt wrote:\n> > static int packed_fsck_ref_content(struct fsck_options *o,\n> > \t\t\t\t struct ref_store *ref_store,\n> > \t\t\t\t const char *start, const char *eof)\n> > {\n> > \tstruct strbuf packed_entry = STRBUF_INIT;\n> > +\tstruct fsck_packed_ref_entry **entries;\n> > \tstruct strbuf refname = STRBUF_INIT;\n> > \tunsigned long line_number = 1;\n> > +\tunsigned int sorted = 0;\n> > +\tsize_t entry_alloc = 20;\n> > +\tsize_t entry_nr = 0;\n> > \tconst char *eol;\n> > \tint ret = 0;\n> > \n> > \tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n> > \tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n> > \tif (*start == '#') {\n> > -\t\tret |= packed_fsck_ref_header(o, start, eol);\n> > +\t\tret |= packed_fsck_ref_header(o, start, eol, &sorted);\n> > \n> > \t\tstart = eol + 1;\n> > \t\tline_number++;\n> > \t}\n> > \n> > +\tALLOC_ARRAY(entries, entry_alloc);\n> > \twhile (start < eof) {\n> > +\t\tstruct fsck_packed_ref_entry *entry\n> > +\t\t\t= create_fsck_packed_ref_entry(line_number, start);\n> \n> Instead of slurping in all entries and allocating them in an array, can\n> we instead remember the last one and just compare that the last record\n> is smaller than the current record?\n> \n\nSorry here, I have missed out this. Actually, the way you say is the\nmost efficient way to check whether the \"packed-refs\" is sorted.\nHowever, there is a concern. When we check each ref entry, we could\ncompare the refname with previous refname. But I don't want to do this\ndue to the reason that I don't want to mix up these two checks. To\nconclude, we have the following call sequences which are independent.\n\n1. check ref entry consistency. (oid, refnames, format...)\n2. check whether the \"packed-refs\" is sorted.\n\nBut I do agree with your concern. The reason why I record them is that I\nthink we have already parsed the file, I think there is no need to parse\nit again. So, I use a way to record the information needed to check. And\nthis would definitely introduce memory burden.\n\nSo we have two choices:\n\n1. Keep the design unchanged (space overhead).\n2. Parse the file again (time overhead). Thus we only have two allocated\nmemory.\n\nFrom my writing, I think 2 will be better. If there are many entries, we\nwould allocate too much memory.\n\nLet me improve this.\n\nThanks,\nJialuo\n"},{"id":"512325","messageId":"xmqq4j0zuk9y.fsf@gitster.g","threadId":"62743","inReplyTo":"Z6xwW0N7GG4NhCzZ@pks.im","subject":"Re: [PATCH v3 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-12T17:48:09Z","receivedAt":"2025-02-12T17:48:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Thu, Feb 06, 2025 at 01:59:04PM +0800, shejialuo wrote:\n>> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n>> index 6401cecd5f..683cfe78dc 100644\n>> --- a/refs/packed-backend.c\n>> +++ b/refs/packed-backend.c\n>> @@ -1749,12 +1749,76 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n>> +static int packed_fsck_ref_header(struct fsck_options *o,\n>> +\t\t\t\t const char *start, const char *eol)\n>> +{\n>> +\tif (!starts_with(start, \"# pack-refs with:\")) {\n>> +\t\tstruct fsck_ref_report report = { 0 };\n>> +\t\treport.path = \"packed-refs.header\";\n>> +\n>> +\t\treturn fsck_report_ref(o, &report,\n>> +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n>> +\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n>> +\t\t\t\t (int)(eol - start), start);\n>> +\t}\n>> +\n>> +\treturn 0;\n>> +}\n>\n> Okay. We still complain about bad headers, but only if there is a line\n> starting with \"#\" and only if the prefix doesn't match. This addresses\n> Junio's comment that packfiles don't have to have a header, and that\n> they may contain capabilities that we don't understand.\n\nWe'd want to also ensure that there is a single trailing whitespace\nafter that colon, which we have always written after \"with:\", no?\n\n>> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n>> index 42c8d4ca1e..da321f16c6 100755\n>> --- a/t/t0602-reffiles-fsck.sh\n>> +++ b/t/t0602-reffiles-fsck.sh\n>> @@ -639,4 +639,29 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n>> \t)\n>> '\n>> \n>> +test_expect_success 'packed-refs header should be checked' '\n>> +\ttest_when_finished \"rm -rf repo\" &&\n>> +\tgit init repo &&\n>> +\t(\n>> +\t\tcd repo &&\n>> +\t\ttest_commit default &&\n>> +\n>> +\t\tgit refs verify 2>err &&\n>> +\t\ttest_must_be_empty err &&\n>> +\n>> +\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n>> +\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n>> +\t\t\t\t \"# pack-refs with a: peeled fully-peeled\"\n>\n> Instead of verifying thrice that we complain about bad header prefixes,\n> should we maybe replace two of these with instances where we check a\n> packed-refs file _without_ a header and one with capabilities that we\n> don't understand?\n\nYup. I also notice that refs/packed-backend.c:create_snapshot()\nwould accept \"# pack-refs with:peeled\" if I am not reading it\ncorrectly, which is an unrelated bug.\n\nThanks.\n"},{"id":"512392","messageId":"Z66-KHC4SP4yqmnW@ArchLinux","threadId":"62743","inReplyTo":"xmqq4j0zuk9y.fsf@gitster.g","subject":"Re: [PATCH v3 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T03:53:12Z","receivedAt":"2025-02-14T03:53:19Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Feb 12, 2025 at 09:48:09AM -0800, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> > On Thu, Feb 06, 2025 at 01:59:04PM +0800, shejialuo wrote:\n> >> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> >> index 6401cecd5f..683cfe78dc 100644\n> >> --- a/refs/packed-backend.c\n> >> +++ b/refs/packed-backend.c\n> >> @@ -1749,12 +1749,76 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> >> +static int packed_fsck_ref_header(struct fsck_options *o,\n> >> +\t\t\t\t const char *start, const char *eol)\n> >> +{\n> >> +\tif (!starts_with(start, \"# pack-refs with:\")) {\n> >> +\t\tstruct fsck_ref_report report = { 0 };\n> >> +\t\treport.path = \"packed-refs.header\";\n> >> +\n> >> +\t\treturn fsck_report_ref(o, &report,\n> >> +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n> >> +\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n> >> +\t\t\t\t (int)(eol - start), start);\n> >> +\t}\n> >> +\n> >> +\treturn 0;\n> >> +}\n> >\n> > Okay. We still complain about bad headers, but only if there is a line\n> > starting with \"#\" and only if the prefix doesn't match. This addresses\n> > Junio's comment that packfiles don't have to have a header, and that\n> > they may contain capabilities that we don't understand.\n> \n> We'd want to also ensure that there is a single trailing whitespace\n> after that colon, which we have always written after \"with:\", no?\n> \n\nAs you have commented below, I don't add this check due to the reason\nthat \"create_snapshot\" method does _not_ check this.\n\n> >> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> >> index 42c8d4ca1e..da321f16c6 100755\n> >> --- a/t/t0602-reffiles-fsck.sh\n> >> +++ b/t/t0602-reffiles-fsck.sh\n> >> @@ -639,4 +639,29 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> >> \t)\n> >> '\n> >> \n> >> +test_expect_success 'packed-refs header should be checked' '\n> >> +\ttest_when_finished \"rm -rf repo\" &&\n> >> +\tgit init repo &&\n> >> +\t(\n> >> +\t\tcd repo &&\n> >> +\t\ttest_commit default &&\n> >> +\n> >> +\t\tgit refs verify 2>err &&\n> >> +\t\ttest_must_be_empty err &&\n> >> +\n> >> +\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n> >> +\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n> >> +\t\t\t\t \"# pack-refs with a: peeled fully-peeled\"\n> >\n> > Instead of verifying thrice that we complain about bad header prefixes,\n> > should we maybe replace two of these with instances where we check a\n> > packed-refs file _without_ a header and one with capabilities that we\n> > don't understand?\n> \n> Yup. I also notice that refs/packed-backend.c:create_snapshot()\n> would accept \"# pack-refs with:peeled\" if I am not reading it\n> correctly, which is an unrelated bug.\n> \n\nYes, you are correct. Let me fix this in the next version.\n\nThanks,\nJialuo\n"},{"id":"512395","messageId":"Z67LkxAFIAeaYr0U@ArchLinux","threadId":"62743","inReplyTo":"Z6RPJI10-2QkwyqH@ArchLinux","subject":"[PATCH v4 0/8] add more ref consistency checks","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T04:50:27Z","receivedAt":"2025-02-14T04:50:35Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis patch enhances the following things:\n\n1. [PATCH v4 4/8]: update the tests to verify that we don't report any\n errors to the user in some cases. Also, suggested by Junio, make sure\n that we check whether there is a trailing space after \"# packed-refs\n with:\".\n2. [PATCH v4 6/8]: instead of greedily calculating the name of the line,\n lazily compute when there is any errors. And use the HERE docs to\n improve the test script.\n3. [PATCH v4 7/8]: instead of storing the states, we parse the file\n again to check whether the file is sorted to avoid allocating too\n much memory. And use the HERE docs to improve the test script.\n4. [PATCH v4 8/8]: update the documentation to emphasis the default. And\n add tests to exercise the code.\n\nshejialuo (8):\n t0602: use subshell to ensure working directory unchanged\n builtin/refs: get worktrees without reading head information\n packed-backend: check whether the \"packed-refs\" is regular file\n packed-backend: add \"packed-refs\" header consistency check\n packed-backend: check whether the refname contains NUL characters\n packed-backend: add \"packed-refs\" entry consistency check\n packed-backend: check whether the \"packed-refs\" is sorted\n builtin/fsck: add `git refs verify` child process\n\n Documentation/fsck-msgids.txt | 14 +\n Documentation/git-fsck.txt | 7 +-\n builtin/fsck.c | 33 +-\n builtin/refs.c | 2 +-\n fsck.h | 4 +\n refs/packed-backend.c | 349 +++++++++-\n t/t0602-reffiles-fsck.sh | 1205 ++++++++++++++++++++-------------\n worktree.c | 5 +\n worktree.h | 6 +\n 9 files changed, 1140 insertions(+), 485 deletions(-)\n\nRange-diff against v3:\n1: 20889b7b18 = 1: 20889b7b18 t0602: use subshell to ensure working directory unchanged\n2: 9d7780e953 = 2: 9d7780e953 builtin/refs: get worktrees without reading head information\n3: 44d26f6440 = 3: 44d26f6440 packed-backend: check whether the \"packed-refs\" is regular file\n4: a9ab7af16a ! 4: 976c5baba0 packed-backend: add \"packed-refs\" header consistency check\n @@ Commit message\n \n In \"packed-backend.c::create_snapshot\", if there is a header (the line\n which starts with '#'), we will check whether the line starts with \"#\n - pack-refs with:\". As we are going to implement the header consistency\n - check, we should port this check into \"packed_fsck\".\n + pack-refs with:\". Before we port this check into \"packed_fsck\", let's\n + fix \"create_snapshot\" to check the prefix \"# packed-ref with: \" instead\n + of \"# packed-ref with:\" due to that we will always write a single\n + trailing space after the colon.\n \n However, we need to consider other situations and discuss whether we\n need to add checks.\n @@ Commit message\n user. This is because in older Git version, we never write header in\n the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n in runtime.\n - 2. If the header content does not start with \"# packed-ref with:\", we\n + 2. If the header content does not start with \"# packed-ref with: \", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n 3. If the header content is not the same as the constant string\n @@ fsck.h: enum fsck_msg_type {\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\n \n ## refs/packed-backend.c ##\n +@@ refs/packed-backend.c: static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n + \n + \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n + \n +-\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n ++\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n + \t\t\tdie_invalid_line(refs->path,\n + \t\t\t\t\t snapshot->buf,\n + \t\t\t\t\t snapshot->eof - snapshot->buf);\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n +static int packed_fsck_ref_next_line(struct fsck_options *o,\n -+\t\t\t\t struct strbuf *packed_entry, const char *start,\n ++\t\t\t\t unsigned long line_number, const char *start,\n +\t\t\t\t const char *eof, const char **eol)\n +{\n +\tint ret = 0;\n +\n +\t*eol = memchr(start, '\\n', eof - start);\n +\tif (!*eol) {\n ++\t\tstruct strbuf packed_entry = STRBUF_INIT;\n +\t\tstruct fsck_ref_report report = { 0 };\n +\n -+\t\treport.path = packed_entry->buf;\n ++\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n ++\t\treport.path = packed_entry.buf;\n +\t\tret = fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n +\t\t\t\t \"'%.*s' is not terminated with a newline\",\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\t\t * the buffer.\n +\t\t */\n +\t\t*eol = eof;\n ++\t\tstrbuf_release(&packed_entry);\n +\t}\n +\n +\treturn ret;\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +static int packed_fsck_ref_header(struct fsck_options *o,\n +\t\t\t\t const char *start, const char *eol)\n +{\n -+\tif (!starts_with(start, \"# pack-refs with:\")) {\n ++\tif (!starts_with(start, \"# pack-refs with: \")) {\n +\t\tstruct fsck_ref_report report = { 0 };\n +\t\treport.path = \"packed-refs.header\";\n +\n +\t\treturn fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n -+\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n ++\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n +\t\t\t\t (int)(eol - start), start);\n +\t}\n +\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +static int packed_fsck_ref_content(struct fsck_options *o,\n +\t\t\t\t const char *start, const char *eof)\n +{\n -+\tstruct strbuf packed_entry = STRBUF_INIT;\n +\tunsigned long line_number = 1;\n +\tconst char *eol;\n +\tint ret = 0;\n +\n -+\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n -+\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n ++\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n +\tif (*start == '#') {\n +\t\tret |= packed_fsck_ref_header(o, start, eol);\n +\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\t\tline_number++;\n +\t}\n +\n -+\tstrbuf_release(&packed_entry);\n +\treturn ret;\n +}\n +\n @@ t/t0602-reffiles-fsck.sh: test_expect_success SYMLINKS 'the filetype of packed-r\n +\n +\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n +\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n -+\t\t\t\t \"# pack-refs with a: peeled fully-peeled\"\n ++\t\t\t\t \"# pack-refs with a: peeled fully-peeled\" \\\n ++\t\t\t\t \"# pack-refs with:peeled fully-peeled sorted\"\n +\t\tdo\n +\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n +\t\t\ttest_must_fail git refs verify 2>err &&\n +\t\t\tcat >expect <<-EOF &&\n -+\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with:'\\''\n ++\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with: '\\''\n +\t\t\tEOF\n +\t\t\trm .git/packed-refs &&\n +\t\t\ttest_cmp expect err || return 1\n +\t\tdone\n +\t)\n +'\n ++\n ++test_expect_success 'packed-refs missing header should not be reported' '\n ++\ttest_when_finished \"rm -rf repo\" &&\n ++\tgit init repo &&\n ++\t(\n ++\t\tcd repo &&\n ++\t\ttest_commit default &&\n ++\n ++\t\tprintf \"$(git rev-parse HEAD) refs/heads/main\\n\" >.git/packed-refs &&\n ++\t\tgit refs verify 2>err &&\n ++\t\ttest_must_be_empty err\n ++\t)\n ++'\n ++\n ++test_expect_success 'packed-refs unknown traits should not be reported' '\n ++\ttest_when_finished \"rm -rf repo\" &&\n ++\tgit init repo &&\n ++\t(\n ++\t\tcd repo &&\n ++\t\ttest_commit default &&\n ++\n ++\t\tprintf \"# pack-refs with: peeled fully-peeled sorted foo\\n\" >.git/packed-refs &&\n ++\t\tgit refs verify 2>err &&\n ++\t\ttest_must_be_empty err\n ++\t)\n ++'\n +\n test_done\n5: 9b075434a1 = 5: b66f142d7f packed-backend: check whether the refname contains NUL characters\n6: a976508319 ! 6: f68028e171 packed-backend: add \"packed-refs\" entry consistency check\n @@ refs/packed-backend.c: static int packed_fsck_ref_header(struct fsck_options *o,\n \n +static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n +\t\t\t\t struct ref_store *ref_store,\n -+\t\t\t\t struct strbuf *packed_entry,\n ++\t\t\t\t unsigned long line_number,\n +\t\t\t\t const char *start, const char *eol)\n +{\n ++\tstruct strbuf packed_entry = STRBUF_INIT;\n +\tstruct fsck_ref_report report = { 0 };\n +\tstruct object_id peeled;\n +\tconst char *p;\n -+\n -+\treport.path = packed_entry->buf;\n ++\tint ret = 0;\n +\n +\t/*\n +\t * Skip the '^' and parse the peeled oid.\n +\t */\n +\tstart++;\n -+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo))\n -+\t\treturn fsck_report_ref(o, &report,\n -+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n -+\t\t\t\t \"'%.*s' has invalid peeled oid\",\n -+\t\t\t\t (int)(eol - start), start);\n -+\n -+\tif (p != eol)\n -+\t\treturn fsck_report_ref(o, &report,\n -+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n -+\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n -+\t\t\t\t (int)(eol - p), p);\n -+\n -+\treturn 0;\n ++\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n ++\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n ++\t\treport.path = packed_entry.buf;\n ++\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n ++\t\t\t\t \"'%.*s' has invalid peeled oid\",\n ++\t\t\t\t (int)(eol - start), start);\n ++\t\tgoto cleanup;\n ++\t}\n ++\n ++\tif (p != eol) {\n ++\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n ++\t\treport.path = packed_entry.buf;\n ++\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n ++\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n ++\t\t\t\t (int)(eol - p), p);\n ++\t\tgoto cleanup;\n ++\t}\n ++cleanup:\n ++\tstrbuf_release(&packed_entry);\n ++\treturn ret;\n +}\n +\n +static int packed_fsck_ref_main_line(struct fsck_options *o,\n +\t\t\t\t struct ref_store *ref_store,\n -+\t\t\t\t struct strbuf *packed_entry,\n ++\t\t\t\t unsigned long line_number,\n +\t\t\t\t struct strbuf *refname,\n +\t\t\t\t const char *start, const char *eol)\n +{\n ++\tstruct strbuf packed_entry = STRBUF_INIT;\n +\tstruct fsck_ref_report report = { 0 };\n +\tstruct object_id oid;\n +\tconst char *p;\n ++\tint ret = 0;\n +\n -+\treport.path = packed_entry->buf;\n ++\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n ++\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n ++\t\treport.path = packed_entry.buf;\n +\n -+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo))\n -+\t\treturn fsck_report_ref(o, &report,\n -+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n -+\t\t\t\t \"'%.*s' has invalid oid\",\n -+\t\t\t\t (int)(eol - start), start);\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n ++\t\t\t\t \"'%.*s' has invalid oid\",\n ++\t\t\t\t (int)(eol - start), start);\n ++\t\tgoto cleanup;\n ++\t}\n +\n -+\tif (p == eol || !isspace(*p))\n -+\t\treturn fsck_report_ref(o, &report,\n -+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n -+\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n -+\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n ++\tif (p == eol || !isspace(*p)) {\n ++\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n ++\t\treport.path = packed_entry.buf;\n ++\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n ++\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n ++\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n ++\t\tgoto cleanup;\n ++\t}\n +\n +\tp++;\n +\tstrbuf_reset(refname);\n +\tstrbuf_add(refname, p, eol - p);\n -+\tif (refname_contains_nul(refname))\n -+\t\treturn fsck_report_ref(o, &report,\n -+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n -+\t\t\t\t \"refname '%s' contains NULL binaries\",\n -+\t\t\t\t refname->buf);\n -+\n -+\tif (check_refname_format(refname->buf, 0))\n -+\t\treturn fsck_report_ref(o, &report,\n -+\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n -+\t\t\t\t \"has bad refname '%s'\", refname->buf);\n -+\n -+\treturn 0;\n ++\tif (refname_contains_nul(refname)) {\n ++\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n ++\t\treport.path = packed_entry.buf;\n ++\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n ++\t\t\t\t \"refname '%s' contains NULL binaries\",\n ++\t\t\t\t refname->buf);\n ++\t}\n ++\n ++\tif (check_refname_format(refname->buf, 0)) {\n ++\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n ++\t\treport.path = packed_entry.buf;\n ++\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n ++\t\t\t\t \"has bad refname '%s'\", refname->buf);\n ++\t}\n ++\n ++cleanup:\n ++\tstrbuf_release(&packed_entry);\n ++\treturn ret;\n +}\n +\n static int packed_fsck_ref_content(struct fsck_options *o,\n +\t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n - \tstruct strbuf packed_entry = STRBUF_INIT;\n +\tstruct strbuf refname = STRBUF_INIT;\n \tunsigned long line_number = 1;\n \tconst char *eol;\n @@ refs/packed-backend.c: static int packed_fsck_ref_content(struct fsck_options *o\n \t}\n \n +\twhile (start < eof) {\n -+\t\tstrbuf_reset(&packed_entry);\n -+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n -+\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n -+\t\tret |= packed_fsck_ref_main_line(o, ref_store, &packed_entry, &refname, start, eol);\n ++\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n ++\t\tret |= packed_fsck_ref_main_line(o, ref_store, line_number, &refname, start, eol);\n +\t\tstart = eol + 1;\n +\t\tline_number++;\n +\t\tif (start < eof && *start == '^') {\n -+\t\t\tstrbuf_reset(&packed_entry);\n -+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n -+\t\t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n -+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, &packed_entry,\n ++\t\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n ++\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, line_number,\n +\t\t\t\t\t\t\t start, eol);\n +\t\t\tstart = eol + 1;\n +\t\t\tline_number++;\n +\t\t}\n +\t}\n +\n -+\tstrbuf_release(&packed_entry);\n +\tstrbuf_release(&refname);\n - \tstrbuf_release(&packed_entry);\n \treturn ret;\n }\n + \n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n cleanup:\n \n ## t/t0602-reffiles-fsck.sh ##\n -@@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs header should be checked' '\n +@@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs unknown traits should not be reported' '\n \t)\n '\n \n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs header should be chec\n +\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n +\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n +\n -+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n -+\t\tprintf \"%s\\n\" \"$short_oid refs/heads/branch-1\" >>.git/packed-refs &&\n -+\t\tprintf \"%sx\\n\" \"$branch_1_oid\" >>.git/packed-refs &&\n -+\t\tprintf \"%s refs/heads/bad-branch\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n -+\t\tprintf \"%s refs/heads/branch.\\n\" \"$branch_2_oid\" >>.git/packed-refs &&\n -+\t\tprintf \"%s refs/tags/annotated-tag-3\\n\" \"$tag_1_oid\" >>.git/packed-refs &&\n -+\t\tprintf \"^%s\\n\" \"$short_oid\" >>.git/packed-refs &&\n -+\t\tprintf \"%s refs/tags/annotated-tag-4.\\n\" \"$tag_2_oid\" >>.git/packed-refs &&\n -+\t\tprintf \"^%s garbage\\n\" \"$tag_2_peeled_oid\" >>.git/packed-refs &&\n ++\t\tcat >.git/packed-refs <<-EOF &&\n ++\t\t# pack-refs with: peeled fully-peeled sorted\n ++\t\t$short_oid refs/heads/branch-1\n ++\t\t${branch_1_oid}x\n ++\t\t$branch_2_oid refs/heads/bad-branch\n ++\t\t$branch_2_oid refs/heads/branch.\n ++\t\t$tag_1_oid refs/tags/annotated-tag-3\n ++\t\t^$short_oid\n ++\t\t$tag_2_oid refs/tags/annotated-tag-4.\n ++\t\t^$tag_2_peeled_oid garbage\n ++\t\tEOF\n +\t\ttest_must_fail git refs verify 2>err &&\n +\t\tcat >expect <<-EOF &&\n +\t\terror: packed-refs line 2: badPackedRefEntry: '\\''$short_oid refs/heads/branch-1'\\'' has invalid oid\n7: 707e3e2151 ! 7: 4a7adf293f packed-backend: check whether the \"packed-refs\" is sorted\n @@ Commit message\n sorted in this case.\n \n Update the \"packed_fsck_ref_header\" to know whether there is a \"sorted\"\n - trail in the header. Then, create a new structure \"fsck_packed_ref_entry\"\n - to store the state during the parsing process for every entry. It may\n - seem that we could just add a new \"struct strbuf refname\" into the\n - \"struct fsck_packed_ref_entry\" and during the parsing process, we could\n - store the refname into this structure and thus we could compare later.\n - However, this is not a good design due to the following reasons:\n + trail in the header. It may seem that we could record all refnames\n + during the parsing process and then compare later. However, this is not\n + a good design due to the following reasons:\n \n 1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n in the \"packed-refs\" file.\n - 2. The most important thing is that we cannot reuse the existing compare\n - functions which cause repetition.\n + 2. We cannot reuse the existing compare function \"cmp_packed_ref_records\"\n + which cause repetition.\n \n - So, instead of storing the \"struct strbuf\", let's use the existing\n - structure \"struct snaphost_record\". And thus we could use the existing\n - function \"cmp_packed_ref_records\".\n + Because \"cmp_packed_ref_records\" needs an extra parameter \"struct\n + snaphost\", extract the common part into a new function\n + \"cmp_packed_ref_records\" to reuse this function to compare.\n \n - However, this function need an extra parameter for \"struct snaphost\".\n - Extract the common part into a new function \"cmp_packed_ref_records\" to\n - reuse this function to compare.\n -\n - Then, create a new function \"packed_fsck_ref_sorted\" to use the new fsck\n - message \"packedRefUnsorted(ERROR)\" to report to the user.\n + Then, create a new function \"packed_fsck_ref_sorted\" to parse the file\n + again and user the new fsck message \"packedRefUnsorted(ERROR)\" to report\n + to the user if the file is not sorted.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n @@ refs/packed-backend.c: static int cmp_packed_ref_records(const void *v1, const v\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n -@@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n - \treturn empty_ref_iterator_begin();\n - }\n - \n -+struct fsck_packed_ref_entry {\n -+\tunsigned long line_number;\n -+\n -+\tstruct snapshot_record record;\n -+};\n -+\n -+static struct fsck_packed_ref_entry *create_fsck_packed_ref_entry(unsigned long line_number,\n -+\t\t\t\t\t\t\t\t const char *start)\n -+{\n -+\tstruct fsck_packed_ref_entry *entry = xcalloc(1, sizeof(*entry));\n -+\tentry->line_number = line_number;\n -+\tentry->record.start = start;\n -+\treturn entry;\n -+}\n -+\n -+static void free_fsck_packed_ref_entries(struct fsck_packed_ref_entry **entries, size_t nr)\n -+{\n -+\tfor (size_t i = 0; i < nr; i++)\n -+\t\tfree(entries[i]);\n -+\tfree(entries);\n -+}\n -+\n - static int packed_fsck_ref_next_line(struct fsck_options *o,\n - \t\t\t\t struct strbuf *packed_entry, const char *start,\n - \t\t\t\t const char *eof, const char **eol)\n @@ refs/packed-backend.c: static int packed_fsck_ref_next_line(struct fsck_options *o,\n }\n \n @@ refs/packed-backend.c: static int packed_fsck_ref_next_line(struct fsck_options\n +\t\t\t\t const char *start, const char *eol,\n +\t\t\t\t unsigned int *sorted)\n {\n --\tif (!starts_with(start, \"# pack-refs with:\")) {\n +-\tif (!starts_with(start, \"# pack-refs with: \")) {\n +\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n +\tchar *tmp_line;\n +\tint ret = 0;\n +\tchar *p;\n +\n +\ttmp_line = xmemdupz(start, eol - start);\n -+\tif (!skip_prefix(tmp_line, \"# pack-refs with:\", (const char **)&p)) {\n ++\tif (!skip_prefix(tmp_line, \"# pack-refs with: \", (const char **)&p)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \t\treport.path = \"packed-refs.header\";\n \n -\t\treturn fsck_report_ref(o, &report,\n -\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n --\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n +-\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n -\t\t\t\t (int)(eol - start), start);\n +\t\tret = fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n -+\t\t\t\t \"'%.*s' does not start with '# pack-refs with:'\",\n ++\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n +\t\t\t\t (int)(eol - start), start);\n +\t\tgoto cleanup;\n \t}\n @@ refs/packed-backend.c: static int packed_fsck_ref_next_line(struct fsck_options\n \n static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n @@ refs/packed-backend.c: static int packed_fsck_ref_main_line(struct fsck_options *o,\n - \treturn 0;\n + \treturn ret;\n }\n \n +static int packed_fsck_ref_sorted(struct fsck_options *o,\n +\t\t\t\t struct ref_store *ref_store,\n -+\t\t\t\t struct fsck_packed_ref_entry **entries,\n -+\t\t\t\t size_t nr)\n ++\t\t\t\t const char *start, const char *eof)\n +{\n +\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n +\tstruct strbuf packed_entry = STRBUF_INIT;\n +\tstruct fsck_ref_report report = { 0 };\n +\tstruct strbuf refname1 = STRBUF_INIT;\n +\tstruct strbuf refname2 = STRBUF_INIT;\n ++\tunsigned long line_number = 1;\n ++\tconst char *former = NULL;\n ++\tconst char *current;\n ++\tconst char *eol;\n +\tint ret = 0;\n +\n -+\tfor (size_t i = 1; i < nr; i++) {\n -+\t\tconst char *r1 = entries[i - 1]->record.start + hexsz + 1;\n -+\t\tconst char *r2 = entries[i]->record.start + hexsz + 1;\n ++\tif (*start == '#') {\n ++\t\teol = memchr(start, '\\n', eof - start);\n ++\t\tstart = eol + 1;\n ++\t\tline_number++;\n ++\t}\n ++\n ++\tfor (; start < eof; line_number++, start = eol + 1) {\n ++\t\teol = memchr(start, '\\n', eof - start);\n ++\n ++\t\tif (*start == '^')\n ++\t\t\tcontinue;\n ++\n ++\t\tif (!former) {\n ++\t\t\tformer = start + hexsz + 1;\n ++\t\t\tcontinue;\n ++\t\t}\n +\n -+\t\tif (cmp_packed_refname(r1, r2) >= 0) {\n ++\t\tcurrent = start + hexsz + 1;\n ++\t\tif (cmp_packed_refname(former, current) >= 0) {\n +\t\t\tconst char *err_fmt =\n -+\t\t\t\t\"refname '%s' is not less than next refname '%s'\";\n -+\t\t\tconst char *eol;\n -+\t\t\teol = memchr(entries[i - 1]->record.start, '\\n',\n -+\t\t\t\t entries[i - 1]->record.len);\n -+\t\t\tstrbuf_add(&refname1, r1, eol - r1);\n -+\t\t\teol = memchr(entries[i]->record.start, '\\n',\n -+\t\t\t\t entries[i]->record.len);\n -+\t\t\tstrbuf_add(&refname2, r2, eol - r2);\n ++\t\t\t\t\"refname '%s' is less than previous refname '%s'\";\n ++\n ++\t\t\teol = memchr(former, '\\n', eof - former);\n ++\t\t\tstrbuf_add(&refname1, former, eol - former);\n ++\t\t\teol = memchr(current, '\\n', eof - current);\n ++\t\t\tstrbuf_add(&refname2, current, eol - current);\n +\n -+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\",\n -+\t\t\t\t entries[i - 1]->line_number);\n ++\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n +\t\t\treport.path = packed_entry.buf;\n +\t\t\tret = fsck_report_ref(o, &report,\n +\t\t\t\t\t FSCK_MSG_PACKED_REF_UNSORTED,\n -+\t\t\t\t\t err_fmt, refname1.buf, refname2.buf);\n ++\t\t\t\t\t err_fmt, refname2.buf, refname1.buf);\n +\t\t\tgoto cleanup;\n +\t\t}\n ++\t\tformer = current;\n +\t}\n +\n +cleanup:\n @@ refs/packed-backend.c: static int packed_fsck_ref_main_line(struct fsck_options\n +\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n ++\t\t\t\t unsigned int *sorted,\n \t\t\t\t const char *start, const char *eof)\n {\n - \tstruct strbuf packed_entry = STRBUF_INIT;\n -+\tstruct fsck_packed_ref_entry **entries;\n \tstruct strbuf refname = STRBUF_INIT;\n - \tunsigned long line_number = 1;\n -+\tunsigned int sorted = 0;\n -+\tsize_t entry_alloc = 20;\n -+\tsize_t entry_nr = 0;\n - \tconst char *eol;\n - \tint ret = 0;\n +@@ refs/packed-backend.c: static int packed_fsck_ref_content(struct fsck_options *o,\n \n - \tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n - \tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n + \tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n \tif (*start == '#') {\n -\t\tret |= packed_fsck_ref_header(o, start, eol);\n -+\t\tret |= packed_fsck_ref_header(o, start, eol, &sorted);\n ++\t\tret |= packed_fsck_ref_header(o, start, eol, sorted);\n \n \t\tstart = eol + 1;\n \t\tline_number++;\n - \t}\n +@@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n + \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n + \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n + \tstruct strbuf packed_ref_content = STRBUF_INIT;\n ++\tunsigned int sorted = 0;\n + \tint ret = 0;\n + \tint fd;\n \n -+\tALLOC_ARRAY(entries, entry_alloc);\n - \twhile (start < eof) {\n -+\t\tstruct fsck_packed_ref_entry *entry\n -+\t\t\t= create_fsck_packed_ref_entry(line_number, start);\n -+\n -+\t\tALLOC_GROW(entries, entry_nr + 1, entry_alloc);\n -+\t\tentries[entry_nr++] = entry;\n - \t\tstrbuf_reset(&packed_entry);\n - \t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n - \t\tret |= packed_fsck_ref_next_line(o, &packed_entry, start, eof, &eol);\n -@@ refs/packed-backend.c: static int packed_fsck_ref_content(struct fsck_options *o,\n - \t\t\tstart = eol + 1;\n - \t\t\tline_number++;\n - \t\t}\n -+\t\tentry->record.len = start - entry->record.start;\n +@@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n + \t\tgoto cleanup;\n \t}\n \n +-\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n ++\tret = packed_fsck_ref_content(o, ref_store, &sorted, packed_ref_content.buf,\n + \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n +\tif (!ret && sorted)\n -+\t\tret |= packed_fsck_ref_sorted(o, ref_store, entries, entry_nr);\n -+\n - \tstrbuf_release(&packed_entry);\n - \tstrbuf_release(&refname);\n - \tstrbuf_release(&packed_entry);\n -+\tfree_fsck_packed_ref_entries(entries, entry_nr);\n - \treturn ret;\n - }\n ++\t\tret = packed_fsck_ref_sorted(o, ref_store, packed_ref_content.buf,\n ++\t\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n + cleanup:\n + \tstrbuf_release(&packed_ref_content);\n \n ## t/t0602-reffiles-fsck.sh ##\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs content should be checked' '\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs content should be che\n +\t\trefname1=\"refs/heads/main\" &&\n +\t\trefname2=\"refs/heads/foo\" &&\n +\t\trefname3=\"refs/tags/foo\" &&\n -+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n -+\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n -+\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n ++\n ++\t\tcat >.git/packed-refs <<-EOF &&\n ++\t\t# pack-refs with: peeled fully-peeled sorted\n ++\t\tEOF\n ++\t\tgit refs verify 2>err &&\n ++\t\trm .git/packed-refs &&\n ++\t\ttest_must_be_empty err &&\n ++\n ++\t\tcat >.git/packed-refs <<-EOF &&\n ++\t\t# pack-refs with: peeled fully-peeled sorted\n ++\t\t$branch_2_oid $refname1\n ++\t\tEOF\n ++\t\tgit refs verify 2>err &&\n ++\t\trm .git/packed-refs &&\n ++\t\ttest_must_be_empty err &&\n ++\n ++\t\tcat >.git/packed-refs <<-EOF &&\n ++\t\t# pack-refs with: peeled fully-peeled sorted\n ++\t\t$branch_2_oid $refname1\n ++\t\t$branch_1_oid $refname2\n ++\t\t$tag_1_oid $refname3\n ++\t\tEOF\n +\t\ttest_must_fail git refs verify 2>err &&\n +\t\tcat >expect <<-EOF &&\n -+\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname1'\\'' is not less than next refname '\\''$refname2'\\''\n ++\t\terror: packed-refs line 3: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname1'\\''\n +\t\tEOF\n +\t\trm .git/packed-refs &&\n +\t\ttest_cmp expect err &&\n +\n -+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted \\n\" >.git/packed-refs &&\n -+\t\tprintf \"%s %s\\n\" \"$tag_1_oid\" \"$refname3\" >>.git/packed-refs &&\n -+\t\tprintf \"^%s\\n\" \"$tag_1_peeled_oid\" >>.git/packed-refs &&\n -+\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname2\" >>.git/packed-refs &&\n ++\t\tcat >.git/packed-refs <<-EOF &&\n ++\t\t# pack-refs with: peeled fully-peeled sorted\n ++\t\t$tag_1_oid $refname3\n ++\t\t^$tag_1_peeled_oid\n ++\t\t$branch_2_oid $refname2\n ++\t\tEOF\n +\t\ttest_must_fail git refs verify 2>err &&\n +\t\tcat >expect <<-EOF &&\n -+\t\terror: packed-refs line 2: packedRefUnsorted: refname '\\''$refname3'\\'' is not less than next refname '\\''$refname2'\\''\n ++\t\terror: packed-refs line 4: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname3'\\''\n +\t\tEOF\n +\t\trm .git/packed-refs &&\n +\t\ttest_cmp expect err\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs content should be che\n +\t\trefname1=\"refs/heads/main\" &&\n +\t\trefname2=\"refs/heads/foo\" &&\n +\t\trefname3=\"refs/tags/foo\" &&\n -+\t\tprintf \"# pack-refs with: peeled fully-peeled \\n\" >.git/packed-refs &&\n -+\t\tprintf \"%s %s\\n\" \"$branch_2_oid\" \"$refname1\" >>.git/packed-refs &&\n -+\t\tprintf \"%s %s\\n\" \"$branch_1_oid\" \"$refname2\" >>.git/packed-refs &&\n ++\n ++\t\tcat >.git/packed-refs <<-EOF &&\n ++\t\t# pack-refs with: peeled fully-peeled\n ++\t\t$branch_2_oid $refname1\n ++\t\t$branch_1_oid $refname2\n ++\t\tEOF\n +\t\tgit refs verify 2>err &&\n +\t\ttest_must_be_empty err\n +\t)\n8: 4f2170aa7c ! 8: 2dd3437478 builtin/fsck: add `git refs verify` child process\n @@ Commit message\n \"git-fsck(1)\" which would implicitly check the consistency of refs to\n die the program.\n \n + Last, update the test to exercise the code.\n +\n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n Signed-off-by: shejialuo \n @@ Documentation/git-fsck.txt: care about this output and want to speed it up furth\n +--[no-]references::\n +\tControl whether to check the references database consistency\n +\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n ++\tThe default is to check the references database.\n +\n CONFIGURATION\n -------------\n @@ builtin/fsck.c: int cmd_fsck(int argc,\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n \t\tfor_each_packed_object(the_repository,\n +\n + ## t/t0602-reffiles-fsck.sh ##\n +@@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-ref without sorted trait should not be checked' '\n + \t)\n + '\n + \n ++test_expect_success '--[no-]references option should apply to fsck' '\n ++\ttest_when_finished \"rm -rf repo\" &&\n ++\tgit init repo &&\n ++\tbranch_dir_prefix=.git/refs/heads &&\n ++\t(\n ++\t\tcd repo &&\n ++\t\ttest_commit default &&\n ++\t\tfor trailing_content in \" garbage\" \" more garbage\"\n ++\t\tdo\n ++\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n ++\t\t\tgit fsck 2>err &&\n ++\t\t\tcat >expect <<-EOF &&\n ++\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n ++\t\t\tEOF\n ++\t\t\trm $branch_dir_prefix/branch-garbage &&\n ++\t\t\ttest_cmp expect err || return 1\n ++\t\tdone &&\n ++\n ++\t\tfor trailing_content in \" garbage\" \" more garbage\"\n ++\t\tdo\n ++\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n ++\t\t\tgit fsck --references 2>err &&\n ++\t\t\tcat >expect <<-EOF &&\n ++\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n ++\t\t\tEOF\n ++\t\t\trm $branch_dir_prefix/branch-garbage &&\n ++\t\t\ttest_cmp expect err || return 1\n ++\t\tdone &&\n ++\n ++\t\tfor trailing_content in \" garbage\" \" more garbage\"\n ++\t\tdo\n ++\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n ++\t\t\tgit fsck --no-references 2>err &&\n ++\t\t\trm $branch_dir_prefix/branch-garbage &&\n ++\t\t\ttest_must_be_empty err || return 1\n ++\t\tdone\n ++\t)\n ++'\n ++\n + test_done\n-- \n2.48.1\n\n"},{"id":"512396","messageId":"Z67L7yU75QxQ0hjm@ArchLinux","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"[PATCH v4 1/8] t0602: use subshell to ensure working directory unchanged","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T04:51:59Z","receivedAt":"2025-02-14T04:52:06Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"For every test, we would execute the command \"cd repo\" in the first but\nwe never execute the command \"cd ..\" to restore the working directory.\nHowever, it's either not a good idea use above way. Because if any test\nfails between \"cd repo\" and \"cd ..\", the \"cd ..\" will never be reached.\nAnd we cannot correctly restore the working directory.\n\nLet's use subshell to ensure that the current working directory could be\nrestored to the correct path.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n t/t0602-reffiles-fsck.sh | 967 ++++++++++++++++++++-------------------\n 1 file changed, 494 insertions(+), 473 deletions(-)\n\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex d4a08b823b..cf7a202d0d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -14,222 +14,229 @@ test_expect_success 'ref name should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b default-branch &&\n-\tgit tag default-tag &&\n-\tgit tag multi_hierarchy/default-tag &&\n-\n-\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n-\trm $branch_dir_prefix/@ &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n-\tgit refs verify 2>err &&\n-\trm $tag_dir_prefix/tag-1.lock &&\n-\ttest_must_be_empty err &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/.lock: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t(\n+\t\tcd repo &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b default-branch &&\n+\t\tgit tag default-tag &&\n+\t\tgit tag multi_hierarchy/default-tag &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\t\trm $branch_dir_prefix/@ &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $tag_dir_prefix/tag-1.lock &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \t\ttest_must_fail git refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\terror: refs/tags/.lock: badRefName: invalid refname format\n \t\tEOF\n-\t\trm -r \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $tag_dir_prefix/.lock &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=warn refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n-\ttest_must_be_empty err\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b branch-1 &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=warn refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm $branch_dir_prefix/.branch-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n '\n \n test_expect_success 'ref name check should work for multiple worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\n-\tcd repo &&\n-\ttest_commit initial &&\n-\tgit checkout -b branch-1 &&\n-\ttest_commit second &&\n-\tgit checkout -b branch-2 &&\n-\ttest_commit third &&\n-\tgit checkout -b branch-3 &&\n-\tgit worktree add ./worktree-1 branch-1 &&\n-\tgit worktree add ./worktree-2 branch-2 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n-\t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n \t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n-\n-\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n-\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err &&\n-\n-\tfor worktree in \"worktree-1\" \"worktree-2\"\n-\tdo\n+\t\tcd repo &&\n+\t\ttest_commit initial &&\n+\t\tgit checkout -b branch-1 &&\n+\t\ttest_commit second &&\n+\t\tgit checkout -b branch-2 &&\n+\t\ttest_commit third &&\n+\t\tgit checkout -b branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-1 &&\n+\t\tgit worktree add ./worktree-2 branch-2 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n \t\t(\n-\t\t\tcd $worktree &&\n-\t\t\ttest_must_fail git refs verify 2>err &&\n-\t\t\tcat >expect <<-EOF &&\n-\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\t\t\tEOF\n-\t\t\tsort err >sorted_err &&\n-\t\t\ttest_cmp expect sorted_err || return 1\n-\t\t)\n-\tdone\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\n+\t\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\t\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err &&\n+\n+\t\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\t\tdo\n+\t\t\t(\n+\t\t\t\tcd $worktree &&\n+\t\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\t\tcat >expect <<-EOF &&\n+\t\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\t\tEOF\n+\t\t\t\tsort err >sorted_err &&\n+\t\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t\t)\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tfor trailing_content in \" garbage\" \" more garbage\"\n-\tdo\n-\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-garbage &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n+\t\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n-\t'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\t'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n \n-\t garbage'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err\n+\t\t garbage'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -237,99 +244,103 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tbad_content_1=$(git rev-parse main)x &&\n-\tbad_content_2=xfsazqfxcadas &&\n-\tbad_content_3=Xfsazqfxcadas &&\n-\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n-\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n-\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n-\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n-\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n-\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tbad_content_1=$(git rev-parse main)x &&\n+\t\tbad_content_2=xfsazqfxcadas &&\n+\t\tbad_content_3=Xfsazqfxcadas &&\n+\t\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\t\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\t\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\t\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\t\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-complicated &&\n-\ttest_cmp expect err\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (aggregate)' '\n@@ -337,32 +348,34 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n-\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'the target of the textual symref should be checked' '\n@@ -370,28 +383,30 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n-\t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n-\n-\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n-\t\tgit refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked' '\n@@ -399,201 +414,207 @@ test_expect_success SYMLINKS 'symlink symref content should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n-\tEOF\n-\trm $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_cmp expect err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-good &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\t\tEOF\n+\t\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tmain_worktree_refdir_prefix=.git/refs/heads &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\n-\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tfor bad_referent_name in \".tag\" \"branch \"\n-\tdo\n-\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tmain_worktree_refdir_prefix=.git/refs/heads &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $worktree1_refdir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor bad_referent_name in \".tag\" \"branch \"\n+\t\tdo\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-garbage &&\n-\ttest_cmp expect err\n+\t\trm $worktree1_refdir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_done\n-- \n2.48.1\n\n"},{"id":"512397","messageId":"Z67L_lmCd6NNXpWZ@ArchLinux","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"[PATCH v4 2/8] builtin/refs: get worktrees without reading head information","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T04:52:14Z","receivedAt":"2025-02-14T04:52:21Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\nand \"next_record\" which would check the correctness of the content of\nthe \"packed-ref\" file. When anything is bad, the program will die.\n\nIt may seem that we have nothing relevant to above feature, because we\nare going to read and parse the raw \"packed-ref\" file without creating\nthe snapshot and using the ref iterator to check the consistency.\n\nHowever, when using \"get_worktrees\" in \"builtin/refs\", we would parse\nthe \"HEAD\" information. If the referent of the \"HEAD\" is inside the\n\"packed-ref\", we will call \"create_snapshot\" function to parse the\n\"packed-ref\" to get the information. No matter whether the entry of\n\"HEAD\" in \"packed-ref\" is correct, \"create_snapshot\" would call\n\"verify_buffer_safe\" to check whether there is a newline in the last\nline of the file. If not, the program will die.\n\nAlthough this behavior has no harm for the program, it will\nshort-circuit the program. When the users execute \"git refs verify\" or\n\"git fsck\", we should avoid reading the head information, which may\nexecute the read operation in packed backend with stricter checks to die\nthe program. Instead, we should continue to check other parts of the\n\"packed-refs\" file completely.\n\nFortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\nworktrees, 2023-12-29), we have introduced a function\n\"get_worktrees_internal\" which allows us to get worktrees without\nreading head information.\n\nCreate a new exposed function \"get_worktrees_without_reading_head\", then\nreplace the \"get_worktrees\" in \"builtin/refs\" with the new created\nfunction.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 2 +-\n worktree.c | 5 +++++\n worktree.h | 6 ++++++\n 3 files changed, 12 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex a29f195834..55ff5dae11 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tworktrees = get_worktrees();\n+\tworktrees = get_worktrees_without_reading_head();\n \tfor (size_t i = 0; worktrees[i]; i++)\n \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n \t\t\t\t &fsck_refs_options, worktrees[i]);\ndiff --git a/worktree.c b/worktree.c\nindex 248bbb39d4..89b7d86cef 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -175,6 +175,11 @@ struct worktree **get_worktrees(void)\n \treturn get_worktrees_internal(0);\n }\n \n+struct worktree **get_worktrees_without_reading_head(void)\n+{\n+\treturn get_worktrees_internal(1);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\ndiff --git a/worktree.h b/worktree.h\nindex 38145df80f..1ba4a161a0 100644\n--- a/worktree.h\n+++ b/worktree.h\n@@ -30,6 +30,12 @@ struct worktree {\n */\n struct worktree **get_worktrees(void);\n \n+/*\n+ * Like `get_worktrees`, but does not read HEAD. This is useful when checking\n+ * the consistency, as reading HEAD may not be necessary.\n+ */\n+struct worktree **get_worktrees_without_reading_head(void);\n+\n /*\n * Returns 1 if linked worktrees exist, 0 otherwise.\n */\n-- \n2.48.1\n\n"},{"id":"512398","messageId":"Z67MDPtjoXQB2sGB@ArchLinux","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"[PATCH v4 3/8] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T04:52:28Z","receivedAt":"2025-02-14T04:52:35Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\nconsistency and correctness of \"packed-refs\" file, they never check the\nfiletype of the \"packed-refs\". The user should always use \"git\npack-refs\" command to create the raw regular \"packed-refs\" file, so we\nneed to explicitly check this in \"git refs verify\".\n\nWe could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file.\nIf the returned \"fd\" value is less than 0, we could check whether the\n\"errno\" is \"ELOOP\" to report an error to the user.\n\nReuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\nthe user if \"packed-refs\" is not a regular file.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 39 +++++++++++++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 22 ++++++++++++++++++++++\n 2 files changed, 57 insertions(+), 4 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a7b6f74b6e..6401cecd5f 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -4,6 +4,7 @@\n #include \"../git-compat-util.h\"\n #include \"../config.h\"\n #include \"../dir.h\"\n+#include \"../fsck.h\"\n #include \"../gettext.h\"\n #include \"../hash.h\"\n #include \"../hex.h\"\n@@ -1748,15 +1749,45 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n-static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED,\n+static int packed_fsck(struct ref_store *ref_store,\n+\t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n+\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n+\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tint ret = 0;\n+\tint fd;\n \n \tif (!is_main_worktree(wt))\n-\t\treturn 0;\n+\t\tgoto cleanup;\n \n-\treturn 0;\n+\tif (o->verbose)\n+\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n+\n+\tfd = open_nofollow(refs->path, O_RDONLY);\n+\tif (fd < 0) {\n+\t\t/*\n+\t\t * If the packed-refs file doesn't exist, there's nothing\n+\t\t * to check.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tif (errno == ELOOP) {\n+\t\t\tstruct fsck_ref_report report = { 0 };\n+\t\t\treport.path = \"packed-refs\";\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t\t \"not a regular file\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tret = error_errno(_(\"unable to open %s\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\treturn ret;\n }\n \n struct ref_storage_be refs_be_packed = {\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex cf7a202d0d..42c8d4ca1e 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -617,4 +617,26 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t)\n '\n \n+test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit pack-refs --all &&\n+\n+\t\tmv .git/packed-refs .git/packed-refs-back &&\n+\t\tln -sf packed-refs-bak .git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs: badRefFiletype: not a regular file\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512399","messageId":"Z67MG8utrQfUrakz@ArchLinux","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"[PATCH v4 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T04:52:43Z","receivedAt":"2025-02-14T04:52:50Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c::create_snapshot\", if there is a header (the line\nwhich starts with '#'), we will check whether the line starts with \"#\npack-refs with:\". Before we port this check into \"packed_fsck\", let's\nfix \"create_snapshot\" to check the prefix \"# packed-ref with: \" instead\nof \"# packed-ref with:\" due to that we will always write a single\ntrailing space after the colon.\n\nHowever, we need to consider other situations and discuss whether we\nneed to add checks.\n\n1. If the header does not exist, we should not report an error to the\n user. This is because in older Git version, we never write header in\n the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n in runtime.\n2. If the header content does not start with \"# packed-ref with: \", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n3. If the header content is not the same as the constant string\n \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n intentionally. So, there is no need to report.\n\nAs we have analyzed, we only need to check the case 2 in the above. In\norder to do this, read the \"packed-refs\" file via \"strbuf_read\". Like\nwhat \"create_snapshot\" and other functions do, we could split the line\nby finding the next newline in the buffer. When we cannot find a\nnewline, we could report an error.\n\nSo, create a function \"packed_fsck_ref_next_line\" to find the next\nnewline and if there is no such newline, use\n\"packedRefEntryNotTerminated(ERROR)\" to report an error to the user.\n\nThen, parse the first line to apply the checks. Update the test to\nexercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 8 ++++\n fsck.h | 2 +\n refs/packed-backend.c | 75 ++++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 52 ++++++++++++++++++++++++\n 4 files changed, 136 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex b14bc44ca4..11906f90fd 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -16,6 +16,10 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefHeader`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid\n+\theader.\n+\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n@@ -176,6 +180,10 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`packedRefEntryNotTerminated`::\n+\t(ERROR) The \"packed-refs\" file contains an entry that is\n+\tnot terminated by a newline.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex a44c231a5f..67e3c97bc0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n@@ -53,6 +54,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE, ERROR) \\\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 6401cecd5f..ff74ab915e 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -694,7 +694,7 @@ static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n \n \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n \n-\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n+\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n \t\t\tdie_invalid_line(refs->path,\n \t\t\t\t\t snapshot->buf,\n \t\t\t\t\t snapshot->eof - snapshot->buf);\n@@ -1749,12 +1749,76 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+static int packed_fsck_ref_next_line(struct fsck_options *o,\n+\t\t\t\t unsigned long line_number, const char *start,\n+\t\t\t\t const char *eof, const char **eol)\n+{\n+\tint ret = 0;\n+\n+\t*eol = memchr(start, '\\n', eof - start);\n+\tif (!*eol) {\n+\t\tstruct strbuf packed_entry = STRBUF_INIT;\n+\t\tstruct fsck_ref_report report = { 0 };\n+\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n+\t\t\t\t \"'%.*s' is not terminated with a newline\",\n+\t\t\t\t (int)(eof - start), start);\n+\n+\t\t/*\n+\t\t * There is no newline but we still want to parse it to the end of\n+\t\t * the buffer.\n+\t\t */\n+\t\t*eol = eof;\n+\t\tstrbuf_release(&packed_entry);\n+\t}\n+\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_header(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs.header\";\n+\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tunsigned long line_number = 1;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\tif (*start == '#') {\n+\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\treturn ret;\n+}\n+\n static int packed_fsck(struct ref_store *ref_store,\n \t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tint ret = 0;\n \tint fd;\n \n@@ -1786,7 +1850,16 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n+\t\tret = error_errno(_(\"unable to read %s\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\n cleanup:\n+\tstrbuf_release(&packed_ref_content);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 42c8d4ca1e..30be1982df 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -639,4 +639,56 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-refs header should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with a: peeled fully-peeled\" \\\n+\t\t\t\t \"# pack-refs with:peeled fully-peeled sorted\"\n+\t\tdo\n+\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with: '\\''\n+\t\t\tEOF\n+\t\t\trm .git/packed-refs &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n+'\n+\n+test_expect_success 'packed-refs missing header should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"$(git rev-parse HEAD) refs/heads/main\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n+test_expect_success 'packed-refs unknown traits should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted foo\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512400","messageId":"Z67MJjI9coLnyi3a@ArchLinux","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"[PATCH v4 5/8] packed-backend: check whether the refname contains NUL characters","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T04:52:54Z","receivedAt":"2025-02-14T04:53:01Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will use \"check_refname_format\" to check\nthe consistency of the refname. If it is not OK, the program will die.\nHowever, it is reported in [1], we cannot catch some corruption. But we\nalready have the code path and we must miss out something.\n\nWe use the following code to get the refname:\n\n strbuf_add(&iter->refname_buf, p, eol - p);\n iter->base.refname = iter->refname_buf.buf\n\nIn the above code, `p` is the start pointer of the refname and `eol` is\nthe next newline pointer. We calculate the length of the refname by\nsubtracting the two pointers. Then we add the memory range between `p`\nand `eol` to get the refname.\n\nHowever, if there are some NUL characters in the memory range between `p`\nand `eol`, we will see the refname as a valid ref name as long as the\nmemory range between `p` and first occurred NUL character is valid.\n\nIn order to catch above corruption, create a new function\n\"refname_contains_nul\" by searching the first NUL character. If it is\nnot at the end of the string, there must be some NUL characters in the\nrefname.\n\nUse this function in \"next_record\" function to die the program if\n\"refname_contains_nul\" returns true.\n\n[1] https://lore.kernel.org/git/6cfee0e4-3285-4f18-91ff-d097da9de737@rd10.de/\n\nReported-by: R. Diez \nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex ff74ab915e..692e315e41 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -494,6 +494,21 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n \t\t\t\t last_line, eof - last_line);\n }\n \n+/*\n+ * When parsing the \"packed-refs\" file, we will parse it line by line.\n+ * Because we know the start pointer of the refname and the next\n+ * newline pointer, we could calculate the length of the refname by\n+ * subtracting the two pointers. However, there is a corner case where\n+ * the refname contains corrupted embedded NUL characters. And\n+ * `check_refname_format()` will not catch this when the truncated\n+ * refname is still a valid refname. To prevent this, we need to check\n+ * whether the refname contains the NUL characters.\n+ */\n+static int refname_contains_nul(struct strbuf *refname)\n+{\n+\treturn !!memchr(refname->buf, '\\0', refname->len);\n+}\n+\n #define SMALL_FILE_SIZE (32*1024)\n \n /*\n@@ -895,6 +910,9 @@ static int next_record(struct packed_ref_iterator *iter)\n \tstrbuf_add(&iter->refname_buf, p, eol - p);\n \titer->base.refname = iter->refname_buf.buf;\n \n+\tif (refname_contains_nul(&iter->refname_buf))\n+\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n+\n \tif (check_refname_format(iter->base.refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!refname_is_safe(iter->base.refname))\n \t\t\tdie(\"packed refname is dangerous: %s\",\n-- \n2.48.1\n\n"},{"id":"512401","messageId":"Z67MMWCk4fA3hkTk@ArchLinux","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"[PATCH v4 6/8] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T04:53:05Z","receivedAt":"2025-02-14T04:53:12Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will parse the ref entry to check the\nconsistency. This function has already checked the following things:\n\n1. Parse the main line of the ref entry to inspect whether the oid is\n not correct. Then, check whether the next character is oid. Then\n check the refname.\n2. If the next line starts with '^', it would continue to parse the\n peeled oid and check whether the last character is '\\n'.\n\nAs we decide to implement the ref consistency check for \"packed-refs\",\nlet's port these two checks and update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 121 +++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 44 +++++++++++++\n 4 files changed, 168 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 11906f90fd..02a7bf0503 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -16,6 +16,9 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefEntry`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid entry.\n+\n `badPackedRefHeader`::\n \t(ERROR) The \"packed-refs\" file contains an invalid\n \theader.\ndiff --git a/fsck.h b/fsck.h\nindex 67e3c97bc0..14d70f6653 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\n \tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 692e315e41..5d1dcfec6f 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1812,9 +1812,113 @@ static int packed_fsck_ref_header(struct fsck_options *o,\n \treturn 0;\n }\n \n+static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id peeled;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\t/*\n+\t * Skip the '^' and parse the peeled oid.\n+\t */\n+\tstart++;\n+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid peeled oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p != eol) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n+\t\t\t\t (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_main_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t struct strbuf *refname,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id oid;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p == eol || !isspace(*p)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n+\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tp++;\n+\tstrbuf_reset(refname);\n+\tstrbuf_add(refname, p, eol - p);\n+\tif (refname_contains_nul(refname)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"refname '%s' contains NULL binaries\",\n+\t\t\t\t refname->buf);\n+\t}\n+\n+\tif (check_refname_format(refname->buf, 0)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n+\t\t\t\t \"has bad refname '%s'\", refname->buf);\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n+\tstruct strbuf refname = STRBUF_INIT;\n \tunsigned long line_number = 1;\n \tconst char *eol;\n \tint ret = 0;\n@@ -1827,6 +1931,21 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\tline_number++;\n \t}\n \n+\twhile (start < eof) {\n+\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\tret |= packed_fsck_ref_main_line(o, ref_store, line_number, &refname, start, eol);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t\tif (start < eof && *start == '^') {\n+\t\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, line_number,\n+\t\t\t\t\t\t\t start, eol);\n+\t\t\tstart = eol + 1;\n+\t\t\tline_number++;\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&refname);\n \treturn ret;\n }\n \n@@ -1873,7 +1992,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 30be1982df..058a783cb7 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -691,4 +691,48 @@ test_expect_success 'packed-refs unknown traits should not be reported' '\n \t)\n '\n \n+test_expect_success 'packed-refs content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tgit tag -a annotated-tag-2 -m tag-2 &&\n+\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n+\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$short_oid refs/heads/branch-1\n+\t\t${branch_1_oid}x\n+\t\t$branch_2_oid refs/heads/bad-branch\n+\t\t$branch_2_oid refs/heads/branch.\n+\t\t$tag_1_oid refs/tags/annotated-tag-3\n+\t\t^$short_oid\n+\t\t$tag_2_oid refs/tags/annotated-tag-4.\n+\t\t^$tag_2_peeled_oid garbage\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: badPackedRefEntry: '\\''$short_oid refs/heads/branch-1'\\'' has invalid oid\n+\t\terror: packed-refs line 3: badPackedRefEntry: has no space after oid '\\''$branch_1_oid'\\'' but with '\\''x'\\''\n+\t\terror: packed-refs line 4: badRefName: has bad refname '\\'' refs/heads/bad-branch'\\''\n+\t\terror: packed-refs line 5: badRefName: has bad refname '\\''refs/heads/branch.'\\''\n+\t\terror: packed-refs line 7: badPackedRefEntry: '\\''$short_oid'\\'' has invalid peeled oid\n+\t\terror: packed-refs line 8: badRefName: has bad refname '\\''refs/tags/annotated-tag-4.'\\''\n+\t\terror: packed-refs line 9: badPackedRefEntry: has trailing garbage after peeled oid '\\'' garbage'\\''\n+\t\tEOF\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512404","messageId":"Z67NrPT9j5C6pzP8@ArchLinux","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"[PATCH v4 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T04:59:24Z","receivedAt":"2025-02-14T04:59:32Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"When there is a \"sorted\" trait in the header of the \"packed-refs\" file,\nit means that each entry is sorted increasingly by comparing the\nrefname. We should add checks to verify whether the \"packed-refs\" is\nsorted in this case.\n\nUpdate the \"packed_fsck_ref_header\" to know whether there is a \"sorted\"\ntrail in the header. It may seem that we could record all refnames\nduring the parsing process and then compare later. However, this is not\na good design due to the following reasons:\n\n1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n in the \"packed-refs\" file.\n2. We cannot reuse the existing compare function \"cmp_packed_ref_records\"\n which cause repetition.\n\nBecause \"cmp_packed_ref_records\" needs an extra parameter \"struct\nsnaphost\", extract the common part into a new function\n\"cmp_packed_ref_records\" to reuse this function to compare.\n\nThen, create a new function \"packed_fsck_ref_sorted\" to parse the file\nagain and user the new fsck message \"packedRefUnsorted(ERROR)\" to report\nto the user if the file is not sorted.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 116 +++++++++++++++++++++++++++++-----\n t/t0602-reffiles-fsck.sh | 87 +++++++++++++++++++++++++\n 4 files changed, 191 insertions(+), 16 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 02a7bf0503..9601fff228 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -187,6 +187,9 @@\n \t(ERROR) The \"packed-refs\" file contains an entry that is\n \tnot terminated by a newline.\n \n+`packedRefUnsorted`::\n+\t(ERROR) The \"packed-refs\" file is not sorted.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex 14d70f6653..19f3cb2773 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -56,6 +56,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n \tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n+\tFUNC(PACKED_REF_UNSORTED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 5d1dcfec6f..391efced54 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -300,14 +300,9 @@ struct snapshot_record {\n \tsize_t len;\n };\n \n-static int cmp_packed_ref_records(const void *v1, const void *v2,\n-\t\t\t\t void *cb_data)\n-{\n-\tconst struct snapshot *snapshot = cb_data;\n-\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n-\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n-\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n \n+static int cmp_packed_refname(const char *r1, const char *r2)\n+{\n \twhile (1) {\n \t\tif (*r1 == '\\n')\n \t\t\treturn *r2 == '\\n' ? 0 : -1;\n@@ -322,6 +317,17 @@ static int cmp_packed_ref_records(const void *v1, const void *v2,\n \t}\n }\n \n+static int cmp_packed_ref_records(const void *v1, const void *v2,\n+\t\t\t\t void *cb_data)\n+{\n+\tconst struct snapshot *snapshot = cb_data;\n+\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n+\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n+\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n+\n+\treturn cmp_packed_refname(r1, r2);\n+}\n+\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n@@ -1797,19 +1803,33 @@ static int packed_fsck_ref_next_line(struct fsck_options *o,\n }\n \n static int packed_fsck_ref_header(struct fsck_options *o,\n-\t\t\t\t const char *start, const char *eol)\n+\t\t\t\t const char *start, const char *eol,\n+\t\t\t\t unsigned int *sorted)\n {\n-\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n+\tchar *tmp_line;\n+\tint ret = 0;\n+\tchar *p;\n+\n+\ttmp_line = xmemdupz(start, eol - start);\n+\tif (!skip_prefix(tmp_line, \"# pack-refs with: \", (const char **)&p)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \t\treport.path = \"packed-refs.header\";\n \n-\t\treturn fsck_report_ref(o, &report,\n-\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n-\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n-\t\t\t\t (int)(eol - start), start);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n \t}\n \n-\treturn 0;\n+\tstring_list_split_in_place(&traits, p, \" \", -1);\n+\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n+\n+cleanup:\n+\tfree(tmp_line);\n+\tstring_list_clear(&traits, 0);\n+\treturn ret;\n }\n \n static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n@@ -1914,8 +1934,68 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n \treturn ret;\n }\n \n+static int packed_fsck_ref_sorted(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct strbuf refname1 = STRBUF_INIT;\n+\tstruct strbuf refname2 = STRBUF_INIT;\n+\tunsigned long line_number = 1;\n+\tconst char *former = NULL;\n+\tconst char *current;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tif (*start == '#') {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\tfor (; start < eof; line_number++, start = eol + 1) {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\n+\t\tif (*start == '^')\n+\t\t\tcontinue;\n+\n+\t\tif (!former) {\n+\t\t\tformer = start + hexsz + 1;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tcurrent = start + hexsz + 1;\n+\t\tif (cmp_packed_refname(former, current) >= 0) {\n+\t\t\tconst char *err_fmt =\n+\t\t\t\t\"refname '%s' is less than previous refname '%s'\";\n+\n+\t\t\teol = memchr(former, '\\n', eof - former);\n+\t\t\tstrbuf_add(&refname1, former, eol - former);\n+\t\t\teol = memchr(current, '\\n', eof - current);\n+\t\t\tstrbuf_add(&refname2, current, eol - current);\n+\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\t\treport.path = packed_entry.buf;\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_PACKED_REF_UNSORTED,\n+\t\t\t\t\t err_fmt, refname2.buf, refname1.buf);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tformer = current;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname1);\n+\tstrbuf_release(&refname2);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned int *sorted,\n \t\t\t\t const char *start, const char *eof)\n {\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -1925,7 +2005,7 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \n \tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n \tif (*start == '#') {\n-\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\t\tret |= packed_fsck_ref_header(o, start, eol, sorted);\n \n \t\tstart = eol + 1;\n \t\tline_number++;\n@@ -1956,6 +2036,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n \tstruct strbuf packed_ref_content = STRBUF_INIT;\n+\tunsigned int sorted = 0;\n \tint ret = 0;\n \tint fd;\n \n@@ -1992,8 +2073,11 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, &sorted, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\tif (!ret && sorted)\n+\t\tret = packed_fsck_ref_sorted(o, ref_store, packed_ref_content.buf,\n+\t\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\n \tstrbuf_release(&packed_ref_content);\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 058a783cb7..f305428f12 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -735,4 +735,91 @@ test_expect_success 'packed-refs content should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-ref with sorted trait should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\t$tag_1_oid $refname3\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 3: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname1'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$tag_1_oid $refname3\n+\t\t^$tag_1_peeled_oid\n+\t\t$branch_2_oid $refname2\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 4: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname3'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n+test_expect_success 'packed-ref without sorted trait should not be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512405","messageId":"Z67Nz4GwAVS3fEds@ArchLinux","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"[PATCH v4 8/8] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T04:59:59Z","receivedAt":"2025-02-14T05:00:06Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"At now, we have already implemented the ref consistency checks for both\n\"files-backend\" and \"packed-backend\". Although we would check some\nredundant things, it won't cause trouble. So, let's integrate it into\nthe \"git-fsck(1)\" command to get feedback from the users. And also by\ncalling \"git refs verify\" in \"git-fsck(1)\", we make sure that the new\nadded checks don't break.\n\nIntroduce a new function \"fsck_refs\" that initializes and runs a child\nprocess to execute the \"git refs verify\" command. In order to provide\nthe user interface create a progress which makes the total task be 1.\nIt's hard to know how many loose refs we will check now. We might\nimprove this later.\n\nThen, introduce the option to allow the user to disable checking ref\ndatabase consistency. Put this function in the very first execution\nsequence of \"git-fsck(1)\" due to that we don't want the existing code of\n\"git-fsck(1)\" which would implicitly check the consistency of refs to\ndie the program.\n\nLast, update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/git-fsck.txt | 7 ++++++-\n builtin/fsck.c | 33 +++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 39 ++++++++++++++++++++++++++++++++++++++\n 3 files changed, 77 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt\nindex 5b82e4605c..5e71a29c3b 100644\n--- a/Documentation/git-fsck.txt\n+++ b/Documentation/git-fsck.txt\n@@ -12,7 +12,7 @@ SYNOPSIS\n 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n-\t [--[no-]name-objects] [...]\n+\t [--[no-]name-objects] [--[no-]references] [...]\n \n DESCRIPTION\n -----------\n@@ -104,6 +104,11 @@ care about this output and want to speed it up further.\n \tprogress status even if the standard error stream is not\n \tdirected to a terminal.\n \n+--[no-]references::\n+\tControl whether to check the references database consistency\n+\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n+\tThe default is to check the references database.\n+\n CONFIGURATION\n -------------\n \ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 7a4dcb0716..f4f395cfbd 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -50,6 +50,7 @@ static int verbose;\n static int show_progress = -1;\n static int show_dangling = 1;\n static int name_objects;\n+static int check_references = 1;\n #define ERROR_OBJECT 01\n #define ERROR_REACHABLE 02\n #define ERROR_PACK 04\n@@ -905,11 +906,37 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n \treturn res;\n }\n \n+static void fsck_refs(struct repository *r)\n+{\n+\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n+\tstruct progress *progress = NULL;\n+\n+\tif (show_progress)\n+\t\tprogress = start_progress(r, _(\"Checking ref database\"), 1);\n+\n+\tif (verbose)\n+\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n+\n+\tchild_process_init(&refs_verify);\n+\trefs_verify.git_cmd = 1;\n+\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n+\tif (verbose)\n+\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n+\tif (check_strict)\n+\t\tstrvec_push(&refs_verify.args, \"--strict\");\n+\n+\tif (run_command(&refs_verify))\n+\t\terrors_found |= ERROR_REFS;\n+\n+\tdisplay_progress(progress, 1);\n+\tstop_progress(&progress);\n+}\n+\n static char const * const fsck_usage[] = {\n \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n \t \" [--[no-]dangling] [--[no-]progress] [--connectivity-only]\\n\"\n-\t \" [--[no-]name-objects] [...]\"),\n+\t \" [--[no-]name-objects] [--[no-]references] [...]\"),\n \tNULL\n };\n \n@@ -928,6 +955,7 @@ static struct option fsck_opts[] = {\n \t\t\t\tN_(\"write dangling objects in .git/lost-found\")),\n \tOPT_BOOL(0, \"progress\", &show_progress, N_(\"show progress\")),\n \tOPT_BOOL(0, \"name-objects\", &name_objects, N_(\"show verbose names for reachable objects\")),\n+\tOPT_BOOL(0, \"references\", &check_references, N_(\"check reference database consistency\")),\n \tOPT_END(),\n };\n \n@@ -970,6 +998,9 @@ int cmd_fsck(int argc,\n \tgit_config(git_fsck_config, &fsck_obj_options);\n \tprepare_repo_settings(the_repository);\n \n+\tif (check_references)\n+\t\tfsck_refs(the_repository);\n+\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n \t\tfor_each_packed_object(the_repository,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex f305428f12..22bd847782 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -822,4 +822,43 @@ test_expect_success 'packed-ref without sorted trait should not be checked' '\n \t)\n '\n \n+test_expect_success '--[no-]references option should apply to fsck' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --references 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --no-references 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512410","messageId":"CAOLa=ZQFLTFNc5AnvDyAaLvY8__R+J9RHZ29TM8COhPxnQs8Zg@mail.gmail.com","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"Re: [PATCH v4 0/8] add more ref consistency checks","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-14T09:04:09Z","receivedAt":"2025-02-14T09:04:11Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> Hi All:\n>\n> This patch enhances the following things:\n>\n> 1. [PATCH v4 4/8]: update the tests to verify that we don't report any\n> errors to the user in some cases. Also, suggested by Junio, make sure\n> that we check whether there is a trailing space after \"# packed-refs\n> with:\".\n> 2. [PATCH v4 6/8]: instead of greedily calculating the name of the line,\n> lazily compute when there is any errors. And use the HERE docs to\n> improve the test script.\n> 3. [PATCH v4 7/8]: instead of storing the states, we parse the file\n> again to check whether the file is sorted to avoid allocating too\n> much memory. And use the HERE docs to improve the test script.\n> 4. [PATCH v4 8/8]: update the documentation to emphasis the default. And\n> add tests to exercise the code.\n>\n\nNit: For someone coming in to review the 4th version directly it would\nbe really nice to see:\n\n1. Summary of what the patch series is about.\n2. Changes built over the last versions.\n\nI know all this information is already spread out over the previous\nversions, but would be nice to have it here (in every version rather).\n\n> shejialuo (8):\n> t0602: use subshell to ensure working directory unchanged\n> builtin/refs: get worktrees without reading head information\n> packed-backend: check whether the \"packed-refs\" is regular file\n> packed-backend: add \"packed-refs\" header consistency check\n> packed-backend: check whether the refname contains NUL characters\n> packed-backend: add \"packed-refs\" entry consistency check\n> packed-backend: check whether the \"packed-refs\" is sorted\n> builtin/fsck: add `git refs verify` child process\n>\n> Documentation/fsck-msgids.txt | 14 +\n> Documentation/git-fsck.txt | 7 +-\n> builtin/fsck.c | 33 +-\n> builtin/refs.c | 2 +-\n> fsck.h | 4 +\n> refs/packed-backend.c | 349 +++++++++-\n> t/t0602-reffiles-fsck.sh | 1205 ++++++++++++++++++++-------------\n> worktree.c | 5 +\n> worktree.h | 6 +\n> 9 files changed, 1140 insertions(+), 485 deletions(-)\n\n[snip]\n"},{"id":"512411","messageId":"CAOLa=ZS3w3KkEoQksXZtLYCT6BJVs6o2+nmpVUapbnqVA4zfng@mail.gmail.com","threadId":"62743","inReplyTo":"Z67L_lmCd6NNXpWZ@ArchLinux","subject":"Re: [PATCH v4 2/8] builtin/refs: get worktrees without reading head information","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-14T09:19:53Z","receivedAt":"2025-02-14T09:19:55Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\n> and \"next_record\" which would check the correctness of the content of\n> the \"packed-ref\" file. When anything is bad, the program will die.\n>\n> It may seem that we have nothing relevant to above feature, because we\n> are going to read and parse the raw \"packed-ref\" file without creating\n> the snapshot and using the ref iterator to check the consistency.\n>\n> However, when using \"get_worktrees\" in \"builtin/refs\", we would parse\n> the \"HEAD\" information. If the referent of the \"HEAD\" is inside the\n> \"packed-ref\", we will call \"create_snapshot\" function to parse the\n> \"packed-ref\" to get the information. No matter whether the entry of\n> \"HEAD\" in \"packed-ref\" is correct, \"create_snapshot\" would call\n> \"verify_buffer_safe\" to check whether there is a newline in the last\n> line of the file. If not, the program will die.\n>\n\nNit: while the second paragraph above makes sense in the context of what\nwe're trying to achieve in this patch series. It doesn't make much sense\nfor this patch in isolation. Perhaps we want to give some more context\naround what we're trying to solve for in the upcoming patches and hence\nhow it hinders that.\n\n> Although this behavior has no harm for the program, it will\n> short-circuit the program. When the users execute \"git refs verify\" or\n> \"git fsck\", we should avoid reading the head information, which may\n> execute the read operation in packed backend with stricter checks to die\n> the program. Instead, we should continue to check other parts of the\n> \"packed-refs\" file completely.\n>\n> Fortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\n> worktrees, 2023-12-29), we have introduced a function\n> \"get_worktrees_internal\" which allows us to get worktrees without\n> reading head information.\n>\n> Create a new exposed function \"get_worktrees_without_reading_head\", then\n> replace the \"get_worktrees\" in \"builtin/refs\" with the new created\n> function.\n>\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> builtin/refs.c | 2 +-\n> worktree.c | 5 +++++\n> worktree.h | 6 ++++++\n> 3 files changed, 12 insertions(+), 1 deletion(-)\n>\n> diff --git a/builtin/refs.c b/builtin/refs.c\n> index a29f195834..55ff5dae11 100644\n> --- a/builtin/refs.c\n> +++ b/builtin/refs.c\n> @@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n> \tgit_config(git_fsck_config, &fsck_refs_options);\n> \tprepare_repo_settings(the_repository);\n>\n> -\tworktrees = get_worktrees();\n> +\tworktrees = get_worktrees_without_reading_head();\n> \tfor (size_t i = 0; worktrees[i]; i++)\n> \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n> \t\t\t\t &fsck_refs_options, worktrees[i]);\n> diff --git a/worktree.c b/worktree.c\n> index 248bbb39d4..89b7d86cef 100644\n> --- a/worktree.c\n> +++ b/worktree.c\n> @@ -175,6 +175,11 @@ struct worktree **get_worktrees(void)\n> \treturn get_worktrees_internal(0);\n> }\n>\n> +struct worktree **get_worktrees_without_reading_head(void)\n> +{\n> +\treturn get_worktrees_internal(1);\n> +}\n> +\n> const char *get_worktree_git_dir(const struct worktree *wt)\n> {\n> \tif (!wt)\n> diff --git a/worktree.h b/worktree.h\n> index 38145df80f..1ba4a161a0 100644\n> --- a/worktree.h\n> +++ b/worktree.h\n> @@ -30,6 +30,12 @@ struct worktree {\n> */\n> struct worktree **get_worktrees(void);\n>\n> +/*\n> + * Like `get_worktrees`, but does not read HEAD. This is useful when checking\n> + * the consistency, as reading HEAD may not be necessary.\n\nChecking what consistency? We should be a bit more verbose here. You can\nmention that skipping HEAD allows to get the worktree without worrying\nabout failures pertaining to parsing the HEAD ref.\n\n> + */\n> +struct worktree **get_worktrees_without_reading_head(void);\n> +\n> /*\n> * Returns 1 if linked worktrees exist, 0 otherwise.\n> */\n> --\n> 2.48.1\n"},{"id":"512412","messageId":"CAOLa=ZQ7CAXP-bYzTv3GJhauwtaL+pFj-2_QPWBh7SMiMsa6bQ@mail.gmail.com","threadId":"62743","inReplyTo":"Z67MDPtjoXQB2sGB@ArchLinux","subject":"Re: [PATCH v4 3/8] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-14T09:50:26Z","receivedAt":"2025-02-14T09:50:29Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\n> consistency and correctness of \"packed-refs\" file, they never check the\n\nBecause you say 'some' here, it made me more curious. Could you state\nexactly what checks are being done here?\n\n> filetype of the \"packed-refs\". The user should always use \"git\n> pack-refs\" command to create the raw regular \"packed-refs\" file, so we\n> need to explicitly check this in \"git refs verify\".\n>\n\nNot sure I understand how the start of this last sentence correlates to\nthe end of it. Is the intention to say that we want to explicitly check\nthe filetype to ensure that the 'packed-refs' file was only created via\n'git pack-refs'? If so, perhaps:\n\n Verify that the 'packed-refs' file has the expected filetype,\n confirming it was created by 'git pack-refs'.\n\n> We could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file.\n> If the returned \"fd\" value is less than 0, we could check whether the\n> \"errno\" is \"ELOOP\" to report an error to the user.\n>\n> Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n> the user if \"packed-refs\" is not a regular file.\n>\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> refs/packed-backend.c | 39 +++++++++++++++++++++++++++++++++++----\n> t/t0602-reffiles-fsck.sh | 22 ++++++++++++++++++++++\n> 2 files changed, 57 insertions(+), 4 deletions(-)\n>\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index a7b6f74b6e..6401cecd5f 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -4,6 +4,7 @@\n> #include \"../git-compat-util.h\"\n> #include \"../config.h\"\n> #include \"../dir.h\"\n> +#include \"../fsck.h\"\n> #include \"../gettext.h\"\n> #include \"../hash.h\"\n> #include \"../hex.h\"\n> @@ -1748,15 +1749,45 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> \treturn empty_ref_iterator_begin();\n> }\n>\n> -static int packed_fsck(struct ref_store *ref_store UNUSED,\n> -\t\t struct fsck_options *o UNUSED,\n> +static int packed_fsck(struct ref_store *ref_store,\n> +\t\t struct fsck_options *o,\n> \t\t struct worktree *wt)\n> {\n> +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> +\tint ret = 0;\n> +\tint fd;\n>\n> \tif (!is_main_worktree(wt))\n> -\t\treturn 0;\n> +\t\tgoto cleanup;\n>\n> -\treturn 0;\n> +\tif (o->verbose)\n> +\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n> +\n> +\tfd = open_nofollow(refs->path, O_RDONLY);\n> +\tif (fd < 0) {\n> +\t\t/*\n> +\t\t * If the packed-refs file doesn't exist, there's nothing\n> +\t\t * to check.\n> +\t\t */\n> +\t\tif (errno == ENOENT)\n> +\t\t\tgoto cleanup;\n> +\n> +\t\tif (errno == ELOOP) {\n> +\t\t\tstruct fsck_ref_report report = { 0 };\n> +\t\t\treport.path = \"packed-refs\";\n> +\t\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n> +\t\t\t\t\t \"not a regular file\");\n> +\t\t\tgoto cleanup;\n> +\t\t}\n> +\n> +\t\tret = error_errno(_(\"unable to open %s\"), refs->path);\n> +\t\tgoto cleanup;\n\nThe paragraph in the commit message:\n\n Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n the user if \"packed-refs\" is not a regular file.\n\nGave me the indication that any error would be reported via\n'fsck_report_ref()', but it seems like we are only reporting for\nsymbolic links. Why is that being singled out?\n\n> +\t}\n> +\n> +cleanup:\n> +\treturn ret;\n> }\n>\n> struct ref_storage_be refs_be_packed = {\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index cf7a202d0d..42c8d4ca1e 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -617,4 +617,26 @@ test_expect_success 'ref content checks should work with worktrees' '\n> \t)\n> '\n>\n> +test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\ttest_commit default &&\n> +\t\tgit branch branch-1 &&\n> +\t\tgit branch branch-2 &&\n> +\t\tgit branch branch-3 &&\n> +\t\tgit pack-refs --all &&\n> +\n> +\t\tmv .git/packed-refs .git/packed-refs-back &&\n> +\t\tln -sf packed-refs-bak .git/packed-refs &&\n\nThis still doesn't make sense to me. 'packed-refs-bak' doesn't exist, is\nthe intention to symlink '.git/packed-refs' -> something which doesn't\nexist?\n\nIn that case why even make the effort to build a packed-refs file, could\nwe simply do 'ln -sf packed-refs-bak .git/packed-refs' in an empty repo?\n\nIf not, then 'packed-refs-bak' is definitely a typo and needs to be made\n'packed-refs-back' which would go in hand with how we setup the test...\n\n> +\t\ttest_must_fail git refs verify 2>err &&\n> +\t\tcat >expect <<-EOF &&\n> +\t\terror: packed-refs: badRefFiletype: not a regular file\n> +\t\tEOF\n> +\t\trm .git/packed-refs &&\n> +\t\ttest_cmp expect err\n> +\t)\n> +'\n> +\n> test_done\n> --\n> 2.48.1\n"},{"id":"512413","messageId":"CAOLa=ZQTJhs+s+4y1DUpGDn7CnM5qwAgicgkcjA6ngmkbhwZyA@mail.gmail.com","threadId":"62743","inReplyTo":"Z67MG8utrQfUrakz@ArchLinux","subject":"Re: [PATCH v4 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-02-14T10:30:45Z","receivedAt":"2025-02-14T10:30:47Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> In \"packed-backend.c::create_snapshot\", if there is a header (the line\n> which starts with '#'), we will check whether the line starts with \"#\n> pack-refs with:\". Before we port this check into \"packed_fsck\", let's\n> fix \"create_snapshot\" to check the prefix \"# packed-ref with: \" instead\n> of \"# packed-ref with:\" due to that we will always write a single\n> trailing space after the colon.\n>\n\nOkay. So we're extending the check to also include the trailing space.\n\n>\n> However, we need to consider other situations and discuss whether we\n> need to add checks.\n>\n> 1. If the header does not exist, we should not report an error to the\n> user. This is because in older Git version, we never write header in\n> the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n> in runtime.\n\nMakes sense.\n\n> 2. If the header content does not start with \"# packed-ref with: \", we\n> should report an error just like what \"create_snapshot\" does. So,\n> create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n> 3. If the header content is not the same as the constant string\n> \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n> intentionally. So, there is no need to report.\n\nDo you think it's worthwhile adding a warning/info here? This would\nallow users to re-run 'git pack-refs' to ensure that they have a more\nup-to date version of 'packed-refs'.\n\n>\n> As we have analyzed, we only need to check the case 2 in the above. In\n> order to do this, read the \"packed-refs\" file via \"strbuf_read\". Like\n> what \"create_snapshot\" and other functions do, we could split the line\n> by finding the next newline in the buffer. When we cannot find a\n> newline, we could report an error.\n>\n> So, create a function \"packed_fsck_ref_next_line\" to find the next\n> newline and if there is no such newline, use\n> \"packedRefEntryNotTerminated(ERROR)\" to report an error to the user.\n>\n> Then, parse the first line to apply the checks. Update the test to\n> exercise the code.\n>\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> Documentation/fsck-msgids.txt | 8 ++++\n> fsck.h | 2 +\n> refs/packed-backend.c | 75 ++++++++++++++++++++++++++++++++++-\n> t/t0602-reffiles-fsck.sh | 52 ++++++++++++++++++++++++\n> 4 files changed, 136 insertions(+), 1 deletion(-)\n>\n> diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> index b14bc44ca4..11906f90fd 100644\n> --- a/Documentation/fsck-msgids.txt\n> +++ b/Documentation/fsck-msgids.txt\n> @@ -16,6 +16,10 @@\n> `badObjectSha1`::\n> \t(ERROR) An object has a bad sha1.\n>\n> +`badPackedRefHeader`::\n> +\t(ERROR) The \"packed-refs\" file contains an invalid\n> +\theader.\n> +\n> `badParentSha1`::\n> \t(ERROR) A commit object has a bad parent sha1.\n>\n> @@ -176,6 +180,10 @@\n> `nullSha1`::\n> \t(WARN) Tree contains entries pointing to a null sha1.\n>\n> +`packedRefEntryNotTerminated`::\n> +\t(ERROR) The \"packed-refs\" file contains an entry that is\n> +\tnot terminated by a newline.\n> +\n> `refMissingNewline`::\n> \t(INFO) A loose ref that does not end with newline(LF). As\n> \tvalid implementations of Git never created such a loose ref\n> diff --git a/fsck.h b/fsck.h\n> index a44c231a5f..67e3c97bc0 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -30,6 +30,7 @@ enum fsck_msg_type {\n> \tFUNC(BAD_EMAIL, ERROR) \\\n> \tFUNC(BAD_NAME, ERROR) \\\n> \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n> +\tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n> \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n> \tFUNC(BAD_REF_CONTENT, ERROR) \\\n> \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n> @@ -53,6 +54,7 @@ enum fsck_msg_type {\n> \tFUNC(MISSING_TYPE, ERROR) \\\n> \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n> \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n> +\tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n> \tFUNC(TREE_NOT_SORTED, ERROR) \\\n> \tFUNC(UNKNOWN_TYPE, ERROR) \\\n> \tFUNC(ZERO_PADDED_DATE, ERROR) \\\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 6401cecd5f..ff74ab915e 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -694,7 +694,7 @@ static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n>\n> \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n>\n> -\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n> +\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n> \t\t\tdie_invalid_line(refs->path,\n> \t\t\t\t\t snapshot->buf,\n> \t\t\t\t\t snapshot->eof - snapshot->buf);\n> @@ -1749,12 +1749,76 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> \treturn empty_ref_iterator_begin();\n> }\n>\n> +static int packed_fsck_ref_next_line(struct fsck_options *o,\n> +\t\t\t\t unsigned long line_number, const char *start,\n> +\t\t\t\t const char *eof, const char **eol)\n> +{\n> +\tint ret = 0;\n> +\n> +\t*eol = memchr(start, '\\n', eof - start);\n> +\tif (!*eol) {\n> +\t\tstruct strbuf packed_entry = STRBUF_INIT;\n> +\t\tstruct fsck_ref_report report = { 0 };\n> +\n> +\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n> +\t\treport.path = packed_entry.buf;\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n> +\t\t\t\t \"'%.*s' is not terminated with a newline\",\n> +\t\t\t\t (int)(eof - start), start);\n> +\n> +\t\t/*\n> +\t\t * There is no newline but we still want to parse it to the end of\n> +\t\t * the buffer.\n> +\t\t */\n> +\t\t*eol = eof;\n> +\t\tstrbuf_release(&packed_entry);\n> +\t}\n> +\n> +\treturn ret;\n> +}\n> +\n> +static int packed_fsck_ref_header(struct fsck_options *o,\n> +\t\t\t\t const char *start, const char *eol)\n> +{\n> +\tif (!starts_with(start, \"# pack-refs with: \")) {\n> +\t\tstruct fsck_ref_report report = { 0 };\n> +\t\treport.path = \"packed-refs.header\";\n> +\n> +\t\treturn fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n> +\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n> +\t\t\t\t (int)(eol - start), start);\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +static int packed_fsck_ref_content(struct fsck_options *o,\n> +\t\t\t\t const char *start, const char *eof)\n> +{\n> +\tunsigned long line_number = 1;\n> +\tconst char *eol;\n> +\tint ret = 0;\n> +\n> +\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n> +\tif (*start == '#') {\n> +\t\tret |= packed_fsck_ref_header(o, start, eol);\n> +\n> +\t\tstart = eol + 1;\n> +\t\tline_number++;\n\nWhy do we increment `line_number` here? There is no usage beyond this.\n\n> +\t}\n> +\n> +\treturn ret;\n> +}\n> +\n> static int packed_fsck(struct ref_store *ref_store,\n> \t\t struct fsck_options *o,\n> \t\t struct worktree *wt)\n> {\n> \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> +\tstruct strbuf packed_ref_content = STRBUF_INIT;\n> \tint ret = 0;\n> \tint fd;\n>\n> @@ -1786,7 +1850,16 @@ static int packed_fsck(struct ref_store *ref_store,\n> \t\tgoto cleanup;\n> \t}\n>\n> +\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n> +\t\tret = error_errno(_(\"unable to read %s\"), refs->path);\n> +\t\tgoto cleanup;\n> +\t}\n> +\n\nSo we want to parse the whole ref content to a buffer, wonder if it\nmakes more sense to use `strbuf_read_line()` here instead. But let's\ncarry on.\n\n> +\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n> +\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n> +\n\nWe pass the entire content and the EOF to the function.\n\n> cleanup:\n> +\tstrbuf_release(&packed_ref_content);\n> \treturn ret;\n> }\n>\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 42c8d4ca1e..30be1982df 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -639,4 +639,56 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> \t)\n> '\n>\n> +test_expect_success 'packed-refs header should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\ttest_commit default &&\n> +\n> +\t\tgit refs verify 2>err &&\n> +\t\ttest_must_be_empty err &&\n> +\n> +\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n> +\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n> +\t\t\t\t \"# pack-refs with a: peeled fully-peeled\" \\\n> +\t\t\t\t \"# pack-refs with:peeled fully-peeled sorted\"\n> +\t\tdo\n> +\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n> +\t\t\ttest_must_fail git refs verify 2>err &&\n> +\t\t\tcat >expect <<-EOF &&\n> +\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with: '\\''\n> +\t\t\tEOF\n> +\t\t\trm .git/packed-refs &&\n> +\t\t\ttest_cmp expect err || return 1\n> +\t\tdone\n> +\t)\n> +'\n> +\n> +test_expect_success 'packed-refs missing header should not be reported' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\ttest_commit default &&\n> +\n> +\t\tprintf \"$(git rev-parse HEAD) refs/heads/main\\n\" >.git/packed-refs &&\n> +\t\tgit refs verify 2>err &&\n> +\t\ttest_must_be_empty err\n> +\t)\n> +'\n> +\n> +test_expect_success 'packed-refs unknown traits should not be reported' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\ttest_commit default &&\n> +\n> +\t\tprintf \"# pack-refs with: peeled fully-peeled sorted foo\\n\" >.git/packed-refs &&\n> +\t\tgit refs verify 2>err &&\n> +\t\ttest_must_be_empty err\n> +\t)\n> +'\n> +\n> test_done\n> --\n> 2.48.1\n"},{"id":"512415","messageId":"Z680Hsu2ov_ETVzl@ArchLinux","threadId":"62743","inReplyTo":"CAOLa=ZQFLTFNc5AnvDyAaLvY8__R+J9RHZ29TM8COhPxnQs8Zg@mail.gmail.com","subject":"Re: [PATCH v4 0/8] add more ref consistency checks","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T12:16:30Z","receivedAt":"2025-02-14T12:14:36Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Feb 14, 2025 at 01:04:09AM -0800, Karthik Nayak wrote:\n> shejialuo writes:\n> \n> > Hi All:\n> >\n> > This patch enhances the following things:\n> >\n> > 1. [PATCH v4 4/8]: update the tests to verify that we don't report any\n> > errors to the user in some cases. Also, suggested by Junio, make sure\n> > that we check whether there is a trailing space after \"# packed-refs\n> > with:\".\n> > 2. [PATCH v4 6/8]: instead of greedily calculating the name of the line,\n> > lazily compute when there is any errors. And use the HERE docs to\n> > improve the test script.\n> > 3. [PATCH v4 7/8]: instead of storing the states, we parse the file\n> > again to check whether the file is sorted to avoid allocating too\n> > much memory. And use the HERE docs to improve the test script.\n> > 4. [PATCH v4 8/8]: update the documentation to emphasis the default. And\n> > add tests to exercise the code.\n> >\n> \n> Nit: For someone coming in to review the 4th version directly it would\n> be really nice to see:\n> \n> 1. Summary of what the patch series is about.\n> 2. Changes built over the last versions.\n> \n> I know all this information is already spread out over the previous\n> versions, but would be nice to have it here (in every version rather).\n> \n\nThanks for your suggestion, I will do this in my later patch.\n\n"},{"id":"512416","messageId":"Z6809gl2Hk73zZJ1@ArchLinux","threadId":"62743","inReplyTo":"CAOLa=ZS3w3KkEoQksXZtLYCT6BJVs6o2+nmpVUapbnqVA4zfng@mail.gmail.com","subject":"Re: [PATCH v4 2/8] builtin/refs: get worktrees without reading head information","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T12:20:06Z","receivedAt":"2025-02-14T12:18:14Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Feb 14, 2025 at 01:19:53AM -0800, Karthik Nayak wrote:\n> shejialuo writes:\n> \n> > In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\n> > and \"next_record\" which would check the correctness of the content of\n> > the \"packed-ref\" file. When anything is bad, the program will die.\n> >\n> > It may seem that we have nothing relevant to above feature, because we\n> > are going to read and parse the raw \"packed-ref\" file without creating\n> > the snapshot and using the ref iterator to check the consistency.\n> >\n> > However, when using \"get_worktrees\" in \"builtin/refs\", we would parse\n> > the \"HEAD\" information. If the referent of the \"HEAD\" is inside the\n> > \"packed-ref\", we will call \"create_snapshot\" function to parse the\n> > \"packed-ref\" to get the information. No matter whether the entry of\n> > \"HEAD\" in \"packed-ref\" is correct, \"create_snapshot\" would call\n> > \"verify_buffer_safe\" to check whether there is a newline in the last\n> > line of the file. If not, the program will die.\n> >\n> \n> Nit: while the second paragraph above makes sense in the context of what\n> we're trying to achieve in this patch series. It doesn't make much sense\n> for this patch in isolation. Perhaps we want to give some more context\n> around what we're trying to solve for in the upcoming patches and hence\n> how it hinders that.\n> \n\nIndeed, I think we should add this paragraph. We need to tell the\ncontext about the motivation.\n\n> > Although this behavior has no harm for the program, it will\n> > short-circuit the program. When the users execute \"git refs verify\" or\n> > \"git fsck\", we should avoid reading the head information, which may\n> > execute the read operation in packed backend with stricter checks to die\n> > the program. Instead, we should continue to check other parts of the\n> > \"packed-refs\" file completely.\n> >\n> > Fortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\n> > worktrees, 2023-12-29), we have introduced a function\n> > \"get_worktrees_internal\" which allows us to get worktrees without\n> > reading head information.\n> >\n> > Create a new exposed function \"get_worktrees_without_reading_head\", then\n> > replace the \"get_worktrees\" in \"builtin/refs\" with the new created\n> > function.\n> >\n> > Mentored-by: Patrick Steinhardt \n> > Mentored-by: Karthik Nayak \n> > Signed-off-by: shejialuo \n> > ---\n> > builtin/refs.c | 2 +-\n> > worktree.c | 5 +++++\n> > worktree.h | 6 ++++++\n> > 3 files changed, 12 insertions(+), 1 deletion(-)\n> >\n> > diff --git a/builtin/refs.c b/builtin/refs.c\n> > index a29f195834..55ff5dae11 100644\n> > --- a/builtin/refs.c\n> > +++ b/builtin/refs.c\n> > @@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n> > \tgit_config(git_fsck_config, &fsck_refs_options);\n> > \tprepare_repo_settings(the_repository);\n> >\n> > -\tworktrees = get_worktrees();\n> > +\tworktrees = get_worktrees_without_reading_head();\n> > \tfor (size_t i = 0; worktrees[i]; i++)\n> > \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n> > \t\t\t\t &fsck_refs_options, worktrees[i]);\n> > diff --git a/worktree.c b/worktree.c\n> > index 248bbb39d4..89b7d86cef 100644\n> > --- a/worktree.c\n> > +++ b/worktree.c\n> > @@ -175,6 +175,11 @@ struct worktree **get_worktrees(void)\n> > \treturn get_worktrees_internal(0);\n> > }\n> >\n> > +struct worktree **get_worktrees_without_reading_head(void)\n> > +{\n> > +\treturn get_worktrees_internal(1);\n> > +}\n> > +\n> > const char *get_worktree_git_dir(const struct worktree *wt)\n> > {\n> > \tif (!wt)\n> > diff --git a/worktree.h b/worktree.h\n> > index 38145df80f..1ba4a161a0 100644\n> > --- a/worktree.h\n> > +++ b/worktree.h\n> > @@ -30,6 +30,12 @@ struct worktree {\n> > */\n> > struct worktree **get_worktrees(void);\n> >\n> > +/*\n> > + * Like `get_worktrees`, but does not read HEAD. This is useful when checking\n> > + * the consistency, as reading HEAD may not be necessary.\n> \n> Checking what consistency? We should be a bit more verbose here. You can\n> mention that skipping HEAD allows to get the worktree without worrying\n> about failures pertaining to parsing the HEAD ref.\n> \n\nGood idea, I will improve this in the next version.\n\n> > + */\n> > +struct worktree **get_worktrees_without_reading_head(void);\n> > +\n> > /*\n> > * Returns 1 if linked worktrees exist, 0 otherwise.\n> > */\n> > --\n> > 2.48.1\n\n\n"},{"id":"512417","messageId":"Z685JSRGeZA2fuFq@ArchLinux","threadId":"62743","inReplyTo":"CAOLa=ZQ7CAXP-bYzTv3GJhauwtaL+pFj-2_QPWBh7SMiMsa6bQ@mail.gmail.com","subject":"Re: [PATCH v4 3/8] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T12:37:57Z","receivedAt":"2025-02-14T12:36:05Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Feb 14, 2025 at 01:50:26AM -0800, Karthik Nayak wrote:\n> shejialuo writes:\n> \n> > Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\n> > consistency and correctness of \"packed-refs\" file, they never check the\n> \n> Because you say 'some' here, it made me more curious. Could you state\n> exactly what checks are being done here?\n> \n\nWell, I don't think we need to elaborate on this at now for the\nfollowing two reasons:\n\n1. We will explain this in the later patches.\n2. Here I just want to emphasis that it does not check the filetype.\n\n> > filetype of the \"packed-refs\". The user should always use \"git\n> > pack-refs\" command to create the raw regular \"packed-refs\" file, so we\n> > need to explicitly check this in \"git refs verify\".\n> >\n> \n> Not sure I understand how the start of this last sentence correlates to\n> the end of it. Is the intention to say that we want to explicitly check\n> the filetype to ensure that the 'packed-refs' file was only created via\n> 'git pack-refs'? If so, perhaps:\n> \n> Verify that the 'packed-refs' file has the expected filetype,\n> confirming it was created by 'git pack-refs'.\n> \n\nThanks for the suggestion, I will improve this in the next version.\n\n> > We could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file.\n> > If the returned \"fd\" value is less than 0, we could check whether the\n> > \"errno\" is \"ELOOP\" to report an error to the user.\n> >\n> > Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n> > the user if \"packed-refs\" is not a regular file.\n> >\n> > Mentored-by: Patrick Steinhardt \n> > Mentored-by: Karthik Nayak \n> > Signed-off-by: shejialuo \n> > ---\n> > refs/packed-backend.c | 39 +++++++++++++++++++++++++++++++++++----\n> > t/t0602-reffiles-fsck.sh | 22 ++++++++++++++++++++++\n> > 2 files changed, 57 insertions(+), 4 deletions(-)\n> >\n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index a7b6f74b6e..6401cecd5f 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -4,6 +4,7 @@\n> > #include \"../git-compat-util.h\"\n> > #include \"../config.h\"\n> > #include \"../dir.h\"\n> > +#include \"../fsck.h\"\n> > #include \"../gettext.h\"\n> > #include \"../hash.h\"\n> > #include \"../hex.h\"\n> > @@ -1748,15 +1749,45 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> > \treturn empty_ref_iterator_begin();\n> > }\n> >\n> > -static int packed_fsck(struct ref_store *ref_store UNUSED,\n> > -\t\t struct fsck_options *o UNUSED,\n> > +static int packed_fsck(struct ref_store *ref_store,\n> > +\t\t struct fsck_options *o,\n> > \t\t struct worktree *wt)\n> > {\n> > +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> > +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> > +\tint ret = 0;\n> > +\tint fd;\n> >\n> > \tif (!is_main_worktree(wt))\n> > -\t\treturn 0;\n> > +\t\tgoto cleanup;\n> >\n> > -\treturn 0;\n> > +\tif (o->verbose)\n> > +\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n> > +\n> > +\tfd = open_nofollow(refs->path, O_RDONLY);\n> > +\tif (fd < 0) {\n> > +\t\t/*\n> > +\t\t * If the packed-refs file doesn't exist, there's nothing\n> > +\t\t * to check.\n> > +\t\t */\n> > +\t\tif (errno == ENOENT)\n> > +\t\t\tgoto cleanup;\n> > +\n> > +\t\tif (errno == ELOOP) {\n> > +\t\t\tstruct fsck_ref_report report = { 0 };\n> > +\t\t\treport.path = \"packed-refs\";\n> > +\t\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n> > +\t\t\t\t\t \"not a regular file\");\n> > +\t\t\tgoto cleanup;\n> > +\t\t}\n> > +\n> > +\t\tret = error_errno(_(\"unable to open %s\"), refs->path);\n> > +\t\tgoto cleanup;\n> \n> The paragraph in the commit message:\n> \n> Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n> the user if \"packed-refs\" is not a regular file.\n> \n> Gave me the indication that any error would be reported via\n> 'fsck_report_ref()', but it seems like we are only reporting for\n> symbolic links. Why is that being singled out?\n> \n\nIIRC, when Patrick told me in first version that if I first stat the\nfile type and then use the `strbuf_read_file` to read the content, there\nis a corner case that the file could be converted into symlink between\nthe `stat` and read.\n\nSo, I use `open_nofollow` to avoid this situation. (Actually, this could\nnot be avoided because in Windows, we would first stat the file and\nthen open the file due to that there is no \"O_NOFOLLOW\" flag for Windows).\n\nI will find a solution to do this in the next version.\n\n> > +\t}\n> > +\n> > +cleanup:\n> > +\treturn ret;\n> > }\n> >\n> > struct ref_storage_be refs_be_packed = {\n> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> > index cf7a202d0d..42c8d4ca1e 100755\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -617,4 +617,26 @@ test_expect_success 'ref content checks should work with worktrees' '\n> > \t)\n> > '\n> >\n> > +test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\t(\n> > +\t\tcd repo &&\n> > +\t\ttest_commit default &&\n> > +\t\tgit branch branch-1 &&\n> > +\t\tgit branch branch-2 &&\n> > +\t\tgit branch branch-3 &&\n> > +\t\tgit pack-refs --all &&\n> > +\n> > +\t\tmv .git/packed-refs .git/packed-refs-back &&\n> > +\t\tln -sf packed-refs-bak .git/packed-refs &&\n> \n> This still doesn't make sense to me. 'packed-refs-bak' doesn't exist, is\n> the intention to symlink '.git/packed-refs' -> something which doesn't\n> exist?\n> \n> In that case why even make the effort to build a packed-refs file, could\n> we simply do 'ln -sf packed-refs-bak .git/packed-refs' in an empty repo?\n> \n\nYou are correct. My intention is not this. If the \"packed-refs\" is a\nsymlink and points to file which we can successfully parse. Current Git\nwon't complain. So my motivation here is to imitate this situation.\n\n> If not, then 'packed-refs-bak' is definitely a typo and needs to be made\n> 'packed-refs-back' which would go in hand with how we setup the test...\n> \n\nThanks for noticing this problem. I definitely made a mistake to type the\n\"packed-refs-back\" to \"packed-refs-bak\".\n\nJialuo\n"},{"id":"512425","messageId":"Z686jytnvdzdG6HJ@ArchLinux","threadId":"62743","inReplyTo":"CAOLa=ZQTJhs+s+4y1DUpGDn7CnM5qwAgicgkcjA6ngmkbhwZyA@mail.gmail.com","subject":"Re: [PATCH v4 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-14T12:43:59Z","receivedAt":"2025-02-14T12:42:06Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Feb 14, 2025 at 02:30:45AM -0800, Karthik Nayak wrote:\n> shejialuo writes:\n\n[snip]\n\n> > 2. If the header content does not start with \"# packed-ref with: \", we\n> > should report an error just like what \"create_snapshot\" does. So,\n> > create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n> > 3. If the header content is not the same as the constant string\n> > \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n> > intentionally. So, there is no need to report.\n> \n> Do you think it's worthwhile adding a warning/info here? This would\n> allow users to re-run 'git pack-refs' to ensure that they have a more\n> up-to date version of 'packed-refs'.\n> \n\nI somehow agree with you here. But Junio worries about the\ncompatibility. You could see [1] about this discussion:\n\n[1] https://lore.kernel.org/git/xmqq1pwkdt7r.fsf@gitster.g/\n\n[snip]\n\n> > +static int packed_fsck_ref_content(struct fsck_options *o,\n> > +\t\t\t\t const char *start, const char *eof)\n> > +{\n> > +\tunsigned long line_number = 1;\n> > +\tconst char *eol;\n> > +\tint ret = 0;\n> > +\n> > +\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n> > +\tif (*start == '#') {\n> > +\t\tret |= packed_fsck_ref_header(o, start, eol);\n> > +\n> > +\t\tstart = eol + 1;\n> > +\t\tline_number++;\n> \n> Why do we increment `line_number` here? There is no usage beyond this.\n> \n\nWe will use this variable when iterating the next line (ref entries). It\nwill be used in next patch.\n\n> > +\t}\n> > +\n> > +\treturn ret;\n> > +}\n> > +\n> > static int packed_fsck(struct ref_store *ref_store,\n> > \t\t struct fsck_options *o,\n> > \t\t struct worktree *wt)\n> > {\n> > \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> > \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> > +\tstruct strbuf packed_ref_content = STRBUF_INIT;\n> > \tint ret = 0;\n> > \tint fd;\n> >\n> > @@ -1786,7 +1850,16 @@ static int packed_fsck(struct ref_store *ref_store,\n> > \t\tgoto cleanup;\n> > \t}\n> >\n> > +\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n> > +\t\tret = error_errno(_(\"unable to read %s\"), refs->path);\n> > +\t\tgoto cleanup;\n> > +\t}\n> > +\n> \n> So we want to parse the whole ref content to a buffer, wonder if it\n> makes more sense to use `strbuf_read_line()` here instead. But let's\n> carry on.\n> \n\nWe may use `strbuf_read_line`. But I don't want to do this. My check\nlogic is the same as the parse logic (\"create_snapshot\" and \"next_record\").\nI want to keep the logic nearly the same. So maybe one day, we may\nrefactor the code to make the parse and check use the same code. But at\nnow, this is difficult.\n\nThanks,\nJialuo\n"},{"id":"512427","messageId":"xmqq4j0wmxqx.fsf@gitster.g","threadId":"62743","inReplyTo":"Z67MG8utrQfUrakz@ArchLinux","subject":"Re: [PATCH v4 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-14T14:01:10Z","receivedAt":"2025-02-14T14:01:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> In \"packed-backend.c::create_snapshot\", if there is a header (the line\n> which starts with '#'), we will check whether the line starts with \"#\n> pack-refs with:\". Before we port this check into \"packed_fsck\", let's\n> fix \"create_snapshot\" to check the prefix \"# packed-ref with: \" instead\n> of \"# packed-ref with:\" due to that we will always write a single\n> trailing space after the colon.\n\nA more important reason to be more strict is not \"we will always\nwrite\", but \"we HAVE ALWAYS written\", I think.\n\n> However, we need to consider other situations and discuss whether we\n> need to add checks.\n>\n> 1. If the header does not exist, we should not report an error to the\n> user. This is because in older Git version, we never write header in\n> the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n> in runtime.\n\nYes.\n\n> 2. If the header content does not start with \"# packed-ref with: \", we\n> should report an error just like what \"create_snapshot\" does. So,\n> create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n\nOK.\n\n> 3. If the header content is not the same as the constant string\n> \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n> intentionally. So, there is no need to report.\n\nNor there is any need to check for literal equality with the\nconstant string. We may want to split the traits that are recorded\non the \"with:\" line and see if there are ones that we do not\nrecognise if only for curiosity, but because create_snapshot(), which\nis the only run-time consumer of this information, only uses the\nones it recognises while ignoring everything else, presence of an\nunknown trait is not an error- or even warning-worthy event. Unless\nwe are curious and want to emit \"info\" level message, there is not\nmuch point in checking the remainder of the header.\n"},{"id":"512512","messageId":"Z7M2m0diauWW2ARQ@ArchLinux","threadId":"62743","inReplyTo":"Z4kQUb7og2Ce1iCo@pks.im","subject":"Re: [PATCH 04/10] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T13:16:11Z","receivedAt":"2025-02-17T13:16:14Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Jan 16, 2025 at 02:57:37PM +0100, Patrick Steinhardt wrote:\n\n[snip]\n\n> > @@ -1779,7 +1867,24 @@ static int packed_fsck(struct ref_store *ref_store,\n> > \t\tgoto cleanup;\n> > \t}\n> > \n> > +\tif (strbuf_read_file(&packed_ref_content, refs->path, 0) < 0) {\n> > +\t\t/*\n> > +\t\t * Although we have checked that the file exists, there is a possibility\n> > +\t\t * that it has been removed between the lstat() and the read attempt by\n> > +\t\t * another process. In that case, we should not report an error.\n> > +\t\t */\n> > +\t\tif (errno == ENOENT)\n> > +\t\t\tgoto cleanup;\n> \n> Unlikely, but good to guard us against that condition regardless. It's\n> still not entirely race-free though because the file could meanwhile\n> have changed into a symlink, and we wouldn't notice now. We could fix\n> that by using open(O_NOFOLLOW), fstat the returne file descriptor and\n> then use `strbuf_read()` to slurp in the file.\n> \n\nI have been looking back to the original discussion. I will follow this\nadvice which eventually avoids the race.\n\nThanks,\nJialuo\n"},{"id":"512517","messageId":"Z7NU5fZfc8vfSvZ0@ArchLinux","threadId":"62743","inReplyTo":"Z67LkxAFIAeaYr0U@ArchLinux","subject":"[PATCH v5 0/8] add more ref consistency checks","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T15:25:25Z","receivedAt":"2025-02-17T15:25:29Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis changes enhances the following things:\n\n1. [PATCH v5 2/8]: enhance the comment suggested by Karthik.\n2. [PATCH v5 3/8]: use lstat to check whether the filetype of\n \"packed-ref\" is a regular file instead of using `open_nofollow`\n to check. And also enhance the commit message suggested by Karthik.\n3. [PATCH v5 4/8]: move \"open_nofollow\" in original [PATCH v4 3/8] to\n this.\n\nAlso, I rebase due to the conflict that all *.txt files have been\nrenamed to *.adoc. However, I don't know whether this is a real\nconflict. But I decide to rebase to make the life of Junio easy.\n\nThanks,\nJialuo\n\n---\n\nThis series mainly does the following things:\n\n1. Fix subshell issues\n2. Add ref checks for packed-backend.\n 1. Check whether the filetype of \"packed-refs\" is correct.\n 2. Check whether the syntax of \"packed-refs\" is correct by using the\n rules from \"packed-backend.c::create_snapshot\" and\n \"packed-backend.c::next_record\".\n 3. Check whether the pointed object exists and whether the\n \"packed-refs\" file is sorted.\n3. Call \"git refs verify\" for \"git-fsck(1)\".\n\nshejialuo (8):\n t0602: use subshell to ensure working directory unchanged\n builtin/refs: get worktrees without reading head information\n packed-backend: check whether the \"packed-refs\" is regular file\n packed-backend: add \"packed-refs\" header consistency check\n packed-backend: check whether the refname contains NUL characters\n packed-backend: add \"packed-refs\" entry consistency check\n packed-backend: check whether the \"packed-refs\" is sorted\n builtin/fsck: add `git refs verify` child process\n\n Documentation/fsck-msgids.adoc | 14 +\n Documentation/git-fsck.adoc | 7 +-\n builtin/fsck.c | 33 +-\n builtin/refs.c | 2 +-\n fsck.h | 4 +\n refs/packed-backend.c | 369 +++++++++-\n t/t0602-reffiles-fsck.sh | 1205 +++++++++++++++++++-------------\n worktree.c | 5 +\n worktree.h | 7 +\n 9 files changed, 1161 insertions(+), 485 deletions(-)\n\nRange-diff against v4:\n1: 20889b7b18 = 1: b3952d80a2 t0602: use subshell to ensure working directory unchanged\n2: 9d7780e953 ! 2: 3695586f58 builtin/refs: get worktrees without reading head information\n @@ worktree.h: struct worktree {\n struct worktree **get_worktrees(void);\n \n +/*\n -+ * Like `get_worktrees`, but does not read HEAD. This is useful when checking\n -+ * the consistency, as reading HEAD may not be necessary.\n ++ * Like `get_worktrees`, but does not read HEAD. Skip reading HEAD allows to\n ++ * get the worktree without worrying about failures pertaining to parsing\n ++ * the HEAD ref. This is useful when we want to check the ref db consistency.\n + */\n +struct worktree **get_worktrees_without_reading_head(void);\n +\n3: 44d26f6440 ! 3: cbaae00e8b packed-backend: check whether the \"packed-refs\" is regular file\n @@ Commit message\n \n Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\n consistency and correctness of \"packed-refs\" file, they never check the\n - filetype of the \"packed-refs\". The user should always use \"git\n - pack-refs\" command to create the raw regular \"packed-refs\" file, so we\n - need to explicitly check this in \"git refs verify\".\n + filetype of the \"packed-refs\". Let's verify that the \"packed-refs\" has\n + the expected filetype, confirming it is created by \"git pack-refs\"\n + command.\n \n - We could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file.\n - If the returned \"fd\" value is less than 0, we could check whether the\n - \"errno\" is \"ELOOP\" to report an error to the user.\n + Use \"lstat\" to check the file mode. If we cannot check the file status\n + due to there is no such file this is OK because there is a possibility\n + that there is no \"packed-refs\" in the repo.\n \n Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n the user if \"packed-refs\" is not a regular file.\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n {\n +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n ++\tstruct stat st;\n +\tint ret = 0;\n -+\tint fd;\n \n \tif (!is_main_worktree(wt))\n -\t\treturn 0;\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\tif (o->verbose)\n +\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n +\n -+\tfd = open_nofollow(refs->path, O_RDONLY);\n -+\tif (fd < 0) {\n ++\tif (lstat(refs->path, &st) < 0) {\n +\t\t/*\n +\t\t * If the packed-refs file doesn't exist, there's nothing\n +\t\t * to check.\n +\t\t */\n +\t\tif (errno == ENOENT)\n +\t\t\tgoto cleanup;\n ++\t\tret = error_errno(_(\"unable to stat %s\"), refs->path);\n ++\t\tgoto cleanup;\n ++\t}\n +\n -+\t\tif (errno == ELOOP) {\n -+\t\t\tstruct fsck_ref_report report = { 0 };\n -+\t\t\treport.path = \"packed-refs\";\n -+\t\t\tret = fsck_report_ref(o, &report,\n -+\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n -+\t\t\t\t\t \"not a regular file\");\n -+\t\t\tgoto cleanup;\n -+\t\t}\n -+\n -+\t\tret = error_errno(_(\"unable to open %s\"), refs->path);\n ++\tif (!S_ISREG(st.st_mode)) {\n ++\t\tstruct fsck_ref_report report = { 0 };\n ++\t\treport.path = \"packed-refs\";\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n ++\t\t\t\t \"not a regular file\");\n +\t\tgoto cleanup;\n +\t}\n +\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref content checks should work wi\n +\t\tgit pack-refs --all &&\n +\n +\t\tmv .git/packed-refs .git/packed-refs-back &&\n -+\t\tln -sf packed-refs-bak .git/packed-refs &&\n ++\t\tln -sf packed-refs-back .git/packed-refs &&\n +\t\ttest_must_fail git refs verify 2>err &&\n +\t\tcat >expect <<-EOF &&\n +\t\terror: packed-refs: badRefFiletype: not a regular file\n4: 976c5baba0 ! 4: b9ce8734ac packed-backend: add \"packed-refs\" header consistency check\n @@ Commit message\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n 3. If the header content is not the same as the constant string\n \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n - intentionally. So, there is no need to report.\n + intentionally and runtime \"create_snapshot\" won't complain about\n + unknown traits. In order to align with the runtime behavior. There is\n + no need to report.\n \n As we have analyzed, we only need to check the case 2 in the above. In\n - order to do this, read the \"packed-refs\" file via \"strbuf_read\". Like\n + order to do this, use \"open_nofollow\" function to get the file\n + descriptor and then read the \"packed-refs\" file via \"strbuf_read\". Like\n what \"create_snapshot\" and other functions do, we could split the line\n by finding the next newline in the buffer. When we cannot find a\n newline, we could report an error.\n @@ Commit message\n Mentored-by: Karthik Nayak \n Signed-off-by: shejialuo \n \n - ## Documentation/fsck-msgids.txt ##\n + ## Documentation/fsck-msgids.adoc ##\n @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n +\tstruct strbuf packed_ref_content = STRBUF_INIT;\n + \tstruct stat st;\n ++\tint fd;\n \tint ret = 0;\n - \tint fd;\n \n + \tif (!is_main_worktree(wt))\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n ++\t/*\n ++\t * There is a chance that \"packed-refs\" file is removed or converted to\n ++\t * a symlink after filetype check and before open. So we need to avoid\n ++\t * this race condition by opening the file.\n ++\t */\n ++\tfd = open_nofollow(refs->path, O_RDONLY);\n ++\tif (fd < 0) {\n ++\t\tif (errno == ENOENT)\n ++\t\t\tgoto cleanup;\n ++\n ++\t\tif (errno == ELOOP) {\n ++\t\t\tstruct fsck_ref_report report = { 0 };\n ++\t\t\treport.path = \"packed-refs\";\n ++\t\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n ++\t\t\t\t\t \"not a regular file\");\n ++\t\t\tgoto cleanup;\n ++\t\t}\n ++\t}\n ++\n +\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n +\t\tret = error_errno(_(\"unable to read %s\"), refs->path);\n +\t\tgoto cleanup;\n5: b66f142d7f = 5: 9f638b3adf packed-backend: check whether the refname contains NUL characters\n6: f68028e171 ! 6: 2c5395bdd0 packed-backend: add \"packed-refs\" entry consistency check\n @@ Commit message\n Mentored-by: Karthik Nayak \n Signed-off-by: shejialuo \n \n - ## Documentation/fsck-msgids.txt ##\n + ## Documentation/fsck-msgids.adoc ##\n @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n @@ refs/packed-backend.c: static int packed_fsck_ref_header(struct fsck_options *o,\n +\t\t\t\t (int)(eol - p), p);\n +\t\tgoto cleanup;\n +\t}\n ++\n +cleanup:\n +\tstrbuf_release(&packed_entry);\n +\treturn ret;\n7: 4a7adf293f ! 7: 648404c60d packed-backend: check whether the \"packed-refs\" is sorted\n @@ Commit message\n Mentored-by: Karthik Nayak \n Signed-off-by: shejialuo \n \n - ## Documentation/fsck-msgids.txt ##\n + ## Documentation/fsck-msgids.adoc ##\n @@\n \t(ERROR) The \"packed-refs\" file contains an entry that is\n \tnot terminated by a newline.\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n \tstruct strbuf packed_ref_content = STRBUF_INIT;\n +\tunsigned int sorted = 0;\n + \tstruct stat st;\n +-\tint fd;\n \tint ret = 0;\n - \tint fd;\n ++\tint fd;\n \n + \tif (!is_main_worktree(wt))\n + \t\tgoto cleanup;\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n8: 2dd3437478 ! 8: 4dbbacf44b builtin/fsck: add `git refs verify` child process\n @@ Commit message\n Mentored-by: Karthik Nayak \n Signed-off-by: shejialuo \n \n - ## Documentation/git-fsck.txt ##\n -@@ Documentation/git-fsck.txt: SYNOPSIS\n + ## Documentation/git-fsck.adoc ##\n +@@ Documentation/git-fsck.adoc: SYNOPSIS\n 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n @@ Documentation/git-fsck.txt: SYNOPSIS\n \n DESCRIPTION\n -----------\n -@@ Documentation/git-fsck.txt: care about this output and want to speed it up further.\n +@@ Documentation/git-fsck.adoc: care about this output and want to speed it up further.\n \tprogress status even if the standard error stream is not\n \tdirected to a terminal.\n \n-- \n2.48.1\n\n"},{"id":"512518","messageId":"Z7NVXehUfi15FA_Y@ArchLinux","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"[PATCH v5 1/8] t0602: use subshell to ensure working directory unchanged","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T15:27:25Z","receivedAt":"2025-02-17T15:27:30Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"For every test, we would execute the command \"cd repo\" in the first but\nwe never execute the command \"cd ..\" to restore the working directory.\nHowever, it's either not a good idea use above way. Because if any test\nfails between \"cd repo\" and \"cd ..\", the \"cd ..\" will never be reached.\nAnd we cannot correctly restore the working directory.\n\nLet's use subshell to ensure that the current working directory could be\nrestored to the correct path.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n t/t0602-reffiles-fsck.sh | 967 ++++++++++++++++++++-------------------\n 1 file changed, 494 insertions(+), 473 deletions(-)\n\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex d4a08b823b..cf7a202d0d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -14,222 +14,229 @@ test_expect_success 'ref name should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b default-branch &&\n-\tgit tag default-tag &&\n-\tgit tag multi_hierarchy/default-tag &&\n-\n-\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n-\trm $branch_dir_prefix/@ &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n-\tgit refs verify 2>err &&\n-\trm $tag_dir_prefix/tag-1.lock &&\n-\ttest_must_be_empty err &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/.lock: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t(\n+\t\tcd repo &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b default-branch &&\n+\t\tgit tag default-tag &&\n+\t\tgit tag multi_hierarchy/default-tag &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\t\trm $branch_dir_prefix/@ &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $tag_dir_prefix/tag-1.lock &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \t\ttest_must_fail git refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\terror: refs/tags/.lock: badRefName: invalid refname format\n \t\tEOF\n-\t\trm -r \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $tag_dir_prefix/.lock &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=warn refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n-\ttest_must_be_empty err\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b branch-1 &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=warn refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm $branch_dir_prefix/.branch-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n '\n \n test_expect_success 'ref name check should work for multiple worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\n-\tcd repo &&\n-\ttest_commit initial &&\n-\tgit checkout -b branch-1 &&\n-\ttest_commit second &&\n-\tgit checkout -b branch-2 &&\n-\ttest_commit third &&\n-\tgit checkout -b branch-3 &&\n-\tgit worktree add ./worktree-1 branch-1 &&\n-\tgit worktree add ./worktree-2 branch-2 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n-\t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n \t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n-\n-\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n-\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err &&\n-\n-\tfor worktree in \"worktree-1\" \"worktree-2\"\n-\tdo\n+\t\tcd repo &&\n+\t\ttest_commit initial &&\n+\t\tgit checkout -b branch-1 &&\n+\t\ttest_commit second &&\n+\t\tgit checkout -b branch-2 &&\n+\t\ttest_commit third &&\n+\t\tgit checkout -b branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-1 &&\n+\t\tgit worktree add ./worktree-2 branch-2 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n \t\t(\n-\t\t\tcd $worktree &&\n-\t\t\ttest_must_fail git refs verify 2>err &&\n-\t\t\tcat >expect <<-EOF &&\n-\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\t\t\tEOF\n-\t\t\tsort err >sorted_err &&\n-\t\t\ttest_cmp expect sorted_err || return 1\n-\t\t)\n-\tdone\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\n+\t\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\t\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err &&\n+\n+\t\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\t\tdo\n+\t\t\t(\n+\t\t\t\tcd $worktree &&\n+\t\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\t\tcat >expect <<-EOF &&\n+\t\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\t\tEOF\n+\t\t\t\tsort err >sorted_err &&\n+\t\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t\t)\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tfor trailing_content in \" garbage\" \" more garbage\"\n-\tdo\n-\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-garbage &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n+\t\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n-\t'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\t'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n \n-\t garbage'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err\n+\t\t garbage'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -237,99 +244,103 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tbad_content_1=$(git rev-parse main)x &&\n-\tbad_content_2=xfsazqfxcadas &&\n-\tbad_content_3=Xfsazqfxcadas &&\n-\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n-\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n-\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n-\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n-\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n-\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tbad_content_1=$(git rev-parse main)x &&\n+\t\tbad_content_2=xfsazqfxcadas &&\n+\t\tbad_content_3=Xfsazqfxcadas &&\n+\t\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\t\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\t\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\t\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\t\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-complicated &&\n-\ttest_cmp expect err\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (aggregate)' '\n@@ -337,32 +348,34 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n-\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'the target of the textual symref should be checked' '\n@@ -370,28 +383,30 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n-\t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n-\n-\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n-\t\tgit refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked' '\n@@ -399,201 +414,207 @@ test_expect_success SYMLINKS 'symlink symref content should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n-\tEOF\n-\trm $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_cmp expect err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-good &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\t\tEOF\n+\t\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tmain_worktree_refdir_prefix=.git/refs/heads &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\n-\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tfor bad_referent_name in \".tag\" \"branch \"\n-\tdo\n-\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tmain_worktree_refdir_prefix=.git/refs/heads &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $worktree1_refdir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor bad_referent_name in \".tag\" \"branch \"\n+\t\tdo\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-garbage &&\n-\ttest_cmp expect err\n+\t\trm $worktree1_refdir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_done\n-- \n2.48.1\n\n"},{"id":"512519","messageId":"Z7NVZvAlNJ_00WX3@ArchLinux","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"[PATCH v5 2/8] builtin/refs: get worktrees without reading head information","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T15:27:34Z","receivedAt":"2025-02-17T15:27:37Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\nand \"next_record\" which would check the correctness of the content of\nthe \"packed-ref\" file. When anything is bad, the program will die.\n\nIt may seem that we have nothing relevant to above feature, because we\nare going to read and parse the raw \"packed-ref\" file without creating\nthe snapshot and using the ref iterator to check the consistency.\n\nHowever, when using \"get_worktrees\" in \"builtin/refs\", we would parse\nthe \"HEAD\" information. If the referent of the \"HEAD\" is inside the\n\"packed-ref\", we will call \"create_snapshot\" function to parse the\n\"packed-ref\" to get the information. No matter whether the entry of\n\"HEAD\" in \"packed-ref\" is correct, \"create_snapshot\" would call\n\"verify_buffer_safe\" to check whether there is a newline in the last\nline of the file. If not, the program will die.\n\nAlthough this behavior has no harm for the program, it will\nshort-circuit the program. When the users execute \"git refs verify\" or\n\"git fsck\", we should avoid reading the head information, which may\nexecute the read operation in packed backend with stricter checks to die\nthe program. Instead, we should continue to check other parts of the\n\"packed-refs\" file completely.\n\nFortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\nworktrees, 2023-12-29), we have introduced a function\n\"get_worktrees_internal\" which allows us to get worktrees without\nreading head information.\n\nCreate a new exposed function \"get_worktrees_without_reading_head\", then\nreplace the \"get_worktrees\" in \"builtin/refs\" with the new created\nfunction.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 2 +-\n worktree.c | 5 +++++\n worktree.h | 7 +++++++\n 3 files changed, 13 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex a29f195834..55ff5dae11 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tworktrees = get_worktrees();\n+\tworktrees = get_worktrees_without_reading_head();\n \tfor (size_t i = 0; worktrees[i]; i++)\n \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n \t\t\t\t &fsck_refs_options, worktrees[i]);\ndiff --git a/worktree.c b/worktree.c\nindex d4a68c9c23..d23482a746 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -198,6 +198,11 @@ struct worktree **get_worktrees(void)\n \treturn get_worktrees_internal(0);\n }\n \n+struct worktree **get_worktrees_without_reading_head(void)\n+{\n+\treturn get_worktrees_internal(1);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\ndiff --git a/worktree.h b/worktree.h\nindex 38145df80f..f7003a9c12 100644\n--- a/worktree.h\n+++ b/worktree.h\n@@ -30,6 +30,13 @@ struct worktree {\n */\n struct worktree **get_worktrees(void);\n \n+/*\n+ * Like `get_worktrees`, but does not read HEAD. Skip reading HEAD allows to\n+ * get the worktree without worrying about failures pertaining to parsing\n+ * the HEAD ref. This is useful when we want to check the ref db consistency.\n+ */\n+struct worktree **get_worktrees_without_reading_head(void);\n+\n /*\n * Returns 1 if linked worktrees exist, 0 otherwise.\n */\n-- \n2.48.1\n\n"},{"id":"512520","messageId":"Z7NVbvyZTxspTjWX@ArchLinux","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"[PATCH v5 3/8] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T15:27:42Z","receivedAt":"2025-02-17T15:27:45Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\nconsistency and correctness of \"packed-refs\" file, they never check the\nfiletype of the \"packed-refs\". Let's verify that the \"packed-refs\" has\nthe expected filetype, confirming it is created by \"git pack-refs\"\ncommand.\n\nUse \"lstat\" to check the file mode. If we cannot check the file status\ndue to there is no such file this is OK because there is a possibility\nthat there is no \"packed-refs\" in the repo.\n\nReuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\nthe user if \"packed-refs\" is not a regular file.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 37 +++++++++++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 22 ++++++++++++++++++++++\n 2 files changed, 55 insertions(+), 4 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a7b6f74b6e..8140a31d07 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -4,6 +4,7 @@\n #include \"../git-compat-util.h\"\n #include \"../config.h\"\n #include \"../dir.h\"\n+#include \"../fsck.h\"\n #include \"../gettext.h\"\n #include \"../hash.h\"\n #include \"../hex.h\"\n@@ -1748,15 +1749,43 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n-static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED,\n+static int packed_fsck(struct ref_store *ref_store,\n+\t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n+\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n+\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct stat st;\n+\tint ret = 0;\n \n \tif (!is_main_worktree(wt))\n-\t\treturn 0;\n+\t\tgoto cleanup;\n \n-\treturn 0;\n+\tif (o->verbose)\n+\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n+\n+\tif (lstat(refs->path, &st) < 0) {\n+\t\t/*\n+\t\t * If the packed-refs file doesn't exist, there's nothing\n+\t\t * to check.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\t\tret = error_errno(_(\"unable to stat %s\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (!S_ISREG(st.st_mode)) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs\";\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t \"not a regular file\");\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\treturn ret;\n }\n \n struct ref_storage_be refs_be_packed = {\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex cf7a202d0d..e65ca341cd 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -617,4 +617,26 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t)\n '\n \n+test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit pack-refs --all &&\n+\n+\t\tmv .git/packed-refs .git/packed-refs-back &&\n+\t\tln -sf packed-refs-back .git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs: badRefFiletype: not a regular file\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512521","messageId":"Z7NVdhM61rhjAHtW@ArchLinux","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"[PATCH v5 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T15:27:50Z","receivedAt":"2025-02-17T15:27:53Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c::create_snapshot\", if there is a header (the line\nwhich starts with '#'), we will check whether the line starts with \"#\npack-refs with:\". Before we port this check into \"packed_fsck\", let's\nfix \"create_snapshot\" to check the prefix \"# packed-ref with: \" instead\nof \"# packed-ref with:\" due to that we will always write a single\ntrailing space after the colon.\n\nHowever, we need to consider other situations and discuss whether we\nneed to add checks.\n\n1. If the header does not exist, we should not report an error to the\n user. This is because in older Git version, we never write header in\n the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n in runtime.\n2. If the header content does not start with \"# packed-ref with: \", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n3. If the header content is not the same as the constant string\n \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n intentionally and runtime \"create_snapshot\" won't complain about\n unknown traits. In order to align with the runtime behavior. There is\n no need to report.\n\nAs we have analyzed, we only need to check the case 2 in the above. In\norder to do this, use \"open_nofollow\" function to get the file\ndescriptor and then read the \"packed-refs\" file via \"strbuf_read\". Like\nwhat \"create_snapshot\" and other functions do, we could split the line\nby finding the next newline in the buffer. When we cannot find a\nnewline, we could report an error.\n\nSo, create a function \"packed_fsck_ref_next_line\" to find the next\nnewline and if there is no such newline, use\n\"packedRefEntryNotTerminated(ERROR)\" to report an error to the user.\n\nThen, parse the first line to apply the checks. Update the test to\nexercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 8 +++\n fsck.h | 2 +\n refs/packed-backend.c | 96 +++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 52 ++++++++++++++++++\n 4 files changed, 157 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex b14bc44ca4..11906f90fd 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -16,6 +16,10 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefHeader`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid\n+\theader.\n+\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n@@ -176,6 +180,10 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`packedRefEntryNotTerminated`::\n+\t(ERROR) The \"packed-refs\" file contains an entry that is\n+\tnot terminated by a newline.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex a44c231a5f..67e3c97bc0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n@@ -53,6 +54,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE, ERROR) \\\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 8140a31d07..09eb3886c3 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -694,7 +694,7 @@ static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n \n \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n \n-\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n+\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n \t\t\tdie_invalid_line(refs->path,\n \t\t\t\t\t snapshot->buf,\n \t\t\t\t\t snapshot->eof - snapshot->buf);\n@@ -1749,13 +1749,78 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+static int packed_fsck_ref_next_line(struct fsck_options *o,\n+\t\t\t\t unsigned long line_number, const char *start,\n+\t\t\t\t const char *eof, const char **eol)\n+{\n+\tint ret = 0;\n+\n+\t*eol = memchr(start, '\\n', eof - start);\n+\tif (!*eol) {\n+\t\tstruct strbuf packed_entry = STRBUF_INIT;\n+\t\tstruct fsck_ref_report report = { 0 };\n+\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n+\t\t\t\t \"'%.*s' is not terminated with a newline\",\n+\t\t\t\t (int)(eof - start), start);\n+\n+\t\t/*\n+\t\t * There is no newline but we still want to parse it to the end of\n+\t\t * the buffer.\n+\t\t */\n+\t\t*eol = eof;\n+\t\tstrbuf_release(&packed_entry);\n+\t}\n+\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_header(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs.header\";\n+\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tunsigned long line_number = 1;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\tif (*start == '#') {\n+\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\treturn ret;\n+}\n+\n static int packed_fsck(struct ref_store *ref_store,\n \t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tstruct stat st;\n+\tint fd;\n \tint ret = 0;\n \n \tif (!is_main_worktree(wt))\n@@ -1784,7 +1849,36 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\t/*\n+\t * There is a chance that \"packed-refs\" file is removed or converted to\n+\t * a symlink after filetype check and before open. So we need to avoid\n+\t * this race condition by opening the file.\n+\t */\n+\tfd = open_nofollow(refs->path, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tif (errno == ELOOP) {\n+\t\t\tstruct fsck_ref_report report = { 0 };\n+\t\t\treport.path = \"packed-refs\";\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t\t \"not a regular file\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n+\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n+\t\tret = error_errno(_(\"unable to read %s\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\n cleanup:\n+\tstrbuf_release(&packed_ref_content);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex e65ca341cd..e055c36e74 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -639,4 +639,56 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-refs header should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with a: peeled fully-peeled\" \\\n+\t\t\t\t \"# pack-refs with:peeled fully-peeled sorted\"\n+\t\tdo\n+\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with: '\\''\n+\t\t\tEOF\n+\t\t\trm .git/packed-refs &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n+'\n+\n+test_expect_success 'packed-refs missing header should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"$(git rev-parse HEAD) refs/heads/main\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n+test_expect_success 'packed-refs unknown traits should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted foo\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512522","messageId":"Z7NVfRVcwcNYRSL5@ArchLinux","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"[PATCH v5 5/8] packed-backend: check whether the refname contains NUL characters","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T15:27:57Z","receivedAt":"2025-02-17T15:28:00Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will use \"check_refname_format\" to check\nthe consistency of the refname. If it is not OK, the program will die.\nHowever, it is reported in [1], we cannot catch some corruption. But we\nalready have the code path and we must miss out something.\n\nWe use the following code to get the refname:\n\n strbuf_add(&iter->refname_buf, p, eol - p);\n iter->base.refname = iter->refname_buf.buf\n\nIn the above code, `p` is the start pointer of the refname and `eol` is\nthe next newline pointer. We calculate the length of the refname by\nsubtracting the two pointers. Then we add the memory range between `p`\nand `eol` to get the refname.\n\nHowever, if there are some NUL characters in the memory range between `p`\nand `eol`, we will see the refname as a valid ref name as long as the\nmemory range between `p` and first occurred NUL character is valid.\n\nIn order to catch above corruption, create a new function\n\"refname_contains_nul\" by searching the first NUL character. If it is\nnot at the end of the string, there must be some NUL characters in the\nrefname.\n\nUse this function in \"next_record\" function to die the program if\n\"refname_contains_nul\" returns true.\n\n[1] https://lore.kernel.org/git/6cfee0e4-3285-4f18-91ff-d097da9de737@rd10.de/\n\nReported-by: R. Diez \nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 09eb3886c3..5edd2136bb 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -494,6 +494,21 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n \t\t\t\t last_line, eof - last_line);\n }\n \n+/*\n+ * When parsing the \"packed-refs\" file, we will parse it line by line.\n+ * Because we know the start pointer of the refname and the next\n+ * newline pointer, we could calculate the length of the refname by\n+ * subtracting the two pointers. However, there is a corner case where\n+ * the refname contains corrupted embedded NUL characters. And\n+ * `check_refname_format()` will not catch this when the truncated\n+ * refname is still a valid refname. To prevent this, we need to check\n+ * whether the refname contains the NUL characters.\n+ */\n+static int refname_contains_nul(struct strbuf *refname)\n+{\n+\treturn !!memchr(refname->buf, '\\0', refname->len);\n+}\n+\n #define SMALL_FILE_SIZE (32*1024)\n \n /*\n@@ -895,6 +910,9 @@ static int next_record(struct packed_ref_iterator *iter)\n \tstrbuf_add(&iter->refname_buf, p, eol - p);\n \titer->base.refname = iter->refname_buf.buf;\n \n+\tif (refname_contains_nul(&iter->refname_buf))\n+\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n+\n \tif (check_refname_format(iter->base.refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!refname_is_safe(iter->base.refname))\n \t\t\tdie(\"packed refname is dangerous: %s\",\n-- \n2.48.1\n\n"},{"id":"512523","messageId":"Z7NVhRckykx13PHq@ArchLinux","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"[PATCH v5 6/8] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T15:28:05Z","receivedAt":"2025-02-17T15:28:08Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will parse the ref entry to check the\nconsistency. This function has already checked the following things:\n\n1. Parse the main line of the ref entry to inspect whether the oid is\n not correct. Then, check whether the next character is oid. Then\n check the refname.\n2. If the next line starts with '^', it would continue to parse the\n peeled oid and check whether the last character is '\\n'.\n\nAs we decide to implement the ref consistency check for \"packed-refs\",\nlet's port these two checks and update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 122 ++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 44 ++++++++++++\n 4 files changed, 169 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 11906f90fd..02a7bf0503 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -16,6 +16,9 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefEntry`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid entry.\n+\n `badPackedRefHeader`::\n \t(ERROR) The \"packed-refs\" file contains an invalid\n \theader.\ndiff --git a/fsck.h b/fsck.h\nindex 67e3c97bc0..14d70f6653 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\n \tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 5edd2136bb..c7138aefff 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1812,9 +1812,114 @@ static int packed_fsck_ref_header(struct fsck_options *o,\n \treturn 0;\n }\n \n+static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id peeled;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\t/*\n+\t * Skip the '^' and parse the peeled oid.\n+\t */\n+\tstart++;\n+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid peeled oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p != eol) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n+\t\t\t\t (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_main_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t struct strbuf *refname,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id oid;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p == eol || !isspace(*p)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n+\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tp++;\n+\tstrbuf_reset(refname);\n+\tstrbuf_add(refname, p, eol - p);\n+\tif (refname_contains_nul(refname)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"refname '%s' contains NULL binaries\",\n+\t\t\t\t refname->buf);\n+\t}\n+\n+\tif (check_refname_format(refname->buf, 0)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n+\t\t\t\t \"has bad refname '%s'\", refname->buf);\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n+\tstruct strbuf refname = STRBUF_INIT;\n \tunsigned long line_number = 1;\n \tconst char *eol;\n \tint ret = 0;\n@@ -1827,6 +1932,21 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\tline_number++;\n \t}\n \n+\twhile (start < eof) {\n+\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\tret |= packed_fsck_ref_main_line(o, ref_store, line_number, &refname, start, eol);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t\tif (start < eof && *start == '^') {\n+\t\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, line_number,\n+\t\t\t\t\t\t\t start, eol);\n+\t\t\tstart = eol + 1;\n+\t\t\tline_number++;\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&refname);\n \treturn ret;\n }\n \n@@ -1892,7 +2012,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex e055c36e74..7421cc1e7f 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -691,4 +691,48 @@ test_expect_success 'packed-refs unknown traits should not be reported' '\n \t)\n '\n \n+test_expect_success 'packed-refs content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tgit tag -a annotated-tag-2 -m tag-2 &&\n+\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n+\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$short_oid refs/heads/branch-1\n+\t\t${branch_1_oid}x\n+\t\t$branch_2_oid refs/heads/bad-branch\n+\t\t$branch_2_oid refs/heads/branch.\n+\t\t$tag_1_oid refs/tags/annotated-tag-3\n+\t\t^$short_oid\n+\t\t$tag_2_oid refs/tags/annotated-tag-4.\n+\t\t^$tag_2_peeled_oid garbage\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: badPackedRefEntry: '\\''$short_oid refs/heads/branch-1'\\'' has invalid oid\n+\t\terror: packed-refs line 3: badPackedRefEntry: has no space after oid '\\''$branch_1_oid'\\'' but with '\\''x'\\''\n+\t\terror: packed-refs line 4: badRefName: has bad refname '\\'' refs/heads/bad-branch'\\''\n+\t\terror: packed-refs line 5: badRefName: has bad refname '\\''refs/heads/branch.'\\''\n+\t\terror: packed-refs line 7: badPackedRefEntry: '\\''$short_oid'\\'' has invalid peeled oid\n+\t\terror: packed-refs line 8: badRefName: has bad refname '\\''refs/tags/annotated-tag-4.'\\''\n+\t\terror: packed-refs line 9: badPackedRefEntry: has trailing garbage after peeled oid '\\'' garbage'\\''\n+\t\tEOF\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512524","messageId":"Z7NVjOW0dodrj2Bo@ArchLinux","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"[PATCH v5 7/8] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T15:28:12Z","receivedAt":"2025-02-17T15:28:16Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"When there is a \"sorted\" trait in the header of the \"packed-refs\" file,\nit means that each entry is sorted increasingly by comparing the\nrefname. We should add checks to verify whether the \"packed-refs\" is\nsorted in this case.\n\nUpdate the \"packed_fsck_ref_header\" to know whether there is a \"sorted\"\ntrail in the header. It may seem that we could record all refnames\nduring the parsing process and then compare later. However, this is not\na good design due to the following reasons:\n\n1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n in the \"packed-refs\" file.\n2. We cannot reuse the existing compare function \"cmp_packed_ref_records\"\n which cause repetition.\n\nBecause \"cmp_packed_ref_records\" needs an extra parameter \"struct\nsnaphost\", extract the common part into a new function\n\"cmp_packed_ref_records\" to reuse this function to compare.\n\nThen, create a new function \"packed_fsck_ref_sorted\" to parse the file\nagain and user the new fsck message \"packedRefUnsorted(ERROR)\" to report\nto the user if the file is not sorted.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 118 ++++++++++++++++++++++++++++-----\n t/t0602-reffiles-fsck.sh | 87 ++++++++++++++++++++++++\n 4 files changed, 192 insertions(+), 17 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 02a7bf0503..9601fff228 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -187,6 +187,9 @@\n \t(ERROR) The \"packed-refs\" file contains an entry that is\n \tnot terminated by a newline.\n \n+`packedRefUnsorted`::\n+\t(ERROR) The \"packed-refs\" file is not sorted.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex 14d70f6653..19f3cb2773 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -56,6 +56,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n \tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n+\tFUNC(PACKED_REF_UNSORTED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex c7138aefff..ae04d8ae80 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -300,14 +300,9 @@ struct snapshot_record {\n \tsize_t len;\n };\n \n-static int cmp_packed_ref_records(const void *v1, const void *v2,\n-\t\t\t\t void *cb_data)\n-{\n-\tconst struct snapshot *snapshot = cb_data;\n-\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n-\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n-\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n \n+static int cmp_packed_refname(const char *r1, const char *r2)\n+{\n \twhile (1) {\n \t\tif (*r1 == '\\n')\n \t\t\treturn *r2 == '\\n' ? 0 : -1;\n@@ -322,6 +317,17 @@ static int cmp_packed_ref_records(const void *v1, const void *v2,\n \t}\n }\n \n+static int cmp_packed_ref_records(const void *v1, const void *v2,\n+\t\t\t\t void *cb_data)\n+{\n+\tconst struct snapshot *snapshot = cb_data;\n+\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n+\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n+\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n+\n+\treturn cmp_packed_refname(r1, r2);\n+}\n+\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n@@ -1797,19 +1803,33 @@ static int packed_fsck_ref_next_line(struct fsck_options *o,\n }\n \n static int packed_fsck_ref_header(struct fsck_options *o,\n-\t\t\t\t const char *start, const char *eol)\n+\t\t\t\t const char *start, const char *eol,\n+\t\t\t\t unsigned int *sorted)\n {\n-\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n+\tchar *tmp_line;\n+\tint ret = 0;\n+\tchar *p;\n+\n+\ttmp_line = xmemdupz(start, eol - start);\n+\tif (!skip_prefix(tmp_line, \"# pack-refs with: \", (const char **)&p)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \t\treport.path = \"packed-refs.header\";\n \n-\t\treturn fsck_report_ref(o, &report,\n-\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n-\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n-\t\t\t\t (int)(eol - start), start);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n \t}\n \n-\treturn 0;\n+\tstring_list_split_in_place(&traits, p, \" \", -1);\n+\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n+\n+cleanup:\n+\tfree(tmp_line);\n+\tstring_list_clear(&traits, 0);\n+\treturn ret;\n }\n \n static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n@@ -1915,8 +1935,68 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n \treturn ret;\n }\n \n+static int packed_fsck_ref_sorted(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct strbuf refname1 = STRBUF_INIT;\n+\tstruct strbuf refname2 = STRBUF_INIT;\n+\tunsigned long line_number = 1;\n+\tconst char *former = NULL;\n+\tconst char *current;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tif (*start == '#') {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\tfor (; start < eof; line_number++, start = eol + 1) {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\n+\t\tif (*start == '^')\n+\t\t\tcontinue;\n+\n+\t\tif (!former) {\n+\t\t\tformer = start + hexsz + 1;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tcurrent = start + hexsz + 1;\n+\t\tif (cmp_packed_refname(former, current) >= 0) {\n+\t\t\tconst char *err_fmt =\n+\t\t\t\t\"refname '%s' is less than previous refname '%s'\";\n+\n+\t\t\teol = memchr(former, '\\n', eof - former);\n+\t\t\tstrbuf_add(&refname1, former, eol - former);\n+\t\t\teol = memchr(current, '\\n', eof - current);\n+\t\t\tstrbuf_add(&refname2, current, eol - current);\n+\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\t\treport.path = packed_entry.buf;\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_PACKED_REF_UNSORTED,\n+\t\t\t\t\t err_fmt, refname2.buf, refname1.buf);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tformer = current;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname1);\n+\tstrbuf_release(&refname2);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned int *sorted,\n \t\t\t\t const char *start, const char *eof)\n {\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -1926,7 +2006,7 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \n \tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n \tif (*start == '#') {\n-\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\t\tret |= packed_fsck_ref_header(o, start, eol, sorted);\n \n \t\tstart = eol + 1;\n \t\tline_number++;\n@@ -1957,9 +2037,10 @@ static int packed_fsck(struct ref_store *ref_store,\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n \tstruct strbuf packed_ref_content = STRBUF_INIT;\n+\tunsigned int sorted = 0;\n \tstruct stat st;\n-\tint fd;\n \tint ret = 0;\n+\tint fd;\n \n \tif (!is_main_worktree(wt))\n \t\tgoto cleanup;\n@@ -2012,8 +2093,11 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, &sorted, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\tif (!ret && sorted)\n+\t\tret = packed_fsck_ref_sorted(o, ref_store, packed_ref_content.buf,\n+\t\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\n \tstrbuf_release(&packed_ref_content);\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 7421cc1e7f..28dc8dcddc 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -735,4 +735,91 @@ test_expect_success 'packed-refs content should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-ref with sorted trait should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\t$tag_1_oid $refname3\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 3: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname1'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$tag_1_oid $refname3\n+\t\t^$tag_1_peeled_oid\n+\t\t$branch_2_oid $refname2\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 4: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname3'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n+test_expect_success 'packed-ref without sorted trait should not be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512525","messageId":"Z7NVlG0coJ9JbuiG@ArchLinux","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"[PATCH v5 8/8] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-17T15:28:20Z","receivedAt":"2025-02-17T15:28:24Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"At now, we have already implemented the ref consistency checks for both\n\"files-backend\" and \"packed-backend\". Although we would check some\nredundant things, it won't cause trouble. So, let's integrate it into\nthe \"git-fsck(1)\" command to get feedback from the users. And also by\ncalling \"git refs verify\" in \"git-fsck(1)\", we make sure that the new\nadded checks don't break.\n\nIntroduce a new function \"fsck_refs\" that initializes and runs a child\nprocess to execute the \"git refs verify\" command. In order to provide\nthe user interface create a progress which makes the total task be 1.\nIt's hard to know how many loose refs we will check now. We might\nimprove this later.\n\nThen, introduce the option to allow the user to disable checking ref\ndatabase consistency. Put this function in the very first execution\nsequence of \"git-fsck(1)\" due to that we don't want the existing code of\n\"git-fsck(1)\" which would implicitly check the consistency of refs to\ndie the program.\n\nLast, update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/git-fsck.adoc | 7 ++++++-\n builtin/fsck.c | 33 ++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 39 +++++++++++++++++++++++++++++++++++++\n 3 files changed, 77 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-fsck.adoc b/Documentation/git-fsck.adoc\nindex 8f32800a83..11203ba925 100644\n--- a/Documentation/git-fsck.adoc\n+++ b/Documentation/git-fsck.adoc\n@@ -12,7 +12,7 @@ SYNOPSIS\n 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n-\t [--[no-]name-objects] [...]\n+\t [--[no-]name-objects] [--[no-]references] [...]\n \n DESCRIPTION\n -----------\n@@ -104,6 +104,11 @@ care about this output and want to speed it up further.\n \tprogress status even if the standard error stream is not\n \tdirected to a terminal.\n \n+--[no-]references::\n+\tControl whether to check the references database consistency\n+\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n+\tThe default is to check the references database.\n+\n CONFIGURATION\n -------------\n \ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 7a4dcb0716..f4f395cfbd 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -50,6 +50,7 @@ static int verbose;\n static int show_progress = -1;\n static int show_dangling = 1;\n static int name_objects;\n+static int check_references = 1;\n #define ERROR_OBJECT 01\n #define ERROR_REACHABLE 02\n #define ERROR_PACK 04\n@@ -905,11 +906,37 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n \treturn res;\n }\n \n+static void fsck_refs(struct repository *r)\n+{\n+\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n+\tstruct progress *progress = NULL;\n+\n+\tif (show_progress)\n+\t\tprogress = start_progress(r, _(\"Checking ref database\"), 1);\n+\n+\tif (verbose)\n+\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n+\n+\tchild_process_init(&refs_verify);\n+\trefs_verify.git_cmd = 1;\n+\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n+\tif (verbose)\n+\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n+\tif (check_strict)\n+\t\tstrvec_push(&refs_verify.args, \"--strict\");\n+\n+\tif (run_command(&refs_verify))\n+\t\terrors_found |= ERROR_REFS;\n+\n+\tdisplay_progress(progress, 1);\n+\tstop_progress(&progress);\n+}\n+\n static char const * const fsck_usage[] = {\n \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n \t \" [--[no-]dangling] [--[no-]progress] [--connectivity-only]\\n\"\n-\t \" [--[no-]name-objects] [...]\"),\n+\t \" [--[no-]name-objects] [--[no-]references] [...]\"),\n \tNULL\n };\n \n@@ -928,6 +955,7 @@ static struct option fsck_opts[] = {\n \t\t\t\tN_(\"write dangling objects in .git/lost-found\")),\n \tOPT_BOOL(0, \"progress\", &show_progress, N_(\"show progress\")),\n \tOPT_BOOL(0, \"name-objects\", &name_objects, N_(\"show verbose names for reachable objects\")),\n+\tOPT_BOOL(0, \"references\", &check_references, N_(\"check reference database consistency\")),\n \tOPT_END(),\n };\n \n@@ -970,6 +998,9 @@ int cmd_fsck(int argc,\n \tgit_config(git_fsck_config, &fsck_obj_options);\n \tprepare_repo_settings(the_repository);\n \n+\tif (check_references)\n+\t\tfsck_refs(the_repository);\n+\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n \t\tfor_each_packed_object(the_repository,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 28dc8dcddc..42e8a84739 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -822,4 +822,43 @@ test_expect_success 'packed-ref without sorted trait should not be checked' '\n \t)\n '\n \n+test_expect_success '--[no-]references option should apply to fsck' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --references 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --no-references 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"512968","messageId":"Z71-zV-RL2niBdrn@pks.im","threadId":"62743","inReplyTo":"Z7NVZvAlNJ_00WX3@ArchLinux","subject":"Re: [PATCH v5 2/8] builtin/refs: get worktrees without reading head information","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-25T08:26:53Z","receivedAt":"2025-02-25T08:27:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Feb 17, 2025 at 11:27:34PM +0800, shejialuo wrote:\n> diff --git a/worktree.h b/worktree.h\n> index 38145df80f..f7003a9c12 100644\n> --- a/worktree.h\n> +++ b/worktree.h\n> @@ -30,6 +30,13 @@ struct worktree {\n> */\n> struct worktree **get_worktrees(void);\n> \n> +/*\n> + * Like `get_worktrees`, but does not read HEAD. Skip reading HEAD allows to\n> + * get the worktree without worrying about failures pertaining to parsing\n> + * the HEAD ref. This is useful when we want to check the ref db consistency.\n\nNit, not worth a reroll: this is highly specific to what you're doing.\nHow about: \"This is useful in contexts where it is assumed that the\nrefdb may not be in a consistent state.\" That would also include cases\nlike e.g. `repair_worktrees()`.\n\nPatrick\n"},{"id":"512969","messageId":"Z71-1EqdYVAu-fp7@pks.im","threadId":"62743","inReplyTo":"Z7NVbvyZTxspTjWX@ArchLinux","subject":"Re: [PATCH v5 3/8] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-25T08:27:00Z","receivedAt":"2025-02-25T08:27:03Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Feb 17, 2025 at 11:27:42PM +0800, shejialuo wrote:\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index a7b6f74b6e..8140a31d07 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -1748,15 +1749,43 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> \treturn empty_ref_iterator_begin();\n> }\n> \n> -static int packed_fsck(struct ref_store *ref_store UNUSED,\n> -\t\t struct fsck_options *o UNUSED,\n> +static int packed_fsck(struct ref_store *ref_store,\n> +\t\t struct fsck_options *o,\n> \t\t struct worktree *wt)\n> {\n> +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> +\tstruct stat st;\n> +\tint ret = 0;\n> \n> \tif (!is_main_worktree(wt))\n> -\t\treturn 0;\n> +\t\tgoto cleanup;\n> \n> -\treturn 0;\n> +\tif (o->verbose)\n> +\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n> +\n> +\tif (lstat(refs->path, &st) < 0) {\n> +\t\t/*\n> +\t\t * If the packed-refs file doesn't exist, there's nothing\n> +\t\t * to check.\n> +\t\t */\n> +\t\tif (errno == ENOENT)\n> +\t\t\tgoto cleanup;\n> +\t\tret = error_errno(_(\"unable to stat %s\"), refs->path);\n\nNit: We should quote the file name: \"unable to stat '%s'\".\n\nPatrick\n"},{"id":"512970","messageId":"Z71-1xa_o39IAo6A@pks.im","threadId":"62743","inReplyTo":"Z7NVdhM61rhjAHtW@ArchLinux","subject":"Re: [PATCH v5 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-25T08:27:03Z","receivedAt":"2025-02-25T08:27:07Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Feb 17, 2025 at 11:27:50PM +0800, shejialuo wrote:\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 8140a31d07..09eb3886c3 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -694,7 +694,7 @@ static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n> \n> \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n> \n> -\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n> +\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n> \t\t\tdie_invalid_line(refs->path,\n> \t\t\t\t\t snapshot->buf,\n> \t\t\t\t\t snapshot->eof - snapshot->buf);\n\nI know that Junio pointed out that we should check for a trailing space\nafter the colon. But do we really feel comfortable to tighten the check\nlike this now? If there was any broken writer of the format that does\nnot include the whitespace we'd now be unable to parse their output.\n\nI scanned through a couple of third-party clients:\n\n - libgit2 is fine and always writes the space. It also expects the\n whitespace to exist.\n\n - JGit does not expect the header to have a trailing space, but\n expects the \"peeled\" capability to have a leading space, which is\n mostly equivalent because that capability is typically the first one\n we write. It always writes the space.\n\n - gitoxide expects the space to exist and writes it.\n\n - go-git doesn't even seem to care about the header? Dunno, maybe I\n was just not able to locate the relevant code.\n\nSo yes, we should be fine, and the fact that other implementations\nexpect the space to exist indicates that being more thorough here is a\ngood thing. It might be a good idea though to split out this change into\na separate commit and then provide more reasoning _why_ it is fine,\nincluding the above info about alternate implementations.\n\nPatrick\n"},{"id":"512971","messageId":"Z71-26jqJH7zea3G@pks.im","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"Re: [PATCH v5 0/8] add more ref consistency checks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-25T08:27:07Z","receivedAt":"2025-02-25T08:27:11Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Feb 17, 2025 at 11:25:25PM +0800, shejialuo wrote:\n> Hi All:\n> \n> This changes enhances the following things:\n> \n> 1. [PATCH v5 2/8]: enhance the comment suggested by Karthik.\n> 2. [PATCH v5 3/8]: use lstat to check whether the filetype of\n> \"packed-ref\" is a regular file instead of using `open_nofollow`\n> to check. And also enhance the commit message suggested by Karthik.\n> 3. [PATCH v5 4/8]: move \"open_nofollow\" in original [PATCH v4 3/8] to\n> this.\n> \n> Also, I rebase due to the conflict that all *.txt files have been\n> renamed to *.adoc. However, I don't know whether this is a real\n> conflict. But I decide to rebase to make the life of Junio easy.\n\nI've got a couple of small nits, but overall I think this series should\nbe almost ready. Thanks!\n\nPatrick\n"},{"id":"513009","messageId":"Z724vnaRqmPAGByt@ArchLinux","threadId":"62743","inReplyTo":"Z71-1xa_o39IAo6A@pks.im","subject":"Re: [PATCH v5 4/8] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T12:34:06Z","receivedAt":"2025-02-25T12:33:59Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Feb 25, 2025 at 09:27:03AM +0100, Patrick Steinhardt wrote:\n> On Mon, Feb 17, 2025 at 11:27:50PM +0800, shejialuo wrote:\n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index 8140a31d07..09eb3886c3 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -694,7 +694,7 @@ static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n> > \n> > \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n> > \n> > -\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n> > +\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n> > \t\t\tdie_invalid_line(refs->path,\n> > \t\t\t\t\t snapshot->buf,\n> > \t\t\t\t\t snapshot->eof - snapshot->buf);\n> \n> I know that Junio pointed out that we should check for a trailing space\n> after the colon. But do we really feel comfortable to tighten the check\n> like this now? If there was any broken writer of the format that does\n> not include the whitespace we'd now be unable to parse their output.\n> \n> I scanned through a couple of third-party clients:\n> \n> - libgit2 is fine and always writes the space. It also expects the\n> whitespace to exist.\n> \n> - JGit does not expect the header to have a trailing space, but\n> expects the \"peeled\" capability to have a leading space, which is\n> mostly equivalent because that capability is typically the first one\n> we write. It always writes the space.\n> \n> - gitoxide expects the space to exist and writes it.\n> \n> - go-git doesn't even seem to care about the header? Dunno, maybe I\n> was just not able to locate the relevant code.\n\nI have searched the code. The go-git implement \"git pack-refs\" in\n`PackRefs`. go-git never writes header for \"packed-refs\" file.\n\nThanks for this wonderful suggestion.\n\n> \n> So yes, we should be fine, and the fact that other implementations\n> expect the space to exist indicates that being more thorough here is a\n> good thing. It might be a good idea though to split out this change into\n> a separate commit and then provide more reasoning _why_ it is fine,\n> including the above info about alternate implementations.\n> \n\nYes, I agree that we should split out this change. Let me do this.\n\n> Patrick\n"},{"id":"513011","messageId":"Z73DTwr9RicKMINe@ArchLinux","threadId":"62743","inReplyTo":"Z7NU5fZfc8vfSvZ0@ArchLinux","subject":"[PATCH v6 0/9] add more ref consistency checks","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:19:11Z","receivedAt":"2025-02-25T13:19:04Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis changes enhances the following things (v6-changed):\n\n1. [PATCH v6 2/9]: enhance the comment.\n2. [PATCH v6 3/9]: use '' to quote the file in the print message.\n2. [PATCH v6 4/9]: a new commit message to explain why we can tighten\nthe rule.\n\nThanks,\nJialuo\n\n---\n\nThis series mainly does the following things:\n\n1. Fix subshell issues\n2. Add ref checks for packed-backend.\n 1. Check whether the filetype of \"packed-refs\" is correct.\n 2. Check whether the syntax of \"packed-refs\" is correct by using the\n rules from \"packed-backend.c::create_snapshot\" and\n \"packed-backend.c::next_record\".\n 3. Check whether the pointed object exists and whether the\n \"packed-refs\" file is sorted.\n3. Call \"git refs verify\" for \"git-fsck(1)\".\n\nshejialuo (9):\n t0602: use subshell to ensure working directory unchanged\n builtin/refs: get worktrees without reading head information\n packed-backend: check whether the \"packed-refs\" is regular file\n packed-backend: check if header starts with \"# pack-refs with: \"\n packed-backend: add \"packed-refs\" header consistency check\n packed-backend: check whether the refname contains NUL characters\n packed-backend: add \"packed-refs\" entry consistency check\n packed-backend: check whether the \"packed-refs\" is sorted\n builtin/fsck: add `git refs verify` child process\n\n Documentation/fsck-msgids.adoc | 14 +\n Documentation/git-fsck.adoc | 7 +-\n builtin/fsck.c | 33 +-\n builtin/refs.c | 2 +-\n fsck.h | 4 +\n refs/packed-backend.c | 369 +++++++++-\n t/t0602-reffiles-fsck.sh | 1205 +++++++++++++++++++-------------\n worktree.c | 5 +\n worktree.h | 8 +\n 9 files changed, 1162 insertions(+), 485 deletions(-)\n\nRange-diff against v5:\n 1: b3952d80a2 = 1: b3952d80a2 t0602: use subshell to ensure working directory unchanged\n 2: 3695586f58 ! 2: fa5ce20bb7 builtin/refs: get worktrees without reading head information\n @@ worktree.h: struct worktree {\n +/*\n + * Like `get_worktrees`, but does not read HEAD. Skip reading HEAD allows to\n + * get the worktree without worrying about failures pertaining to parsing\n -+ * the HEAD ref. This is useful when we want to check the ref db consistency.\n ++ * the HEAD ref. This is useful in contexts where it is assumed that the\n ++ * refdb may not be in a consistent state.\n + */\n +struct worktree **get_worktrees_without_reading_head(void);\n +\n 3: cbaae00e8b ! 3: 787645a700 packed-backend: check whether the \"packed-refs\" is regular file\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\t\t */\n +\t\tif (errno == ENOENT)\n +\t\t\tgoto cleanup;\n -+\t\tret = error_errno(_(\"unable to stat %s\"), refs->path);\n ++\t\tret = error_errno(_(\"unable to stat '%s'\"), refs->path);\n +\t\tgoto cleanup;\n +\t}\n +\n -: ---------- > 4: f097e0f093 packed-backend: check if header starts with \"# pack-refs with: \"\n 4: b9ce8734ac ! 5: a589a38b68 packed-backend: add \"packed-refs\" header consistency check\n @@ Commit message\n \n In \"packed-backend.c::create_snapshot\", if there is a header (the line\n which starts with '#'), we will check whether the line starts with \"#\n - pack-refs with:\". Before we port this check into \"packed_fsck\", let's\n - fix \"create_snapshot\" to check the prefix \"# packed-ref with: \" instead\n - of \"# packed-ref with:\" due to that we will always write a single\n - trailing space after the colon.\n -\n - However, we need to consider other situations and discuss whether we\n - need to add checks.\n + pack-refs with: \". However, we need to consider other situations and\n + discuss whether we need to add checks.\n \n 1. If the header does not exist, we should not report an error to the\n user. This is because in older Git version, we never write header in\n @@ fsck.h: enum fsck_msg_type {\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\n \n ## refs/packed-backend.c ##\n -@@ refs/packed-backend.c: static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n - \n - \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n - \n --\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n -+\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n - \t\t\tdie_invalid_line(refs->path,\n - \t\t\t\t\t snapshot->buf,\n - \t\t\t\t\t snapshot->eof - snapshot->buf);\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n 5: 9f638b3adf = 6: 7255c2b597 packed-backend: check whether the refname contains NUL characters\n 6: 2c5395bdd0 = 7: 7794a2ebfd packed-backend: add \"packed-refs\" entry consistency check\n 7: 648404c60d = 8: 2a9138b14d packed-backend: check whether the \"packed-refs\" is sorted\n 8: 4dbbacf44b = 9: ccde32491f builtin/fsck: add `git refs verify` child process\n-- \n2.48.1\n\n"},{"id":"513013","messageId":"Z73DyBq67rp6vWOH@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v6 1/9] t0602: use subshell to ensure working directory unchanged","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:21:12Z","receivedAt":"2025-02-25T13:21:06Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"For every test, we would execute the command \"cd repo\" in the first but\nwe never execute the command \"cd ..\" to restore the working directory.\nHowever, it's either not a good idea use above way. Because if any test\nfails between \"cd repo\" and \"cd ..\", the \"cd ..\" will never be reached.\nAnd we cannot correctly restore the working directory.\n\nLet's use subshell to ensure that the current working directory could be\nrestored to the correct path.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n t/t0602-reffiles-fsck.sh | 967 ++++++++++++++++++++-------------------\n 1 file changed, 494 insertions(+), 473 deletions(-)\n\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex d4a08b823b..cf7a202d0d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -14,222 +14,229 @@ test_expect_success 'ref name should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b default-branch &&\n-\tgit tag default-tag &&\n-\tgit tag multi_hierarchy/default-tag &&\n-\n-\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n-\trm $branch_dir_prefix/@ &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n-\tgit refs verify 2>err &&\n-\trm $tag_dir_prefix/tag-1.lock &&\n-\ttest_must_be_empty err &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/.lock: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t(\n+\t\tcd repo &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b default-branch &&\n+\t\tgit tag default-tag &&\n+\t\tgit tag multi_hierarchy/default-tag &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\t\trm $branch_dir_prefix/@ &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $tag_dir_prefix/tag-1.lock &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \t\ttest_must_fail git refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\terror: refs/tags/.lock: badRefName: invalid refname format\n \t\tEOF\n-\t\trm -r \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $tag_dir_prefix/.lock &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=warn refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n-\ttest_must_be_empty err\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b branch-1 &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=warn refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm $branch_dir_prefix/.branch-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n '\n \n test_expect_success 'ref name check should work for multiple worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\n-\tcd repo &&\n-\ttest_commit initial &&\n-\tgit checkout -b branch-1 &&\n-\ttest_commit second &&\n-\tgit checkout -b branch-2 &&\n-\ttest_commit third &&\n-\tgit checkout -b branch-3 &&\n-\tgit worktree add ./worktree-1 branch-1 &&\n-\tgit worktree add ./worktree-2 branch-2 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n-\t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n \t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n-\n-\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n-\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err &&\n-\n-\tfor worktree in \"worktree-1\" \"worktree-2\"\n-\tdo\n+\t\tcd repo &&\n+\t\ttest_commit initial &&\n+\t\tgit checkout -b branch-1 &&\n+\t\ttest_commit second &&\n+\t\tgit checkout -b branch-2 &&\n+\t\ttest_commit third &&\n+\t\tgit checkout -b branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-1 &&\n+\t\tgit worktree add ./worktree-2 branch-2 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n \t\t(\n-\t\t\tcd $worktree &&\n-\t\t\ttest_must_fail git refs verify 2>err &&\n-\t\t\tcat >expect <<-EOF &&\n-\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\t\t\tEOF\n-\t\t\tsort err >sorted_err &&\n-\t\t\ttest_cmp expect sorted_err || return 1\n-\t\t)\n-\tdone\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\n+\t\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\t\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err &&\n+\n+\t\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\t\tdo\n+\t\t\t(\n+\t\t\t\tcd $worktree &&\n+\t\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\t\tcat >expect <<-EOF &&\n+\t\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\t\tEOF\n+\t\t\t\tsort err >sorted_err &&\n+\t\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t\t)\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tfor trailing_content in \" garbage\" \" more garbage\"\n-\tdo\n-\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-garbage &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n+\t\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n-\t'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\t'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n \n-\t garbage'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err\n+\t\t garbage'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -237,99 +244,103 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tbad_content_1=$(git rev-parse main)x &&\n-\tbad_content_2=xfsazqfxcadas &&\n-\tbad_content_3=Xfsazqfxcadas &&\n-\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n-\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n-\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n-\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n-\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n-\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tbad_content_1=$(git rev-parse main)x &&\n+\t\tbad_content_2=xfsazqfxcadas &&\n+\t\tbad_content_3=Xfsazqfxcadas &&\n+\t\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\t\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\t\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\t\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\t\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-complicated &&\n-\ttest_cmp expect err\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (aggregate)' '\n@@ -337,32 +348,34 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n-\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'the target of the textual symref should be checked' '\n@@ -370,28 +383,30 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n-\t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n-\n-\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n-\t\tgit refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked' '\n@@ -399,201 +414,207 @@ test_expect_success SYMLINKS 'symlink symref content should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n-\tEOF\n-\trm $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_cmp expect err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-good &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\t\tEOF\n+\t\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tmain_worktree_refdir_prefix=.git/refs/heads &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\n-\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tfor bad_referent_name in \".tag\" \"branch \"\n-\tdo\n-\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tmain_worktree_refdir_prefix=.git/refs/heads &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $worktree1_refdir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor bad_referent_name in \".tag\" \"branch \"\n+\t\tdo\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-garbage &&\n-\ttest_cmp expect err\n+\t\trm $worktree1_refdir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_done\n-- \n2.48.1\n\n"},{"id":"513014","messageId":"Z73D00MHpcQRUYs-@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v6 2/9] builtin/refs: get worktrees without reading head information","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:21:23Z","receivedAt":"2025-02-25T13:21:17Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\nand \"next_record\" which would check the correctness of the content of\nthe \"packed-ref\" file. When anything is bad, the program will die.\n\nIt may seem that we have nothing relevant to above feature, because we\nare going to read and parse the raw \"packed-ref\" file without creating\nthe snapshot and using the ref iterator to check the consistency.\n\nHowever, when using \"get_worktrees\" in \"builtin/refs\", we would parse\nthe \"HEAD\" information. If the referent of the \"HEAD\" is inside the\n\"packed-ref\", we will call \"create_snapshot\" function to parse the\n\"packed-ref\" to get the information. No matter whether the entry of\n\"HEAD\" in \"packed-ref\" is correct, \"create_snapshot\" would call\n\"verify_buffer_safe\" to check whether there is a newline in the last\nline of the file. If not, the program will die.\n\nAlthough this behavior has no harm for the program, it will\nshort-circuit the program. When the users execute \"git refs verify\" or\n\"git fsck\", we should avoid reading the head information, which may\nexecute the read operation in packed backend with stricter checks to die\nthe program. Instead, we should continue to check other parts of the\n\"packed-refs\" file completely.\n\nFortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\nworktrees, 2023-12-29), we have introduced a function\n\"get_worktrees_internal\" which allows us to get worktrees without\nreading head information.\n\nCreate a new exposed function \"get_worktrees_without_reading_head\", then\nreplace the \"get_worktrees\" in \"builtin/refs\" with the new created\nfunction.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 2 +-\n worktree.c | 5 +++++\n worktree.h | 8 ++++++++\n 3 files changed, 14 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex a29f195834..55ff5dae11 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tworktrees = get_worktrees();\n+\tworktrees = get_worktrees_without_reading_head();\n \tfor (size_t i = 0; worktrees[i]; i++)\n \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n \t\t\t\t &fsck_refs_options, worktrees[i]);\ndiff --git a/worktree.c b/worktree.c\nindex d4a68c9c23..d23482a746 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -198,6 +198,11 @@ struct worktree **get_worktrees(void)\n \treturn get_worktrees_internal(0);\n }\n \n+struct worktree **get_worktrees_without_reading_head(void)\n+{\n+\treturn get_worktrees_internal(1);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\ndiff --git a/worktree.h b/worktree.h\nindex 38145df80f..a305c7e2c7 100644\n--- a/worktree.h\n+++ b/worktree.h\n@@ -30,6 +30,14 @@ struct worktree {\n */\n struct worktree **get_worktrees(void);\n \n+/*\n+ * Like `get_worktrees`, but does not read HEAD. Skip reading HEAD allows to\n+ * get the worktree without worrying about failures pertaining to parsing\n+ * the HEAD ref. This is useful in contexts where it is assumed that the\n+ * refdb may not be in a consistent state.\n+ */\n+struct worktree **get_worktrees_without_reading_head(void);\n+\n /*\n * Returns 1 if linked worktrees exist, 0 otherwise.\n */\n-- \n2.48.1\n\n"},{"id":"513015","messageId":"Z73D3PdEFk_nciH7@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v6 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:21:32Z","receivedAt":"2025-02-25T13:21:25Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\nconsistency and correctness of \"packed-refs\" file, they never check the\nfiletype of the \"packed-refs\". Let's verify that the \"packed-refs\" has\nthe expected filetype, confirming it is created by \"git pack-refs\"\ncommand.\n\nUse \"lstat\" to check the file mode. If we cannot check the file status\ndue to there is no such file this is OK because there is a possibility\nthat there is no \"packed-refs\" in the repo.\n\nReuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\nthe user if \"packed-refs\" is not a regular file.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 37 +++++++++++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 22 ++++++++++++++++++++++\n 2 files changed, 55 insertions(+), 4 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a7b6f74b6e..6c118119a0 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -4,6 +4,7 @@\n #include \"../git-compat-util.h\"\n #include \"../config.h\"\n #include \"../dir.h\"\n+#include \"../fsck.h\"\n #include \"../gettext.h\"\n #include \"../hash.h\"\n #include \"../hex.h\"\n@@ -1748,15 +1749,43 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n-static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED,\n+static int packed_fsck(struct ref_store *ref_store,\n+\t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n+\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n+\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct stat st;\n+\tint ret = 0;\n \n \tif (!is_main_worktree(wt))\n-\t\treturn 0;\n+\t\tgoto cleanup;\n \n-\treturn 0;\n+\tif (o->verbose)\n+\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n+\n+\tif (lstat(refs->path, &st) < 0) {\n+\t\t/*\n+\t\t * If the packed-refs file doesn't exist, there's nothing\n+\t\t * to check.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\t\tret = error_errno(_(\"unable to stat '%s'\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (!S_ISREG(st.st_mode)) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs\";\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t \"not a regular file\");\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\treturn ret;\n }\n \n struct ref_storage_be refs_be_packed = {\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex cf7a202d0d..e65ca341cd 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -617,4 +617,26 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t)\n '\n \n+test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit pack-refs --all &&\n+\n+\t\tmv .git/packed-refs .git/packed-refs-back &&\n+\t\tln -sf packed-refs-back .git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs: badRefFiletype: not a regular file\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513016","messageId":"Z73D5XLzzg-OPmdT@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v6 4/9] packed-backend: check if header starts with \"# pack-refs with: \"","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:21:41Z","receivedAt":"2025-02-25T13:21:34Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We always write a space after \"# pack-refs with:\". However, when\ncreating the packed-ref snapshot, we only check whether the header\nstarts with \"# pack-refs with:\". However, we need to make sure that we\nwould not break compatibility by tightening the rule. The following is\nhow some third-party libraries handle the header of \"packed-ref\" file.\n\n1. libgit2 is fine and always writes the space. It also expects the\n whitespace to exist.\n2. JGit does not expect th header to have a trailing space, but expects\n the \"peeled\" capability to have a leading space, which is mostly\n equivalent because that capability is typically the first one we\n write. It always writes the space.\n3. gitoxide expects the space t exist and writes it.\n4. go-git doesn't create the header by default.\n\nSo, we are safe to tighten the rule by checking whether the header\nstarts with \"# pack-refs with: \".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 6c118119a0..9dabb5e556 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -694,7 +694,7 @@ static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n \n \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n \n-\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n+\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n \t\t\tdie_invalid_line(refs->path,\n \t\t\t\t\t snapshot->buf,\n \t\t\t\t\t snapshot->eof - snapshot->buf);\n-- \n2.48.1\n\n"},{"id":"513017","messageId":"Z73D7Qug4lgA8owW@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v6 5/9] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:21:49Z","receivedAt":"2025-02-25T13:21:43Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c::create_snapshot\", if there is a header (the line\nwhich starts with '#'), we will check whether the line starts with \"#\npack-refs with: \". However, we need to consider other situations and\ndiscuss whether we need to add checks.\n\n1. If the header does not exist, we should not report an error to the\n user. This is because in older Git version, we never write header in\n the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n in runtime.\n2. If the header content does not start with \"# packed-ref with: \", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n3. If the header content is not the same as the constant string\n \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n intentionally and runtime \"create_snapshot\" won't complain about\n unknown traits. In order to align with the runtime behavior. There is\n no need to report.\n\nAs we have analyzed, we only need to check the case 2 in the above. In\norder to do this, use \"open_nofollow\" function to get the file\ndescriptor and then read the \"packed-refs\" file via \"strbuf_read\". Like\nwhat \"create_snapshot\" and other functions do, we could split the line\nby finding the next newline in the buffer. When we cannot find a\nnewline, we could report an error.\n\nSo, create a function \"packed_fsck_ref_next_line\" to find the next\nnewline and if there is no such newline, use\n\"packedRefEntryNotTerminated(ERROR)\" to report an error to the user.\n\nThen, parse the first line to apply the checks. Update the test to\nexercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 8 +++\n fsck.h | 2 +\n refs/packed-backend.c | 94 ++++++++++++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 52 +++++++++++++++++++\n 4 files changed, 156 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex b14bc44ca4..11906f90fd 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -16,6 +16,10 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefHeader`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid\n+\theader.\n+\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n@@ -176,6 +180,10 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`packedRefEntryNotTerminated`::\n+\t(ERROR) The \"packed-refs\" file contains an entry that is\n+\tnot terminated by a newline.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex a44c231a5f..67e3c97bc0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n@@ -53,6 +54,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE, ERROR) \\\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 9dabb5e556..4891c86a5a 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1749,13 +1749,78 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+static int packed_fsck_ref_next_line(struct fsck_options *o,\n+\t\t\t\t unsigned long line_number, const char *start,\n+\t\t\t\t const char *eof, const char **eol)\n+{\n+\tint ret = 0;\n+\n+\t*eol = memchr(start, '\\n', eof - start);\n+\tif (!*eol) {\n+\t\tstruct strbuf packed_entry = STRBUF_INIT;\n+\t\tstruct fsck_ref_report report = { 0 };\n+\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n+\t\t\t\t \"'%.*s' is not terminated with a newline\",\n+\t\t\t\t (int)(eof - start), start);\n+\n+\t\t/*\n+\t\t * There is no newline but we still want to parse it to the end of\n+\t\t * the buffer.\n+\t\t */\n+\t\t*eol = eof;\n+\t\tstrbuf_release(&packed_entry);\n+\t}\n+\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_header(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs.header\";\n+\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tunsigned long line_number = 1;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\tif (*start == '#') {\n+\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\treturn ret;\n+}\n+\n static int packed_fsck(struct ref_store *ref_store,\n \t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tstruct stat st;\n+\tint fd;\n \tint ret = 0;\n \n \tif (!is_main_worktree(wt))\n@@ -1784,7 +1849,36 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\t/*\n+\t * There is a chance that \"packed-refs\" file is removed or converted to\n+\t * a symlink after filetype check and before open. So we need to avoid\n+\t * this race condition by opening the file.\n+\t */\n+\tfd = open_nofollow(refs->path, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tif (errno == ELOOP) {\n+\t\t\tstruct fsck_ref_report report = { 0 };\n+\t\t\treport.path = \"packed-refs\";\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t\t \"not a regular file\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n+\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n+\t\tret = error_errno(_(\"unable to read %s\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\n cleanup:\n+\tstrbuf_release(&packed_ref_content);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex e65ca341cd..e055c36e74 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -639,4 +639,56 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-refs header should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with a: peeled fully-peeled\" \\\n+\t\t\t\t \"# pack-refs with:peeled fully-peeled sorted\"\n+\t\tdo\n+\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with: '\\''\n+\t\t\tEOF\n+\t\t\trm .git/packed-refs &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n+'\n+\n+test_expect_success 'packed-refs missing header should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"$(git rev-parse HEAD) refs/heads/main\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n+test_expect_success 'packed-refs unknown traits should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted foo\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513018","messageId":"Z73D9X_9Xsptgaif@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v6 6/9] packed-backend: check whether the refname contains NUL characters","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:21:57Z","receivedAt":"2025-02-25T13:21:51Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will use \"check_refname_format\" to check\nthe consistency of the refname. If it is not OK, the program will die.\nHowever, it is reported in [1], we cannot catch some corruption. But we\nalready have the code path and we must miss out something.\n\nWe use the following code to get the refname:\n\n strbuf_add(&iter->refname_buf, p, eol - p);\n iter->base.refname = iter->refname_buf.buf\n\nIn the above code, `p` is the start pointer of the refname and `eol` is\nthe next newline pointer. We calculate the length of the refname by\nsubtracting the two pointers. Then we add the memory range between `p`\nand `eol` to get the refname.\n\nHowever, if there are some NUL characters in the memory range between `p`\nand `eol`, we will see the refname as a valid ref name as long as the\nmemory range between `p` and first occurred NUL character is valid.\n\nIn order to catch above corruption, create a new function\n\"refname_contains_nul\" by searching the first NUL character. If it is\nnot at the end of the string, there must be some NUL characters in the\nrefname.\n\nUse this function in \"next_record\" function to die the program if\n\"refname_contains_nul\" returns true.\n\n[1] https://lore.kernel.org/git/6cfee0e4-3285-4f18-91ff-d097da9de737@rd10.de/\n\nReported-by: R. Diez \nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 4891c86a5a..a74ee57776 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -494,6 +494,21 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n \t\t\t\t last_line, eof - last_line);\n }\n \n+/*\n+ * When parsing the \"packed-refs\" file, we will parse it line by line.\n+ * Because we know the start pointer of the refname and the next\n+ * newline pointer, we could calculate the length of the refname by\n+ * subtracting the two pointers. However, there is a corner case where\n+ * the refname contains corrupted embedded NUL characters. And\n+ * `check_refname_format()` will not catch this when the truncated\n+ * refname is still a valid refname. To prevent this, we need to check\n+ * whether the refname contains the NUL characters.\n+ */\n+static int refname_contains_nul(struct strbuf *refname)\n+{\n+\treturn !!memchr(refname->buf, '\\0', refname->len);\n+}\n+\n #define SMALL_FILE_SIZE (32*1024)\n \n /*\n@@ -895,6 +910,9 @@ static int next_record(struct packed_ref_iterator *iter)\n \tstrbuf_add(&iter->refname_buf, p, eol - p);\n \titer->base.refname = iter->refname_buf.buf;\n \n+\tif (refname_contains_nul(&iter->refname_buf))\n+\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n+\n \tif (check_refname_format(iter->base.refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!refname_is_safe(iter->base.refname))\n \t\t\tdie(\"packed refname is dangerous: %s\",\n-- \n2.48.1\n\n"},{"id":"513019","messageId":"Z73D_eE8dnzfAcBD@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v6 7/9] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:22:05Z","receivedAt":"2025-02-25T13:21:59Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will parse the ref entry to check the\nconsistency. This function has already checked the following things:\n\n1. Parse the main line of the ref entry to inspect whether the oid is\n not correct. Then, check whether the next character is oid. Then\n check the refname.\n2. If the next line starts with '^', it would continue to parse the\n peeled oid and check whether the last character is '\\n'.\n\nAs we decide to implement the ref consistency check for \"packed-refs\",\nlet's port these two checks and update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 122 ++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 44 ++++++++++++\n 4 files changed, 169 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 11906f90fd..02a7bf0503 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -16,6 +16,9 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefEntry`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid entry.\n+\n `badPackedRefHeader`::\n \t(ERROR) The \"packed-refs\" file contains an invalid\n \theader.\ndiff --git a/fsck.h b/fsck.h\nindex 67e3c97bc0..14d70f6653 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\n \tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a74ee57776..dd3f7ab255 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1812,9 +1812,114 @@ static int packed_fsck_ref_header(struct fsck_options *o,\n \treturn 0;\n }\n \n+static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id peeled;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\t/*\n+\t * Skip the '^' and parse the peeled oid.\n+\t */\n+\tstart++;\n+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid peeled oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p != eol) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n+\t\t\t\t (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_main_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t struct strbuf *refname,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id oid;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p == eol || !isspace(*p)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n+\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tp++;\n+\tstrbuf_reset(refname);\n+\tstrbuf_add(refname, p, eol - p);\n+\tif (refname_contains_nul(refname)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"refname '%s' contains NULL binaries\",\n+\t\t\t\t refname->buf);\n+\t}\n+\n+\tif (check_refname_format(refname->buf, 0)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n+\t\t\t\t \"has bad refname '%s'\", refname->buf);\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n+\tstruct strbuf refname = STRBUF_INIT;\n \tunsigned long line_number = 1;\n \tconst char *eol;\n \tint ret = 0;\n@@ -1827,6 +1932,21 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\tline_number++;\n \t}\n \n+\twhile (start < eof) {\n+\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\tret |= packed_fsck_ref_main_line(o, ref_store, line_number, &refname, start, eol);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t\tif (start < eof && *start == '^') {\n+\t\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, line_number,\n+\t\t\t\t\t\t\t start, eol);\n+\t\t\tstart = eol + 1;\n+\t\t\tline_number++;\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&refname);\n \treturn ret;\n }\n \n@@ -1892,7 +2012,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex e055c36e74..7421cc1e7f 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -691,4 +691,48 @@ test_expect_success 'packed-refs unknown traits should not be reported' '\n \t)\n '\n \n+test_expect_success 'packed-refs content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tgit tag -a annotated-tag-2 -m tag-2 &&\n+\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n+\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$short_oid refs/heads/branch-1\n+\t\t${branch_1_oid}x\n+\t\t$branch_2_oid refs/heads/bad-branch\n+\t\t$branch_2_oid refs/heads/branch.\n+\t\t$tag_1_oid refs/tags/annotated-tag-3\n+\t\t^$short_oid\n+\t\t$tag_2_oid refs/tags/annotated-tag-4.\n+\t\t^$tag_2_peeled_oid garbage\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: badPackedRefEntry: '\\''$short_oid refs/heads/branch-1'\\'' has invalid oid\n+\t\terror: packed-refs line 3: badPackedRefEntry: has no space after oid '\\''$branch_1_oid'\\'' but with '\\''x'\\''\n+\t\terror: packed-refs line 4: badRefName: has bad refname '\\'' refs/heads/bad-branch'\\''\n+\t\terror: packed-refs line 5: badRefName: has bad refname '\\''refs/heads/branch.'\\''\n+\t\terror: packed-refs line 7: badPackedRefEntry: '\\''$short_oid'\\'' has invalid peeled oid\n+\t\terror: packed-refs line 8: badRefName: has bad refname '\\''refs/tags/annotated-tag-4.'\\''\n+\t\terror: packed-refs line 9: badPackedRefEntry: has trailing garbage after peeled oid '\\'' garbage'\\''\n+\t\tEOF\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513020","messageId":"Z73EBmp3VT1ahS96@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v6 8/9] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:22:14Z","receivedAt":"2025-02-25T13:22:07Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"When there is a \"sorted\" trait in the header of the \"packed-refs\" file,\nit means that each entry is sorted increasingly by comparing the\nrefname. We should add checks to verify whether the \"packed-refs\" is\nsorted in this case.\n\nUpdate the \"packed_fsck_ref_header\" to know whether there is a \"sorted\"\ntrail in the header. It may seem that we could record all refnames\nduring the parsing process and then compare later. However, this is not\na good design due to the following reasons:\n\n1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n in the \"packed-refs\" file.\n2. We cannot reuse the existing compare function \"cmp_packed_ref_records\"\n which cause repetition.\n\nBecause \"cmp_packed_ref_records\" needs an extra parameter \"struct\nsnaphost\", extract the common part into a new function\n\"cmp_packed_ref_records\" to reuse this function to compare.\n\nThen, create a new function \"packed_fsck_ref_sorted\" to parse the file\nagain and user the new fsck message \"packedRefUnsorted(ERROR)\" to report\nto the user if the file is not sorted.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 118 ++++++++++++++++++++++++++++-----\n t/t0602-reffiles-fsck.sh | 87 ++++++++++++++++++++++++\n 4 files changed, 192 insertions(+), 17 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 02a7bf0503..9601fff228 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -187,6 +187,9 @@\n \t(ERROR) The \"packed-refs\" file contains an entry that is\n \tnot terminated by a newline.\n \n+`packedRefUnsorted`::\n+\t(ERROR) The \"packed-refs\" file is not sorted.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex 14d70f6653..19f3cb2773 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -56,6 +56,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n \tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n+\tFUNC(PACKED_REF_UNSORTED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex dd3f7ab255..75f28e283a 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -300,14 +300,9 @@ struct snapshot_record {\n \tsize_t len;\n };\n \n-static int cmp_packed_ref_records(const void *v1, const void *v2,\n-\t\t\t\t void *cb_data)\n-{\n-\tconst struct snapshot *snapshot = cb_data;\n-\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n-\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n-\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n \n+static int cmp_packed_refname(const char *r1, const char *r2)\n+{\n \twhile (1) {\n \t\tif (*r1 == '\\n')\n \t\t\treturn *r2 == '\\n' ? 0 : -1;\n@@ -322,6 +317,17 @@ static int cmp_packed_ref_records(const void *v1, const void *v2,\n \t}\n }\n \n+static int cmp_packed_ref_records(const void *v1, const void *v2,\n+\t\t\t\t void *cb_data)\n+{\n+\tconst struct snapshot *snapshot = cb_data;\n+\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n+\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n+\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n+\n+\treturn cmp_packed_refname(r1, r2);\n+}\n+\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n@@ -1797,19 +1803,33 @@ static int packed_fsck_ref_next_line(struct fsck_options *o,\n }\n \n static int packed_fsck_ref_header(struct fsck_options *o,\n-\t\t\t\t const char *start, const char *eol)\n+\t\t\t\t const char *start, const char *eol,\n+\t\t\t\t unsigned int *sorted)\n {\n-\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n+\tchar *tmp_line;\n+\tint ret = 0;\n+\tchar *p;\n+\n+\ttmp_line = xmemdupz(start, eol - start);\n+\tif (!skip_prefix(tmp_line, \"# pack-refs with: \", (const char **)&p)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \t\treport.path = \"packed-refs.header\";\n \n-\t\treturn fsck_report_ref(o, &report,\n-\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n-\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n-\t\t\t\t (int)(eol - start), start);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n \t}\n \n-\treturn 0;\n+\tstring_list_split_in_place(&traits, p, \" \", -1);\n+\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n+\n+cleanup:\n+\tfree(tmp_line);\n+\tstring_list_clear(&traits, 0);\n+\treturn ret;\n }\n \n static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n@@ -1915,8 +1935,68 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n \treturn ret;\n }\n \n+static int packed_fsck_ref_sorted(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct strbuf refname1 = STRBUF_INIT;\n+\tstruct strbuf refname2 = STRBUF_INIT;\n+\tunsigned long line_number = 1;\n+\tconst char *former = NULL;\n+\tconst char *current;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tif (*start == '#') {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\tfor (; start < eof; line_number++, start = eol + 1) {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\n+\t\tif (*start == '^')\n+\t\t\tcontinue;\n+\n+\t\tif (!former) {\n+\t\t\tformer = start + hexsz + 1;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tcurrent = start + hexsz + 1;\n+\t\tif (cmp_packed_refname(former, current) >= 0) {\n+\t\t\tconst char *err_fmt =\n+\t\t\t\t\"refname '%s' is less than previous refname '%s'\";\n+\n+\t\t\teol = memchr(former, '\\n', eof - former);\n+\t\t\tstrbuf_add(&refname1, former, eol - former);\n+\t\t\teol = memchr(current, '\\n', eof - current);\n+\t\t\tstrbuf_add(&refname2, current, eol - current);\n+\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\t\treport.path = packed_entry.buf;\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_PACKED_REF_UNSORTED,\n+\t\t\t\t\t err_fmt, refname2.buf, refname1.buf);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tformer = current;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname1);\n+\tstrbuf_release(&refname2);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned int *sorted,\n \t\t\t\t const char *start, const char *eof)\n {\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -1926,7 +2006,7 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \n \tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n \tif (*start == '#') {\n-\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\t\tret |= packed_fsck_ref_header(o, start, eol, sorted);\n \n \t\tstart = eol + 1;\n \t\tline_number++;\n@@ -1957,9 +2037,10 @@ static int packed_fsck(struct ref_store *ref_store,\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n \tstruct strbuf packed_ref_content = STRBUF_INIT;\n+\tunsigned int sorted = 0;\n \tstruct stat st;\n-\tint fd;\n \tint ret = 0;\n+\tint fd;\n \n \tif (!is_main_worktree(wt))\n \t\tgoto cleanup;\n@@ -2012,8 +2093,11 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, &sorted, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\tif (!ret && sorted)\n+\t\tret = packed_fsck_ref_sorted(o, ref_store, packed_ref_content.buf,\n+\t\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\n \tstrbuf_release(&packed_ref_content);\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 7421cc1e7f..28dc8dcddc 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -735,4 +735,91 @@ test_expect_success 'packed-refs content should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-ref with sorted trait should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\t$tag_1_oid $refname3\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 3: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname1'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$tag_1_oid $refname3\n+\t\t^$tag_1_peeled_oid\n+\t\t$branch_2_oid $refname2\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 4: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname3'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n+test_expect_success 'packed-ref without sorted trait should not be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513021","messageId":"Z73EDq-xCm1GO28-@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v6 9/9] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-25T13:22:22Z","receivedAt":"2025-02-25T13:22:16Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"At now, we have already implemented the ref consistency checks for both\n\"files-backend\" and \"packed-backend\". Although we would check some\nredundant things, it won't cause trouble. So, let's integrate it into\nthe \"git-fsck(1)\" command to get feedback from the users. And also by\ncalling \"git refs verify\" in \"git-fsck(1)\", we make sure that the new\nadded checks don't break.\n\nIntroduce a new function \"fsck_refs\" that initializes and runs a child\nprocess to execute the \"git refs verify\" command. In order to provide\nthe user interface create a progress which makes the total task be 1.\nIt's hard to know how many loose refs we will check now. We might\nimprove this later.\n\nThen, introduce the option to allow the user to disable checking ref\ndatabase consistency. Put this function in the very first execution\nsequence of \"git-fsck(1)\" due to that we don't want the existing code of\n\"git-fsck(1)\" which would implicitly check the consistency of refs to\ndie the program.\n\nLast, update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/git-fsck.adoc | 7 ++++++-\n builtin/fsck.c | 33 ++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 39 +++++++++++++++++++++++++++++++++++++\n 3 files changed, 77 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-fsck.adoc b/Documentation/git-fsck.adoc\nindex 8f32800a83..11203ba925 100644\n--- a/Documentation/git-fsck.adoc\n+++ b/Documentation/git-fsck.adoc\n@@ -12,7 +12,7 @@ SYNOPSIS\n 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n-\t [--[no-]name-objects] [...]\n+\t [--[no-]name-objects] [--[no-]references] [...]\n \n DESCRIPTION\n -----------\n@@ -104,6 +104,11 @@ care about this output and want to speed it up further.\n \tprogress status even if the standard error stream is not\n \tdirected to a terminal.\n \n+--[no-]references::\n+\tControl whether to check the references database consistency\n+\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n+\tThe default is to check the references database.\n+\n CONFIGURATION\n -------------\n \ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 7a4dcb0716..f4f395cfbd 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -50,6 +50,7 @@ static int verbose;\n static int show_progress = -1;\n static int show_dangling = 1;\n static int name_objects;\n+static int check_references = 1;\n #define ERROR_OBJECT 01\n #define ERROR_REACHABLE 02\n #define ERROR_PACK 04\n@@ -905,11 +906,37 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n \treturn res;\n }\n \n+static void fsck_refs(struct repository *r)\n+{\n+\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n+\tstruct progress *progress = NULL;\n+\n+\tif (show_progress)\n+\t\tprogress = start_progress(r, _(\"Checking ref database\"), 1);\n+\n+\tif (verbose)\n+\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n+\n+\tchild_process_init(&refs_verify);\n+\trefs_verify.git_cmd = 1;\n+\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n+\tif (verbose)\n+\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n+\tif (check_strict)\n+\t\tstrvec_push(&refs_verify.args, \"--strict\");\n+\n+\tif (run_command(&refs_verify))\n+\t\terrors_found |= ERROR_REFS;\n+\n+\tdisplay_progress(progress, 1);\n+\tstop_progress(&progress);\n+}\n+\n static char const * const fsck_usage[] = {\n \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n \t \" [--[no-]dangling] [--[no-]progress] [--connectivity-only]\\n\"\n-\t \" [--[no-]name-objects] [...]\"),\n+\t \" [--[no-]name-objects] [--[no-]references] [...]\"),\n \tNULL\n };\n \n@@ -928,6 +955,7 @@ static struct option fsck_opts[] = {\n \t\t\t\tN_(\"write dangling objects in .git/lost-found\")),\n \tOPT_BOOL(0, \"progress\", &show_progress, N_(\"show progress\")),\n \tOPT_BOOL(0, \"name-objects\", &name_objects, N_(\"show verbose names for reachable objects\")),\n+\tOPT_BOOL(0, \"references\", &check_references, N_(\"check reference database consistency\")),\n \tOPT_END(),\n };\n \n@@ -970,6 +998,9 @@ int cmd_fsck(int argc,\n \tgit_config(git_fsck_config, &fsck_obj_options);\n \tprepare_repo_settings(the_repository);\n \n+\tif (check_references)\n+\t\tfsck_refs(the_repository);\n+\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n \t\tfor_each_packed_object(the_repository,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 28dc8dcddc..42e8a84739 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -822,4 +822,43 @@ test_expect_success 'packed-ref without sorted trait should not be checked' '\n \t)\n '\n \n+test_expect_success '--[no-]references option should apply to fsck' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --references 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --no-references 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513031","messageId":"xmqq1pvlly1f.fsf@gitster.g","threadId":"62743","inReplyTo":"Z73D3PdEFk_nciH7@ArchLinux","subject":"Re: [PATCH v6 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-25T17:44:12Z","receivedAt":"2025-02-25T17:44:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\n> consistency and correctness of \"packed-refs\" file, they never check the\n> filetype of the \"packed-refs\". Let's verify that the \"packed-refs\" has\n> the expected filetype, confirming it is created by \"git pack-refs\"\n> command.\n>\n> Use \"lstat\" to check the file mode. If we cannot check the file status\n> due to there is no such file this is OK because there is a possibility\n> that there is no \"packed-refs\" in the repo.\n\nCan this be done _after_ the open_nofollow() check you had in the\nprevious round noticed a problem? Even though we are trying to\nnotice and find problems in the given repository, it is generally\na good idea to optimize for the more common case (i.e. the file is a\nregular one and not a symbolic link or directory or anything funny).\nSomething along the lines of\n\n\tfd = open_nofollow(...);\n\tif (fd < 0) {\n\t\tlstat() to inspect the details\n\t} else if (fstat(fd, &st) < 0) {\n\t\t... cannot tell what we opened ...\n\t} else if (!S_ISREG(st.st_mode)) {\n\t\t... we opened something funny ...\n\t} else {\n\t\t... the thing is a regular file as expected ...\n\t}\n\nperhaps?\n\n"},{"id":"513054","messageId":"Z77MAMqVGrT2jgcf@pks.im","threadId":"62743","inReplyTo":"Z73D5XLzzg-OPmdT@ArchLinux","subject":"Re: [PATCH v6 4/9] packed-backend: check if header starts with \"# pack-refs with: \"","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-26T08:08:32Z","receivedAt":"2025-02-26T08:08:40Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Feb 25, 2025 at 09:21:41PM +0800, shejialuo wrote:\n> We always write a space after \"# pack-refs with:\". However, when\n> creating the packed-ref snapshot, we only check whether the header\n> starts with \"# pack-refs with:\". However, we need to make sure that we\n> would not break compatibility by tightening the rule. The following is\n> how some third-party libraries handle the header of \"packed-ref\" file.\n> \n> 1. libgit2 is fine and always writes the space. It also expects the\n> whitespace to exist.\n> 2. JGit does not expect th header to have a trailing space, but expects\n> the \"peeled\" capability to have a leading space, which is mostly\n> equivalent because that capability is typically the first one we\n> write. It always writes the space.\n> 3. gitoxide expects the space t exist and writes it.\n> 4. go-git doesn't create the header by default.\n> \n> So, we are safe to tighten the rule by checking whether the header\n> starts with \"# pack-refs with: \".\n\nThe commit message nicely describes why it's safe to do the change, but\nit doesn't describe why it's something we _want_ to do.\n\nIdeally, we'd be able to argue with a technical spec of the format, but\nunless I'm mistaken such a document does not exist. The next-best thing\nis to do what everyone can agree on, and that seems to be to both write\nand expect a space after the colon. By not following consensus that\nexists in other libraries we're being more loose.\n\nSo if we for example started to stop writing the space due to a bug,\nwe'd still continue to parse the header alright and thus not notice the\nproblem, but now we have broken other implementations. That may be a\ngood enough justification for the change itself.\n\nPatrick\n"},{"id":"513074","messageId":"Z78Dk1WDdqnuPQhX@ArchLinux","threadId":"62743","inReplyTo":"xmqq1pvlly1f.fsf@gitster.g","subject":"Re: [PATCH v6 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T12:05:39Z","receivedAt":"2025-02-26T12:05:33Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Feb 25, 2025 at 09:44:12AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\n> > consistency and correctness of \"packed-refs\" file, they never check the\n> > filetype of the \"packed-refs\". Let's verify that the \"packed-refs\" has\n> > the expected filetype, confirming it is created by \"git pack-refs\"\n> > command.\n> >\n> > Use \"lstat\" to check the file mode. If we cannot check the file status\n> > due to there is no such file this is OK because there is a possibility\n> > that there is no \"packed-refs\" in the repo.\n> \n> Can this be done _after_ the open_nofollow() check you had in the\n> previous round noticed a problem? Even though we are trying to\n> notice and find problems in the given repository, it is generally\n> a good idea to optimize for the more common case (i.e. the file is a\n> regular one and not a symbolic link or directory or anything funny).\n> Something along the lines of\n> \n> \tfd = open_nofollow(...);\n> \tif (fd < 0) {\n> \t\tlstat() to inspect the details\n> \t} else if (fstat(fd, &st) < 0) {\n> \t\t... cannot tell what we opened ...\n> \t} else if (!S_ISREG(st.st_mode)) {\n> \t\t... we opened something funny ...\n> \t} else {\n> \t\t... the thing is a regular file as expected ...\n> \t}\n> \n\nGood idea, by using this way, the code would be more clean. I will\nimprove this in the next version.\n\n> perhaps?\n"},{"id":"513075","messageId":"Z78JAaKTTivF5M0A@ArchLinux","threadId":"62743","inReplyTo":"Z77MAMqVGrT2jgcf@pks.im","subject":"Re: [PATCH v6 4/9] packed-backend: check if header starts with \"# pack-refs with: \"","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T12:28:49Z","receivedAt":"2025-02-26T12:28:41Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Feb 26, 2025 at 09:08:32AM +0100, Patrick Steinhardt wrote:\n> On Tue, Feb 25, 2025 at 09:21:41PM +0800, shejialuo wrote:\n> > We always write a space after \"# pack-refs with:\". However, when\n> > creating the packed-ref snapshot, we only check whether the header\n> > starts with \"# pack-refs with:\". However, we need to make sure that we\n> > would not break compatibility by tightening the rule. The following is\n> > how some third-party libraries handle the header of \"packed-ref\" file.\n> > \n> > 1. libgit2 is fine and always writes the space. It also expects the\n> > whitespace to exist.\n> > 2. JGit does not expect th header to have a trailing space, but expects\n> > the \"peeled\" capability to have a leading space, which is mostly\n> > equivalent because that capability is typically the first one we\n> > write. It always writes the space.\n> > 3. gitoxide expects the space t exist and writes it.\n> > 4. go-git doesn't create the header by default.\n> > \n> > So, we are safe to tighten the rule by checking whether the header\n> > starts with \"# pack-refs with: \".\n> \n> The commit message nicely describes why it's safe to do the change, but\n> it doesn't describe why it's something we _want_ to do.\n> \n\nYes, as you have said below. We don't have document about the header\nformat. It's an internal implementation of Git.\n\n> Ideally, we'd be able to argue with a technical spec of the format, but\n> unless I'm mistaken such a document does not exist. The next-best thing\n> is to do what everyone can agree on, and that seems to be to both write\n> and expect a space after the colon. By not following consensus that\n> exists in other libraries we're being more loose.\n> \n> So if we for example started to stop writing the space due to a bug,\n> we'd still continue to parse the header alright and thus not notice the\n> problem, but now we have broken other implementations. That may be a\n> good enough justification for the change itself.\n> \n\nThanks, I will improve the commit message in the next version.\n\n> Patrick\n"},{"id":"513081","messageId":"Z78bmBSrDR20GY6g@ArchLinux","threadId":"62743","inReplyTo":"Z73DTwr9RicKMINe@ArchLinux","subject":"[PATCH v7 0/9] add more ref consistency checks","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:48:08Z","receivedAt":"2025-02-26T13:48:00Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis changes enhances the following things:\n\n1. [PATCH v7 3/9]: use \"open_nofollow\" with \"fstat\" to check whether the\nfile is regular. And update the test to improve coverage.\n2. [PACTH v7 4/9]: improve the commit message suggested by Patrick.\n\nThanks,\nJialuo\n\n---\n\nThis series mainly does the following things:\n\n1. Fix subshell issues\n2. Add ref checks for packed-backend.\n 1. Check whether the filetype of \"packed-refs\" is correct.\n 2. Check whether the syntax of \"packed-refs\" is correct by using the\n rules from \"packed-backend.c::create_snapshot\" and\n \"packed-backend.c::next_record\".\n 3. Check whether the pointed object exists and whether the\n \"packed-refs\" file is sorted.\n3. Call \"git refs verify\" for \"git-fsck(1)\".\n\nshejialuo (9):\n t0602: use subshell to ensure working directory unchanged\n builtin/refs: get worktrees without reading head information\n packed-backend: check whether the \"packed-refs\" is regular file\n packed-backend: check if header starts with \"# pack-refs with: \"\n packed-backend: add \"packed-refs\" header consistency check\n packed-backend: check whether the refname contains NUL characters\n packed-backend: add \"packed-refs\" entry consistency check\n packed-backend: check whether the \"packed-refs\" is sorted\n builtin/fsck: add `git refs verify` child process\n\n Documentation/fsck-msgids.adoc | 14 +\n Documentation/git-fsck.adoc | 7 +-\n builtin/fsck.c | 33 +-\n builtin/refs.c | 2 +-\n fsck.h | 4 +\n refs/packed-backend.c | 361 +++++++++-\n t/t0602-reffiles-fsck.sh | 1209 +++++++++++++++++++-------------\n worktree.c | 5 +\n worktree.h | 8 +\n 9 files changed, 1161 insertions(+), 482 deletions(-)\n\nRange-diff against v6:\n 1: b3952d80a2 = 1: b3952d80a2 t0602: use subshell to ensure working directory unchanged\n 2: fa5ce20bb7 = 2: fa5ce20bb7 builtin/refs: get worktrees without reading head information\n 3: 787645a700 ! 3: 861583f417 packed-backend: check whether the \"packed-refs\" is regular file\n @@ Commit message\n the expected filetype, confirming it is created by \"git pack-refs\"\n command.\n \n - Use \"lstat\" to check the file mode. If we cannot check the file status\n - due to there is no such file this is OK because there is a possibility\n - that there is no \"packed-refs\" in the repo.\n + We could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file.\n + If the returned \"fd\" value is less than 0, we could check whether the\n + \"errno\" is \"ELOOP\" to report an error to the user. And then we use\n + \"fstat\" to check whether the \"packed-refs\" file is a regular file.\n \n Reuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\n the user if \"packed-refs\" is not a regular file.\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n +\tstruct stat st;\n +\tint ret = 0;\n ++\tint fd;\n \n \tif (!is_main_worktree(wt))\n --\t\treturn 0;\n -+\t\tgoto cleanup;\n + \t\treturn 0;\n \n -\treturn 0;\n +\tif (o->verbose)\n +\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n +\n -+\tif (lstat(refs->path, &st) < 0) {\n ++\tfd = open_nofollow(refs->path, O_RDONLY);\n ++\tif (fd < 0) {\n +\t\t/*\n +\t\t * If the packed-refs file doesn't exist, there's nothing\n +\t\t * to check.\n +\t\t */\n +\t\tif (errno == ENOENT)\n +\t\t\tgoto cleanup;\n ++\n ++\t\tif (errno == ELOOP) {\n ++\t\t\tstruct fsck_ref_report report = { 0 };\n ++\t\t\treport.path = \"packed-refs\";\n ++\t\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n ++\t\t\t\t\t \"not a regular file but a symlink\");\n ++\t\t\tgoto cleanup;\n ++\t\t}\n ++\n ++\t\tret = error_errno(_(\"unable to open '%s'\"), refs->path);\n ++\t\tgoto cleanup;\n ++\t} else if (fstat(fd, &st) < 0) {\n +\t\tret = error_errno(_(\"unable to stat '%s'\"), refs->path);\n +\t\tgoto cleanup;\n -+\t}\n -+\n -+\tif (!S_ISREG(st.st_mode)) {\n ++\t} else if (!S_ISREG(st.st_mode)) {\n +\t\tstruct fsck_ref_report report = { 0 };\n +\t\treport.path = \"packed-refs\";\n +\t\tret = fsck_report_ref(o, &report,\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\t}\n +\n +cleanup:\n ++\tif (fd >= 0)\n ++\t\tclose(fd);\n +\treturn ret;\n }\n \n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref content checks should work wi\n +\t\tln -sf packed-refs-back .git/packed-refs &&\n +\t\ttest_must_fail git refs verify 2>err &&\n +\t\tcat >expect <<-EOF &&\n -+\t\terror: packed-refs: badRefFiletype: not a regular file\n ++\t\terror: packed-refs: badRefFiletype: not a regular file but a symlink\n +\t\tEOF\n +\t\trm .git/packed-refs &&\n ++\t\ttest_cmp expect err &&\n ++\n ++\t\tmkdir .git/packed-refs &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: packed-refs: badRefFiletype: not a regular file\n ++\t\tEOF\n ++\t\trm -r .git/packed-refs &&\n +\t\ttest_cmp expect err\n +\t)\n +'\n 4: f097e0f093 ! 4: 5f54cb05c3 packed-backend: check if header starts with \"# pack-refs with: \"\n @@ Metadata\n ## Commit message ##\n packed-backend: check if header starts with \"# pack-refs with: \"\n \n - We always write a space after \"# pack-refs with:\". However, when\n - creating the packed-ref snapshot, we only check whether the header\n - starts with \"# pack-refs with:\". However, we need to make sure that we\n - would not break compatibility by tightening the rule. The following is\n - how some third-party libraries handle the header of \"packed-ref\" file.\n + We always write a space after \"# pack-refs with:\" but we don't align\n + with this rule in the \"create_snapshot\" method where we would check\n + whether header starts with \"# pack-refs with:\". It might seem that we\n + should undoubtedly tighten this rule, however, we don't have any\n + technical documentation about this and there is a possibility that we\n + would break the compatibility for other third-party libraries.\n +\n + By investigating influential third-party libraries, we could conclude\n + how these libraries handle the header of \"packed-refs\" file:\n \n 1. libgit2 is fine and always writes the space. It also expects the\n whitespace to exist.\n @@ Commit message\n 3. gitoxide expects the space t exist and writes it.\n 4. go-git doesn't create the header by default.\n \n - So, we are safe to tighten the rule by checking whether the header\n - starts with \"# pack-refs with: \".\n + As many third-party libraries expect a single space after \"# pack-refs\n + with:\", if we forget to write the space after the colon,\n + \"create_snapshot\" won't catch this. And we would break other\n + re-implementations. So, we'd better tighten the rule by checking whether\n + the header starts with \"# pack-refs with: \".\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n 5: a589a38b68 ! 5: 7d7dc899ad packed-backend: add \"packed-refs\" header consistency check\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n +\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tstruct stat st;\n -+\tint fd;\n \tint ret = 0;\n - \n - \tif (!is_main_worktree(wt))\n + \tint fd;\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n -+\t/*\n -+\t * There is a chance that \"packed-refs\" file is removed or converted to\n -+\t * a symlink after filetype check and before open. So we need to avoid\n -+\t * this race condition by opening the file.\n -+\t */\n -+\tfd = open_nofollow(refs->path, O_RDONLY);\n -+\tif (fd < 0) {\n -+\t\tif (errno == ENOENT)\n -+\t\t\tgoto cleanup;\n -+\n -+\t\tif (errno == ELOOP) {\n -+\t\t\tstruct fsck_ref_report report = { 0 };\n -+\t\t\treport.path = \"packed-refs\";\n -+\t\t\tret = fsck_report_ref(o, &report,\n -+\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n -+\t\t\t\t\t \"not a regular file\");\n -+\t\t\tgoto cleanup;\n -+\t\t}\n -+\t}\n -+\n +\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n -+\t\tret = error_errno(_(\"unable to read %s\"), refs->path);\n ++\t\tret = error_errno(_(\"unable to read '%s'\"), refs->path);\n +\t\tgoto cleanup;\n +\t}\n +\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n +\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n +\n cleanup:\n + \tif (fd >= 0)\n + \t\tclose(fd);\n +\tstrbuf_release(&packed_ref_content);\n \treturn ret;\n }\n 6: 7255c2b597 = 6: 571479d3e7 packed-backend: check whether the refname contains NUL characters\n 7: 7794a2ebfd = 7: e498a57286 packed-backend: add \"packed-refs\" entry consistency check\n 8: 2a9138b14d ! 8: 3638cb118d packed-backend: check whether the \"packed-refs\" is sorted\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n \tstruct strbuf packed_ref_content = STRBUF_INIT;\n +\tunsigned int sorted = 0;\n \tstruct stat st;\n --\tint fd;\n \tint ret = 0;\n -+\tint fd;\n - \n - \tif (!is_main_worktree(wt))\n - \t\tgoto cleanup;\n + \tint fd;\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n +\t\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\n - \tstrbuf_release(&packed_ref_content);\n + \tif (fd >= 0)\n \n ## t/t0602-reffiles-fsck.sh ##\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'packed-refs content should be checked' '\n 9: ccde32491f = 9: 5d87e76d28 builtin/fsck: add `git refs verify` child process\n-- \n2.48.1\n\n"},{"id":"513082","messageId":"Z78b7hfgq-RWtp-q@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v7 1/9] t0602: use subshell to ensure working directory unchanged","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:49:34Z","receivedAt":"2025-02-26T13:49:28Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"For every test, we would execute the command \"cd repo\" in the first but\nwe never execute the command \"cd ..\" to restore the working directory.\nHowever, it's either not a good idea use above way. Because if any test\nfails between \"cd repo\" and \"cd ..\", the \"cd ..\" will never be reached.\nAnd we cannot correctly restore the working directory.\n\nLet's use subshell to ensure that the current working directory could be\nrestored to the correct path.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n t/t0602-reffiles-fsck.sh | 967 ++++++++++++++++++++-------------------\n 1 file changed, 494 insertions(+), 473 deletions(-)\n\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex d4a08b823b..cf7a202d0d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -14,222 +14,229 @@ test_expect_success 'ref name should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b default-branch &&\n-\tgit tag default-tag &&\n-\tgit tag multi_hierarchy/default-tag &&\n-\n-\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n-\trm $branch_dir_prefix/@ &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n-\tgit refs verify 2>err &&\n-\trm $tag_dir_prefix/tag-1.lock &&\n-\ttest_must_be_empty err &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/.lock: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t(\n+\t\tcd repo &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b default-branch &&\n+\t\tgit tag default-tag &&\n+\t\tgit tag multi_hierarchy/default-tag &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\t\trm $branch_dir_prefix/@ &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $tag_dir_prefix/tag-1.lock &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \t\ttest_must_fail git refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\terror: refs/tags/.lock: badRefName: invalid refname format\n \t\tEOF\n-\t\trm -r \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $tag_dir_prefix/.lock &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=warn refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n-\ttest_must_be_empty err\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b branch-1 &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=warn refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm $branch_dir_prefix/.branch-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n '\n \n test_expect_success 'ref name check should work for multiple worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\n-\tcd repo &&\n-\ttest_commit initial &&\n-\tgit checkout -b branch-1 &&\n-\ttest_commit second &&\n-\tgit checkout -b branch-2 &&\n-\ttest_commit third &&\n-\tgit checkout -b branch-3 &&\n-\tgit worktree add ./worktree-1 branch-1 &&\n-\tgit worktree add ./worktree-2 branch-2 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n-\t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n \t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n-\n-\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n-\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err &&\n-\n-\tfor worktree in \"worktree-1\" \"worktree-2\"\n-\tdo\n+\t\tcd repo &&\n+\t\ttest_commit initial &&\n+\t\tgit checkout -b branch-1 &&\n+\t\ttest_commit second &&\n+\t\tgit checkout -b branch-2 &&\n+\t\ttest_commit third &&\n+\t\tgit checkout -b branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-1 &&\n+\t\tgit worktree add ./worktree-2 branch-2 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n \t\t(\n-\t\t\tcd $worktree &&\n-\t\t\ttest_must_fail git refs verify 2>err &&\n-\t\t\tcat >expect <<-EOF &&\n-\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\t\t\tEOF\n-\t\t\tsort err >sorted_err &&\n-\t\t\ttest_cmp expect sorted_err || return 1\n-\t\t)\n-\tdone\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\n+\t\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\t\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err &&\n+\n+\t\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\t\tdo\n+\t\t\t(\n+\t\t\t\tcd $worktree &&\n+\t\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\t\tcat >expect <<-EOF &&\n+\t\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\t\tEOF\n+\t\t\t\tsort err >sorted_err &&\n+\t\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t\t)\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tfor trailing_content in \" garbage\" \" more garbage\"\n-\tdo\n-\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-garbage &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n+\t\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n-\t'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\t'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n \n-\t garbage'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err\n+\t\t garbage'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -237,99 +244,103 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tbad_content_1=$(git rev-parse main)x &&\n-\tbad_content_2=xfsazqfxcadas &&\n-\tbad_content_3=Xfsazqfxcadas &&\n-\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n-\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n-\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n-\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n-\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n-\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tbad_content_1=$(git rev-parse main)x &&\n+\t\tbad_content_2=xfsazqfxcadas &&\n+\t\tbad_content_3=Xfsazqfxcadas &&\n+\t\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\t\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\t\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\t\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\t\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-complicated &&\n-\ttest_cmp expect err\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (aggregate)' '\n@@ -337,32 +348,34 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n-\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'the target of the textual symref should be checked' '\n@@ -370,28 +383,30 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n-\t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n-\n-\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n-\t\tgit refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked' '\n@@ -399,201 +414,207 @@ test_expect_success SYMLINKS 'symlink symref content should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n-\tEOF\n-\trm $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_cmp expect err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-good &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\t\tEOF\n+\t\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tmain_worktree_refdir_prefix=.git/refs/heads &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\n-\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tfor bad_referent_name in \".tag\" \"branch \"\n-\tdo\n-\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tmain_worktree_refdir_prefix=.git/refs/heads &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $worktree1_refdir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor bad_referent_name in \".tag\" \"branch \"\n+\t\tdo\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-garbage &&\n-\ttest_cmp expect err\n+\t\trm $worktree1_refdir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_done\n-- \n2.48.1\n\n"},{"id":"513083","messageId":"Z78b-XOIvgbQvufQ@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v7 2/9] builtin/refs: get worktrees without reading head information","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:49:45Z","receivedAt":"2025-02-26T13:49:37Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\nand \"next_record\" which would check the correctness of the content of\nthe \"packed-ref\" file. When anything is bad, the program will die.\n\nIt may seem that we have nothing relevant to above feature, because we\nare going to read and parse the raw \"packed-ref\" file without creating\nthe snapshot and using the ref iterator to check the consistency.\n\nHowever, when using \"get_worktrees\" in \"builtin/refs\", we would parse\nthe \"HEAD\" information. If the referent of the \"HEAD\" is inside the\n\"packed-ref\", we will call \"create_snapshot\" function to parse the\n\"packed-ref\" to get the information. No matter whether the entry of\n\"HEAD\" in \"packed-ref\" is correct, \"create_snapshot\" would call\n\"verify_buffer_safe\" to check whether there is a newline in the last\nline of the file. If not, the program will die.\n\nAlthough this behavior has no harm for the program, it will\nshort-circuit the program. When the users execute \"git refs verify\" or\n\"git fsck\", we should avoid reading the head information, which may\nexecute the read operation in packed backend with stricter checks to die\nthe program. Instead, we should continue to check other parts of the\n\"packed-refs\" file completely.\n\nFortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\nworktrees, 2023-12-29), we have introduced a function\n\"get_worktrees_internal\" which allows us to get worktrees without\nreading head information.\n\nCreate a new exposed function \"get_worktrees_without_reading_head\", then\nreplace the \"get_worktrees\" in \"builtin/refs\" with the new created\nfunction.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 2 +-\n worktree.c | 5 +++++\n worktree.h | 8 ++++++++\n 3 files changed, 14 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex a29f195834..55ff5dae11 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tworktrees = get_worktrees();\n+\tworktrees = get_worktrees_without_reading_head();\n \tfor (size_t i = 0; worktrees[i]; i++)\n \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n \t\t\t\t &fsck_refs_options, worktrees[i]);\ndiff --git a/worktree.c b/worktree.c\nindex d4a68c9c23..d23482a746 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -198,6 +198,11 @@ struct worktree **get_worktrees(void)\n \treturn get_worktrees_internal(0);\n }\n \n+struct worktree **get_worktrees_without_reading_head(void)\n+{\n+\treturn get_worktrees_internal(1);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\ndiff --git a/worktree.h b/worktree.h\nindex 38145df80f..a305c7e2c7 100644\n--- a/worktree.h\n+++ b/worktree.h\n@@ -30,6 +30,14 @@ struct worktree {\n */\n struct worktree **get_worktrees(void);\n \n+/*\n+ * Like `get_worktrees`, but does not read HEAD. Skip reading HEAD allows to\n+ * get the worktree without worrying about failures pertaining to parsing\n+ * the HEAD ref. This is useful in contexts where it is assumed that the\n+ * refdb may not be in a consistent state.\n+ */\n+struct worktree **get_worktrees_without_reading_head(void);\n+\n /*\n * Returns 1 if linked worktrees exist, 0 otherwise.\n */\n-- \n2.48.1\n\n"},{"id":"513084","messageId":"Z78cAU69IUSDgpuD@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v7 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:49:53Z","receivedAt":"2025-02-26T13:49:46Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\nconsistency and correctness of \"packed-refs\" file, they never check the\nfiletype of the \"packed-refs\". Let's verify that the \"packed-refs\" has\nthe expected filetype, confirming it is created by \"git pack-refs\"\ncommand.\n\nWe could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file.\nIf the returned \"fd\" value is less than 0, we could check whether the\n\"errno\" is \"ELOOP\" to report an error to the user. And then we use\n\"fstat\" to check whether the \"packed-refs\" file is a regular file.\n\nReuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\nthe user if \"packed-refs\" is not a regular file.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 50 +++++++++++++++++++++++++++++++++++++---\n t/t0602-reffiles-fsck.sh | 30 ++++++++++++++++++++++++\n 2 files changed, 77 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a7b6f74b6e..f69a0598c7 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -4,6 +4,7 @@\n #include \"../git-compat-util.h\"\n #include \"../config.h\"\n #include \"../dir.h\"\n+#include \"../fsck.h\"\n #include \"../gettext.h\"\n #include \"../hash.h\"\n #include \"../hex.h\"\n@@ -1748,15 +1749,58 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n-static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED,\n+static int packed_fsck(struct ref_store *ref_store,\n+\t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n+\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n+\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct stat st;\n+\tint ret = 0;\n+\tint fd;\n \n \tif (!is_main_worktree(wt))\n \t\treturn 0;\n \n-\treturn 0;\n+\tif (o->verbose)\n+\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n+\n+\tfd = open_nofollow(refs->path, O_RDONLY);\n+\tif (fd < 0) {\n+\t\t/*\n+\t\t * If the packed-refs file doesn't exist, there's nothing\n+\t\t * to check.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tif (errno == ELOOP) {\n+\t\t\tstruct fsck_ref_report report = { 0 };\n+\t\t\treport.path = \"packed-refs\";\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t\t \"not a regular file but a symlink\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tret = error_errno(_(\"unable to open '%s'\"), refs->path);\n+\t\tgoto cleanup;\n+\t} else if (fstat(fd, &st) < 0) {\n+\t\tret = error_errno(_(\"unable to stat '%s'\"), refs->path);\n+\t\tgoto cleanup;\n+\t} else if (!S_ISREG(st.st_mode)) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs\";\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t \"not a regular file\");\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tif (fd >= 0)\n+\t\tclose(fd);\n+\treturn ret;\n }\n \n struct ref_storage_be refs_be_packed = {\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex cf7a202d0d..68b7d4999e 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -617,4 +617,34 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t)\n '\n \n+test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit pack-refs --all &&\n+\n+\t\tmv .git/packed-refs .git/packed-refs-back &&\n+\t\tln -sf packed-refs-back .git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs: badRefFiletype: not a regular file but a symlink\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tmkdir .git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs: badRefFiletype: not a regular file\n+\t\tEOF\n+\t\trm -r .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513085","messageId":"Z78cC7LY7PxEvDlB@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v7 4/9] packed-backend: check if header starts with \"# pack-refs with: \"","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:50:03Z","receivedAt":"2025-02-26T13:49:55Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We always write a space after \"# pack-refs with:\" but we don't align\nwith this rule in the \"create_snapshot\" method where we would check\nwhether header starts with \"# pack-refs with:\". It might seem that we\nshould undoubtedly tighten this rule, however, we don't have any\ntechnical documentation about this and there is a possibility that we\nwould break the compatibility for other third-party libraries.\n\nBy investigating influential third-party libraries, we could conclude\nhow these libraries handle the header of \"packed-refs\" file:\n\n1. libgit2 is fine and always writes the space. It also expects the\n whitespace to exist.\n2. JGit does not expect th header to have a trailing space, but expects\n the \"peeled\" capability to have a leading space, which is mostly\n equivalent because that capability is typically the first one we\n write. It always writes the space.\n3. gitoxide expects the space t exist and writes it.\n4. go-git doesn't create the header by default.\n\nAs many third-party libraries expect a single space after \"# pack-refs\nwith:\", if we forget to write the space after the colon,\n\"create_snapshot\" won't catch this. And we would break other\nre-implementations. So, we'd better tighten the rule by checking whether\nthe header starts with \"# pack-refs with: \".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex f69a0598c7..3dd3fec459 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -694,7 +694,7 @@ static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n \n \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n \n-\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n+\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n \t\t\tdie_invalid_line(refs->path,\n \t\t\t\t\t snapshot->buf,\n \t\t\t\t\t snapshot->eof - snapshot->buf);\n-- \n2.48.1\n\n"},{"id":"513086","messageId":"Z78cFPZvptoJfReH@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v7 5/9] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:50:12Z","receivedAt":"2025-02-26T13:50:04Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c::create_snapshot\", if there is a header (the line\nwhich starts with '#'), we will check whether the line starts with \"#\npack-refs with: \". However, we need to consider other situations and\ndiscuss whether we need to add checks.\n\n1. If the header does not exist, we should not report an error to the\n user. This is because in older Git version, we never write header in\n the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n in runtime.\n2. If the header content does not start with \"# packed-ref with: \", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n3. If the header content is not the same as the constant string\n \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n intentionally and runtime \"create_snapshot\" won't complain about\n unknown traits. In order to align with the runtime behavior. There is\n no need to report.\n\nAs we have analyzed, we only need to check the case 2 in the above. In\norder to do this, use \"open_nofollow\" function to get the file\ndescriptor and then read the \"packed-refs\" file via \"strbuf_read\". Like\nwhat \"create_snapshot\" and other functions do, we could split the line\nby finding the next newline in the buffer. When we cannot find a\nnewline, we could report an error.\n\nSo, create a function \"packed_fsck_ref_next_line\" to find the next\nnewline and if there is no such newline, use\n\"packedRefEntryNotTerminated(ERROR)\" to report an error to the user.\n\nThen, parse the first line to apply the checks. Update the test to\nexercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 8 ++++\n fsck.h | 2 +\n refs/packed-backend.c | 73 ++++++++++++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 52 ++++++++++++++++++++++++\n 4 files changed, 135 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex b14bc44ca4..11906f90fd 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -16,6 +16,10 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefHeader`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid\n+\theader.\n+\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n@@ -176,6 +180,10 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`packedRefEntryNotTerminated`::\n+\t(ERROR) The \"packed-refs\" file contains an entry that is\n+\tnot terminated by a newline.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex a44c231a5f..67e3c97bc0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n@@ -53,6 +54,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE, ERROR) \\\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 3dd3fec459..b00fca6501 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1749,12 +1749,76 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+static int packed_fsck_ref_next_line(struct fsck_options *o,\n+\t\t\t\t unsigned long line_number, const char *start,\n+\t\t\t\t const char *eof, const char **eol)\n+{\n+\tint ret = 0;\n+\n+\t*eol = memchr(start, '\\n', eof - start);\n+\tif (!*eol) {\n+\t\tstruct strbuf packed_entry = STRBUF_INIT;\n+\t\tstruct fsck_ref_report report = { 0 };\n+\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n+\t\t\t\t \"'%.*s' is not terminated with a newline\",\n+\t\t\t\t (int)(eof - start), start);\n+\n+\t\t/*\n+\t\t * There is no newline but we still want to parse it to the end of\n+\t\t * the buffer.\n+\t\t */\n+\t\t*eol = eof;\n+\t\tstrbuf_release(&packed_entry);\n+\t}\n+\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_header(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs.header\";\n+\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tunsigned long line_number = 1;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\tif (*start == '#') {\n+\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\treturn ret;\n+}\n+\n static int packed_fsck(struct ref_store *ref_store,\n \t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tstruct stat st;\n \tint ret = 0;\n \tint fd;\n@@ -1797,9 +1861,18 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n+\t\tret = error_errno(_(\"unable to read '%s'\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\n cleanup:\n \tif (fd >= 0)\n \t\tclose(fd);\n+\tstrbuf_release(&packed_ref_content);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 68b7d4999e..74d876984d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -647,4 +647,56 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-refs header should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with a: peeled fully-peeled\" \\\n+\t\t\t\t \"# pack-refs with:peeled fully-peeled sorted\"\n+\t\tdo\n+\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with: '\\''\n+\t\t\tEOF\n+\t\t\trm .git/packed-refs &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n+'\n+\n+test_expect_success 'packed-refs missing header should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"$(git rev-parse HEAD) refs/heads/main\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n+test_expect_success 'packed-refs unknown traits should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted foo\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513087","messageId":"Z78cHmY7WZvwCpl6@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v7 6/9] packed-backend: check whether the refname contains NUL characters","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:50:22Z","receivedAt":"2025-02-26T13:50:14Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will use \"check_refname_format\" to check\nthe consistency of the refname. If it is not OK, the program will die.\nHowever, it is reported in [1], we cannot catch some corruption. But we\nalready have the code path and we must miss out something.\n\nWe use the following code to get the refname:\n\n strbuf_add(&iter->refname_buf, p, eol - p);\n iter->base.refname = iter->refname_buf.buf\n\nIn the above code, `p` is the start pointer of the refname and `eol` is\nthe next newline pointer. We calculate the length of the refname by\nsubtracting the two pointers. Then we add the memory range between `p`\nand `eol` to get the refname.\n\nHowever, if there are some NUL characters in the memory range between `p`\nand `eol`, we will see the refname as a valid ref name as long as the\nmemory range between `p` and first occurred NUL character is valid.\n\nIn order to catch above corruption, create a new function\n\"refname_contains_nul\" by searching the first NUL character. If it is\nnot at the end of the string, there must be some NUL characters in the\nrefname.\n\nUse this function in \"next_record\" function to die the program if\n\"refname_contains_nul\" returns true.\n\n[1] https://lore.kernel.org/git/6cfee0e4-3285-4f18-91ff-d097da9de737@rd10.de/\n\nReported-by: R. Diez \nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex b00fca6501..6e7d08c565 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -494,6 +494,21 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n \t\t\t\t last_line, eof - last_line);\n }\n \n+/*\n+ * When parsing the \"packed-refs\" file, we will parse it line by line.\n+ * Because we know the start pointer of the refname and the next\n+ * newline pointer, we could calculate the length of the refname by\n+ * subtracting the two pointers. However, there is a corner case where\n+ * the refname contains corrupted embedded NUL characters. And\n+ * `check_refname_format()` will not catch this when the truncated\n+ * refname is still a valid refname. To prevent this, we need to check\n+ * whether the refname contains the NUL characters.\n+ */\n+static int refname_contains_nul(struct strbuf *refname)\n+{\n+\treturn !!memchr(refname->buf, '\\0', refname->len);\n+}\n+\n #define SMALL_FILE_SIZE (32*1024)\n \n /*\n@@ -895,6 +910,9 @@ static int next_record(struct packed_ref_iterator *iter)\n \tstrbuf_add(&iter->refname_buf, p, eol - p);\n \titer->base.refname = iter->refname_buf.buf;\n \n+\tif (refname_contains_nul(&iter->refname_buf))\n+\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n+\n \tif (check_refname_format(iter->base.refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!refname_is_safe(iter->base.refname))\n \t\t\tdie(\"packed refname is dangerous: %s\",\n-- \n2.48.1\n\n"},{"id":"513088","messageId":"Z78cKDRtgYykccPH@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v7 7/9] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:50:32Z","receivedAt":"2025-02-26T13:50:25Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will parse the ref entry to check the\nconsistency. This function has already checked the following things:\n\n1. Parse the main line of the ref entry to inspect whether the oid is\n not correct. Then, check whether the next character is oid. Then\n check the refname.\n2. If the next line starts with '^', it would continue to parse the\n peeled oid and check whether the last character is '\\n'.\n\nAs we decide to implement the ref consistency check for \"packed-refs\",\nlet's port these two checks and update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 122 ++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 44 ++++++++++++\n 4 files changed, 169 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 11906f90fd..02a7bf0503 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -16,6 +16,9 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefEntry`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid entry.\n+\n `badPackedRefHeader`::\n \t(ERROR) The \"packed-refs\" file contains an invalid\n \theader.\ndiff --git a/fsck.h b/fsck.h\nindex 67e3c97bc0..14d70f6653 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\n \tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 6e7d08c565..8c410fca77 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1812,9 +1812,114 @@ static int packed_fsck_ref_header(struct fsck_options *o,\n \treturn 0;\n }\n \n+static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id peeled;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\t/*\n+\t * Skip the '^' and parse the peeled oid.\n+\t */\n+\tstart++;\n+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid peeled oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p != eol) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n+\t\t\t\t (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_main_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t struct strbuf *refname,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id oid;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p == eol || !isspace(*p)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n+\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tp++;\n+\tstrbuf_reset(refname);\n+\tstrbuf_add(refname, p, eol - p);\n+\tif (refname_contains_nul(refname)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"refname '%s' contains NULL binaries\",\n+\t\t\t\t refname->buf);\n+\t}\n+\n+\tif (check_refname_format(refname->buf, 0)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n+\t\t\t\t \"has bad refname '%s'\", refname->buf);\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n+\tstruct strbuf refname = STRBUF_INIT;\n \tunsigned long line_number = 1;\n \tconst char *eol;\n \tint ret = 0;\n@@ -1827,6 +1932,21 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\tline_number++;\n \t}\n \n+\twhile (start < eof) {\n+\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\tret |= packed_fsck_ref_main_line(o, ref_store, line_number, &refname, start, eol);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t\tif (start < eof && *start == '^') {\n+\t\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, line_number,\n+\t\t\t\t\t\t\t start, eol);\n+\t\t\tstart = eol + 1;\n+\t\t\tline_number++;\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&refname);\n \treturn ret;\n }\n \n@@ -1884,7 +2004,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 74d876984d..a88c792ce1 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -699,4 +699,48 @@ test_expect_success 'packed-refs unknown traits should not be reported' '\n \t)\n '\n \n+test_expect_success 'packed-refs content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tgit tag -a annotated-tag-2 -m tag-2 &&\n+\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n+\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$short_oid refs/heads/branch-1\n+\t\t${branch_1_oid}x\n+\t\t$branch_2_oid refs/heads/bad-branch\n+\t\t$branch_2_oid refs/heads/branch.\n+\t\t$tag_1_oid refs/tags/annotated-tag-3\n+\t\t^$short_oid\n+\t\t$tag_2_oid refs/tags/annotated-tag-4.\n+\t\t^$tag_2_peeled_oid garbage\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: badPackedRefEntry: '\\''$short_oid refs/heads/branch-1'\\'' has invalid oid\n+\t\terror: packed-refs line 3: badPackedRefEntry: has no space after oid '\\''$branch_1_oid'\\'' but with '\\''x'\\''\n+\t\terror: packed-refs line 4: badRefName: has bad refname '\\'' refs/heads/bad-branch'\\''\n+\t\terror: packed-refs line 5: badRefName: has bad refname '\\''refs/heads/branch.'\\''\n+\t\terror: packed-refs line 7: badPackedRefEntry: '\\''$short_oid'\\'' has invalid peeled oid\n+\t\terror: packed-refs line 8: badRefName: has bad refname '\\''refs/tags/annotated-tag-4.'\\''\n+\t\terror: packed-refs line 9: badPackedRefEntry: has trailing garbage after peeled oid '\\'' garbage'\\''\n+\t\tEOF\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513089","messageId":"Z78cQEM0xkeJ5b2X@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v7 8/9] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:50:56Z","receivedAt":"2025-02-26T13:50:48Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"When there is a \"sorted\" trait in the header of the \"packed-refs\" file,\nit means that each entry is sorted increasingly by comparing the\nrefname. We should add checks to verify whether the \"packed-refs\" is\nsorted in this case.\n\nUpdate the \"packed_fsck_ref_header\" to know whether there is a \"sorted\"\ntrail in the header. It may seem that we could record all refnames\nduring the parsing process and then compare later. However, this is not\na good design due to the following reasons:\n\n1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n in the \"packed-refs\" file.\n2. We cannot reuse the existing compare function \"cmp_packed_ref_records\"\n which cause repetition.\n\nBecause \"cmp_packed_ref_records\" needs an extra parameter \"struct\nsnaphost\", extract the common part into a new function\n\"cmp_packed_ref_records\" to reuse this function to compare.\n\nThen, create a new function \"packed_fsck_ref_sorted\" to parse the file\nagain and user the new fsck message \"packedRefUnsorted(ERROR)\" to report\nto the user if the file is not sorted.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 116 ++++++++++++++++++++++++++++-----\n t/t0602-reffiles-fsck.sh | 87 +++++++++++++++++++++++++\n 4 files changed, 191 insertions(+), 16 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 02a7bf0503..9601fff228 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -187,6 +187,9 @@\n \t(ERROR) The \"packed-refs\" file contains an entry that is\n \tnot terminated by a newline.\n \n+`packedRefUnsorted`::\n+\t(ERROR) The \"packed-refs\" file is not sorted.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex 14d70f6653..19f3cb2773 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -56,6 +56,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n \tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n+\tFUNC(PACKED_REF_UNSORTED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 8c410fca77..a1710d7c2a 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -300,14 +300,9 @@ struct snapshot_record {\n \tsize_t len;\n };\n \n-static int cmp_packed_ref_records(const void *v1, const void *v2,\n-\t\t\t\t void *cb_data)\n-{\n-\tconst struct snapshot *snapshot = cb_data;\n-\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n-\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n-\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n \n+static int cmp_packed_refname(const char *r1, const char *r2)\n+{\n \twhile (1) {\n \t\tif (*r1 == '\\n')\n \t\t\treturn *r2 == '\\n' ? 0 : -1;\n@@ -322,6 +317,17 @@ static int cmp_packed_ref_records(const void *v1, const void *v2,\n \t}\n }\n \n+static int cmp_packed_ref_records(const void *v1, const void *v2,\n+\t\t\t\t void *cb_data)\n+{\n+\tconst struct snapshot *snapshot = cb_data;\n+\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n+\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n+\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n+\n+\treturn cmp_packed_refname(r1, r2);\n+}\n+\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n@@ -1797,19 +1803,33 @@ static int packed_fsck_ref_next_line(struct fsck_options *o,\n }\n \n static int packed_fsck_ref_header(struct fsck_options *o,\n-\t\t\t\t const char *start, const char *eol)\n+\t\t\t\t const char *start, const char *eol,\n+\t\t\t\t unsigned int *sorted)\n {\n-\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n+\tchar *tmp_line;\n+\tint ret = 0;\n+\tchar *p;\n+\n+\ttmp_line = xmemdupz(start, eol - start);\n+\tif (!skip_prefix(tmp_line, \"# pack-refs with: \", (const char **)&p)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \t\treport.path = \"packed-refs.header\";\n \n-\t\treturn fsck_report_ref(o, &report,\n-\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n-\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n-\t\t\t\t (int)(eol - start), start);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n \t}\n \n-\treturn 0;\n+\tstring_list_split_in_place(&traits, p, \" \", -1);\n+\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n+\n+cleanup:\n+\tfree(tmp_line);\n+\tstring_list_clear(&traits, 0);\n+\treturn ret;\n }\n \n static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n@@ -1915,8 +1935,68 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n \treturn ret;\n }\n \n+static int packed_fsck_ref_sorted(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct strbuf refname1 = STRBUF_INIT;\n+\tstruct strbuf refname2 = STRBUF_INIT;\n+\tunsigned long line_number = 1;\n+\tconst char *former = NULL;\n+\tconst char *current;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tif (*start == '#') {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\tfor (; start < eof; line_number++, start = eol + 1) {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\n+\t\tif (*start == '^')\n+\t\t\tcontinue;\n+\n+\t\tif (!former) {\n+\t\t\tformer = start + hexsz + 1;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tcurrent = start + hexsz + 1;\n+\t\tif (cmp_packed_refname(former, current) >= 0) {\n+\t\t\tconst char *err_fmt =\n+\t\t\t\t\"refname '%s' is less than previous refname '%s'\";\n+\n+\t\t\teol = memchr(former, '\\n', eof - former);\n+\t\t\tstrbuf_add(&refname1, former, eol - former);\n+\t\t\teol = memchr(current, '\\n', eof - current);\n+\t\t\tstrbuf_add(&refname2, current, eol - current);\n+\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\t\treport.path = packed_entry.buf;\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_PACKED_REF_UNSORTED,\n+\t\t\t\t\t err_fmt, refname2.buf, refname1.buf);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tformer = current;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname1);\n+\tstrbuf_release(&refname2);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned int *sorted,\n \t\t\t\t const char *start, const char *eof)\n {\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -1926,7 +2006,7 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \n \tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n \tif (*start == '#') {\n-\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\t\tret |= packed_fsck_ref_header(o, start, eol, sorted);\n \n \t\tstart = eol + 1;\n \t\tline_number++;\n@@ -1957,6 +2037,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n \tstruct strbuf packed_ref_content = STRBUF_INIT;\n+\tunsigned int sorted = 0;\n \tstruct stat st;\n \tint ret = 0;\n \tint fd;\n@@ -2004,8 +2085,11 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, &sorted, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\tif (!ret && sorted)\n+\t\tret = packed_fsck_ref_sorted(o, ref_store, packed_ref_content.buf,\n+\t\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\n \tif (fd >= 0)\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex a88c792ce1..767e2bd4a0 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -743,4 +743,91 @@ test_expect_success 'packed-refs content should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-ref with sorted trait should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\t$tag_1_oid $refname3\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 3: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname1'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$tag_1_oid $refname3\n+\t\t^$tag_1_peeled_oid\n+\t\t$branch_2_oid $refname2\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 4: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname3'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n+test_expect_success 'packed-ref without sorted trait should not be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513090","messageId":"Z78cSA6qOF5KaATM@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v7 9/9] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-26T13:51:04Z","receivedAt":"2025-02-26T13:50:56Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"At now, we have already implemented the ref consistency checks for both\n\"files-backend\" and \"packed-backend\". Although we would check some\nredundant things, it won't cause trouble. So, let's integrate it into\nthe \"git-fsck(1)\" command to get feedback from the users. And also by\ncalling \"git refs verify\" in \"git-fsck(1)\", we make sure that the new\nadded checks don't break.\n\nIntroduce a new function \"fsck_refs\" that initializes and runs a child\nprocess to execute the \"git refs verify\" command. In order to provide\nthe user interface create a progress which makes the total task be 1.\nIt's hard to know how many loose refs we will check now. We might\nimprove this later.\n\nThen, introduce the option to allow the user to disable checking ref\ndatabase consistency. Put this function in the very first execution\nsequence of \"git-fsck(1)\" due to that we don't want the existing code of\n\"git-fsck(1)\" which would implicitly check the consistency of refs to\ndie the program.\n\nLast, update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/git-fsck.adoc | 7 ++++++-\n builtin/fsck.c | 33 ++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 39 +++++++++++++++++++++++++++++++++++++\n 3 files changed, 77 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-fsck.adoc b/Documentation/git-fsck.adoc\nindex 8f32800a83..11203ba925 100644\n--- a/Documentation/git-fsck.adoc\n+++ b/Documentation/git-fsck.adoc\n@@ -12,7 +12,7 @@ SYNOPSIS\n 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n-\t [--[no-]name-objects] [...]\n+\t [--[no-]name-objects] [--[no-]references] [...]\n \n DESCRIPTION\n -----------\n@@ -104,6 +104,11 @@ care about this output and want to speed it up further.\n \tprogress status even if the standard error stream is not\n \tdirected to a terminal.\n \n+--[no-]references::\n+\tControl whether to check the references database consistency\n+\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n+\tThe default is to check the references database.\n+\n CONFIGURATION\n -------------\n \ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 7a4dcb0716..f4f395cfbd 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -50,6 +50,7 @@ static int verbose;\n static int show_progress = -1;\n static int show_dangling = 1;\n static int name_objects;\n+static int check_references = 1;\n #define ERROR_OBJECT 01\n #define ERROR_REACHABLE 02\n #define ERROR_PACK 04\n@@ -905,11 +906,37 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n \treturn res;\n }\n \n+static void fsck_refs(struct repository *r)\n+{\n+\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n+\tstruct progress *progress = NULL;\n+\n+\tif (show_progress)\n+\t\tprogress = start_progress(r, _(\"Checking ref database\"), 1);\n+\n+\tif (verbose)\n+\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n+\n+\tchild_process_init(&refs_verify);\n+\trefs_verify.git_cmd = 1;\n+\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n+\tif (verbose)\n+\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n+\tif (check_strict)\n+\t\tstrvec_push(&refs_verify.args, \"--strict\");\n+\n+\tif (run_command(&refs_verify))\n+\t\terrors_found |= ERROR_REFS;\n+\n+\tdisplay_progress(progress, 1);\n+\tstop_progress(&progress);\n+}\n+\n static char const * const fsck_usage[] = {\n \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n \t \" [--[no-]dangling] [--[no-]progress] [--connectivity-only]\\n\"\n-\t \" [--[no-]name-objects] [...]\"),\n+\t \" [--[no-]name-objects] [--[no-]references] [...]\"),\n \tNULL\n };\n \n@@ -928,6 +955,7 @@ static struct option fsck_opts[] = {\n \t\t\t\tN_(\"write dangling objects in .git/lost-found\")),\n \tOPT_BOOL(0, \"progress\", &show_progress, N_(\"show progress\")),\n \tOPT_BOOL(0, \"name-objects\", &name_objects, N_(\"show verbose names for reachable objects\")),\n+\tOPT_BOOL(0, \"references\", &check_references, N_(\"check reference database consistency\")),\n \tOPT_END(),\n };\n \n@@ -970,6 +998,9 @@ int cmd_fsck(int argc,\n \tgit_config(git_fsck_config, &fsck_obj_options);\n \tprepare_repo_settings(the_repository);\n \n+\tif (check_references)\n+\t\tfsck_refs(the_repository);\n+\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n \t\tfor_each_packed_object(the_repository,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 767e2bd4a0..9d1dc2144c 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -830,4 +830,43 @@ test_expect_success 'packed-ref without sorted trait should not be checked' '\n \t)\n '\n \n+test_expect_success '--[no-]references option should apply to fsck' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --references 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --no-references 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513114","messageId":"xmqq5xkwd042.fsf@gitster.g","threadId":"62743","inReplyTo":"Z78cAU69IUSDgpuD@ArchLinux","subject":"Re: [PATCH v7 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-26T18:36:29Z","receivedAt":"2025-02-26T18:36:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> +static int packed_fsck(struct ref_store *ref_store,\n> +\t\t struct fsck_options *o,\n> \t\t struct worktree *wt)\n> {\n> +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> +\tstruct stat st;\n> +\tint ret = 0;\n> +\tint fd;\n> \n> \tif (!is_main_worktree(wt))\n> \t\treturn 0;\n\nI do not think it is worth a reroll only to improve this one, but\nfor future reference, initializing \"fd = -1\" and jumping to cleanup\nhere instead of \"return 0\" would future-proof the code better. This\nis especially so, given that in a few patches later, we would add a\nstrbuf that is initialized before this \"we do not do anything\noutside the primary worktree\" short-cut, and many \"goto cleanup\"s we\nsee in this patch below would jump to cleanup to strbuf_release() on\nthat initialized but unused strbuf. Jumping there with negative fd\nto cleanup that already avoids close(fd) for negative fd would be\nlike jumping there with initialized but unused strbuf. Having a\nsingle exit point (\"cleanup:\" label) would help future evolution of\nthe code, by making it easier to add more resource-acquriing code to\nthis function in the future.\n\n> -\treturn 0;\n> +\tif (o->verbose)\n> +\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n> +\n> +\tfd = open_nofollow(refs->path, O_RDONLY);\n> +\tif (fd < 0) {\n> +\t\t/*\n> +\t\t * If the packed-refs file doesn't exist, there's nothing\n> +\t\t * to check.\n> +\t\t */\n> +\t\tif (errno == ENOENT)\n> +\t\t\tgoto cleanup;\n> +\n> +\t\tif (errno == ELOOP) {\n> +\t\t\tstruct fsck_ref_report report = { 0 };\n> +\t\t\treport.path = \"packed-refs\";\n> +\t\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n> +\t\t\t\t\t \"not a regular file but a symlink\");\n> +\t\t\tgoto cleanup;\n> +\t\t}\n> +\n> +\t\tret = error_errno(_(\"unable to open '%s'\"), refs->path);\n> +\t\tgoto cleanup;\n> +\t} else if (fstat(fd, &st) < 0) {\n> +\t\tret = error_errno(_(\"unable to stat '%s'\"), refs->path);\n> +\t\tgoto cleanup;\n> +\t} else if (!S_ISREG(st.st_mode)) {\n> +\t\tstruct fsck_ref_report report = { 0 };\n> +\t\treport.path = \"packed-refs\";\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n> +\t\t\t\t \"not a regular file\");\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +cleanup:\n> +\tif (fd >= 0)\n> +\t\tclose(fd);\n> +\treturn ret;\n> }\n"},{"id":"513130","messageId":"Z7-4XRCVvLjFCFR8@ArchLinux","threadId":"62743","inReplyTo":"xmqq5xkwd042.fsf@gitster.g","subject":"Re: [PATCH v7 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T00:57:01Z","receivedAt":"2025-02-27T00:56:52Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Feb 26, 2025 at 10:36:29AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > +static int packed_fsck(struct ref_store *ref_store,\n> > +\t\t struct fsck_options *o,\n> > \t\t struct worktree *wt)\n> > {\n> > +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> > +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> > +\tstruct stat st;\n> > +\tint ret = 0;\n> > +\tint fd;\n> > \n> > \tif (!is_main_worktree(wt))\n> > \t\treturn 0;\n> \n> I do not think it is worth a reroll only to improve this one, but\n> for future reference, initializing \"fd = -1\" and jumping to cleanup\n> here instead of \"return 0\" would future-proof the code better. This\n> is especially so, given that in a few patches later, we would add a\n> strbuf that is initialized before this \"we do not do anything\n> outside the primary worktree\" short-cut, and many \"goto cleanup\"s we\n> see in this patch below would jump to cleanup to strbuf_release() on\n> that initialized but unused strbuf. Jumping there with negative fd\n> to cleanup that already avoids close(fd) for negative fd would be\n> like jumping there with initialized but unused strbuf. Having a\n> single exit point (\"cleanup:\" label) would help future evolution of\n> the code, by making it easier to add more resource-acquriing code to\n> this function in the future.\n> \n\nYou are right. Actually, I just want to avoid assigning the `fd` to -1.\nHowever, I didn't realize that I would initialize the strbuf later.\nAfter waking up, I have suddenly realized this problem.\n\nIf other reviewers don't have any comments for this new version, I will\nsend out a reroll. We have already iterated many times, if we could make\nit better, why not?\n\nThanks,\nJialuo\n"},{"id":"513148","messageId":"Z8ByUakK-YNyfwuL@pks.im","threadId":"62743","inReplyTo":"Z7-4XRCVvLjFCFR8@ArchLinux","subject":"Re: [PATCH v7 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-27T14:10:25Z","receivedAt":"2025-02-27T14:10:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Feb 27, 2025 at 08:57:01AM +0800, shejialuo wrote:\n> On Wed, Feb 26, 2025 at 10:36:29AM -0800, Junio C Hamano wrote:\n> > shejialuo writes:\n> > \n> > > +static int packed_fsck(struct ref_store *ref_store,\n> > > +\t\t struct fsck_options *o,\n> > > \t\t struct worktree *wt)\n> > > {\n> > > +\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n> > > +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n> > > +\tstruct stat st;\n> > > +\tint ret = 0;\n> > > +\tint fd;\n> > > \n> > > \tif (!is_main_worktree(wt))\n> > > \t\treturn 0;\n> > \n> > I do not think it is worth a reroll only to improve this one, but\n> > for future reference, initializing \"fd = -1\" and jumping to cleanup\n> > here instead of \"return 0\" would future-proof the code better. This\n> > is especially so, given that in a few patches later, we would add a\n> > strbuf that is initialized before this \"we do not do anything\n> > outside the primary worktree\" short-cut, and many \"goto cleanup\"s we\n> > see in this patch below would jump to cleanup to strbuf_release() on\n> > that initialized but unused strbuf. Jumping there with negative fd\n> > to cleanup that already avoids close(fd) for negative fd would be\n> > like jumping there with initialized but unused strbuf. Having a\n> > single exit point (\"cleanup:\" label) would help future evolution of\n> > the code, by making it easier to add more resource-acquriing code to\n> > this function in the future.\n> > \n> \n> You are right. Actually, I just want to avoid assigning the `fd` to -1.\n> However, I didn't realize that I would initialize the strbuf later.\n> After waking up, I have suddenly realized this problem.\n> \n> If other reviewers don't have any comments for this new version, I will\n> send out a reroll. We have already iterated many times, if we could make\n> it better, why not?\n\nI don't have anything else to add to this version, thanks!\n\nPatrick\n"},{"id":"513154","messageId":"Z8CMx7O19PMs9sVY@ArchLinux","threadId":"62743","inReplyTo":"Z78bmBSrDR20GY6g@ArchLinux","subject":"[PATCH v8 0/9] add more ref consistency checks","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:03:19Z","receivedAt":"2025-02-27T16:03:11Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis changes enhances the following things:\n\n1. [PATCH v8 3/9]: initialize fd = -1 for two purposes:\n 1. We could use unified `goto cleanup` where we have one only\n control.\n 2. We should not close the fd when we cannot open the file.\n\nHope that this version would be final. Thank for the effort of every\nreviewer.\n\nThanks,\nJialuo\n\n---\n\nThis series mainly does the following things:\n\n1. Fix subshell issues\n2. Add ref checks for packed-backend.\n 1. Check whether the filetype of \"packed-refs\" is correct.\n 2. Check whether the syntax of \"packed-refs\" is correct by using the\n rules from \"packed-backend.c::create_snapshot\" and\n \"packed-backend.c::next_record\".\n 3. Check whether the pointed object exists and whether the\n \"packed-refs\" file is sorted.\n3. Call \"git refs verify\" for \"git-fsck(1)\".\n\nshejialuo (9):\n t0602: use subshell to ensure working directory unchanged\n builtin/refs: get worktrees without reading head information\n packed-backend: check whether the \"packed-refs\" is regular file\n packed-backend: check if header starts with \"# pack-refs with: \"\n packed-backend: add \"packed-refs\" header consistency check\n packed-backend: check whether the refname contains NUL characters\n packed-backend: add \"packed-refs\" entry consistency check\n packed-backend: check whether the \"packed-refs\" is sorted\n builtin/fsck: add `git refs verify` child process\n\n Documentation/fsck-msgids.adoc | 14 +\n Documentation/git-fsck.adoc | 7 +-\n builtin/fsck.c | 33 +-\n builtin/refs.c | 2 +-\n fsck.h | 4 +\n refs/packed-backend.c | 363 +++++++++-\n t/t0602-reffiles-fsck.sh | 1209 +++++++++++++++++++-------------\n worktree.c | 5 +\n worktree.h | 8 +\n 9 files changed, 1162 insertions(+), 483 deletions(-)\n\nRange-diff against v7:\n 1: b3952d80a2 = 1: b3952d80a2 t0602: use subshell to ensure working directory unchanged\n 2: fa5ce20bb7 = 2: fa5ce20bb7 builtin/refs: get worktrees without reading head information\n 3: 861583f417 ! 3: b3686a9695 packed-backend: check whether the \"packed-refs\" is regular file\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n +\tstruct stat st;\n +\tint ret = 0;\n -+\tint fd;\n ++\tint fd = -1;\n \n \tif (!is_main_worktree(wt))\n - \t\treturn 0;\n +-\t\treturn 0;\n ++\t\tgoto cleanup;\n \n -\treturn 0;\n +\tif (o->verbose)\n 4: 5f54cb05c3 = 4: 2638d5043f packed-backend: check if header starts with \"# pack-refs with: \"\n 5: 7d7dc899ad ! 5: 13e34de350 packed-backend: add \"packed-refs\" header consistency check\n @@ refs/packed-backend.c: static struct ref_iterator *packed_reflog_iterator_begin(\n +\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tstruct stat st;\n \tint ret = 0;\n - \tint fd;\n + \tint fd = -1;\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n 6: 571479d3e7 = 6: 0632a1d5e2 packed-backend: check whether the refname contains NUL characters\n 7: e498a57286 = 7: 4618da3199 packed-backend: add \"packed-refs\" entry consistency check\n 8: 3638cb118d ! 8: 355e43d251 packed-backend: check whether the \"packed-refs\" is sorted\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n +\tunsigned int sorted = 0;\n \tstruct stat st;\n \tint ret = 0;\n - \tint fd;\n + \tint fd = -1;\n @@ refs/packed-backend.c: static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n 9: 5d87e76d28 = 9: 57dac06151 builtin/fsck: add `git refs verify` child process\n-- \n2.48.1\n\n"},{"id":"513155","messageId":"Z8CNY140eAG-xCt5@ArchLinux","threadId":"62743","inReplyTo":"Z8CMx7O19PMs9sVY@ArchLinux","subject":"[PATCH v8 1/9] t0602: use subshell to ensure working directory unchanged","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:05:55Z","receivedAt":"2025-02-27T16:05:47Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"For every test, we would execute the command \"cd repo\" in the first but\nwe never execute the command \"cd ..\" to restore the working directory.\nHowever, it's either not a good idea use above way. Because if any test\nfails between \"cd repo\" and \"cd ..\", the \"cd ..\" will never be reached.\nAnd we cannot correctly restore the working directory.\n\nLet's use subshell to ensure that the current working directory could be\nrestored to the correct path.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n t/t0602-reffiles-fsck.sh | 967 ++++++++++++++++++++-------------------\n 1 file changed, 494 insertions(+), 473 deletions(-)\n\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex d4a08b823b..cf7a202d0d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -14,222 +14,229 @@ test_expect_success 'ref name should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b default-branch &&\n-\tgit tag default-tag &&\n-\tgit tag multi_hierarchy/default-tag &&\n-\n-\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n-\trm $branch_dir_prefix/@ &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n-\tgit refs verify 2>err &&\n-\trm $tag_dir_prefix/tag-1.lock &&\n-\ttest_must_be_empty err &&\n-\n-\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/.lock: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t(\n+\t\tcd repo &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b default-branch &&\n+\t\tgit tag default-tag &&\n+\t\tgit tag multi_hierarchy/default-tag &&\n \n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n-\t\tEOF\n-\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n-\tdo\n-\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n-\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\t\trm $branch_dir_prefix/@ &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $tag_dir_prefix/tag-1.lock &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \t\ttest_must_fail git refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\terror: refs/tags/.lock: badRefName: invalid refname format\n \t\tEOF\n-\t\trm -r \"$branch_dir_prefix/$refname\" &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $tag_dir_prefix/.lock &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\t\tdo\n+\t\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=warn refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n-\ttest_must_be_empty err\n+\t(\n+\t\tcd repo &&\n+\t\tgit commit --allow-empty -m initial &&\n+\t\tgit checkout -b branch-1 &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=warn refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/.branch-1: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm $branch_dir_prefix/.branch-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n+\t\tgit -c fsck.badRefName=ignore refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n '\n \n test_expect_success 'ref name check should work for multiple worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\n-\tcd repo &&\n-\ttest_commit initial &&\n-\tgit checkout -b branch-1 &&\n-\ttest_commit second &&\n-\tgit checkout -b branch-2 &&\n-\ttest_commit third &&\n-\tgit checkout -b branch-3 &&\n-\tgit worktree add ./worktree-1 branch-1 &&\n-\tgit worktree add ./worktree-2 branch-2 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n-\t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n \t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n-\t) &&\n-\n-\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n-\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err &&\n-\n-\tfor worktree in \"worktree-1\" \"worktree-2\"\n-\tdo\n+\t\tcd repo &&\n+\t\ttest_commit initial &&\n+\t\tgit checkout -b branch-1 &&\n+\t\ttest_commit second &&\n+\t\tgit checkout -b branch-2 &&\n+\t\ttest_commit third &&\n+\t\tgit checkout -b branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-1 &&\n+\t\tgit worktree add ./worktree-2 branch-2 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n \t\t(\n-\t\t\tcd $worktree &&\n-\t\t\ttest_must_fail git refs verify 2>err &&\n-\t\t\tcat >expect <<-EOF &&\n-\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n-\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n-\t\t\tEOF\n-\t\t\tsort err >sorted_err &&\n-\t\t\ttest_cmp expect sorted_err || return 1\n-\t\t)\n-\tdone\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t\t) &&\n+\n+\t\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\t\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err &&\n+\n+\t\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\t\tdo\n+\t\t\t(\n+\t\t\t\tcd $worktree &&\n+\t\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\t\tcat >expect <<-EOF &&\n+\t\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\t\tEOF\n+\t\t\t\tsort err >sorted_err &&\n+\t\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t\t)\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n-\t\tEOF\n-\t\trm $branch_dir_prefix/a/b/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tfor trailing_content in \" garbage\" \" more garbage\"\n-\tdo\n-\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-garbage &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n+\t\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n-\t'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err &&\n \n-\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\t\t'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n \n \n-\t garbage'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-garbage-special &&\n-\ttest_cmp expect err\n+\t\t garbage'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage-special &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -237,99 +244,103 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tbad_content_1=$(git rev-parse main)x &&\n-\tbad_content_2=xfsazqfxcadas &&\n-\tbad_content_3=Xfsazqfxcadas &&\n-\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n-\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n-\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n-\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n-\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n-\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n-\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n-\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tbad_content_1=$(git rev-parse main)x &&\n+\t\tbad_content_2=xfsazqfxcadas &&\n+\t\tbad_content_3=Xfsazqfxcadas &&\n+\t\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\t\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\t\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\t\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\t\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\t\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (individual)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n \n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n \t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n \t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $branch_dir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\tEOF\n-\trm $branch_dir_prefix/a/b/branch-complicated &&\n-\ttest_cmp expect err\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success 'textual symref content should be checked (aggregate)' '\n@@ -337,32 +348,34 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n-\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n-\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n-\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n-\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n-\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n-\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n-\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n-\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n-\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n-\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n-\tEOF\n-\tsort err >sorted_err &&\n-\ttest_cmp expect sorted_err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\t\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\t\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\t\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\t\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\t\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\t\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\t\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\t\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\t\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\t\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n '\n \n test_expect_success 'the target of the textual symref should be checked' '\n@@ -370,28 +383,30 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n-\t\tgit refs verify 2>err &&\n-\t\trm $branch_dir_prefix/branch-good &&\n-\t\ttest_must_be_empty err || return 1\n-\tdone &&\n-\n-\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n-\tdo\n-\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n-\t\tgit refs verify 2>err &&\n-\t\tcat >expect <<-EOF &&\n-\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n-\t\tEOF\n-\t\trm $branch_dir_prefix/branch-bad-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-good &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone &&\n+\n+\t\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\t\tdo\n+\t\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\t\tgit refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked' '\n@@ -399,201 +414,207 @@ test_expect_success SYMLINKS 'symlink symref content should be checked' '\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n \ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n-\tEOF\n-\trm $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_cmp expect err\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\t\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-good &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\t\tEOF\n+\t\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tmain_worktree_refdir_prefix=.git/refs/heads &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\n-\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tfor bad_referent_name in \".tag\" \"branch \"\n-\tdo\n-\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tmain_worktree_refdir_prefix=.git/refs/heads &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n \t\ttest_cmp expect err &&\n \n-\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n-\t\ttest_cmp expect err || return 1\n-\tdone\n+\t\trm $worktree1_refdir_prefix/branch-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tfor bad_referent_name in \".tag\" \"branch \"\n+\t\tdo\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err &&\n+\n+\t\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n '\n \n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n-\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n-\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n-\n \t(\n-\t\tcd worktree-1 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n-\t) &&\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit worktree add ./worktree-1 branch-2 &&\n+\t\tgit worktree add ./worktree-2 branch-3 &&\n+\t\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\t\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\t(\n+\t\t\tcd worktree-1 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\t\t(\n+\t\t\tcd worktree-2 &&\n+\t\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t\t) &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\t\tdo\n+\t\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\t\tEOF\n+\t\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \t\tEOF\n-\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n+\t\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\t\ttest_cmp expect err &&\n \n-\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n-\tdo\n-\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_must_fail git refs verify 2>err &&\n+\t\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \t\tEOF\n-\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n-\t\ttest_cmp expect err || return 1\n-\tdone &&\n-\n-\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-no-newline &&\n-\ttest_cmp expect err &&\n-\n-\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n-\tEOF\n-\trm $worktree1_refdir_prefix/branch-garbage &&\n-\ttest_cmp expect err\n+\t\trm $worktree1_refdir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err\n+\t)\n '\n \n test_done\n-- \n2.48.1\n\n"},{"id":"513156","messageId":"Z8CNbjZ3igC2XK7k@ArchLinux","threadId":"62743","inReplyTo":"Z8CMx7O19PMs9sVY@ArchLinux","subject":"[PATCH v8 2/9] builtin/refs: get worktrees without reading head information","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:06:06Z","receivedAt":"2025-02-27T16:05:58Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c\", there are some functions such as \"create_snapshot\"\nand \"next_record\" which would check the correctness of the content of\nthe \"packed-ref\" file. When anything is bad, the program will die.\n\nIt may seem that we have nothing relevant to above feature, because we\nare going to read and parse the raw \"packed-ref\" file without creating\nthe snapshot and using the ref iterator to check the consistency.\n\nHowever, when using \"get_worktrees\" in \"builtin/refs\", we would parse\nthe \"HEAD\" information. If the referent of the \"HEAD\" is inside the\n\"packed-ref\", we will call \"create_snapshot\" function to parse the\n\"packed-ref\" to get the information. No matter whether the entry of\n\"HEAD\" in \"packed-ref\" is correct, \"create_snapshot\" would call\n\"verify_buffer_safe\" to check whether there is a newline in the last\nline of the file. If not, the program will die.\n\nAlthough this behavior has no harm for the program, it will\nshort-circuit the program. When the users execute \"git refs verify\" or\n\"git fsck\", we should avoid reading the head information, which may\nexecute the read operation in packed backend with stricter checks to die\nthe program. Instead, we should continue to check other parts of the\n\"packed-refs\" file completely.\n\nFortunately, in 465a22b338 (worktree: skip reading HEAD when repairing\nworktrees, 2023-12-29), we have introduced a function\n\"get_worktrees_internal\" which allows us to get worktrees without\nreading head information.\n\nCreate a new exposed function \"get_worktrees_without_reading_head\", then\nreplace the \"get_worktrees\" in \"builtin/refs\" with the new created\nfunction.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 2 +-\n worktree.c | 5 +++++\n worktree.h | 8 ++++++++\n 3 files changed, 14 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex a29f195834..55ff5dae11 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -88,7 +88,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix,\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tworktrees = get_worktrees();\n+\tworktrees = get_worktrees_without_reading_head();\n \tfor (size_t i = 0; worktrees[i]; i++)\n \t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n \t\t\t\t &fsck_refs_options, worktrees[i]);\ndiff --git a/worktree.c b/worktree.c\nindex d4a68c9c23..d23482a746 100644\n--- a/worktree.c\n+++ b/worktree.c\n@@ -198,6 +198,11 @@ struct worktree **get_worktrees(void)\n \treturn get_worktrees_internal(0);\n }\n \n+struct worktree **get_worktrees_without_reading_head(void)\n+{\n+\treturn get_worktrees_internal(1);\n+}\n+\n const char *get_worktree_git_dir(const struct worktree *wt)\n {\n \tif (!wt)\ndiff --git a/worktree.h b/worktree.h\nindex 38145df80f..a305c7e2c7 100644\n--- a/worktree.h\n+++ b/worktree.h\n@@ -30,6 +30,14 @@ struct worktree {\n */\n struct worktree **get_worktrees(void);\n \n+/*\n+ * Like `get_worktrees`, but does not read HEAD. Skip reading HEAD allows to\n+ * get the worktree without worrying about failures pertaining to parsing\n+ * the HEAD ref. This is useful in contexts where it is assumed that the\n+ * refdb may not be in a consistent state.\n+ */\n+struct worktree **get_worktrees_without_reading_head(void);\n+\n /*\n * Returns 1 if linked worktrees exist, 0 otherwise.\n */\n-- \n2.48.1\n\n"},{"id":"513157","messageId":"Z8CNgB_9UWWNhAzy@ArchLinux","threadId":"62743","inReplyTo":"Z8CMx7O19PMs9sVY@ArchLinux","subject":"[PATCH v8 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:06:24Z","receivedAt":"2025-02-27T16:06:15Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Although \"git-fsck(1)\" and \"packed-backend.c\" will check some\nconsistency and correctness of \"packed-refs\" file, they never check the\nfiletype of the \"packed-refs\". Let's verify that the \"packed-refs\" has\nthe expected filetype, confirming it is created by \"git pack-refs\"\ncommand.\n\nWe could use \"open_nofollow\" wrapper to open the raw \"packed-refs\" file.\nIf the returned \"fd\" value is less than 0, we could check whether the\n\"errno\" is \"ELOOP\" to report an error to the user. And then we use\n\"fstat\" to check whether the \"packed-refs\" file is a regular file.\n\nReuse \"FSCK_MSG_BAD_REF_FILETYPE\" fsck message id to report the error to\nthe user if \"packed-refs\" is not a regular file.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 52 ++++++++++++++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 30 +++++++++++++++++++++++\n 2 files changed, 78 insertions(+), 4 deletions(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex a7b6f74b6e..1fba804a2a 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -4,6 +4,7 @@\n #include \"../git-compat-util.h\"\n #include \"../config.h\"\n #include \"../dir.h\"\n+#include \"../fsck.h\"\n #include \"../gettext.h\"\n #include \"../hash.h\"\n #include \"../hex.h\"\n@@ -1748,15 +1749,58 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n-static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED,\n+static int packed_fsck(struct ref_store *ref_store,\n+\t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n+\tstruct packed_ref_store *refs = packed_downcast(ref_store,\n+\t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct stat st;\n+\tint ret = 0;\n+\tint fd = -1;\n \n \tif (!is_main_worktree(wt))\n-\t\treturn 0;\n+\t\tgoto cleanup;\n \n-\treturn 0;\n+\tif (o->verbose)\n+\t\tfprintf_ln(stderr, \"Checking packed-refs file %s\", refs->path);\n+\n+\tfd = open_nofollow(refs->path, O_RDONLY);\n+\tif (fd < 0) {\n+\t\t/*\n+\t\t * If the packed-refs file doesn't exist, there's nothing\n+\t\t * to check.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tif (errno == ELOOP) {\n+\t\t\tstruct fsck_ref_report report = { 0 };\n+\t\t\treport.path = \"packed-refs\";\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t\t \"not a regular file but a symlink\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tret = error_errno(_(\"unable to open '%s'\"), refs->path);\n+\t\tgoto cleanup;\n+\t} else if (fstat(fd, &st) < 0) {\n+\t\tret = error_errno(_(\"unable to stat '%s'\"), refs->path);\n+\t\tgoto cleanup;\n+\t} else if (!S_ISREG(st.st_mode)) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs\";\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_FILETYPE,\n+\t\t\t\t \"not a regular file\");\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tif (fd >= 0)\n+\t\tclose(fd);\n+\treturn ret;\n }\n \n struct ref_storage_be refs_be_packed = {\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex cf7a202d0d..68b7d4999e 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -617,4 +617,34 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t)\n '\n \n+test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit branch branch-3 &&\n+\t\tgit pack-refs --all &&\n+\n+\t\tmv .git/packed-refs .git/packed-refs-back &&\n+\t\tln -sf packed-refs-back .git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs: badRefFiletype: not a regular file but a symlink\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tmkdir .git/packed-refs &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs: badRefFiletype: not a regular file\n+\t\tEOF\n+\t\trm -r .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513158","messageId":"Z8CNkGJjKl0eTolM@ArchLinux","threadId":"62743","inReplyTo":"Z8CMx7O19PMs9sVY@ArchLinux","subject":"[PATCH v8 4/9] packed-backend: check if header starts with \"# pack-refs with: \"","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:06:40Z","receivedAt":"2025-02-27T16:06:32Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We always write a space after \"# pack-refs with:\" but we don't align\nwith this rule in the \"create_snapshot\" method where we would check\nwhether header starts with \"# pack-refs with:\". It might seem that we\nshould undoubtedly tighten this rule, however, we don't have any\ntechnical documentation about this and there is a possibility that we\nwould break the compatibility for other third-party libraries.\n\nBy investigating influential third-party libraries, we could conclude\nhow these libraries handle the header of \"packed-refs\" file:\n\n1. libgit2 is fine and always writes the space. It also expects the\n whitespace to exist.\n2. JGit does not expect th header to have a trailing space, but expects\n the \"peeled\" capability to have a leading space, which is mostly\n equivalent because that capability is typically the first one we\n write. It always writes the space.\n3. gitoxide expects the space t exist and writes it.\n4. go-git doesn't create the header by default.\n\nAs many third-party libraries expect a single space after \"# pack-refs\nwith:\", if we forget to write the space after the colon,\n\"create_snapshot\" won't catch this. And we would break other\nre-implementations. So, we'd better tighten the rule by checking whether\nthe header starts with \"# pack-refs with: \".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 1fba804a2a..eaa8746f3e 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -694,7 +694,7 @@ static struct snapshot *create_snapshot(struct packed_ref_store *refs)\n \n \t\ttmp = xmemdupz(snapshot->buf, eol - snapshot->buf);\n \n-\t\tif (!skip_prefix(tmp, \"# pack-refs with:\", (const char **)&p))\n+\t\tif (!skip_prefix(tmp, \"# pack-refs with: \", (const char **)&p))\n \t\t\tdie_invalid_line(refs->path,\n \t\t\t\t\t snapshot->buf,\n \t\t\t\t\t snapshot->eof - snapshot->buf);\n-- \n2.48.1\n\n"},{"id":"513159","messageId":"Z8CNmSAzQYpLw6ZK@ArchLinux","threadId":"62743","inReplyTo":"Z8CMx7O19PMs9sVY@ArchLinux","subject":"[PATCH v8 5/9] packed-backend: add \"packed-refs\" header consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:06:49Z","receivedAt":"2025-02-27T16:06:41Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"packed-backend.c::create_snapshot\", if there is a header (the line\nwhich starts with '#'), we will check whether the line starts with \"#\npack-refs with: \". However, we need to consider other situations and\ndiscuss whether we need to add checks.\n\n1. If the header does not exist, we should not report an error to the\n user. This is because in older Git version, we never write header in\n the \"packed-refs\" file. Also, we do allow no header in \"packed-refs\"\n in runtime.\n2. If the header content does not start with \"# packed-ref with: \", we\n should report an error just like what \"create_snapshot\" does. So,\n create a new fsck message \"badPackedRefHeader(ERROR)\" for this.\n3. If the header content is not the same as the constant string\n \"PACKED_REFS_HEADER\". This is expected because we make it extensible\n intentionally and runtime \"create_snapshot\" won't complain about\n unknown traits. In order to align with the runtime behavior. There is\n no need to report.\n\nAs we have analyzed, we only need to check the case 2 in the above. In\norder to do this, use \"open_nofollow\" function to get the file\ndescriptor and then read the \"packed-refs\" file via \"strbuf_read\". Like\nwhat \"create_snapshot\" and other functions do, we could split the line\nby finding the next newline in the buffer. When we cannot find a\nnewline, we could report an error.\n\nSo, create a function \"packed_fsck_ref_next_line\" to find the next\nnewline and if there is no such newline, use\n\"packedRefEntryNotTerminated(ERROR)\" to report an error to the user.\n\nThen, parse the first line to apply the checks. Update the test to\nexercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 8 ++++\n fsck.h | 2 +\n refs/packed-backend.c | 73 ++++++++++++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 52 ++++++++++++++++++++++++\n 4 files changed, 135 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex b14bc44ca4..11906f90fd 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -16,6 +16,10 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefHeader`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid\n+\theader.\n+\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n@@ -176,6 +180,10 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`packedRefEntryNotTerminated`::\n+\t(ERROR) The \"packed-refs\" file contains an entry that is\n+\tnot terminated by a newline.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex a44c231a5f..67e3c97bc0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n@@ -53,6 +54,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE, ERROR) \\\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex eaa8746f3e..07154bccae 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1749,12 +1749,76 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n \treturn empty_ref_iterator_begin();\n }\n \n+static int packed_fsck_ref_next_line(struct fsck_options *o,\n+\t\t\t\t unsigned long line_number, const char *start,\n+\t\t\t\t const char *eof, const char **eol)\n+{\n+\tint ret = 0;\n+\n+\t*eol = memchr(start, '\\n', eof - start);\n+\tif (!*eol) {\n+\t\tstruct strbuf packed_entry = STRBUF_INIT;\n+\t\tstruct fsck_ref_report report = { 0 };\n+\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_PACKED_REF_ENTRY_NOT_TERMINATED,\n+\t\t\t\t \"'%.*s' is not terminated with a newline\",\n+\t\t\t\t (int)(eof - start), start);\n+\n+\t\t/*\n+\t\t * There is no newline but we still want to parse it to the end of\n+\t\t * the buffer.\n+\t\t */\n+\t\t*eol = eof;\n+\t\tstrbuf_release(&packed_entry);\n+\t}\n+\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_header(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\t\tstruct fsck_ref_report report = { 0 };\n+\t\treport.path = \"packed-refs.header\";\n+\n+\t\treturn fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tunsigned long line_number = 1;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\tif (*start == '#') {\n+\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\treturn ret;\n+}\n+\n static int packed_fsck(struct ref_store *ref_store,\n \t\t struct fsck_options *o,\n \t\t struct worktree *wt)\n {\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n+\tstruct strbuf packed_ref_content = STRBUF_INIT;\n \tstruct stat st;\n \tint ret = 0;\n \tint fd = -1;\n@@ -1797,9 +1861,18 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (strbuf_read(&packed_ref_content, fd, 0) < 0) {\n+\t\tret = error_errno(_(\"unable to read '%s'\"), refs->path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\n cleanup:\n \tif (fd >= 0)\n \t\tclose(fd);\n+\tstrbuf_release(&packed_ref_content);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 68b7d4999e..74d876984d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -647,4 +647,56 @@ test_expect_success SYMLINKS 'the filetype of packed-refs should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-refs header should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tfor bad_header in \"# pack-refs wit: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with traits: peeled fully-peeled sorted \" \\\n+\t\t\t\t \"# pack-refs with a: peeled fully-peeled\" \\\n+\t\t\t\t \"# pack-refs with:peeled fully-peeled sorted\"\n+\t\tdo\n+\t\t\tprintf \"%s\\n\" \"$bad_header\" >.git/packed-refs &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: packed-refs.header: badPackedRefHeader: '\\''$bad_header'\\'' does not start with '\\''# pack-refs with: '\\''\n+\t\t\tEOF\n+\t\t\trm .git/packed-refs &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone\n+\t)\n+'\n+\n+test_expect_success 'packed-refs missing header should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"$(git rev-parse HEAD) refs/heads/main\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n+test_expect_success 'packed-refs unknown traits should not be reported' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\n+\t\tprintf \"# pack-refs with: peeled fully-peeled sorted foo\\n\" >.git/packed-refs &&\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513160","messageId":"Z8CNpE0FdCLwaSpU@ArchLinux","threadId":"62743","inReplyTo":"Z8CMx7O19PMs9sVY@ArchLinux","subject":"[PATCH v8 6/9] packed-backend: check whether the refname contains NUL characters","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:07:00Z","receivedAt":"2025-02-27T16:06:51Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will use \"check_refname_format\" to check\nthe consistency of the refname. If it is not OK, the program will die.\nHowever, it is reported in [1], we cannot catch some corruption. But we\nalready have the code path and we must miss out something.\n\nWe use the following code to get the refname:\n\n strbuf_add(&iter->refname_buf, p, eol - p);\n iter->base.refname = iter->refname_buf.buf\n\nIn the above code, `p` is the start pointer of the refname and `eol` is\nthe next newline pointer. We calculate the length of the refname by\nsubtracting the two pointers. Then we add the memory range between `p`\nand `eol` to get the refname.\n\nHowever, if there are some NUL characters in the memory range between `p`\nand `eol`, we will see the refname as a valid ref name as long as the\nmemory range between `p` and first occurred NUL character is valid.\n\nIn order to catch above corruption, create a new function\n\"refname_contains_nul\" by searching the first NUL character. If it is\nnot at the end of the string, there must be some NUL characters in the\nrefname.\n\nUse this function in \"next_record\" function to die the program if\n\"refname_contains_nul\" returns true.\n\n[1] https://lore.kernel.org/git/6cfee0e4-3285-4f18-91ff-d097da9de737@rd10.de/\n\nReported-by: R. Diez \nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/packed-backend.c | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 07154bccae..9a90c52f70 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -494,6 +494,21 @@ static void verify_buffer_safe(struct snapshot *snapshot)\n \t\t\t\t last_line, eof - last_line);\n }\n \n+/*\n+ * When parsing the \"packed-refs\" file, we will parse it line by line.\n+ * Because we know the start pointer of the refname and the next\n+ * newline pointer, we could calculate the length of the refname by\n+ * subtracting the two pointers. However, there is a corner case where\n+ * the refname contains corrupted embedded NUL characters. And\n+ * `check_refname_format()` will not catch this when the truncated\n+ * refname is still a valid refname. To prevent this, we need to check\n+ * whether the refname contains the NUL characters.\n+ */\n+static int refname_contains_nul(struct strbuf *refname)\n+{\n+\treturn !!memchr(refname->buf, '\\0', refname->len);\n+}\n+\n #define SMALL_FILE_SIZE (32*1024)\n \n /*\n@@ -895,6 +910,9 @@ static int next_record(struct packed_ref_iterator *iter)\n \tstrbuf_add(&iter->refname_buf, p, eol - p);\n \titer->base.refname = iter->refname_buf.buf;\n \n+\tif (refname_contains_nul(&iter->refname_buf))\n+\t\tdie(\"packed refname contains embedded NULL: %s\", iter->base.refname);\n+\n \tif (check_refname_format(iter->base.refname, REFNAME_ALLOW_ONELEVEL)) {\n \t\tif (!refname_is_safe(iter->base.refname))\n \t\t\tdie(\"packed refname is dangerous: %s\",\n-- \n2.48.1\n\n"},{"id":"513161","messageId":"Z8CNtcTbJrCbLXeT@ArchLinux","threadId":"62743","inReplyTo":"Z8CMx7O19PMs9sVY@ArchLinux","subject":"[PATCH v8 7/9] packed-backend: add \"packed-refs\" entry consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:07:17Z","receivedAt":"2025-02-27T16:07:09Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"packed-backend.c::next_record\" will parse the ref entry to check the\nconsistency. This function has already checked the following things:\n\n1. Parse the main line of the ref entry to inspect whether the oid is\n not correct. Then, check whether the next character is oid. Then\n check the refname.\n2. If the next line starts with '^', it would continue to parse the\n peeled oid and check whether the last character is '\\n'.\n\nAs we decide to implement the ref consistency check for \"packed-refs\",\nlet's port these two checks and update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 122 ++++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 44 ++++++++++++\n 4 files changed, 169 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 11906f90fd..02a7bf0503 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -16,6 +16,9 @@\n `badObjectSha1`::\n \t(ERROR) An object has a bad sha1.\n \n+`badPackedRefEntry`::\n+\t(ERROR) The \"packed-refs\" file contains an invalid entry.\n+\n `badPackedRefHeader`::\n \t(ERROR) The \"packed-refs\" file contains an invalid\n \theader.\ndiff --git a/fsck.h b/fsck.h\nindex 67e3c97bc0..14d70f6653 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -30,6 +30,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_EMAIL, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\n \tFUNC(BAD_PACKED_REF_HEADER, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 9a90c52f70..ef20300fd3 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -1812,9 +1812,114 @@ static int packed_fsck_ref_header(struct fsck_options *o,\n \treturn 0;\n }\n \n+static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id peeled;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\t/*\n+\t * Skip the '^' and parse the peeled oid.\n+\t */\n+\tstart++;\n+\tif (parse_oid_hex_algop(start, &peeled, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid peeled oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p != eol) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has trailing garbage after peeled oid '%.*s'\",\n+\t\t\t\t (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n+static int packed_fsck_ref_main_line(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned long line_number,\n+\t\t\t\t struct strbuf *refname,\n+\t\t\t\t const char *start, const char *eol)\n+{\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct object_id oid;\n+\tconst char *p;\n+\tint ret = 0;\n+\n+\tif (parse_oid_hex_algop(start, &oid, &p, ref_store->repo->hash_algo)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"'%.*s' has invalid oid\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (p == eol || !isspace(*p)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"has no space after oid '%s' but with '%.*s'\",\n+\t\t\t\t oid_to_hex(&oid), (int)(eol - p), p);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tp++;\n+\tstrbuf_reset(refname);\n+\tstrbuf_add(refname, p, eol - p);\n+\tif (refname_contains_nul(refname)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_ENTRY,\n+\t\t\t\t \"refname '%s' contains NULL binaries\",\n+\t\t\t\t refname->buf);\n+\t}\n+\n+\tif (check_refname_format(refname->buf, 0)) {\n+\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\treport.path = packed_entry.buf;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_NAME,\n+\t\t\t\t \"has bad refname '%s'\", refname->buf);\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n \t\t\t\t const char *start, const char *eof)\n {\n+\tstruct strbuf refname = STRBUF_INIT;\n \tunsigned long line_number = 1;\n \tconst char *eol;\n \tint ret = 0;\n@@ -1827,6 +1932,21 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \t\tline_number++;\n \t}\n \n+\twhile (start < eof) {\n+\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\tret |= packed_fsck_ref_main_line(o, ref_store, line_number, &refname, start, eol);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t\tif (start < eof && *start == '^') {\n+\t\t\tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n+\t\t\tret |= packed_fsck_ref_peeled_line(o, ref_store, line_number,\n+\t\t\t\t\t\t\t start, eol);\n+\t\t\tstart = eol + 1;\n+\t\t\tline_number++;\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&refname);\n \treturn ret;\n }\n \n@@ -1884,7 +2004,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 74d876984d..a88c792ce1 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -699,4 +699,48 @@ test_expect_success 'packed-refs unknown traits should not be reported' '\n \t)\n '\n \n+test_expect_success 'packed-refs content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tgit tag -a annotated-tag-2 -m tag-2 &&\n+\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_2_oid=$(git rev-parse annotated-tag-2) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\ttag_2_peeled_oid=$(git rev-parse annotated-tag-2^{}) &&\n+\t\tshort_oid=$(printf \"%s\" $tag_1_peeled_oid | cut -c 1-4) &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$short_oid refs/heads/branch-1\n+\t\t${branch_1_oid}x\n+\t\t$branch_2_oid refs/heads/bad-branch\n+\t\t$branch_2_oid refs/heads/branch.\n+\t\t$tag_1_oid refs/tags/annotated-tag-3\n+\t\t^$short_oid\n+\t\t$tag_2_oid refs/tags/annotated-tag-4.\n+\t\t^$tag_2_peeled_oid garbage\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 2: badPackedRefEntry: '\\''$short_oid refs/heads/branch-1'\\'' has invalid oid\n+\t\terror: packed-refs line 3: badPackedRefEntry: has no space after oid '\\''$branch_1_oid'\\'' but with '\\''x'\\''\n+\t\terror: packed-refs line 4: badRefName: has bad refname '\\'' refs/heads/bad-branch'\\''\n+\t\terror: packed-refs line 5: badRefName: has bad refname '\\''refs/heads/branch.'\\''\n+\t\terror: packed-refs line 7: badPackedRefEntry: '\\''$short_oid'\\'' has invalid peeled oid\n+\t\terror: packed-refs line 8: badRefName: has bad refname '\\''refs/tags/annotated-tag-4.'\\''\n+\t\terror: packed-refs line 9: badPackedRefEntry: has trailing garbage after peeled oid '\\'' garbage'\\''\n+\t\tEOF\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513162","messageId":"Z8CNzNlWdBN0t0ox@ArchLinux","threadId":"62743","inReplyTo":"Z8CMx7O19PMs9sVY@ArchLinux","subject":"[PATCH v8 8/9] packed-backend: check whether the \"packed-refs\" is sorted","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:07:40Z","receivedAt":"2025-02-27T16:07:32Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"When there is a \"sorted\" trait in the header of the \"packed-refs\" file,\nit means that each entry is sorted increasingly by comparing the\nrefname. We should add checks to verify whether the \"packed-refs\" is\nsorted in this case.\n\nUpdate the \"packed_fsck_ref_header\" to know whether there is a \"sorted\"\ntrail in the header. It may seem that we could record all refnames\nduring the parsing process and then compare later. However, this is not\na good design due to the following reasons:\n\n1. Because we need to store the state across the whole checking\n lifetime, we would consume a lot of memory if there are many entries\n in the \"packed-refs\" file.\n2. We cannot reuse the existing compare function \"cmp_packed_ref_records\"\n which cause repetition.\n\nBecause \"cmp_packed_ref_records\" needs an extra parameter \"struct\nsnaphost\", extract the common part into a new function\n\"cmp_packed_ref_records\" to reuse this function to compare.\n\nThen, create a new function \"packed_fsck_ref_sorted\" to parse the file\nagain and user the new fsck message \"packedRefUnsorted(ERROR)\" to report\nto the user if the file is not sorted.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.adoc | 3 +\n fsck.h | 1 +\n refs/packed-backend.c | 116 ++++++++++++++++++++++++++++-----\n t/t0602-reffiles-fsck.sh | 87 +++++++++++++++++++++++++\n 4 files changed, 191 insertions(+), 16 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 02a7bf0503..9601fff228 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -187,6 +187,9 @@\n \t(ERROR) The \"packed-refs\" file contains an entry that is\n \tnot terminated by a newline.\n \n+`packedRefUnsorted`::\n+\t(ERROR) The \"packed-refs\" file is not sorted.\n+\n `refMissingNewline`::\n \t(INFO) A loose ref that does not end with newline(LF). As\n \tvalid implementations of Git never created such a loose ref\ndiff --git a/fsck.h b/fsck.h\nindex 14d70f6653..19f3cb2773 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -56,6 +56,7 @@ enum fsck_msg_type {\n \tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n \tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n \tFUNC(PACKED_REF_ENTRY_NOT_TERMINATED, ERROR) \\\n+\tFUNC(PACKED_REF_UNSORTED, ERROR) \\\n \tFUNC(TREE_NOT_SORTED, ERROR) \\\n \tFUNC(UNKNOWN_TYPE, ERROR) \\\n \tFUNC(ZERO_PADDED_DATE, ERROR) \\\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex ef20300fd3..813e5020e4 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -300,14 +300,9 @@ struct snapshot_record {\n \tsize_t len;\n };\n \n-static int cmp_packed_ref_records(const void *v1, const void *v2,\n-\t\t\t\t void *cb_data)\n-{\n-\tconst struct snapshot *snapshot = cb_data;\n-\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n-\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n-\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n \n+static int cmp_packed_refname(const char *r1, const char *r2)\n+{\n \twhile (1) {\n \t\tif (*r1 == '\\n')\n \t\t\treturn *r2 == '\\n' ? 0 : -1;\n@@ -322,6 +317,17 @@ static int cmp_packed_ref_records(const void *v1, const void *v2,\n \t}\n }\n \n+static int cmp_packed_ref_records(const void *v1, const void *v2,\n+\t\t\t\t void *cb_data)\n+{\n+\tconst struct snapshot *snapshot = cb_data;\n+\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n+\tconst char *r1 = e1->start + snapshot_hexsz(snapshot) + 1;\n+\tconst char *r2 = e2->start + snapshot_hexsz(snapshot) + 1;\n+\n+\treturn cmp_packed_refname(r1, r2);\n+}\n+\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n@@ -1797,19 +1803,33 @@ static int packed_fsck_ref_next_line(struct fsck_options *o,\n }\n \n static int packed_fsck_ref_header(struct fsck_options *o,\n-\t\t\t\t const char *start, const char *eol)\n+\t\t\t\t const char *start, const char *eol,\n+\t\t\t\t unsigned int *sorted)\n {\n-\tif (!starts_with(start, \"# pack-refs with: \")) {\n+\tstruct string_list traits = STRING_LIST_INIT_NODUP;\n+\tchar *tmp_line;\n+\tint ret = 0;\n+\tchar *p;\n+\n+\ttmp_line = xmemdupz(start, eol - start);\n+\tif (!skip_prefix(tmp_line, \"# pack-refs with: \", (const char **)&p)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \t\treport.path = \"packed-refs.header\";\n \n-\t\treturn fsck_report_ref(o, &report,\n-\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n-\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n-\t\t\t\t (int)(eol - start), start);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_PACKED_REF_HEADER,\n+\t\t\t\t \"'%.*s' does not start with '# pack-refs with: '\",\n+\t\t\t\t (int)(eol - start), start);\n+\t\tgoto cleanup;\n \t}\n \n-\treturn 0;\n+\tstring_list_split_in_place(&traits, p, \" \", -1);\n+\t*sorted = unsorted_string_list_has_string(&traits, \"sorted\");\n+\n+cleanup:\n+\tfree(tmp_line);\n+\tstring_list_clear(&traits, 0);\n+\treturn ret;\n }\n \n static int packed_fsck_ref_peeled_line(struct fsck_options *o,\n@@ -1915,8 +1935,68 @@ static int packed_fsck_ref_main_line(struct fsck_options *o,\n \treturn ret;\n }\n \n+static int packed_fsck_ref_sorted(struct fsck_options *o,\n+\t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t const char *start, const char *eof)\n+{\n+\tsize_t hexsz = ref_store->repo->hash_algo->hexsz;\n+\tstruct strbuf packed_entry = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tstruct strbuf refname1 = STRBUF_INIT;\n+\tstruct strbuf refname2 = STRBUF_INIT;\n+\tunsigned long line_number = 1;\n+\tconst char *former = NULL;\n+\tconst char *current;\n+\tconst char *eol;\n+\tint ret = 0;\n+\n+\tif (*start == '#') {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\t\tstart = eol + 1;\n+\t\tline_number++;\n+\t}\n+\n+\tfor (; start < eof; line_number++, start = eol + 1) {\n+\t\teol = memchr(start, '\\n', eof - start);\n+\n+\t\tif (*start == '^')\n+\t\t\tcontinue;\n+\n+\t\tif (!former) {\n+\t\t\tformer = start + hexsz + 1;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tcurrent = start + hexsz + 1;\n+\t\tif (cmp_packed_refname(former, current) >= 0) {\n+\t\t\tconst char *err_fmt =\n+\t\t\t\t\"refname '%s' is less than previous refname '%s'\";\n+\n+\t\t\teol = memchr(former, '\\n', eof - former);\n+\t\t\tstrbuf_add(&refname1, former, eol - former);\n+\t\t\teol = memchr(current, '\\n', eof - current);\n+\t\t\tstrbuf_add(&refname2, current, eol - current);\n+\n+\t\t\tstrbuf_addf(&packed_entry, \"packed-refs line %lu\", line_number);\n+\t\t\treport.path = packed_entry.buf;\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_PACKED_REF_UNSORTED,\n+\t\t\t\t\t err_fmt, refname2.buf, refname1.buf);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tformer = current;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&packed_entry);\n+\tstrbuf_release(&refname1);\n+\tstrbuf_release(&refname2);\n+\treturn ret;\n+}\n+\n static int packed_fsck_ref_content(struct fsck_options *o,\n \t\t\t\t struct ref_store *ref_store,\n+\t\t\t\t unsigned int *sorted,\n \t\t\t\t const char *start, const char *eof)\n {\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -1926,7 +2006,7 @@ static int packed_fsck_ref_content(struct fsck_options *o,\n \n \tret |= packed_fsck_ref_next_line(o, line_number, start, eof, &eol);\n \tif (*start == '#') {\n-\t\tret |= packed_fsck_ref_header(o, start, eol);\n+\t\tret |= packed_fsck_ref_header(o, start, eol, sorted);\n \n \t\tstart = eol + 1;\n \t\tline_number++;\n@@ -1957,6 +2037,7 @@ static int packed_fsck(struct ref_store *ref_store,\n \tstruct packed_ref_store *refs = packed_downcast(ref_store,\n \t\t\t\t\t\t\tREF_STORE_READ, \"fsck\");\n \tstruct strbuf packed_ref_content = STRBUF_INIT;\n+\tunsigned int sorted = 0;\n \tstruct stat st;\n \tint ret = 0;\n \tint fd = -1;\n@@ -2004,8 +2085,11 @@ static int packed_fsck(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n-\tret = packed_fsck_ref_content(o, ref_store, packed_ref_content.buf,\n+\tret = packed_fsck_ref_content(o, ref_store, &sorted, packed_ref_content.buf,\n \t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n+\tif (!ret && sorted)\n+\t\tret = packed_fsck_ref_sorted(o, ref_store, packed_ref_content.buf,\n+\t\t\t\t\t packed_ref_content.buf + packed_ref_content.len);\n \n cleanup:\n \tif (fd >= 0)\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex a88c792ce1..767e2bd4a0 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -743,4 +743,91 @@ test_expect_success 'packed-refs content should be checked' '\n \t)\n '\n \n+test_expect_success 'packed-ref with sorted trait should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\trm .git/packed-refs &&\n+\t\ttest_must_be_empty err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\t$tag_1_oid $refname3\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 3: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname1'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled sorted\n+\t\t$tag_1_oid $refname3\n+\t\t^$tag_1_peeled_oid\n+\t\t$branch_2_oid $refname2\n+\t\tEOF\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: packed-refs line 4: packedRefUnsorted: refname '\\''$refname2'\\'' is less than previous refname '\\''$refname3'\\''\n+\t\tEOF\n+\t\trm .git/packed-refs &&\n+\t\ttest_cmp expect err\n+\t)\n+'\n+\n+test_expect_success 'packed-ref without sorted trait should not be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tgit branch branch-1 &&\n+\t\tgit branch branch-2 &&\n+\t\tgit tag -a annotated-tag-1 -m tag-1 &&\n+\t\tbranch_1_oid=$(git rev-parse branch-1) &&\n+\t\tbranch_2_oid=$(git rev-parse branch-2) &&\n+\t\ttag_1_oid=$(git rev-parse annotated-tag-1) &&\n+\t\ttag_1_peeled_oid=$(git rev-parse annotated-tag-1^{}) &&\n+\t\trefname1=\"refs/heads/main\" &&\n+\t\trefname2=\"refs/heads/foo\" &&\n+\t\trefname3=\"refs/tags/foo\" &&\n+\n+\t\tcat >.git/packed-refs <<-EOF &&\n+\t\t# pack-refs with: peeled fully-peeled\n+\t\t$branch_2_oid $refname1\n+\t\t$branch_1_oid $refname2\n+\t\tEOF\n+\t\tgit refs verify 2>err &&\n+\t\ttest_must_be_empty err\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513163","messageId":"Z8CN1K51oJliGCO9@ArchLinux","threadId":"62743","inReplyTo":"Z8CMx7O19PMs9sVY@ArchLinux","subject":"[PATCH v8 9/9] builtin/fsck: add `git refs verify` child process","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-27T16:07:48Z","receivedAt":"2025-02-27T16:07:41Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"At now, we have already implemented the ref consistency checks for both\n\"files-backend\" and \"packed-backend\". Although we would check some\nredundant things, it won't cause trouble. So, let's integrate it into\nthe \"git-fsck(1)\" command to get feedback from the users. And also by\ncalling \"git refs verify\" in \"git-fsck(1)\", we make sure that the new\nadded checks don't break.\n\nIntroduce a new function \"fsck_refs\" that initializes and runs a child\nprocess to execute the \"git refs verify\" command. In order to provide\nthe user interface create a progress which makes the total task be 1.\nIt's hard to know how many loose refs we will check now. We might\nimprove this later.\n\nThen, introduce the option to allow the user to disable checking ref\ndatabase consistency. Put this function in the very first execution\nsequence of \"git-fsck(1)\" due to that we don't want the existing code of\n\"git-fsck(1)\" which would implicitly check the consistency of refs to\ndie the program.\n\nLast, update the test to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/git-fsck.adoc | 7 ++++++-\n builtin/fsck.c | 33 ++++++++++++++++++++++++++++++-\n t/t0602-reffiles-fsck.sh | 39 +++++++++++++++++++++++++++++++++++++\n 3 files changed, 77 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-fsck.adoc b/Documentation/git-fsck.adoc\nindex 8f32800a83..11203ba925 100644\n--- a/Documentation/git-fsck.adoc\n+++ b/Documentation/git-fsck.adoc\n@@ -12,7 +12,7 @@ SYNOPSIS\n 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n \t [--[no-]full] [--strict] [--verbose] [--lost-found]\n \t [--[no-]dangling] [--[no-]progress] [--connectivity-only]\n-\t [--[no-]name-objects] [...]\n+\t [--[no-]name-objects] [--[no-]references] [...]\n \n DESCRIPTION\n -----------\n@@ -104,6 +104,11 @@ care about this output and want to speed it up further.\n \tprogress status even if the standard error stream is not\n \tdirected to a terminal.\n \n+--[no-]references::\n+\tControl whether to check the references database consistency\n+\tvia 'git refs verify'. See linkgit:git-refs[1] for details.\n+\tThe default is to check the references database.\n+\n CONFIGURATION\n -------------\n \ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 7a4dcb0716..f4f395cfbd 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -50,6 +50,7 @@ static int verbose;\n static int show_progress = -1;\n static int show_dangling = 1;\n static int name_objects;\n+static int check_references = 1;\n #define ERROR_OBJECT 01\n #define ERROR_REACHABLE 02\n #define ERROR_PACK 04\n@@ -905,11 +906,37 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress)\n \treturn res;\n }\n \n+static void fsck_refs(struct repository *r)\n+{\n+\tstruct child_process refs_verify = CHILD_PROCESS_INIT;\n+\tstruct progress *progress = NULL;\n+\n+\tif (show_progress)\n+\t\tprogress = start_progress(r, _(\"Checking ref database\"), 1);\n+\n+\tif (verbose)\n+\t\tfprintf_ln(stderr, _(\"Checking ref database\"));\n+\n+\tchild_process_init(&refs_verify);\n+\trefs_verify.git_cmd = 1;\n+\tstrvec_pushl(&refs_verify.args, \"refs\", \"verify\", NULL);\n+\tif (verbose)\n+\t\tstrvec_push(&refs_verify.args, \"--verbose\");\n+\tif (check_strict)\n+\t\tstrvec_push(&refs_verify.args, \"--strict\");\n+\n+\tif (run_command(&refs_verify))\n+\t\terrors_found |= ERROR_REFS;\n+\n+\tdisplay_progress(progress, 1);\n+\tstop_progress(&progress);\n+}\n+\n static char const * const fsck_usage[] = {\n \tN_(\"git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\\n\"\n \t \" [--[no-]full] [--strict] [--verbose] [--lost-found]\\n\"\n \t \" [--[no-]dangling] [--[no-]progress] [--connectivity-only]\\n\"\n-\t \" [--[no-]name-objects] [...]\"),\n+\t \" [--[no-]name-objects] [--[no-]references] [...]\"),\n \tNULL\n };\n \n@@ -928,6 +955,7 @@ static struct option fsck_opts[] = {\n \t\t\t\tN_(\"write dangling objects in .git/lost-found\")),\n \tOPT_BOOL(0, \"progress\", &show_progress, N_(\"show progress\")),\n \tOPT_BOOL(0, \"name-objects\", &name_objects, N_(\"show verbose names for reachable objects\")),\n+\tOPT_BOOL(0, \"references\", &check_references, N_(\"check reference database consistency\")),\n \tOPT_END(),\n };\n \n@@ -970,6 +998,9 @@ int cmd_fsck(int argc,\n \tgit_config(git_fsck_config, &fsck_obj_options);\n \tprepare_repo_settings(the_repository);\n \n+\tif (check_references)\n+\t\tfsck_refs(the_repository);\n+\n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\n \t\tfor_each_packed_object(the_repository,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 767e2bd4a0..9d1dc2144c 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -830,4 +830,43 @@ test_expect_success 'packed-ref without sorted trait should not be checked' '\n \t)\n '\n \n+test_expect_success '--[no-]references option should apply to fsck' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\t(\n+\t\tcd repo &&\n+\t\ttest_commit default &&\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --references 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\t\tEOF\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_cmp expect err || return 1\n+\t\tdone &&\n+\n+\t\tfor trailing_content in \" garbage\" \" more garbage\"\n+\t\tdo\n+\t\t\tprintf \"%s\" \"$(git rev-parse HEAD)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\t\tgit fsck --no-references 2>err &&\n+\t\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\t\ttest_must_be_empty err || return 1\n+\t\tdone\n+\t)\n+'\n+\n test_done\n-- \n2.48.1\n\n"},{"id":"513166","messageId":"xmqqeczj9vh4.fsf@gitster.g","threadId":"62743","inReplyTo":"Z7-4XRCVvLjFCFR8@ArchLinux","subject":"Re: [PATCH v7 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-27T16:57:11Z","receivedAt":"2025-02-27T16:57:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> You are right. Actually, I just want to avoid assigning the `fd` to -1.\n\nWhy not?\n\nBetween leaving it uninitialized and explicitly initializing it to\nsignal that it is invalid, the only difference is that you can\nprogrammatically check if fd is invalid and refrain from calling\nclose(fd), for example, with the latter, while with the former you\ncannot.\n\n> However, I didn't realize that I would initialize the strbuf later.\n> After waking up, I have suddenly realized this problem.\n\nGiven that initialized-but-never-used strbuf does not hold any\nacquired resources, the current code at the end of the series is\nstill OK. So there is technically nothing to fix. I'll take a\nreroll if you later send one, but as I said, I do not think it is\nnecessary to reroll only to add fd=-1 initialization.\n"},{"id":"513223","messageId":"Z8FDX9-BbAt4H1RV@ArchLinux","threadId":"62743","inReplyTo":"xmqqeczj9vh4.fsf@gitster.g","subject":"Re: [PATCH v7 3/9] packed-backend: check whether the \"packed-refs\" is regular file","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2025-02-28T05:02:23Z","receivedAt":"2025-02-28T05:02:13Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Feb 27, 2025 at 08:57:11AM -0800, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > You are right. Actually, I just want to avoid assigning the `fd` to -1.\n> \n> Why not?\n> \n> Between leaving it uninitialized and explicitly initializing it to\n> signal that it is invalid, the only difference is that you can\n> programmatically check if fd is invalid and refrain from calling\n> close(fd), for example, with the latter, while with the former you\n> cannot.\n> \n\nYes, that's correct.\n\n> > However, I didn't realize that I would initialize the strbuf later.\n> > After waking up, I have suddenly realized this problem.\n> \n> Given that initialized-but-never-used strbuf does not hold any\n> acquired resources, the current code at the end of the series is\n> still OK. So there is technically nothing to fix. I'll take a\n> reroll if you later send one, but as I said, I do not think it is\n> necessary to reroll only to add fd=-1 initialization.\n\nYes, as you have said, there is nothing wrong at now. And as Patrick has\nnothing comment. I have sent out a reroll to make code better.\n\nThanks,\nJialuo\n"}]}