{"thread":{"id":"61943","subject":"[RFC] Implement ref content consistency check","startedAt":"2024-08-13T14:18:21Z","lastAt":"2024-11-20T23:21:04Z","messageCount":209,"participants":["shejialuo","karthik nayak","Patrick Steinhardt","Junio C Hamano","Jeff King","Karthik Nayak","Taylor Blau"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"500784","messageId":"ZrtrT1CPI4YUf5db@ArchLinux","threadId":"61943","inReplyTo":null,"subject":"[RFC] Implement ref content consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-13T14:18:55Z","receivedAt":"2024-08-13T14:18:21Z","isPatch":false,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nWe have already set up the infrastructure of the ref consistency.\nHowever, we have only add ref name check when establishing the\ninfrastructure in below:\n\n https://lore.kernel.org/git/ZrSqMmD-quQ18a9F@ArchLinux.localdomain/\n\nActually, we already have a patch here which has already implemented the\nref content consistency check. But during the review process, we have\nencountered some problems. The intention of this RFC is to make sure\nwhat content we should check and also to what extend.\n\nI conclude the following info:\n\n1. For the regular ref which has a trailing garbage, we should warn the\nuser. This is the most simplest situation, we could reply on\n\"parse_loose_ref_content\" to do this.\n2. For the symref, we could also rely on \"parse_loose_ref_content\" to\nget the \"pointee\", and check the location of the \"pointee\", check the\nname of the \"pointee\" and the file type of the \"pointee\".\n3. FOr the symbolic ref, we could follow the idea of 2.\n\nBut Patrick gives a question here:\n\n> In case the ref ends with a newline, should we check that the next\n> character is `\\0`? Otherwise, it may contain multiple lines, which is\n> not allowed for a normal ref.\n>\n> Also, shouldn't the ref always end with a newline?\n\nFor symref, I guess we have no spec here. From my experiments, a symref\ncould have a newline or no newline, even multiple newlines. And also\nsymref could have multiple spaces. But the following is a bad symref\n\n ref: refs/heads/main garbage\n\nI think we should fully discuss what we should check here. Thus I will\nimplement the code.\n\nThanks,\nJialuo\n"},{"id":"500968","messageId":"CAOLa=ZQVkmyVWAxyjrEQoEJ+gKJoJjfFqsDvr_A15FHGX1w=rQ@mail.gmail.com","threadId":"61943","inReplyTo":"ZrtrT1CPI4YUf5db@ArchLinux","subject":"Re: [RFC] Implement ref content consistency check","fromName":"karthik nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-08-15T10:19:50Z","receivedAt":"2024-08-15T10:19:52Z","isPatch":false,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> Hi All:\n>\n> We have already set up the infrastructure of the ref consistency.\n> However, we have only add ref name check when establishing the\n> infrastructure in below:\n>\n> https://lore.kernel.org/git/ZrSqMmD-quQ18a9F@ArchLinux.localdomain/\n>\n> Actually, we already have a patch here which has already implemented the\n> ref content consistency check. But during the review process, we have\n> encountered some problems. The intention of this RFC is to make sure\n> what content we should check and also to what extend.\n>\n> I conclude the following info:\n>\n> 1. For the regular ref which has a trailing garbage, we should warn the\n> user. This is the most simplest situation, we could reply on\n> \"parse_loose_ref_content\" to do this.\n> 2. For the symref, we could also rely on \"parse_loose_ref_content\" to\n> get the \"pointee\", and check the location of the \"pointee\", check the\n> name of the \"pointee\" and the file type of the \"pointee\".\n> 3. FOr the symbolic ref, we could follow the idea of 2.\n>\n\nJust to understand clearly, when you're talking about 'symbolic ref' you\nare referring to symbolic links?\n\nI ask because, as per our documentation in\n'Documentation/git-symbolic-ref.txt':\n\n In the past, `.git/HEAD` was a symbolic link pointing at\n `refs/heads/master`. When we wanted to switch to another branch, we\n did `ln -sf refs/heads/newbranch .git/HEAD`, and when we wanted to\n find out which branch we are on, we did `readlink .git/HEAD`. But\n symbolic links are not entirely portable, so they are now deprecated\n and symbolic refs (as described above) are used by default.\n\n> But Patrick gives a question here:\n>\n>> In case the ref ends with a newline, should we check that the next\n>> character is `\\0`? Otherwise, it may contain multiple lines, which is\n>> not allowed for a normal ref.\n>>\n>> Also, shouldn't the ref always end with a newline?\n>\n> For symref, I guess we have no spec here. From my experiments, a symref\n> could have a newline or no newline, even multiple newlines. And also\n> symref could have multiple spaces. But the following is a bad symref\n>\n> ref: refs/heads/main garbage\n>\n> I think we should fully discuss what we should check here. Thus I will\n> implement the code.\n>\n\nAgreed, in refs/files-backend.c:create_symref_lock, we write symrefs as\n\"ref: %s\\n\" so it makes sense to validate that there is nothing extra.\n"},{"id":"501004","messageId":"Zr4EqESHSnQET1Xg@ArchLinux","threadId":"61943","inReplyTo":"CAOLa=ZQVkmyVWAxyjrEQoEJ+gKJoJjfFqsDvr_A15FHGX1w=rQ@mail.gmail.com","subject":"Re: [RFC] Implement ref content consistency check","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-15T13:37:44Z","receivedAt":"2024-08-15T13:37:07Z","isPatch":false,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Aug 15, 2024 at 03:19:50AM -0700, karthik nayak wrote:\n> shejialuo writes:\n> \n> > Hi All:\n> >\n> > We have already set up the infrastructure of the ref consistency.\n> > However, we have only add ref name check when establishing the\n> > infrastructure in below:\n> >\n> > https://lore.kernel.org/git/ZrSqMmD-quQ18a9F@ArchLinux.localdomain/\n> >\n> > Actually, we already have a patch here which has already implemented the\n> > ref content consistency check. But during the review process, we have\n> > encountered some problems. The intention of this RFC is to make sure\n> > what content we should check and also to what extend.\n> >\n> > I conclude the following info:\n> >\n> > 1. For the regular ref which has a trailing garbage, we should warn the\n> > user. This is the most simplest situation, we could reply on\n> > \"parse_loose_ref_content\" to do this.\n> > 2. For the symref, we could also rely on \"parse_loose_ref_content\" to\n> > get the \"pointee\", and check the location of the \"pointee\", check the\n> > name of the \"pointee\" and the file type of the \"pointee\".\n> > 3. FOr the symbolic ref, we could follow the idea of 2.\n> >\n> \n> Just to understand clearly, when you're talking about 'symbolic ref' you\n> are referring to symbolic links?\n> \n\nI am sorry about this. It's symbolic links here.\n\n> > But Patrick gives a question here:\n> >\n> >> In case the ref ends with a newline, should we check that the next\n> >> character is `\\0`? Otherwise, it may contain multiple lines, which is\n> >> not allowed for a normal ref.\n> >>\n> >> Also, shouldn't the ref always end with a newline?\n> >\n> > For symref, I guess we have no spec here. From my experiments, a symref\n> > could have a newline or no newline, even multiple newlines. And also\n> > symref could have multiple spaces. But the following is a bad symref\n> >\n> > ref: refs/heads/main garbage\n> >\n> > I think we should fully discuss what we should check here. Thus I will\n> > implement the code.\n> >\n> \n> Agreed, in refs/files-backend.c:create_symref_lock, we write symrefs as\n> \"ref: %s\\n\" so it makes sense to validate that there is nothing extra.\n\nYes, we should do this. I will implement the code and the send the\npatches to the mailing list.\n\nThanks\n\n"},{"id":"501115","messageId":"Zr8Wkudjn5n1Zm0y@tanuki","threadId":"61943","inReplyTo":"Zr4EqESHSnQET1Xg@ArchLinux","subject":"Re: [RFC] Implement ref content consistency check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-16T09:06:31Z","receivedAt":"2024-08-16T09:06:36Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Aug 15, 2024 at 09:37:44PM +0800, shejialuo wrote:\n> On Thu, Aug 15, 2024 at 03:19:50AM -0700, karthik nayak wrote:\n> > shejialuo writes:\n> > \n> > > Hi All:\n> > >\n> > > We have already set up the infrastructure of the ref consistency.\n> > > However, we have only add ref name check when establishing the\n> > > infrastructure in below:\n> > >\n> > > https://lore.kernel.org/git/ZrSqMmD-quQ18a9F@ArchLinux.localdomain/\n> > >\n> > > Actually, we already have a patch here which has already implemented the\n> > > ref content consistency check. But during the review process, we have\n> > > encountered some problems. The intention of this RFC is to make sure\n> > > what content we should check and also to what extend.\n> > >\n> > > I conclude the following info:\n> > >\n> > > 1. For the regular ref which has a trailing garbage, we should warn the\n> > > user. This is the most simplest situation, we could reply on\n> > > \"parse_loose_ref_content\" to do this.\n> > > 2. For the symref, we could also rely on \"parse_loose_ref_content\" to\n> > > get the \"pointee\", and check the location of the \"pointee\", check the\n> > > name of the \"pointee\" and the file type of the \"pointee\".\n> > > 3. FOr the symbolic ref, we could follow the idea of 2.\n> > >\n> > \n> > Just to understand clearly, when you're talking about 'symbolic ref' you\n> > are referring to symbolic links?\n> > \n> \n> I am sorry about this. It's symbolic links here.\n\nWait, is it really symbolic link? I don't think so, you actually were\ntalking about symbolic refs correctly. The fact that symbolic refs have\nbeen implemented as a symbolic link in the past (and still can be used\nfor that purpose) is rather an implementation detail. But the overall\ncontext, and what we actually want to check on disk, is a symbolic ref\nin its modern incarnation.\n\nAnd checking the format of both normal and symbolic refs does make sense\nin my opinion.\n\n> > > But Patrick gives a question here:\n> > >\n> > >> In case the ref ends with a newline, should we check that the next\n> > >> character is `\\0`? Otherwise, it may contain multiple lines, which is\n> > >> not allowed for a normal ref.\n> > >>\n> > >> Also, shouldn't the ref always end with a newline?\n> > >\n> > > For symref, I guess we have no spec here. From my experiments, a symref\n> > > could have a newline or no newline, even multiple newlines. And also\n> > > symref could have multiple spaces. But the following is a bad symref\n> > >\n> > > ref: refs/heads/main garbage\n> > >\n> > > I think we should fully discuss what we should check here. Thus I will\n> > > implement the code.\n> > >\n> > \n> > Agreed, in refs/files-backend.c:create_symref_lock, we write symrefs as\n> > \"ref: %s\\n\" so it makes sense to validate that there is nothing extra.\n> \n> Yes, we should do this. I will implement the code and the send the\n> patches to the mailing list.\n\nAgreed. We have to exclude pseudorefs (FETCH_HEAD, MERGE_HEAD, see\ngitglossary(7)) from these checks, as those _are_ allowed to contain\nextra data. But no other reference should carry more data than that.\nNamely, a regular ref should always be \"hex * hash_len + \\n\", while a\nsymbolic ref should always be \"ref: $valid_refname\\n\".\n\nA ref that does not conform to this is not a properly formatted\nreference and thus worth being warned about.\n\nPatrick\n"},{"id":"501143","messageId":"xmqqzfpcl8wf.fsf@gitster.g","threadId":"61943","inReplyTo":"Zr8Wkudjn5n1Zm0y@tanuki","subject":"Re: [RFC] Implement ref content consistency check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-16T16:39:12Z","receivedAt":"2024-08-16T16:39:20Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n>> > > 1. For the regular ref which has a trailing garbage, we should warn the\n>> > > user. This is the most simplest situation, we could reply on\n>> > > \"parse_loose_ref_content\" to do this.\n>> > > 2. For the symref, we could also rely on \"parse_loose_ref_content\" to\n>> > > get the \"pointee\", and check the location of the \"pointee\", check the\n>> > > name of the \"pointee\" and the file type of the \"pointee\".\n>> > > 3. FOr the symbolic ref, we could follow the idea of 2.\n>> > \n>> > Just to understand clearly, when you're talking about 'symbolic ref' you\n>> > are referring to symbolic links?\n>> \n>> I am sorry about this. It's symbolic links here.\n>\n> Wait, is it really symbolic link? I don't think so, you actually were\n> talking about symbolic refs correctly.\n\nIn #2, yes. I think #3 is about what to do with a random symbolic\nlink inside or near .git/refs/ hierarchy, which may or may not meant\nas a symref. I agree that we should assume that the user meant them\nto be used as symrefs, check its validity the same way as a textual\nsymrefs, and complain if they look bogus.\n\n> And checking the format of both normal and symbolic refs does make sense\n> in my opinion.\n\nYup.\n"},{"id":"501235","messageId":"ZsIMc6cJ-kzMzW_8@ArchLinux","threadId":"61943","inReplyTo":"ZrtrT1CPI4YUf5db@ArchLinux","subject":"[PATCH v1 0/4] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-18T15:00:03Z","receivedAt":"2024-08-18T14:59:23Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis patch aims at adding ref content check for files backend. By the\nRFC we have discussed, I add three types of checks.\n\n1. Check regular ref content. I enhance the \"parse_loose_ref_contents\"\nto validate the content of the regular ref and warn the user about the\ntrailing garbage.\n2. Check symbolic ref content. Check the trailing garbage and content.\n3. Check symlink ref by reusing the function introduced by #2.\n\nThe CI is passed:\n\n https://github.com/shejialuo/git/pull/14\n\nThanks,\nJialuo\n\nshejialuo (4):\n fsck: introduce \"FSCK_REF_REPORT_DEFAULT\" macro\n ref: add regular ref content check for files backend\n ref: add symbolic ref content check for files backend\n ref: add symlink ref consistency check for files backend\n\n Documentation/fsck-msgids.txt | 12 +++\n fsck.h | 10 ++\n refs.c | 2 +-\n refs/files-backend.c | 188 +++++++++++++++++++++++++++++++++-\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 183 +++++++++++++++++++++++++++++++++\n 6 files changed, 392 insertions(+), 5 deletions(-)\n\n-- \n2.46.0\n\n"},{"id":"501236","messageId":"ZsIM0L72bei9Fudt@ArchLinux","threadId":"61943","inReplyTo":"ZsIMc6cJ-kzMzW_8@ArchLinux","subject":"[PATCH v1 1/4] fsck: introduce \"FSCK_REF_REPORT_DEFAULT\" macro","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-18T15:01:36Z","receivedAt":"2024-08-18T15:00:56Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n\"referent\" is NULL. So, we need to always initialize these parameters to\nNULL instead of letting them point to anywhere when creating a new\n\"fsck_ref_report\" structure.\n\nIn order to conveniently create a new \"fsck_ref_report\", add a new macro\n\"FSCK_REF_REPORT_DEFAULT\".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n fsck.h | 6 ++++++\n refs/files-backend.c | 2 +-\n 2 files changed, 7 insertions(+), 1 deletion(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 500b4c04d2..8894394d16 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -152,6 +152,12 @@ struct fsck_ref_report {\n \tconst char *referent;\n };\n \n+#define FSCK_REF_REPORT_DEFAULT { \\\n+\t.path = NULL, \\\n+\t.oid = NULL, \\\n+\t.referent = NULL, \\\n+}\n+\n struct fsck_options {\n \tfsck_walk_func walk;\n \tfsck_error error_func;\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8d6ec9458d..725a4f52e3 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3446,7 +3446,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tstruct fsck_ref_report report = { .path = NULL };\n+\t\tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n-- \n2.46.0\n\n"},{"id":"501237","messageId":"ZsIM2DRDbJsvNjAM@ArchLinux","threadId":"61943","inReplyTo":"ZsIMc6cJ-kzMzW_8@ArchLinux","subject":"[PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-18T15:01:44Z","receivedAt":"2024-08-18T15:01:07Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We implicitly reply on \"git-fsck(1)\" to check the consistency of regular\nrefs. However, when parsing the regular refs for files backend, we allow\nthe ref content to end with no newline or contain some garbages. We\nshould warn the user about above situations.\n\nIn order to provide above functionality, enhance the \"git-refs verify\"\ncommand by adding consistency check for regular refs for files backend.\n\nAdd the following three fsck messages to represent the above situations:\n\n1. \"badRefContent(ERROR)\": A ref has a bad content.\n2. \"refMissingNewline(WARN)\": A valid ref does not end with newline.\n3. \"trailingRefContent(WARN)\": A ref has trailing contents.\n\nIn order to tell whether the ref has trailing content, add a new\nparameter \"trailing\" to \"parse_loose_ref_contents\". Then introduce a new\nfunction \"files_fsck_refs_content\" to check the regular refs.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 9 ++++\n fsck.h | 3 ++\n refs.c | 2 +-\n refs/files-backend.c | 67 ++++++++++++++++++++++++++-\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 87 +++++++++++++++++++++++++++++++++++\n 6 files changed, 166 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 68a2801f15..1688c2f1fe 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -19,6 +19,9 @@\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n+`badRefContent`::\n+\t(ERROR) A ref has a bad content.\n+\n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n \n@@ -170,6 +173,12 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`refMissingNewline`::\n+\t(WARN) A valid ref does not end with newline.\n+\n+`trailingRefContent`::\n+\t(WARN) A ref has trailing contents.\n+\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \ndiff --git a/fsck.h b/fsck.h\nindex 8894394d16..975d9b9da9 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -31,6 +31,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n@@ -73,6 +74,8 @@ enum fsck_msg_type {\n \tFUNC(HAS_DOTDOT, WARN) \\\n \tFUNC(HAS_DOTGIT, WARN) \\\n \tFUNC(NULL_SHA1, WARN) \\\n+\tFUNC(REF_MISSING_NEWLINE, WARN) \\\n+\tFUNC(TRAILING_REF_CONTENT, WARN) \\\n \tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n \tFUNC(NUL_IN_COMMIT, WARN) \\\n \tFUNC(LARGE_PATHNAME, WARN) \\\ndiff --git a/refs.c b/refs.c\nindex 74de3d3009..5e74881945 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1758,7 +1758,7 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \t}\n \n \tresult = parse_loose_ref_contents(ref_store->repo->hash_algo, content.buf,\n-\t\t\t\t\t oid, referent, type, failure_errno);\n+\t\t\t\t\t oid, referent, type, NULL, failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 725a4f52e3..ae71692f36 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -560,7 +560,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname,\n \tbuf = sb_contents.buf;\n \n \tret = parse_loose_ref_contents(ref_store->repo->hash_algo, buf,\n-\t\t\t\t oid, referent, type, &myerr);\n+\t\t\t\t oid, referent, type, NULL, &myerr);\n \n out:\n \tif (ret && !myerr)\n@@ -597,7 +597,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno)\n+\t\t\t const char **trailing, int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -619,6 +619,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n+\n+\tif (trailing)\n+\t\t*trailing = p;\n+\n \treturn 0;\n }\n \n@@ -3430,6 +3434,64 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_refs_content(struct ref_store *ref_store,\n+\t\t\t\t struct fsck_options *o,\n+\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t struct dir_iterator *iter)\n+{\n+\tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n+\tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tstruct strbuf refname = STRBUF_INIT;\n+\tconst char *trailing = NULL;\n+\tunsigned int type = 0;\n+\tint failure_errno = 0;\n+\tstruct object_id oid;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n+\treport.path = refname.buf;\n+\n+\tif (S_ISREG(iter->st.st_mode)) {\n+\t\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n+\t\t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n+\t\t\t\t\t refs_check_dir, iter->relative_path);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n+\t\t\t\t\t ref_content.buf, &oid, &referent,\n+\t\t\t\t\t &type, &trailing, &failure_errno)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t\t \"invalid ref content\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tif (!(type & REF_ISSYMREF)) {\n+\t\t\tif (*trailing == '\\0') {\n+\t\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t\t \"missing newline\");\n+\t\t\t\tgoto cleanup;\n+\t\t\t}\n+\n+\t\t\tif (*trailing != '\\n' || (*(trailing + 1) != '\\0')) {\n+\t\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t\t \"trailing garbage in ref\");\n+\t\t\t\tgoto cleanup;\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&refname);\n+\tstrbuf_release(&ref_content);\n+\tstrbuf_release(&referent);\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n \t\t\t\tconst char *refs_check_dir,\n@@ -3512,6 +3574,7 @@ static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n+\t\tfiles_fsck_refs_content,\n \t\tNULL,\n \t};\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 2313c830d8..73b05f971b 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -715,7 +715,7 @@ struct ref_store {\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno);\n+\t\t\t const char **trailing, int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 71a4d1a5ae..7c1910d784 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -89,4 +89,91 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'regular ref content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\tgit commit --allow-empty -m initial &&\n+\tgit checkout -b branch-1 &&\n+\tgit tag tag-1 &&\n+\tgit commit --allow-empty -m second &&\n+\tgit checkout -b branch-2 &&\n+\tgit tag tag-2 &&\n+\tgit checkout -b a/b/tag-2 &&\n+\n+\tprintf \"%s\" \"$(git rev-parse branch-1)\" > $branch_dir_prefix/branch-1-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline\n+\tEOF\n+\trm $branch_dir_prefix/branch-1-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse branch-1)\" > $branch_dir_prefix/branch-1-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-1-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/branch-1-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-1-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-1-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\\n\\na\" \"$(git rev-parse tag-2)\" > $tag_dir_prefix/tag-2-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-2-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-2-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-garbage &&\n+\ttest_must_fail git -c fsck.trailingRefContent=error refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-1-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%sx\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-1-bad: badRefContent: invalid ref content\n+\tEOF\n+\trm $tag_dir_prefix/tag-1-bad &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"xfsazqfxcadas\" > $tag_dir_prefix/tag-2-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-2-bad: badRefContent: invalid ref content\n+\tEOF\n+\trm $tag_dir_prefix/tag-2-bad &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"xfsazqfxcadas\" > $branch_dir_prefix/a/b/branch-2-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-2-bad: badRefContent: invalid ref content\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-2-bad &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"501238","messageId":"ZsIM4OZWfylcP5Ix@ArchLinux","threadId":"61943","inReplyTo":"ZsIMc6cJ-kzMzW_8@ArchLinux","subject":"[PATCH v1 3/4] ref: add symbolic ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-18T15:01:52Z","receivedAt":"2024-08-18T15:01:13Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already introduced the checks for regular refs. There is no need\nto check the consistency of the target which the symbolic ref points to.\nInstead, we just check the content of the symbolic ref itself.\n\nIn order to check the content of the symbolic ref, create a function\n\"files_fsck_symref_target\". It will first check whether the \"pointee\" is\nunder the \"refs/\" directory and then we will check the \"pointee\" itself.\n\nThere is no specification about the content of the symbolic ref.\nAlthough we do write \"ref: %s\\n\" to create a symbolic ref by using\n\"git-symbolic-ref(1)\" command. However, this is not mandatory. We still\naccept symbolic refs with null trailing garbage. Put it more specific,\nthe following are correct:\n\n1. \"ref: refs/heads/master \"\n2. \"ref: refs/heads/master \\n \\n\"\n3. \"ref: refs/heads/master\\n\\n\"\n\nBut we do not allow any non-null trailing garbage. The following are bad\nsymbolic contents.\n\n1. \"ref: refs/heads/master garbage\\n\"\n2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n\nIn order to provide above checks, we will traverse the \"pointee\" to\nreport the user whether this is null-garbage or no newline. And if\nsymbolic refs contain non-null garbage, we will report\n\"FSCK_MSG_BAD_REF_CONTENT\" to the user.\n\nThen, we will check the name of the \"pointee\" is correct by using\n\"check_refname_format\". And then if we can access the \"pointee_path\" in\nthe file system, we should ensure that the file type is correct.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 ++\n fsck.h | 1 +\n refs/files-backend.c | 87 +++++++++++++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 52 +++++++++++++++++++++\n 4 files changed, 143 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 1688c2f1fe..73587661dc 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,6 +28,9 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n+`badSymrefPointee`::\n+\t(ERROR) The pointee of a symref is bad.\n+\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n \ndiff --git a/fsck.h b/fsck.h\nindex 975d9b9da9..985b674dd9 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,6 +34,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n+\tFUNC(BAD_SYMREF_POINTEE, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex ae71692f36..bfb8d338d2 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3434,12 +3434,92 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+/*\n+ * Check the symref \"pointee_name\" and \"pointee_path\". The caller should\n+ * make sure that \"pointee_path\" is absolute. For symbolic ref, \"pointee_name\"\n+ * would be the content after \"refs:\".\n+ */\n+static int files_fsck_symref_target(struct fsck_options *o,\n+\t\t\t\t struct fsck_ref_report *report,\n+\t\t\t\t const char *refname,\n+\t\t\t\t struct strbuf *pointee_name,\n+\t\t\t\t struct strbuf *pointee_path)\n+{\n+\tunsigned int newline_num = 0;\n+\tunsigned int space_num = 0;\n+\tconst char *p = NULL;\n+\tstruct stat st;\n+\tint ret = 0;\n+\n+\tif (!skip_prefix(pointee_name->buf, \"refs/\", &p)) {\n+\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n+\t\t\t\t \"points to ref outside the refs directory\");\n+\t\tgoto out;\n+\t}\n+\n+\twhile (*p != '\\0') {\n+\t\tif ((space_num || newline_num) && !isspace(*p)) {\n+\t\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t\t \"contains non-null garbage\");\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (*p == '\\n') {\n+\t\t\tnewline_num++;\n+\t\t} else if (*p == ' ') {\n+\t\t\tspace_num++;\n+\t\t}\n+\t\tp++;\n+\t}\n+\n+\tif (space_num || newline_num > 1) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t \"trailing null-garbage\");\n+\t} else if (!newline_num) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t \"missing newline\");\n+\t}\n+\n+\tstrbuf_rtrim(pointee_name);\n+\n+\tif (check_refname_format(pointee_name->buf, 0)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n+\t\t\t\t \"points to refname with invalid format\");\n+\t}\n+\n+\t/*\n+\t * Missing target should not be treated as any error worthy event and\n+\t * not even warn. It is a common case that a symbolic ref points to a\n+\t * ref that does not exist yet. If the target ref does not exist, just\n+\t * skip the check for the file type.\n+\t */\n+\tif (lstat(pointee_path->buf, &st) < 0)\n+\t\tgoto out;\n+\n+\tif (!S_ISREG(st.st_mode) && !S_ISLNK(st.st_mode)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n+\t\t\t\t \"points to an invalid file type\");\n+\t\tgoto out;\n+\t}\n+\n+out:\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter)\n {\n \tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n+\tstruct strbuf pointee_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -3482,6 +3562,12 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t\t\t \"trailing garbage in ref\");\n \t\t\t\tgoto cleanup;\n \t\t\t}\n+\t\t} else {\n+\t\t\tstrbuf_addf(&pointee_path, \"%s/%s\",\n+\t\t\t\t ref_store->gitdir, referent.buf);\n+\t\t\tret = files_fsck_symref_target(o, &report, refname.buf,\n+\t\t\t\t\t\t &referent,\n+\t\t\t\t\t\t &pointee_path);\n \t\t}\n \t}\n \n@@ -3489,6 +3575,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n+\tstrbuf_release(&pointee_path);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 7c1910d784..e8fc2ef015 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -176,4 +176,56 @@ test_expect_success 'regular ref content should be checked' '\n \ttest_cmp expect err\n '\n \n+test_expect_success 'symbolic ref content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\tgit commit --allow-empty -m initial &&\n+\tgit checkout -b branch-1 &&\n+\tgit tag tag-1 &&\n+\tgit checkout -b a/b/branch-2 &&\n+\n+\tprintf \"ref: refs/heads/branch\" > $branch_dir_prefix/branch-1-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline\n+\tEOF\n+\trm $branch_dir_prefix/branch-1-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \" > $branch_dir_prefix/a/b/branch-trailing &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\\n\" > $branch_dir_prefix/a/b/branch-trailing &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n\\n \" > $branch_dir_prefix/a/b/branch-trailing &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/.branch\\n\" > $branch_dir_prefix/branch-2-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-2-bad: badSymrefPointee: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-2-bad &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"501239","messageId":"ZsIM6JZ7miA3j09j@ArchLinux","threadId":"61943","inReplyTo":"ZsIMc6cJ-kzMzW_8@ArchLinux","subject":"[PATCH v1 4/4] ref: add symlink ref consistency check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-18T15:02:00Z","receivedAt":"2024-08-18T15:01:20Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already introduced \"files_fsck_symref_target\". We should reuse\nthis function to handle the symrefs which are legacy symbolic links. We\nshould not check the trailing garbage for symbolic links. Add a new\nparameter \"symbolic_link\" to disable some checks which should only be\nused for symbolic ref.\n\nWe firstly use the \"strbuf_add_real_path\" to resolve the symlinks and\nget the absolute path \"pointee_path\" which the symlink ref points to.\nThen we can get the absolute path \"abs_gitdir\" of the \"gitdir\". By\ncombining \"pointee_path\" and \"abs_gitdir\", we can extract the\n\"referent\". Thus, we can reuse \"files_fsck_symref_target\" function to\nseamlessly check the symlink refs.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 82 ++++++++++++++++++++++++++++------------\n t/t0602-reffiles-fsck.sh | 44 +++++++++++++++++++++\n 2 files changed, 101 insertions(+), 25 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex bfb8d338d2..398afedaf0 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1,4 +1,5 @@\n #include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n #include \"../copy.h\"\n #include \"../environment.h\"\n #include \"../gettext.h\"\n@@ -3437,13 +3438,15 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n /*\n * Check the symref \"pointee_name\" and \"pointee_path\". The caller should\n * make sure that \"pointee_path\" is absolute. For symbolic ref, \"pointee_name\"\n- * would be the content after \"refs:\".\n+ * would be the content after \"refs:\". For symblic link, \"pointee_name\" would\n+ * be the relative path agaignst \"gitdir\".\n */\n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n \t\t\t\t const char *refname,\n \t\t\t\t struct strbuf *pointee_name,\n-\t\t\t\t struct strbuf *pointee_path)\n+\t\t\t\t struct strbuf *pointee_path,\n+\t\t\t\t unsigned int symbolic_link)\n {\n \tunsigned int newline_num = 0;\n \tunsigned int space_num = 0;\n@@ -3459,34 +3462,36 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n-\twhile (*p != '\\0') {\n-\t\tif ((space_num || newline_num) && !isspace(*p)) {\n-\t\t\tret = fsck_report_ref(o, report,\n-\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n-\t\t\t\t\t \"contains non-null garbage\");\n-\t\t\tgoto out;\n+\tif (!symbolic_link) {\n+\t\twhile (*p != '\\0') {\n+\t\t\tif ((space_num || newline_num) && !isspace(*p)) {\n+\t\t\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t\t\t \"contains non-null garbage\");\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\n+\t\t\tif (*p == '\\n') {\n+\t\t\t\tnewline_num++;\n+\t\t\t} else if (*p == ' ') {\n+\t\t\t\tspace_num++;\n+\t\t\t}\n+\t\t\tp++;\n \t\t}\n \n-\t\tif (*p == '\\n') {\n-\t\t\tnewline_num++;\n-\t\t} else if (*p == ' ') {\n-\t\t\tspace_num++;\n+\t\tif (space_num || newline_num > 1) {\n+\t\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t \"trailing null-garbage\");\n+\t\t} else if (!newline_num) {\n+\t\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t \"missing newline\");\n \t\t}\n-\t\tp++;\n-\t}\n \n-\tif (space_num || newline_num > 1) {\n-\t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n-\t\t\t\t \"trailing null-garbage\");\n-\t} else if (!newline_num) {\n-\t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n-\t\t\t\t \"missing newline\");\n+\t\tstrbuf_rtrim(pointee_name);\n \t}\n \n-\tstrbuf_rtrim(pointee_name);\n-\n \tif (check_refname_format(pointee_name->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n@@ -3521,8 +3526,10 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n \tstruct strbuf pointee_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n+\tunsigned int symbolic_link = 0;\n \tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n@@ -3567,8 +3574,32 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t ref_store->gitdir, referent.buf);\n \t\t\tret = files_fsck_symref_target(o, &report, refname.buf,\n \t\t\t\t\t\t &referent,\n-\t\t\t\t\t\t &pointee_path);\n+\t\t\t\t\t\t &pointee_path,\n+\t\t\t\t\t\t symbolic_link);\n+\t\t}\n+\t} else if (S_ISLNK(iter->st.st_mode)) {\n+\t\tconst char *pointee_name = NULL;\n+\n+\t\tsymbolic_link = 1;\n+\n+\t\tstrbuf_add_real_path(&pointee_path, iter->path.buf);\n+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n+\t\tstrbuf_normalize_path(&abs_gitdir);\n+\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n+\t\t\tstrbuf_addch(&abs_gitdir, '/');\n+\n+\t\tif (!skip_prefix(pointee_path.buf,\n+\t\t\t\t abs_gitdir.buf, &pointee_name)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n+\t\t\t\t\t \"point to target outside gitdir\");\n+\t\t\tgoto cleanup;\n \t\t}\n+\n+\t\tstrbuf_addstr(&referent, pointee_name);\n+\t\tret = files_fsck_symref_target(o, &report, refname.buf,\n+\t\t\t\t\t &referent, &pointee_path,\n+\t\t\t\t\t symbolic_link);\n \t}\n \n cleanup:\n@@ -3576,6 +3607,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n \tstrbuf_release(&pointee_path);\n+\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex e8fc2ef015..c6e93e4757 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -228,4 +228,48 @@ test_expect_success 'symbolic ref content should be checked' '\n \ttest_cmp expect err\n '\n \n+test_expect_success SYMLINKS 'symbolic ref (symbolic link) content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\tgit commit --allow-empty -m initial &&\n+\tgit checkout -b branch-1 &&\n+\tgit tag tag-1 &&\n+\tgit checkout -b a/b/branch-2 &&\n+\n+\tln -sf ../../../../branch $branch_dir_prefix/branch-symbolic &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic: badSymrefPointee: point to target outside gitdir\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../logs/branch-bad $branch_dir_prefix/branch-symbolic &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic: badSymrefPointee: points to ref outside the refs directory\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic: badSymrefPointee: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\".branch\" $branch_dir_prefix/branch-symbolic &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic: badSymrefPointee: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"501384","messageId":"xmqq4j7fb1pn.fsf@gitster.g","threadId":"61943","inReplyTo":"ZsIM0L72bei9Fudt@ArchLinux","subject":"Re: [PATCH v1 1/4] fsck: introduce \"FSCK_REF_REPORT_DEFAULT\" macro","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-20T16:25:56Z","receivedAt":"2024-08-20T16:26:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n> \"referent\" is NULL. So, we need to always initialize these parameters to\n> NULL instead of letting them point to anywhere when creating a new\n> \"fsck_ref_report\" structure.\n\nThe above is correct, but ...\n\n> \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n> -\t\tstruct fsck_ref_report report = { .path = NULL };\n> +\t\tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n\n... the code without this patch is already doing so.\n\nWhen designated initializers are used to initialize a struct, all\nmembers that are not initialized explicitly are implicitly\ninitialized the same as for objects that have static storage\nduration (meaning: pointers are initialized to NULL, arithmetics are\ninitialized to zero).\n\nSo I do not quite see why this change is needed. By hiding the fact\nthat the \"report\" structure is zero-initialized behind the macro, it\nmakes it less obvious that we are clearing everything.\n\nIf the patch were to rewrite the above like so:\n\n\t\tstruct fsck_ref_report report = { 0 }\n\nit would make it even more clear that everything is zero\ninitialized, and also makes it obvious that .path member is not any\nspecial.\n\nThanks.\n"},{"id":"501385","messageId":"xmqqed6j9m24.fsf@gitster.g","threadId":"61943","inReplyTo":"ZsIM2DRDbJsvNjAM@ArchLinux","subject":"Re: [PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-20T16:49:23Z","receivedAt":"2024-08-20T16:49:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> We implicitly reply on \"git-fsck(1)\" to check the consistency of regular\n\n\"reply\" -> \"rely\", I think.\n\n> refs. However, when parsing the regular refs for files backend, we allow\n> the ref content to end with no newline or contain some garbages. We\n> should warn the user about above situations.\n\nHmph, should we? \n\nIf the content is short (e.g., in SHA-1 repository it only has 39\nhexdigit) even if that may be sufficient to uniquely name the\nobject, we should warn about it, of course. A file that has\n64-hexdigit with a terminating LF at the end may be a valid file to\nbe in $GIT_DIR/refs/ hierarchy in a SHA-256 repository, but such a\nfile in a SHA-1 repository should also be subject to a warning, as\nit could be a sign that somebody screwed up object format\nconversion.\n\nBut a file that has only 40-hexdigit without a terminating LF at the\nend? Or a file that has 40-hexdigit followed by a CRLF instead of\nLF? Or a file that has the identical content as a valid ref on its\nfirst line, but has extra stuff on its second and subsequent lines?\n\nWhat does the name-to-object-name-mapping layer (aka \"get_oid\" API)\ndo when they see such a file in the $GIT_DIR/refs/ hierarchy? If\nthey are treated as valid ref in the \"normal\" code path, it needs a\nstrong justification to tighten the rules retroactively, much\nstronger than \"Our current code, and any of our older versions,\nwould have written such a file as a loose ref with our code.\"\n\n\"What are we protecting us from with this tightening?\" is the\nquestion we should be asking ourselves, when evaluating each of\nthese new rules that fsck used not to care about.\n"},{"id":"501434","messageId":"ZsXiXi4Nt1eSKLad@ArchLinux","threadId":"61943","inReplyTo":"xmqq4j7fb1pn.fsf@gitster.g","subject":"Re: [PATCH v1 1/4] fsck: introduce \"FSCK_REF_REPORT_DEFAULT\" macro","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-21T12:49:34Z","receivedAt":"2024-08-21T12:48:50Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Aug 20, 2024 at 09:25:56AM -0700, Junio C Hamano wrote:\n> > \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n> > -\t\tstruct fsck_ref_report report = { .path = NULL };\n> > +\t\tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n> \n> ... the code without this patch is already doing so.\n> \n> When designated initializers are used to initialize a struct, all\n> members that are not initialized explicitly are implicitly\n> initialized the same as for objects that have static storage\n> duration (meaning: pointers are initialized to NULL, arithmetics are\n> initialized to zero).\n> \n> So I do not quite see why this change is needed. By hiding the fact\n> that the \"report\" structure is zero-initialized behind the macro, it\n> makes it less obvious that we are clearing everything.\n> \n> If the patch were to rewrite the above like so:\n> \n> \t\tstruct fsck_ref_report report = { 0 }\n> \n> it would make it even more clear that everything is zero\n> initialized, and also makes it obvious that .path member is not any\n> special.\n> \n\nYes, I should use this way. Thanks.\n\n> Thanks.\n"},{"id":"501435","messageId":"ZsX3-yU52X2fe6JT@ArchLinux","threadId":"61943","inReplyTo":"xmqqed6j9m24.fsf@gitster.g","subject":"Re: [PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-21T14:21:47Z","receivedAt":"2024-08-21T14:21:04Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Aug 20, 2024 at 09:49:23AM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > We implicitly reply on \"git-fsck(1)\" to check the consistency of regular\n> \n> \"reply\" -> \"rely\", I think.\n\nI will fix in the next version.\n\n> > refs. However, when parsing the regular refs for files backend, we allow\n> > the ref content to end with no newline or contain some garbages. We\n> > should warn the user about above situations.\n> \n> Hmph, should we? \n>\n\nI am very sorry about this. Actually, I should not use \"should\". I don't\ngive compelling reasons here why we need to introduce such checks. I\njust told the reviewer \"we should warn\". I will try to avoid above\nmistakes where I didn't give enough motivation.\n\n> What does the name-to-object-name-mapping layer (aka \"get_oid\" API)\n> do when they see such a file in the $GIT_DIR/refs/ hierarchy? If\n> they are treated as valid ref in the \"normal\" code path, it needs a\n> strong justification to tighten the rules retroactively, much\n> stronger than \"Our current code, and any of our older versions,\n> would have written such a file as a loose ref with our code.\"\n> \n\nLet me first talk about what will happen when we use the following\ncommand:\n\n $ git checkout bad-branch\n\nI use \"gdb\" to find the following call sequence:\n\n \"cmd_checkout\" -> \"checkout_main\" -> \"parse_branchname_arg\" ->\n ... -> \"get_oid_basic\" -> \"repo_dwim_ref\" -> ... ->\n \"parse_loose_ref_contents\" -> \"parse_oid_hex_algop\" ->\n \"get_oid_hex_algop\"\n\nI dive into the \"object-name.c::get_oid_basic\" function. If we pass the\nactually \"oid\", it will call the \"get_oid_hex_algop\" directly.\nOtherwise, it will execute the following code:\n\n if (!len && reflog_len)\n refs_found = ...;\n else if (reflog_len)\n refs_found = ...\n else\n refs_found = repo_dwim_ref(r, str, len, oid, &real_ref, !fatal);\n\n if (!refs_found)\n return -1;\n\nAs we can see, when there is no corresponding refs found by calling\n\"repo_dwim_ref\" function, \"get_oid_basic\" function will return -1. And\nhere we could have one important conclusion:\n\n The \"get_oid_basic\" function relies on \"repo_dwim_ref\" function to\n parse the ref and get the pointee \"oid\". So, it uses the interfaces\n provided by ref backend.\n\nNext, we look at what will \"parse_loose_ref_contents\" do for regular\nrefs.\n\n int parse_loose_ref_contents(...)\n {\n ...\n if (parse_oid_hex_algop(buf, oid, *p, algop) ||\n (*p != '\\0' && !isspace(*p))) {\n *type |= REF_ISBROKEN;\n *failure_errno = EINVAL;\n return -1;\n }\n return 0;\n }\n\nLet's continue to see what \"parse_oid_hex_algop\" will do:\n\n int parse_oid_hex_algop(...)\n {\n int ret = get_oid_hex_algop(hex, oid, algop);\n if (!ret) {\n *end = hex + algop->hexsz;\n }\n return ret;\n }\n\nIf the result of \"get_oid_hex_algop\" is successful. We will set the\n\"end\" pointer here. The \"get_oid_hex_algop\" will eventually call the\n\"get_hash_hex_algop\" function\n\n static int get_hash_hex_algop(...)\n {\n int i;\n for (i = 0; i < algop->rawsz; i++) {\n int val = hex2chr(hex);\n if (val < 0)\n return -1;\n *hash+= = val;\n hex += 2;\n }\n return 0;\n }\n\nThis function will convert the hex to char by the raw size of the\nalgorithm. And by the following code, we could conclude the following\nthings:\n\n1. \"41053a9084501db79c72b14e8a5a0b67de3f91ae\" is correct, because it\nwill be parsed successfully by \"get_hash_hex_algop\" and \"*p == '\\0'\".\n2. \"41053a9084501db79c72b14e8a5a0b67de3f91aef\" is not correct, it will\nbe parsed successfully by \"get_hash_hex_algop\" but \"*p != '\\0'\"\nand \"isspace(*p)\" is false. So the check in \"parse_loose_ref_contents\"\ncannot be passed.\n3. \"1053a9084501db79c72b14e8a5a0b67de3f91a\" is not correct, it cannot be\nparsed successfully by \"get_hash_hex_algop\".\n4. \"41053a9084501db79c72b14e8a5a0b67de3f91ae garbage\" is correct,\nbecause it will be parsed successfully by \"get_hash_hex_algop\" and\n\"isspace(*p)\" is true.\n\nBy the above discussion, I could answer you comments one by one.\n\n> If the content is short (e.g., in SHA-1 repository it only has 39\n> hexdigit) even if that may be sufficient to uniquely name the\n> object, we should warn about it, of course.\n\nWhen the content is short, although it may be sufficient to identify the\nobject, we should still report an error here. This is because we care\nabout the ref. As we can see from above discussion, the \"object-name.c\"\ntotally relies on the interfaces provided by the ref backend. And\n\"get_hash_hex_algop\" is unhappy about this situation. And eventually the\n\"object-name.c::get_oid_basic\" will be unhappy, return -1.\n\n> A file that has 64-hexdigit with a terminating LF at the end may be\n> a valid file to be in $GIT_DIR/refs/ hierarchy in a SHA-256\n> repository, but such a file in a SHA-1 repository should also be\n> subject to a warning, as it could be a sign that somebody screwed up\n> object format conversion.\n\nI agree with this idea. But in this implementation, we want to reuse the\n\"parse_loose_ref_contents\" to check the consistency of the regular refs.\nIf we are in a SHA-1 repository, \"parse_loose_ref_contents\" will be\nunhappy about this. However, I don't think we need to provide user that\n\"the content is 64-hexdigit ...\". We just report \"bad ref content\" to\nthe user. This will also indicate the user something is wrong, you need\nto check the ref database.\n\n> But a file that has only 40-hexdigit without a terminating LF at the\n> end? Or a file that has 40-hexdigit followed by a CRLF instead of\n> LF? Or a file that has the identical content as a valid ref on its\n> first line, but has extra stuff on its second and subsequent lines?\n\nThis is the core problem why we want to introduce more strict check.\nBecause in the current \"parse_loose_ref_contents\" function, as long as\nthe next byte of the end of the hex is '\\0', spaces, LF, CRLF. We could\nknow that the content of the ref is OK.\n\nBut in my view, we should warn the user about this situation. This is\nbecause in the original code, we do not check the ref strictly for files\nbackend. And I think at current, the normal user should not interact\nwith the git database. If there are some garbages we found in the ref\ndatabase, I guess this could be a sign for the user: \"Watch out! there\nmay be something wrong\".\n\n> \"What are we protecting us from with this tightening?\" is the\n> question we should be asking ourselves, when evaluating each of\n> these new rules that fsck used not to care about.\n\nThat's a hard question, really. I find it hard to know what should we\ndo? The motivation is hard to describe. But I think this reply could\nmake thing more clear here.\n\nThanks,\nJialuo\n"},{"id":"501501","messageId":"Zsb64NTyqc_oHNFO@tanuki","threadId":"61943","inReplyTo":"xmqqed6j9m24.fsf@gitster.g","subject":"Re: [PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-22T08:46:31Z","receivedAt":"2024-08-22T08:46:36Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Aug 20, 2024 at 09:49:23AM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > We implicitly reply on \"git-fsck(1)\" to check the consistency of regular\n> \n> \"reply\" -> \"rely\", I think.\n> \n> > refs. However, when parsing the regular refs for files backend, we allow\n> > the ref content to end with no newline or contain some garbages. We\n> > should warn the user about above situations.\n> \n> Hmph, should we? \n> \n> If the content is short (e.g., in SHA-1 repository it only has 39\n> hexdigit) even if that may be sufficient to uniquely name the\n> object, we should warn about it, of course. A file that has\n> 64-hexdigit with a terminating LF at the end may be a valid file to\n> be in $GIT_DIR/refs/ hierarchy in a SHA-256 repository, but such a\n> file in a SHA-1 repository should also be subject to a warning, as\n> it could be a sign that somebody screwed up object format\n> conversion.\n> \n> But a file that has only 40-hexdigit without a terminating LF at the\n> end? Or a file that has 40-hexdigit followed by a CRLF instead of\n> LF? Or a file that has the identical content as a valid ref on its\n> first line, but has extra stuff on its second and subsequent lines?\n> \n> What does the name-to-object-name-mapping layer (aka \"get_oid\" API)\n> do when they see such a file in the $GIT_DIR/refs/ hierarchy? If\n> they are treated as valid ref in the \"normal\" code path, it needs a\n> strong justification to tighten the rules retroactively, much\n> stronger than \"Our current code, and any of our older versions,\n> would have written such a file as a loose ref with our code.\"\n> \n> \"What are we protecting us from with this tightening?\" is the\n> question we should be asking ourselves, when evaluating each of\n> these new rules that fsck used not to care about.\n\nI'd say filesystem corruption, buggy implementations and compatibility\nwith other implementations of Git. The format for refs does not allow\nfor any other information than either an object ID for plain refs, and\nthe referee for symbolic refs. The fact that we do accept that is a mere\nimplementation detail because we reuse the same function to parse refs\nthat we also use for pseudorefs. And these _can_ have additional data.\n\nSo any reference that contains additional data is not a proper ref and\nthus should be warned about from my point of view. No Git tooling should\nwrite them, so if something does it's a red flag to me.\n\nPatrick\n"},{"id":"501502","messageId":"Zsb7XgX4Lt_4LX_P@tanuki","threadId":"61943","inReplyTo":"ZsIM2DRDbJsvNjAM@ArchLinux","subject":"Re: [PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-22T08:48:30Z","receivedAt":"2024-08-22T08:48:35Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Aug 18, 2024 at 11:01:44PM +0800, shejialuo wrote:\n> +static int files_fsck_refs_content(struct ref_store *ref_store,\n> +\t\t\t\t struct fsck_options *o,\n> +\t\t\t\t const char *refs_check_dir,\n> +\t\t\t\t struct dir_iterator *iter)\n> +{\n> +\tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n> +\tstruct strbuf ref_content = STRBUF_INIT;\n> +\tstruct strbuf referent = STRBUF_INIT;\n> +\tstruct strbuf refname = STRBUF_INIT;\n> +\tconst char *trailing = NULL;\n> +\tunsigned int type = 0;\n> +\tint failure_errno = 0;\n> +\tstruct object_id oid;\n> +\tint ret = 0;\n> +\n> +\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n> +\treport.path = refname.buf;\n> +\n> +\tif (S_ISREG(iter->st.st_mode)) {\n\nWe can avoid having to indent the remainder of this function if we `goto\ncleanup` here.\n\nPatrick\n"},{"id":"501503","messageId":"Zsb8oDA-vyLxNY0U@tanuki","threadId":"61943","inReplyTo":"ZsIM4OZWfylcP5Ix@ArchLinux","subject":"Re: [PATCH v1 3/4] ref: add symbolic ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-22T08:53:57Z","receivedAt":"2024-08-22T08:54:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Aug 18, 2024 at 11:01:52PM +0800, shejialuo wrote:\n> We have already introduced the checks for regular refs. There is no need\n> to check the consistency of the target which the symbolic ref points to.\n> Instead, we just check the content of the symbolic ref itself.\n> \n> In order to check the content of the symbolic ref, create a function\n> \"files_fsck_symref_target\". It will first check whether the \"pointee\" is\n> under the \"refs/\" directory and then we will check the \"pointee\" itself.\n> \n> There is no specification about the content of the symbolic ref.\n> Although we do write \"ref: %s\\n\" to create a symbolic ref by using\n> \"git-symbolic-ref(1)\" command. However, this is not mandatory. We still\n> accept symbolic refs with null trailing garbage. Put it more specific,\n> the following are correct:\n> \n> 1. \"ref: refs/heads/master \"\n> 2. \"ref: refs/heads/master \\n \\n\"\n> 3. \"ref: refs/heads/master\\n\\n\"\n> \n> But we do not allow any non-null trailing garbage. The following are bad\n> symbolic contents.\n> \n> 1. \"ref: refs/heads/master garbage\\n\"\n> 2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n> \n> In order to provide above checks, we will traverse the \"pointee\" to\n> report the user whether this is null-garbage or no newline. And if\n> symbolic refs contain non-null garbage, we will report\n> \"FSCK_MSG_BAD_REF_CONTENT\" to the user.\n> \n> Then, we will check the name of the \"pointee\" is correct by using\n> \"check_refname_format\". And then if we can access the \"pointee_path\" in\n> the file system, we should ensure that the file type is correct.\n> \n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> Documentation/fsck-msgids.txt | 3 ++\n> fsck.h | 1 +\n> refs/files-backend.c | 87 +++++++++++++++++++++++++++++++++++\n> t/t0602-reffiles-fsck.sh | 52 +++++++++++++++++++++\n> 4 files changed, 143 insertions(+)\n> \n> diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> index 1688c2f1fe..73587661dc 100644\n> --- a/Documentation/fsck-msgids.txt\n> +++ b/Documentation/fsck-msgids.txt\n> @@ -28,6 +28,9 @@\n> `badRefName`::\n> \t(ERROR) A ref has an invalid format.\n> \n> +`badSymrefPointee`::\n> +\t(ERROR) The pointee of a symref is bad.\n> +\n> `badTagName`::\n> \t(INFO) A tag has an invalid format.\n> \n> diff --git a/fsck.h b/fsck.h\n> index 975d9b9da9..985b674dd9 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -34,6 +34,7 @@ enum fsck_msg_type {\n> \tFUNC(BAD_REF_CONTENT, ERROR) \\\n> \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n> \tFUNC(BAD_REF_NAME, ERROR) \\\n> +\tFUNC(BAD_SYMREF_POINTEE, ERROR) \\\n> \tFUNC(BAD_TIMEZONE, ERROR) \\\n> \tFUNC(BAD_TREE, ERROR) \\\n> \tFUNC(BAD_TREE_SHA1, ERROR) \\\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index ae71692f36..bfb8d338d2 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3434,12 +3434,92 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> \t\t\t\t const char *refs_check_dir,\n> \t\t\t\t struct dir_iterator *iter);\n> \n> +/*\n> + * Check the symref \"pointee_name\" and \"pointee_path\". The caller should\n> + * make sure that \"pointee_path\" is absolute. For symbolic ref, \"pointee_name\"\n> + * would be the content after \"refs:\".\n> + */\n> +static int files_fsck_symref_target(struct fsck_options *o,\n> +\t\t\t\t struct fsck_ref_report *report,\n> +\t\t\t\t const char *refname,\n> +\t\t\t\t struct strbuf *pointee_name,\n> +\t\t\t\t struct strbuf *pointee_path)\n> +{\n> +\tunsigned int newline_num = 0;\n> +\tunsigned int space_num = 0;\n> +\tconst char *p = NULL;\n> +\tstruct stat st;\n> +\tint ret = 0;\n> +\n> +\tif (!skip_prefix(pointee_name->buf, \"refs/\", &p)) {\n> +\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> +\t\t\t\t \"points to ref outside the refs directory\");\n> +\t\tgoto out;\n> +\t}\n> +\n> +\twhile (*p != '\\0') {\n\nWe typically write this `while (*p)`.\n\n> +\t\tif ((space_num || newline_num) && !isspace(*p)) {\n> +\t\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> +\t\t\t\t\t \"contains non-null garbage\");\n> +\t\t\tgoto out;\n> +\t\t}\n> +\n> +\t\tif (*p == '\\n') {\n> +\t\t\tnewline_num++;\n> +\t\t} else if (*p == ' ') {\n> +\t\t\tspace_num++;\n> +\t\t}\n> +\t\tp++;\n> +\t}\n\nCan't we replace this with a single `strchr('\\n')` call to check for the\nnewline and then verify that the next character is a `\\0`? The check for\nspaces would then be handled by `check_refname_format()`.\n\n> +\t/*\n> +\t * Missing target should not be treated as any error worthy event and\n> +\t * not even warn. It is a common case that a symbolic ref points to a\n> +\t * ref that does not exist yet. If the target ref does not exist, just\n> +\t * skip the check for the file type.\n> +\t */\n> +\tif (lstat(pointee_path->buf, &st) < 0)\n> +\t\tgoto out;\n> +\n> +\tif (!S_ISREG(st.st_mode) && !S_ISLNK(st.st_mode)) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> +\t\t\t\t \"points to an invalid file type\");\n> +\t\tgoto out;\n> +\t}\n\nWhat exactly are we guarding against here? Don't we already verify that\nfiles in `refs/` have the correct type? Or are we checking that it does\nnot point to a directory?\n\nPatrick\n"},{"id":"501530","messageId":"ZscpxXaHLpycuMGH@ArchLinux","threadId":"61943","inReplyTo":"Zsb7XgX4Lt_4LX_P@tanuki","subject":"Re: [PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-22T12:06:29Z","receivedAt":"2024-08-22T12:05:45Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Aug 22, 2024 at 10:48:30AM +0200, Patrick Steinhardt wrote:\n>\n> We can avoid having to indent the remainder of this function if we `goto\n> cleanup` here.\n> \n\nYes, actually I have thought about this way. But I don't want to use\n\"goto\". However, ident is noisy too. I will fix in the next version.\n"},{"id":"501532","messageId":"ZscyGg8M8TbJVKNS@ArchLinux","threadId":"61943","inReplyTo":"Zsb8oDA-vyLxNY0U@tanuki","subject":"Re: [PATCH v1 3/4] ref: add symbolic ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-22T12:42:02Z","receivedAt":"2024-08-22T12:41:18Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Thu, Aug 22, 2024 at 10:53:57AM +0200, Patrick Steinhardt wrote:\n\n\n> > +\t\tif ((space_num || newline_num) && !isspace(*p)) {\n> > +\t\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> > +\t\t\t\t\t \"contains non-null garbage\");\n> > +\t\t\tgoto out;\n> > +\t\t}\n> > +\n> > +\t\tif (*p == '\\n') {\n> > +\t\t\tnewline_num++;\n> > +\t\t} else if (*p == ' ') {\n> > +\t\t\tspace_num++;\n> > +\t\t}\n> > +\t\tp++;\n> > +\t}\n> \n> Can't we replace this with a single `strchr('\\n')` call to check for the\n> newline and then verify that the next character is a `\\0`? The check for\n> spaces would then be handled by `check_refname_format()`.\n> \n\nWe cannot. Think about this situation.\n\n \"ref: refs/heads/master \\n \"\n\nWe find that the next character of '\\n' is not '\\0'. Then we leave it to\n\"check_refname_format\". But \"check_refname_format\" will report an error\nhere, but this is an allowed symref.\n\nBut I think using `strchr` is a nice way. I will try to find an elegant\nway here to handle this logic here.\n\n> > +\t/*\n> > +\t * Missing target should not be treated as any error worthy event and\n> > +\t * not even warn. It is a common case that a symbolic ref points to a\n> > +\t * ref that does not exist yet. If the target ref does not exist, just\n> > +\t * skip the check for the file type.\n> > +\t */\n> > +\tif (lstat(pointee_path->buf, &st) < 0)\n> > +\t\tgoto out;\n> > +\n> > +\tif (!S_ISREG(st.st_mode) && !S_ISLNK(st.st_mode)) {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> > +\t\t\t\t \"points to an invalid file type\");\n> > +\t\tgoto out;\n> > +\t}\n> \n> What exactly are we guarding against here? Don't we already verify that\n> files in `refs/` have the correct type? Or are we checking that it does\n> not point to a directory?\n> \n\nWhen scanning the \"refs\" directory, we will check the file in the ref\ndatabase, but we ignore the directory. So we are checking to know\nwhether it does not point to a directory. If the ref points to a bad\nfile type for example \"ref/heads/bad-file\"\n\nIf it is a block type file. We will first report that \"refs/heads/bad-file\"\nis a bad file and then report ref points to bad file\n\"refs/heads/bad-file\".\n\nActually, I think this is a little redundant here, but we can be\ntolerant about this because we need to guard against directory. We need\nto consider this situation.\n\nSo we could let this be.\n\n> Patrick\n"},{"id":"501546","messageId":"xmqq1q2gtu19.fsf@gitster.g","threadId":"61943","inReplyTo":"Zsb64NTyqc_oHNFO@tanuki","subject":"Re: [PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-22T16:13:38Z","receivedAt":"2024-08-22T16:13:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> So any reference that contains additional data is not a proper ref and\n> thus should be warned about from my point of view. No Git tooling should\n> write them, so if something does it's a red flag to me.\n\nIf you find such a file in $GIT_DIR/refs/ hierarchy, because our\nconsumer side has been looser than necessary forever, and we never\nhave written such a file ourselves, it is a sign that a third-party\ntool wrote it, and that the third-party tool used our reader\nimplementation as the specification. That is why I am hesitant to\nretroactively tighten the rules like this patch does.\n\nThanks.\n\n"},{"id":"501547","messageId":"xmqqwmk8sfaz.fsf@gitster.g","threadId":"61943","inReplyTo":"xmqq1q2gtu19.fsf@gitster.g","subject":"Re: [PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-22T16:17:08Z","receivedAt":"2024-08-22T16:17:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Patrick Steinhardt writes:\n>\n>> So any reference that contains additional data is not a proper ref and\n>> thus should be warned about from my point of view. No Git tooling should\n>> write them, so if something does it's a red flag to me.\n>\n> If you find such a file in $GIT_DIR/refs/ hierarchy, because our\n> consumer side has been looser than necessary forever, and we never\n> have written such a file ourselves, it is a sign that a third-party\n> tool wrote it, and that the third-party tool used our reader\n> implementation as the specification. That is why I am hesitant to\n> retroactively tighten the rules like this patch does.\n\nI forgot to add my recommended course of action, without which a\nreview is worth much less X-<.\n\nI am OK if we tightened the rules retroactively, as long as it\nstarts as a probing check (i.e. \"info: we found an unusual thing\nin the wild. Please report this to us so that we can ask you for\nmore details like how such a ref that would violate a rule that was\nretroactively tightened got there\", not \"error: malformed ref\").\n\nThanks.\n"},{"id":"501572","messageId":"ZsgfyogIFC1ECN32@tanuki","threadId":"61943","inReplyTo":"ZscyGg8M8TbJVKNS@ArchLinux","subject":"Re: [PATCH v1 3/4] ref: add symbolic ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-23T05:36:10Z","receivedAt":"2024-08-23T05:36:17Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Aug 22, 2024 at 08:42:02PM +0800, shejialuo wrote:\n> On Thu, Aug 22, 2024 at 10:53:57AM +0200, Patrick Steinhardt wrote:\n> \n> \n> > > +\t\tif ((space_num || newline_num) && !isspace(*p)) {\n> > > +\t\t\tret = fsck_report_ref(o, report,\n> > > +\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> > > +\t\t\t\t\t \"contains non-null garbage\");\n> > > +\t\t\tgoto out;\n> > > +\t\t}\n> > > +\n> > > +\t\tif (*p == '\\n') {\n> > > +\t\t\tnewline_num++;\n> > > +\t\t} else if (*p == ' ') {\n> > > +\t\t\tspace_num++;\n> > > +\t\t}\n> > > +\t\tp++;\n> > > +\t}\n> > \n> > Can't we replace this with a single `strchr('\\n')` call to check for the\n> > newline and then verify that the next character is a `\\0`? The check for\n> > spaces would then be handled by `check_refname_format()`.\n> > \n> \n> We cannot. Think about this situation.\n> \n> \"ref: refs/heads/master \\n \"\n> \n> We find that the next character of '\\n' is not '\\0'. Then we leave it to\n> \"check_refname_format\". But \"check_refname_format\" will report an error\n> here, but this is an allowed symref.\n\nWouldn't it be correct to warn about this? To me the above very much\nlooks like garbage after the refname, same like we'd also warn about\nsuch garbage for direct refs.\n\nPatrick\n"},{"id":"501574","messageId":"Zsg4ZLHFD4nzISc6@tanuki","threadId":"61943","inReplyTo":"xmqqwmk8sfaz.fsf@gitster.g","subject":"Re: [PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-23T07:21:14Z","receivedAt":"2024-08-23T07:21:20Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Aug 22, 2024 at 09:17:08AM -0700, Junio C Hamano wrote:\n> Junio C Hamano writes:\n> \n> > Patrick Steinhardt writes:\n> >\n> >> So any reference that contains additional data is not a proper ref and\n> >> thus should be warned about from my point of view. No Git tooling should\n> >> write them, so if something does it's a red flag to me.\n> >\n> > If you find such a file in $GIT_DIR/refs/ hierarchy, because our\n> > consumer side has been looser than necessary forever, and we never\n> > have written such a file ourselves, it is a sign that a third-party\n> > tool wrote it, and that the third-party tool used our reader\n> > implementation as the specification. That is why I am hesitant to\n> > retroactively tighten the rules like this patch does.\n> \n> I forgot to add my recommended course of action, without which a\n> review is worth much less X-<.\n> \n> I am OK if we tightened the rules retroactively, as long as it\n> starts as a probing check (i.e. \"info: we found an unusual thing\n> in the wild. Please report this to us so that we can ask you for\n> more details like how such a ref that would violate a rule that was\n> retroactively tightened got there\", not \"error: malformed ref\").\n\nOkay, that makes sense. The fsck infrastructure does have info message\ntypes, so this should certainly be doable. I'd argue that we might want\nto make this an `FSCK_WARN`, but I'm also fine with iteratively bumping\nup the severity from INFO to WARN to ERROR when we don't observe any\ncomplaints about this tightening.\n\nPatrick\n"},{"id":"501580","messageId":"ZshyuH55IP_a55z0@ArchLinux","threadId":"61943","inReplyTo":"Zsg4ZLHFD4nzISc6@tanuki","subject":"Re: [PATCH v1 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-23T11:30:00Z","receivedAt":"2024-08-23T11:29:15Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Aug 23, 2024 at 09:21:14AM +0200, Patrick Steinhardt wrote:\n> On Thu, Aug 22, 2024 at 09:17:08AM -0700, Junio C Hamano wrote:\n> > Junio C Hamano writes:\n> > \n> > > Patrick Steinhardt writes:\n> > >\n> > >> So any reference that contains additional data is not a proper ref and\n> > >> thus should be warned about from my point of view. No Git tooling should\n> > >> write them, so if something does it's a red flag to me.\n> > >\n> > > If you find such a file in $GIT_DIR/refs/ hierarchy, because our\n> > > consumer side has been looser than necessary forever, and we never\n> > > have written such a file ourselves, it is a sign that a third-party\n> > > tool wrote it, and that the third-party tool used our reader\n> > > implementation as the specification. That is why I am hesitant to\n> > > retroactively tighten the rules like this patch does.\n> > \n> > I forgot to add my recommended course of action, without which a\n> > review is worth much less X-<.\n> > \n> > I am OK if we tightened the rules retroactively, as long as it\n> > starts as a probing check (i.e. \"info: we found an unusual thing\n> > in the wild. Please report this to us so that we can ask you for\n> > more details like how such a ref that would violate a rule that was\n> > retroactively tightened got there\", not \"error: malformed ref\").\n> \n> Okay, that makes sense. The fsck infrastructure does have info message\n> types, so this should certainly be doable. I'd argue that we might want\n> to make this an `FSCK_WARN`, but I'm also fine with iteratively bumping\n> up the severity from INFO to WARN to ERROR when we don't observe any\n> complaints about this tightening.\n> \n\nFrom the perspective of the implementation, there is no difference\nbetween the info and warn. But I have a doubt here. Do we really\ndistinguish the info and warn in code?\n\nLet's see the \"fsck_vreport\" (although this is a new function, but I\nnever change the implementation) function:\n\n static int fsck_vreport(...)\n {\n enum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);\n\n if (msg_type == FSCK_FATAL)\n msg_type = FSCK_ERROR;\n if (msg_type == FSCK_INFO)\n msg_type = FSCK_WARN;\n\n ...\n }\n\nWe eventually convert the \"FSCK_INFO\" to \"FSCK_WARN\". Confusing.\n"},{"id":"501581","messageId":"Zsh0YZDHQPfdkzhd@ArchLinux","threadId":"61943","inReplyTo":"ZsgfyogIFC1ECN32@tanuki","subject":"Re: [PATCH v1 3/4] ref: add symbolic ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-23T11:37:05Z","receivedAt":"2024-08-23T11:36:20Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Aug 23, 2024 at 07:36:10AM +0200, Patrick Steinhardt wrote:\n> On Thu, Aug 22, 2024 at 08:42:02PM +0800, shejialuo wrote:\n> > On Thu, Aug 22, 2024 at 10:53:57AM +0200, Patrick Steinhardt wrote:\n> > \n> > \n> > > > +\t\tif ((space_num || newline_num) && !isspace(*p)) {\n> > > > +\t\t\tret = fsck_report_ref(o, report,\n> > > > +\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> > > > +\t\t\t\t\t \"contains non-null garbage\");\n> > > > +\t\t\tgoto out;\n> > > > +\t\t}\n> > > > +\n> > > > +\t\tif (*p == '\\n') {\n> > > > +\t\t\tnewline_num++;\n> > > > +\t\t} else if (*p == ' ') {\n> > > > +\t\t\tspace_num++;\n> > > > +\t\t}\n> > > > +\t\tp++;\n> > > > +\t}\n> > > \n> > > Can't we replace this with a single `strchr('\\n')` call to check for the\n> > > newline and then verify that the next character is a `\\0`? The check for\n> > > spaces would then be handled by `check_refname_format()`.\n> > > \n> > \n> > We cannot. Think about this situation.\n> > \n> > \"ref: refs/heads/master \\n \"\n> > \n> > We find that the next character of '\\n' is not '\\0'. Then we leave it to\n> > \"check_refname_format\". But \"check_refname_format\" will report an error\n> > here, but this is an allowed symref.\n> \n> Wouldn't it be correct to warn about this? To me the above very much\n> looks like garbage after the refname, same like we'd also warn about\n> such garbage for direct refs.\n> \n\nYes, we should warn about this. But only null-garbage is allowed for\nsymref. The following situation is bad:\n\n \"ref: refs/heads/master \\n garbage\\n\"\n\nWe should report error here, from my perspective, it's a FATAL ERROR.\nHowever, let's decide how to do this when we know what fsck error level\nwe should set.\n\n> Patrick\n"},{"id":"501719","messageId":"Zs348uXMBdCuwF-2@ArchLinux","threadId":"61943","inReplyTo":"ZsIMc6cJ-kzMzW_8@ArchLinux","subject":"[PATCH v2 0/4] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-27T16:04:02Z","receivedAt":"2024-08-27T16:03:13Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis new version handles the following reviews:\n\n1. According to the advice from the Junio, we should just use \"{0}\" to\ninitialize the zero structure \"fsck_ref_report\". This version handles\nthis in [PATCH v2 1/4].\n2. According to the advice from the Patrick, use \"strrchr\" instead of\nlooping to make the code more clean in [PATCH v2 3/4].\n3. Use \"goto\" to remove ident.\n\nHowever, the most important thing for this patch is which fsck message\ntype I choose. I have recorded the reason in the commit message. But I\nwanna explain the motivation in cover letter for making the reviewers\neasy to understand.\n\nActually, in the review process of the first version. Junio thought we\nshould use \"FSCK_INFO\" and Patrick thought we should use \"FSCK_WARN\".\nAnd I raised a question here, what is the difference between the\n\"FSCK_INFO\" and \"FSCK_WARN\" because in \"fsck.c::fsck_vreport\" function,\nwe will convert \"FSCK_INFO\" to \"FSCK_WARN\" like the following:\n\n static int fsck_vreport(...)\n {\n enum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);\n\n if (msg_type == FSCK_FATAL)\n msg_type = FSCK_ERROR;\n if (msg_type == FSCK_INFO)\n msg_type = FSCK_WARN;\n ...\n }\n\nAnd I have gone back to the history. Actually the first time the fsck\nmessage type was set up at f27d05b170 (fsck: allow upgrading fsck\nwarnings to errors, 2015-06-22):\n\n https://lore.kernel.org/git/cover.1418055173.git.johannes.schindelin@gmx.de/\n\nAnd I have understood why we need \"FSCK_INFO\". This is because when\nsetting the \"strict\" filed in \"fsck_options\", all the fsck warns will\nbecome fsck errors. For example, this change verifies my thinking:\n4dd3b045f5 (fsck: downgrade tree badFilemode to \"info\", 2022-08-10).\n\nAs you can see, this restriction makes the code safer. So, I agree with\nJunio, at now, we should use \"FSCK_INFO\" for trailing garbage and ref\ncontent ends without newline.\n\nBut we should report fsck errors for the following two situations for\n\"git-fsck(1)\" will report fsck errors by implicitly checking the ref\ndatabase consistency.\n\n1. \"parse_loose_ref_contents\" fail.\n2. symref content is bad (cannot parse).\n\nThanks,\nJialuo\n\nshejialuo (4):\n ref: initialize \"fsck_ref_report\" with zero\n ref: add regular ref content check for files backend\n ref: add symbolic ref content check for files backend\n ref: add symlink ref check for files backend\n\n Documentation/fsck-msgids.txt | 12 +++\n fsck.h | 4 +\n refs.c | 2 +-\n refs/files-backend.c | 179 +++++++++++++++++++++++++++++++-\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 185 ++++++++++++++++++++++++++++++++++\n 6 files changed, 379 insertions(+), 5 deletions(-)\n\nRange-diff against v1:\n1: 9ed3026ac5 ! 1: 0367904c81 fsck: introduce \"FSCK_REF_REPORT_DEFAULT\" macro\n @@ Metadata\n Author: shejialuo \n \n ## Commit message ##\n - fsck: introduce \"FSCK_REF_REPORT_DEFAULT\" macro\n + ref: initialize \"fsck_ref_report\" with zero\n \n In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n \"referent\" is NULL. So, we need to always initialize these parameters to\n NULL instead of letting them point to anywhere when creating a new\n \"fsck_ref_report\" structure.\n \n - In order to conveniently create a new \"fsck_ref_report\", add a new macro\n - \"FSCK_REF_REPORT_DEFAULT\".\n + The original code explicitly specifies the \".path\" field to initialize\n + the \"fsck_ref_report\" structure. However, it introduces confusion how we\n + initialize the other fields. In order to avoid this, initialize the\n + \"fsck_ref_report\" with zero to make clear that everything in\n + \"fsck_ref_report\" is zero initialized.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n Signed-off-by: shejialuo \n \n - ## fsck.h ##\n -@@ fsck.h: struct fsck_ref_report {\n - \tconst char *referent;\n - };\n - \n -+#define FSCK_REF_REPORT_DEFAULT { \\\n -+\t.path = NULL, \\\n -+\t.oid = NULL, \\\n -+\t.referent = NULL, \\\n -+}\n -+\n - struct fsck_options {\n - \tfsck_walk_func walk;\n - \tfsck_error error_func;\n -\n ## refs/files-backend.c ##\n @@ refs/files-backend.c: static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n -\t\tstruct fsck_ref_report report = { .path = NULL };\n -+\t\tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n ++\t\tstruct fsck_ref_report report = {0};\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n2: 714284cf2b ! 2: 7b6f4145cd ref: add regular ref content check for files backend\n @@ Metadata\n ## Commit message ##\n ref: add regular ref content check for files backend\n \n - We implicitly reply on \"git-fsck(1)\" to check the consistency of regular\n - refs. However, when parsing the regular refs for files backend, we allow\n - the ref content to end with no newline or contain some garbages. We\n - should warn the user about above situations.\n + We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\n + refs. However, when parsing the regular refs for files backend by using\n + \"files-backend.c::parse_loose_ref_contents\", we allow the ref content to\n + be end with no newline or contain some garbages.\n \n - In order to provide above functionality, enhance the \"git-refs verify\"\n - command by adding consistency check for regular refs for files backend.\n + It may seem that we should report an error or warn fsck message to the\n + user about above situations. However, there may be some third-party\n + tools customizing the content of refs. We should not report an error\n + fsck message.\n \n - Add the following three fsck messages to represent the above situations:\n + And we cannot either report a warn fsck message to the user. This is\n + because for \"git-receive-pack(1)\" and \"git-fetch-pack(1)\", they will\n + parse the fsck message type and check the message type by\n + \"fsck.c::is_valid_msg_type\". Only the fsck infos are not valid. If we\n + make the fsck message type to be warn, the user could upgrade the fsck\n + warnings to errors. And the user can also set the \"strict\" field in\n + \"fsck_options\" to upgrade the fsck warnings to errors.\n \n - 1. \"badRefContent(ERROR)\": A ref has a bad content.\n - 2. \"refMissingNewline(WARN)\": A valid ref does not end with newline.\n - 3. \"trailingRefContent(WARN)\": A ref has trailing contents.\n + We should not allow the user to upgrade the fsck warnings to errors. It\n + might cause compatibility issue which will break the legacy repository.\n + So we add the following two fsck infos to represent the situation where\n + the ref content ends without newline or has garbages:\n +\n + 1. \"refMissingNewline(INFO)\": A valid ref does not end with newline.\n + 2. \"trailingRefContent(INFO)\": A ref has trailing contents.\n +\n + In \"fsck.c::fsck_vreport\", we will convert \"FSCK_INFO\" to \"FSCK_WARN\",\n + and we can still warn the user about these situations when using\n + \"git-refs verify\" without introducing compatibility issue.\n +\n + In current \"git-fsck(1)\", it will report an error when the ref content\n + is bad, so we should following this to report an error to the user when\n + \"parse_loose_ref_contents\" fails. And we add a new fsck error message\n + called \"badRefContent(ERROR)\" to represent that a ref has a bad content.\n \n In order to tell whether the ref has trailing content, add a new\n parameter \"trailing\" to \"parse_loose_ref_contents\". Then introduce a new\n - function \"files_fsck_refs_content\" to check the regular refs.\n + function \"files_fsck_refs_content\" to check the regular refs to enhance\n + the \"git-refs verify\".\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n @@ Documentation/fsck-msgids.txt\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n +`refMissingNewline`::\n -+\t(WARN) A valid ref does not end with newline.\n ++\t(INFO) A valid ref does not end with newline.\n +\n +`trailingRefContent`::\n -+\t(WARN) A ref has trailing contents.\n ++\t(INFO) A ref has trailing contents.\n +\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n @@ fsck.h: enum fsck_msg_type {\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n @@ fsck.h: enum fsck_msg_type {\n - \tFUNC(HAS_DOTDOT, WARN) \\\n - \tFUNC(HAS_DOTGIT, WARN) \\\n - \tFUNC(NULL_SHA1, WARN) \\\n -+\tFUNC(REF_MISSING_NEWLINE, WARN) \\\n -+\tFUNC(TRAILING_REF_CONTENT, WARN) \\\n - \tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n - \tFUNC(NUL_IN_COMMIT, WARN) \\\n - \tFUNC(LARGE_PATHNAME, WARN) \\\n + \tFUNC(MAILMAP_SYMLINK, INFO) \\\n + \tFUNC(BAD_TAG_NAME, INFO) \\\n + \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n ++\tFUNC(REF_MISSING_NEWLINE, INFO) \\\n ++\tFUNC(TRAILING_REF_CONTENT, INFO) \\\n + \t/* ignored (elevated when requested) */ \\\n + \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n + \n \n ## refs.c ##\n @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\t\t\t\t const char *refs_check_dir,\n +\t\t\t\t struct dir_iterator *iter)\n +{\n -+\tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n +\tstruct strbuf ref_content = STRBUF_INIT;\n +\tstruct strbuf referent = STRBUF_INIT;\n +\tstruct strbuf refname = STRBUF_INIT;\n ++\tstruct fsck_ref_report report = {0};\n +\tconst char *trailing = NULL;\n +\tunsigned int type = 0;\n +\tint failure_errno = 0;\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\t\t}\n +\n +\t\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n -+\t\t\t\t\t ref_content.buf, &oid, &referent,\n -+\t\t\t\t\t &type, &trailing, &failure_errno)) {\n ++\t\t\t\t\t ref_content.buf, &oid, &referent,\n ++\t\t\t\t\t &type, &trailing, &failure_errno)) {\n +\t\t\tret = fsck_report_ref(o, &report,\n +\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n +\t\t\t\t\t \"invalid ref content\");\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\t\t\t\tgoto cleanup;\n +\t\t\t}\n +\t\t}\n ++\t\tgoto cleanup;\n +\t}\n +\n +cleanup:\n3: 032b0d6a64 ! 3: 20d8556902 ref: add symbolic ref content check for files backend\n @@ Commit message\n 3. \"ref: refs/heads/master\\n\\n\"\n \n But we do not allow any non-null trailing garbage. The following are bad\n - symbolic contents.\n + symbolic contents which will be reported as fsck error by \"git-fsck(1)\".\n \n 1. \"ref: refs/heads/master garbage\\n\"\n 2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n \n - In order to provide above checks, we will traverse the \"pointee\" to\n - report the user whether this is null-garbage or no newline. And if\n - symbolic refs contain non-null garbage, we will report\n - \"FSCK_MSG_BAD_REF_CONTENT\" to the user.\n -\n - Then, we will check the name of the \"pointee\" is correct by using\n - \"check_refname_format\". And then if we can access the \"pointee_path\" in\n - the file system, we should ensure that the file type is correct.\n + In order to provide above checks, we will use \"strrchr\" to check whether\n + we have newline in the ref content. Then we will check the name of the\n + \"pointee\" is correct by using \"check_refname_format\". If the function\n + fails, we need to trim the \"pointee\" to see whether the null-garbage\n + causes the function fails. If so, we need to report that there is\n + null-garbage in the symref content. Otherwise, we should report the user\n + the \"pointee\" is bad.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\t\t\t\t struct strbuf *pointee_name,\n +\t\t\t\t struct strbuf *pointee_path)\n +{\n -+\tunsigned int newline_num = 0;\n -+\tunsigned int space_num = 0;\n ++\tconst char *newline_pos = NULL;\n +\tconst char *p = NULL;\n +\tstruct stat st;\n +\tint ret = 0;\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\t\tgoto out;\n +\t}\n +\n -+\twhile (*p != '\\0') {\n -+\t\tif ((space_num || newline_num) && !isspace(*p)) {\n -+\t\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n -+\t\t\t\t\t \"contains non-null garbage\");\n -+\t\t\tgoto out;\n -+\t\t}\n -+\n -+\t\tif (*p == '\\n') {\n -+\t\t\tnewline_num++;\n -+\t\t} else if (*p == ' ') {\n -+\t\t\tspace_num++;\n -+\t\t}\n -+\t\tp++;\n -+\t}\n -+\n -+\tif (space_num || newline_num > 1) {\n -+\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n -+\t\t\t\t \"trailing null-garbage\");\n -+\t} else if (!newline_num) {\n ++\tnewline_pos = strrchr(p, '\\n');\n ++\tif (!newline_pos || *(newline_pos + 1)) {\n +\t\tret = fsck_report_ref(o, report,\n +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n +\t\t\t\t \"missing newline\");\n +\t}\n +\n -+\tstrbuf_rtrim(pointee_name);\n -+\n +\tif (check_refname_format(pointee_name->buf, 0)) {\n ++\t\t/*\n ++\t\t * When containing null-garbage, \"check_refname_format\" will\n ++\t\t * fail, we should trim the \"pointee\" to check again.\n ++\t\t */\n ++\t\tstrbuf_rtrim(pointee_name);\n ++\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n ++\t\t\tret = fsck_report_ref(o, report,\n ++\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n ++\t\t\t\t\t \"trailing null-garbage\");\n ++\t\t\tgoto out;\n ++\t\t}\n ++\n +\t\tret = fsck_report_ref(o, report,\n +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n +\t\t\t\t \"points to refname with invalid format\");\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter)\n {\n - \tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n +\tstruct strbuf pointee_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_s\n +\t\t\t\t\t\t &referent,\n +\t\t\t\t\t\t &pointee_path);\n \t\t}\n + \t\tgoto cleanup;\n \t}\n - \n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be che\n +\tprintf \"ref: refs/heads/branch \" > $branch_dir_prefix/a/b/branch-trailing &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n ++\twarning: refs/heads/a/b/branch-trailing: refMissingNewline: missing newline\n +\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n +\tEOF\n +\trm $branch_dir_prefix/a/b/branch-trailing &&\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be che\n +\tprintf \"ref: refs/heads/branch \\n\\n \" > $branch_dir_prefix/a/b/branch-trailing &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n ++\twarning: refs/heads/a/b/branch-trailing: refMissingNewline: missing newline\n +\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n +\tEOF\n +\trm $branch_dir_prefix/a/b/branch-trailing &&\n4: 147a873958 ! 4: d9867c5f87 ref: add symlink ref consistency check for files backend\n @@ Metadata\n Author: shejialuo \n \n ## Commit message ##\n - ref: add symlink ref consistency check for files backend\n + ref: add symlink ref check for files backend\n \n We have already introduced \"files_fsck_symref_target\". We should reuse\n this function to handle the symrefs which are legacy symbolic links. We\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\t\t\t\t struct strbuf *pointee_path,\n +\t\t\t\t unsigned int symbolic_link)\n {\n - \tunsigned int newline_num = 0;\n - \tunsigned int space_num = 0;\n + \tconst char *newline_pos = NULL;\n + \tconst char *p = NULL;\n @@ refs/files-backend.c: static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n --\twhile (*p != '\\0') {\n --\t\tif ((space_num || newline_num) && !isspace(*p)) {\n --\t\t\tret = fsck_report_ref(o, report,\n --\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n --\t\t\t\t\t \"contains non-null garbage\");\n --\t\t\tgoto out;\n +-\tnewline_pos = strrchr(p, '\\n');\n +-\tif (!newline_pos || *(newline_pos + 1)) {\n +-\t\tret = fsck_report_ref(o, report,\n +-\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n +-\t\t\t\t \"missing newline\");\n +\tif (!symbolic_link) {\n -+\t\twhile (*p != '\\0') {\n -+\t\t\tif ((space_num || newline_num) && !isspace(*p)) {\n -+\t\t\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n -+\t\t\t\t\t\t \"contains non-null garbage\");\n -+\t\t\t\tgoto out;\n -+\t\t\t}\n -+\n -+\t\t\tif (*p == '\\n') {\n -+\t\t\t\tnewline_num++;\n -+\t\t\t} else if (*p == ' ') {\n -+\t\t\t\tspace_num++;\n -+\t\t\t}\n -+\t\t\tp++;\n - \t\t}\n - \n --\t\tif (*p == '\\n') {\n --\t\t\tnewline_num++;\n --\t\t} else if (*p == ' ') {\n --\t\t\tspace_num++;\n -+\t\tif (space_num || newline_num > 1) {\n -+\t\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n -+\t\t\t\t\t \"trailing null-garbage\");\n -+\t\t} else if (!newline_num) {\n ++\t\tnewline_pos = strrchr(p, '\\n');\n ++\t\tif (!newline_pos || *(newline_pos + 1)) {\n +\t\t\tret = fsck_report_ref(o, report,\n +\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n +\t\t\t\t\t \"missing newline\");\n - \t\t}\n --\t\tp++;\n --\t}\n - \n --\tif (space_num || newline_num > 1) {\n --\t\tret = fsck_report_ref(o, report,\n --\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n --\t\t\t\t \"trailing null-garbage\");\n --\t} else if (!newline_num) {\n --\t\tret = fsck_report_ref(o, report,\n --\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n --\t\t\t\t \"missing newline\");\n -+\t\tstrbuf_rtrim(pointee_name);\n ++\t\t}\n \t}\n \n --\tstrbuf_rtrim(pointee_name);\n --\n \tif (check_refname_format(pointee_name->buf, 0)) {\n +-\t\t/*\n +-\t\t * When containing null-garbage, \"check_refname_format\" will\n +-\t\t * fail, we should trim the \"pointee\" to check again.\n +-\t\t */\n +-\t\tstrbuf_rtrim(pointee_name);\n +-\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n +-\t\t\tret = fsck_report_ref(o, report,\n +-\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n +-\t\t\t\t\t \"trailing null-garbage\");\n +-\t\t\tgoto out;\n ++\t\tif (!symbolic_link) {\n ++\t\t\t/*\n ++\t\t\t* When containing null-garbage, \"check_refname_format\" will\n ++\t\t\t* fail, we should trim the \"pointee\" to check again.\n ++\t\t\t*/\n ++\t\t\tstrbuf_rtrim(pointee_name);\n ++\t\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n ++\t\t\t\tret = fsck_report_ref(o, report,\n ++\t\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n ++\t\t\t\t\t\t \"trailing null-garbage\");\n ++\t\t\t\tgoto out;\n ++\t\t\t}\n + \t\t}\n + \n \t\tret = fsck_report_ref(o, report,\n - \t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n - \tstruct fsck_ref_report report = FSCK_REF_REPORT_DEFAULT;\n + {\n \tstruct strbuf pointee_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n +\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n + \tstruct fsck_ref_report report = {0};\n ++\tconst char *pointee_name = NULL;\n +\tunsigned int symbolic_link = 0;\n \tconst char *trailing = NULL;\n \tunsigned int type = 0;\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_s\n -\t\t\t\t\t\t &pointee_path);\n +\t\t\t\t\t\t &pointee_path,\n +\t\t\t\t\t\t symbolic_link);\n -+\t\t}\n -+\t} else if (S_ISLNK(iter->st.st_mode)) {\n -+\t\tconst char *pointee_name = NULL;\n + \t\t}\n + \t\tgoto cleanup;\n + \t}\n + \n ++\tsymbolic_link = 1;\n +\n -+\t\tsymbolic_link = 1;\n ++\tstrbuf_add_real_path(&pointee_path, iter->path.buf);\n ++\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n ++\tstrbuf_normalize_path(&abs_gitdir);\n ++\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n ++\t\tstrbuf_addch(&abs_gitdir, '/');\n +\n -+\t\tstrbuf_add_real_path(&pointee_path, iter->path.buf);\n -+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n -+\t\tstrbuf_normalize_path(&abs_gitdir);\n -+\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n -+\t\t\tstrbuf_addch(&abs_gitdir, '/');\n ++\tif (!skip_prefix(pointee_path.buf, abs_gitdir.buf, &pointee_name)) {\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n ++\t\t\t\t \"point to target outside gitdir\");\n ++\t\tgoto cleanup;\n ++\t}\n +\n -+\t\tif (!skip_prefix(pointee_path.buf,\n -+\t\t\t\t abs_gitdir.buf, &pointee_name)) {\n -+\t\t\tret = fsck_report_ref(o, &report,\n -+\t\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n -+\t\t\t\t\t \"point to target outside gitdir\");\n -+\t\t\tgoto cleanup;\n - \t\t}\n ++\tstrbuf_addstr(&referent, pointee_name);\n ++\tret = files_fsck_symref_target(o, &report, refname.buf,\n ++\t\t\t\t &referent, &pointee_path,\n ++\t\t\t\t symbolic_link);\n +\n -+\t\tstrbuf_addstr(&referent, pointee_name);\n -+\t\tret = files_fsck_symref_target(o, &report, refname.buf,\n -+\t\t\t\t\t &referent, &pointee_path,\n -+\t\t\t\t\t symbolic_link);\n - \t}\n - \n cleanup:\n -@@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n + \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n \tstrbuf_release(&pointee_path);\n-- \n2.46.0\n\n"},{"id":"501720","messageId":"Zs351iV2HbdhNvEz@ArchLinux","threadId":"61943","inReplyTo":"Zs348uXMBdCuwF-2@ArchLinux","subject":"[PATCH v2 1/4] ref: initialize \"fsck_ref_report\" with zero","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-27T16:07:50Z","receivedAt":"2024-08-27T16:07:00Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n\"referent\" is NULL. So, we need to always initialize these parameters to\nNULL instead of letting them point to anywhere when creating a new\n\"fsck_ref_report\" structure.\n\nThe original code explicitly specifies the \".path\" field to initialize\nthe \"fsck_ref_report\" structure. However, it introduces confusion how we\ninitialize the other fields. In order to avoid this, initialize the\n\"fsck_ref_report\" with zero to make clear that everything in\n\"fsck_ref_report\" is zero initialized.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8d6ec9458d..d6fc3bd67c 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3446,7 +3446,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tstruct fsck_ref_report report = { .path = NULL };\n+\t\tstruct fsck_ref_report report = {0};\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n-- \n2.46.0\n\n"},{"id":"501721","messageId":"Zs353oLDaw2SbNQs@ArchLinux","threadId":"61943","inReplyTo":"Zs348uXMBdCuwF-2@ArchLinux","subject":"[PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-27T16:07:58Z","receivedAt":"2024-08-27T16:07:09Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\nrefs. However, when parsing the regular refs for files backend by using\n\"files-backend.c::parse_loose_ref_contents\", we allow the ref content to\nbe end with no newline or contain some garbages.\n\nIt may seem that we should report an error or warn fsck message to the\nuser about above situations. However, there may be some third-party\ntools customizing the content of refs. We should not report an error\nfsck message.\n\nAnd we cannot either report a warn fsck message to the user. This is\nbecause if the caller set the \"strict\" field in \"fsck_options\" to\nto upgrade the fsck warnings to errors.\n\nWe should not allow the user to upgrade the fsck warnings to errors. It\nmight cause compatibility issue which will break the legacy repository.\nSo we add the following two fsck infos to represent the situation where\nthe ref content ends without newline or has garbages:\n\n1. \"refMissingNewline(INFO)\": A valid ref does not end with newline.\n2. \"trailingRefContent(INFO)\": A ref has trailing contents.\n\nIn \"fsck.c::fsck_vreport\", we will convert \"FSCK_INFO\" to \"FSCK_WARN\",\nand we can still warn the user about these situations when using\n\"git-refs verify\" without introducing compatibility issue.\n\nIn current \"git-fsck(1)\", it will report an error when the ref content\nis bad, so we should following this to report an error to the user when\n\"parse_loose_ref_contents\" fails. And we add a new fsck error message\ncalled \"badRefContent(ERROR)\" to represent that a ref has a bad content.\n\nIn order to tell whether the ref has trailing content, add a new\nparameter \"trailing\" to \"parse_loose_ref_contents\". Then introduce a new\nfunction \"files_fsck_refs_content\" to check the regular refs to enhance\nthe \"git-refs verify\".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 9 ++++\n fsck.h | 3 ++\n refs.c | 2 +-\n refs/files-backend.c | 68 ++++++++++++++++++++++++++-\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 87 +++++++++++++++++++++++++++++++++++\n 6 files changed, 167 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 68a2801f15..fc074fc571 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -19,6 +19,9 @@\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n+`badRefContent`::\n+\t(ERROR) A ref has a bad content.\n+\n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n \n@@ -170,6 +173,12 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`refMissingNewline`::\n+\t(INFO) A valid ref does not end with newline.\n+\n+`trailingRefContent`::\n+\t(INFO) A ref has trailing contents.\n+\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \ndiff --git a/fsck.h b/fsck.h\nindex 500b4c04d2..b85072df57 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -31,6 +31,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n@@ -84,6 +85,8 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs.c b/refs.c\nindex 74de3d3009..5e74881945 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1758,7 +1758,7 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \t}\n \n \tresult = parse_loose_ref_contents(ref_store->repo->hash_algo, content.buf,\n-\t\t\t\t\t oid, referent, type, failure_errno);\n+\t\t\t\t\t oid, referent, type, NULL, failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex d6fc3bd67c..69c00073eb 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -560,7 +560,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname,\n \tbuf = sb_contents.buf;\n \n \tret = parse_loose_ref_contents(ref_store->repo->hash_algo, buf,\n-\t\t\t\t oid, referent, type, &myerr);\n+\t\t\t\t oid, referent, type, NULL, &myerr);\n \n out:\n \tif (ret && !myerr)\n@@ -597,7 +597,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno)\n+\t\t\t const char **trailing, int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -619,6 +619,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n+\n+\tif (trailing)\n+\t\t*trailing = p;\n+\n \treturn 0;\n }\n \n@@ -3430,6 +3434,65 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_refs_content(struct ref_store *ref_store,\n+\t\t\t\t struct fsck_options *o,\n+\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t struct dir_iterator *iter)\n+{\n+\tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tstruct strbuf refname = STRBUF_INIT;\n+\tstruct fsck_ref_report report = {0};\n+\tconst char *trailing = NULL;\n+\tunsigned int type = 0;\n+\tint failure_errno = 0;\n+\tstruct object_id oid;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n+\treport.path = refname.buf;\n+\n+\tif (S_ISREG(iter->st.st_mode)) {\n+\t\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n+\t\t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n+\t\t\t\t\t refs_check_dir, iter->relative_path);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n+\t\t\t\t\t ref_content.buf, &oid, &referent,\n+\t\t\t\t\t &type, &trailing, &failure_errno)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t\t \"invalid ref content\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tif (!(type & REF_ISSYMREF)) {\n+\t\t\tif (*trailing == '\\0') {\n+\t\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t\t \"missing newline\");\n+\t\t\t\tgoto cleanup;\n+\t\t\t}\n+\n+\t\t\tif (*trailing != '\\n' || (*(trailing + 1) != '\\0')) {\n+\t\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t\t \"trailing garbage in ref\");\n+\t\t\t\tgoto cleanup;\n+\t\t\t}\n+\t\t}\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&refname);\n+\tstrbuf_release(&ref_content);\n+\tstrbuf_release(&referent);\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n \t\t\t\tconst char *refs_check_dir,\n@@ -3512,6 +3575,7 @@ static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n+\t\tfiles_fsck_refs_content,\n \t\tNULL,\n \t};\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 2313c830d8..73b05f971b 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -715,7 +715,7 @@ struct ref_store {\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno);\n+\t\t\t const char **trailing, int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 71a4d1a5ae..7c1910d784 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -89,4 +89,91 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'regular ref content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\tgit commit --allow-empty -m initial &&\n+\tgit checkout -b branch-1 &&\n+\tgit tag tag-1 &&\n+\tgit commit --allow-empty -m second &&\n+\tgit checkout -b branch-2 &&\n+\tgit tag tag-2 &&\n+\tgit checkout -b a/b/tag-2 &&\n+\n+\tprintf \"%s\" \"$(git rev-parse branch-1)\" > $branch_dir_prefix/branch-1-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline\n+\tEOF\n+\trm $branch_dir_prefix/branch-1-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse branch-1)\" > $branch_dir_prefix/branch-1-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-1-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/branch-1-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-1-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-1-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\\n\\na\" \"$(git rev-parse tag-2)\" > $tag_dir_prefix/tag-2-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-2-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-2-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-garbage &&\n+\ttest_must_fail git -c fsck.trailingRefContent=error refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-1-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%sx\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-1-bad: badRefContent: invalid ref content\n+\tEOF\n+\trm $tag_dir_prefix/tag-1-bad &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"xfsazqfxcadas\" > $tag_dir_prefix/tag-2-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-2-bad: badRefContent: invalid ref content\n+\tEOF\n+\trm $tag_dir_prefix/tag-2-bad &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"xfsazqfxcadas\" > $branch_dir_prefix/a/b/branch-2-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-2-bad: badRefContent: invalid ref content\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-2-bad &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"501722","messageId":"Zs3558scHssaG_XS@ArchLinux","threadId":"61943","inReplyTo":"Zs348uXMBdCuwF-2@ArchLinux","subject":"[PATCH v2 3/4] ref: add symbolic ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-27T16:08:07Z","receivedAt":"2024-08-27T16:07:19Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already introduced the checks for regular refs. There is no need\nto check the consistency of the target which the symbolic ref points to.\nInstead, we just check the content of the symbolic ref itself.\n\nIn order to check the content of the symbolic ref, create a function\n\"files_fsck_symref_target\". It will first check whether the \"pointee\" is\nunder the \"refs/\" directory and then we will check the \"pointee\" itself.\n\nThere is no specification about the content of the symbolic ref.\nAlthough we do write \"ref: %s\\n\" to create a symbolic ref by using\n\"git-symbolic-ref(1)\" command. However, this is not mandatory. We still\naccept symbolic refs with null trailing garbage. Put it more specific,\nthe following are correct:\n\n1. \"ref: refs/heads/master \"\n2. \"ref: refs/heads/master \\n \\n\"\n3. \"ref: refs/heads/master\\n\\n\"\n\nBut we do not allow any non-null trailing garbage. The following are bad\nsymbolic contents which will be reported as fsck error by \"git-fsck(1)\".\n\n1. \"ref: refs/heads/master garbage\\n\"\n2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n\nIn order to provide above checks, we will use \"strrchr\" to check whether\nwe have newline in the ref content. Then we will check the name of the\n\"pointee\" is correct by using \"check_refname_format\". If the function\nfails, we need to trim the \"pointee\" to see whether the null-garbage\ncauses the function fails. If so, we need to report that there is\nnull-garbage in the symref content. Otherwise, we should report the user\nthe \"pointee\" is bad.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 ++\n fsck.h | 1 +\n refs/files-backend.c | 77 +++++++++++++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 54 ++++++++++++++++++++++++\n 4 files changed, 135 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex fc074fc571..85fd058c81 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,6 +28,9 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n+`badSymrefPointee`::\n+\t(ERROR) The pointee of a symref is bad.\n+\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n \ndiff --git a/fsck.h b/fsck.h\nindex b85072df57..cbe837f84c 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,6 +34,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n+\tFUNC(BAD_SYMREF_POINTEE, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 69c00073eb..382c73fcf7 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3434,11 +3434,81 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+/*\n+ * Check the symref \"pointee_name\" and \"pointee_path\". The caller should\n+ * make sure that \"pointee_path\" is absolute. For symbolic ref, \"pointee_name\"\n+ * would be the content after \"refs:\".\n+ */\n+static int files_fsck_symref_target(struct fsck_options *o,\n+\t\t\t\t struct fsck_ref_report *report,\n+\t\t\t\t const char *refname,\n+\t\t\t\t struct strbuf *pointee_name,\n+\t\t\t\t struct strbuf *pointee_path)\n+{\n+\tconst char *newline_pos = NULL;\n+\tconst char *p = NULL;\n+\tstruct stat st;\n+\tint ret = 0;\n+\n+\tif (!skip_prefix(pointee_name->buf, \"refs/\", &p)) {\n+\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n+\t\t\t\t \"points to ref outside the refs directory\");\n+\t\tgoto out;\n+\t}\n+\n+\tnewline_pos = strrchr(p, '\\n');\n+\tif (!newline_pos || *(newline_pos + 1)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t \"missing newline\");\n+\t}\n+\n+\tif (check_refname_format(pointee_name->buf, 0)) {\n+\t\t/*\n+\t\t * When containing null-garbage, \"check_refname_format\" will\n+\t\t * fail, we should trim the \"pointee\" to check again.\n+\t\t */\n+\t\tstrbuf_rtrim(pointee_name);\n+\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n+\t\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t \"trailing null-garbage\");\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n+\t\t\t\t \"points to refname with invalid format\");\n+\t}\n+\n+\t/*\n+\t * Missing target should not be treated as any error worthy event and\n+\t * not even warn. It is a common case that a symbolic ref points to a\n+\t * ref that does not exist yet. If the target ref does not exist, just\n+\t * skip the check for the file type.\n+\t */\n+\tif (lstat(pointee_path->buf, &st) < 0)\n+\t\tgoto out;\n+\n+\tif (!S_ISREG(st.st_mode) && !S_ISLNK(st.st_mode)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n+\t\t\t\t \"points to an invalid file type\");\n+\t\tgoto out;\n+\t}\n+\n+out:\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter)\n {\n+\tstruct strbuf pointee_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -3482,6 +3552,12 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t\t\t \"trailing garbage in ref\");\n \t\t\t\tgoto cleanup;\n \t\t\t}\n+\t\t} else {\n+\t\t\tstrbuf_addf(&pointee_path, \"%s/%s\",\n+\t\t\t\t ref_store->gitdir, referent.buf);\n+\t\t\tret = files_fsck_symref_target(o, &report, refname.buf,\n+\t\t\t\t\t\t &referent,\n+\t\t\t\t\t\t &pointee_path);\n \t\t}\n \t\tgoto cleanup;\n \t}\n@@ -3490,6 +3566,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n+\tstrbuf_release(&pointee_path);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 7c1910d784..69280795ca 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -176,4 +176,58 @@ test_expect_success 'regular ref content should be checked' '\n \ttest_cmp expect err\n '\n \n+test_expect_success 'symbolic ref content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\tgit commit --allow-empty -m initial &&\n+\tgit checkout -b branch-1 &&\n+\tgit tag tag-1 &&\n+\tgit checkout -b a/b/branch-2 &&\n+\n+\tprintf \"ref: refs/heads/branch\" > $branch_dir_prefix/branch-1-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline\n+\tEOF\n+\trm $branch_dir_prefix/branch-1-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \" > $branch_dir_prefix/a/b/branch-trailing &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\\n\" > $branch_dir_prefix/a/b/branch-trailing &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n\\n \" > $branch_dir_prefix/a/b/branch-trailing &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/.branch\\n\" > $branch_dir_prefix/branch-2-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-2-bad: badSymrefPointee: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-2-bad &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"501723","messageId":"Zs358SAI5AYnX75v@ArchLinux","threadId":"61943","inReplyTo":"Zs348uXMBdCuwF-2@ArchLinux","subject":"[PATCH v2 4/4] ref: add symlink ref check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-27T16:08:17Z","receivedAt":"2024-08-27T16:07:29Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already introduced \"files_fsck_symref_target\". We should reuse\nthis function to handle the symrefs which are legacy symbolic links. We\nshould not check the trailing garbage for symbolic links. Add a new\nparameter \"symbolic_link\" to disable some checks which should only be\nused for symbolic ref.\n\nWe firstly use the \"strbuf_add_real_path\" to resolve the symlinks and\nget the absolute path \"pointee_path\" which the symlink ref points to.\nThen we can get the absolute path \"abs_gitdir\" of the \"gitdir\". By\ncombining \"pointee_path\" and \"abs_gitdir\", we can extract the\n\"referent\". Thus, we can reuse \"files_fsck_symref_target\" function to\nseamlessly check the symlink refs.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 68 +++++++++++++++++++++++++++++-----------\n t/t0602-reffiles-fsck.sh | 44 ++++++++++++++++++++++++++\n 2 files changed, 94 insertions(+), 18 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 382c73fcf7..8641e3ba65 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1,4 +1,5 @@\n #include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n #include \"../copy.h\"\n #include \"../environment.h\"\n #include \"../gettext.h\"\n@@ -3437,13 +3438,15 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n /*\n * Check the symref \"pointee_name\" and \"pointee_path\". The caller should\n * make sure that \"pointee_path\" is absolute. For symbolic ref, \"pointee_name\"\n- * would be the content after \"refs:\".\n+ * would be the content after \"refs:\". For symblic link, \"pointee_name\" would\n+ * be the relative path agaignst \"gitdir\".\n */\n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n \t\t\t\t const char *refname,\n \t\t\t\t struct strbuf *pointee_name,\n-\t\t\t\t struct strbuf *pointee_path)\n+\t\t\t\t struct strbuf *pointee_path,\n+\t\t\t\t unsigned int symbolic_link)\n {\n \tconst char *newline_pos = NULL;\n \tconst char *p = NULL;\n@@ -3458,24 +3461,28 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n-\tnewline_pos = strrchr(p, '\\n');\n-\tif (!newline_pos || *(newline_pos + 1)) {\n-\t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n-\t\t\t\t \"missing newline\");\n+\tif (!symbolic_link) {\n+\t\tnewline_pos = strrchr(p, '\\n');\n+\t\tif (!newline_pos || *(newline_pos + 1)) {\n+\t\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t \"missing newline\");\n+\t\t}\n \t}\n \n \tif (check_refname_format(pointee_name->buf, 0)) {\n-\t\t/*\n-\t\t * When containing null-garbage, \"check_refname_format\" will\n-\t\t * fail, we should trim the \"pointee\" to check again.\n-\t\t */\n-\t\tstrbuf_rtrim(pointee_name);\n-\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n-\t\t\tret = fsck_report_ref(o, report,\n-\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n-\t\t\t\t\t \"trailing null-garbage\");\n-\t\t\tgoto out;\n+\t\tif (!symbolic_link) {\n+\t\t\t/*\n+\t\t\t* When containing null-garbage, \"check_refname_format\" will\n+\t\t\t* fail, we should trim the \"pointee\" to check again.\n+\t\t\t*/\n+\t\t\tstrbuf_rtrim(pointee_name);\n+\t\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n+\t\t\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t\t \"trailing null-garbage\");\n+\t\t\t\tgoto out;\n+\t\t\t}\n \t\t}\n \n \t\tret = fsck_report_ref(o, report,\n@@ -3510,9 +3517,12 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n {\n \tstruct strbuf pointee_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n \tstruct fsck_ref_report report = {0};\n+\tconst char *pointee_name = NULL;\n+\tunsigned int symbolic_link = 0;\n \tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n@@ -3557,16 +3567,38 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t ref_store->gitdir, referent.buf);\n \t\t\tret = files_fsck_symref_target(o, &report, refname.buf,\n \t\t\t\t\t\t &referent,\n-\t\t\t\t\t\t &pointee_path);\n+\t\t\t\t\t\t &pointee_path,\n+\t\t\t\t\t\t symbolic_link);\n \t\t}\n \t\tgoto cleanup;\n \t}\n \n+\tsymbolic_link = 1;\n+\n+\tstrbuf_add_real_path(&pointee_path, iter->path.buf);\n+\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n+\tstrbuf_normalize_path(&abs_gitdir);\n+\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n+\t\tstrbuf_addch(&abs_gitdir, '/');\n+\n+\tif (!skip_prefix(pointee_path.buf, abs_gitdir.buf, &pointee_name)) {\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n+\t\t\t\t \"point to target outside gitdir\");\n+\t\tgoto cleanup;\n+\t}\n+\n+\tstrbuf_addstr(&referent, pointee_name);\n+\tret = files_fsck_symref_target(o, &report, refname.buf,\n+\t\t\t\t &referent, &pointee_path,\n+\t\t\t\t symbolic_link);\n+\n cleanup:\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n \tstrbuf_release(&pointee_path);\n+\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 69280795ca..36992fbc7f 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -230,4 +230,48 @@ test_expect_success 'symbolic ref content should be checked' '\n \ttest_cmp expect err\n '\n \n+test_expect_success SYMLINKS 'symbolic ref (symbolic link) content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\tgit commit --allow-empty -m initial &&\n+\tgit checkout -b branch-1 &&\n+\tgit tag tag-1 &&\n+\tgit checkout -b a/b/branch-2 &&\n+\n+\tln -sf ../../../../branch $branch_dir_prefix/branch-symbolic &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic: badSymrefPointee: point to target outside gitdir\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../logs/branch-bad $branch_dir_prefix/branch-symbolic &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic: badSymrefPointee: points to ref outside the refs directory\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic: badSymrefPointee: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\".branch\" $branch_dir_prefix/branch-symbolic &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic: badSymrefPointee: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"501725","messageId":"Zs38frYexwFt0xlP@ArchLinux","threadId":"61943","inReplyTo":"Zs353oLDaw2SbNQs@ArchLinux","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-27T16:19:10Z","receivedAt":"2024-08-27T16:18:20Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Aug 28, 2024 at 12:08:03AM +0800, shejialuo wrote:\n\n> And we cannot either report a warn fsck message to the user. This is\n> because if the caller set the \"strict\" field in \"fsck_options\" to\n> to upgrade the fsck warnings to errors.\n> \n\nSorry for this paragraph, because I have changed commit message for this\npatch, the range-diff part would be outdated. But the code is still the\nsame. So I hope this will not cause much trouble. And this paragraph\nshould be the following:\n\n And we cannot either report a warn fsck message to the user. This is\n because if the caller set the \"strict\" field in \"fsck_options\", fsck\n warnings will be converted to errors.\n\nI will fix this in the next version until I receive enough feedback.\n\nThanks.\n\n"},{"id":"501726","messageId":"xmqqbk1dc0v5.fsf@gitster.g","threadId":"61943","inReplyTo":"Zs351iV2HbdhNvEz@ArchLinux","subject":"Re: [PATCH v2 1/4] ref: initialize \"fsck_ref_report\" with zero","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-27T17:49:18Z","receivedAt":"2024-08-27T17:49:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> The original code explicitly specifies the \".path\" field to initialize\n> the \"fsck_ref_report\" structure. However, it introduces confusion how we\n> initialize the other fields.\n\nThe above description is a bit too strong than what this patch is\nactually fixing. If you explicitly initialize any member of an\naggregate type, other members not mentioned will be implicitly\n0-initialized, so the original does not give any confusion to\nreaders who know what they are reading.\n\nWhat the patch improves is that the common idiom used in this\ncode base (and possibly elsewhere) is to use \"{ 0 }\", instead\nof explicitly saying \"this particular member is 0-initialized\".\n\n The original code explicitly initializes the \"path\" member in\n the \"struct fsck_ref_report\" to NULL (which implicitly\n 0-initializes other members in the struct). It is more\n customary to use \"{ 0 }\" to express that we are 0-initializing\n everything.\n\nThe patch is correct, but spelling it like \"{ 0 }\" with a space on both\nsides is more common [*], and because this patch is all about making it\nmore idiomatic, let's write it that way.\n\nThanks.\n\n[Footnote]\n\n * \"git grep -e '{0};' -e '{ 0 };' '*.[ch]'\" tells us so.\n\n\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> refs/files-backend.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 8d6ec9458d..d6fc3bd67c 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3446,7 +3446,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n> \t\tgoto cleanup;\n> \n> \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n> -\t\tstruct fsck_ref_report report = { .path = NULL };\n> +\t\tstruct fsck_ref_report report = {0};\n> \n> \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n> \t\treport.path = sb.buf;\n"},{"id":"501727","messageId":"xmqqjzg1aksx.fsf@gitster.g","threadId":"61943","inReplyTo":"Zs353oLDaw2SbNQs@ArchLinux","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-27T18:21:34Z","receivedAt":"2024-08-27T18:21:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\n> refs. However, when parsing the regular refs for files backend by using\n> \"files-backend.c::parse_loose_ref_contents\", we allow the ref content to\n> be end with no newline or contain some garbages.\n\n\"to be end with\" -> \"to end with\".\n\"or contain\" -> \"or to contain\" (optional, I think).\n\nOr \"... the ref content without terminating newline, or with extra\nbytes after the terminating newline.\"\n\n> It may seem that we should report an error or warn fsck message to the\n> user about above situations. However, there may be some third-party\n> tools customizing the content of refs. We should not report an error\n> fsck message.\n\n Even though we never created such loose refs ourselves, we have\n accepted such loose refs forever, so it is entirely possible\n that third-party tools may rely on such loose refs being valid.\n Let's notice such a \"curiously formatted\" loose ref files and\n tell the users our findings, so that we can assess the possible\n extent of damage if/when we retroactively tightened the parsing\n rules in the future.\n\n> We should not allow the user to upgrade the fsck warnings to errors. It\n> might cause compatibility issue which will break the legacy repository.\n\nI am not sure this is a right thing to say. If the user wants to\nensure that the tool they use in their repository, which may include\nsome third-party reimplementation of Git, would never create such a\n(semi-)malformed loose ref files, it is within their right, and it\nis the most reasonable way, to promote these \"curiously formatted\nloose ref\" fsck warnings to errors.\n\nIs your \"We should not allow\" above backed by code that prevents\nthem from promoting the warnings to errors, or is it merely a\ndeclaration of your intention?\n\n> So we add the following two fsck infos to represent the situation where\n> the ref content ends without newline or has garbages:\n>\n> 1. \"refMissingNewline(INFO)\": A valid ref does not end with newline.\n> 2. \"trailingRefContent(INFO)\": A ref has trailing contents.\n\nOK.\n\n> In \"fsck.c::fsck_vreport\", we will convert \"FSCK_INFO\" to \"FSCK_WARN\",\n> and we can still warn the user about these situations when using\n> \"git-refs verify\" without introducing compatibility issue.\n\nOK.\n\n> In current \"git-fsck(1)\", it will report an error when the ref content\n> is bad, so we should following this to report an error to the user when\n> \"parse_loose_ref_contents\" fails. And we add a new fsck error message\n> called \"badRefContent(ERROR)\" to represent that a ref has a bad content.\n\nGood.\n\n> @@ -170,6 +173,12 @@\n> `nullSha1`::\n> \t(WARN) Tree contains entries pointing to a null sha1.\n> \n> +`refMissingNewline`::\n> +\t(INFO) A valid ref does not end with newline.\n> +\n> +`trailingRefContent`::\n> +\t(INFO) A ref has trailing contents.\n> +\n> `treeNotSorted`::\n> \t(ERROR) A tree is not properly sorted.\n\nThere is no mention of \"you shouldn't promote these to error\" here,\nwhich is good. But wouldn't we want to tell users to report such\ncuriously formatted loose refs, after figuring out who created them,\nto help us to eventually make the check stricter in the future?\n\nGit 3.0 boundary might be a good time to tighten interoperability\nrules such that we won't accept anything we wouldn't have written\nourselves (not limited to loose ref format, but this applies to\nanything on-disk or on-wire), but we'd need enough preparation if we\nwant to be able to do so in the future.\n\nThanks.\n\n\n"},{"id":"501728","messageId":"xmqq1q2993kg.fsf@gitster.g","threadId":"61943","inReplyTo":"Zs3558scHssaG_XS@ArchLinux","subject":"Re: [PATCH v2 3/4] ref: add symbolic ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-27T19:19:11Z","receivedAt":"2024-08-27T19:19:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> We have already introduced the checks for regular refs. There is no need\n> to check the consistency of the target which the symbolic ref points to.\n> Instead, we just check the content of the symbolic ref itself.\n\nJust in case you need it in the future, if you ever need to refer to\na symbolic ref in a way that it is clear which of the two kinds you\nare talking about, you can say \"textual symref\" (a regular file\nwhose contents is \"ref: \" followed by the target), to contrast them\nwith \"symbolic link used as symref\".\n\nIn the proposed log message of this commit, all references to\n\"symbolic ref\" talk about textual ones, so I do not see any need to\nbe extra explicit by saying \"textual symref\".\n\n> In order to check the content of the symbolic ref, create a function\n> \"files_fsck_symref_target\". It will first check whether the \"pointee\" is\n> under the \"refs/\" directory and then we will check the \"pointee\" itself.\n\nHmph, as the pointee must be within the usual places that you would\nfind refs (either in refs/ directory or pseudo ref files immediately\nbelow $GIT_DIR), wouldn't we check the pointee when fsck (or \"git\nrefs verify\") run and check everything? The pointee will have its\nturn to be checked, and I am not sure why you need to check the\npointee when you find a symbolic ref is pointing at it, which will\nlead for it to be checked twice (or more).\n\nI however did not find an additional code to \"check the pointee itself\"\nin the patch, so perhaps it is OK---the only thing that needs fixing\nmay be the above paragraph if that is the case.\n\n> There is no specification about the content of the symbolic ref.\n> Although we do write \"ref: %s\\n\" to create a symbolic ref by using\n> \"git-symbolic-ref(1)\" command. However, this is not mandatory. We still\n> accept symbolic refs with null trailing garbage. Put it more specific,\n> the following are correct:\n>\n> 1. \"ref: refs/heads/master \"\n> 2. \"ref: refs/heads/master \\n \\n\"\n> 3. \"ref: refs/heads/master\\n\\n\"\n>\n> But we do not allow any non-null trailing garbage.\n\nYour use of word \"null\" is probably too confusing to contributors to\nthis project. None of the above has NUL bytes in them. I think you\nwant to say something like this:\n\n A regular file is accepted as a textual symbolic ref if it\n begins with \"ref:\", followed by zero or more whitespaces,\n followed by the full refname (e.g. \"refs/heads/master\",\n \"refs/tags/v1.0\"), followed only by whitespace characters. We\n always write a single SP after \"ref:\" and a single LF after the\n full refname, but third-party reimplementations of Git may have\n taken advantage of the looser syntax that is allowed as above.\n\n> The following are bad\n> symbolic contents which will be reported as fsck error by \"git-fsck(1)\".\n>\n> 1. \"ref: refs/heads/master garbage\\n\"\n> 2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n>\n> In order to provide above checks, we will use \"strrchr\" to check whether\n> we have newline in the ref content.\n\nstrrchr() to look for only LF is overly strict. You need to match\nwhat refs/files-backend.c:read_ref_internal() does to the contents\nread from such a loose ref file, i.e. strbuf_rtrim(). Any isspace()\nbytes are trimmed at the end, including SP, HT, CR and LF.\n\n> +static int files_fsck_symref_target(struct fsck_options *o,\n> +\t\t\t\t struct fsck_ref_report *report,\n> +\t\t\t\t const char *refname,\n> +\t\t\t\t struct strbuf *pointee_name,\n> +\t\t\t\t struct strbuf *pointee_path)\n> +{\n> +\tconst char *newline_pos = NULL;\n> +\tconst char *p = NULL;\n> +\tstruct stat st;\n> +\tint ret = 0;\n> +\n> +\tif (!skip_prefix(pointee_name->buf, \"refs/\", &p)) {\n> +\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> +\t\t\t\t \"points to ref outside the refs directory\");\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tnewline_pos = strrchr(p, '\\n');\n> +\tif (!newline_pos || *(newline_pos + 1)) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> +\t\t\t\t \"missing newline\");\n\nIf newline_pos is NULL, it is truly a \"missing newline\" situation.\nIf I am reading the code correctly, the severity level is set to\nINFO, which is good.\n\nIf newline_pos is not NULL but newline_pos[1] is not NUL, however,\nthat is not a \"missing newline\". \"refs: refs/heads/master\\n \" would\ntrigger this report, for example.\n\nAs far as I can tell, such a textual symbolic ref is taken as a\nvalid symbolic ref pointing at \"refs/heads/master\" by\nrefs/files-backend.c:read_ref_internal(), so we are trying to detect\na valid but curiously formatted textual symbolic ref file with the\nabove code?\n\nAnd strrchr() to find the last LF is not sufficient for that\npurpose. We would never write \"refs: refs/head/master \\n\",\nbut the above code will find the LF, be satisified that the LF is\nfollowed by NUL, without realizing that SP there is not something we\nwould have written!\n\nI am not sure if that is worth detecting that if it is something we\nwould have written, but if that were the case, then you would\nprobably need to do\n\n (1) check the last byte of pointee_name.buf[] to make sure that\n it is LF; and\n (2) remember pointee_name.len, run strbuf_rtrim() on pointee_name,\n and that LF at the end was the only thing that was trimmed by\n checking the pointee_name.len after trimming.\n\nor something like that. Then you do not have to have an ugly \"oh we\nneed to check again\"---the production code would not do that, either.\n\n> +\tif (check_refname_format(pointee_name->buf, 0)) {\n> +\t\t/*\n> +\t\t * When containing null-garbage, \"check_refname_format\" will\n> +\t\t * fail, we should trim the \"pointee\" to check again.\n> +\t\t */\n> +\t\tstrbuf_rtrim(pointee_name);\n> +\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n> +\t\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> +\t\t\t\t\t \"trailing null-garbage\");\n> +\t\t\tgoto out;\n> +\t\t}\n\nIOW, the above \"let's retry\" feels totally wrong. You shouldn't\nhave to do so, and that comes from running check_refname_format()\nbefore rtrimming the pointee_name.\n\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> +\t\t\t\t \"points to refname with invalid format\");\n> +\t}\n\nGood. With this check, we know that the referent, if exists, is\nwell-formed. The contents of the referent will then be checked just\nlike all other refs that may not be pointed by any symbolic ref.\n\n> +\t/*\n> +\t * Missing target should not be treated as any error worthy event and\n> +\t * not even warn. It is a common case that a symbolic ref points to a\n> +\t * ref that does not exist yet. If the target ref does not exist, just\n> +\t * skip the check for the file type.\n> +\t */\n> +\tif (lstat(pointee_path->buf, &st) < 0)\n> +\t\tgoto out;\n\nGood.\n\n> +\tif (!S_ISREG(st.st_mode) && !S_ISLNK(st.st_mode)) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> +\t\t\t\t \"points to an invalid file type\");\n> +\t\tgoto out;\n\nI do not think it is wrong per se, but I am not sure if this check\nis needed, either. When \"git fsck\" or \"git refs verify\" is told to\ncheck the loose refs, wouldn't it walk the refs directory and report\nsuch an unusual filesystem entity that is not a regular file,\nsymbolic link, or a directory as \"there is unusual cruft exist\nhere\"?\n"},{"id":"501759","messageId":"Zs8c81Z-zb4uQpp6@tanuki","threadId":"61943","inReplyTo":"Zs353oLDaw2SbNQs@ArchLinux","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-28T12:50:01Z","receivedAt":"2024-08-28T12:50:06Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Aug 28, 2024 at 12:07:58AM +0800, shejialuo wrote:\n> @@ -170,6 +173,12 @@\n> `nullSha1`::\n> \t(WARN) Tree contains entries pointing to a null sha1.\n> \n> +`refMissingNewline`::\n> +\t(INFO) A valid ref does not end with newline.\n\nThis reads a bit funny to me. If the ref is valid, why do we complain?\n\nMaybe this would read better if you said \"An otherwise valid ref does\nnot end with a newline\".\n\n> @@ -3430,6 +3434,65 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> \t\t\t\t const char *refs_check_dir,\n> \t\t\t\t struct dir_iterator *iter);\n> \n> +static int files_fsck_refs_content(struct ref_store *ref_store,\n> +\t\t\t\t struct fsck_options *o,\n> +\t\t\t\t const char *refs_check_dir,\n> +\t\t\t\t struct dir_iterator *iter)\n> +{\n> +\tstruct strbuf ref_content = STRBUF_INIT;\n> +\tstruct strbuf referent = STRBUF_INIT;\n> +\tstruct strbuf refname = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = {0};\n> +\tconst char *trailing = NULL;\n> +\tunsigned int type = 0;\n> +\tint failure_errno = 0;\n> +\tstruct object_id oid;\n> +\tint ret = 0;\n> +\n> +\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n> +\treport.path = refname.buf;\n> +\n> +\tif (S_ISREG(iter->st.st_mode)) {\n\nThis is still indenting the whole body. You mentioned that you don't\nwant to use `goto`, but in our codebase it's actually quite idiomatic.\nAnd you already use it anyway.\n\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 71a4d1a5ae..7c1910d784 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -89,4 +89,91 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n> \ttest_must_be_empty err\n> '\n> \n> +test_expect_success 'regular ref content should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\tbranch_dir_prefix=.git/refs/heads &&\n> +\ttag_dir_prefix=.git/refs/tags &&\n> +\tcd repo &&\n> +\tgit commit --allow-empty -m initial &&\n> +\tgit checkout -b branch-1 &&\n> +\tgit tag tag-1 &&\n> +\tgit commit --allow-empty -m second &&\n> +\tgit checkout -b branch-2 &&\n> +\tgit tag tag-2 &&\n> +\tgit checkout -b a/b/tag-2 &&\n\nWouldn't it be sufficient to only create a single commit, e.g. via\n`test_commit`? From all I can see all you need is some object ID, so\ncreating the tags and second commit doesn't seem to be necessary.\n\n> +\tprintf \"%s\" \"$(git rev-parse branch-1)\" > $branch_dir_prefix/branch-1-no-newline &&\n\nWe don't typically have spaces after the redirect. So you should remove\nthem here and in all the subsequent instances.\n\n> +\tgit refs verify 2>err &&\n> +\tcat >expect <<-EOF &&\n> +\twarning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline\n> +\tEOF\n> +\trm $branch_dir_prefix/branch-1-no-newline &&\n> +\ttest_cmp expect err &&\n\nI was wondering whether each of these cases should be a separate test,\nbut that may be a bit wasteful. Alternatively, can we maybe set up a\nsingle repository with all the garbage that we want to verify and then\ndouble check that executing `git refs verify` surfaces them all in a\nsingle invocation?\n\nPatrick\n"},{"id":"501760","messageId":"Zs8c_vuqOSSWJjpd@tanuki","threadId":"61943","inReplyTo":"xmqqjzg1aksx.fsf@gitster.g","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-28T12:50:06Z","receivedAt":"2024-08-28T12:50:09Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Aug 27, 2024 at 11:21:34AM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> > @@ -170,6 +173,12 @@\n> > `nullSha1`::\n> > \t(WARN) Tree contains entries pointing to a null sha1.\n> > \n> > +`refMissingNewline`::\n> > +\t(INFO) A valid ref does not end with newline.\n> > +\n> > +`trailingRefContent`::\n> > +\t(INFO) A ref has trailing contents.\n> > +\n> > `treeNotSorted`::\n> > \t(ERROR) A tree is not properly sorted.\n> \n> There is no mention of \"you shouldn't promote these to error\" here,\n> which is good. But wouldn't we want to tell users to report such\n> curiously formatted loose refs, after figuring out who created them,\n> to help us to eventually make the check stricter in the future?\n> \n> Git 3.0 boundary might be a good time to tighten interoperability\n> rules such that we won't accept anything we wouldn't have written\n> ourselves (not limited to loose ref format, but this applies to\n> anything on-disk or on-wire), but we'd need enough preparation if we\n> want to be able to do so in the future.\n\nI quite like this idea. Jialuo, would you maybe want to include another\npatch on top that adds a paragraph to Documentation/BreakingChanges.txt?\nIt should note that this is not yet settled and depends on whether or\nnot we see complaints with your new checks.\n\nI guess another prereq for the change is to integrate `git refs verify`\nwith git-fsck(1), because otherwise people likely wouldn't see the new\nmessages in the first place.\n\nPatrick\n"},{"id":"501761","messageId":"Zs8dAc0ss9KbwIDs@tanuki","threadId":"61943","inReplyTo":"Zs3558scHssaG_XS@ArchLinux","subject":"Re: [PATCH v2 3/4] ref: add symbolic ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-28T12:50:09Z","receivedAt":"2024-08-28T12:50:13Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Aug 28, 2024 at 12:08:07AM +0800, shejialuo wrote:\n> We have already introduced the checks for regular refs. There is no need\n> to check the consistency of the target which the symbolic ref points to.\n> Instead, we just check the content of the symbolic ref itself.\n> \n> In order to check the content of the symbolic ref, create a function\n> \"files_fsck_symref_target\". It will first check whether the \"pointee\" is\n> under the \"refs/\" directory and then we will check the \"pointee\" itself.\n> \n> There is no specification about the content of the symbolic ref.\n> Although we do write \"ref: %s\\n\" to create a symbolic ref by using\n> \"git-symbolic-ref(1)\" command. However, this is not mandatory. We still\n> accept symbolic refs with null trailing garbage. Put it more specific,\n> the following are correct:\n> \n> 1. \"ref: refs/heads/master \"\n> 2. \"ref: refs/heads/master \\n \\n\"\n> 3. \"ref: refs/heads/master\\n\\n\"\n\nNow that we're talking about tightening the rules for direct refs, I\nwonder whether we'd also want to apply the same rules to symrefs.\nNamely, when there is trailing whitespace we should generate an\nINFO-level message about that, too. This is mostly for the sake of\nconsistency.\n\n[snip]\n> diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> index fc074fc571..85fd058c81 100644\n> --- a/Documentation/fsck-msgids.txt\n> +++ b/Documentation/fsck-msgids.txt\n> @@ -28,6 +28,9 @@\n> `badRefName`::\n> \t(ERROR) A ref has an invalid format.\n> \n> +`badSymrefPointee`::\n> +\t(ERROR) The pointee of a symref is bad.\n\nI think we'd want to clarify what \"bad\" is supposed to mean. Like, is a\nmissing symref pointee bad? If this is about the format of the pointee's\nname, we might want to call this \"badSymrefPointeeName\".\n\nAlso, I think we don't typically call the value of a symbolic ref\n\"pointee\", but \"target\". Searching for \"pointee\" in our codebase only\ngives a single hit, and that one is not related to symbolic refs.\n\n> diff --git a/fsck.h b/fsck.h\n> index b85072df57..cbe837f84c 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -34,6 +34,7 @@ enum fsck_msg_type {\n> \tFUNC(BAD_REF_CONTENT, ERROR) \\\n> \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n> \tFUNC(BAD_REF_NAME, ERROR) \\\n> +\tFUNC(BAD_SYMREF_POINTEE, ERROR) \\\n> \tFUNC(BAD_TIMEZONE, ERROR) \\\n> \tFUNC(BAD_TREE, ERROR) \\\n> \tFUNC(BAD_TREE_SHA1, ERROR) \\\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 69c00073eb..382c73fcf7 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3434,11 +3434,81 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> \t\t\t\t const char *refs_check_dir,\n> \t\t\t\t struct dir_iterator *iter);\n> \n> +/*\n> + * Check the symref \"pointee_name\" and \"pointee_path\". The caller should\n> + * make sure that \"pointee_path\" is absolute. For symbolic ref, \"pointee_name\"\n> + * would be the content after \"refs:\".\n> + */\n> +static int files_fsck_symref_target(struct fsck_options *o,\n> +\t\t\t\t struct fsck_ref_report *report,\n> +\t\t\t\t const char *refname,\n> +\t\t\t\t struct strbuf *pointee_name,\n> +\t\t\t\t struct strbuf *pointee_path)\n> +{\n> +\tconst char *newline_pos = NULL;\n> +\tconst char *p = NULL;\n> +\tstruct stat st;\n> +\tint ret = 0;\n> +\n> +\tif (!skip_prefix(pointee_name->buf, \"refs/\", &p)) {\n> +\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> +\t\t\t\t \"points to ref outside the refs directory\");\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tnewline_pos = strrchr(p, '\\n');\n> +\tif (!newline_pos || *(newline_pos + 1)) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> +\t\t\t\t \"missing newline\");\n> +\t}\n\nThe second condition `*(newline_pos + 1)` checks whether there is any\ndata after the newline, doesn't it? That indicates a different kind of\nerror than \"missing newline\", namely that there is trailing garbage. I\nguess we'd want to report a separate info-level message for this.\n\nAlso, shouldn't we use `strchr` instead of `strrchr()`? Otherwise, we're\nonly checking for trailing garbage after the _last_ newline, not after\nthe first one.\n\n> +\tif (check_refname_format(pointee_name->buf, 0)) {\n> +\t\t/*\n> +\t\t * When containing null-garbage, \"check_refname_format\" will\n> +\t\t * fail, we should trim the \"pointee\" to check again.\n> +\t\t */\n> +\t\tstrbuf_rtrim(pointee_name);\n> +\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n> +\t\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> +\t\t\t\t\t \"trailing null-garbage\");\n> +\t\t\tgoto out;\n> +\t\t}\n\nAh, I didn't get at first that we're doing the check a second time here.\nAs mentioned above, I think we should check for trailing garbage further\nup already and more explicitly.\n\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 7c1910d784..69280795ca 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -176,4 +176,58 @@ test_expect_success 'regular ref content should be checked' '\n> \ttest_cmp expect err\n> '\n> \n> +test_expect_success 'symbolic ref content should be checked' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\tbranch_dir_prefix=.git/refs/heads &&\n> +\ttag_dir_prefix=.git/refs/tags &&\n> +\tcd repo &&\n> +\tgit commit --allow-empty -m initial &&\n> +\tgit checkout -b branch-1 &&\n> +\tgit tag tag-1 &&\n> +\tgit checkout -b a/b/branch-2 &&\n> +\n> +\tprintf \"ref: refs/heads/branch\" > $branch_dir_prefix/branch-1-no-newline &&\n> +\tgit refs verify 2>err &&\n> +\tcat >expect <<-EOF &&\n> +\twarning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline\n> +\tEOF\n> +\trm $branch_dir_prefix/branch-1-no-newline &&\n> +\ttest_cmp expect err &&\n\nSame comments here as in the preceding patch for the tests.\n\nPatrick\n"},{"id":"501769","messageId":"Zs801HqHg45v_q6X@ArchLinux","threadId":"61943","inReplyTo":"xmqqjzg1aksx.fsf@gitster.g","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-28T14:31:48Z","receivedAt":"2024-08-28T14:30:57Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Aug 27, 2024 at 11:21:34AM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\n> > refs. However, when parsing the regular refs for files backend by using\n> > \"files-backend.c::parse_loose_ref_contents\", we allow the ref content to\n> > be end with no newline or contain some garbages.\n> \n> \"to be end with\" -> \"to end with\".\n> \"or contain\" -> \"or to contain\" (optional, I think).\n> \n> Or \"... the ref content without terminating newline, or with extra\n> bytes after the terminating newline.\"\n> \n\nThanks, I will fix this in the next version.\n\n> > It may seem that we should report an error or warn fsck message to the\n> > user about above situations. However, there may be some third-party\n> > tools customizing the content of refs. We should not report an error\n> > fsck message.\n> \n> Even though we never created such loose refs ourselves, we have\n> accepted such loose refs forever, so it is entirely possible\n> that third-party tools may rely on such loose refs being valid.\n> Let's notice such a \"curiously formatted\" loose ref files and\n> tell the users our findings, so that we can assess the possible\n> extent of damage if/when we retroactively tightened the parsing\n> rules in the future.\n> \n\nI think I could organize the above to the commit message to better show\nthe motivation why we should not report an error fsck message.\n\n> > We should not allow the user to upgrade the fsck warnings to errors. It\n> > might cause compatibility issue which will break the legacy repository.\n> \n> I am not sure this is a right thing to say. If the user wants to\n> ensure that the tool they use in their repository, which may include\n> some third-party reimplementation of Git, would never create such a\n> (semi-)malformed loose ref files, it is within their right, and it\n> is the most reasonable way, to promote these \"curiously formatted\n> loose ref\" fsck warnings to errors.\n> \n> Is your \"We should not allow\" above backed by code that prevents\n> them from promoting the warnings to errors, or is it merely a\n> declaration of your intention?\n> \n\nI have introduced some misunderstanding here. In the previous paragraph,\nI have mentioned that if the caller set the \"strict\" field in\n\"fsck_options\", the fsck warns would be automatically converted to fsck\nerrors which may cause some trouble.\n\nSo I think here we should move this paragraph just after the previous\nparagraph to indicate why we do want to make a info fsck message here.\nActually, the user could still explicitly use the following command\n\n git -c fsck.refMissingNewline=error refs verify\n\nto upgrade the fsck info to fsck error. But if the user use \"--strict\"\nlike the following:\n\n git refs verify --strict\n\nThe fsck warns would be automatically converted to fsck errors. But\nactually at current, we do not want to the user implicitly upgrade fsck\nwarns to fsck errors by using \"--strict\" flag. That's why we need to\nintroduce the \"FSCK_INO\" here.\n\nActually, I was inspired by the Jeff King's commit:\n\n 4dd3b045f5 (fsck: downgrade tree badFilemode to \"info\", 2022-08-10)\n\nIn this commit, Jeff downgrades badFilemode to \"info\" to avoid above\nsituation. I will improve the commit message to make things clearer.\n\nHowever, from my perspective, the semantic of \"FSCK_INFO\" is a little\nunsuitable here. The comment says:\n\n /* infos (reported as warnings, but ignored by default) */\n\nThe \"ignored by default\" here is very confusing. Actually, we make the\n\"info\" lower than the \"warn\" to avoid automatically converting the \"warn\"\nto \"error\" by setting \"strict\" field in \"fsck_options\".\n\nBut \"ignored by default\" will make the user think \"oh, it's info, but we\nreport it as warnings\". We cannot know the real intention of the\n\"FSCK_INFO\" unless we have above context.\n\nBut I guess this is too far from the intention of this patch. We may\nimprove this later.\n\n> > @@ -170,6 +173,12 @@\n> > `nullSha1`::\n> > \t(WARN) Tree contains entries pointing to a null sha1.\n> > \n> > +`refMissingNewline`::\n> > +\t(INFO) A valid ref does not end with newline.\n> > +\n> > +`trailingRefContent`::\n> > +\t(INFO) A ref has trailing contents.\n> > +\n> > `treeNotSorted`::\n> > \t(ERROR) A tree is not properly sorted.\n> \n> There is no mention of \"you shouldn't promote these to error\" here,\n> which is good. But wouldn't we want to tell users to report such\n> curiously formatted loose refs, after figuring out who created them,\n> to help us to eventually make the check stricter in the future?\n> \n\nFrom the review from the Patrick, I will add another patch in the\n\"Documentation/BreakingChanges.txt\" later.\n\n> Git 3.0 boundary might be a good time to tighten interoperability\n> rules such that we won't accept anything we wouldn't have written\n> ourselves (not limited to loose ref format, but this applies to\n> anything on-disk or on-wire), but we'd need enough preparation if we\n> want to be able to do so in the future.\n> \n> Thanks.\n> \n\nThanks,\nJialuo\n"},{"id":"501770","messageId":"Zs83HCZjM1gycfv1@ArchLinux","threadId":"61943","inReplyTo":"Zs8c81Z-zb4uQpp6@tanuki","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-28T14:41:32Z","receivedAt":"2024-08-28T14:40:42Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Aug 28, 2024 at 02:50:01PM +0200, Patrick Steinhardt wrote:\n> On Wed, Aug 28, 2024 at 12:07:58AM +0800, shejialuo wrote:\n> > @@ -170,6 +173,12 @@\n> > `nullSha1`::\n> > \t(WARN) Tree contains entries pointing to a null sha1.\n> > \n> > +`refMissingNewline`::\n> > +\t(INFO) A valid ref does not end with newline.\n> \n> This reads a bit funny to me. If the ref is valid, why do we complain?\n> \n> Maybe this would read better if you said \"An otherwise valid ref does\n> not end with a newline\".\n> \n\nI think we should just drop the \"valid\" here. Because for symref, it\nmay miss newline and is NOT valid.\n\nI will improve this in the next version.\n\n> > @@ -3430,6 +3434,65 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> > \t\t\t\t const char *refs_check_dir,\n> > \t\t\t\t struct dir_iterator *iter);\n> > \n> > +static int files_fsck_refs_content(struct ref_store *ref_store,\n> > +\t\t\t\t struct fsck_options *o,\n> > +\t\t\t\t const char *refs_check_dir,\n> > +\t\t\t\t struct dir_iterator *iter)\n> > +{\n> > +\tstruct strbuf ref_content = STRBUF_INIT;\n> > +\tstruct strbuf referent = STRBUF_INIT;\n> > +\tstruct strbuf refname = STRBUF_INIT;\n> > +\tstruct fsck_ref_report report = {0};\n> > +\tconst char *trailing = NULL;\n> > +\tunsigned int type = 0;\n> > +\tint failure_errno = 0;\n> > +\tstruct object_id oid;\n> > +\tint ret = 0;\n> > +\n> > +\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n> > +\treport.path = refname.buf;\n> > +\n> > +\tif (S_ISREG(iter->st.st_mode)) {\n> \n> This is still indenting the whole body. You mentioned that you don't\n> want to use `goto`, but in our codebase it's actually quite idiomatic.\n> And you already use it anyway.\n> \n\nI think so, indenting is noisy. Will use \"goto\" to avoid indenting.\n\n> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> > index 71a4d1a5ae..7c1910d784 100755\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -89,4 +89,91 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n> > \ttest_must_be_empty err\n> > '\n> > \n> > +test_expect_success 'regular ref content should be checked' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\tbranch_dir_prefix=.git/refs/heads &&\n> > +\ttag_dir_prefix=.git/refs/tags &&\n> > +\tcd repo &&\n> > +\tgit commit --allow-empty -m initial &&\n> > +\tgit checkout -b branch-1 &&\n> > +\tgit tag tag-1 &&\n> > +\tgit commit --allow-empty -m second &&\n> > +\tgit checkout -b branch-2 &&\n> > +\tgit tag tag-2 &&\n> > +\tgit checkout -b a/b/tag-2 &&\n> \n> Wouldn't it be sufficient to only create a single commit, e.g. via\n> `test_commit`? From all I can see all you need is some object ID, so\n> creating the tags and second commit doesn't seem to be necessary.\n> \n\nI agree with this. I will clean the code for the next version.\n\n> > +\tprintf \"%s\" \"$(git rev-parse branch-1)\" > $branch_dir_prefix/branch-1-no-newline &&\n> \n> We don't typically have spaces after the redirect. So you should remove\n> them here and in all the subsequent instances.\n> \n\nI will clean the code style here.\n\n> > +\tgit refs verify 2>err &&\n> > +\tcat >expect <<-EOF &&\n> > +\twarning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline\n> > +\tEOF\n> > +\trm $branch_dir_prefix/branch-1-no-newline &&\n> > +\ttest_cmp expect err &&\n> \n> I was wondering whether each of these cases should be a separate test,\n> but that may be a bit wasteful. Alternatively, can we maybe set up a\n> single repository with all the garbage that we want to verify and then\n> double check that executing `git refs verify` surfaces them all in a\n> single invocation?\n> \n\nActually, I have also thought about separating the tests which may\nclear and I dropped this idea due to the reason the same as yours. I DO\nagree that we should set up a single repository with all the garbage\nthat we want to verify. This is necessary.\n\nThanks,\nJialuo\n"},{"id":"501775","messageId":"Zs9BwpUbWBLvsFMZ@ArchLinux","threadId":"61943","inReplyTo":"xmqq1q2993kg.fsf@gitster.g","subject":"Re: [PATCH v2 3/4] ref: add symbolic ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-28T15:26:58Z","receivedAt":"2024-08-28T15:26:07Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Aug 27, 2024 at 12:19:11PM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > In order to check the content of the symbolic ref, create a function\n> > \"files_fsck_symref_target\". It will first check whether the \"pointee\" is\n> > under the \"refs/\" directory and then we will check the \"pointee\" itself.\n> \n> Hmph, as the pointee must be within the usual places that you would\n> find refs (either in refs/ directory or pseudo ref files immediately\n> below $GIT_DIR), wouldn't we check the pointee when fsck (or \"git\n> refs verify\") run and check everything? The pointee will have its\n> turn to be checked, and I am not sure why you need to check the\n> pointee when you find a symbolic ref is pointing at it, which will\n> lead for it to be checked twice (or more).\n> \n> I however did not find an additional code to \"check the pointee itself\"\n> in the patch, so perhaps it is OK---the only thing that needs fixing\n> may be the above paragraph if that is the case.\n> \n\nYes, \"we will check the 'pointee'\" itself makes the reader confused. I\nwill fix the above paragraph. Actually we do not check the \"pointee\",\nbut check the symref content. Will fix this in the next version.\n\n> > There is no specification about the content of the symbolic ref.\n> > Although we do write \"ref: %s\\n\" to create a symbolic ref by using\n> > \"git-symbolic-ref(1)\" command. However, this is not mandatory. We still\n> > accept symbolic refs with null trailing garbage. Put it more specific,\n> > the following are correct:\n> >\n> > 1. \"ref: refs/heads/master \"\n> > 2. \"ref: refs/heads/master \\n \\n\"\n> > 3. \"ref: refs/heads/master\\n\\n\"\n> >\n> > But we do not allow any non-null trailing garbage.\n> \n> Your use of word \"null\" is probably too confusing to contributors to\n> this project. None of the above has NUL bytes in them. I think you\n> want to say something like this:\n> \n> A regular file is accepted as a textual symbolic ref if it\n> begins with \"ref:\", followed by zero or more whitespaces,\n> followed by the full refname (e.g. \"refs/heads/master\",\n> \"refs/tags/v1.0\"), followed only by whitespace characters. We\n> always write a single SP after \"ref:\" and a single LF after the\n> full refname, but third-party reimplementations of Git may have\n> taken advantage of the looser syntax that is allowed as above.\n> \n\nThanks for your suggestion. I will improve this in the next version.\n\n> > The following are bad\n> > symbolic contents which will be reported as fsck error by \"git-fsck(1)\".\n> >\n> > 1. \"ref: refs/heads/master garbage\\n\"\n> > 2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n> >\n> > In order to provide above checks, we will use \"strrchr\" to check whether\n> > we have newline in the ref content.\n> \n> strrchr() to look for only LF is overly strict. You need to match\n> what refs/files-backend.c:read_ref_internal() does to the contents\n> read from such a loose ref file, i.e. strbuf_rtrim(). Any isspace()\n> bytes are trimmed at the end, including SP, HT, CR and LF.\n> \n\nI will look into how \"strbuf_rtrim\" does to see whether we can reuse\nsome functions to avoid repetition.\n\n> > +static int files_fsck_symref_target(struct fsck_options *o,\n> > +\t\t\t\t struct fsck_ref_report *report,\n> > +\t\t\t\t const char *refname,\n> > +\t\t\t\t struct strbuf *pointee_name,\n> > +\t\t\t\t struct strbuf *pointee_path)\n> > +{\n> > +\tconst char *newline_pos = NULL;\n> > +\tconst char *p = NULL;\n> > +\tstruct stat st;\n> > +\tint ret = 0;\n> > +\n> > +\tif (!skip_prefix(pointee_name->buf, \"refs/\", &p)) {\n> > +\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> > +\t\t\t\t \"points to ref outside the refs directory\");\n> > +\t\tgoto out;\n> > +\t}\n> > +\n> > +\tnewline_pos = strrchr(p, '\\n');\n> > +\tif (!newline_pos || *(newline_pos + 1)) {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> > +\t\t\t\t \"missing newline\");\n> \n> If newline_pos is NULL, it is truly a \"missing newline\" situation.\n> If I am reading the code correctly, the severity level is set to\n> INFO, which is good.\n> \n> If newline_pos is not NULL but newline_pos[1] is not NUL, however,\n> that is not a \"missing newline\". \"refs: refs/heads/master\\n \" would\n> trigger this report, for example.\n> \n\nWhen I design this, I actually consider \"ref: refs/heads/master\\n \" is\nstill missing the newline. And then we also report that it has garbage.\nI think \"ref: refs/heads/master\\n \\n\" is not missing the newline. But, I\ndon't think this is good.\n\nI will find a good way to handle this.\n\n> As far as I can tell, such a textual symbolic ref is taken as a\n> valid symbolic ref pointing at \"refs/heads/master\" by\n> refs/files-backend.c:read_ref_internal(), so we are trying to detect\n> a valid but curiously formatted textual symbolic ref file with the\n> above code?\n\nYes, these situations will be taken as a valid symbolic ref but actually\nthere are something wrong. So this is what we need to care about.\n\n> \n> And strrchr() to find the last LF is not sufficient for that\n> purpose. We would never write \"refs: refs/head/master \\n\",\n> but the above code will find the LF, be satisified that the LF is\n> followed by NUL, without realizing that SP there is not something we\n> would have written!\n\nI totally ignored this situation, and in current patch, we cannot check\nthis. I know why Patrick lets me use \"strchr\" but not \"strrchr\". I think\nwe should find the last '\\n'. But instead we need to find the first\n'\\n'. However, in this example, we will still fail by using \"strchr\".\nThis part should be totally re-designed.\n\n> \n> I am not sure if that is worth detecting that if it is something we\n> would have written, but if that were the case, then you would\n> probably need to do\n> \n> (1) check the last byte of pointee_name.buf[] to make sure that\n> it is LF; and\n> (2) remember pointee_name.len, run strbuf_rtrim() on pointee_name,\n> and that LF at the end was the only thing that was trimmed by\n> checking the pointee_name.len after trimming.\n> \n> or something like that. Then you do not have to have an ugly \"oh we\n> need to check again\"---the production code would not do that, either.\n> \n\nYes, this is a good idea.\n\n> > +\tif (check_refname_format(pointee_name->buf, 0)) {\n> > +\t\t/*\n> > +\t\t * When containing null-garbage, \"check_refname_format\" will\n> > +\t\t * fail, we should trim the \"pointee\" to check again.\n> > +\t\t */\n> > +\t\tstrbuf_rtrim(pointee_name);\n> > +\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n> > +\t\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> > +\t\t\t\t\t \"trailing null-garbage\");\n> > +\t\t\tgoto out;\n> > +\t\t}\n> \n> IOW, the above \"let's retry\" feels totally wrong. You shouldn't\n> have to do so, and that comes from running check_refname_format()\n> before rtrimming the pointee_name.\n> \n\nYes, actually, I have thought I could compare the length change after\nexecuting the \"strbuf_rtrim\". I don't want to create two new variables,\nso I call \"check_refname_format\" twice.\n\nWill fix this in the next version.\n\n> > +\tif (!S_ISREG(st.st_mode) && !S_ISLNK(st.st_mode)) {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> > +\t\t\t\t \"points to an invalid file type\");\n> > +\t\tgoto out;\n> \n> I do not think it is wrong per se, but I am not sure if this check\n> is needed, either. When \"git fsck\" or \"git refs verify\" is told to\n> check the loose refs, wouldn't it walk the refs directory and report\n> such an unusual filesystem entity that is not a regular file,\n> symbolic link, or a directory as \"there is unusual cruft exist\n> here\"?\n\nWhen setting up the infrastructure, actually we DO report filesystem\nentity that is not a regular file or symbolic link like the following:\n\n if (S_ISDIR(iter->st.st_mode)) {\n continue;\n } else if (S_ISREG(iter->st.st_mode) ||\n S_ISLNK(iter->st.st_mode)) {\n ...;\n } else {\n // report file system error\n }\n\nWe do not check the directory, because the directory will be always\nvalid in the filesystem. we could not say that\n\n \"refs/heads/a/\" is a bad ref.\n\nSo, this check mainly need to check whether the symref points to a\ndirectory. Actually, Patrick has also gave the review about this\nquestion in the previous version:\n\n> What exactly are we guarding against here? Don't we already verify that\n> files in `refs/` have the correct type? Or are we checking that it does\n> not point to a directory?\n\nHowever, we should remove this line, because \"check_refname_format\" will\ntake care for us.\n\n git check-ref-format 'refs/heads/'\n\nIt will generate an error. So, we could entirely remove this line and\nlet \"check_refname_format\" do this. And we could also remove the\n\n if (lstat(pointee_path->buf, &st) < 0)\n goto out;\n\nThe code will be much more clean.\n\nThanks,\nJialuo\n"},{"id":"501776","messageId":"xmqqy14g3brp.fsf@gitster.g","threadId":"61943","inReplyTo":"Zs8c81Z-zb4uQpp6@tanuki","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-28T15:30:50Z","receivedAt":"2024-08-28T15:31:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Wed, Aug 28, 2024 at 12:07:58AM +0800, shejialuo wrote:\n>> @@ -170,6 +173,12 @@\n>> `nullSha1`::\n>> \t(WARN) Tree contains entries pointing to a null sha1.\n>> \n>> +`refMissingNewline`::\n>> +\t(INFO) A valid ref does not end with newline.\n>\n> This reads a bit funny to me. If the ref is valid, why do we complain?\n\nI think you understood after reading the series through and\nresponded to my \"curiously formatted\" comment. I understand that\nthese marked as INFO are not about \"to complain\" but are for us to\nask the user to report so that we can learn of any third-party tools\nthat may get in our way to later tighten the parsing rules\nretroactively. \n\n> Maybe this would read better if you said \"An otherwise valid ref does\n> not end with a newline\".\n\nSo I do agree that the text above is less than optimal. It is \"this\nis valid, but something we wouldn't have written. Who creates such\na ref?\"\n"},{"id":"501777","messageId":"Zs9ECY-EOyz3M6LQ@ArchLinux","threadId":"61943","inReplyTo":"Zs8dAc0ss9KbwIDs@tanuki","subject":"Re: [PATCH v2 3/4] ref: add symbolic ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-28T15:36:41Z","receivedAt":"2024-08-28T15:35:50Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Aug 28, 2024 at 02:50:09PM +0200, Patrick Steinhardt wrote:\n> On Wed, Aug 28, 2024 at 12:08:07AM +0800, shejialuo wrote:\n> > We have already introduced the checks for regular refs. There is no need\n> > to check the consistency of the target which the symbolic ref points to.\n> > Instead, we just check the content of the symbolic ref itself.\n> > \n> > In order to check the content of the symbolic ref, create a function\n> > \"files_fsck_symref_target\". It will first check whether the \"pointee\" is\n> > under the \"refs/\" directory and then we will check the \"pointee\" itself.\n> > \n> > There is no specification about the content of the symbolic ref.\n> > Although we do write \"ref: %s\\n\" to create a symbolic ref by using\n> > \"git-symbolic-ref(1)\" command. However, this is not mandatory. We still\n> > accept symbolic refs with null trailing garbage. Put it more specific,\n> > the following are correct:\n> > \n> > 1. \"ref: refs/heads/master \"\n> > 2. \"ref: refs/heads/master \\n \\n\"\n> > 3. \"ref: refs/heads/master\\n\\n\"\n> \n> Now that we're talking about tightening the rules for direct refs, I\n> wonder whether we'd also want to apply the same rules to symrefs.\n> Namely, when there is trailing whitespace we should generate an\n> INFO-level message about that, too. This is mostly for the sake of\n> consistency.\n> \n\nYes, actually this patch does this. I think I need to mention we reuse\nthe \"FSCK_INFO\" message id defined in the [PATCH v2 2/4].\n\n> [snip]\n> > diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> > index fc074fc571..85fd058c81 100644\n> > --- a/Documentation/fsck-msgids.txt\n> > +++ b/Documentation/fsck-msgids.txt\n> > @@ -28,6 +28,9 @@\n> > `badRefName`::\n> > \t(ERROR) A ref has an invalid format.\n> > \n> > +`badSymrefPointee`::\n> > +\t(ERROR) The pointee of a symref is bad.\n> \n> I think we'd want to clarify what \"bad\" is supposed to mean. Like, is a\n> missing symref pointee bad? If this is about the format of the pointee's\n> name, we might want to call this \"badSymrefPointeeName\".\n> \n\nI agree, bad is too general here, we need to make it concrete.\n\n> Also, I think we don't typically call the value of a symbolic ref\n> \"pointee\", but \"target\". Searching for \"pointee\" in our codebase only\n> gives a single hit, and that one is not related to symbolic refs.\n> \n\nThanks, I will fix this in the next version.\n\n> > +/*\n> > + * Check the symref \"pointee_name\" and \"pointee_path\". The caller should\n> > + * make sure that \"pointee_path\" is absolute. For symbolic ref, \"pointee_name\"\n> > + * would be the content after \"refs:\".\n> > + */\n> > +static int files_fsck_symref_target(struct fsck_options *o,\n> > +\t\t\t\t struct fsck_ref_report *report,\n> > +\t\t\t\t const char *refname,\n> > +\t\t\t\t struct strbuf *pointee_name,\n> > +\t\t\t\t struct strbuf *pointee_path)\n> > +{\n> > +\tconst char *newline_pos = NULL;\n> > +\tconst char *p = NULL;\n> > +\tstruct stat st;\n> > +\tint ret = 0;\n> > +\n> > +\tif (!skip_prefix(pointee_name->buf, \"refs/\", &p)) {\n> > +\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n> > +\t\t\t\t \"points to ref outside the refs directory\");\n> > +\t\tgoto out;\n> > +\t}\n> > +\n> > +\tnewline_pos = strrchr(p, '\\n');\n> > +\tif (!newline_pos || *(newline_pos + 1)) {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> > +\t\t\t\t \"missing newline\");\n> > +\t}\n> \n> The second condition `*(newline_pos + 1)` checks whether there is any\n> data after the newline, doesn't it? That indicates a different kind of\n> error than \"missing newline\", namely that there is trailing garbage. I\n> guess we'd want to report a separate info-level message for this.\n> \n> Also, shouldn't we use `strchr` instead of `strrchr()`? Otherwise, we're\n> only checking for trailing garbage after the _last_ newline, not after\n> the first one.\n> \n\nYes, I totally made a mistake here. I will try to think about a new\ndesign. I have already replied to Junio like the following:\n\n> > And strrchr() to find the last LF is not sufficient for that\n> > purpose. We would never write \"refs: refs/head/master \\n\",\n> > but the above code will find the LF, be satisified that the LF is\n> > followed by NUL, without realizing that SP there is not something we\n> > would have written!\n\n> I totally ignored this situation, and in current patch, we cannot check\n> this. I know why Patrick lets me use \"strchr\" but not \"strrchr\". I think\n> we should find the last '\\n'. But instead we need to find the first\n> '\\n'. However, in this example, we will still fail by using \"strchr\".\n> This part should be totally re-designed.\n\n> > +\tif (check_refname_format(pointee_name->buf, 0)) {\n> > +\t\t/*\n> > +\t\t * When containing null-garbage, \"check_refname_format\" will\n> > +\t\t * fail, we should trim the \"pointee\" to check again.\n> > +\t\t */\n> > +\t\tstrbuf_rtrim(pointee_name);\n> > +\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n> > +\t\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> > +\t\t\t\t\t \"trailing null-garbage\");\n> > +\t\t\tgoto out;\n> > +\t\t}\n> \n> Ah, I didn't get at first that we're doing the check a second time here.\n> As mentioned above, I think we should check for trailing garbage further\n> up already and more explicitly.\n> \n\nWell, I guess the implementation about this is totally wrong, which will\nmake the reviewers hard to understand. I will drop this way to\nexplicitly check the garbage.\n\nThanks,\nJialuo\n"},{"id":"501778","messageId":"xmqqbk1c3baj.fsf@gitster.g","threadId":"61943","inReplyTo":"Zs8dAc0ss9KbwIDs@tanuki","subject":"Re: [PATCH v2 3/4] ref: add symbolic ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-28T15:41:08Z","receivedAt":"2024-08-28T15:41:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Also, I think we don't typically call the value of a symbolic ref\n> \"pointee\", but \"target\". Searching for \"pointee\" in our codebase only\n> gives a single hit, and that one is not related to symbolic refs.\n\nYesterday while I was studying for reviewing this series, I saw some\nexisting code that call them \"referent\". There may also be \"target\".\n\n>> +\tif (!newline_pos || *(newline_pos + 1)) {\n>> +\t\tret = fsck_report_ref(o, report,\n>> +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n>> +\t\t\t\t \"missing newline\");\n>> +\t}\n>\n> The second condition `*(newline_pos + 1)` checks whether there is any\n> data after the newline, doesn't it? That indicates a different kind of\n> error than \"missing newline\", namely that there is trailing garbage. I\n> guess we'd want to report a separate info-level message for this.\n>\n> Also, shouldn't we use `strchr` instead of `strrchr()`? Otherwise, we're\n> only checking for trailing garbage after the _last_ newline, not after\n> the first one.\n\nNone of the above. It should strbuf_rtrim() and if we removed\nanything but just a single terminating LF, we are looking at\nsomething we wouldn't ahve written. The next check_refname_format()\ncall would then find \"trailing garbage\".\n\n - \"refs/heads/master \\n \" gets rtrimmed to \"refs/heads/master\",\n which is \"valid but curious\".\n\n - \"refs/heads/main trash\\n \" becomes \"refs/heads/main trash\",\n which is outright bad.\n\n"},{"id":"501781","messageId":"xmqqv7zk1ucv.fsf@gitster.g","threadId":"61943","inReplyTo":"Zs8c_vuqOSSWJjpd@tanuki","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-28T16:32:16Z","receivedAt":"2024-08-28T16:32:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n>> Git 3.0 boundary might be a good time to tighten interoperability\n>> rules such that we won't accept anything we wouldn't have written\n>> ourselves (not limited to loose ref format, but this applies to\n>> anything on-disk or on-wire), but we'd need enough preparation if we\n>> want to be able to do so in the future.\n>\n> I quite like this idea.\n\nI wouldn't say that I wrote it as a devil's advocate comment, but I\nwas hoping that somebody quote Postel in response, as the above\nadvocates a directly opposite position, which I wouldn't usually\ntake.\n\n> I guess another prereq for the change is to integrate `git refs verify`\n> with git-fsck(1), because otherwise people likely wouldn't see the new\n> messages in the first place.\n\nAbsolutely.\n"},{"id":"501782","messageId":"xmqqr0a81tqp.fsf@gitster.g","threadId":"61943","inReplyTo":"Zs801HqHg45v_q6X@ArchLinux","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-28T16:45:34Z","receivedAt":"2024-08-28T16:45:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n>> > @@ -170,6 +173,12 @@\n>> > `nullSha1`::\n>> > \t(WARN) Tree contains entries pointing to a null sha1.\n>> > \n>> > +`refMissingNewline`::\n>> > +\t(INFO) A valid ref does not end with newline.\n>> > +\n>> > +`trailingRefContent`::\n>> > +\t(INFO) A ref has trailing contents.\n>> > +\n>> > `treeNotSorted`::\n>> > \t(ERROR) A tree is not properly sorted.\n>> \n>> There is no mention of \"you shouldn't promote these to error\" here,\n>> which is good. But wouldn't we want to tell users to report such\n>> curiously formatted loose refs, after figuring out who created them,\n>> to help us to eventually make the check stricter in the future?\n>\n> From the review from the Patrick, I will add another patch in the\n> \"Documentation/BreakingChanges.txt\" later.\n\nAs that documentation is not end-user facing, it is orthogonal and\nunrelated.\n\nWhat I meant was that we need to tell the user that the refs they\nhave (and the third-party tools they used to create them) may be\ndeclared invalid in a future version of Git and they would want to\nreport it, in order to influence our possible future direction. And\nwe need to do so in an end-user facing documentation (i.e. the part\nof the patch quoted above) and/or in the info messages themselves.\n\n"},{"id":"501788","messageId":"xmqqle0gzdyh.fsf_-_@gitster.g","threadId":"61943","inReplyTo":"Zs348uXMBdCuwF-2@ArchLinux","subject":"[PATCH] SQUASH??? remove unused parameters","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-28T18:42:30Z","receivedAt":"2024-08-28T18:42:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"With -Wunused-parameter, the compiler notices that many parameters\nare unused. They are truly unused, and the signatures for the\nfunctions involved are not constrained externally, so we can simply\ndrop the parameters from the definition of these functions and their\ncallers.\n\nPlease squash these in when the topic gets rerolled. Thanks.\n\n refs/files-backend.c | 13 +++++--------\n 1 file changed, 5 insertions(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8641e3ba65..69dd283c9d 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1966,9 +1966,8 @@ static int create_ref_symlink(struct ref_lock *lock, const char *target)\n \treturn ret;\n }\n \n-static int create_symref_lock(struct files_ref_store *refs,\n-\t\t\t struct ref_lock *lock, const char *refname,\n-\t\t\t const char *target, struct strbuf *err)\n+static int create_symref_lock(struct ref_lock *lock, const char *target,\n+\t\t\t struct strbuf *err)\n {\n \tif (!fdopen_lock_file(&lock->lk, \"w\")) {\n \t\tstrbuf_addf(err, \"unable to fdopen %s: %s\",\n@@ -2584,8 +2583,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t}\n \n \tif (update->new_target && !(update->flags & REF_LOG_ONLY)) {\n-\t\tif (create_symref_lock(refs, lock, update->refname,\n-\t\t\t\t update->new_target, err)) {\n+\t\tif (create_symref_lock(lock, update->new_target, err)) {\n \t\t\tret = TRANSACTION_GENERIC_ERROR;\n \t\t\tgoto out;\n \t\t}\n@@ -3443,7 +3441,6 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n */\n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n-\t\t\t\t const char *refname,\n \t\t\t\t struct strbuf *pointee_name,\n \t\t\t\t struct strbuf *pointee_path,\n \t\t\t\t unsigned int symbolic_link)\n@@ -3565,7 +3562,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t} else {\n \t\t\tstrbuf_addf(&pointee_path, \"%s/%s\",\n \t\t\t\t ref_store->gitdir, referent.buf);\n-\t\t\tret = files_fsck_symref_target(o, &report, refname.buf,\n+\t\t\tret = files_fsck_symref_target(o, &report,\n \t\t\t\t\t\t &referent,\n \t\t\t\t\t\t &pointee_path,\n \t\t\t\t\t\t symbolic_link);\n@@ -3589,7 +3586,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t}\n \n \tstrbuf_addstr(&referent, pointee_name);\n-\tret = files_fsck_symref_target(o, &report, refname.buf,\n+\tret = files_fsck_symref_target(o, &report,\n \t\t\t\t &referent, &pointee_path,\n \t\t\t\t symbolic_link);\n \n-- \n2.46.0-563-gaeb9b172ce\n\n"},{"id":"501794","messageId":"xmqqbk1cz69c.fsf@gitster.g","threadId":"61943","inReplyTo":"Zs348uXMBdCuwF-2@ArchLinux","subject":"Re: [PATCH v2 0/4] add ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-28T21:28:47Z","receivedAt":"2024-08-28T21:28:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Here is another one.\n\nBy the way, Peff, do we have MAYBE_UNUSED that can be used in a case\nlike this one? Platforms without symbolic links supported may well\ndefine NO_SYMLINK_HEAD, which makes the incoming parameters unused.\n\nstatic int create_ref_symlink(struct ref_lock *lock, const char *target)\n{\n\tint ret = -1;\n#ifndef NO_SYMLINK_HEAD\n\tchar *ref_path = get_locked_file_path(&lock->lk);\n\tunlink(ref_path);\n\tret = symlink(target, ref_path);\n\tfree(ref_path);\n\n\tif (ret)\n\t\tfprintf(stderr, \"no symlink - falling back to symbolic ref\\n\");\n#endif\n\treturn ret;\n}\n\nWe can of course do the attached, which I'll let shejialuo to squash\ninto an appropriate patch in the series.\n\nThanks.\n\n\n refs/files-backend.c | 7 +++++--\n 1 file changed, 5 insertions(+), 2 deletions(-)\n\ndiff --git c/refs/files-backend.c w/refs/files-backend.c\nindex 69dd283c9d..110af32788 100644\n--- c/refs/files-backend.c\n+++ w/refs/files-backend.c\n@@ -1951,10 +1951,13 @@ static int commit_ref_update(struct files_ref_store *refs,\n \treturn 0;\n }\n \n+#ifdef NO_SYMLINK_HEAD\n+#define create_ref_symlink(lock, referent) (-1)\n+#else\n static int create_ref_symlink(struct ref_lock *lock, const char *target)\n {\n \tint ret = -1;\n-#ifndef NO_SYMLINK_HEAD\n+\n \tchar *ref_path = get_locked_file_path(&lock->lk);\n \tunlink(ref_path);\n \tret = symlink(target, ref_path);\n@@ -1962,9 +1965,9 @@ static int create_ref_symlink(struct ref_lock *lock, const char *target)\n \n \tif (ret)\n \t\tfprintf(stderr, \"no symlink - falling back to symbolic ref\\n\");\n-#endif\n \treturn ret;\n }\n+#endif\n \n static int create_symref_lock(struct ref_lock *lock, const char *target,\n \t\t\t struct strbuf *err)\n"},{"id":"501798","messageId":"20240829040215.GA4054823@coredump.intra.peff.net","threadId":"61943","inReplyTo":"xmqqbk1cz69c.fsf@gitster.g","subject":"Re: [PATCH v2 0/4] add ref content check for files backend","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-08-29T04:02:15Z","receivedAt":"2024-08-29T04:02:23Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Aug 28, 2024 at 02:28:47PM -0700, Junio C Hamano wrote:\n\n> By the way, Peff, do we have MAYBE_UNUSED that can be used in a case\n> like this one? Platforms without symbolic links supported may well\n> define NO_SYMLINK_HEAD, which makes the incoming parameters unused.\n\nYes, it would be fine to use MAYBE_UNUSED in a case like this.\n\nThe other option, and what I did for a conditional compilation in\nimap-send.c, is to just mention the variable like:\n\n /* mark as used to appease -Wunused-parameter with NO_SYMLINK_HEAD */\n (void)lock;\n (void)target;\n\nIn retrospect I think MAYBE_UNUSED is probably a little less magical,\nand I perhaps should have used it there.\n\nIn this particular case, though, where there's no actual code in one\nhalf of the #ifdef, I think just defining two separate functions is\ncleaner. I.e., what you did with a macro below, though I'd probably have\njust used a real function with UNUSED markers.\n\nAs an aside, I wonder if we should consider deprecating and eventually\ndropping support for core.prefersymlinkrefs. I can't think of a reason\nanybody would want to use it, and of course it makes no sense as we move\non to alternate backends like reftables. I sent patches ages ago:\n\n https://lore.kernel.org/git/20151229060055.GA17047@sigill.intra.peff.net/\n\nbut I think it may have just gotten lost in the shuffle, and I've\nsomehow been meaning to re-submit them for 9 years. :-/\n\n-Peff\n"},{"id":"501799","messageId":"xmqq5xrjzzxt.fsf@gitster.g","threadId":"61943","inReplyTo":"20240829040215.GA4054823@coredump.intra.peff.net","subject":"Re: [PATCH v2 0/4] add ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-29T04:59:58Z","receivedAt":"2024-08-29T05:00:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> As an aside, I wonder if we should consider deprecating and eventually\n> dropping support for core.prefersymlinkrefs. I can't think of a reason\n> anybody would want to use it, and of course it makes no sense as we move\n> on to alternate backends like reftables.\n\nYup. Perhaps add an entry or two to BreakingChanges document?\n\n Documentation/BreakingChanges.txt | 6 ++++++\n 1 file changed, 6 insertions(+)\n\ndiff --git c/Documentation/BreakingChanges.txt w/Documentation/BreakingChanges.txt\nindex 0532bfcf7f..2a85740f3c 100644\n--- c/Documentation/BreakingChanges.txt\n+++ w/Documentation/BreakingChanges.txt\n@@ -115,6 +115,12 @@ info/grafts as outdated, 2014-03-05) and will be removed.\n +\n Cf. <20140304174806.GA11561@sigill.intra.peff.net>.\n \n+* Support for core.prefersymlinkrefs will be dropped. Support for\n+ existing repositories that use symbolic links to represent a\n+ symbolic ref may or may not be dropped.\n++\n+Cf. <20240829040215.GA4054823@coredump.intra.peff.net>\n+\n == Superseded features that will not be deprecated\n \n Some features have gained newer replacements that aim to improve the design in\n"},{"id":"501801","messageId":"ZtAcowXWinP2Iguj@tanuki","threadId":"61943","inReplyTo":"xmqq5xrjzzxt.fsf@gitster.g","subject":"Re: [PATCH v2 0/4] add ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-29T07:00:58Z","receivedAt":"2024-08-29T07:01:03Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Aug 28, 2024 at 09:59:58PM -0700, Junio C Hamano wrote:\n> Jeff King writes:\n> \n> > As an aside, I wonder if we should consider deprecating and eventually\n> > dropping support for core.prefersymlinkrefs. I can't think of a reason\n> > anybody would want to use it, and of course it makes no sense as we move\n> > on to alternate backends like reftables.\n> \n> Yup. Perhaps add an entry or two to BreakingChanges document?\n> \n> Documentation/BreakingChanges.txt | 6 ++++++\n> 1 file changed, 6 insertions(+)\n> \n> diff --git c/Documentation/BreakingChanges.txt w/Documentation/BreakingChanges.txt\n> index 0532bfcf7f..2a85740f3c 100644\n> --- c/Documentation/BreakingChanges.txt\n> +++ w/Documentation/BreakingChanges.txt\n> @@ -115,6 +115,12 @@ info/grafts as outdated, 2014-03-05) and will be removed.\n> +\n> Cf. <20140304174806.GA11561@sigill.intra.peff.net>.\n> \n> +* Support for core.prefersymlinkrefs will be dropped. Support for\n> + existing repositories that use symbolic links to represent a\n> + symbolic ref may or may not be dropped.\n> ++\n> +Cf. <20240829040215.GA4054823@coredump.intra.peff.net>\n> +\n> == Superseded features that will not be deprecated\n\nYes, I'm very much in favor of that. As Peff said, I don't see a single\nreason why it would make sense to use symlinks nowadays. We have also\nsupported the \"new\" syntax for ages now, and I'd be surprised if there\nwere repos out there using it on purpose.\n\nWe should probably do the above together with a new check that starts to\nwarn about symbolic links in \"refs/\" such that users become aware of\nthis deprecation. We'd have to grow the infrastructure to also scan root\nrefs though, which to the best of my knowledge we don't currently scan.\n\nPatrick\n"},{"id":"501831","messageId":"ZtBJLVou0UnImuIn@tanuki","threadId":"61943","inReplyTo":"xmqqbk1c3baj.fsf@gitster.g","subject":"Re: [PATCH v2 3/4] ref: add symbolic ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-29T10:11:02Z","receivedAt":"2024-08-29T10:11:06Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Aug 28, 2024 at 08:41:08AM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> > Also, I think we don't typically call the value of a symbolic ref\n> > \"pointee\", but \"target\". Searching for \"pointee\" in our codebase only\n> > gives a single hit, and that one is not related to symbolic refs.\n> \n> Yesterday while I was studying for reviewing this series, I saw some\n> existing code that call them \"referent\". There may also be \"target\".\n\nAh, true, I totally forgot about \"referent\". I guess we use both, but it\nwould of course be great if we only had a single term to refer them.\nReferent seems to be used more widely, at least in the refs subsystem.\n\n> >> +\tif (!newline_pos || *(newline_pos + 1)) {\n> >> +\t\tret = fsck_report_ref(o, report,\n> >> +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> >> +\t\t\t\t \"missing newline\");\n> >> +\t}\n> >\n> > The second condition `*(newline_pos + 1)` checks whether there is any\n> > data after the newline, doesn't it? That indicates a different kind of\n> > error than \"missing newline\", namely that there is trailing garbage. I\n> > guess we'd want to report a separate info-level message for this.\n> >\n> > Also, shouldn't we use `strchr` instead of `strrchr()`? Otherwise, we're\n> > only checking for trailing garbage after the _last_ newline, not after\n> > the first one.\n> \n> None of the above. It should strbuf_rtrim() and if we removed\n> anything but just a single terminating LF, we are looking at\n> something we wouldn't ahve written. The next check_refname_format()\n> call would then find \"trailing garbage\".\n\nFair.\n\n> - \"refs/heads/master \\n \" gets rtrimmed to \"refs/heads/master\",\n> which is \"valid but curious\".\n\nOkay. This _may_ be something to generate an info message for, mostly in\nthe same spirit as we want to do it for direct refs.\n\n> - \"refs/heads/main trash\\n \" becomes \"refs/heads/main trash\",\n> which is outright bad.\n\nYeah, this one should be an error indeed.\n\nPatrick\n"},{"id":"501832","messageId":"ZtBLHD-sXeNutI0j@tanuki","threadId":"61943","inReplyTo":"xmqqv7zk1ucv.fsf@gitster.g","subject":"Re: [PATCH v2 2/4] ref: add regular ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-08-29T10:19:13Z","receivedAt":"2024-08-29T10:19:19Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Aug 28, 2024 at 09:32:16AM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> >> Git 3.0 boundary might be a good time to tighten interoperability\n> >> rules such that we won't accept anything we wouldn't have written\n> >> ourselves (not limited to loose ref format, but this applies to\n> >> anything on-disk or on-wire), but we'd need enough preparation if we\n> >> want to be able to do so in the future.\n> >\n> > I quite like this idea.\n> \n> I wouldn't say that I wrote it as a devil's advocate comment, but I\n> was hoping that somebody quote Postel in response, as the above\n> advocates a directly opposite position, which I wouldn't usually\n> take.\n\nFor context, this is the quote you probably refer to: \"be conservative\nin what you do, be liberal in what you accept from others\".\n\nIn any case, I still think it is sensible to at least warn about refs\nlike this. It is unexpected to me and may indicate real issues in the\nunderstanding of others that end up writing to the refdb. If there are\nimplementations of Git out there that intentionally use our lax parsing\nto e.g. stuff additional metadata into refs, then we need to tell them\nthat this is not okay.\n\nThis may have been fine in the past where there was only a single ref\nbackend, but now with multiple ref backends the picture has changed in\nmy opinion.\n\nPatrick\n"},{"id":"501835","messageId":"xmqqseunxtks.fsf_-_@gitster.g","threadId":"61943","inReplyTo":"20240829040215.GA4054823@coredump.intra.peff.net","subject":"[PATCH 8/6] CodingGuidelines: also mention MAYBE_UNUSED","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-29T15:00:19Z","receivedAt":"2024-08-29T15:00:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Wed, Aug 28, 2024 at 02:28:47PM -0700, Junio C Hamano wrote:\n>\n>> By the way, Peff, do we have MAYBE_UNUSED that can be used in a case\n>> like this one? Platforms without symbolic links supported may well\n>> define NO_SYMLINK_HEAD, which makes the incoming parameters unused.\n>\n> Yes, it would be fine to use MAYBE_UNUSED in a case like this.\n\nIt turns out that I was, without realizing it myself, making an\noblique reference to your patch 7/6 ;-)\n\nPerhaps something along this line?\n\n---- >8 ----\nSubject: CodingGuidelines: also mention MAYBE_UNUSED\n\nA function that uses a parameter in one build may lose all uses of\nthe parameter in another build, depending on the configuration. A\nworkaround for such a case, MAYBE_UNUSED, should also be mentioned\nwhen we recommend the use of UNUSED to our developers.\n\nSigned-off-by: Junio C Hamano \n---\n Documentation/CodingGuidelines | 5 +++--\n git-compat-util.h | 21 +++++++++++++++++++++\n 2 files changed, 24 insertions(+), 2 deletions(-)\n\ndiff --git c/Documentation/CodingGuidelines w/Documentation/CodingGuidelines\nindex d0fc7cfe60..3263245b03 100644\n--- c/Documentation/CodingGuidelines\n+++ w/Documentation/CodingGuidelines\n@@ -262,8 +262,9 @@ For C programs:\n like \"error: unused parameter 'foo' [-Werror=unused-parameter]\",\n which indicates that a function ignores its argument. If the unused\n parameter can't be removed (e.g., because the function is used as a\n- callback and has to match a certain interface), you can annotate the\n- individual parameters with the UNUSED keyword, like \"int foo UNUSED\".\n+ callback and has to match a certain interface), you can annotate\n+ the individual parameters with the UNUSED (or MAYBE_UNUSED)\n+ keyword, like \"int foo UNUSED\".\n \n - We try to support a wide range of C compilers to compile Git with,\n including old ones. As of Git v2.35.0 Git requires C99 (we check\ndiff --git c/git-compat-util.h w/git-compat-util.h\nindex 71b4d23f03..23307ce780 100644\n--- c/git-compat-util.h\n+++ w/git-compat-util.h\n@@ -195,6 +195,17 @@ struct strbuf;\n #define _NETBSD_SOURCE 1\n #define _SGI_SOURCE 1\n \n+/*\n+ * UNUSED marks a function parameter that is always unused.\n+ *\n+ * A callback interface may dictate that a function accepts a\n+ * parameter at that position, but the implementation of the function\n+ * may not need to use the parameter. In such a case, mark the parameter\n+ * with UNUSED.\n+ *\n+ * When a parameter may be used or unused, depending on conditional\n+ * compilation, consider using MAYBE_UNUSED instead.\n+ */\n #if GIT_GNUC_PREREQ(4, 5)\n #define UNUSED __attribute__((unused)) \\\n \t__attribute__((deprecated (\"parameter declared as UNUSED\")))\n@@ -649,6 +660,16 @@ static inline int git_has_dir_sep(const char *path)\n #define RESULT_MUST_BE_USED\n #endif\n \n+/*\n+ * MAYBE_UNUSED marks a function parameter that may be unused, but\n+ * whose use is not an error.\n+ *\n+ * Depending on a configuration, all uses of a function parameter may\n+ * become #ifdef'ed away. Marking such a parameter with UNUSED would\n+ * give a warning in a compilation where the parameter is indeed used,\n+ * and not marking such a parameter would give a warning in a\n+ * compilation where the parameter is unused.\n+ */\n #define MAYBE_UNUSED __attribute__((__unused__))\n \n #include \"compat/bswap.h\"\n"},{"id":"501836","messageId":"xmqqikvjxt98.fsf@gitster.g","threadId":"61943","inReplyTo":"ZtAcowXWinP2Iguj@tanuki","subject":"Re: [PATCH v2 0/4] add ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-29T15:07:15Z","receivedAt":"2024-08-29T15:07:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n>> +* Support for core.prefersymlinkrefs will be dropped. Support for\n>> + existing repositories that use symbolic links to represent a\n>> + symbolic ref may or may not be dropped.\n>> ++\n>> +Cf. <20240829040215.GA4054823@coredump.intra.peff.net>\n>> +\n>> == Superseded features that will not be deprecated\n> ...\n> We should probably do the above together with a new check that starts to\n> warn about symbolic links in \"refs/\" such that users become aware of\n> this deprecation. We'd have to grow the infrastructure to also scan root\n> refs though, which to the best of my knowledge we don't currently scan.\n\nYup, that is why the above suggestion is on _this_ thread that is\nabout the \"check for curiously formatted symrefs, in the hope that\nwe can retroactively tighten our checks later\" topic.\n"},{"id":"501838","messageId":"ZtCYMiOXVUM7SD3v@ArchLinux","threadId":"61943","inReplyTo":"xmqq5xrjzzxt.fsf@gitster.g","subject":"Re: [PATCH v2 0/4] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-08-29T15:48:02Z","receivedAt":"2024-08-29T15:47:09Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Aug 28, 2024 at 09:59:58PM -0700, Junio C Hamano wrote:\n> Jeff King writes:\n> \n> > As an aside, I wonder if we should consider deprecating and eventually\n> > dropping support for core.prefersymlinkrefs. I can't think of a reason\n> > anybody would want to use it, and of course it makes no sense as we move\n> > on to alternate backends like reftables.\n> \n> Yup. Perhaps add an entry or two to BreakingChanges document?\n> \n> Documentation/BreakingChanges.txt | 6 ++++++\n> 1 file changed, 6 insertions(+)\n> \n> diff --git c/Documentation/BreakingChanges.txt w/Documentation/BreakingChanges.txt\n> index 0532bfcf7f..2a85740f3c 100644\n> --- c/Documentation/BreakingChanges.txt\n> +++ w/Documentation/BreakingChanges.txt\n> @@ -115,6 +115,12 @@ info/grafts as outdated, 2014-03-05) and will be removed.\n> +\n> Cf. <20140304174806.GA11561@sigill.intra.peff.net>.\n> \n> +* Support for core.prefersymlinkrefs will be dropped. Support for\n> + existing repositories that use symbolic links to represent a\n> + symbolic ref may or may not be dropped.\n> ++\n> +Cf. <20240829040215.GA4054823@coredump.intra.peff.net>\n> +\n> == Superseded features that will not be deprecated\n> \n> Some features have gained newer replacements that aim to improve the design in\n\nFrom my current understanding, I think I need to rebase two patches\nprovided by your here:\n\n https://lore.kernel.org/git/xmqqle0gzdyh.fsf_-_@gitster.g/\n https://lore.kernel.org/git/xmqqbk1cz69c.fsf@gitster.g/\n\nI think in this patch, we just info the user that we will drop\n\"core.prefersymlinkrefs\" later, so I should not concern about this\npatch and also the [PATCH 8/6].\n\nThanks,\nJialuo\n"},{"id":"501841","messageId":"xmqqmskvwbnp.fsf@gitster.g","threadId":"61943","inReplyTo":"ZtCYMiOXVUM7SD3v@ArchLinux","subject":"Re: [PATCH v2 0/4] add ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-29T16:12:42Z","receivedAt":"2024-08-29T16:12:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> From my current understanding, I think I need to rebase two patches\n> provided by your here:\n>\n> https://lore.kernel.org/git/xmqqle0gzdyh.fsf_-_@gitster.g/\n> https://lore.kernel.org/git/xmqqbk1cz69c.fsf@gitster.g/\n\nThey are to be squashed into your patch, \"suggested edit\" for your\nchanges, not \"to be rebased\". In other words, we do not want to see\na patch (from your v2 as-is) to create problems and then another\npatch (taken from one of these links) applied on top to remedy them.\nWe instead want to see a patch (start from your v2 but with the\nchanges from these links) that does not introduce problems in the\nfirst place.\n"},{"id":"501845","messageId":"20240829175215.GA415423@coredump.intra.peff.net","threadId":"61943","inReplyTo":"xmqqseunxtks.fsf_-_@gitster.g","subject":"Re: [PATCH 8/6] CodingGuidelines: also mention MAYBE_UNUSED","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-08-29T17:52:15Z","receivedAt":"2024-08-29T17:52:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Aug 29, 2024 at 08:00:19AM -0700, Junio C Hamano wrote:\n\n> > Yes, it would be fine to use MAYBE_UNUSED in a case like this.\n> \n> It turns out that I was, without realizing it myself, making an\n> oblique reference to your patch 7/6 ;-)\n> \n> Perhaps something along this line?\n\nYeah, this looks good. A few small comments below (but I'm not sure\nanything needs to be changed).\n\n> diff --git c/Documentation/CodingGuidelines w/Documentation/CodingGuidelines\n> index d0fc7cfe60..3263245b03 100644\n> --- c/Documentation/CodingGuidelines\n> +++ w/Documentation/CodingGuidelines\n> @@ -262,8 +262,9 @@ For C programs:\n> like \"error: unused parameter 'foo' [-Werror=unused-parameter]\",\n> which indicates that a function ignores its argument. If the unused\n> parameter can't be removed (e.g., because the function is used as a\n> - callback and has to match a certain interface), you can annotate the\n> - individual parameters with the UNUSED keyword, like \"int foo UNUSED\".\n> + callback and has to match a certain interface), you can annotate\n> + the individual parameters with the UNUSED (or MAYBE_UNUSED)\n> + keyword, like \"int foo UNUSED\".\n\nHere I was going to suggest explaining why you'd use one or the other\n(because I'm afraid of people using MAYBE_UNUSED when UNUSED would be\nmore appropriate). But I think the extra comments you added later are\neven better, as it lets us explain without cluttering up the\nCodingGuidelines document.\n\n> +/*\n> + * UNUSED marks a function parameter that is always unused.\n> + *\n> + * A callback interface may dictate that a function accepts a\n> + * parameter at that position, but the implementation of the function\n> + * may not need to use the parameter. In such a case, mark the parameter\n> + * with UNUSED.\n> + *\n> + * When a parameter may be used or unused, depending on conditional\n> + * compilation, consider using MAYBE_UNUSED instead.\n> + */\n\nLooks good.\n\n> +/*\n> + * MAYBE_UNUSED marks a function parameter that may be unused, but\n> + * whose use is not an error.\n> + *\n> + * Depending on a configuration, all uses of a function parameter may\n> + * become #ifdef'ed away. Marking such a parameter with UNUSED would\n> + * give a warning in a compilation where the parameter is indeed used,\n> + * and not marking such a parameter would give a warning in a\n> + * compilation where the parameter is unused.\n> + */\n> #define MAYBE_UNUSED __attribute__((__unused__))\n\nThis is all good as pertains to function parameters. But the original\nreason we added MAYBE_UNUSED was actually for static functions that were\nauto-generated by the commit-slab macros. Saying \"...marks a function\nparameter\" implies to me that it's the only use. I don't know if we want\nto be more expansive here or not. Adding auto-generated macro functions\nshould be quite a rarity, I'd think.\n\n-Peff\n"},{"id":"501847","messageId":"xmqq8qwfw6e9.fsf@gitster.g","threadId":"61943","inReplyTo":"20240829175215.GA415423@coredump.intra.peff.net","subject":"Re: [PATCH 8/6] CodingGuidelines: also mention MAYBE_UNUSED","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-29T18:06:22Z","receivedAt":"2024-08-29T18:06:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n>> +/*\n>> + * MAYBE_UNUSED marks a function parameter that may be unused, but\n>> + * whose use is not an error.\n>> + *\n>> + * Depending on a configuration, all uses of a function parameter may\n>> + * become #ifdef'ed away. Marking such a parameter with UNUSED would\n>> + * give a warning in a compilation where the parameter is indeed used,\n>> + * and not marking such a parameter would give a warning in a\n>> + * compilation where the parameter is unused.\n>> + */\n>> #define MAYBE_UNUSED __attribute__((__unused__))\n>\n> This is all good as pertains to function parameters. But the original\n> reason we added MAYBE_UNUSED was actually for static functions that were\n> auto-generated by the commit-slab macros. Saying \"...marks a function\n> parameter\" implies to me that it's the only use. I don't know if we want\n> to be more expansive here or not. Adding auto-generated macro functions\n> should be quite a rarity, I'd think.\n\nTrue. You can annotate types, variables, and functions with the\nattributes as well. How about saying something like this\n\n MAYBE_UNUSED marks a function parameter that may be unused but\n whose use is not an error. It also can be applied to functions,\n types and variables.\n\nand then keep the explanation of why you may want to use the maybe-\nvariant as-is, using a function parameter as an example? Or I could\nrewrite \"parameter\" and \"function parameter\" in it with \"thing\"\n(with double quotes around), like:\n\n Depending on a configuration, all uses of a \"thing\" may become\n #ifdef'ed away....\n\nUnlike the use of deprecated attribute, our definition of\nMAYBE_UNUSED is not guarded with anything. Shouldn't we at least do\n\n #if defined(__GNUC__)\n #define MAYBE_UNUSED __attribute__((__unused__))\n #else\n #define MAYBE_UNUSED /* noop */\n #endif\n\nor something, by the way?\n\nThanks.\n"},{"id":"501849","messageId":"xmqq4j73w5up.fsf_-_@gitster.g","threadId":"61943","inReplyTo":"xmqq8qwfw6e9.fsf@gitster.g","subject":"[PATCH v2] CodingGuidelines: also mention MAYBE_UNUSED","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-29T18:18:06Z","receivedAt":"2024-08-29T18:18:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"A function that uses a parameter in one build may lose all uses of\nthe parameter in another build, depending on the configuration. A\nworkaround for such a case, MAYBE_UNUSED, should also be mentioned\nwhen we recommend the use of UNUSED to our developers.\n\nKeep the addition to the guideline short and document the criteria\nto choose between UNUSED and MAYBE_UNUSED near their definition.\n\nSigned-off-by: Junio C Hamano \n---\n Documentation/CodingGuidelines | 5 +++--\n git-compat-util.h | 24 ++++++++++++++++++++++++\n 2 files changed, 27 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines\nindex d0fc7cfe60..3263245b03 100644\n--- a/Documentation/CodingGuidelines\n+++ b/Documentation/CodingGuidelines\n@@ -262,8 +262,9 @@ For C programs:\n like \"error: unused parameter 'foo' [-Werror=unused-parameter]\",\n which indicates that a function ignores its argument. If the unused\n parameter can't be removed (e.g., because the function is used as a\n- callback and has to match a certain interface), you can annotate the\n- individual parameters with the UNUSED keyword, like \"int foo UNUSED\".\n+ callback and has to match a certain interface), you can annotate\n+ the individual parameters with the UNUSED (or MAYBE_UNUSED)\n+ keyword, like \"int foo UNUSED\".\n \n - We try to support a wide range of C compilers to compile Git with,\n including old ones. As of Git v2.35.0 Git requires C99 (we check\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 71b4d23f03..e4a306dd56 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -195,6 +195,19 @@ struct strbuf;\n #define _NETBSD_SOURCE 1\n #define _SGI_SOURCE 1\n \n+/*\n+ * UNUSED marks a function parameter that is always unused. It also\n+ * can be used to annotate a function, a variable, or a type that is\n+ * always unused.\n+ *\n+ * A callback interface may dictate that a function accepts a\n+ * parameter at that position, but the implementation of the function\n+ * may not need to use the parameter. In such a case, mark the parameter\n+ * with UNUSED.\n+ *\n+ * When a parameter may be used or unused, depending on conditional\n+ * compilation, consider using MAYBE_UNUSED instead.\n+ */\n #if GIT_GNUC_PREREQ(4, 5)\n #define UNUSED __attribute__((unused)) \\\n \t__attribute__((deprecated (\"parameter declared as UNUSED\")))\n@@ -649,6 +662,17 @@ static inline int git_has_dir_sep(const char *path)\n #define RESULT_MUST_BE_USED\n #endif\n \n+/*\n+ * MAYBE_UNUSED marks a function parameter that may be unused, but\n+ * whose use is not an error. It also can be used to annotate a\n+ * function, a variable, or a type that may be unused.\n+ *\n+ * Depending on a configuration, all uses of such a thing may become\n+ * #ifdef'ed away. Marking it with UNUSED would give a warning in a\n+ * compilation where it is indeed used, and not marking it at all\n+ * would give a warning in a compilation where it is unused. In such\n+ * a case, MAYBE_UNUSED is the appropriate annotation to use.\n+ */\n #define MAYBE_UNUSED __attribute__((__unused__))\n \n #include \"compat/bswap.h\"\n\nInterdiff against v1:\n diff --git a/git-compat-util.h b/git-compat-util.h\n index 23307ce780..e4a306dd56 100644\n --- a/git-compat-util.h\n +++ b/git-compat-util.h\n @@ -196,7 +196,9 @@ struct strbuf;\n #define _SGI_SOURCE 1\n \n /*\n - * UNUSED marks a function parameter that is always unused.\n + * UNUSED marks a function parameter that is always unused. It also\n + * can be used to annotate a function, a variable, or a type that is\n + * always unused.\n *\n * A callback interface may dictate that a function accepts a\n * parameter at that position, but the implementation of the function\n @@ -662,13 +664,14 @@ static inline int git_has_dir_sep(const char *path)\n \n /*\n * MAYBE_UNUSED marks a function parameter that may be unused, but\n - * whose use is not an error.\n + * whose use is not an error. It also can be used to annotate a\n + * function, a variable, or a type that may be unused.\n *\n - * Depending on a configuration, all uses of a function parameter may\n - * become #ifdef'ed away. Marking such a parameter with UNUSED would\n - * give a warning in a compilation where the parameter is indeed used,\n - * and not marking such a parameter would give a warning in a\n - * compilation where the parameter is unused.\n + * Depending on a configuration, all uses of such a thing may become\n + * #ifdef'ed away. Marking it with UNUSED would give a warning in a\n + * compilation where it is indeed used, and not marking it at all\n + * would give a warning in a compilation where it is unused. In such\n + * a case, MAYBE_UNUSED is the appropriate annotation to use.\n */\n #define MAYBE_UNUSED __attribute__((__unused__))\n \n-- \n2.46.0-563-gaeb9b172ce\n\n"},{"id":"501850","messageId":"xmqqttf3uquc.fsf_-_@gitster.g","threadId":"61943","inReplyTo":"xmqq4j73w5up.fsf_-_@gitster.g","subject":"[PATCH 9/6] git-compat-util: guard definition of MAYBE_UNUSED with __GNUC__","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-29T18:27:39Z","receivedAt":"2024-08-29T18:27:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Just like we only define UNUSED macro when __GNUC__ is defined,\nand fall back to an empty definition otherwise, we should do the\nsame for MAYBE_UNUSED.\n\nSigned-off-by: Junio C Hamano \n---\n * Before I forget that we have discussed this, just as a\n documentation (read: this is not a patch to be applied).\n\n I think this only matters when a compiler satisfies all three\n traits:\n\n - does not define __GNUC__\n - does have its own __attribute__() macro\n - barfs on __attribute__((__unused__))\n\n Otherwise we will define __attribute__(x) away to empty to cause\n no harm.\n\n Since we have survived without complaints without such a guard\n for quite some time, it may be a sign that no compiler that knows\n __attribute__() that people ever tried to compile Git with barfs\n with __attribute__((__unused__)). I dunno.\n\n git-compat-util.h | 4 ++++\n 1 file changed, 4 insertions(+)\n\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex e4a306dd56..74ed581b5d 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -673,7 +673,11 @@ static inline int git_has_dir_sep(const char *path)\n * would give a warning in a compilation where it is unused. In such\n * a case, MAYBE_UNUSED is the appropriate annotation to use.\n */\n+#ifdef __GNUC__\n #define MAYBE_UNUSED __attribute__((__unused__))\n+#else\n+#define MAYBE_UNUSED\n+#endif\n \n #include \"compat/bswap.h\"\n \n-- \n2.46.0-563-gaeb9b172ce\n\n"},{"id":"501861","messageId":"20240829194054.GC423429@coredump.intra.peff.net","threadId":"61943","inReplyTo":"xmqq4j73w5up.fsf_-_@gitster.g","subject":"Re: [PATCH v2] CodingGuidelines: also mention MAYBE_UNUSED","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-08-29T19:40:54Z","receivedAt":"2024-08-29T19:40:55Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Aug 29, 2024 at 11:18:06AM -0700, Junio C Hamano wrote:\n\n> +/*\n> + * MAYBE_UNUSED marks a function parameter that may be unused, but\n> + * whose use is not an error. It also can be used to annotate a\n> + * function, a variable, or a type that may be unused.\n> + *\n> + * Depending on a configuration, all uses of such a thing may become\n> + * #ifdef'ed away. Marking it with UNUSED would give a warning in a\n> + * compilation where it is indeed used, and not marking it at all\n> + * would give a warning in a compilation where it is unused. In such\n> + * a case, MAYBE_UNUSED is the appropriate annotation to use.\n> + */\n> #define MAYBE_UNUSED __attribute__((__unused__))\n\nThanks, I think this is good. There's more nuanced discussion about when\nthe \"MAYBE\" variant could be used for non-parameters, but I don't know\nthat it's worth trying to enumerate every place we've found it useful.\n\n-Peff\n"},{"id":"501862","messageId":"20240829194536.GD423429@coredump.intra.peff.net","threadId":"61943","inReplyTo":"xmqqttf3uquc.fsf_-_@gitster.g","subject":"Re: [PATCH 9/6] git-compat-util: guard definition of MAYBE_UNUSED with __GNUC__","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-08-29T19:45:36Z","receivedAt":"2024-08-29T19:45:38Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Aug 29, 2024 at 11:27:39AM -0700, Junio C Hamano wrote:\n\n> Just like we only define UNUSED macro when __GNUC__ is defined,\n> and fall back to an empty definition otherwise, we should do the\n> same for MAYBE_UNUSED.\n> \n> Signed-off-by: Junio C Hamano \n> ---\n> * Before I forget that we have discussed this, just as a\n> documentation (read: this is not a patch to be applied).\n> \n> I think this only matters when a compiler satisfies all three\n> traits:\n> \n> - does not define __GNUC__\n> - does have its own __attribute__() macro\n> - barfs on __attribute__((__unused__))\n> \n> Otherwise we will define __attribute__(x) away to empty to cause\n> no harm.\n> \n> Since we have survived without complaints without such a guard\n> for quite some time, it may be a sign that no compiler that knows\n> __attribute__() that people ever tried to compile Git with barfs\n> with __attribute__((__unused__)). I dunno.\n\nYeah, I was surprised that this didn't have a guard and was not\ncurrently barfing on other compilers. And the answer is that we already\nturn __attribute__ into a noop on non-GNUC platforms.\n\nWhich made me wonder if UNUSED really needs its guards. It does, because\nit is defined early in the file, before the __attribute__ handling. I\ndon't think we want to move it down, since it needs to be available for\nuse by inline'd compat wrappers. But arguably we should move the\nattribute macro earlier in the file?\n\nI don't know that it is really worth spending too much time futzing\nwith, though.\n\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index e4a306dd56..74ed581b5d 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -673,7 +673,11 @@ static inline int git_has_dir_sep(const char *path)\n> * would give a warning in a compilation where it is unused. In such\n> * a case, MAYBE_UNUSED is the appropriate annotation to use.\n> */\n> +#ifdef __GNUC__\n> #define MAYBE_UNUSED __attribute__((__unused__))\n> +#else\n> +#define MAYBE_UNUSED\n> +#endif\n\nSo yeah, I'm not necessarily opposed to this, but I don't think it's\nreally doing anything in practice.\n\n-Peff\n"},{"id":"501863","messageId":"20240829194855.GE423429@coredump.intra.peff.net","threadId":"61943","inReplyTo":"ZtAcowXWinP2Iguj@tanuki","subject":"Re: [PATCH v2 0/4] add ref content check for files backend","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-08-29T19:48:55Z","receivedAt":"2024-08-29T19:48:56Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Aug 29, 2024 at 09:00:58AM +0200, Patrick Steinhardt wrote:\n\n> > diff --git c/Documentation/BreakingChanges.txt w/Documentation/BreakingChanges.txt\n> > index 0532bfcf7f..2a85740f3c 100644\n> > --- c/Documentation/BreakingChanges.txt\n> > +++ w/Documentation/BreakingChanges.txt\n> > @@ -115,6 +115,12 @@ info/grafts as outdated, 2014-03-05) and will be removed.\n> > +\n> > Cf. <20140304174806.GA11561@sigill.intra.peff.net>.\n> > \n> > +* Support for core.prefersymlinkrefs will be dropped. Support for\n> > + existing repositories that use symbolic links to represent a\n> > + symbolic ref may or may not be dropped.\n> > ++\n> > +Cf. <20240829040215.GA4054823@coredump.intra.peff.net>\n> > +\n> > == Superseded features that will not be deprecated\n> \n> Yes, I'm very much in favor of that. As Peff said, I don't see a single\n> reason why it would make sense to use symlinks nowadays. We have also\n> supported the \"new\" syntax for ages now, and I'd be surprised if there\n> were repos out there using it on purpose.\n> \n> We should probably do the above together with a new check that starts to\n> warn about symbolic links in \"refs/\" such that users become aware of\n> this deprecation. We'd have to grow the infrastructure to also scan root\n> refs though, which to the best of my knowledge we don't currently scan.\n\nI think the first step of the proposal (and what I had written in the\npatches that I linked) was just that we would stop _writing_ symlinks.\nAnd there we'd only need to warn people who have that config option set.\n\nWhether to drop the reading side is less clear to me. I think in the\nlong run it is good as a cleanup (and one less source of weird behavior\nthat malicious local repos can trigger). But that decision can be made\nseparately. I think it would be OK to just issue a deprecation warning\nwhenever we actually follow a symlink (because I think we do so\nmanually, since we need to know the target name as part of the\nresolution process).\n\n-Peff\n"},{"id":"501869","messageId":"xmqqcylrulnd.fsf@gitster.g","threadId":"61943","inReplyTo":"20240829194536.GD423429@coredump.intra.peff.net","subject":"Re: [PATCH 9/6] git-compat-util: guard definition of MAYBE_UNUSED with __GNUC__","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-08-29T20:19:50Z","receivedAt":"2024-08-29T20:19:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Aug 29, 2024 at 11:27:39AM -0700, Junio C Hamano wrote:\n>\n>> Just like we only define UNUSED macro when __GNUC__ is defined,\n>> and fall back to an empty definition otherwise, we should do the\n>> same for MAYBE_UNUSED.\n>> \n>> Signed-off-by: Junio C Hamano \n>> ---\n>> * Before I forget that we have discussed this, just as a\n>> documentation (read: this is not a patch to be applied).\n>> \n>> I think this only matters when a compiler satisfies all three\n>> traits:\n>> \n>> - does not define __GNUC__\n>> - does have its own __attribute__() macro\n>> - barfs on __attribute__((__unused__))\n>> \n>> Otherwise we will define __attribute__(x) away to empty to cause\n>> no harm.\n>> \n>> Since we have survived without complaints without such a guard\n>> for quite some time, it may be a sign that no compiler that knows\n>> __attribute__() that people ever tried to compile Git with barfs\n>> with __attribute__((__unused__)). I dunno.\n>\n> Yeah, I was surprised that this didn't have a guard and was not\n> currently barfing on other compilers. And the answer is that we already\n> turn __attribute__ into a noop on non-GNUC platforms.\n\nPlus these non-GNUC platforms either\n\n (1) do not have their own __attribute__, which lets us turn\n __attribute__() into noop, or\n\n (2) have their own __attribute__, but they happen to support\n __attribute__((__unused__)).\n\nIf somebody has __attribute__() and does not support (__unused__) in\nit, use of MAYBE_UNUSED would be broken (maybe their __attribute__()\nsupports other things but not unused).\n\n> Which made me wonder if UNUSED really needs its guards. It does, because\n> it is defined early in the file, before the __attribute__ handling. I\n> don't think we want to move it down, since it needs to be available for\n> use by inline'd compat wrappers. But arguably we should move the\n> attribute macro earlier in the file?\n\nAnd moving __attribute__ definition earlier in the file would not\nhelp such a platform with broken __attribute__((__unused__))\n\n> I don't know that it is really worth spending too much time futzing\n> with, though.\n\nI am inclined to think it is not. So let's scrap the patch. The\nlist archive will hopefully remember when it becomes necessary ;-)\n"},{"id":"502029","messageId":"Ztb-mgl50cwGVO8A@ArchLinux","threadId":"61943","inReplyTo":"Zs348uXMBdCuwF-2@ArchLinux","subject":"[PATCH v3 0/4] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-03T12:18:34Z","receivedAt":"2024-09-03T12:17:39Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis new version does the following things:\n\n1. [PATCH v3 1/4]\n\n + the motivation of the previous commit message is too strong, this\n version improves this.\n\n2. [PATCH v3 2/4]\n\n + Enhance the commit message to make things clearer.\n + Enhance the descriptions of the fsck message ids. Tell the user we\n may consider converting info to error later to let the user know\n this and report some feedback to us.\n + Use \"goto\" to avoid unnecessary indentation.\n + Use \"test_commit\" to create a single commit in the test file to\n avoid unnecessary setups.\n + Enhance the test cases by adding normal situation case test and\n add a new aggregation test to double verify the functionality.\n + Clean the \"> $file\" to \">$file\" to make the code style correct.\n\n3. [PATCH v3 3/4]\n\n + Enhance the commit message by better describing the motivation.\n + Change the fsck message name \"badSymrefPointee\" to\n \"badSymrefTarget\" to be align with the codebase. And talking more\n about what is the \"bad\" in the documentation.\n + Use idea from Junio to check the textual symref content.\n + Still keep the following code:\n\n if (lstat(referent->buf, &st))\n goto out;\n\n if (S_ISDIR(st.st_mode)) {\n ret = report(...);\n goto out;\n }\n\n This is because that we cannot know whether \"refs/heads/a\" is a\n regular ref or a directory by using \"check_refname_format\". So we\n have to add this check. It may seem we have done this when\n iterating the \"refs\" directory. However, we do report error for\n other NON-symlink and NON-regular file type but for directory, we\n omit. We cannot say oh this is not right. So we need to explicitly\n check here.\n\n + Like [PATCH v3 2/4], enhance the test code.\n\n4. [PATCH v3 4/4]\n\n + Enhance the commit message\n + Introduce a new fsck info \"symlinkRef\" to warn the user that we\n will see this warning as an error when we drop the symlink ref\n support.\n + Squash the following two patches into this patch:\n https://lore.kernel.org/git/xmqqle0gzdyh.fsf_-_@gitster.g/\n https://lore.kernel.org/git/xmqqbk1cz69c.fsf@gitster.g/\n\nThanks,\nJialuo\n\n\nshejialuo (4):\n ref: initialize \"fsck_ref_report\" with zero\n ref: add regular ref content check for files backend\n ref: add symref content check for files backend\n ref: add symlink ref content check for files backend\n\n Documentation/fsck-msgids.txt | 20 ++\n fsck.h | 5 +\n refs.c | 2 +-\n refs/files-backend.c | 205 ++++++++++++++++++++-\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 334 ++++++++++++++++++++++++++++++++++\n 6 files changed, 556 insertions(+), 12 deletions(-)\n\nRange-diff against v2:\n1: c49a216b70 ! 1: 9fdab751c1 ref: initialize \"fsck_ref_report\" with zero\n @@ Commit message\n NULL instead of letting them point to anywhere when creating a new\n \"fsck_ref_report\" structure.\n \n - The original code explicitly specifies the \".path\" field to initialize\n - the \"fsck_ref_report\" structure. However, it introduces confusion how we\n - initialize the other fields. In order to avoid this, initialize the\n - \"fsck_ref_report\" with zero to make clear that everything in\n - \"fsck_ref_report\" is zero initialized.\n + The original code explicitly initializes the \"path\" member in the\n + \"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\n + members in the struct). It is more customary to use \" {0} \" to express\n + that we are 0-initializing everything. In order to be align with the the\n + codebase, initialize \"fsck_ref_report\" with zero.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n @@ refs/files-backend.c: static int files_fsck_refs_name(struct ref_store *ref_stor\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n -\t\tstruct fsck_ref_report report = { .path = NULL };\n -+\t\tstruct fsck_ref_report report = {0};\n ++\t\tstruct fsck_ref_report report = { 0 };\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n2: 99e37b0304 ! 2: 4640b6e345 ref: add regular ref content check for files backend\n @@ Commit message\n We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\n refs. However, when parsing the regular refs for files backend by using\n \"files-backend.c::parse_loose_ref_contents\", we allow the ref content to\n - be end with no newline or contain some garbages.\n + end with no newline or to contain some garbages.\n \n - It may seem that we should report an error or warn fsck message to the\n - user about above situations. However, there may be some third-party\n - tools customizing the content of refs. We should not report an error\n - fsck message.\n + Even though we never create such loose refs ourselves, we have accepted\n + such loose refs. So, it is entirely possible that some third-party tools\n + may rely on such loose refs being valid. We should not report an error\n + fsck message at current. But let's notice such a \"curiously formatted\"\n + loose refs being valid and tell the user our findings, so we can access\n + the possible extent of damage when we tighten the parsing rules in the\n + future.\n \n - And we cannot either report a warn fsck message to the user. This is\n - because if the caller set the \"strict\" field in \"fsck_options\" to\n - to upgrade the fsck warnings to errors.\n + And it's not suitable to either report a warn fsck message to the user.\n + This is because if the caller set the \"strict\" field in \"fsck_options\",\n + fsck warns will be automatically upgraded to errors. We should not allow\n + user to specify the \"--strict\" flag to upgrade the fsck warnings to\n + errors at current. It might cause compatibility issue which may break\n + the legacy repository. So we add the following two fsck infos to\n + represent the situation where the ref content ends without newline or has\n + garbages:\n \n - We should not allow the user to upgrade the fsck warnings to errors. It\n - might cause compatibility issue which will break the legacy repository.\n - So we add the following two fsck infos to represent the situation where\n - the ref content ends without newline or has garbages:\n + 1. \"refMissingNewline(INFO)\": A ref does not end with newline. This kind\n + of ref may be considered ERROR in the future.\n + 2. \"trailingRefContent(INFO)\": A ref has trailing contents. This kind of\n + ref may be considered ERROR in the future.\n \n - 1. \"refMissingNewline(INFO)\": A valid ref does not end with newline.\n - 2. \"trailingRefContent(INFO)\": A ref has trailing contents.\n -\n - In \"fsck.c::fsck_vreport\", we will convert \"FSCK_INFO\" to \"FSCK_WARN\",\n - and we can still warn the user about these situations when using\n - \"git-refs verify\" without introducing compatibility issue.\n + It may seem that we could not give the user any warnings by creating\n + fsck infos. However, in \"fsck.c::fsck_vreport\", we will convert\n + \"FSCK_INFO\" to \"FSCK_WARN\" and we can still warn the user about these\n + situations when using \"git-refs verify\" without introducing\n + compatibility issue.\n \n In current \"git-fsck(1)\", it will report an error when the ref content\n is bad, so we should following this to report an error to the user when\n @@ Documentation/fsck-msgids.txt\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n +`refMissingNewline`::\n -+\t(INFO) A valid ref does not end with newline.\n ++\t(INFO) A ref does not end with newline. This kind of ref may\n ++\tbe considered ERROR in the future.\n +\n +`trailingRefContent`::\n -+\t(INFO) A ref has trailing contents.\n ++\t(INFO) A ref has trailing contents. This kind of ref may be\n ++\tconsidered ERROR in the future.\n +\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n +\treport.path = refname.buf;\n +\n -+\tif (S_ISREG(iter->st.st_mode)) {\n -+\t\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n -+\t\t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n -+\t\t\t\t\t refs_check_dir, iter->relative_path);\n -+\t\t\tgoto cleanup;\n -+\t\t}\n ++\tif (S_ISLNK(iter->st.st_mode))\n ++\t\tgoto cleanup;\n ++\n ++\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n ++\t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n ++\t\t\t\t refs_check_dir, iter->relative_path);\n ++\t\tgoto cleanup;\n ++\t}\n +\n -+\t\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n -+\t\t\t\t\t ref_content.buf, &oid, &referent,\n -+\t\t\t\t\t &type, &trailing, &failure_errno)) {\n ++\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n ++\t\t\t\t ref_content.buf, &oid, &referent,\n ++\t\t\t\t &type, &trailing, &failure_errno)) {\n ++\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n ++\t\t\t\t \"invalid ref content\");\n ++\t\tgoto cleanup;\n ++\t}\n ++\n ++\tif (!(type & REF_ISSYMREF)) {\n ++\t\tif (*trailing == '\\0') {\n +\t\t\tret = fsck_report_ref(o, &report,\n -+\t\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n -+\t\t\t\t\t \"invalid ref content\");\n ++\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n ++\t\t\t\t\t \"missing newline\");\n +\t\t\tgoto cleanup;\n +\t\t}\n +\n -+\t\tif (!(type & REF_ISSYMREF)) {\n -+\t\t\tif (*trailing == '\\0') {\n -+\t\t\t\tret = fsck_report_ref(o, &report,\n -+\t\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n -+\t\t\t\t\t\t \"missing newline\");\n -+\t\t\t\tgoto cleanup;\n -+\t\t\t}\n -+\n -+\t\t\tif (*trailing != '\\n' || (*(trailing + 1) != '\\0')) {\n -+\t\t\t\tret = fsck_report_ref(o, &report,\n -+\t\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n -+\t\t\t\t\t\t \"trailing garbage in ref\");\n -+\t\t\t\tgoto cleanup;\n -+\t\t\t}\n ++\t\tif (*trailing != '\\n' || (*(trailing + 1) != '\\0')) {\n ++\t\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n ++\t\t\t\t\t \"trailing garbage in ref\");\n ++\t\t\tgoto cleanup;\n +\t\t}\n -+\t\tgoto cleanup;\n +\t}\n +\n +cleanup:\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref name check should be adapted\n \ttest_must_be_empty err\n '\n \n -+test_expect_success 'regular ref content should be checked' '\n ++test_expect_success 'regular ref content should be checked (individual)' '\n +\ttest_when_finished \"rm -rf repo\" &&\n +\tgit init repo &&\n +\tbranch_dir_prefix=.git/refs/heads &&\n +\ttag_dir_prefix=.git/refs/tags &&\n +\tcd repo &&\n -+\tgit commit --allow-empty -m initial &&\n -+\tgit checkout -b branch-1 &&\n -+\tgit tag tag-1 &&\n -+\tgit commit --allow-empty -m second &&\n -+\tgit checkout -b branch-2 &&\n -+\tgit tag tag-2 &&\n -+\tgit checkout -b a/b/tag-2 &&\n ++\ttest_commit default &&\n ++\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n +\n -+\tprintf \"%s\" \"$(git rev-parse branch-1)\" > $branch_dir_prefix/branch-1-no-newline &&\n ++\tgit refs verify 2>err &&\n ++\ttest_must_be_empty err &&\n ++\n ++\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline\n ++\twarning: refs/heads/branch-no-newline: refMissingNewline: missing newline\n +\tEOF\n -+\trm $branch_dir_prefix/branch-1-no-newline &&\n ++\trm $branch_dir_prefix/branch-no-newline &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"%s garbage\" \"$(git rev-parse branch-1)\" > $branch_dir_prefix/branch-1-garbage &&\n ++\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/heads/branch-1-garbage: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/heads/branch-garbage: trailingRefContent: trailing garbage in ref\n +\tEOF\n -+\trm $branch_dir_prefix/branch-1-garbage &&\n ++\trm $branch_dir_prefix/branch-garbage &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-garbage &&\n ++\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/tags/tag-garbage-1: trailingRefContent: trailing garbage in ref\n +\tEOF\n -+\trm $tag_dir_prefix/tag-1-garbage &&\n ++\trm $tag_dir_prefix/tag-garbage-1 &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-garbage &&\n ++\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/tags/tag-garbage-2: trailingRefContent: trailing garbage in ref\n +\tEOF\n -+\trm $tag_dir_prefix/tag-1-garbage &&\n ++\trm $tag_dir_prefix/tag-garbage-2 &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"%s garbage\\n\\na\" \"$(git rev-parse tag-2)\" > $tag_dir_prefix/tag-2-garbage &&\n ++\tprintf \"%s garbage\\n\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/tags/tag-2-garbage: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/tags/tag-garbage-3: trailingRefContent: trailing garbage in ref\n +\tEOF\n -+\trm $tag_dir_prefix/tag-2-garbage &&\n ++\trm $tag_dir_prefix/tag-garbage-3 &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"%s garbage\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-garbage &&\n ++\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n +\ttest_must_fail git -c fsck.trailingRefContent=error refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\terror: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref\n ++\terror: refs/tags/tag-garbage-4: trailingRefContent: trailing garbage in ref\n +\tEOF\n -+\trm $tag_dir_prefix/tag-1-garbage &&\n ++\trm $tag_dir_prefix/tag-garbage-4 &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"%sx\" \"$(git rev-parse tag-1)\" > $tag_dir_prefix/tag-1-bad &&\n ++\tprintf \"%sx\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-bad-1 &&\n +\ttest_must_fail git refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\terror: refs/tags/tag-1-bad: badRefContent: invalid ref content\n ++\terror: refs/tags/tag-bad-1: badRefContent: invalid ref content\n +\tEOF\n -+\trm $tag_dir_prefix/tag-1-bad &&\n ++\trm $tag_dir_prefix/tag-bad-1 &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"xfsazqfxcadas\" > $tag_dir_prefix/tag-2-bad &&\n ++\tprintf \"xfsazqfxcadas\" >$tag_dir_prefix/tag-bad-2 &&\n +\ttest_must_fail git refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\terror: refs/tags/tag-2-bad: badRefContent: invalid ref content\n ++\terror: refs/tags/tag-bad-2: badRefContent: invalid ref content\n +\tEOF\n -+\trm $tag_dir_prefix/tag-2-bad &&\n ++\trm $tag_dir_prefix/tag-bad-2 &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"xfsazqfxcadas\" > $branch_dir_prefix/a/b/branch-2-bad &&\n ++\tprintf \"xfsazqfxcadas\" >$branch_dir_prefix/a/b/branch-bad &&\n +\ttest_must_fail git refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\terror: refs/heads/a/b/branch-2-bad: badRefContent: invalid ref content\n ++\terror: refs/heads/a/b/branch-bad: badRefContent: invalid ref content\n +\tEOF\n -+\trm $branch_dir_prefix/a/b/branch-2-bad &&\n ++\trm $branch_dir_prefix/a/b/branch-bad &&\n +\ttest_cmp expect err\n +'\n ++\n ++test_expect_success 'regular ref content should be checked (aggregate)' '\n ++\ttest_when_finished \"rm -rf repo\" &&\n ++\tgit init repo &&\n ++\tbranch_dir_prefix=.git/refs/heads &&\n ++\ttag_dir_prefix=.git/refs/tags &&\n ++\tcd repo &&\n ++\ttest_commit default &&\n ++\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n ++\n ++\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n ++\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n ++\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n ++\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n ++\tprintf \"%s garbage\\n\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n ++\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n ++\tprintf \"%sx\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-bad-1 &&\n ++\tprintf \"xfsazqfxcadas\" >$tag_dir_prefix/tag-bad-2 &&\n ++\tprintf \"xfsazqfxcadas\" >$branch_dir_prefix/a/b/branch-bad &&\n ++\n ++\ttest_must_fail git refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\terror: refs/heads/a/b/branch-bad: badRefContent: invalid ref content\n ++\terror: refs/tags/tag-bad-1: badRefContent: invalid ref content\n ++\terror: refs/tags/tag-bad-2: badRefContent: invalid ref content\n ++\twarning: refs/heads/branch-garbage: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/heads/branch-no-newline: refMissingNewline: missing newline\n ++\twarning: refs/tags/tag-garbage-1: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/tags/tag-garbage-2: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/tags/tag-garbage-3: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/tags/tag-garbage-4: trailingRefContent: trailing garbage in ref\n ++\tEOF\n ++\tsort err >sorted_err &&\n ++\ttest_cmp expect sorted_err\n ++'\n +\n test_done\n3: 76dcf6bf58 ! 3: 0691e2960d ref: add symbolic ref content check for files backend\n @@ Metadata\n Author: shejialuo \n \n ## Commit message ##\n - ref: add symbolic ref content check for files backend\n + ref: add symref content check for files backend\n \n We have already introduced the checks for regular refs. There is no need\n - to check the consistency of the target which the symbolic ref points to.\n - Instead, we just check the content of the symbolic ref itself.\n + to check the consistency of the target which the symref points to.\n + Instead, we just need to check the content of teh symref itself.\n \n - In order to check the content of the symbolic ref, create a function\n - \"files_fsck_symref_target\". It will first check whether the \"pointee\" is\n - under the \"refs/\" directory and then we will check the \"pointee\" itself.\n + In order to check the content of the symref, create a function\n + \"files_fsck_symref_target\". It will first check whether the \"referent\"\n + is under the \"refs/\" directory and then we will check the symref\n + contents.\n \n - There is no specification about the content of the symbolic ref.\n - Although we do write \"ref: %s\\n\" to create a symbolic ref by using\n - \"git-symbolic-ref(1)\" command. However, this is not mandatory. We still\n - accept symbolic refs with null trailing garbage. Put it more specific,\n - the following are correct:\n + A regular file is accepted as a textual symref if it begins with\n + \"ref:\", followed by zero or more whitespaces, followed by the full\n + refname, followed only by whitespace characters. We always write\n + a single SP after \"ref:\" and a single LF after the refname, but\n + third-party reimplementations of Git may have taken advantage of the\n + looser syntax. Put it more specific, we accept the following contents\n + of the symref:\n \n 1. \"ref: refs/heads/master \"\n 2. \"ref: refs/heads/master \\n \\n\"\n 3. \"ref: refs/heads/master\\n\\n\"\n \n - But we do not allow any non-null trailing garbage. The following are bad\n - symbolic contents which will be reported as fsck error by \"git-fsck(1)\".\n + But we do not allow any other trailing garbage. The followings are bad\n + symref contents which will be reported as fsck error by \"git-fsck(1)\".\n \n 1. \"ref: refs/heads/master garbage\\n\"\n 2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n \n - In order to provide above checks, we will use \"strrchr\" to check whether\n - we have newline in the ref content. Then we will check the name of the\n - \"pointee\" is correct by using \"check_refname_format\". If the function\n - fails, we need to trim the \"pointee\" to see whether the null-garbage\n - causes the function fails. If so, we need to report that there is\n - null-garbage in the symref content. Otherwise, we should report the user\n - the \"pointee\" is bad.\n + In order to provide above checks, we will first check whether the symref\n + content misses the newline by peeking the last byte of the \"referent\" to\n + see whether it is '\\n'.\n +\n + And we will remember the untrimmed length of the \"referent\" and call\n + \"strbuf_rtrim()\" on \"referent\". Then, we will call \"check_refname_format\"\n + to chceck whether the trimmed referent format is valid. If not, we will\n + report to the user that the symref points to referent which has invalid\n + format. If it is valid, we will compare the untrimmed length and trimmed\n + length, if they are not the same, we need to warn the user there is some\n + trailing garbage in the symref content.\n +\n + At last, we need to check whether the referent is the directory. We\n + cannot distinguish whether the \"refs/heads/a\" is a directory or not by\n + using \"check_refname_format\". We have already checked bad file type when\n + iterating the \"refs/\" directory but we ignore the directory. Thus, we\n + need to explicitly add check here.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n @@ Documentation/fsck-msgids.txt\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n -+`badSymrefPointee`::\n -+\t(ERROR) The pointee of a symref is bad.\n ++`badSymrefTarget`::\n ++\t(ERROR) The symref target points outside the ref directory or\n ++\tthe name of the symref target is invalid.\n +\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n @@ fsck.h: enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n -+\tFUNC(BAD_SYMREF_POINTEE, ERROR) \\\n ++\tFUNC(BAD_SYMREF_TARGET, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n \t\t\t\t struct dir_iterator *iter);\n \n +/*\n -+ * Check the symref \"pointee_name\" and \"pointee_path\". The caller should\n -+ * make sure that \"pointee_path\" is absolute. For symbolic ref, \"pointee_name\"\n -+ * would be the content after \"refs:\".\n ++ * Check the symref \"referent\" and \"referent_path\". For textual symref,\n ++ * \"referent\" would be the content after \"refs:\".\n + */\n +static int files_fsck_symref_target(struct fsck_options *o,\n +\t\t\t\t struct fsck_ref_report *report,\n -+\t\t\t\t const char *refname,\n -+\t\t\t\t struct strbuf *pointee_name,\n -+\t\t\t\t struct strbuf *pointee_path)\n ++\t\t\t\t struct strbuf *referent,\n ++\t\t\t\t struct strbuf *referent_path)\n +{\n -+\tconst char *newline_pos = NULL;\n ++\tsize_t len = referent->len - 1;\n +\tconst char *p = NULL;\n +\tstruct stat st;\n +\tint ret = 0;\n +\n -+\tif (!skip_prefix(pointee_name->buf, \"refs/\", &p)) {\n ++\tif (!skip_prefix(referent->buf, \"refs/\", &p)) {\n +\n +\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n ++\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n +\t\t\t\t \"points to ref outside the refs directory\");\n +\t\tgoto out;\n +\t}\n +\n -+\tnewline_pos = strrchr(p, '\\n');\n -+\tif (!newline_pos || *(newline_pos + 1)) {\n ++\tif (referent->buf[referent->len - 1] != '\\n') {\n +\t\tret = fsck_report_ref(o, report,\n +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n +\t\t\t\t \"missing newline\");\n ++\t\tlen++;\n +\t}\n +\n -+\tif (check_refname_format(pointee_name->buf, 0)) {\n -+\t\t/*\n -+\t\t * When containing null-garbage, \"check_refname_format\" will\n -+\t\t * fail, we should trim the \"pointee\" to check again.\n -+\t\t */\n -+\t\tstrbuf_rtrim(pointee_name);\n -+\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n -+\t\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n -+\t\t\t\t\t \"trailing null-garbage\");\n -+\t\t\tgoto out;\n -+\t\t}\n -+\n ++\tstrbuf_rtrim(referent);\n ++\tif (check_refname_format(referent->buf, 0)) {\n +\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n ++\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n +\t\t\t\t \"points to refname with invalid format\");\n ++\t\tgoto out;\n ++\t}\n ++\n ++\tif (len != referent->len) {\n ++\t\tret = fsck_report_ref(o, report,\n ++\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n ++\t\t\t\t \"trailing garbage in ref\");\n +\t}\n +\n +\t/*\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\t * ref that does not exist yet. If the target ref does not exist, just\n +\t * skip the check for the file type.\n +\t */\n -+\tif (lstat(pointee_path->buf, &st) < 0)\n ++\tif (lstat(referent_path->buf, &st))\n +\t\tgoto out;\n +\n -+\tif (!S_ISREG(st.st_mode) && !S_ISLNK(st.st_mode)) {\n ++\t/*\n ++\t * We cannot distinguish whether \"refs/heads/a\" is directory or nots by\n ++\t * using \"check_refname_format(referent->buf, 0)\". Instead, we need to\n ++\t * check the file type of the target.\n ++\t */\n ++\tif (S_ISDIR(st.st_mode)) {\n +\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n -+\t\t\t\t \"points to an invalid file type\");\n ++\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n ++\t\t\t\t \"points to the directory\");\n +\t\tgoto out;\n +\t}\n +\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter)\n {\n -+\tstruct strbuf pointee_path = STRBUF_INIT;\n ++\tstruct strbuf referent_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n - \t\t\t\t\t\t \"trailing garbage in ref\");\n - \t\t\t\tgoto cleanup;\n - \t\t\t}\n -+\t\t} else {\n -+\t\t\tstrbuf_addf(&pointee_path, \"%s/%s\",\n -+\t\t\t\t ref_store->gitdir, referent.buf);\n -+\t\t\tret = files_fsck_symref_target(o, &report, refname.buf,\n -+\t\t\t\t\t\t &referent,\n -+\t\t\t\t\t\t &pointee_path);\n + \t\t\t\t\t \"trailing garbage in ref\");\n + \t\t\tgoto cleanup;\n \t\t}\n - \t\tgoto cleanup;\n ++\t} else {\n ++\t\tstrbuf_addf(&referent_path, \"%s/%s\",\n ++\t\t\t ref_store->gitdir, referent.buf);\n ++\t\t/*\n ++\t\t * the referent may contain the spaces and the newline, need to\n ++\t\t * trim for path.\n ++\t\t */\n ++\t\tstrbuf_rtrim(&referent_path);\n ++\t\tret = files_fsck_symref_target(o, &report,\n ++\t\t\t\t\t &referent,\n ++\t\t\t\t\t &referent_path);\n \t}\n -@@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n + \n + cleanup:\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n -+\tstrbuf_release(&pointee_path);\n ++\tstrbuf_release(&referent_path);\n \treturn ret;\n }\n \n \n ## t/t0602-reffiles-fsck.sh ##\n -@@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be checked' '\n - \ttest_cmp expect err\n +@@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be checked (aggregate)' '\n + \ttest_cmp expect sorted_err\n '\n \n -+test_expect_success 'symbolic ref content should be checked' '\n ++test_expect_success 'textual symref content should be checked (individual)' '\n +\ttest_when_finished \"rm -rf repo\" &&\n +\tgit init repo &&\n +\tbranch_dir_prefix=.git/refs/heads &&\n +\ttag_dir_prefix=.git/refs/tags &&\n +\tcd repo &&\n -+\tgit commit --allow-empty -m initial &&\n -+\tgit checkout -b branch-1 &&\n -+\tgit tag tag-1 &&\n -+\tgit checkout -b a/b/branch-2 &&\n ++\ttest_commit default &&\n ++\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n ++\n ++\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n ++\tgit refs verify 2>err &&\n ++\trm $branch_dir_prefix/branch-good &&\n ++\ttest_must_be_empty err &&\n ++\n ++\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n ++\tgit refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\twarning: refs/heads/branch-no-newline-1: refMissingNewline: missing newline\n ++\tEOF\n ++\trm $branch_dir_prefix/branch-no-newline-1 &&\n ++\ttest_cmp expect err &&\n +\n -+\tprintf \"ref: refs/heads/branch\" > $branch_dir_prefix/branch-1-no-newline &&\n ++\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/heads/branch-1-no-newline: refMissingNewline: missing newline\n ++\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: missing newline\n ++\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: trailing garbage in ref\n +\tEOF\n -+\trm $branch_dir_prefix/branch-1-no-newline &&\n ++\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"ref: refs/heads/branch \" > $branch_dir_prefix/a/b/branch-trailing &&\n ++\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/heads/a/b/branch-trailing: refMissingNewline: missing newline\n -+\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n ++\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: trailing garbage in ref\n +\tEOF\n -+\trm $branch_dir_prefix/a/b/branch-trailing &&\n ++\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"ref: refs/heads/branch\\n\\n\" > $branch_dir_prefix/a/b/branch-trailing &&\n ++\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n ++\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: trailing garbage in ref\n +\tEOF\n -+\trm $branch_dir_prefix/a/b/branch-trailing &&\n ++\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"ref: refs/heads/branch \\n\\n \" > $branch_dir_prefix/a/b/branch-trailing &&\n ++\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/heads/a/b/branch-trailing: refMissingNewline: missing newline\n -+\twarning: refs/heads/a/b/branch-trailing: trailingRefContent: trailing null-garbage\n ++\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: missing newline\n ++\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: trailing garbage in ref\n +\tEOF\n -+\trm $branch_dir_prefix/a/b/branch-trailing &&\n ++\trm $branch_dir_prefix/a/b/branch-complicated &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"ref: refs/heads/.branch\\n\" > $branch_dir_prefix/branch-2-bad &&\n ++\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n +\ttest_must_fail git refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\terror: refs/heads/branch-2-bad: badSymrefPointee: points to refname with invalid format\n ++\terror: refs/heads/branch-bad-1: badSymrefTarget: points to refname with invalid format\n +\tEOF\n -+\trm $branch_dir_prefix/branch-2-bad &&\n ++\trm $branch_dir_prefix/branch-bad-1 &&\n ++\ttest_cmp expect err &&\n ++\n ++\tprintf \"ref: reflogs/heads/main\\n\" >$branch_dir_prefix/branch-bad-2 &&\n ++\ttest_must_fail git refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\terror: refs/heads/branch-bad-2: badSymrefTarget: points to ref outside the refs directory\n ++\tEOF\n ++\trm $branch_dir_prefix/branch-bad-2 &&\n ++\ttest_cmp expect err &&\n ++\n ++\tprintf \"ref: refs/heads/a\\n\" >$branch_dir_prefix/branch-bad-3 &&\n ++\ttest_must_fail git refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\terror: refs/heads/branch-bad-3: badSymrefTarget: points to the directory\n ++\tEOF\n ++\trm $branch_dir_prefix/branch-bad-3 &&\n +\ttest_cmp expect err\n +'\n ++\n ++test_expect_success 'textual symref content should be checked (aggregate)' '\n ++\ttest_when_finished \"rm -rf repo\" &&\n ++\tgit init repo &&\n ++\tbranch_dir_prefix=.git/refs/heads &&\n ++\ttag_dir_prefix=.git/refs/tags &&\n ++\tcd repo &&\n ++\ttest_commit default &&\n ++\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n ++\n ++\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n ++\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n ++\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n ++\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n ++\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n ++\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n ++\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n ++\tprintf \"ref: reflogs/heads/main\\n\" >$branch_dir_prefix/branch-bad-2 &&\n ++\tprintf \"ref: refs/heads/a\\n\" >$branch_dir_prefix/branch-bad-3 &&\n ++\n ++\ttest_must_fail git refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\terror: refs/heads/branch-bad-1: badSymrefTarget: points to refname with invalid format\n ++\terror: refs/heads/branch-bad-2: badSymrefTarget: points to ref outside the refs directory\n ++\terror: refs/heads/branch-bad-3: badSymrefTarget: points to the directory\n ++\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: missing newline\n ++\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: missing newline\n ++\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: trailing garbage in ref\n ++\twarning: refs/heads/branch-no-newline-1: refMissingNewline: missing newline\n ++\tEOF\n ++\tsort err >sorted_err &&\n ++\ttest_cmp expect sorted_err\n ++'\n +\n test_done\n4: 2008f8635c ! 4: 4105bfa1e3 ref: add symlink ref check for files backend\n @@ Metadata\n Author: shejialuo \n \n ## Commit message ##\n - ref: add symlink ref check for files backend\n + ref: add symlink ref content check for files backend\n \n We have already introduced \"files_fsck_symref_target\". We should reuse\n - this function to handle the symrefs which are legacy symbolic links. We\n - should not check the trailing garbage for symbolic links. Add a new\n + this function to handle the symrefs which use legacy symbolic links. We\n + should not check the trailing garbage for symbolic refs. Add a new\n parameter \"symbolic_link\" to disable some checks which should only be\n - used for symbolic ref.\n + executed for textual symrefs.\n \n - We firstly use the \"strbuf_add_real_path\" to resolve the symlinks and\n - get the absolute path \"pointee_path\" which the symlink ref points to.\n - Then we can get the absolute path \"abs_gitdir\" of the \"gitdir\". By\n - combining \"pointee_path\" and \"abs_gitdir\", we can extract the\n + We firstly use the \"strbuf_add_real_path\" to resolve the symlink and\n + get the absolute path \"referent_path\" which the symlink ref points\n + to. Then we can get the absolute path \"abs_gitdir\" of the \"gitdir\".\n + By combining \"referent_path\" and \"abs_gitdir\", we can extract the\n \"referent\". Thus, we can reuse \"files_fsck_symref_target\" function to\n seamlessly check the symlink refs.\n \n + Because we are going to drop support for \"core.prefersymlinkrefs\", add a\n + new fsck message \"symlinkRef\" to let the user be aware of this\n + information.\n +\n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n Signed-off-by: shejialuo \n \n + ## Documentation/fsck-msgids.txt ##\n +@@\n + \t(INFO) A ref does not end with newline. This kind of ref may\n + \tbe considered ERROR in the future.\n + \n ++`symlinkRef`::\n ++\t(INFO) A symref uses the symbolic link. This kind of symref may\n ++\tbe considered ERROR in the future when totally dropping the\n ++\tsymlink support.\n ++\n + `trailingRefContent`::\n + \t(INFO) A ref has trailing contents. This kind of ref may be\n + \tconsidered ERROR in the future.\n +\n + ## fsck.h ##\n +@@ fsck.h: enum fsck_msg_type {\n + \tFUNC(BAD_TAG_NAME, INFO) \\\n + \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n + \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n ++\tFUNC(SYMLINK_REF, INFO) \\\n + \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n + \t/* ignored (elevated when requested) */ \\\n + \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n +\n ## refs/files-backend.c ##\n @@\n #include \"../git-compat-util.h\"\n @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref\n \t\t\tgoto out;\n \t\t}\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n + \n /*\n - * Check the symref \"pointee_name\" and \"pointee_path\". The caller should\n - * make sure that \"pointee_path\" is absolute. For symbolic ref, \"pointee_name\"\n -- * would be the content after \"refs:\".\n -+ * would be the content after \"refs:\". For symblic link, \"pointee_name\" would\n -+ * be the relative path agaignst \"gitdir\".\n + * Check the symref \"referent\" and \"referent_path\". For textual symref,\n +- * \"referent\" would be the content after \"refs:\".\n ++ * \"referent\" would be the content after \"refs:\". For symlink ref,\n ++ * \"referent\" would be the relative path agaignst \"gitdir\" which should\n ++ * be the same as the textual symref literally.\n */\n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n --\t\t\t\t const char *refname,\n - \t\t\t\t struct strbuf *pointee_name,\n --\t\t\t\t struct strbuf *pointee_path)\n -+\t\t\t\t struct strbuf *pointee_path,\n + \t\t\t\t struct strbuf *referent,\n +-\t\t\t\t struct strbuf *referent_path)\n ++\t\t\t\t struct strbuf *referent_path,\n +\t\t\t\t unsigned int symbolic_link)\n {\n - \tconst char *newline_pos = NULL;\n + \tsize_t len = referent->len - 1;\n \tconst char *p = NULL;\n @@ refs/files-backend.c: static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n --\tnewline_pos = strrchr(p, '\\n');\n --\tif (!newline_pos || *(newline_pos + 1)) {\n --\t\tret = fsck_report_ref(o, report,\n --\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n --\t\t\t\t \"missing newline\");\n -+\tif (!symbolic_link) {\n -+\t\tnewline_pos = strrchr(p, '\\n');\n -+\t\tif (!newline_pos || *(newline_pos + 1)) {\n -+\t\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n -+\t\t\t\t\t \"missing newline\");\n -+\t\t}\n +-\tif (referent->buf[referent->len - 1] != '\\n') {\n ++\tif (!symbolic_link && referent->buf[referent->len - 1] != '\\n') {\n + \t\tret = fsck_report_ref(o, report,\n + \t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n + \t\t\t\t \"missing newline\");\n + \t\tlen++;\n \t}\n \n - \tif (check_refname_format(pointee_name->buf, 0)) {\n --\t\t/*\n --\t\t * When containing null-garbage, \"check_refname_format\" will\n --\t\t * fail, we should trim the \"pointee\" to check again.\n --\t\t */\n --\t\tstrbuf_rtrim(pointee_name);\n --\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n --\t\t\tret = fsck_report_ref(o, report,\n --\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n --\t\t\t\t\t \"trailing null-garbage\");\n --\t\t\tgoto out;\n -+\t\tif (!symbolic_link) {\n -+\t\t\t/*\n -+\t\t\t* When containing null-garbage, \"check_refname_format\" will\n -+\t\t\t* fail, we should trim the \"pointee\" to check again.\n -+\t\t\t*/\n -+\t\t\tstrbuf_rtrim(pointee_name);\n -+\t\t\tif (!check_refname_format(pointee_name->buf, 0)) {\n -+\t\t\t\tret = fsck_report_ref(o, report,\n -+\t\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n -+\t\t\t\t\t\t \"trailing null-garbage\");\n -+\t\t\t\tgoto out;\n -+\t\t\t}\n - \t\t}\n +-\tstrbuf_rtrim(referent);\n ++\tif (!symbolic_link)\n ++\t\tstrbuf_rtrim(referent);\n ++\n + \tif (check_refname_format(referent->buf, 0)) {\n + \t\tret = fsck_report_ref(o, report,\n + \t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n +@@ refs/files-backend.c: static int files_fsck_symref_target(struct fsck_options *o,\n + \t\tgoto out;\n + \t}\n \n +-\tif (len != referent->len) {\n ++\tif (!symbolic_link && len != referent->len) {\n \t\tret = fsck_report_ref(o, report,\n + \t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n + \t\t\t\t \"trailing garbage in ref\");\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n {\n - \tstruct strbuf pointee_path = STRBUF_INIT;\n + \tstruct strbuf referent_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n +\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n \tstruct fsck_ref_report report = {0};\n -+\tconst char *pointee_name = NULL;\n +\tunsigned int symbolic_link = 0;\n \tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n - \t\t} else {\n - \t\t\tstrbuf_addf(&pointee_path, \"%s/%s\",\n - \t\t\t\t ref_store->gitdir, referent.buf);\n --\t\t\tret = files_fsck_symref_target(o, &report, refname.buf,\n -+\t\t\tret = files_fsck_symref_target(o, &report,\n - \t\t\t\t\t\t &referent,\n --\t\t\t\t\t\t &pointee_path);\n -+\t\t\t\t\t\t &pointee_path,\n -+\t\t\t\t\t\t symbolic_link);\n - \t\t}\n - \t\tgoto cleanup;\n - \t}\n + \tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n + \treport.path = refname.buf;\n \n -+\tsymbolic_link = 1;\n -+\n -+\tstrbuf_add_real_path(&pointee_path, iter->path.buf);\n -+\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n -+\tstrbuf_normalize_path(&abs_gitdir);\n -+\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n -+\t\tstrbuf_addch(&abs_gitdir, '/');\n +-\tif (S_ISLNK(iter->st.st_mode))\n ++\tif (S_ISLNK(iter->st.st_mode)) {\n ++\t\tconst char* relative_referent_path;\n +\n -+\tif (!skip_prefix(pointee_path.buf, abs_gitdir.buf, &pointee_name)) {\n ++\t\tsymbolic_link = 1;\n +\t\tret = fsck_report_ref(o, &report,\n -+\t\t\t\t FSCK_MSG_BAD_SYMREF_POINTEE,\n -+\t\t\t\t \"point to target outside gitdir\");\n -+\t\tgoto cleanup;\n -+\t}\n ++\t\t\t\t FSCK_MSG_SYMLINK_REF,\n ++\t\t\t\t \"use deprecated symbolic link for symref\");\n +\n -+\tstrbuf_addstr(&referent, pointee_name);\n -+\tret = files_fsck_symref_target(o, &report,\n -+\t\t\t\t &referent, &pointee_path,\n -+\t\t\t\t symbolic_link);\n ++\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n ++\t\tstrbuf_normalize_path(&abs_gitdir);\n ++\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n ++\t\t\tstrbuf_addch(&abs_gitdir, '/');\n +\n ++\t\tstrbuf_add_real_path(&referent_path, iter->path.buf);\n ++\n ++\t\tif (!skip_prefix(referent_path.buf,\n ++\t\t\t\t abs_gitdir.buf,\n ++\t\t\t\t &relative_referent_path)) {\n ++\t\t\tret = fsck_report_ref(o, &report,\n ++\t\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n ++\t\t\t\t\t \"point to target outside gitdir\");\n ++\t\t\tgoto cleanup;\n ++\t\t}\n ++\n ++\t\tstrbuf_addstr(&referent, relative_referent_path);\n ++\t\tret = files_fsck_symref_target(o, &report,\n ++\t\t\t\t\t &referent, &referent_path,\n ++\t\t\t\t\t symbolic_link);\n ++\n + \t\tgoto cleanup;\n ++\t}\n + \n + \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n + \t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n +@@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n + \t\tstrbuf_rtrim(&referent_path);\n + \t\tret = files_fsck_symref_target(o, &report,\n + \t\t\t\t\t &referent,\n +-\t\t\t\t\t &referent_path);\n ++\t\t\t\t\t &referent_path,\n ++\t\t\t\t\t symbolic_link);\n + \t}\n + \n cleanup:\n - \tstrbuf_release(&refname);\n +@@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n - \tstrbuf_release(&pointee_path);\n + \tstrbuf_release(&referent_path);\n +\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \n \n ## t/t0602-reffiles-fsck.sh ##\n -@@ t/t0602-reffiles-fsck.sh: test_expect_success 'symbolic ref content should be checked' '\n - \ttest_cmp expect err\n +@@ t/t0602-reffiles-fsck.sh: test_expect_success 'textual symref content should be checked (aggregate)' '\n + \ttest_cmp expect sorted_err\n '\n \n -+test_expect_success SYMLINKS 'symbolic ref (symbolic link) content should be checked' '\n ++test_expect_success SYMLINKS 'symlink symref content should be checked (individual)' '\n +\ttest_when_finished \"rm -rf repo\" &&\n +\tgit init repo &&\n +\tbranch_dir_prefix=.git/refs/heads &&\n +\ttag_dir_prefix=.git/refs/tags &&\n +\tcd repo &&\n -+\tgit commit --allow-empty -m initial &&\n -+\tgit checkout -b branch-1 &&\n -+\tgit tag tag-1 &&\n -+\tgit checkout -b a/b/branch-2 &&\n ++\ttest_commit default &&\n ++\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n ++\n ++\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n ++\tgit refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n ++\tEOF\n ++\trm $branch_dir_prefix/branch-symbolic-good &&\n ++\ttest_cmp expect err &&\n ++\n ++\tln -sf ../../../../branch $branch_dir_prefix/branch-symbolic-1 &&\n ++\ttest_must_fail git refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\twarning: refs/heads/branch-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n ++\terror: refs/heads/branch-symbolic-1: badSymrefTarget: point to target outside gitdir\n ++\tEOF\n ++\trm $branch_dir_prefix/branch-symbolic-1 &&\n ++\ttest_cmp expect err &&\n +\n -+\tln -sf ../../../../branch $branch_dir_prefix/branch-symbolic &&\n ++\tln -sf ../../logs/branch-bad $branch_dir_prefix/branch-symbolic-2 &&\n +\ttest_must_fail git refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\terror: refs/heads/branch-symbolic: badSymrefPointee: point to target outside gitdir\n ++\twarning: refs/heads/branch-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n ++\terror: refs/heads/branch-symbolic-2: badSymrefTarget: points to ref outside the refs directory\n +\tEOF\n -+\trm $branch_dir_prefix/branch-symbolic &&\n ++\trm $branch_dir_prefix/branch-symbolic-2 &&\n +\ttest_cmp expect err &&\n +\n -+\tln -sf ../../logs/branch-bad $branch_dir_prefix/branch-symbolic &&\n ++\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic-3 &&\n +\ttest_must_fail git refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\terror: refs/heads/branch-symbolic: badSymrefPointee: points to ref outside the refs directory\n ++\twarning: refs/heads/branch-symbolic-3: symlinkRef: use deprecated symbolic link for symref\n ++\terror: refs/heads/branch-symbolic-3: badSymrefTarget: points to refname with invalid format\n +\tEOF\n -+\trm $branch_dir_prefix/branch-symbolic &&\n ++\trm $branch_dir_prefix/branch-symbolic-3 &&\n +\ttest_cmp expect err &&\n +\n -+\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic &&\n ++\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n +\ttest_must_fail git refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\terror: refs/heads/branch-symbolic: badSymrefPointee: points to refname with invalid format\n ++\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n ++\terror: refs/tags/tag-symbolic-1: badSymrefTarget: points to refname with invalid format\n +\tEOF\n -+\trm $branch_dir_prefix/branch-symbolic &&\n ++\trm $tag_dir_prefix/tag-symbolic-1 &&\n +\ttest_cmp expect err &&\n +\n -+\tln -sf ./\".branch\" $branch_dir_prefix/branch-symbolic &&\n ++\tln -sf ./ $tag_dir_prefix/tag-symbolic-2 &&\n +\ttest_must_fail git refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\terror: refs/heads/branch-symbolic: badSymrefPointee: points to refname with invalid format\n ++\twarning: refs/tags/tag-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n ++\terror: refs/tags/tag-symbolic-2: badSymrefTarget: points to the directory\n +\tEOF\n -+\trm $branch_dir_prefix/branch-symbolic &&\n ++\trm $tag_dir_prefix/tag-symbolic-2 &&\n +\ttest_cmp expect err\n +'\n ++\n ++test_expect_success SYMLINKS 'symlink symref content should be checked (aggregate)' '\n ++\ttest_when_finished \"rm -rf repo\" &&\n ++\tgit init repo &&\n ++\tbranch_dir_prefix=.git/refs/heads &&\n ++\ttag_dir_prefix=.git/refs/tags &&\n ++\tcd repo &&\n ++\ttest_commit default &&\n ++\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n ++\n ++\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n ++\tln -sf ../../../../branch $branch_dir_prefix/branch-symbolic-1 &&\n ++\tln -sf ../../logs/branch-bad $branch_dir_prefix/branch-symbolic-2 &&\n ++\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic-3 &&\n ++\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n ++\tln -sf ./ $tag_dir_prefix/tag-symbolic-2 &&\n ++\n ++\ttest_must_fail git refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\terror: refs/heads/branch-symbolic-1: badSymrefTarget: point to target outside gitdir\n ++\terror: refs/heads/branch-symbolic-2: badSymrefTarget: points to ref outside the refs directory\n ++\terror: refs/heads/branch-symbolic-3: badSymrefTarget: points to refname with invalid format\n ++\terror: refs/tags/tag-symbolic-1: badSymrefTarget: points to refname with invalid format\n ++\terror: refs/tags/tag-symbolic-2: badSymrefTarget: points to the directory\n ++\twarning: refs/heads/branch-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n ++\twarning: refs/heads/branch-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n ++\twarning: refs/heads/branch-symbolic-3: symlinkRef: use deprecated symbolic link for symref\n ++\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n ++\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n ++\twarning: refs/tags/tag-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n ++\tEOF\n ++\tsort err >sorted_err &&\n ++\ttest_cmp expect sorted_err\n ++'\n +\n test_done\n-- \n2.46.0\n\n"},{"id":"502030","messageId":"Ztb_FZl_xaIot-GK@ArchLinux","threadId":"61943","inReplyTo":"Ztb-mgl50cwGVO8A@ArchLinux","subject":"[PATCH v3 1/4] ref: initialize \"fsck_ref_report\" with zero","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-03T12:20:37Z","receivedAt":"2024-09-03T12:19:40Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n\"referent\" is NULL. So, we need to always initialize these parameters to\nNULL instead of letting them point to anywhere when creating a new\n\"fsck_ref_report\" structure.\n\nThe original code explicitly initializes the \"path\" member in the\n\"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\nmembers in the struct). It is more customary to use \" {0} \" to express\nthat we are 0-initializing everything. In order to be align with the the\ncodebase, initialize \"fsck_ref_report\" with zero.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8d6ec9458d..890d0324e1 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3446,7 +3446,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tstruct fsck_ref_report report = { .path = NULL };\n+\t\tstruct fsck_ref_report report = { 0 };\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n-- \n2.46.0\n\n"},{"id":"502031","messageId":"Ztb_HqLg-WvwA2I0@ArchLinux","threadId":"61943","inReplyTo":"Ztb-mgl50cwGVO8A@ArchLinux","subject":"[PATCH v3 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-03T12:20:46Z","receivedAt":"2024-09-03T12:19:49Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\nrefs. However, when parsing the regular refs for files backend by using\n\"files-backend.c::parse_loose_ref_contents\", we allow the ref content to\nend with no newline or to contain some garbages.\n\nEven though we never create such loose refs ourselves, we have accepted\nsuch loose refs. So, it is entirely possible that some third-party tools\nmay rely on such loose refs being valid. We should not report an error\nfsck message at current. But let's notice such a \"curiously formatted\"\nloose refs being valid and tell the user our findings, so we can access\nthe possible extent of damage when we tighten the parsing rules in the\nfuture.\n\nAnd it's not suitable to either report a warn fsck message to the user.\nThis is because if the caller set the \"strict\" field in \"fsck_options\",\nfsck warns will be automatically upgraded to errors. We should not allow\nuser to specify the \"--strict\" flag to upgrade the fsck warnings to\nerrors at current. It might cause compatibility issue which may break\nthe legacy repository. So we add the following two fsck infos to\nrepresent the situation where the ref content ends without newline or has\ngarbages:\n\n1. \"refMissingNewline(INFO)\": A ref does not end with newline. This kind\n of ref may be considered ERROR in the future.\n2. \"trailingRefContent(INFO)\": A ref has trailing contents. This kind of\n ref may be considered ERROR in the future.\n\nIt may seem that we could not give the user any warnings by creating\nfsck infos. However, in \"fsck.c::fsck_vreport\", we will convert\n\"FSCK_INFO\" to \"FSCK_WARN\" and we can still warn the user about these\nsituations when using \"git-refs verify\" without introducing\ncompatibility issue.\n\nIn current \"git-fsck(1)\", it will report an error when the ref content\nis bad, so we should following this to report an error to the user when\n\"parse_loose_ref_contents\" fails. And we add a new fsck error message\ncalled \"badRefContent(ERROR)\" to represent that a ref has a bad content.\n\nIn order to tell whether the ref has trailing content, add a new\nparameter \"trailing\" to \"parse_loose_ref_contents\". Then introduce a new\nfunction \"files_fsck_refs_content\" to check the regular refs to enhance\nthe \"git-refs verify\".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 11 ++++\n fsck.h | 3 +\n refs.c | 2 +-\n refs/files-backend.c | 68 ++++++++++++++++++-\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 120 ++++++++++++++++++++++++++++++++++\n 6 files changed, 202 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 68a2801f15..06d045ac48 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -19,6 +19,9 @@\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n+`badRefContent`::\n+\t(ERROR) A ref has a bad content.\n+\n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n \n@@ -170,6 +173,14 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`refMissingNewline`::\n+\t(INFO) A ref does not end with newline. This kind of ref may\n+\tbe considered ERROR in the future.\n+\n+`trailingRefContent`::\n+\t(INFO) A ref has trailing contents. This kind of ref may be\n+\tconsidered ERROR in the future.\n+\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \ndiff --git a/fsck.h b/fsck.h\nindex 500b4c04d2..b85072df57 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -31,6 +31,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n@@ -84,6 +85,8 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs.c b/refs.c\nindex 74de3d3009..5e74881945 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1758,7 +1758,7 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \t}\n \n \tresult = parse_loose_ref_contents(ref_store->repo->hash_algo, content.buf,\n-\t\t\t\t\t oid, referent, type, failure_errno);\n+\t\t\t\t\t oid, referent, type, NULL, failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 890d0324e1..0187b85c5f 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -560,7 +560,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname,\n \tbuf = sb_contents.buf;\n \n \tret = parse_loose_ref_contents(ref_store->repo->hash_algo, buf,\n-\t\t\t\t oid, referent, type, &myerr);\n+\t\t\t\t oid, referent, type, NULL, &myerr);\n \n out:\n \tif (ret && !myerr)\n@@ -597,7 +597,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno)\n+\t\t\t const char **trailing, int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -619,6 +619,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n+\n+\tif (trailing)\n+\t\t*trailing = p;\n+\n \treturn 0;\n }\n \n@@ -3430,6 +3434,65 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_refs_content(struct ref_store *ref_store,\n+\t\t\t\t struct fsck_options *o,\n+\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t struct dir_iterator *iter)\n+{\n+\tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tstruct strbuf refname = STRBUF_INIT;\n+\tstruct fsck_ref_report report = {0};\n+\tconst char *trailing = NULL;\n+\tunsigned int type = 0;\n+\tint failure_errno = 0;\n+\tstruct object_id oid;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n+\treport.path = refname.buf;\n+\n+\tif (S_ISLNK(iter->st.st_mode))\n+\t\tgoto cleanup;\n+\n+\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n+\t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n+\t\t\t\t refs_check_dir, iter->relative_path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n+\t\t\t\t ref_content.buf, &oid, &referent,\n+\t\t\t\t &type, &trailing, &failure_errno)) {\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"invalid ref content\");\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (!(type & REF_ISSYMREF)) {\n+\t\tif (*trailing == '\\0') {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t \"missing newline\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tif (*trailing != '\\n' || (*(trailing + 1) != '\\0')) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t \"trailing garbage in ref\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&refname);\n+\tstrbuf_release(&ref_content);\n+\tstrbuf_release(&referent);\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n \t\t\t\tconst char *refs_check_dir,\n@@ -3512,6 +3575,7 @@ static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n+\t\tfiles_fsck_refs_content,\n \t\tNULL,\n \t};\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 2313c830d8..73b05f971b 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -715,7 +715,7 @@ struct ref_store {\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno);\n+\t\t\t const char **trailing, int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 71a4d1a5ae..a06ad044f2 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -89,4 +89,124 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'regular ref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: missing newline\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-1: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-2: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\\n\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-3: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n+\ttest_must_fail git -c fsck.trailingRefContent=error refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-garbage-4: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-4 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%sx\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-bad-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-bad-1: badRefContent: invalid ref content\n+\tEOF\n+\trm $tag_dir_prefix/tag-bad-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"xfsazqfxcadas\" >$tag_dir_prefix/tag-bad-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-bad-2: badRefContent: invalid ref content\n+\tEOF\n+\trm $tag_dir_prefix/tag-bad-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"xfsazqfxcadas\" >$branch_dir_prefix/a/b/branch-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: invalid ref content\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-bad &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'regular ref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n+\tprintf \"%s garbage\\n\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n+\tprintf \"%sx\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-bad-1 &&\n+\tprintf \"xfsazqfxcadas\" >$tag_dir_prefix/tag-bad-2 &&\n+\tprintf \"xfsazqfxcadas\" >$branch_dir_prefix/a/b/branch-bad &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: invalid ref content\n+\terror: refs/tags/tag-bad-1: badRefContent: invalid ref content\n+\terror: refs/tags/tag-bad-2: badRefContent: invalid ref content\n+\twarning: refs/heads/branch-garbage: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: missing newline\n+\twarning: refs/tags/tag-garbage-1: trailingRefContent: trailing garbage in ref\n+\twarning: refs/tags/tag-garbage-2: trailingRefContent: trailing garbage in ref\n+\twarning: refs/tags/tag-garbage-3: trailingRefContent: trailing garbage in ref\n+\twarning: refs/tags/tag-garbage-4: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"502032","messageId":"Ztb_JuMjaoAbIZXq@ArchLinux","threadId":"61943","inReplyTo":"Ztb-mgl50cwGVO8A@ArchLinux","subject":"[PATCH v3 3/4] ref: add symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-03T12:20:54Z","receivedAt":"2024-09-03T12:19:58Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already introduced the checks for regular refs. There is no need\nto check the consistency of the target which the symref points to.\nInstead, we just need to check the content of teh symref itself.\n\nIn order to check the content of the symref, create a function\n\"files_fsck_symref_target\". It will first check whether the \"referent\"\nis under the \"refs/\" directory and then we will check the symref\ncontents.\n\nA regular file is accepted as a textual symref if it begins with\n\"ref:\", followed by zero or more whitespaces, followed by the full\nrefname, followed only by whitespace characters. We always write\na single SP after \"ref:\" and a single LF after the refname, but\nthird-party reimplementations of Git may have taken advantage of the\nlooser syntax. Put it more specific, we accept the following contents\nof the symref:\n\n1. \"ref: refs/heads/master \"\n2. \"ref: refs/heads/master \\n \\n\"\n3. \"ref: refs/heads/master\\n\\n\"\n\nBut we do not allow any other trailing garbage. The followings are bad\nsymref contents which will be reported as fsck error by \"git-fsck(1)\".\n\n1. \"ref: refs/heads/master garbage\\n\"\n2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n\nIn order to provide above checks, we will first check whether the symref\ncontent misses the newline by peeking the last byte of the \"referent\" to\nsee whether it is '\\n'.\n\nAnd we will remember the untrimmed length of the \"referent\" and call\n\"strbuf_rtrim()\" on \"referent\". Then, we will call \"check_refname_format\"\nto chceck whether the trimmed referent format is valid. If not, we will\nreport to the user that the symref points to referent which has invalid\nformat. If it is valid, we will compare the untrimmed length and trimmed\nlength, if they are not the same, we need to warn the user there is some\ntrailing garbage in the symref content.\n\nAt last, we need to check whether the referent is the directory. We\ncannot distinguish whether the \"refs/heads/a\" is a directory or not by\nusing \"check_refname_format\". We have already checked bad file type when\niterating the \"refs/\" directory but we ignore the directory. Thus, we\nneed to explicitly add check here.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 4 ++\n fsck.h | 1 +\n refs/files-backend.c | 81 +++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 117 ++++++++++++++++++++++++++++++++++\n 4 files changed, 203 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 06d045ac48..beb6c4e49e 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,6 +28,10 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n+`badSymrefTarget`::\n+\t(ERROR) The symref target points outside the ref directory or\n+\tthe name of the symref target is invalid.\n+\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n \ndiff --git a/fsck.h b/fsck.h\nindex b85072df57..5ea874916d 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,6 +34,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n+\tFUNC(BAD_SYMREF_TARGET, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0187b85c5f..fef32e607f 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3434,11 +3434,80 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+/*\n+ * Check the symref \"referent\" and \"referent_path\". For textual symref,\n+ * \"referent\" would be the content after \"refs:\".\n+ */\n+static int files_fsck_symref_target(struct fsck_options *o,\n+\t\t\t\t struct fsck_ref_report *report,\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t struct strbuf *referent_path)\n+{\n+\tsize_t len = referent->len - 1;\n+\tconst char *p = NULL;\n+\tstruct stat st;\n+\tint ret = 0;\n+\n+\tif (!skip_prefix(referent->buf, \"refs/\", &p)) {\n+\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n+\t\t\t\t \"points to ref outside the refs directory\");\n+\t\tgoto out;\n+\t}\n+\n+\tif (referent->buf[referent->len - 1] != '\\n') {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t \"missing newline\");\n+\t\tlen++;\n+\t}\n+\n+\tstrbuf_rtrim(referent);\n+\tif (check_refname_format(referent->buf, 0)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n+\t\t\t\t \"points to refname with invalid format\");\n+\t\tgoto out;\n+\t}\n+\n+\tif (len != referent->len) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t \"trailing garbage in ref\");\n+\t}\n+\n+\t/*\n+\t * Missing target should not be treated as any error worthy event and\n+\t * not even warn. It is a common case that a symbolic ref points to a\n+\t * ref that does not exist yet. If the target ref does not exist, just\n+\t * skip the check for the file type.\n+\t */\n+\tif (lstat(referent_path->buf, &st))\n+\t\tgoto out;\n+\n+\t/*\n+\t * We cannot distinguish whether \"refs/heads/a\" is directory or nots by\n+\t * using \"check_refname_format(referent->buf, 0)\". Instead, we need to\n+\t * check the file type of the target.\n+\t */\n+\tif (S_ISDIR(st.st_mode)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n+\t\t\t\t \"points to the directory\");\n+\t\tgoto out;\n+\t}\n+\n+out:\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter)\n {\n+\tstruct strbuf referent_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -3484,12 +3553,24 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t\t \"trailing garbage in ref\");\n \t\t\tgoto cleanup;\n \t\t}\n+\t} else {\n+\t\tstrbuf_addf(&referent_path, \"%s/%s\",\n+\t\t\t ref_store->gitdir, referent.buf);\n+\t\t/*\n+\t\t * the referent may contain the spaces and the newline, need to\n+\t\t * trim for path.\n+\t\t */\n+\t\tstrbuf_rtrim(&referent_path);\n+\t\tret = files_fsck_symref_target(o, &report,\n+\t\t\t\t\t &referent,\n+\t\t\t\t\t &referent_path);\n \t}\n \n cleanup:\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n+\tstrbuf_release(&referent_path);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex a06ad044f2..e0bf8c8c8b 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -209,4 +209,121 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'textual symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-good &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: missing newline\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badSymrefTarget: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: reflogs/heads/main\\n\" >$branch_dir_prefix/branch-bad-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-2: badSymrefTarget: points to ref outside the refs directory\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/a\\n\" >$branch_dir_prefix/branch-bad-3 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-3: badSymrefTarget: points to the directory\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-3 &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'textual symref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\tprintf \"ref: reflogs/heads/main\\n\" >$branch_dir_prefix/branch-bad-2 &&\n+\tprintf \"ref: refs/heads/a\\n\" >$branch_dir_prefix/branch-bad-3 &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badSymrefTarget: points to refname with invalid format\n+\terror: refs/heads/branch-bad-2: badSymrefTarget: points to ref outside the refs directory\n+\terror: refs/heads/branch-bad-3: badSymrefTarget: points to the directory\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: missing newline\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"502033","messageId":"Ztb_Lzxgla2FHICH@ArchLinux","threadId":"61943","inReplyTo":"Ztb-mgl50cwGVO8A@ArchLinux","subject":"[PATCH v3 4/4] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-03T12:21:03Z","receivedAt":"2024-09-03T12:20:07Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already introduced \"files_fsck_symref_target\". We should reuse\nthis function to handle the symrefs which use legacy symbolic links. We\nshould not check the trailing garbage for symbolic refs. Add a new\nparameter \"symbolic_link\" to disable some checks which should only be\nexecuted for textual symrefs.\n\nWe firstly use the \"strbuf_add_real_path\" to resolve the symlink and\nget the absolute path \"referent_path\" which the symlink ref points\nto. Then we can get the absolute path \"abs_gitdir\" of the \"gitdir\".\nBy combining \"referent_path\" and \"abs_gitdir\", we can extract the\n\"referent\". Thus, we can reuse \"files_fsck_symref_target\" function to\nseamlessly check the symlink refs.\n\nBecause we are going to drop support for \"core.prefersymlinkrefs\", add a\nnew fsck message \"symlinkRef\" to let the user be aware of this\ninformation.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 5 ++\n fsck.h | 1 +\n refs/files-backend.c | 68 +++++++++++++++++++-----\n t/t0602-reffiles-fsck.sh | 97 +++++++++++++++++++++++++++++++++++\n 4 files changed, 157 insertions(+), 14 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex beb6c4e49e..9e8e1ac7f0 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -181,6 +181,11 @@\n \t(INFO) A ref does not end with newline. This kind of ref may\n \tbe considered ERROR in the future.\n \n+`symlinkRef`::\n+\t(INFO) A symref uses the symbolic link. This kind of symref may\n+\tbe considered ERROR in the future when totally dropping the\n+\tsymlink support.\n+\n `trailingRefContent`::\n \t(INFO) A ref has trailing contents. This kind of ref may be\n \tconsidered ERROR in the future.\ndiff --git a/fsck.h b/fsck.h\nindex 5ea874916d..1c6f750812 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -87,6 +87,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(SYMLINK_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex fef32e607f..2a1b952f0d 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1,4 +1,5 @@\n #include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n #include \"../copy.h\"\n #include \"../environment.h\"\n #include \"../gettext.h\"\n@@ -1950,10 +1951,13 @@ static int commit_ref_update(struct files_ref_store *refs,\n \treturn 0;\n }\n \n+#ifdef NO_SYMLINK_HEAD\n+#define create_ref_symlink(a, b) (-1)\n+#else\n static int create_ref_symlink(struct ref_lock *lock, const char *target)\n {\n \tint ret = -1;\n-#ifndef NO_SYMLINK_HEAD\n+\n \tchar *ref_path = get_locked_file_path(&lock->lk);\n \tunlink(ref_path);\n \tret = symlink(target, ref_path);\n@@ -1961,13 +1965,12 @@ static int create_ref_symlink(struct ref_lock *lock, const char *target)\n \n \tif (ret)\n \t\tfprintf(stderr, \"no symlink - falling back to symbolic ref\\n\");\n-#endif\n \treturn ret;\n }\n+#endif\n \n-static int create_symref_lock(struct files_ref_store *refs,\n-\t\t\t struct ref_lock *lock, const char *refname,\n-\t\t\t const char *target, struct strbuf *err)\n+static int create_symref_lock(struct ref_lock *lock, const char *target,\n+\t\t\t struct strbuf *err)\n {\n \tif (!fdopen_lock_file(&lock->lk, \"w\")) {\n \t\tstrbuf_addf(err, \"unable to fdopen %s: %s\",\n@@ -2583,8 +2586,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t}\n \n \tif (update->new_target && !(update->flags & REF_LOG_ONLY)) {\n-\t\tif (create_symref_lock(refs, lock, update->refname,\n-\t\t\t\t update->new_target, err)) {\n+\t\tif (create_symref_lock(lock, update->new_target, err)) {\n \t\t\tret = TRANSACTION_GENERIC_ERROR;\n \t\t\tgoto out;\n \t\t}\n@@ -3436,12 +3438,15 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \n /*\n * Check the symref \"referent\" and \"referent_path\". For textual symref,\n- * \"referent\" would be the content after \"refs:\".\n+ * \"referent\" would be the content after \"refs:\". For symlink ref,\n+ * \"referent\" would be the relative path agaignst \"gitdir\" which should\n+ * be the same as the textual symref literally.\n */\n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n \t\t\t\t struct strbuf *referent,\n-\t\t\t\t struct strbuf *referent_path)\n+\t\t\t\t struct strbuf *referent_path,\n+\t\t\t\t unsigned int symbolic_link)\n {\n \tsize_t len = referent->len - 1;\n \tconst char *p = NULL;\n@@ -3456,14 +3461,16 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n-\tif (referent->buf[referent->len - 1] != '\\n') {\n+\tif (!symbolic_link && referent->buf[referent->len - 1] != '\\n') {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n \t\t\t\t \"missing newline\");\n \t\tlen++;\n \t}\n \n-\tstrbuf_rtrim(referent);\n+\tif (!symbolic_link)\n+\t\tstrbuf_rtrim(referent);\n+\n \tif (check_refname_format(referent->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n@@ -3471,7 +3478,7 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n-\tif (len != referent->len) {\n+\tif (!symbolic_link && len != referent->len) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n \t\t\t\t \"trailing garbage in ref\");\n@@ -3509,9 +3516,11 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n {\n \tstruct strbuf referent_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n \tstruct fsck_ref_report report = {0};\n+\tunsigned int symbolic_link = 0;\n \tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n@@ -3521,8 +3530,37 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n \treport.path = refname.buf;\n \n-\tif (S_ISLNK(iter->st.st_mode))\n+\tif (S_ISLNK(iter->st.st_mode)) {\n+\t\tconst char* relative_referent_path;\n+\n+\t\tsymbolic_link = 1;\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_SYMLINK_REF,\n+\t\t\t\t \"use deprecated symbolic link for symref\");\n+\n+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n+\t\tstrbuf_normalize_path(&abs_gitdir);\n+\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n+\t\t\tstrbuf_addch(&abs_gitdir, '/');\n+\n+\t\tstrbuf_add_real_path(&referent_path, iter->path.buf);\n+\n+\t\tif (!skip_prefix(referent_path.buf,\n+\t\t\t\t abs_gitdir.buf,\n+\t\t\t\t &relative_referent_path)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n+\t\t\t\t\t \"point to target outside gitdir\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tstrbuf_addstr(&referent, relative_referent_path);\n+\t\tret = files_fsck_symref_target(o, &report,\n+\t\t\t\t\t &referent, &referent_path,\n+\t\t\t\t\t symbolic_link);\n+\n \t\tgoto cleanup;\n+\t}\n \n \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n \t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n@@ -3563,7 +3601,8 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\tstrbuf_rtrim(&referent_path);\n \t\tret = files_fsck_symref_target(o, &report,\n \t\t\t\t\t &referent,\n-\t\t\t\t\t &referent_path);\n+\t\t\t\t\t &referent_path,\n+\t\t\t\t\t symbolic_link);\n \t}\n \n cleanup:\n@@ -3571,6 +3610,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n \tstrbuf_release(&referent_path);\n+\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex e0bf8c8c8b..e735816d5b 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -326,4 +326,101 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success SYMLINKS 'symlink symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../../../branch $branch_dir_prefix/branch-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-1: badSymrefTarget: point to target outside gitdir\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-1 &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../logs/branch-bad $branch_dir_prefix/branch-symbolic-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-2: badSymrefTarget: points to ref outside the refs directory\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-2 &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic-3 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-3: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-3: badSymrefTarget: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-3 &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-1: badSymrefTarget: points to refname with invalid format\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./ $tag_dir_prefix/tag-symbolic-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-2: badSymrefTarget: points to the directory\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-2 &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success SYMLINKS 'symlink symref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tln -sf ../../../../branch $branch_dir_prefix/branch-symbolic-1 &&\n+\tln -sf ../../logs/branch-bad $branch_dir_prefix/branch-symbolic-2 &&\n+\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic-3 &&\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\tln -sf ./ $tag_dir_prefix/tag-symbolic-2 &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic-1: badSymrefTarget: point to target outside gitdir\n+\terror: refs/heads/branch-symbolic-2: badSymrefTarget: points to ref outside the refs directory\n+\terror: refs/heads/branch-symbolic-3: badSymrefTarget: points to refname with invalid format\n+\terror: refs/tags/tag-symbolic-1: badSymrefTarget: points to refname with invalid format\n+\terror: refs/tags/tag-symbolic-2: badSymrefTarget: points to the directory\n+\twarning: refs/heads/branch-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic-3: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/tags/tag-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"502453","messageId":"Zt8OZywRAxYaWqpo@pks.im","threadId":"61943","inReplyTo":"Ztb_HqLg-WvwA2I0@ArchLinux","subject":"Re: [PATCH v3 2/4] ref: add regular ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-09-09T15:04:07Z","receivedAt":"2024-09-09T15:04:11Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Sep 03, 2024 at 08:20:46PM +0800, shejialuo wrote:\n> We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\n> refs. However, when parsing the regular refs for files backend by using\n> \"files-backend.c::parse_loose_ref_contents\", we allow the ref content to\n> end with no newline or to contain some garbages.\n> \n> Even though we never create such loose refs ourselves, we have accepted\n> such loose refs. So, it is entirely possible that some third-party tools\n> may rely on such loose refs being valid. We should not report an error\n> fsck message at current. But let's notice such a \"curiously formatted\"\n> loose refs being valid and tell the user our findings, so we can access\n> the possible extent of damage when we tighten the parsing rules in the\n> future.\n> \n> And it's not suitable to either report a warn fsck message to the user.\n\ns/to either/either to\n\n> This is because if the caller set the \"strict\" field in \"fsck_options\",\n> fsck warns will be automatically upgraded to errors. We should not allow\n> user to specify the \"--strict\" flag to upgrade the fsck warnings to\n> errors at current.\n\nThis is formulated a bit curiously: it reads as if we wanted to limit\nwhat the user can do, but what we really want to ensure is that the\n`--strict` flag doesn't convert it into an error. So maybe something\nlike this instead of the second sentence:\n\n We don't (yet) want the \"--strict\" flag that controls this bit to\n end up generating errors for such weirdly-formatted reference\n contents, as we first want to assess whether this retroactive\n tightening will cause issues for any tools out there.\n\n> It might cause compatibility issue which may break\n\ns/issue/issues\n\n> the legacy repository. So we add the following two fsck infos to\n\nI wouldn't call it \"legacy\" just yet, as we didn't yet decide whether\nwe're going to make this formatting invalid in the first place. It's\nrather a test balloon.\n\n> represent the situation where the ref content ends without newline or has\n> garbages:\n\ns/garbages/trailing garbage\n\n> 1. \"refMissingNewline(INFO)\": A ref does not end with newline. This kind\n> of ref may be considered ERROR in the future.\n> 2. \"trailingRefContent(INFO)\": A ref has trailing contents. This kind of\n> ref may be considered ERROR in the future.\n\nIn both cases, \"may be considered ERROR\" -> \"may be considered an\nerror\". Also in the actual messages.\n\n> It may seem that we could not give the user any warnings by creating\n> fsck infos. However, in \"fsck.c::fsck_vreport\", we will convert\n> \"FSCK_INFO\" to \"FSCK_WARN\" and we can still warn the user about these\n> situations when using \"git-refs verify\" without introducing\n\ns/\"git-refs verify\"/\"git refs verify\". We don't use dashed builtins\nnowadays anymore.\n\n> compatibility issue.\n\ns/issue/issues\n\n> In current \"git-fsck(1)\", it will report an error when the ref content\n> is bad, so we should following this to report an error to the user when\n> \"parse_loose_ref_contents\" fails. And we add a new fsck error message\n> called \"badRefContent(ERROR)\" to represent that a ref has a bad content.\n\nOkay, so this is basically porting over behaviour that git-fsck(1)\nalready has to `git refs verify` and should thus not cause new issues\nanywhere. I think it would have made sense to do so in a first step and\nthen introduce the tightened rules in a separate commit.\n\nWill we eventually remove those checks from git-fsck(1) when we adapt it\nto call `git refs verify`? If so, we should likely note that in the\ncommit message.\n\n> In order to tell whether the ref has trailing content, add a new\n> parameter \"trailing\" to \"parse_loose_ref_contents\". Then introduce a new\n> function \"files_fsck_refs_content\" to check the regular refs to enhance\n> the \"git-refs verify\".\n\nThis paragraph only re-explains what the diff already tells us, so it\ncan likely be removed.\n\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> Documentation/fsck-msgids.txt | 11 ++++\n> fsck.h | 3 +\n> refs.c | 2 +-\n> refs/files-backend.c | 68 ++++++++++++++++++-\n> refs/refs-internal.h | 2 +-\n> t/t0602-reffiles-fsck.sh | 120 ++++++++++++++++++++++++++++++++++\n> 6 files changed, 202 insertions(+), 4 deletions(-)\n> \n> diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> index 68a2801f15..06d045ac48 100644\n> --- a/Documentation/fsck-msgids.txt\n> +++ b/Documentation/fsck-msgids.txt\n> @@ -19,6 +19,9 @@\n> `badParentSha1`::\n> \t(ERROR) A commit object has a bad parent sha1.\n> \n> +`badRefContent`::\n> +\t(ERROR) A ref has a bad content.\n> +\n\ns/a bad content/bad content\n\n> `badRefFiletype`::\n> \t(ERROR) A ref has a bad file type.\n> \n> @@ -170,6 +173,14 @@\n> `nullSha1`::\n> \t(WARN) Tree contains entries pointing to a null sha1.\n> \n> +`refMissingNewline`::\n> +\t(INFO) A ref does not end with newline. This kind of ref may\n> +\tbe considered ERROR in the future.\n> +\n\nI'd reformulate the second sentence to \"This will be considered an error\nin the future\". This indicates that we have the intent to tighten this\ncheck to any user and would urge them to speak up in case they disagree\nwith such a tightening.\n\n> +`trailingRefContent`::\n> +\t(INFO) A ref has trailing contents. This kind of ref may be\n> +\tconsidered ERROR in the future.\n\nSame.\n\n> @@ -3430,6 +3434,65 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> \t\t\t\t const char *refs_check_dir,\n> \t\t\t\t struct dir_iterator *iter);\n> \n> +static int files_fsck_refs_content(struct ref_store *ref_store,\n> +\t\t\t\t struct fsck_options *o,\n> +\t\t\t\t const char *refs_check_dir,\n> +\t\t\t\t struct dir_iterator *iter)\n> +{\n> +\tstruct strbuf ref_content = STRBUF_INIT;\n> +\tstruct strbuf referent = STRBUF_INIT;\n> +\tstruct strbuf refname = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = {0};\n> +\tconst char *trailing = NULL;\n> +\tunsigned int type = 0;\n> +\tint failure_errno = 0;\n> +\tstruct object_id oid;\n> +\tint ret = 0;\n> +\n> +\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n> +\treport.path = refname.buf;\n> +\n> +\tif (S_ISLNK(iter->st.st_mode))\n> +\t\tgoto cleanup;\n> +\n> +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> +\t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n> +\t\t\t\t refs_check_dir, iter->relative_path);\n\nWe typically have the name of things we read trailing and not leading in\nerror messages. So this should rather be \"unable do read ref '%s/%s'\".\n\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n> +\t\t\t\t ref_content.buf, &oid, &referent,\n> +\t\t\t\t &type, &trailing, &failure_errno)) {\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> +\t\t\t\t \"invalid ref content\");\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +\tif (!(type & REF_ISSYMREF)) {\n\nComing back to my comment further up, I guess this whole block here\ncould be introduced in a separate commit. So the first commit introduces\nthe infra to check loose ref contents as an obvious step because we\nsimply port over rules that already exist in git-fsck(1). And the second\nstep could then do this retroactive tightening with the justification\nyou have spelt out in the commit message.\n\n> +\t\tif (*trailing == '\\0') {\n\n`if (!*trailing)`\n\nPatrick\n"},{"id":"502454","messageId":"Zt8Oa4-N_8QMqUNJ@pks.im","threadId":"61943","inReplyTo":"Ztb_JuMjaoAbIZXq@ArchLinux","subject":"Re: [PATCH v3 3/4] ref: add symref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-09-09T15:04:11Z","receivedAt":"2024-09-09T15:04:14Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Sep 03, 2024 at 08:20:54PM +0800, shejialuo wrote:\n> We have already introduced the checks for regular refs. There is no need\n> to check the consistency of the target which the symref points to.\n> Instead, we just need to check the content of teh symref itself.\n\ns/teh/the\n\n> In order to check the content of the symref, create a function\n> \"files_fsck_symref_target\". It will first check whether the \"referent\"\n> is under the \"refs/\" directory and then we will check the symref\n> contents.\n> \n> A regular file is accepted as a textual symref if it begins with\n> \"ref:\", followed by zero or more whitespaces, followed by the full\n> refname, followed only by whitespace characters. We always write\n> a single SP after \"ref:\" and a single LF after the refname, but\n> third-party reimplementations of Git may have taken advantage of the\n> looser syntax. Put it more specific, we accept the following contents\n> of the symref:\n> \n> 1. \"ref: refs/heads/master \"\n> 2. \"ref: refs/heads/master \\n \\n\"\n> 3. \"ref: refs/heads/master\\n\\n\"\n> \n> But we do not allow any other trailing garbage. The followings are bad\n> symref contents which will be reported as fsck error by \"git-fsck(1)\".\n> \n> 1. \"ref: refs/heads/master garbage\\n\"\n> 2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n> \n> In order to provide above checks, we will first check whether the symref\n> content misses the newline by peeking the last byte of the \"referent\" to\n> see whether it is '\\n'.\n\nI'd still argue that we should do the same retroactive tightening as we\nintroduce for normal references, also with an INFO level at first.\nOtherwise we're being inconsistent across the ref types.\n\n> And we will remember the untrimmed length of the \"referent\" and call\n> \"strbuf_rtrim()\" on \"referent\". Then, we will call \"check_refname_format\"\n> to chceck whether the trimmed referent format is valid. If not, we will\n> report to the user that the symref points to referent which has invalid\n> format. If it is valid, we will compare the untrimmed length and trimmed\n> length, if they are not the same, we need to warn the user there is some\n> trailing garbage in the symref content.\n> \n> At last, we need to check whether the referent is the directory. We\n> cannot distinguish whether the \"refs/heads/a\" is a directory or not by\n> using \"check_refname_format\". We have already checked bad file type when\n> iterating the \"refs/\" directory but we ignore the directory. Thus, we\n> need to explicitly add check here.\n> \n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> Documentation/fsck-msgids.txt | 4 ++\n> fsck.h | 1 +\n> refs/files-backend.c | 81 +++++++++++++++++++++++\n> t/t0602-reffiles-fsck.sh | 117 ++++++++++++++++++++++++++++++++++\n> 4 files changed, 203 insertions(+)\n> \n> diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> index 06d045ac48..beb6c4e49e 100644\n> --- a/Documentation/fsck-msgids.txt\n> +++ b/Documentation/fsck-msgids.txt\n> @@ -28,6 +28,10 @@\n> `badRefName`::\n> \t(ERROR) A ref has an invalid format.\n> \n> +`badSymrefTarget`::\n> +\t(ERROR) The symref target points outside the ref directory or\n> +\tthe name of the symref target is invalid.\n\nThese are two separate error cases, and we even have different code\npaths raising them. Shouldn't we thus also have two different diagnostic\ncodes for this?\n\n> `badTagName`::\n> \t(INFO) A tag has an invalid format.\n> \n> diff --git a/fsck.h b/fsck.h\n> index b85072df57..5ea874916d 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -34,6 +34,7 @@ enum fsck_msg_type {\n> \tFUNC(BAD_REF_CONTENT, ERROR) \\\n> \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n> \tFUNC(BAD_REF_NAME, ERROR) \\\n> +\tFUNC(BAD_SYMREF_TARGET, ERROR) \\\n> \tFUNC(BAD_TIMEZONE, ERROR) \\\n> \tFUNC(BAD_TREE, ERROR) \\\n> \tFUNC(BAD_TREE_SHA1, ERROR) \\\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 0187b85c5f..fef32e607f 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3434,11 +3434,80 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> \t\t\t\t const char *refs_check_dir,\n> \t\t\t\t struct dir_iterator *iter);\n> \n> +/*\n> + * Check the symref \"referent\" and \"referent_path\". For textual symref,\n> + * \"referent\" would be the content after \"refs:\".\n> + */\n> +static int files_fsck_symref_target(struct fsck_options *o,\n> +\t\t\t\t struct fsck_ref_report *report,\n> +\t\t\t\t struct strbuf *referent,\n> +\t\t\t\t struct strbuf *referent_path)\n> +{\n> +\tsize_t len = referent->len - 1;\n> +\tconst char *p = NULL;\n> +\tstruct stat st;\n> +\tint ret = 0;\n> +\n> +\tif (!skip_prefix(referent->buf, \"refs/\", &p)) {\n> +\n\nThere's a superfluous newline here.\n\nAlso, you never use the value of `p`, so you can instead use\n`starts_with()`.\n\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n> +\t\t\t\t \"points to ref outside the refs directory\");\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tif (referent->buf[referent->len - 1] != '\\n') {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> +\t\t\t\t \"missing newline\");\n> +\t\tlen++;\n> +\t}\n> +\n> +\tstrbuf_rtrim(referent);\n> +\tif (check_refname_format(referent->buf, 0)) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n> +\t\t\t\t \"points to refname with invalid format\");\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tif (len != referent->len) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> +\t\t\t\t \"trailing garbage in ref\");\n> +\t}\n> +\n> +\t/*\n> +\t * Missing target should not be treated as any error worthy event and\n> +\t * not even warn. It is a common case that a symbolic ref points to a\n> +\t * ref that does not exist yet. If the target ref does not exist, just\n> +\t * skip the check for the file type.\n> +\t */\n> +\tif (lstat(referent_path->buf, &st))\n> +\t\tgoto out;\n\nWe may also want to verify that `errno == ENOENT` here.\n\nPatrick\n"},{"id":"502455","messageId":"Zt8OcPTzYg3raQlN@pks.im","threadId":"61943","inReplyTo":"Ztb_Lzxgla2FHICH@ArchLinux","subject":"Re: [PATCH v3 4/4] ref: add symlink ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-09-09T15:04:17Z","receivedAt":"2024-09-09T15:04:20Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Sep 03, 2024 at 08:21:03PM +0800, shejialuo wrote:\n> We have already introduced \"files_fsck_symref_target\". We should reuse\n> this function to handle the symrefs which use legacy symbolic links. We\n> should not check the trailing garbage for symbolic refs. Add a new\n> parameter \"symbolic_link\" to disable some checks which should only be\n> executed for textual symrefs.\n> \n> We firstly use the \"strbuf_add_real_path\" to resolve the symlink and\n> get the absolute path \"referent_path\" which the symlink ref points\n> to. Then we can get the absolute path \"abs_gitdir\" of the \"gitdir\".\n> By combining \"referent_path\" and \"abs_gitdir\", we can extract the\n> \"referent\". Thus, we can reuse \"files_fsck_symref_target\" function to\n> seamlessly check the symlink refs.\n> \n> Because we are going to drop support for \"core.prefersymlinkrefs\", add a\n> new fsck message \"symlinkRef\" to let the user be aware of this\n> information.\n\nI don't we fully decided to drop support for symrefs via symbolic links\nyet, so this is a tad too strong of a statement. I'd rather say that we\nconsider deprecating it in the future, but first need to asses whether\nthey may still be used.\n\nAlso, didn't we say that we'd want to remove support for _writing_\nsymbolic links, but not for reading them? Not a 100% sure though.\n\n> @@ -1961,13 +1965,12 @@ static int create_ref_symlink(struct ref_lock *lock, const char *target)\n> \n> \tif (ret)\n> \t\tfprintf(stderr, \"no symlink - falling back to symbolic ref\\n\");\n> -#endif\n> \treturn ret;\n> }\n> +#endif\n> \n> -static int create_symref_lock(struct files_ref_store *refs,\n> -\t\t\t struct ref_lock *lock, const char *refname,\n> -\t\t\t const char *target, struct strbuf *err)\n> +static int create_symref_lock(struct ref_lock *lock, const char *target,\n> +\t\t\t struct strbuf *err)\n> {\n> \tif (!fdopen_lock_file(&lock->lk, \"w\")) {\n> \t\tstrbuf_addf(err, \"unable to fdopen %s: %s\",\n> @@ -2583,8 +2586,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n> \t}\n> \n> \tif (update->new_target && !(update->flags & REF_LOG_ONLY)) {\n> -\t\tif (create_symref_lock(refs, lock, update->refname,\n> -\t\t\t\t update->new_target, err)) {\n> +\t\tif (create_symref_lock(lock, update->new_target, err)) {\n> \t\t\tret = TRANSACTION_GENERIC_ERROR;\n> \t\t\tgoto out;\n> \t\t}\n\nWhy does the writing side need to change?\n\n> @@ -3509,9 +3516,11 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> {\n> \tstruct strbuf referent_path = STRBUF_INIT;\n> \tstruct strbuf ref_content = STRBUF_INIT;\n> +\tstruct strbuf abs_gitdir = STRBUF_INIT;\n> \tstruct strbuf referent = STRBUF_INIT;\n> \tstruct strbuf refname = STRBUF_INIT;\n> \tstruct fsck_ref_report report = {0};\n> +\tunsigned int symbolic_link = 0;\n\nThis variable isn't used, as both code paths that end up using it could\njust statically set it to `1` or `0`.\n\n> \tconst char *trailing = NULL;\n> \tunsigned int type = 0;\n> \tint failure_errno = 0;\n> @@ -3521,8 +3530,37 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> \tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n> \treport.path = refname.buf;\n> \n> -\tif (S_ISLNK(iter->st.st_mode))\n> +\tif (S_ISLNK(iter->st.st_mode)) {\n> +\t\tconst char* relative_referent_path;\n> +\n> +\t\tsymbolic_link = 1;\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_SYMLINK_REF,\n> +\t\t\t\t \"use deprecated symbolic link for symref\");\n> +\n> +\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n> +\t\tstrbuf_normalize_path(&abs_gitdir);\n> +\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n> +\t\t\tstrbuf_addch(&abs_gitdir, '/');\n> +\n> +\t\tstrbuf_add_real_path(&referent_path, iter->path.buf);\n> +\n> +\t\tif (!skip_prefix(referent_path.buf,\n> +\t\t\t\t abs_gitdir.buf,\n> +\t\t\t\t &relative_referent_path)) {\n> +\t\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n> +\t\t\t\t\t \"point to target outside gitdir\");\n> +\t\t\tgoto cleanup;\n> +\t\t}\n> +\n> +\t\tstrbuf_addstr(&referent, relative_referent_path);\n> +\t\tret = files_fsck_symref_target(o, &report,\n> +\t\t\t\t\t &referent, &referent_path,\n> +\t\t\t\t\t symbolic_link);\n> +\n> \t\tgoto cleanup;\n> +\t}\n> \n> \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> \t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n\nPatrick\n"},{"id":"502548","messageId":"Zt_4ghf3YfumEG-j@ArchLinux","threadId":"61943","inReplyTo":"Zt8OZywRAxYaWqpo@pks.im","subject":"Re: [PATCH v3 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-10T07:42:58Z","receivedAt":"2024-09-10T07:41:55Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Sep 09, 2024 at 05:04:07PM +0200, Patrick Steinhardt wrote:\n> > This is because if the caller set the \"strict\" field in \"fsck_options\",\n> > fsck warns will be automatically upgraded to errors. We should not allow\n> > user to specify the \"--strict\" flag to upgrade the fsck warnings to\n> > errors at current.\n> \n> This is formulated a bit curiously: it reads as if we wanted to limit\n> what the user can do, but what we really want to ensure is that the\n> `--strict` flag doesn't convert it into an error. So maybe something\n> like this instead of the second sentence:\n> \n> We don't (yet) want the \"--strict\" flag that controls this bit to\n> end up generating errors for such weirdly-formatted reference\n> contents, as we first want to assess whether this retroactive\n> tightening will cause issues for any tools out there.\n> \n\nThanks, I will improve this in the next version.\n\n> > the legacy repository. So we add the following two fsck infos to\n> \n> I wouldn't call it \"legacy\" just yet, as we didn't yet decide whether\n> we're going to make this formatting invalid in the first place. It's\n> rather a test balloon.\n> \n\nI agree, we should drop \"legacy\" here.\n\n> > In current \"git-fsck(1)\", it will report an error when the ref content\n> > is bad, so we should following this to report an error to the user when\n> > \"parse_loose_ref_contents\" fails. And we add a new fsck error message\n> > called \"badRefContent(ERROR)\" to represent that a ref has a bad content.\n> \n> Okay, so this is basically porting over behaviour that git-fsck(1)\n> already has to `git refs verify` and should thus not cause new issues\n> anywhere. I think it would have made sense to do so in a first step and\n> then introduce the tightened rules in a separate commit.\n> \n\nBy reading the whole comments, we'd better create a commit which ports\nthe existing checks to \"git refs verify\" both for regular refs and\nsymrefs.\n\nSo, I will add more commits in the next version with the following\nsequences:\n\n1. Set up the infrastructure to check the contents for refs.\n2. Port existing checks in \"git-fsck(1)\" to \"git refs verify\".\n3. Introduce the tightened rules.\n\n> Will we eventually remove those checks from git-fsck(1) when we adapt it\n> to call `git refs verify`? If so, we should likely note that in the\n> commit message.\n\nWe should do this, as we have discussed before, \"git-fsck(1)\" implicitly\nchecks some refs which makes the code hard to understand.\n\n> Coming back to my comment further up, I guess this whole block here\n> could be introduced in a separate commit. So the first commit introduces\n> the infra to check loose ref contents as an obvious step because we\n> simply port over rules that already exist in git-fsck(1). And the second\n> step could then do this retroactive tightening with the justification\n> you have spelt out in the commit message.\n\nYes, it will be much more clear. So, I should not simply classify the\nsituations by the type of refs.\n\nThanks,\nJialuo\n"},{"id":"502551","messageId":"Zt_9C1LbqO4oT70w@ArchLinux","threadId":"61943","inReplyTo":"Zt8Oa4-N_8QMqUNJ@pks.im","subject":"Re: [PATCH v3 3/4] ref: add symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-10T08:02:19Z","receivedAt":"2024-09-10T08:01:15Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Sep 09, 2024 at 05:04:11PM +0200, Patrick Steinhardt wrote:\n> > In order to provide above checks, we will first check whether the symref\n> > content misses the newline by peeking the last byte of the \"referent\" to\n> > see whether it is '\\n'.\n> \n> I'd still argue that we should do the same retroactive tightening as we\n> introduce for normal references, also with an INFO level at first.\n> Otherwise we're being inconsistent across the ref types.\n> \n\nActually, for above situations, we will use the same fsck error message\nids introduce in [PATCH v3 2/4]. And I think we must refer to this in\nthis commit message.\n\nBut it makes me wonder should we use a new commit to introduce these\ntwo fsck message ids?\n\n> > diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> > index 06d045ac48..beb6c4e49e 100644\n> > --- a/Documentation/fsck-msgids.txt\n> > +++ b/Documentation/fsck-msgids.txt\n> > @@ -28,6 +28,10 @@\n> > `badRefName`::\n> > \t(ERROR) A ref has an invalid format.\n> > \n> > +`badSymrefTarget`::\n> > +\t(ERROR) The symref target points outside the ref directory or\n> > +\tthe name of the symref target is invalid.\n> \n> These are two separate error cases, and we even have different code\n> paths raising them. Shouldn't we thus also have two different diagnostic\n> codes for this?\n> \n\nI agree. I will improve in the next version.\n\n> > +\tif (!skip_prefix(referent->buf, \"refs/\", &p)) {\n> > +\n> \n> There's a superfluous newline here.\n> \n> Also, you never use the value of `p`, so you can instead use\n> `starts_with()`.\n> \n\nThanks, actually I have searched the code with \"is_prefix\". Well, I\ndidn't think about \"starts_<>\".\n\n> > +\t/*\n> > +\t * Missing target should not be treated as any error worthy event and\n> > +\t * not even warn. It is a common case that a symbolic ref points to a\n> > +\t * ref that does not exist yet. If the target ref does not exist, just\n> > +\t * skip the check for the file type.\n> > +\t */\n> > +\tif (lstat(referent_path->buf, &st))\n> > +\t\tgoto out;\n> \n> We may also want to verify that `errno == ENOENT` here.\n> \n\nI agree, if \"errno != ENOENT\", we should report to the user about this\nref-unrelated failure.\n\nThanks,\nJialuo\n"},{"id":"502553","messageId":"ZuADKU3Lzpee4KJQ@ArchLinux","threadId":"61943","inReplyTo":"Zt8OcPTzYg3raQlN@pks.im","subject":"Re: [PATCH v3 4/4] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-10T08:28:25Z","receivedAt":"2024-09-10T08:27:21Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Sep 09, 2024 at 05:04:17PM +0200, Patrick Steinhardt wrote:\n> > Because we are going to drop support for \"core.prefersymlinkrefs\", add a\n> > new fsck message \"symlinkRef\" to let the user be aware of this\n> > information.\n> \n> I don't we fully decided to drop support for symrefs via symbolic links\n> yet, so this is a tad too strong of a statement. I'd rather say that we\n> consider deprecating it in the future, but first need to asses whether\n> they may still be used.\n> \n\nYes, that will be much better.\n\n> Also, didn't we say that we'd want to remove support for _writing_\n> symbolic links, but not for reading them? Not a 100% sure though.\n> \n\nI have re-read the Junio's patch about the breaking change. We will drop\nthe support for writing. But for reading we may or may not. I will\nimprove this in the next version.\n\n> > \tif (update->new_target && !(update->flags & REF_LOG_ONLY)) {\n> > -\t\tif (create_symref_lock(refs, lock, update->refname,\n> > -\t\t\t\t update->new_target, err)) {\n> > +\t\tif (create_symref_lock(lock, update->new_target, err)) {\n> > \t\t\tret = TRANSACTION_GENERIC_ERROR;\n> > \t\t\tgoto out;\n> > \t\t}\n> \n> Why does the writing side need to change?\n> \n\nI squash two patches provided by Junio to sync with the \"master\" branch\nto make sure the build could be passed. This is because Peff has\nintroduced the \"UNUSED\" check when building.\n\nSo we could just ignore this part.\n\nThanks,\nJialuo\n"},{"id":"502564","messageId":"CAOLa=ZT8N7TRSVNhqGrjskMTTFgO16Q4VKMVM1LPHtEorkT6cg@mail.gmail.com","threadId":"61943","inReplyTo":"Ztb_HqLg-WvwA2I0@ArchLinux","subject":"Re: [PATCH v3 2/4] ref: add regular ref content check for files backend","fromName":"karthik nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-09-10T16:07:15Z","receivedAt":"2024-09-10T16:07:17Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\n> refs. However, when parsing the regular refs for files backend by using\n\nNit: s/for/in the/\n\n> \"files-backend.c::parse_loose_ref_contents\", we allow the ref content to\n> end with no newline or to contain some garbages.\n\nThe 'no newline' reads a bit odd, perhaps, \"we allow the ref's content\nto end with garbage or without a newline.\"\n\n\n> Even though we never create such loose refs ourselves, we have accepted\n> such loose refs. So, it is entirely possible that some third-party tools\n> may rely on such loose refs being valid. We should not report an error\n> fsck message at current. But let's notice such a \"curiously formatted\"\n\ns/such a/such/ since the next line uses 'refs' in plural form.\n\n> loose refs being valid and tell the user our findings, so we can access\n\ns/access/assess\n\n> the possible extent of damage when we tighten the parsing rules in the\n> future.\n>\n\nWe could also rewrite the last sentence to make it a little more clearer\nas \"We should notify the users about such 'curiously formatted' loose\nrefs so that adequate care is taken before we decide to tighter the rules\nin the future.\"\n\n> And it's not suitable to either report a warn fsck message to the user.\n> This is because if the caller set the \"strict\" field in \"fsck_options\",\n> fsck warns will be automatically upgraded to errors. We should not allow\n> user to specify the \"--strict\" flag to upgrade the fsck warnings to\n> errors at current. It might cause compatibility issue which may break\n> the legacy repository. So we add the following two fsck infos to\n\nI think Patrick touched base here and I agree with his comments.\n\n> represent the situation where the ref content ends without newline or has\n> garbages:\n>\n> 1. \"refMissingNewline(INFO)\": A ref does not end with newline. This kind\n> of ref may be considered ERROR in the future.\n> 2. \"trailingRefContent(INFO)\": A ref has trailing contents. This kind of\n\ns/contents/content\n\n> ref may be considered ERROR in the future.\n>\n> It may seem that we could not give the user any warnings by creating\n\ns/could/would\n\n> fsck infos. However, in \"fsck.c::fsck_vreport\", we will convert\n\nI think we can also rephrase this first sentence a little better,\nperhaps:\n\n It might appear that we can't provide the user with any warnings by\n using FSCK_INFO.\n\n> \"FSCK_INFO\" to \"FSCK_WARN\" and we can still warn the user about these\n> situations when using \"git-refs verify\" without introducing\n> compatibility issue.\n\ns/issue/issues\n\n> In current \"git-fsck(1)\", it will report an error when the ref content\n> is bad, so we should following this to report an error to the user when\n> \"parse_loose_ref_contents\" fails. And we add a new fsck error message\n> called \"badRefContent(ERROR)\" to represent that a ref has a bad content.\n\nI would rephrase this a bit, as:\n\n The \"git-fsck(1)\" command reports an error when the ref content is\n invalid. Following this, add a similar check to \"git refs verify\". A\n a new fsck error message called \"badRefContent(ERROR)\" to represent\n that a ref has a invalid content.\n\n[snip]\n\n> +static int files_fsck_refs_content(struct ref_store *ref_store,\n> +\t\t\t\t struct fsck_options *o,\n> +\t\t\t\t const char *refs_check_dir,\n> +\t\t\t\t struct dir_iterator *iter)\n> +{\n> +\tstruct strbuf ref_content = STRBUF_INIT;\n> +\tstruct strbuf referent = STRBUF_INIT;\n> +\tstruct strbuf refname = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = {0};\n> +\tconst char *trailing = NULL;\n> +\tunsigned int type = 0;\n> +\tint failure_errno = 0;\n> +\tstruct object_id oid;\n> +\tint ret = 0;\n> +\n> +\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n> +\treport.path = refname.buf;\n> +\n> +\tif (S_ISLNK(iter->st.st_mode))\n> +\t\tgoto cleanup;\n\nSince we iterate over all refs, we don't need to check the target for a\nsymbolic link. So we skip all symbolic links. Makes sense. Would be nice\nto have a comment here.\n\n[snip]\n"},{"id":"502602","messageId":"CAOLa=ZS2TsRAeAHJ6B9h82-H2tSG-vZMRBSpspQ3hOW5GBdciw@mail.gmail.com","threadId":"61943","inReplyTo":"Ztb_JuMjaoAbIZXq@ArchLinux","subject":"Re: [PATCH v3 3/4] ref: add symref content check for files backend","fromName":"karthik nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-09-10T22:19:49Z","receivedAt":"2024-09-10T22:19:51Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n[snip]\n\n> And we will remember the untrimmed length of the \"referent\" and call\n> \"strbuf_rtrim()\" on \"referent\". Then, we will call \"check_refname_format\"\n> to chceck whether the trimmed referent format is valid. If not, we will\n\ns/chceck/check\n\n> report to the user that the symref points to referent which has invalid\n> format. If it is valid, we will compare the untrimmed length and trimmed\n> length, if they are not the same, we need to warn the user there is some\n> trailing garbage in the symref content.\n>\n> At last, we need to check whether the referent is the directory. We\n\ns/is the/is a/\n\n> cannot distinguish whether the \"refs/heads/a\" is a directory or not by\n\nIt would be a little clearer if we say\n\n We cannot distinguish whether a given reference like 'refs/heads/a'\n is a file or a directory.\n\n> using \"check_refname_format\". We have already checked bad file type when\n> iterating the \"refs/\" directory but we ignore the directory. Thus, we\n> need to explicitly add check here.\n>\n\n[snip]\n\n> +/*\n> + * Check the symref \"referent\" and \"referent_path\". For textual symref,\n> + * \"referent\" would be the content after \"refs:\".\n> + */\n> +static int files_fsck_symref_target(struct fsck_options *o,\n> +\t\t\t\t struct fsck_ref_report *report,\n> +\t\t\t\t struct strbuf *referent,\n> +\t\t\t\t struct strbuf *referent_path)\n> +{\n> +\tsize_t len = referent->len - 1;\n> +\tconst char *p = NULL;\n> +\tstruct stat st;\n> +\tint ret = 0;\n> +\n> +\tif (!skip_prefix(referent->buf, \"refs/\", &p)) {\n> +\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n> +\t\t\t\t \"points to ref outside the refs directory\");\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tif (referent->buf[referent->len - 1] != '\\n') {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> +\t\t\t\t \"missing newline\");\n> +\t\tlen++;\n> +\t}\n> +\n> +\tstrbuf_rtrim(referent);\n> +\tif (check_refname_format(referent->buf, 0)) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n> +\t\t\t\t \"points to refname with invalid format\");\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tif (len != referent->len) {\n\nWould this work with a symref containing:\n\n ref: refs/heads/feature\\ngarbage\\n\n\nSince we check last character and rtrim, wouldn't this bypass our\nchecks? Isn't it better to find the first `\\n` and check if the index <\nreferent->len?\n\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> +\t\t\t\t \"trailing garbage in ref\");\n> +\t}\n> +\n> +\t/*\n> +\t * Missing target should not be treated as any error worthy event and\n> +\t * not even warn. It is a common case that a symbolic ref points to a\n> +\t * ref that does not exist yet. If the target ref does not exist, just\n> +\t * skip the check for the file type.\n> +\t */\n\nI think the common terminology for this is 'dangling symref'. Perhaps we\ncould shorten this to simply say:\n\n Dangling symrefs are common and so we don't report them.\n\n> +\tif (lstat(referent_path->buf, &st))\n> +\t\tgoto out;\n> +\n> +\t/*\n> +\t * We cannot distinguish whether \"refs/heads/a\" is directory or nots by\n\ns/is/is a/\ns/nots/not/\n\n> +\t * using \"check_refname_format(referent->buf, 0)\". Instead, we need to\n> +\t * check the file type of the target.\n> +\t */\n> +\tif (S_ISDIR(st.st_mode)) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n> +\t\t\t\t \"points to the directory\");\n> +\t\tgoto out;\n> +\t}\n> +\n> +out:\n> +\treturn ret;\n> +}\n> +\n\n[snip]\n"},{"id":"502652","messageId":"ZuJnSHXJw6AVzbxL@ArchLinux","threadId":"61943","inReplyTo":"CAOLa=ZS2TsRAeAHJ6B9h82-H2tSG-vZMRBSpspQ3hOW5GBdciw@mail.gmail.com","subject":"Re: [PATCH v3 3/4] ref: add symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-12T04:00:08Z","receivedAt":"2024-09-12T03:59:03Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Sep 10, 2024 at 03:19:49PM -0700, karthik nayak wrote:\n\n[snip]\n\n> > +\tif (referent->buf[referent->len - 1] != '\\n') {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> > +\t\t\t\t \"missing newline\");\n> > +\t\tlen++;\n> > +\t}\n> > +\n> > +\tstrbuf_rtrim(referent);\n> > +\tif (check_refname_format(referent->buf, 0)) {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n> > +\t\t\t\t \"points to refname with invalid format\");\n> > +\t\tgoto out;\n> > +\t}\n> > +\n> > +\tif (len != referent->len) {\n> \n> Would this work with a symref containing:\n> \n> ref: refs/heads/feature\\ngarbage\\n\n> \n> Since we check last character and rtrim, wouldn't this bypass our\n> checks? Isn't it better to find the first `\\n` and check if the index <\n> referent->len?\n> \n\nWe will check the above example by \"check_refname_format\". It will\nreport the following message:\n\n error: ... : badSymrefTarget: points to refname with invalid format\n\nFrom the context, I guess you suggest that we should report there is a\ntrailing garbage in the ref. However, for the above situation, we should\nreport an error which is align with the behavior of the \"git-fsck(1)\".\n\nSo there is no need to check whether there is a trailing garbage when we\nencounter an error.\n\nAnd we cannot use this way, for example:\n\n ref: refs/heads/feature \\n\n\nIf we find the first '\\n' index. In this example, index will be equal to\n\"referent->len\". And we totally ignore this case.\n\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> > +\t\t\t\t \"trailing garbage in ref\");\n> > +\t}\n> > +\n> > +\t/*\n> > +\t * Missing target should not be treated as any error worthy event and\n> > +\t * not even warn. It is a common case that a symbolic ref points to a\n> > +\t * ref that does not exist yet. If the target ref does not exist, just\n> > +\t * skip the check for the file type.\n> > +\t */\n> \n> I think the common terminology for this is 'dangling symref'. Perhaps we\n> could shorten this to simply say:\n> \n> Dangling symrefs are common and so we don't report them.\n> \n\nThanks, I will improve this in the next version.\n"},{"id":"502746","messageId":"ZuQTApWlM8jOt9Ev@ArchLinux","threadId":"61943","inReplyTo":"CAOLa=ZT8N7TRSVNhqGrjskMTTFgO16Q4VKMVM1LPHtEorkT6cg@mail.gmail.com","subject":"Re: [PATCH v3 2/4] ref: add regular ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-13T10:25:06Z","receivedAt":"2024-09-13T10:23:59Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Sep 10, 2024 at 09:07:15AM -0700, karthik nayak wrote:\n\n[snip]\n\n> > +static int files_fsck_refs_content(struct ref_store *ref_store,\n> > +\t\t\t\t struct fsck_options *o,\n> > +\t\t\t\t const char *refs_check_dir,\n> > +\t\t\t\t struct dir_iterator *iter)\n> > +{\n> > +\tstruct strbuf ref_content = STRBUF_INIT;\n> > +\tstruct strbuf referent = STRBUF_INIT;\n> > +\tstruct strbuf refname = STRBUF_INIT;\n> > +\tstruct fsck_ref_report report = {0};\n> > +\tconst char *trailing = NULL;\n> > +\tunsigned int type = 0;\n> > +\tint failure_errno = 0;\n> > +\tstruct object_id oid;\n> > +\tint ret = 0;\n> > +\n> > +\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n> > +\treport.path = refname.buf;\n> > +\n> > +\tif (S_ISLNK(iter->st.st_mode))\n> > +\t\tgoto cleanup;\n> \n> Since we iterate over all refs, we don't need to check the target for a\n> symbolic link. So we skip all symbolic links. Makes sense. Would be nice\n> to have a comment here.\n> \n\nToday I am handling the reviews, there is a misunderstanding here. It's\ncorrect that \"We don't need to check the target for a symbolic link\".\nBut we do need to check the symbolic links. It might be a symlink\nsymref. In here, we just ignore the implementation and will be\nimplemented in the later patch.\n\n"},{"id":"502759","messageId":"ZuRzCyjQFilGhj8j@ArchLinux","threadId":"61943","inReplyTo":"Ztb-mgl50cwGVO8A@ArchLinux","subject":"[PATCH v4 0/5] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-13T17:14:51Z","receivedAt":"2024-09-13T17:13:45Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis version handles some minor problems mainly focus at the improving\ncommit messages, comments and some minor problems.\n\n1. Split [PATCH v3 2/4] into two commits [PATCH v4 2/5] and [PATCH v4\n3/5]. [PATCH v4 2/5] integrates \"git-fsck(1)\"'s check and [PATCH v4 3/5]\ntightens rules to check the refs with trailing garbage and refs without\nnewline.\n\n2. Handle a lot of typo errors in original [PATCH v3 2/4]. And improve\nthe fsck-msgids documentation.\n\n3. Improve [PATCH v4 4/5]'s commit message to first introduce the\ntighten rules to be consistent with the [PATCH v4 3/5].\n\n4. Remove \"badSymrefTarget(ERROR)\" fsck message. Add three new messages\nto be more specific:\n\n 1. badReferentFiletype(ERROR): The referent of a symref has a bad file\n type.\n\n 2. badReferentName(ERROR): The referent name of a symref is invalid.\n\n 3. escapeReferent(ERROR): The referent of a symref is outside the\n ref directory\n\n5. Handle typos and some minor problems.\n\nBecause I add more commits, I provide the \"--interdiff\" here to make the\nreviewer's life easier.\n\nHowever, because I have not merged the latest ci fixup, so I cannot\nverify some jobs in CIs. May need the help from Junio to verify.\n\nThanks,\nJialuo\n\nshejialuo (5):\n ref: initialize \"fsck_ref_report\" with zero\n ref: port git-fsck(1) regular refs check for files backend\n ref: add more strict checks for regular refs\n ref: add symref content check for files backend\n ref: add symlink ref content check for files backend\n\n Documentation/fsck-msgids.txt | 25 +++\n fsck.h | 7 +\n refs.c | 2 +-\n refs/files-backend.c | 202 +++++++++++++++++++-\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 334 ++++++++++++++++++++++++++++++++++\n 6 files changed, 560 insertions(+), 12 deletions(-)\n\nInterdiff against v3:\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 9e8e1ac7f0..31626e765b 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -20,7 +20,7 @@\n \t(ERROR) A commit object has a bad parent sha1.\n \n `badRefContent`::\n-\t(ERROR) A ref has a bad content.\n+\t(ERROR) A ref has bad content.\n \n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n@@ -28,9 +28,11 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n-`badSymrefTarget`::\n-\t(ERROR) The symref target points outside the ref directory or\n-\tthe name of the symref target is invalid.\n+`badReferentFiletype`::\n+\t(ERROR) The referent of a symref has a bad file type.\n+\n+`badReferentName`::\n+\t(ERROR) The referent name of a symref is invalid.\n \n `badTagName`::\n \t(INFO) A tag has an invalid format.\n@@ -53,6 +55,9 @@\n `emptyName`::\n \t(WARN) A path contains an empty name.\n \n+`escapeReferent`::\n+\t(ERROR) The referent of a symref is outside the \"ref\" directory.\n+\n `extraHeaderEntry`::\n \t(IGNORE) Extra headers found after `tagger`.\n \n@@ -178,8 +183,8 @@\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n `refMissingNewline`::\n-\t(INFO) A ref does not end with newline. This kind of ref may\n-\tbe considered ERROR in the future.\n+\t(INFO) A ref does not end with newline. This will be\n+\tconsidered an error in the future.\n \n `symlinkRef`::\n \t(INFO) A symref uses the symbolic link. This kind of symref may\n@@ -187,8 +192,8 @@\n \tsymlink support.\n \n `trailingRefContent`::\n-\t(INFO) A ref has trailing contents. This kind of ref may be\n-\tconsidered ERROR in the future.\n+\t(INFO) A ref has trailing content. This will be\n+\tconsidered an error in the future.\n \n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\ndiff --git a/fsck.h b/fsck.h\nindex 1c6f750812..b72ee632a4 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,12 +34,14 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n-\tFUNC(BAD_SYMREF_TARGET, ERROR) \\\n+\tFUNC(BAD_REFERENT_FILETYPE, ERROR) \\\n+\tFUNC(BAD_REFERENT_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\n \tFUNC(BAD_TYPE, ERROR) \\\n \tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n+\tFUNC(ESCAPE_REFERENT, ERROR) \\\n \tFUNC(MISSING_AUTHOR, ERROR) \\\n \tFUNC(MISSING_COMMITTER, ERROR) \\\n \tFUNC(MISSING_EMAIL, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 2a1b952f0d..c511deb509 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3449,14 +3449,12 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t unsigned int symbolic_link)\n {\n \tsize_t len = referent->len - 1;\n-\tconst char *p = NULL;\n \tstruct stat st;\n \tint ret = 0;\n \n-\tif (!skip_prefix(referent->buf, \"refs/\", &p)) {\n-\n+\tif (!starts_with(referent->buf, \"refs/\")) {\n \t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n+\t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n \t\t\t\t \"points to ref outside the refs directory\");\n \t\tgoto out;\n \t}\n@@ -3473,7 +3471,7 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \n \tif (check_refname_format(referent->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n \t\t\t\t \"points to refname with invalid format\");\n \t\tgoto out;\n \t}\n@@ -3485,22 +3483,24 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t}\n \n \t/*\n-\t * Missing target should not be treated as any error worthy event and\n-\t * not even warn. It is a common case that a symbolic ref points to a\n-\t * ref that does not exist yet. If the target ref does not exist, just\n-\t * skip the check for the file type.\n+\t * Dangling symrefs are common and so we don't report them.\n \t */\n-\tif (lstat(referent_path->buf, &st))\n+\tif (lstat(referent_path->buf, &st)) {\n+\t\tif (errno != ENOENT) {\n+\t\t\tret = error_errno(_(\"unable to stat '%s'\"),\n+\t\t\t\t\t referent_path->buf);\n+\t\t}\n \t\tgoto out;\n+\t}\n \n \t/*\n-\t * We cannot distinguish whether \"refs/heads/a\" is directory or nots by\n+\t * We cannot distinguish whether \"refs/heads/a\" is a directory or not by\n \t * using \"check_refname_format(referent->buf, 0)\". Instead, we need to\n \t * check the file type of the target.\n \t */\n \tif (S_ISDIR(st.st_mode)) {\n \t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT_FILETYPE,\n \t\t\t\t \"points to the directory\");\n \t\tgoto out;\n \t}\n@@ -3520,7 +3520,6 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n \tstruct fsck_ref_report report = {0};\n-\tunsigned int symbolic_link = 0;\n \tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n@@ -3533,7 +3532,6 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tif (S_ISLNK(iter->st.st_mode)) {\n \t\tconst char* relative_referent_path;\n \n-\t\tsymbolic_link = 1;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_SYMLINK_REF,\n \t\t\t\t \"use deprecated symbolic link for symref\");\n@@ -3549,21 +3547,20 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t abs_gitdir.buf,\n \t\t\t\t &relative_referent_path)) {\n \t\t\tret = fsck_report_ref(o, &report,\n-\t\t\t\t\t FSCK_MSG_BAD_SYMREF_TARGET,\n+\t\t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n \t\t\t\t\t \"point to target outside gitdir\");\n \t\t\tgoto cleanup;\n \t\t}\n \n \t\tstrbuf_addstr(&referent, relative_referent_path);\n \t\tret = files_fsck_symref_target(o, &report,\n-\t\t\t\t\t &referent, &referent_path,\n-\t\t\t\t\t symbolic_link);\n+\t\t\t\t\t &referent, &referent_path, 1);\n \n \t\tgoto cleanup;\n \t}\n \n \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n-\t\tret = error_errno(_(\"%s/%s: unable to read the ref\"),\n+\t\tret = error_errno(_(\"unable to read ref '%s/%s'\"),\n \t\t\t\t refs_check_dir, iter->relative_path);\n \t\tgoto cleanup;\n \t}\n@@ -3578,14 +3575,14 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t}\n \n \tif (!(type & REF_ISSYMREF)) {\n-\t\tif (*trailing == '\\0') {\n+\t\tif (!*trailing) {\n \t\t\tret = fsck_report_ref(o, &report,\n \t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n \t\t\t\t\t \"missing newline\");\n \t\t\tgoto cleanup;\n \t\t}\n \n-\t\tif (*trailing != '\\n' || (*(trailing + 1) != '\\0')) {\n+\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n \t\t\tret = fsck_report_ref(o, &report,\n \t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n \t\t\t\t\t \"trailing garbage in ref\");\n@@ -3602,7 +3599,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\tret = files_fsck_symref_target(o, &report,\n \t\t\t\t\t &referent,\n \t\t\t\t\t &referent_path,\n-\t\t\t\t\t symbolic_link);\n+\t\t\t\t\t 0);\n \t}\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex e735816d5b..7c3579705f 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -268,7 +268,7 @@ test_expect_success 'textual symref content should be checked (individual)' '\n \tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badSymrefTarget: points to refname with invalid format\n+\terror: refs/heads/branch-bad-1: badReferentName: points to refname with invalid format\n \tEOF\n \trm $branch_dir_prefix/branch-bad-1 &&\n \ttest_cmp expect err &&\n@@ -276,7 +276,7 @@ test_expect_success 'textual symref content should be checked (individual)' '\n \tprintf \"ref: reflogs/heads/main\\n\" >$branch_dir_prefix/branch-bad-2 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-2: badSymrefTarget: points to ref outside the refs directory\n+\terror: refs/heads/branch-bad-2: escapeReferent: points to ref outside the refs directory\n \tEOF\n \trm $branch_dir_prefix/branch-bad-2 &&\n \ttest_cmp expect err &&\n@@ -284,7 +284,7 @@ test_expect_success 'textual symref content should be checked (individual)' '\n \tprintf \"ref: refs/heads/a\\n\" >$branch_dir_prefix/branch-bad-3 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-3: badSymrefTarget: points to the directory\n+\terror: refs/heads/branch-bad-3: badReferentFiletype: points to the directory\n \tEOF\n \trm $branch_dir_prefix/branch-bad-3 &&\n \ttest_cmp expect err\n@@ -311,9 +311,9 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badSymrefTarget: points to refname with invalid format\n-\terror: refs/heads/branch-bad-2: badSymrefTarget: points to ref outside the refs directory\n-\terror: refs/heads/branch-bad-3: badSymrefTarget: points to the directory\n+\terror: refs/heads/branch-bad-1: badReferentName: points to refname with invalid format\n+\terror: refs/heads/branch-bad-2: escapeReferent: points to ref outside the refs directory\n+\terror: refs/heads/branch-bad-3: badReferentFiletype: points to the directory\n \twarning: refs/heads/a/b/branch-complicated: refMissingNewline: missing newline\n \twarning: refs/heads/a/b/branch-complicated: trailingRefContent: trailing garbage in ref\n \twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: missing newline\n@@ -347,7 +347,7 @@ test_expect_success SYMLINKS 'symlink symref content should be checked (individu\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \twarning: refs/heads/branch-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-1: badSymrefTarget: point to target outside gitdir\n+\terror: refs/heads/branch-symbolic-1: escapeReferent: point to target outside gitdir\n \tEOF\n \trm $branch_dir_prefix/branch-symbolic-1 &&\n \ttest_cmp expect err &&\n@@ -356,7 +356,7 @@ test_expect_success SYMLINKS 'symlink symref content should be checked (individu\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \twarning: refs/heads/branch-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-2: badSymrefTarget: points to ref outside the refs directory\n+\terror: refs/heads/branch-symbolic-2: escapeReferent: points to ref outside the refs directory\n \tEOF\n \trm $branch_dir_prefix/branch-symbolic-2 &&\n \ttest_cmp expect err &&\n@@ -365,7 +365,7 @@ test_expect_success SYMLINKS 'symlink symref content should be checked (individu\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \twarning: refs/heads/branch-symbolic-3: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-3: badSymrefTarget: points to refname with invalid format\n+\terror: refs/heads/branch-symbolic-3: badReferentName: points to refname with invalid format\n \tEOF\n \trm $branch_dir_prefix/branch-symbolic-3 &&\n \ttest_cmp expect err &&\n@@ -374,7 +374,7 @@ test_expect_success SYMLINKS 'symlink symref content should be checked (individu\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-1: badSymrefTarget: points to refname with invalid format\n+\terror: refs/tags/tag-symbolic-1: badReferentName: points to refname with invalid format\n \tEOF\n \trm $tag_dir_prefix/tag-symbolic-1 &&\n \ttest_cmp expect err &&\n@@ -383,7 +383,7 @@ test_expect_success SYMLINKS 'symlink symref content should be checked (individu\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \twarning: refs/tags/tag-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-2: badSymrefTarget: points to the directory\n+\terror: refs/tags/tag-symbolic-2: badReferentFiletype: points to the directory\n \tEOF\n \trm $tag_dir_prefix/tag-symbolic-2 &&\n \ttest_cmp expect err\n@@ -407,11 +407,11 @@ test_expect_success SYMLINKS 'symlink symref content should be checked (aggregat\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-symbolic-1: badSymrefTarget: point to target outside gitdir\n-\terror: refs/heads/branch-symbolic-2: badSymrefTarget: points to ref outside the refs directory\n-\terror: refs/heads/branch-symbolic-3: badSymrefTarget: points to refname with invalid format\n-\terror: refs/tags/tag-symbolic-1: badSymrefTarget: points to refname with invalid format\n-\terror: refs/tags/tag-symbolic-2: badSymrefTarget: points to the directory\n+\terror: refs/heads/branch-symbolic-1: escapeReferent: point to target outside gitdir\n+\terror: refs/heads/branch-symbolic-2: escapeReferent: points to ref outside the refs directory\n+\terror: refs/heads/branch-symbolic-3: badReferentName: points to refname with invalid format\n+\terror: refs/tags/tag-symbolic-1: badReferentName: points to refname with invalid format\n+\terror: refs/tags/tag-symbolic-2: badReferentFiletype: points to the directory\n \twarning: refs/heads/branch-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n \twarning: refs/heads/branch-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n \twarning: refs/heads/branch-symbolic-3: symlinkRef: use deprecated symbolic link for symref\n-- \n2.46.0\n\n"},{"id":"502760","messageId":"ZuRztuVQSVY6SiXF@ArchLinux","threadId":"61943","inReplyTo":"ZuRzCyjQFilGhj8j@ArchLinux","subject":"[PATCH v4 1/5] ref: initialize \"fsck_ref_report\" with zero","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-13T17:17:42Z","receivedAt":"2024-09-13T17:16:36Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n\"referent\" is NULL. So, we need to always initialize these parameters to\nNULL instead of letting them point to anywhere when creating a new\n\"fsck_ref_report\" structure.\n\nThe original code explicitly initializes the \"path\" member in the\n\"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\nmembers in the struct). It is more customary to use \" {0} \" to express\nthat we are 0-initializing everything. In order to be align with the the\ncodebase, initialize \"fsck_ref_report\" with zero.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8d6ec9458d..890d0324e1 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3446,7 +3446,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tstruct fsck_ref_report report = { .path = NULL };\n+\t\tstruct fsck_ref_report report = { 0 };\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n-- \n2.46.0\n\n"},{"id":"502761","messageId":"ZuRzwKTFd65RL4HC@ArchLinux","threadId":"61943","inReplyTo":"ZuRzCyjQFilGhj8j@ArchLinux","subject":"[PATCH v4 2/5] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-13T17:17:52Z","receivedAt":"2024-09-13T17:16:45Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\nrefs. However, we have already set up the infrastructure of the ref\nconsistency checks. We need to port original checks from \"git-fsck(1)\".\nThus, we could clean the \"git-fsck(1)\" code by removing these implicit\nchecks.\n\nThe \"git-fsck(1)\" command reports an error when the ref content is\ninvalid. Following this, add a similar check to \"git refs verify\".\nAdd a new fsck error message called \"badRefContent(ERROR)\" to represent\nthat a ref has an invalid content.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 ++\n fsck.h | 1 +\n refs/files-backend.c | 43 +++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 60 +++++++++++++++++++++++++++++++++++\n 4 files changed, 107 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 68a2801f15..22c385ea22 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -19,6 +19,9 @@\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n+`badRefContent`::\n+\t(ERROR) A ref has bad content.\n+\n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n \ndiff --git a/fsck.h b/fsck.h\nindex 500b4c04d2..0d99a87911 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -31,6 +31,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 890d0324e1..b1ed2e5c04 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3430,6 +3430,48 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_refs_content(struct ref_store *ref_store,\n+\t\t\t\t struct fsck_options *o,\n+\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t struct dir_iterator *iter)\n+{\n+\tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tstruct strbuf refname = STRBUF_INIT;\n+\tstruct fsck_ref_report report = {0};\n+\tunsigned int type = 0;\n+\tint failure_errno = 0;\n+\tstruct object_id oid;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n+\treport.path = refname.buf;\n+\n+\tif (S_ISLNK(iter->st.st_mode))\n+\t\tgoto cleanup;\n+\n+\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n+\t\tret = error_errno(_(\"unable to read ref '%s/%s'\"),\n+\t\t\t\t refs_check_dir, iter->relative_path);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n+\t\t\t\t ref_content.buf, &oid, &referent,\n+\t\t\t\t &type, &failure_errno)) {\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"invalid ref content\");\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&refname);\n+\tstrbuf_release(&ref_content);\n+\tstrbuf_release(&referent);\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n \t\t\t\tconst char *refs_check_dir,\n@@ -3512,6 +3554,7 @@ static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n+\t\tfiles_fsck_refs_content,\n \t\tNULL,\n \t};\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 71a4d1a5ae..a1205b3a3b 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -89,4 +89,64 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'regular ref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"%sx\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-bad-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-bad-1: badRefContent: invalid ref content\n+\tEOF\n+\trm $tag_dir_prefix/tag-bad-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"xfsazqfxcadas\" >$tag_dir_prefix/tag-bad-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-bad-2: badRefContent: invalid ref content\n+\tEOF\n+\trm $tag_dir_prefix/tag-bad-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"xfsazqfxcadas\" >$branch_dir_prefix/a/b/branch-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: invalid ref content\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-bad &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'regular ref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"%sx\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-bad-1 &&\n+\tprintf \"xfsazqfxcadas\" >$tag_dir_prefix/tag-bad-2 &&\n+\tprintf \"xfsazqfxcadas\" >$branch_dir_prefix/a/b/branch-bad &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: invalid ref content\n+\terror: refs/tags/tag-bad-1: badRefContent: invalid ref content\n+\terror: refs/tags/tag-bad-2: badRefContent: invalid ref content\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"502762","messageId":"ZuRzxyjAI3tp4uLK@ArchLinux","threadId":"61943","inReplyTo":"ZuRzCyjQFilGhj8j@ArchLinux","subject":"[PATCH v4 3/5] ref: add more strict checks for regular refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-13T17:17:59Z","receivedAt":"2024-09-13T17:16:53Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already used \"parse_loose_ref_contents\" function to check\nwhether the ref content is valid in files backend. However, by\nusing \"parse_loose_ref_contents\", we allow the ref's content to end with\ngarbage or without a newline.\n\nEven though we never create such loose refs ourselves, we have accepted\nsuch loose refs. So, it is entirely possible that some third-party tools\nmay rely on such loose refs being valid. We should not report an error\nfsck message at current. We should notify the users about such\n\"curiously formatted\" loose refs so that adequate care is taken before\nwe decide to tighten the rules in the future.\n\nAnd it's not suitable either to report a warn fsck message to the user.\nWe don't yet want the \"--strict\" flag that controls this bit to end up\ngenerating errors for such weirdly-formatted reference contents, as we\nfirst want to assess whether this retroactive tightening will cause\nissues for any tools out there. It may cause compatibility issues which\nmay break the repository. So we add the following two fsck infos to\nrepresent the situation where the ref content ends without newline or\nhas trailing garbages:\n\n1. refMissingNewline(INFO): A ref does not end with newline. This will\n be considered an error in the future.\n2. trailingRefContent(INFO): A ref has trailing content. This will be\n considered an error in the future.\n\nIt might appear that we can't provide the user with any warnings by\nusing FSCK_INFO. However, in \"fsck.c::fsck_vreport\", we will convert\nFSCK_INFO to FSCK_WARN and we can still warn the user about these\nsituations when using \"git refs verify\" without introducing\ncompatibility issues.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 8 +++++\n fsck.h | 2 ++\n refs.c | 2 +-\n refs/files-backend.c | 27 ++++++++++++++--\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 60 +++++++++++++++++++++++++++++++++++\n 6 files changed, 96 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 22c385ea22..8827137ef0 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -173,6 +173,14 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`refMissingNewline`::\n+\t(INFO) A ref does not end with newline. This will be\n+\tconsidered an error in the future.\n+\n+`trailingRefContent`::\n+\t(INFO) A ref has trailing content. This will be\n+\tconsidered an error in the future.\n+\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \ndiff --git a/fsck.h b/fsck.h\nindex 0d99a87911..b85072df57 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -85,6 +85,8 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs.c b/refs.c\nindex 74de3d3009..5e74881945 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1758,7 +1758,7 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \t}\n \n \tresult = parse_loose_ref_contents(ref_store->repo->hash_algo, content.buf,\n-\t\t\t\t\t oid, referent, type, failure_errno);\n+\t\t\t\t\t oid, referent, type, NULL, failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b1ed2e5c04..df4ce270ae 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -560,7 +560,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname,\n \tbuf = sb_contents.buf;\n \n \tret = parse_loose_ref_contents(ref_store->repo->hash_algo, buf,\n-\t\t\t\t oid, referent, type, &myerr);\n+\t\t\t\t oid, referent, type, NULL, &myerr);\n \n out:\n \tif (ret && !myerr)\n@@ -597,7 +597,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno)\n+\t\t\t const char **trailing, int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -619,6 +619,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n+\n+\tif (trailing)\n+\t\t*trailing = p;\n+\n \treturn 0;\n }\n \n@@ -3439,6 +3443,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n \tstruct fsck_ref_report report = {0};\n+\tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n \tstruct object_id oid;\n@@ -3458,13 +3463,29 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n \t\t\t\t ref_content.buf, &oid, &referent,\n-\t\t\t\t &type, &failure_errno)) {\n+\t\t\t\t &type, &trailing, &failure_errno)) {\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n \t\t\t\t \"invalid ref content\");\n \t\tgoto cleanup;\n \t}\n \n+\tif (!(type & REF_ISSYMREF)) {\n+\t\tif (!*trailing) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t \"missing newline\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t \"trailing garbage in ref\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n cleanup:\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 2313c830d8..73b05f971b 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -715,7 +715,7 @@ struct ref_store {\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno);\n+\t\t\t const char **trailing, int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex a1205b3a3b..a06ad044f2 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -101,6 +101,54 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \tgit refs verify 2>err &&\n \ttest_must_be_empty err &&\n \n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: missing newline\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-1: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-2: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\\n\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-3: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n+\ttest_must_fail git -c fsck.trailingRefContent=error refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-garbage-4: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-4 &&\n+\ttest_cmp expect err &&\n+\n \tprintf \"%sx\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-bad-1 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n@@ -135,6 +183,12 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \ttest_commit default &&\n \tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n+\tprintf \"%s garbage\\n\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n \tprintf \"%sx\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-bad-1 &&\n \tprintf \"xfsazqfxcadas\" >$tag_dir_prefix/tag-bad-2 &&\n \tprintf \"xfsazqfxcadas\" >$branch_dir_prefix/a/b/branch-bad &&\n@@ -144,6 +198,12 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \terror: refs/heads/a/b/branch-bad: badRefContent: invalid ref content\n \terror: refs/tags/tag-bad-1: badRefContent: invalid ref content\n \terror: refs/tags/tag-bad-2: badRefContent: invalid ref content\n+\twarning: refs/heads/branch-garbage: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: missing newline\n+\twarning: refs/tags/tag-garbage-1: trailingRefContent: trailing garbage in ref\n+\twarning: refs/tags/tag-garbage-2: trailingRefContent: trailing garbage in ref\n+\twarning: refs/tags/tag-garbage-3: trailingRefContent: trailing garbage in ref\n+\twarning: refs/tags/tag-garbage-4: trailingRefContent: trailing garbage in ref\n \tEOF\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err\n-- \n2.46.0\n\n"},{"id":"502763","messageId":"ZuRzzwZds8ys-JEN@ArchLinux","threadId":"61943","inReplyTo":"ZuRzCyjQFilGhj8j@ArchLinux","subject":"[PATCH v4 4/5] ref: add symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-13T17:18:07Z","receivedAt":"2024-09-13T17:17:00Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already introduced the checks for regular refs. There is no need\nto check the consistency of the target which the symref points to.\nInstead, we just need to check the content of the symref itself.\n\nA regular file is accepted as a textual symref if it begins with\n\"ref:\", followed by zero or more whitespaces, followed by the full\nrefname, followed only by whitespace characters. We always write\na single SP after \"ref:\" and a single LF after the refname, but\nthird-party reimplementations of Git may have taken advantage of the\nlooser syntax. Put it more specific, we accept the following contents\nof the symref:\n\n1. \"ref: refs/heads/master \"\n2. \"ref: refs/heads/master \\n \\n\"\n3. \"ref: refs/heads/master\\n\\n\"\n\nThus, we could reuse \"refMissingNewline\" and \"trailingRefContent\"\nFSCK_INFOs to do the same retroactive tightening as we introduce for\nregular references.\n\nBut we do not allow any other trailing garbage. The followings are bad\nsymref contents which will be reported as fsck error by \"git-fsck(1)\".\n\n1. \"ref: refs/heads/master garbage\\n\"\n2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n\nAnd we introduce a new \"badReferentName(ERROR)\" fsck message to report\nabove errors to the user.\n\nIn order to check the content of the symref, create a function\n\"files_fsck_symref_target\". It will first check whether the \"referent\"\nis under the \"refs/\" directory, if not, we will report \"escapeReferent\"\nfsck error message to notify the user this situation.\n\nThen, we will first check whether the symref content misses the newline\nby peeking the last byte of the \"referent\" to see whether it is '\\n'.\n\nAnd we will remember the untrimmed length of the \"referent\" and call\n\"strbuf_rtrim()\" on \"referent\". Then, we will call \"check_refname_format\"\nto check whether the trimmed referent format is valid. If not, we will\nreport to the user that the symref points to referent which has invalid\nformat. If it is valid, we will compare the untrimmed length and trimmed\nlength, if they are not the same, we need to warn the user there is some\ntrailing garbage in the symref content.\n\nAt last, we need to check whether the referent is a directory. We cannot\ndistinguish whether a given reference like \"refs/heads/a\" is a file or a\ndirectory by using \"check_refname_format\". We have already checked bad\nfile type when iterating the \"refs/\" directory but we ignore the\ndirectory. Thus, we need to explicitly add check here.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 9 +++\n fsck.h | 3 +\n refs/files-backend.c | 81 +++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 117 ++++++++++++++++++++++++++++++++++\n 4 files changed, 210 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 8827137ef0..03bcb77972 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,6 +28,12 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n+`badReferentFiletype`::\n+\t(ERROR) The referent of a symref has a bad file type.\n+\n+`badReferentName`::\n+\t(ERROR) The referent name of a symref is invalid.\n+\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n \n@@ -49,6 +55,9 @@\n `emptyName`::\n \t(WARN) A path contains an empty name.\n \n+`escapeReferent`::\n+\t(ERROR) The referent of a symref is outside the \"ref\" directory.\n+\n `extraHeaderEntry`::\n \t(IGNORE) Extra headers found after `tagger`.\n \ndiff --git a/fsck.h b/fsck.h\nindex b85072df57..c90561c6db 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,11 +34,14 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n+\tFUNC(BAD_REFERENT_FILETYPE, ERROR) \\\n+\tFUNC(BAD_REFERENT_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\n \tFUNC(BAD_TYPE, ERROR) \\\n \tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n+\tFUNC(ESCAPE_REFERENT, ERROR) \\\n \tFUNC(MISSING_AUTHOR, ERROR) \\\n \tFUNC(MISSING_COMMITTER, ERROR) \\\n \tFUNC(MISSING_EMAIL, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex df4ce270ae..0cb4a2da73 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3434,11 +3434,80 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+/*\n+ * Check the symref \"referent\" and \"referent_path\". For textual symref,\n+ * \"referent\" would be the content after \"refs:\".\n+ */\n+static int files_fsck_symref_target(struct fsck_options *o,\n+\t\t\t\t struct fsck_ref_report *report,\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t struct strbuf *referent_path)\n+{\n+\tsize_t len = referent->len - 1;\n+\tstruct stat st;\n+\tint ret = 0;\n+\n+\tif (!starts_with(referent->buf, \"refs/\")) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n+\t\t\t\t \"points to ref outside the refs directory\");\n+\t\tgoto out;\n+\t}\n+\n+\tif (referent->buf[referent->len - 1] != '\\n') {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t \"missing newline\");\n+\t\tlen++;\n+\t}\n+\n+\tstrbuf_rtrim(referent);\n+\tif (check_refname_format(referent->buf, 0)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n+\t\t\t\t \"points to refname with invalid format\");\n+\t\tgoto out;\n+\t}\n+\n+\tif (len != referent->len) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t \"trailing garbage in ref\");\n+\t}\n+\n+\t/*\n+\t * Dangling symrefs are common and so we don't report them.\n+\t */\n+\tif (lstat(referent_path->buf, &st)) {\n+\t\tif (errno != ENOENT) {\n+\t\t\tret = error_errno(_(\"unable to stat '%s'\"),\n+\t\t\t\t\t referent_path->buf);\n+\t\t}\n+\t\tgoto out;\n+\t}\n+\n+\t/*\n+\t * We cannot distinguish whether \"refs/heads/a\" is a directory or not by\n+\t * using \"check_refname_format(referent->buf, 0)\". Instead, we need to\n+\t * check the file type of the target.\n+\t */\n+\tif (S_ISDIR(st.st_mode)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT_FILETYPE,\n+\t\t\t\t \"points to the directory\");\n+\t\tgoto out;\n+\t}\n+\n+out:\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter)\n {\n+\tstruct strbuf referent_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -3484,12 +3553,24 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t\t \"trailing garbage in ref\");\n \t\t\tgoto cleanup;\n \t\t}\n+\t} else {\n+\t\tstrbuf_addf(&referent_path, \"%s/%s\",\n+\t\t\t ref_store->gitdir, referent.buf);\n+\t\t/*\n+\t\t * the referent may contain the spaces and the newline, need to\n+\t\t * trim for path.\n+\t\t */\n+\t\tstrbuf_rtrim(&referent_path);\n+\t\tret = files_fsck_symref_target(o, &report,\n+\t\t\t\t\t &referent,\n+\t\t\t\t\t &referent_path);\n \t}\n \n cleanup:\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n+\tstrbuf_release(&referent_path);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex a06ad044f2..9580c340ab 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -209,4 +209,121 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'textual symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-good &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: missing newline\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: trailing garbage in ref\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badReferentName: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: reflogs/heads/main\\n\" >$branch_dir_prefix/branch-bad-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-2: escapeReferent: points to ref outside the refs directory\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/a\\n\" >$branch_dir_prefix/branch-bad-3 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-3: badReferentFiletype: points to the directory\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-3 &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'textual symref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\tprintf \"ref: reflogs/heads/main\\n\" >$branch_dir_prefix/branch-bad-2 &&\n+\tprintf \"ref: refs/heads/a\\n\" >$branch_dir_prefix/branch-bad-3 &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badReferentName: points to refname with invalid format\n+\terror: refs/heads/branch-bad-2: escapeReferent: points to ref outside the refs directory\n+\terror: refs/heads/branch-bad-3: badReferentFiletype: points to the directory\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: missing newline\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: trailing garbage in ref\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: missing newline\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"502764","messageId":"ZuRz1_cHDnr_pWzo@ArchLinux","threadId":"61943","inReplyTo":"ZuRzCyjQFilGhj8j@ArchLinux","subject":"[PATCH v4 5/5] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-13T17:18:15Z","receivedAt":"2024-09-13T17:17:09Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already introduced \"files_fsck_symref_target\". We should reuse\nthis function to handle the symrefs which use legacy symbolic links. We\nshould not check the trailing garbage for symbolic refs. Add a new\nparameter \"symbolic_link\" to disable some checks which should only be\nexecuted for textual symrefs.\n\nWe firstly use the \"strbuf_add_real_path\" to resolve the symlink and\nget the absolute path \"referent_path\" which the symlink ref points\nto. Then we can get the absolute path \"abs_gitdir\" of the \"gitdir\".\nBy combining \"referent_path\" and \"abs_gitdir\", we can extract the\n\"referent\". Thus, we can reuse \"files_fsck_symref_target\" function to\nseamlessly check the symlink refs.\n\nBecause we consider deprecating writing the symbolic links and for\nreading, we may or may not deprecate. We first need to asses whether\nsymbolic links may still be used. So, add a new fsck message\n\"symlinkRef(INFO)\" to let the user be aware of this information.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 5 ++\n fsck.h | 1 +\n refs/files-backend.c | 65 ++++++++++++++++++-----\n t/t0602-reffiles-fsck.sh | 97 +++++++++++++++++++++++++++++++++++\n 4 files changed, 154 insertions(+), 14 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 03bcb77972..31626e765b 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -186,6 +186,11 @@\n \t(INFO) A ref does not end with newline. This will be\n \tconsidered an error in the future.\n \n+`symlinkRef`::\n+\t(INFO) A symref uses the symbolic link. This kind of symref may\n+\tbe considered ERROR in the future when totally dropping the\n+\tsymlink support.\n+\n `trailingRefContent`::\n \t(INFO) A ref has trailing content. This will be\n \tconsidered an error in the future.\ndiff --git a/fsck.h b/fsck.h\nindex c90561c6db..b72ee632a4 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -89,6 +89,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(SYMLINK_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0cb4a2da73..c511deb509 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1,4 +1,5 @@\n #include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n #include \"../copy.h\"\n #include \"../environment.h\"\n #include \"../gettext.h\"\n@@ -1950,10 +1951,13 @@ static int commit_ref_update(struct files_ref_store *refs,\n \treturn 0;\n }\n \n+#ifdef NO_SYMLINK_HEAD\n+#define create_ref_symlink(a, b) (-1)\n+#else\n static int create_ref_symlink(struct ref_lock *lock, const char *target)\n {\n \tint ret = -1;\n-#ifndef NO_SYMLINK_HEAD\n+\n \tchar *ref_path = get_locked_file_path(&lock->lk);\n \tunlink(ref_path);\n \tret = symlink(target, ref_path);\n@@ -1961,13 +1965,12 @@ static int create_ref_symlink(struct ref_lock *lock, const char *target)\n \n \tif (ret)\n \t\tfprintf(stderr, \"no symlink - falling back to symbolic ref\\n\");\n-#endif\n \treturn ret;\n }\n+#endif\n \n-static int create_symref_lock(struct files_ref_store *refs,\n-\t\t\t struct ref_lock *lock, const char *refname,\n-\t\t\t const char *target, struct strbuf *err)\n+static int create_symref_lock(struct ref_lock *lock, const char *target,\n+\t\t\t struct strbuf *err)\n {\n \tif (!fdopen_lock_file(&lock->lk, \"w\")) {\n \t\tstrbuf_addf(err, \"unable to fdopen %s: %s\",\n@@ -2583,8 +2586,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,\n \t}\n \n \tif (update->new_target && !(update->flags & REF_LOG_ONLY)) {\n-\t\tif (create_symref_lock(refs, lock, update->refname,\n-\t\t\t\t update->new_target, err)) {\n+\t\tif (create_symref_lock(lock, update->new_target, err)) {\n \t\t\tret = TRANSACTION_GENERIC_ERROR;\n \t\t\tgoto out;\n \t\t}\n@@ -3436,12 +3438,15 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \n /*\n * Check the symref \"referent\" and \"referent_path\". For textual symref,\n- * \"referent\" would be the content after \"refs:\".\n+ * \"referent\" would be the content after \"refs:\". For symlink ref,\n+ * \"referent\" would be the relative path agaignst \"gitdir\" which should\n+ * be the same as the textual symref literally.\n */\n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n \t\t\t\t struct strbuf *referent,\n-\t\t\t\t struct strbuf *referent_path)\n+\t\t\t\t struct strbuf *referent_path,\n+\t\t\t\t unsigned int symbolic_link)\n {\n \tsize_t len = referent->len - 1;\n \tstruct stat st;\n@@ -3454,14 +3459,16 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n-\tif (referent->buf[referent->len - 1] != '\\n') {\n+\tif (!symbolic_link && referent->buf[referent->len - 1] != '\\n') {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n \t\t\t\t \"missing newline\");\n \t\tlen++;\n \t}\n \n-\tstrbuf_rtrim(referent);\n+\tif (!symbolic_link)\n+\t\tstrbuf_rtrim(referent);\n+\n \tif (check_refname_format(referent->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n@@ -3469,7 +3476,7 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n-\tif (len != referent->len) {\n+\tif (!symbolic_link && len != referent->len) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n \t\t\t\t \"trailing garbage in ref\");\n@@ -3509,6 +3516,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n {\n \tstruct strbuf referent_path = STRBUF_INIT;\n \tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n \tstruct fsck_ref_report report = {0};\n@@ -3521,8 +3529,35 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n \treport.path = refname.buf;\n \n-\tif (S_ISLNK(iter->st.st_mode))\n+\tif (S_ISLNK(iter->st.st_mode)) {\n+\t\tconst char* relative_referent_path;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_SYMLINK_REF,\n+\t\t\t\t \"use deprecated symbolic link for symref\");\n+\n+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n+\t\tstrbuf_normalize_path(&abs_gitdir);\n+\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n+\t\t\tstrbuf_addch(&abs_gitdir, '/');\n+\n+\t\tstrbuf_add_real_path(&referent_path, iter->path.buf);\n+\n+\t\tif (!skip_prefix(referent_path.buf,\n+\t\t\t\t abs_gitdir.buf,\n+\t\t\t\t &relative_referent_path)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n+\t\t\t\t\t \"point to target outside gitdir\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\n+\t\tstrbuf_addstr(&referent, relative_referent_path);\n+\t\tret = files_fsck_symref_target(o, &report,\n+\t\t\t\t\t &referent, &referent_path, 1);\n+\n \t\tgoto cleanup;\n+\t}\n \n \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n \t\tret = error_errno(_(\"unable to read ref '%s/%s'\"),\n@@ -3563,7 +3598,8 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\tstrbuf_rtrim(&referent_path);\n \t\tret = files_fsck_symref_target(o, &report,\n \t\t\t\t\t &referent,\n-\t\t\t\t\t &referent_path);\n+\t\t\t\t\t &referent_path,\n+\t\t\t\t\t 0);\n \t}\n \n cleanup:\n@@ -3571,6 +3607,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n \tstrbuf_release(&referent_path);\n+\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 9580c340ab..7c3579705f 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -326,4 +326,101 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success SYMLINKS 'symlink symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../../../branch $branch_dir_prefix/branch-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-1: escapeReferent: point to target outside gitdir\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-1 &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../logs/branch-bad $branch_dir_prefix/branch-symbolic-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-2: escapeReferent: points to ref outside the refs directory\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-2 &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic-3 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-3: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-3: badReferentName: points to refname with invalid format\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-3 &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-1: badReferentName: points to refname with invalid format\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./ $tag_dir_prefix/tag-symbolic-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-2: badReferentFiletype: points to the directory\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-2 &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success SYMLINKS 'symlink symref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tln -sf ../../../../branch $branch_dir_prefix/branch-symbolic-1 &&\n+\tln -sf ../../logs/branch-bad $branch_dir_prefix/branch-symbolic-2 &&\n+\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic-3 &&\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\tln -sf ./ $tag_dir_prefix/tag-symbolic-2 &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-symbolic-1: escapeReferent: point to target outside gitdir\n+\terror: refs/heads/branch-symbolic-2: escapeReferent: points to ref outside the refs directory\n+\terror: refs/heads/branch-symbolic-3: badReferentName: points to refname with invalid format\n+\terror: refs/tags/tag-symbolic-1: badReferentName: points to refname with invalid format\n+\terror: refs/tags/tag-symbolic-2: badReferentFiletype: points to the directory\n+\twarning: refs/heads/branch-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic-3: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/tags/tag-symbolic-2: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_done\n-- \n2.46.0\n\n"},{"id":"503008","messageId":"xmqqfrpwj4mo.fsf@gitster.g","threadId":"61943","inReplyTo":"ZuRztuVQSVY6SiXF@ArchLinux","subject":"Re: [PATCH v4 1/5] ref: initialize \"fsck_ref_report\" with zero","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-18T16:41:51Z","receivedAt":"2024-09-18T16:41:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n> \"referent\" is NULL. So, we need to always initialize these parameters to\n> NULL instead of letting them point to anywhere when creating a new\n> \"fsck_ref_report\" structure.\n>\n> The original code explicitly initializes the \"path\" member in the\n> \"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\n> members in the struct). It is more customary to use \" {0} \" to express\n\n\" {0} \" -> \"{ 0 }\" \n\n> that we are 0-initializing everything. In order to be align with the the\n\n\"be align with the the\" -> \"align with the\"\n\n> codebase, initialize \"fsck_ref_report\" with zero.\n\nBoth I'll amend in-place so no need to reroll just for these.\n\nThanks.\n\n>\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> refs/files-backend.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 8d6ec9458d..890d0324e1 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3446,7 +3446,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n> \t\tgoto cleanup;\n> \n> \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n> -\t\tstruct fsck_ref_report report = { .path = NULL };\n> +\t\tstruct fsck_ref_report report = { 0 };\n> \n> \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n> \t\treport.path = sb.buf;\n"},{"id":"503009","messageId":"xmqqa5g4j4ap.fsf@gitster.g","threadId":"61943","inReplyTo":"ZuRzCyjQFilGhj8j@ArchLinux","subject":"Re: [PATCH v4 0/5] add ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-18T16:49:02Z","receivedAt":"2024-09-18T16:49:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> Because I add more commits, I provide the \"--interdiff\" here to make the\n> reviewer's life easier.\n\nYeah, for the changes from the previous iteration of this series,\nrange-diff comparison is pretty much useless. Interdiff is indeed\nmore usable, but essentially this iteration deserves reviews with\nfresh sets of eyes.\n\n> However, because I have not merged the latest ci fixup, so I cannot\n> verify some jobs in CIs. May need the help from Junio to verify.\n\nA good way to do so is to fork a temporary branch at the tip of\nthese 5 commits, and then either merge or cherry-pick the CI fixup.\nSuch a temporary branch should be usable for CI testing, right?\n\nThanks.\n\nPS.\n\nI am not feeling well today; please expect delayed and/or sparse\nresponses.\n\n"},{"id":"503012","messageId":"xmqqh6acdbz2.fsf@gitster.g","threadId":"61943","inReplyTo":"ZuRzwKTFd65RL4HC@ArchLinux","subject":"Re: [PATCH v4 2/5] ref: port git-fsck(1) regular refs check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-18T18:59:45Z","receivedAt":"2024-09-18T18:59:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> We implicitly rely on \"git-fsck(1)\" to check the consistency of regular\n> refs. However, we have already set up the infrastructure of the ref\n> consistency checks. We need to port original checks from \"git-fsck(1)\".\n> Thus, we could clean the \"git-fsck(1)\" code by removing these implicit\n> checks.\n\nThe above reads as if you are, in preparation to \"port\" the checks\nwe have in \"fsck\" to elsewhere (presumably to \"refs verify\"), you\nare removing the checks that _will_ become redundant from \"fsck\".\n\nBut that does not seem to be what is happening. Let me try to\nparaphrase, in order to check my understanding of what you wanted to\nsay:\n\n \"git-fsck(1) has some consistency checks for regular refs. As\n we want to align the checks \"git refs verify\" performs with\n them (and eventually call the unified code that checks refs from\n both), port the logic \"git fsck\" has to \"git refs verify\".\n\nIf we fail to achieve the \"a single unified code to check called by\nboth fsck and refs-verify\" at the end of this series, and instead\nend up with duplicated code that implements the checks in two\nseparate code, risking them to be slightly different and drift away\nover time from each other, that is fine, as long as our intention is\nto continue the effort for unification in a follow up series. \n\nBut such a plan needs to be spelled out.\n\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 890d0324e1..b1ed2e5c04 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3430,6 +3430,48 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> \t\t\t\t const char *refs_check_dir,\n> \t\t\t\t struct dir_iterator *iter);\n> \n> +static int files_fsck_refs_content(struct ref_store *ref_store,\n> +\t\t\t\t struct fsck_options *o,\n> +\t\t\t\t const char *refs_check_dir,\n> +\t\t\t\t struct dir_iterator *iter)\n> +{\n> +\tstruct strbuf ref_content = STRBUF_INIT;\n> +\tstruct strbuf referent = STRBUF_INIT;\n> +\tstruct strbuf refname = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = {0};\n> +\tunsigned int type = 0;\n> +\tint failure_errno = 0;\n> +\tstruct object_id oid;\n> +\tint ret = 0;\n> +\n> +\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n> +\treport.path = refname.buf;\n> +\n> +\tif (S_ISLNK(iter->st.st_mode))\n> +\t\tgoto cleanup;\n\n\"symbolic links are OK\" for now. We'll add sanity checks for them\nin later steps. OK.\n\n> +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> +\t\tret = error_errno(_(\"unable to read ref '%s/%s'\"),\n> +\t\t\t\t refs_check_dir, iter->relative_path);\n\nIs there a reason why we cannot to use report.path aka refname.buf,\nand instead we have to recompute the same path again?\n\nShould this error be propagated back to the caller, not just to the\nend-user, by a call to fsck_report_ref(), like you do for a ref file\nthat has questionable contents? If ref iteration (like for-each-ref)\nclaims there is this ref, and you cannot read its value when you try\nto use it, it is just as bad as having a loose ref file that has\nunusable contents, isn't it?\n\nIt is a separate matter if such a failure mode deserves its own\nerror code (FSCK_MSG_UNREADABLE_REF) or can be rolled into the same\nFSCK_MSG_BAD_REF_CONTENT. I can see arguments for both sides and\noffhand have no strong preference either way.\n\nThanks.\n\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n> +\t\t\t\t ref_content.buf, &oid, &referent,\n> +\t\t\t\t &type, &failure_errno)) {\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> +\t\t\t\t \"invalid ref content\");\n> +\t\tgoto cleanup;\n> +\t}\n> +\n> +cleanup:\n> +\tstrbuf_release(&refname);\n> +\tstrbuf_release(&ref_content);\n> +\tstrbuf_release(&referent);\n> +\treturn ret;\n> +}\n"},{"id":"503016","messageId":"xmqqr09gbvku.fsf@gitster.g","threadId":"61943","inReplyTo":"ZuRzxyjAI3tp4uLK@ArchLinux","subject":"Re: [PATCH v4 3/5] ref: add more strict checks for regular refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-18T19:39:13Z","receivedAt":"2024-09-18T19:39:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> +`refMissingNewline`::\n> +\t(INFO) A ref does not end with newline. This will be\n> +\tconsidered an error in the future.\n\nIt is ONLY files backend's loose-ref representation to store the\nobject name that is the value of the ref as hexadecimal text\nterminated with a newline. With packed backend, even if the file\nends with an incomplete line, it would be confusing to say that such\nlack of terminating LF is associated with a particular ref. With\nreftable backend, the object name may not even be hexadecimal but\nbinary without any terminating LF.\n\nAt least you should say \"A loose ref file that does not end with...\",\nbecause a ref NEVER ends or contains newline, and what you are\nexpecting to be terminated with LF is not even a ref, but the value\nof it.\n\nAlso, isn't it too strong to say \"will be\" without giving any\nfurther information, like:\n\n As valid implementations of Git never created such a loose ref\n file, it may become an error in the future. Report to the\n git@vger.kernel.org mailing list if you see this error, as we\n need to know what tools created such a file.\n\nor something?\n\nThe same comment applies to the next entry.\n\n> @@ -619,6 +619,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n> \t\t*failure_errno = EINVAL;\n> \t\treturn -1;\n> \t}\n> +\n> +\tif (trailing)\n> +\t\t*trailing = p;\n> +\n> \treturn 0;\n\nIn the pre-context of this hunk, if parse_oid_hex_algoph() failed to\nrecognise the initial segment of buf as a valid hexadecimal object\nname, it would have already returned, so we know 'p' is always valid\nhere. It is the byte that comes immediately after the hexadecimal\nobject name.\n\nOK.\n\n> \tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n> \t\t\t\t ref_content.buf, &oid, &referent,\n> -\t\t\t\t &type, &failure_errno)) {\n> +\t\t\t\t &type, &trailing, &failure_errno)) {\n> \t\tret = fsck_report_ref(o, &report,\n> \t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> \t\t\t\t \"invalid ref content\");\n> \t\tgoto cleanup;\n> \t}\n> \n> +\tif (!(type & REF_ISSYMREF)) {\n\nJust like we punted for S_ISLNK() in an earlier step, we for now\ndeal with regular refs in this step. OK.\n\n> +\t\tif (!*trailing) {\n> +\t\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> +\t\t\t\t\t \"missing newline\");\n> +\t\t\tgoto cleanup;\n> +\t\t}\n> +\n> +\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n> +\t\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> +\t\t\t\t\t \"trailing garbage in ref\");\n> +\t\t\tgoto cleanup;\n> +\t\t}\n\nNot limited to this patch, but isn't fsck_report_ref() misdesigned,\nor is it just they are used poorly in these patches? In these two\ncallsites, the message string parameter does not give any more\ninformation than what the FSCK_MSG_* enum gives.\n\nIn fact, MSG_REF_MISSING_NEWLINE at least says that the complaint is\nabout refs, but \"missing newline\" does not even say from what the\nnewline is missing. For TRAILING_REF_CONTENT, people may expect to\nsee what garbage follows the expected contents, but that information\n(i.e. contents of *trailing) is lost here.\n"},{"id":"503018","messageId":"xmqqldzobtq6.fsf@gitster.g","threadId":"61943","inReplyTo":"ZuRzzwZds8ys-JEN@ArchLinux","subject":"Re: [PATCH v4 4/5] ref: add symref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-18T20:19:13Z","receivedAt":"2024-09-18T20:19:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\nExpect that people do not read the body of the message as completing\na paragrpah the title started. I.e. ...\n\n> We have already introduced the checks for regular refs. There is no need\n> to check the consistency of the target which the symref points to.\n> Instead, we just need to check the content of the symref itself.\n\n... this needs a bit of preamble, like\n\n We have code that check regular ref contents, but we do not yet\n check contents of symbolic refs.\n\n> A regular file is accepted as a textual symref if it begins with\n> \"ref:\", followed by zero or more whitespaces, followed by the full\n> refname, followed only by whitespace characters. We always write\n> a single SP after \"ref:\" and a single LF after the refname, but\n> third-party reimplementations of Git may have taken advantage of the\n> looser syntax. Put it more specific, we accept the following contents\n> of the symref:\n>\n> 1. \"ref: refs/heads/master \"\n> 2. \"ref: refs/heads/master \\n \\n\"\n> 3. \"ref: refs/heads/master\\n\\n\"\n>\n> Thus, we could reuse \"refMissingNewline\" and \"trailingRefContent\"\n> FSCK_INFOs to do the same retroactive tightening as we introduce for\n> regular references.\n>\n> But we do not allow any other trailing garbage. The followings are bad\n> symref contents which will be reported as fsck error by \"git-fsck(1)\".\n\nThis description needs to be updated, as it is unclear if you are\ntalking about errors we already detect, or if you are planning to\nupdate fsck to notice and report these errors.\n\n> 1. \"ref: refs/heads/master garbage\\n\"\n> 2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n>\n> And we introduce a new \"badReferentName(ERROR)\" fsck message to report\n> above errors to the user.\n\nOK.\n\n> In order to check the content of the symref, create a function\n> \"files_fsck_symref_target\". It will first check whether the \"referent\"\n> is under the \"refs/\" directory, if not, we will report \"escapeReferent\"\n> fsck error message to notify the user this situation.\n>\n> Then, we will first check whether the symref content misses the newline\n> by peeking the last byte of the \"referent\" to see whether it is '\\n'.\n\n\"Then, we will first\" -> \"Then it checks\" or something.\n\nYou already consumed \"first\" for the check to limit the referent to\nthose under \"refs/\" hierarchy.\n\n> And we will remember the untrimmed length of the \"referent\" and call\n> \"strbuf_rtrim()\" on \"referent\". Then, we will call \"check_refname_format\"\n> to check whether the trimmed referent format is valid. If not, we will\n> report to the user that the symref points to referent which has invalid\n> format. If it is valid, we will compare the untrimmed length and trimmed\n> length, if they are not the same, we need to warn the user there is some\n> trailing garbage in the symref content.\n\nThat is an implementation detail of what you did. But if the\nimplementation were buggy and did not exactly what you intended to\ndo, the above description gives no information to help others to fix\nit up so that it works as you intended it to work, because you do\nnot explain it.\n\nSo what did you want to achieve in the third step (the first being\n\"limit to refs/ hiararchy\", the second being \"no incomplete lines\nallowed\")?\n\n Third, we want to make sure that the contents of a textual\n symref MUST have a single LF after the target refname and\n NOTHING ELSE.\n\nor something.\n\n> At last, we need to check whether the referent is a directory. We cannot\n\n\"a directory\" -> \"an existing directory\"?\n\nI am not comfortable to see the word \"directory\" used in this\nproposed log message, as some refs could be stored in the packed\nbackend and are referenced by the symbolic ref you are inspecting\n(this comment also refers to the \"refs/ directory\" you mentioned\nearlier as \"the first check\").\n\n Lastly, a symbolic ref MUST either point to an existing ref,\n or if the referent does not exist, it MUST NOT be a leading\n subpath for another existing ref (e.g., when \"refs/heads/main\"\n exists, a symbolic ref that points at \"refs/heads\" is a no-no).\n\nor something (but again, I am open to a phrasing better than\n\"subpath\").\n\nDesign question. What do we want to do when we have no loose refs\nunder the \"refs/heads/historical/\" hiearchy, (i.e. all of them are\nin packed-refs file) hence \".git/refs/heads/historical\" directory\ndoes not exist on the filesystem. And a symbolic ref points at\n\"refs/heads/historical\". Shouldn't we give the same error whether\nthe .git/refs/heads/historical directory exist or not, as long as\nthe refs/heads/historical/main branch exists (in the packed-refs\nbackend)?\n\n> diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> index 8827137ef0..03bcb77972 100644\n> --- a/Documentation/fsck-msgids.txt\n> +++ b/Documentation/fsck-msgids.txt\n> @@ -28,6 +28,12 @@\n> `badRefName`::\n> \t(ERROR) A ref has an invalid format.\n> \n> +`badReferentFiletype`::\n> +\t(ERROR) The referent of a symref has a bad file type.\n> +\n> +`badReferentName`::\n> +\t(ERROR) The referent name of a symref is invalid.\n> +\n> `badTagName`::\n> \t(INFO) A tag has an invalid format.\n> \n> @@ -49,6 +55,9 @@\n> `emptyName`::\n> \t(WARN) A path contains an empty name.\n> \n> +`escapeReferent`::\n> +\t(ERROR) The referent of a symref is outside the \"ref\" directory.\n\nI am not sure starting this as ERROR is wise. Users and third-party\ntools make creative uses of the system and I cannot offhand think of\nan argument why it should be forbidden to create a symbolic link to\nour own HEAD or to some worktree-specific ref in another worktree.\n\n> +\tsize_t len = referent->len - 1;\n> +\tstruct stat st;\n> +\tint ret = 0;\n> +\n> +\tif (!starts_with(referent->buf, \"refs/\")) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n> +\t\t\t\t \"points to ref outside the refs directory\");\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tif (referent->buf[referent->len - 1] != '\\n') {\n\nAs you initialized \"len\" to \"referent->len-1\" earlier, wouldn't it\nmore natural to use it here? That would match the incrementing of\nlen++ later in this block.\n\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> +\t\t\t\t \"missing newline\");\n> +\t\tlen++;\n> +\t}\n\nHaving said that, the above should be simplified more like:\n\n * declare but not initialize \"len\". better yet, declare \"orig_len\"\n and leave it uninitialized.\n\n * do not touch \"len++\" in the above block (actually, you can\n discard the above \"if(it does not end with LF)\" block, see\n below).\n\n * instead grab \"referent->len\" in \"len\" (or \"orig_len\") immediately\n before you first modify referent, i.e. before strbuf_rtrim() call.\n\n\torig_len = referent->len;\n\torig_last_byte = referent->buf[orig_len - 1];\n\n> +\tstrbuf_rtrim(referent);\n> +\tif (check_refname_format(referent->buf, 0)) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n> +\t\t\t\t \"points to refname with invalid format\");\n\nSimilar to an earlier step, the message does not give any more\ninformation than the enum. Wouldn't the user who got this error\nwant to learn what referent->buf said and which part of it was bad\nin the same message, instead of having to look it up on their own\nafter fsck finishes?\n\n> +\t\tgoto out;\n> +\t}\n\nAt this point we know check_refname_format() is happy with what is\nleft after rtrimming the referent. There are four cases:\n\n - rtrim() did not trim anything (orig_len == referent->len); the file\n lacked the terminating LF.\n\n - rtrim() trimmed one byte (orig_len - 1 == referent->len) and\n the byte was not LF (orig_last_byte != '\\n'). The file lacked\n the terminating LF.\n\n - rtrim() trimmed exactly one byte (orig_len - 1 == referent->len)\n and the byte was LF (orig_last_byte == '\\n'). There is no error.\n\n - all other cases, i.e., rtrim() trimmed two or more bytes. The\n file had trailing whitespaces after a valid referent that passed\n check_refname_format().\n\nSo in short,\n\n\tif (referent->len == orig_len ||\n\t referent->len == orig_len - 1 && orig_last_byte != '\\n') {\n\t\tFSCK_MSG_REF_MISSING_NEWLINE;\n\t} else if (referent->len < orig_len - 1) {\n\t\tFSCK_MSG_REF_TRAILING_WHITESPACE;\n\t}\n\ncan replace the next block you wrote, and we can also remove the\nearlier \"it is an error if it does not end with '\\n'\", I think.\n\n> +\tif (len != referent->len) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> +\t\t\t\t \"trailing garbage in ref\");\n\nAs check_refname_format() was happy, the difference between orig_len\nand referent->len are only coming from trailing whitespaces, i.e. it\nis not that it had arbitrary garbage. Shouldn't we be more explicit\nabout that?\n\n> +\t/*\n> +\t * Dangling symrefs are common and so we don't report them.\n> +\t */\n> +\tif (lstat(referent_path->buf, &st)) {\n> +\t\tif (errno != ENOENT) {\n> +\t\t\tret = error_errno(_(\"unable to stat '%s'\"),\n> +\t\t\t\t\t referent_path->buf);\n> +\t\t}\n> +\t\tgoto out;\n> +\t}\n> +\n> +\t/*\n> +\t * We cannot distinguish whether \"refs/heads/a\" is a directory or not by\n> +\t * using \"check_refname_format(referent->buf, 0)\". Instead, we need to\n> +\t * check the file type of the target.\n> +\t */\n> +\tif (S_ISDIR(st.st_mode)) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_BAD_REFERENT_FILETYPE,\n> +\t\t\t\t \"points to the directory\");\n> +\t\tgoto out;\n> +\t}\n\nIf referent_path->buf refers to \"refs/heads/historical/\", and all\nthe branches under the hierarchy have been sent to packed-refs,\nthen this check will not trigger.\n\nI wonder if this check is the right thing to enforce in the first\nplace, though.\n\nAs far as the end user is concerned, refs/heads/historical/master\nbranch stil exists, and there is no refs/heads/historical branch, so\nsuch a symbolic ref, for all intents and purposes, is the same as\nany other dangling symbolic refs, no?\n\nOf course, \"git update-ref SUCH_A_SYMREF HEAD\" will complain because\nthere is refs/heads/historical, with something like \n\n \"refs/heads/historical/master\" exists, cannot create \"refs/heads/historical\"\n\nbut that is to be expected. If you remove the last branch in the\nrefs/heads/historical hierarchy, you should be able to do such an\nupdate-ref to instanciate refs/heads/historical as a regular ref.\n\n> @@ -3484,12 +3553,24 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> \t\t\t\t\t \"trailing garbage in ref\");\n> \t\t\tgoto cleanup;\n> \t\t}\n> +\t} else {\n> +\t\tstrbuf_addf(&referent_path, \"%s/%s\",\n> +\t\t\t ref_store->gitdir, referent.buf);\n> +\t\t/*\n> +\t\t * the referent may contain the spaces and the newline, need to\n> +\t\t * trim for path.\n> +\t\t */\n> +\t\tstrbuf_rtrim(&referent_path);\n\nI doubt this is a good design. We have referent, and the symbolic\nref checker knows that the true referent refname may be followed by\nwhitespaces, so instead of inventing referent _path here, it would\nbe a better design to let the files_fsck_symref_target() to decide\nwhat file to open and check based on referent, no? Give it the\nrefstore or refstore's gitdir and have the concatenation with the\nrtrimmed contents in the referent->buf after it inspected it\ninstead, perhaps?\n\n> +\t\tret = files_fsck_symref_target(o, &report,\n> +\t\t\t\t\t &referent,\n> +\t\t\t\t\t &referent_path);\n"},{"id":"503033","messageId":"xmqqo74ka7l7.fsf@gitster.g","threadId":"61943","inReplyTo":"ZuRz1_cHDnr_pWzo@ArchLinux","subject":"Re: [PATCH v4 5/5] ref: add symlink ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-18T23:02:44Z","receivedAt":"2024-09-18T23:02:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> Because we consider deprecating writing the symbolic links and for\n> reading, we may or may not deprecate. We first need to asses whether\n> symbolic links may still be used. So, add a new fsck message\n> \"symlinkRef(INFO)\" to let the user be aware of this information.\n\nIf that is the intention, the the documentation entry is somewhat\nout of line.\n\n> +`symlinkRef`::\n> +\t(INFO) A symref uses the symbolic link. This kind of symref may\n> +\tbe considered ERROR in the future when totally dropping the\n> +\tsymlink support.\n\n A symbolic link is used as a symref. Report to the\n git@vger.kernel.org mailing list if you see this error, as we\n are assessing the feasibility of dropping the support to use\n symbolic links as a symref.\n\nBut quite honestly, I do not think it is necessary to deprecate (let\nalone remove) the support for reading side.\n"},{"id":"503216","messageId":"ZvAwr6uj2WRD7L7y@ArchLinux","threadId":"61943","inReplyTo":"xmqqh6acdbz2.fsf@gitster.g","subject":"Re: [PATCH v4 2/5] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-22T14:58:55Z","receivedAt":"2024-09-22T14:57:44Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Sep 18, 2024 at 11:59:45AM -0700, Junio C Hamano wrote:\n\n[snip]\n\n> The above reads as if you are, in preparation to \"port\" the checks\n> we have in \"fsck\" to elsewhere (presumably to \"refs verify\"), you\n> are removing the checks that _will_ become redundant from \"fsck\".\n> \n> But that does not seem to be what is happening. Let me try to\n> paraphrase, in order to check my understanding of what you wanted to\n> say:\n> \n> \"git-fsck(1) has some consistency checks for regular refs. As\n> we want to align the checks \"git refs verify\" performs with\n> them (and eventually call the unified code that checks refs from\n> both), port the logic \"git fsck\" has to \"git refs verify\".\n> \n\nThanks, I have re-read my words, I did not explain this thing well.\n\n> > +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> > +\t\tret = error_errno(_(\"unable to read ref '%s/%s'\"),\n> > +\t\t\t\t refs_check_dir, iter->relative_path);\n> \n> Is there a reason why we cannot to use report.path aka refname.buf,\n> and instead we have to recompute the same path again?\n> \n\nThanks for pointing out this, because this part I wrote a long time ago\nand I think it's unrelated to the fsck part. So, I forgot to change.\n\n> Should this error be propagated back to the caller, not just to the\n> end-user, by a call to fsck_report_ref(), like you do for a ref file\n> that has questionable contents? If ref iteration (like for-each-ref)\n> claims there is this ref, and you cannot read its value when you try\n> to use it, it is just as bad as having a loose ref file that has\n> unusable contents, isn't it?\n> \n\nI agree. The initial motivation for this design is that I think this is\nOS-specific issue (It may be read successfully in the next time). So, I\ndon't put it into the fsck part. But It make senses that we should\nreport this.\n\n> It is a separate matter if such a failure mode deserves its own\n> error code (FSCK_MSG_UNREADABLE_REF) or can be rolled into the same\n> FSCK_MSG_BAD_REF_CONTENT. I can see arguments for both sides and\n> offhand have no strong preference either way.\n> \n\nWe could just use \"FSCK_MSG_BAD_REF_CONTENT\" and add a message \"cannot\nopen this file\". I guess this should be enough.\n\n"},{"id":"503217","messageId":"ZvAyf8Uy6R33mUda@ArchLinux","threadId":"61943","inReplyTo":"xmqqr09gbvku.fsf@gitster.g","subject":"Re: [PATCH v4 3/5] ref: add more strict checks for regular refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-22T15:06:39Z","receivedAt":"2024-09-22T15:05:28Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Sep 18, 2024 at 12:39:13PM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > +`refMissingNewline`::\n> > +\t(INFO) A ref does not end with newline. This will be\n> > +\tconsidered an error in the future.\n> \n> It is ONLY files backend's loose-ref representation to store the\n> object name that is the value of the ref as hexadecimal text\n> terminated with a newline. With packed backend, even if the file\n> ends with an incomplete line, it would be confusing to say that such\n> lack of terminating LF is associated with a particular ref. With\n> reftable backend, the object name may not even be hexadecimal but\n> binary without any terminating LF.\n> \n> At least you should say \"A loose ref file that does not end with...\",\n> because a ref NEVER ends or contains newline, and what you are\n> expecting to be terminated with LF is not even a ref, but the value\n> of it.\n> \n\nThanks, I will improve this in the next version.\n\n> Also, isn't it too strong to say \"will be\" without giving any\n> further information, like:\n> \n> As valid implementations of Git never created such a loose ref\n> file, it may become an error in the future. Report to the\n> git@vger.kernel.org mailing list if you see this error, as we\n> need to know what tools created such a file.\n> \n> or something?\n> \n\nThis is nice. I know the intention here.\n\n> > +\t\tif (!*trailing) {\n> > +\t\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> > +\t\t\t\t\t \"missing newline\");\n> > +\t\t\tgoto cleanup;\n> > +\t\t}\n> > +\n> > +\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n> > +\t\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> > +\t\t\t\t\t \"trailing garbage in ref\");\n> > +\t\t\tgoto cleanup;\n> > +\t\t}\n> \n> Not limited to this patch, but isn't fsck_report_ref() misdesigned,\n> or is it just they are used poorly in these patches? In these two\n> callsites, the message string parameter does not give any more\n> information than what the FSCK_MSG_* enum gives.\n> \n> In fact, MSG_REF_MISSING_NEWLINE at least says that the complaint is\n> about refs, but \"missing newline\" does not even say from what the\n> newline is missing. For TRAILING_REF_CONTENT, people may expect to\n> see what garbage follows the expected contents, but that information\n> (i.e. contents of *trailing) is lost here.\n\nI agree with you here, I use way too general words to describe what\nhappens. I will improve this. Actually, I feel hard to find words for\n\"MSG_REF_MISSING_NEWLINE\". I think we should say:\n\n\tLF should be at the end of the file.\n\nThanks,\nJialuo\n"},{"id":"503218","messageId":"ZvA9agbGaGnF6nxW@ArchLinux","threadId":"61943","inReplyTo":"xmqqldzobtq6.fsf@gitster.g","subject":"Re: [PATCH v4 4/5] ref: add symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-22T15:53:14Z","receivedAt":"2024-09-22T15:52:03Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Sep 18, 2024 at 01:19:13PM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> Expect that people do not read the body of the message as completing\n> a paragrpah the title started. I.e. ...\n> \n> > We have already introduced the checks for regular refs. There is no need\n> > to check the consistency of the target which the symref points to.\n> > Instead, we just need to check the content of the symref itself.\n> \n> ... this needs a bit of preamble, like\n> \n> We have code that check regular ref contents, but we do not yet\n> check contents of symbolic refs.\n> \n\nThanks, I will improve this in the next version.\n\n> > A regular file is accepted as a textual symref if it begins with\n> > \"ref:\", followed by zero or more whitespaces, followed by the full\n> > refname, followed only by whitespace characters. We always write\n> > a single SP after \"ref:\" and a single LF after the refname, but\n> > third-party reimplementations of Git may have taken advantage of the\n> > looser syntax. Put it more specific, we accept the following contents\n> > of the symref:\n> >\n> > 1. \"ref: refs/heads/master \"\n> > 2. \"ref: refs/heads/master \\n \\n\"\n> > 3. \"ref: refs/heads/master\\n\\n\"\n> >\n> > Thus, we could reuse \"refMissingNewline\" and \"trailingRefContent\"\n> > FSCK_INFOs to do the same retroactive tightening as we introduce for\n> > regular references.\n> >\n> > But we do not allow any other trailing garbage. The followings are bad\n> > symref contents which will be reported as fsck error by \"git-fsck(1)\".\n> \n> This description needs to be updated, as it is unclear if you are\n> talking about errors we already detect, or if you are planning to\n> update fsck to notice and report these errors.\n> \n\nYes, When I was writing this part, I felt a little painful to express my\nwords. I have thought how could I express the connection between the\ncurrent patch and the previous one.\n\n> > And we will remember the untrimmed length of the \"referent\" and call\n> > \"strbuf_rtrim()\" on \"referent\". Then, we will call \"check_refname_format\"\n> > to check whether the trimmed referent format is valid. If not, we will\n> > report to the user that the symref points to referent which has invalid\n> > format. If it is valid, we will compare the untrimmed length and trimmed\n> > length, if they are not the same, we need to warn the user there is some\n> > trailing garbage in the symref content.\n> \n> That is an implementation detail of what you did. But if the\n> implementation were buggy and did not exactly what you intended to\n> do, the above description gives no information to help others to fix\n> it up so that it works as you intended it to work, because you do\n> not explain it.\n> \n> So what did you want to achieve in the third step (the first being\n> \"limit to refs/ hiararchy\", the second being \"no incomplete lines\n> allowed\")?\n> \n> Third, we want to make sure that the contents of a textual\n> symref MUST have a single LF after the target refname and\n> NOTHING ELSE.\n> \n> or something.\n> \n\nFrom the above comments, I need to organize the commit message of\nthis patch to make things clear here.\n\n> \"a directory\" -> \"an existing directory\"?\n> \n> I am not comfortable to see the word \"directory\" used in this\n> proposed log message, as some refs could be stored in the packed\n> backend and are referenced by the symbolic ref you are inspecting\n> (this comment also refers to the \"refs/ directory\" you mentioned\n> earlier as \"the first check\").\n> \n> Lastly, a symbolic ref MUST either point to an existing ref,\n> or if the referent does not exist, it MUST NOT be a leading\n> subpath for another existing ref (e.g., when \"refs/heads/main\"\n> exists, a symbolic ref that points at \"refs/heads\" is a no-no).\n> \n> or something (but again, I am open to a phrasing better than\n> \"subpath\").\n> \n> Design question. What do we want to do when we have no loose refs\n> under the \"refs/heads/historical/\" hiearchy, (i.e. all of them are\n> in packed-refs file) hence \".git/refs/heads/historical\" directory\n> does not exist on the filesystem. And a symbolic ref points at\n> \"refs/heads/historical\". Shouldn't we give the same error whether\n> the .git/refs/heads/historical directory exist or not, as long as\n> the refs/heads/historical/main branch exists (in the packed-refs\n> backend)?\n> \n\nI guess I need to think carefully here. Actually, my intention is that I\nwant to concentrate on the loose refs and then take consideration about\nthe packed refs.\n\nHowever, from what you have said above, it seems I could not do this.\nThey are connected. But at current, I am not so familiar with packed\nrefs behavior, I could not answer all the questions above.\n\nI decide to understand what packed-ref done. So, this series may be\nstalled sometime until I have a good knowledge and re-think the design\nhere.\n\n> > +`escapeReferent`::\n> > +\t(ERROR) The referent of a symref is outside the \"ref\" directory.\n> \n> I am not sure starting this as ERROR is wise. Users and third-party\n> tools make creative uses of the system and I cannot offhand think of\n> an argument why it should be forbidden to create a symbolic link to\n> our own HEAD or to some worktree-specific ref in another worktree.\n> \n\nDo we allow this cross-access (hack)? It might cause some trouble from\nmy perspective.\n\n> > +\tif (referent->buf[referent->len - 1] != '\\n') {\n> \n> As you initialized \"len\" to \"referent->len-1\" earlier, wouldn't it\n> more natural to use it here? That would match the incrementing of\n> len++ later in this block.\n> \n\nYes, exactly.\n\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n> > +\t\t\t\t \"missing newline\");\n> > +\t\tlen++;\n> > +\t}\n> \n> Having said that, the above should be simplified more like:\n> \n> * declare but not initialize \"len\". better yet, declare \"orig_len\"\n> and leave it uninitialized.\n> \n> * do not touch \"len++\" in the above block (actually, you can\n> discard the above \"if(it does not end with LF)\" block, see\n> below).\n> \n> * instead grab \"referent->len\" in \"len\" (or \"orig_len\") immediately\n> before you first modify referent, i.e. before strbuf_rtrim() call.\n> \n> \torig_len = referent->len;\n> \torig_last_byte = referent->buf[orig_len - 1];\n> \n\nI agree.\n\n> > +\tstrbuf_rtrim(referent);\n> > +\tif (check_refname_format(referent->buf, 0)) {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n> > +\t\t\t\t \"points to refname with invalid format\");\n> \n> Similar to an earlier step, the message does not give any more\n> information than the enum. Wouldn't the user who got this error\n> want to learn what referent->buf said and which part of it was bad\n> in the same message, instead of having to look it up on their own\n> after fsck finishes?\n> \n\nYes, I agree. I will improve this.\n\n> > +\t\tgoto out;\n> > +\t}\n> \n> At this point we know check_refname_format() is happy with what is\n> left after rtrimming the referent. There are four cases:\n> \n> - rtrim() did not trim anything (orig_len == referent->len); the file\n> lacked the terminating LF.\n> \n> - rtrim() trimmed one byte (orig_len - 1 == referent->len) and\n> the byte was not LF (orig_last_byte != '\\n'). The file lacked\n> the terminating LF.\n> \n> - rtrim() trimmed exactly one byte (orig_len - 1 == referent->len)\n> and the byte was LF (orig_last_byte == '\\n'). There is no error.\n> \n> - all other cases, i.e., rtrim() trimmed two or more bytes. The\n> file had trailing whitespaces after a valid referent that passed\n> check_refname_format().\n> \n\nThat's so clear. My implementation is not good compared with this.\n\n> So in short,\n> \n> \tif (referent->len == orig_len ||\n> \t referent->len == orig_len - 1 && orig_last_byte != '\\n') {\n> \t\tFSCK_MSG_REF_MISSING_NEWLINE;\n> \t} else if (referent->len < orig_len - 1) {\n> \t\tFSCK_MSG_REF_TRAILING_WHITESPACE;\n> \t}\n> \n> can replace the next block you wrote, and we can also remove the\n> earlier \"it is an error if it does not end with '\\n'\", I think.\n> \n> > +\tif (len != referent->len) {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n> > +\t\t\t\t \"trailing garbage in ref\");\n> \n> As check_refname_format() was happy, the difference between orig_len\n> and referent->len are only coming from trailing whitespaces, i.e. it\n> is not that it had arbitrary garbage. Shouldn't we be more explicit\n> about that?\n> \n\nYes, I made a lot of mistakes when calling the \"fsck_report_ref\". I will\nreport the exact garbage content to the user.\n\n> > +\t/*\n> > +\t * Dangling symrefs are common and so we don't report them.\n> > +\t */\n> > +\tif (lstat(referent_path->buf, &st)) {\n> > +\t\tif (errno != ENOENT) {\n> > +\t\t\tret = error_errno(_(\"unable to stat '%s'\"),\n> > +\t\t\t\t\t referent_path->buf);\n> > +\t\t}\n> > +\t\tgoto out;\n> > +\t}\n> > +\n> > +\t/*\n> > +\t * We cannot distinguish whether \"refs/heads/a\" is a directory or not by\n> > +\t * using \"check_refname_format(referent->buf, 0)\". Instead, we need to\n> > +\t * check the file type of the target.\n> > +\t */\n> > +\tif (S_ISDIR(st.st_mode)) {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_BAD_REFERENT_FILETYPE,\n> > +\t\t\t\t \"points to the directory\");\n> > +\t\tgoto out;\n> > +\t}\n> \n> If referent_path->buf refers to \"refs/heads/historical/\", and all\n> the branches under the hierarchy have been sent to packed-refs,\n> then this check will not trigger.\n> \n\nYes, because \"refs/heads/historical\" will not appear in the filesystem.\n\n> I wonder if this check is the right thing to enforce in the first\n> place, though.\n> \n> As far as the end user is concerned, refs/heads/historical/master\n> branch stil exists, and there is no refs/heads/historical branch, so\n> such a symbolic ref, for all intents and purposes, is the same as\n> any other dangling symbolic refs, no?\n> \n> Of course, \"git update-ref SUCH_A_SYMREF HEAD\" will complain because\n> there is refs/heads/historical, with something like \n> \n> \"refs/heads/historical/master\" exists, cannot create \"refs/heads/historical\"\n> \n> but that is to be expected. If you remove the last branch in the\n> refs/heads/historical hierarchy, you should be able to do such an\n> update-ref to instanciate refs/heads/historical as a regular ref.\n> \n\nI am a little shocked here. I do this in action and find the directory\nwill be automatically converted to a regular file in the filesystem. So,\nI agree with you here. We should never check this, because we allow\nsymref to point to a directory. As long as there is no loose refs and\npacked refs under this directory, we could use \"git update-ref\" for this\nsymref.\n\nThanks,\n\n> > @@ -3484,12 +3553,24 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> > \t\t\t\t\t \"trailing garbage in ref\");\n> > \t\t\tgoto cleanup;\n> > \t\t}\n> > +\t} else {\n> > +\t\tstrbuf_addf(&referent_path, \"%s/%s\",\n> > +\t\t\t ref_store->gitdir, referent.buf);\n> > +\t\t/*\n> > +\t\t * the referent may contain the spaces and the newline, need to\n> > +\t\t * trim for path.\n> > +\t\t */\n> > +\t\tstrbuf_rtrim(&referent_path);\n> \n> I doubt this is a good design. We have referent, and the symbolic\n> ref checker knows that the true referent refname may be followed by\n> whitespaces, so instead of inventing referent _path here, it would\n> be a better design to let the files_fsck_symref_target() to decide\n> what file to open and check based on referent, no? Give it the\n> refstore or refstore's gitdir and have the concatenation with the\n> rtrimmed contents in the referent->buf after it inspected it\n> instead, perhaps?\n> \n\nYes, I agree with you here. We should use \"files_fsck_symref_target\" to\ndo this.\n\n\n----\n\nFrom this review, I think I need to understand more behaviors about\nfiles backend and packed backend. Thanks for your so dedicated reviews.\nI may spend more time to send the next version. And there may be some\ndelay.\n\nThanks,\nJialuo\n"},{"id":"503220","messageId":"xmqqcykv8wiz.fsf@gitster.g","threadId":"61943","inReplyTo":"ZvAyf8Uy6R33mUda@ArchLinux","subject":"Re: [PATCH v4 3/5] ref: add more strict checks for regular refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-22T16:48:20Z","receivedAt":"2024-09-22T16:48:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> I agree with you here, I use way too general words to describe what\n> happens. I will improve this. Actually, I feel hard to find words for\n> \"MSG_REF_MISSING_NEWLINE\". I think we should say:\n>\n> \tLF should be at the end of the file.\n\nGiving a human-readable message when we have an enum can be done at\na lot higher layer with the current way the fsck_report_ref()\nfunction is used (i.e. in that function, not by its callers).\n\nThat is what I meant by \"misdesigned\"---if one message enum always\ncorresponds to one human-readable message, there is not much point\nin forcing callers to supply both, is there?\n"},{"id":"503221","messageId":"xmqq5xqn8w6r.fsf@gitster.g","threadId":"61943","inReplyTo":"ZvA9agbGaGnF6nxW@ArchLinux","subject":"Re: [PATCH v4 4/5] ref: add symref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-22T16:55:40Z","receivedAt":"2024-09-22T16:55:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n>> > +`escapeReferent`::\n>> > +\t(ERROR) The referent of a symref is outside the \"ref\" directory.\n>> \n>> I am not sure starting this as ERROR is wise. Users and third-party\n>> tools make creative uses of the system and I cannot offhand think of\n>> an argument why it should be forbidden to create a symbolic link to\n>> our own HEAD or to some worktree-specific ref in another worktree.\n>> \n> Do we allow this cross-access (hack)? It might cause some trouble from\n> my perspective.\n\nIf the current implementation allows users to set up and take\nadvantage of, then it is not a hack. It would cause breakage\nif we make it an error. Does such a symref successfully refer\nto the referent right now? I think it does.\n\nThanks.\n"},{"id":"503643","messageId":"Zvj-DgHqtC30KjJe@ArchLinux","threadId":"61943","inReplyTo":"ZuRzCyjQFilGhj8j@ArchLinux","subject":"[PATCH v5 0/9] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:13:18Z","receivedAt":"2024-09-29T07:13:21Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis version handles a lot of review from Junio.\n\n1. [PATCH v5 1/9] enhances the commit message compared with the previous\n[PATCH v4 1/5].\n\n2. [PATCH v5 2/9] is a new topic which has not never been introduced in\nthe previous. It supports multiple worktrees check for refs. During the\nGSoC PATCH: , I do not implement\nthe code to support worktree check. However, we need to add this due to\nthe review from Junio:\n > > +`escapeReferent`::\n > > +\t(ERROR) The referent of a symref is outside the \"ref\" directory.\n >\n > I am not sure starting this as ERROR is wise. Users and third-party\n > tools make creative uses of the system and I cannot offhand think of\n > an argument why it should be forbidden to create a symbolic link to\n > our own HEAD or to some worktree-specific ref in another worktree.\nWhen checking the escape situation of the referent, I didn't consider\nthe worktree. So, I decide to first add checks for multiple worktree.\nAnd then add a new test for multiple worktrees.\n\n3. The intention of the [PATCH v5 3/9] is the same as the [PATCH v4\n2/5].\n\n + Enhance the commit message suggested by Junio.\n + Use \"fsck_ref_report\" to tell the user we cannot read the file\n instead of reporting general error.\n + For \"FSCK_MSG_BAD_REF_CONTENT\" message id, instead of just reporting\n the no-information message \"invalid ref content\", report the actual\n content of the ref, i.e., \"ref_content.buf\".\n\n4. The intention of the [PATCH v5 4/9] is the same as the [PATCH v4\n3/5].\n\n + Instead of using the concrete \"refMissingNewline\" and\n \"trailingRefContent\" fsck messages, create a fsck info message\n \"unofficialFormattedRef\"\n + Follow the advice from Junio, use \"fsck_ref_report\" to report more\n useful information. For example, what is the trailing garbage.\n\n5. The PATCH[v4 4/5] is split into 4 commits from [PATCH v5 5/9] to\n[PATCH v5 8/9]. The reason why I decide to do this is that I introduce\nthe check for worktree and the version 4 is a little messy for the\ncommit message. Although the C code is not changed too much, the commit\nmessage is hard to write and make the reviewer confused.\n\n6. [PATCH v5 5/9] aims to add checks for textual symref except escape\nsituation.\n\n + Because I split commit here, it's easy to write the clean commit\n message, which should be changed according to the review from Junio.\n + Followed the advice from Junio to gracefully check the symref. Thus,\n the commit message is more clean.\n + Drop the check for \"referent\" pointing to a directory. We allow\n this, it's a dangling symref. No need to check this. So we could drop\n the parameter \"referent_path\" in \"files_fsck_symref_target()\".\n + Enhance the \"fsck_ref_report\" to report more useful information.\n\n7. [PATCH v5 6/9] enhances the check for escape situation. Introduce a\nnew fsck message \"escapeReferent(INFO)\".\n\n8. [PATCH v5 7/9] enhances the situation where we use multiple\nworktrees. In practice, we allow point to ref of one of the linked\nworktrees from primary worktree or one of the linked worktrees. We\nshould not warn about this.\n\n9. [PATCH v5 8/9] enhances the test script for worktrees.\n\n10. The intention of [PATCH v5 9/9] is the same as the [PATCH v4 5/5].\nNot so much change.\n\nBecause I do not sync the upstream for a long time. For this series, I\nsync the latest upstream and generate the patch, it is based on\n\n 3857aae53f (Git 2.47-rc0, 2024-09-25)\n\nAnd I don't think range-diff is useful, it is messy for the reviewers.\nActually, there are not so many logic changes in this new version.\n\nThanks,\nJialuo\n\nshejialuo (9):\n ref: initialize \"fsck_ref_report\" with zero\n builtin/refs: support multiple worktrees check for refs.\n ref: port git-fsck(1) regular refs check for files backend\n ref: add more strict checks for regular refs\n ref: add basic symref content check for files backend\n ref: add escape check for the referent of symref\n ref: enhance escape situation for worktrees\n t0602: add ref content checks for worktrees\n ref: add symlink ref content check for files backend\n\n Documentation/fsck-msgids.txt | 28 +++\n builtin/refs.c | 11 +-\n fsck.h | 5 +\n refs.c | 2 +-\n refs/files-backend.c | 168 ++++++++++++-\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 442 ++++++++++++++++++++++++++++++++++\n 7 files changed, 646 insertions(+), 12 deletions(-)\n\n-- \n2.46.2\n\n"},{"id":"503644","messageId":"Zvj-hjBXlWr803Us@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v5 1/9] ref: initialize \"fsck_ref_report\" with zero","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:15:18Z","receivedAt":"2024-09-29T07:15:20Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n\"referent\" is NULL. So, we need to always initialize these parameters to\nNULL instead of letting them point to anywhere when creating a new\n\"fsck_ref_report\" structure.\n\nThe original code explicitly initializes the \"path\" member in the\n\"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\nmembers in the struct). It is more customary to use \"{ 0 }\" to express\nthat we are 0-initializing everything. In order to align with the the\ncodebase, initialize \"fsck_ref_report\" with zero.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0824c0b8a9..03d2503276 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3520,7 +3520,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tstruct fsck_ref_report report = { .path = NULL };\n+\t\tstruct fsck_ref_report report = { 0 };\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n-- \n2.46.2\n\n"},{"id":"503645","messageId":"Zvj-jkFE9NN30uDl@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v5 2/9] builtin/refs: support multiple worktrees check for refs.","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:15:26Z","receivedAt":"2024-09-29T07:15:29Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already set up the infrastructure to check the consistency for\nrefs, but we do not support multiple worktrees. As we decide to add more\nchecks for ref content, we need to set up support for multiple\nworktrees. Use \"get_worktrees\" and \"get_worktree_ref_store\" to check\nrefs under the worktrees.\n\nBecause we should only check once for \"packed-refs\", let's call the fsck\nfunction for packed-backend when in the main worktree. In order to know\nwhich directory we check, we should default print this information\ninstead of specifying \"--verbose\".\n\nIt's not suitable to print these information to the stderr. So, change\nto stdout.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 11 ++++++--\n refs/files-backend.c | 18 ++++++++----\n t/t0602-reffiles-fsck.sh | 59 ++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 81 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex 24978a7b7b..3c492ea922 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -5,6 +5,7 @@\n #include \"parse-options.h\"\n #include \"refs.h\"\n #include \"strbuf.h\"\n+#include \"worktree.h\"\n \n #define REFS_MIGRATE_USAGE \\\n \tN_(\"git refs migrate --ref-format= [--dry-run]\")\n@@ -66,6 +67,7 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)\n static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n {\n \tstruct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;\n+\tstruct worktree **worktrees, **p;\n \tconst char * const verify_usage[] = {\n \t\tREFS_VERIFY_USAGE,\n \t\tNULL,\n@@ -75,7 +77,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \t\tOPT_BOOL(0, \"strict\", &fsck_refs_options.strict, N_(\"enable strict checking\")),\n \t\tOPT_END(),\n \t};\n-\tint ret;\n+\tint ret = 0;\n \n \targc = parse_options(argc, argv, prefix, options, verify_usage, 0);\n \tif (argc)\n@@ -84,9 +86,14 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);\n+\tworktrees = get_worktrees();\n+\tfor (p = worktrees; *p; p++) {\n+\t\tstruct worktree *wt = *p;\n+\t\tret += refs_fsck(get_worktree_ref_store(wt), &fsck_refs_options);\n+\t}\n \n \tfsck_options_clear(&fsck_refs_options);\n+\tfree_worktrees(worktrees);\n \treturn ret;\n }\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 03d2503276..57318b4c4e 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3558,7 +3558,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n \t\t\tif (o->verbose)\n-\t\t\t\tfprintf_ln(stderr, \"Checking %s/%s\",\n+\t\t\t\tfprintf_ln(stdout, \"Checking %s/%s\",\n \t\t\t\t\t refs_check_dir, iter->relative_path);\n \t\t\tfor (size_t i = 0; fsck_refs_fn[i]; i++) {\n \t\t\t\tif (fsck_refs_fn[i](ref_store, o, refs_check_dir, iter))\n@@ -3589,8 +3589,8 @@ static int files_fsck_refs(struct ref_store *ref_store,\n \t\tNULL,\n \t};\n \n-\tif (o->verbose)\n-\t\tfprintf_ln(stderr, _(\"Checking references consistency\"));\n+\tfprintf_ln(stdout, _(\"Checking references consistency in %s\"),\n+\t\t ref_store->gitdir);\n \treturn files_fsck_refs_dir(ref_store, o, \"refs\", fsck_refs_fn);\n }\n \n@@ -3600,8 +3600,16 @@ static int files_fsck(struct ref_store *ref_store,\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"fsck\");\n \n-\treturn files_fsck_refs(ref_store, o) |\n-\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n+\tint ret = files_fsck_refs(ref_store, o);\n+\n+\t/*\n+\t * packed-refs should only be checked once because it is shared\n+\t * between all worktrees.\n+\t */\n+\tif (!strcmp(ref_store->gitdir, ref_store->repo->gitdir))\n+\t\tret += refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n+\n+\treturn ret;\n }\n \n struct ref_storage_be refs_be_files = {\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 71a4d1a5ae..4c6cd6f7d0 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -89,4 +89,63 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'ref name check should work for multiple worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\n+\tcd repo &&\n+\ttest_commit initial &&\n+\tgit checkout -b branch-1 &&\n+\ttest_commit second &&\n+\tgit checkout -b branch-2 &&\n+\ttest_commit third &&\n+\tgit checkout -b branch-3 &&\n+\tgit worktree add ./worktree-1 branch-1 &&\n+\tgit worktree add ./worktree-2 branch-2 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\n+\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/.branch-2 &&\n+\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/@ &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/worktree/.branch-2: badRefName: invalid refname format\n+\terror: refs/worktree/@: badRefName: invalid refname format\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/worktree/.branch-2: badRefName: invalid refname format\n+\t\terror: refs/worktree/@: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t) &&\n+\n+\t(\n+\t\tcd worktree-2 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/worktree/.branch-2: badRefName: invalid refname format\n+\t\terror: refs/worktree/@: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n+'\n+\n test_done\n-- \n2.46.2\n\n"},{"id":"503646","messageId":"Zvj-osCNDMrUQv83@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:15:46Z","receivedAt":"2024-09-29T07:15:48Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"git-fsck(1)\" has some consistency checks for regular refs. As we want\nto align the checks \"git refs verify\" performs with them (and eventually\ncall the unified code that checks refs from both), port the logic\n\"git-fsck\" has to \"git refs verify\".\n\n\"git-fsck(1)\" will report an error when the ref content is invalid.\nFollowing this, add a similar check to \"git refs verify\". Then add a new\nfsck error message \"badRefContent(ERROR)\" to represent that a ref has an\ninvalid content.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 ++\n fsck.h | 1 +\n refs/files-backend.c | 45 ++++++++++++++++++++++++\n t/t0602-reffiles-fsck.sh | 66 +++++++++++++++++++++++++++++++++++\n 4 files changed, 115 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 68a2801f15..22c385ea22 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -19,6 +19,9 @@\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n+`badRefContent`::\n+\t(ERROR) A ref has bad content.\n+\n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n \ndiff --git a/fsck.h b/fsck.h\nindex 500b4c04d2..0d99a87911 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -31,6 +31,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 57318b4c4e..35b3fa983e 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3504,6 +3504,50 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_refs_content(struct ref_store *ref_store,\n+\t\t\t\t struct fsck_options *o,\n+\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t struct dir_iterator *iter)\n+{\n+\tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tstruct strbuf refname = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tunsigned int type = 0;\n+\tint failure_errno = 0;\n+\tstruct object_id oid;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n+\treport.path = refname.buf;\n+\n+\tif (S_ISLNK(iter->st.st_mode))\n+\t\tgoto cleanup;\n+\n+\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"cannot read ref file\");\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n+\t\t\t\t ref_content.buf, &oid, &referent,\n+\t\t\t\t &type, &failure_errno)) {\n+\t\tstrbuf_rtrim(&ref_content);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"%s\", ref_content.buf);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&refname);\n+\tstrbuf_release(&ref_content);\n+\tstrbuf_release(&referent);\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n \t\t\t\tconst char *refs_check_dir,\n@@ -3586,6 +3630,7 @@ static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n+\t\tfiles_fsck_refs_content,\n \t\tNULL,\n \t};\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 4c6cd6f7d0..628f9bcc46 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -148,4 +148,70 @@ test_expect_success 'ref name check should work for multiple worktrees' '\n \t)\n '\n \n+test_expect_success 'regular ref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n+\n+\tbad_content=$(git rev-parse main)x &&\n+\tprintf \"%s\" $bad_content >$tag_dir_prefix/tag-bad-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-bad-1: badRefContent: $bad_content\n+\tEOF\n+\trm $tag_dir_prefix/tag-bad-1 &&\n+\ttest_cmp expect err &&\n+\n+\tbad_content=xfsazqfxcadas &&\n+\tprintf \"%s\" $bad_content >$tag_dir_prefix/tag-bad-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-bad-2: badRefContent: $bad_content\n+\tEOF\n+\trm $tag_dir_prefix/tag-bad-2 &&\n+\ttest_cmp expect err &&\n+\n+\tbad_content=Xfsazqfxcadas &&\n+\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-bad &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'regular ref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tbad_content_1=$(git rev-parse main)x &&\n+\tbad_content_2=xfsazqfxcadas &&\n+\tbad_content_3=Xfsazqfxcadas &&\n+\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_done\n-- \n2.46.2\n\n"},{"id":"503647","messageId":"Zvj-sBX-0AFsuFDC@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v5 4/9] ref: add more strict checks for regular refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:16:00Z","receivedAt":"2024-09-29T07:16:02Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already used \"parse_loose_ref_contents\" function to check\nwhether the ref content is valid in files backend. However, by\nusing \"parse_loose_ref_contents\", we allow the ref's content to end with\ngarbage or without a newline.\n\nEven though we never create such loose refs ourselves, we have accepted\nsuch loose refs. So, it is entirely possible that some third-party tools\nmay rely on such loose refs being valid. We should not report an error\nfsck message at current. We should notify the users about such\n\"curiously formatted\" loose refs so that adequate care is taken before\nwe decide to tighten the rules in the future.\n\nAnd it's not suitable either to report a warn fsck message to the user.\nWe don't yet want the \"--strict\" flag that controls this bit to end up\ngenerating errors for such weirdly-formatted reference contents, as we\nfirst want to assess whether this retroactive tightening will cause\nissues for any tools out there. It may cause compatibility issues which\nmay break the repository. So we add the \"unofficialFormattedRef(INFO)\"\nfsck message to represent the situation where the ref format is not\nofficially created by us and notify the users it may become an error in\nthe future.\n\nIt might appear that we can't provide the user with any warnings by\nusing FSCK_INFO. However, in \"fsck.c::fsck_vreport\", we will convert\nFSCK_INFO to FSCK_WARN and we can still warn the user about these\nsituations when using \"git refs verify\" without introducing\ncompatibility issues.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 8 +++++\n fsck.h | 1 +\n refs.c | 2 +-\n refs/files-backend.c | 26 +++++++++++++--\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 59 +++++++++++++++++++++++++++++++++++\n 6 files changed, 93 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 22c385ea22..e310b5bce9 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -179,6 +179,14 @@\n `unknownType`::\n \t(ERROR) Found an unknown object type.\n \n+`unofficialFormattedRef`::\n+\t(INFO) The content of a loose ref file is not in the official\n+\tformat such as not having a LF at the end or having trailing\n+\tgarbage. As valid implementations of Git never created such a\n+\tloose ref file, it may become an error in the future. Report\n+\tto the git@vger.kernel.org mailing list if you see this error,\n+\tas we need to know what tools created such a file.\n+\n `unterminatedHeader`::\n \t(FATAL) Missing end-of-line in the object header.\n \ndiff --git a/fsck.h b/fsck.h\nindex 0d99a87911..7420add5c0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -85,6 +85,7 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(UNOFFICIAL_FORMATTED_REF, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs.c b/refs.c\nindex 5f729ed412..6ba1bb1aa1 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1788,7 +1788,7 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \t}\n \n \tresult = parse_loose_ref_contents(ref_store->repo->hash_algo, content.buf,\n-\t\t\t\t\t oid, referent, type, failure_errno);\n+\t\t\t\t\t oid, referent, type, NULL, failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 35b3fa983e..b2a790c884 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -568,7 +568,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname,\n \tbuf = sb_contents.buf;\n \n \tret = parse_loose_ref_contents(ref_store->repo->hash_algo, buf,\n-\t\t\t\t oid, referent, type, &myerr);\n+\t\t\t\t oid, referent, type, NULL, &myerr);\n \n out:\n \tif (ret && !myerr)\n@@ -605,7 +605,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno)\n+\t\t\t const char **trailing, int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -627,6 +627,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n+\n+\tif (trailing)\n+\t\t*trailing = p;\n+\n \treturn 0;\n }\n \n@@ -3513,6 +3517,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n+\tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n \tstruct object_id oid;\n@@ -3533,7 +3538,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n \t\t\t\t ref_content.buf, &oid, &referent,\n-\t\t\t\t &type, &failure_errno)) {\n+\t\t\t\t &type, &trailing, &failure_errno)) {\n \t\tstrbuf_rtrim(&ref_content);\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n@@ -3541,6 +3546,21 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (!(type & REF_ISSYMREF)) {\n+\t\tif (!*trailing) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n+\t\t\t\t\t \"misses LF at the end\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n+\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n cleanup:\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 2313c830d8..73b05f971b 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -715,7 +715,7 @@ struct ref_store {\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno);\n+\t\t\t const char **trailing, int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 628f9bcc46..2f5c4a1926 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -185,6 +185,61 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n \tEOF\n \trm $branch_dir_prefix/a/b/branch-bad &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: unofficialFormattedRef: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage: unofficialFormattedRef: has trailing garbage: '\\'' garbage'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-1: unofficialFormattedRef: has trailing garbage: '\\''\n+\n+\n+\t'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-2: unofficialFormattedRef: has trailing garbage: '\\''\n+\n+\n+\t garbage'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-3: unofficialFormattedRef: has trailing garbage: '\\'' garbage\n+\ta'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n+\ttest_must_fail git -c fsck.unofficialFormattedRef=error refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-garbage-4: unofficialFormattedRef: has trailing garbage: '\\'' garbage'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-4 &&\n \ttest_cmp expect err\n '\n \n@@ -203,12 +258,16 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n \tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n \tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n \terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n \terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\twarning: refs/heads/branch-garbage: unofficialFormattedRef: has trailing garbage: '\\'' garbage'\\''\n+\twarning: refs/heads/branch-no-newline: unofficialFormattedRef: misses LF at the end\n \tEOF\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err\n-- \n2.46.2\n\n"},{"id":"503648","messageId":"Zvj-vbvQym1R4KJk@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v5 5/9] ref: add basic symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:16:13Z","receivedAt":"2024-09-29T07:16:15Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have code that checks regular ref contents, but we do not yet check\nthe contents of symbolic refs. By using \"parse_loose_ref_content\" for\nsymbolic refs, we will get the information of the \"referent\".\n\nWe do not need to check the \"referent\" by opening the file. This is\nbecause if \"referent\" exists in the file system, we will eventually\ncheck its correctness by inspecting every file in the \"refs\" directory.\nIf the \"referent\" does not exist in the filesystem, this is OK as it is\nseen as the dangling symref.\n\nSo we just need to check the \"referent\" string content. A regular could\nbe accepted as a textual symref if it begins with \"ref:\", followed by\nzero or more whitespaces, followed by the full refname, followed only by\nwhitespace characters. However, we always write a single SP after \"ref:\"\nand a single LF after the refname. It may seem that we should report a\nfsck error message when the \"referent\" does not apply above rules and we\nshould not be so aggressive because third-party reimplementations of Git\nmay have taken advantage of the looser syntax. Put it more specific, we\naccept the following \"referent\":\n\n1. \"ref: refs/heads/master \"\n2. \"ref: refs/heads/master \\n \\n\"\n3. \"ref: refs/heads/master\\n\\n\"\n\nWhen introducing the regular ref content checks, we created a new fsck\nmessage \"unofficialFormattedRef\" which exactly represents above\nsituation. So we will reuse this fsck message to write checks to info\nthe user about these situations.\n\nBut we do not allow any other trailing garbage. The followings are bad\nsymref contents which will be reported as fsck error by \"git-fsck(1)\".\n\n1. \"ref: refs/heads/master garbage\\n\"\n2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n\nAnd we introduce a new \"badReferent(ERROR)\" fsck message to report above\nerrors by using \"ref.c::check_refname_format\". But we cannot just pass\nthe \"referent\" to this function because the \"referent\" might contain\nsome whitespaces which will cause \"check_refname_format\" failing.\n\nIn order to add checks, we will do the following things:\n\n1. Record the untrimmed length \"orig_len\" and untrimmed last byte\n \"orig_last_byte\".\n2. Use \"strbuf_rtrim\" to trim the whitespaces or newlines to make sure\n \"check_refname_format\" won't be failed by them.\n3. Use \"orig_len\" and \"orig_last_byte\" to check whether the \"referent\"\n misses '\\n' at the end or it has trailing whitespaces or newlines.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 ++\n fsck.h | 1 +\n refs/files-backend.c | 40 +++++++++++++++\n t/t0602-reffiles-fsck.sh | 97 +++++++++++++++++++++++++++++++++++\n 4 files changed, 141 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex e310b5bce9..e0e4519334 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,6 +28,9 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n+`badReferent`::\n+\t(ERROR) The referent of a ref is invalid.\n+\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n \ndiff --git a/fsck.h b/fsck.h\nindex 7420add5c0..979d75cb53 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,6 +34,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n+\tFUNC(BAD_REFERENT, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b2a790c884..57ac466b64 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3508,6 +3508,43 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refs_check_dir,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_symref_target(struct fsck_options *o,\n+\t\t\t\t struct fsck_ref_report *report,\n+\t\t\t\t struct strbuf *referent)\n+{\n+\tchar orig_last_byte;\n+\tsize_t orig_len;\n+\tint ret = 0;\n+\n+\torig_len = referent->len;\n+\torig_last_byte = referent->buf[orig_len - 1];\n+\tstrbuf_rtrim(referent);\n+\n+\tif (check_refname_format(referent->buf, 0)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT,\n+\t\t\t\t \"points to invalid refname '%s'\", referent->buf);\n+\t\tgoto out;\n+\t}\n+\n+\n+\tif (referent->len == orig_len ||\n+\t (referent->len < orig_len && orig_last_byte != '\\n')) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n+\t\t\t\t \"misses LF at the end\");\n+\t}\n+\n+\tif (referent->len != orig_len && referent->len != orig_len - 1) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n+\t\t\t\t \"has trailing whitespaces or newlines\");\n+\t}\n+\n+out:\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *refs_check_dir,\n@@ -3559,6 +3596,9 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n \t\t\tgoto cleanup;\n \t\t}\n+\t} else {\n+\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tgoto cleanup;\n \t}\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 2f5c4a1926..718f6abb71 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -273,4 +273,101 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'textual symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-good &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline-1: unofficialFormattedRef: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-1: unofficialFormattedRef: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: unofficialFormattedRef: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-2: unofficialFormattedRef: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-3: unofficialFormattedRef: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-complicated: unofficialFormattedRef: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: unofficialFormattedRef: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badReferent: points to invalid refname '\\''refs/heads/.branch'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-1 &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'textual symref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badReferent: points to invalid refname '\\''refs/heads/.branch'\\''\n+\twarning: refs/heads/a/b/branch-complicated: unofficialFormattedRef: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-complicated: unofficialFormattedRef: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: unofficialFormattedRef: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-1: unofficialFormattedRef: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-2: unofficialFormattedRef: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-3: unofficialFormattedRef: has trailing whitespaces or newlines\n+\twarning: refs/heads/branch-no-newline-1: unofficialFormattedRef: misses LF at the end\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_done\n-- \n2.46.2\n\n"},{"id":"503649","messageId":"Zvj-xaa_j26Auig7@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v5 6/9] ref: add escape check for the referent of symref","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:16:21Z","receivedAt":"2024-09-29T07:16:23Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Ideally, we want to the users use \"git symbolic-ref\" to create symrefs\ninstead of writing raw contents into the filesystem. However, \"git\nsymbolic-ref\" is strict with the refname but not strict with the\nreferent. For example, we can make the \"referent\" located at the\n\"$(gitdir)/logs/aaa\" and manually write the content into this where we\ncan still successfully parse this symref by using \"git rev-parse\".\n\n $ git init repo && cd repo && git commit --allow-empty -mx\n $ git symbolic-ref refs/heads/test logs/aaa\n $ echo $(git rev-parse HEAD) > .git/logs/aaa\n $ git rev-parse test\n\nWe may need to add some restrictions for \"referent\" parameter when using\n\"git symbolic-ref\" to create symrefs because ideally all the\nnonpeudo-refs should be located under the \"refs\" directory and we may\ntighten this in the future.\n\nIn order to tell the user we may tighten the \"escape\" situation, create\na new fsck message \"escapeReferent\" to notify the user that this may\nbecome an error in the future.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 8 ++++++++\n fsck.h | 1 +\n refs/files-backend.c | 7 +++++++\n t/t0602-reffiles-fsck.sh | 18 ++++++++++++++++++\n 4 files changed, 34 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex e0e4519334..223974057d 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -52,6 +52,14 @@\n `emptyName`::\n \t(WARN) A path contains an empty name.\n \n+`escapeReferent`::\n+\t(INFO) The referent of a symref is outside the \"ref\" directory.\n+\tAlthough we allow create a symref pointing to the referent which\n+\tis outside the \"ref\" by using `git symbolic-ref`, we may tighten\n+\tthe rule in the future. Report to the git@vger.kernel.org\n+\tmailing list if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n `extraHeaderEntry`::\n \t(IGNORE) Extra headers found after `tagger`.\n \ndiff --git a/fsck.h b/fsck.h\nindex 979d75cb53..5ecee0fda5 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -80,6 +80,7 @@ enum fsck_msg_type {\n \tFUNC(LARGE_PATHNAME, WARN) \\\n \t/* infos (reported as warnings, but ignored by default) */ \\\n \tFUNC(BAD_FILEMODE, INFO) \\\n+\tFUNC(ESCAPE_REFERENT, INFO) \\\n \tFUNC(GITMODULES_PARSE, INFO) \\\n \tFUNC(GITIGNORE_SYMLINK, INFO) \\\n \tFUNC(GITATTRIBUTES_SYMLINK, INFO) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 57ac466b64..bd215c8d08 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3520,6 +3520,13 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \torig_last_byte = referent->buf[orig_len - 1];\n \tstrbuf_rtrim(referent);\n \n+\tif (!starts_with(referent->buf, \"refs/\")) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n+\t\t\t\t \"referent '%s' is outside of refs/\",\n+\t\t\t\t referent->buf);\n+\t}\n+\n \tif (check_refname_format(referent->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_BAD_REFERENT,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 718f6abb71..585f562245 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -370,4 +370,22 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'textual symref should be checked whether it is escaped' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs-back/heads/main\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-bad-1: escapeReferent: referent '\\''refs-back/heads/main'\\'' is outside of refs/\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-1 &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.46.2\n\n"},{"id":"503650","messageId":"Zvj-7Rx8ZT_27UpE@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v5 7/9] ref: enhance escape situation for worktrees","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:17:01Z","receivedAt":"2024-09-29T07:17:03Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We do allow users to use \"git symbolic-ref\" to create symrefs which\npoint to one of the linked worktrees from the primary worktree or one of\nthe linked worktrees.\n\nWe should not info the user about the escape for above situation. So,\nenhance \"files_fsck_symref_target\" function to check whether the \"referent\"\nstarts with the \"worktrees/\" to make sure that we won't warn the user\nwhen symrefs point to \"referent\" in the linked worktrees.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 5 +++--\n t/t0602-reffiles-fsck.sh | 34 +++++++++++++++++++++++++++++++++-\n 2 files changed, 36 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex bd215c8d08..1182bca108 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3520,10 +3520,11 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \torig_last_byte = referent->buf[orig_len - 1];\n \tstrbuf_rtrim(referent);\n \n-\tif (!starts_with(referent->buf, \"refs/\")) {\n+\tif (!starts_with(referent->buf, \"refs/\") &&\n+\t !starts_with(referent->buf, \"worktrees/\")) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n-\t\t\t\t \"referent '%s' is outside of refs/\",\n+\t\t\t\t \"referent '%s' is outside of refs/ or worktrees/\",\n \t\t\t\t referent->buf);\n \t}\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 585f562245..936448f780 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -382,10 +382,42 @@ test_expect_success 'textual symref should be checked whether it is escaped' '\n \tprintf \"ref: refs-back/heads/main\\n\" >$branch_dir_prefix/branch-bad-1 &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-bad-1: escapeReferent: referent '\\''refs-back/heads/main'\\'' is outside of refs/\n+\twarning: refs/heads/branch-bad-1: escapeReferent: referent '\\''refs-back/heads/main'\\'' is outside of refs/ or worktrees/\n \tEOF\n \trm $branch_dir_prefix/branch-bad-1 &&\n \ttest_cmp expect err\n '\n \n+test_expect_success 'textual symref escape check should work with worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit branch branch-3 &&\n+\tgit worktree add ./worktree-1 branch-2 &&\n+\tgit worktree add ./worktree-2 branch-3 &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit branch refs/worktree/w1-branch &&\n+\t\tgit symbolic-ref refs/worktree/branch-4 refs/heads/branch-1 &&\n+\t\tgit symbolic-ref refs/worktree/branch-5 worktrees/worktree-2/refs/worktree/w2-branch\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit branch refs/worktree/w2-branch &&\n+\t\tgit symbolic-ref refs/worktree/branch-4 refs/heads/branch-1 &&\n+\t\tgit symbolic-ref refs/worktree/branch-5 worktrees/worktree-1/refs/worktree/w1-branch\n+\t) &&\n+\n+\n+\tgit symbolic-ref refs/heads/branch-5 worktrees/worktree-1/refs/worktree/w1-branch &&\n+\tgit symbolic-ref refs/heads/branch-6 worktrees/worktree-2/refs/worktree/w2-branch &&\n+\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err\n+'\n+\n test_done\n-- \n2.46.2\n\n"},{"id":"503651","messageId":"Zvj-_tO_Qtp6EDBy@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v5 8/9] t0602: add ref content checks for worktrees","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:17:18Z","receivedAt":"2024-09-29T07:17:21Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already added content tests, but we don't have tests when there\nare worktrees in the repository. Add a new test to test all the\nfunctionalities we have added for worktrees.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n t/t0602-reffiles-fsck.sh | 66 ++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 66 insertions(+)\n\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 936448f780..97bbcd3f13 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -420,4 +420,70 @@ test_expect_success 'textual symref escape check should work with worktrees' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'all textual symref checks should work with worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit branch branch-3 &&\n+\tgit worktree add ./worktree-1 branch-2 &&\n+\tgit worktree add ./worktree-2 branch-3 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\n+\tbad_content_1=$(git rev-parse HEAD)x &&\n+\tbad_content_2=xfsazqfxcadas &&\n+\tbad_content_3=Xfsazqfxcadas &&\n+\n+\tprintf \"%s\" $bad_content_1 >$worktree1_refdir_prefix/bad-branch-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/worktree/bad-branch-1: badRefContent: $bad_content_1\n+\tEOF\n+\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\" $bad_content_2 >$worktree2_refdir_prefix/bad-branch-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/worktree/bad-branch-2: badRefContent: $bad_content_2\n+\tEOF\n+\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\" $bad_content_3 >$worktree1_refdir_prefix/bad-branch-3 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/worktree/bad-branch-3: badRefContent: $bad_content_3\n+\tEOF\n+\trm $worktree1_refdir_prefix/bad-branch-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/worktree/branch-no-newline: unofficialFormattedRef: misses LF at the end\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree2_refdir_prefix/branch-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/worktree/branch-garbage: unofficialFormattedRef: has trailing garbage: '\\'' garbage'\\''\n+\tEOF\n+\trm $worktree2_refdir_prefix/branch-garbage\n+'\n+\n test_done\n-- \n2.46.2\n\n"},{"id":"503652","messageId":"Zvj_EELQdMsN7j2w@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v5 9/9] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-09-29T07:17:36Z","receivedAt":"2024-09-29T07:17:38Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already introduced \"files_fsck_symref_target\". We should reuse\nthis function to handle the symrefs which use legacy symbolic links. We\nshould not check the trailing garbage for symbolic refs. Add a new\nparameter \"symbolic_link\" to disable some checks which should only be\nexecuted for textual symrefs.\n\nWe firstly use the \"strbuf_add_real_path\" to resolve the symlink and\nget the absolute path as the \"ref_content\" which the symlink ref points\nto. Then we can use the absolute \"abs_gitdir\" of the \"gitdir\" and then\ncombine \"ref_content\" and \"abs_gitdir\" to extract the relative path\n\"referent\". If \"ref_content\" is outside of \"gitdir\", we just use the\n\"ref_content\" as the \"referent\". Thus, we can reuse\n\"files_fsck_symref_target\" function to seamlessly check the symlink\nrefs.\n\nBecause we consider deprecating writing the symbolic links. We first\nneed to asses whether symbolic links may still be used. So, add a new\nfsck message \"symlinkRef(INFO)\" to tell the user be aware of this\ninformation.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 6 +++++\n fsck.h | 1 +\n refs/files-backend.c | 43 ++++++++++++++++++++++++++++-----\n t/t0602-reffiles-fsck.sh | 45 +++++++++++++++++++++++++++++++++++\n 4 files changed, 89 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 223974057d..ffe9d6a2f6 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -184,6 +184,12 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`symlinkRef`::\n+\t(INFO) A symbolic link is used as a symref. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as we\n+\tare assessing the feasibility of dropping the support to drop\n+\tcreating symblinks as symrefs.\n+\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \ndiff --git a/fsck.h b/fsck.h\nindex 5ecee0fda5..f1da5c8a77 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -87,6 +87,7 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(SYMLINK_REF, INFO) \\\n \tFUNC(UNOFFICIAL_FORMATTED_REF, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 1182bca108..5a5327a146 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1,6 +1,7 @@\n #define USE_THE_REPOSITORY_VARIABLE\n \n #include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n #include \"../config.h\"\n #include \"../copy.h\"\n #include \"../environment.h\"\n@@ -3510,15 +3511,18 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n-\t\t\t\t struct strbuf *referent)\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t unsigned int symbolic_link)\n {\n \tchar orig_last_byte;\n \tsize_t orig_len;\n \tint ret = 0;\n \n-\torig_len = referent->len;\n-\torig_last_byte = referent->buf[orig_len - 1];\n-\tstrbuf_rtrim(referent);\n+\tif (!symbolic_link) {\n+\t\torig_len = referent->len;\n+\t\torig_last_byte = referent->buf[orig_len - 1];\n+\t\tstrbuf_rtrim(referent);\n+\t}\n \n \tif (!starts_with(referent->buf, \"refs/\") &&\n \t !starts_with(referent->buf, \"worktrees/\")) {\n@@ -3535,6 +3539,9 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n+\tif (symbolic_link)\n+\t\tgoto out;\n+\n \n \tif (referent->len == orig_len ||\n \t (referent->len < orig_len && orig_last_byte != '\\n')) {\n@@ -3559,6 +3566,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct dir_iterator *iter)\n {\n \tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct strbuf refname = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n@@ -3571,8 +3579,30 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n \treport.path = refname.buf;\n \n-\tif (S_ISLNK(iter->st.st_mode))\n+\tif (S_ISLNK(iter->st.st_mode)) {\n+\t\tconst char* relative_referent_path = NULL;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_SYMLINK_REF,\n+\t\t\t\t \"use deprecated symbolic link for symref\");\n+\n+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n+\t\tstrbuf_normalize_path(&abs_gitdir);\n+\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n+\t\t\tstrbuf_addch(&abs_gitdir, '/');\n+\n+\t\tstrbuf_add_real_path(&ref_content, iter->path.buf);\n+\t\tskip_prefix(ref_content.buf, abs_gitdir.buf,\n+\t\t\t &relative_referent_path);\n+\n+\t\tif (relative_referent_path)\n+\t\t\tstrbuf_addstr(&referent, relative_referent_path);\n+\t\telse\n+\t\t\tstrbuf_addbuf(&referent, &ref_content);\n+\n+\t\tret += files_fsck_symref_target(o, &report, &referent, 1);\n \t\tgoto cleanup;\n+\t}\n \n \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n \t\tret = fsck_report_ref(o, &report,\n@@ -3605,7 +3635,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\tgoto cleanup;\n \t\t}\n \t} else {\n-\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tret = files_fsck_symref_target(o, &report, &referent, 0);\n \t\tgoto cleanup;\n \t}\n \n@@ -3613,6 +3643,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n+\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 97bbcd3f13..be4c064b3c 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -486,4 +486,49 @@ test_expect_success 'all textual symref checks should work with worktrees' '\n \trm $worktree2_refdir_prefix/branch-garbage\n '\n \n+test_expect_success SYMLINKS 'symlink symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic: escapeReferent: referent '\\''logs/branch-escape'\\'' is outside of refs/ or worktrees/\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-bad: badReferent: points to invalid refname '\\''refs/heads/branch space'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-1: badReferent: points to invalid refname '\\''refs/tags/.tag'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.46.2\n\n"},{"id":"503759","messageId":"xmqqikud6kc0.fsf@gitster.g","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"Re: [PATCH v5 0/9] add ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-30T18:57:19Z","receivedAt":"2024-09-30T18:57:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> Because I do not sync the upstream for a long time. For this series, I\n> sync the latest upstream and generate the patch, it is based on\n>\n> 3857aae53f (Git 2.47-rc0, 2024-09-25)\n\nDoes this help to reduce conflicts when merging the topic to say\n'next' or 'seen'? If so, such a rebase and noting it in the cover\nletter message, like you just did, is very much appreciated.\n\nIf not, please don't ;-).\n\n> And I don't think range-diff is useful, it is messy for the reviewers.\n> Actually, there are not so many logic changes in this new version.\n\nOK, so this needs a fresh full review. Thanks.\n"},{"id":"503794","messageId":"ZvtvQBdnDWzVgtYk@ArchLinux","threadId":"61943","inReplyTo":"xmqqikud6kc0.fsf@gitster.g","subject":"Re: [PATCH v5 0/9] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-01T03:40:48Z","receivedAt":"2024-10-01T03:40:48Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Sep 30, 2024 at 11:57:19AM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > Because I do not sync the upstream for a long time. For this series, I\n> > sync the latest upstream and generate the patch, it is based on\n> >\n> > 3857aae53f (Git 2.47-rc0, 2024-09-25)\n> \n> Does this help to reduce conflicts when merging the topic to say\n> 'next' or 'seen'? If so, such a rebase and noting it in the cover\n> letter message, like you just did, is very much appreciated.\n> \n> If not, please don't ;-).\n> \n\nActually, I am sure that there is no conflicts after squashing the\nfollowing two patches.\n\n \n \n\nThe reason why I just sync the upstream is that the build system (such\nas warning about unused parameters) and CIs are all changed.\n\nI will remember this.\n\nThanks,\nJialuo\n"},{"id":"504255","messageId":"ZwOBwxiSZpxJlsfT@pks.im","threadId":"61943","inReplyTo":"Zvj-jkFE9NN30uDl@ArchLinux","subject":"Re: [PATCH v5 2/9] builtin/refs: support multiple worktrees check for refs.","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T06:58:30Z","receivedAt":"2024-10-07T06:58:37Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Sep 29, 2024 at 03:15:26PM +0800, shejialuo wrote:\n> We have already set up the infrastructure to check the consistency for\n> refs, but we do not support multiple worktrees. As we decide to add more\n> checks for ref content, we need to set up support for multiple\n> worktrees. Use \"get_worktrees\" and \"get_worktree_ref_store\" to check\n> refs under the worktrees.\n\nMakes sense.\n\n> Because we should only check once for \"packed-refs\", let's call the fsck\n> function for packed-backend when in the main worktree. In order to know\n> which directory we check, we should default print this information\n> instead of specifying \"--verbose\".\n\nThis change should likely be evicted into its own commit with a bit more\nexplanation.\n\n> It's not suitable to print these information to the stderr. So, change\n> to stdout.\n\nThis one, too. Why exactly is in not suitable to print to stderr?\n\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> builtin/refs.c | 11 ++++++--\n> refs/files-backend.c | 18 ++++++++----\n> t/t0602-reffiles-fsck.sh | 59 ++++++++++++++++++++++++++++++++++++++++\n> 3 files changed, 81 insertions(+), 7 deletions(-)\n> \n> diff --git a/builtin/refs.c b/builtin/refs.c\n> index 24978a7b7b..3c492ea922 100644\n> --- a/builtin/refs.c\n> +++ b/builtin/refs.c\n> @@ -5,6 +5,7 @@\n> #include \"parse-options.h\"\n> #include \"refs.h\"\n> #include \"strbuf.h\"\n> +#include \"worktree.h\"\n> \n> #define REFS_MIGRATE_USAGE \\\n> \tN_(\"git refs migrate --ref-format= [--dry-run]\")\n> @@ -66,6 +67,7 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)\n> static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n> {\n> \tstruct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;\n> +\tstruct worktree **worktrees, **p;\n> \tconst char * const verify_usage[] = {\n> \t\tREFS_VERIFY_USAGE,\n> \t\tNULL,\n> @@ -75,7 +77,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n> \t\tOPT_BOOL(0, \"strict\", &fsck_refs_options.strict, N_(\"enable strict checking\")),\n> \t\tOPT_END(),\n> \t};\n> -\tint ret;\n> +\tint ret = 0;\n> \n> \targc = parse_options(argc, argv, prefix, options, verify_usage, 0);\n> \tif (argc)\n> @@ -84,9 +86,14 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n> \tgit_config(git_fsck_config, &fsck_refs_options);\n> \tprepare_repo_settings(the_repository);\n> \n> -\tret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);\n> +\tworktrees = get_worktrees();\n> +\tfor (p = worktrees; *p; p++) {\n> +\t\tstruct worktree *wt = *p;\n> +\t\tret += refs_fsck(get_worktree_ref_store(wt), &fsck_refs_options);\n> +\t}\n\nI think it is more customary to say `ret |=` instead of `ref +=`.\nOtherwise we could at least in theory wrap around and even land at `ret\n== 0`, even though this is quite unlikely.\n\n> \tfsck_options_clear(&fsck_refs_options);\n> +\tfree_worktrees(worktrees);\n> \treturn ret;\n> }\n> \n[snip]\n> @@ -3600,8 +3600,16 @@ static int files_fsck(struct ref_store *ref_store,\n> \tstruct files_ref_store *refs =\n> \t\tfiles_downcast(ref_store, REF_STORE_READ, \"fsck\");\n> \n> -\treturn files_fsck_refs(ref_store, o) |\n> -\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n> +\tint ret = files_fsck_refs(ref_store, o);\n> +\n> +\t/*\n> +\t * packed-refs should only be checked once because it is shared\n> +\t * between all worktrees.\n> +\t */\n> +\tif (!strcmp(ref_store->gitdir, ref_store->repo->gitdir))\n> +\t\tret += refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n> +\n> +\treturn ret;\n> }\n> \n> struct ref_storage_be refs_be_files = {\n\nWhat is the current behaviour? Is it that we verify the packed-refs file\nmultiple times, or rather that we call `packed_ref_store->be->fsck()`\nmany times even though we know it won't do anything for anything except\nfor the main worktree?\n\nIf it is the former I very much agree that we should make this\nconditional. If it's the latter I'm more in the camp of letting it be\nsuch that if worktrees were to ever gain support for \"packed-refs\" we\nwouldn't have to change anything.\n\nIn any case, as proposed I think it would make sense to evict this into\na standalone commit such that these details can be explained in the\ncommit message.\n\nPatrick\n"},{"id":"504254","messageId":"ZwOGmoX5ner_F3Ac@pks.im","threadId":"61943","inReplyTo":"Zvj-osCNDMrUQv83@ArchLinux","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T06:58:34Z","receivedAt":"2024-10-07T06:58:38Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Sep 29, 2024 at 03:15:46PM +0800, shejialuo wrote:\n> \"git-fsck(1)\" has some consistency checks for regular refs. As we want\n> to align the checks \"git refs verify\" performs with them (and eventually\n> call the unified code that checks refs from both), port the logic\n> \"git-fsck\" has to \"git refs verify\".\n\nWhat's missing here is the actual intent of this commit, namely why we\nwant to align the checks. I assume that this prepares us for calling\n`git refs verify` as part of git-fsck(1), but readers not familiar with\nthe larger picture may be left wondering.\n\n> \"git-fsck(1)\" will report an error when the ref content is invalid.\n> Following this, add a similar check to \"git refs verify\". Then add a new\n> fsck error message \"badRefContent(ERROR)\" to represent that a ref has an\n> invalid content.\n\nIt would help readers to know where the code is that you're porting over\nto `git refs verify` so that one can double check that the port is done\nfaithfully to the original.\n\nPatrick\n"},{"id":"504256","messageId":"ZwOGnQSqmwALK-9z@pks.im","threadId":"61943","inReplyTo":"Zvj-sBX-0AFsuFDC@ArchLinux","subject":"Re: [PATCH v5 4/9] ref: add more strict checks for regular refs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T06:58:37Z","receivedAt":"2024-10-07T06:58:41Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Sep 29, 2024 at 03:16:00PM +0800, shejialuo wrote:\n> diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> index 22c385ea22..e310b5bce9 100644\n> --- a/Documentation/fsck-msgids.txt\n> +++ b/Documentation/fsck-msgids.txt\n> @@ -179,6 +179,14 @@\n> `unknownType`::\n> \t(ERROR) Found an unknown object type.\n> \n> +`unofficialFormattedRef`::\n> +\t(INFO) The content of a loose ref file is not in the official\n> +\tformat such as not having a LF at the end or having trailing\n> +\tgarbage. As valid implementations of Git never created such a\n> +\tloose ref file, it may become an error in the future. Report\n> +\tto the git@vger.kernel.org mailing list if you see this error,\n> +\tas we need to know what tools created such a file.\n> +\n\nI find \"unofficial\" to be a tad weird. Do we rather want to say\nsomething like \"badRefTrailingGarbage\"?\n\n> @@ -3541,6 +3546,21 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> \t\tgoto cleanup;\n> \t}\n> \n> +\tif (!(type & REF_ISSYMREF)) {\n> +\t\tif (!*trailing) {\n> +\t\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n> +\t\t\t\t\t \"misses LF at the end\");\n> +\t\t\tgoto cleanup;\n> +\t\t}\n> +\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n> +\t\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n> +\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n> +\t\t\tgoto cleanup;\n> +\t\t}\n> +\t}\n> +\n\nI think we should discern these two error cases and provide different\nmessage IDs.\n\nPatrick\n"},{"id":"504257","messageId":"ZwOGoOyW6HGuneMG@pks.im","threadId":"61943","inReplyTo":"Zvj-7Rx8ZT_27UpE@ArchLinux","subject":"Re: [PATCH v5 7/9] ref: enhance escape situation for worktrees","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T06:58:40Z","receivedAt":"2024-10-07T06:58:44Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Sep 29, 2024 at 03:17:01PM +0800, shejialuo wrote:\n> We do allow users to use \"git symbolic-ref\" to create symrefs which\n> point to one of the linked worktrees from the primary worktree or one of\n> the linked worktrees.\n> \n> We should not info the user about the escape for above situation. So,\n> enhance \"files_fsck_symref_target\" function to check whether the \"referent\"\n> starts with the \"worktrees/\" to make sure that we won't warn the user\n> when symrefs point to \"referent\" in the linked worktrees.\n\nShouldn't this commit be squashed into the former one, as it immediately\nfixes an edge case that was introduced with the parent commit?\n\nPatrick\n"},{"id":"504258","messageId":"ZwOGoxXmn_J6pgh1@pks.im","threadId":"61943","inReplyTo":"Zvj-_tO_Qtp6EDBy@ArchLinux","subject":"Re: [PATCH v5 8/9] t0602: add ref content checks for worktrees","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T06:58:43Z","receivedAt":"2024-10-07T06:58:51Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Sep 29, 2024 at 03:17:18PM +0800, shejialuo wrote:\n> We have already added content tests, but we don't have tests when there\n> are worktrees in the repository. Add a new test to test all the\n> functionalities we have added for worktrees.\n\nI'd squash this commit into the one where you introduced checks for\nworktrees. Or if this exercises errors that you have added in subsequent\ncommits I'd squash it into the respective commit that introduces those\nchecks.\n\nPatrick\n"},{"id":"504259","messageId":"ZwOGqpeYiKITgsoV@pks.im","threadId":"61943","inReplyTo":"Zvj_EELQdMsN7j2w@ArchLinux","subject":"Re: [PATCH v5 9/9] ref: add symlink ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T06:58:50Z","receivedAt":"2024-10-07T06:58:55Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Sep 29, 2024 at 03:17:36PM +0800, shejialuo wrote:\n> We have already introduced \"files_fsck_symref_target\". We should reuse\n> this function to handle the symrefs which use legacy symbolic links. We\n> should not check the trailing garbage for symbolic refs. Add a new\n> parameter \"symbolic_link\" to disable some checks which should only be\n> executed for textual symrefs.\n\nYou're getting into implementation details before noting what the actual\nproblem is. So I'd recommend first describing the problem at a higher\nlevel, and then note that we can reuse parts of preexisting infra to\naddress the issue.\n\nPatrick\n"},{"id":"504260","messageId":"ZwOGr4Tv8K_wemtD@pks.im","threadId":"61943","inReplyTo":"Zvj-xaa_j26Auig7@ArchLinux","subject":"Re: [PATCH v5 6/9] ref: add escape check for the referent of symref","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T06:58:55Z","receivedAt":"2024-10-07T06:58:58Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Sep 29, 2024 at 03:16:21PM +0800, shejialuo wrote:\n> Ideally, we want to the users use \"git symbolic-ref\" to create symrefs\n> instead of writing raw contents into the filesystem. However, \"git\n> symbolic-ref\" is strict with the refname but not strict with the\n> referent. For example, we can make the \"referent\" located at the\n> \"$(gitdir)/logs/aaa\" and manually write the content into this where we\n> can still successfully parse this symref by using \"git rev-parse\".\n> \n> $ git init repo && cd repo && git commit --allow-empty -mx\n> $ git symbolic-ref refs/heads/test logs/aaa\n> $ echo $(git rev-parse HEAD) > .git/logs/aaa\n> $ git rev-parse test\n\nOh, curious. This should definitely be tightened in git-symbolic-ref(1)\nitself. The target should either be a root ref or something starting\nwith \"refs/\". Anyway, that is of course outside of the scope of this\npatch series.\n\n> We may need to add some restrictions for \"referent\" parameter when using\n> \"git symbolic-ref\" to create symrefs because ideally all the\n> nonpeudo-refs should be located under the \"refs\" directory and we may\n> tighten this in the future.\n\nAgreed.\n\n> In order to tell the user we may tighten the \"escape\" situation, create\n> a new fsck message \"escapeReferent\" to notify the user that this may\n> become an error in the future.\n> \n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> Documentation/fsck-msgids.txt | 8 ++++++++\n> fsck.h | 1 +\n> refs/files-backend.c | 7 +++++++\n> t/t0602-reffiles-fsck.sh | 18 ++++++++++++++++++\n> 4 files changed, 34 insertions(+)\n> \n> diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> index e0e4519334..223974057d 100644\n> --- a/Documentation/fsck-msgids.txt\n> +++ b/Documentation/fsck-msgids.txt\n> @@ -52,6 +52,14 @@\n> `emptyName`::\n> \t(WARN) A path contains an empty name.\n> \n> +`escapeReferent`::\n> +\t(INFO) The referent of a symref is outside the \"ref\" directory.\n\nProposal: 'The referent of a symbolic reference points neither to a root\nreference nor to a reference starting with \"refs/\".'\n\nI'd also rename this to e.g. \"symrefTargetIsNotAReference\" or something\nlike that, because it's not really about whether or not the referent is\n\"escaping\". It's a bit of a mouthful, but I don't really have a better\nname. So feel free to pick something different that describes the error\nbetter.\n\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 57ac466b64..bd215c8d08 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3520,6 +3520,13 @@ static int files_fsck_symref_target(struct fsck_options *o,\n> \torig_last_byte = referent->buf[orig_len - 1];\n> \tstrbuf_rtrim(referent);\n> \n> +\tif (!starts_with(referent->buf, \"refs/\")) {\n> +\t\tret = fsck_report_ref(o, report,\n> +\t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n> +\t\t\t\t \"referent '%s' is outside of refs/\",\n> +\t\t\t\t referent->buf);\n> +\t}\n> +\n> \tif (check_refname_format(referent->buf, 0)) {\n> \t\tret = fsck_report_ref(o, report,\n> \t\t\t\t FSCK_MSG_BAD_REFERENT,\n\nThis check is invalid, because referents can also point to root refs. So\nyou should probably also add a call to `is_root_ref()` here.\n\nWe also have `is_pseudo_ref()`, and one might be tempted to also allow\nthat. But pseudo refs aren't proper refs, so I'd argue that a symref\npointing to a pseudo ref is invalid, too.\n\nPatrick\n"},{"id":"504270","messageId":"ZwOe7YVWmhshRhI9@ArchLinux","threadId":"61943","inReplyTo":"ZwOBwxiSZpxJlsfT@pks.im","subject":"Re: [PATCH v5 2/9] builtin/refs: support multiple worktrees check for refs.","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T08:42:21Z","receivedAt":"2024-10-07T08:42:16Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 08:58:30AM +0200, Patrick Steinhardt wrote:\n> On Sun, Sep 29, 2024 at 03:15:26PM +0800, shejialuo wrote:\n> > We have already set up the infrastructure to check the consistency for\n> > refs, but we do not support multiple worktrees. As we decide to add more\n> > checks for ref content, we need to set up support for multiple\n> > worktrees. Use \"get_worktrees\" and \"get_worktree_ref_store\" to check\n> > refs under the worktrees.\n> \n> Makes sense.\n> \n> > Because we should only check once for \"packed-refs\", let's call the fsck\n> > function for packed-backend when in the main worktree. In order to know\n> > which directory we check, we should default print this information\n> > instead of specifying \"--verbose\".\n> \n> This change should likely be evicted into its own commit with a bit more\n> explanation.\n> \n> > It's not suitable to print these information to the stderr. So, change\n> > to stdout.\n> \n> This one, too. Why exactly is in not suitable to print to stderr?\n> \n\nI am sorry for the confusion. We should not print which directory we\ncheck here into stderr. Because I think this will make test script\ncontain many unrelated info when using \"git refs verify 2>err\".\n\nThe reason here is when checking the consistency of refs in multiple\nworktrees. The ref name could be repeat. For example, worktree A\nhas its own ref called \"test\" under \".git/worktrees/A/refs/worktree/test\"\nand worktree B has its own ref still called \"test\" under\n\".git/worktrees/B/refs/worktree/test\".\n\nHowever, the refname would be printed to \"refs/worktree/test\". It will\nmake the user confused which \"refs/worktree/test\" is checked. So, we\nshould print this information like:\n\n Checking references consistency in .git\n ...\n checking references consistency in .git/worktrees/A\n ...\n checking references consistency in .git/worktrees/B\n\nHowever, when writing this, I feel a \".git\" is a bad usage. It will make\nthe user think it will check everything here. This should be improved in\nthe next version.\n\n> > @@ -75,7 +77,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n> > \t\tOPT_BOOL(0, \"strict\", &fsck_refs_options.strict, N_(\"enable strict checking\")),\n> > \t\tOPT_END(),\n> > \t};\n> > -\tint ret;\n> > +\tint ret = 0;\n> > \n> > \targc = parse_options(argc, argv, prefix, options, verify_usage, 0);\n> > \tif (argc)\n> > @@ -84,9 +86,14 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n> > \tgit_config(git_fsck_config, &fsck_refs_options);\n> > \tprepare_repo_settings(the_repository);\n> > \n> > -\tret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);\n> > +\tworktrees = get_worktrees();\n> > +\tfor (p = worktrees; *p; p++) {\n> > +\t\tstruct worktree *wt = *p;\n> > +\t\tret += refs_fsck(get_worktree_ref_store(wt), &fsck_refs_options);\n> > +\t}\n> \n> I think it is more customary to say `ret |=` instead of `ref +=`.\n> Otherwise we could at least in theory wrap around and even land at `ret\n> == 0`, even though this is quite unlikely.\n> \n\nI agree here. I will improve this in the next version.\n\n[snip]\n\n> > @@ -3600,8 +3600,16 @@ static int files_fsck(struct ref_store *ref_store,\n> > \tstruct files_ref_store *refs =\n> > \t\tfiles_downcast(ref_store, REF_STORE_READ, \"fsck\");\n> > \n> > -\treturn files_fsck_refs(ref_store, o) |\n> > -\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n> > +\tint ret = files_fsck_refs(ref_store, o);\n> > +\n> > +\t/*\n> > +\t * packed-refs should only be checked once because it is shared\n> > +\t * between all worktrees.\n> > +\t */\n> > +\tif (!strcmp(ref_store->gitdir, ref_store->repo->gitdir))\n> > +\t\tret += refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n> > +\n> > +\treturn ret;\n> > }\n> > \n> > struct ref_storage_be refs_be_files = {\n> \n> What is the current behaviour? Is it that we verify the packed-refs file\n> multiple times, or rather that we call `packed_ref_store->be->fsck()`\n> many times even though we know it won't do anything for anything except\n> for the main worktree?\n> \n\nThat's a good question. I think the second is the current behaviour. We\nwill call `packed_ref_store->be->fsck()` many times. I understand what\nyou mean here, we just put the check into `packed_ref_store->be->fsck()`\nfunction.\n\n> If it is the former I very much agree that we should make this\n> conditional. If it's the latter I'm more in the camp of letting it be\n> such that if worktrees were to ever gain support for \"packed-refs\" we\n> wouldn't have to change anything.\n> \n\nI agree.\n\n> In any case, as proposed I think it would make sense to evict this into\n> a standalone commit such that these details can be explained in the\n> commit message.\n> \n\nYes, the current commit message lacks of details.\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"504271","messageId":"ZwOfBCSTO4Qvtos3@ArchLinux","threadId":"61943","inReplyTo":"ZwOGmoX5ner_F3Ac@pks.im","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T08:42:44Z","receivedAt":"2024-10-07T08:42:38Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 08:58:34AM +0200, Patrick Steinhardt wrote:\n> On Sun, Sep 29, 2024 at 03:15:46PM +0800, shejialuo wrote:\n> > \"git-fsck(1)\" has some consistency checks for regular refs. As we want\n> > to align the checks \"git refs verify\" performs with them (and eventually\n> > call the unified code that checks refs from both), port the logic\n> > \"git-fsck\" has to \"git refs verify\".\n> \n> What's missing here is the actual intent of this commit, namely why we\n> want to align the checks. I assume that this prepares us for calling\n> `git refs verify` as part of git-fsck(1), but readers not familiar with\n> the larger picture may be left wondering.\n> \n\nIndeed, I will improve this in the next version.\n\n> > \"git-fsck(1)\" will report an error when the ref content is invalid.\n> > Following this, add a similar check to \"git refs verify\". Then add a new\n> > fsck error message \"badRefContent(ERROR)\" to represent that a ref has an\n> > invalid content.\n> \n> It would help readers to know where the code is that you're porting over\n> to `git refs verify` so that one can double check that the port is done\n> faithfully to the original.\n> \n\nI am a little confused here. There are too many codes in \"git-fsck(1)\"\nto check the ref consistency. How could I accurately express this info\nin the commit message?\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"504272","messageId":"ZwOfYGi21oa302sS@ArchLinux","threadId":"61943","inReplyTo":"ZwOGnQSqmwALK-9z@pks.im","subject":"Re: [PATCH v5 4/9] ref: add more strict checks for regular refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T08:44:16Z","receivedAt":"2024-10-07T08:44:10Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 08:58:37AM +0200, Patrick Steinhardt wrote:\n> On Sun, Sep 29, 2024 at 03:16:00PM +0800, shejialuo wrote:\n> > diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> > index 22c385ea22..e310b5bce9 100644\n> > --- a/Documentation/fsck-msgids.txt\n> > +++ b/Documentation/fsck-msgids.txt\n> > @@ -179,6 +179,14 @@\n> > `unknownType`::\n> > \t(ERROR) Found an unknown object type.\n> > \n> > +`unofficialFormattedRef`::\n> > +\t(INFO) The content of a loose ref file is not in the official\n> > +\tformat such as not having a LF at the end or having trailing\n> > +\tgarbage. As valid implementations of Git never created such a\n> > +\tloose ref file, it may become an error in the future. Report\n> > +\tto the git@vger.kernel.org mailing list if you see this error,\n> > +\tas we need to know what tools created such a file.\n> > +\n> \n> I find \"unofficial\" to be a tad weird. Do we rather want to say\n> something like \"badRefTrailingGarbage\"?\n> \n\nWell, I will answer this question just in below question together.\n\n> > @@ -3541,6 +3546,21 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> > \t\tgoto cleanup;\n> > \t}\n> > \n> > +\tif (!(type & REF_ISSYMREF)) {\n> > +\t\tif (!*trailing) {\n> > +\t\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n> > +\t\t\t\t\t \"misses LF at the end\");\n> > +\t\t\tgoto cleanup;\n> > +\t\t}\n> > +\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n> > +\t\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n> > +\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n> > +\t\t\tgoto cleanup;\n> > +\t\t}\n> > +\t}\n> > +\n> \n> I think we should discern these two error cases and provide different\n> message IDs.\n> \n\nActually, in the previous versions, I have mapped one message id to one\nerror case. But, in the v4, Junio asked a question\n\n Not limited to this patch, but isn't fsck_report_ref() misdesigned,\n or is it just they are used poorly in these patches? In these two\n callsites, the message string parameter does not give any more\n information than what the FSCK_MSG_* enum gives.\n\n That is what I meant by \"misdesigned\"---if one message enum always\n corresponds to one human-readable message, there is not much point\n in forcing callers to supply both, is there?\n\nIn my opinion, we should have only one case here for trailing garbage\nand not end with a newline. When writing the code, I chose the name\n\"unofficialFormattedRef\" for the following reason:\n\n 1. If we use two message ids here, for every message id, we need write\n to info the user \"please report this to git mailing list\".\n\n 2. If we decide to make this as an error. We could just classify them\n into \"badRefContent\" message category.\n\n 3. The semantic is correct here, they are truly curious formatted\n refs, and eventually we will give the info to the user what is\n curious.\n\nSo, I think we should not always map one message to one error case.\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"504273","messageId":"ZwOffN9UWX1gP0gy@ArchLinux","threadId":"61943","inReplyTo":"ZwOGr4Tv8K_wemtD@pks.im","subject":"Re: [PATCH v5 6/9] ref: add escape check for the referent of symref","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T08:44:44Z","receivedAt":"2024-10-07T08:44:38Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 08:58:55AM +0200, Patrick Steinhardt wrote:\n> On Sun, Sep 29, 2024 at 03:16:21PM +0800, shejialuo wrote:\n> > Ideally, we want to the users use \"git symbolic-ref\" to create symrefs\n> > instead of writing raw contents into the filesystem. However, \"git\n> > symbolic-ref\" is strict with the refname but not strict with the\n> > referent. For example, we can make the \"referent\" located at the\n> > \"$(gitdir)/logs/aaa\" and manually write the content into this where we\n> > can still successfully parse this symref by using \"git rev-parse\".\n> > \n> > $ git init repo && cd repo && git commit --allow-empty -mx\n> > $ git symbolic-ref refs/heads/test logs/aaa\n> > $ echo $(git rev-parse HEAD) > .git/logs/aaa\n> > $ git rev-parse test\n> \n> Oh, curious. This should definitely be tightened in git-symbolic-ref(1)\n> itself. The target should either be a root ref or something starting\n> with \"refs/\". Anyway, that is of course outside of the scope of this\n> patch series.\n> \n\nI am curious here too when I did experiments when writing the code.\nBecause Junio have told me this could happen, so I dive into this.\nHowever, it's not reasonable. If we want to tighten the rule, we need to\nalso let \"git symbolic-ref\" to align with the behavior. That's another\nquestion though.\n\n[snip]\n\n> > diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> > index e0e4519334..223974057d 100644\n> > --- a/Documentation/fsck-msgids.txt\n> > +++ b/Documentation/fsck-msgids.txt\n> > @@ -52,6 +52,14 @@\n> > `emptyName`::\n> > \t(WARN) A path contains an empty name.\n> > \n> > +`escapeReferent`::\n> > +\t(INFO) The referent of a symref is outside the \"ref\" directory.\n> \n> Proposal: 'The referent of a symbolic reference points neither to a root\n> reference nor to a reference starting with \"refs/\".'\n> \n\nThat's much better.\n\n> I'd also rename this to e.g. \"symrefTargetIsNotAReference\" or something\n> like that, because it's not really about whether or not the referent is\n> \"escaping\". It's a bit of a mouthful, but I don't really have a better\n> name. So feel free to pick something different that describes the error\n> better.\n> \n\nI guess \"symrefTargetIsNotAReference\" is a little too long. If we decide\nto convert it to error later. Why not just put it into the \"badReferent\"\nfsck message?\n\nSo, I do not think we need to rename. As I have talked about, we don't\nneed to map error case to fsck message id one by one.\n\n> > diff --git a/refs/files-backend.c b/refs/files-backend.c\n> > index 57ac466b64..bd215c8d08 100644\n> > --- a/refs/files-backend.c\n> > +++ b/refs/files-backend.c\n> > @@ -3520,6 +3520,13 @@ static int files_fsck_symref_target(struct fsck_options *o,\n> > \torig_last_byte = referent->buf[orig_len - 1];\n> > \tstrbuf_rtrim(referent);\n> > \n> > +\tif (!starts_with(referent->buf, \"refs/\")) {\n> > +\t\tret = fsck_report_ref(o, report,\n> > +\t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n> > +\t\t\t\t \"referent '%s' is outside of refs/\",\n> > +\t\t\t\t referent->buf);\n> > +\t}\n> > +\n> > \tif (check_refname_format(referent->buf, 0)) {\n> > \t\tret = fsck_report_ref(o, report,\n> > \t\t\t\t FSCK_MSG_BAD_REFERENT,\n> \n> This check is invalid, because referents can also point to root refs. So\n> you should probably also add a call to `is_root_ref()` here.\n> \n\nThanks, I omit this situation here.\n\n> We also have `is_pseudo_ref()`, and one might be tempted to also allow\n> that. But pseudo refs aren't proper refs, so I'd argue that a symref\n> pointing to a pseudo ref is invalid, too.\n> \n\nI agree.\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"504274","messageId":"ZwOfpr7g9ZJ0Xub6@ArchLinux","threadId":"61943","inReplyTo":"ZwOGoOyW6HGuneMG@pks.im","subject":"Re: [PATCH v5 7/9] ref: enhance escape situation for worktrees","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T08:45:26Z","receivedAt":"2024-10-07T08:45:20Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 08:58:40AM +0200, Patrick Steinhardt wrote:\n> On Sun, Sep 29, 2024 at 03:17:01PM +0800, shejialuo wrote:\n> > We do allow users to use \"git symbolic-ref\" to create symrefs which\n> > point to one of the linked worktrees from the primary worktree or one of\n> > the linked worktrees.\n> > \n> > We should not info the user about the escape for above situation. So,\n> > enhance \"files_fsck_symref_target\" function to check whether the \"referent\"\n> > starts with the \"worktrees/\" to make sure that we won't warn the user\n> > when symrefs point to \"referent\" in the linked worktrees.\n> \n> Shouldn't this commit be squashed into the former one, as it immediately\n> fixes an edge case that was introduced with the parent commit?\n> \n\nI partially agree here. I don't think this is an edge case that was\nintroduced with the parent commit. The reason why I use a new commit\nhere is that I want to emphasis the behavior.\n\nThis is because Junio asked me in the v4 about \"escapeReferent\"\n\n I am not sure starting this as ERROR is wise. Users and third-party\n tools make creative uses of the system and I cannot offhand think of\n an argument why it should be forbidden to create a symbolic link to\n our own HEAD or to some worktree-specific ref in another worktree.\n\nActually, I have never thought we could do this. So, this is my\nintention. But I do agree that this commit is highly relevant with the\nparent commit.\n\nI will improve this in the next version.\n\n> Patrick\n"},{"id":"504275","messageId":"ZwOftTrlYZh9Oz63@ArchLinux","threadId":"61943","inReplyTo":"ZwOGoxXmn_J6pgh1@pks.im","subject":"Re: [PATCH v5 8/9] t0602: add ref content checks for worktrees","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T08:45:41Z","receivedAt":"2024-10-07T08:45:34Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 08:58:43AM +0200, Patrick Steinhardt wrote:\n> On Sun, Sep 29, 2024 at 03:17:18PM +0800, shejialuo wrote:\n> > We have already added content tests, but we don't have tests when there\n> > are worktrees in the repository. Add a new test to test all the\n> > functionalities we have added for worktrees.\n> \n> I'd squash this commit into the one where you introduced checks for\n> worktrees. Or if this exercises errors that you have added in subsequent\n> commits I'd squash it into the respective commit that introduces those\n> checks.\n> \n\nYes, make sense. I will improve this in the next version.\n\n> Patrick\n"},{"id":"504276","messageId":"ZwOfxGjuKmzm00ry@ArchLinux","threadId":"61943","inReplyTo":"ZwOGqpeYiKITgsoV@pks.im","subject":"Re: [PATCH v5 9/9] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T08:45:56Z","receivedAt":"2024-10-07T08:45:50Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 08:58:50AM +0200, Patrick Steinhardt wrote:\n> On Sun, Sep 29, 2024 at 03:17:36PM +0800, shejialuo wrote:\n> > We have already introduced \"files_fsck_symref_target\". We should reuse\n> > this function to handle the symrefs which use legacy symbolic links. We\n> > should not check the trailing garbage for symbolic refs. Add a new\n> > parameter \"symbolic_link\" to disable some checks which should only be\n> > executed for textual symrefs.\n> \n> You're getting into implementation details before noting what the actual\n> problem is. So I'd recommend first describing the problem at a higher\n> level, and then note that we can reuse parts of preexisting infra to\n> address the issue.\n> \n\nThanks, I will improve this in the next version.\n\n> Patrick\n"},{"id":"504282","messageId":"ZwOm43a2ZmpvnlWc@pks.im","threadId":"61943","inReplyTo":"ZwOe7YVWmhshRhI9@ArchLinux","subject":"Re: [PATCH v5 2/9] builtin/refs: support multiple worktrees check for refs.","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T09:16:19Z","receivedAt":"2024-10-07T09:16:26Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 07, 2024 at 04:42:21PM +0800, shejialuo wrote:\n> On Mon, Oct 07, 2024 at 08:58:30AM +0200, Patrick Steinhardt wrote:\n> > On Sun, Sep 29, 2024 at 03:15:26PM +0800, shejialuo wrote:\n> > > We have already set up the infrastructure to check the consistency for\n> > > refs, but we do not support multiple worktrees. As we decide to add more\n> > > checks for ref content, we need to set up support for multiple\n> > > worktrees. Use \"get_worktrees\" and \"get_worktree_ref_store\" to check\n> > > refs under the worktrees.\n> > \n> > Makes sense.\n> > \n> > > Because we should only check once for \"packed-refs\", let's call the fsck\n> > > function for packed-backend when in the main worktree. In order to know\n> > > which directory we check, we should default print this information\n> > > instead of specifying \"--verbose\".\n> > \n> > This change should likely be evicted into its own commit with a bit more\n> > explanation.\n> > \n> > > It's not suitable to print these information to the stderr. So, change\n> > > to stdout.\n> > \n> > This one, too. Why exactly is in not suitable to print to stderr?\n> > \n> \n> I am sorry for the confusion. We should not print which directory we\n> check here into stderr. Because I think this will make test script\n> contain many unrelated info when using \"git refs verify 2>err\".\n> \n> The reason here is when checking the consistency of refs in multiple\n> worktrees. The ref name could be repeat. For example, worktree A\n> has its own ref called \"test\" under \".git/worktrees/A/refs/worktree/test\"\n> and worktree B has its own ref still called \"test\" under\n> \".git/worktrees/B/refs/worktree/test\".\n> \n> However, the refname would be printed to \"refs/worktree/test\". It will\n> make the user confused which \"refs/worktree/test\" is checked. So, we\n> should print this information like:\n> \n> Checking references consistency in .git\n> ...\n> checking references consistency in .git/worktrees/A\n> ...\n> checking references consistency in .git/worktrees/B\n> \n> However, when writing this, I feel a \".git\" is a bad usage. It will make\n> the user think it will check everything here. This should be improved in\n> the next version.\n\nBut wouldn't it be the better solution if we printed the fully-qualified\nreference name \"worktrees/worktree/refs/worktree/test\" instead? That\nwould remove the need to say which directory we're currently verifying\nin the first place.\n\nPatrick\n"},{"id":"504283","messageId":"ZwOnYAx_h9uGzPst@pks.im","threadId":"61943","inReplyTo":"ZwOfBCSTO4Qvtos3@ArchLinux","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T09:18:24Z","receivedAt":"2024-10-07T09:18:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 07, 2024 at 04:42:44PM +0800, shejialuo wrote:\n> On Mon, Oct 07, 2024 at 08:58:34AM +0200, Patrick Steinhardt wrote:\n> > On Sun, Sep 29, 2024 at 03:15:46PM +0800, shejialuo wrote:\n> > > \"git-fsck(1)\" will report an error when the ref content is invalid.\n> > > Following this, add a similar check to \"git refs verify\". Then add a new\n> > > fsck error message \"badRefContent(ERROR)\" to represent that a ref has an\n> > > invalid content.\n> > \n> > It would help readers to know where the code is that you're porting over\n> > to `git refs verify` so that one can double check that the port is done\n> > faithfully to the original.\n> > \n> \n> I am a little confused here. There are too many codes in \"git-fsck(1)\"\n> to check the ref consistency. How could I accurately express this info\n> in the commit message?\n\nWell, you say you ported over a specific consistency check from\ngit-fsck(1) to `git refs verify` in the commit message. So I assume that\nit should match a specific check in git-fsck(1), shouldn't it?\n\nPatrick\n"},{"id":"504284","messageId":"ZwOo9dQSr8Xu-PBb@pks.im","threadId":"61943","inReplyTo":"ZwOfYGi21oa302sS@ArchLinux","subject":"Re: [PATCH v5 4/9] ref: add more strict checks for regular refs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T09:25:17Z","receivedAt":"2024-10-07T09:25:22Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 07, 2024 at 04:44:16PM +0800, shejialuo wrote:\n> On Mon, Oct 07, 2024 at 08:58:37AM +0200, Patrick Steinhardt wrote:\n> > On Sun, Sep 29, 2024 at 03:16:00PM +0800, shejialuo wrote:\n> > > diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\n> > > index 22c385ea22..e310b5bce9 100644\n> > > --- a/Documentation/fsck-msgids.txt\n> > > +++ b/Documentation/fsck-msgids.txt\n> > > @@ -3541,6 +3546,21 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> > > \t\tgoto cleanup;\n> > > \t}\n> > > \n> > > +\tif (!(type & REF_ISSYMREF)) {\n> > > +\t\tif (!*trailing) {\n> > > +\t\t\tret = fsck_report_ref(o, &report,\n> > > +\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n> > > +\t\t\t\t\t \"misses LF at the end\");\n> > > +\t\t\tgoto cleanup;\n> > > +\t\t}\n> > > +\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n> > > +\t\t\tret = fsck_report_ref(o, &report,\n> > > +\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n> > > +\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n> > > +\t\t\tgoto cleanup;\n> > > +\t\t}\n> > > +\t}\n> > > +\n> > \n> > I think we should discern these two error cases and provide different\n> > message IDs.\n> > \n> \n> Actually, in the previous versions, I have mapped one message id to one\n> error case. But, in the v4, Junio asked a question\n> \n> Not limited to this patch, but isn't fsck_report_ref() misdesigned,\n> or is it just they are used poorly in these patches? In these two\n> callsites, the message string parameter does not give any more\n> information than what the FSCK_MSG_* enum gives.\n> \n> That is what I meant by \"misdesigned\"---if one message enum always\n> corresponds to one human-readable message, there is not much point\n> in forcing callers to supply both, is there?\n> \n> In my opinion, we should have only one case here for trailing garbage\n> and not end with a newline. When writing the code, I chose the name\n> \"unofficialFormattedRef\" for the following reason:\n> \n> 1. If we use two message ids here, for every message id, we need write\n> to info the user \"please report this to git mailing list\".\n> \n> 2. If we decide to make this as an error. We could just classify them\n> into \"badRefContent\" message category.\n> \n> 3. The semantic is correct here, they are truly curious formatted\n> refs, and eventually we will give the info to the user what is\n> curious.\n> \n> So, I think we should not always map one message to one error case.\n\nFrom my point of view the error codes should be the single source of\ntruth, as this is what a user can use to disable specific checks. So if\none code maps to multiple messages they have the problem that they can\nonly disable all of those messages.\n\nI don't disagree with what Junio is saying. It is somewhat duplicate\nthat the user has to pass both a code and a message in the current\nform-- it should be sufficient for them to pass the code, and the\nmessage can then e.g. be extracted from a central array that maps codes\nto messages.\n\nBut you can also make the reverse argument: messages can be dynamic, so\nthat the caller may include additional details around why specfically\nthe check failed. The code and message would still be 1:1, but we may\ninclude additional details like that to guide the user.\n\nPatrick\n"},{"id":"504285","messageId":"ZwOpR2kQ0cWb_7Kq@pks.im","threadId":"61943","inReplyTo":"ZwOffN9UWX1gP0gy@ArchLinux","subject":"Re: [PATCH v5 6/9] ref: add escape check for the referent of symref","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-07T09:26:31Z","receivedAt":"2024-10-07T09:26:41Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 07, 2024 at 04:44:44PM +0800, shejialuo wrote:\n> On Mon, Oct 07, 2024 at 08:58:55AM +0200, Patrick Steinhardt wrote:\n> > On Sun, Sep 29, 2024 at 03:16:21PM +0800, shejialuo wrote:\n> > I'd also rename this to e.g. \"symrefTargetIsNotAReference\" or something\n> > like that, because it's not really about whether or not the referent is\n> > \"escaping\". It's a bit of a mouthful, but I don't really have a better\n> > name. So feel free to pick something different that describes the error\n> > better.\n> > \n> \n> I guess \"symrefTargetIsNotAReference\" is a little too long. If we decide\n> to convert it to error later. Why not just put it into the \"badReferent\"\n> fsck message?\n> \n> So, I do not think we need to rename. As I have talked about, we don't\n> need to map error case to fsck message id one by one.\n\nMostly because I disagree with this here. I think there should be a 1:1\nmapping, and \"badReferent\" is too generic to provide that.\n\nPatrick\n"},{"id":"504322","messageId":"ZwPOwgXITjoUejp5@ArchLinux","threadId":"61943","inReplyTo":"ZwOm43a2ZmpvnlWc@pks.im","subject":"Re: [PATCH v5 2/9] builtin/refs: support multiple worktrees check for refs.","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T12:06:26Z","receivedAt":"2024-10-07T12:06:20Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 11:16:19AM +0200, Patrick Steinhardt wrote:\n\n[snip]\n\n> > However, the refname would be printed to \"refs/worktree/test\". It will\n> > make the user confused which \"refs/worktree/test\" is checked. So, we\n> > should print this information like:\n> > \n> > Checking references consistency in .git\n> > ...\n> > checking references consistency in .git/worktrees/A\n> > ...\n> > checking references consistency in .git/worktrees/B\n> > \n> > However, when writing this, I feel a \".git\" is a bad usage. It will make\n> > the user think it will check everything here. This should be improved in\n> > the next version.\n> \n> But wouldn't it be the better solution if we printed the fully-qualified\n> reference name \"worktrees/worktree/refs/worktree/test\" instead? That\n> would remove the need to say which directory we're currently verifying\n> in the first place.\n> \n\nGood idea. I will use this way in the next version.\n\n> Patrick\n\nThanks\n"},{"id":"504323","messageId":"ZwPPSe3KGjU_XAPY@ArchLinux","threadId":"61943","inReplyTo":"ZwOnYAx_h9uGzPst@pks.im","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T12:08:41Z","receivedAt":"2024-10-07T12:08:34Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 11:18:24AM +0200, Patrick Steinhardt wrote:\n> On Mon, Oct 07, 2024 at 04:42:44PM +0800, shejialuo wrote:\n> > On Mon, Oct 07, 2024 at 08:58:34AM +0200, Patrick Steinhardt wrote:\n> > > On Sun, Sep 29, 2024 at 03:15:46PM +0800, shejialuo wrote:\n> > > > \"git-fsck(1)\" will report an error when the ref content is invalid.\n> > > > Following this, add a similar check to \"git refs verify\". Then add a new\n> > > > fsck error message \"badRefContent(ERROR)\" to represent that a ref has an\n> > > > invalid content.\n> > > \n> > > It would help readers to know where the code is that you're porting over\n> > > to `git refs verify` so that one can double check that the port is done\n> > > faithfully to the original.\n> > > \n> > \n> > I am a little confused here. There are too many codes in \"git-fsck(1)\"\n> > to check the ref consistency. How could I accurately express this info\n> > in the commit message?\n> \n> Well, you say you ported over a specific consistency check from\n> git-fsck(1) to `git refs verify` in the commit message. So I assume that\n> it should match a specific check in git-fsck(1), shouldn't it?\n> \n\nI understand your meaning here. I will improve the commit message in the\nnext version.\n\n> Patrick\n"},{"id":"504324","messageId":"ZwPRvENH0TA5YX6J@ArchLinux","threadId":"61943","inReplyTo":"ZwOo9dQSr8Xu-PBb@pks.im","subject":"Re: [PATCH v5 4/9] ref: add more strict checks for regular refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T12:19:08Z","receivedAt":"2024-10-07T12:19:02Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 07, 2024 at 11:25:17AM +0200, Patrick Steinhardt wrote:\n\n[snip]\n\n> > \n> > Actually, in the previous versions, I have mapped one message id to one\n> > error case. But, in the v4, Junio asked a question\n> > \n> > Not limited to this patch, but isn't fsck_report_ref() misdesigned,\n> > or is it just they are used poorly in these patches? In these two\n> > callsites, the message string parameter does not give any more\n> > information than what the FSCK_MSG_* enum gives.\n> > \n> > That is what I meant by \"misdesigned\"---if one message enum always\n> > corresponds to one human-readable message, there is not much point\n> > in forcing callers to supply both, is there?\n> > \n> > In my opinion, we should have only one case here for trailing garbage\n> > and not end with a newline. When writing the code, I chose the name\n> > \"unofficialFormattedRef\" for the following reason:\n> > \n> > 1. If we use two message ids here, for every message id, we need write\n> > to info the user \"please report this to git mailing list\".\n> > \n> > 2. If we decide to make this as an error. We could just classify them\n> > into \"badRefContent\" message category.\n> > \n> > 3. The semantic is correct here, they are truly curious formatted\n> > refs, and eventually we will give the info to the user what is\n> > curious.\n> > \n> > So, I think we should not always map one message to one error case.\n> \n> From my point of view the error codes should be the single source of\n> truth, as this is what a user can use to disable specific checks. So if\n> one code maps to multiple messages they have the problem that they can\n> only disable all of those messages.\n> \n\nThanks for your remind here. I totally forgot this. I have changed my\nmind now, we should use one to one mapping here. As you said, if we do\nnot, we will give the user the bad experience.\n\n> I don't disagree with what Junio is saying. It is somewhat duplicate\n> that the user has to pass both a code and a message in the current\n> form-- it should be sufficient for them to pass the code, and the\n> message can then e.g. be extracted from a central array that maps codes\n> to messages.\n> \n> But you can also make the reverse argument: messages can be dynamic, so\n> that the caller may include additional details around why specfically\n> the check failed. The code and message would still be 1:1, but we may\n> include additional details like that to guide the user.\n> \n\nYes, I will refactor the \"fsck_report\" to allow the user pass the \"NULL\"\nmessage if the fsck message id is clear enough to indicate the error\ncase.\n\nSo, more things to do here.\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"504325","messageId":"ZwPYwbeYJn30pPnM@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"Re: [PATCH v5 0/9] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-07T12:49:05Z","receivedAt":"2024-10-07T12:49:00Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"From the discussion with Patrick in v5 and Junio in v4. I conclude the\nfollow things:\n\n1. \"fsck_ref_report\" should not be refactored to accept `NULL`. There\nwould be only one situation where it will be a little bad (the content\nof a ref does not end with a newline). In the other situations, the\nmessage part will be useful, such as:\n\n refs/heads/garbage-branch: trailingRefContent: ' garbage'.\n refs/heads/escape: escapeReferent: referent 'xxx' is outside.\n\nAlthough for some messages, only use fsck message id is enough. But we\ncould also specify the message. It's not harmful anyway.\n\n2. The mapping from fsck message id to error case should be one to one.\nThis is essentially important because the user could set the fsck error\nlevels. If we use multiple to one, we will give the user a bad\nexperience. We should avoid this.\n\nI will wait for more comments to ensure the next version will be better.\n\nThanks,\nJialuo\n"},{"id":"504395","messageId":"CAOLa=ZQ4VO52nGjTX1nbzj3zxSjijmBUAsEq=OsOFo8xjMwVYQ@mail.gmail.com","threadId":"61943","inReplyTo":"Zvj-hjBXlWr803Us@ArchLinux","subject":"Re: [PATCH v5 1/9] ref: initialize \"fsck_ref_report\" with zero","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-10-08T07:29:54Z","receivedAt":"2024-10-08T07:29:56Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n> \"referent\" is NULL. So, we need to always initialize these parameters to\n> NULL instead of letting them point to anywhere when creating a new\n> \"fsck_ref_report\" structure.\n>\n> The original code explicitly initializes the \"path\" member in the\n> \"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\n> members in the struct). It is more customary to use \"{ 0 }\" to express\n> that we are 0-initializing everything. In order to align with the the\n\ns/the//\n\n[snip]\n"},{"id":"504396","messageId":"CAOLa=ZQ3Gytt4Lsttxws3DWqbjteJS8mXvZSPzDwBJi_ALS03Q@mail.gmail.com","threadId":"61943","inReplyTo":"Zvj-osCNDMrUQv83@ArchLinux","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-10-08T07:43:20Z","receivedAt":"2024-10-08T07:43:22Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n[snip]\n\n> +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> +\t\t\t\t \"cannot read ref file\");\n> +\t\tgoto cleanup;\n> +\t}\n> +\n\nShouldn't we use `die_errno` here instead? I mean, this is not really a\nbad ref content issue. If we don't want to die here, it would still\nprobably be nice to get the actual issue using `strerror` instead and\nuse that instead of the generic message we have here.\n\n[snip]\n"},{"id":"504397","messageId":"CAOLa=ZQGaLJTJ-bcSYD1mcsgCD8ayHYzXatmK2iyz4gDodvvKA@mail.gmail.com","threadId":"61943","inReplyTo":"Zvj-vbvQym1R4KJk@ArchLinux","subject":"Re: [PATCH v5 5/9] ref: add basic symref content check for files backend","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-10-08T07:58:16Z","receivedAt":"2024-10-08T07:58:18Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> We have code that checks regular ref contents, but we do not yet check\n> the contents of symbolic refs. By using \"parse_loose_ref_content\" for\n> symbolic refs, we will get the information of the \"referent\".\n>\n> We do not need to check the \"referent\" by opening the file. This is\n> because if \"referent\" exists in the file system, we will eventually\n> check its correctness by inspecting every file in the \"refs\" directory.\n> If the \"referent\" does not exist in the filesystem, this is OK as it is\n> seen as the dangling symref.\n>\n> So we just need to check the \"referent\" string content. A regular could\n\nseems like we're missing the noun here, a regular what?\n\n> be accepted as a textual symref if it begins with \"ref:\", followed by\n> zero or more whitespaces, followed by the full refname, followed only by\n> whitespace characters. However, we always write a single SP after \"ref:\"\n> and a single LF after the refname. It may seem that we should report a\n> fsck error message when the \"referent\" does not apply above rules and we\n> should not be so aggressive because third-party reimplementations of Git\n> may have taken advantage of the looser syntax. Put it more specific, we\n> accept the following \"referent\":\n>\n> 1. \"ref: refs/heads/master \"\n> 2. \"ref: refs/heads/master \\n \\n\"\n> 3. \"ref: refs/heads/master\\n\\n\"\n>\n> When introducing the regular ref content checks, we created a new fsck\n> message \"unofficialFormattedRef\" which exactly represents above\n> situation. So we will reuse this fsck message to write checks to info\n> the user about these situations.\n>\n\nPlus to what Patrick said in the previous commit, it would be nice to\nseparate these issues with different message IDs.\n\n> But we do not allow any other trailing garbage. The followings are bad\n> symref contents which will be reported as fsck error by \"git-fsck(1)\".\n>\n> 1. \"ref: refs/heads/master garbage\\n\"\n> 2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n>\n> And we introduce a new \"badReferent(ERROR)\" fsck message to report above\n> errors by using \"ref.c::check_refname_format\". But we cannot just pass\n> the \"referent\" to this function because the \"referent\" might contain\n> some whitespaces which will cause \"check_refname_format\" failing.\n>\n\nIt would be nice if you could elaborate here, or rather restructure to\nsay something like..\n\n Since 'check_refname_format' doesn't work with whitespaces, we use\n the trimmed version of 'referent' with the function.\n\n[snip]\n"},{"id":"504415","messageId":"ZwUjCnxPw0NObj5d@ArchLinux","threadId":"61943","inReplyTo":"CAOLa=ZQGaLJTJ-bcSYD1mcsgCD8ayHYzXatmK2iyz4gDodvvKA@mail.gmail.com","subject":"Re: [PATCH v5 5/9] ref: add basic symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-08T12:18:18Z","receivedAt":"2024-10-08T12:18:10Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Oct 08, 2024 at 12:58:16AM -0700, Karthik Nayak wrote:\n> shejialuo writes:\n> \n> > We have code that checks regular ref contents, but we do not yet check\n> > the contents of symbolic refs. By using \"parse_loose_ref_content\" for\n> > symbolic refs, we will get the information of the \"referent\".\n> >\n> > We do not need to check the \"referent\" by opening the file. This is\n> > because if \"referent\" exists in the file system, we will eventually\n> > check its correctness by inspecting every file in the \"refs\" directory.\n> > If the \"referent\" does not exist in the filesystem, this is OK as it is\n> > seen as the dangling symref.\n> >\n> > So we just need to check the \"referent\" string content. A regular could\n> \n> seems like we're missing the noun here, a regular what?\n> \n\nIt should be \"a regular ref\". I copied the original commit message and\nmay carelessly type \"daw\" in vim to delete the \"ref\". Thanks.\n"},{"id":"504417","messageId":"ZwUkZuCtYu7niuFM@ArchLinux","threadId":"61943","inReplyTo":"CAOLa=ZQ3Gytt4Lsttxws3DWqbjteJS8mXvZSPzDwBJi_ALS03Q@mail.gmail.com","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-08T12:24:06Z","receivedAt":"2024-10-08T12:23:58Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Oct 08, 2024 at 12:43:20AM -0700, Karthik Nayak wrote:\n> shejialuo writes:\n> \n> [snip]\n> \n> > +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> > +\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> > +\t\t\t\t \"cannot read ref file\");\n> > +\t\tgoto cleanup;\n> > +\t}\n> > +\n> \n> Shouldn't we use `die_errno` here instead? I mean, this is not really a\n> bad ref content issue. If we don't want to die here, it would still\n> probably be nice to get the actual issue using `strerror` instead and\n> use that instead of the generic message we have here.\n> \n\nWell, I think I need to dive into the \"open\" system call here. Actually,\nwe have two opinions now. Junio thought that we should use\n\"fsck_report_ref\" to report. Karthik, Patrick and I thought that we\nshould report using \"*errno\" because this is a general error.\n\nLet me investigate what situations will make \"open\" system call fail.\nThus, we could fully explain which choice we will choose.\n\nThanks,\nJialuo\n"},{"id":"504460","messageId":"xmqq5xq232wa.fsf@gitster.g","threadId":"61943","inReplyTo":"ZwUkZuCtYu7niuFM@ArchLinux","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-10-08T17:44:53Z","receivedAt":"2024-10-08T17:44:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"shejialuo writes:\n\n> On Tue, Oct 08, 2024 at 12:43:20AM -0700, Karthik Nayak wrote:\n>> shejialuo writes:\n>> \n>> [snip]\n>> \n>> > +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n>> > +\t\tret = fsck_report_ref(o, &report,\n>> > +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n>> > +\t\t\t\t \"cannot read ref file\");\n>> > +\t\tgoto cleanup;\n>> > +\t}\n>> > +\n>> \n>> Shouldn't we use `die_errno` here instead? I mean, this is not really a\n>> bad ref content issue. If we don't want to die here, it would still\n>> probably be nice to get the actual issue using `strerror` instead and\n>> use that instead of the generic message we have here.\n>> \n>\n> Well, I think I need to dive into the \"open\" system call here. Actually,\n> we have two opinions now. Junio thought that we should use\n> \"fsck_report_ref\" to report. Karthik, Patrick and I thought that we\n> should report using \"*errno\" because this is a general error.\n\nWhat do you mean by \"a general error\"? It is true that we failed to\nread a ref file, so even if it is an I/O error, I'd think it is OK\nto report it as an error while reading one particular ref.\n\nGiving more information is a separate issue. If fsck_report_ref()\ncan be extended to take something like\n\n \"cannot read ref file '%s': (%s)\", iter->path.buf, strerror(errno)\n\nthat would give the user necessary information.\n\nAnd I agree with half-of what Karthik said, i.e., we do not want to\ndie here if this is meant to run as a part of \"git fsck\".\n\nI may have said this before, but quite frankly, the API into the\nfsck_report_ref() function is misdesigned. If the single constant\nstring \"cannot read ref file\" cnanot give more information than\nFSCK_MSG_BAD_REF_CONTENT, the parameter has no value.\n\nThe fsck.c:report() function, which is the main function to report\nfsck's findings before fsck_report_ref() was introduced, did not\nhave such a problem, as it allowed \"const char *fmt, ...\" at the\nend. Is it too late to fix the fsck_report_ref()?\n\nThanks.\n\n"},{"id":"504531","messageId":"ZwY5O63OI01LZX1u@pks.im","threadId":"61943","inReplyTo":"xmqq5xq232wa.fsf@gitster.g","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-09T08:05:19Z","receivedAt":"2024-10-09T08:05:25Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Oct 08, 2024 at 10:44:53AM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > On Tue, Oct 08, 2024 at 12:43:20AM -0700, Karthik Nayak wrote:\n> >> shejialuo writes:\n> >> \n> >> [snip]\n> >> \n> >> > +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> >> > +\t\tret = fsck_report_ref(o, &report,\n> >> > +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> >> > +\t\t\t\t \"cannot read ref file\");\n> >> > +\t\tgoto cleanup;\n> >> > +\t}\n> >> > +\n> >> \n> >> Shouldn't we use `die_errno` here instead? I mean, this is not really a\n> >> bad ref content issue. If we don't want to die here, it would still\n> >> probably be nice to get the actual issue using `strerror` instead and\n> >> use that instead of the generic message we have here.\n> >> \n> >\n> > Well, I think I need to dive into the \"open\" system call here. Actually,\n> > we have two opinions now. Junio thought that we should use\n> > \"fsck_report_ref\" to report. Karthik, Patrick and I thought that we\n> > should report using \"*errno\" because this is a general error.\n> \n> What do you mean by \"a general error\"? It is true that we failed to\n> read a ref file, so even if it is an I/O error, I'd think it is OK\n> to report it as an error while reading one particular ref.\n> \n> Giving more information is a separate issue. If fsck_report_ref()\n> can be extended to take something like\n> \n> \"cannot read ref file '%s': (%s)\", iter->path.buf, strerror(errno)\n> \n> that would give the user necessary information.\n\nYeah, this is also in line with what I proposed elsewhere, where we have\nbeen discussing the 1:1 mapping between error codes and error messages.\nIf the error messages were dynamic they'd be a whole lot useful overall\nand could provide more context.\n\n> And I agree with half-of what Karthik said, i.e., we do not want to\n> die here if this is meant to run as a part of \"git fsck\".\n> \n> I may have said this before, but quite frankly, the API into the\n> fsck_report_ref() function is misdesigned. If the single constant\n> string \"cannot read ref file\" cnanot give more information than\n> FSCK_MSG_BAD_REF_CONTENT, the parameter has no value.\n\nTrue in the current form, yeah. If `fsck_report_ref()` learned to take a\nvararg argument and treat its first argument as a string format it would\nbe justified though, as the message is now dynamic and can contain more\ncontext around the specific failure that cannot be provided statically\nvia the 1:1 mapping between error code and message.\n\n> The fsck.c:report() function, which is the main function to report\n> fsck's findings before fsck_report_ref() was introduced, did not\n> have such a problem, as it allowed \"const char *fmt, ...\" at the\n> end. Is it too late to fix the fsck_report_ref()?\n\nI don't think so, I think we should be able to refactor the code rather\neasily to do so.\n\nPatrick\n"},{"id":"504546","messageId":"ZwZvTKG625P_ncl7@ArchLinux","threadId":"61943","inReplyTo":"xmqq5xq232wa.fsf@gitster.g","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-09T11:55:56Z","receivedAt":"2024-10-09T11:55:48Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Oct 08, 2024 at 10:44:53AM -0700, Junio C Hamano wrote:\n> shejialuo writes:\n> \n> > On Tue, Oct 08, 2024 at 12:43:20AM -0700, Karthik Nayak wrote:\n> >> shejialuo writes:\n> >> \n> >> [snip]\n> >> \n> >> > +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> >> > +\t\tret = fsck_report_ref(o, &report,\n> >> > +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> >> > +\t\t\t\t \"cannot read ref file\");\n> >> > +\t\tgoto cleanup;\n> >> > +\t}\n> >> > +\n> >> \n> >> Shouldn't we use `die_errno` here instead? I mean, this is not really a\n> >> bad ref content issue. If we don't want to die here, it would still\n> >> probably be nice to get the actual issue using `strerror` instead and\n> >> use that instead of the generic message we have here.\n> >> \n> >\n> > Well, I think I need to dive into the \"open\" system call here. Actually,\n> > we have two opinions now. Junio thought that we should use\n> > \"fsck_report_ref\" to report. Karthik, Patrick and I thought that we\n> > should report using \"*errno\" because this is a general error.\n> \n> What do you mean by \"a general error\"? It is true that we failed to\n> read a ref file, so even if it is an I/O error, I'd think it is OK\n> to report it as an error while reading one particular ref.\n\nMake sense.\n\n> Giving more information is a separate issue. If fsck_report_ref()\n> can be extended to take something like\n> \n> \"cannot read ref file '%s': (%s)\", iter->path.buf, strerror(errno)\n> \n> that would give the user necessary information.\n\nAt current, the `fsck_report_ref` can do this. I think I used\n`fsck_report_ref` function badly in this case.\n\n> And I agree with half-of what Karthik said, i.e., we do not want to\n> die here if this is meant to run as a part of \"git fsck\".\n\nYes, we should not die the program. Instead, we need to continuously\ncheck other refs.\n\n> I may have said this before, but quite frankly, the API into the\n> fsck_report_ref() function is misdesigned. If the single constant\n> string \"cannot read ref file\" cnanot give more information than\n> FSCK_MSG_BAD_REF_CONTENT, the parameter has no value.\n> \n> The fsck.c:report() function, which is the main function to report\n> fsck's findings before fsck_report_ref() was introduced, did not\n> have such a problem, as it allowed \"const char *fmt, ...\" at the\n> end. Is it too late to fix the fsck_report_ref()?\n\nI agree that if the FSCK message id could explain the error well, there\nis no need for us to provide extra message. But, I want to say the\n`fsck_report_ref` is not misdesigned here. It is just the same as the\n\"fsck.c::report\" function which has \"const char *fmt, ...\" at the end\nlike the following shows:\n\n int fsck_report_ref(struct fsck_options *options,\n struct fsck_ref_report *report,\n enum fsck_msg_id msg_id,\n const char *fmt, ...)\n\nAnd I do think \"fsck.c::report\" function also has the above problems.\nLet me give you some examples here in \"fsck.c\":\n\n report(options, tree_oid, OBJ_TREE,\n FSCK_MSG_BAD_FILEMODE,\n \"contains bad file modes\");\n\n report(options, tree_oid, OBJ_TREE,\n FSCK_MSG_DUPLICATE_ENTRIES,\n \"contains duplicate file entries\");\n\n ...\n\nSo, I want to say there is no difference between \"fsck_ref_report\" and\n\"fsck.c::report\". When I refactored the code in GSoC journey, the main\nproblem is that we should reuse the original \"fsck.c::report\" code\ninstead of writing redundant codes.\n\nThe final result is I extract a new function \"fsck_vreport\" here (I\nleverage the original \"fsck.c::report\" function) which will be called by\n\"fsck_ref_report\" and \"fsck.c::report\".\n\n static int fsck_vreport(struct fsck_options *options,\n void *fsck_report,\n enum fsck_msg_id msg_id,\n const char *fmt, va_list ap)\n\nFrom my perspective, if we decide to refactor, we should allow the user\ncall the followings:\n\n fsck_ref_report(..., FSCK_MSG_BAD_REF_CONTENT, NULL);\n report(..., FSCK_MSG_DUPLICATE_ENTRIES, NULL);\n\nSo, we should check whether `fmt` is NULL in the `fsck_vreport`\nfunction to make sure that if FSCK message is good enough to explain\nwhat happens, we should not pass any message.\n\n> Thanks.\n\nThanks,\nJialuo\n"},{"id":"504547","messageId":"ZwZwGAxkbQtnozh9@ArchLinux","threadId":"61943","inReplyTo":"ZwY5O63OI01LZX1u@pks.im","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-09T11:59:20Z","receivedAt":"2024-10-09T11:59:14Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Oct 09, 2024 at 10:05:19AM +0200, Patrick Steinhardt wrote:\n\n[snip]\n\n> > I may have said this before, but quite frankly, the API into the\n> > fsck_report_ref() function is misdesigned. If the single constant\n> > string \"cannot read ref file\" cnanot give more information than\n> > FSCK_MSG_BAD_REF_CONTENT, the parameter has no value.\n> \n> True in the current form, yeah. If `fsck_report_ref()` learned to take a\n> vararg argument and treat its first argument as a string format it would\n> be justified though, as the message is now dynamic and can contain more\n> context around the specific failure that cannot be provided statically\n> via the 1:1 mapping between error code and message.\n> \n\nIt is not \"learned\". At current, `fsck_report_ref` can do this and is\nthe same as \"fsck.c::report\". I have explained this when replying to\nJunio.\n\n> > The fsck.c:report() function, which is the main function to report\n> > fsck's findings before fsck_report_ref() was introduced, did not\n> > have such a problem, as it allowed \"const char *fmt, ...\" at the\n> > end. Is it too late to fix the fsck_report_ref()?\n> \n> I don't think so, I think we should be able to refactor the code rather\n> easily to do so.\n> \n\nIt's not hard to refactor the code. But this is not the problem. I am a\nlittle confused here. Because we already allowed \"fsck_report_ref\"\nhaving \"const char *fmt, ...\" at the end.\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"504678","messageId":"Zwd5kiZZME3ygM0K@pks.im","threadId":"61943","inReplyTo":"ZwZwGAxkbQtnozh9@ArchLinux","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-10-10T06:52:07Z","receivedAt":"2024-10-10T06:52:12Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Oct 09, 2024 at 07:59:20PM +0800, shejialuo wrote:\n> On Wed, Oct 09, 2024 at 10:05:19AM +0200, Patrick Steinhardt wrote:\n> > > The fsck.c:report() function, which is the main function to report\n> > > fsck's findings before fsck_report_ref() was introduced, did not\n> > > have such a problem, as it allowed \"const char *fmt, ...\" at the\n> > > end. Is it too late to fix the fsck_report_ref()?\n> > \n> > I don't think so, I think we should be able to refactor the code rather\n> > easily to do so.\n> > \n> \n> It's not hard to refactor the code. But this is not the problem. I am a\n> little confused here. Because we already allowed \"fsck_report_ref\"\n> having \"const char *fmt, ...\" at the end.\n\nAh, I didn't double check, but was operating on what I understood from\nthis thread. In that case I think that the current interface is okay.\n\nPatrick\n"},{"id":"504710","messageId":"xmqqr08olzgl.fsf@gitster.g","threadId":"61943","inReplyTo":"Zwd5kiZZME3ygM0K@pks.im","subject":"Re: [PATCH v5 3/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-10-10T16:00:58Z","receivedAt":"2024-10-10T16:01:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Wed, Oct 09, 2024 at 07:59:20PM +0800, shejialuo wrote:\n>> On Wed, Oct 09, 2024 at 10:05:19AM +0200, Patrick Steinhardt wrote:\n>> > > The fsck.c:report() function, which is the main function to report\n>> > > fsck's findings before fsck_report_ref() was introduced, did not\n>> > > have such a problem, as it allowed \"const char *fmt, ...\" at the\n>> > > end. Is it too late to fix the fsck_report_ref()?\n>> > \n>> > I don't think so, I think we should be able to refactor the code rather\n>> > easily to do so.\n>> > \n>> \n>> It's not hard to refactor the code. But this is not the problem. I am a\n>> little confused here. Because we already allowed \"fsck_report_ref\"\n>> having \"const char *fmt, ...\" at the end.\n>\n> Ah, I didn't double check, but was operating on what I understood from\n> this thread. In that case I think that the current interface is okay.\n\nI didn't, either. So there is an obvious way out for \"why aren't we\ntelling the errno to users\" issue? That's good.\n"},{"id":"505677","messageId":"ZxZX5HDdq_R0C77b@ArchLinux","threadId":"61943","inReplyTo":"Zvj-DgHqtC30KjJe@ArchLinux","subject":"[PATCH v6 0/9] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:32:20Z","receivedAt":"2024-10-21T13:32:19Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis new version updates the following things.\n\nFirst, I want to talk about the new things. [PATCH v6 2/9] and [PATCH v6\n3/9] are used to solve a bug when I implemented the checks for refname\nfor the following code:\n\n if (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n ret = fsck_report(...);\n }\n\nSo, the code will wrongly report an error for \"refs/heads/@\". And I fix\nthis issue by using two commits.\n\nFor the difference against the previous version\n\n1. Split [PATCH v5 8/9] into every related commit.\n\n2. In [PATCH v6 4/9], print the worktree ref fullname to avoid\nambiguous.\n\n3. Use one-to-one mapping fsck message.\n\n4. Enhance the commit message and the usage of \"fsck_report_ref\" to\nprovide more useful information.\n\n5. Rename \"escapeReferent\" to \"symrefTargetIsNotARef\". I agree that we\nshould use this. \"escpae\" is not right. However, I cannot find an\nelegant name. So I follow the advice from Patrick.\n\nI provide the \"interdiff\" here which will be helpful for reviewers.\n\nThanks,\nJialuo\n\nshejialuo (9):\n ref: initialize \"fsck_ref_report\" with zero\n ref: check the full refname instead of basename\n ref: initialize target name outside of check functions\n ref: support multiple worktrees check for refs\n ref: port git-fsck(1) regular refs check for files backend\n ref: add more strict checks for regular refs\n ref: add basic symref content check for files backend\n ref: check whether the target of the symref is a ref\n ref: add symlink ref content check for files backend\n\n Documentation/fsck-msgids.txt | 35 +++\n builtin/refs.c | 12 +-\n fsck.h | 6 +\n refs.c | 7 +-\n refs.h | 3 +-\n refs/debug.c | 5 +-\n refs/files-backend.c | 187 ++++++++++++--\n refs/packed-backend.c | 8 +-\n refs/refs-internal.h | 5 +-\n refs/reftable-backend.c | 3 +-\n t/t0602-reffiles-fsck.sh | 457 +++++++++++++++++++++++++++++++++-\n 11 files changed, 693 insertions(+), 35 deletions(-)\n\nInterdiff against v5:\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex ffe9d6a2f6..b14bc44ca4 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,8 +28,8 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n-`badReferent`::\n-\t(ERROR) The referent of a ref is invalid.\n+`badReferentName`::\n+\t(ERROR) The referent name of a symref is invalid.\n \n `badTagName`::\n \t(INFO) A tag has an invalid format.\n@@ -52,14 +52,6 @@\n `emptyName`::\n \t(WARN) A path contains an empty name.\n \n-`escapeReferent`::\n-\t(INFO) The referent of a symref is outside the \"ref\" directory.\n-\tAlthough we allow create a symref pointing to the referent which\n-\tis outside the \"ref\" by using `git symbolic-ref`, we may tighten\n-\tthe rule in the future. Report to the git@vger.kernel.org\n-\tmailing list if you see this error, as we need to know what tools\n-\tcreated such a file.\n-\n `extraHeaderEntry`::\n \t(IGNORE) Extra headers found after `tagger`.\n \n@@ -184,11 +176,34 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`refMissingNewline`::\n+\t(INFO) A loose ref that does not end with newline(LF). As\n+\tvalid implementations of Git never created such a loose ref\n+\tfile, it may become an error in the future. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as\n+\twe need to know what tools created such a file.\n+\n `symlinkRef`::\n-\t(INFO) A symbolic link is used as a symref. Report to the\n+\t(INFO) A symbolic link is used as a symref. Report to the\n \tgit@vger.kernel.org mailing list if you see this error, as we\n \tare assessing the feasibility of dropping the support to drop\n-\tcreating symblinks as symrefs.\n+\tcreating symbolic links as symrefs.\n+\n+`symrefTargetIsNotARef`::\n+\t(INFO) The target of a symbolic reference points neither to\n+\ta root reference nor to a reference starting with \"refs/\".\n+\tAlthough we allow create a symref pointing to the referent which\n+\tis outside the \"ref\" by using `git symbolic-ref`, we may tighten\n+\tthe rule in the future. Report to the git@vger.kernel.org\n+\tmailing list if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n+`trailingRefContent`::\n+\t(INFO) A loose ref has trailing content. As valid implementations\n+\tof Git never created such a loose ref file, it may become an\n+\terror in the future. Report to the git@vger.kernel.org mailing\n+\tlist if you see this error, as we need to know what tools\n+\tcreated such a file.\n \n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n@@ -196,14 +211,6 @@\n `unknownType`::\n \t(ERROR) Found an unknown object type.\n \n-`unofficialFormattedRef`::\n-\t(INFO) The content of a loose ref file is not in the official\n-\tformat such as not having a LF at the end or having trailing\n-\tgarbage. As valid implementations of Git never created such a\n-\tloose ref file, it may become an error in the future. Report\n-\tto the git@vger.kernel.org mailing list if you see this error,\n-\tas we need to know what tools created such a file.\n-\n `unterminatedHeader`::\n \t(FATAL) Missing end-of-line in the object header.\n \ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex 3c492ea922..886c4ceae3 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -89,9 +89,10 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \tworktrees = get_worktrees();\n \tfor (p = worktrees; *p; p++) {\n \t\tstruct worktree *wt = *p;\n-\t\tret += refs_fsck(get_worktree_ref_store(wt), &fsck_refs_options);\n+\t\tret |= refs_fsck(get_worktree_ref_store(wt), &fsck_refs_options, wt);\n \t}\n \n+\n \tfsck_options_clear(&fsck_refs_options);\n \tfree_worktrees(worktrees);\n \treturn ret;\ndiff --git a/fsck.h b/fsck.h\nindex f1da5c8a77..a44c231a5f 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,7 +34,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n-\tFUNC(BAD_REFERENT, ERROR) \\\n+\tFUNC(BAD_REFERENT_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\n@@ -80,7 +80,6 @@ enum fsck_msg_type {\n \tFUNC(LARGE_PATHNAME, WARN) \\\n \t/* infos (reported as warnings, but ignored by default) */ \\\n \tFUNC(BAD_FILEMODE, INFO) \\\n-\tFUNC(ESCAPE_REFERENT, INFO) \\\n \tFUNC(GITMODULES_PARSE, INFO) \\\n \tFUNC(GITIGNORE_SYMLINK, INFO) \\\n \tFUNC(GITATTRIBUTES_SYMLINK, INFO) \\\n@@ -88,7 +87,9 @@ enum fsck_msg_type {\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n \tFUNC(SYMLINK_REF, INFO) \\\n-\tFUNC(UNOFFICIAL_FORMATTED_REF, INFO) \\\n+\tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n+\tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs.c b/refs.c\nindex 6ba1bb1aa1..f88b32a633 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -318,9 +318,10 @@ int check_refname_format(const char *refname, int flags)\n \treturn check_or_sanitize_refname(refname, flags, NULL);\n }\n \n-int refs_fsck(struct ref_store *refs, struct fsck_options *o)\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt)\n {\n-\treturn refs->be->fsck(refs, o);\n+\treturn refs->be->fsck(refs, o, wt);\n }\n \n void sanitize_refname_component(const char *refname, struct strbuf *out)\ndiff --git a/refs.h b/refs.h\nindex 108dfc93b3..341d43239c 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -549,7 +549,8 @@ int check_refname_format(const char *refname, int flags);\n * reflogs are consistent, and non-zero otherwise. The errors will be\n * written to stderr.\n */\n-int refs_fsck(struct ref_store *refs, struct fsck_options *o);\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt);\n \n /*\n * Apply the rules from check_refname_format, but mutate the result until it\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 45e2e784a0..72e80ddd6d 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -420,10 +420,11 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,\n }\n \n static int debug_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n-\tint res = drefs->refs->be->fsck(drefs->refs, o);\n+\tint res = drefs->refs->be->fsck(drefs->refs, o, wt);\n \ttrace_printf_key(&trace_refs, \"fsck: %d\\n\", res);\n \treturn res;\n }\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 5a5327a146..180f8e28b7 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -24,6 +24,7 @@\n #include \"../dir.h\"\n #include \"../chdir-notify.h\"\n #include \"../setup.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../revision.h\"\n@@ -3506,7 +3507,7 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store,\n */\n typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n-\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t const char *target_name,\n \t\t\t\t struct dir_iterator *iter);\n \n static int files_fsck_symref_target(struct fsck_options *o,\n@@ -3514,27 +3515,29 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct strbuf *referent,\n \t\t\t\t unsigned int symbolic_link)\n {\n+\tint is_referent_root;\n \tchar orig_last_byte;\n \tsize_t orig_len;\n \tint ret = 0;\n \n-\tif (!symbolic_link) {\n-\t\torig_len = referent->len;\n-\t\torig_last_byte = referent->buf[orig_len - 1];\n+\torig_len = referent->len;\n+\torig_last_byte = referent->buf[orig_len - 1];\n+\tif (!symbolic_link)\n \t\tstrbuf_rtrim(referent);\n-\t}\n \n-\tif (!starts_with(referent->buf, \"refs/\") &&\n+\tis_referent_root = is_root_ref(referent->buf);\n+\tif (!is_referent_root &&\n+\t !starts_with(referent->buf, \"refs/\") &&\n \t !starts_with(referent->buf, \"worktrees/\")) {\n \t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_ESCAPE_REFERENT,\n-\t\t\t\t \"referent '%s' is outside of refs/ or worktrees/\",\n-\t\t\t\t referent->buf);\n+\t\t\t\t FSCK_MSG_SYMREF_TARGET_IS_NOT_A_REF,\n+\t\t\t\t \"points to non-ref target '%s'\", referent->buf);\n+\n \t}\n \n-\tif (check_refname_format(referent->buf, 0)) {\n+\tif (!is_referent_root && check_refname_format(referent->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_BAD_REFERENT,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n \t\t\t\t \"points to invalid refname '%s'\", referent->buf);\n \t\tgoto out;\n \t}\n@@ -3542,17 +3545,16 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \tif (symbolic_link)\n \t\tgoto out;\n \n-\n \tif (referent->len == orig_len ||\n \t (referent->len < orig_len && orig_last_byte != '\\n')) {\n \t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n+\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n \t\t\t\t \"misses LF at the end\");\n \t}\n \n \tif (referent->len != orig_len && referent->len != orig_len - 1) {\n \t\tret = fsck_report_ref(o, report,\n-\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n+\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n \t\t\t\t \"has trailing whitespaces or newlines\");\n \t}\n \n@@ -3562,13 +3564,12 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n-\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t const char *target_name,\n \t\t\t\t struct dir_iterator *iter)\n {\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n-\tstruct strbuf refname = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n \tconst char *trailing = NULL;\n \tunsigned int type = 0;\n@@ -3576,8 +3577,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstruct object_id oid;\n \tint ret = 0;\n \n-\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir, iter->relative_path);\n-\treport.path = refname.buf;\n+\treport.path = target_name;\n \n \tif (S_ISLNK(iter->st.st_mode)) {\n \t\tconst char* relative_referent_path = NULL;\n@@ -3600,14 +3600,15 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\telse\n \t\t\tstrbuf_addbuf(&referent, &ref_content);\n \n-\t\tret += files_fsck_symref_target(o, &report, &referent, 1);\n+\t\tret |= files_fsck_symref_target(o, &report, &referent, 1);\n \t\tgoto cleanup;\n \t}\n \n \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n-\t\t\t\t \"cannot read ref file\");\n+\t\t\t\t \"cannot read ref file '%s': (%s)\",\n+\t\t\t\t iter->path.buf, strerror(errno));\n \t\tgoto cleanup;\n \t}\n \n@@ -3624,13 +3625,13 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tif (!(type & REF_ISSYMREF)) {\n \t\tif (!*trailing) {\n \t\t\tret = fsck_report_ref(o, &report,\n-\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n \t\t\t\t\t \"misses LF at the end\");\n \t\t\tgoto cleanup;\n \t\t}\n \t\tif (*trailing != '\\n' || *(trailing + 1)) {\n \t\t\tret = fsck_report_ref(o, &report,\n-\t\t\t\t\t FSCK_MSG_UNOFFICIAL_FORMATTED_REF,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n \t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n \t\t\tgoto cleanup;\n \t\t}\n@@ -3640,7 +3641,6 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t}\n \n cleanup:\n-\tstrbuf_release(&refname);\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n \tstrbuf_release(&abs_gitdir);\n@@ -3649,7 +3649,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n-\t\t\t\tconst char *refs_check_dir,\n+\t\t\t\tconst char *target_name,\n \t\t\t\tstruct dir_iterator *iter)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -3662,11 +3662,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \tif (iter->basename[0] != '.' && ends_with(iter->basename, \".lock\"))\n \t\tgoto cleanup;\n \n-\tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n+\tif (check_refname_format(target_name, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n-\t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n-\t\treport.path = sb.buf;\n+\t\treport.path = target_name;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\n \t\t\t\t \"invalid refname format\");\n@@ -3680,8 +3679,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\t struct fsck_options *o,\n \t\t\t const char *refs_check_dir,\n+\t\t\t struct worktree *wt,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n+\tstruct strbuf target_name = STRBUF_INIT;\n \tstruct strbuf sb = STRBUF_INIT;\n \tstruct dir_iterator *iter;\n \tint iter_status;\n@@ -3700,11 +3701,18 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\tcontinue;\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n+\t\t\tstrbuf_reset(&target_name);\n+\n+\t\t\tif (!is_main_worktree(wt))\n+\t\t\t\tstrbuf_addf(&target_name, \"worktrees/%s/\", wt->id);\n+\t\t\tstrbuf_addf(&target_name, \"%s/%s\", refs_check_dir,\n+\t\t\t\t iter->relative_path);\n+\n \t\t\tif (o->verbose)\n-\t\t\t\tfprintf_ln(stdout, \"Checking %s/%s\",\n-\t\t\t\t\t refs_check_dir, iter->relative_path);\n+\t\t\t\tfprintf_ln(stderr, \"Checking %s\", target_name.buf);\n+\n \t\t\tfor (size_t i = 0; fsck_refs_fn[i]; i++) {\n-\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refs_check_dir, iter))\n+\t\t\t\tif (fsck_refs_fn[i](ref_store, o, target_name.buf, iter))\n \t\t\t\t\tret = -1;\n \t\t\t}\n \t\t} else {\n@@ -3721,11 +3729,13 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \n out:\n \tstrbuf_release(&sb);\n+\tstrbuf_release(&target_name);\n \treturn ret;\n }\n \n static int files_fsck_refs(struct ref_store *ref_store,\n-\t\t\t struct fsck_options *o)\n+\t\t\t struct fsck_options *o,\n+\t\t\t struct worktree *wt)\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n@@ -3733,27 +3743,20 @@ static int files_fsck_refs(struct ref_store *ref_store,\n \t\tNULL,\n \t};\n \n-\tfprintf_ln(stdout, _(\"Checking references consistency in %s\"),\n-\t\t ref_store->gitdir);\n-\treturn files_fsck_refs_dir(ref_store, o, \"refs\", fsck_refs_fn);\n+\tif (o->verbose)\n+\t\tfprintf_ln(stderr, _(\"Checking references consistency\"));\n+\treturn files_fsck_refs_dir(ref_store, o, \"refs\", wt, fsck_refs_fn);\n }\n \n static int files_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"fsck\");\n \n-\tint ret = files_fsck_refs(ref_store, o);\n-\n-\t/*\n-\t * packed-refs should only be checked once because it is shared\n-\t * between all worktrees.\n-\t */\n-\tif (!strcmp(ref_store->gitdir, ref_store->repo->gitdir))\n-\t\tret += refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n-\n-\treturn ret;\n+\treturn files_fsck_refs(ref_store, o, wt) |\n+\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o, wt);\n }\n \n struct ref_storage_be refs_be_files = {\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 07c57fd541..46dcaec654 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -13,6 +13,7 @@\n #include \"../lockfile.h\"\n #include \"../chdir-notify.h\"\n #include \"../statinfo.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../trace2.h\"\n@@ -1754,8 +1755,13 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n }\n \n static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED)\n+\t\t struct fsck_options *o UNUSED,\n+\t\t struct worktree *wt)\n {\n+\n+\tif (!is_main_worktree(wt))\n+\t\treturn 0;\n+\n \treturn 0;\n }\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 73b05f971b..125f1fe735 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -653,7 +653,8 @@ typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refnam\n \t\t\t\t struct strbuf *referent);\n \n typedef int fsck_fn(struct ref_store *ref_store,\n-\t\t struct fsck_options *o);\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt);\n \n struct ref_storage_be {\n \tconst char *name;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex f5f957e6de..b6a63c1015 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -2443,7 +2443,8 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n }\n \n static int reftable_be_fsck(struct ref_store *ref_store UNUSED,\n-\t\t\t struct fsck_options *o UNUSED)\n+\t\t\t struct fsck_options *o UNUSED,\n+\t\t\t struct worktree *wt UNUSED)\n {\n \treturn 0;\n }\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex be4c064b3c..aee7e04b82 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '\n \tgit tag tag-2 &&\n \tgit tag multi_hierarchy/tag-2 &&\n \n+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\tEOF\n+\ttest_must_be_empty err &&\n+\trm $branch_dir_prefix/@ &&\n+\n \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n@@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '\n \trm $branch_dir_prefix/.branch-1 &&\n \ttest_cmp expect err &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\'' branch-1'\\'' &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/heads/@: badRefName: invalid refname format\n+\terror: refs/heads/ branch-1: badRefName: invalid refname format\n \tEOF\n-\trm $branch_dir_prefix/@ &&\n+\trm $branch_dir_prefix/'\\'' branch-1'\\'' &&\n \ttest_cmp expect err &&\n \n-\tcp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/@ &&\n+\tcp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/'\\''~tag-2'\\'' &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/tags/multi_hierarchy/@: badRefName: invalid refname format\n+\terror: refs/tags/multi_hierarchy/~tag-2: badRefName: invalid refname format\n \tEOF\n-\trm $tag_dir_prefix/multi_hierarchy/@ &&\n+\trm $tag_dir_prefix/multi_hierarchy/'\\''~tag-2'\\'' &&\n \ttest_cmp expect err &&\n \n \tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock &&\n@@ -60,6 +67,15 @@ test_expect_success 'ref name should be checked' '\n \terror: refs/tags/.lock: badRefName: invalid refname format\n \tEOF\n \trm $tag_dir_prefix/.lock &&\n+\ttest_cmp expect err &&\n+\n+\tmkdir $tag_dir_prefix/'\\''~new-feature'\\'' &&\n+\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/'\\''~new-feature'\\''/tag-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/~new-feature/tag-1: badRefName: invalid refname format\n+\tEOF\n+\trm -rf $tag_dir_prefix/'\\''~new-feature'\\'' &&\n \ttest_cmp expect err\n '\n \n@@ -84,7 +100,7 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \trm $branch_dir_prefix/.branch-1 &&\n \ttest_cmp expect err &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\''~branch-1'\\'' &&\n \tgit -c fsck.badRefName=ignore refs verify 2>err &&\n \ttest_must_be_empty err\n '\n@@ -114,13 +130,13 @@ test_expect_success 'ref name check should work for multiple worktrees' '\n \t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n \t) &&\n \n-\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/.branch-2 &&\n-\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/@ &&\n+\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/worktree/.branch-2: badRefName: invalid refname format\n-\terror: refs/worktree/@: badRefName: invalid refname format\n+\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n \tEOF\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err &&\n@@ -129,8 +145,8 @@ test_expect_success 'ref name check should work for multiple worktrees' '\n \t\tcd worktree-1 &&\n \t\ttest_must_fail git refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/worktree/.branch-2: badRefName: invalid refname format\n-\t\terror: refs/worktree/@: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n \t\tEOF\n \t\tsort err >sorted_err &&\n \t\ttest_cmp expect sorted_err\n@@ -140,8 +156,8 @@ test_expect_success 'ref name check should work for multiple worktrees' '\n \t\tcd worktree-2 &&\n \t\ttest_must_fail git refs verify 2>err &&\n \t\tcat >expect <<-EOF &&\n-\t\terror: refs/worktree/.branch-2: badRefName: invalid refname format\n-\t\terror: refs/worktree/@: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n \t\tEOF\n \t\tsort err >sorted_err &&\n \t\ttest_cmp expect sorted_err\n@@ -190,7 +206,7 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline: unofficialFormattedRef: misses LF at the end\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \trm $branch_dir_prefix/branch-no-newline &&\n \ttest_cmp expect err &&\n@@ -198,7 +214,7 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-garbage: unofficialFormattedRef: has trailing garbage: '\\'' garbage'\\''\n+\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \tEOF\n \trm $branch_dir_prefix/branch-garbage &&\n \ttest_cmp expect err &&\n@@ -206,7 +222,7 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-garbage-1: unofficialFormattedRef: has trailing garbage: '\\''\n+\twarning: refs/tags/tag-garbage-1: trailingRefContent: has trailing garbage: '\\''\n \n \n \t'\\''\n@@ -217,7 +233,7 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-garbage-2: unofficialFormattedRef: has trailing garbage: '\\''\n+\twarning: refs/tags/tag-garbage-2: trailingRefContent: has trailing garbage: '\\''\n \n \n \t garbage'\\''\n@@ -228,16 +244,16 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \tprintf \"%s garbage\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-garbage-3: unofficialFormattedRef: has trailing garbage: '\\'' garbage\n+\twarning: refs/tags/tag-garbage-3: trailingRefContent: has trailing garbage: '\\'' garbage\n \ta'\\''\n \tEOF\n \trm $tag_dir_prefix/tag-garbage-3 &&\n \ttest_cmp expect err &&\n \n \tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n-\ttest_must_fail git -c fsck.unofficialFormattedRef=error refs verify 2>err &&\n+\ttest_must_fail git -c fsck.trailingRefContent=error refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/tags/tag-garbage-4: unofficialFormattedRef: has trailing garbage: '\\'' garbage'\\''\n+\terror: refs/tags/tag-garbage-4: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \tEOF\n \trm $tag_dir_prefix/tag-garbage-4 &&\n \ttest_cmp expect err\n@@ -266,8 +282,8 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n \terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n \terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n-\twarning: refs/heads/branch-garbage: unofficialFormattedRef: has trailing garbage: '\\'' garbage'\\''\n-\twarning: refs/heads/branch-no-newline: unofficialFormattedRef: misses LF at the end\n+\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err\n@@ -287,10 +303,15 @@ test_expect_success 'textual symref content should be checked (individual)' '\n \trm $branch_dir_prefix/branch-good &&\n \ttest_must_be_empty err &&\n \n+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-head &&\n+\ttest_must_be_empty err &&\n+\n \tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-no-newline-1: unofficialFormattedRef: misses LF at the end\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n \tEOF\n \trm $branch_dir_prefix/branch-no-newline-1 &&\n \ttest_cmp expect err &&\n@@ -298,8 +319,8 @@ test_expect_success 'textual symref content should be checked (individual)' '\n \tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-1: unofficialFormattedRef: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: unofficialFormattedRef: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n \tEOF\n \trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n \ttest_cmp expect err &&\n@@ -307,7 +328,7 @@ test_expect_success 'textual symref content should be checked (individual)' '\n \tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-2: unofficialFormattedRef: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n \tEOF\n \trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n \ttest_cmp expect err &&\n@@ -315,7 +336,7 @@ test_expect_success 'textual symref content should be checked (individual)' '\n \tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-trailing-3: unofficialFormattedRef: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n \tEOF\n \trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n \ttest_cmp expect err &&\n@@ -323,8 +344,8 @@ test_expect_success 'textual symref content should be checked (individual)' '\n \tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/a/b/branch-complicated: unofficialFormattedRef: misses LF at the end\n-\twarning: refs/heads/a/b/branch-complicated: unofficialFormattedRef: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n \tEOF\n \trm $branch_dir_prefix/a/b/branch-complicated &&\n \ttest_cmp expect err &&\n@@ -332,7 +353,7 @@ test_expect_success 'textual symref content should be checked (individual)' '\n \tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badReferent: points to invalid refname '\\''refs/heads/.branch'\\''\n+\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n \tEOF\n \trm $branch_dir_prefix/branch-bad-1 &&\n \ttest_cmp expect err\n@@ -348,6 +369,7 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n \tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n \tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n \tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n \tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n@@ -357,20 +379,20 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/heads/branch-bad-1: badReferent: points to invalid refname '\\''refs/heads/.branch'\\''\n-\twarning: refs/heads/a/b/branch-complicated: unofficialFormattedRef: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-complicated: unofficialFormattedRef: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-1: unofficialFormattedRef: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-1: unofficialFormattedRef: misses LF at the end\n-\twarning: refs/heads/a/b/branch-trailing-2: unofficialFormattedRef: has trailing whitespaces or newlines\n-\twarning: refs/heads/a/b/branch-trailing-3: unofficialFormattedRef: has trailing whitespaces or newlines\n-\twarning: refs/heads/branch-no-newline-1: unofficialFormattedRef: misses LF at the end\n+\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n \tEOF\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err\n '\n \n-test_expect_success 'textual symref should be checked whether it is escaped' '\n+test_expect_success 'the target of the textual symref should be checked' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n@@ -379,48 +401,71 @@ test_expect_success 'textual symref should be checked whether it is escaped' '\n \ttest_commit default &&\n \tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-good &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-good &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"ref: refs/foo\\n\" >$branch_dir_prefix/branch-good &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-good &&\n+\ttest_must_be_empty err &&\n+\n \tprintf \"ref: refs-back/heads/main\\n\" >$branch_dir_prefix/branch-bad-1 &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-bad-1: escapeReferent: referent '\\''refs-back/heads/main'\\'' is outside of refs/ or worktrees/\n+\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''refs-back/heads/main'\\''\n \tEOF\n \trm $branch_dir_prefix/branch-bad-1 &&\n \ttest_cmp expect err\n '\n \n-test_expect_success 'textual symref escape check should work with worktrees' '\n+test_expect_success SYMLINKS 'symlink symref content should be checked' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n \tcd repo &&\n \ttest_commit default &&\n-\tgit branch branch-1 &&\n-\tgit branch branch-2 &&\n-\tgit branch branch-3 &&\n-\tgit worktree add ./worktree-1 branch-2 &&\n-\tgit worktree add ./worktree-2 branch-3 &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n \n-\t(\n-\t\tcd worktree-1 &&\n-\t\tgit branch refs/worktree/w1-branch &&\n-\t\tgit symbolic-ref refs/worktree/branch-4 refs/heads/branch-1 &&\n-\t\tgit symbolic-ref refs/worktree/branch-5 worktrees/worktree-2/refs/worktree/w2-branch\n-\t) &&\n-\t(\n-\t\tcd worktree-2 &&\n-\t\tgit branch refs/worktree/w2-branch &&\n-\t\tgit symbolic-ref refs/worktree/branch-4 refs/heads/branch-1 &&\n-\t\tgit symbolic-ref refs/worktree/branch-5 worktrees/worktree-1/refs/worktree/w1-branch\n-\t) &&\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n \n+\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n \n-\tgit symbolic-ref refs/heads/branch-5 worktrees/worktree-1/refs/worktree/w1-branch &&\n-\tgit symbolic-ref refs/heads/branch-6 worktrees/worktree-2/refs/worktree/w2-branch &&\n+\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_cmp expect err &&\n \n-\tgit refs verify 2>err &&\n-\ttest_must_be_empty err\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_cmp expect err\n '\n \n-test_expect_success 'all textual symref checks should work with worktrees' '\n+test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tcd repo &&\n@@ -449,7 +494,7 @@ test_expect_success 'all textual symref checks should work with worktrees' '\n \tprintf \"%s\" $bad_content_1 >$worktree1_refdir_prefix/bad-branch-1 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/worktree/bad-branch-1: badRefContent: $bad_content_1\n+\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content_1\n \tEOF\n \trm $worktree1_refdir_prefix/bad-branch-1 &&\n \ttest_cmp expect err &&\n@@ -457,7 +502,7 @@ test_expect_success 'all textual symref checks should work with worktrees' '\n \tprintf \"%s\" $bad_content_2 >$worktree2_refdir_prefix/bad-branch-2 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/worktree/bad-branch-2: badRefContent: $bad_content_2\n+\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content_2\n \tEOF\n \trm $worktree2_refdir_prefix/bad-branch-2 &&\n \ttest_cmp expect err &&\n@@ -465,7 +510,7 @@ test_expect_success 'all textual symref checks should work with worktrees' '\n \tprintf \"%s\" $bad_content_3 >$worktree1_refdir_prefix/bad-branch-3 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/worktree/bad-branch-3: badRefContent: $bad_content_3\n+\terror: worktrees/worktree-1/refs/worktree/bad-branch-3: badRefContent: $bad_content_3\n \tEOF\n \trm $worktree1_refdir_prefix/bad-branch-3 &&\n \ttest_cmp expect err &&\n@@ -473,61 +518,17 @@ test_expect_success 'all textual symref checks should work with worktrees' '\n \tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/worktree/branch-no-newline: unofficialFormattedRef: misses LF at the end\n+\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \trm $worktree1_refdir_prefix/branch-no-newline &&\n \ttest_cmp expect err &&\n \n-\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree2_refdir_prefix/branch-garbage &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/worktree/branch-garbage: unofficialFormattedRef: has trailing garbage: '\\'' garbage'\\''\n-\tEOF\n-\trm $worktree2_refdir_prefix/branch-garbage\n-'\n-\n-test_expect_success SYMLINKS 'symlink symref content should be checked (individual)' '\n-\ttest_when_finished \"rm -rf repo\" &&\n-\tgit init repo &&\n-\tbranch_dir_prefix=.git/refs/heads &&\n-\ttag_dir_prefix=.git/refs/tags &&\n-\tcd repo &&\n-\ttest_commit default &&\n-\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n-\n-\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n-\tgit refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-good &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n \tgit refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n-\twarning: refs/heads/branch-symbolic: escapeReferent: referent '\\''logs/branch-escape'\\'' is outside of refs/ or worktrees/\n+\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n \tEOF\n-\trm $branch_dir_prefix/branch-symbolic &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\"branch space\" $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/heads/branch-symbolic-bad: badReferent: points to invalid refname '\\''refs/heads/branch space'\\''\n-\tEOF\n-\trm $branch_dir_prefix/branch-symbolic-bad &&\n-\ttest_cmp expect err &&\n-\n-\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n-\terror: refs/tags/tag-symbolic-1: badReferent: points to invalid refname '\\''refs/tags/.tag'\\''\n-\tEOF\n-\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\trm $worktree1_refdir_prefix/branch-garbage &&\n \ttest_cmp expect err\n '\n \n-- \n2.47.0\n\n"},{"id":"505678","messageId":"ZxZYVfgJlk8JxC94@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v6 1/9] ref: initialize \"fsck_ref_report\" with zero","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:34:13Z","receivedAt":"2024-10-21T13:34:11Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n\"referent\" is NULL. So, we need to always initialize these parameters to\nNULL instead of letting them point to anywhere when creating a new\n\"fsck_ref_report\" structure.\n\nThe original code explicitly initializes the \"path\" member in the\n\"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\nmembers in the struct). It is more customary to use \"{ 0 }\" to express\nthat we are 0-initializing everything. In order to align with the\ncodebase, initialize \"fsck_ref_report\" with zero.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0824c0b8a9..03d2503276 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3520,7 +3520,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tstruct fsck_ref_report report = { .path = NULL };\n+\t\tstruct fsck_ref_report report = { 0 };\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n-- \n2.47.0\n\n"},{"id":"505679","messageId":"ZxZYXpuCD2I_3bNh@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v6 2/9] ref: check the full refname instead of basename","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:34:22Z","receivedAt":"2024-10-21T13:34:20Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"files-backend.c::files_fsck_refs_name\", we validate the refname\nformat by using \"check_refname_format\" to check the basename of the\niterator with \"REFNAME_ALLOW_ONELEVEL\" flag.\n\nHowever, this is a bad implementation. Although we doesn't allow a\nsingle \"@\" in \".git\" directory, we do allow \"refs/heads/@\". So, we will\nreport an error wrongly when there is a \"refs/heads/@\" ref by using one\nlevel refname \"@\".\n\nBecause we just check one level refname, we either cannot check the\nother parts of the full refname. And we will ignore the following\nerrors:\n\n \"refs/heads/ new-feature/test\"\n \"refs/heads/~new-feature/test\"\n\nIn order to fix the above problem, enhance \"files_fsck_refs_name\" to use\nthe full name for \"check_refname_format\". Then, replace the tests which\nare related to \"@\" and add tests to exercise the above situations.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 4 ++--\n t/t0602-reffiles-fsck.sh | 30 +++++++++++++++++++++++-------\n 2 files changed, 25 insertions(+), 9 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 03d2503276..f246c92684 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3519,10 +3519,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \tif (iter->basename[0] != '.' && ends_with(iter->basename, \".lock\"))\n \t\tgoto cleanup;\n \n-\tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n+\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n+\tif (check_refname_format(sb.buf, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n-\t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 71a4d1a5ae..0aee377439 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '\n \tgit tag tag-2 &&\n \tgit tag multi_hierarchy/tag-2 &&\n \n+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\tEOF\n+\ttest_must_be_empty err &&\n+\trm $branch_dir_prefix/@ &&\n+\n \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n@@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '\n \trm $branch_dir_prefix/.branch-1 &&\n \ttest_cmp expect err &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\'' branch-1'\\'' &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/heads/@: badRefName: invalid refname format\n+\terror: refs/heads/ branch-1: badRefName: invalid refname format\n \tEOF\n-\trm $branch_dir_prefix/@ &&\n+\trm $branch_dir_prefix/'\\'' branch-1'\\'' &&\n \ttest_cmp expect err &&\n \n-\tcp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/@ &&\n+\tcp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/'\\''~tag-2'\\'' &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n-\terror: refs/tags/multi_hierarchy/@: badRefName: invalid refname format\n+\terror: refs/tags/multi_hierarchy/~tag-2: badRefName: invalid refname format\n \tEOF\n-\trm $tag_dir_prefix/multi_hierarchy/@ &&\n+\trm $tag_dir_prefix/multi_hierarchy/'\\''~tag-2'\\'' &&\n \ttest_cmp expect err &&\n \n \tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock &&\n@@ -60,6 +67,15 @@ test_expect_success 'ref name should be checked' '\n \terror: refs/tags/.lock: badRefName: invalid refname format\n \tEOF\n \trm $tag_dir_prefix/.lock &&\n+\ttest_cmp expect err &&\n+\n+\tmkdir $tag_dir_prefix/'\\''~new-feature'\\'' &&\n+\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/'\\''~new-feature'\\''/tag-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/~new-feature/tag-1: badRefName: invalid refname format\n+\tEOF\n+\trm -rf $tag_dir_prefix/'\\''~new-feature'\\'' &&\n \ttest_cmp expect err\n '\n \n@@ -84,7 +100,7 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \trm $branch_dir_prefix/.branch-1 &&\n \ttest_cmp expect err &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\''~branch-1'\\'' &&\n \tgit -c fsck.badRefName=ignore refs verify 2>err &&\n \ttest_must_be_empty err\n '\n-- \n2.47.0\n\n"},{"id":"505680","messageId":"ZxZYZy-9deyT6I9a@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v6 3/9] ref: initialize target name outside of check functions","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:34:31Z","receivedAt":"2024-10-21T13:34:28Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We passes \"refs_check_dir\" to the \"files_fsck_refs_name\" function which\nallows it to create the checked ref name later. However, when we\nintroduce a new check function, we have to re-calculate the target name.\nIt's bad for us to do repeat calculation. Instead, we should calculate\nit only once and pass the target name to the check functions.\n\nIn order not to do repeat calculation, rename \"refs_check_dir\" to\n\"target_name\". And in \"files_fsck_refs_dir\", create a new strbuf\n\"target_name\", thus whenever we handle a new target, calculate the\nname and call the check functions one by one.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 21 +++++++++++++--------\n 1 file changed, 13 insertions(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f246c92684..fbfcd1115c 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3501,12 +3501,12 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store,\n */\n typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n-\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t const char *target_name,\n \t\t\t\t struct dir_iterator *iter);\n \n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n-\t\t\t\tconst char *refs_check_dir,\n+\t\t\t\tconst char *target_name,\n \t\t\t\tstruct dir_iterator *iter)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -3519,11 +3519,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \tif (iter->basename[0] != '.' && ends_with(iter->basename, \".lock\"))\n \t\tgoto cleanup;\n \n-\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n-\tif (check_refname_format(sb.buf, 0)) {\n+\tif (check_refname_format(target_name, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n-\t\treport.path = sb.buf;\n+\t\treport.path = target_name;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\n \t\t\t\t \"invalid refname format\");\n@@ -3539,6 +3538,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\t const char *refs_check_dir,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n+\tstruct strbuf target_name = STRBUF_INIT;\n \tstruct strbuf sb = STRBUF_INIT;\n \tstruct dir_iterator *iter;\n \tint iter_status;\n@@ -3557,11 +3557,15 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\tcontinue;\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n+\t\t\tstrbuf_reset(&target_name);\n+\t\t\tstrbuf_addf(&target_name, \"%s/%s\", refs_check_dir,\n+\t\t\t\t iter->relative_path);\n+\n \t\t\tif (o->verbose)\n-\t\t\t\tfprintf_ln(stderr, \"Checking %s/%s\",\n-\t\t\t\t\t refs_check_dir, iter->relative_path);\n+\t\t\t\tfprintf_ln(stderr, \"Checking %s\", target_name.buf);\n+\n \t\t\tfor (size_t i = 0; fsck_refs_fn[i]; i++) {\n-\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refs_check_dir, iter))\n+\t\t\t\tif (fsck_refs_fn[i](ref_store, o, target_name.buf, iter))\n \t\t\t\t\tret = -1;\n \t\t\t}\n \t\t} else {\n@@ -3578,6 +3582,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \n out:\n \tstrbuf_release(&sb);\n+\tstrbuf_release(&target_name);\n \treturn ret;\n }\n \n-- \n2.47.0\n\n"},{"id":"505681","messageId":"ZxZYcPwLB5oLTFUo@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v6 4/9] ref: support multiple worktrees check for refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:34:40Z","receivedAt":"2024-10-21T13:34:38Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already set up the infrastructure to check the consistency for\nrefs, but we do not support multiple worktrees. As we decide to add more\nchecks for ref content, we need to set up support for multiple\nworktrees.\n\nBecause each worktree has its own specific refs, instead of just showing\nthe users \"refs/worktree/foo\", we need to display the full name such as\n\"worktrees//refs/worktree/foo\". So we should know the id of the\nworktree to get the full name. Add a new parameter \"struct worktree *\"\nfor \"refs-internal.h::fsck_fn\". Then change the related functions to\nfollow this new interface.\n\nThe \"packed-refs\" only exists in the main worktree, so we should only\ncheck \"packed-refs\" in the main worktree. Use \"is_main_worktree\" method\nto skip checking \"packed-refs\" in \"packed_fsck\" function.\n\nThen, enhance the \"files-backend.c::files_fsck_refs_dir\" function to add\n\"worktree//\" prefix when we are not in the main worktree.\n\nLast, add a new test to check the refname when there are multiple\nworktrees to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 12 ++++++--\n refs.c | 5 ++--\n refs.h | 3 +-\n refs/debug.c | 5 ++--\n refs/files-backend.c | 17 ++++++++----\n refs/packed-backend.c | 8 +++++-\n refs/refs-internal.h | 3 +-\n refs/reftable-backend.c | 3 +-\n t/t0602-reffiles-fsck.sh | 59 ++++++++++++++++++++++++++++++++++++++++\n 9 files changed, 100 insertions(+), 15 deletions(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex 24978a7b7b..886c4ceae3 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -5,6 +5,7 @@\n #include \"parse-options.h\"\n #include \"refs.h\"\n #include \"strbuf.h\"\n+#include \"worktree.h\"\n \n #define REFS_MIGRATE_USAGE \\\n \tN_(\"git refs migrate --ref-format= [--dry-run]\")\n@@ -66,6 +67,7 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)\n static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n {\n \tstruct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;\n+\tstruct worktree **worktrees, **p;\n \tconst char * const verify_usage[] = {\n \t\tREFS_VERIFY_USAGE,\n \t\tNULL,\n@@ -75,7 +77,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \t\tOPT_BOOL(0, \"strict\", &fsck_refs_options.strict, N_(\"enable strict checking\")),\n \t\tOPT_END(),\n \t};\n-\tint ret;\n+\tint ret = 0;\n \n \targc = parse_options(argc, argv, prefix, options, verify_usage, 0);\n \tif (argc)\n@@ -84,9 +86,15 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);\n+\tworktrees = get_worktrees();\n+\tfor (p = worktrees; *p; p++) {\n+\t\tstruct worktree *wt = *p;\n+\t\tret |= refs_fsck(get_worktree_ref_store(wt), &fsck_refs_options, wt);\n+\t}\n+\n \n \tfsck_options_clear(&fsck_refs_options);\n+\tfree_worktrees(worktrees);\n \treturn ret;\n }\n \ndiff --git a/refs.c b/refs.c\nindex 5f729ed412..395a17273c 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -318,9 +318,10 @@ int check_refname_format(const char *refname, int flags)\n \treturn check_or_sanitize_refname(refname, flags, NULL);\n }\n \n-int refs_fsck(struct ref_store *refs, struct fsck_options *o)\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt)\n {\n-\treturn refs->be->fsck(refs, o);\n+\treturn refs->be->fsck(refs, o, wt);\n }\n \n void sanitize_refname_component(const char *refname, struct strbuf *out)\ndiff --git a/refs.h b/refs.h\nindex 108dfc93b3..341d43239c 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -549,7 +549,8 @@ int check_refname_format(const char *refname, int flags);\n * reflogs are consistent, and non-zero otherwise. The errors will be\n * written to stderr.\n */\n-int refs_fsck(struct ref_store *refs, struct fsck_options *o);\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt);\n \n /*\n * Apply the rules from check_refname_format, but mutate the result until it\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 45e2e784a0..72e80ddd6d 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -420,10 +420,11 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,\n }\n \n static int debug_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n-\tint res = drefs->refs->be->fsck(drefs->refs, o);\n+\tint res = drefs->refs->be->fsck(drefs->refs, o, wt);\n \ttrace_printf_key(&trace_refs, \"fsck: %d\\n\", res);\n \treturn res;\n }\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex fbfcd1115c..24ad73faba 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -23,6 +23,7 @@\n #include \"../dir.h\"\n #include \"../chdir-notify.h\"\n #include \"../setup.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../revision.h\"\n@@ -3536,6 +3537,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\t struct fsck_options *o,\n \t\t\t const char *refs_check_dir,\n+\t\t\t struct worktree *wt,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n \tstruct strbuf target_name = STRBUF_INIT;\n@@ -3558,6 +3560,9 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n \t\t\tstrbuf_reset(&target_name);\n+\n+\t\t\tif (!is_main_worktree(wt))\n+\t\t\t\tstrbuf_addf(&target_name, \"worktrees/%s/\", wt->id);\n \t\t\tstrbuf_addf(&target_name, \"%s/%s\", refs_check_dir,\n \t\t\t\t iter->relative_path);\n \n@@ -3587,7 +3592,8 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n }\n \n static int files_fsck_refs(struct ref_store *ref_store,\n-\t\t\t struct fsck_options *o)\n+\t\t\t struct fsck_options *o,\n+\t\t\t struct worktree *wt)\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n@@ -3596,17 +3602,18 @@ static int files_fsck_refs(struct ref_store *ref_store,\n \n \tif (o->verbose)\n \t\tfprintf_ln(stderr, _(\"Checking references consistency\"));\n-\treturn files_fsck_refs_dir(ref_store, o, \"refs\", fsck_refs_fn);\n+\treturn files_fsck_refs_dir(ref_store, o, \"refs\", wt, fsck_refs_fn);\n }\n \n static int files_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"fsck\");\n \n-\treturn files_fsck_refs(ref_store, o) |\n-\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n+\treturn files_fsck_refs(ref_store, o, wt) |\n+\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o, wt);\n }\n \n struct ref_storage_be refs_be_files = {\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 07c57fd541..46dcaec654 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -13,6 +13,7 @@\n #include \"../lockfile.h\"\n #include \"../chdir-notify.h\"\n #include \"../statinfo.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../trace2.h\"\n@@ -1754,8 +1755,13 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n }\n \n static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED)\n+\t\t struct fsck_options *o UNUSED,\n+\t\t struct worktree *wt)\n {\n+\n+\tif (!is_main_worktree(wt))\n+\t\treturn 0;\n+\n \treturn 0;\n }\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 2313c830d8..037d7991cd 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -653,7 +653,8 @@ typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refnam\n \t\t\t\t struct strbuf *referent);\n \n typedef int fsck_fn(struct ref_store *ref_store,\n-\t\t struct fsck_options *o);\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt);\n \n struct ref_storage_be {\n \tconst char *name;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex f5f957e6de..b6a63c1015 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -2443,7 +2443,8 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n }\n \n static int reftable_be_fsck(struct ref_store *ref_store UNUSED,\n-\t\t\t struct fsck_options *o UNUSED)\n+\t\t\t struct fsck_options *o UNUSED,\n+\t\t\t struct worktree *wt UNUSED)\n {\n \treturn 0;\n }\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 0aee377439..6eb1385c50 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -105,4 +105,63 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'ref name check should work for multiple worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\n+\tcd repo &&\n+\ttest_commit initial &&\n+\tgit checkout -b branch-1 &&\n+\ttest_commit second &&\n+\tgit checkout -b branch-2 &&\n+\ttest_commit third &&\n+\tgit checkout -b branch-3 &&\n+\tgit worktree add ./worktree-1 branch-1 &&\n+\tgit worktree add ./worktree-2 branch-2 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\n+\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t) &&\n+\n+\t(\n+\t\tcd worktree-2 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\tEOF\n+\t\tsort err >sorted_err &&\n+\t\ttest_cmp expect sorted_err\n+\t)\n+'\n+\n test_done\n-- \n2.47.0\n\n"},{"id":"505682","messageId":"ZxZYd1qL6LxAc9-Y@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v6 5/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:34:47Z","receivedAt":"2024-10-21T13:34:47Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"git-fsck(1)\" implicitly checks the ref content by passing the\ncallback \"fsck_handle_ref\" to the \"refs.c::refs_for_each_rawref\".\nThen, it will check whether the ref content (eventually \"oid\")\nis valid. If not, it will report the following error to the user.\n\n error: refs/heads/main: invalid sha1 pointer 0000...\n\nAnd it will also report above errors when there are dangling symrefs\nin the repository wrongly. This does not align with the behavior of\nthe \"git symbolic-ref\" command which allows users to create dangling\nsymrefs.\n\nAs we have already introduced the \"git refs verify\" command, we'd better\ncheck the ref content explicitly in the \"git refs verify\" command thus\nlater we could remove these checks in \"git-fsck(1)\" and launch a\nsubprocess to call \"git refs verify\" in \"git-fsck(1)\" to make the\n\"git-fsck(1)\" more clean.\n\nFollowing what \"git-fsck(1)\" does, add a similar check to \"git refs\nverify\". Then add a new fsck error message \"badRefContent(ERROR)\" to\nrepresent that a ref has an invalid content.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/files-backend.c | 43 +++++++++++++\n t/t0602-reffiles-fsck.sh | 117 ++++++++++++++++++++++++++++++++++\n 4 files changed, 164 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 68a2801f15..22c385ea22 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -19,6 +19,9 @@\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n+`badRefContent`::\n+\t(ERROR) A ref has bad content.\n+\n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n \ndiff --git a/fsck.h b/fsck.h\nindex 500b4c04d2..0d99a87911 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -31,6 +31,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 24ad73faba..2861980bdd 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3505,6 +3505,48 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *target_name,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_refs_content(struct ref_store *ref_store,\n+\t\t\t\t struct fsck_options *o,\n+\t\t\t\t const char *target_name,\n+\t\t\t\t struct dir_iterator *iter)\n+{\n+\tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tunsigned int type = 0;\n+\tint failure_errno = 0;\n+\tstruct object_id oid;\n+\tint ret = 0;\n+\n+\treport.path = target_name;\n+\n+\tif (S_ISLNK(iter->st.st_mode))\n+\t\tgoto cleanup;\n+\n+\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"cannot read ref file '%s': (%s)\",\n+\t\t\t\t iter->path.buf, strerror(errno));\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n+\t\t\t\t ref_content.buf, &oid, &referent,\n+\t\t\t\t &type, &failure_errno)) {\n+\t\tstrbuf_rtrim(&ref_content);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"%s\", ref_content.buf);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&ref_content);\n+\tstrbuf_release(&referent);\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n \t\t\t\tconst char *target_name,\n@@ -3597,6 +3639,7 @@ static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n+\t\tfiles_fsck_refs_content,\n \t\tNULL,\n \t};\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 6eb1385c50..29bdd3fc01 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -164,4 +164,121 @@ test_expect_success 'ref name check should work for multiple worktrees' '\n \t)\n '\n \n+test_expect_success 'regular ref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n+\n+\tbad_content=$(git rev-parse main)x &&\n+\tprintf \"%s\" $bad_content >$tag_dir_prefix/tag-bad-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-bad-1: badRefContent: $bad_content\n+\tEOF\n+\trm $tag_dir_prefix/tag-bad-1 &&\n+\ttest_cmp expect err &&\n+\n+\tbad_content=xfsazqfxcadas &&\n+\tprintf \"%s\" $bad_content >$tag_dir_prefix/tag-bad-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-bad-2: badRefContent: $bad_content\n+\tEOF\n+\trm $tag_dir_prefix/tag-bad-2 &&\n+\ttest_cmp expect err &&\n+\n+\tbad_content=Xfsazqfxcadas &&\n+\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-bad &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'regular ref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tbad_content_1=$(git rev-parse main)x &&\n+\tbad_content_2=xfsazqfxcadas &&\n+\tbad_content_3=Xfsazqfxcadas &&\n+\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n+test_expect_success 'ref content checks should work with worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit branch branch-3 &&\n+\tgit worktree add ./worktree-1 branch-2 &&\n+\tgit worktree add ./worktree-2 branch-3 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\n+\tbad_content_1=$(git rev-parse HEAD)x &&\n+\tbad_content_2=xfsazqfxcadas &&\n+\tbad_content_3=Xfsazqfxcadas &&\n+\n+\tprintf \"%s\" $bad_content_1 >$worktree1_refdir_prefix/bad-branch-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content_1\n+\tEOF\n+\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\" $bad_content_2 >$worktree2_refdir_prefix/bad-branch-2 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content_2\n+\tEOF\n+\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\" $bad_content_3 >$worktree1_refdir_prefix/bad-branch-3 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: worktrees/worktree-1/refs/worktree/bad-branch-3: badRefContent: $bad_content_3\n+\tEOF\n+\trm $worktree1_refdir_prefix/bad-branch-3 &&\n+\ttest_cmp expect err\n+'\n+\n test_done\n-- \n2.47.0\n\n"},{"id":"505683","messageId":"ZxZYf-_M4GvzrFpO@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v6 6/9] ref: add more strict checks for regular refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:34:55Z","receivedAt":"2024-10-21T13:34:53Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already used \"parse_loose_ref_contents\" function to check\nwhether the ref content is valid in files backend. However, by\nusing \"parse_loose_ref_contents\", we allow the ref's content to end with\ngarbage or without a newline.\n\nEven though we never create such loose refs ourselves, we have accepted\nsuch loose refs. So, it is entirely possible that some third-party tools\nmay rely on such loose refs being valid. We should not report an error\nfsck message at current. We should notify the users about such\n\"curiously formatted\" loose refs so that adequate care is taken before\nwe decide to tighten the rules in the future.\n\nAnd it's not suitable either to report a warn fsck message to the user.\nWe don't yet want the \"--strict\" flag that controls this bit to end up\ngenerating errors for such weirdly-formatted reference contents, as we\nfirst want to assess whether this retroactive tightening will cause\nissues for any tools out there. It may cause compatibility issues which\nmay break the repository. So, we add the following two fsck infos to\nrepresent the situation where the ref content ends without newline or\nhas trailing garbages:\n\n1. refMissingNewline(INFO): A loose ref that does not end with\n newline(LF).\n2. trailingRefContent(INFO): A loose ref has trailing content.\n\nIt might appear that we can't provide the user with any warnings by\nusing FSCK_INFO. However, in \"fsck.c::fsck_vreport\", we will convert\nFSCK_INFO to FSCK_WARN and we can still warn the user about these\nsituations when using \"git refs verify\" without introducing\ncompatibility issues.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 14 ++++++++\n fsck.h | 2 ++\n refs.c | 2 +-\n refs/files-backend.c | 26 ++++++++++++--\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 67 +++++++++++++++++++++++++++++++++++\n 6 files changed, 108 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 22c385ea22..6db0eaa84a 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -173,6 +173,20 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`refMissingNewline`::\n+\t(INFO) A loose ref that does not end with newline(LF). As\n+\tvalid implementations of Git never created such a loose ref\n+\tfile, it may become an error in the future. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as\n+\twe need to know what tools created such a file.\n+\n+`trailingRefContent`::\n+\t(INFO) A loose ref has trailing content. As valid implementations\n+\tof Git never created such a loose ref file, it may become an\n+\terror in the future. Report to the git@vger.kernel.org mailing\n+\tlist if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \ndiff --git a/fsck.h b/fsck.h\nindex 0d99a87911..b85072df57 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -85,6 +85,8 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs.c b/refs.c\nindex 395a17273c..f88b32a633 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1789,7 +1789,7 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \t}\n \n \tresult = parse_loose_ref_contents(ref_store->repo->hash_algo, content.buf,\n-\t\t\t\t\t oid, referent, type, failure_errno);\n+\t\t\t\t\t oid, referent, type, NULL, failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 2861980bdd..b1fba92e5f 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -569,7 +569,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname,\n \tbuf = sb_contents.buf;\n \n \tret = parse_loose_ref_contents(ref_store->repo->hash_algo, buf,\n-\t\t\t\t oid, referent, type, &myerr);\n+\t\t\t\t oid, referent, type, NULL, &myerr);\n \n out:\n \tif (ret && !myerr)\n@@ -606,7 +606,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno)\n+\t\t\t const char **trailing, int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -628,6 +628,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n+\n+\tif (trailing)\n+\t\t*trailing = p;\n+\n \treturn 0;\n }\n \n@@ -3513,6 +3517,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n+\tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n \tstruct object_id oid;\n@@ -3533,7 +3538,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n \t\t\t\t ref_content.buf, &oid, &referent,\n-\t\t\t\t &type, &failure_errno)) {\n+\t\t\t\t &type, &trailing, &failure_errno)) {\n \t\tstrbuf_rtrim(&ref_content);\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n@@ -3541,6 +3546,21 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (!(type & REF_ISSYMREF)) {\n+\t\tif (!*trailing) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t \"misses LF at the end\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n cleanup:\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 037d7991cd..125f1fe735 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -716,7 +716,7 @@ struct ref_store {\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno);\n+\t\t\t const char **trailing, int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 29bdd3fc01..0418d79c4f 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -201,6 +201,61 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n \tEOF\n \trm $branch_dir_prefix/a/b/branch-bad &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-1: trailingRefContent: has trailing garbage: '\\''\n+\n+\n+\t'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-2: trailingRefContent: has trailing garbage: '\\''\n+\n+\n+\t garbage'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-garbage-3: trailingRefContent: has trailing garbage: '\\'' garbage\n+\ta'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n+\ttest_must_fail git -c fsck.trailingRefContent=error refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/tags/tag-garbage-4: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-garbage-4 &&\n \ttest_cmp expect err\n '\n \n@@ -219,12 +274,16 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n \tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n \tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n \terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n \terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err\n@@ -278,6 +337,14 @@ test_expect_success 'ref content checks should work with worktrees' '\n \terror: worktrees/worktree-1/refs/worktree/bad-branch-3: badRefContent: $bad_content_3\n \tEOF\n \trm $worktree1_refdir_prefix/bad-branch-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-no-newline &&\n \ttest_cmp expect err\n '\n \n-- \n2.47.0\n\n"},{"id":"505684","messageId":"ZxZYh1D3rLGrDZjN@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v6 7/9] ref: add basic symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:35:03Z","receivedAt":"2024-10-21T13:35:01Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have code that checks regular ref contents, but we do not yet check\nthe contents of symbolic refs. By using \"parse_loose_ref_content\" for\nsymbolic refs, we will get the information of the \"referent\".\n\nWe do not need to check the \"referent\" by opening the file. This is\nbecause if \"referent\" exists in the file system, we will eventually\ncheck its correctness by inspecting every file in the \"refs\" directory.\nIf the \"referent\" does not exist in the filesystem, this is OK as it is\nseen as the dangling symref.\n\nSo we just need to check the \"referent\" string content. A regular ref\ncould be accepted as a textual symref if it begins with \"ref:\", followed\nby zero or more whitespaces, followed by the full refname, followed only\nby whitespace characters. However, we always write a single SP after\n\"ref:\" and a single LF after the refname. It may seem that we should\nreport a fsck error message when the \"referent\" does not apply above\nrules and we should not be so aggressive because third-party\nreimplementations of Git may have taken advantage of the looser syntax.\nPut it more specific, we accept the following contents:\n\n1. \"ref: refs/heads/master \"\n2. \"ref: refs/heads/master \\n \\n\"\n3. \"ref: refs/heads/master\\n\\n\"\n\nWhen introducing the regular ref content checks, we created two fsck\ninfos \"refMissingNewline\" and \"trailingRefContent\" which exactly\nrepresents above situations. So we will reuse these two fsck messages to\nwrite checks to info the user about these situations.\n\nBut we do not allow any other trailing garbage. The followings are bad\nsymref contents which will be reported as fsck error by \"git-fsck(1)\".\n\n1. \"ref: refs/heads/master garbage\\n\"\n2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n\nAnd we introduce a new \"badReferentName(ERROR)\" fsck message to report\nabove errors by using \"is_root_ref\" and \"check_refname_format\" to check\nthe \"referent\". Since both \"is_root_ref\" and \"check_refname_format\"\ndon't work with whitespaces, we use the trimmed version of \"referent\"\nwith these functions.\n\nIn order to add checks, we will do the following things:\n\n1. Record the untrimmed length \"orig_len\" and untrimmed last byte\n \"orig_last_byte\".\n2. Use \"strbuf_rtrim\" to trim the whitespaces or newlines to make sure\n \"is_root_ref\" and \"check_refname_format\" won't be failed by them.\n3. Use \"orig_len\" and \"orig_last_byte\" to check whether the \"referent\"\n misses '\\n' at the end or it has trailing whitespaces or newlines.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/files-backend.c | 40 ++++++++++++\n t/t0602-reffiles-fsck.sh | 111 ++++++++++++++++++++++++++++++++++\n 4 files changed, 155 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 6db0eaa84a..dcea05edfc 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,6 +28,9 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n+`badReferentName`::\n+\t(ERROR) The referent name of a symref is invalid.\n+\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n \ndiff --git a/fsck.h b/fsck.h\nindex b85072df57..5227dfdef2 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,6 +34,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n+\tFUNC(BAD_REFERENT_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b1fba92e5f..1a267547f2 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3509,6 +3509,43 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *target_name,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_symref_target(struct fsck_options *o,\n+\t\t\t\t struct fsck_ref_report *report,\n+\t\t\t\t struct strbuf *referent)\n+{\n+\tchar orig_last_byte;\n+\tsize_t orig_len;\n+\tint ret = 0;\n+\n+\torig_len = referent->len;\n+\torig_last_byte = referent->buf[orig_len - 1];\n+\tstrbuf_rtrim(referent);\n+\n+\tif (!is_root_ref(referent->buf) &&\n+\t check_refname_format(referent->buf, 0)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n+\t\t\t\t \"points to invalid refname '%s'\", referent->buf);\n+\t\tgoto out;\n+\t}\n+\n+\tif (referent->len == orig_len ||\n+\t (referent->len < orig_len && orig_last_byte != '\\n')) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t \"misses LF at the end\");\n+\t}\n+\n+\tif (referent->len != orig_len && referent->len != orig_len - 1) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t \"has trailing whitespaces or newlines\");\n+\t}\n+\n+out:\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *target_name,\n@@ -3559,6 +3596,9 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n \t\t\tgoto cleanup;\n \t\t}\n+\t} else {\n+\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tgoto cleanup;\n \t}\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 0418d79c4f..f475966d7b 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -289,6 +289,109 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'textual symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-good &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-head &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-1 &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'textual symref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n@@ -345,6 +448,14 @@ test_expect_success 'ref content checks should work with worktrees' '\n \twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \trm $worktree1_refdir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-garbage &&\n \ttest_cmp expect err\n '\n \n-- \n2.47.0\n\n"},{"id":"505685","messageId":"ZxZYjq64HLhuk_Pf@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v6 8/9] ref: check whether the target of the symref is a ref","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:35:10Z","receivedAt":"2024-10-21T13:35:09Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Ideally, we want to the users use \"git symbolic-ref\" to create symrefs\ninstead of writing raw contents into the filesystem. However, \"git\nsymbolic-ref\" is strict with the refname but not strict with the\nreferent. For example, we can make the \"referent\" located at the\n\"$(gitdir)/logs/aaa\" and manually write the content into this where we\ncan still successfully parse this symref by using \"git rev-parse\".\n\n $ git init repo && cd repo && git commit --allow-empty -mx\n $ git symbolic-ref refs/heads/test logs/aaa\n $ echo $(git rev-parse HEAD) > .git/logs/aaa\n $ git rev-parse test\n\nWe may need to add some restrictions for \"referent\" parameter when using\n\"git symbolic-ref\" to create symrefs because ideally all the\nnonpseudo-refs should be located under the \"refs\" directory and we may\ntighten this in the future.\n\nIn order to tell the user we may tighten the above situation, create\na new fsck message \"symrefTargetIsNotARef\" to notify the user that this\nmay become an error in the future.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 9 +++++++++\n fsck.h | 1 +\n refs/files-backend.c | 14 ++++++++++++--\n t/t0602-reffiles-fsck.sh | 28 ++++++++++++++++++++++++++++\n 4 files changed, 50 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex dcea05edfc..f82ebc58e8 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -183,6 +183,15 @@\n \tgit@vger.kernel.org mailing list if you see this error, as\n \twe need to know what tools created such a file.\n \n+`symrefTargetIsNotARef`::\n+\t(INFO) The target of a symbolic reference points neither to\n+\ta root reference nor to a reference starting with \"refs/\".\n+\tAlthough we allow create a symref pointing to the referent which\n+\tis outside the \"ref\" by using `git symbolic-ref`, we may tighten\n+\tthe rule in the future. Report to the git@vger.kernel.org\n+\tmailing list if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n `trailingRefContent`::\n \t(INFO) A loose ref has trailing content. As valid implementations\n \tof Git never created such a loose ref file, it may become an\ndiff --git a/fsck.h b/fsck.h\nindex 5227dfdef2..53a47612e6 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -87,6 +87,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 1a267547f2..b4912af3b5 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3513,6 +3513,7 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n \t\t\t\t struct strbuf *referent)\n {\n+\tint is_referent_root;\n \tchar orig_last_byte;\n \tsize_t orig_len;\n \tint ret = 0;\n@@ -3521,8 +3522,17 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \torig_last_byte = referent->buf[orig_len - 1];\n \tstrbuf_rtrim(referent);\n \n-\tif (!is_root_ref(referent->buf) &&\n-\t check_refname_format(referent->buf, 0)) {\n+\tis_referent_root = is_root_ref(referent->buf);\n+\tif (!is_referent_root &&\n+\t !starts_with(referent->buf, \"refs/\") &&\n+\t !starts_with(referent->buf, \"worktrees/\")) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_SYMREF_TARGET_IS_NOT_A_REF,\n+\t\t\t\t \"points to non-ref target '%s'\", referent->buf);\n+\n+\t}\n+\n+\tif (!is_referent_root && check_refname_format(referent->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n \t\t\t\t \"points to invalid refname '%s'\", referent->buf);\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex f475966d7b..c6d40ce9a1 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -392,6 +392,34 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'the target of the textual symref should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-good &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-good &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"ref: refs/foo\\n\" >$branch_dir_prefix/branch-good &&\n+\tgit refs verify 2>err &&\n+\trm $branch_dir_prefix/branch-good &&\n+\ttest_must_be_empty err &&\n+\n+\tprintf \"ref: refs-back/heads/main\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''refs-back/heads/main'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-bad-1 &&\n+\ttest_cmp expect err\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-- \n2.47.0\n\n"},{"id":"505686","messageId":"ZxZYlsHBSEhTcsHG@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v6 9/9] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-21T13:35:18Z","receivedAt":"2024-10-21T13:35:16Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Besides the textual symref, we also allow symbolic links as the symref.\nSo, we should also provide the consistency check as what we have done\nfor textual symref. And also we consider deprecating writing the\nsymbolic links. We first need to access whether symbolic links still\nbe used. So, add a new fsck message \"symlinkRef(INFO)\" to tell the\nuser be aware of this information.\n\nWe have already introduced \"files_fsck_symref_target\". We should reuse\nthis function to handle the symrefs which use legacy symbolic links. We\nshould not check the trailing garbage for symbolic refs. Add a new\nparameter \"symbolic_link\" to disable some checks which should only be\nexecuted for textual symrefs.\n\nAnd we need to also generate the \"referent\" parameter for reusing\n\"files_fsck_symref_target\" by the following steps:\n\n1. Use \"strbuf_add_real_path\" to resolve the symlink and get the\n absolute path \"ref_content\" which the symlink ref points to.\n2. Generate the absolute path \"abs_gitdir\" of \"gitdir\" and combine\n \"ref_content\" and \"abs_gitdir\" to extract the relative path\n \"relative_referent_path\".\n3. If \"ref_content\" is outside of \"gitdir\", we just set \"referent\" with\n \"ref_content\". Instead, we set \"referent\" with\n \"relative_referent_path\".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 6 +++++\n fsck.h | 1 +\n refs/files-backend.c | 38 +++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 45 +++++++++++++++++++++++++++++++++++\n 4 files changed, 86 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex f82ebc58e8..b14bc44ca4 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -183,6 +183,12 @@\n \tgit@vger.kernel.org mailing list if you see this error, as\n \twe need to know what tools created such a file.\n \n+`symlinkRef`::\n+\t(INFO) A symbolic link is used as a symref. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as we\n+\tare assessing the feasibility of dropping the support to drop\n+\tcreating symbolic links as symrefs.\n+\n `symrefTargetIsNotARef`::\n \t(INFO) The target of a symbolic reference points neither to\n \ta root reference nor to a reference starting with \"refs/\".\ndiff --git a/fsck.h b/fsck.h\nindex 53a47612e6..a44c231a5f 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -86,6 +86,7 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(SYMLINK_REF, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n \tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b4912af3b5..180f8e28b7 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1,6 +1,7 @@\n #define USE_THE_REPOSITORY_VARIABLE\n \n #include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n #include \"../config.h\"\n #include \"../copy.h\"\n #include \"../environment.h\"\n@@ -3511,7 +3512,8 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n-\t\t\t\t struct strbuf *referent)\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t unsigned int symbolic_link)\n {\n \tint is_referent_root;\n \tchar orig_last_byte;\n@@ -3520,7 +3522,8 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \n \torig_len = referent->len;\n \torig_last_byte = referent->buf[orig_len - 1];\n-\tstrbuf_rtrim(referent);\n+\tif (!symbolic_link)\n+\t\tstrbuf_rtrim(referent);\n \n \tis_referent_root = is_root_ref(referent->buf);\n \tif (!is_referent_root &&\n@@ -3539,6 +3542,9 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n+\tif (symbolic_link)\n+\t\tgoto out;\n+\n \tif (referent->len == orig_len ||\n \t (referent->len < orig_len && orig_last_byte != '\\n')) {\n \t\tret = fsck_report_ref(o, report,\n@@ -3562,6 +3568,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct dir_iterator *iter)\n {\n \tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n \tconst char *trailing = NULL;\n@@ -3572,8 +3579,30 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \treport.path = target_name;\n \n-\tif (S_ISLNK(iter->st.st_mode))\n+\tif (S_ISLNK(iter->st.st_mode)) {\n+\t\tconst char* relative_referent_path = NULL;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_SYMLINK_REF,\n+\t\t\t\t \"use deprecated symbolic link for symref\");\n+\n+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n+\t\tstrbuf_normalize_path(&abs_gitdir);\n+\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n+\t\t\tstrbuf_addch(&abs_gitdir, '/');\n+\n+\t\tstrbuf_add_real_path(&ref_content, iter->path.buf);\n+\t\tskip_prefix(ref_content.buf, abs_gitdir.buf,\n+\t\t\t &relative_referent_path);\n+\n+\t\tif (relative_referent_path)\n+\t\t\tstrbuf_addstr(&referent, relative_referent_path);\n+\t\telse\n+\t\t\tstrbuf_addbuf(&referent, &ref_content);\n+\n+\t\tret |= files_fsck_symref_target(o, &report, &referent, 1);\n \t\tgoto cleanup;\n+\t}\n \n \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n \t\tret = fsck_report_ref(o, &report,\n@@ -3607,13 +3636,14 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\tgoto cleanup;\n \t\t}\n \t} else {\n-\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tret = files_fsck_symref_target(o, &report, &referent, 0);\n \t\tgoto cleanup;\n \t}\n \n cleanup:\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n+\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex c6d40ce9a1..aee7e04b82 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -420,6 +420,51 @@ test_expect_success 'the target of the textual symref should be checked' '\n \ttest_cmp expect err\n '\n \n+test_expect_success SYMLINKS 'symlink symref content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_cmp expect err\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-- \n2.47.0\n\n"},{"id":"505700","messageId":"CAOLa=ZSoB77JvuEJkWLL=eiDTsKysM8sxcYddUEbTSt1LziY1A@mail.gmail.com","threadId":"61943","inReplyTo":"ZxZYXpuCD2I_3bNh@ArchLinux","subject":"Re: [PATCH v6 2/9] ref: check the full refname instead of basename","fromName":"karthik nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-10-21T15:38:02Z","receivedAt":"2024-10-21T15:38:05Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n[snip]\n\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 71a4d1a5ae..0aee377439 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '\n> \tgit tag tag-2 &&\n> \tgit tag multi_hierarchy/tag-2 &&\n>\n> +\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n> +\tgit refs verify 2>err &&\n> +\tcat >expect <<-EOF &&\n> +\tEOF\n> +\ttest_must_be_empty err &&\n> +\trm $branch_dir_prefix/@ &&\n> +\n> \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n> \ttest_must_fail git refs verify 2>err &&\n> \tcat >expect <<-EOF &&\n> @@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '\n> \trm $branch_dir_prefix/.branch-1 &&\n> \ttest_cmp expect err &&\n>\n> -\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n> +\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\'' branch-1'\\'' &&\n\nNit: Here and below we could use ${SQ} instead.\n\n[snip]\n"},{"id":"505701","messageId":"CAOLa=ZQ9b1NVng1=3E8tyGfuASv+FeuNo5cNfh6Pb_xvUok-xw@mail.gmail.com","threadId":"61943","inReplyTo":"ZxZYZy-9deyT6I9a@ArchLinux","subject":"Re: [PATCH v6 3/9] ref: initialize target name outside of check functions","fromName":"karthik nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-10-21T15:49:11Z","receivedAt":"2024-10-21T15:49:12Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n> We passes \"refs_check_dir\" to the \"files_fsck_refs_name\" function which\n> allows it to create the checked ref name later. However, when we\n> introduce a new check function, we have to re-calculate the target name.\n> It's bad for us to do repeat calculation. Instead, we should calculate\n> it only once and pass the target name to the check functions.\n>\n> In order not to do repeat calculation, rename \"refs_check_dir\" to\n> \"target_name\". And in \"files_fsck_refs_dir\", create a new strbuf\n\nNit: Why `target_name` and not simply `target`?\n\n> \"target_name\", thus whenever we handle a new target, calculate the\n> name and call the check functions one by one.\n>\n> Mentored-by: Patrick Steinhardt \n> Mentored-by: Karthik Nayak \n> Signed-off-by: shejialuo \n> ---\n> refs/files-backend.c | 21 +++++++++++++--------\n> 1 file changed, 13 insertions(+), 8 deletions(-)\n>\n\n[snip]\n"},{"id":"505702","messageId":"CAOLa=ZSGuBsLxUaA_gvXrYzR=Abzno5PEMZZD+dAs_smcyoqLg@mail.gmail.com","threadId":"61943","inReplyTo":"ZxZYcPwLB5oLTFUo@ArchLinux","subject":"Re: [PATCH v6 4/9] ref: support multiple worktrees check for refs","fromName":"karthik nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2024-10-21T15:56:30Z","receivedAt":"2024-10-21T15:56:32Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"shejialuo writes:\n\n[snip]\n\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 0aee377439..6eb1385c50 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -105,4 +105,63 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n> \ttest_must_be_empty err\n> '\n>\n> +test_expect_success 'ref name check should work for multiple worktrees' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\n> +\tcd repo &&\n> +\ttest_commit initial &&\n> +\tgit checkout -b branch-1 &&\n> +\ttest_commit second &&\n> +\tgit checkout -b branch-2 &&\n> +\ttest_commit third &&\n> +\tgit checkout -b branch-3 &&\n> +\tgit worktree add ./worktree-1 branch-1 &&\n> +\tgit worktree add ./worktree-2 branch-2 &&\n> +\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n> +\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n> +\n> +\t(\n> +\t\tcd worktree-1 &&\n> +\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n> +\t) &&\n> +\t(\n> +\t\tcd worktree-2 &&\n> +\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n> +\t) &&\n> +\n> +\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n> +\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n> +\n> +\ttest_must_fail git refs verify 2>err &&\n> +\tcat >expect <<-EOF &&\n> +\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n> +\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n> +\tEOF\n> +\tsort err >sorted_err &&\n> +\ttest_cmp expect sorted_err &&\n> +\n> +\t(\n> +\t\tcd worktree-1 &&\n> +\t\ttest_must_fail git refs verify 2>err &&\n> +\t\tcat >expect <<-EOF &&\n> +\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n> +\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n> +\t\tEOF\n> +\t\tsort err >sorted_err &&\n> +\t\ttest_cmp expect sorted_err\n> +\t) &&\n> +\n> +\t(\n> +\t\tcd worktree-2 &&\n> +\t\ttest_must_fail git refs verify 2>err &&\n> +\t\tcat >expect <<-EOF &&\n> +\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n> +\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n> +\t\tEOF\n> +\t\tsort err >sorted_err &&\n> +\t\ttest_cmp expect sorted_err\n> +\t)\n\nThese last three loops are the same, couldn't we loop?\n\nfor dir in \".\" \"worktree-1\" \"worktree-2\"\ndo\n ...\ndone\n\n> +'\n> +\n> test_done\n> --\n> 2.47.0\n"},{"id":"505703","messageId":"ZxZ8yNBZWNEhLgND@nand.local","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"Re: [PATCH v6 0/9] add ref content check for files backend","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2024-10-21T16:09:44Z","receivedAt":"2024-10-21T16:09:46Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Mon, Oct 21, 2024 at 09:32:20PM +0800, shejialuo wrote:\n> Hi All:\n>\n> This new version updates the following things.\n\nI am assuming that this new round was rebased onto the tip of 'master',\nsince I could not apply it on top of its original base\n\n b3d175409d9 (Merge branch 'sj/ref-fsck', 2024-08-16)\n\nIn the future, please indicate when you rebase your series so that I\nknow what the correct base is for that round.\n\nThanks,\nTaylor\n"},{"id":"505704","messageId":"ZxZ+xteOnm0im5vC@nand.local","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"Re: [PATCH v6 0/9] add ref content check for files backend","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2024-10-21T16:18:14Z","receivedAt":"2024-10-21T16:18:17Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Mon, Oct 21, 2024 at 09:32:20PM +0800, shejialuo wrote:\n> shejialuo (9):\n> ref: initialize \"fsck_ref_report\" with zero\n> ref: check the full refname instead of basename\n> ref: initialize target name outside of check functions\n> ref: support multiple worktrees check for refs\n> ref: port git-fsck(1) regular refs check for files backend\n> ref: add more strict checks for regular refs\n> ref: add basic symref content check for files backend\n> ref: check whether the target of the symref is a ref\n> ref: add symlink ref content check for files backend\n>\n> Documentation/fsck-msgids.txt | 35 +++\n> builtin/refs.c | 12 +-\n> fsck.h | 6 +\n> refs.c | 7 +-\n> refs.h | 3 +-\n> refs/debug.c | 5 +-\n> refs/files-backend.c | 187 ++++++++++++--\n> refs/packed-backend.c | 8 +-\n> refs/refs-internal.h | 5 +-\n> refs/reftable-backend.c | 3 +-\n> t/t0602-reffiles-fsck.sh | 457 +++++++++++++++++++++++++++++++++-\n> 11 files changed, 693 insertions(+), 35 deletions(-)\n\nGreat, thanks for the new round. Looking at the inter-diff, it looks\nlike this round also needs a fresh review. I'm catching up on new\nthreads from the weekend, so I'll put this on my review queue. But in\nthe meantime, if your mentors can look at it, that would be much\nappreciated.\n\nThanks,\nTaylor\n"},{"id":"505818","messageId":"ZxePhMFmUd89Ibx0@ArchLinux","threadId":"61943","inReplyTo":"ZxZ8yNBZWNEhLgND@nand.local","subject":"Re: [PATCH v6 0/9] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-22T11:41:56Z","receivedAt":"2024-10-22T11:41:52Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 21, 2024 at 12:09:44PM -0400, Taylor Blau wrote:\n> On Mon, Oct 21, 2024 at 09:32:20PM +0800, shejialuo wrote:\n> > Hi All:\n> >\n> > This new version updates the following things.\n> \n> I am assuming that this new round was rebased onto the tip of 'master',\n> since I could not apply it on top of its original base\n> \n> b3d175409d9 (Merge branch 'sj/ref-fsck', 2024-08-16)\n> \n> In the future, please indicate when you rebase your series so that I\n> know what the correct base is for that round.\n> \n\nSorry for that Taylor. I have told Junio that I rebased the series in\nthe previous version. And I forgot you have become the intermediate\nmaintainer and didn't provide this information for you.\n\nThanks,\nJiauo\n\n> Thanks,\n> Taylor\n"},{"id":"505819","messageId":"ZxePqCgV3RQDtwe-@ArchLinux","threadId":"61943","inReplyTo":"CAOLa=ZSoB77JvuEJkWLL=eiDTsKysM8sxcYddUEbTSt1LziY1A@mail.gmail.com","subject":"Re: [PATCH v6 2/9] ref: check the full refname instead of basename","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-22T11:42:32Z","receivedAt":"2024-10-22T11:42:28Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 21, 2024 at 10:38:02AM -0500, karthik nayak wrote:\n> shejialuo writes:\n> \n> [snip]\n> \n> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> > index 71a4d1a5ae..0aee377439 100755\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '\n> > \tgit tag tag-2 &&\n> > \tgit tag multi_hierarchy/tag-2 &&\n> >\n> > +\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n> > +\tgit refs verify 2>err &&\n> > +\tcat >expect <<-EOF &&\n> > +\tEOF\n> > +\ttest_must_be_empty err &&\n> > +\trm $branch_dir_prefix/@ &&\n> > +\n> > \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n> > \ttest_must_fail git refs verify 2>err &&\n> > \tcat >expect <<-EOF &&\n> > @@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '\n> > \trm $branch_dir_prefix/.branch-1 &&\n> > \ttest_cmp expect err &&\n> >\n> > -\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n> > +\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\'' branch-1'\\'' &&\n> \n> Nit: Here and below we could use ${SQ} instead.\n> \n\nI agree.\n\n> [snip]\n\n\n"},{"id":"505820","messageId":"ZxeQCW6iP0x1P1o4@ArchLinux","threadId":"61943","inReplyTo":"CAOLa=ZSGuBsLxUaA_gvXrYzR=Abzno5PEMZZD+dAs_smcyoqLg@mail.gmail.com","subject":"Re: [PATCH v6 4/9] ref: support multiple worktrees check for refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-10-22T11:44:09Z","receivedAt":"2024-10-22T11:44:05Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Mon, Oct 21, 2024 at 10:56:30AM -0500, karthik nayak wrote:\n> shejialuo writes:\n> \n> [snip]\n> \n> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> > index 0aee377439..6eb1385c50 100755\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -105,4 +105,63 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n> > \ttest_must_be_empty err\n> > '\n> >\n> > +test_expect_success 'ref name check should work for multiple worktrees' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\n> > +\tcd repo &&\n> > +\ttest_commit initial &&\n> > +\tgit checkout -b branch-1 &&\n> > +\ttest_commit second &&\n> > +\tgit checkout -b branch-2 &&\n> > +\ttest_commit third &&\n> > +\tgit checkout -b branch-3 &&\n> > +\tgit worktree add ./worktree-1 branch-1 &&\n> > +\tgit worktree add ./worktree-2 branch-2 &&\n> > +\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n> > +\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n> > +\n> > +\t(\n> > +\t\tcd worktree-1 &&\n> > +\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n> > +\t) &&\n> > +\t(\n> > +\t\tcd worktree-2 &&\n> > +\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n> > +\t) &&\n> > +\n> > +\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n> > +\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n> > +\n> > +\ttest_must_fail git refs verify 2>err &&\n> > +\tcat >expect <<-EOF &&\n> > +\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n> > +\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n> > +\tEOF\n> > +\tsort err >sorted_err &&\n> > +\ttest_cmp expect sorted_err &&\n> > +\n> > +\t(\n> > +\t\tcd worktree-1 &&\n> > +\t\ttest_must_fail git refs verify 2>err &&\n> > +\t\tcat >expect <<-EOF &&\n> > +\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n> > +\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n> > +\t\tEOF\n> > +\t\tsort err >sorted_err &&\n> > +\t\ttest_cmp expect sorted_err\n> > +\t) &&\n> > +\n> > +\t(\n> > +\t\tcd worktree-2 &&\n> > +\t\ttest_must_fail git refs verify 2>err &&\n> > +\t\tcat >expect <<-EOF &&\n> > +\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n> > +\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n> > +\t\tEOF\n> > +\t\tsort err >sorted_err &&\n> > +\t\ttest_cmp expect sorted_err\n> > +\t)\n> \n> These last three loops are the same, couldn't we loop?\n> \n> for dir in \".\" \"worktree-1\" \"worktree-2\"\n> do\n> ...\n> done\n> \n\nActually, I guess all the tests could be written with that way. I need\nto refactor in the next version to make the tests cleaner.\n\nThanks,\nJialuo\n"},{"id":"506636","messageId":"ZynFKPFcNQILce3E@pks.im","threadId":"61943","inReplyTo":"ZxZYXpuCD2I_3bNh@ArchLinux","subject":"Re: [PATCH v6 2/9] ref: check the full refname instead of basename","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-05T07:11:42Z","receivedAt":"2024-11-05T07:11:53Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 21, 2024 at 09:34:22PM +0800, shejialuo wrote:\n> In \"files-backend.c::files_fsck_refs_name\", we validate the refname\n> format by using \"check_refname_format\" to check the basename of the\n> iterator with \"REFNAME_ALLOW_ONELEVEL\" flag.\n> \n> However, this is a bad implementation. Although we doesn't allow a\n> single \"@\" in \".git\" directory, we do allow \"refs/heads/@\". So, we will\n> report an error wrongly when there is a \"refs/heads/@\" ref by using one\n> level refname \"@\".\n> \n> Because we just check one level refname, we either cannot check the\n> other parts of the full refname. And we will ignore the following\n> errors:\n> \n> \"refs/heads/ new-feature/test\"\n> \"refs/heads/~new-feature/test\"\n> \n> In order to fix the above problem, enhance \"files_fsck_refs_name\" to use\n> the full name for \"check_refname_format\". Then, replace the tests which\n> are related to \"@\" and add tests to exercise the above situations.\n\nOkay, makes sense.\n\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 03d2503276..f246c92684 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3519,10 +3519,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n> \tif (iter->basename[0] != '.' && ends_with(iter->basename, \".lock\"))\n> \t\tgoto cleanup;\n> \n> -\tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n> +\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n> +\tif (check_refname_format(sb.buf, 0)) {\n> \t\tstruct fsck_ref_report report = { 0 };\n> \n> -\t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n> \t\treport.path = sb.buf;\n> \t\tret = fsck_report_ref(o, &report,\n> \t\t\t\t FSCK_MSG_BAD_REF_NAME,\n\nSo this only works right now because we never check root refs in the\nfirst place? Maybe that is worth a comment.\n\n> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> index 71a4d1a5ae..0aee377439 100755\n> --- a/t/t0602-reffiles-fsck.sh\n> +++ b/t/t0602-reffiles-fsck.sh\n> @@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '\n> \tgit tag tag-2 &&\n> \tgit tag multi_hierarchy/tag-2 &&\n> \n> +\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n> +\tgit refs verify 2>err &&\n> +\tcat >expect <<-EOF &&\n> +\tEOF\n> +\ttest_must_be_empty err &&\n> +\trm $branch_dir_prefix/@ &&\n\n`expect` isn't used here as you use `test_must_be_empty`.\n\n> \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n> \ttest_must_fail git refs verify 2>err &&\n> \tcat >expect <<-EOF &&\n> @@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '\n> \trm $branch_dir_prefix/.branch-1 &&\n> \ttest_cmp expect err &&\n> \n> -\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n> +\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\'' branch-1'\\'' &&\n> \ttest_must_fail git refs verify 2>err &&\n> \tcat >expect <<-EOF &&\n> -\terror: refs/heads/@: badRefName: invalid refname format\n> +\terror: refs/heads/ branch-1: badRefName: invalid refname format\n> \tEOF\n> -\trm $branch_dir_prefix/@ &&\n> +\trm $branch_dir_prefix/'\\'' branch-1'\\'' &&\n> \ttest_cmp expect err &&\n\nOkay, we now allow `refs/heads/@`, but still don't allow other bad\nformatting like spaces in the refname.\n\nPatrick\n"},{"id":"506637","messageId":"ZynFMnYgpCbYwQOs@pks.im","threadId":"61943","inReplyTo":"ZxZYZy-9deyT6I9a@ArchLinux","subject":"Re: [PATCH v6 3/9] ref: initialize target name outside of check functions","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-05T07:11:46Z","receivedAt":"2024-11-05T07:11:54Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 21, 2024 at 09:34:31PM +0800, shejialuo wrote:\n> We passes \"refs_check_dir\" to the \"files_fsck_refs_name\" function which\n> allows it to create the checked ref name later. However, when we\n> introduce a new check function, we have to re-calculate the target name.\n> It's bad for us to do repeat calculation. Instead, we should calculate\n> it only once and pass the target name to the check functions.\n\nIt would be nice to clarify what exactly is bad about it. Does it create\nextra memory churn? Or is this about not duplicating logic?\n\n> In order not to do repeat calculation, rename \"refs_check_dir\" to\n> \"target_name\". And in \"files_fsck_refs_dir\", create a new strbuf\n> \"target_name\", thus whenever we handle a new target, calculate the\n> name and call the check functions one by one.\n\n\"target_name\" is somewhat of a weird name. I'd expect that this is\neither the path to the reference, in which case I'd call this \"path\", or\nthe name of the reference that is to be checked, in which case I'd call\nthis \"refname\".\n\n> @@ -3539,6 +3538,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n> \t\t\t const char *refs_check_dir,\n> \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n> {\n> +\tstruct strbuf target_name = STRBUF_INIT;\n> \tstruct strbuf sb = STRBUF_INIT;\n> \tstruct dir_iterator *iter;\n> \tint iter_status;\n> @@ -3557,11 +3557,15 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n> \t\t\tcontinue;\n> \t\t} else if (S_ISREG(iter->st.st_mode) ||\n> \t\t\t S_ISLNK(iter->st.st_mode)) {\n> +\t\t\tstrbuf_reset(&target_name);\n> +\t\t\tstrbuf_addf(&target_name, \"%s/%s\", refs_check_dir,\n> +\t\t\t\t iter->relative_path);\n> +\n> \t\t\tif (o->verbose)\n> -\t\t\t\tfprintf_ln(stderr, \"Checking %s/%s\",\n> -\t\t\t\t\t refs_check_dir, iter->relative_path);\n> +\t\t\t\tfprintf_ln(stderr, \"Checking %s\", target_name.buf);\n> +\n> \t\t\tfor (size_t i = 0; fsck_refs_fn[i]; i++) {\n> -\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refs_check_dir, iter))\n> +\t\t\t\tif (fsck_refs_fn[i](ref_store, o, target_name.buf, iter))\n> \t\t\t\t\tret = -1;\n> \t\t\t}\n> \t\t} else {\n\nThe change itself does make sense though. We indeed avoid reallocating\nthe array for every single ref, which is a worthwhile change.\n\nI was wondering whether we could reuse `sb` here, but we do use it at\nthe end of the function to potentially print an error message.\n\nPatrick\n"},{"id":"506638","messageId":"ZynFNQ8SnvTJlVdN@pks.im","threadId":"61943","inReplyTo":"ZxZYcPwLB5oLTFUo@ArchLinux","subject":"Re: [PATCH v6 4/9] ref: support multiple worktrees check for refs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-05T07:11:49Z","receivedAt":"2024-11-05T07:11:58Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 21, 2024 at 09:34:40PM +0800, shejialuo wrote:\n> We have already set up the infrastructure to check the consistency for\n> refs, but we do not support multiple worktrees. As we decide to add more\n> checks for ref content, we need to set up support for multiple\n> worktrees.\n\nI don't quite follow that logic: the fact that we perform more checks\nfor the ref content doesn't necessarily mean that we also have to check\nworktree refs. We rather want to do that so that we get feature parity\nwith git-fsck(1) eventually, don't we?\n\n> @@ -66,6 +67,7 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)\n> static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n> {\n> \tstruct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;\n> +\tstruct worktree **worktrees, **p;\n> \tconst char * const verify_usage[] = {\n> \t\tREFS_VERIFY_USAGE,\n> \t\tNULL,\n\nInstead of declaring the `**p` variable we can instead...\n\n> @@ -84,9 +86,15 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n> \tgit_config(git_fsck_config, &fsck_refs_options);\n> \tprepare_repo_settings(the_repository);\n> \n> -\tret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);\n> +\tworktrees = get_worktrees();\n> +\tfor (p = worktrees; *p; p++) {\n> +\t\tstruct worktree *wt = *p;\n> +\t\tret |= refs_fsck(get_worktree_ref_store(wt), &fsck_refs_options, wt);\n> +\t}\n> +\n\n... refactor this loop like this:\n\n for (size_t i = 0; worktrees[i]; i++)\n ret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n &fsck_refs_options, worktrees[i]);\n\nI was briefly wondering whether we also get worktrees in case the repo\nis bare, as we don't actually have a proper worktree there. But the\nanswer seems to be \"yes\".\n\n> @@ -3558,6 +3560,9 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n> \t\t} else if (S_ISREG(iter->st.st_mode) ||\n> \t\t\t S_ISLNK(iter->st.st_mode)) {\n> \t\t\tstrbuf_reset(&target_name);\n> +\n> +\t\t\tif (!is_main_worktree(wt))\n> +\t\t\t\tstrbuf_addf(&target_name, \"worktrees/%s/\", wt->id);\n> \t\t\tstrbuf_addf(&target_name, \"%s/%s\", refs_check_dir,\n> \t\t\t\t iter->relative_path);\n> \n\nHm. Isn't it somewhat duplicate to pass both the prepended target name\n_and_ the worktree to the callback? I imagine that we'd have to\neventually strip the worktree prefix to find the correct ref, unless we\nend up using the main ref store to look up the ref.\n\n> diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> index 07c57fd541..46dcaec654 100644\n> --- a/refs/packed-backend.c\n> +++ b/refs/packed-backend.c\n> @@ -13,6 +13,7 @@\n> #include \"../lockfile.h\"\n> #include \"../chdir-notify.h\"\n> #include \"../statinfo.h\"\n> +#include \"../worktree.h\"\n> #include \"../wrapper.h\"\n> #include \"../write-or-die.h\"\n> #include \"../trace2.h\"\n> @@ -1754,8 +1755,13 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> }\n> \n> static int packed_fsck(struct ref_store *ref_store UNUSED,\n> -\t\t struct fsck_options *o UNUSED)\n> +\t\t struct fsck_options *o UNUSED,\n> +\t\t struct worktree *wt)\n> {\n> +\n> +\tif (!is_main_worktree(wt))\n> +\t\treturn 0;\n> +\n> \treturn 0;\n> }\n\nIt's somewhat funny to have this condition here, but it does make sense\noverall as worktrees never have packed refs in the first place.\n\nPatrick\n"},{"id":"506639","messageId":"ZynFOUSUXAtTtWTk@pks.im","threadId":"61943","inReplyTo":"ZxZYd1qL6LxAc9-Y@ArchLinux","subject":"Re: [PATCH v6 5/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-05T07:11:53Z","receivedAt":"2024-11-05T07:12:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 21, 2024 at 09:34:47PM +0800, shejialuo wrote:\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 24ad73faba..2861980bdd 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3505,6 +3505,48 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> \t\t\t\t const char *target_name,\n> \t\t\t\t struct dir_iterator *iter);\n> \n> +static int files_fsck_refs_content(struct ref_store *ref_store,\n> +\t\t\t\t struct fsck_options *o,\n> +\t\t\t\t const char *target_name,\n> +\t\t\t\t struct dir_iterator *iter)\n> +{\n> +\tstruct strbuf ref_content = STRBUF_INIT;\n> +\tstruct strbuf referent = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = { 0 };\n> +\tunsigned int type = 0;\n> +\tint failure_errno = 0;\n> +\tstruct object_id oid;\n> +\tint ret = 0;\n> +\n> +\treport.path = target_name;\n> +\n> +\tif (S_ISLNK(iter->st.st_mode))\n> +\t\tgoto cleanup;\n> +\n> +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> +\t\t\t\t \"cannot read ref file '%s': (%s)\",\n> +\t\t\t\t iter->path.buf, strerror(errno));\n> +\t\tgoto cleanup;\n> +\t}\n\nLet's drop the braces around `(%s)`, we don't print such braces in\n`warning_errno()` or `die_errno()`, either.\n\nPatrick\n"},{"id":"506659","messageId":"ZyoVA-p4JXPaKTny@ArchLinux","threadId":"61943","inReplyTo":"ZynFNQ8SnvTJlVdN@pks.im","subject":"Re: [PATCH v6 4/9] ref: support multiple worktrees check for refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-05T12:52:19Z","receivedAt":"2024-11-05T12:52:00Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Nov 05, 2024 at 08:11:49AM +0100, Patrick Steinhardt wrote:\n> On Mon, Oct 21, 2024 at 09:34:40PM +0800, shejialuo wrote:\n> > We have already set up the infrastructure to check the consistency for\n> > refs, but we do not support multiple worktrees. As we decide to add more\n> > checks for ref content, we need to set up support for multiple\n> > worktrees.\n> \n> I don't quite follow that logic: the fact that we perform more checks\n> for the ref content doesn't necessarily mean that we also have to check\n> worktree refs. We rather want to do that so that we get feature parity\n> with git-fsck(1) eventually, don't we?\n> \n\nYes, I agree. I come across why I wrote such message. Actually, in the\nvery early implementation, I didn't consider about worktree situation\nfor the \"escape\". And I thought I should add support for worktree. So, I\nmade a mistake.\n\n[snip]\n\n> > @@ -3558,6 +3560,9 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n> > \t\t} else if (S_ISREG(iter->st.st_mode) ||\n> > \t\t\t S_ISLNK(iter->st.st_mode)) {\n> > \t\t\tstrbuf_reset(&target_name);\n> > +\n> > +\t\t\tif (!is_main_worktree(wt))\n> > +\t\t\t\tstrbuf_addf(&target_name, \"worktrees/%s/\", wt->id);\n> > \t\t\tstrbuf_addf(&target_name, \"%s/%s\", refs_check_dir,\n> > \t\t\t\t iter->relative_path);\n> > \n> \n> Hm. Isn't it somewhat duplicate to pass both the prepended target name\n> _and_ the worktree to the callback? I imagine that we'd have to\n> eventually strip the worktree prefix to find the correct ref, unless we\n> end up using the main ref store to look up the ref.\n> \n\nActually, the worktree won't be passed to the callback. The\n`fsck_refs_fn` function will never use worktree `wt`. The reason why I\nuse `wt` is that we need to print _full_ path information to the user\nwhen error happens for the situation where worktree A and worktree B has\nthe same ref name \"refs/worktree/foo\".\n\nI agree that we will strip the worktree prefix to find the correct ref\nin the file system. This is done by the following statement:\n\n\tstrbuf_addf(&sb, \"%s/%s\", ref_store->gitdir, refs_check_dir);\n\nFor worktree, `ref_store->gitdir` will automatically be\n`.git/worktrees/`.\n\nIn the v5, I didn't print the full path and we even didn't need the\nparameter `wt`. However, if we want to print the following info:\n\n\tworktrees//refs/worktree/a\n\nSo, just because we need the `worktrees/` information. Actually, we\ncould also get the information by using \"ref_store->gitdir\" and\n\"ref_store->repo->gitdir\". However, this is cumbersome and it's a bad\nidea. So I change the prototype of \"fsck_fn\" to add a new parameter\n\"struct worktree *\".\n\n> > diff --git a/refs/packed-backend.c b/refs/packed-backend.c\n> > index 07c57fd541..46dcaec654 100644\n> > --- a/refs/packed-backend.c\n> > +++ b/refs/packed-backend.c\n> > @@ -13,6 +13,7 @@\n> > #include \"../lockfile.h\"\n> > #include \"../chdir-notify.h\"\n> > #include \"../statinfo.h\"\n> > +#include \"../worktree.h\"\n> > #include \"../wrapper.h\"\n> > #include \"../write-or-die.h\"\n> > #include \"../trace2.h\"\n> > @@ -1754,8 +1755,13 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n> > }\n> > \n> > static int packed_fsck(struct ref_store *ref_store UNUSED,\n> > -\t\t struct fsck_options *o UNUSED)\n> > +\t\t struct fsck_options *o UNUSED,\n> > +\t\t struct worktree *wt)\n> > {\n> > +\n> > +\tif (!is_main_worktree(wt))\n> > +\t\treturn 0;\n> > +\n> > \treturn 0;\n> > }\n> \n> It's somewhat funny to have this condition here, but it does make sense\n> overall as worktrees never have packed refs in the first place.\n> \n\nYes, there is no packed-refs in the worktree. And we need to prevent\ncalling multiple times.\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"506701","messageId":"ZysN4GIc9JELkIKS@pks.im","threadId":"61943","inReplyTo":"ZyoVA-p4JXPaKTny@ArchLinux","subject":"Re: [PATCH v6 4/9] ref: support multiple worktrees check for refs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-06T06:34:08Z","receivedAt":"2024-11-06T06:34:20Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Nov 05, 2024 at 08:52:19PM +0800, shejialuo wrote:\n> On Tue, Nov 05, 2024 at 08:11:49AM +0100, Patrick Steinhardt wrote:\n> > On Mon, Oct 21, 2024 at 09:34:40PM +0800, shejialuo wrote:\n> > > @@ -3558,6 +3560,9 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n> > > \t\t} else if (S_ISREG(iter->st.st_mode) ||\n> > > \t\t\t S_ISLNK(iter->st.st_mode)) {\n> > > \t\t\tstrbuf_reset(&target_name);\n> > > +\n> > > +\t\t\tif (!is_main_worktree(wt))\n> > > +\t\t\t\tstrbuf_addf(&target_name, \"worktrees/%s/\", wt->id);\n> > > \t\t\tstrbuf_addf(&target_name, \"%s/%s\", refs_check_dir,\n> > > \t\t\t\t iter->relative_path);\n> > > \n> > \n> > Hm. Isn't it somewhat duplicate to pass both the prepended target name\n> > _and_ the worktree to the callback? I imagine that we'd have to\n> > eventually strip the worktree prefix to find the correct ref, unless we\n> > end up using the main ref store to look up the ref.\n> > \n> \n> Actually, the worktree won't be passed to the callback. The\n> `fsck_refs_fn` function will never use worktree `wt`. The reason why I\n> use `wt` is that we need to print _full_ path information to the user\n> when error happens for the situation where worktree A and worktree B has\n> the same ref name \"refs/worktree/foo\".\n> \n> I agree that we will strip the worktree prefix to find the correct ref\n> in the file system. This is done by the following statement:\n> \n> \tstrbuf_addf(&sb, \"%s/%s\", ref_store->gitdir, refs_check_dir);\n> \n> For worktree, `ref_store->gitdir` will automatically be\n> `.git/worktrees/`.\n> \n> In the v5, I didn't print the full path and we even didn't need the\n> parameter `wt`. However, if we want to print the following info:\n> \n> \tworktrees//refs/worktree/a\n> \n> So, just because we need the `worktrees/` information. Actually, we\n> could also get the information by using \"ref_store->gitdir\" and\n> \"ref_store->repo->gitdir\". However, this is cumbersome and it's a bad\n> idea. So I change the prototype of \"fsck_fn\" to add a new parameter\n> \"struct worktree *\".\n\nIn practice you can also derive that full refname from the worktree\nitself, as the ID is stored in `struct worktree::id`. Would that maybe\nbe a better solution?\n\nPatrick\n"},{"id":"506713","messageId":"ZytfBSz56ocwjjWq@ArchLinux","threadId":"61943","inReplyTo":"ZysN4GIc9JELkIKS@pks.im","subject":"Re: [PATCH v6 4/9] ref: support multiple worktrees check for refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-06T12:20:21Z","receivedAt":"2024-11-06T12:19:59Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Nov 06, 2024 at 07:34:08AM +0100, Patrick Steinhardt wrote:\n\n[snip]\n\n> \n> In practice you can also derive that full refname from the worktree\n> itself, as the ID is stored in `struct worktree::id`. Would that maybe\n> be a better solution?\n> \n\nI think we are on the same boat. This is exactly what I have done in\nthis patch.\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"506714","messageId":"Zyth0-rUAj83Rz6F@ArchLinux","threadId":"61943","inReplyTo":"ZynFMnYgpCbYwQOs@pks.im","subject":"Re: [PATCH v6 3/9] ref: initialize target name outside of check functions","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-06T12:32:19Z","receivedAt":"2024-11-06T12:31:58Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Nov 05, 2024 at 08:11:46AM +0100, Patrick Steinhardt wrote:\n> On Mon, Oct 21, 2024 at 09:34:31PM +0800, shejialuo wrote:\n> > We passes \"refs_check_dir\" to the \"files_fsck_refs_name\" function which\n> > allows it to create the checked ref name later. However, when we\n> > introduce a new check function, we have to re-calculate the target name.\n> > It's bad for us to do repeat calculation. Instead, we should calculate\n> > it only once and pass the target name to the check functions.\n> \n> It would be nice to clarify what exactly is bad about it. Does it create\n> extra memory churn? Or is this about not duplicating logic?\n> \n\nThanks, I will improve this in the next version.\n\n> > In order not to do repeat calculation, rename \"refs_check_dir\" to\n> > \"target_name\". And in \"files_fsck_refs_dir\", create a new strbuf\n> > \"target_name\", thus whenever we handle a new target, calculate the\n> > name and call the check functions one by one.\n> \n> \"target_name\" is somewhat of a weird name. I'd expect that this is\n> either the path to the reference, in which case I'd call this \"path\", or\n> the name of the reference that is to be checked, in which case I'd call\n> this \"refname\".\n> \n\nI felt quite hard to name this variable when I wrote the code. \"refname\"\nis not suitable due to we may check the reflog later by calling\n\"files_fsck_refs_dir\" function.\n\nSo, we should use \"path\" here.\n\nThanks,\nJialuo\n"},{"id":"506715","messageId":"Zyti_WrBfP8wA2ny@ArchLinux","threadId":"61943","inReplyTo":"ZynFKPFcNQILce3E@pks.im","subject":"Re: [PATCH v6 2/9] ref: check the full refname instead of basename","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-06T12:37:17Z","receivedAt":"2024-11-06T12:36:55Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Tue, Nov 05, 2024 at 08:11:42AM +0100, Patrick Steinhardt wrote:\n\n[snip]\n\n> > diff --git a/refs/files-backend.c b/refs/files-backend.c\n> > index 03d2503276..f246c92684 100644\n> > --- a/refs/files-backend.c\n> > +++ b/refs/files-backend.c\n> > @@ -3519,10 +3519,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n> > \tif (iter->basename[0] != '.' && ends_with(iter->basename, \".lock\"))\n> > \t\tgoto cleanup;\n> > \n> > -\tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n> > +\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n> > +\tif (check_refname_format(sb.buf, 0)) {\n> > \t\tstruct fsck_ref_report report = { 0 };\n> > \n> > -\t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n> > \t\treport.path = sb.buf;\n> > \t\tret = fsck_report_ref(o, &report,\n> > \t\t\t\t FSCK_MSG_BAD_REF_NAME,\n> \n> So this only works right now because we never check root refs in the\n> first place? Maybe that is worth a comment.\n> \n\nYes, I agree. I will improve this in the next version.\n\n> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\n> > index 71a4d1a5ae..0aee377439 100755\n> > --- a/t/t0602-reffiles-fsck.sh\n> > +++ b/t/t0602-reffiles-fsck.sh\n> > @@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '\n> > \tgit tag tag-2 &&\n> > \tgit tag multi_hierarchy/tag-2 &&\n> > \n> > +\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n> > +\tgit refs verify 2>err &&\n> > +\tcat >expect <<-EOF &&\n> > +\tEOF\n> > +\ttest_must_be_empty err &&\n> > +\trm $branch_dir_prefix/@ &&\n> \n> `expect` isn't used here as you use `test_must_be_empty`.\n> \n\nThanks, I will improve this in the next version.\n\n> > \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n> > \ttest_must_fail git refs verify 2>err &&\n> > \tcat >expect <<-EOF &&\n> > @@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '\n> > \trm $branch_dir_prefix/.branch-1 &&\n> > \ttest_cmp expect err &&\n> > \n> > -\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n> > +\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\'' branch-1'\\'' &&\n> > \ttest_must_fail git refs verify 2>err &&\n> > \tcat >expect <<-EOF &&\n> > -\terror: refs/heads/@: badRefName: invalid refname format\n> > +\terror: refs/heads/ branch-1: badRefName: invalid refname format\n> > \tEOF\n> > -\trm $branch_dir_prefix/@ &&\n> > +\trm $branch_dir_prefix/'\\'' branch-1'\\'' &&\n> > \ttest_cmp expect err &&\n> \n> Okay, we now allow `refs/heads/@`, but still don't allow other bad\n> formatting like spaces in the refname.\n> \n\nYes, this is a mistake. Junio have told me in this patch and I have\nrealized this.\n\n https://lore.kernel.org/git/xmqqjzei1mtb.fsf@gitster.g/\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"506716","messageId":"Zytrn-WWLIupKZ6m@pks.im","threadId":"61943","inReplyTo":"Zyth0-rUAj83Rz6F@ArchLinux","subject":"Re: [PATCH v6 3/9] ref: initialize target name outside of check functions","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-06T13:14:07Z","receivedAt":"2024-11-06T13:14:19Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Nov 06, 2024 at 08:32:19PM +0800, shejialuo wrote:\n> On Tue, Nov 05, 2024 at 08:11:46AM +0100, Patrick Steinhardt wrote:\n> > On Mon, Oct 21, 2024 at 09:34:31PM +0800, shejialuo wrote:\n> > > In order not to do repeat calculation, rename \"refs_check_dir\" to\n> > > \"target_name\". And in \"files_fsck_refs_dir\", create a new strbuf\n> > > \"target_name\", thus whenever we handle a new target, calculate the\n> > > name and call the check functions one by one.\n> > \n> > \"target_name\" is somewhat of a weird name. I'd expect that this is\n> > either the path to the reference, in which case I'd call this \"path\", or\n> > the name of the reference that is to be checked, in which case I'd call\n> > this \"refname\".\n> > \n> \n> I felt quite hard to name this variable when I wrote the code. \"refname\"\n> is not suitable due to we may check the reflog later by calling\n> \"files_fsck_refs_dir\" function.\n\nI anticipate that we'll likely have separate infra for checking reflogs\nas they are both stored in a different directory and because their\nformat is completely different compared to normal refs. So there isn't\nreally too much of a point to plan ahead for sharing logic here, I'd\nthink, and thus \"refname\" might be a better fit. If that changes in the\nfuture we can still refactor the code.\n\nPatrick\n"},{"id":"506928","messageId":"ZzCiCGxL4Adnd_eq@ArchLinux","threadId":"61943","inReplyTo":"ZxZX5HDdq_R0C77b@ArchLinux","subject":"[PATCH v7 0/9] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:07:36Z","receivedAt":"2024-11-10T12:07:40Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis new version solves the follow problems:\n\n1. Enhance the commit message suggested by Patrick.\n2. Rename \"target_name\" to \"refname\".\n3. Enhance the shell scripts to use `for in` to avoid repetition. And\nthis is the main change of this new version.\n\nThanks,\nJialuo\n\nshejialuo (9):\n ref: initialize \"fsck_ref_report\" with zero\n ref: check the full refname instead of basename\n ref: initialize ref name outside of check functions\n ref: support multiple worktrees check for refs\n ref: port git-fsck(1) regular refs check for files backend\n ref: add more strict checks for regular refs\n ref: add basic symref content check for files backend\n ref: check whether the target of the symref is a ref\n ref: add symlink ref content check for files backend\n\n Documentation/fsck-msgids.txt | 35 +++\n builtin/refs.c | 10 +-\n fsck.h | 6 +\n refs.c | 7 +-\n refs.h | 3 +-\n refs/debug.c | 5 +-\n refs/files-backend.c | 190 ++++++++++++--\n refs/packed-backend.c | 8 +-\n refs/refs-internal.h | 5 +-\n refs/reftable-backend.c | 3 +-\n t/t0602-reffiles-fsck.sh | 480 +++++++++++++++++++++++++++++++---\n 11 files changed, 690 insertions(+), 62 deletions(-)\n\nRange-diff against v6:\n 1: 319f384f1c = 1: bfb2a21af4 ref: initialize \"fsck_ref_report\" with zero\n 2: 8662fc9679 ! 2: 9efc83f7ea ref: check the full refname instead of basename\n @@ Commit message\n \n In order to fix the above problem, enhance \"files_fsck_refs_name\" to use\n the full name for \"check_refname_format\". Then, replace the tests which\n - are related to \"@\" and add tests to exercise the above situations.\n + are related to \"@\" and add tests to exercise the above situations using\n + for loop to avoid repetition.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n @@ refs/files-backend.c: static int files_fsck_refs_name(struct ref_store *ref_stor\n \t\tgoto cleanup;\n \n -\tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n ++\t/*\n ++\t * This works right now because we never check the root refs.\n ++\t */\n +\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n +\tif (check_refname_format(sb.buf, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n @@ refs/files-backend.c: static int files_fsck_refs_name(struct ref_store *ref_stor\n \n ## t/t0602-reffiles-fsck.sh ##\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref name should be checked' '\n - \tgit tag tag-2 &&\n - \tgit tag multi_hierarchy/tag-2 &&\n + \tcd repo &&\n \n -+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n -+\tgit refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\tEOF\n -+\ttest_must_be_empty err &&\n -+\trm $branch_dir_prefix/@ &&\n -+\n - \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n - \ttest_must_fail git refs verify 2>err &&\n - \tcat >expect <<-EOF &&\n -@@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref name should be checked' '\n - \trm $branch_dir_prefix/.branch-1 &&\n - \ttest_cmp expect err &&\n + \tgit commit --allow-empty -m initial &&\n +-\tgit checkout -b branch-1 &&\n +-\tgit tag tag-1 &&\n +-\tgit commit --allow-empty -m second &&\n +-\tgit checkout -b branch-2 &&\n +-\tgit tag tag-2 &&\n +-\tgit tag multi_hierarchy/tag-2 &&\n ++\tgit checkout -b default-branch &&\n ++\tgit tag default-tag &&\n ++\tgit tag multi_hierarchy/default-tag &&\n \n +-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n +-\ttest_must_fail git refs verify 2>err &&\n +-\tcat >expect <<-EOF &&\n +-\terror: refs/heads/.branch-1: badRefName: invalid refname format\n +-\tEOF\n +-\trm $branch_dir_prefix/.branch-1 &&\n +-\ttest_cmp expect err &&\n +-\n -\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n -+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\'' branch-1'\\'' &&\n - \ttest_must_fail git refs verify 2>err &&\n - \tcat >expect <<-EOF &&\n +-\ttest_must_fail git refs verify 2>err &&\n +-\tcat >expect <<-EOF &&\n -\terror: refs/heads/@: badRefName: invalid refname format\n -+\terror: refs/heads/ branch-1: badRefName: invalid refname format\n - \tEOF\n --\trm $branch_dir_prefix/@ &&\n -+\trm $branch_dir_prefix/'\\'' branch-1'\\'' &&\n - \ttest_cmp expect err &&\n +-\tEOF\n ++\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n ++\tgit refs verify 2>err &&\n ++\ttest_must_be_empty err &&\n + \trm $branch_dir_prefix/@ &&\n +-\ttest_cmp expect err &&\n \n -\tcp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/@ &&\n -+\tcp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/'\\''~tag-2'\\'' &&\n - \ttest_must_fail git refs verify 2>err &&\n - \tcat >expect <<-EOF &&\n +-\ttest_must_fail git refs verify 2>err &&\n +-\tcat >expect <<-EOF &&\n -\terror: refs/tags/multi_hierarchy/@: badRefName: invalid refname format\n -+\terror: refs/tags/multi_hierarchy/~tag-2: badRefName: invalid refname format\n - \tEOF\n +-\tEOF\n -\trm $tag_dir_prefix/multi_hierarchy/@ &&\n -+\trm $tag_dir_prefix/multi_hierarchy/'\\''~tag-2'\\'' &&\n - \ttest_cmp expect err &&\n +-\ttest_cmp expect err &&\n +-\n +-\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock &&\n ++\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n + \tgit refs verify 2>err &&\n + \trm $tag_dir_prefix/tag-1.lock &&\n + \ttest_must_be_empty err &&\n \n - \tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock &&\n -@@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref name should be checked' '\n +-\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/.lock &&\n ++\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n + \ttest_must_fail git refs verify 2>err &&\n + \tcat >expect <<-EOF &&\n \terror: refs/tags/.lock: badRefName: invalid refname format\n \tEOF\n \trm $tag_dir_prefix/.lock &&\n +-\ttest_cmp expect err\n +\ttest_cmp expect err &&\n +\n -+\tmkdir $tag_dir_prefix/'\\''~new-feature'\\'' &&\n -+\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/'\\''~new-feature'\\''/tag-1 &&\n -+\ttest_must_fail git refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\terror: refs/tags/~new-feature/tag-1: badRefName: invalid refname format\n -+\tEOF\n -+\trm -rf $tag_dir_prefix/'\\''~new-feature'\\'' &&\n - \ttest_cmp expect err\n ++\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n ++\tdo\n ++\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: refs/heads/$refname: badRefName: invalid refname format\n ++\t\tEOF\n ++\t\trm \"$branch_dir_prefix/$refname\" &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone &&\n ++\n ++\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n ++\tdo\n ++\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: refs/tags/$refname: badRefName: invalid refname format\n ++\t\tEOF\n ++\t\trm \"$tag_dir_prefix/$refname\" &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone &&\n ++\n ++\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n ++\tdo\n ++\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n ++\t\tEOF\n ++\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone &&\n ++\n ++\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n ++\tdo\n ++\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n ++\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n ++\t\tEOF\n ++\t\trm -r \"$branch_dir_prefix/$refname\" &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone\n '\n \n + test_expect_success 'ref name check should be adapted into fsck messages' '\n + \ttest_when_finished \"rm -rf repo\" &&\n + \tgit init repo &&\n + \tbranch_dir_prefix=.git/refs/heads &&\n +-\ttag_dir_prefix=.git/refs/tags &&\n + \tcd repo &&\n + \tgit commit --allow-empty -m initial &&\n + \tgit checkout -b branch-1 &&\n +-\tgit tag tag-1 &&\n +-\tgit commit --allow-empty -m second &&\n +-\tgit checkout -b branch-2 &&\n +-\tgit tag tag-2 &&\n + \n + \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n + \tgit -c fsck.badRefName=warn refs verify 2>err &&\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref name check should be adapted into fsck messages' '\n \trm $branch_dir_prefix/.branch-1 &&\n \ttest_cmp expect err &&\n \n -\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n -+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\\''~branch-1'\\'' &&\n ++\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n \tgit -c fsck.badRefName=ignore refs verify 2>err &&\n \ttest_must_be_empty err\n '\n 3: 96144756fe ! 3: 5ea7d18203 ref: initialize target name outside of check functions\n @@ Metadata\n Author: shejialuo \n \n ## Commit message ##\n - ref: initialize target name outside of check functions\n + ref: initialize ref name outside of check functions\n \n We passes \"refs_check_dir\" to the \"files_fsck_refs_name\" function which\n allows it to create the checked ref name later. However, when we\n - introduce a new check function, we have to re-calculate the target name.\n - It's bad for us to do repeat calculation. Instead, we should calculate\n - it only once and pass the target name to the check functions.\n + introduce a new check function, we have to allocate redundant memory and\n + re-calculate the ref name. It's bad for us to allocate redundant memory\n + and duplicate logic. Instead, we should allocate and calculate it only\n + once and pass the ref name to the check functions.\n \n In order not to do repeat calculation, rename \"refs_check_dir\" to\n - \"target_name\". And in \"files_fsck_refs_dir\", create a new strbuf\n - \"target_name\", thus whenever we handle a new target, calculate the\n - name and call the check functions one by one.\n + \"refname\". And in \"files_fsck_refs_dir\", create a new strbuf \"refname\",\n + thus whenever we handle a new ref, calculate the name and call the check\n + functions one by one.\n \n Mentored-by: Patrick Steinhardt \n Mentored-by: Karthik Nayak \n @@ refs/files-backend.c: static int files_ref_store_remove_on_disk(struct ref_store\n typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n -\t\t\t\t const char *refs_check_dir,\n -+\t\t\t\t const char *target_name,\n ++\t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n -\t\t\t\tconst char *refs_check_dir,\n -+\t\t\t\tconst char *target_name,\n ++\t\t\t\tconst char *refname,\n \t\t\t\tstruct dir_iterator *iter)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n @@ refs/files-backend.c: static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n - \tif (iter->basename[0] != '.' && ends_with(iter->basename, \".lock\"))\n - \t\tgoto cleanup;\n - \n + \t/*\n + \t * This works right now because we never check the root refs.\n + \t */\n -\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n -\tif (check_refname_format(sb.buf, 0)) {\n -+\tif (check_refname_format(target_name, 0)) {\n ++\tif (check_refname_format(refname, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n -\t\treport.path = sb.buf;\n -+\t\treport.path = target_name;\n ++\t\treport.path = refname;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\n \t\t\t\t \"invalid refname format\");\n @@ refs/files-backend.c: static int files_fsck_refs_dir(struct ref_store *ref_store\n \t\t\t const char *refs_check_dir,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n -+\tstruct strbuf target_name = STRBUF_INIT;\n ++\tstruct strbuf refname = STRBUF_INIT;\n \tstruct strbuf sb = STRBUF_INIT;\n \tstruct dir_iterator *iter;\n \tint iter_status;\n @@ refs/files-backend.c: static int files_fsck_refs_dir(struct ref_store *ref_store\n \t\t\tcontinue;\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n -+\t\t\tstrbuf_reset(&target_name);\n -+\t\t\tstrbuf_addf(&target_name, \"%s/%s\", refs_check_dir,\n ++\t\t\tstrbuf_reset(&refname);\n ++\t\t\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir,\n +\t\t\t\t iter->relative_path);\n +\n \t\t\tif (o->verbose)\n -\t\t\t\tfprintf_ln(stderr, \"Checking %s/%s\",\n -\t\t\t\t\t refs_check_dir, iter->relative_path);\n -+\t\t\t\tfprintf_ln(stderr, \"Checking %s\", target_name.buf);\n ++\t\t\t\tfprintf_ln(stderr, \"Checking %s\", refname.buf);\n +\n \t\t\tfor (size_t i = 0; fsck_refs_fn[i]; i++) {\n -\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refs_check_dir, iter))\n -+\t\t\t\tif (fsck_refs_fn[i](ref_store, o, target_name.buf, iter))\n ++\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refname.buf, iter))\n \t\t\t\t\tret = -1;\n \t\t\t}\n \t\t} else {\n @@ refs/files-backend.c: static int files_fsck_refs_dir(struct ref_store *ref_store\n \n out:\n \tstrbuf_release(&sb);\n -+\tstrbuf_release(&target_name);\n ++\tstrbuf_release(&refname);\n \treturn ret;\n }\n \n 4: b396bf6bc2 ! 4: cb4669b64d ref: support multiple worktrees check for refs\n @@ Commit message\n ref: support multiple worktrees check for refs\n \n We have already set up the infrastructure to check the consistency for\n - refs, but we do not support multiple worktrees. As we decide to add more\n - checks for ref content, we need to set up support for multiple\n - worktrees.\n + refs, but we do not support multiple worktrees. However, \"git-fsck(1)\"\n + will check the refs of worktrees. As we decide to get feature parity\n + with \"git-fsck(1)\", we need to set up support for multiple worktrees.\n \n Because each worktree has its own specific refs, instead of just showing\n the users \"refs/worktree/foo\", we need to display the full name such as\n @@ builtin/refs.c: static int cmd_refs_migrate(int argc, const char **argv, const c\n static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n {\n \tstruct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;\n -+\tstruct worktree **worktrees, **p;\n ++\tstruct worktree **worktrees;\n \tconst char * const verify_usage[] = {\n \t\tREFS_VERIFY_USAGE,\n \t\tNULL,\n @@ builtin/refs.c: static int cmd_refs_verify(int argc, const char **argv, const ch\n \n -\tret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);\n +\tworktrees = get_worktrees();\n -+\tfor (p = worktrees; *p; p++) {\n -+\t\tstruct worktree *wt = *p;\n -+\t\tret |= refs_fsck(get_worktree_ref_store(wt), &fsck_refs_options, wt);\n -+\t}\n -+\n ++\tfor (size_t i = 0; worktrees[i]; i++)\n ++\t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n ++\t\t\t\t &fsck_refs_options, worktrees[i]);\n \n \tfsck_options_clear(&fsck_refs_options);\n +\tfree_worktrees(worktrees);\n @@ refs/files-backend.c: static int files_fsck_refs_name(struct ref_store *ref_stor\n +\t\t\t struct worktree *wt,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n - \tstruct strbuf target_name = STRBUF_INIT;\n + \tstruct strbuf refname = STRBUF_INIT;\n @@ refs/files-backend.c: static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n - \t\t\tstrbuf_reset(&target_name);\n + \t\t\tstrbuf_reset(&refname);\n +\n +\t\t\tif (!is_main_worktree(wt))\n -+\t\t\t\tstrbuf_addf(&target_name, \"worktrees/%s/\", wt->id);\n - \t\t\tstrbuf_addf(&target_name, \"%s/%s\", refs_check_dir,\n ++\t\t\t\tstrbuf_addf(&refname, \"worktrees/%s/\", wt->id);\n + \t\t\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir,\n \t\t\t\t iter->relative_path);\n \n @@ refs/files-backend.c: static int files_fsck_refs_dir(struct ref_store *ref_store,\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref name check should be adapted\n +\tsort err >sorted_err &&\n +\ttest_cmp expect sorted_err &&\n +\n -+\t(\n -+\t\tcd worktree-1 &&\n -+\t\ttest_must_fail git refs verify 2>err &&\n -+\t\tcat >expect <<-EOF &&\n -+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n -+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n -+\t\tEOF\n -+\t\tsort err >sorted_err &&\n -+\t\ttest_cmp expect sorted_err\n -+\t) &&\n -+\n -+\t(\n -+\t\tcd worktree-2 &&\n -+\t\ttest_must_fail git refs verify 2>err &&\n -+\t\tcat >expect <<-EOF &&\n -+\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n -+\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n -+\t\tEOF\n -+\t\tsort err >sorted_err &&\n -+\t\ttest_cmp expect sorted_err\n -+\t)\n ++\tfor worktree in \"worktree-1\" \"worktree-2\"\n ++\tdo\n ++\t\t(\n ++\t\t\tcd $worktree &&\n ++\t\t\ttest_must_fail git refs verify 2>err &&\n ++\t\t\tcat >expect <<-EOF &&\n ++\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n ++\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n ++\t\t\tEOF\n ++\t\t\tsort err >sorted_err &&\n ++\t\t\ttest_cmp expect sorted_err || return 1\n ++\t\t)\n ++\tdone\n +'\n +\n test_done\n 5: 6a9e297dfc ! 5: 4e1add6465 ref: port git-fsck(1) regular refs check for files backend\n @@ fsck.h: enum fsck_msg_type {\n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n - \t\t\t\t const char *target_name,\n + \t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n +static int files_fsck_refs_content(struct ref_store *ref_store,\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n +\t\tret = fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n -+\t\t\t\t \"cannot read ref file '%s': (%s)\",\n ++\t\t\t\t \"cannot read ref file '%s': %s\",\n +\t\t\t\t iter->path.buf, strerror(errno));\n +\t\tgoto cleanup;\n +\t}\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n - \t\t\t\tconst char *target_name,\n + \t\t\t\tconst char *refname,\n @@ refs/files-backend.c: static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n @@ refs/files-backend.c: static int files_fsck_refs(struct ref_store *ref_store,\n \n ## t/t0602-reffiles-fsck.sh ##\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref name check should work for multiple worktrees' '\n - \t)\n + \tdone\n '\n \n +test_expect_success 'regular ref content should be checked (individual)' '\n +\ttest_when_finished \"rm -rf repo\" &&\n +\tgit init repo &&\n +\tbranch_dir_prefix=.git/refs/heads &&\n -+\ttag_dir_prefix=.git/refs/tags &&\n +\tcd repo &&\n +\ttest_commit default &&\n +\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref name check should work for mu\n +\tgit refs verify 2>err &&\n +\ttest_must_be_empty err &&\n +\n -+\tbad_content=$(git rev-parse main)x &&\n -+\tprintf \"%s\" $bad_content >$tag_dir_prefix/tag-bad-1 &&\n -+\ttest_must_fail git refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\terror: refs/tags/tag-bad-1: badRefContent: $bad_content\n -+\tEOF\n -+\trm $tag_dir_prefix/tag-bad-1 &&\n -+\ttest_cmp expect err &&\n -+\n -+\tbad_content=xfsazqfxcadas &&\n -+\tprintf \"%s\" $bad_content >$tag_dir_prefix/tag-bad-2 &&\n -+\ttest_must_fail git refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\terror: refs/tags/tag-bad-2: badRefContent: $bad_content\n -+\tEOF\n -+\trm $tag_dir_prefix/tag-bad-2 &&\n -+\ttest_cmp expect err &&\n -+\n -+\tbad_content=Xfsazqfxcadas &&\n -+\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n -+\ttest_must_fail git refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n -+\tEOF\n -+\trm $branch_dir_prefix/a/b/branch-bad &&\n -+\ttest_cmp expect err\n ++\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n ++\tdo\n ++\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n ++\t\tEOF\n ++\t\trm $branch_dir_prefix/branch-bad &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone &&\n ++\n ++\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n ++\tdo\n ++\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n ++\t\tEOF\n ++\t\trm $branch_dir_prefix/a/b/branch-bad &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone\n +'\n +\n +test_expect_success 'regular ref content should be checked (aggregate)' '\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref name check should work for mu\n +\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n +\t) &&\n +\n -+\tbad_content_1=$(git rev-parse HEAD)x &&\n -+\tbad_content_2=xfsazqfxcadas &&\n -+\tbad_content_3=Xfsazqfxcadas &&\n -+\n -+\tprintf \"%s\" $bad_content_1 >$worktree1_refdir_prefix/bad-branch-1 &&\n -+\ttest_must_fail git refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content_1\n -+\tEOF\n -+\trm $worktree1_refdir_prefix/bad-branch-1 &&\n -+\ttest_cmp expect err &&\n -+\n -+\tprintf \"%s\" $bad_content_2 >$worktree2_refdir_prefix/bad-branch-2 &&\n -+\ttest_must_fail git refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content_2\n -+\tEOF\n -+\trm $worktree2_refdir_prefix/bad-branch-2 &&\n -+\ttest_cmp expect err &&\n -+\n -+\tprintf \"%s\" $bad_content_3 >$worktree1_refdir_prefix/bad-branch-3 &&\n -+\ttest_must_fail git refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\terror: worktrees/worktree-1/refs/worktree/bad-branch-3: badRefContent: $bad_content_3\n -+\tEOF\n -+\trm $worktree1_refdir_prefix/bad-branch-3 &&\n -+\ttest_cmp expect err\n ++\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n ++\tdo\n ++\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n ++\t\tEOF\n ++\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone &&\n ++\n ++\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n ++\tdo\n ++\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n ++\t\tEOF\n ++\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone\n +'\n +\n test_done\n 6: 7eea024182 ! 6: 945322fab7 ref: add more strict checks for regular refs\n @@ refs/refs-internal.h: struct ref_store {\n \n ## t/t0602-reffiles-fsck.sh ##\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be checked (individual)' '\n - \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n - \tEOF\n - \trm $branch_dir_prefix/a/b/branch-bad &&\n -+\ttest_cmp expect err &&\n + \t\tEOF\n + \t\trm $branch_dir_prefix/a/b/branch-bad &&\n + \t\ttest_cmp expect err || return 1\n +-\tdone\n ++\tdone &&\n +\n +\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n +\tgit refs verify 2>err &&\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be che\n +\trm $branch_dir_prefix/branch-no-newline &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n -+\tgit refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n -+\tEOF\n -+\trm $branch_dir_prefix/branch-garbage &&\n -+\ttest_cmp expect err &&\n -+\n -+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-1 &&\n ++\tfor trailing_content in \" garbage\" \" more garbage\"\n ++\tdo\n ++\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n ++\t\tgit refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n ++\t\tEOF\n ++\t\trm $branch_dir_prefix/branch-garbage &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone &&\n ++\n ++\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/tags/tag-garbage-1: trailingRefContent: has trailing garbage: '\\''\n ++\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n +\n +\n +\t'\\''\n +\tEOF\n -+\trm $tag_dir_prefix/tag-garbage-1 &&\n ++\trm $branch_dir_prefix/branch-garbage-special &&\n +\ttest_cmp expect err &&\n +\n -+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-2 &&\n ++\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/tags/tag-garbage-2: trailingRefContent: has trailing garbage: '\\''\n ++\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n +\n +\n +\t garbage'\\''\n +\tEOF\n -+\trm $tag_dir_prefix/tag-garbage-2 &&\n -+\ttest_cmp expect err &&\n -+\n -+\tprintf \"%s garbage\\na\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-3 &&\n -+\tgit refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\twarning: refs/tags/tag-garbage-3: trailingRefContent: has trailing garbage: '\\'' garbage\n -+\ta'\\''\n -+\tEOF\n -+\trm $tag_dir_prefix/tag-garbage-3 &&\n -+\ttest_cmp expect err &&\n -+\n -+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$tag_dir_prefix/tag-garbage-4 &&\n -+\ttest_must_fail git -c fsck.trailingRefContent=error refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\terror: refs/tags/tag-garbage-4: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n -+\tEOF\n -+\trm $tag_dir_prefix/tag-garbage-4 &&\n - \ttest_cmp expect err\n ++\trm $branch_dir_prefix/branch-garbage-special &&\n ++\ttest_cmp expect err\n '\n \n + test_expect_success 'regular ref content should be checked (aggregate)' '\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be checked (aggregate)' '\n \tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n \tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be che\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref content checks should work with worktrees' '\n - \terror: worktrees/worktree-1/refs/worktree/bad-branch-3: badRefContent: $bad_content_3\n - \tEOF\n - \trm $worktree1_refdir_prefix/bad-branch-3 &&\n -+\ttest_cmp expect err &&\n + \t\tEOF\n + \t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n + \t\ttest_cmp expect err || return 1\n +-\tdone\n ++\tdone &&\n +\n +\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n +\tgit refs verify 2>err &&\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'ref content checks should work wi\n +\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n +\tEOF\n +\trm $worktree1_refdir_prefix/branch-no-newline &&\n - \ttest_cmp expect err\n ++\ttest_cmp expect err\n '\n \n + test_done\n 7: 1bf36dd644 ! 7: 3006eb9431 ref: add basic symref content check for files backend\n @@ fsck.h: enum fsck_msg_type {\n \n ## refs/files-backend.c ##\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n - \t\t\t\t const char *target_name,\n + \t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n +static int files_fsck_symref_target(struct fsck_options *o,\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be che\n +\ttest_when_finished \"rm -rf repo\" &&\n +\tgit init repo &&\n +\tbranch_dir_prefix=.git/refs/heads &&\n -+\ttag_dir_prefix=.git/refs/tags &&\n +\tcd repo &&\n +\ttest_commit default &&\n +\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n +\n -+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n -+\tgit refs verify 2>err &&\n -+\trm $branch_dir_prefix/branch-good &&\n -+\ttest_must_be_empty err &&\n ++\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n ++\tdo\n ++\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n ++\t\tgit refs verify 2>err &&\n ++\t\trm $branch_dir_prefix/branch-good &&\n ++\t\ttest_must_be_empty err || return 1\n ++\tdone &&\n +\n -+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n -+\tgit refs verify 2>err &&\n -+\trm $branch_dir_prefix/branch-head &&\n -+\ttest_must_be_empty err &&\n ++\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n ++\tdo\n ++\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n ++\t\tEOF\n ++\t\trm $branch_dir_prefix/branch-bad &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone &&\n +\n -+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n ++\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n +\tgit refs verify 2>err &&\n +\tcat >expect <<-EOF &&\n -+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n ++\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n +\tEOF\n -+\trm $branch_dir_prefix/branch-no-newline-1 &&\n ++\trm $branch_dir_prefix/branch-no-newline &&\n +\ttest_cmp expect err &&\n +\n +\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'regular ref content should be che\n +\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n +\tEOF\n +\trm $branch_dir_prefix/a/b/branch-complicated &&\n -+\ttest_cmp expect err &&\n -+\n -+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n -+\ttest_must_fail git refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n -+\tEOF\n -+\trm $branch_dir_prefix/branch-bad-1 &&\n +\ttest_cmp expect err\n +'\n +\n 8: 1d200f2ade ! 8: c59d003d78 ref: check whether the target of the symref is a ref\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'textual symref content should be\n +\ttest_commit default &&\n +\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n +\n -+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-good &&\n -+\tgit refs verify 2>err &&\n -+\trm $branch_dir_prefix/branch-good &&\n -+\ttest_must_be_empty err &&\n ++\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n ++\tdo\n ++\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n ++\t\tgit refs verify 2>err &&\n ++\t\trm $branch_dir_prefix/branch-good &&\n ++\t\ttest_must_be_empty err || return 1\n ++\tdone &&\n +\n -+\tprintf \"ref: refs/foo\\n\" >$branch_dir_prefix/branch-good &&\n -+\tgit refs verify 2>err &&\n -+\trm $branch_dir_prefix/branch-good &&\n -+\ttest_must_be_empty err &&\n -+\n -+\tprintf \"ref: refs-back/heads/main\\n\" >$branch_dir_prefix/branch-bad-1 &&\n -+\tgit refs verify 2>err &&\n -+\tcat >expect <<-EOF &&\n -+\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''refs-back/heads/main'\\''\n -+\tEOF\n -+\trm $branch_dir_prefix/branch-bad-1 &&\n -+\ttest_cmp expect err\n ++\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n ++\tdo\n ++\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n ++\t\tgit refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n ++\t\tEOF\n ++\t\trm $branch_dir_prefix/branch-bad-1 &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone\n +'\n +\n test_expect_success 'ref content checks should work with worktrees' '\n 9: 752f0ad22e ! 9: bb6d7f3323 ref: add symlink ref content check for files backend\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_s\n \n ## t/t0602-reffiles-fsck.sh ##\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'the target of the textual symref should be checked' '\n - \ttest_cmp expect err\n + \tdone\n '\n \n +test_expect_success SYMLINKS 'symlink symref content should be checked' '\n-- \n2.47.0\n\n"},{"id":"506929","messageId":"ZzCibqcJzTkXLBap@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v7 1/9] ref: initialize \"fsck_ref_report\" with zero","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:09:18Z","receivedAt":"2024-11-10T12:09:21Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n\"referent\" is NULL. So, we need to always initialize these parameters to\nNULL instead of letting them point to anywhere when creating a new\n\"fsck_ref_report\" structure.\n\nThe original code explicitly initializes the \"path\" member in the\n\"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\nmembers in the struct). It is more customary to use \"{ 0 }\" to express\nthat we are 0-initializing everything. In order to align with the\ncodebase, initialize \"fsck_ref_report\" with zero.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0824c0b8a9..03d2503276 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3520,7 +3520,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tstruct fsck_ref_report report = { .path = NULL };\n+\t\tstruct fsck_ref_report report = { 0 };\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n-- \n2.47.0\n\n"},{"id":"506930","messageId":"ZzCidukownju3fx3@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v7 2/9] ref: check the full refname instead of basename","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:09:26Z","receivedAt":"2024-11-10T12:09:30Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"files-backend.c::files_fsck_refs_name\", we validate the refname\nformat by using \"check_refname_format\" to check the basename of the\niterator with \"REFNAME_ALLOW_ONELEVEL\" flag.\n\nHowever, this is a bad implementation. Although we doesn't allow a\nsingle \"@\" in \".git\" directory, we do allow \"refs/heads/@\". So, we will\nreport an error wrongly when there is a \"refs/heads/@\" ref by using one\nlevel refname \"@\".\n\nBecause we just check one level refname, we either cannot check the\nother parts of the full refname. And we will ignore the following\nerrors:\n\n \"refs/heads/ new-feature/test\"\n \"refs/heads/~new-feature/test\"\n\nIn order to fix the above problem, enhance \"files_fsck_refs_name\" to use\nthe full name for \"check_refname_format\". Then, replace the tests which\nare related to \"@\" and add tests to exercise the above situations using\nfor loop to avoid repetition.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 7 ++-\n t/t0602-reffiles-fsck.sh | 92 ++++++++++++++++++++++++----------------\n 2 files changed, 60 insertions(+), 39 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 03d2503276..b055edc061 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3519,10 +3519,13 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \tif (iter->basename[0] != '.' && ends_with(iter->basename, \".lock\"))\n \t\tgoto cleanup;\n \n-\tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n+\t/*\n+\t * This works right now because we never check the root refs.\n+\t */\n+\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n+\tif (check_refname_format(sb.buf, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n-\t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 71a4d1a5ae..2a172c913d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -18,63 +18,81 @@ test_expect_success 'ref name should be checked' '\n \tcd repo &&\n \n \tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\tgit tag tag-1 &&\n-\tgit commit --allow-empty -m second &&\n-\tgit checkout -b branch-2 &&\n-\tgit tag tag-2 &&\n-\tgit tag multi_hierarchy/tag-2 &&\n+\tgit checkout -b default-branch &&\n+\tgit tag default-tag &&\n+\tgit tag multi_hierarchy/default-tag &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/@: badRefName: invalid refname format\n-\tEOF\n+\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n \trm $branch_dir_prefix/@ &&\n-\ttest_cmp expect err &&\n \n-\tcp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/@ &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/multi_hierarchy/@: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/multi_hierarchy/@ &&\n-\ttest_cmp expect err &&\n-\n-\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock &&\n+\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n \tgit refs verify 2>err &&\n \trm $tag_dir_prefix/tag-1.lock &&\n \ttest_must_be_empty err &&\n \n-\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/.lock &&\n+\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \terror: refs/tags/.lock: badRefName: invalid refname format\n \tEOF\n \trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err\n+\ttest_cmp expect err &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\ttag_dir_prefix=.git/refs/tags &&\n \tcd repo &&\n \tgit commit --allow-empty -m initial &&\n \tgit checkout -b branch-1 &&\n-\tgit tag tag-1 &&\n-\tgit commit --allow-empty -m second &&\n-\tgit checkout -b branch-2 &&\n-\tgit tag tag-2 &&\n \n \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n \tgit -c fsck.badRefName=warn refs verify 2>err &&\n@@ -84,7 +102,7 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \trm $branch_dir_prefix/.branch-1 &&\n \ttest_cmp expect err &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n \tgit -c fsck.badRefName=ignore refs verify 2>err &&\n \ttest_must_be_empty err\n '\n-- \n2.47.0\n\n"},{"id":"506931","messageId":"ZzCifksT_MShWeoy@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v7 3/9] ref: initialize ref name outside of check functions","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:09:34Z","receivedAt":"2024-11-10T12:09:37Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We passes \"refs_check_dir\" to the \"files_fsck_refs_name\" function which\nallows it to create the checked ref name later. However, when we\nintroduce a new check function, we have to allocate redundant memory and\nre-calculate the ref name. It's bad for us to allocate redundant memory\nand duplicate logic. Instead, we should allocate and calculate it only\nonce and pass the ref name to the check functions.\n\nIn order not to do repeat calculation, rename \"refs_check_dir\" to\n\"refname\". And in \"files_fsck_refs_dir\", create a new strbuf \"refname\",\nthus whenever we handle a new ref, calculate the name and call the check\nfunctions one by one.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 21 +++++++++++++--------\n 1 file changed, 13 insertions(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b055edc061..8edb700568 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3501,12 +3501,12 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store,\n */\n typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n-\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n-\t\t\t\tconst char *refs_check_dir,\n+\t\t\t\tconst char *refname,\n \t\t\t\tstruct dir_iterator *iter)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -3522,11 +3522,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t/*\n \t * This works right now because we never check the root refs.\n \t */\n-\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n-\tif (check_refname_format(sb.buf, 0)) {\n+\tif (check_refname_format(refname, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n-\t\treport.path = sb.buf;\n+\t\treport.path = refname;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\n \t\t\t\t \"invalid refname format\");\n@@ -3542,6 +3541,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\t const char *refs_check_dir,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n+\tstruct strbuf refname = STRBUF_INIT;\n \tstruct strbuf sb = STRBUF_INIT;\n \tstruct dir_iterator *iter;\n \tint iter_status;\n@@ -3560,11 +3560,15 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\tcontinue;\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n+\t\t\tstrbuf_reset(&refname);\n+\t\t\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir,\n+\t\t\t\t iter->relative_path);\n+\n \t\t\tif (o->verbose)\n-\t\t\t\tfprintf_ln(stderr, \"Checking %s/%s\",\n-\t\t\t\t\t refs_check_dir, iter->relative_path);\n+\t\t\t\tfprintf_ln(stderr, \"Checking %s\", refname.buf);\n+\n \t\t\tfor (size_t i = 0; fsck_refs_fn[i]; i++) {\n-\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refs_check_dir, iter))\n+\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refname.buf, iter))\n \t\t\t\t\tret = -1;\n \t\t\t}\n \t\t} else {\n@@ -3581,6 +3585,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \n out:\n \tstrbuf_release(&sb);\n+\tstrbuf_release(&refname);\n \treturn ret;\n }\n \n-- \n2.47.0\n\n"},{"id":"506932","messageId":"ZzCih9B53nTqj7wh@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v7 4/9] ref: support multiple worktrees check for refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:09:43Z","receivedAt":"2024-11-10T12:09:46Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already set up the infrastructure to check the consistency for\nrefs, but we do not support multiple worktrees. However, \"git-fsck(1)\"\nwill check the refs of worktrees. As we decide to get feature parity\nwith \"git-fsck(1)\", we need to set up support for multiple worktrees.\n\nBecause each worktree has its own specific refs, instead of just showing\nthe users \"refs/worktree/foo\", we need to display the full name such as\n\"worktrees//refs/worktree/foo\". So we should know the id of the\nworktree to get the full name. Add a new parameter \"struct worktree *\"\nfor \"refs-internal.h::fsck_fn\". Then change the related functions to\nfollow this new interface.\n\nThe \"packed-refs\" only exists in the main worktree, so we should only\ncheck \"packed-refs\" in the main worktree. Use \"is_main_worktree\" method\nto skip checking \"packed-refs\" in \"packed_fsck\" function.\n\nThen, enhance the \"files-backend.c::files_fsck_refs_dir\" function to add\n\"worktree//\" prefix when we are not in the main worktree.\n\nLast, add a new test to check the refname when there are multiple\nworktrees to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 10 ++++++--\n refs.c | 5 ++--\n refs.h | 3 ++-\n refs/debug.c | 5 ++--\n refs/files-backend.c | 17 ++++++++++----\n refs/packed-backend.c | 8 ++++++-\n refs/refs-internal.h | 3 ++-\n refs/reftable-backend.c | 3 ++-\n t/t0602-reffiles-fsck.sh | 51 ++++++++++++++++++++++++++++++++++++++++\n 9 files changed, 90 insertions(+), 15 deletions(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex 24978a7b7b..394b4101c6 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -5,6 +5,7 @@\n #include \"parse-options.h\"\n #include \"refs.h\"\n #include \"strbuf.h\"\n+#include \"worktree.h\"\n \n #define REFS_MIGRATE_USAGE \\\n \tN_(\"git refs migrate --ref-format= [--dry-run]\")\n@@ -66,6 +67,7 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)\n static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n {\n \tstruct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;\n+\tstruct worktree **worktrees;\n \tconst char * const verify_usage[] = {\n \t\tREFS_VERIFY_USAGE,\n \t\tNULL,\n@@ -75,7 +77,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \t\tOPT_BOOL(0, \"strict\", &fsck_refs_options.strict, N_(\"enable strict checking\")),\n \t\tOPT_END(),\n \t};\n-\tint ret;\n+\tint ret = 0;\n \n \targc = parse_options(argc, argv, prefix, options, verify_usage, 0);\n \tif (argc)\n@@ -84,9 +86,13 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);\n+\tworktrees = get_worktrees();\n+\tfor (size_t i = 0; worktrees[i]; i++)\n+\t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n+\t\t\t\t &fsck_refs_options, worktrees[i]);\n \n \tfsck_options_clear(&fsck_refs_options);\n+\tfree_worktrees(worktrees);\n \treturn ret;\n }\n \ndiff --git a/refs.c b/refs.c\nindex 5f729ed412..395a17273c 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -318,9 +318,10 @@ int check_refname_format(const char *refname, int flags)\n \treturn check_or_sanitize_refname(refname, flags, NULL);\n }\n \n-int refs_fsck(struct ref_store *refs, struct fsck_options *o)\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt)\n {\n-\treturn refs->be->fsck(refs, o);\n+\treturn refs->be->fsck(refs, o, wt);\n }\n \n void sanitize_refname_component(const char *refname, struct strbuf *out)\ndiff --git a/refs.h b/refs.h\nindex 108dfc93b3..341d43239c 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -549,7 +549,8 @@ int check_refname_format(const char *refname, int flags);\n * reflogs are consistent, and non-zero otherwise. The errors will be\n * written to stderr.\n */\n-int refs_fsck(struct ref_store *refs, struct fsck_options *o);\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt);\n \n /*\n * Apply the rules from check_refname_format, but mutate the result until it\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 45e2e784a0..72e80ddd6d 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -420,10 +420,11 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,\n }\n \n static int debug_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n-\tint res = drefs->refs->be->fsck(drefs->refs, o);\n+\tint res = drefs->refs->be->fsck(drefs->refs, o, wt);\n \ttrace_printf_key(&trace_refs, \"fsck: %d\\n\", res);\n \treturn res;\n }\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8edb700568..8bfdce64bc 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -23,6 +23,7 @@\n #include \"../dir.h\"\n #include \"../chdir-notify.h\"\n #include \"../setup.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../revision.h\"\n@@ -3539,6 +3540,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\t struct fsck_options *o,\n \t\t\t const char *refs_check_dir,\n+\t\t\t struct worktree *wt,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -3561,6 +3563,9 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n \t\t\tstrbuf_reset(&refname);\n+\n+\t\t\tif (!is_main_worktree(wt))\n+\t\t\t\tstrbuf_addf(&refname, \"worktrees/%s/\", wt->id);\n \t\t\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir,\n \t\t\t\t iter->relative_path);\n \n@@ -3590,7 +3595,8 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n }\n \n static int files_fsck_refs(struct ref_store *ref_store,\n-\t\t\t struct fsck_options *o)\n+\t\t\t struct fsck_options *o,\n+\t\t\t struct worktree *wt)\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n@@ -3599,17 +3605,18 @@ static int files_fsck_refs(struct ref_store *ref_store,\n \n \tif (o->verbose)\n \t\tfprintf_ln(stderr, _(\"Checking references consistency\"));\n-\treturn files_fsck_refs_dir(ref_store, o, \"refs\", fsck_refs_fn);\n+\treturn files_fsck_refs_dir(ref_store, o, \"refs\", wt, fsck_refs_fn);\n }\n \n static int files_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"fsck\");\n \n-\treturn files_fsck_refs(ref_store, o) |\n-\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n+\treturn files_fsck_refs(ref_store, o, wt) |\n+\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o, wt);\n }\n \n struct ref_storage_be refs_be_files = {\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 07c57fd541..46dcaec654 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -13,6 +13,7 @@\n #include \"../lockfile.h\"\n #include \"../chdir-notify.h\"\n #include \"../statinfo.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../trace2.h\"\n@@ -1754,8 +1755,13 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n }\n \n static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED)\n+\t\t struct fsck_options *o UNUSED,\n+\t\t struct worktree *wt)\n {\n+\n+\tif (!is_main_worktree(wt))\n+\t\treturn 0;\n+\n \treturn 0;\n }\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 2313c830d8..037d7991cd 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -653,7 +653,8 @@ typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refnam\n \t\t\t\t struct strbuf *referent);\n \n typedef int fsck_fn(struct ref_store *ref_store,\n-\t\t struct fsck_options *o);\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt);\n \n struct ref_storage_be {\n \tconst char *name;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex f5f957e6de..b6a63c1015 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -2443,7 +2443,8 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n }\n \n static int reftable_be_fsck(struct ref_store *ref_store UNUSED,\n-\t\t\t struct fsck_options *o UNUSED)\n+\t\t\t struct fsck_options *o UNUSED,\n+\t\t\t struct worktree *wt UNUSED)\n {\n \treturn 0;\n }\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 2a172c913d..1e17393a3d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -107,4 +107,55 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'ref name check should work for multiple worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\n+\tcd repo &&\n+\ttest_commit initial &&\n+\tgit checkout -b branch-1 &&\n+\ttest_commit second &&\n+\tgit checkout -b branch-2 &&\n+\ttest_commit third &&\n+\tgit checkout -b branch-3 &&\n+\tgit worktree add ./worktree-1 branch-1 &&\n+\tgit worktree add ./worktree-2 branch-2 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\n+\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err &&\n+\n+\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\tdo\n+\t\t(\n+\t\t\tcd $worktree &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\tsort err >sorted_err &&\n+\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t)\n+\tdone\n+'\n+\n test_done\n-- \n2.47.0\n\n"},{"id":"506933","messageId":"ZzCij4ilPLhlKXS6@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v7 5/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:09:51Z","receivedAt":"2024-11-10T12:09:54Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"git-fsck(1)\" implicitly checks the ref content by passing the\ncallback \"fsck_handle_ref\" to the \"refs.c::refs_for_each_rawref\".\nThen, it will check whether the ref content (eventually \"oid\")\nis valid. If not, it will report the following error to the user.\n\n error: refs/heads/main: invalid sha1 pointer 0000...\n\nAnd it will also report above errors when there are dangling symrefs\nin the repository wrongly. This does not align with the behavior of\nthe \"git symbolic-ref\" command which allows users to create dangling\nsymrefs.\n\nAs we have already introduced the \"git refs verify\" command, we'd better\ncheck the ref content explicitly in the \"git refs verify\" command thus\nlater we could remove these checks in \"git-fsck(1)\" and launch a\nsubprocess to call \"git refs verify\" in \"git-fsck(1)\" to make the\n\"git-fsck(1)\" more clean.\n\nFollowing what \"git-fsck(1)\" does, add a similar check to \"git refs\nverify\". Then add a new fsck error message \"badRefContent(ERROR)\" to\nrepresent that a ref has an invalid content.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/files-backend.c | 43 ++++++++++++++\n t/t0602-reffiles-fsck.sh | 105 ++++++++++++++++++++++++++++++++++\n 4 files changed, 152 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 68a2801f15..22c385ea22 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -19,6 +19,9 @@\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n+`badRefContent`::\n+\t(ERROR) A ref has bad content.\n+\n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n \ndiff --git a/fsck.h b/fsck.h\nindex 500b4c04d2..0d99a87911 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -31,6 +31,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8bfdce64bc..2d126ecbbe 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3505,6 +3505,48 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_refs_content(struct ref_store *ref_store,\n+\t\t\t\t struct fsck_options *o,\n+\t\t\t\t const char *target_name,\n+\t\t\t\t struct dir_iterator *iter)\n+{\n+\tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tunsigned int type = 0;\n+\tint failure_errno = 0;\n+\tstruct object_id oid;\n+\tint ret = 0;\n+\n+\treport.path = target_name;\n+\n+\tif (S_ISLNK(iter->st.st_mode))\n+\t\tgoto cleanup;\n+\n+\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"cannot read ref file '%s': %s\",\n+\t\t\t\t iter->path.buf, strerror(errno));\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n+\t\t\t\t ref_content.buf, &oid, &referent,\n+\t\t\t\t &type, &failure_errno)) {\n+\t\tstrbuf_rtrim(&ref_content);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"%s\", ref_content.buf);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&ref_content);\n+\tstrbuf_release(&referent);\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n \t\t\t\tconst char *refname,\n@@ -3600,6 +3642,7 @@ static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n+\t\tfiles_fsck_refs_content,\n \t\tNULL,\n \t};\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 1e17393a3d..162370077b 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -158,4 +158,109 @@ test_expect_success 'ref name check should work for multiple worktrees' '\n \tdone\n '\n \n+test_expect_success 'regular ref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n+\n+\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-bad &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n+test_expect_success 'regular ref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tbad_content_1=$(git rev-parse main)x &&\n+\tbad_content_2=xfsazqfxcadas &&\n+\tbad_content_3=Xfsazqfxcadas &&\n+\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n+test_expect_success 'ref content checks should work with worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit branch branch-3 &&\n+\tgit worktree add ./worktree-1 branch-2 &&\n+\tgit worktree add ./worktree-2 branch-3 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\n+\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_done\n-- \n2.47.0\n\n"},{"id":"506934","messageId":"ZzCim_sgaFHF3FIM@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v7 6/9] ref: add more strict checks for regular refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:10:03Z","receivedAt":"2024-11-10T12:10:07Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already used \"parse_loose_ref_contents\" function to check\nwhether the ref content is valid in files backend. However, by\nusing \"parse_loose_ref_contents\", we allow the ref's content to end with\ngarbage or without a newline.\n\nEven though we never create such loose refs ourselves, we have accepted\nsuch loose refs. So, it is entirely possible that some third-party tools\nmay rely on such loose refs being valid. We should not report an error\nfsck message at current. We should notify the users about such\n\"curiously formatted\" loose refs so that adequate care is taken before\nwe decide to tighten the rules in the future.\n\nAnd it's not suitable either to report a warn fsck message to the user.\nWe don't yet want the \"--strict\" flag that controls this bit to end up\ngenerating errors for such weirdly-formatted reference contents, as we\nfirst want to assess whether this retroactive tightening will cause\nissues for any tools out there. It may cause compatibility issues which\nmay break the repository. So, we add the following two fsck infos to\nrepresent the situation where the ref content ends without newline or\nhas trailing garbages:\n\n1. refMissingNewline(INFO): A loose ref that does not end with\n newline(LF).\n2. trailingRefContent(INFO): A loose ref has trailing content.\n\nIt might appear that we can't provide the user with any warnings by\nusing FSCK_INFO. However, in \"fsck.c::fsck_vreport\", we will convert\nFSCK_INFO to FSCK_WARN and we can still warn the user about these\nsituations when using \"git refs verify\" without introducing\ncompatibility issues.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 14 +++++++++\n fsck.h | 2 ++\n refs.c | 2 +-\n refs/files-backend.c | 26 ++++++++++++++--\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 57 +++++++++++++++++++++++++++++++++--\n 6 files changed, 96 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 22c385ea22..6db0eaa84a 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -173,6 +173,20 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`refMissingNewline`::\n+\t(INFO) A loose ref that does not end with newline(LF). As\n+\tvalid implementations of Git never created such a loose ref\n+\tfile, it may become an error in the future. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as\n+\twe need to know what tools created such a file.\n+\n+`trailingRefContent`::\n+\t(INFO) A loose ref has trailing content. As valid implementations\n+\tof Git never created such a loose ref file, it may become an\n+\terror in the future. Report to the git@vger.kernel.org mailing\n+\tlist if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \ndiff --git a/fsck.h b/fsck.h\nindex 0d99a87911..b85072df57 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -85,6 +85,8 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs.c b/refs.c\nindex 395a17273c..f88b32a633 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1789,7 +1789,7 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \t}\n \n \tresult = parse_loose_ref_contents(ref_store->repo->hash_algo, content.buf,\n-\t\t\t\t\t oid, referent, type, failure_errno);\n+\t\t\t\t\t oid, referent, type, NULL, failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 2d126ecbbe..871c8946f8 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -569,7 +569,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname,\n \tbuf = sb_contents.buf;\n \n \tret = parse_loose_ref_contents(ref_store->repo->hash_algo, buf,\n-\t\t\t\t oid, referent, type, &myerr);\n+\t\t\t\t oid, referent, type, NULL, &myerr);\n \n out:\n \tif (ret && !myerr)\n@@ -606,7 +606,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno)\n+\t\t\t const char **trailing, int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -628,6 +628,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n+\n+\tif (trailing)\n+\t\t*trailing = p;\n+\n \treturn 0;\n }\n \n@@ -3513,6 +3517,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n+\tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n \tstruct object_id oid;\n@@ -3533,7 +3538,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n \t\t\t\t ref_content.buf, &oid, &referent,\n-\t\t\t\t &type, &failure_errno)) {\n+\t\t\t\t &type, &trailing, &failure_errno)) {\n \t\tstrbuf_rtrim(&ref_content);\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n@@ -3541,6 +3546,21 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (!(type & REF_ISSYMREF)) {\n+\t\tif (!*trailing) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t \"misses LF at the end\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n cleanup:\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 037d7991cd..125f1fe735 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -716,7 +716,7 @@ struct ref_store {\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno);\n+\t\t\t const char **trailing, int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 162370077b..33e7a390ad 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -189,7 +189,48 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \t\tEOF\n \t\trm $branch_dir_prefix/a/b/branch-bad &&\n \t\ttest_cmp expect err || return 1\n-\tdone\n+\tdone &&\n+\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tfor trailing_content in \" garbage\" \" more garbage\"\n+\tdo\n+\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\n+\n+\t'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage-special &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\n+\n+\t garbage'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage-special &&\n+\ttest_cmp expect err\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -207,12 +248,16 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n \tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n \tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n \terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n \terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err\n@@ -260,7 +305,15 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t\tEOF\n \t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n \t\ttest_cmp expect err || return 1\n-\tdone\n+\tdone &&\n+\n+\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\ttest_cmp expect err\n '\n \n test_done\n-- \n2.47.0\n\n"},{"id":"506935","messageId":"ZzCioxuAOnyol9gt@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v7 7/9] ref: add basic symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:10:11Z","receivedAt":"2024-11-10T12:10:15Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have code that checks regular ref contents, but we do not yet check\nthe contents of symbolic refs. By using \"parse_loose_ref_content\" for\nsymbolic refs, we will get the information of the \"referent\".\n\nWe do not need to check the \"referent\" by opening the file. This is\nbecause if \"referent\" exists in the file system, we will eventually\ncheck its correctness by inspecting every file in the \"refs\" directory.\nIf the \"referent\" does not exist in the filesystem, this is OK as it is\nseen as the dangling symref.\n\nSo we just need to check the \"referent\" string content. A regular ref\ncould be accepted as a textual symref if it begins with \"ref:\", followed\nby zero or more whitespaces, followed by the full refname, followed only\nby whitespace characters. However, we always write a single SP after\n\"ref:\" and a single LF after the refname. It may seem that we should\nreport a fsck error message when the \"referent\" does not apply above\nrules and we should not be so aggressive because third-party\nreimplementations of Git may have taken advantage of the looser syntax.\nPut it more specific, we accept the following contents:\n\n1. \"ref: refs/heads/master \"\n2. \"ref: refs/heads/master \\n \\n\"\n3. \"ref: refs/heads/master\\n\\n\"\n\nWhen introducing the regular ref content checks, we created two fsck\ninfos \"refMissingNewline\" and \"trailingRefContent\" which exactly\nrepresents above situations. So we will reuse these two fsck messages to\nwrite checks to info the user about these situations.\n\nBut we do not allow any other trailing garbage. The followings are bad\nsymref contents which will be reported as fsck error by \"git-fsck(1)\".\n\n1. \"ref: refs/heads/master garbage\\n\"\n2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n\nAnd we introduce a new \"badReferentName(ERROR)\" fsck message to report\nabove errors by using \"is_root_ref\" and \"check_refname_format\" to check\nthe \"referent\". Since both \"is_root_ref\" and \"check_refname_format\"\ndon't work with whitespaces, we use the trimmed version of \"referent\"\nwith these functions.\n\nIn order to add checks, we will do the following things:\n\n1. Record the untrimmed length \"orig_len\" and untrimmed last byte\n \"orig_last_byte\".\n2. Use \"strbuf_rtrim\" to trim the whitespaces or newlines to make sure\n \"is_root_ref\" and \"check_refname_format\" won't be failed by them.\n3. Use \"orig_len\" and \"orig_last_byte\" to check whether the \"referent\"\n misses '\\n' at the end or it has trailing whitespaces or newlines.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/files-backend.c | 40 ++++++++++++\n t/t0602-reffiles-fsck.sh | 111 ++++++++++++++++++++++++++++++++++\n 4 files changed, 155 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 6db0eaa84a..dcea05edfc 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,6 +28,9 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n+`badReferentName`::\n+\t(ERROR) The referent name of a symref is invalid.\n+\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n \ndiff --git a/fsck.h b/fsck.h\nindex b85072df57..5227dfdef2 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,6 +34,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n+\tFUNC(BAD_REFERENT_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 871c8946f8..8bc7c6e0c2 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3509,6 +3509,43 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_symref_target(struct fsck_options *o,\n+\t\t\t\t struct fsck_ref_report *report,\n+\t\t\t\t struct strbuf *referent)\n+{\n+\tchar orig_last_byte;\n+\tsize_t orig_len;\n+\tint ret = 0;\n+\n+\torig_len = referent->len;\n+\torig_last_byte = referent->buf[orig_len - 1];\n+\tstrbuf_rtrim(referent);\n+\n+\tif (!is_root_ref(referent->buf) &&\n+\t check_refname_format(referent->buf, 0)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n+\t\t\t\t \"points to invalid refname '%s'\", referent->buf);\n+\t\tgoto out;\n+\t}\n+\n+\tif (referent->len == orig_len ||\n+\t (referent->len < orig_len && orig_last_byte != '\\n')) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t \"misses LF at the end\");\n+\t}\n+\n+\tif (referent->len != orig_len && referent->len != orig_len - 1) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t \"has trailing whitespaces or newlines\");\n+\t}\n+\n+out:\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *target_name,\n@@ -3559,6 +3596,9 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n \t\t\tgoto cleanup;\n \t\t}\n+\t} else {\n+\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tgoto cleanup;\n \t}\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 33e7a390ad..ee1e5f2864 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -263,6 +263,109 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'textual symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $branch_dir_prefix/branch-good &&\n+\t\ttest_must_be_empty err || return 1\n+\tdone &&\n+\n+\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-bad &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'textual symref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n@@ -313,6 +416,14 @@ test_expect_success 'ref content checks should work with worktrees' '\n \twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \trm $worktree1_refdir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-garbage &&\n \ttest_cmp expect err\n '\n \n-- \n2.47.0\n\n"},{"id":"506936","messageId":"ZzCirMhZe4vLOxuO@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v7 8/9] ref: check whether the target of the symref is a ref","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:10:20Z","receivedAt":"2024-11-10T12:10:23Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Ideally, we want to the users use \"git symbolic-ref\" to create symrefs\ninstead of writing raw contents into the filesystem. However, \"git\nsymbolic-ref\" is strict with the refname but not strict with the\nreferent. For example, we can make the \"referent\" located at the\n\"$(gitdir)/logs/aaa\" and manually write the content into this where we\ncan still successfully parse this symref by using \"git rev-parse\".\n\n $ git init repo && cd repo && git commit --allow-empty -mx\n $ git symbolic-ref refs/heads/test logs/aaa\n $ echo $(git rev-parse HEAD) > .git/logs/aaa\n $ git rev-parse test\n\nWe may need to add some restrictions for \"referent\" parameter when using\n\"git symbolic-ref\" to create symrefs because ideally all the\nnonpseudo-refs should be located under the \"refs\" directory and we may\ntighten this in the future.\n\nIn order to tell the user we may tighten the above situation, create\na new fsck message \"symrefTargetIsNotARef\" to notify the user that this\nmay become an error in the future.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 9 +++++++++\n fsck.h | 1 +\n refs/files-backend.c | 14 ++++++++++++--\n t/t0602-reffiles-fsck.sh | 29 +++++++++++++++++++++++++++++\n 4 files changed, 51 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex dcea05edfc..f82ebc58e8 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -183,6 +183,15 @@\n \tgit@vger.kernel.org mailing list if you see this error, as\n \twe need to know what tools created such a file.\n \n+`symrefTargetIsNotARef`::\n+\t(INFO) The target of a symbolic reference points neither to\n+\ta root reference nor to a reference starting with \"refs/\".\n+\tAlthough we allow create a symref pointing to the referent which\n+\tis outside the \"ref\" by using `git symbolic-ref`, we may tighten\n+\tthe rule in the future. Report to the git@vger.kernel.org\n+\tmailing list if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n `trailingRefContent`::\n \t(INFO) A loose ref has trailing content. As valid implementations\n \tof Git never created such a loose ref file, it may become an\ndiff --git a/fsck.h b/fsck.h\nindex 5227dfdef2..53a47612e6 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -87,6 +87,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8bc7c6e0c2..b3ec409920 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3513,6 +3513,7 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n \t\t\t\t struct strbuf *referent)\n {\n+\tint is_referent_root;\n \tchar orig_last_byte;\n \tsize_t orig_len;\n \tint ret = 0;\n@@ -3521,8 +3522,17 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \torig_last_byte = referent->buf[orig_len - 1];\n \tstrbuf_rtrim(referent);\n \n-\tif (!is_root_ref(referent->buf) &&\n-\t check_refname_format(referent->buf, 0)) {\n+\tis_referent_root = is_root_ref(referent->buf);\n+\tif (!is_referent_root &&\n+\t !starts_with(referent->buf, \"refs/\") &&\n+\t !starts_with(referent->buf, \"worktrees/\")) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_SYMREF_TARGET_IS_NOT_A_REF,\n+\t\t\t\t \"points to non-ref target '%s'\", referent->buf);\n+\n+\t}\n+\n+\tif (!is_referent_root && check_refname_format(referent->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n \t\t\t\t \"points to invalid refname '%s'\", referent->buf);\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex ee1e5f2864..692b30727a 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -366,6 +366,35 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'the target of the textual symref should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $branch_dir_prefix/branch-good &&\n+\t\ttest_must_be_empty err || return 1\n+\tdone &&\n+\n+\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-- \n2.47.0\n\n"},{"id":"506937","messageId":"ZzCis8E49O10O1zr@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v7 9/9] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-10T12:10:27Z","receivedAt":"2024-11-10T12:10:32Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Besides the textual symref, we also allow symbolic links as the symref.\nSo, we should also provide the consistency check as what we have done\nfor textual symref. And also we consider deprecating writing the\nsymbolic links. We first need to access whether symbolic links still\nbe used. So, add a new fsck message \"symlinkRef(INFO)\" to tell the\nuser be aware of this information.\n\nWe have already introduced \"files_fsck_symref_target\". We should reuse\nthis function to handle the symrefs which use legacy symbolic links. We\nshould not check the trailing garbage for symbolic refs. Add a new\nparameter \"symbolic_link\" to disable some checks which should only be\nexecuted for textual symrefs.\n\nAnd we need to also generate the \"referent\" parameter for reusing\n\"files_fsck_symref_target\" by the following steps:\n\n1. Use \"strbuf_add_real_path\" to resolve the symlink and get the\n absolute path \"ref_content\" which the symlink ref points to.\n2. Generate the absolute path \"abs_gitdir\" of \"gitdir\" and combine\n \"ref_content\" and \"abs_gitdir\" to extract the relative path\n \"relative_referent_path\".\n3. If \"ref_content\" is outside of \"gitdir\", we just set \"referent\" with\n \"ref_content\". Instead, we set \"referent\" with\n \"relative_referent_path\".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 6 +++++\n fsck.h | 1 +\n refs/files-backend.c | 38 +++++++++++++++++++++++++----\n t/t0602-reffiles-fsck.sh | 45 +++++++++++++++++++++++++++++++++++\n 4 files changed, 86 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex f82ebc58e8..b14bc44ca4 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -183,6 +183,12 @@\n \tgit@vger.kernel.org mailing list if you see this error, as\n \twe need to know what tools created such a file.\n \n+`symlinkRef`::\n+\t(INFO) A symbolic link is used as a symref. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as we\n+\tare assessing the feasibility of dropping the support to drop\n+\tcreating symbolic links as symrefs.\n+\n `symrefTargetIsNotARef`::\n \t(INFO) The target of a symbolic reference points neither to\n \ta root reference nor to a reference starting with \"refs/\".\ndiff --git a/fsck.h b/fsck.h\nindex 53a47612e6..a44c231a5f 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -86,6 +86,7 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(SYMLINK_REF, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n \tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b3ec409920..37c669a30f 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1,6 +1,7 @@\n #define USE_THE_REPOSITORY_VARIABLE\n \n #include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n #include \"../config.h\"\n #include \"../copy.h\"\n #include \"../environment.h\"\n@@ -3511,7 +3512,8 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n-\t\t\t\t struct strbuf *referent)\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t unsigned int symbolic_link)\n {\n \tint is_referent_root;\n \tchar orig_last_byte;\n@@ -3520,7 +3522,8 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \n \torig_len = referent->len;\n \torig_last_byte = referent->buf[orig_len - 1];\n-\tstrbuf_rtrim(referent);\n+\tif (!symbolic_link)\n+\t\tstrbuf_rtrim(referent);\n \n \tis_referent_root = is_root_ref(referent->buf);\n \tif (!is_referent_root &&\n@@ -3539,6 +3542,9 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n+\tif (symbolic_link)\n+\t\tgoto out;\n+\n \tif (referent->len == orig_len ||\n \t (referent->len < orig_len && orig_last_byte != '\\n')) {\n \t\tret = fsck_report_ref(o, report,\n@@ -3562,6 +3568,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct dir_iterator *iter)\n {\n \tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n \tconst char *trailing = NULL;\n@@ -3572,8 +3579,30 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \treport.path = target_name;\n \n-\tif (S_ISLNK(iter->st.st_mode))\n+\tif (S_ISLNK(iter->st.st_mode)) {\n+\t\tconst char* relative_referent_path = NULL;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_SYMLINK_REF,\n+\t\t\t\t \"use deprecated symbolic link for symref\");\n+\n+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n+\t\tstrbuf_normalize_path(&abs_gitdir);\n+\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n+\t\t\tstrbuf_addch(&abs_gitdir, '/');\n+\n+\t\tstrbuf_add_real_path(&ref_content, iter->path.buf);\n+\t\tskip_prefix(ref_content.buf, abs_gitdir.buf,\n+\t\t\t &relative_referent_path);\n+\n+\t\tif (relative_referent_path)\n+\t\t\tstrbuf_addstr(&referent, relative_referent_path);\n+\t\telse\n+\t\t\tstrbuf_addbuf(&referent, &ref_content);\n+\n+\t\tret |= files_fsck_symref_target(o, &report, &referent, 1);\n \t\tgoto cleanup;\n+\t}\n \n \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n \t\tret = fsck_report_ref(o, &report,\n@@ -3607,13 +3636,14 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\tgoto cleanup;\n \t\t}\n \t} else {\n-\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tret = files_fsck_symref_target(o, &report, &referent, 0);\n \t\tgoto cleanup;\n \t}\n \n cleanup:\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n+\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 692b30727a..0d5eda6d22 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -395,6 +395,51 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tdone\n '\n \n+test_expect_success SYMLINKS 'symlink symref content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_cmp expect err\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-- \n2.47.0\n\n"},{"id":"507190","messageId":"ZzRW2zFHUNeeT7Jb@pks.im","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"Re: [PATCH v7 0/9] add ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-13T07:36:08Z","receivedAt":"2024-11-13T07:36:21Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Nov 10, 2024 at 08:07:36PM +0800, shejialuo wrote:\n> Hi All:\n> \n> This new version solves the follow problems:\n> \n> 1. Enhance the commit message suggested by Patrick.\n> 2. Rename \"target_name\" to \"refname\".\n> 3. Enhance the shell scripts to use `for in` to avoid repetition. And\n> this is the main change of this new version.\n> \n> Thanks,\n> Jialuo\n\nI've got two more comments, but otherwise this series looks close now.\nThanks!\n\nPatrick\n"},{"id":"507191","messageId":"ZzRW7E2Z0ZYQ7i20@pks.im","threadId":"61943","inReplyTo":"ZzCij4ilPLhlKXS6@ArchLinux","subject":"Re: [PATCH v7 5/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-13T07:36:12Z","receivedAt":"2024-11-13T07:36:22Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Nov 10, 2024 at 08:09:51PM +0800, shejialuo wrote:\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 8bfdce64bc..2d126ecbbe 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3505,6 +3505,48 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> \t\t\t\t const char *refname,\n> \t\t\t\t struct dir_iterator *iter);\n> \n> +static int files_fsck_refs_content(struct ref_store *ref_store,\n> +\t\t\t\t struct fsck_options *o,\n> +\t\t\t\t const char *target_name,\n> +\t\t\t\t struct dir_iterator *iter)\n> +{\n> +\tstruct strbuf ref_content = STRBUF_INIT;\n> +\tstruct strbuf referent = STRBUF_INIT;\n> +\tstruct fsck_ref_report report = { 0 };\n> +\tunsigned int type = 0;\n> +\tint failure_errno = 0;\n> +\tstruct object_id oid;\n> +\tint ret = 0;\n> +\n> +\treport.path = target_name;\n> +\n> +\tif (S_ISLNK(iter->st.st_mode))\n> +\t\tgoto cleanup;\n> +\n> +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> +\t\t\t\t \"cannot read ref file '%s': %s\",\n> +\t\t\t\t iter->path.buf, strerror(errno));\n> +\t\tgoto cleanup;\n> +\t}\n\nI didn't catch this in previous rounds, but it's a little dubious\nwhether we should report this as an actual fsck error. I can expect\nmultiple situations:\n\n - The file has weird permissions and thus cannot be read, failing with\n EPERM, which doesn't match well with BAD_REF_CONTENT.\n\n - The file does not exist anymore because we were racing with a\n concurrent writer, failing with ENOENT. This is benign and expected\n to happen in busy repos, so generating an error here feels wrong.\n\n - The file cannot be read at all due to an I/O error. This may be\n reported with BAD_REF_CONTENT, but conflating this with the case\n where we have actually bad content may not be the best idea.\n\nSo maybe we should ignore ENOENT, report bad permissions and otherwise\nreturn an actual error to the caller?\n\nPatrick\n"},{"id":"507192","messageId":"ZzRW8M39-hw1E9-h@pks.im","threadId":"61943","inReplyTo":"ZzCis8E49O10O1zr@ArchLinux","subject":"Re: [PATCH v7 9/9] ref: add symlink ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-13T07:36:16Z","receivedAt":"2024-11-13T07:36:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Nov 10, 2024 at 08:10:27PM +0800, shejialuo wrote:\n> @@ -3572,8 +3579,30 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> \n> \treport.path = target_name;\n> \n> -\tif (S_ISLNK(iter->st.st_mode))\n> +\tif (S_ISLNK(iter->st.st_mode)) {\n> +\t\tconst char* relative_referent_path = NULL;\n\nNit: the asterisk should stick with the variable name.\n\n> +\t\tret = fsck_report_ref(o, &report,\n> +\t\t\t\t FSCK_MSG_SYMLINK_REF,\n> +\t\t\t\t \"use deprecated symbolic link for symref\");\n> +\n> +\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n> +\t\tstrbuf_normalize_path(&abs_gitdir);\n> +\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n> +\t\t\tstrbuf_addch(&abs_gitdir, '/');\n> +\n> +\t\tstrbuf_add_real_path(&ref_content, iter->path.buf);\n> +\t\tskip_prefix(ref_content.buf, abs_gitdir.buf,\n> +\t\t\t &relative_referent_path);\n> +\n> +\t\tif (relative_referent_path)\n> +\t\t\tstrbuf_addstr(&referent, relative_referent_path);\n> +\t\telse\n> +\t\t\tstrbuf_addbuf(&referent, &ref_content);\n> +\n> +\t\tret |= files_fsck_symref_target(o, &report, &referent, 1);\n> \t\tgoto cleanup;\n> +\t}\n\nI wonder whether this logic works as expected with per-worktree symbolic\nrefs which are a symlink. On the other hand I wonder whether those work\nas expected in the first place. Probably not. *shrug*\n\nIn any case, it would be nice to have a test for this.\n\nPatrick\n"},{"id":"507266","messageId":"ZzXocdnqigGYFXQ_@ArchLinux","threadId":"61943","inReplyTo":"ZzRW7E2Z0ZYQ7i20@pks.im","subject":"Re: [PATCH v7 5/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T12:09:21Z","receivedAt":"2024-11-14T12:09:20Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Nov 13, 2024 at 08:36:12AM +0100, Patrick Steinhardt wrote:\n> On Sun, Nov 10, 2024 at 08:09:51PM +0800, shejialuo wrote:\n> > diff --git a/refs/files-backend.c b/refs/files-backend.c\n> > index 8bfdce64bc..2d126ecbbe 100644\n> > --- a/refs/files-backend.c\n> > +++ b/refs/files-backend.c\n> > @@ -3505,6 +3505,48 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n> > \t\t\t\t const char *refname,\n> > \t\t\t\t struct dir_iterator *iter);\n> > \n> > +static int files_fsck_refs_content(struct ref_store *ref_store,\n> > +\t\t\t\t struct fsck_options *o,\n> > +\t\t\t\t const char *target_name,\n> > +\t\t\t\t struct dir_iterator *iter)\n> > +{\n> > +\tstruct strbuf ref_content = STRBUF_INIT;\n> > +\tstruct strbuf referent = STRBUF_INIT;\n> > +\tstruct fsck_ref_report report = { 0 };\n> > +\tunsigned int type = 0;\n> > +\tint failure_errno = 0;\n> > +\tstruct object_id oid;\n> > +\tint ret = 0;\n> > +\n> > +\treport.path = target_name;\n> > +\n> > +\tif (S_ISLNK(iter->st.st_mode))\n> > +\t\tgoto cleanup;\n> > +\n> > +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n> > +\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n> > +\t\t\t\t \"cannot read ref file '%s': %s\",\n> > +\t\t\t\t iter->path.buf, strerror(errno));\n> > +\t\tgoto cleanup;\n> > +\t}\n> \n> I didn't catch this in previous rounds, but it's a little dubious\n> whether we should report this as an actual fsck error. I can expect\n> multiple situations:\n> \n> - The file has weird permissions and thus cannot be read, failing with\n> EPERM, which doesn't match well with BAD_REF_CONTENT.\n> \n> - The file does not exist anymore because we were racing with a\n> concurrent writer, failing with ENOENT. This is benign and expected\n> to happen in busy repos, so generating an error here feels wrong.\n> \n> - The file cannot be read at all due to an I/O error. This may be\n> reported with BAD_REF_CONTENT, but conflating this with the case\n> where we have actually bad content may not be the best idea.\n> \n> So maybe we should ignore ENOENT, report bad permissions and otherwise\n> return an actual error to the caller?\n> \n\nSo, I think we should just use \"error_errno\" method to report the actual\nerror to the caller. And we also need to add some comments.\n\nThanks for this wonderful suggestion.\n\n\n> Patrick\n"},{"id":"507267","messageId":"ZzXqeLGF5dfSlh1s@ArchLinux","threadId":"61943","inReplyTo":"ZzRW8M39-hw1E9-h@pks.im","subject":"Re: [PATCH v7 9/9] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T12:18:00Z","receivedAt":"2024-11-14T12:17:58Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Wed, Nov 13, 2024 at 08:36:16AM +0100, Patrick Steinhardt wrote:\n> On Sun, Nov 10, 2024 at 08:10:27PM +0800, shejialuo wrote:\n> > @@ -3572,8 +3579,30 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n> > \n> > \treport.path = target_name;\n> > \n> > -\tif (S_ISLNK(iter->st.st_mode))\n> > +\tif (S_ISLNK(iter->st.st_mode)) {\n> > +\t\tconst char* relative_referent_path = NULL;\n> \n> Nit: the asterisk should stick with the variable name.\n> \n\nI will improve this in the next version.\n\n> > +\t\tret = fsck_report_ref(o, &report,\n> > +\t\t\t\t FSCK_MSG_SYMLINK_REF,\n> > +\t\t\t\t \"use deprecated symbolic link for symref\");\n> > +\n> > +\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n> > +\t\tstrbuf_normalize_path(&abs_gitdir);\n> > +\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n> > +\t\t\tstrbuf_addch(&abs_gitdir, '/');\n> > +\n> > +\t\tstrbuf_add_real_path(&ref_content, iter->path.buf);\n> > +\t\tskip_prefix(ref_content.buf, abs_gitdir.buf,\n> > +\t\t\t &relative_referent_path);\n> > +\n> > +\t\tif (relative_referent_path)\n> > +\t\t\tstrbuf_addstr(&referent, relative_referent_path);\n> > +\t\telse\n> > +\t\t\tstrbuf_addbuf(&referent, &ref_content);\n> > +\n> > +\t\tret |= files_fsck_symref_target(o, &report, &referent, 1);\n> > \t\tgoto cleanup;\n> > +\t}\n> \n> I wonder whether this logic works as expected with per-worktree symbolic\n> refs which are a symlink. On the other hand I wonder whether those work\n> as expected in the first place. Probably not. *shrug*\n> \n> In any case, it would be nice to have a test for this.\n> \n\nCorrect, I have ignored because I add worktree support in the later\nversion. Let me add a new test to verify this.\n\n> Patrick\n\nThanks,\nJialuo\n"},{"id":"507273","messageId":"ZzYqoai8X_Wdtbmt@ArchLinux","threadId":"61943","inReplyTo":"ZzCiCGxL4Adnd_eq@ArchLinux","subject":"[PATCH v8 0/9] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:51:45Z","receivedAt":"2024-11-14T16:51:44Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi all:\n\nThis new version solves the following problem:\n\n1. when reading the content of the ref file, we do not use\n\"fsck_report_ref\" function. It's not suitable.\n2. Add a new test for symlink worktree test in the last patch. After\nwriting the tets, find a bug. Fix the bug described below.\n\nBecause we have introduced the check for worktrees, we should not use\n\"ref_store->gitdir\", instead we need to use \"ref_store->repo->gitdir\" to\nget the main worktree \"gitdir\". After fixing this, the test is passed.\n\nThank Patrick to remind me about this. I forgot to add test thus making\nmistakes.\n\nThanks,\nJialuo\n\nshejialuo (9):\n ref: initialize \"fsck_ref_report\" with zero\n ref: check the full refname instead of basename\n ref: initialize ref name outside of check functions\n ref: support multiple worktrees check for refs\n ref: port git-fsck(1) regular refs check for files backend\n ref: add more strict checks for regular refs\n ref: add basic symref content check for files backend\n ref: check whether the target of the symref is a ref\n ref: add symlink ref content check for files backend\n\n Documentation/fsck-msgids.txt | 35 +++\n builtin/refs.c | 10 +-\n fsck.h | 6 +\n refs.c | 7 +-\n refs.h | 3 +-\n refs/debug.c | 5 +-\n refs/files-backend.c | 195 +++++++++++-\n refs/packed-backend.c | 8 +-\n refs/refs-internal.h | 5 +-\n refs/reftable-backend.c | 3 +-\n t/t0602-reffiles-fsck.sh | 576 ++++++++++++++++++++++++++++++++--\n 11 files changed, 791 insertions(+), 62 deletions(-)\n\nRange-diff against v7:\n 1: bfb2a21af4 = 1: bfb2a21af4 ref: initialize \"fsck_ref_report\" with zero\n 2: 9efc83f7ea = 2: 9efc83f7ea ref: check the full refname instead of basename\n 3: 5ea7d18203 = 3: 5ea7d18203 ref: initialize ref name outside of check functions\n 4: cb4669b64d = 4: cb4669b64d ref: support multiple worktrees check for refs\n 5: 4e1add6465 ! 5: c6c128c922 ref: port git-fsck(1) regular refs check for files backend\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\tif (S_ISLNK(iter->st.st_mode))\n +\t\tgoto cleanup;\n +\n -+\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n -+\t\tret = fsck_report_ref(o, &report,\n -+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n -+\t\t\t\t \"cannot read ref file '%s': %s\",\n -+\t\t\t\t iter->path.buf, strerror(errno));\n ++\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0 ) {\n ++\t\t/*\n ++\t\t * Ref file could be removed by another concurrent process. We should\n ++\t\t * ignore this error and continue to the next ref.\n ++\t\t */\n ++\t\tif (errno == ENOENT)\n ++\t\t\tgoto cleanup;\n ++\n ++\t\tret = error_errno(_(\"cannot read ref file '%s': %s\"),\n ++\t\t\t\t iter->path.buf, strerror(errno));\n +\t\tgoto cleanup;\n +\t}\n +\n 6: 945322fab7 = 6: 911fa42717 ref: add more strict checks for regular refs\n 7: 3006eb9431 = 7: 7aa6a99206 ref: add basic symref content check for files backend\n 8: c59d003d78 = 8: dbb0787ad1 ref: check whether the target of the symref is a ref\n 9: bb6d7f3323 ! 9: a6d85b4864 ref: add symlink ref content check for files backend\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_s\n \n -\tif (S_ISLNK(iter->st.st_mode))\n +\tif (S_ISLNK(iter->st.st_mode)) {\n -+\t\tconst char* relative_referent_path = NULL;\n ++\t\tconst char *relative_referent_path = NULL;\n +\n +\t\tret = fsck_report_ref(o, &report,\n +\t\t\t\t FSCK_MSG_SYMLINK_REF,\n +\t\t\t\t \"use deprecated symbolic link for symref\");\n +\n -+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->gitdir);\n ++\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->repo->gitdir);\n +\t\tstrbuf_normalize_path(&abs_gitdir);\n +\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n +\t\t\tstrbuf_addch(&abs_gitdir, '/');\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_s\n \t\tgoto cleanup;\n +\t}\n \n - \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n - \t\tret = fsck_report_ref(o, &report,\n + \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0 ) {\n + \t\t/*\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\tgoto cleanup;\n \t\t}\n @@ t/t0602-reffiles-fsck.sh: test_expect_success 'the target of the textual symref\n +\trm $tag_dir_prefix/tag-symbolic-1 &&\n +\ttest_cmp expect err\n +'\n ++\n ++test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n ++\ttest_when_finished \"rm -rf repo\" &&\n ++\tgit init repo &&\n ++\tcd repo &&\n ++\ttest_commit default &&\n ++\tgit branch branch-1 &&\n ++\tgit branch branch-2 &&\n ++\tgit branch branch-3 &&\n ++\tgit worktree add ./worktree-1 branch-2 &&\n ++\tgit worktree add ./worktree-2 branch-3 &&\n ++\tmain_worktree_refdir_prefix=.git/refs/heads &&\n ++\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n ++\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n ++\n ++\t(\n ++\t\tcd worktree-1 &&\n ++\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n ++\t) &&\n ++\t(\n ++\t\tcd worktree-2 &&\n ++\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n ++\t) &&\n ++\n ++\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n ++\tgit refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n ++\tEOF\n ++\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n ++\ttest_cmp expect err &&\n ++\n ++\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n ++\tgit refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n ++\tEOF\n ++\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n ++\ttest_cmp expect err &&\n ++\n ++\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n ++\tgit refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n ++\tEOF\n ++\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n ++\ttest_cmp expect err &&\n ++\n ++\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n ++\tgit refs verify 2>err &&\n ++\tcat >expect <<-EOF &&\n ++\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n ++\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n ++\tEOF\n ++\trm $worktree1_refdir_prefix/branch-symbolic &&\n ++\ttest_cmp expect err &&\n ++\n ++\tfor bad_referent_name in \".tag\" \"branch \"\n ++\tdo\n ++\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n ++\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n ++\t\tEOF\n ++\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n ++\t\ttest_cmp expect err &&\n ++\n ++\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n ++\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n ++\t\tEOF\n ++\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n ++\t\ttest_cmp expect err &&\n ++\n ++\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n ++\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n ++\t\tEOF\n ++\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n ++\t\ttest_cmp expect err &&\n ++\n ++\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n ++\t\ttest_must_fail git refs verify 2>err &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n ++\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n ++\t\tEOF\n ++\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n ++\t\ttest_cmp expect err || return 1\n ++\tdone\n ++'\n +\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n-- \n2.47.0\n\n"},{"id":"507274","messageId":"ZzYrJT58VlU7slBo@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v8 1/9] ref: initialize \"fsck_ref_report\" with zero","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:53:57Z","receivedAt":"2024-11-14T16:53:55Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n\"referent\" is NULL. So, we need to always initialize these parameters to\nNULL instead of letting them point to anywhere when creating a new\n\"fsck_ref_report\" structure.\n\nThe original code explicitly initializes the \"path\" member in the\n\"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\nmembers in the struct). It is more customary to use \"{ 0 }\" to express\nthat we are 0-initializing everything. In order to align with the\ncodebase, initialize \"fsck_ref_report\" with zero.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0824c0b8a9..03d2503276 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3520,7 +3520,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tstruct fsck_ref_report report = { .path = NULL };\n+\t\tstruct fsck_ref_report report = { 0 };\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n-- \n2.47.0\n\n"},{"id":"507275","messageId":"ZzYrLDkcm3tgeCDe@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v8 2/9] ref: check the full refname instead of basename","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:54:04Z","receivedAt":"2024-11-14T16:54:03Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"files-backend.c::files_fsck_refs_name\", we validate the refname\nformat by using \"check_refname_format\" to check the basename of the\niterator with \"REFNAME_ALLOW_ONELEVEL\" flag.\n\nHowever, this is a bad implementation. Although we doesn't allow a\nsingle \"@\" in \".git\" directory, we do allow \"refs/heads/@\". So, we will\nreport an error wrongly when there is a \"refs/heads/@\" ref by using one\nlevel refname \"@\".\n\nBecause we just check one level refname, we either cannot check the\nother parts of the full refname. And we will ignore the following\nerrors:\n\n \"refs/heads/ new-feature/test\"\n \"refs/heads/~new-feature/test\"\n\nIn order to fix the above problem, enhance \"files_fsck_refs_name\" to use\nthe full name for \"check_refname_format\". Then, replace the tests which\nare related to \"@\" and add tests to exercise the above situations using\nfor loop to avoid repetition.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 7 ++-\n t/t0602-reffiles-fsck.sh | 92 ++++++++++++++++++++++++----------------\n 2 files changed, 60 insertions(+), 39 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 03d2503276..b055edc061 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3519,10 +3519,13 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \tif (iter->basename[0] != '.' && ends_with(iter->basename, \".lock\"))\n \t\tgoto cleanup;\n \n-\tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n+\t/*\n+\t * This works right now because we never check the root refs.\n+\t */\n+\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n+\tif (check_refname_format(sb.buf, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n-\t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 71a4d1a5ae..2a172c913d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -18,63 +18,81 @@ test_expect_success 'ref name should be checked' '\n \tcd repo &&\n \n \tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\tgit tag tag-1 &&\n-\tgit commit --allow-empty -m second &&\n-\tgit checkout -b branch-2 &&\n-\tgit tag tag-2 &&\n-\tgit tag multi_hierarchy/tag-2 &&\n+\tgit checkout -b default-branch &&\n+\tgit tag default-tag &&\n+\tgit tag multi_hierarchy/default-tag &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/@: badRefName: invalid refname format\n-\tEOF\n+\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n \trm $branch_dir_prefix/@ &&\n-\ttest_cmp expect err &&\n \n-\tcp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/@ &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/multi_hierarchy/@: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/multi_hierarchy/@ &&\n-\ttest_cmp expect err &&\n-\n-\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock &&\n+\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n \tgit refs verify 2>err &&\n \trm $tag_dir_prefix/tag-1.lock &&\n \ttest_must_be_empty err &&\n \n-\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/.lock &&\n+\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \terror: refs/tags/.lock: badRefName: invalid refname format\n \tEOF\n \trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err\n+\ttest_cmp expect err &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\ttag_dir_prefix=.git/refs/tags &&\n \tcd repo &&\n \tgit commit --allow-empty -m initial &&\n \tgit checkout -b branch-1 &&\n-\tgit tag tag-1 &&\n-\tgit commit --allow-empty -m second &&\n-\tgit checkout -b branch-2 &&\n-\tgit tag tag-2 &&\n \n \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n \tgit -c fsck.badRefName=warn refs verify 2>err &&\n@@ -84,7 +102,7 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \trm $branch_dir_prefix/.branch-1 &&\n \ttest_cmp expect err &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n \tgit -c fsck.badRefName=ignore refs verify 2>err &&\n \ttest_must_be_empty err\n '\n-- \n2.47.0\n\n"},{"id":"507276","messageId":"ZzYrNBzugqMRU_Ty@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v8 3/9] ref: initialize ref name outside of check functions","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:54:12Z","receivedAt":"2024-11-14T16:54:12Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We passes \"refs_check_dir\" to the \"files_fsck_refs_name\" function which\nallows it to create the checked ref name later. However, when we\nintroduce a new check function, we have to allocate redundant memory and\nre-calculate the ref name. It's bad for us to allocate redundant memory\nand duplicate logic. Instead, we should allocate and calculate it only\nonce and pass the ref name to the check functions.\n\nIn order not to do repeat calculation, rename \"refs_check_dir\" to\n\"refname\". And in \"files_fsck_refs_dir\", create a new strbuf \"refname\",\nthus whenever we handle a new ref, calculate the name and call the check\nfunctions one by one.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 21 +++++++++++++--------\n 1 file changed, 13 insertions(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b055edc061..8edb700568 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3501,12 +3501,12 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store,\n */\n typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n-\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n-\t\t\t\tconst char *refs_check_dir,\n+\t\t\t\tconst char *refname,\n \t\t\t\tstruct dir_iterator *iter)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -3522,11 +3522,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t/*\n \t * This works right now because we never check the root refs.\n \t */\n-\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n-\tif (check_refname_format(sb.buf, 0)) {\n+\tif (check_refname_format(refname, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n-\t\treport.path = sb.buf;\n+\t\treport.path = refname;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\n \t\t\t\t \"invalid refname format\");\n@@ -3542,6 +3541,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\t const char *refs_check_dir,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n+\tstruct strbuf refname = STRBUF_INIT;\n \tstruct strbuf sb = STRBUF_INIT;\n \tstruct dir_iterator *iter;\n \tint iter_status;\n@@ -3560,11 +3560,15 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\tcontinue;\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n+\t\t\tstrbuf_reset(&refname);\n+\t\t\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir,\n+\t\t\t\t iter->relative_path);\n+\n \t\t\tif (o->verbose)\n-\t\t\t\tfprintf_ln(stderr, \"Checking %s/%s\",\n-\t\t\t\t\t refs_check_dir, iter->relative_path);\n+\t\t\t\tfprintf_ln(stderr, \"Checking %s\", refname.buf);\n+\n \t\t\tfor (size_t i = 0; fsck_refs_fn[i]; i++) {\n-\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refs_check_dir, iter))\n+\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refname.buf, iter))\n \t\t\t\t\tret = -1;\n \t\t\t}\n \t\t} else {\n@@ -3581,6 +3585,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \n out:\n \tstrbuf_release(&sb);\n+\tstrbuf_release(&refname);\n \treturn ret;\n }\n \n-- \n2.47.0\n\n"},{"id":"507277","messageId":"ZzYrOyvgyS8NWSzO@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v8 4/9] ref: support multiple worktrees check for refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:54:19Z","receivedAt":"2024-11-14T16:54:18Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already set up the infrastructure to check the consistency for\nrefs, but we do not support multiple worktrees. However, \"git-fsck(1)\"\nwill check the refs of worktrees. As we decide to get feature parity\nwith \"git-fsck(1)\", we need to set up support for multiple worktrees.\n\nBecause each worktree has its own specific refs, instead of just showing\nthe users \"refs/worktree/foo\", we need to display the full name such as\n\"worktrees//refs/worktree/foo\". So we should know the id of the\nworktree to get the full name. Add a new parameter \"struct worktree *\"\nfor \"refs-internal.h::fsck_fn\". Then change the related functions to\nfollow this new interface.\n\nThe \"packed-refs\" only exists in the main worktree, so we should only\ncheck \"packed-refs\" in the main worktree. Use \"is_main_worktree\" method\nto skip checking \"packed-refs\" in \"packed_fsck\" function.\n\nThen, enhance the \"files-backend.c::files_fsck_refs_dir\" function to add\n\"worktree//\" prefix when we are not in the main worktree.\n\nLast, add a new test to check the refname when there are multiple\nworktrees to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 10 ++++++--\n refs.c | 5 ++--\n refs.h | 3 ++-\n refs/debug.c | 5 ++--\n refs/files-backend.c | 17 ++++++++++----\n refs/packed-backend.c | 8 ++++++-\n refs/refs-internal.h | 3 ++-\n refs/reftable-backend.c | 3 ++-\n t/t0602-reffiles-fsck.sh | 51 ++++++++++++++++++++++++++++++++++++++++\n 9 files changed, 90 insertions(+), 15 deletions(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex 24978a7b7b..394b4101c6 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -5,6 +5,7 @@\n #include \"parse-options.h\"\n #include \"refs.h\"\n #include \"strbuf.h\"\n+#include \"worktree.h\"\n \n #define REFS_MIGRATE_USAGE \\\n \tN_(\"git refs migrate --ref-format= [--dry-run]\")\n@@ -66,6 +67,7 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)\n static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n {\n \tstruct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;\n+\tstruct worktree **worktrees;\n \tconst char * const verify_usage[] = {\n \t\tREFS_VERIFY_USAGE,\n \t\tNULL,\n@@ -75,7 +77,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \t\tOPT_BOOL(0, \"strict\", &fsck_refs_options.strict, N_(\"enable strict checking\")),\n \t\tOPT_END(),\n \t};\n-\tint ret;\n+\tint ret = 0;\n \n \targc = parse_options(argc, argv, prefix, options, verify_usage, 0);\n \tif (argc)\n@@ -84,9 +86,13 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);\n+\tworktrees = get_worktrees();\n+\tfor (size_t i = 0; worktrees[i]; i++)\n+\t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n+\t\t\t\t &fsck_refs_options, worktrees[i]);\n \n \tfsck_options_clear(&fsck_refs_options);\n+\tfree_worktrees(worktrees);\n \treturn ret;\n }\n \ndiff --git a/refs.c b/refs.c\nindex 5f729ed412..395a17273c 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -318,9 +318,10 @@ int check_refname_format(const char *refname, int flags)\n \treturn check_or_sanitize_refname(refname, flags, NULL);\n }\n \n-int refs_fsck(struct ref_store *refs, struct fsck_options *o)\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt)\n {\n-\treturn refs->be->fsck(refs, o);\n+\treturn refs->be->fsck(refs, o, wt);\n }\n \n void sanitize_refname_component(const char *refname, struct strbuf *out)\ndiff --git a/refs.h b/refs.h\nindex 108dfc93b3..341d43239c 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -549,7 +549,8 @@ int check_refname_format(const char *refname, int flags);\n * reflogs are consistent, and non-zero otherwise. The errors will be\n * written to stderr.\n */\n-int refs_fsck(struct ref_store *refs, struct fsck_options *o);\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt);\n \n /*\n * Apply the rules from check_refname_format, but mutate the result until it\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 45e2e784a0..72e80ddd6d 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -420,10 +420,11 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,\n }\n \n static int debug_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n-\tint res = drefs->refs->be->fsck(drefs->refs, o);\n+\tint res = drefs->refs->be->fsck(drefs->refs, o, wt);\n \ttrace_printf_key(&trace_refs, \"fsck: %d\\n\", res);\n \treturn res;\n }\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8edb700568..8bfdce64bc 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -23,6 +23,7 @@\n #include \"../dir.h\"\n #include \"../chdir-notify.h\"\n #include \"../setup.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../revision.h\"\n@@ -3539,6 +3540,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\t struct fsck_options *o,\n \t\t\t const char *refs_check_dir,\n+\t\t\t struct worktree *wt,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -3561,6 +3563,9 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n \t\t\tstrbuf_reset(&refname);\n+\n+\t\t\tif (!is_main_worktree(wt))\n+\t\t\t\tstrbuf_addf(&refname, \"worktrees/%s/\", wt->id);\n \t\t\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir,\n \t\t\t\t iter->relative_path);\n \n@@ -3590,7 +3595,8 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n }\n \n static int files_fsck_refs(struct ref_store *ref_store,\n-\t\t\t struct fsck_options *o)\n+\t\t\t struct fsck_options *o,\n+\t\t\t struct worktree *wt)\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n@@ -3599,17 +3605,18 @@ static int files_fsck_refs(struct ref_store *ref_store,\n \n \tif (o->verbose)\n \t\tfprintf_ln(stderr, _(\"Checking references consistency\"));\n-\treturn files_fsck_refs_dir(ref_store, o, \"refs\", fsck_refs_fn);\n+\treturn files_fsck_refs_dir(ref_store, o, \"refs\", wt, fsck_refs_fn);\n }\n \n static int files_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"fsck\");\n \n-\treturn files_fsck_refs(ref_store, o) |\n-\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n+\treturn files_fsck_refs(ref_store, o, wt) |\n+\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o, wt);\n }\n \n struct ref_storage_be refs_be_files = {\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 07c57fd541..46dcaec654 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -13,6 +13,7 @@\n #include \"../lockfile.h\"\n #include \"../chdir-notify.h\"\n #include \"../statinfo.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../trace2.h\"\n@@ -1754,8 +1755,13 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n }\n \n static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED)\n+\t\t struct fsck_options *o UNUSED,\n+\t\t struct worktree *wt)\n {\n+\n+\tif (!is_main_worktree(wt))\n+\t\treturn 0;\n+\n \treturn 0;\n }\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 2313c830d8..037d7991cd 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -653,7 +653,8 @@ typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refnam\n \t\t\t\t struct strbuf *referent);\n \n typedef int fsck_fn(struct ref_store *ref_store,\n-\t\t struct fsck_options *o);\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt);\n \n struct ref_storage_be {\n \tconst char *name;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex f5f957e6de..b6a63c1015 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -2443,7 +2443,8 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n }\n \n static int reftable_be_fsck(struct ref_store *ref_store UNUSED,\n-\t\t\t struct fsck_options *o UNUSED)\n+\t\t\t struct fsck_options *o UNUSED,\n+\t\t\t struct worktree *wt UNUSED)\n {\n \treturn 0;\n }\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 2a172c913d..1e17393a3d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -107,4 +107,55 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'ref name check should work for multiple worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\n+\tcd repo &&\n+\ttest_commit initial &&\n+\tgit checkout -b branch-1 &&\n+\ttest_commit second &&\n+\tgit checkout -b branch-2 &&\n+\ttest_commit third &&\n+\tgit checkout -b branch-3 &&\n+\tgit worktree add ./worktree-1 branch-1 &&\n+\tgit worktree add ./worktree-2 branch-2 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\n+\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err &&\n+\n+\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\tdo\n+\t\t(\n+\t\t\tcd $worktree &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\tsort err >sorted_err &&\n+\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t)\n+\tdone\n+'\n+\n test_done\n-- \n2.47.0\n\n"},{"id":"507278","messageId":"ZzYrRExrs17rapOb@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v8 5/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:54:28Z","receivedAt":"2024-11-14T16:54:27Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"git-fsck(1)\" implicitly checks the ref content by passing the\ncallback \"fsck_handle_ref\" to the \"refs.c::refs_for_each_rawref\".\nThen, it will check whether the ref content (eventually \"oid\")\nis valid. If not, it will report the following error to the user.\n\n error: refs/heads/main: invalid sha1 pointer 0000...\n\nAnd it will also report above errors when there are dangling symrefs\nin the repository wrongly. This does not align with the behavior of\nthe \"git symbolic-ref\" command which allows users to create dangling\nsymrefs.\n\nAs we have already introduced the \"git refs verify\" command, we'd better\ncheck the ref content explicitly in the \"git refs verify\" command thus\nlater we could remove these checks in \"git-fsck(1)\" and launch a\nsubprocess to call \"git refs verify\" in \"git-fsck(1)\" to make the\n\"git-fsck(1)\" more clean.\n\nFollowing what \"git-fsck(1)\" does, add a similar check to \"git refs\nverify\". Then add a new fsck error message \"badRefContent(ERROR)\" to\nrepresent that a ref has an invalid content.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/files-backend.c | 48 ++++++++++++++++\n t/t0602-reffiles-fsck.sh | 105 ++++++++++++++++++++++++++++++++++\n 4 files changed, 157 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 68a2801f15..22c385ea22 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -19,6 +19,9 @@\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n+`badRefContent`::\n+\t(ERROR) A ref has bad content.\n+\n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n \ndiff --git a/fsck.h b/fsck.h\nindex 500b4c04d2..0d99a87911 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -31,6 +31,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8bfdce64bc..f81b4c8dd5 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3505,6 +3505,53 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_refs_content(struct ref_store *ref_store,\n+\t\t\t\t struct fsck_options *o,\n+\t\t\t\t const char *target_name,\n+\t\t\t\t struct dir_iterator *iter)\n+{\n+\tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tunsigned int type = 0;\n+\tint failure_errno = 0;\n+\tstruct object_id oid;\n+\tint ret = 0;\n+\n+\treport.path = target_name;\n+\n+\tif (S_ISLNK(iter->st.st_mode))\n+\t\tgoto cleanup;\n+\n+\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0 ) {\n+\t\t/*\n+\t\t * Ref file could be removed by another concurrent process. We should\n+\t\t * ignore this error and continue to the next ref.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tret = error_errno(_(\"cannot read ref file '%s': %s\"),\n+\t\t\t\t iter->path.buf, strerror(errno));\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n+\t\t\t\t ref_content.buf, &oid, &referent,\n+\t\t\t\t &type, &failure_errno)) {\n+\t\tstrbuf_rtrim(&ref_content);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"%s\", ref_content.buf);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&ref_content);\n+\tstrbuf_release(&referent);\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n \t\t\t\tconst char *refname,\n@@ -3600,6 +3647,7 @@ static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n+\t\tfiles_fsck_refs_content,\n \t\tNULL,\n \t};\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 1e17393a3d..162370077b 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -158,4 +158,109 @@ test_expect_success 'ref name check should work for multiple worktrees' '\n \tdone\n '\n \n+test_expect_success 'regular ref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n+\n+\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-bad &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n+test_expect_success 'regular ref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tbad_content_1=$(git rev-parse main)x &&\n+\tbad_content_2=xfsazqfxcadas &&\n+\tbad_content_3=Xfsazqfxcadas &&\n+\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n+test_expect_success 'ref content checks should work with worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit branch branch-3 &&\n+\tgit worktree add ./worktree-1 branch-2 &&\n+\tgit worktree add ./worktree-2 branch-3 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\n+\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_done\n-- \n2.47.0\n\n"},{"id":"507279","messageId":"ZzYrTNiAoDHf4Qz_@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v8 6/9] ref: add more strict checks for regular refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:54:36Z","receivedAt":"2024-11-14T16:54:36Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already used \"parse_loose_ref_contents\" function to check\nwhether the ref content is valid in files backend. However, by\nusing \"parse_loose_ref_contents\", we allow the ref's content to end with\ngarbage or without a newline.\n\nEven though we never create such loose refs ourselves, we have accepted\nsuch loose refs. So, it is entirely possible that some third-party tools\nmay rely on such loose refs being valid. We should not report an error\nfsck message at current. We should notify the users about such\n\"curiously formatted\" loose refs so that adequate care is taken before\nwe decide to tighten the rules in the future.\n\nAnd it's not suitable either to report a warn fsck message to the user.\nWe don't yet want the \"--strict\" flag that controls this bit to end up\ngenerating errors for such weirdly-formatted reference contents, as we\nfirst want to assess whether this retroactive tightening will cause\nissues for any tools out there. It may cause compatibility issues which\nmay break the repository. So, we add the following two fsck infos to\nrepresent the situation where the ref content ends without newline or\nhas trailing garbages:\n\n1. refMissingNewline(INFO): A loose ref that does not end with\n newline(LF).\n2. trailingRefContent(INFO): A loose ref has trailing content.\n\nIt might appear that we can't provide the user with any warnings by\nusing FSCK_INFO. However, in \"fsck.c::fsck_vreport\", we will convert\nFSCK_INFO to FSCK_WARN and we can still warn the user about these\nsituations when using \"git refs verify\" without introducing\ncompatibility issues.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 14 +++++++++\n fsck.h | 2 ++\n refs.c | 2 +-\n refs/files-backend.c | 26 ++++++++++++++--\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 57 +++++++++++++++++++++++++++++++++--\n 6 files changed, 96 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 22c385ea22..6db0eaa84a 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -173,6 +173,20 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`refMissingNewline`::\n+\t(INFO) A loose ref that does not end with newline(LF). As\n+\tvalid implementations of Git never created such a loose ref\n+\tfile, it may become an error in the future. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as\n+\twe need to know what tools created such a file.\n+\n+`trailingRefContent`::\n+\t(INFO) A loose ref has trailing content. As valid implementations\n+\tof Git never created such a loose ref file, it may become an\n+\terror in the future. Report to the git@vger.kernel.org mailing\n+\tlist if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \ndiff --git a/fsck.h b/fsck.h\nindex 0d99a87911..b85072df57 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -85,6 +85,8 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs.c b/refs.c\nindex 395a17273c..f88b32a633 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1789,7 +1789,7 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \t}\n \n \tresult = parse_loose_ref_contents(ref_store->repo->hash_algo, content.buf,\n-\t\t\t\t\t oid, referent, type, failure_errno);\n+\t\t\t\t\t oid, referent, type, NULL, failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f81b4c8dd5..a325b102b8 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -569,7 +569,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname,\n \tbuf = sb_contents.buf;\n \n \tret = parse_loose_ref_contents(ref_store->repo->hash_algo, buf,\n-\t\t\t\t oid, referent, type, &myerr);\n+\t\t\t\t oid, referent, type, NULL, &myerr);\n \n out:\n \tif (ret && !myerr)\n@@ -606,7 +606,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno)\n+\t\t\t const char **trailing, int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -628,6 +628,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n+\n+\tif (trailing)\n+\t\t*trailing = p;\n+\n \treturn 0;\n }\n \n@@ -3513,6 +3517,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n+\tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n \tstruct object_id oid;\n@@ -3538,7 +3543,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n \t\t\t\t ref_content.buf, &oid, &referent,\n-\t\t\t\t &type, &failure_errno)) {\n+\t\t\t\t &type, &trailing, &failure_errno)) {\n \t\tstrbuf_rtrim(&ref_content);\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n@@ -3546,6 +3551,21 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (!(type & REF_ISSYMREF)) {\n+\t\tif (!*trailing) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t \"misses LF at the end\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n cleanup:\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 037d7991cd..125f1fe735 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -716,7 +716,7 @@ struct ref_store {\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno);\n+\t\t\t const char **trailing, int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 162370077b..33e7a390ad 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -189,7 +189,48 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \t\tEOF\n \t\trm $branch_dir_prefix/a/b/branch-bad &&\n \t\ttest_cmp expect err || return 1\n-\tdone\n+\tdone &&\n+\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tfor trailing_content in \" garbage\" \" more garbage\"\n+\tdo\n+\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\n+\n+\t'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage-special &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\n+\n+\t garbage'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage-special &&\n+\ttest_cmp expect err\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -207,12 +248,16 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n \tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n \tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n \terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n \terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err\n@@ -260,7 +305,15 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t\tEOF\n \t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n \t\ttest_cmp expect err || return 1\n-\tdone\n+\tdone &&\n+\n+\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\ttest_cmp expect err\n '\n \n test_done\n-- \n2.47.0\n\n"},{"id":"507280","messageId":"ZzYrVO-a5vQS9CGv@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v8 7/9] ref: add basic symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:54:44Z","receivedAt":"2024-11-14T16:54:43Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have code that checks regular ref contents, but we do not yet check\nthe contents of symbolic refs. By using \"parse_loose_ref_content\" for\nsymbolic refs, we will get the information of the \"referent\".\n\nWe do not need to check the \"referent\" by opening the file. This is\nbecause if \"referent\" exists in the file system, we will eventually\ncheck its correctness by inspecting every file in the \"refs\" directory.\nIf the \"referent\" does not exist in the filesystem, this is OK as it is\nseen as the dangling symref.\n\nSo we just need to check the \"referent\" string content. A regular ref\ncould be accepted as a textual symref if it begins with \"ref:\", followed\nby zero or more whitespaces, followed by the full refname, followed only\nby whitespace characters. However, we always write a single SP after\n\"ref:\" and a single LF after the refname. It may seem that we should\nreport a fsck error message when the \"referent\" does not apply above\nrules and we should not be so aggressive because third-party\nreimplementations of Git may have taken advantage of the looser syntax.\nPut it more specific, we accept the following contents:\n\n1. \"ref: refs/heads/master \"\n2. \"ref: refs/heads/master \\n \\n\"\n3. \"ref: refs/heads/master\\n\\n\"\n\nWhen introducing the regular ref content checks, we created two fsck\ninfos \"refMissingNewline\" and \"trailingRefContent\" which exactly\nrepresents above situations. So we will reuse these two fsck messages to\nwrite checks to info the user about these situations.\n\nBut we do not allow any other trailing garbage. The followings are bad\nsymref contents which will be reported as fsck error by \"git-fsck(1)\".\n\n1. \"ref: refs/heads/master garbage\\n\"\n2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n\nAnd we introduce a new \"badReferentName(ERROR)\" fsck message to report\nabove errors by using \"is_root_ref\" and \"check_refname_format\" to check\nthe \"referent\". Since both \"is_root_ref\" and \"check_refname_format\"\ndon't work with whitespaces, we use the trimmed version of \"referent\"\nwith these functions.\n\nIn order to add checks, we will do the following things:\n\n1. Record the untrimmed length \"orig_len\" and untrimmed last byte\n \"orig_last_byte\".\n2. Use \"strbuf_rtrim\" to trim the whitespaces or newlines to make sure\n \"is_root_ref\" and \"check_refname_format\" won't be failed by them.\n3. Use \"orig_len\" and \"orig_last_byte\" to check whether the \"referent\"\n misses '\\n' at the end or it has trailing whitespaces or newlines.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/files-backend.c | 40 ++++++++++++\n t/t0602-reffiles-fsck.sh | 111 ++++++++++++++++++++++++++++++++++\n 4 files changed, 155 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 6db0eaa84a..dcea05edfc 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,6 +28,9 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n+`badReferentName`::\n+\t(ERROR) The referent name of a symref is invalid.\n+\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n \ndiff --git a/fsck.h b/fsck.h\nindex b85072df57..5227dfdef2 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,6 +34,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n+\tFUNC(BAD_REFERENT_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex a325b102b8..c496006db1 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3509,6 +3509,43 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_symref_target(struct fsck_options *o,\n+\t\t\t\t struct fsck_ref_report *report,\n+\t\t\t\t struct strbuf *referent)\n+{\n+\tchar orig_last_byte;\n+\tsize_t orig_len;\n+\tint ret = 0;\n+\n+\torig_len = referent->len;\n+\torig_last_byte = referent->buf[orig_len - 1];\n+\tstrbuf_rtrim(referent);\n+\n+\tif (!is_root_ref(referent->buf) &&\n+\t check_refname_format(referent->buf, 0)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n+\t\t\t\t \"points to invalid refname '%s'\", referent->buf);\n+\t\tgoto out;\n+\t}\n+\n+\tif (referent->len == orig_len ||\n+\t (referent->len < orig_len && orig_last_byte != '\\n')) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t \"misses LF at the end\");\n+\t}\n+\n+\tif (referent->len != orig_len && referent->len != orig_len - 1) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t \"has trailing whitespaces or newlines\");\n+\t}\n+\n+out:\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *target_name,\n@@ -3564,6 +3601,9 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n \t\t\tgoto cleanup;\n \t\t}\n+\t} else {\n+\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tgoto cleanup;\n \t}\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 33e7a390ad..ee1e5f2864 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -263,6 +263,109 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'textual symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $branch_dir_prefix/branch-good &&\n+\t\ttest_must_be_empty err || return 1\n+\tdone &&\n+\n+\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-bad &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'textual symref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n@@ -313,6 +416,14 @@ test_expect_success 'ref content checks should work with worktrees' '\n \twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \trm $worktree1_refdir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-garbage &&\n \ttest_cmp expect err\n '\n \n-- \n2.47.0\n\n"},{"id":"507281","messageId":"ZzYrXZWGadkGzrv4@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v8 8/9] ref: check whether the target of the symref is a ref","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:54:53Z","receivedAt":"2024-11-14T16:54:52Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Ideally, we want to the users use \"git symbolic-ref\" to create symrefs\ninstead of writing raw contents into the filesystem. However, \"git\nsymbolic-ref\" is strict with the refname but not strict with the\nreferent. For example, we can make the \"referent\" located at the\n\"$(gitdir)/logs/aaa\" and manually write the content into this where we\ncan still successfully parse this symref by using \"git rev-parse\".\n\n $ git init repo && cd repo && git commit --allow-empty -mx\n $ git symbolic-ref refs/heads/test logs/aaa\n $ echo $(git rev-parse HEAD) > .git/logs/aaa\n $ git rev-parse test\n\nWe may need to add some restrictions for \"referent\" parameter when using\n\"git symbolic-ref\" to create symrefs because ideally all the\nnonpseudo-refs should be located under the \"refs\" directory and we may\ntighten this in the future.\n\nIn order to tell the user we may tighten the above situation, create\na new fsck message \"symrefTargetIsNotARef\" to notify the user that this\nmay become an error in the future.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 9 +++++++++\n fsck.h | 1 +\n refs/files-backend.c | 14 ++++++++++++--\n t/t0602-reffiles-fsck.sh | 29 +++++++++++++++++++++++++++++\n 4 files changed, 51 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex dcea05edfc..f82ebc58e8 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -183,6 +183,15 @@\n \tgit@vger.kernel.org mailing list if you see this error, as\n \twe need to know what tools created such a file.\n \n+`symrefTargetIsNotARef`::\n+\t(INFO) The target of a symbolic reference points neither to\n+\ta root reference nor to a reference starting with \"refs/\".\n+\tAlthough we allow create a symref pointing to the referent which\n+\tis outside the \"ref\" by using `git symbolic-ref`, we may tighten\n+\tthe rule in the future. Report to the git@vger.kernel.org\n+\tmailing list if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n `trailingRefContent`::\n \t(INFO) A loose ref has trailing content. As valid implementations\n \tof Git never created such a loose ref file, it may become an\ndiff --git a/fsck.h b/fsck.h\nindex 5227dfdef2..53a47612e6 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -87,6 +87,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex c496006db1..edf73d6cce 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3513,6 +3513,7 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n \t\t\t\t struct strbuf *referent)\n {\n+\tint is_referent_root;\n \tchar orig_last_byte;\n \tsize_t orig_len;\n \tint ret = 0;\n@@ -3521,8 +3522,17 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \torig_last_byte = referent->buf[orig_len - 1];\n \tstrbuf_rtrim(referent);\n \n-\tif (!is_root_ref(referent->buf) &&\n-\t check_refname_format(referent->buf, 0)) {\n+\tis_referent_root = is_root_ref(referent->buf);\n+\tif (!is_referent_root &&\n+\t !starts_with(referent->buf, \"refs/\") &&\n+\t !starts_with(referent->buf, \"worktrees/\")) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_SYMREF_TARGET_IS_NOT_A_REF,\n+\t\t\t\t \"points to non-ref target '%s'\", referent->buf);\n+\n+\t}\n+\n+\tif (!is_referent_root && check_refname_format(referent->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n \t\t\t\t \"points to invalid refname '%s'\", referent->buf);\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex ee1e5f2864..692b30727a 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -366,6 +366,35 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'the target of the textual symref should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $branch_dir_prefix/branch-good &&\n+\t\ttest_must_be_empty err || return 1\n+\tdone &&\n+\n+\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-- \n2.47.0\n\n"},{"id":"507282","messageId":"ZzYraA3_4STrj3Jm@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v8 9/9] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-14T16:55:04Z","receivedAt":"2024-11-14T16:55:03Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Besides the textual symref, we also allow symbolic links as the symref.\nSo, we should also provide the consistency check as what we have done\nfor textual symref. And also we consider deprecating writing the\nsymbolic links. We first need to access whether symbolic links still\nbe used. So, add a new fsck message \"symlinkRef(INFO)\" to tell the\nuser be aware of this information.\n\nWe have already introduced \"files_fsck_symref_target\". We should reuse\nthis function to handle the symrefs which use legacy symbolic links. We\nshould not check the trailing garbage for symbolic refs. Add a new\nparameter \"symbolic_link\" to disable some checks which should only be\nexecuted for textual symrefs.\n\nAnd we need to also generate the \"referent\" parameter for reusing\n\"files_fsck_symref_target\" by the following steps:\n\n1. Use \"strbuf_add_real_path\" to resolve the symlink and get the\n absolute path \"ref_content\" which the symlink ref points to.\n2. Generate the absolute path \"abs_gitdir\" of \"gitdir\" and combine\n \"ref_content\" and \"abs_gitdir\" to extract the relative path\n \"relative_referent_path\".\n3. If \"ref_content\" is outside of \"gitdir\", we just set \"referent\" with\n \"ref_content\". Instead, we set \"referent\" with\n \"relative_referent_path\".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 6 ++\n fsck.h | 1 +\n refs/files-backend.c | 38 ++++++++-\n t/t0602-reffiles-fsck.sh | 141 ++++++++++++++++++++++++++++++++++\n 4 files changed, 182 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex f82ebc58e8..b14bc44ca4 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -183,6 +183,12 @@\n \tgit@vger.kernel.org mailing list if you see this error, as\n \twe need to know what tools created such a file.\n \n+`symlinkRef`::\n+\t(INFO) A symbolic link is used as a symref. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as we\n+\tare assessing the feasibility of dropping the support to drop\n+\tcreating symbolic links as symrefs.\n+\n `symrefTargetIsNotARef`::\n \t(INFO) The target of a symbolic reference points neither to\n \ta root reference nor to a reference starting with \"refs/\".\ndiff --git a/fsck.h b/fsck.h\nindex 53a47612e6..a44c231a5f 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -86,6 +86,7 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(SYMLINK_REF, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n \tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex edf73d6cce..c715e411f3 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1,6 +1,7 @@\n #define USE_THE_REPOSITORY_VARIABLE\n \n #include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n #include \"../config.h\"\n #include \"../copy.h\"\n #include \"../environment.h\"\n@@ -3511,7 +3512,8 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n-\t\t\t\t struct strbuf *referent)\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t unsigned int symbolic_link)\n {\n \tint is_referent_root;\n \tchar orig_last_byte;\n@@ -3520,7 +3522,8 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \n \torig_len = referent->len;\n \torig_last_byte = referent->buf[orig_len - 1];\n-\tstrbuf_rtrim(referent);\n+\tif (!symbolic_link)\n+\t\tstrbuf_rtrim(referent);\n \n \tis_referent_root = is_root_ref(referent->buf);\n \tif (!is_referent_root &&\n@@ -3539,6 +3542,9 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n+\tif (symbolic_link)\n+\t\tgoto out;\n+\n \tif (referent->len == orig_len ||\n \t (referent->len < orig_len && orig_last_byte != '\\n')) {\n \t\tret = fsck_report_ref(o, report,\n@@ -3562,6 +3568,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct dir_iterator *iter)\n {\n \tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n \tconst char *trailing = NULL;\n@@ -3572,8 +3579,30 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \treport.path = target_name;\n \n-\tif (S_ISLNK(iter->st.st_mode))\n+\tif (S_ISLNK(iter->st.st_mode)) {\n+\t\tconst char *relative_referent_path = NULL;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_SYMLINK_REF,\n+\t\t\t\t \"use deprecated symbolic link for symref\");\n+\n+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->repo->gitdir);\n+\t\tstrbuf_normalize_path(&abs_gitdir);\n+\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n+\t\t\tstrbuf_addch(&abs_gitdir, '/');\n+\n+\t\tstrbuf_add_real_path(&ref_content, iter->path.buf);\n+\t\tskip_prefix(ref_content.buf, abs_gitdir.buf,\n+\t\t\t &relative_referent_path);\n+\n+\t\tif (relative_referent_path)\n+\t\t\tstrbuf_addstr(&referent, relative_referent_path);\n+\t\telse\n+\t\t\tstrbuf_addbuf(&referent, &ref_content);\n+\n+\t\tret |= files_fsck_symref_target(o, &report, &referent, 1);\n \t\tgoto cleanup;\n+\t}\n \n \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0 ) {\n \t\t/*\n@@ -3612,13 +3641,14 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\tgoto cleanup;\n \t\t}\n \t} else {\n-\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tret = files_fsck_symref_target(o, &report, &referent, 0);\n \t\tgoto cleanup;\n \t}\n \n cleanup:\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n+\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 692b30727a..f8f27cfc6c 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -395,6 +395,147 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tdone\n '\n \n+test_expect_success SYMLINKS 'symlink symref content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit branch branch-3 &&\n+\tgit worktree add ./worktree-1 branch-2 &&\n+\tgit worktree add ./worktree-2 branch-3 &&\n+\tmain_worktree_refdir_prefix=.git/refs/heads &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\n+\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tfor bad_referent_name in \".tag\" \"branch \"\n+\tdo\n+\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\tEOF\n+\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\tEOF\n+\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\tEOF\n+\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\tEOF\n+\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-- \n2.47.0\n\n"},{"id":"507320","messageId":"Zzbz_4Xqt39JZfgU@pks.im","threadId":"61943","inReplyTo":"ZzYrRExrs17rapOb@ArchLinux","subject":"Re: [PATCH v8 5/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-15T07:11:01Z","receivedAt":"2024-11-15T07:11:14Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Nov 15, 2024 at 12:54:28AM +0800, shejialuo wrote:\n> +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0 ) {\n\nNit: there's a space too much here now.\n\n> +\t\t/*\n> +\t\t * Ref file could be removed by another concurrent process. We should\n> +\t\t * ignore this error and continue to the next ref.\n> +\t\t */\n> +\t\tif (errno == ENOENT)\n> +\t\t\tgoto cleanup;\n> +\n> +\t\tret = error_errno(_(\"cannot read ref file '%s': %s\"),\n> +\t\t\t\t iter->path.buf, strerror(errno));\n> +\t\tgoto cleanup;\n> +\t}\n\nYou report `errno` twice. This should be:\n\n\tret = error_errno(_(\"cannot read ref file '%s'\"), iter->path.buf);\n\nOther than that this version looks good to me, thanks!\n\nPatrick\n"},{"id":"507356","messageId":"Zzcrkd0aetRjUHKA@ArchLinux","threadId":"61943","inReplyTo":"Zzbz_4Xqt39JZfgU@pks.im","subject":"Re: [PATCH v8 5/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-15T11:08:01Z","receivedAt":"2024-11-15T11:07:58Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Nov 15, 2024 at 08:11:01AM +0100, Patrick Steinhardt wrote:\n> On Fri, Nov 15, 2024 at 12:54:28AM +0800, shejialuo wrote:\n> > +\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0 ) {\n> \n> Nit: there's a space too much here now.\n> \n\nI will improve this in the next version.\n\n> > +\t\t/*\n> > +\t\t * Ref file could be removed by another concurrent process. We should\n> > +\t\t * ignore this error and continue to the next ref.\n> > +\t\t */\n> > +\t\tif (errno == ENOENT)\n> > +\t\t\tgoto cleanup;\n> > +\n> > +\t\tret = error_errno(_(\"cannot read ref file '%s': %s\"),\n> > +\t\t\t\t iter->path.buf, strerror(errno));\n> > +\t\tgoto cleanup;\n> > +\t}\n> \n> You report `errno` twice. This should be:\n> \n> \tret = error_errno(_(\"cannot read ref file '%s'\"), iter->path.buf);\n> \n> Other than that this version looks good to me, thanks!\n> \n\nOpps, I didn't think about it, I just copied it. I will fix this in the\nnext version.\n\n> Patrick\n"},{"id":"507357","messageId":"ZzcsC6MHi-BPyXLK@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"Re: [PATCH v8 0/9] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-15T11:10:03Z","receivedAt":"2024-11-15T11:10:01Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"On Fri, Nov 15, 2024 at 12:51:49AM +0800, shejialuo wrote:\n> Hi all:\n> \n> This new version solves the following problem:\n> \n> 1. when reading the content of the ref file, we do not use\n> \"fsck_report_ref\" function. It's not suitable.\n> 2. Add a new test for symlink worktree test in the last patch. After\n> writing the tets, find a bug. Fix the bug described below.\n> \n> Because we have introduced the check for worktrees, we should not use\n> \"ref_store->gitdir\", instead we need to use \"ref_store->repo->gitdir\" to\n> get the main worktree \"gitdir\". After fixing this, the test is passed.\n> \n> Thank Patrick to remind me about this. I forgot to add test thus making\n> mistakes.\n> \n> Thanks,\n> Jialuo\n\nI'd like to wait for couple of days for more reviews and comments from\nJunio and Karthik.\n\n"},{"id":"507717","messageId":"Zz3MON9_9DGD6nsy@ArchLinux","threadId":"61943","inReplyTo":"ZzYqoai8X_Wdtbmt@ArchLinux","subject":"[PATCH v9 0/9] add ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:47:04Z","receivedAt":"2024-11-20T11:46:57Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Hi All:\n\nThis version fixes two problems:\n\n1. Remove unnecessary space.\n2. Drop extra \"strerror(errno)\".\n\nThanks,\nJialuo\n\nshejialuo (9):\n ref: initialize \"fsck_ref_report\" with zero\n ref: check the full refname instead of basename\n ref: initialize ref name outside of check functions\n ref: support multiple worktrees check for refs\n ref: port git-fsck(1) regular refs check for files backend\n ref: add more strict checks for regular refs\n ref: add basic symref content check for files backend\n ref: check whether the target of the symref is a ref\n ref: add symlink ref content check for files backend\n\n Documentation/fsck-msgids.txt | 35 +++\n builtin/refs.c | 10 +-\n fsck.h | 6 +\n refs.c | 7 +-\n refs.h | 3 +-\n refs/debug.c | 5 +-\n refs/files-backend.c | 194 +++++++++++-\n refs/packed-backend.c | 8 +-\n refs/refs-internal.h | 5 +-\n refs/reftable-backend.c | 3 +-\n t/t0602-reffiles-fsck.sh | 576 ++++++++++++++++++++++++++++++++--\n 11 files changed, 790 insertions(+), 62 deletions(-)\n\nRange-diff against v8:\n 1: bfb2a21af4 = 1: bfb2a21af4 ref: initialize \"fsck_ref_report\" with zero\n 2: 9efc83f7ea = 2: 9efc83f7ea ref: check the full refname instead of basename\n 3: 5ea7d18203 = 3: 5ea7d18203 ref: initialize ref name outside of check functions\n 4: cb4669b64d = 4: cb4669b64d ref: support multiple worktrees check for refs\n 5: c6c128c922 ! 5: d6188063d9 ref: port git-fsck(1) regular refs check for files backend\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\tif (S_ISLNK(iter->st.st_mode))\n +\t\tgoto cleanup;\n +\n -+\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0 ) {\n ++\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n +\t\t/*\n +\t\t * Ref file could be removed by another concurrent process. We should\n +\t\t * ignore this error and continue to the next ref.\n @@ refs/files-backend.c: typedef int (*files_fsck_refs_fn)(struct ref_store *ref_st\n +\t\tif (errno == ENOENT)\n +\t\t\tgoto cleanup;\n +\n -+\t\tret = error_errno(_(\"cannot read ref file '%s': %s\"),\n -+\t\t\t\t iter->path.buf, strerror(errno));\n ++\t\tret = error_errno(_(\"cannot read ref file '%s'\"), iter->path.buf);\n +\t\tgoto cleanup;\n +\t}\n +\n 6: 911fa42717 = 6: e5e97ba3ad ref: add more strict checks for regular refs\n 7: 7aa6a99206 = 7: 1dec0a56d2 ref: add basic symref content check for files backend\n 8: dbb0787ad1 = 8: dcc4a02102 ref: check whether the target of the symref is a ref\n 9: a6d85b4864 ! 9: fc10862f6f ref: add symlink ref content check for files backend\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_s\n \t\tgoto cleanup;\n +\t}\n \n - \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0 ) {\n + \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n \t\t/*\n @@ refs/files-backend.c: static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\tgoto cleanup;\n-- \n2.47.0\n\n"},{"id":"507718","messageId":"Zz3NLB_Y-9ct9Kad@ArchLinux","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"[PATCH v9 1/9] ref: initialize \"fsck_ref_report\" with zero","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:51:08Z","receivedAt":"2024-11-20T11:51:00Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"fsck.c::fsck_refs_error_function\", we need to tell whether \"oid\" and\n\"referent\" is NULL. So, we need to always initialize these parameters to\nNULL instead of letting them point to anywhere when creating a new\n\"fsck_ref_report\" structure.\n\nThe original code explicitly initializes the \"path\" member in the\n\"struct fsck_ref_report\" to NULL (which implicitly 0-initializes other\nmembers in the struct). It is more customary to use \"{ 0 }\" to express\nthat we are 0-initializing everything. In order to align with the\ncodebase, initialize \"fsck_ref_report\" with zero.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 0824c0b8a9..03d2503276 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3520,7 +3520,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\tgoto cleanup;\n \n \tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n-\t\tstruct fsck_ref_report report = { .path = NULL };\n+\t\tstruct fsck_ref_report report = { 0 };\n \n \t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n-- \n2.47.0\n\n"},{"id":"507719","messageId":"Zz3NNLYmCKNyF06i@ArchLinux","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"[PATCH v9 2/9] ref: check the full refname instead of basename","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:51:16Z","receivedAt":"2024-11-20T11:51:08Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"In \"files-backend.c::files_fsck_refs_name\", we validate the refname\nformat by using \"check_refname_format\" to check the basename of the\niterator with \"REFNAME_ALLOW_ONELEVEL\" flag.\n\nHowever, this is a bad implementation. Although we doesn't allow a\nsingle \"@\" in \".git\" directory, we do allow \"refs/heads/@\". So, we will\nreport an error wrongly when there is a \"refs/heads/@\" ref by using one\nlevel refname \"@\".\n\nBecause we just check one level refname, we either cannot check the\nother parts of the full refname. And we will ignore the following\nerrors:\n\n \"refs/heads/ new-feature/test\"\n \"refs/heads/~new-feature/test\"\n\nIn order to fix the above problem, enhance \"files_fsck_refs_name\" to use\nthe full name for \"check_refname_format\". Then, replace the tests which\nare related to \"@\" and add tests to exercise the above situations using\nfor loop to avoid repetition.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 7 ++-\n t/t0602-reffiles-fsck.sh | 92 ++++++++++++++++++++++++----------------\n 2 files changed, 60 insertions(+), 39 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 03d2503276..b055edc061 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3519,10 +3519,13 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \tif (iter->basename[0] != '.' && ends_with(iter->basename, \".lock\"))\n \t\tgoto cleanup;\n \n-\tif (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {\n+\t/*\n+\t * This works right now because we never check the root refs.\n+\t */\n+\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n+\tif (check_refname_format(sb.buf, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n-\t\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n \t\treport.path = sb.buf;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 71a4d1a5ae..2a172c913d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -18,63 +18,81 @@ test_expect_success 'ref name should be checked' '\n \tcd repo &&\n \n \tgit commit --allow-empty -m initial &&\n-\tgit checkout -b branch-1 &&\n-\tgit tag tag-1 &&\n-\tgit commit --allow-empty -m second &&\n-\tgit checkout -b branch-2 &&\n-\tgit tag tag-2 &&\n-\tgit tag multi_hierarchy/tag-2 &&\n+\tgit checkout -b default-branch &&\n+\tgit tag default-tag &&\n+\tgit tag multi_hierarchy/default-tag &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/.branch-1: badRefName: invalid refname format\n-\tEOF\n-\trm $branch_dir_prefix/.branch-1 &&\n-\ttest_cmp expect err &&\n-\n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/heads/@: badRefName: invalid refname format\n-\tEOF\n+\tcp $branch_dir_prefix/default-branch $branch_dir_prefix/@ &&\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n \trm $branch_dir_prefix/@ &&\n-\ttest_cmp expect err &&\n \n-\tcp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/@ &&\n-\ttest_must_fail git refs verify 2>err &&\n-\tcat >expect <<-EOF &&\n-\terror: refs/tags/multi_hierarchy/@: badRefName: invalid refname format\n-\tEOF\n-\trm $tag_dir_prefix/multi_hierarchy/@ &&\n-\ttest_cmp expect err &&\n-\n-\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock &&\n+\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/tag-1.lock &&\n \tgit refs verify 2>err &&\n \trm $tag_dir_prefix/tag-1.lock &&\n \ttest_must_be_empty err &&\n \n-\tcp $tag_dir_prefix/tag-1 $tag_dir_prefix/.lock &&\n+\tcp $tag_dir_prefix/default-tag $tag_dir_prefix/.lock &&\n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \terror: refs/tags/.lock: badRefName: invalid refname format\n \tEOF\n \trm $tag_dir_prefix/.lock &&\n-\ttest_cmp expect err\n+\ttest_cmp expect err &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/$refname: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm \"$branch_dir_prefix/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tcp $tag_dir_prefix/default-tag \"$tag_dir_prefix/$refname\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/tags/$refname: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm \"$tag_dir_prefix/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tcp $tag_dir_prefix/multi_hierarchy/default-tag \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/tags/multi_hierarchy/$refname: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm \"$tag_dir_prefix/multi_hierarchy/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor refname in \".refname-starts-with-dot\" \"~refname-has-stride\"\n+\tdo\n+\t\tmkdir \"$branch_dir_prefix/$refname\" &&\n+\t\tcp $branch_dir_prefix/default-branch \"$branch_dir_prefix/$refname/default-branch\" &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/$refname/default-branch: badRefName: invalid refname format\n+\t\tEOF\n+\t\trm -r \"$branch_dir_prefix/$refname\" &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n '\n \n test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n \tbranch_dir_prefix=.git/refs/heads &&\n-\ttag_dir_prefix=.git/refs/tags &&\n \tcd repo &&\n \tgit commit --allow-empty -m initial &&\n \tgit checkout -b branch-1 &&\n-\tgit tag tag-1 &&\n-\tgit commit --allow-empty -m second &&\n-\tgit checkout -b branch-2 &&\n-\tgit tag tag-2 &&\n \n \tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n \tgit -c fsck.badRefName=warn refs verify 2>err &&\n@@ -84,7 +102,7 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \trm $branch_dir_prefix/.branch-1 &&\n \ttest_cmp expect err &&\n \n-\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&\n+\tcp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&\n \tgit -c fsck.badRefName=ignore refs verify 2>err &&\n \ttest_must_be_empty err\n '\n-- \n2.47.0\n\n"},{"id":"507720","messageId":"Zz3NPE52NcfKYE0Q@ArchLinux","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"[PATCH v9 3/9] ref: initialize ref name outside of check functions","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:51:24Z","receivedAt":"2024-11-20T11:51:16Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We passes \"refs_check_dir\" to the \"files_fsck_refs_name\" function which\nallows it to create the checked ref name later. However, when we\nintroduce a new check function, we have to allocate redundant memory and\nre-calculate the ref name. It's bad for us to allocate redundant memory\nand duplicate logic. Instead, we should allocate and calculate it only\nonce and pass the ref name to the check functions.\n\nIn order not to do repeat calculation, rename \"refs_check_dir\" to\n\"refname\". And in \"files_fsck_refs_dir\", create a new strbuf \"refname\",\nthus whenever we handle a new ref, calculate the name and call the check\nfunctions one by one.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n refs/files-backend.c | 21 +++++++++++++--------\n 1 file changed, 13 insertions(+), 8 deletions(-)\n\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex b055edc061..8edb700568 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3501,12 +3501,12 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store,\n */\n typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n-\t\t\t\t const char *refs_check_dir,\n+\t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n-\t\t\t\tconst char *refs_check_dir,\n+\t\t\t\tconst char *refname,\n \t\t\t\tstruct dir_iterator *iter)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -3522,11 +3522,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t/*\n \t * This works right now because we never check the root refs.\n \t */\n-\tstrbuf_addf(&sb, \"%s/%s\", refs_check_dir, iter->relative_path);\n-\tif (check_refname_format(sb.buf, 0)) {\n+\tif (check_refname_format(refname, 0)) {\n \t\tstruct fsck_ref_report report = { 0 };\n \n-\t\treport.path = sb.buf;\n+\t\treport.path = refname;\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_NAME,\n \t\t\t\t \"invalid refname format\");\n@@ -3542,6 +3541,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\t const char *refs_check_dir,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n+\tstruct strbuf refname = STRBUF_INIT;\n \tstruct strbuf sb = STRBUF_INIT;\n \tstruct dir_iterator *iter;\n \tint iter_status;\n@@ -3560,11 +3560,15 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\tcontinue;\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n+\t\t\tstrbuf_reset(&refname);\n+\t\t\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir,\n+\t\t\t\t iter->relative_path);\n+\n \t\t\tif (o->verbose)\n-\t\t\t\tfprintf_ln(stderr, \"Checking %s/%s\",\n-\t\t\t\t\t refs_check_dir, iter->relative_path);\n+\t\t\t\tfprintf_ln(stderr, \"Checking %s\", refname.buf);\n+\n \t\t\tfor (size_t i = 0; fsck_refs_fn[i]; i++) {\n-\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refs_check_dir, iter))\n+\t\t\t\tif (fsck_refs_fn[i](ref_store, o, refname.buf, iter))\n \t\t\t\t\tret = -1;\n \t\t\t}\n \t\t} else {\n@@ -3581,6 +3585,7 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \n out:\n \tstrbuf_release(&sb);\n+\tstrbuf_release(&refname);\n \treturn ret;\n }\n \n-- \n2.47.0\n\n"},{"id":"507721","messageId":"Zz3NRHF472oFdiH8@ArchLinux","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"[PATCH v9 4/9] ref: support multiple worktrees check for refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:51:32Z","receivedAt":"2024-11-20T11:51:24Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already set up the infrastructure to check the consistency for\nrefs, but we do not support multiple worktrees. However, \"git-fsck(1)\"\nwill check the refs of worktrees. As we decide to get feature parity\nwith \"git-fsck(1)\", we need to set up support for multiple worktrees.\n\nBecause each worktree has its own specific refs, instead of just showing\nthe users \"refs/worktree/foo\", we need to display the full name such as\n\"worktrees//refs/worktree/foo\". So we should know the id of the\nworktree to get the full name. Add a new parameter \"struct worktree *\"\nfor \"refs-internal.h::fsck_fn\". Then change the related functions to\nfollow this new interface.\n\nThe \"packed-refs\" only exists in the main worktree, so we should only\ncheck \"packed-refs\" in the main worktree. Use \"is_main_worktree\" method\nto skip checking \"packed-refs\" in \"packed_fsck\" function.\n\nThen, enhance the \"files-backend.c::files_fsck_refs_dir\" function to add\n\"worktree//\" prefix when we are not in the main worktree.\n\nLast, add a new test to check the refname when there are multiple\nworktrees to exercise the code.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n builtin/refs.c | 10 ++++++--\n refs.c | 5 ++--\n refs.h | 3 ++-\n refs/debug.c | 5 ++--\n refs/files-backend.c | 17 ++++++++++----\n refs/packed-backend.c | 8 ++++++-\n refs/refs-internal.h | 3 ++-\n refs/reftable-backend.c | 3 ++-\n t/t0602-reffiles-fsck.sh | 51 ++++++++++++++++++++++++++++++++++++++++\n 9 files changed, 90 insertions(+), 15 deletions(-)\n\ndiff --git a/builtin/refs.c b/builtin/refs.c\nindex 24978a7b7b..394b4101c6 100644\n--- a/builtin/refs.c\n+++ b/builtin/refs.c\n@@ -5,6 +5,7 @@\n #include \"parse-options.h\"\n #include \"refs.h\"\n #include \"strbuf.h\"\n+#include \"worktree.h\"\n \n #define REFS_MIGRATE_USAGE \\\n \tN_(\"git refs migrate --ref-format= [--dry-run]\")\n@@ -66,6 +67,7 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix)\n static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n {\n \tstruct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT;\n+\tstruct worktree **worktrees;\n \tconst char * const verify_usage[] = {\n \t\tREFS_VERIFY_USAGE,\n \t\tNULL,\n@@ -75,7 +77,7 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \t\tOPT_BOOL(0, \"strict\", &fsck_refs_options.strict, N_(\"enable strict checking\")),\n \t\tOPT_END(),\n \t};\n-\tint ret;\n+\tint ret = 0;\n \n \targc = parse_options(argc, argv, prefix, options, verify_usage, 0);\n \tif (argc)\n@@ -84,9 +86,13 @@ static int cmd_refs_verify(int argc, const char **argv, const char *prefix)\n \tgit_config(git_fsck_config, &fsck_refs_options);\n \tprepare_repo_settings(the_repository);\n \n-\tret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options);\n+\tworktrees = get_worktrees();\n+\tfor (size_t i = 0; worktrees[i]; i++)\n+\t\tret |= refs_fsck(get_worktree_ref_store(worktrees[i]),\n+\t\t\t\t &fsck_refs_options, worktrees[i]);\n \n \tfsck_options_clear(&fsck_refs_options);\n+\tfree_worktrees(worktrees);\n \treturn ret;\n }\n \ndiff --git a/refs.c b/refs.c\nindex 5f729ed412..395a17273c 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -318,9 +318,10 @@ int check_refname_format(const char *refname, int flags)\n \treturn check_or_sanitize_refname(refname, flags, NULL);\n }\n \n-int refs_fsck(struct ref_store *refs, struct fsck_options *o)\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt)\n {\n-\treturn refs->be->fsck(refs, o);\n+\treturn refs->be->fsck(refs, o, wt);\n }\n \n void sanitize_refname_component(const char *refname, struct strbuf *out)\ndiff --git a/refs.h b/refs.h\nindex 108dfc93b3..341d43239c 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -549,7 +549,8 @@ int check_refname_format(const char *refname, int flags);\n * reflogs are consistent, and non-zero otherwise. The errors will be\n * written to stderr.\n */\n-int refs_fsck(struct ref_store *refs, struct fsck_options *o);\n+int refs_fsck(struct ref_store *refs, struct fsck_options *o,\n+\t struct worktree *wt);\n \n /*\n * Apply the rules from check_refname_format, but mutate the result until it\ndiff --git a/refs/debug.c b/refs/debug.c\nindex 45e2e784a0..72e80ddd6d 100644\n--- a/refs/debug.c\n+++ b/refs/debug.c\n@@ -420,10 +420,11 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname,\n }\n \n static int debug_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;\n-\tint res = drefs->refs->be->fsck(drefs->refs, o);\n+\tint res = drefs->refs->be->fsck(drefs->refs, o, wt);\n \ttrace_printf_key(&trace_refs, \"fsck: %d\\n\", res);\n \treturn res;\n }\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8edb700568..8bfdce64bc 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -23,6 +23,7 @@\n #include \"../dir.h\"\n #include \"../chdir-notify.h\"\n #include \"../setup.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../revision.h\"\n@@ -3539,6 +3540,7 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t\t struct fsck_options *o,\n \t\t\t const char *refs_check_dir,\n+\t\t\t struct worktree *wt,\n \t\t\t files_fsck_refs_fn *fsck_refs_fn)\n {\n \tstruct strbuf refname = STRBUF_INIT;\n@@ -3561,6 +3563,9 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n \t\t} else if (S_ISREG(iter->st.st_mode) ||\n \t\t\t S_ISLNK(iter->st.st_mode)) {\n \t\t\tstrbuf_reset(&refname);\n+\n+\t\t\tif (!is_main_worktree(wt))\n+\t\t\t\tstrbuf_addf(&refname, \"worktrees/%s/\", wt->id);\n \t\t\tstrbuf_addf(&refname, \"%s/%s\", refs_check_dir,\n \t\t\t\t iter->relative_path);\n \n@@ -3590,7 +3595,8 @@ static int files_fsck_refs_dir(struct ref_store *ref_store,\n }\n \n static int files_fsck_refs(struct ref_store *ref_store,\n-\t\t\t struct fsck_options *o)\n+\t\t\t struct fsck_options *o,\n+\t\t\t struct worktree *wt)\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n@@ -3599,17 +3605,18 @@ static int files_fsck_refs(struct ref_store *ref_store,\n \n \tif (o->verbose)\n \t\tfprintf_ln(stderr, _(\"Checking references consistency\"));\n-\treturn files_fsck_refs_dir(ref_store, o, \"refs\", fsck_refs_fn);\n+\treturn files_fsck_refs_dir(ref_store, o, \"refs\", wt, fsck_refs_fn);\n }\n \n static int files_fsck(struct ref_store *ref_store,\n-\t\t struct fsck_options *o)\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt)\n {\n \tstruct files_ref_store *refs =\n \t\tfiles_downcast(ref_store, REF_STORE_READ, \"fsck\");\n \n-\treturn files_fsck_refs(ref_store, o) |\n-\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o);\n+\treturn files_fsck_refs(ref_store, o, wt) |\n+\t refs->packed_ref_store->be->fsck(refs->packed_ref_store, o, wt);\n }\n \n struct ref_storage_be refs_be_files = {\ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 07c57fd541..46dcaec654 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -13,6 +13,7 @@\n #include \"../lockfile.h\"\n #include \"../chdir-notify.h\"\n #include \"../statinfo.h\"\n+#include \"../worktree.h\"\n #include \"../wrapper.h\"\n #include \"../write-or-die.h\"\n #include \"../trace2.h\"\n@@ -1754,8 +1755,13 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s\n }\n \n static int packed_fsck(struct ref_store *ref_store UNUSED,\n-\t\t struct fsck_options *o UNUSED)\n+\t\t struct fsck_options *o UNUSED,\n+\t\t struct worktree *wt)\n {\n+\n+\tif (!is_main_worktree(wt))\n+\t\treturn 0;\n+\n \treturn 0;\n }\n \ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 2313c830d8..037d7991cd 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -653,7 +653,8 @@ typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refnam\n \t\t\t\t struct strbuf *referent);\n \n typedef int fsck_fn(struct ref_store *ref_store,\n-\t\t struct fsck_options *o);\n+\t\t struct fsck_options *o,\n+\t\t struct worktree *wt);\n \n struct ref_storage_be {\n \tconst char *name;\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex f5f957e6de..b6a63c1015 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -2443,7 +2443,8 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n }\n \n static int reftable_be_fsck(struct ref_store *ref_store UNUSED,\n-\t\t\t struct fsck_options *o UNUSED)\n+\t\t\t struct fsck_options *o UNUSED,\n+\t\t\t struct worktree *wt UNUSED)\n {\n \treturn 0;\n }\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 2a172c913d..1e17393a3d 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -107,4 +107,55 @@ test_expect_success 'ref name check should be adapted into fsck messages' '\n \ttest_must_be_empty err\n '\n \n+test_expect_success 'ref name check should work for multiple worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\n+\tcd repo &&\n+\ttest_commit initial &&\n+\tgit checkout -b branch-1 &&\n+\ttest_commit second &&\n+\tgit checkout -b branch-2 &&\n+\ttest_commit third &&\n+\tgit checkout -b branch-3 &&\n+\tgit worktree add ./worktree-1 branch-1 &&\n+\tgit worktree add ./worktree-2 branch-2 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-3\n+\t) &&\n+\n+\tcp $worktree1_refdir_prefix/branch-4 $worktree1_refdir_prefix/'\\'' branch-5'\\'' &&\n+\tcp $worktree2_refdir_prefix/branch-4 $worktree2_refdir_prefix/'\\''~branch-6'\\'' &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err &&\n+\n+\tfor worktree in \"worktree-1\" \"worktree-2\"\n+\tdo\n+\t\t(\n+\t\t\tcd $worktree &&\n+\t\t\ttest_must_fail git refs verify 2>err &&\n+\t\t\tcat >expect <<-EOF &&\n+\t\t\terror: worktrees/worktree-1/refs/worktree/ branch-5: badRefName: invalid refname format\n+\t\t\terror: worktrees/worktree-2/refs/worktree/~branch-6: badRefName: invalid refname format\n+\t\t\tEOF\n+\t\t\tsort err >sorted_err &&\n+\t\t\ttest_cmp expect sorted_err || return 1\n+\t\t)\n+\tdone\n+'\n+\n test_done\n-- \n2.47.0\n\n"},{"id":"507722","messageId":"Zz3NTtPCe3gdzX8-@ArchLinux","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"[PATCH v9 5/9] ref: port git-fsck(1) regular refs check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:51:42Z","receivedAt":"2024-11-20T11:51:34Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"\"git-fsck(1)\" implicitly checks the ref content by passing the\ncallback \"fsck_handle_ref\" to the \"refs.c::refs_for_each_rawref\".\nThen, it will check whether the ref content (eventually \"oid\")\nis valid. If not, it will report the following error to the user.\n\n error: refs/heads/main: invalid sha1 pointer 0000...\n\nAnd it will also report above errors when there are dangling symrefs\nin the repository wrongly. This does not align with the behavior of\nthe \"git symbolic-ref\" command which allows users to create dangling\nsymrefs.\n\nAs we have already introduced the \"git refs verify\" command, we'd better\ncheck the ref content explicitly in the \"git refs verify\" command thus\nlater we could remove these checks in \"git-fsck(1)\" and launch a\nsubprocess to call \"git refs verify\" in \"git-fsck(1)\" to make the\n\"git-fsck(1)\" more clean.\n\nFollowing what \"git-fsck(1)\" does, add a similar check to \"git refs\nverify\". Then add a new fsck error message \"badRefContent(ERROR)\" to\nrepresent that a ref has an invalid content.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/files-backend.c | 47 +++++++++++++++\n t/t0602-reffiles-fsck.sh | 105 ++++++++++++++++++++++++++++++++++\n 4 files changed, 156 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 68a2801f15..22c385ea22 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -19,6 +19,9 @@\n `badParentSha1`::\n \t(ERROR) A commit object has a bad parent sha1.\n \n+`badRefContent`::\n+\t(ERROR) A ref has bad content.\n+\n `badRefFiletype`::\n \t(ERROR) A ref has a bad file type.\n \ndiff --git a/fsck.h b/fsck.h\nindex 500b4c04d2..0d99a87911 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -31,6 +31,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8bfdce64bc..9f300a7d3c 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3505,6 +3505,52 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_refs_content(struct ref_store *ref_store,\n+\t\t\t\t struct fsck_options *o,\n+\t\t\t\t const char *target_name,\n+\t\t\t\t struct dir_iterator *iter)\n+{\n+\tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf referent = STRBUF_INIT;\n+\tstruct fsck_ref_report report = { 0 };\n+\tunsigned int type = 0;\n+\tint failure_errno = 0;\n+\tstruct object_id oid;\n+\tint ret = 0;\n+\n+\treport.path = target_name;\n+\n+\tif (S_ISLNK(iter->st.st_mode))\n+\t\tgoto cleanup;\n+\n+\tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n+\t\t/*\n+\t\t * Ref file could be removed by another concurrent process. We should\n+\t\t * ignore this error and continue to the next ref.\n+\t\t */\n+\t\tif (errno == ENOENT)\n+\t\t\tgoto cleanup;\n+\n+\t\tret = error_errno(_(\"cannot read ref file '%s'\"), iter->path.buf);\n+\t\tgoto cleanup;\n+\t}\n+\n+\tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n+\t\t\t\t ref_content.buf, &oid, &referent,\n+\t\t\t\t &type, &failure_errno)) {\n+\t\tstrbuf_rtrim(&ref_content);\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n+\t\t\t\t \"%s\", ref_content.buf);\n+\t\tgoto cleanup;\n+\t}\n+\n+cleanup:\n+\tstrbuf_release(&ref_content);\n+\tstrbuf_release(&referent);\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,\n \t\t\t\tstruct fsck_options *o,\n \t\t\t\tconst char *refname,\n@@ -3600,6 +3646,7 @@ static int files_fsck_refs(struct ref_store *ref_store,\n {\n \tfiles_fsck_refs_fn fsck_refs_fn[]= {\n \t\tfiles_fsck_refs_name,\n+\t\tfiles_fsck_refs_content,\n \t\tNULL,\n \t};\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 1e17393a3d..162370077b 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -158,4 +158,109 @@ test_expect_success 'ref name check should work for multiple worktrees' '\n \tdone\n '\n \n+test_expect_success 'regular ref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tgit refs verify 2>err &&\n+\ttest_must_be_empty err &&\n+\n+\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/branch-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-bad &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor bad_content in \"$(git rev-parse main)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$branch_dir_prefix/a/b/branch-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $branch_dir_prefix/a/b/branch-bad &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n+test_expect_success 'regular ref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tbad_content_1=$(git rev-parse main)x &&\n+\tbad_content_2=xfsazqfxcadas &&\n+\tbad_content_3=Xfsazqfxcadas &&\n+\tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n+\tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n+\tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n+\terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n+\terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n+test_expect_success 'ref content checks should work with worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit branch branch-3 &&\n+\tgit worktree add ./worktree-1 branch-2 &&\n+\tgit worktree add ./worktree-2 branch-3 &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\n+\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$worktree1_refdir_prefix/bad-branch-1 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-1/refs/worktree/bad-branch-1: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $worktree1_refdir_prefix/bad-branch-1 &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tfor bad_content in \"$(git rev-parse HEAD)x\" \"xfsazqfxcadas\" \"Xfsazqfxcadas\"\n+\tdo\n+\t\tprintf \"%s\" $bad_content >$worktree2_refdir_prefix/bad-branch-2 &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: worktrees/worktree-2/refs/worktree/bad-branch-2: badRefContent: $bad_content\n+\t\tEOF\n+\t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_done\n-- \n2.47.0\n\n"},{"id":"507723","messageId":"Zz3NVRoP4JTQxTve@ArchLinux","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"[PATCH v9 6/9] ref: add more strict checks for regular refs","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:51:49Z","receivedAt":"2024-11-20T11:51:42Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have already used \"parse_loose_ref_contents\" function to check\nwhether the ref content is valid in files backend. However, by\nusing \"parse_loose_ref_contents\", we allow the ref's content to end with\ngarbage or without a newline.\n\nEven though we never create such loose refs ourselves, we have accepted\nsuch loose refs. So, it is entirely possible that some third-party tools\nmay rely on such loose refs being valid. We should not report an error\nfsck message at current. We should notify the users about such\n\"curiously formatted\" loose refs so that adequate care is taken before\nwe decide to tighten the rules in the future.\n\nAnd it's not suitable either to report a warn fsck message to the user.\nWe don't yet want the \"--strict\" flag that controls this bit to end up\ngenerating errors for such weirdly-formatted reference contents, as we\nfirst want to assess whether this retroactive tightening will cause\nissues for any tools out there. It may cause compatibility issues which\nmay break the repository. So, we add the following two fsck infos to\nrepresent the situation where the ref content ends without newline or\nhas trailing garbages:\n\n1. refMissingNewline(INFO): A loose ref that does not end with\n newline(LF).\n2. trailingRefContent(INFO): A loose ref has trailing content.\n\nIt might appear that we can't provide the user with any warnings by\nusing FSCK_INFO. However, in \"fsck.c::fsck_vreport\", we will convert\nFSCK_INFO to FSCK_WARN and we can still warn the user about these\nsituations when using \"git refs verify\" without introducing\ncompatibility issues.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 14 +++++++++\n fsck.h | 2 ++\n refs.c | 2 +-\n refs/files-backend.c | 26 ++++++++++++++--\n refs/refs-internal.h | 2 +-\n t/t0602-reffiles-fsck.sh | 57 +++++++++++++++++++++++++++++++++--\n 6 files changed, 96 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 22c385ea22..6db0eaa84a 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -173,6 +173,20 @@\n `nullSha1`::\n \t(WARN) Tree contains entries pointing to a null sha1.\n \n+`refMissingNewline`::\n+\t(INFO) A loose ref that does not end with newline(LF). As\n+\tvalid implementations of Git never created such a loose ref\n+\tfile, it may become an error in the future. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as\n+\twe need to know what tools created such a file.\n+\n+`trailingRefContent`::\n+\t(INFO) A loose ref has trailing content. As valid implementations\n+\tof Git never created such a loose ref file, it may become an\n+\terror in the future. Report to the git@vger.kernel.org mailing\n+\tlist if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n `treeNotSorted`::\n \t(ERROR) A tree is not properly sorted.\n \ndiff --git a/fsck.h b/fsck.h\nindex 0d99a87911..b85072df57 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -85,6 +85,8 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n \ndiff --git a/refs.c b/refs.c\nindex 395a17273c..f88b32a633 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1789,7 +1789,7 @@ static int refs_read_special_head(struct ref_store *ref_store,\n \t}\n \n \tresult = parse_loose_ref_contents(ref_store->repo->hash_algo, content.buf,\n-\t\t\t\t\t oid, referent, type, failure_errno);\n+\t\t\t\t\t oid, referent, type, NULL, failure_errno);\n \n done:\n \tstrbuf_release(&full_path);\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 9f300a7d3c..3d4d612420 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -569,7 +569,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname,\n \tbuf = sb_contents.buf;\n \n \tret = parse_loose_ref_contents(ref_store->repo->hash_algo, buf,\n-\t\t\t\t oid, referent, type, &myerr);\n+\t\t\t\t oid, referent, type, NULL, &myerr);\n \n out:\n \tif (ret && !myerr)\n@@ -606,7 +606,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno)\n+\t\t\t const char **trailing, int *failure_errno)\n {\n \tconst char *p;\n \tif (skip_prefix(buf, \"ref:\", &buf)) {\n@@ -628,6 +628,10 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t*failure_errno = EINVAL;\n \t\treturn -1;\n \t}\n+\n+\tif (trailing)\n+\t\t*trailing = p;\n+\n \treturn 0;\n }\n \n@@ -3513,6 +3517,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \tstruct strbuf ref_content = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n+\tconst char *trailing = NULL;\n \tunsigned int type = 0;\n \tint failure_errno = 0;\n \tstruct object_id oid;\n@@ -3537,7 +3542,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \tif (parse_loose_ref_contents(ref_store->repo->hash_algo,\n \t\t\t\t ref_content.buf, &oid, &referent,\n-\t\t\t\t &type, &failure_errno)) {\n+\t\t\t\t &type, &trailing, &failure_errno)) {\n \t\tstrbuf_rtrim(&ref_content);\n \t\tret = fsck_report_ref(o, &report,\n \t\t\t\t FSCK_MSG_BAD_REF_CONTENT,\n@@ -3545,6 +3550,21 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\tgoto cleanup;\n \t}\n \n+\tif (!(type & REF_ISSYMREF)) {\n+\t\tif (!*trailing) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t\t \"misses LF at the end\");\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t\tif (*trailing != '\\n' || *(trailing + 1)) {\n+\t\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n cleanup:\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\ndiff --git a/refs/refs-internal.h b/refs/refs-internal.h\nindex 037d7991cd..125f1fe735 100644\n--- a/refs/refs-internal.h\n+++ b/refs/refs-internal.h\n@@ -716,7 +716,7 @@ struct ref_store {\n int parse_loose_ref_contents(const struct git_hash_algo *algop,\n \t\t\t const char *buf, struct object_id *oid,\n \t\t\t struct strbuf *referent, unsigned int *type,\n-\t\t\t int *failure_errno);\n+\t\t\t const char **trailing, int *failure_errno);\n \n /*\n * Fill in the generic part of refs and add it to our collection of\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 162370077b..33e7a390ad 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -189,7 +189,48 @@ test_expect_success 'regular ref content should be checked (individual)' '\n \t\tEOF\n \t\trm $branch_dir_prefix/a/b/branch-bad &&\n \t\ttest_cmp expect err || return 1\n-\tdone\n+\tdone &&\n+\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tfor trailing_content in \" garbage\" \" more garbage\"\n+\tdo\n+\t\tprintf \"%s\" \"$(git rev-parse main)$trailing_content\" >$branch_dir_prefix/branch-garbage &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\''$trailing_content'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-garbage &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tprintf \"%s\\n\\n\\n\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\n+\n+\t'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage-special &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s\\n\\n\\n garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage-special &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-garbage-special: trailingRefContent: has trailing garbage: '\\''\n+\n+\n+\t garbage'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-garbage-special &&\n+\ttest_cmp expect err\n '\n \n test_expect_success 'regular ref content should be checked (aggregate)' '\n@@ -207,12 +248,16 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \tprintf \"%s\" $bad_content_1 >$tag_dir_prefix/tag-bad-1 &&\n \tprintf \"%s\" $bad_content_2 >$tag_dir_prefix/tag-bad-2 &&\n \tprintf \"%s\" $bad_content_3 >$branch_dir_prefix/a/b/branch-bad &&\n+\tprintf \"%s\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-no-newline &&\n+\tprintf \"%s garbage\" \"$(git rev-parse main)\" >$branch_dir_prefix/branch-garbage &&\n \n \ttest_must_fail git refs verify 2>err &&\n \tcat >expect <<-EOF &&\n \terror: refs/heads/a/b/branch-bad: badRefContent: $bad_content_3\n \terror: refs/tags/tag-bad-1: badRefContent: $bad_content_1\n \terror: refs/tags/tag-bad-2: badRefContent: $bad_content_2\n+\twarning: refs/heads/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \tsort err >sorted_err &&\n \ttest_cmp expect sorted_err\n@@ -260,7 +305,15 @@ test_expect_success 'ref content checks should work with worktrees' '\n \t\tEOF\n \t\trm $worktree2_refdir_prefix/bad-branch-2 &&\n \t\ttest_cmp expect err || return 1\n-\tdone\n+\tdone &&\n+\n+\tprintf \"%s\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-no-newline &&\n+\ttest_cmp expect err\n '\n \n test_done\n-- \n2.47.0\n\n"},{"id":"507724","messageId":"Zz3NYMFZ37uSqP0K@ArchLinux","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"[PATCH v9 7/9] ref: add basic symref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:52:00Z","receivedAt":"2024-11-20T11:51:53Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"We have code that checks regular ref contents, but we do not yet check\nthe contents of symbolic refs. By using \"parse_loose_ref_content\" for\nsymbolic refs, we will get the information of the \"referent\".\n\nWe do not need to check the \"referent\" by opening the file. This is\nbecause if \"referent\" exists in the file system, we will eventually\ncheck its correctness by inspecting every file in the \"refs\" directory.\nIf the \"referent\" does not exist in the filesystem, this is OK as it is\nseen as the dangling symref.\n\nSo we just need to check the \"referent\" string content. A regular ref\ncould be accepted as a textual symref if it begins with \"ref:\", followed\nby zero or more whitespaces, followed by the full refname, followed only\nby whitespace characters. However, we always write a single SP after\n\"ref:\" and a single LF after the refname. It may seem that we should\nreport a fsck error message when the \"referent\" does not apply above\nrules and we should not be so aggressive because third-party\nreimplementations of Git may have taken advantage of the looser syntax.\nPut it more specific, we accept the following contents:\n\n1. \"ref: refs/heads/master \"\n2. \"ref: refs/heads/master \\n \\n\"\n3. \"ref: refs/heads/master\\n\\n\"\n\nWhen introducing the regular ref content checks, we created two fsck\ninfos \"refMissingNewline\" and \"trailingRefContent\" which exactly\nrepresents above situations. So we will reuse these two fsck messages to\nwrite checks to info the user about these situations.\n\nBut we do not allow any other trailing garbage. The followings are bad\nsymref contents which will be reported as fsck error by \"git-fsck(1)\".\n\n1. \"ref: refs/heads/master garbage\\n\"\n2. \"ref: refs/heads/master \\n\\n\\n garbage \"\n\nAnd we introduce a new \"badReferentName(ERROR)\" fsck message to report\nabove errors by using \"is_root_ref\" and \"check_refname_format\" to check\nthe \"referent\". Since both \"is_root_ref\" and \"check_refname_format\"\ndon't work with whitespaces, we use the trimmed version of \"referent\"\nwith these functions.\n\nIn order to add checks, we will do the following things:\n\n1. Record the untrimmed length \"orig_len\" and untrimmed last byte\n \"orig_last_byte\".\n2. Use \"strbuf_rtrim\" to trim the whitespaces or newlines to make sure\n \"is_root_ref\" and \"check_refname_format\" won't be failed by them.\n3. Use \"orig_len\" and \"orig_last_byte\" to check whether the \"referent\"\n misses '\\n' at the end or it has trailing whitespaces or newlines.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 3 +\n fsck.h | 1 +\n refs/files-backend.c | 40 ++++++++++++\n t/t0602-reffiles-fsck.sh | 111 ++++++++++++++++++++++++++++++++++\n 4 files changed, 155 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex 6db0eaa84a..dcea05edfc 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -28,6 +28,9 @@\n `badRefName`::\n \t(ERROR) A ref has an invalid format.\n \n+`badReferentName`::\n+\t(ERROR) The referent name of a symref is invalid.\n+\n `badTagName`::\n \t(INFO) A tag has an invalid format.\n \ndiff --git a/fsck.h b/fsck.h\nindex b85072df57..5227dfdef2 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -34,6 +34,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_REF_CONTENT, ERROR) \\\n \tFUNC(BAD_REF_FILETYPE, ERROR) \\\n \tFUNC(BAD_REF_NAME, ERROR) \\\n+\tFUNC(BAD_REFERENT_NAME, ERROR) \\\n \tFUNC(BAD_TIMEZONE, ERROR) \\\n \tFUNC(BAD_TREE, ERROR) \\\n \tFUNC(BAD_TREE_SHA1, ERROR) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 3d4d612420..f4342e3f3e 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3509,6 +3509,43 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \t\t\t\t const char *refname,\n \t\t\t\t struct dir_iterator *iter);\n \n+static int files_fsck_symref_target(struct fsck_options *o,\n+\t\t\t\t struct fsck_ref_report *report,\n+\t\t\t\t struct strbuf *referent)\n+{\n+\tchar orig_last_byte;\n+\tsize_t orig_len;\n+\tint ret = 0;\n+\n+\torig_len = referent->len;\n+\torig_last_byte = referent->buf[orig_len - 1];\n+\tstrbuf_rtrim(referent);\n+\n+\tif (!is_root_ref(referent->buf) &&\n+\t check_refname_format(referent->buf, 0)) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n+\t\t\t\t \"points to invalid refname '%s'\", referent->buf);\n+\t\tgoto out;\n+\t}\n+\n+\tif (referent->len == orig_len ||\n+\t (referent->len < orig_len && orig_last_byte != '\\n')) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_REF_MISSING_NEWLINE,\n+\t\t\t\t \"misses LF at the end\");\n+\t}\n+\n+\tif (referent->len != orig_len && referent->len != orig_len - 1) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_TRAILING_REF_CONTENT,\n+\t\t\t\t \"has trailing whitespaces or newlines\");\n+\t}\n+\n+out:\n+\treturn ret;\n+}\n+\n static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct fsck_options *o,\n \t\t\t\t const char *target_name,\n@@ -3563,6 +3600,9 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t\t \"has trailing garbage: '%s'\", trailing);\n \t\t\tgoto cleanup;\n \t\t}\n+\t} else {\n+\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tgoto cleanup;\n \t}\n \n cleanup:\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 33e7a390ad..ee1e5f2864 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -263,6 +263,109 @@ test_expect_success 'regular ref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'textual symref content should be checked (individual)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tfor good_referent in \"refs/heads/branch\" \"HEAD\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $branch_dir_prefix/branch-good &&\n+\t\ttest_must_be_empty err || return 1\n+\tdone &&\n+\n+\tfor bad_referent in \"refs/heads/.branch\" \"refs/heads/~branch\" \"refs/heads/?branch\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $bad_referent >$branch_dir_prefix/branch-bad &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\terror: refs/heads/branch-bad: badReferentName: points to invalid refname '\\''$bad_referent'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-bad &&\n+\t\ttest_cmp expect err || return 1\n+\tdone &&\n+\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-no-newline: refMissingNewline: misses LF at the end\n+\tEOF\n+\trm $branch_dir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-1 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-2 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-trailing-3 &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\tEOF\n+\trm $branch_dir_prefix/a/b/branch-complicated &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success 'textual symref content should be checked (aggregate)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tprintf \"ref: refs/heads/branch\\n\" >$branch_dir_prefix/branch-good &&\n+\tprintf \"ref: HEAD\\n\" >$branch_dir_prefix/branch-head &&\n+\tprintf \"ref: refs/heads/branch\" >$branch_dir_prefix/branch-no-newline-1 &&\n+\tprintf \"ref: refs/heads/branch \" >$branch_dir_prefix/a/b/branch-trailing-1 &&\n+\tprintf \"ref: refs/heads/branch\\n\\n\" >$branch_dir_prefix/a/b/branch-trailing-2 &&\n+\tprintf \"ref: refs/heads/branch \\n\" >$branch_dir_prefix/a/b/branch-trailing-3 &&\n+\tprintf \"ref: refs/heads/branch \\n \" >$branch_dir_prefix/a/b/branch-complicated &&\n+\tprintf \"ref: refs/heads/.branch\\n\" >$branch_dir_prefix/branch-bad-1 &&\n+\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\terror: refs/heads/branch-bad-1: badReferentName: points to invalid refname '\\''refs/heads/.branch'\\''\n+\twarning: refs/heads/a/b/branch-complicated: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-complicated: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-1: refMissingNewline: misses LF at the end\n+\twarning: refs/heads/a/b/branch-trailing-1: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-2: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/a/b/branch-trailing-3: trailingRefContent: has trailing whitespaces or newlines\n+\twarning: refs/heads/branch-no-newline-1: refMissingNewline: misses LF at the end\n+\tEOF\n+\tsort err >sorted_err &&\n+\ttest_cmp expect sorted_err\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n@@ -313,6 +416,14 @@ test_expect_success 'ref content checks should work with worktrees' '\n \twarning: worktrees/worktree-1/refs/worktree/branch-no-newline: refMissingNewline: misses LF at the end\n \tEOF\n \trm $worktree1_refdir_prefix/branch-no-newline &&\n+\ttest_cmp expect err &&\n+\n+\tprintf \"%s garbage\" \"$(git rev-parse HEAD)\" >$worktree1_refdir_prefix/branch-garbage &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-garbage: trailingRefContent: has trailing garbage: '\\'' garbage'\\''\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-garbage &&\n \ttest_cmp expect err\n '\n \n-- \n2.47.0\n\n"},{"id":"507725","messageId":"Zz3NaXuIrVjDsJri@ArchLinux","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"[PATCH v9 8/9] ref: check whether the target of the symref is a ref","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:52:09Z","receivedAt":"2024-11-20T11:52:01Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Ideally, we want to the users use \"git symbolic-ref\" to create symrefs\ninstead of writing raw contents into the filesystem. However, \"git\nsymbolic-ref\" is strict with the refname but not strict with the\nreferent. For example, we can make the \"referent\" located at the\n\"$(gitdir)/logs/aaa\" and manually write the content into this where we\ncan still successfully parse this symref by using \"git rev-parse\".\n\n $ git init repo && cd repo && git commit --allow-empty -mx\n $ git symbolic-ref refs/heads/test logs/aaa\n $ echo $(git rev-parse HEAD) > .git/logs/aaa\n $ git rev-parse test\n\nWe may need to add some restrictions for \"referent\" parameter when using\n\"git symbolic-ref\" to create symrefs because ideally all the\nnonpseudo-refs should be located under the \"refs\" directory and we may\ntighten this in the future.\n\nIn order to tell the user we may tighten the above situation, create\na new fsck message \"symrefTargetIsNotARef\" to notify the user that this\nmay become an error in the future.\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 9 +++++++++\n fsck.h | 1 +\n refs/files-backend.c | 14 ++++++++++++--\n t/t0602-reffiles-fsck.sh | 29 +++++++++++++++++++++++++++++\n 4 files changed, 51 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex dcea05edfc..f82ebc58e8 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -183,6 +183,15 @@\n \tgit@vger.kernel.org mailing list if you see this error, as\n \twe need to know what tools created such a file.\n \n+`symrefTargetIsNotARef`::\n+\t(INFO) The target of a symbolic reference points neither to\n+\ta root reference nor to a reference starting with \"refs/\".\n+\tAlthough we allow create a symref pointing to the referent which\n+\tis outside the \"ref\" by using `git symbolic-ref`, we may tighten\n+\tthe rule in the future. Report to the git@vger.kernel.org\n+\tmailing list if you see this error, as we need to know what tools\n+\tcreated such a file.\n+\n `trailingRefContent`::\n \t(INFO) A loose ref has trailing content. As valid implementations\n \tof Git never created such a loose ref file, it may become an\ndiff --git a/fsck.h b/fsck.h\nindex 5227dfdef2..53a47612e6 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -87,6 +87,7 @@ enum fsck_msg_type {\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n+\tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\n \t/* ignored (elevated when requested) */ \\\n \tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex f4342e3f3e..c2b99fdf40 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3513,6 +3513,7 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n \t\t\t\t struct strbuf *referent)\n {\n+\tint is_referent_root;\n \tchar orig_last_byte;\n \tsize_t orig_len;\n \tint ret = 0;\n@@ -3521,8 +3522,17 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \torig_last_byte = referent->buf[orig_len - 1];\n \tstrbuf_rtrim(referent);\n \n-\tif (!is_root_ref(referent->buf) &&\n-\t check_refname_format(referent->buf, 0)) {\n+\tis_referent_root = is_root_ref(referent->buf);\n+\tif (!is_referent_root &&\n+\t !starts_with(referent->buf, \"refs/\") &&\n+\t !starts_with(referent->buf, \"worktrees/\")) {\n+\t\tret = fsck_report_ref(o, report,\n+\t\t\t\t FSCK_MSG_SYMREF_TARGET_IS_NOT_A_REF,\n+\t\t\t\t \"points to non-ref target '%s'\", referent->buf);\n+\n+\t}\n+\n+\tif (!is_referent_root && check_refname_format(referent->buf, 0)) {\n \t\tret = fsck_report_ref(o, report,\n \t\t\t\t FSCK_MSG_BAD_REFERENT_NAME,\n \t\t\t\t \"points to invalid refname '%s'\", referent->buf);\ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex ee1e5f2864..692b30727a 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -366,6 +366,35 @@ test_expect_success 'textual symref content should be checked (aggregate)' '\n \ttest_cmp expect sorted_err\n '\n \n+test_expect_success 'the target of the textual symref should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tfor good_referent in \"refs/heads/branch\" \"HEAD\" \"refs/tags/tag\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $good_referent >$branch_dir_prefix/branch-good &&\n+\t\tgit refs verify 2>err &&\n+\t\trm $branch_dir_prefix/branch-good &&\n+\t\ttest_must_be_empty err || return 1\n+\tdone &&\n+\n+\tfor nonref_referent in \"refs-back/heads/branch\" \"refs-back/tags/tag\" \"reflogs/refs/heads/branch\"\n+\tdo\n+\t\tprintf \"ref: %s\\n\" $nonref_referent >$branch_dir_prefix/branch-bad-1 &&\n+\t\tgit refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: refs/heads/branch-bad-1: symrefTargetIsNotARef: points to non-ref target '\\''$nonref_referent'\\''\n+\t\tEOF\n+\t\trm $branch_dir_prefix/branch-bad-1 &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-- \n2.47.0\n\n"},{"id":"507726","messageId":"Zz3Ncs8RwsqTvj2K@ArchLinux","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"[PATCH v9 9/9] ref: add symlink ref content check for files backend","fromName":"shejialuo","fromEmail":"shejialuo@gmail.com","sentAt":"2024-11-20T11:52:18Z","receivedAt":"2024-11-20T11:52:10Z","isPatch":true,"sender":{"key":"shejialuo@gmail.com","avatar":"https://avatars.githubusercontent.com/u/56911263?v=4"},"body":"Besides the textual symref, we also allow symbolic links as the symref.\nSo, we should also provide the consistency check as what we have done\nfor textual symref. And also we consider deprecating writing the\nsymbolic links. We first need to access whether symbolic links still\nbe used. So, add a new fsck message \"symlinkRef(INFO)\" to tell the\nuser be aware of this information.\n\nWe have already introduced \"files_fsck_symref_target\". We should reuse\nthis function to handle the symrefs which use legacy symbolic links. We\nshould not check the trailing garbage for symbolic refs. Add a new\nparameter \"symbolic_link\" to disable some checks which should only be\nexecuted for textual symrefs.\n\nAnd we need to also generate the \"referent\" parameter for reusing\n\"files_fsck_symref_target\" by the following steps:\n\n1. Use \"strbuf_add_real_path\" to resolve the symlink and get the\n absolute path \"ref_content\" which the symlink ref points to.\n2. Generate the absolute path \"abs_gitdir\" of \"gitdir\" and combine\n \"ref_content\" and \"abs_gitdir\" to extract the relative path\n \"relative_referent_path\".\n3. If \"ref_content\" is outside of \"gitdir\", we just set \"referent\" with\n \"ref_content\". Instead, we set \"referent\" with\n \"relative_referent_path\".\n\nMentored-by: Patrick Steinhardt \nMentored-by: Karthik Nayak \nSigned-off-by: shejialuo \n---\n Documentation/fsck-msgids.txt | 6 ++\n fsck.h | 1 +\n refs/files-backend.c | 38 ++++++++-\n t/t0602-reffiles-fsck.sh | 141 ++++++++++++++++++++++++++++++++++\n 4 files changed, 182 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt\nindex f82ebc58e8..b14bc44ca4 100644\n--- a/Documentation/fsck-msgids.txt\n+++ b/Documentation/fsck-msgids.txt\n@@ -183,6 +183,12 @@\n \tgit@vger.kernel.org mailing list if you see this error, as\n \twe need to know what tools created such a file.\n \n+`symlinkRef`::\n+\t(INFO) A symbolic link is used as a symref. Report to the\n+\tgit@vger.kernel.org mailing list if you see this error, as we\n+\tare assessing the feasibility of dropping the support to drop\n+\tcreating symbolic links as symrefs.\n+\n `symrefTargetIsNotARef`::\n \t(INFO) The target of a symbolic reference points neither to\n \ta root reference nor to a reference starting with \"refs/\".\ndiff --git a/fsck.h b/fsck.h\nindex 53a47612e6..a44c231a5f 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -86,6 +86,7 @@ enum fsck_msg_type {\n \tFUNC(MAILMAP_SYMLINK, INFO) \\\n \tFUNC(BAD_TAG_NAME, INFO) \\\n \tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\tFUNC(SYMLINK_REF, INFO) \\\n \tFUNC(REF_MISSING_NEWLINE, INFO) \\\n \tFUNC(SYMREF_TARGET_IS_NOT_A_REF, INFO) \\\n \tFUNC(TRAILING_REF_CONTENT, INFO) \\\ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex c2b99fdf40..ea5961e48c 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -1,6 +1,7 @@\n #define USE_THE_REPOSITORY_VARIABLE\n \n #include \"../git-compat-util.h\"\n+#include \"../abspath.h\"\n #include \"../config.h\"\n #include \"../copy.h\"\n #include \"../environment.h\"\n@@ -3511,7 +3512,8 @@ typedef int (*files_fsck_refs_fn)(struct ref_store *ref_store,\n \n static int files_fsck_symref_target(struct fsck_options *o,\n \t\t\t\t struct fsck_ref_report *report,\n-\t\t\t\t struct strbuf *referent)\n+\t\t\t\t struct strbuf *referent,\n+\t\t\t\t unsigned int symbolic_link)\n {\n \tint is_referent_root;\n \tchar orig_last_byte;\n@@ -3520,7 +3522,8 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \n \torig_len = referent->len;\n \torig_last_byte = referent->buf[orig_len - 1];\n-\tstrbuf_rtrim(referent);\n+\tif (!symbolic_link)\n+\t\tstrbuf_rtrim(referent);\n \n \tis_referent_root = is_root_ref(referent->buf);\n \tif (!is_referent_root &&\n@@ -3539,6 +3542,9 @@ static int files_fsck_symref_target(struct fsck_options *o,\n \t\tgoto out;\n \t}\n \n+\tif (symbolic_link)\n+\t\tgoto out;\n+\n \tif (referent->len == orig_len ||\n \t (referent->len < orig_len && orig_last_byte != '\\n')) {\n \t\tret = fsck_report_ref(o, report,\n@@ -3562,6 +3568,7 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\t\t struct dir_iterator *iter)\n {\n \tstruct strbuf ref_content = STRBUF_INIT;\n+\tstruct strbuf abs_gitdir = STRBUF_INIT;\n \tstruct strbuf referent = STRBUF_INIT;\n \tstruct fsck_ref_report report = { 0 };\n \tconst char *trailing = NULL;\n@@ -3572,8 +3579,30 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \n \treport.path = target_name;\n \n-\tif (S_ISLNK(iter->st.st_mode))\n+\tif (S_ISLNK(iter->st.st_mode)) {\n+\t\tconst char *relative_referent_path = NULL;\n+\n+\t\tret = fsck_report_ref(o, &report,\n+\t\t\t\t FSCK_MSG_SYMLINK_REF,\n+\t\t\t\t \"use deprecated symbolic link for symref\");\n+\n+\t\tstrbuf_add_absolute_path(&abs_gitdir, ref_store->repo->gitdir);\n+\t\tstrbuf_normalize_path(&abs_gitdir);\n+\t\tif (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1]))\n+\t\t\tstrbuf_addch(&abs_gitdir, '/');\n+\n+\t\tstrbuf_add_real_path(&ref_content, iter->path.buf);\n+\t\tskip_prefix(ref_content.buf, abs_gitdir.buf,\n+\t\t\t &relative_referent_path);\n+\n+\t\tif (relative_referent_path)\n+\t\t\tstrbuf_addstr(&referent, relative_referent_path);\n+\t\telse\n+\t\t\tstrbuf_addbuf(&referent, &ref_content);\n+\n+\t\tret |= files_fsck_symref_target(o, &report, &referent, 1);\n \t\tgoto cleanup;\n+\t}\n \n \tif (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) {\n \t\t/*\n@@ -3611,13 +3640,14 @@ static int files_fsck_refs_content(struct ref_store *ref_store,\n \t\t\tgoto cleanup;\n \t\t}\n \t} else {\n-\t\tret = files_fsck_symref_target(o, &report, &referent);\n+\t\tret = files_fsck_symref_target(o, &report, &referent, 0);\n \t\tgoto cleanup;\n \t}\n \n cleanup:\n \tstrbuf_release(&ref_content);\n \tstrbuf_release(&referent);\n+\tstrbuf_release(&abs_gitdir);\n \treturn ret;\n }\n \ndiff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh\nindex 692b30727a..f8f27cfc6c 100755\n--- a/t/t0602-reffiles-fsck.sh\n+++ b/t/t0602-reffiles-fsck.sh\n@@ -395,6 +395,147 @@ test_expect_success 'the target of the textual symref should be checked' '\n \tdone\n '\n \n+test_expect_success SYMLINKS 'symlink symref content should be checked' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tbranch_dir_prefix=.git/refs/heads &&\n+\ttag_dir_prefix=.git/refs/tags &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tmkdir -p \"$branch_dir_prefix/a/b\" &&\n+\n+\tln -sf ./main $branch_dir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../logs/branch-escape $branch_dir_prefix/branch-symbolic &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\twarning: refs/heads/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\"branch \" $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-bad: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/heads/branch-symbolic-bad: badReferentName: points to invalid refname '\\''refs/heads/branch '\\''\n+\tEOF\n+\trm $branch_dir_prefix/branch-symbolic-bad &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ./\".tag\" $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_must_fail git refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/tags/tag-symbolic-1: symlinkRef: use deprecated symbolic link for symref\n+\terror: refs/tags/tag-symbolic-1: badReferentName: points to invalid refname '\\''refs/tags/.tag'\\''\n+\tEOF\n+\trm $tag_dir_prefix/tag-symbolic-1 &&\n+\ttest_cmp expect err\n+'\n+\n+test_expect_success SYMLINKS 'symlink symref content should be checked (worktree)' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\tcd repo &&\n+\ttest_commit default &&\n+\tgit branch branch-1 &&\n+\tgit branch branch-2 &&\n+\tgit branch branch-3 &&\n+\tgit worktree add ./worktree-1 branch-2 &&\n+\tgit worktree add ./worktree-2 branch-3 &&\n+\tmain_worktree_refdir_prefix=.git/refs/heads &&\n+\tworktree1_refdir_prefix=.git/worktrees/worktree-1/refs/worktree &&\n+\tworktree2_refdir_prefix=.git/worktrees/worktree-2/refs/worktree &&\n+\n+\t(\n+\t\tcd worktree-1 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\t(\n+\t\tcd worktree-2 &&\n+\t\tgit update-ref refs/worktree/branch-4 refs/heads/branch-1\n+\t) &&\n+\n+\tln -sf ../../../../refs/heads/good-branch $worktree1_refdir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../../../worktrees/worktree-1/good-branch $worktree2_refdir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-2/refs/worktree/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $worktree2_refdir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../worktrees/worktree-2/good-branch $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: refs/heads/branch-symbolic-good: symlinkRef: use deprecated symbolic link for symref\n+\tEOF\n+\trm $main_worktree_refdir_prefix/branch-symbolic-good &&\n+\ttest_cmp expect err &&\n+\n+\tln -sf ../../../../logs/branch-escape $worktree1_refdir_prefix/branch-symbolic &&\n+\tgit refs verify 2>err &&\n+\tcat >expect <<-EOF &&\n+\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\twarning: worktrees/worktree-1/refs/worktree/branch-symbolic: symrefTargetIsNotARef: points to non-ref target '\\''logs/branch-escape'\\''\n+\tEOF\n+\trm $worktree1_refdir_prefix/branch-symbolic &&\n+\ttest_cmp expect err &&\n+\n+\tfor bad_referent_name in \".tag\" \"branch \"\n+\tdo\n+\t\tln -sf ./\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-1/refs/worktree/$bad_referent_name'\\''\n+\t\tEOF\n+\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree1_refdir_prefix/bad-symbolic &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: worktrees/worktree-1/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: worktrees/worktree-1/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\tEOF\n+\t\trm $worktree1_refdir_prefix/bad-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ./\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''worktrees/worktree-2/refs/worktree/$bad_referent_name'\\''\n+\t\tEOF\n+\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\ttest_cmp expect err &&\n+\n+\t\tln -sf ../../../../refs/heads/\"$bad_referent_name\" $worktree2_refdir_prefix/bad-symbolic &&\n+\t\ttest_must_fail git refs verify 2>err &&\n+\t\tcat >expect <<-EOF &&\n+\t\twarning: worktrees/worktree-2/refs/worktree/bad-symbolic: symlinkRef: use deprecated symbolic link for symref\n+\t\terror: worktrees/worktree-2/refs/worktree/bad-symbolic: badReferentName: points to invalid refname '\\''refs/heads/$bad_referent_name'\\''\n+\t\tEOF\n+\t\trm $worktree2_refdir_prefix/bad-symbolic &&\n+\t\ttest_cmp expect err || return 1\n+\tdone\n+'\n+\n test_expect_success 'ref content checks should work with worktrees' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n-- \n2.47.0\n\n"},{"id":"507764","messageId":"Zz3xpMdzeeUssH6n@pks.im","threadId":"61943","inReplyTo":"Zz3MON9_9DGD6nsy@ArchLinux","subject":"Re: [PATCH v9 0/9] add ref content check for files backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-20T14:26:53Z","receivedAt":"2024-11-20T14:27:07Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Nov 20, 2024 at 07:47:04PM +0800, shejialuo wrote:\n> Hi All:\n> \n> This version fixes two problems:\n> \n> 1. Remove unnecessary space.\n> 2. Drop extra \"strerror(errno)\".\n> \n> Thanks,\n> Jialuo\n\nThe range-diff looks as expected, so this version lokos good to me.\n\nThanks!\n\nPatrick\n"},{"id":"507795","messageId":"xmqq8qtdijaa.fsf@gitster.g","threadId":"61943","inReplyTo":"Zz3xpMdzeeUssH6n@pks.im","subject":"Re: [PATCH v9 0/9] add ref content check for files backend","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-11-20T23:21:01Z","receivedAt":"2024-11-20T23:21:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Wed, Nov 20, 2024 at 07:47:04PM +0800, shejialuo wrote:\n>> Hi All:\n>> \n>> This version fixes two problems:\n>> \n>> 1. Remove unnecessary space.\n>> 2. Drop extra \"strerror(errno)\".\n>> \n>> Thanks,\n>> Jialuo\n>\n> The range-diff looks as expected, so this version lokos good to me.\n\nThanks, both. Looking good.\n"}]}