{"thread":{"id":"54994","subject":"RFC on packfile URIs and .gitmodules check","startedAt":"2021-01-15T23:43:52Z","lastAt":"2021-03-29T02:07:14Z","messageCount":229,"participants":["Jonathan Tan","Junio C Hamano","Taylor Blau","Derrick Stolee","Ævar Arnfjörð Bjarmason","Josh Steadmon","Jeff King","Jonathan Nieder","Johannes Schindelin","Ramsay Jones"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"414478","messageId":"20210115234300.350442-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":null,"subject":"RFC on packfile URIs and .gitmodules check","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-15T23:43:00Z","receivedAt":"2021-01-15T23:43:52Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Someone at $DAYJOB noticed that if a .gitmodules-containing tree and the\n.gitmodules blob itself are sent in 2 separate packfiles during a fetch\n(which can happen when packfile URIs are used), transfer.fsckobjects\ncauses the fetch to fail. You can reproduce it as follows (as of the\ntime of writing):\n\n $ git -c fetch.uriprotocols=https -c transfer.fsckobjects=true clone https://chromium.googlesource.com/chromiumos/codesearch\n Cloning into 'codesearch'...\n remote: Total 2242 (delta 0), reused 2242 (delta 0)\n Receiving objects: 100% (2242/2242), 1.77 MiB | 4.62 MiB/s, done.\n error: object 1f155c20935ee1154a813a814f03ef2b3976680f: gitmodulesMissing: unable to read .gitmodules blob\n fatal: fsck error in pack objects\n fatal: index-pack failed\n\nThis happens because the fsck part is currently being done in\nindex-pack, which operates on one pack at a time. When index-pack sees\nthe tree, it runs fsck on it (like any other object), and the fsck\nsubsystem remembers the .gitmodules target (specifically, in\ngitmodules_found in fsck.c). Later, index-pack runs fsck_finish() which\nchecks if the target exists, but it doesn't, so it reports the failure.\n\nOne option is for fetch to do its own pass of checking all downloaded\nobjects once all packfiles have been downloaded, but that seems wasteful\nas all trees would have to be re-inflated.\n\nAnother option is to do it within the connectivity check instead - so,\nupdate rev-list and the object walking mechanism to be able to detect\n.gitmodules in trees and fsck the target blob whenever such an entry\noccurs. This has the advantage that there is no extra re-inflation,\nalthough it might be strange to have object walking be able to fsck.\n\nThe simplest solution would be to just relax this - check the blob if it\nexists, but if it doesn't, it's OK. Some things in favor of this\nsolution:\n\n - This is something we already do in the partial clone case (although\n it could be argued that in this case, we're already trusting the\n server for far more than .gitmodules, so just because it's OK in the\n partial clone case doesn't mean that it's OK in the regular case).\n\n - Also, the commit message for this feature (from ed8b10f631 (\"fsck: check\n .gitmodules content\", 2018-05-21)) gives a rationale of a newer\n server being able to protect older clients.\n - Servers using receive-pack (instead of fetch-pack) to obtain\n objects would still be protected, since receive-pack still only\n accepts one packfile at a time (and there are currently no plans\n to expand this).\n - Also, malicious .gitobjects files could still be crafted that pass\n fsck checking - for example, by containing a URL (of another\n server) that refers to a repo with a .gitobjects that would fail\n fsck.\n\nSo I would rather go with just relaxing the check, but if consensus is\nthat we should still do it, I'll investigate doing it in the\nconnectivity check.\n"},{"id":"414479","messageId":"xmqqim7xraeo.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"20210115234300.350442-1-jonathantanmy@google.com","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-16T00:30:07Z","receivedAt":"2021-01-16T00:30:55Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> Someone at $DAYJOB noticed that if a .gitmodules-containing tree and the\n> .gitmodules blob itself are sent in 2 separate packfiles during a fetch\n> (which can happen when packfile URIs are used), transfer.fsckobjects\n> causes the fetch to fail. You can reproduce it as follows (as of the\n> time of writing):\n>\n> $ git -c fetch.uriprotocols=https -c transfer.fsckobjects=true clone https://chromium.googlesource.com/chromiumos/codesearch\n> Cloning into 'codesearch'...\n> remote: Total 2242 (delta 0), reused 2242 (delta 0)\n> Receiving objects: 100% (2242/2242), 1.77 MiB | 4.62 MiB/s, done.\n> error: object 1f155c20935ee1154a813a814f03ef2b3976680f: gitmodulesMissing: unable to read .gitmodules blob\n> fatal: fsck error in pack objects\n> fatal: index-pack failed\n>\n> This happens because the fsck part is currently being done in\n> index-pack, which operates on one pack at a time. When index-pack sees\n> the tree, it runs fsck on it (like any other object), and the fsck\n> subsystem remembers the .gitmodules target (specifically, in\n> gitmodules_found in fsck.c). Later, index-pack runs fsck_finish() which\n> checks if the target exists, but it doesn't, so it reports the failure.\n\nIs this because the gitmodules blob is contained in the base image\nserved via the pack URI mechansim, and the \"dynamic\" packfile for\nthe latest part of the history refers to the gitmodules file that is\nunchanged, hence the latter one lacks it?\n\n> Another option is to do it within the connectivity check instead - so,\n> update rev-list and the object walking mechanism to be able to detect\n> .gitmodules in trees and fsck the target blob whenever such an entry\n> occurs. This has the advantage that there is no extra re-inflation,\n> although it might be strange to have object walking be able to fsck.\n>\n> The simplest solution would be to just relax this - check the blob if it\n> exists, but if it doesn't, it's OK. Some things in favor of this\n> solution:\n>\n> - This is something we already do in the partial clone case (although\n> it could be argued that in this case, we're already trusting the\n> server for far more than .gitmodules, so just because it's OK in the\n> partial clone case doesn't mean that it's OK in the regular case).\n>\n> - Also, the commit message for this feature (from ed8b10f631 (\"fsck: check\n> .gitmodules content\", 2018-05-21)) gives a rationale of a newer\n> server being able to protect older clients.\n> - Servers using receive-pack (instead of fetch-pack) to obtain\n> objects would still be protected, since receive-pack still only\n> accepts one packfile at a time (and there are currently no plans\n> to expand this).\n> - Also, malicious .gitobjects files could still be crafted that pass\n> fsck checking - for example, by containing a URL (of another\n> server) that refers to a repo with a .gitobjects that would fail\n> fsck.\n>\n> So I would rather go with just relaxing the check, but if consensus is\n> that we should still do it, I'll investigate doing it in the\n> connectivity check.\n\nYou've listed two possible solutions, i.e.\n\n (1) punt and declare that we assume an missing and uncheckable blob\n is OK,\n\n (2) defer the check after transfer completes.\n\nBetween the two, my gut feeling is that the latter is preferrable.\nIf we assume an missing and uncheckable one is OK, then even if a\nblob is available to be checked, there is not much point in\nchecking, no?\n\nAs long as the quarantine of incoming pack works correctly,\nstreaming the incoming packdata (and packfile downloaded out of line\nvia a separate mechanism like pack URI) to index-pack that does not\ncheck to complete the transfer, with a separate step to check the\nsanity of these packs as a whole, should not harm the repository\neven if it is interrupted in the middle, after transfer is done but\nbefore checking says it is OK.\n\nAs a potential third option, I wonder if it is easier for everybody\ninvolved (including third-party implementation of their\nindex-pack/fsck equivalent) if we made it a rule that a pack that\nhas a tree that refers to .git must include the blob for\nit?\n\nThanks.\n"},{"id":"414483","messageId":"YAJcDr0EPkKk0aR/@nand.local","threadId":"54994","inReplyTo":"xmqqim7xraeo.fsf@gitster.c.googlers.com","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-01-16T03:22:54Z","receivedAt":"2021-01-16T03:25:20Z","isPatch":false,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Jan 15, 2021 at 04:30:07PM -0800, Junio C Hamano wrote:\n> Jonathan Tan writes:\n>\n> > Someone at $DAYJOB noticed that if a .gitmodules-containing tree and the\n> > .gitmodules blob itself are sent in 2 separate packfiles during a fetch\n> > (which can happen when packfile URIs are used), transfer.fsckobjects\n> > causes the fetch to fail. You can reproduce it as follows (as of the\n> > time of writing):\n> >\n> > $ git -c fetch.uriprotocols=https -c transfer.fsckobjects=true clone https://chromium.googlesource.com/chromiumos/codesearch\n> > Cloning into 'codesearch'...\n> > remote: Total 2242 (delta 0), reused 2242 (delta 0)\n> > Receiving objects: 100% (2242/2242), 1.77 MiB | 4.62 MiB/s, done.\n> > error: object 1f155c20935ee1154a813a814f03ef2b3976680f: gitmodulesMissing: unable to read .gitmodules blob\n> > fatal: fsck error in pack objects\n> > fatal: index-pack failed\n> >\n> > This happens because the fsck part is currently being done in\n> > index-pack, which operates on one pack at a time. When index-pack sees\n> > the tree, it runs fsck on it (like any other object), and the fsck\n> > subsystem remembers the .gitmodules target (specifically, in\n> > gitmodules_found in fsck.c). Later, index-pack runs fsck_finish() which\n> > checks if the target exists, but it doesn't, so it reports the failure.\n>\n> Is this because the gitmodules blob is contained in the base image\n> served via the pack URI mechansim, and the \"dynamic\" packfile for\n> the latest part of the history refers to the gitmodules file that is\n> unchanged, hence the latter one lacks it?\n\nThat seems like a likely explanation, although this seems ultimately up\nto what the pack CDN serves.\n> You've listed two possible solutions, i.e.\n>\n> (1) punt and declare that we assume an missing and uncheckable blob\n> is OK,\n>\n> (2) defer the check after transfer completes.\n>\n> Between the two, my gut feeling is that the latter is preferrable.\n> If we assume an missing and uncheckable one is OK, then even if a\n> blob is available to be checked, there is not much point in\n> checking, no?\n\nI'm going to second this. If this were a more benign check, then I'd\nperhaps feel differently, but .gitmodules fsck checks seem to get\nhardened fairly often during security releases, and so it seems\nimportant to keep performing them when the user asked for it.\n\n> As long as the quarantine of incoming pack works correctly,\n> streaming the incoming packdata (and packfile downloaded out of line\n> via a separate mechanism like pack URI) to index-pack that does not\n> check to complete the transfer, with a separate step to check the\n> sanity of these packs as a whole, should not harm the repository\n> even if it is interrupted in the middle, after transfer is done but\n> before checking says it is OK.\n\nAgreed. Bear in mind that I am pretty unfamiliar with this code, and so\nI'm not sure if it's 'easy' or not to change it in this way. The obvious\ndownside, which Jonathan notes, is that you almost certainly have to\nreinflate all of the trees again.\n\nBut, since the user is asking for transfer.fsckObjects explicitly, I\ndon't think that it's a problem.\n\n> As a potential third option, I wonder if it is easier for everybody\n> involved (including third-party implementation of their\n> index-pack/fsck equivalent) if we made it a rule that a pack that\n> has a tree that refers to .git must include the blob for\n> it?\n\nInteresting, but I'm sure CDN administrators would prefer to have as few\nrestrictions in place as possible.\n\nA potential fourth option that I can think of is that we can try to\neagerly perform the .gitmodules fsck checks as we receive objects, under\nthe assumption that the .gitmoudles blob and the tree which contains it\nappear in the same pack.\n\nIf they do, then we ought to be able to check them as we currently do\n(and avoid leaving them to the slow post-processing step). Any blobs\nthat we _can't_ find get placed into an array, and then that array is\niterated over after we have received all packs, including from the CDN.\nAny blobs that couldn't be found in the pack transferred from the\nremote, the CDN, or the local repository (and isn't explicitly excluded\nvia an object --filter) is declared missing.\n\nThoughts?\n\n> Thanks.\n\nThanks,\nTaylor\n"},{"id":"414682","messageId":"d2ca2fec-a353-787a-15a7-3831a665523e@gmail.com","threadId":"54994","inReplyTo":"YAJcDr0EPkKk0aR/@nand.local","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-01-19T12:56:07Z","receivedAt":"2021-01-19T14:52:31Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 1/15/2021 10:22 PM, Taylor Blau wrote:\n> On Fri, Jan 15, 2021 at 04:30:07PM -0800, Junio C Hamano wrote:\n>> Jonathan Tan writes:\n>>\n>>> Someone at $DAYJOB noticed that if a .gitmodules-containing tree and the\n>>> .gitmodules blob itself are sent in 2 separate packfiles during a fetch\n>>> (which can happen when packfile URIs are used), transfer.fsckobjects\n>>> causes the fetch to fail. You can reproduce it as follows (as of the\n>>> time of writing):\n>>>\n>>> $ git -c fetch.uriprotocols=https -c transfer.fsckobjects=true clone https://chromium.googlesource.com/chromiumos/codesearch\n>>> Cloning into 'codesearch'...\n>>> remote: Total 2242 (delta 0), reused 2242 (delta 0)\n>>> Receiving objects: 100% (2242/2242), 1.77 MiB | 4.62 MiB/s, done.\n>>> error: object 1f155c20935ee1154a813a814f03ef2b3976680f: gitmodulesMissing: unable to read .gitmodules blob\n>>> fatal: fsck error in pack objects\n>>> fatal: index-pack failed\n\nI'm contributing a quick suggestion for just this item:\n\n>>> This happens because the fsck part is currently being done in\n>>> index-pack, which operates on one pack at a time. When index-pack sees\n>>> the tree, it runs fsck on it (like any other object), and the fsck\n>>> subsystem remembers the .gitmodules target (specifically, in\n>>> gitmodules_found in fsck.c). Later, index-pack runs fsck_finish() which\n>>> checks if the target exists, but it doesn't, so it reports the failure.\n>>\n>> Is this because the gitmodules blob is contained in the base image\n>> served via the pack URI mechansim, and the \"dynamic\" packfile for\n>> the latest part of the history refers to the gitmodules file that is\n>> unchanged, hence the latter one lacks it?\n> \n> That seems like a likely explanation, although this seems ultimately up\n> to what the pack CDN serves.\n>> You've listed two possible solutions, i.e.\n>>\n>> (1) punt and declare that we assume an missing and uncheckable blob\n>> is OK,\n>>\n>> (2) defer the check after transfer completes.\n>>\n>> Between the two, my gut feeling is that the latter is preferrable.\n>> If we assume an missing and uncheckable one is OK, then even if a\n>> blob is available to be checked, there is not much point in\n>> checking, no?\n> \n> I'm going to second this. If this were a more benign check, then I'd\n> perhaps feel differently, but .gitmodules fsck checks seem to get\n> hardened fairly often during security releases, and so it seems\n> important to keep performing them when the user asked for it.\n\nIt might be nice to teach 'index-pack' a mode that says certain\nerrors should be reported as warnings by writing the problematic\nOIDs to stdout/stderr. Then, the second check after all packs are\npresent can focus on those problematic objects instead of\nre-scanning everything.\n\nThanks,\n-Stolee\n"},{"id":"414703","messageId":"20210119191306.1500480-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"d2ca2fec-a353-787a-15a7-3831a665523e@gmail.com","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-19T19:13:06Z","receivedAt":"2021-01-19T19:23:14Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> I'm contributing a quick suggestion for just this item:\n> \n> >>> This happens because the fsck part is currently being done in\n> >>> index-pack, which operates on one pack at a time. When index-pack sees\n> >>> the tree, it runs fsck on it (like any other object), and the fsck\n> >>> subsystem remembers the .gitmodules target (specifically, in\n> >>> gitmodules_found in fsck.c). Later, index-pack runs fsck_finish() which\n> >>> checks if the target exists, but it doesn't, so it reports the failure.\n> >>\n> >> Is this because the gitmodules blob is contained in the base image\n> >> served via the pack URI mechansim, and the \"dynamic\" packfile for\n> >> the latest part of the history refers to the gitmodules file that is\n> >> unchanged, hence the latter one lacks it?\n> > \n> > That seems like a likely explanation, although this seems ultimately up\n> > to what the pack CDN serves.\n> >> You've listed two possible solutions, i.e.\n> >>\n> >> (1) punt and declare that we assume an missing and uncheckable blob\n> >> is OK,\n> >>\n> >> (2) defer the check after transfer completes.\n> >>\n> >> Between the two, my gut feeling is that the latter is preferrable.\n> >> If we assume an missing and uncheckable one is OK, then even if a\n> >> blob is available to be checked, there is not much point in\n> >> checking, no?\n> > \n> > I'm going to second this. If this were a more benign check, then I'd\n> > perhaps feel differently, but .gitmodules fsck checks seem to get\n> > hardened fairly often during security releases, and so it seems\n> > important to keep performing them when the user asked for it.\n> \n> It might be nice to teach 'index-pack' a mode that says certain\n> errors should be reported as warnings by writing the problematic\n> OIDs to stdout/stderr. Then, the second check after all packs are\n> present can focus on those problematic objects instead of\n> re-scanning everything.\n\nMy initial reaction was that stdout is already used to report the hash\npart of the generated name and that stderr is already used for whatever\nwarnings there are, but looking at the documentation, index-pack\n--fsck-objects is \"[for] internal use only\", so it might be fine to\nextend the output format in this case and report the problematic OIDs\nafter the hash. I'll take a look.\n"},{"id":"414722","messageId":"20210119190209.1497399-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"YAJcDr0EPkKk0aR/@nand.local","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-19T19:02:09Z","receivedAt":"2021-01-19T20:16:50Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> > Is this because the gitmodules blob is contained in the base image\n> > served via the pack URI mechansim, and the \"dynamic\" packfile for\n> > the latest part of the history refers to the gitmodules file that is\n> > unchanged, hence the latter one lacks it?\n> \n> That seems like a likely explanation, although this seems ultimately up\n> to what the pack CDN serves.\n\nIn this case, yes, that is what is happening.\n\n> > You've listed two possible solutions, i.e.\n> >\n> > (1) punt and declare that we assume an missing and uncheckable blob\n> > is OK,\n> >\n> > (2) defer the check after transfer completes.\n> >\n> > Between the two, my gut feeling is that the latter is preferrable.\n> > If we assume an missing and uncheckable one is OK, then even if a\n> > blob is available to be checked, there is not much point in\n> > checking, no?\n> \n> I'm going to second this. If this were a more benign check, then I'd\n> perhaps feel differently, but .gitmodules fsck checks seem to get\n> hardened fairly often during security releases, and so it seems\n> important to keep performing them when the user asked for it.\n\nThat makes sense.\n\n> > As long as the quarantine of incoming pack works correctly,\n> > streaming the incoming packdata (and packfile downloaded out of line\n> > via a separate mechanism like pack URI) to index-pack that does not\n> > check to complete the transfer, with a separate step to check the\n> > sanity of these packs as a whole, should not harm the repository\n> > even if it is interrupted in the middle, after transfer is done but\n> > before checking says it is OK.\n> \n> Agreed. Bear in mind that I am pretty unfamiliar with this code, and so\n> I'm not sure if it's 'easy' or not to change it in this way. The obvious\n> downside, which Jonathan notes, is that you almost certainly have to\n> reinflate all of the trees again.\n> \n> But, since the user is asking for transfer.fsckObjects explicitly, I\n> don't think that it's a problem.\n\nWe might be able to avoid the reinflate if we do it as part of the\nconnectivity check or somehow teach index-pack a way to communicate the\ndangling .gitmodules links (as you suggest below).\n\n> > As a potential third option, I wonder if it is easier for everybody\n> > involved (including third-party implementation of their\n> > index-pack/fsck equivalent) if we made it a rule that a pack that\n> > has a tree that refers to .git must include the blob for\n> > it?\n> \n> Interesting, but I'm sure CDN administrators would prefer to have as few\n> restrictions in place as possible.\n\nThat rule would help, but it also seems inelegant in that if we put\ncommits that have the same .gitmodules in 2 or more different packs,\nthere would be identical objects across those packs (besides the reason\nTaylor mentioned).\n\n> A potential fourth option that I can think of is that we can try to\n> eagerly perform the .gitmodules fsck checks as we receive objects, under\n> the assumption that the .gitmoudles blob and the tree which contains it\n> appear in the same pack.\n> \n> If they do, then we ought to be able to check them as we currently do\n> (and avoid leaving them to the slow post-processing step). Any blobs\n> that we _can't_ find get placed into an array, and then that array is\n> iterated over after we have received all packs, including from the CDN.\n> Any blobs that couldn't be found in the pack transferred from the\n> remote, the CDN, or the local repository (and isn't explicitly excluded\n> via an object --filter) is declared missing.\n> \n> Thoughts?\n\nThe hard part is communicating this array to the parent fetch process.\nStolee has a suggestion [1] which I will reply to directly.\n\n[1] https://lore.kernel.org/git/d2ca2fec-a353-787a-15a7-3831a665523e@gmail.com/\n"},{"id":"414769","messageId":"xmqqsg6wh110.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"20210119191306.1500480-1-jonathantanmy@google.com","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-20T01:04:11Z","receivedAt":"2021-01-20T01:05:18Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n>> It might be nice to teach 'index-pack' a mode that says certain\n>> errors should be reported as warnings by writing the problematic\n>> OIDs to stdout/stderr. Then, the second check after all packs are\n>> present can focus on those problematic objects instead of\n>> re-scanning everything.\n>\n> My initial reaction was that stdout is already used to report the hash\n> part of the generated name and that stderr is already used for whatever\n> warnings there are, but looking at the documentation, index-pack\n> --fsck-objects is \"[for] internal use only\", so it might be fine to\n> extend the output format in this case and report the problematic OIDs\n> after the hash. I'll take a look.\n\nIf I am not mistaken, Taylor also mentioned the possibility to give\n\"these objects need reinspecting\" to a later process, and it is an\nexcellent suggestion. And I think it is perfectly fine to adjust\nthe internal format used purely for internal use.\n\nThanks.\n"},{"id":"414781","messageId":"87o8hk820f.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"20210115234300.350442-1-jonathantanmy@google.com","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-01-20T08:07:44Z","receivedAt":"2021-01-20T08:12:23Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sat, Jan 16 2021, Jonathan Tan wrote:\n\n> Someone at $DAYJOB noticed that if a .gitmodules-containing tree and the\n> .gitmodules blob itself are sent in 2 separate packfiles during a fetch\n> (which can happen when packfile URIs are used), transfer.fsckobjects\n> causes the fetch to fail. You can reproduce it as follows (as of the\n> time of writing):\n>\n> $ git -c fetch.uriprotocols=https -c transfer.fsckobjects=true clone https://chromium.googlesource.com/chromiumos/codesearch\n> Cloning into 'codesearch'...\n> remote: Total 2242 (delta 0), reused 2242 (delta 0)\n> Receiving objects: 100% (2242/2242), 1.77 MiB | 4.62 MiB/s, done.\n> error: object 1f155c20935ee1154a813a814f03ef2b3976680f: gitmodulesMissing: unable to read .gitmodules blob\n> fatal: fsck error in pack objects\n> fatal: index-pack failed\n>\n> This happens because the fsck part is currently being done in\n> index-pack, which operates on one pack at a time. When index-pack sees\n> the tree, it runs fsck on it (like any other object), and the fsck\n> subsystem remembers the .gitmodules target (specifically, in\n> gitmodules_found in fsck.c). Later, index-pack runs fsck_finish() which\n> checks if the target exists, but it doesn't, so it reports the failure.\n>\n> One option is for fetch to do its own pass of checking all downloaded\n> objects once all packfiles have been downloaded, but that seems wasteful\n> as all trees would have to be re-inflated.\n>\n> Another option is to do it within the connectivity check instead - so,\n> update rev-list and the object walking mechanism to be able to detect\n> .gitmodules in trees and fsck the target blob whenever such an entry\n> occurs. This has the advantage that there is no extra re-inflation,\n> although it might be strange to have object walking be able to fsck.\n>\n> The simplest solution would be to just relax this - check the blob if it\n> exists, but if it doesn't, it's OK. Some things in favor of this\n> solution:\n>\n> - This is something we already do in the partial clone case (although\n> it could be argued that in this case, we're already trusting the\n> server for far more than .gitmodules, so just because it's OK in the\n> partial clone case doesn't mean that it's OK in the regular case).\n>\n> - Also, the commit message for this feature (from ed8b10f631 (\"fsck: check\n> .gitmodules content\", 2018-05-21)) gives a rationale of a newer\n> server being able to protect older clients.\n> - Servers using receive-pack (instead of fetch-pack) to obtain\n> objects would still be protected, since receive-pack still only\n> accepts one packfile at a time (and there are currently no plans\n> to expand this).\n> - Also, malicious .gitobjects files could still be crafted that pass\n> fsck checking - for example, by containing a URL (of another\n> server) that refers to a repo with a .gitobjects that would fail\n> fsck.\n>\n> So I would rather go with just relaxing the check, but if consensus is\n> that we should still do it, I'll investigate doing it in the\n> connectivity check.\n\nWould this still behave if the $DAYJOB's packfile-uri server support was\nbehaving as documented in packfile-uri.txt, or just because it has\noutside-spec behavior?\n\nI.e. the spec[1] says this:\n\n This is the implementation: a feature, marked experimental, that\n allows the server to be configured by one or more\n `uploadpack.blobPackfileUri= ` entries. Whenever the list\n of objects to be sent is assembled, all such blobs are excluded,\n replaced with URIs. The client will download those URIs, expecting\n them to each point to packfiles containing single blobs.\n\nWhich I can't see leaving an opening for more than packfile-uri being to\nserve up packfiles which each contain a single blob.\n\nIn that case it seems to me we'd be OK (but I haven't tested), because\nfsck_finish() will call read_object_file() which'll try to read that\n\"blob from the object store when it encounters the \".gitmodules\" tree,\nand because we'd have already downloaded the packfile with the blob\nbefore moving onto the main dialog.\n\nBut as we discussed on-list before[2] this isn't the way packfile-uri\nactually works in the wild. It's really just sending some arbitrary data\nin a pack in that URI, with a server that knows what's in that pack and\nwill send the rest in such a way that everything ends up being\nconnected.\n\nAs far as I can tell the only reason this is called \"packfile URI\" and\nbehaves this way in git.git is because of the convenience of\nintrumenting pack-objects.c with an \"oidset excluded_by_config\" to not\nstream those blobs in a pack, but it isn't how the only (I'm pretty\nsure) production server implementation in the wild behaves at all.\n\nSo *poke* about the reply I had in [3] late last year. I think the first\nthing worth doing here is fixing the docs so they describe how this\nworks. You didn't get back on that (and I also forgot about it until\nthis thread), but it would be nice to know what you think about the\nsuggested prose there.\n\nRe-reading it I'd add something like this to the spec:\n\n A. That the config is called \"uploadpack.blobPackfileUri\" in git.git\n has nothing to do with how this is expected to behave on the\n wire. It's just to serve the narrow support pack-objects.c has for\n crafting such a pack.\n\n B. It's then called \"packfile-uris\" on the wire, nothing to do with\n blobs. Just packs with a checksum that we'll validate. An older\n versions of this spec said \"[a] packfiles containing single blobs\"\n but it can be any combination of blob/tree/commit data.\n\n C. A client is then expected to deal with any combination of data\n ordered/sliced/split up etc. in any possible way from such a\n combination of \"packfile-uris\" and PACK dialog, as long as the end\n result is valid.\n\nExcept that the result of this discussion will perhaps be a more narrow\ndefinition for \"C\".\n\n1. https://github.com/git/git/blob/cd8402e0fd8cfc0ec9fb10e22ffb6aabd992eae1/Documentation/technical/packfile-uri.txt#L37-L41\n2. https://lore.kernel.org/git/20201125190957.1113461-1-jonathantanmy@google.com/\n3. https://lore.kernel.org/git/87tut5vghw.fsf@evledraar.gmail.com/\n"},{"id":"414844","messageId":"20210120193657.1807392-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"87o8hk820f.fsf@evledraar.gmail.com","subject":"[PATCH] Doc: clarify contents of packfile sent as URI","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-20T19:36:57Z","receivedAt":"2021-01-20T19:39:52Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Clarify that, when the packfile-uri feature is used, the client should\nnot assume that the extra packfiles downloaded would only contain a\nsingle blob, but support packfiles containing multiple objects of all\ntypes.\n\nSigned-off-by: Jonathan Tan \n---\n Documentation/technical/packfile-uri.txt | 7 +++++--\n 1 file changed, 5 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/technical/packfile-uri.txt b/Documentation/technical/packfile-uri.txt\nindex 318713abc3..f7eabc6c76 100644\n--- a/Documentation/technical/packfile-uri.txt\n+++ b/Documentation/technical/packfile-uri.txt\n@@ -37,8 +37,11 @@ at least so that we can test the client.\n This is the implementation: a feature, marked experimental, that allows the\n server to be configured by one or more `uploadpack.blobPackfileUri=\n ` entries. Whenever the list of objects to be sent is assembled, all such\n-blobs are excluded, replaced with URIs. The client will download those URIs,\n-expecting them to each point to packfiles containing single blobs.\n+blobs are excluded, replaced with URIs. As noted in \"Future work\" below, the\n+server can evolve in the future to support excluding other objects (or other\n+implementations of servers could be made that support excluding other objects)\n+without needing a protocol change, so clients should not expect that packfiles\n+downloaded in this way only contain single blobs.\n \n Client design\n -------------\n-- \n2.30.0.284.gd98b1dd5eaa7-goog\n\n"},{"id":"414845","messageId":"20210120193053.1804670-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"87o8hk820f.fsf@evledraar.gmail.com","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-20T19:30:53Z","receivedAt":"2021-01-20T19:42:07Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Would this still behave if the $DAYJOB's packfile-uri server support was\n> behaving as documented in packfile-uri.txt, or just because it has\n> outside-spec behavior?\n> \n> I.e. the spec[1] says this:\n> \n> This is the implementation: a feature, marked experimental, that\n> allows the server to be configured by one or more\n> `uploadpack.blobPackfileUri= ` entries. Whenever the list\n> of objects to be sent is assembled, all such blobs are excluded,\n> replaced with URIs. The client will download those URIs, expecting\n> them to each point to packfiles containing single blobs.\n> \n> Which I can't see leaving an opening for more than packfile-uri being to\n> serve up packfiles which each contain a single blob.\n\nI meant to leave an opening by referring to this just as a Minimum\nViable Product and by explaining in Future Work that the protocol allows\nevolution of (among other things) which objects the server sends through\na URI without any protocol changes.\n\nBut in any case, this will also happen even if we constrain ourselves to\nexcluding single blobs and sending them via other packfiles instead -\nsee below.\n\n> In that case it seems to me we'd be OK (but I haven't tested), because\n> fsck_finish() will call read_object_file() which'll try to read that\n> \"blob from the object store when it encounters the \".gitmodules\" tree,\n> and because we'd have already downloaded the packfile with the blob\n> before moving onto the main dialog.\n\nWe wouldn't be OK, actually. Suppose we have a separate packfile\ncontaining only the \".gitmodules\" blob - when we call fsck_finish(), we\nwould not have downloaded the other packfile yet. Git processes the\nentire fetch response by piping the inline packfile (after demux) into\nindex-pack (which is the one that calls fsck_finish()) before it\ndownloads any of the other packfile(s).\n\n> But as we discussed on-list before[2] this isn't the way packfile-uri\n> actually works in the wild. It's really just sending some arbitrary data\n> in a pack in that URI, with a server that knows what's in that pack and\n> will send the rest in such a way that everything ends up being\n> connected.\n> \n> As far as I can tell the only reason this is called \"packfile URI\" and\n> behaves this way in git.git is because of the convenience of\n> intrumenting pack-objects.c with an \"oidset excluded_by_config\" to not\n> stream those blobs in a pack, but it isn't how the only (I'm pretty\n> sure) production server implementation in the wild behaves at all.\n\nI don't know if this is the only production server implementation, but\nyes, this particular one (googlesource.com) can put objects of multiple\ntypes in the other packfile, not only a single blob. There is some JGit\ncode here [1] that can send a URI corresponding to a \"CachedPack\" (which\nmay contain all objects, not only blobs) if that pack is also available\nthrough a URI.\n\n[1] https://gerrit.googlesource.com/jgit/+/a004820858b54d18c6f72fc94dc33bce8b606d66\n\n> So *poke* about the reply I had in [3] late last year. I think the first\n> thing worth doing here is fixing the docs so they describe how this\n> works. You didn't get back on that (and I also forgot about it until\n> this thread), but it would be nice to know what you think about the\n> suggested prose there.\n\nRereading that, the issue is that uploadpack.blobPackfileUri is indeed\nhow the current Git server handles it - it excludes a blob and sends a\nURI instead. The client is not supposed to see how the server has\nconfigured it, and should not be constrained by the fact that the server\nthat is being shipped with it only excludes single blobs.\n\n> Re-reading it I'd add something like this to the spec:\n> \n> A. That the config is called \"uploadpack.blobPackfileUri\" in git.git\n> has nothing to do with how this is expected to behave on the\n> wire. It's just to serve the narrow support pack-objects.c has for\n> crafting such a pack.\n\nYes, that's true.\n\n> B. It's then called \"packfile-uris\" on the wire, nothing to do with\n> blobs. Just packs with a checksum that we'll validate. An older\n> versions of this spec said \"[a] packfiles containing single blobs\"\n> but it can be any combination of blob/tree/commit data.\n\nYes, we can delete that line.\n\n> C. A client is then expected to deal with any combination of data\n> ordered/sliced/split up etc. in any possible way from such a\n> combination of \"packfile-uris\" and PACK dialog, as long as the end\n> result is valid.\n> \n> Except that the result of this discussion will perhaps be a more narrow\n> definition for \"C\".\n\nYes. I think all these can be done just by changing the last sentence in\n\"Server design\" - I'll send a patch.\n\n> 1. https://github.com/git/git/blob/cd8402e0fd8cfc0ec9fb10e22ffb6aabd992eae1/Documentation/technical/packfile-uri.txt#L37-L41\n> 2. https://lore.kernel.org/git/20201125190957.1113461-1-jonathantanmy@google.com/\n> 3. https://lore.kernel.org/git/87tut5vghw.fsf@evledraar.gmail.com/\n"},{"id":"414891","messageId":"xmqq1refdm58.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"20210120193053.1804670-1-jonathantanmy@google.com","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-21T03:06:11Z","receivedAt":"2021-01-21T05:59:56Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> We wouldn't be OK, actually. Suppose we have a separate packfile\n> containing only the \".gitmodules\" blob - when we call fsck_finish(), we\n> would not have downloaded the other packfile yet. Git processes the\n> entire fetch response by piping the inline packfile (after demux) into\n> index-pack (which is the one that calls fsck_finish()) before it\n> downloads any of the other packfile(s).\n\nIs that order documented as a requirement for implementation?\n\nNaïvely, I would expect that a CDN offload would be to relieve\nservers from the burden of having to repack ancient part of the\nhistory all the time for any new \"clone\" clients and that is what\nthe \"here is a URI, go fetch it because I won't give you objects\nthat already appear there\" feature is about. Because we expect that\nthe offloaded contents would not be up-to-date, the traditional\npackfile transfer would then is used to complete the history with\nobjects necessary for the parts of the history newer than the\noffloaded contents.\n\nAnd from that viewpoint, it sounds totally backwards to start\nprocessing the up-to-the-minute fresh packfile that came via the\ntraditional packfile transfer before the CDN offloaded contents are\nfetched and stored safely in our repository.\n\nWe probably want to finish interaction with the live server as\nquickly as possible---it would go counter to that wish if we force\nthe live part of the history hang in flight, unprocessed, while the\nclient downloads offloaded bulk from CDN and processes it, making\nthe server side stuck waiting for some write(2) to go through.\n\nBut I still wonder if it is an option to locally delay the\nprocessing of the up-to-the-minute-fresh part.\n\nInstead of feeding what comes from them directly to \"index-pack\n--fsck-objects\", would it make sense to spool it to a temporary, so\nthat we can release the server early, but then make sure to fetch\nand process packfile URI material before coming back to process the\nspooled packdata. That would allow the newer part of the history to\nhave newer trees that still reference the same old .gitmodules that\nis found in the frozen packfile that comes from CDN, no?\n\nOr can there be a situation where some objects in CDN pack are\nreferred to by objects in the up-to-the-minute-fresh pack (e.g. a\n\".gitmodules\" blob in CDN pack is still unchanged and used in an\nupdated tree in the latest revision) and some other objects in CDN\npack refer to an object in the live part of the history? If there\nis such a cyclic dependency, \"index-pack --fsck\" one pack at a time\nwould not work, but I doubt such a cycle can arise.\n\nThanks.\n"},{"id":"414914","messageId":"20210121183238.1780849-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqq1refdm58.fsf@gitster.c.googlers.com","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-21T18:32:38Z","receivedAt":"2021-01-21T18:34:34Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Jonathan Tan writes:\n> \n> > We wouldn't be OK, actually. Suppose we have a separate packfile\n> > containing only the \".gitmodules\" blob - when we call fsck_finish(), we\n> > would not have downloaded the other packfile yet. Git processes the\n> > entire fetch response by piping the inline packfile (after demux) into\n> > index-pack (which is the one that calls fsck_finish()) before it\n> > downloads any of the other packfile(s).\n> \n> Is that order documented as a requirement for implementation?\n> \n> Naïvely, I would expect that a CDN offload would be to relieve\n> servers from the burden of having to repack ancient part of the\n> history all the time for any new \"clone\" clients and that is what\n> the \"here is a URI, go fetch it because I won't give you objects\n> that already appear there\" feature is about. Because we expect that\n> the offloaded contents would not be up-to-date, the traditional\n> packfile transfer would then is used to complete the history with\n> objects necessary for the parts of the history newer than the\n> offloaded contents.\n> \n> And from that viewpoint, it sounds totally backwards to start\n> processing the up-to-the-minute fresh packfile that came via the\n> traditional packfile transfer before the CDN offloaded contents are\n> fetched and stored safely in our repository.\n> \n> We probably want to finish interaction with the live server as\n> quickly as possible---it would go counter to that wish if we force\n> the live part of the history hang in flight, unprocessed, while the\n> client downloads offloaded bulk from CDN and processes it, making\n> the server side stuck waiting for some write(2) to go through.\n> \n> But I still wonder if it is an option to locally delay the\n> processing of the up-to-the-minute-fresh part.\n> \n> Instead of feeding what comes from them directly to \"index-pack\n> --fsck-objects\", would it make sense to spool it to a temporary, so\n> that we can release the server early, but then make sure to fetch\n> and process packfile URI material before coming back to process the\n> spooled packdata. That would allow the newer part of the history to\n> have newer trees that still reference the same old .gitmodules that\n> is found in the frozen packfile that comes from CDN, no?\n> \n> Or can there be a situation where some objects in CDN pack are\n> referred to by objects in the up-to-the-minute-fresh pack (e.g. a\n> \".gitmodules\" blob in CDN pack is still unchanged and used in an\n> updated tree in the latest revision) and some other objects in CDN\n> pack refer to an object in the live part of the history? If there\n> is such a cyclic dependency, \"index-pack --fsck\" one pack at a time\n> would not work, but I doubt such a cycle can arise.\n\nMy intention is that the order of the packfiles (and cyclic\ndependencies) would not matter, so we wouldn't need to delay any\nprocessing of the up-to-the-minute-fresh part. I'm currently working on\ngetting index-pack to output a list of the dangling .gitmodules files,\nso that fetch-pack (its consumer) can do one final fsck on those files.\n\nAnother way, as you said, is to say that the order of the packfiles\nmatters (which potentially allows some simplification on the client\nside) but I don't think that we need to lose this flexibility.\n"},{"id":"414917","messageId":"xmqqbldicexd.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"20210121183238.1780849-1-jonathantanmy@google.com","subject":"Re: RFC on packfile URIs and .gitmodules check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-21T18:39:42Z","receivedAt":"2021-01-21T18:45:12Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n>> Jonathan Tan writes:\n>> \n>> Or can there be a situation where some objects in CDN pack are\n>> referred to by objects in the up-to-the-minute-fresh pack (e.g. a\n>> \".gitmodules\" blob in CDN pack is still unchanged and used in an\n>> updated tree in the latest revision) and some other objects in CDN\n>> pack refer to an object in the live part of the history? If there\n>> is such a cyclic dependency, \"index-pack --fsck\" one pack at a time\n>> would not work, but I doubt such a cycle can arise.\n>\n> My intention is that the order of the packfiles (and cyclic\n> dependencies) would not matter...\n> I'm currently working on\n> getting index-pack to output a list of the dangling .gitmodules files,\n> so that fetch-pack (its consumer) can do one final fsck on those files.\n\nIn other words, it essentially becomes \"we check everything we\nobtained as a single unit across multiple packs, but for performance\nwe'll let index-pack work as much as possible on each individual\npack while it has necessary data in its core, and then we conclude\nby checking the objects on the 'boundaries' that cannot be validated\nusing info that is only in one pack\".\n\nThat does sound like the right approach. THanks.\n"},{"id":"415117","messageId":"cover.1611455251.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"20210115234300.350442-1-jonathantanmy@google.com","subject":"[PATCH 0/4] Check .gitmodules when using packfile URIs","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-24T02:34:26Z","receivedAt":"2021-01-24T02:35:40Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This patch set resolves the .gitmodules-and-tree-in-separate-packfiles\nissue I mentioned in [1] by having index-pack print out all dangling\n.gitmodules (instead of returning with an error code) and then teaching\nfetch-pack to read those and run its own fsck checks after all\nindex-pack invocations are complete.\n\nAs part of this, index-pack has to output (1) the hash that goes into\nthe name of the .pack/.idx file and (2) the hashes of all dangling\n.gitmodules. I just had (2) come after (1). If anyone has a better idea,\nI'm interested.\n\nI also discovered a bug in that different index-pack arguments were used\nwhen processing the inline packfile and when processing the ones\nreferenced by URIs. Patch 1-3 fixes that bug by passing the arguments to\nuse as a space-separated URL-encoded list. (URL-encoded so that we can\nhave spaces in the arguments.) Again, if anyone has a better idea, I'm\ninterested. It is only in patch 4 that we have the dangling .gitmodules\nfix.\n\n[1] https://lore.kernel.org/git/20210115234300.350442-1-jonathantanmy@google.com/\n\nJonathan Tan (4):\n http: allow custom index-pack args\n http-fetch: allow custom index-pack args\n fetch-pack: with packfile URIs, use index-pack arg\n fetch-pack: print and use dangling .gitmodules\n\n Documentation/git-http-fetch.txt | 9 ++-\n Documentation/git-index-pack.txt | 7 +-\n builtin/index-pack.c | 9 ++-\n builtin/receive-pack.c | 2 +-\n fetch-pack.c | 106 ++++++++++++++++++++++++++-----\n fsck.c | 16 +++--\n fsck.h | 8 +++\n http-fetch.c | 35 +++++++++-\n http.c | 15 +++--\n http.h | 10 +--\n pack-write.c | 8 ++-\n pack.h | 2 +-\n t/t5550-http-fetch-dumb.sh | 3 +-\n t/t5702-protocol-v2.sh | 47 ++++++++++++++\n 14 files changed, 232 insertions(+), 45 deletions(-)\n\n-- \n2.30.0.280.ga3ce27912f-goog\n\n"},{"id":"415118","messageId":"e5d84a2d946f2770cc2034a100e741607faf62ee.1611455251.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"cover.1611455251.git.jonathantanmy@google.com","subject":"[PATCH 1/4] http: allow custom index-pack args","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-24T02:34:27Z","receivedAt":"2021-01-24T02:35:40Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Currently, when fetching, packfiles referenced by URIs are run through\nindex-pack without any arguments other than --stdin and --keep, no\nmatter what arguments are used for the packfile that is inline in the\nfetch response. As a preparation for ensuring that all packs (whether\ninline or not) use the same index-pack arguments, teach the http\nsubsystem to allow custom index-pack arguments.\n\nhttp-fetch has been updated to use the new API. For now, it passes\n--keep alone instead of --keep with a process ID, but this is only\ntemporary because http-fetch itself will be taught to accept index-pack\nparameters (instead of using a hardcoded constant) in a subsequent\ncommit.\n\nSigned-off-by: Jonathan Tan \n---\n http-fetch.c | 6 +++++-\n http.c | 15 ++++++++-------\n http.h | 10 +++++-----\n 3 files changed, 18 insertions(+), 13 deletions(-)\n\ndiff --git a/http-fetch.c b/http-fetch.c\nindex c4ccc5fea9..2d1d9d054f 100644\n--- a/http-fetch.c\n+++ b/http-fetch.c\n@@ -43,6 +43,9 @@ static int fetch_using_walker(const char *raw_url, int get_verbosely,\n \treturn rc;\n }\n \n+static const char *index_pack_args[] =\n+\t{\"index-pack\", \"--stdin\", \"--keep\", NULL};\n+\n static void fetch_single_packfile(struct object_id *packfile_hash,\n \t\t\t\t const char *url) {\n \tstruct http_pack_request *preq;\n@@ -55,7 +58,8 @@ static void fetch_single_packfile(struct object_id *packfile_hash,\n \tif (preq == NULL)\n \t\tdie(\"couldn't create http pack request\");\n \tpreq->slot->results = &results;\n-\tpreq->generate_keep = 1;\n+\tpreq->index_pack_args = index_pack_args;\n+\tpreq->preserve_index_pack_stdout = 1;\n \n \tif (start_active_slot(preq->slot)) {\n \t\trun_active_slot(preq->slot);\ndiff --git a/http.c b/http.c\nindex 8b23a546af..f8ea28bb2e 100644\n--- a/http.c\n+++ b/http.c\n@@ -2259,6 +2259,9 @@ void release_http_pack_request(struct http_pack_request *preq)\n \tfree(preq);\n }\n \n+static const char *default_index_pack_args[] =\n+\t{\"index-pack\", \"--stdin\", NULL};\n+\n int finish_http_pack_request(struct http_pack_request *preq)\n {\n \tstruct child_process ip = CHILD_PROCESS_INIT;\n@@ -2270,17 +2273,15 @@ int finish_http_pack_request(struct http_pack_request *preq)\n \n \ttmpfile_fd = xopen(preq->tmpfile.buf, O_RDONLY);\n \n-\tstrvec_push(&ip.args, \"index-pack\");\n-\tstrvec_push(&ip.args, \"--stdin\");\n \tip.git_cmd = 1;\n \tip.in = tmpfile_fd;\n-\tif (preq->generate_keep) {\n-\t\tstrvec_pushf(&ip.args, \"--keep=git %\"PRIuMAX,\n-\t\t\t (uintmax_t)getpid());\n+\tip.argv = preq->index_pack_args ? preq->index_pack_args\n+\t\t\t\t\t: default_index_pack_args;\n+\n+\tif (preq->preserve_index_pack_stdout)\n \t\tip.out = 0;\n-\t} else {\n+\telse\n \t\tip.no_stdout = 1;\n-\t}\n \n \tif (run_command(&ip)) {\n \t\tret = -1;\ndiff --git a/http.h b/http.h\nindex 5de792ef3f..bf3d1270ad 100644\n--- a/http.h\n+++ b/http.h\n@@ -218,12 +218,12 @@ struct http_pack_request {\n \tchar *url;\n \n \t/*\n-\t * If this is true, finish_http_pack_request() will pass \"--keep\" to\n-\t * index-pack, resulting in the creation of a keep file, and will not\n-\t * suppress its stdout (that is, the \"keep\\t\\n\" line will be\n-\t * printed to stdout).\n+\t * index-pack command to run. Must be terminated by NULL.\n+\t *\n+\t * If NULL, defaults to\t{\"index-pack\", \"--stdin\", NULL}.\n \t */\n-\tunsigned generate_keep : 1;\n+\tconst char **index_pack_args;\n+\tunsigned preserve_index_pack_stdout : 1;\n \n \tFILE *packfile;\n \tstruct strbuf tmpfile;\n-- \n2.30.0.280.ga3ce27912f-goog\n\n"},{"id":"415119","messageId":"3032117ee747e72f6208eab7f960737d25f8a82a.1611455251.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"cover.1611455251.git.jonathantanmy@google.com","subject":"[PATCH 2/4] http-fetch: allow custom index-pack args","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-24T02:34:28Z","receivedAt":"2021-01-24T02:35:40Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is the next step in teaching fetch-pack to pass its index-pack\narguments when processing packfiles referenced by URIs.\n\nThe \"--keep\" in fetch-pack.c will be replaced with a full message in a\nsubsequent commit.\n\nSigned-off-by: Jonathan Tan \n---\n Documentation/git-http-fetch.txt | 9 ++++++--\n fetch-pack.c | 1 +\n http-fetch.c | 35 +++++++++++++++++++++++++++-----\n t/t5550-http-fetch-dumb.sh | 3 ++-\n 4 files changed, 40 insertions(+), 8 deletions(-)\n\ndiff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt\nindex 4deb4893f5..aa171088e8 100644\n--- a/Documentation/git-http-fetch.txt\n+++ b/Documentation/git-http-fetch.txt\n@@ -41,11 +41,16 @@ commit-id::\n \t\t['\\t']\n \n --packfile=::\n-\tInstead of a commit id on the command line (which is not expected in\n+\tFor internal use only. Instead of a commit id on the command line (which is not expected in\n \tthis case), 'git http-fetch' fetches the packfile directly at the given\n \tURL and uses index-pack to generate corresponding .idx and .keep files.\n \tThe hash is used to determine the name of the temporary file and is\n-\tarbitrary. The output of index-pack is printed to stdout.\n+\tarbitrary. The output of index-pack is printed to stdout. Requires\n+\t--index-pack-args.\n+\n+--index-pack-args=::\n+\tFor internal use only. The command to run on the contents of the\n+\tdownloaded pack. Arguments are URL-encoded separated by spaces.\n \n --recover::\n \tVerify that everything reachable from target is fetched. Used after\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 876f90c759..274ae602f7 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -1645,6 +1645,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n \t\t\t (int) the_hash_algo->hexsz,\n \t\t\t packfile_uris.items[i].string);\n+\t\tstrvec_push(&cmd.args, \"--index-pack-args=index-pack --stdin --keep\");\n \t\tstrvec_push(&cmd.args, uri);\n \t\tcmd.git_cmd = 1;\n \t\tcmd.no_stdin = 1;\ndiff --git a/http-fetch.c b/http-fetch.c\nindex 2d1d9d054f..12feb84e71 100644\n--- a/http-fetch.c\n+++ b/http-fetch.c\n@@ -3,6 +3,7 @@\n #include \"exec-cmd.h\"\n #include \"http.h\"\n #include \"walker.h\"\n+#include \"strvec.h\"\n \n static const char http_fetch_usage[] = \"git http-fetch \"\n \"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin | --packfile=hash | commit-id] url\";\n@@ -43,11 +44,9 @@ static int fetch_using_walker(const char *raw_url, int get_verbosely,\n \treturn rc;\n }\n \n-static const char *index_pack_args[] =\n-\t{\"index-pack\", \"--stdin\", \"--keep\", NULL};\n-\n static void fetch_single_packfile(struct object_id *packfile_hash,\n-\t\t\t\t const char *url) {\n+\t\t\t\t const char *url,\n+\t\t\t\t const char **index_pack_args) {\n \tstruct http_pack_request *preq;\n \tstruct slot_results results;\n \tint ret;\n@@ -90,6 +89,7 @@ int cmd_main(int argc, const char **argv)\n \tint packfile = 0;\n \tint nongit;\n \tstruct object_id packfile_hash;\n+\tconst char *index_pack_args = NULL;\n \n \tsetup_git_directory_gently(&nongit);\n \n@@ -116,6 +116,8 @@ int cmd_main(int argc, const char **argv)\n \t\t\tpackfile = 1;\n \t\t\tif (parse_oid_hex(p, &packfile_hash, &end) || *end)\n \t\t\t\tdie(_(\"argument to --packfile must be a valid hash (got '%s')\"), p);\n+\t\t} else if (skip_prefix(argv[arg], \"--index-pack-args=\", &p)) {\n+\t\t\tindex_pack_args = p;\n \t\t}\n \t\targ++;\n \t}\n@@ -128,10 +130,33 @@ int cmd_main(int argc, const char **argv)\n \tgit_config(git_default_config, NULL);\n \n \tif (packfile) {\n-\t\tfetch_single_packfile(&packfile_hash, argv[arg]);\n+\t\tstruct strvec encoded = STRVEC_INIT;\n+\t\tchar **raw;\n+\t\tint i;\n+\n+\t\tif (!index_pack_args)\n+\t\t\tdie(_(\"--packfile requires --index-pack-args\"));\n+\n+\t\tstrvec_split(&encoded, index_pack_args);\n+\n+\t\tCALLOC_ARRAY(raw, encoded.nr + 1);\n+\t\tfor (i = 0; i < encoded.nr; i++)\n+\t\t\traw[i] = url_percent_decode(encoded.v[i]);\n+\n+\t\tfetch_single_packfile(&packfile_hash, argv[arg],\n+\t\t\t\t (const char **) raw);\n+\n+\t\tfor (i = 0; i < encoded.nr; i++)\n+\t\t\tfree(raw[i]);\n+\t\tfree(raw);\n+\t\tstrvec_clear(&encoded);\n+\n \t\treturn 0;\n \t}\n \n+\tif (index_pack_args)\n+\t\tdie(_(\"--index-pack-args can only be used with --packfile\"));\n+\n \tif (commits_on_stdin) {\n \t\tcommits = walker_targets_stdin(&commit_id, &write_ref);\n \t} else {\ndiff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh\nindex 483578b2d7..af90e7efed 100755\n--- a/t/t5550-http-fetch-dumb.sh\n+++ b/t/t5550-http-fetch-dumb.sh\n@@ -224,7 +224,8 @@ test_expect_success 'http-fetch --packfile' '\n \n \tgit init packfileclient &&\n \tp=$(cd \"$HTTPD_DOCUMENT_ROOT_PATH\"/repo_pack.git && ls objects/pack/pack-*.pack) &&\n-\tgit -C packfileclient http-fetch --packfile=$ARBITRARY \"$HTTPD_URL\"/dumb/repo_pack.git/$p >out &&\n+\tgit -C packfileclient http-fetch --packfile=$ARBITRARY \\\n+\t\t--index-pack-args=\"index-pack --stdin --keep\" \"$HTTPD_URL\"/dumb/repo_pack.git/$p >out &&\n \n \tgrep \"^keep.[0-9a-f]\\{16,\\}$\" out &&\n \tcut -c6- out >packhash &&\n-- \n2.30.0.280.ga3ce27912f-goog\n\n"},{"id":"415120","messageId":"c985fc598240cb9566345cf71bb353508f525469.1611455251.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"cover.1611455251.git.jonathantanmy@google.com","subject":"[PATCH 3/4] fetch-pack: with packfile URIs, use index-pack arg","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-24T02:34:29Z","receivedAt":"2021-01-24T02:35:40Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Unify the index-pack arguments used when processing the inline pack and\nwhen downloading packfiles referenced by URIs. This is done by teaching\nget_pack() to also store the index-pack arguments whenever at least one\npackfile URI is given, and then when processing the packfile URI(s),\nusing the stored arguments.\n\nSigned-off-by: Jonathan Tan \n---\n fetch-pack.c | 35 ++++++++++++++++++++++++++---------\n 1 file changed, 26 insertions(+), 9 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 274ae602f7..fe69635eb5 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -797,12 +797,13 @@ static void write_promisor_file(const char *keep_name,\n }\n \n /*\n- * Pass 1 as \"only_packfile\" if the pack received is the only pack in this\n- * fetch request (that is, if there were no packfile URIs provided).\n+ * If packfile URIs were provided, pass a non-NULL pointer to index_pack_args.\n+ * The string to pass as the --index-pack-args argument to http-fetch will be\n+ * stored there. (It must be freed by the caller.)\n */\n static int get_pack(struct fetch_pack_args *args,\n \t\t int xd[2], struct string_list *pack_lockfiles,\n-\t\t int only_packfile,\n+\t\t char **index_pack_args,\n \t\t struct ref **sought, int nr_sought)\n {\n \tstruct async demux;\n@@ -845,7 +846,7 @@ static int get_pack(struct fetch_pack_args *args,\n \t\tstrvec_push(&cmd.args, alternate_shallow_file);\n \t}\n \n-\tif (do_keep || args->from_promisor) {\n+\tif (do_keep || args->from_promisor || index_pack_args) {\n \t\tif (pack_lockfiles)\n \t\t\tcmd.out = -1;\n \t\tcmd_name = \"index-pack\";\n@@ -863,7 +864,7 @@ static int get_pack(struct fetch_pack_args *args,\n \t\t\t\t \"--keep=fetch-pack %\"PRIuMAX \" on %s\",\n \t\t\t\t (uintmax_t)getpid(), hostname);\n \t\t}\n-\t\tif (only_packfile && args->check_self_contained_and_connected)\n+\t\tif (!index_pack_args && args->check_self_contained_and_connected)\n \t\t\tstrvec_push(&cmd.args, \"--check-self-contained-and-connected\");\n \t\telse\n \t\t\t/*\n@@ -901,7 +902,7 @@ static int get_pack(struct fetch_pack_args *args,\n \t : transfer_fsck_objects >= 0\n \t ? transfer_fsck_objects\n \t : 0) {\n-\t\tif (args->from_promisor || !only_packfile)\n+\t\tif (args->from_promisor || index_pack_args)\n \t\t\t/*\n \t\t\t * We cannot use --strict in index-pack because it\n \t\t\t * checks both broken objects and links, but we only\n@@ -913,6 +914,19 @@ static int get_pack(struct fetch_pack_args *args,\n \t\t\t\t fsck_msg_types.buf);\n \t}\n \n+\tif (index_pack_args) {\n+\t\tstruct strbuf joined = STRBUF_INIT;\n+\t\tint i;\n+\n+\t\tfor (i = 0; i < cmd.args.nr; i++) {\n+\t\t\tif (i)\n+\t\t\t\tstrbuf_addch(&joined, ' ');\n+\t\t\tstrbuf_addstr_urlencode(&joined, cmd.args.v[i],\n+\t\t\t\t\t\tis_rfc3986_unreserved);\n+\t\t}\n+\t\t*index_pack_args = strbuf_detach(&joined, NULL);\n+\t}\n+\n \tcmd.in = demux.out;\n \tcmd.git_cmd = 1;\n \tif (start_command(&cmd))\n@@ -1084,7 +1098,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\talternate_shallow_file = setup_temporary_shallow(si->shallow);\n \telse\n \t\talternate_shallow_file = NULL;\n-\tif (get_pack(args, fd, pack_lockfiles, 1, sought, nr_sought))\n+\tif (get_pack(args, fd, pack_lockfiles, NULL, sought, nr_sought))\n \t\tdie(_(\"git fetch-pack: fetch failed.\"));\n \n all_done:\n@@ -1535,6 +1549,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tint seen_ack = 0;\n \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n \tint i;\n+\tchar *index_pack_args = NULL;\n \n \tnegotiator = &negotiator_alloc;\n \tfetch_negotiator_init(r, negotiator);\n@@ -1624,7 +1639,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\t\treceive_packfile_uris(&reader, &packfile_uris);\n \t\t\tprocess_section_header(&reader, \"packfile\", 0);\n \t\t\tif (get_pack(args, fd, pack_lockfiles,\n-\t\t\t\t !packfile_uris.nr, sought, nr_sought))\n+\t\t\t\t packfile_uris.nr ? &index_pack_args : NULL,\n+\t\t\t\t sought, nr_sought))\n \t\t\t\tdie(_(\"git fetch-pack: fetch failed.\"));\n \t\t\tdo_check_stateless_delimiter(args, &reader);\n \n@@ -1645,7 +1661,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n \t\t\t (int) the_hash_algo->hexsz,\n \t\t\t packfile_uris.items[i].string);\n-\t\tstrvec_push(&cmd.args, \"--index-pack-args=index-pack --stdin --keep\");\n+\t\tstrvec_pushf(&cmd.args, \"--index-pack-args=%s\", index_pack_args);\n \t\tstrvec_push(&cmd.args, uri);\n \t\tcmd.git_cmd = 1;\n \t\tcmd.no_stdin = 1;\n@@ -1681,6 +1697,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\t\t\t\t packname));\n \t}\n \tstring_list_clear(&packfile_uris, 0);\n+\tFREE_AND_NULL(index_pack_args);\n \n \tif (negotiator)\n \t\tnegotiator->release(negotiator);\n-- \n2.30.0.280.ga3ce27912f-goog\n\n"},{"id":"415121","messageId":"2d6d8c2671fe424c752994dcb5277d4d923e17a0.1611455251.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"cover.1611455251.git.jonathantanmy@google.com","subject":"[PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-24T02:34:30Z","receivedAt":"2021-01-24T02:35:40Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Teach index-pack to print dangling .gitmodules links after its \"keep\" or\n\"pack\" line instead of declaring an error, and teach fetch-pack to check\nsuch lines printed.\n\nThis allows the tree side of the .gitmodules link to be in one packfile\nand the blob side to be in another without failing the fsck check,\nbecause it is now fetch-pack which checks such objects after all\npackfiles have been downloaded and indexed (and not index-pack on an\nindividual packfile, as it is before this commit).\n\nSigned-off-by: Jonathan Tan \n---\n Documentation/git-index-pack.txt | 7 ++-\n builtin/index-pack.c | 9 +++-\n builtin/receive-pack.c | 2 +-\n fetch-pack.c | 78 +++++++++++++++++++++++++++-----\n fsck.c | 16 +++++--\n fsck.h | 8 ++++\n pack-write.c | 8 +++-\n pack.h | 2 +-\n t/t5702-protocol-v2.sh | 47 +++++++++++++++++++\n 9 files changed, 155 insertions(+), 22 deletions(-)\n\ndiff --git a/Documentation/git-index-pack.txt b/Documentation/git-index-pack.txt\nindex af0c26232c..e74a4a1eda 100644\n--- a/Documentation/git-index-pack.txt\n+++ b/Documentation/git-index-pack.txt\n@@ -78,7 +78,12 @@ OPTIONS\n \tDie if the pack contains broken links. For internal use only.\n \n --fsck-objects::\n-\tDie if the pack contains broken objects. For internal use only.\n+\tFor internal use only.\n++\n+Die if the pack contains broken objects. If the pack contains a tree\n+pointing to a .gitmodules blob that does not exist, prints the hash of\n+that blob (for the caller to check) after the hash that goes into the\n+name of the pack/idx file (see \"Notes\").\n \n --threads=::\n \tSpecifies the number of threads to spawn when resolving\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 557bd2f348..f995c15115 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1888,8 +1888,13 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \telse\n \t\tclose(input_fd);\n \n-\tif (do_fsck_object && fsck_finish(&fsck_options))\n-\t\tdie(_(\"fsck error in pack objects\"));\n+\tif (do_fsck_object) {\n+\t\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n+\n+\t\tfo.print_dangling_gitmodules = 1;\n+\t\tif (fsck_finish(&fo))\n+\t\t\tdie(_(\"fsck error in pack objects\"));\n+\t}\n \n \tfree(objects);\n \tstrbuf_release(&index_name_buf);\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex d49d050e6e..ed2c9b42e9 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -2275,7 +2275,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)\n \t\tstatus = start_command(&child);\n \t\tif (status)\n \t\t\treturn \"index-pack fork failed\";\n-\t\tpack_lockfile = index_pack_lockfile(child.out);\n+\t\tpack_lockfile = index_pack_lockfile(child.out, NULL);\n \t\tclose(child.out);\n \t\tstatus = finish_command(&child);\n \t\tif (status)\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex fe69635eb5..128362e0ba 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -796,6 +796,26 @@ static void write_promisor_file(const char *keep_name,\n \tstrbuf_release(&promisor_name);\n }\n \n+static void parse_gitmodules_oids(int fd, struct oidset *gitmodules_oids)\n+{\n+\tint len = the_hash_algo->hexsz + 1; /* hash + NL */\n+\n+\tdo {\n+\t\tchar hex_hash[GIT_MAX_HEXSZ + 1];\n+\t\tint read_len = read_in_full(fd, hex_hash, len);\n+\t\tstruct object_id oid;\n+\t\tconst char *end;\n+\n+\t\tif (!read_len)\n+\t\t\treturn;\n+\t\tif (read_len != len)\n+\t\t\tdie(\"invalid length read %d\", read_len);\n+\t\tif (parse_oid_hex(hex_hash, &oid, &end) || *end != '\\n')\n+\t\t\tdie(\"invalid hash\");\n+\t\toidset_insert(gitmodules_oids, &oid);\n+\t} while (1);\n+}\n+\n /*\n * If packfile URIs were provided, pass a non-NULL pointer to index_pack_args.\n * The string to pass as the --index-pack-args argument to http-fetch will be\n@@ -804,7 +824,8 @@ static void write_promisor_file(const char *keep_name,\n static int get_pack(struct fetch_pack_args *args,\n \t\t int xd[2], struct string_list *pack_lockfiles,\n \t\t char **index_pack_args,\n-\t\t struct ref **sought, int nr_sought)\n+\t\t struct ref **sought, int nr_sought,\n+\t\t struct oidset *gitmodules_oids)\n {\n \tstruct async demux;\n \tint do_keep = args->keep_pack;\n@@ -812,6 +833,7 @@ static int get_pack(struct fetch_pack_args *args,\n \tstruct pack_header header;\n \tint pass_header = 0;\n \tstruct child_process cmd = CHILD_PROCESS_INIT;\n+\tint fsck_objects = 0;\n \tint ret;\n \n \tmemset(&demux, 0, sizeof(demux));\n@@ -846,8 +868,15 @@ static int get_pack(struct fetch_pack_args *args,\n \t\tstrvec_push(&cmd.args, alternate_shallow_file);\n \t}\n \n-\tif (do_keep || args->from_promisor || index_pack_args) {\n-\t\tif (pack_lockfiles)\n+\tif (fetch_fsck_objects >= 0\n+\t ? fetch_fsck_objects\n+\t : transfer_fsck_objects >= 0\n+\t ? transfer_fsck_objects\n+\t : 0)\n+\t\tfsck_objects = 1;\n+\n+\tif (do_keep || args->from_promisor || index_pack_args || fsck_objects) {\n+\t\tif (pack_lockfiles || fsck_objects)\n \t\t\tcmd.out = -1;\n \t\tcmd_name = \"index-pack\";\n \t\tstrvec_push(&cmd.args, cmd_name);\n@@ -897,11 +926,7 @@ static int get_pack(struct fetch_pack_args *args,\n \t\tstrvec_pushf(&cmd.args, \"--pack_header=%\"PRIu32\",%\"PRIu32,\n \t\t\t ntohl(header.hdr_version),\n \t\t\t\t ntohl(header.hdr_entries));\n-\tif (fetch_fsck_objects >= 0\n-\t ? fetch_fsck_objects\n-\t : transfer_fsck_objects >= 0\n-\t ? transfer_fsck_objects\n-\t : 0) {\n+\tif (fsck_objects) {\n \t\tif (args->from_promisor || index_pack_args)\n \t\t\t/*\n \t\t\t * We cannot use --strict in index-pack because it\n@@ -931,10 +956,15 @@ static int get_pack(struct fetch_pack_args *args,\n \tcmd.git_cmd = 1;\n \tif (start_command(&cmd))\n \t\tdie(_(\"fetch-pack: unable to fork off %s\"), cmd_name);\n-\tif (do_keep && pack_lockfiles) {\n-\t\tchar *pack_lockfile = index_pack_lockfile(cmd.out);\n+\tif (do_keep && (pack_lockfiles || fsck_objects)) {\n+\t\tint is_well_formed;\n+\t\tchar *pack_lockfile = index_pack_lockfile(cmd.out, &is_well_formed);\n+\n+\t\tif (!is_well_formed)\n+\t\t\tdie(_(\"fetch-pack: invalid index-pack output\"));\n \t\tif (pack_lockfile)\n \t\t\tstring_list_append_nodup(pack_lockfiles, pack_lockfile);\n+\t\tparse_gitmodules_oids(cmd.out, gitmodules_oids);\n \t\tclose(cmd.out);\n \t}\n \n@@ -969,6 +999,22 @@ static int cmp_ref_by_name(const void *a_, const void *b_)\n \treturn strcmp(a->name, b->name);\n }\n \n+static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n+{\n+\tstruct oidset_iter iter;\n+\tconst struct object_id *oid;\n+\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n+\n+\tif (!oidset_size(gitmodules_oids))\n+\t\treturn;\n+\n+\toidset_iter_init(gitmodules_oids, &iter);\n+\twhile ((oid = oidset_iter_next(&iter)))\n+\t\tregister_found_gitmodules(oid);\n+\tif (fsck_finish(&fo))\n+\t\tdie(\"fsck failed\");\n+}\n+\n static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\t\t\t int fd[2],\n \t\t\t\t const struct ref *orig_ref,\n@@ -983,6 +1029,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \tint agent_len;\n \tstruct fetch_negotiator negotiator_alloc;\n \tstruct fetch_negotiator *negotiator;\n+\tstruct oidset gitmodules_oids = OIDSET_INIT;\n \n \tnegotiator = &negotiator_alloc;\n \tfetch_negotiator_init(r, negotiator);\n@@ -1098,8 +1145,10 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\talternate_shallow_file = setup_temporary_shallow(si->shallow);\n \telse\n \t\talternate_shallow_file = NULL;\n-\tif (get_pack(args, fd, pack_lockfiles, NULL, sought, nr_sought))\n+\tif (get_pack(args, fd, pack_lockfiles, NULL, sought, nr_sought,\n+\t\t &gitmodules_oids))\n \t\tdie(_(\"git fetch-pack: fetch failed.\"));\n+\tfsck_gitmodules_oids(&gitmodules_oids);\n \n all_done:\n \tif (negotiator)\n@@ -1550,6 +1599,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n \tint i;\n \tchar *index_pack_args = NULL;\n+\tstruct oidset gitmodules_oids = OIDSET_INIT;\n \n \tnegotiator = &negotiator_alloc;\n \tfetch_negotiator_init(r, negotiator);\n@@ -1640,7 +1690,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\tprocess_section_header(&reader, \"packfile\", 0);\n \t\t\tif (get_pack(args, fd, pack_lockfiles,\n \t\t\t\t packfile_uris.nr ? &index_pack_args : NULL,\n-\t\t\t\t sought, nr_sought))\n+\t\t\t\t sought, nr_sought, &gitmodules_oids))\n \t\t\t\tdie(_(\"git fetch-pack: fetch failed.\"));\n \t\t\tdo_check_stateless_delimiter(args, &reader);\n \n@@ -1680,6 +1730,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \n \t\tpackname[the_hash_algo->hexsz] = '\\0';\n \n+\t\tparse_gitmodules_oids(cmd.out, &gitmodules_oids);\n+\n \t\tclose(cmd.out);\n \n \t\tif (finish_command(&cmd))\n@@ -1699,6 +1751,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstring_list_clear(&packfile_uris, 0);\n \tFREE_AND_NULL(index_pack_args);\n \n+\tfsck_gitmodules_oids(&gitmodules_oids);\n+\n \tif (negotiator)\n \t\tnegotiator->release(negotiator);\n \ndiff --git a/fsck.c b/fsck.c\nindex f82e2fe9e3..04f3d342af 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1243,6 +1243,11 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n+void register_found_gitmodules(const struct object_id *oid)\n+{\n+\toidset_insert(&gitmodules_found, oid);\n+}\n+\n int fsck_finish(struct fsck_options *options)\n {\n \tint ret = 0;\n@@ -1262,10 +1267,13 @@ int fsck_finish(struct fsck_options *options)\n \t\tif (!buf) {\n \t\t\tif (is_promisor_object(oid))\n \t\t\t\tcontinue;\n-\t\t\tret |= report(options,\n-\t\t\t\t oid, OBJ_BLOB,\n-\t\t\t\t FSCK_MSG_GITMODULES_MISSING,\n-\t\t\t\t \"unable to read .gitmodules blob\");\n+\t\t\tif (options->print_dangling_gitmodules)\n+\t\t\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n+\t\t\telse\n+\t\t\t\tret |= report(options,\n+\t\t\t\t\t oid, OBJ_BLOB,\n+\t\t\t\t\t FSCK_MSG_GITMODULES_MISSING,\n+\t\t\t\t\t \"unable to read .gitmodules blob\");\n \t\t\tcontinue;\n \t\t}\n \ndiff --git a/fsck.h b/fsck.h\nindex 69cf715e79..4b8cf03445 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -41,6 +41,12 @@ struct fsck_options {\n \tint *msg_type;\n \tstruct oidset skiplist;\n \tkh_oid_map_t *object_names;\n+\n+\t/*\n+\t * If 1, print the hashes of missing .gitmodules blobs instead of\n+\t * considering them to be errors.\n+\t */\n+\tunsigned print_dangling_gitmodules:1;\n };\n \n #define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n@@ -62,6 +68,8 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n+void register_found_gitmodules(const struct object_id *oid);\n+\n /*\n * Some fsck checks are context-dependent, and may end up queued; run this\n * after completing all fsck_object() calls in order to resolve any remaining\ndiff --git a/pack-write.c b/pack-write.c\nindex 3513665e1e..f66ea8e5a1 100644\n--- a/pack-write.c\n+++ b/pack-write.c\n@@ -272,7 +272,7 @@ void fixup_pack_header_footer(int pack_fd,\n \tfsync_or_die(pack_fd, pack_name);\n }\n \n-char *index_pack_lockfile(int ip_out)\n+char *index_pack_lockfile(int ip_out, int *is_well_formed)\n {\n \tchar packname[GIT_MAX_HEXSZ + 6];\n \tconst int len = the_hash_algo->hexsz + 6;\n@@ -286,11 +286,17 @@ char *index_pack_lockfile(int ip_out)\n \t */\n \tif (read_in_full(ip_out, packname, len) == len && packname[len-1] == '\\n') {\n \t\tconst char *name;\n+\n+\t\tif (is_well_formed)\n+\t\t\t*is_well_formed = 1;\n \t\tpackname[len-1] = 0;\n \t\tif (skip_prefix(packname, \"keep\\t\", &name))\n \t\t\treturn xstrfmt(\"%s/pack/pack-%s.keep\",\n \t\t\t\t get_object_directory(), name);\n+\t\treturn NULL;\n \t}\n+\tif (is_well_formed)\n+\t\t*is_well_formed = 0;\n \treturn NULL;\n }\n \ndiff --git a/pack.h b/pack.h\nindex 9fc0945ac9..09cffec395 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -85,7 +85,7 @@ int verify_pack_index(struct packed_git *);\n int verify_pack(struct repository *, struct packed_git *, verify_fn fn, struct progress *, uint32_t);\n off_t write_pack_header(struct hashfile *f, uint32_t);\n void fixup_pack_header_footer(int, unsigned char *, const char *, uint32_t, unsigned char *, off_t);\n-char *index_pack_lockfile(int fd);\n+char *index_pack_lockfile(int fd, int *is_well_formed);\n \n /*\n * The \"hdr\" output buffer should be at least this big, which will handle sizes\ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 7d5b17909b..8b8fb43dbc 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -936,6 +936,53 @@ test_expect_success 'packfile-uri with transfer.fsckobjects fails on bad object'\n \ttest_i18ngrep \"invalid author/committer line - missing email\" error\n '\n \n+test_expect_success 'packfile-uri with transfer.fsckobjects succeeds when .gitmodules is separate from tree' '\n+\tP=\"$HTTPD_DOCUMENT_ROOT_PATH/http_parent\" &&\n+\trm -rf \"$P\" http_child &&\n+\n+\tgit init \"$P\" &&\n+\tgit -C \"$P\" config \"uploadpack.allowsidebandall\" \"true\" &&\n+\n+\techo \"[submodule libfoo]\" >\"$P/.gitmodules\" &&\n+\techo \"path = include/foo\" >>\"$P/.gitmodules\" &&\n+\techo \"url = git://example.com/git/lib.git\" >>\"$P/.gitmodules\" &&\n+\tgit -C \"$P\" add .gitmodules &&\n+\tgit -C \"$P\" commit -m x &&\n+\n+\tconfigure_exclusion \"$P\" .gitmodules >h &&\n+\n+\tsane_unset GIT_TEST_SIDEBAND_ALL &&\n+\tgit -c protocol.version=2 -c transfer.fsckobjects=1 \\\n+\t\t-c fetch.uriprotocols=http,https \\\n+\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n+\n+\t# Ensure that there are exactly 4 files (2 .pack and 2 .idx).\n+\tls http_child/.git/objects/pack/* >filelist &&\n+\ttest_line_count = 4 filelist\n+'\n+\n+test_expect_success 'packfile-uri with transfer.fsckobjects fails when .gitmodules separate from tree is invalid' '\n+\tP=\"$HTTPD_DOCUMENT_ROOT_PATH/http_parent\" &&\n+\trm -rf \"$P\" http_child err &&\n+\n+\tgit init \"$P\" &&\n+\tgit -C \"$P\" config \"uploadpack.allowsidebandall\" \"true\" &&\n+\n+\techo \"[submodule \\\"..\\\"]\" >\"$P/.gitmodules\" &&\n+\techo \"path = include/foo\" >>\"$P/.gitmodules\" &&\n+\techo \"url = git://example.com/git/lib.git\" >>\"$P/.gitmodules\" &&\n+\tgit -C \"$P\" add .gitmodules &&\n+\tgit -C \"$P\" commit -m x &&\n+\n+\tconfigure_exclusion \"$P\" .gitmodules >h &&\n+\n+\tsane_unset GIT_TEST_SIDEBAND_ALL &&\n+\ttest_must_fail git -c protocol.version=2 -c transfer.fsckobjects=1 \\\n+\t\t-c fetch.uriprotocols=http,https \\\n+\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child 2>err &&\n+\ttest_i18ngrep \"disallowed submodule name\" err\n+'\n+\n # DO NOT add non-httpd-specific tests here, because the last part of this\n # test script is only executed when httpd is available and enabled.\n \n-- \n2.30.0.280.ga3ce27912f-goog\n\n"},{"id":"415133","messageId":"xmqqa6sy26gq.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"cover.1611455251.git.jonathantanmy@google.com","subject":"Re: [PATCH 0/4] Check .gitmodules when using packfile URIs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-24T06:29:25Z","receivedAt":"2021-01-24T06:30:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> As part of this, index-pack has to output (1) the hash that goes into\n> the name of the .pack/.idx file and (2) the hashes of all dangling\n> .gitmodules. I just had (2) come after (1). If anyone has a better idea,\n> I'm interested.\n\nI have this feeling that the \"blobs that need to be validated across\npacks\" will *not* be the last enhancement we'd need to make to the\noutput from index-pack to allow richer communication between it and\nits invoker. While there is no reason to change how the first line\nof the output looks like, we'd probably want to make sure that the\nfuture versions of Git can easily tell \"list of blobs that require\nfurther validation\" from other additional information.\n\nI am not comfortable to recommend \"ok, then let's add a delimiter\nline '---\\n' if/when we need to have something after the list of\nblobs and append more stuff in future versions of Git\", because we\nmay find need to emit new kinds of info before the list of blobs\nthat needs further validation, for example, in future versions of\nGit.\n\nHaving said all that, the internal communication between the\nindex-pack and its caller do not need as much care about\ncompatibility across versions as output visible to end-users, so\nwhen a future version of Git needs to send different kinds of\ninformation in different order from what you created here, we can do\nso pretty much freely, I would guess.\n\nThanks.\n"},{"id":"415137","messageId":"xmqqy2gizs1s.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"2d6d8c2671fe424c752994dcb5277d4d923e17a0.1611455251.git.jonathantanmy@google.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-24T07:56:47Z","receivedAt":"2021-01-24T07:58:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\n> index 7d5b17909b..8b8fb43dbc 100755\n> ...\n> +\tsane_unset GIT_TEST_SIDEBAND_ALL &&\n> +\tgit -c protocol.version=2 -c transfer.fsckobjects=1 \\\n> +\t\t-c fetch.uriprotocols=http,https \\\n> +\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n> +\n> +\t# Ensure that there are exactly 4 files (2 .pack and 2 .idx).\n\nEhh, please don't. We may add multi-pack-index there, or perhaps\nreverse index files in the future. If you care about having two\npacks logically because you are exercising the out-of-band\nprepackaged packfile plus the dynamic transfer, make sure you have\ntwo packs (and probably the idx files that go with them). Don't\nassume there will be one .idx each for them *AND* nothing else\nthere.\n\n> +\tls http_child/.git/objects/pack/* >filelist &&\n> +\ttest_line_count = 4 filelist\n> +'\n\nIOW,\n\n\td=http_child/.git/objects/pack/\n\tls \"$d\"/*.pack \"$d\"/*.idx >filelist &&\n\ttest_line_count = 4 filelist\n\nor something like that.\n"},{"id":"415153","messageId":"87im7m7drv.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"3032117ee747e72f6208eab7f960737d25f8a82a.1611455251.git.jonathantanmy@google.com","subject":"Re: [PATCH 2/4] http-fetch: allow custom index-pack args","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-01-24T11:52:36Z","receivedAt":"2021-01-24T11:54:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sun, Jan 24 2021, Jonathan Tan wrote:\n\n> --packfile=::\n> -\tInstead of a commit id on the command line (which is not expected in\n> +\tFor internal use only. Instead of a commit id on the command line (which is not expected in\n\nLeaves the rest at ~79 and this long line at ~100. Perhaps a follow-up\nchange to re-word-wrap would be in order?\n"},{"id":"415154","messageId":"87ft2q7ck4.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"2d6d8c2671fe424c752994dcb5277d4d923e17a0.1611455251.git.jonathantanmy@google.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-01-24T12:18:51Z","receivedAt":"2021-01-24T12:19:54Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sun, Jan 24 2021, Jonathan Tan wrote:\n\n> +void register_found_gitmodules(const struct object_id *oid)\n> +{\n> +\toidset_insert(&gitmodules_found, oid);\n> +}\n> +\n\nIn fsck.c we only use this variable to insert into it, or in fsck_blob()\nto do the actual check, but then we either abort early if we've found\nit, or right after that:\n\n if (object_on_skiplist(options, oid))\n return 0;\n\nSo (along with comments I have below...) you could just use the existing\n\"skiplist\" option instead, no?\n\n> int fsck_finish(struct fsck_options *options)\n> {\n> \tint ret = 0;\n> @@ -1262,10 +1267,13 @@ int fsck_finish(struct fsck_options *options)\n> \t\tif (!buf) {\n> \t\t\tif (is_promisor_object(oid))\n> \t\t\t\tcontinue;\n> -\t\t\tret |= report(options,\n> -\t\t\t\t oid, OBJ_BLOB,\n> -\t\t\t\t FSCK_MSG_GITMODULES_MISSING,\n> -\t\t\t\t \"unable to read .gitmodules blob\");\n> +\t\t\tif (options->print_dangling_gitmodules)\n> +\t\t\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n> +\t\t\telse\n> +\t\t\t\tret |= report(options,\n> +\t\t\t\t\t oid, OBJ_BLOB,\n> +\t\t\t\t\t FSCK_MSG_GITMODULES_MISSING,\n> +\t\t\t\t\t \"unable to read .gitmodules blob\");\n> \t\t\tcontinue;\n> \t\t}\n> \n> diff --git a/fsck.h b/fsck.h\n> index 69cf715e79..4b8cf03445 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -41,6 +41,12 @@ struct fsck_options {\n> \tint *msg_type;\n> \tstruct oidset skiplist;\n> \tkh_oid_map_t *object_names;\n> +\n> +\t/*\n> +\t * If 1, print the hashes of missing .gitmodules blobs instead of\n> +\t * considering them to be errors.\n> +\t */\n> +\tunsigned print_dangling_gitmodules:1;\n> };\n> \n> #define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n> @@ -62,6 +68,8 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n> int fsck_object(struct object *obj, void *data, unsigned long size,\n> \tstruct fsck_options *options);\n> \n> +void register_found_gitmodules(const struct object_id *oid);\n> +\n> /*\n> * Some fsck checks are context-dependent, and may end up queued; run this\n> * after completing all fsck_object() calls in order to resolve any remaining\n\n\nThis whole thing seems just like the bad path I took in earlier rounds\nof my in-flight mktag series. You don't need this new custom API. You\njust setup an error handler for your fsck which ignores / prints / logs\n/ whatever the OIDs you want if you get a FSCK_MSG_GITMODULES_MISSING\nerror, which you then \"return 0\" on.\n\nIf you don't have FSCK_MSG_GITMODULES_MISSING punt and call\nfsck_error_function().\n"},{"id":"415155","messageId":"87czxu7c15.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"2d6d8c2671fe424c752994dcb5277d4d923e17a0.1611455251.git.jonathantanmy@google.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-01-24T12:30:14Z","receivedAt":"2021-01-24T12:31:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sun, Jan 24 2021, Jonathan Tan wrote:\n> --fsck-objects::\n> -\tDie if the pack contains broken objects. For internal use only.\n> +\tFor internal use only.\n> ++\n> +Die if the pack contains broken objects. If the pack contains a tree\n> +pointing to a .gitmodules blob that does not exist, prints the hash of\n> +that blob (for the caller to check) after the hash that goes into the\n> +name of the pack/idx file (see \"Notes\").\n\n[I should have waited a bit and sent one E-Mail]\n\nIs this really generally usable as an IPC mechanism, what if we need\nanother set of OIDs we care about? Shouldn't it at least be hidden\nbehind some option so you don't get a deluge of output from index-pack\nif you're not in this packfile-uri mode?\n\nBut, along with my other E-Mail...\n\n> [...]\n> +static void parse_gitmodules_oids(int fd, struct oidset *gitmodules_oids)\n> +{\n> +\tint len = the_hash_algo->hexsz + 1; /* hash + NL */\n> +\n> +\tdo {\n> +\t\tchar hex_hash[GIT_MAX_HEXSZ + 1];\n> +\t\tint read_len = read_in_full(fd, hex_hash, len);\n> +\t\tstruct object_id oid;\n> +\t\tconst char *end;\n> +\n> +\t\tif (!read_len)\n> +\t\t\treturn;\n> +\t\tif (read_len != len)\n> +\t\t\tdie(\"invalid length read %d\", read_len);\n> +\t\tif (parse_oid_hex(hex_hash, &oid, &end) || *end != '\\n')\n> +\t\t\tdie(\"invalid hash\");\n> +\t\toidset_insert(gitmodules_oids, &oid);\n> +\t} while (1);\n> +}\n> +\n\nDoesn't this IPC mechanism already exist in the form of fsck.skipList?\nSee my 1f3299fda9 (fsck: make fsck_config() re-usable, 2021-01-05) on\n\"next\". I.e. as noted in my just-sent-E-Mail you could probably just\nre-use skiplist as-is.\n\nOr if not it seems to me that this whole IPC mechanism would be better\ndone with a tempfile and passing it along like we already pass the\nfsck.skipList between these processes.\n\nI doubt it's going to be large enough to matter, we could just put it in\n.git/ somewhere, like we put gc.log etc (but created with a mktemp()\nname...).\n\nOr if we want to keep the \"print | process\" model we can refactor\nthe existing fsck IPC noted in 1f3299fda9 a bit, so e.g. you pass some\nversion of \"lines prefixed with \"fsck-skiplist: \" go into list xyz via a\ncommand-line option. And then existing option(s) and your potential new\nlist (which as noted, I think is probably redundant to the skiplist) can\nuse it.\n\n\n\n"},{"id":"415276","messageId":"xmqqmtwwwjbw.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"xmqqy2gizs1s.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-01-26T01:57:55Z","receivedAt":"2021-01-26T05:06:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Jonathan Tan writes:\n>\n>> diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\n>> index 7d5b17909b..8b8fb43dbc 100755\n>> ...\n>> +\tsane_unset GIT_TEST_SIDEBAND_ALL &&\n>> +\tgit -c protocol.version=2 -c transfer.fsckobjects=1 \\\n>> +\t\t-c fetch.uriprotocols=http,https \\\n>> +\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n>> +\n>> +\t# Ensure that there are exactly 4 files (2 .pack and 2 .idx).\n>\n> Ehh, please don't. We may add multi-pack-index there, or perhaps\n> reverse index files in the future. If you care about having two\n> packs logically because you are exercising the out-of-band\n> prepackaged packfile plus the dynamic transfer, make sure you have\n> two packs (and probably the idx files that go with them). Don't\n> assume there will be one .idx each for them *AND* nothing else\n> there.\n>\n>> +\tls http_child/.git/objects/pack/* >filelist &&\n>> +\ttest_line_count = 4 filelist\n>> +'\n>\n> IOW,\n>\n> \td=http_child/.git/objects/pack/\n> \tls \"$d\"/*.pack \"$d\"/*.idx >filelist &&\n> \ttest_line_count = 4 filelist\n>\n> or something like that.\n\nFYI, I have the following queued to make the tip of 'seen' pass the\ntests.\n\n---- >8 -------- >8 -------- >8 -------- >8 -------- >8 -------- >8 ----\nFrom: Junio C Hamano \nDate: Mon, 25 Jan 2021 17:27:10 -0800\nSubject: [PATCH] SQUASH??? test fix\n\n---\n t/t5702-protocol-v2.sh | 15 +++++++++------\n 1 file changed, 9 insertions(+), 6 deletions(-)\n\ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 8b8fb43dbc..b1bc73a9a9 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -847,8 +847,9 @@ test_expect_success 'part of packfile response provided as URI' '\n \ttest -f hfound &&\n \ttest -f h2found &&\n \n-\t# Ensure that there are exactly 6 files (3 .pack and 3 .idx).\n-\tls http_child/.git/objects/pack/* >filelist &&\n+\t# Ensure that there are exactly 3 packfiles with associated .idx\n+\tls http_child/.git/objects/pack/*.pack \\\n+\t http_child/.git/objects/pack/*.idx >filelist &&\n \ttest_line_count = 6 filelist\n '\n \n@@ -901,8 +902,9 @@ test_expect_success 'packfile-uri with transfer.fsckobjects' '\n \t\t-c fetch.uriprotocols=http,https \\\n \t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n \n-\t# Ensure that there are exactly 4 files (2 .pack and 2 .idx).\n-\tls http_child/.git/objects/pack/* >filelist &&\n+\t# Ensure that there are exactly 2 packfiles with associated .idx\n+\tls http_child/.git/objects/pack/*.pack \\\n+\t http_child/.git/objects/pack/*.idx >filelist &&\n \ttest_line_count = 4 filelist\n '\n \n@@ -956,8 +958,9 @@ test_expect_success 'packfile-uri with transfer.fsckobjects succeeds when .gitmo\n \t\t-c fetch.uriprotocols=http,https \\\n \t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n \n-\t# Ensure that there are exactly 4 files (2 .pack and 2 .idx).\n-\tls http_child/.git/objects/pack/* >filelist &&\n+\t# Ensure that there are exactly 2 packfiles with associated .idx\n+\tls http_child/.git/objects/pack/*.pack \\\n+\t http_child/.git/objects/pack/*.idx >filelist &&\n \ttest_line_count = 4 filelist\n '\n \n-- \n2.30.0-509-gbbf2750a06\n\n"},{"id":"415471","messageId":"20210128003255.3872508-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"87im7m7drv.fsf@evledraar.gmail.com","subject":"Re: [PATCH 2/4] http-fetch: allow custom index-pack args","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-28T00:32:54Z","receivedAt":"2021-01-28T00:33:58Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> On Sun, Jan 24 2021, Jonathan Tan wrote:\n> \n> > --packfile=::\n> > -\tInstead of a commit id on the command line (which is not expected in\n> > +\tFor internal use only. Instead of a commit id on the command line (which is not expected in\n> \n> Leaves the rest at ~79 and this long line at ~100. Perhaps a follow-up\n> change to re-word-wrap would be in order?\n\nHmm...I'll split that onto two lines then. I don't think it's worth the\nextra commit in history to have it exactly wrapped right, so I'll forgo\nthe follow-up change for now.\n"},{"id":"415472","messageId":"20210128003536.3874866-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqqa6sy26gq.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 0/4] Check .gitmodules when using packfile URIs","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-28T00:35:36Z","receivedAt":"2021-01-28T00:37:13Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Jonathan Tan writes:\n> \n> > As part of this, index-pack has to output (1) the hash that goes into\n> > the name of the .pack/.idx file and (2) the hashes of all dangling\n> > .gitmodules. I just had (2) come after (1). If anyone has a better idea,\n> > I'm interested.\n> \n> I have this feeling that the \"blobs that need to be validated across\n> packs\" will *not* be the last enhancement we'd need to make to the\n> output from index-pack to allow richer communication between it and\n> its invoker. While there is no reason to change how the first line\n> of the output looks like, we'd probably want to make sure that the\n> future versions of Git can easily tell \"list of blobs that require\n> further validation\" from other additional information.\n> \n> I am not comfortable to recommend \"ok, then let's add a delimiter\n> line '---\\n' if/when we need to have something after the list of\n> blobs and append more stuff in future versions of Git\", because we\n> may find need to emit new kinds of info before the list of blobs\n> that needs further validation, for example, in future versions of\n> Git.\n> \n> Having said all that, the internal communication between the\n> index-pack and its caller do not need as much care about\n> compatibility across versions as output visible to end-users, so\n> when a future version of Git needs to send different kinds of\n> information in different order from what you created here, we can do\n> so pretty much freely, I would guess.\n\nYeah, that's what I thought too - since this is an internal interface,\nwe can evolve them in lockstep. If we're really worried about the Git\nbinaries (on a user's system) getting out of sync, we could just make\nsure that subsequent updates to this protocol are\nnon-backwards-compatible (e.g. have index-pack emit \"foo \", where\n\"foo\" is a string that describes the new check, so that current\nfetch-pack will reject \"foo\" since it is not a hash).\n"},{"id":"415473","messageId":"20210128010320.3880242-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"87ft2q7ck4.fsf@evledraar.gmail.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-28T01:03:20Z","receivedAt":"2021-01-28T01:07:27Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> On Sun, Jan 24 2021, Jonathan Tan wrote:\n> \n> > +void register_found_gitmodules(const struct object_id *oid)\n> > +{\n> > +\toidset_insert(&gitmodules_found, oid);\n> > +}\n> > +\n> \n> In fsck.c we only use this variable to insert into it, or in fsck_blob()\n> to do the actual check, but then we either abort early if we've found\n> it, or right after that:\n\nBy \"this variable\", do you mean gitmodules_found? fsck_finish() consumes\nit.\n\n> if (object_on_skiplist(options, oid))\n> return 0;\n> \n> So (along with comments I have below...) you could just use the existing\n> \"skiplist\" option instead, no?\n\nI don't understand this part (in particular, the part you quoted). About\n\"skiplist\", I'll reply to your other email [1] which has more details.\n\n[1] https://lore.kernel.org/git/87czxu7c15.fsf@evledraar.gmail.com/\n\n> This whole thing seems just like the bad path I took in earlier rounds\n> of my in-flight mktag series. You don't need this new custom API. You\n> just setup an error handler for your fsck which ignores / prints / logs\n> / whatever the OIDs you want if you get a FSCK_MSG_GITMODULES_MISSING\n> error, which you then \"return 0\" on.\n> \n> If you don't have FSCK_MSG_GITMODULES_MISSING punt and call\n> fsck_error_function().\n\nI tried that first, and the issue is that IDs like\nFSCK_MSG_GITMODULES_MISSING are internal to fsck.c. As for whether we\nshould start exposing the IDs publicly, I think we should wait until a\nfew new cases like this come up, so that we more fully understand the\nrequirements first.\n"},{"id":"415474","messageId":"20210128010426.3882500-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqqmtwwwjbw.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-28T01:04:26Z","receivedAt":"2021-01-28T01:08:00Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> > Ehh, please don't. We may add multi-pack-index there, or perhaps\n> > reverse index files in the future. If you care about having two\n> > packs logically because you are exercising the out-of-band\n> > prepackaged packfile plus the dynamic transfer, make sure you have\n> > two packs (and probably the idx files that go with them). Don't\n> > assume there will be one .idx each for them *AND* nothing else\n> > there.\n> >\n> >> +\tls http_child/.git/objects/pack/* >filelist &&\n> >> +\ttest_line_count = 4 filelist\n> >> +'\n> >\n> > IOW,\n> >\n> > \td=http_child/.git/objects/pack/\n> > \tls \"$d\"/*.pack \"$d\"/*.idx >filelist &&\n> > \ttest_line_count = 4 filelist\n> >\n> > or something like that.\n> \n> FYI, I have the following queued to make the tip of 'seen' pass the\n> tests.\n\n[snip]\n\nOK - I'll include these changes in the next version.\n"},{"id":"415475","messageId":"20210128011525.3886106-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"87czxu7c15.fsf@evledraar.gmail.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-01-28T01:15:25Z","receivedAt":"2021-01-28T01:18:18Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> On Sun, Jan 24 2021, Jonathan Tan wrote:\n> > --fsck-objects::\n> > -\tDie if the pack contains broken objects. For internal use only.\n> > +\tFor internal use only.\n> > ++\n> > +Die if the pack contains broken objects. If the pack contains a tree\n> > +pointing to a .gitmodules blob that does not exist, prints the hash of\n> > +that blob (for the caller to check) after the hash that goes into the\n> > +name of the pack/idx file (see \"Notes\").\n> \n> [I should have waited a bit and sent one E-Mail]\n> \n> Is this really generally usable as an IPC mechanism, what if we need\n> another set of OIDs we care about? Shouldn't it at least be hidden\n> behind some option so you don't get a deluge of output from index-pack\n> if you're not in this packfile-uri mode?\n\n--fsck-objects is only for internal use, and it's only used by\nfetch-pack.c. So its only consumer does want the output.\n\nJunio also mentioned the possibility of another set of OIDs, and I\nreplied [1].\n\n[1] https://lore.kernel.org/git/20210128003536.3874866-1-jonathantanmy@google.com/\n\n> But, along with my other E-Mail...\n> \n> > [...]\n> > +static void parse_gitmodules_oids(int fd, struct oidset *gitmodules_oids)\n> > +{\n> > +\tint len = the_hash_algo->hexsz + 1; /* hash + NL */\n> > +\n> > +\tdo {\n> > +\t\tchar hex_hash[GIT_MAX_HEXSZ + 1];\n> > +\t\tint read_len = read_in_full(fd, hex_hash, len);\n> > +\t\tstruct object_id oid;\n> > +\t\tconst char *end;\n> > +\n> > +\t\tif (!read_len)\n> > +\t\t\treturn;\n> > +\t\tif (read_len != len)\n> > +\t\t\tdie(\"invalid length read %d\", read_len);\n> > +\t\tif (parse_oid_hex(hex_hash, &oid, &end) || *end != '\\n')\n> > +\t\t\tdie(\"invalid hash\");\n> > +\t\toidset_insert(gitmodules_oids, &oid);\n> > +\t} while (1);\n> > +}\n> > +\n> \n> Doesn't this IPC mechanism already exist in the form of fsck.skipList?\n> See my 1f3299fda9 (fsck: make fsck_config() re-usable, 2021-01-05) on\n> \"next\". I.e. as noted in my just-sent-E-Mail you could probably just\n> re-use skiplist as-is.\n\nI'm not sure how fsck.skipList could be used here. Before running\nfsck_finish() for the first time, we don't know which .gitmodules are\nmissing and which are not. And when running fsck_finish() for the second\ntime, we definitely do not want to skip any blobs.\n\n> Or if not it seems to me that this whole IPC mechanism would be better\n> done with a tempfile and passing it along like we already pass the\n> fsck.skipList between these processes.\n> \n> I doubt it's going to be large enough to matter, we could just put it in\n> .git/ somewhere, like we put gc.log etc (but created with a mktemp()\n> name...).\n> \n> Or if we want to keep the \"print | process\" model we can refactor\n> the existing fsck IPC noted in 1f3299fda9 a bit, so e.g. you pass some\n> version of \"lines prefixed with \"fsck-skiplist: \" go into list xyz via a\n> command-line option. And then existing option(s) and your potential new\n> list (which as noted, I think is probably redundant to the skiplist) can\n> use it.\n\nI think using stdout is superior to using a tempfile - we don't have to\nworry about interrupted invocations, for example.\n\nWhat do you mean by \"the existing fsck IPC noted in 1f3299fda9\"? If you\nmean the ability to pass a list of OIDs, for example using \"-c\nfsck.skipList=filename.txt\", I'm not sure that it solves anything.\nFirstly, I don't think that the skipList is useful here (as I said\nearlier). And secondly, I don't think that OID input is the issue -\nright now, the design is a process (index-pack, calling fsck_finish())\nwriting to its output which is then picked up by the calling process\n(fetch-pack). We are not sending the dangling .gitmodules through stdin\nanywhere.\n"},{"id":"417106","messageId":"YCwvycbylJ9O4qx8@google.com","threadId":"54994","inReplyTo":"3032117ee747e72f6208eab7f960737d25f8a82a.1611455251.git.jonathantanmy@google.com","subject":"Re: [PATCH 2/4] http-fetch: allow custom index-pack args","fromName":"Josh Steadmon","fromEmail":"steadmon@google.com","sentAt":"2021-02-16T20:49:13Z","receivedAt":"2021-02-16T20:50:03Z","isPatch":true,"sender":{"key":"steadmon@google.com","avatar":"https://avatars.githubusercontent.com/u/2654920?v=4"},"body":"On 2021.01.23 18:34, Jonathan Tan wrote:\n> This is the next step in teaching fetch-pack to pass its index-pack\n> arguments when processing packfiles referenced by URIs.\n> \n> The \"--keep\" in fetch-pack.c will be replaced with a full message in a\n> subsequent commit.\n> \n> Signed-off-by: Jonathan Tan \n> ---\n> Documentation/git-http-fetch.txt | 9 ++++++--\n> fetch-pack.c | 1 +\n> http-fetch.c | 35 +++++++++++++++++++++++++++-----\n> t/t5550-http-fetch-dumb.sh | 3 ++-\n> 4 files changed, 40 insertions(+), 8 deletions(-)\n> \n> diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt\n> index 4deb4893f5..aa171088e8 100644\n> --- a/Documentation/git-http-fetch.txt\n> +++ b/Documentation/git-http-fetch.txt\n> @@ -41,11 +41,16 @@ commit-id::\n> \t\t['\\t']\n> \n> --packfile=::\n> -\tInstead of a commit id on the command line (which is not expected in\n> +\tFor internal use only. Instead of a commit id on the command line (which is not expected in\n> \tthis case), 'git http-fetch' fetches the packfile directly at the given\n> \tURL and uses index-pack to generate corresponding .idx and .keep files.\n> \tThe hash is used to determine the name of the temporary file and is\n> -\tarbitrary. The output of index-pack is printed to stdout.\n> +\tarbitrary. The output of index-pack is printed to stdout. Requires\n> +\t--index-pack-args.\n> +\n> +--index-pack-args=::\n> +\tFor internal use only. The command to run on the contents of the\n> +\tdownloaded pack. Arguments are URL-encoded separated by spaces.\n\nI'm a bit skeptical of using URL encoding to work around embedded\nspaces. I believe in Emily's config-based hooks series, she wrote an\nargument parser to pull repeated arguments into a strvec, could you do\nsomething like that here?\n\nI'm sympathetic to the idea that since this is an internal-only flag, we\ncan be a bit weird with the argument format, though.\n\n> --recover::\n> \tVerify that everything reachable from target is fetched. Used after\n> diff --git a/fetch-pack.c b/fetch-pack.c\n> index 876f90c759..274ae602f7 100644\n> --- a/fetch-pack.c\n> +++ b/fetch-pack.c\n> @@ -1645,6 +1645,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n> \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n> \t\t\t (int) the_hash_algo->hexsz,\n> \t\t\t packfile_uris.items[i].string);\n> +\t\tstrvec_push(&cmd.args, \"--index-pack-args=index-pack --stdin --keep\");\n> \t\tstrvec_push(&cmd.args, uri);\n> \t\tcmd.git_cmd = 1;\n> \t\tcmd.no_stdin = 1;\n> diff --git a/http-fetch.c b/http-fetch.c\n> index 2d1d9d054f..12feb84e71 100644\n> --- a/http-fetch.c\n> +++ b/http-fetch.c\n> @@ -3,6 +3,7 @@\n> #include \"exec-cmd.h\"\n> #include \"http.h\"\n> #include \"walker.h\"\n> +#include \"strvec.h\"\n> \n> static const char http_fetch_usage[] = \"git http-fetch \"\n> \"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin | --packfile=hash | commit-id] url\";\n> @@ -43,11 +44,9 @@ static int fetch_using_walker(const char *raw_url, int get_verbosely,\n> \treturn rc;\n> }\n> \n> -static const char *index_pack_args[] =\n> -\t{\"index-pack\", \"--stdin\", \"--keep\", NULL};\n> -\n> static void fetch_single_packfile(struct object_id *packfile_hash,\n> -\t\t\t\t const char *url) {\n> +\t\t\t\t const char *url,\n> +\t\t\t\t const char **index_pack_args) {\n> \tstruct http_pack_request *preq;\n> \tstruct slot_results results;\n> \tint ret;\n> @@ -90,6 +89,7 @@ int cmd_main(int argc, const char **argv)\n> \tint packfile = 0;\n> \tint nongit;\n> \tstruct object_id packfile_hash;\n> +\tconst char *index_pack_args = NULL;\n> \n> \tsetup_git_directory_gently(&nongit);\n> \n> @@ -116,6 +116,8 @@ int cmd_main(int argc, const char **argv)\n> \t\t\tpackfile = 1;\n> \t\t\tif (parse_oid_hex(p, &packfile_hash, &end) || *end)\n> \t\t\t\tdie(_(\"argument to --packfile must be a valid hash (got '%s')\"), p);\n> +\t\t} else if (skip_prefix(argv[arg], \"--index-pack-args=\", &p)) {\n> +\t\t\tindex_pack_args = p;\n> \t\t}\n> \t\targ++;\n> \t}\n> @@ -128,10 +130,33 @@ int cmd_main(int argc, const char **argv)\n> \tgit_config(git_default_config, NULL);\n> \n> \tif (packfile) {\n> -\t\tfetch_single_packfile(&packfile_hash, argv[arg]);\n> +\t\tstruct strvec encoded = STRVEC_INIT;\n> +\t\tchar **raw;\n> +\t\tint i;\n> +\n> +\t\tif (!index_pack_args)\n> +\t\t\tdie(_(\"--packfile requires --index-pack-args\"));\n> +\n> +\t\tstrvec_split(&encoded, index_pack_args);\n> +\n> +\t\tCALLOC_ARRAY(raw, encoded.nr + 1);\n> +\t\tfor (i = 0; i < encoded.nr; i++)\n> +\t\t\traw[i] = url_percent_decode(encoded.v[i]);\n> +\n> +\t\tfetch_single_packfile(&packfile_hash, argv[arg],\n> +\t\t\t\t (const char **) raw);\n> +\n> +\t\tfor (i = 0; i < encoded.nr; i++)\n> +\t\t\tfree(raw[i]);\n> +\t\tfree(raw);\n> +\t\tstrvec_clear(&encoded);\n> +\n> \t\treturn 0;\n> \t}\n> \n> +\tif (index_pack_args)\n> +\t\tdie(_(\"--index-pack-args can only be used with --packfile\"));\n> +\n> \tif (commits_on_stdin) {\n> \t\tcommits = walker_targets_stdin(&commit_id, &write_ref);\n> \t} else {\n> diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh\n> index 483578b2d7..af90e7efed 100755\n> --- a/t/t5550-http-fetch-dumb.sh\n> +++ b/t/t5550-http-fetch-dumb.sh\n> @@ -224,7 +224,8 @@ test_expect_success 'http-fetch --packfile' '\n> \n> \tgit init packfileclient &&\n> \tp=$(cd \"$HTTPD_DOCUMENT_ROOT_PATH\"/repo_pack.git && ls objects/pack/pack-*.pack) &&\n> -\tgit -C packfileclient http-fetch --packfile=$ARBITRARY \"$HTTPD_URL\"/dumb/repo_pack.git/$p >out &&\n> +\tgit -C packfileclient http-fetch --packfile=$ARBITRARY \\\n> +\t\t--index-pack-args=\"index-pack --stdin --keep\" \"$HTTPD_URL\"/dumb/repo_pack.git/$p >out &&\n> \n> \tgrep \"^keep.[0-9a-f]\\{16,\\}$\" out &&\n> \tcut -c6- out >packhash &&\n> -- \n> 2.30.0.280.ga3ce27912f-goog\n> \n"},{"id":"417119","messageId":"xmqqpn0zoc2l.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"YCwvycbylJ9O4qx8@google.com","subject":"Re: [PATCH 2/4] http-fetch: allow custom index-pack args","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-16T22:57:38Z","receivedAt":"2021-02-16T22:58:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Josh Steadmon writes:\n\n>> +--index-pack-args=::\n>> +\tFor internal use only. The command to run on the contents of the\n>> +\tdownloaded pack. Arguments are URL-encoded separated by spaces.\n>\n> I'm a bit skeptical of using URL encoding to work around embedded\n> spaces. I believe in Emily's config-based hooks series, she wrote an\n> argument parser to pull repeated arguments into a strvec, could you do\n> something like that here?\n>\n> I'm sympathetic to the idea that since this is an internal-only flag, we\n> can be a bit weird with the argument format, though.\n\nWe tend to prefer quote.c::sq_quote*() suite of quoting; does this\ncodepath have very different constraints that require different\nencoding?\n\nThanks.\n"},{"id":"417135","messageId":"87blcja2ha.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"20210128010320.3880242-1-jonathantanmy@google.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T01:48:33Z","receivedAt":"2021-02-17T01:49:34Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 28 2021, Jonathan Tan wrote:\n\nSorry I managed to miss this at the time. Hopefully a late reply is\nbetter than never.\n\n>> On Sun, Jan 24 2021, Jonathan Tan wrote:\n>> \n>> > +void register_found_gitmodules(const struct object_id *oid)\n>> > +{\n>> > +\toidset_insert(&gitmodules_found, oid);\n>> > +}\n>> > +\n>> \n>> In fsck.c we only use this variable to insert into it, or in fsck_blob()\n>> to do the actual check, but then we either abort early if we've found\n>> it, or right after that:\n>\n> By \"this variable\", do you mean gitmodules_found? fsck_finish() consumes\n> it.\n\nYes, consumes it to emit errors with report(), no?\n\n>> if (object_on_skiplist(options, oid))\n>> return 0;\n>> \n>> So (along with comments I have below...) you could just use the existing\n>> \"skiplist\" option instead, no?\n>\n> I don't understand this part (in particular, the part you quoted). About\n> \"skiplist\", I'll reply to your other email [1] which has more details.\n>\n> [1] https://lore.kernel.org/git/87czxu7c15.fsf@evledraar.gmail.com/\n\n*nod*\n\n>> This whole thing seems just like the bad path I took in earlier rounds\n>> of my in-flight mktag series. You don't need this new custom API. You\n>> just setup an error handler for your fsck which ignores / prints / logs\n>> / whatever the OIDs you want if you get a FSCK_MSG_GITMODULES_MISSING\n>> error, which you then \"return 0\" on.\n>> \n>> If you don't have FSCK_MSG_GITMODULES_MISSING punt and call\n>> fsck_error_function().\n>\n> I tried that first, and the issue is that IDs like\n> FSCK_MSG_GITMODULES_MISSING are internal to fsck.c. As for whether we\n> should start exposing the IDs publicly, I think we should wait until a\n> few new cases like this come up, so that we more fully understand the\n> requirements first.\n\nThe requirement is that you want the objects ids we'd otherwise error\nabout in fsck_finish(). Yeah we don't pass the \"fsck_msg_id\" down in the\n\"report()\" function, but you can reliably strstr() it out of the\nmessage. We document & hard rely on that already, since it's also a\nconfig key.\n\nBut yeah, we could just change the report function to pass down the id\nand move the relevant macros from fsck.c to fsck.h. I think that would\nbe a smaller change conceptually than a special-case flag in\nfsck_options for something we could otherwise do with the error\nreporting.\n\n"},{"id":"417138","messageId":"878s7na1h6.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"20210128011525.3886106-1-jonathantanmy@google.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T02:10:13Z","receivedAt":"2021-02-17T02:11:32Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 28 2021, Jonathan Tan wrote:\n\n>> On Sun, Jan 24 2021, Jonathan Tan wrote:\n>> > --fsck-objects::\n>> > -\tDie if the pack contains broken objects. For internal use only.\n>> > +\tFor internal use only.\n>> > ++\n>> > +Die if the pack contains broken objects. If the pack contains a tree\n>> > +pointing to a .gitmodules blob that does not exist, prints the hash of\n>> > +that blob (for the caller to check) after the hash that goes into the\n>> > +name of the pack/idx file (see \"Notes\").\n>> \n>> [I should have waited a bit and sent one E-Mail]\n>> \n>> Is this really generally usable as an IPC mechanism, what if we need\n>> another set of OIDs we care about? Shouldn't it at least be hidden\n>> behind some option so you don't get a deluge of output from index-pack\n>> if you're not in this packfile-uri mode?\n>\n> --fsck-objects is only for internal use, and it's only used by\n> fetch-pack.c. So its only consumer does want the output.\n>\n> Junio also mentioned the possibility of another set of OIDs, and I\n> replied [1].\n>\n> [1] https://lore.kernel.org/git/20210128003536.3874866-1-jonathantanmy@google.com/\n>\n>> But, along with my other E-Mail...\n>> \n>> > [...]\n>> > +static void parse_gitmodules_oids(int fd, struct oidset *gitmodules_oids)\n>> > +{\n>> > +\tint len = the_hash_algo->hexsz + 1; /* hash + NL */\n>> > +\n>> > +\tdo {\n>> > +\t\tchar hex_hash[GIT_MAX_HEXSZ + 1];\n>> > +\t\tint read_len = read_in_full(fd, hex_hash, len);\n>> > +\t\tstruct object_id oid;\n>> > +\t\tconst char *end;\n>> > +\n>> > +\t\tif (!read_len)\n>> > +\t\t\treturn;\n>> > +\t\tif (read_len != len)\n>> > +\t\t\tdie(\"invalid length read %d\", read_len);\n>> > +\t\tif (parse_oid_hex(hex_hash, &oid, &end) || *end != '\\n')\n>> > +\t\t\tdie(\"invalid hash\");\n>> > +\t\toidset_insert(gitmodules_oids, &oid);\n>> > +\t} while (1);\n>> > +}\n>> > +\n>> \n>> Doesn't this IPC mechanism already exist in the form of fsck.skipList?\n>> See my 1f3299fda9 (fsck: make fsck_config() re-usable, 2021-01-05) on\n>> \"next\". I.e. as noted in my just-sent-E-Mail you could probably just\n>> re-use skiplist as-is.\n>\n> I'm not sure how fsck.skipList could be used here. Before running\n> fsck_finish() for the first time, we don't know which .gitmodules are\n> missing and which are not. And when running fsck_finish() for the second\n> time, we definitely do not want to skip any blobs.\n>\n>> Or if not it seems to me that this whole IPC mechanism would be better\n>> done with a tempfile and passing it along like we already pass the\n>> fsck.skipList between these processes.\n>> \n>> I doubt it's going to be large enough to matter, we could just put it in\n>> .git/ somewhere, like we put gc.log etc (but created with a mktemp()\n>> name...).\n>> \n>> Or if we want to keep the \"print | process\" model we can refactor\n>> the existing fsck IPC noted in 1f3299fda9 a bit, so e.g. you pass some\n>> version of \"lines prefixed with \"fsck-skiplist: \" go into list xyz via a\n>> command-line option. And then existing option(s) and your potential new\n>> list (which as noted, I think is probably redundant to the skiplist) can\n>> use it.\n>\n> I think using stdout is superior to using a tempfile - we don't have to\n> worry about interrupted invocations, for example.\n>\n> What do you mean by \"the existing fsck IPC noted in 1f3299fda9\"? If you\n> mean the ability to pass a list of OIDs, for example using \"-c\n> fsck.skipList=filename.txt\", I'm not sure that it solves anything.\n> Firstly, I don't think that the skipList is useful here (as I said\n> earlier). And secondly, I don't think that OID input is the issue -\n> right now, the design is a process (index-pack, calling fsck_finish())\n> writing to its output which is then picked up by the calling process\n> (fetch-pack). We are not sending the dangling .gitmodules through stdin\n> anywhere.\n\nSorry for being unclear here. I don't think (honestly I don't remember,\nit's been almost a month) that I meant to you should use the skipList.\n\nLooking at that code again we use object_on_skiplist() to do an early\npunt in report(), but also fsck_blob(), presumably you never want the\nlatter, and that early punting wouldn't be needed if your report()\nfunction intercepted the modules blob id for stashing it away / later\nreporting / whatever.\n\nSo yeah, I'm 99% sure now that's not what I meant :)\n\nWhat I meant with:\n\n Or if we want to keep the \"print | process\"[...]\n\nIs that we have an existing ad-hoc IPC model for these commands in\npassing along the skipList, which is made more complex because sometimes\nthe initial process reads the file, sometimes it passes it along as-is\nto the child.\n\nAnd then there's this patch that passes OIDs too, but through a\ndifferent mechanism.\n\nI was suggesting that perhaps it made more sense to refactor both so\nthey could use the same mechanism, because we're potentially passing two\nlists of OIDs between the two. Just one goes via line-at-a-time in the\noutput, the other via a config option on the command-line.\n"},{"id":"417179","messageId":"8735xua40w.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"2d6d8c2671fe424c752994dcb5277d4d923e17a0.1611455251.git.jonathantanmy@google.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:27:27Z","receivedAt":"2021-02-17T19:28:14Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sun, Jan 24 2021, Jonathan Tan wrote:\n\n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index 557bd2f348..f995c15115 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -1888,8 +1888,13 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n> \telse\n> \t\tclose(input_fd);\n> \n> -\tif (do_fsck_object && fsck_finish(&fsck_options))\n> -\t\tdie(_(\"fsck error in pack objects\"));\n> +\tif (do_fsck_object) {\n> +\t\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n> +\n> +\t\tfo.print_dangling_gitmodules = 1;\n> +\t\tif (fsck_finish(&fo))\n> +\t\t\tdie(_(\"fsck error in pack objects\"));\n> +\t}\n> [...]\n> +static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n> +{\n> +\tstruct oidset_iter iter;\n> +\tconst struct object_id *oid;\n> +\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n> +\n> +\tif (!oidset_size(gitmodules_oids))\n> +\t\treturn;\n> +\n> +\toidset_iter_init(gitmodules_oids, &iter);\n> +\twhile ((oid = oidset_iter_next(&iter)))\n> +\t\tregister_found_gitmodules(oid);\n> +\tif (fsck_finish(&fo))\n> +\t\tdie(\"fsck failed\");\n> +}\n> +\n\nWhat's the need for STRICT here & can't the former use the existing\nfsck_options in index-pack.c? With this on top we pass all tests:\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 18531199242..5464edf4778 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1933,10 +1933,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \t\tclose(input_fd);\n \n \tif (do_fsck_object) {\n-\t\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n-\n-\t\tfo.print_dangling_gitmodules = 1;\n-\t\tif (fsck_finish(&fo))\n+\t\tfsck_options.print_dangling_gitmodules = 1;\n+\t\tif (fsck_finish(&fsck_options))\n \t\t\tdie(_(\"fsck error in pack objects\"));\n \t}\n \ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 0a337a04f1f..a8754d97e3d 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -997,7 +997,7 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n {\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n-\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n+\tstruct fsck_options fo = FSCK_OPTIONS_DEFAULT;\n \n \tif (!oidset_size(gitmodules_oids))\n \t\treturn;\n"},{"id":"417182","messageId":"20210217194246.25342-1-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 00/14] fsck: API improvements","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:32Z","receivedAt":"2021-02-17T19:44:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Jonathan Tan pointed out that the fsck error_func doesn't pass you the\nID of the fsck failure in [1]. This series improves the API so it\ndoes, and moves the gitmodules_{found,done} variables into the\nfsck_options struct.\n\nThe result is that instead of the \"print_dangling_gitmodules\" member\nin that series we can just implement that with the diff at the end of\nthis cover letter (goes on top of a merge of this series & \"seen\"),\nand without any changes to fsck_finish().\n\nThis conflicts with other in-flight fsck changes but the conflict is\nrather trivial. Jeff King has another concurrent series to add a\ncouple of new fsck checks, those need to be moved to fsck.h, and\nthere's another trivial conflict in 2 hunks due to the\ngitmodules_{found,done} move.\n\n1. https://lore.kernel.org/git/87blcja2ha.fsf@evledraar.gmail.com/\n\nÆvar Arnfjörð Bjarmason (14):\n fsck.h: indent arguments to of fsck_set_msg_type\n fsck.h: use use \"enum object_type\" instead of \"int\"\n fsck.c: rename variables in fsck_set_msg_type() for less confusion\n fsck.c: move definition of msg_id into append_msg_id()\n fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n fsck.c: undefine temporary STR macro after use\n fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n fsck.c: pass along the fsck_msg_id in the fsck_error callback\n fsck.c: add an fsck_set_msg_type() API that takes enums\n fsck.h: update FSCK_OPTIONS_* for object_name\n fsck.c: move gitmodules_{found,done} into fsck_options\n\n builtin/fsck.c | 7 +-\n builtin/index-pack.c | 3 +-\n builtin/mktag.c | 7 +-\n builtin/unpack-objects.c | 3 +-\n fsck.c | 160 ++++++++++++---------------------------\n fsck.h | 98 +++++++++++++++++++++---\n 6 files changed, 152 insertions(+), 126 deletions(-)\n\n-- \n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 82f381f854..22dfcfc5de 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1713,6 +1713,20 @@ static void show_pack_info(int stat_only)\n \t}\n }\n \n+static int index_pack_fsck_error_func(struct fsck_options *o,\n+\t\t\t\t const struct object_id *oid,\n+\t\t\t\t enum object_type object_type,\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t const char *message)\n+{\n+\tif (msg_id == FSCK_MSG_GITMODULES_MISSING) {\n+\t\tputs(oid_to_hex(oid));\n+\t\treturn 0;\n+\t}\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n+}\n+\n int cmd_index_pack(int argc, const char **argv, const char *prefix)\n {\n \tint i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;\n@@ -1934,10 +1948,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \t\tclose(input_fd);\n \n \tif (do_fsck_object) {\n-\t\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n-\n-\t\tfo.print_dangling_gitmodules = 1;\n-\t\tif (fsck_finish(&fo))\n+\t\tfsck_options.error_func = index_pack_fsck_error_func;\n+\t\tif (fsck_finish(&fsck_options))\n \t\t\tdie(_(\"fsck error in pack objects\"));\n \t}\n \ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 0a337a04f1..9fc2ce86e4 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -40,6 +40,7 @@ static struct shallow_lock shallow_lock;\n static const char *alternate_shallow_file;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n \n /* Remember to update object flag allocation in object.h */\n #define COMPLETE\t(1U << 0)\n@@ -993,19 +994,34 @@ static int cmp_ref_by_name(const void *a_, const void *b_)\n \treturn strcmp(a->name, b->name);\n }\n \n+static int fetch_pack_fsck_error_func(struct fsck_options *o,\n+\t\t\t\t const struct object_id *oid,\n+\t\t\t\t enum object_type object_type,\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t const char *message)\n+{\n+\tif (msg_id == FSCK_MSG_GITMODULES_MISSING) {\n+\t\tputs(oid_to_hex(oid));\n+\t\treturn 0;\n+\t}\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n+}\n+\n static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n {\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n-\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n \n \tif (!oidset_size(gitmodules_oids))\n \t\treturn;\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(oid);\n-\tif (fsck_finish(&fo))\n+\t\toidset_insert(&fsck_options.gitmodules_found, oid);\n+\n+\tfsck_options.error_func = fetch_pack_fsck_error_func;\n+\tif (fsck_finish(&fsck_options))\n \t\tdie(\"fsck failed\");\n }\n \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417183","messageId":"20210217194246.25342-3-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 02/14] fsck.h: use use \"enum object_type\" instead of \"int\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:34Z","receivedAt":"2021-02-17T19:44:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_walk_func to use an \"enum object_type\" instead of an\n\"int\" type. The types are compatible, and ever since this was added in\n355885d5315 (add generic, type aware object chain walker, 2008-02-25)\nwe've used entries from object_type (OBJ_BLOB etc.).\n\nSo this doesn't really change anything as far as the generated code is\nconcerned, it just gives the compiler more information and makes this\neasier to read.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 3 ++-\n builtin/index-pack.c | 3 ++-\n builtin/unpack-objects.c | 3 ++-\n fsck.h | 3 ++-\n 4 files changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 821e7798c70..68f0329e69e 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -197,7 +197,8 @@ static int traverse_reachable(void)\n \treturn !!result;\n }\n \n-static int mark_used(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_used(struct object *obj, enum object_type object_type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn 1;\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 54f74c48741..2f291a14d4a 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -212,7 +212,8 @@ static void cleanup_thread(void)\n \tfree(thread_data);\n }\n \n-static int mark_link(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_link(struct object *obj, enum object_type type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn -1;\ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex dd4a75e030d..ca54fd16688 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -187,7 +187,8 @@ static void write_cached_object(struct object *obj, struct obj_buffer *obj_buf)\n * that have reachability requirements and calls this function.\n * Verify its reachability and validity recursively and write it out.\n */\n-static int check_object(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int check_object(struct object *obj, enum object_type type,\n+\t\t\tvoid *data, struct fsck_options *options)\n {\n \tstruct obj_buffer *obj_buf;\n \ndiff --git a/fsck.h b/fsck.h\nindex df0b64a2163..0c75789d219 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -23,7 +23,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type);\n * <0\terror signaled and abort\n * >0\terror signaled and do not abort\n */\n-typedef int (*fsck_walk_func)(struct object *obj, int type, void *data, struct fsck_options *options);\n+typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n+\t\t\t void *data, struct fsck_options *options);\n \n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417184","messageId":"20210217194246.25342-4-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 03/14] fsck.c: rename variables in fsck_set_msg_type() for less confusion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:35Z","receivedAt":"2021-02-17T19:44:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename variables in a function added in 0282f4dced0 (fsck: offer a\nfunction to demote fsck errors to warnings, 2015-06-22).\n\nIt was needlessly confusing that it took a \"msg_type\" argument, but\nthen later declared another \"msg_type\" of a different type.\n\nLet's rename that to \"tmp\", and rename \"id\" to \"msg_id\" and \"msg_id\"\nto \"msg_id_str\" etc. This will make a follow-up change smaller.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 24 ++++++++++++------------\n 1 file changed, 12 insertions(+), 12 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 4b7f0b73d73..acccad243ec 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -203,27 +203,27 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n }\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type)\n+\t\tconst char *msg_id_str, const char *msg_type_str)\n {\n-\tint id = parse_msg_id(msg_id), type;\n+\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n \n-\tif (id < 0)\n-\t\tdie(\"Unhandled message id: %s\", msg_id);\n-\ttype = parse_msg_type(msg_type);\n+\tif (msg_id < 0)\n+\t\tdie(\"Unhandled message id: %s\", msg_id_str);\n+\tmsg_type = parse_msg_type(msg_type_str);\n \n-\tif (type != FSCK_ERROR && msg_id_info[id].msg_type == FSCK_FATAL)\n-\t\tdie(\"Cannot demote %s to %s\", msg_id, msg_type);\n+\tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n+\t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *msg_type;\n-\t\tALLOC_ARRAY(msg_type, FSCK_MSG_MAX);\n+\t\tint *tmp;\n+\t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tmsg_type[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = msg_type;\n+\t\t\ttmp[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = tmp;\n \t}\n \n-\toptions->msg_type[id] = type;\n+\toptions->msg_type[msg_id] = msg_type;\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417185","messageId":"20210217194246.25342-2-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 01/14] fsck.h: indent arguments to of fsck_set_msg_type","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:33Z","receivedAt":"2021-02-17T19:44:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Signed-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 423c467feb7..df0b64a2163 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -11,7 +11,7 @@ struct fsck_options;\n struct object;\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type);\n+\t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n int is_valid_msg_type(const char *msg_id, const char *msg_type);\n \n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417186","messageId":"20210217194246.25342-5-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 04/14] fsck.c: move definition of msg_id into append_msg_id()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:36Z","receivedAt":"2021-02-17T19:45:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor code added in 71ab8fa840f (fsck: report the ID of the\nerror/warning, 2015-06-22) to resolve the msg_id to a string in the\nfunction that wants it, instead of doing it in report().\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex acccad243ec..1070071ffec 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,8 +264,9 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, const char *msg_id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n {\n+\tconst char *msg_id = msg_id_info[id].id_string;\n \tfor (;;) {\n \t\tchar c = *(msg_id)++;\n \n@@ -308,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, msg_id_info[id].id_string);\n+\tappend_msg_id(&sb, id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417187","messageId":"20210217194246.25342-8-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 07/14] fsck.c: call parse_msg_type() early in fsck_set_msg_type()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:39Z","receivedAt":"2021-02-17T19:45:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"There's no reason to defer the calling of parse_msg_type() until after\nwe've checked if the \"id < 0\". This is not a hot codepath, and\nparse_msg_type() itself may die on invalid input.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 00e0fef21ca..7c53080ad48 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -203,11 +203,10 @@ void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n \tint msg_id = parse_msg_id(msg_id_str);\n-\tenum fsck_msg_type msg_type;\n+\tenum fsck_msg_type msg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n-\tmsg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417188","messageId":"20210217194246.25342-6-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 05/14] fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:37Z","receivedAt":"2021-02-17T19:45:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the remaining variables of type fsck_msg_id from \"id\" to\n\"msg_id\". This change is relatively small, and is worth the churn for\na later change where we have different id's in the \"report\" function.\n---\n fsck.c | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 1070071ffec..dbb6f7c4ee2 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,19 +264,19 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id msg_id)\n {\n-\tconst char *msg_id = msg_id_info[id].id_string;\n+\tconst char *msg_id_str = msg_id_info[msg_id].id_string;\n \tfor (;;) {\n-\t\tchar c = *(msg_id)++;\n+\t\tchar c = *(msg_id_str)++;\n \n \t\tif (!c)\n \t\t\tbreak;\n \t\tif (c != '_')\n \t\t\tstrbuf_addch(sb, tolower(c));\n \t\telse {\n-\t\t\tassert(*msg_id);\n-\t\t\tstrbuf_addch(sb, *(msg_id)++);\n+\t\t\tassert(*msg_id_str);\n+\t\t\tstrbuf_addch(sb, *(msg_id_str)++);\n \t\t}\n \t}\n \n@@ -292,11 +292,11 @@ static int object_on_skiplist(struct fsck_options *opts,\n __attribute__((format (printf, 5, 6)))\n static int report(struct fsck_options *options,\n \t\t const struct object_id *oid, enum object_type object_type,\n-\t\t enum fsck_msg_id id, const char *fmt, ...)\n+\t\t enum fsck_msg_id msg_id, const char *fmt, ...)\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(id, options), result;\n+\tint msg_type = fsck_msg_type(msg_id, options), result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -309,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, id);\n+\tappend_msg_id(&sb, msg_id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417189","messageId":"20210217194246.25342-9-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 08/14] fsck.c: undefine temporary STR macro after use","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:40Z","receivedAt":"2021-02-17T19:45:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In f417eed8cde (fsck: provide a function to parse fsck message IDs,\n2015-06-22) the \"STR\" macro was introduced, but that short macro name\nwas not undefined after use as was done earlier in the same series for\nthe MSG_ID macro in c99ba492f1c (fsck: introduce identifiers for fsck\nmessages, 2015-06-22).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/fsck.c b/fsck.c\nindex 7c53080ad48..88884e91c89 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -100,6 +100,7 @@ static struct {\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n+#undef STR\n \n static void prepare_msg_ids(void)\n {\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417190","messageId":"20210217194246.25342-11-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 10/14] fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:42Z","receivedAt":"2021-02-17T19:45:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FOREACH_FSCK_MSG_ID macro and the fsck_msg_id enum it helps\ndefine from fsck.c to fsck.h. This is in preparation for having\nnon-static functions take the fsck_msg_id as an argument.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 66 ---------------------------------------------------------\n fsck.h | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 67 insertions(+), 66 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 1730acd698d..980ef2cb8fa 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,72 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_FSCK_MSG_ID(FUNC) \\\n-\t/* fatal errors */ \\\n-\tFUNC(NUL_IN_HEADER, FATAL) \\\n-\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n-\t/* errors */ \\\n-\tFUNC(BAD_DATE, ERROR) \\\n-\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n-\tFUNC(BAD_EMAIL, ERROR) \\\n-\tFUNC(BAD_NAME, ERROR) \\\n-\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n-\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n-\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n-\tFUNC(BAD_TIMEZONE, ERROR) \\\n-\tFUNC(BAD_TREE, ERROR) \\\n-\tFUNC(BAD_TREE_SHA1, ERROR) \\\n-\tFUNC(BAD_TYPE, ERROR) \\\n-\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n-\tFUNC(MISSING_AUTHOR, ERROR) \\\n-\tFUNC(MISSING_COMMITTER, ERROR) \\\n-\tFUNC(MISSING_EMAIL, ERROR) \\\n-\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_OBJECT, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_TAG, ERROR) \\\n-\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n-\tFUNC(MISSING_TREE, ERROR) \\\n-\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n-\tFUNC(MISSING_TYPE, ERROR) \\\n-\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n-\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n-\tFUNC(TREE_NOT_SORTED, ERROR) \\\n-\tFUNC(UNKNOWN_TYPE, ERROR) \\\n-\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n-\tFUNC(GITMODULES_MISSING, ERROR) \\\n-\tFUNC(GITMODULES_BLOB, ERROR) \\\n-\tFUNC(GITMODULES_LARGE, ERROR) \\\n-\tFUNC(GITMODULES_NAME, ERROR) \\\n-\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n-\tFUNC(GITMODULES_URL, ERROR) \\\n-\tFUNC(GITMODULES_PATH, ERROR) \\\n-\tFUNC(GITMODULES_UPDATE, ERROR) \\\n-\t/* warnings */ \\\n-\tFUNC(BAD_FILEMODE, WARN) \\\n-\tFUNC(EMPTY_NAME, WARN) \\\n-\tFUNC(FULL_PATHNAME, WARN) \\\n-\tFUNC(HAS_DOT, WARN) \\\n-\tFUNC(HAS_DOTDOT, WARN) \\\n-\tFUNC(HAS_DOTGIT, WARN) \\\n-\tFUNC(NULL_SHA1, WARN) \\\n-\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n-\tFUNC(NUL_IN_COMMIT, WARN) \\\n-\t/* infos (reported as warnings, but ignored by default) */ \\\n-\tFUNC(GITMODULES_PARSE, INFO) \\\n-\tFUNC(BAD_TAG_NAME, INFO) \\\n-\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n-\t/* ignored (elevated when requested) */ \\\n-\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n-\n-#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n-enum fsck_msg_id {\n-\tFOREACH_FSCK_MSG_ID(MSG_ID)\n-\tFSCK_MSG_MAX\n-};\n-#undef MSG_ID\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\ndiff --git a/fsck.h b/fsck.h\nindex c77e8ddf10b..b4c53aaa08c 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -10,6 +10,73 @@ enum fsck_msg_type {\n \tFSCK_WARN,\n \tFSCK_IGNORE\n };\n+\n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n+\t/* fatal errors */ \\\n+\tFUNC(NUL_IN_HEADER, FATAL) \\\n+\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n+\t/* errors */ \\\n+\tFUNC(BAD_DATE, ERROR) \\\n+\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n+\tFUNC(BAD_EMAIL, ERROR) \\\n+\tFUNC(BAD_NAME, ERROR) \\\n+\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n+\tFUNC(BAD_TIMEZONE, ERROR) \\\n+\tFUNC(BAD_TREE, ERROR) \\\n+\tFUNC(BAD_TREE_SHA1, ERROR) \\\n+\tFUNC(BAD_TYPE, ERROR) \\\n+\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n+\tFUNC(MISSING_AUTHOR, ERROR) \\\n+\tFUNC(MISSING_COMMITTER, ERROR) \\\n+\tFUNC(MISSING_EMAIL, ERROR) \\\n+\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_OBJECT, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_TAG, ERROR) \\\n+\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n+\tFUNC(MISSING_TREE, ERROR) \\\n+\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n+\tFUNC(MISSING_TYPE, ERROR) \\\n+\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n+\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(TREE_NOT_SORTED, ERROR) \\\n+\tFUNC(UNKNOWN_TYPE, ERROR) \\\n+\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n+\tFUNC(GITMODULES_MISSING, ERROR) \\\n+\tFUNC(GITMODULES_BLOB, ERROR) \\\n+\tFUNC(GITMODULES_LARGE, ERROR) \\\n+\tFUNC(GITMODULES_NAME, ERROR) \\\n+\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n+\tFUNC(GITMODULES_URL, ERROR) \\\n+\tFUNC(GITMODULES_PATH, ERROR) \\\n+\tFUNC(GITMODULES_UPDATE, ERROR) \\\n+\t/* warnings */ \\\n+\tFUNC(BAD_FILEMODE, WARN) \\\n+\tFUNC(EMPTY_NAME, WARN) \\\n+\tFUNC(FULL_PATHNAME, WARN) \\\n+\tFUNC(HAS_DOT, WARN) \\\n+\tFUNC(HAS_DOTDOT, WARN) \\\n+\tFUNC(HAS_DOTGIT, WARN) \\\n+\tFUNC(NULL_SHA1, WARN) \\\n+\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n+\tFUNC(NUL_IN_COMMIT, WARN) \\\n+\t/* infos (reported as warnings, but ignored by default) */ \\\n+\tFUNC(GITMODULES_PARSE, INFO) \\\n+\tFUNC(BAD_TAG_NAME, INFO) \\\n+\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\t/* ignored (elevated when requested) */ \\\n+\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n+\n+#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n+enum fsck_msg_id {\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n+\tFSCK_MSG_MAX\n+};\n+#undef MSG_ID\n+\n struct fsck_options;\n struct object;\n \n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417191","messageId":"20210217194246.25342-13-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 12/14] fsck.c: add an fsck_set_msg_type() API that takes enums","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:44Z","receivedAt":"2021-02-17T19:45:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code I added in acf9de4c94e (mktag: use fsck instead of custom\nverify_tag(), 2021-01-05) to make use of a new API function that takes\nthe fsck_msg_{id,type} types, instead of arbitrary strings that\nwe'll (hopefully) parse into those types.\n\nAt the time that the fsck_set_msg_type() API was introduced in\n0282f4dced0 (fsck: offer a function to demote fsck errors to warnings,\n2015-06-22) it was only intended to be used to parse user-supplied\ndata.\n\nFor things that are purely internal to the C code it makes sense to\nhave the compiler check these arguments, and to skip the sanity\nchecking of the data in fsck_set_msg_type() which is redundant to\nchecks we get from the compiler.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/mktag.c | 3 ++-\n fsck.c | 27 +++++++++++++++++----------\n fsck.h | 3 +++\n 3 files changed, 22 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex dc989c356f5..de67a94f24e 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -93,7 +93,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)\n \t\tdie_errno(_(\"could not read from stdin\"));\n \n \tfsck_options.error_func = mktag_fsck_error_func;\n-\tfsck_set_msg_type(&fsck_options, \"extraheaderentry\", \"warn\");\n+\tfsck_set_msg_type_from_ids(&fsck_options, FSCK_MSG_EXTRA_HEADER_ENTRY,\n+\t\t\t\t FSCK_WARN);\n \t/* config might set fsck.extraHeaderEntry=* again */\n \tgit_config(mktag_config, NULL);\n \tif (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,\ndiff --git a/fsck.c b/fsck.c\nindex 007f02b556a..54632404de5 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -134,6 +134,22 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n \treturn 1;\n }\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type)\n+{\n+\tif (!options->msg_type) {\n+\t\tint i;\n+\t\tenum fsck_msg_type *tmp;\n+\t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n+\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n+\t\t\ttmp[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = tmp;\n+\t}\n+\n+\toptions->msg_type[msg_id] = msg_type;\n+}\n+\n void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n@@ -146,16 +162,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n-\tif (!options->msg_type) {\n-\t\tint i;\n-\t\tenum fsck_msg_type *tmp;\n-\t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n-\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\ttmp[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = tmp;\n-\t}\n-\n-\toptions->msg_type[msg_id] = msg_type;\n+\tfsck_set_msg_type_from_ids(options, msg_id, msg_type);\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\ndiff --git a/fsck.h b/fsck.h\nindex 56536d7f29e..af145bb4596 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -80,6 +80,9 @@ enum fsck_msg_id {\n struct fsck_options;\n struct object;\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type);\n void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417192","messageId":"20210217194246.25342-12-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 11/14] fsck.c: pass along the fsck_msg_id in the fsck_error callback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:43Z","receivedAt":"2021-02-17T19:45:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_error callback to also pass along the\nfsck_msg_id. Before this change the only way to get the message id was\nto parse it back out of the \"message\".\n\nLet's pass it down explicitly for the benefit of callers that might\nwant to use it, as discussed in [1].\n\nPassing the msg_type is now redundant, as you can always get it back\nfrom the msg_id, but I'm not changing that convention. It's really\ncommon to need the msg_type, and the report() function itself (which\ncalls \"fsck_error\") needs to call fsck_msg_type() to discover\nit. Let's not needlessly re-do that work in the user callback.\n\n1. https://lore.kernel.org/git/87blcja2ha.fsf@evledraar.gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 4 +++-\n builtin/mktag.c | 1 +\n fsck.c | 6 ++++--\n fsck.h | 6 ++++--\n 4 files changed, 12 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex d6d745dc702..b71fac4ceca 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -89,7 +89,9 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 1834394a9b6..dc989c356f5 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -23,6 +23,7 @@ static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n \t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n \t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\ndiff --git a/fsck.c b/fsck.c\nindex 980ef2cb8fa..007f02b556a 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -247,7 +247,7 @@ static int report(struct fsck_options *options,\n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n \tresult = options->error_func(options, oid, object_type,\n-\t\t\t\t msg_type, sb.buf);\n+\t\t\t\t msg_type, msg_id, sb.buf);\n \tstrbuf_release(&sb);\n \tva_end(ap);\n \n@@ -1195,7 +1195,9 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type,\n+\t\t\tenum fsck_msg_id msg_id,\n+\t\t\tconst char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex b4c53aaa08c..56536d7f29e 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -99,11 +99,13 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\t const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\tconst char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417193","messageId":"20210217194246.25342-7-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 06/14] fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:38Z","receivedAt":"2021-02-17T19:45:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} defines into a new\nfsck_msg_type enum.\n\nThese defines were originally introduced in:\n\n - ba002f3b28a (builtin-fsck: move common object checking code to\n fsck.c, 2008-02-25)\n - f50c4407305 (fsck: disallow demoting grave fsck errors to warnings,\n 2015-06-22)\n - efaba7cc77f (fsck: optionally ignore specific fsck issues\n completely, 2015-06-22)\n - f27d05b1704 (fsck: allow upgrading fsck warnings to errors,\n 2015-06-22)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 2 +-\n builtin/mktag.c | 3 ++-\n fsck.c | 21 ++++++++++-----------\n fsck.h | 17 ++++++++++-------\n 4 files changed, 23 insertions(+), 20 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 68f0329e69e..d6d745dc702 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -89,7 +89,7 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t int msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type, const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 41a399a69e4..1834394a9b6 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -22,7 +22,8 @@ static int mktag_config(const char *var, const char *value, void *cb)\n static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/fsck.c b/fsck.c\nindex dbb6f7c4ee2..00e0fef21ca 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,9 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FSCK_FATAL -1\n-#define FSCK_INFO -2\n-\n #define FOREACH_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n@@ -97,7 +94,7 @@ static struct {\n \tconst char *id_string;\n \tconst char *downcased;\n \tconst char *camelcased;\n-\tint msg_type;\n+\tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n \tFOREACH_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n@@ -164,10 +161,10 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n \t\tlist_config_item(list, prefix, msg_id_info[i].camelcased);\n }\n \n-static int fsck_msg_type(enum fsck_msg_id msg_id,\n+static enum fsck_msg_type fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n-\tint msg_type;\n+\tenum fsck_msg_type msg_type;\n \n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n@@ -182,7 +179,7 @@ static int fsck_msg_type(enum fsck_msg_id msg_id,\n \treturn msg_type;\n }\n \n-static int parse_msg_type(const char *str)\n+static enum fsck_msg_type parse_msg_type(const char *str)\n {\n \tif (!strcmp(str, \"error\"))\n \t\treturn FSCK_ERROR;\n@@ -205,7 +202,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n-\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n+\tint msg_id = parse_msg_id(msg_id_str);\n+\tenum fsck_msg_type msg_type;\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n@@ -216,7 +214,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *tmp;\n+\t\tenum fsck_msg_type *tmp;\n \t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n \t\t\ttmp[i] = fsck_msg_type(i, options);\n@@ -296,7 +294,8 @@ static int report(struct fsck_options *options,\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(msg_id, options), result;\n+\tenum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);\n+\tint result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -1262,7 +1261,7 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tint msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type, const char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex 0c75789d219..c77e8ddf10b 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -3,10 +3,13 @@\n \n #include \"oidset.h\"\n \n-#define FSCK_ERROR 1\n-#define FSCK_WARN 2\n-#define FSCK_IGNORE 3\n-\n+enum fsck_msg_type {\n+\tFSCK_INFO = -2,\n+\tFSCK_FATAL = -1,\n+\tFSCK_ERROR = 1,\n+\tFSCK_WARN,\n+\tFSCK_IGNORE\n+};\n struct fsck_options;\n struct object;\n \n@@ -29,17 +32,17 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t int msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tint msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, const char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n \tfsck_error error_func;\n \tunsigned strict:1;\n-\tint *msg_type;\n+\tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n \tkh_oid_map_t *object_names;\n };\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417194","messageId":"20210217194246.25342-14-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 13/14] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:45Z","receivedAt":"2021-02-17T19:45:51Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add the object_name member to the initialization macro. This was\nomitted in 7b35efd734e (fsck_walk(): optionally name objects on the\ngo, 2016-07-17) when the field was added.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex af145bb4596..28137a77df0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -119,8 +119,8 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n-#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n+#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n+#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417195","messageId":"20210217194246.25342-15-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 14/14] fsck.c: move gitmodules_{found,done} into fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:46Z","receivedAt":"2021-02-17T19:46:10Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the gitmodules_{found,done} static variables added in\n159e7b080bf (fsck: detect gitmodules files, 2018-05-02) into the\nfsck_options struct. It makes sense to keep all the context in the\nsame place.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 19 ++++++++-----------\n fsck.h | 6 ++++--\n 2 files changed, 12 insertions(+), 13 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 54632404de5..f344b6be3d3 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -19,9 +19,6 @@\n #include \"credential.h\"\n #include \"help.h\"\n \n-static struct oidset gitmodules_found = OIDSET_INIT;\n-static struct oidset gitmodules_done = OIDSET_INIT;\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\n@@ -621,7 +618,7 @@ static int fsck_tree(const struct object_id *oid,\n \n \t\tif (is_hfs_dotgitmodules(name) || is_ntfs_dotgitmodules(name)) {\n \t\t\tif (!S_ISLNK(mode))\n-\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\telse\n \t\t\t\tretval += report(options,\n \t\t\t\t\t\t oid, OBJ_TREE,\n@@ -635,7 +632,7 @@ static int fsck_tree(const struct object_id *oid,\n \t\t\t\thas_dotgit |= is_ntfs_dotgit(backslash);\n \t\t\t\tif (is_ntfs_dotgitmodules(backslash)) {\n \t\t\t\t\tif (!S_ISLNK(mode))\n-\t\t\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\t\t\telse\n \t\t\t\t\t\tretval += report(options, oid, OBJ_TREE,\n \t\t\t\t\t\t\t\t FSCK_MSG_GITMODULES_SYMLINK,\n@@ -1147,9 +1144,9 @@ static int fsck_blob(const struct object_id *oid, const char *buf,\n \tstruct fsck_gitmodules_data data;\n \tstruct config_options config_opts = { 0 };\n \n-\tif (!oidset_contains(&gitmodules_found, oid))\n+\tif (!oidset_contains(&options->gitmodules_found, oid))\n \t\treturn 0;\n-\toidset_insert(&gitmodules_done, oid);\n+\toidset_insert(&options->gitmodules_done, oid);\n \n \tif (object_on_skiplist(options, oid))\n \t\treturn 0;\n@@ -1220,13 +1217,13 @@ int fsck_finish(struct fsck_options *options)\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n \n-\toidset_iter_init(&gitmodules_found, &iter);\n+\toidset_iter_init(&options->gitmodules_found, &iter);\n \twhile ((oid = oidset_iter_next(&iter))) {\n \t\tenum object_type type;\n \t\tunsigned long size;\n \t\tchar *buf;\n \n-\t\tif (oidset_contains(&gitmodules_done, oid))\n+\t\tif (oidset_contains(&options->gitmodules_done, oid))\n \t\t\tcontinue;\n \n \t\tbuf = read_object_file(oid, &type, &size);\n@@ -1251,8 +1248,8 @@ int fsck_finish(struct fsck_options *options)\n \t}\n \n \n-\toidset_clear(&gitmodules_found);\n-\toidset_clear(&gitmodules_done);\n+\toidset_clear(&options->gitmodules_found);\n+\toidset_clear(&options->gitmodules_done);\n \treturn ret;\n }\n \ndiff --git a/fsck.h b/fsck.h\nindex 28137a77df0..99c77289688 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -116,11 +116,13 @@ struct fsck_options {\n \tunsigned strict:1;\n \tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n+\tstruct oidset gitmodules_found;\n+\tstruct oidset gitmodules_done;\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n-#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n+#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, OIDSET_INIT, OIDSET_INIT, NULL }\n+#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, OIDSET_INIT, OIDSET_INIT, NULL }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417196","messageId":"20210217194246.25342-10-avarab@gmail.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"[PATCH 09/14] fsck.c: give \"FOREACH_MSG_ID\" a more specific name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-17T19:42:41Z","receivedAt":"2021-02-17T19:46:13Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the FOREACH_MSG_ID macro to FOREACH_FSCK_MSG_ID in preparation\nfor moving it over to fsck.h. It's good convention to name macros\nin *.h files in such a way as to clearly not clash with any other\nnames in other files.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 88884e91c89..1730acd698d 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,7 +22,7 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_MSG_ID(FUNC) \\\n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n \tFUNC(UNTERMINATED_HEADER, FATAL) \\\n@@ -83,7 +83,7 @@ static struct oidset gitmodules_done = OIDSET_INIT;\n \n #define MSG_ID(id, msg_type) FSCK_MSG_##id,\n enum fsck_msg_id {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \tFSCK_MSG_MAX\n };\n #undef MSG_ID\n@@ -96,7 +96,7 @@ static struct {\n \tconst char *camelcased;\n \tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417197","messageId":"20210217194604.871194-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqqpn0zoc2l.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 2/4] http-fetch: allow custom index-pack args","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-17T19:46:04Z","receivedAt":"2021-02-17T19:47:18Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Josh Steadmon writes:\n> \n> >> +--index-pack-args=::\n> >> +\tFor internal use only. The command to run on the contents of the\n> >> +\tdownloaded pack. Arguments are URL-encoded separated by spaces.\n> >\n> > I'm a bit skeptical of using URL encoding to work around embedded\n> > spaces. I believe in Emily's config-based hooks series, she wrote an\n> > argument parser to pull repeated arguments into a strvec, could you do\n> > something like that here?\n> >\n> > I'm sympathetic to the idea that since this is an internal-only flag, we\n> > can be a bit weird with the argument format, though.\n> \n> We tend to prefer quote.c::sq_quote*() suite of quoting; does this\n> codepath have very different constraints that require different\n> encoding?\n\nMy main issue was that I needed to join arbitrary strings and then split\nthem, which is why I URL-encoded them (so that they would no longer\ncontain spaces) and then used spaces as the \"join\" separator. With\nJosh's suggestion, I wouldn't need any sort of encoding or quoting, so I\nthink I'll use that.\n"},{"id":"417207","messageId":"20210217200522.875107-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"87blcja2ha.fsf@evledraar.gmail.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-17T20:05:22Z","receivedAt":"2021-02-17T20:06:07Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> > I tried that first, and the issue is that IDs like\n> > FSCK_MSG_GITMODULES_MISSING are internal to fsck.c. As for whether we\n> > should start exposing the IDs publicly, I think we should wait until a\n> > few new cases like this come up, so that we more fully understand the\n> > requirements first.\n> \n> The requirement is that you want the objects ids we'd otherwise error\n> about in fsck_finish(). Yeah we don't pass the \"fsck_msg_id\" down in the\n> \"report()\" function, but you can reliably strstr() it out of the\n> message.\n\nWe can't strstr() because of false positives (if, e.g. there is a\nsubmodule name that contains the string we're looking for), but looking\nat report() in fsck.c, the message ID is the very first thing appended,\nso I think we can use starts_with().\n\n> We document & hard rely on that already, since it's also a\n> config key.\n\nAh, good point.\n\n> But yeah, we could just change the report function to pass down the id\n> and move the relevant macros from fsck.c to fsck.h. I think that would\n> be a smaller change conceptually than a special-case flag in\n> fsck_options for something we could otherwise do with the error\n> reporting.\n\nI agree - I thought this wouldn't be possible, but like you said, we can\nreliably make use of the string in report() (or pass the ID, like your\npatch set [1] does) so we should do this.\n\nWhat would be the best way to proceed, now that we have at least 2 patch\nsets (mine and yours) in play? I was thinking that I should update my\none to use the string reported in report() (with starts_with()), so that\nboth our patch sets can be reviewed and merged in parallel, and after\nthat, update the fsck code to use the ID instead of the string.\n\n[1] https://lore.kernel.org/git/87blcja2ha.fsf@evledraar.gmail.com/\n"},{"id":"417208","messageId":"20210217201006.877438-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"878s7na1h6.fsf@evledraar.gmail.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-17T20:10:06Z","receivedAt":"2021-02-17T20:10:52Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Sorry for being unclear here. I don't think (honestly I don't remember,\n> it's been almost a month) that I meant to you should use the skipList.\n> \n> Looking at that code again we use object_on_skiplist() to do an early\n> punt in report(), but also fsck_blob(), presumably you never want the\n> latter, and that early punting wouldn't be needed if your report()\n> function intercepted the modules blob id for stashing it away / later\n> reporting / whatever.\n> \n> So yeah, I'm 99% sure now that's not what I meant :)\n> \n> What I meant with:\n> \n> Or if we want to keep the \"print | process\"[...]\n> \n> Is that we have an existing ad-hoc IPC model for these commands in\n> passing along the skipList, which is made more complex because sometimes\n> the initial process reads the file, sometimes it passes it along as-is\n> to the child.\n> \n> And then there's this patch that passes OIDs too, but through a\n> different mechanism.\n> \n> I was suggesting that perhaps it made more sense to refactor both so\n> they could use the same mechanism, because we're potentially passing two\n> lists of OIDs between the two. Just one goes via line-at-a-time in the\n> output, the other via a config option on the command-line.\n\nThanks for your explanation. I still think that they are quite different\n- skiplist is a user-written file containing a list of OIDs that will\nlikely never change, whereas my list of dangling .gitmodules is a list\nof OIDs dynamically generated (and thus, always different) whenever a\nfetch is done. So I think it's quite reasonable to pass skiplist as a\nfile name, and my list should be passed line-by-line.\n"},{"id":"417209","messageId":"20210217201115.879491-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"8735xua40w.fsf@evledraar.gmail.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-17T20:11:15Z","receivedAt":"2021-02-17T20:12:00Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> What's the need for STRICT here & can't the former use the existing\n> fsck_options in index-pack.c? With this on top we pass all tests:\n\n[snip code]\n\nGood point - I'll do that.\n"},{"id":"417218","messageId":"xmqqo8gijtl7.fsf@gitster.g","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"Re: [PATCH 00/14] fsck: API improvements","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-17T21:02:44Z","receivedAt":"2021-02-17T21:03:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Jonathan Tan pointed out that the fsck error_func doesn't pass you the\n> ID of the fsck failure in [1]. This series improves the API so it\n> does, and moves the gitmodules_{found,done} variables into the\n> fsck_options struct.\n>\n> The result is that instead of the \"print_dangling_gitmodules\" member\n> in that series we can just implement that with the diff at the end of\n> this cover letter (goes on top of a merge of this series & \"seen\"),\n> and without any changes to fsck_finish().\n>\n> This conflicts with other in-flight fsck changes but the conflict is\n> rather trivial. Jeff King has another concurrent series to add a\n> couple of new fsck checks, those need to be moved to fsck.h, and\n> there's another trivial conflict in 2 hunks due to the\n> gitmodules_{found,done} move.\n>\n> 1. https://lore.kernel.org/git/87blcja2ha.fsf@evledraar.gmail.com/\n\nLet's get this reviewed now, but with expectation that it will be\nrebased after the dust settles.\n"},{"id":"417248","messageId":"xmqqczwyi7qf.fsf@gitster.g","threadId":"54994","inReplyTo":"20210217194246.25342-3-avarab@gmail.com","subject":"Re: [PATCH 02/14] fsck.h: use use \"enum object_type\" instead of \"int\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-17T23:40:08Z","receivedAt":"2021-02-17T23:41:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Subject: Re: [PATCH 02/14] fsck.h: use use \"enum object_type\" instead of \"int\"\n\nuse use.\n\n> Change the fsck_walk_func to use an \"enum object_type\" instead of an\n> \"int\" type. The types are compatible, and ever since this was added in\n> 355885d5315 (add generic, type aware object chain walker, 2008-02-25)\n> we've used entries from object_type (OBJ_BLOB etc.).\n>\n> So this doesn't really change anything as far as the generated code is\n> concerned, it just gives the compiler more information and makes this\n> easier to read.\n\nYup, as long as we won't trick the compiler into complaining \"ah,\nbut you are not covering OBJ_OFS_DELTA or OBJ_BAD values in your\nswitch statement\", I think a change like this is a good thing.\n\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> builtin/fsck.c | 3 ++-\n> builtin/index-pack.c | 3 ++-\n> builtin/unpack-objects.c | 3 ++-\n> fsck.h | 3 ++-\n> 4 files changed, 8 insertions(+), 4 deletions(-)\n>\n> diff --git a/builtin/fsck.c b/builtin/fsck.c\n> index 821e7798c70..68f0329e69e 100644\n> --- a/builtin/fsck.c\n> +++ b/builtin/fsck.c\n> @@ -197,7 +197,8 @@ static int traverse_reachable(void)\n> \treturn !!result;\n> }\n> \n> -static int mark_used(struct object *obj, int type, void *data, struct fsck_options *options)\n> +static int mark_used(struct object *obj, enum object_type object_type,\n> +\t\t void *data, struct fsck_options *options)\n> {\n> \tif (!obj)\n> \t\treturn 1;\n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index 54f74c48741..2f291a14d4a 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -212,7 +212,8 @@ static void cleanup_thread(void)\n> \tfree(thread_data);\n> }\n> \n> -static int mark_link(struct object *obj, int type, void *data, struct fsck_options *options)\n> +static int mark_link(struct object *obj, enum object_type type,\n> +\t\t void *data, struct fsck_options *options)\n> {\n> \tif (!obj)\n> \t\treturn -1;\n> diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\n> index dd4a75e030d..ca54fd16688 100644\n> --- a/builtin/unpack-objects.c\n> +++ b/builtin/unpack-objects.c\n> @@ -187,7 +187,8 @@ static void write_cached_object(struct object *obj, struct obj_buffer *obj_buf)\n> * that have reachability requirements and calls this function.\n> * Verify its reachability and validity recursively and write it out.\n> */\n> -static int check_object(struct object *obj, int type, void *data, struct fsck_options *options)\n> +static int check_object(struct object *obj, enum object_type type,\n> +\t\t\tvoid *data, struct fsck_options *options)\n> {\n> \tstruct obj_buffer *obj_buf;\n> \n> diff --git a/fsck.h b/fsck.h\n> index df0b64a2163..0c75789d219 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -23,7 +23,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type);\n> * <0\terror signaled and abort\n> * >0\terror signaled and do not abort\n> */\n> -typedef int (*fsck_walk_func)(struct object *obj, int type, void *data, struct fsck_options *options);\n> +typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n> +\t\t\t void *data, struct fsck_options *options);\n> \n> /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n> typedef int (*fsck_error)(struct fsck_options *o,\n"},{"id":"417250","messageId":"87zh028ctp.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"xmqqo8gijtl7.fsf@gitster.g","subject":"Re: [PATCH 00/14] fsck: API improvements","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T00:00:18Z","receivedAt":"2021-02-18T00:01:10Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Feb 17 2021, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> Jonathan Tan pointed out that the fsck error_func doesn't pass you the\n>> ID of the fsck failure in [1]. This series improves the API so it\n>> does, and moves the gitmodules_{found,done} variables into the\n>> fsck_options struct.\n>>\n>> The result is that instead of the \"print_dangling_gitmodules\" member\n>> in that series we can just implement that with the diff at the end of\n>> this cover letter (goes on top of a merge of this series & \"seen\"),\n>> and without any changes to fsck_finish().\n>>\n>> This conflicts with other in-flight fsck changes but the conflict is\n>> rather trivial. Jeff King has another concurrent series to add a\n>> couple of new fsck checks, those need to be moved to fsck.h, and\n>> there's another trivial conflict in 2 hunks due to the\n>> gitmodules_{found,done} move.\n>>\n>> 1. https://lore.kernel.org/git/87blcja2ha.fsf@evledraar.gmail.com/\n>\n> Let's get this reviewed now, but with expectation that it will be\n> rebased after the dust settles.\n\nMakes sense. Pending a review of this would you be interested in queuing\na v2 of this that doesn't conflict with in-flight topics?\n\nPatches 01..09 & 13/14 can live conflict-free with what's in \"seen\" now\n(I'd have made the 13th the 10th in v1 if I'd noticed). Then I could\nre-roll the remainder of this once the other topics land.\n"},{"id":"417295","messageId":"20210218105840.11989-4-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 03/10] fsck.c: rename variables in fsck_set_msg_type() for less confusion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:33Z","receivedAt":"2021-02-18T12:27:31Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename variables in a function added in 0282f4dced0 (fsck: offer a\nfunction to demote fsck errors to warnings, 2015-06-22).\n\nIt was needlessly confusing that it took a \"msg_type\" argument, but\nthen later declared another \"msg_type\" of a different type.\n\nLet's rename that to \"tmp\", and rename \"id\" to \"msg_id\" and \"msg_id\"\nto \"msg_id_str\" etc. This will make a follow-up change smaller.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 24 ++++++++++++------------\n 1 file changed, 12 insertions(+), 12 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 4b7f0b73d73..acccad243ec 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -203,27 +203,27 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n }\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type)\n+\t\tconst char *msg_id_str, const char *msg_type_str)\n {\n-\tint id = parse_msg_id(msg_id), type;\n+\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n \n-\tif (id < 0)\n-\t\tdie(\"Unhandled message id: %s\", msg_id);\n-\ttype = parse_msg_type(msg_type);\n+\tif (msg_id < 0)\n+\t\tdie(\"Unhandled message id: %s\", msg_id_str);\n+\tmsg_type = parse_msg_type(msg_type_str);\n \n-\tif (type != FSCK_ERROR && msg_id_info[id].msg_type == FSCK_FATAL)\n-\t\tdie(\"Cannot demote %s to %s\", msg_id, msg_type);\n+\tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n+\t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *msg_type;\n-\t\tALLOC_ARRAY(msg_type, FSCK_MSG_MAX);\n+\t\tint *tmp;\n+\t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tmsg_type[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = msg_type;\n+\t\t\ttmp[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = tmp;\n \t}\n \n-\toptions->msg_type[id] = type;\n+\toptions->msg_type[msg_id] = msg_type;\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417296","messageId":"20210218105840.11989-6-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 05/10] fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:35Z","receivedAt":"2021-02-18T12:28:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the remaining variables of type fsck_msg_id from \"id\" to\n\"msg_id\". This change is relatively small, and is worth the churn for\na later change where we have different id's in the \"report\" function.\n---\n fsck.c | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 1070071ffec..dbb6f7c4ee2 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,19 +264,19 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id msg_id)\n {\n-\tconst char *msg_id = msg_id_info[id].id_string;\n+\tconst char *msg_id_str = msg_id_info[msg_id].id_string;\n \tfor (;;) {\n-\t\tchar c = *(msg_id)++;\n+\t\tchar c = *(msg_id_str)++;\n \n \t\tif (!c)\n \t\t\tbreak;\n \t\tif (c != '_')\n \t\t\tstrbuf_addch(sb, tolower(c));\n \t\telse {\n-\t\t\tassert(*msg_id);\n-\t\t\tstrbuf_addch(sb, *(msg_id)++);\n+\t\t\tassert(*msg_id_str);\n+\t\t\tstrbuf_addch(sb, *(msg_id_str)++);\n \t\t}\n \t}\n \n@@ -292,11 +292,11 @@ static int object_on_skiplist(struct fsck_options *opts,\n __attribute__((format (printf, 5, 6)))\n static int report(struct fsck_options *options,\n \t\t const struct object_id *oid, enum object_type object_type,\n-\t\t enum fsck_msg_id id, const char *fmt, ...)\n+\t\t enum fsck_msg_id msg_id, const char *fmt, ...)\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(id, options), result;\n+\tint msg_type = fsck_msg_type(msg_id, options), result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -309,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, id);\n+\tappend_msg_id(&sb, msg_id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417297","messageId":"20210218105840.11989-10-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 09/10] fsck.c: give \"FOREACH_MSG_ID\" a more specific name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:39Z","receivedAt":"2021-02-18T12:29:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the FOREACH_MSG_ID macro to FOREACH_FSCK_MSG_ID in preparation\nfor moving it over to fsck.h. It's good convention to name macros\nin *.h files in such a way as to clearly not clash with any other\nnames in other files.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 88884e91c89..1730acd698d 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,7 +22,7 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_MSG_ID(FUNC) \\\n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n \tFUNC(UNTERMINATED_HEADER, FATAL) \\\n@@ -83,7 +83,7 @@ static struct oidset gitmodules_done = OIDSET_INIT;\n \n #define MSG_ID(id, msg_type) FSCK_MSG_##id,\n enum fsck_msg_id {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \tFSCK_MSG_MAX\n };\n #undef MSG_ID\n@@ -96,7 +96,7 @@ static struct {\n \tconst char *camelcased;\n \tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417298","messageId":"20210218105840.11989-7-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 06/10] fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:36Z","receivedAt":"2021-02-18T12:30:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} defines into a new\nfsck_msg_type enum.\n\nThese defines were originally introduced in:\n\n - ba002f3b28a (builtin-fsck: move common object checking code to\n fsck.c, 2008-02-25)\n - f50c4407305 (fsck: disallow demoting grave fsck errors to warnings,\n 2015-06-22)\n - efaba7cc77f (fsck: optionally ignore specific fsck issues\n completely, 2015-06-22)\n - f27d05b1704 (fsck: allow upgrading fsck warnings to errors,\n 2015-06-22)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 2 +-\n builtin/mktag.c | 3 ++-\n fsck.c | 21 ++++++++++-----------\n fsck.h | 17 ++++++++++-------\n 4 files changed, 23 insertions(+), 20 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 68f0329e69e..d6d745dc702 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -89,7 +89,7 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t int msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type, const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 41a399a69e4..1834394a9b6 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -22,7 +22,8 @@ static int mktag_config(const char *var, const char *value, void *cb)\n static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/fsck.c b/fsck.c\nindex dbb6f7c4ee2..00e0fef21ca 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,9 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FSCK_FATAL -1\n-#define FSCK_INFO -2\n-\n #define FOREACH_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n@@ -97,7 +94,7 @@ static struct {\n \tconst char *id_string;\n \tconst char *downcased;\n \tconst char *camelcased;\n-\tint msg_type;\n+\tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n \tFOREACH_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n@@ -164,10 +161,10 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n \t\tlist_config_item(list, prefix, msg_id_info[i].camelcased);\n }\n \n-static int fsck_msg_type(enum fsck_msg_id msg_id,\n+static enum fsck_msg_type fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n-\tint msg_type;\n+\tenum fsck_msg_type msg_type;\n \n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n@@ -182,7 +179,7 @@ static int fsck_msg_type(enum fsck_msg_id msg_id,\n \treturn msg_type;\n }\n \n-static int parse_msg_type(const char *str)\n+static enum fsck_msg_type parse_msg_type(const char *str)\n {\n \tif (!strcmp(str, \"error\"))\n \t\treturn FSCK_ERROR;\n@@ -205,7 +202,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n-\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n+\tint msg_id = parse_msg_id(msg_id_str);\n+\tenum fsck_msg_type msg_type;\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n@@ -216,7 +214,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *tmp;\n+\t\tenum fsck_msg_type *tmp;\n \t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n \t\t\ttmp[i] = fsck_msg_type(i, options);\n@@ -296,7 +294,8 @@ static int report(struct fsck_options *options,\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(msg_id, options), result;\n+\tenum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);\n+\tint result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -1262,7 +1261,7 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tint msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type, const char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex 0c75789d219..c77e8ddf10b 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -3,10 +3,13 @@\n \n #include \"oidset.h\"\n \n-#define FSCK_ERROR 1\n-#define FSCK_WARN 2\n-#define FSCK_IGNORE 3\n-\n+enum fsck_msg_type {\n+\tFSCK_INFO = -2,\n+\tFSCK_FATAL = -1,\n+\tFSCK_ERROR = 1,\n+\tFSCK_WARN,\n+\tFSCK_IGNORE\n+};\n struct fsck_options;\n struct object;\n \n@@ -29,17 +32,17 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t int msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tint msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, const char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n \tfsck_error error_func;\n \tunsigned strict:1;\n-\tint *msg_type;\n+\tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n \tkh_oid_map_t *object_names;\n };\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417299","messageId":"20210218105840.11989-11-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 10/10] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:40Z","receivedAt":"2021-02-18T12:30:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add the object_name member to the initialization macro. This was\nomitted in 7b35efd734e (fsck_walk(): optionally name objects on the\ngo, 2016-07-17) when the field was added.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex c77e8ddf10b..5d44ff1c8e3 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -47,8 +47,8 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n-#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n+#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n+#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417300","messageId":"20210218105840.11989-9-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 08/10] fsck.c: undefine temporary STR macro after use","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:38Z","receivedAt":"2021-02-18T12:31:01Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In f417eed8cde (fsck: provide a function to parse fsck message IDs,\n2015-06-22) the \"STR\" macro was introduced, but that short macro name\nwas not undefined after use as was done earlier in the same series for\nthe MSG_ID macro in c99ba492f1c (fsck: introduce identifiers for fsck\nmessages, 2015-06-22).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/fsck.c b/fsck.c\nindex 7c53080ad48..88884e91c89 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -100,6 +100,7 @@ static struct {\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n+#undef STR\n \n static void prepare_msg_ids(void)\n {\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417301","messageId":"20210218105840.11989-8-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 07/10] fsck.c: call parse_msg_type() early in fsck_set_msg_type()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:37Z","receivedAt":"2021-02-18T12:31:59Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"There's no reason to defer the calling of parse_msg_type() until after\nwe've checked if the \"id < 0\". This is not a hot codepath, and\nparse_msg_type() itself may die on invalid input.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 00e0fef21ca..7c53080ad48 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -203,11 +203,10 @@ void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n \tint msg_id = parse_msg_id(msg_id_str);\n-\tenum fsck_msg_type msg_type;\n+\tenum fsck_msg_type msg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n-\tmsg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417302","messageId":"20210218105840.11989-5-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 04/10] fsck.c: move definition of msg_id into append_msg_id()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:34Z","receivedAt":"2021-02-18T12:37:11Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor code added in 71ab8fa840f (fsck: report the ID of the\nerror/warning, 2015-06-22) to resolve the msg_id to a string in the\nfunction that wants it, instead of doing it in report().\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex acccad243ec..1070071ffec 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,8 +264,9 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, const char *msg_id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n {\n+\tconst char *msg_id = msg_id_info[id].id_string;\n \tfor (;;) {\n \t\tchar c = *(msg_id)++;\n \n@@ -308,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, msg_id_info[id].id_string);\n+\tappend_msg_id(&sb, id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417303","messageId":"20210218105840.11989-2-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 01/10] fsck.h: indent arguments to of fsck_set_msg_type","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:31Z","receivedAt":"2021-02-18T12:37:14Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Signed-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 423c467feb7..df0b64a2163 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -11,7 +11,7 @@ struct fsck_options;\n struct object;\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type);\n+\t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n int is_valid_msg_type(const char *msg_id, const char *msg_type);\n \n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417304","messageId":"20210218105840.11989-3-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 02/10] fsck.h: use \"enum object_type\" instead of \"int\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:32Z","receivedAt":"2021-02-18T12:37:14Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_walk_func to use an \"enum object_type\" instead of an\n\"int\" type. The types are compatible, and ever since this was added in\n355885d5315 (add generic, type aware object chain walker, 2008-02-25)\nwe've used entries from object_type (OBJ_BLOB etc.).\n\nSo this doesn't really change anything as far as the generated code is\nconcerned, it just gives the compiler more information and makes this\neasier to read.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 3 ++-\n builtin/index-pack.c | 3 ++-\n builtin/unpack-objects.c | 3 ++-\n fsck.h | 3 ++-\n 4 files changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 821e7798c70..68f0329e69e 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -197,7 +197,8 @@ static int traverse_reachable(void)\n \treturn !!result;\n }\n \n-static int mark_used(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_used(struct object *obj, enum object_type object_type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn 1;\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 54f74c48741..2f291a14d4a 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -212,7 +212,8 @@ static void cleanup_thread(void)\n \tfree(thread_data);\n }\n \n-static int mark_link(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_link(struct object *obj, enum object_type type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn -1;\ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex dd4a75e030d..ca54fd16688 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -187,7 +187,8 @@ static void write_cached_object(struct object *obj, struct obj_buffer *obj_buf)\n * that have reachability requirements and calls this function.\n * Verify its reachability and validity recursively and write it out.\n */\n-static int check_object(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int check_object(struct object *obj, enum object_type type,\n+\t\t\tvoid *data, struct fsck_options *options)\n {\n \tstruct obj_buffer *obj_buf;\n \ndiff --git a/fsck.h b/fsck.h\nindex df0b64a2163..0c75789d219 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -23,7 +23,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type);\n * <0\terror signaled and abort\n * >0\terror signaled and do not abort\n */\n-typedef int (*fsck_walk_func)(struct object *obj, int type, void *data, struct fsck_options *options);\n+typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n+\t\t\t void *data, struct fsck_options *options);\n \n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417305","messageId":"20210218105840.11989-1-avarab@gmail.com","threadId":"54994","inReplyTo":"20210217194246.25342-1-avarab@gmail.com","subject":"[PATCH v2 00/10] fsck: API improvements (no conflicts with 'seen')","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T10:58:30Z","receivedAt":"2021-02-18T12:37:14Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"As suggested in\nhttps://lore.kernel.org/git/87zh028ctp.fsf@evledraar.gmail.com/ a\nversion of this that doesn't conflict with other in-flight topics. I\ncan submit the rest later.\n\nÆvar Arnfjörð Bjarmason (10):\n fsck.h: indent arguments to of fsck_set_msg_type\n fsck.h: use \"enum object_type\" instead of \"int\"\n fsck.c: rename variables in fsck_set_msg_type() for less confusion\n fsck.c: move definition of msg_id into append_msg_id()\n fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n fsck.c: undefine temporary STR macro after use\n fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n fsck.h: update FSCK_OPTIONS_* for object_name\n\n builtin/fsck.c | 5 ++--\n builtin/index-pack.c | 3 +-\n builtin/mktag.c | 3 +-\n builtin/unpack-objects.c | 3 +-\n fsck.c | 60 ++++++++++++++++++++--------------------\n fsck.h | 26 +++++++++--------\n 6 files changed, 54 insertions(+), 46 deletions(-)\n\nRange-diff:\n -: ----------- > 1: 88b347b74ed fsck.h: indent arguments to of fsck_set_msg_type\n 1: 1a60d65d2ca ! 2: 868eac3d4d1 fsck.h: use use \"enum object_type\" instead of \"int\"\n @@ Metadata\n Author: Ævar Arnfjörð Bjarmason \n \n ## Commit message ##\n - fsck.h: use use \"enum object_type\" instead of \"int\"\n + fsck.h: use \"enum object_type\" instead of \"int\"\n \n Change the fsck_walk_func to use an \"enum object_type\" instead of an\n \"int\" type. The types are compatible, and ever since this was added in\n 2: 24761f269b7 = 3: f599dc6c8f3 fsck.c: rename variables in fsck_set_msg_type() for less confusion\n 3: fb4c66f9305 = 4: 33f3b1942c1 fsck.c: move definition of msg_id into append_msg_id()\n 4: a129dbd9964 = 5: 28c9245e418 fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n 5: d9bee41072e = 6: d25037c6f18 fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n 6: 423568026c3 = 7: 66d0f1047cc fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n 7: cb43e832738 = 8: 7643a5bf211 fsck.c: undefine temporary STR macro after use\n 8: 2cd14cb4e2a = 9: 7c64e2267ce fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n 9: 1ada154ef23 < -: ----------- fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n10: c4179445f22 < -: ----------- fsck.c: pass along the fsck_msg_id in the fsck_error callback\n11: c1fc724f0e8 < -: ----------- fsck.c: add an fsck_set_msg_type() API that takes enums\n12: 8de91fac068 = 10: a98a3512629 fsck.h: update FSCK_OPTIONS_* for object_name\n13: 29ff97856ff < -: ----------- fsck.c: move gitmodules_{found,done} into fsck_options\n-- \n2.30.0.284.gd98b1dd5eaa7\n\n"},{"id":"417306","messageId":"87r1ld8vdv.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"20210128003536.3874866-1-jonathantanmy@google.com","subject":"Re: [PATCH 0/4] Check .gitmodules when using packfile URIs","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T11:31:40Z","receivedAt":"2021-02-18T12:37:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jan 28 2021, Jonathan Tan wrote:\n\n>> Jonathan Tan writes:\n>> \n>> > As part of this, index-pack has to output (1) the hash that goes into\n>> > the name of the .pack/.idx file and (2) the hashes of all dangling\n>> > .gitmodules. I just had (2) come after (1). If anyone has a better idea,\n>> > I'm interested.\n>> \n>> I have this feeling that the \"blobs that need to be validated across\n>> packs\" will *not* be the last enhancement we'd need to make to the\n>> output from index-pack to allow richer communication between it and\n>> its invoker. While there is no reason to change how the first line\n>> of the output looks like, we'd probably want to make sure that the\n>> future versions of Git can easily tell \"list of blobs that require\n>> further validation\" from other additional information.\n>> \n>> I am not comfortable to recommend \"ok, then let's add a delimiter\n>> line '---\\n' if/when we need to have something after the list of\n>> blobs and append more stuff in future versions of Git\", because we\n>> may find need to emit new kinds of info before the list of blobs\n>> that needs further validation, for example, in future versions of\n>> Git.\n>> \n>> Having said all that, the internal communication between the\n>> index-pack and its caller do not need as much care about\n>> compatibility across versions as output visible to end-users, so\n>> when a future version of Git needs to send different kinds of\n>> information in different order from what you created here, we can do\n>> so pretty much freely, I would guess.\n>\n> Yeah, that's what I thought too - since this is an internal interface,\n> we can evolve them in lockstep. If we're really worried about the Git\n> binaries (on a user's system) getting out of sync, \n\nI'm thinking in reading \"getting out of sync\" that you may be missing an\naspect of the issue here.\n\nWe're not talking about some abnormal error in some packaging system,\nbut how we'd expect all installations of git to behave if you update\nthem with *.rpm, *.deb etc, e.g. when your binaries are in\n/usr/libexec/git-core. I suppose NixOS or something where there's\nhash-based paths may be exempt from this.\n\nOn those systems if you've got a server serving concurrent traffic and\nupdate the \"git\" package you could expect failure if any git process\ninvoked by another is incompatible during such an upgrade.\n\nIf you browse some of the recent GIT_CONFIG_PARAMETERS discussion this\nwas discussed there. I.e. even if GIT_CONFIG_PARAMETERS is internal-only\nwe bent over backwards not to change it in such a way as to have process\nA invoking process B and the two not understanding each other because of\nsuch an upgrade.\n\nThat's exactly because of this case, where receive-pack may be started\non version A, someone runs \"apt install git\" in the background\nconcurrently, and now a version A of that program is talking to a\nversion B index-pack.\n\n> we could just make sure that subsequent updates to this protocol are\n> non-backwards-compatible (e.g. have index-pack emit \"foo \",\n> where \"foo\" is a string that describes the new check, so that current\n> fetch-pack will reject \"foo\" since it is not a hash).\n\nAnd then presumably index-pack would die and receive-pack would die on\nthe push or whatever, so the push fails for the end user.\n"},{"id":"417307","messageId":"87o8gh8tq5.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"20210217201006.877438-1-jonathantanmy@google.com","subject":"Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-18T12:07:30Z","receivedAt":"2021-02-18T12:37:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Feb 17 2021, Jonathan Tan wrote:\n\n>> Sorry for being unclear here. I don't think (honestly I don't remember,\n>> it's been almost a month) that I meant to you should use the skipList.\n>> \n>> Looking at that code again we use object_on_skiplist() to do an early\n>> punt in report(), but also fsck_blob(), presumably you never want the\n>> latter, and that early punting wouldn't be needed if your report()\n>> function intercepted the modules blob id for stashing it away / later\n>> reporting / whatever.\n>> \n>> So yeah, I'm 99% sure now that's not what I meant :)\n>> \n>> What I meant with:\n>> \n>> Or if we want to keep the \"print | process\"[...]\n>> \n>> Is that we have an existing ad-hoc IPC model for these commands in\n>> passing along the skipList, which is made more complex because sometimes\n>> the initial process reads the file, sometimes it passes it along as-is\n>> to the child.\n>> \n>> And then there's this patch that passes OIDs too, but through a\n>> different mechanism.\n>> \n>> I was suggesting that perhaps it made more sense to refactor both so\n>> they could use the same mechanism, because we're potentially passing two\n>> lists of OIDs between the two. Just one goes via line-at-a-time in the\n>> output, the other via a config option on the command-line.\n>\n> Thanks for your explanation. I still think that they are quite different\n> - skiplist is a user-written file containing a list of OIDs that will\n> likely never change, whereas my list of dangling .gitmodules is a list\n> of OIDs dynamically generated (and thus, always different) whenever a\n> fetch is done. So I think it's quite reasonable to pass skiplist as a\n> file name, and my list should be passed line-by-line.\n\nSure, but I'm not talking about passing it as a tempfile.\n\nYes, I suggested that in the third-to-last paragraph of [1] but then\nwent on to say that we could also move to some IPC mechanism where you\nspew in the list of dangling .gitmodules, and we also spew in the\nskipList and anything else we want to pass in.\n\nI'm not saying this needs to be part of this series. But let me\nrephrase:\n\nWe now have some combination of\n{receive-pack,upload-pack,send-pack,fetch-pack,unpack-objects} that need\nto communicate locally or pass data back & forth, passing data either\nvia a CLI option to read a file, packnames/refs on --stdin, or (now) a\nsingle list of OIDs on stdout.\n\nLet's say we don't just need to pass the .gitmodules OIDs, but also\ne.g. .mailmap OIDs or whatever (due to some future vulnerability).\n\nWould this IPC mechanism deal with that, or would we need to introduce a\nbreaking change (Re: my recently send mail about concurrent updates of\nlibexec programs)? Can we use soemething like pkt-line to talk back &\nforth in an extensible way?\n\nNot needed now, just food for thought...\n\n1. https://lore.kernel.org/git/87czxu7c15.fsf@evledraar.gmail.com/\n"},{"id":"417339","messageId":"xmqqtuq9faw5.fsf@gitster.g","threadId":"54994","inReplyTo":"87zh028ctp.fsf@evledraar.gmail.com","subject":"Re: [PATCH 00/14] fsck: API improvements","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T19:12:26Z","receivedAt":"2021-02-18T19:33:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n>> Let's get this reviewed now, but with expectation that it will be\n>> rebased after the dust settles.\n>\n> Makes sense. Pending a review of this would you be interested in queuing\n> a v2 of this that doesn't conflict with in-flight topics?\n\nNot really. I am not sure your recent patches are getting\nsufficient review bandwidth they deserve.\n\n> Patches 01..09 & 13/14 can live conflict-free with what's in \"seen\" now\n> (I'd have made the 13th the 10th in v1 if I'd noticed). Then I could\n> re-roll the remainder of this once the other topics land.\n"},{"id":"417346","messageId":"YC7D6j0A9AW0X9Z9@coredump.intra.peff.net","threadId":"54994","inReplyTo":"20210218105840.11989-4-avarab@gmail.com","subject":"Re: [PATCH v2 03/10] fsck.c: rename variables in fsck_set_msg_type() for less confusion","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-02-18T19:45:46Z","receivedAt":"2021-02-18T19:50:28Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 18, 2021 at 11:58:33AM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> Rename variables in a function added in 0282f4dced0 (fsck: offer a\n> function to demote fsck errors to warnings, 2015-06-22).\n> \n> It was needlessly confusing that it took a \"msg_type\" argument, but\n> then later declared another \"msg_type\" of a different type.\n> \n> Let's rename that to \"tmp\", and rename \"id\" to \"msg_id\" and \"msg_id\"\n> to \"msg_id_str\" etc. This will make a follow-up change smaller.\n\nI think this is an improvement, though maybe \"severity\" would be a\nless-generic term than \"type\".\n\n> void fsck_set_msg_type(struct fsck_options *options,\n> -\t\tconst char *msg_id, const char *msg_type)\n> +\t\tconst char *msg_id_str, const char *msg_type_str)\n> {\n> -\tint id = parse_msg_id(msg_id), type;\n> +\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n\nI always get nervous when a refactoring renames something away from\n\"foo\", and then renames another thing _to_ \"foo\". Any untouched bits of\ncode are vulnerable to confusing them.\n\nBut I think the types are sufficiently different that we can mostly rely\non the compiler (though things like numeric or bool comparisons can work\nwith either pointers or ints), and the fact that we can see the entire\nfunction is small enough that we can see the entire thing in the context\nhere.\n\nSo I think it is OK.\n\n-Peff\n"},{"id":"417347","messageId":"YC7FcTnduQiaF+W5@coredump.intra.peff.net","threadId":"54994","inReplyTo":"20210218105840.11989-7-avarab@gmail.com","subject":"Re: [PATCH v2 06/10] fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-02-18T19:52:17Z","receivedAt":"2021-02-18T19:55:50Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 18, 2021 at 11:58:36AM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> Move the FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} defines into a new\n> fsck_msg_type enum.\n\nMakes sense. As with my previous comment, I wonder if \"severity\" is a\nmore descriptive term.\n\n> diff --git a/fsck.h b/fsck.h\n> index 0c75789d219..c77e8ddf10b 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -3,10 +3,13 @@\n> \n> #include \"oidset.h\"\n> \n> -#define FSCK_ERROR 1\n> -#define FSCK_WARN 2\n> -#define FSCK_IGNORE 3\n> -\n> +enum fsck_msg_type {\n> +\tFSCK_INFO = -2,\n> +\tFSCK_FATAL = -1,\n> +\tFSCK_ERROR = 1,\n> +\tFSCK_WARN,\n> +\tFSCK_IGNORE\n> +};\n\nYou kept the values the same as they were before, which is good in a\nrefactoring step, but...wow, the ordering is weird and confusing.\n\nIn FATAL/ERROR/WARN/IGNORE the number increases as severity decreases.\nMaybe reversed from how I'd do it, but at least the order makes sense.\nBut somehow INFO is on the far side of FATAL?\n\nAgain, not something to address in this patch, but I hope something we\ncould maybe deal with in the longer term (perhaps along with fixing the\nweird \"INFO is a warning from the user's perspective, but WARNING is\ngenerally an error\" behavior).\n\nI also know that this is assigning WARN and IGNORE based on\ncounting-by-one from ERROR, so it's correct. But I think it would be\nmore obvious if you simply filled in the values manually, so a reader\ndoes not have to wonder why some are assigned and some are not.\n\n-Peff\n"},{"id":"417349","messageId":"YC7GW8L+nJLFYHpY@coredump.intra.peff.net","threadId":"54994","inReplyTo":"20210218105840.11989-11-avarab@gmail.com","subject":"Re: [PATCH v2 10/10] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-02-18T19:56:11Z","receivedAt":"2021-02-18T19:59:46Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 18, 2021 at 11:58:40AM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> Add the object_name member to the initialization macro. This was\n> omitted in 7b35efd734e (fsck_walk(): optionally name objects on the\n> go, 2016-07-17) when the field was added.\n\nWe're correct either way here, because trailing fields that are not\ninitialized will get the usual zero-initialization. But I don't mind\ntrying to be more complete.\n\nThat said, we have embraced designated initializers these days, in which\ncase we usually omit the NULL ones. So perhaps:\n\n #define FSCK_OPTIONS_DEFAULT { \\\n\t.walk = fsck_error_function, \\\n\t.skiplist = OIDSET_INIT, \\\n }\n #define FSCK_OPTIONS_STRICT { \\\n\t.walk = fsck_error_function, \\\n\t.skiplist = OIDSET_INIT, \\\n\t.strict = 1, \\\n }\n\nwould be more readable still?\n\n-Peff\n"},{"id":"417350","messageId":"YC7GgoVre3hFF1Se@coredump.intra.peff.net","threadId":"54994","inReplyTo":"20210218105840.11989-10-avarab@gmail.com","subject":"Re: [PATCH v2 09/10] fsck.c: give \"FOREACH_MSG_ID\" a more specific name","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-02-18T19:56:50Z","receivedAt":"2021-02-18T20:00:37Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 18, 2021 at 11:58:39AM +0100, Ævar Arnfjörð Bjarmason wrote:\n\n> Rename the FOREACH_MSG_ID macro to FOREACH_FSCK_MSG_ID in preparation\n> for moving it over to fsck.h. It's good convention to name macros\n> in *.h files in such a way as to clearly not clash with any other\n> names in other files.\n\nThe patch to move it is not in this v2 of the series, so arguably this\nis less interesting. However, I think the resulting code is equally or\nmore readable, so I don't mind it standing on its own.\n\n-Peff\n"},{"id":"417351","messageId":"YC7GupBLb4IoDLj6@coredump.intra.peff.net","threadId":"54994","inReplyTo":"xmqqtuq9faw5.fsf@gitster.g","subject":"Re: [PATCH 00/14] fsck: API improvements","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-02-18T19:57:46Z","receivedAt":"2021-02-18T20:01:04Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 18, 2021 at 11:12:26AM -0800, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n> \n> >> Let's get this reviewed now, but with expectation that it will be\n> >> rebased after the dust settles.\n> >\n> > Makes sense. Pending a review of this would you be interested in queuing\n> > a v2 of this that doesn't conflict with in-flight topics?\n> \n> Not really. I am not sure your recent patches are getting\n> sufficient review bandwidth they deserve.\n\nFWIW, I just read through v2 (without having looked at all at v1 yet!),\nand they all seemed like quite reasonable cleanups. I left a few small\ncomments that might be worth a quick re-roll, but I would also be OK\nwith the patches being picked up as-is.\n\n-Peff\n"},{"id":"417355","messageId":"xmqqtuq9dsu3.fsf@gitster.g","threadId":"54994","inReplyTo":"YC7GupBLb4IoDLj6@coredump.intra.peff.net","subject":"Re: [PATCH 00/14] fsck: API improvements","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T20:27:48Z","receivedAt":"2021-02-18T20:28:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Feb 18, 2021 at 11:12:26AM -0800, Junio C Hamano wrote:\n>\n>> Ævar Arnfjörð Bjarmason writes:\n>> \n>> >> Let's get this reviewed now, but with expectation that it will be\n>> >> rebased after the dust settles.\n>> >\n>> > Makes sense. Pending a review of this would you be interested in queuing\n>> > a v2 of this that doesn't conflict with in-flight topics?\n>> \n>> Not really. I am not sure your recent patches are getting\n>> sufficient review bandwidth they deserve.\n>\n> FWIW, I just read through v2 (without having looked at all at v1 yet!),\n> and they all seemed like quite reasonable cleanups. I left a few small\n> comments that might be worth a quick re-roll, but I would also be OK\n> with the patches being picked up as-is.\n\nThat's good to hear. I shouldn't even have bothered to answer the\nquestion, if the v2 were to have sent to the list without waiting\nfor my reply ;-)\n"},{"id":"417361","messageId":"xmqq7dn5dnop.fsf@gitster.g","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"Re: [PATCH v2 00/10] fsck: API improvements (no conflicts with 'seen')","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T22:19:02Z","receivedAt":"2021-02-18T22:20:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> As suggested in\n> https://lore.kernel.org/git/87zh028ctp.fsf@evledraar.gmail.com/ a\n> version of this that doesn't conflict with other in-flight topics. I\n> can submit the rest later.\n\nAnd a bystander does not have a clue what this thing is about,\nbeyond that it tweaks fsck API, how urgent it would be, what benefit\nit brings to us?\n\nThat kind of things are expected to be described here.\n\nThe cover letter of v1 does not do much better job, either, but is\nit fair to understand that this primarily is about allowing the\ncallback functions (which handle various problems fsck machinery\nfinds) to learn what error it encountered, so that things like\n\"enumerate missing .gitmodules blobs\" 384c9d1c (fetch-pack: print\nand use dangling .gitmodules, 2021-01-23) wants to do does not have\nto be written by inserting a very narrow custom code into the\ngeneral error reporting codepath, but by customizing the error\nreporting function?\n\nIf so, can we at least say something a bit more specific and\nfocused, than the overly broad \"API improvements\"?\n\nTHanks.\n"},{"id":"417363","messageId":"xmqq35xtdnhf.fsf@gitster.g","threadId":"54994","inReplyTo":"20210218105840.11989-6-avarab@gmail.com","subject":"Re: [PATCH v2 05/10] fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T22:23:24Z","receivedAt":"2021-02-18T22:25:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Rename the remaining variables of type fsck_msg_id from \"id\" to\n> \"msg_id\". This change is relatively small, and is worth the churn for\n> a later change where we have different id's in the \"report\" function.\n> ---\n> fsck.c | 16 ++++++++--------\n> 1 file changed, 8 insertions(+), 8 deletions(-)\n\nUp to this point I have no objections to the patches themselves, but\nthis one is not signed off.\n\n> diff --git a/fsck.c b/fsck.c\n> index 1070071ffec..dbb6f7c4ee2 100644\n> --- a/fsck.c\n> +++ b/fsck.c\n> @@ -264,19 +264,19 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n> \tfree(to_free);\n> }\n> \n> -static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n> +static void append_msg_id(struct strbuf *sb, enum fsck_msg_id msg_id)\n> {\n> -\tconst char *msg_id = msg_id_info[id].id_string;\n> +\tconst char *msg_id_str = msg_id_info[msg_id].id_string;\n> \tfor (;;) {\n> -\t\tchar c = *(msg_id)++;\n> +\t\tchar c = *(msg_id_str)++;\n> \n> \t\tif (!c)\n> \t\t\tbreak;\n> \t\tif (c != '_')\n> \t\t\tstrbuf_addch(sb, tolower(c));\n> \t\telse {\n> -\t\t\tassert(*msg_id);\n> -\t\t\tstrbuf_addch(sb, *(msg_id)++);\n> +\t\t\tassert(*msg_id_str);\n> +\t\t\tstrbuf_addch(sb, *(msg_id_str)++);\n> \t\t}\n> \t}\n> \n> @@ -292,11 +292,11 @@ static int object_on_skiplist(struct fsck_options *opts,\n> __attribute__((format (printf, 5, 6)))\n> static int report(struct fsck_options *options,\n> \t\t const struct object_id *oid, enum object_type object_type,\n> -\t\t enum fsck_msg_id id, const char *fmt, ...)\n> +\t\t enum fsck_msg_id msg_id, const char *fmt, ...)\n> {\n> \tva_list ap;\n> \tstruct strbuf sb = STRBUF_INIT;\n> -\tint msg_type = fsck_msg_type(id, options), result;\n> +\tint msg_type = fsck_msg_type(msg_id, options), result;\n> \n> \tif (msg_type == FSCK_IGNORE)\n> \t\treturn 0;\n> @@ -309,7 +309,7 @@ static int report(struct fsck_options *options,\n> \telse if (msg_type == FSCK_INFO)\n> \t\tmsg_type = FSCK_WARN;\n> \n> -\tappend_msg_id(&sb, id);\n> +\tappend_msg_id(&sb, msg_id);\n> \n> \tva_start(ap, fmt);\n> \tstrbuf_vaddf(&sb, fmt, ap);\n"},{"id":"417364","messageId":"xmqqy2flc8pk.fsf@gitster.g","threadId":"54994","inReplyTo":"YC7FcTnduQiaF+W5@coredump.intra.peff.net","subject":"Re: [PATCH v2 06/10] fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T22:27:51Z","receivedAt":"2021-02-18T22:28:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Feb 18, 2021 at 11:58:36AM +0100, Ævar Arnfjörð Bjarmason wrote:\n>\n>> Move the FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} defines into a new\n>> fsck_msg_type enum.\n>\n> Makes sense. As with my previous comment, I wonder if \"severity\" is a\n> more descriptive term.\n>\n>> diff --git a/fsck.h b/fsck.h\n>> index 0c75789d219..c77e8ddf10b 100644\n>> --- a/fsck.h\n>> +++ b/fsck.h\n>> @@ -3,10 +3,13 @@\n>> \n>> #include \"oidset.h\"\n>> \n>> -#define FSCK_ERROR 1\n>> -#define FSCK_WARN 2\n>> -#define FSCK_IGNORE 3\n>> -\n>> +enum fsck_msg_type {\n>> +\tFSCK_INFO = -2,\n>> +\tFSCK_FATAL = -1,\n>> +\tFSCK_ERROR = 1,\n>> +\tFSCK_WARN,\n>> +\tFSCK_IGNORE\n>> +};\n>\n> You kept the values the same as they were before, which is good in a\n> refactoring step, but...wow, the ordering is weird and confusing.\n>\n> In FATAL/ERROR/WARN/IGNORE the number increases as severity decreases.\n> Maybe reversed from how I'd do it, but at least the order makes sense.\n> But somehow INFO is on the far side of FATAL?\n>\n> Again, not something to address in this patch, but I hope something we\n> could maybe deal with in the longer term (perhaps along with fixing the\n> weird \"INFO is a warning from the user's perspective, but WARNING is\n> generally an error\" behavior).\n>\n> I also know that this is assigning WARN and IGNORE based on\n> counting-by-one from ERROR, so it's correct. But I think it would be\n> more obvious if you simply filled in the values manually, so a reader\n> does not have to wonder why some are assigned and some are not.\n\nI had the same reaction, plus \"Wow, we had FSCK_* constants in two\ndifferent places and without colliding? Have we been lucky?\nDeclaring it in one place, whether we use enum or not (as enum is\nnot very useful in C as a type checking vehicle), makes a lot of\nsense but why does this come this late in the series, instead of\nbeing at the front as a trivial low-hanging fruit?\"\n\nThanks.\n"},{"id":"417365","messageId":"xmqqtuq9c8mv.fsf@gitster.g","threadId":"54994","inReplyTo":"20210218105840.11989-8-avarab@gmail.com","subject":"Re: [PATCH v2 07/10] fsck.c: call parse_msg_type() early in fsck_set_msg_type()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T22:29:28Z","receivedAt":"2021-02-18T22:30:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> There's no reason to defer the calling of parse_msg_type() until after\n> we've checked if the \"id < 0\". This is not a hot codepath, and\n> parse_msg_type() itself may die on invalid input.\n\nThat explains why this change can be done, but does not justify why\nit is a good change. Unlike all the previous steps, I would rather\nsay this is borderline needless churn.\n\nLet's keep reading as the picture may change as we touch more code\naround this area.\n\nThanks.\n\n\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.c | 3 +--\n> 1 file changed, 1 insertion(+), 2 deletions(-)\n>\n> diff --git a/fsck.c b/fsck.c\n> index 00e0fef21ca..7c53080ad48 100644\n> --- a/fsck.c\n> +++ b/fsck.c\n> @@ -203,11 +203,10 @@ void fsck_set_msg_type(struct fsck_options *options,\n> \t\tconst char *msg_id_str, const char *msg_type_str)\n> {\n> \tint msg_id = parse_msg_id(msg_id_str);\n> -\tenum fsck_msg_type msg_type;\n> +\tenum fsck_msg_type msg_type = parse_msg_type(msg_type_str);\n> \n> \tif (msg_id < 0)\n> \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n> -\tmsg_type = parse_msg_type(msg_type_str);\n> \n> \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n> \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n"},{"id":"417366","messageId":"xmqqpn0xc8lp.fsf@gitster.g","threadId":"54994","inReplyTo":"20210218105840.11989-9-avarab@gmail.com","subject":"Re: [PATCH v2 08/10] fsck.c: undefine temporary STR macro after use","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T22:30:10Z","receivedAt":"2021-02-18T22:31:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> In f417eed8cde (fsck: provide a function to parse fsck message IDs,\n> 2015-06-22) the \"STR\" macro was introduced, but that short macro name\n> was not undefined after use as was done earlier in the same series for\n> the MSG_ID macro in c99ba492f1c (fsck: introduce identifiers for fsck\n> messages, 2015-06-22).\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.c | 1 +\n> 1 file changed, 1 insertion(+)\n>\n> diff --git a/fsck.c b/fsck.c\n> index 7c53080ad48..88884e91c89 100644\n> --- a/fsck.c\n> +++ b/fsck.c\n> @@ -100,6 +100,7 @@ static struct {\n> \t{ NULL, NULL, NULL, -1 }\n> };\n> #undef MSG_ID\n> +#undef STR\n\nGood clean-up.\n\n> \n> static void prepare_msg_ids(void)\n> {\n"},{"id":"417368","messageId":"xmqqlfblc8hr.fsf@gitster.g","threadId":"54994","inReplyTo":"20210218105840.11989-11-avarab@gmail.com","subject":"Re: [PATCH v2 10/10] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T22:32:32Z","receivedAt":"2021-02-18T22:33:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Add the object_name member to the initialization macro. This was\n> omitted in 7b35efd734e (fsck_walk(): optionally name objects on the\n> go, 2016-07-17) when the field was added.\n\nThis is more of a Meh to me. If this were to change us to\ndesignated initializers and omit NULL and 0 initialization, it would\nbe more interesting.\n\nThanks.\n\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.h | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/fsck.h b/fsck.h\n> index c77e8ddf10b..5d44ff1c8e3 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -47,8 +47,8 @@ struct fsck_options {\n> \tkh_oid_map_t *object_names;\n> };\n> \n> -#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n> -#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n> +#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n> +#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n> \n> /* descend in all linked child objects\n> * the return value is:\n"},{"id":"417369","messageId":"xmqqh7m9c8gx.fsf@gitster.g","threadId":"54994","inReplyTo":"YC7GW8L+nJLFYHpY@coredump.intra.peff.net","subject":"Re: [PATCH v2 10/10] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T22:33:02Z","receivedAt":"2021-02-18T22:33:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Feb 18, 2021 at 11:58:40AM +0100, Ævar Arnfjörð Bjarmason wrote:\n>\n>> Add the object_name member to the initialization macro. This was\n>> omitted in 7b35efd734e (fsck_walk(): optionally name objects on the\n>> go, 2016-07-17) when the field was added.\n>\n> We're correct either way here, because trailing fields that are not\n> initialized will get the usual zero-initialization. But I don't mind\n> trying to be more complete.\n>\n> That said, we have embraced designated initializers these days, in which\n> case we usually omit the NULL ones. So perhaps:\n>\n> #define FSCK_OPTIONS_DEFAULT { \\\n> \t.walk = fsck_error_function, \\\n> \t.skiplist = OIDSET_INIT, \\\n> }\n> #define FSCK_OPTIONS_STRICT { \\\n> \t.walk = fsck_error_function, \\\n> \t.skiplist = OIDSET_INIT, \\\n> \t.strict = 1, \\\n> }\n>\n> would be more readable still?\n\nAhh, I should probably have read your reviews first before reading\npatches myself ;-)\n\nThanks.\n"},{"id":"417370","messageId":"xmqqczwxc8bw.fsf@gitster.g","threadId":"54994","inReplyTo":"YC7GupBLb4IoDLj6@coredump.intra.peff.net","subject":"Re: [PATCH 00/14] fsck: API improvements","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T22:36:03Z","receivedAt":"2021-02-18T22:36:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Feb 18, 2021 at 11:12:26AM -0800, Junio C Hamano wrote:\n>\n>> Ævar Arnfjörð Bjarmason writes:\n>> \n>> >> Let's get this reviewed now, but with expectation that it will be\n>> >> rebased after the dust settles.\n>> >\n>> > Makes sense. Pending a review of this would you be interested in queuing\n>> > a v2 of this that doesn't conflict with in-flight topics?\n>> \n>> Not really. I am not sure your recent patches are getting\n>> sufficient review bandwidth they deserve.\n>\n> FWIW, I just read through v2 (without having looked at all at v1 yet!),\n> and they all seemed like quite reasonable cleanups. I left a few small\n> comments that might be worth a quick re-roll, but I would also be OK\n> with the patches being picked up as-is.\n\nYeah, all except for a handful minor nits looked good.\n\nThanks for writing and reviewing. Perhaps a final reroll to tie the\nloose ends, or is it just a matter of signing off one of them and\ndroping a couple of other ones (which other ones)?\n\n\n\n"},{"id":"417374","messageId":"xmqqzh01ar2s.fsf@gitster.g","threadId":"54994","inReplyTo":"cover.1611455251.git.jonathantanmy@google.com","subject":"Re: [PATCH 0/4] Check .gitmodules when using packfile URIs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-18T23:34:03Z","receivedAt":"2021-02-18T23:35:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> This patch set resolves the .gitmodules-and-tree-in-separate-packfiles\n> issue I mentioned in [1] by having index-pack print out all dangling\n> .gitmodules (instead of returning with an error code) and then teaching\n> fetch-pack to read those and run its own fsck checks after all\n> index-pack invocations are complete.\n>\n> As part of this, index-pack has to output (1) the hash that goes into\n> the name of the .pack/.idx file and (2) the hashes of all dangling\n> .gitmodules. I just had (2) come after (1). If anyone has a better idea,\n> I'm interested.\n>\n> I also discovered a bug in that different index-pack arguments were used\n> when processing the inline packfile and when processing the ones\n> referenced by URIs. Patch 1-3 fixes that bug by passing the arguments to\n> use as a space-separated URL-encoded list. (URL-encoded so that we can\n> have spaces in the arguments.) Again, if anyone has a better idea, I'm\n> interested. It is only in patch 4 that we have the dangling .gitmodules\n> fix.\n\nThis seems to have been stalled but I think it would be a better\napproach to use a custom callback for error reporting, suggested by\nÆvar, which would be where his fsck API clean-up topic would lead\nto.\n\nIf it is not ultra-urgent, perhaps you can retract the ones that are\nqueued right now, work with Ævar to finish the error-callback work\nand rebuild this topic on top of it? Thanks.\n"},{"id":"417378","messageId":"20210219004612.1181920-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqqzh01ar2s.fsf@gitster.g","subject":"Re: [PATCH 0/4] Check .gitmodules when using packfile URIs","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-19T00:46:12Z","receivedAt":"2021-02-19T00:47:00Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> This seems to have been stalled but I think it would be a better\n> approach to use a custom callback for error reporting, suggested by\n> Ævar, which would be where his fsck API clean-up topic would lead\n> to.\n> \n> If it is not ultra-urgent, perhaps you can retract the ones that are\n> queued right now, work with Ævar to finish the error-callback work\n> and rebuild this topic on top of it? Thanks.\n\nOK - that works. My original idea was to rewrite it using an\nerror-callback but using starts_with() instead of the ID that Ævar's\nwork will provide, but seeing that at least one other contributor (Peff)\nseems OK with the patches, rebasing mine on top of his works too. I'll\nalso take a look at his patches.\n"},{"id":"417379","messageId":"87lfbk98sq.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"xmqqtuq9dsu3.fsf@gitster.g","subject":"Re: [PATCH 00/14] fsck: API improvements","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-19T00:54:13Z","receivedAt":"2021-02-19T00:54:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Feb 18 2021, Junio C Hamano wrote:\n\n> Jeff King writes:\n>\n>> On Thu, Feb 18, 2021 at 11:12:26AM -0800, Junio C Hamano wrote:\n>>\n>>> Ævar Arnfjörð Bjarmason writes:\n>>> \n>>> >> Let's get this reviewed now, but with expectation that it will be\n>>> >> rebased after the dust settles.\n>>> >\n>>> > Makes sense. Pending a review of this would you be interested in queuing\n>>> > a v2 of this that doesn't conflict with in-flight topics?\n>>> \n>>> Not really. I am not sure your recent patches are getting\n>>> sufficient review bandwidth they deserve.\n>>\n>> FWIW, I just read through v2 (without having looked at all at v1 yet!),\n>> and they all seemed like quite reasonable cleanups. I left a few small\n>> comments that might be worth a quick re-roll, but I would also be OK\n>> with the patches being picked up as-is.\n>\n> That's good to hear. I shouldn't even have bothered to answer the\n> question, if the v2 were to have sent to the list without waiting\n> for my reply ;-)\n\nFWIW it's not that I didn't care about the reply, but I'm somewhat\nintermittently available time/network wise in the coming days. And\nthere's the TZ difference between us.\n\nI sent v1 thinking you might be willing to pick it up & resolve the\nconflict, but since you expressed an interest in deferring it until\nconflicting work landed figured I'd ask (and then just sent the patches)\nif you'd be interested in a conflict-free version to queue alongside\nthose changes.\n\nIf it was still \"nah\" fair enough, I'd just wait. But if not those\npatches would be there to pickup.\n\nThanks a lot to you & Jeff for the review on v2. I won't have time to\naddress all that today, and in any case I got the message that maybe I\nshould stop firehosing the list with patch series's for a bit :)\n"},{"id":"417380","messageId":"87im6o984g.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"xmqqzh01ar2s.fsf@gitster.g","subject":"Re: [PATCH 0/4] Check .gitmodules when using packfile URIs","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-19T01:08:47Z","receivedAt":"2021-02-19T01:13:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Feb 19 2021, Junio C Hamano wrote:\n\n> Jonathan Tan writes:\n>\n>> This patch set resolves the .gitmodules-and-tree-in-separate-packfiles\n>> issue I mentioned in [1] by having index-pack print out all dangling\n>> .gitmodules (instead of returning with an error code) and then teaching\n>> fetch-pack to read those and run its own fsck checks after all\n>> index-pack invocations are complete.\n>>\n>> As part of this, index-pack has to output (1) the hash that goes into\n>> the name of the .pack/.idx file and (2) the hashes of all dangling\n>> .gitmodules. I just had (2) come after (1). If anyone has a better idea,\n>> I'm interested.\n>>\n>> I also discovered a bug in that different index-pack arguments were used\n>> when processing the inline packfile and when processing the ones\n>> referenced by URIs. Patch 1-3 fixes that bug by passing the arguments to\n>> use as a space-separated URL-encoded list. (URL-encoded so that we can\n>> have spaces in the arguments.) Again, if anyone has a better idea, I'm\n>> interested. It is only in patch 4 that we have the dangling .gitmodules\n>> fix.\n>\n> This seems to have been stalled but I think it would be a better\n> approach to use a custom callback for error reporting, suggested by\n> Ævar, which would be where his fsck API clean-up topic would lead\n> to.\n>\n> If it is not ultra-urgent, perhaps you can retract the ones that are\n> queued right now, work with Ævar to finish the error-callback work\n> and rebuild this topic on top of it? Thanks.\n\nIf my vote counts for something I think it makes sense to have\nJonathan's series go first and just ignore my fsck API improvement\npatches (well, the part of my v1[1] which conflicts with his work).\n\nI'm also happy to help him queue his on top of a v1 version of my\nseries.\n\nBut the end result of doing so (shown after the \"--\" in [1]) is just a\nsmall re-arrangement of code to get a cleaner fsck API use, it doesn't\nactually matter to anyone using git.\n\nWhereas his patches actually do, we have in-the-wild server/repo/clone\nsetups that are getting on-clone errors, and the window for 2.31 is\ngetting closer.\n\nWe can always do the small API use refactoring later. My interest in\nbarking up that tree was just that I've been poking at that part of the\nfsck API and have some follow-up work that hasn't made it onto the list\nyet that makes other use of the fsck API.\n\nSo in the longer term I wanted us to think about not needing N special\ncases like \"print_dangling_gitmodules\" if we could help it, but in the\nshorter term having it is a non-issue.\n\n1. https://lore.kernel.org/git/20210217194246.25342-1-avarab@gmail.com/\n"},{"id":"417428","messageId":"xmqqr1lba02w.fsf@gitster.g","threadId":"54994","inReplyTo":"87im6o984g.fsf@evledraar.gmail.com","subject":"Re: [PATCH 0/4] Check .gitmodules when using packfile URIs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-20T03:29:27Z","receivedAt":"2021-02-20T03:30:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Fri, Feb 19 2021, Junio C Hamano wrote:\n>\n>> This seems to have been stalled but I think it would be a better\n>> approach to use a custom callback for error reporting, suggested by\n>> Ævar, which would be where his fsck API clean-up topic would lead\n>> to.\n>>\n>> If it is not ultra-urgent, perhaps you can retract the ones that are\n>> queued right now, work with Ævar to finish the error-callback work\n>> and rebuild this topic on top of it? Thanks.\n>\n> If my vote counts for something I think it makes sense to have\n> Jonathan's series go first and just ignore my fsck API improvement\n> patches (well, the part of my v1[1] which conflicts with his work).\n>\n> I'm also happy to help him queue his on top of a v1 version of my\n> series.\n\nEither would work for us, I would think.\n\nThanks.\n"},{"id":"417429","messageId":"xmqqmtvza007.fsf@gitster.g","threadId":"54994","inReplyTo":"20210219004612.1181920-1-jonathantanmy@google.com","subject":"Re: [PATCH 0/4] Check .gitmodules when using packfile URIs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-20T03:31:04Z","receivedAt":"2021-02-20T03:31:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n>> This seems to have been stalled but I think it would be a better\n>> approach to use a custom callback for error reporting, suggested by\n>> Ævar, which would be where his fsck API clean-up topic would lead\n>> to.\n>> \n>> If it is not ultra-urgent, perhaps you can retract the ones that are\n>> queued right now, work with Ævar to finish the error-callback work\n>> and rebuild this topic on top of it? Thanks.\n>\n> OK - that works. My original idea was to rewrite it using an\n> error-callback but using starts_with() instead of the ID that Ævar's\n> work will provide, but seeing that at least one other contributor (Peff)\n> seems OK with the patches, rebasing mine on top of his works too. I'll\n> also take a look at his patches.\n\nThanks, either way would work for me, but if the suggested route\nforces you review Ævar's code and work together, that would be a\ngood bonus point ;-)\n\n"},{"id":"417485","messageId":"cover.1614021092.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"20210115234300.350442-1-jonathantanmy@google.com","subject":"[PATCH v2 0/4] Check .gitmodules when using packfile URIs","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-22T19:20:05Z","receivedAt":"2021-02-22T19:25:37Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Here's v2. I think I've addressed all the review comments, including\npassing the index-pack args as separate arguments (to avoid the\nnecessity to somehow encode in order to get rid of spaces), and by using\na custom error function instead of a specific option in fsck.\n\nThis applies on master. I mentioned earlier [1] that I was planning to\nimplement this on Ævar's fsck API improvements, but after looking at the\nlatest v2, I see that it omits patch 11 from v1 (which is the one I\nneed), so what I've done is to use a string check in the meantime.\n\n[1] https://lore.kernel.org/git/20210219004612.1181920-1-jonathantanmy@google.com/\n\nJonathan Tan (4):\n http: allow custom index-pack args\n http-fetch: allow custom index-pack args\n fetch-pack: with packfile URIs, use index-pack arg\n fetch-pack: print and use dangling .gitmodules\n\n Documentation/git-http-fetch.txt | 10 ++-\n Documentation/git-index-pack.txt | 7 ++-\n builtin/index-pack.c | 25 +++++++-\n builtin/receive-pack.c | 2 +-\n fetch-pack.c | 103 ++++++++++++++++++++++++++-----\n fsck.c | 5 ++\n fsck.h | 2 +\n http-fetch.c | 20 +++++-\n http.c | 15 ++---\n http.h | 10 +--\n pack-write.c | 8 ++-\n pack.h | 2 +-\n t/t5550-http-fetch-dumb.sh | 5 +-\n t/t5702-protocol-v2.sh | 58 +++++++++++++++--\n 14 files changed, 227 insertions(+), 45 deletions(-)\n\nRange-diff against v1:\n-: ---------- > 1: b7e376be16 http: allow custom index-pack args\n1: 9fba6c9bcc ! 2: 57220ceb84 http-fetch: allow custom index-pack args\n @@ Documentation/git-http-fetch.txt: commit-id::\n \n --packfile=::\n -\tInstead of a commit id on the command line (which is not expected in\n -+\tFor internal use only. Instead of a commit id on the command line (which is not expected in\n ++\tFor internal use only. Instead of a commit id on the command\n ++\tline (which is not expected in\n \tthis case), 'git http-fetch' fetches the packfile directly at the given\n \tURL and uses index-pack to generate corresponding .idx and .keep files.\n \tThe hash is used to determine the name of the temporary file and is\n @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n \t\t\t (int) the_hash_algo->hexsz,\n \t\t\t packfile_uris.items[i].string);\n -+\t\tstrvec_push(&cmd.args, \"--index-pack-args=index-pack --stdin --keep\");\n ++\t\tstrvec_push(&cmd.args, \"--index-pack-arg=index-pack\");\n ++\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--stdin\");\n ++\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--keep\");\n \t\tstrvec_push(&cmd.args, uri);\n \t\tcmd.git_cmd = 1;\n \t\tcmd.no_stdin = 1;\n @@ http-fetch.c: int cmd_main(int argc, const char **argv)\n \tint packfile = 0;\n \tint nongit;\n \tstruct object_id packfile_hash;\n -+\tconst char *index_pack_args = NULL;\n ++\tstruct strvec index_pack_args = STRVEC_INIT;\n \n \tsetup_git_directory_gently(&nongit);\n \n @@ http-fetch.c: int cmd_main(int argc, const char **argv)\n \t\t\tpackfile = 1;\n \t\t\tif (parse_oid_hex(p, &packfile_hash, &end) || *end)\n \t\t\t\tdie(_(\"argument to --packfile must be a valid hash (got '%s')\"), p);\n -+\t\t} else if (skip_prefix(argv[arg], \"--index-pack-args=\", &p)) {\n -+\t\t\tindex_pack_args = p;\n ++\t\t} else if (skip_prefix(argv[arg], \"--index-pack-arg=\", &p)) {\n ++\t\t\tstrvec_push(&index_pack_args, p);\n \t\t}\n \t\targ++;\n \t}\n @@ http-fetch.c: int cmd_main(int argc, const char **argv)\n \n \tif (packfile) {\n -\t\tfetch_single_packfile(&packfile_hash, argv[arg]);\n -+\t\tstruct strvec encoded = STRVEC_INIT;\n -+\t\tchar **raw;\n -+\t\tint i;\n -+\n -+\t\tif (!index_pack_args)\n ++\t\tif (!index_pack_args.nr)\n +\t\t\tdie(_(\"--packfile requires --index-pack-args\"));\n +\n -+\t\tstrvec_split(&encoded, index_pack_args);\n -+\n -+\t\tCALLOC_ARRAY(raw, encoded.nr + 1);\n -+\t\tfor (i = 0; i < encoded.nr; i++)\n -+\t\t\traw[i] = url_percent_decode(encoded.v[i]);\n -+\n +\t\tfetch_single_packfile(&packfile_hash, argv[arg],\n -+\t\t\t\t (const char **) raw);\n -+\n -+\t\tfor (i = 0; i < encoded.nr; i++)\n -+\t\t\tfree(raw[i]);\n -+\t\tfree(raw);\n -+\t\tstrvec_clear(&encoded);\n ++\t\t\t\t index_pack_args.v);\n +\n \t\treturn 0;\n \t}\n \n -+\tif (index_pack_args)\n ++\tif (index_pack_args.nr)\n +\t\tdie(_(\"--index-pack-args can only be used with --packfile\"));\n +\n \tif (commits_on_stdin) {\n @@ t/t5550-http-fetch-dumb.sh: test_expect_success 'http-fetch --packfile' '\n \tp=$(cd \"$HTTPD_DOCUMENT_ROOT_PATH\"/repo_pack.git && ls objects/pack/pack-*.pack) &&\n -\tgit -C packfileclient http-fetch --packfile=$ARBITRARY \"$HTTPD_URL\"/dumb/repo_pack.git/$p >out &&\n +\tgit -C packfileclient http-fetch --packfile=$ARBITRARY \\\n -+\t\t--index-pack-args=\"index-pack --stdin --keep\" \"$HTTPD_URL\"/dumb/repo_pack.git/$p >out &&\n ++\t\t--index-pack-arg=index-pack --index-pack-arg=--stdin \\\n ++\t\t--index-pack-arg=--keep \\\n ++\t\t\"$HTTPD_URL\"/dumb/repo_pack.git/$p >out &&\n \n \tgrep \"^keep.[0-9a-f]\\{16,\\}$\" out &&\n \tcut -c6- out >packhash &&\n2: 7c3244e79f ! 3: aa87335464 fetch-pack: with packfile URIs, use index-pack arg\n @@ fetch-pack.c: static void write_promisor_file(const char *keep_name,\n - * Pass 1 as \"only_packfile\" if the pack received is the only pack in this\n - * fetch request (that is, if there were no packfile URIs provided).\n + * If packfile URIs were provided, pass a non-NULL pointer to index_pack_args.\n -+ * The string to pass as the --index-pack-args argument to http-fetch will be\n ++ * The strings to pass as the --index-pack-arg arguments to http-fetch will be\n + * stored there. (It must be freed by the caller.)\n */\n static int get_pack(struct fetch_pack_args *args,\n \t\t int xd[2], struct string_list *pack_lockfiles,\n -\t\t int only_packfile,\n -+\t\t char **index_pack_args,\n ++\t\t struct strvec *index_pack_args,\n \t\t struct ref **sought, int nr_sought)\n {\n \tstruct async demux;\n @@ fetch-pack.c: static int get_pack(struct fetch_pack_args *args,\n \t}\n \n +\tif (index_pack_args) {\n -+\t\tstruct strbuf joined = STRBUF_INIT;\n +\t\tint i;\n +\n -+\t\tfor (i = 0; i < cmd.args.nr; i++) {\n -+\t\t\tif (i)\n -+\t\t\t\tstrbuf_addch(&joined, ' ');\n -+\t\t\tstrbuf_addstr_urlencode(&joined, cmd.args.v[i],\n -+\t\t\t\t\t\tis_rfc3986_unreserved);\n -+\t\t}\n -+\t\t*index_pack_args = strbuf_detach(&joined, NULL);\n ++\t\tfor (i = 0; i < cmd.args.nr; i++)\n ++\t\t\tstrvec_push(index_pack_args, cmd.args.v[i]);\n +\t}\n +\n \tcmd.in = demux.out;\n @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tint seen_ack = 0;\n \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n \tint i;\n -+\tchar *index_pack_args = NULL;\n ++\tstruct strvec index_pack_args = STRVEC_INIT;\n \n \tnegotiator = &negotiator_alloc;\n \tfetch_negotiator_init(r, negotiator);\n @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\t\tdie(_(\"git fetch-pack: fetch failed.\"));\n \t\t\tdo_check_stateless_delimiter(args, &reader);\n \n +@@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n + \t}\n + \n + \tfor (i = 0; i < packfile_uris.nr; i++) {\n ++\t\tint j;\n + \t\tstruct child_process cmd = CHILD_PROCESS_INIT;\n + \t\tchar packname[GIT_MAX_HEXSZ + 1];\n + \t\tconst char *uri = packfile_uris.items[i].string +\n @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n \t\t\t (int) the_hash_algo->hexsz,\n \t\t\t packfile_uris.items[i].string);\n --\t\tstrvec_push(&cmd.args, \"--index-pack-args=index-pack --stdin --keep\");\n -+\t\tstrvec_pushf(&cmd.args, \"--index-pack-args=%s\", index_pack_args);\n +-\t\tstrvec_push(&cmd.args, \"--index-pack-arg=index-pack\");\n +-\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--stdin\");\n +-\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--keep\");\n ++\t\tfor (j = 0; j < index_pack_args.nr; j++)\n ++\t\t\tstrvec_pushf(&cmd.args, \"--index-pack-arg=%s\",\n ++\t\t\t\t index_pack_args.v[j]);\n \t\tstrvec_push(&cmd.args, uri);\n \t\tcmd.git_cmd = 1;\n \t\tcmd.no_stdin = 1;\n @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\t\t\t\t packname));\n \t}\n \tstring_list_clear(&packfile_uris, 0);\n -+\tFREE_AND_NULL(index_pack_args);\n ++\tstrvec_clear(&index_pack_args);\n \n \tif (negotiator)\n \t\tnegotiator->release(negotiator);\n3: 384c9d1c73 ! 4: e8b18d02e6 fetch-pack: print and use dangling .gitmodules\n @@ Documentation/git-index-pack.txt: OPTIONS\n \tSpecifies the number of threads to spawn when resolving\n \n ## builtin/index-pack.c ##\n +@@ builtin/index-pack.c: static void show_pack_info(int stat_only)\n + \t}\n + }\n + \n ++static int print_dangling_gitmodules(struct fsck_options *o,\n ++\t\t\t\t const struct object_id *oid,\n ++\t\t\t\t enum object_type object_type,\n ++\t\t\t\t int msg_type, const char *message)\n ++{\n ++\t/*\n ++\t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\n ++\t * instead of relying on this string check.\n ++\t */\n ++\tif (starts_with(message, \"gitmodulesMissing\")) {\n ++\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n ++\t\treturn 0;\n ++\t}\n ++\treturn fsck_error_function(o, oid, object_type, msg_type, message);\n ++}\n ++\n + int cmd_index_pack(int argc, const char **argv, const char *prefix)\n + {\n + \tint i, fix_thin_pack = 0, verify = 0, stat_only = 0;\n @@ builtin/index-pack.c: int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \telse\n \t\tclose(input_fd);\n @@ builtin/index-pack.c: int cmd_index_pack(int argc, const char **argv, const char\n -\tif (do_fsck_object && fsck_finish(&fsck_options))\n -\t\tdie(_(\"fsck error in pack objects\"));\n +\tif (do_fsck_object) {\n -+\t\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n ++\t\tstruct fsck_options fo = fsck_options;\n +\n -+\t\tfo.print_dangling_gitmodules = 1;\n ++\t\tfo.error_func = print_dangling_gitmodules;\n +\t\tif (fsck_finish(&fo))\n +\t\t\tdie(_(\"fsck error in pack objects\"));\n +\t}\n @@ fetch-pack.c: static void write_promisor_file(const char *keep_name,\n +\n /*\n * If packfile URIs were provided, pass a non-NULL pointer to index_pack_args.\n - * The string to pass as the --index-pack-args argument to http-fetch will be\n + * The strings to pass as the --index-pack-arg arguments to http-fetch will be\n @@ fetch-pack.c: static void write_promisor_file(const char *keep_name,\n static int get_pack(struct fetch_pack_args *args,\n \t\t int xd[2], struct string_list *pack_lockfiles,\n - \t\t char **index_pack_args,\n + \t\t struct strvec *index_pack_args,\n -\t\t struct ref **sought, int nr_sought)\n +\t\t struct ref **sought, int nr_sought,\n +\t\t struct oidset *gitmodules_oids)\n @@ fetch-pack.c: static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n \tint i;\n - \tchar *index_pack_args = NULL;\n + \tstruct strvec index_pack_args = STRVEC_INIT;\n +\tstruct oidset gitmodules_oids = OIDSET_INIT;\n \n \tnegotiator = &negotiator_alloc;\n @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\tif (finish_command(&cmd))\n @@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstring_list_clear(&packfile_uris, 0);\n - \tFREE_AND_NULL(index_pack_args);\n + \tstrvec_clear(&index_pack_args);\n \n +\tfsck_gitmodules_oids(&gitmodules_oids);\n +\n @@ fsck.c: int fsck_error_function(struct fsck_options *o,\n int fsck_finish(struct fsck_options *options)\n {\n \tint ret = 0;\n -@@ fsck.c: int fsck_finish(struct fsck_options *options)\n - \t\tif (!buf) {\n - \t\t\tif (is_promisor_object(oid))\n - \t\t\t\tcontinue;\n --\t\t\tret |= report(options,\n --\t\t\t\t oid, OBJ_BLOB,\n --\t\t\t\t FSCK_MSG_GITMODULES_MISSING,\n --\t\t\t\t \"unable to read .gitmodules blob\");\n -+\t\t\tif (options->print_dangling_gitmodules)\n -+\t\t\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n -+\t\t\telse\n -+\t\t\t\tret |= report(options,\n -+\t\t\t\t\t oid, OBJ_BLOB,\n -+\t\t\t\t\t FSCK_MSG_GITMODULES_MISSING,\n -+\t\t\t\t\t \"unable to read .gitmodules blob\");\n - \t\t\tcontinue;\n - \t\t}\n - \n \n ## fsck.h ##\n -@@ fsck.h: struct fsck_options {\n - \tint *msg_type;\n - \tstruct oidset skiplist;\n - \tkh_oid_map_t *object_names;\n -+\n -+\t/*\n -+\t * If 1, print the hashes of missing .gitmodules blobs instead of\n -+\t * considering them to be errors.\n -+\t */\n -+\tunsigned print_dangling_gitmodules:1;\n - };\n - \n - #define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n @@ fsck.h: int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n @@ pack.h: int verify_pack_index(struct packed_git *);\n * The \"hdr\" output buffer should be at least this big, which will handle sizes\n \n ## t/t5702-protocol-v2.sh ##\n +@@ t/t5702-protocol-v2.sh: test_expect_success 'part of packfile response provided as URI' '\n + \ttest -f hfound &&\n + \ttest -f h2found &&\n + \n +-\t# Ensure that there are exactly 6 files (3 .pack and 3 .idx).\n +-\tls http_child/.git/objects/pack/* >filelist &&\n ++\t# Ensure that there are exactly 3 packfiles with associated .idx\n ++\tls http_child/.git/objects/pack/*.pack \\\n ++\t http_child/.git/objects/pack/*.idx >filelist &&\n + \ttest_line_count = 6 filelist\n + '\n + \n +@@ t/t5702-protocol-v2.sh: test_expect_success 'packfile-uri with transfer.fsckobjects' '\n + \t\t-c fetch.uriprotocols=http,https \\\n + \t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n + \n +-\t# Ensure that there are exactly 4 files (2 .pack and 2 .idx).\n +-\tls http_child/.git/objects/pack/* >filelist &&\n ++\t# Ensure that there are exactly 2 packfiles with associated .idx\n ++\tls http_child/.git/objects/pack/*.pack \\\n ++\t http_child/.git/objects/pack/*.idx >filelist &&\n + \ttest_line_count = 4 filelist\n + '\n + \n @@ t/t5702-protocol-v2.sh: test_expect_success 'packfile-uri with transfer.fsckobjects fails on bad object'\n \ttest_i18ngrep \"invalid author/committer line - missing email\" error\n '\n @@ t/t5702-protocol-v2.sh: test_expect_success 'packfile-uri with transfer.fsckobje\n +\t\t-c fetch.uriprotocols=http,https \\\n +\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n +\n -+\t# Ensure that there are exactly 4 files (2 .pack and 2 .idx).\n -+\tls http_child/.git/objects/pack/* >filelist &&\n ++\t# Ensure that there are exactly 2 packfiles with associated .idx\n ++\tls http_child/.git/objects/pack/*.pack \\\n ++\t http_child/.git/objects/pack/*.idx >filelist &&\n +\ttest_line_count = 4 filelist\n +'\n +\n4: da0d7b38ae < -: ---------- SQUASH??? test fix\n-- \n2.30.0.617.g56c4b15f3c-goog\n\n"},{"id":"417486","messageId":"b7e376be160a569e25405153c6d0f21fa28ca03a.1614021093.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"cover.1614021092.git.jonathantanmy@google.com","subject":"[PATCH v2 1/4] http: allow custom index-pack args","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-22T19:20:06Z","receivedAt":"2021-02-22T19:26:31Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Currently, when fetching, packfiles referenced by URIs are run through\nindex-pack without any arguments other than --stdin and --keep, no\nmatter what arguments are used for the packfile that is inline in the\nfetch response. As a preparation for ensuring that all packs (whether\ninline or not) use the same index-pack arguments, teach the http\nsubsystem to allow custom index-pack arguments.\n\nhttp-fetch has been updated to use the new API. For now, it passes\n--keep alone instead of --keep with a process ID, but this is only\ntemporary because http-fetch itself will be taught to accept index-pack\nparameters (instead of using a hardcoded constant) in a subsequent\ncommit.\n\nSigned-off-by: Jonathan Tan \nSigned-off-by: Junio C Hamano \n---\n http-fetch.c | 6 +++++-\n http.c | 15 ++++++++-------\n http.h | 10 +++++-----\n 3 files changed, 18 insertions(+), 13 deletions(-)\n\ndiff --git a/http-fetch.c b/http-fetch.c\nindex c4ccc5fea9..2d1d9d054f 100644\n--- a/http-fetch.c\n+++ b/http-fetch.c\n@@ -43,6 +43,9 @@ static int fetch_using_walker(const char *raw_url, int get_verbosely,\n \treturn rc;\n }\n \n+static const char *index_pack_args[] =\n+\t{\"index-pack\", \"--stdin\", \"--keep\", NULL};\n+\n static void fetch_single_packfile(struct object_id *packfile_hash,\n \t\t\t\t const char *url) {\n \tstruct http_pack_request *preq;\n@@ -55,7 +58,8 @@ static void fetch_single_packfile(struct object_id *packfile_hash,\n \tif (preq == NULL)\n \t\tdie(\"couldn't create http pack request\");\n \tpreq->slot->results = &results;\n-\tpreq->generate_keep = 1;\n+\tpreq->index_pack_args = index_pack_args;\n+\tpreq->preserve_index_pack_stdout = 1;\n \n \tif (start_active_slot(preq->slot)) {\n \t\trun_active_slot(preq->slot);\ndiff --git a/http.c b/http.c\nindex 8b23a546af..f8ea28bb2e 100644\n--- a/http.c\n+++ b/http.c\n@@ -2259,6 +2259,9 @@ void release_http_pack_request(struct http_pack_request *preq)\n \tfree(preq);\n }\n \n+static const char *default_index_pack_args[] =\n+\t{\"index-pack\", \"--stdin\", NULL};\n+\n int finish_http_pack_request(struct http_pack_request *preq)\n {\n \tstruct child_process ip = CHILD_PROCESS_INIT;\n@@ -2270,17 +2273,15 @@ int finish_http_pack_request(struct http_pack_request *preq)\n \n \ttmpfile_fd = xopen(preq->tmpfile.buf, O_RDONLY);\n \n-\tstrvec_push(&ip.args, \"index-pack\");\n-\tstrvec_push(&ip.args, \"--stdin\");\n \tip.git_cmd = 1;\n \tip.in = tmpfile_fd;\n-\tif (preq->generate_keep) {\n-\t\tstrvec_pushf(&ip.args, \"--keep=git %\"PRIuMAX,\n-\t\t\t (uintmax_t)getpid());\n+\tip.argv = preq->index_pack_args ? preq->index_pack_args\n+\t\t\t\t\t: default_index_pack_args;\n+\n+\tif (preq->preserve_index_pack_stdout)\n \t\tip.out = 0;\n-\t} else {\n+\telse\n \t\tip.no_stdout = 1;\n-\t}\n \n \tif (run_command(&ip)) {\n \t\tret = -1;\ndiff --git a/http.h b/http.h\nindex 5de792ef3f..bf3d1270ad 100644\n--- a/http.h\n+++ b/http.h\n@@ -218,12 +218,12 @@ struct http_pack_request {\n \tchar *url;\n \n \t/*\n-\t * If this is true, finish_http_pack_request() will pass \"--keep\" to\n-\t * index-pack, resulting in the creation of a keep file, and will not\n-\t * suppress its stdout (that is, the \"keep\\t\\n\" line will be\n-\t * printed to stdout).\n+\t * index-pack command to run. Must be terminated by NULL.\n+\t *\n+\t * If NULL, defaults to\t{\"index-pack\", \"--stdin\", NULL}.\n \t */\n-\tunsigned generate_keep : 1;\n+\tconst char **index_pack_args;\n+\tunsigned preserve_index_pack_stdout : 1;\n \n \tFILE *packfile;\n \tstruct strbuf tmpfile;\n-- \n2.30.0.617.g56c4b15f3c-goog\n\n"},{"id":"417487","messageId":"57220ceb841056aade08705ca0ac73ccc69f05ab.1614021093.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"cover.1614021092.git.jonathantanmy@google.com","subject":"[PATCH v2 2/4] http-fetch: allow custom index-pack args","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-22T19:20:07Z","receivedAt":"2021-02-22T19:26:32Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"This is the next step in teaching fetch-pack to pass its index-pack\narguments when processing packfiles referenced by URIs.\n\nThe \"--keep\" in fetch-pack.c will be replaced with a full message in a\nsubsequent commit.\n\nSigned-off-by: Jonathan Tan \nSigned-off-by: Junio C Hamano \n---\n Documentation/git-http-fetch.txt | 10 ++++++++--\n fetch-pack.c | 3 +++\n http-fetch.c | 20 +++++++++++++++-----\n t/t5550-http-fetch-dumb.sh | 5 ++++-\n 4 files changed, 30 insertions(+), 8 deletions(-)\n\ndiff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt\nindex 4deb4893f5..9fa17b60e4 100644\n--- a/Documentation/git-http-fetch.txt\n+++ b/Documentation/git-http-fetch.txt\n@@ -41,11 +41,17 @@ commit-id::\n \t\t['\\t']\n \n --packfile=::\n-\tInstead of a commit id on the command line (which is not expected in\n+\tFor internal use only. Instead of a commit id on the command\n+\tline (which is not expected in\n \tthis case), 'git http-fetch' fetches the packfile directly at the given\n \tURL and uses index-pack to generate corresponding .idx and .keep files.\n \tThe hash is used to determine the name of the temporary file and is\n-\tarbitrary. The output of index-pack is printed to stdout.\n+\tarbitrary. The output of index-pack is printed to stdout. Requires\n+\t--index-pack-args.\n+\n+--index-pack-args=::\n+\tFor internal use only. The command to run on the contents of the\n+\tdownloaded pack. Arguments are URL-encoded separated by spaces.\n \n --recover::\n \tVerify that everything reachable from target is fetched. Used after\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 876f90c759..aeac010b0b 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -1645,6 +1645,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n \t\t\t (int) the_hash_algo->hexsz,\n \t\t\t packfile_uris.items[i].string);\n+\t\tstrvec_push(&cmd.args, \"--index-pack-arg=index-pack\");\n+\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--stdin\");\n+\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--keep\");\n \t\tstrvec_push(&cmd.args, uri);\n \t\tcmd.git_cmd = 1;\n \t\tcmd.no_stdin = 1;\ndiff --git a/http-fetch.c b/http-fetch.c\nindex 2d1d9d054f..fa642462a9 100644\n--- a/http-fetch.c\n+++ b/http-fetch.c\n@@ -3,6 +3,7 @@\n #include \"exec-cmd.h\"\n #include \"http.h\"\n #include \"walker.h\"\n+#include \"strvec.h\"\n \n static const char http_fetch_usage[] = \"git http-fetch \"\n \"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin | --packfile=hash | commit-id] url\";\n@@ -43,11 +44,9 @@ static int fetch_using_walker(const char *raw_url, int get_verbosely,\n \treturn rc;\n }\n \n-static const char *index_pack_args[] =\n-\t{\"index-pack\", \"--stdin\", \"--keep\", NULL};\n-\n static void fetch_single_packfile(struct object_id *packfile_hash,\n-\t\t\t\t const char *url) {\n+\t\t\t\t const char *url,\n+\t\t\t\t const char **index_pack_args) {\n \tstruct http_pack_request *preq;\n \tstruct slot_results results;\n \tint ret;\n@@ -90,6 +89,7 @@ int cmd_main(int argc, const char **argv)\n \tint packfile = 0;\n \tint nongit;\n \tstruct object_id packfile_hash;\n+\tstruct strvec index_pack_args = STRVEC_INIT;\n \n \tsetup_git_directory_gently(&nongit);\n \n@@ -116,6 +116,8 @@ int cmd_main(int argc, const char **argv)\n \t\t\tpackfile = 1;\n \t\t\tif (parse_oid_hex(p, &packfile_hash, &end) || *end)\n \t\t\t\tdie(_(\"argument to --packfile must be a valid hash (got '%s')\"), p);\n+\t\t} else if (skip_prefix(argv[arg], \"--index-pack-arg=\", &p)) {\n+\t\t\tstrvec_push(&index_pack_args, p);\n \t\t}\n \t\targ++;\n \t}\n@@ -128,10 +130,18 @@ int cmd_main(int argc, const char **argv)\n \tgit_config(git_default_config, NULL);\n \n \tif (packfile) {\n-\t\tfetch_single_packfile(&packfile_hash, argv[arg]);\n+\t\tif (!index_pack_args.nr)\n+\t\t\tdie(_(\"--packfile requires --index-pack-args\"));\n+\n+\t\tfetch_single_packfile(&packfile_hash, argv[arg],\n+\t\t\t\t index_pack_args.v);\n+\n \t\treturn 0;\n \t}\n \n+\tif (index_pack_args.nr)\n+\t\tdie(_(\"--index-pack-args can only be used with --packfile\"));\n+\n \tif (commits_on_stdin) {\n \t\tcommits = walker_targets_stdin(&commit_id, &write_ref);\n \t} else {\ndiff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh\nindex 483578b2d7..358b322e05 100755\n--- a/t/t5550-http-fetch-dumb.sh\n+++ b/t/t5550-http-fetch-dumb.sh\n@@ -224,7 +224,10 @@ test_expect_success 'http-fetch --packfile' '\n \n \tgit init packfileclient &&\n \tp=$(cd \"$HTTPD_DOCUMENT_ROOT_PATH\"/repo_pack.git && ls objects/pack/pack-*.pack) &&\n-\tgit -C packfileclient http-fetch --packfile=$ARBITRARY \"$HTTPD_URL\"/dumb/repo_pack.git/$p >out &&\n+\tgit -C packfileclient http-fetch --packfile=$ARBITRARY \\\n+\t\t--index-pack-arg=index-pack --index-pack-arg=--stdin \\\n+\t\t--index-pack-arg=--keep \\\n+\t\t\"$HTTPD_URL\"/dumb/repo_pack.git/$p >out &&\n \n \tgrep \"^keep.[0-9a-f]\\{16,\\}$\" out &&\n \tcut -c6- out >packhash &&\n-- \n2.30.0.617.g56c4b15f3c-goog\n\n"},{"id":"417488","messageId":"aa873354649b73c83544399a92dc8d8ecc489e44.1614021093.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"cover.1614021092.git.jonathantanmy@google.com","subject":"[PATCH v2 3/4] fetch-pack: with packfile URIs, use index-pack arg","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-22T19:20:08Z","receivedAt":"2021-02-22T19:26:35Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Unify the index-pack arguments used when processing the inline pack and\nwhen downloading packfiles referenced by URIs. This is done by teaching\nget_pack() to also store the index-pack arguments whenever at least one\npackfile URI is given, and then when processing the packfile URI(s),\nusing the stored arguments.\n\nSigned-off-by: Jonathan Tan \nSigned-off-by: Junio C Hamano \n---\n fetch-pack.c | 34 +++++++++++++++++++++++-----------\n 1 file changed, 23 insertions(+), 11 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex aeac010b0b..dd0a6c4b34 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -797,12 +797,13 @@ static void write_promisor_file(const char *keep_name,\n }\n \n /*\n- * Pass 1 as \"only_packfile\" if the pack received is the only pack in this\n- * fetch request (that is, if there were no packfile URIs provided).\n+ * If packfile URIs were provided, pass a non-NULL pointer to index_pack_args.\n+ * The strings to pass as the --index-pack-arg arguments to http-fetch will be\n+ * stored there. (It must be freed by the caller.)\n */\n static int get_pack(struct fetch_pack_args *args,\n \t\t int xd[2], struct string_list *pack_lockfiles,\n-\t\t int only_packfile,\n+\t\t struct strvec *index_pack_args,\n \t\t struct ref **sought, int nr_sought)\n {\n \tstruct async demux;\n@@ -845,7 +846,7 @@ static int get_pack(struct fetch_pack_args *args,\n \t\tstrvec_push(&cmd.args, alternate_shallow_file);\n \t}\n \n-\tif (do_keep || args->from_promisor) {\n+\tif (do_keep || args->from_promisor || index_pack_args) {\n \t\tif (pack_lockfiles)\n \t\t\tcmd.out = -1;\n \t\tcmd_name = \"index-pack\";\n@@ -863,7 +864,7 @@ static int get_pack(struct fetch_pack_args *args,\n \t\t\t\t \"--keep=fetch-pack %\"PRIuMAX \" on %s\",\n \t\t\t\t (uintmax_t)getpid(), hostname);\n \t\t}\n-\t\tif (only_packfile && args->check_self_contained_and_connected)\n+\t\tif (!index_pack_args && args->check_self_contained_and_connected)\n \t\t\tstrvec_push(&cmd.args, \"--check-self-contained-and-connected\");\n \t\telse\n \t\t\t/*\n@@ -901,7 +902,7 @@ static int get_pack(struct fetch_pack_args *args,\n \t : transfer_fsck_objects >= 0\n \t ? transfer_fsck_objects\n \t : 0) {\n-\t\tif (args->from_promisor || !only_packfile)\n+\t\tif (args->from_promisor || index_pack_args)\n \t\t\t/*\n \t\t\t * We cannot use --strict in index-pack because it\n \t\t\t * checks both broken objects and links, but we only\n@@ -913,6 +914,13 @@ static int get_pack(struct fetch_pack_args *args,\n \t\t\t\t fsck_msg_types.buf);\n \t}\n \n+\tif (index_pack_args) {\n+\t\tint i;\n+\n+\t\tfor (i = 0; i < cmd.args.nr; i++)\n+\t\t\tstrvec_push(index_pack_args, cmd.args.v[i]);\n+\t}\n+\n \tcmd.in = demux.out;\n \tcmd.git_cmd = 1;\n \tif (start_command(&cmd))\n@@ -1084,7 +1092,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\talternate_shallow_file = setup_temporary_shallow(si->shallow);\n \telse\n \t\talternate_shallow_file = NULL;\n-\tif (get_pack(args, fd, pack_lockfiles, 1, sought, nr_sought))\n+\tif (get_pack(args, fd, pack_lockfiles, NULL, sought, nr_sought))\n \t\tdie(_(\"git fetch-pack: fetch failed.\"));\n \n all_done:\n@@ -1535,6 +1543,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tint seen_ack = 0;\n \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n \tint i;\n+\tstruct strvec index_pack_args = STRVEC_INIT;\n \n \tnegotiator = &negotiator_alloc;\n \tfetch_negotiator_init(r, negotiator);\n@@ -1624,7 +1633,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\t\treceive_packfile_uris(&reader, &packfile_uris);\n \t\t\tprocess_section_header(&reader, \"packfile\", 0);\n \t\t\tif (get_pack(args, fd, pack_lockfiles,\n-\t\t\t\t !packfile_uris.nr, sought, nr_sought))\n+\t\t\t\t packfile_uris.nr ? &index_pack_args : NULL,\n+\t\t\t\t sought, nr_sought))\n \t\t\t\tdie(_(\"git fetch-pack: fetch failed.\"));\n \t\t\tdo_check_stateless_delimiter(args, &reader);\n \n@@ -1636,6 +1646,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t}\n \n \tfor (i = 0; i < packfile_uris.nr; i++) {\n+\t\tint j;\n \t\tstruct child_process cmd = CHILD_PROCESS_INIT;\n \t\tchar packname[GIT_MAX_HEXSZ + 1];\n \t\tconst char *uri = packfile_uris.items[i].string +\n@@ -1645,9 +1656,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n \t\t\t (int) the_hash_algo->hexsz,\n \t\t\t packfile_uris.items[i].string);\n-\t\tstrvec_push(&cmd.args, \"--index-pack-arg=index-pack\");\n-\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--stdin\");\n-\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--keep\");\n+\t\tfor (j = 0; j < index_pack_args.nr; j++)\n+\t\t\tstrvec_pushf(&cmd.args, \"--index-pack-arg=%s\",\n+\t\t\t\t index_pack_args.v[j]);\n \t\tstrvec_push(&cmd.args, uri);\n \t\tcmd.git_cmd = 1;\n \t\tcmd.no_stdin = 1;\n@@ -1683,6 +1694,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\t\t\t\t packname));\n \t}\n \tstring_list_clear(&packfile_uris, 0);\n+\tstrvec_clear(&index_pack_args);\n \n \tif (negotiator)\n \t\tnegotiator->release(negotiator);\n-- \n2.30.0.617.g56c4b15f3c-goog\n\n"},{"id":"417489","messageId":"e8b18d02e61e4307af859e74789b0c191d8e75f7.1614021093.git.jonathantanmy@google.com","threadId":"54994","inReplyTo":"cover.1614021092.git.jonathantanmy@google.com","subject":"[PATCH v2 4/4] fetch-pack: print and use dangling .gitmodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-22T19:20:09Z","receivedAt":"2021-02-22T19:26:47Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Teach index-pack to print dangling .gitmodules links after its \"keep\" or\n\"pack\" line instead of declaring an error, and teach fetch-pack to check\nsuch lines printed.\n\nThis allows the tree side of the .gitmodules link to be in one packfile\nand the blob side to be in another without failing the fsck check,\nbecause it is now fetch-pack which checks such objects after all\npackfiles have been downloaded and indexed (and not index-pack on an\nindividual packfile, as it is before this commit).\n\nSigned-off-by: Jonathan Tan \nSigned-off-by: Junio C Hamano \n---\n Documentation/git-index-pack.txt | 7 ++-\n builtin/index-pack.c | 25 +++++++++-\n builtin/receive-pack.c | 2 +-\n fetch-pack.c | 78 +++++++++++++++++++++++++++-----\n fsck.c | 5 ++\n fsck.h | 2 +\n pack-write.c | 8 +++-\n pack.h | 2 +-\n t/t5702-protocol-v2.sh | 58 ++++++++++++++++++++++--\n 9 files changed, 165 insertions(+), 22 deletions(-)\n\ndiff --git a/Documentation/git-index-pack.txt b/Documentation/git-index-pack.txt\nindex af0c26232c..e74a4a1eda 100644\n--- a/Documentation/git-index-pack.txt\n+++ b/Documentation/git-index-pack.txt\n@@ -78,7 +78,12 @@ OPTIONS\n \tDie if the pack contains broken links. For internal use only.\n \n --fsck-objects::\n-\tDie if the pack contains broken objects. For internal use only.\n+\tFor internal use only.\n++\n+Die if the pack contains broken objects. If the pack contains a tree\n+pointing to a .gitmodules blob that does not exist, prints the hash of\n+that blob (for the caller to check) after the hash that goes into the\n+name of the pack/idx file (see \"Notes\").\n \n --threads=::\n \tSpecifies the number of threads to spawn when resolving\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 557bd2f348..0444febeee 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1693,6 +1693,22 @@ static void show_pack_info(int stat_only)\n \t}\n }\n \n+static int print_dangling_gitmodules(struct fsck_options *o,\n+\t\t\t\t const struct object_id *oid,\n+\t\t\t\t enum object_type object_type,\n+\t\t\t\t int msg_type, const char *message)\n+{\n+\t/*\n+\t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\n+\t * instead of relying on this string check.\n+\t */\n+\tif (starts_with(message, \"gitmodulesMissing\")) {\n+\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n+\t\treturn 0;\n+\t}\n+\treturn fsck_error_function(o, oid, object_type, msg_type, message);\n+}\n+\n int cmd_index_pack(int argc, const char **argv, const char *prefix)\n {\n \tint i, fix_thin_pack = 0, verify = 0, stat_only = 0;\n@@ -1888,8 +1904,13 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \telse\n \t\tclose(input_fd);\n \n-\tif (do_fsck_object && fsck_finish(&fsck_options))\n-\t\tdie(_(\"fsck error in pack objects\"));\n+\tif (do_fsck_object) {\n+\t\tstruct fsck_options fo = fsck_options;\n+\n+\t\tfo.error_func = print_dangling_gitmodules;\n+\t\tif (fsck_finish(&fo))\n+\t\t\tdie(_(\"fsck error in pack objects\"));\n+\t}\n \n \tfree(objects);\n \tstrbuf_release(&index_name_buf);\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex d49d050e6e..ed2c9b42e9 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -2275,7 +2275,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)\n \t\tstatus = start_command(&child);\n \t\tif (status)\n \t\t\treturn \"index-pack fork failed\";\n-\t\tpack_lockfile = index_pack_lockfile(child.out);\n+\t\tpack_lockfile = index_pack_lockfile(child.out, NULL);\n \t\tclose(child.out);\n \t\tstatus = finish_command(&child);\n \t\tif (status)\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex dd0a6c4b34..f9def5ac74 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -796,6 +796,26 @@ static void write_promisor_file(const char *keep_name,\n \tstrbuf_release(&promisor_name);\n }\n \n+static void parse_gitmodules_oids(int fd, struct oidset *gitmodules_oids)\n+{\n+\tint len = the_hash_algo->hexsz + 1; /* hash + NL */\n+\n+\tdo {\n+\t\tchar hex_hash[GIT_MAX_HEXSZ + 1];\n+\t\tint read_len = read_in_full(fd, hex_hash, len);\n+\t\tstruct object_id oid;\n+\t\tconst char *end;\n+\n+\t\tif (!read_len)\n+\t\t\treturn;\n+\t\tif (read_len != len)\n+\t\t\tdie(\"invalid length read %d\", read_len);\n+\t\tif (parse_oid_hex(hex_hash, &oid, &end) || *end != '\\n')\n+\t\t\tdie(\"invalid hash\");\n+\t\toidset_insert(gitmodules_oids, &oid);\n+\t} while (1);\n+}\n+\n /*\n * If packfile URIs were provided, pass a non-NULL pointer to index_pack_args.\n * The strings to pass as the --index-pack-arg arguments to http-fetch will be\n@@ -804,7 +824,8 @@ static void write_promisor_file(const char *keep_name,\n static int get_pack(struct fetch_pack_args *args,\n \t\t int xd[2], struct string_list *pack_lockfiles,\n \t\t struct strvec *index_pack_args,\n-\t\t struct ref **sought, int nr_sought)\n+\t\t struct ref **sought, int nr_sought,\n+\t\t struct oidset *gitmodules_oids)\n {\n \tstruct async demux;\n \tint do_keep = args->keep_pack;\n@@ -812,6 +833,7 @@ static int get_pack(struct fetch_pack_args *args,\n \tstruct pack_header header;\n \tint pass_header = 0;\n \tstruct child_process cmd = CHILD_PROCESS_INIT;\n+\tint fsck_objects = 0;\n \tint ret;\n \n \tmemset(&demux, 0, sizeof(demux));\n@@ -846,8 +868,15 @@ static int get_pack(struct fetch_pack_args *args,\n \t\tstrvec_push(&cmd.args, alternate_shallow_file);\n \t}\n \n-\tif (do_keep || args->from_promisor || index_pack_args) {\n-\t\tif (pack_lockfiles)\n+\tif (fetch_fsck_objects >= 0\n+\t ? fetch_fsck_objects\n+\t : transfer_fsck_objects >= 0\n+\t ? transfer_fsck_objects\n+\t : 0)\n+\t\tfsck_objects = 1;\n+\n+\tif (do_keep || args->from_promisor || index_pack_args || fsck_objects) {\n+\t\tif (pack_lockfiles || fsck_objects)\n \t\t\tcmd.out = -1;\n \t\tcmd_name = \"index-pack\";\n \t\tstrvec_push(&cmd.args, cmd_name);\n@@ -897,11 +926,7 @@ static int get_pack(struct fetch_pack_args *args,\n \t\tstrvec_pushf(&cmd.args, \"--pack_header=%\"PRIu32\",%\"PRIu32,\n \t\t\t ntohl(header.hdr_version),\n \t\t\t\t ntohl(header.hdr_entries));\n-\tif (fetch_fsck_objects >= 0\n-\t ? fetch_fsck_objects\n-\t : transfer_fsck_objects >= 0\n-\t ? transfer_fsck_objects\n-\t : 0) {\n+\tif (fsck_objects) {\n \t\tif (args->from_promisor || index_pack_args)\n \t\t\t/*\n \t\t\t * We cannot use --strict in index-pack because it\n@@ -925,10 +950,15 @@ static int get_pack(struct fetch_pack_args *args,\n \tcmd.git_cmd = 1;\n \tif (start_command(&cmd))\n \t\tdie(_(\"fetch-pack: unable to fork off %s\"), cmd_name);\n-\tif (do_keep && pack_lockfiles) {\n-\t\tchar *pack_lockfile = index_pack_lockfile(cmd.out);\n+\tif (do_keep && (pack_lockfiles || fsck_objects)) {\n+\t\tint is_well_formed;\n+\t\tchar *pack_lockfile = index_pack_lockfile(cmd.out, &is_well_formed);\n+\n+\t\tif (!is_well_formed)\n+\t\t\tdie(_(\"fetch-pack: invalid index-pack output\"));\n \t\tif (pack_lockfile)\n \t\t\tstring_list_append_nodup(pack_lockfiles, pack_lockfile);\n+\t\tparse_gitmodules_oids(cmd.out, gitmodules_oids);\n \t\tclose(cmd.out);\n \t}\n \n@@ -963,6 +993,22 @@ static int cmp_ref_by_name(const void *a_, const void *b_)\n \treturn strcmp(a->name, b->name);\n }\n \n+static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n+{\n+\tstruct oidset_iter iter;\n+\tconst struct object_id *oid;\n+\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n+\n+\tif (!oidset_size(gitmodules_oids))\n+\t\treturn;\n+\n+\toidset_iter_init(gitmodules_oids, &iter);\n+\twhile ((oid = oidset_iter_next(&iter)))\n+\t\tregister_found_gitmodules(oid);\n+\tif (fsck_finish(&fo))\n+\t\tdie(\"fsck failed\");\n+}\n+\n static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\t\t\t int fd[2],\n \t\t\t\t const struct ref *orig_ref,\n@@ -977,6 +1023,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \tint agent_len;\n \tstruct fetch_negotiator negotiator_alloc;\n \tstruct fetch_negotiator *negotiator;\n+\tstruct oidset gitmodules_oids = OIDSET_INIT;\n \n \tnegotiator = &negotiator_alloc;\n \tfetch_negotiator_init(r, negotiator);\n@@ -1092,8 +1139,10 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\talternate_shallow_file = setup_temporary_shallow(si->shallow);\n \telse\n \t\talternate_shallow_file = NULL;\n-\tif (get_pack(args, fd, pack_lockfiles, NULL, sought, nr_sought))\n+\tif (get_pack(args, fd, pack_lockfiles, NULL, sought, nr_sought,\n+\t\t &gitmodules_oids))\n \t\tdie(_(\"git fetch-pack: fetch failed.\"));\n+\tfsck_gitmodules_oids(&gitmodules_oids);\n \n all_done:\n \tif (negotiator)\n@@ -1544,6 +1593,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n \tint i;\n \tstruct strvec index_pack_args = STRVEC_INIT;\n+\tstruct oidset gitmodules_oids = OIDSET_INIT;\n \n \tnegotiator = &negotiator_alloc;\n \tfetch_negotiator_init(r, negotiator);\n@@ -1634,7 +1684,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\tprocess_section_header(&reader, \"packfile\", 0);\n \t\t\tif (get_pack(args, fd, pack_lockfiles,\n \t\t\t\t packfile_uris.nr ? &index_pack_args : NULL,\n-\t\t\t\t sought, nr_sought))\n+\t\t\t\t sought, nr_sought, &gitmodules_oids))\n \t\t\t\tdie(_(\"git fetch-pack: fetch failed.\"));\n \t\t\tdo_check_stateless_delimiter(args, &reader);\n \n@@ -1677,6 +1727,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \n \t\tpackname[the_hash_algo->hexsz] = '\\0';\n \n+\t\tparse_gitmodules_oids(cmd.out, &gitmodules_oids);\n+\n \t\tclose(cmd.out);\n \n \t\tif (finish_command(&cmd))\n@@ -1696,6 +1748,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstring_list_clear(&packfile_uris, 0);\n \tstrvec_clear(&index_pack_args);\n \n+\tfsck_gitmodules_oids(&gitmodules_oids);\n+\n \tif (negotiator)\n \t\tnegotiator->release(negotiator);\n \ndiff --git a/fsck.c b/fsck.c\nindex f82e2fe9e3..49ef6569e8 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1243,6 +1243,11 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n+void register_found_gitmodules(const struct object_id *oid)\n+{\n+\toidset_insert(&gitmodules_found, oid);\n+}\n+\n int fsck_finish(struct fsck_options *options)\n {\n \tint ret = 0;\ndiff --git a/fsck.h b/fsck.h\nindex 69cf715e79..d75b723bd5 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -62,6 +62,8 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n+void register_found_gitmodules(const struct object_id *oid);\n+\n /*\n * Some fsck checks are context-dependent, and may end up queued; run this\n * after completing all fsck_object() calls in order to resolve any remaining\ndiff --git a/pack-write.c b/pack-write.c\nindex 3513665e1e..f66ea8e5a1 100644\n--- a/pack-write.c\n+++ b/pack-write.c\n@@ -272,7 +272,7 @@ void fixup_pack_header_footer(int pack_fd,\n \tfsync_or_die(pack_fd, pack_name);\n }\n \n-char *index_pack_lockfile(int ip_out)\n+char *index_pack_lockfile(int ip_out, int *is_well_formed)\n {\n \tchar packname[GIT_MAX_HEXSZ + 6];\n \tconst int len = the_hash_algo->hexsz + 6;\n@@ -286,11 +286,17 @@ char *index_pack_lockfile(int ip_out)\n \t */\n \tif (read_in_full(ip_out, packname, len) == len && packname[len-1] == '\\n') {\n \t\tconst char *name;\n+\n+\t\tif (is_well_formed)\n+\t\t\t*is_well_formed = 1;\n \t\tpackname[len-1] = 0;\n \t\tif (skip_prefix(packname, \"keep\\t\", &name))\n \t\t\treturn xstrfmt(\"%s/pack/pack-%s.keep\",\n \t\t\t\t get_object_directory(), name);\n+\t\treturn NULL;\n \t}\n+\tif (is_well_formed)\n+\t\t*is_well_formed = 0;\n \treturn NULL;\n }\n \ndiff --git a/pack.h b/pack.h\nindex 9fc0945ac9..09cffec395 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -85,7 +85,7 @@ int verify_pack_index(struct packed_git *);\n int verify_pack(struct repository *, struct packed_git *, verify_fn fn, struct progress *, uint32_t);\n off_t write_pack_header(struct hashfile *f, uint32_t);\n void fixup_pack_header_footer(int, unsigned char *, const char *, uint32_t, unsigned char *, off_t);\n-char *index_pack_lockfile(int fd);\n+char *index_pack_lockfile(int fd, int *is_well_formed);\n \n /*\n * The \"hdr\" output buffer should be at least this big, which will handle sizes\ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 7d5b17909b..b1bc73a9a9 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -847,8 +847,9 @@ test_expect_success 'part of packfile response provided as URI' '\n \ttest -f hfound &&\n \ttest -f h2found &&\n \n-\t# Ensure that there are exactly 6 files (3 .pack and 3 .idx).\n-\tls http_child/.git/objects/pack/* >filelist &&\n+\t# Ensure that there are exactly 3 packfiles with associated .idx\n+\tls http_child/.git/objects/pack/*.pack \\\n+\t http_child/.git/objects/pack/*.idx >filelist &&\n \ttest_line_count = 6 filelist\n '\n \n@@ -901,8 +902,9 @@ test_expect_success 'packfile-uri with transfer.fsckobjects' '\n \t\t-c fetch.uriprotocols=http,https \\\n \t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n \n-\t# Ensure that there are exactly 4 files (2 .pack and 2 .idx).\n-\tls http_child/.git/objects/pack/* >filelist &&\n+\t# Ensure that there are exactly 2 packfiles with associated .idx\n+\tls http_child/.git/objects/pack/*.pack \\\n+\t http_child/.git/objects/pack/*.idx >filelist &&\n \ttest_line_count = 4 filelist\n '\n \n@@ -936,6 +938,54 @@ test_expect_success 'packfile-uri with transfer.fsckobjects fails on bad object'\n \ttest_i18ngrep \"invalid author/committer line - missing email\" error\n '\n \n+test_expect_success 'packfile-uri with transfer.fsckobjects succeeds when .gitmodules is separate from tree' '\n+\tP=\"$HTTPD_DOCUMENT_ROOT_PATH/http_parent\" &&\n+\trm -rf \"$P\" http_child &&\n+\n+\tgit init \"$P\" &&\n+\tgit -C \"$P\" config \"uploadpack.allowsidebandall\" \"true\" &&\n+\n+\techo \"[submodule libfoo]\" >\"$P/.gitmodules\" &&\n+\techo \"path = include/foo\" >>\"$P/.gitmodules\" &&\n+\techo \"url = git://example.com/git/lib.git\" >>\"$P/.gitmodules\" &&\n+\tgit -C \"$P\" add .gitmodules &&\n+\tgit -C \"$P\" commit -m x &&\n+\n+\tconfigure_exclusion \"$P\" .gitmodules >h &&\n+\n+\tsane_unset GIT_TEST_SIDEBAND_ALL &&\n+\tgit -c protocol.version=2 -c transfer.fsckobjects=1 \\\n+\t\t-c fetch.uriprotocols=http,https \\\n+\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child &&\n+\n+\t# Ensure that there are exactly 2 packfiles with associated .idx\n+\tls http_child/.git/objects/pack/*.pack \\\n+\t http_child/.git/objects/pack/*.idx >filelist &&\n+\ttest_line_count = 4 filelist\n+'\n+\n+test_expect_success 'packfile-uri with transfer.fsckobjects fails when .gitmodules separate from tree is invalid' '\n+\tP=\"$HTTPD_DOCUMENT_ROOT_PATH/http_parent\" &&\n+\trm -rf \"$P\" http_child err &&\n+\n+\tgit init \"$P\" &&\n+\tgit -C \"$P\" config \"uploadpack.allowsidebandall\" \"true\" &&\n+\n+\techo \"[submodule \\\"..\\\"]\" >\"$P/.gitmodules\" &&\n+\techo \"path = include/foo\" >>\"$P/.gitmodules\" &&\n+\techo \"url = git://example.com/git/lib.git\" >>\"$P/.gitmodules\" &&\n+\tgit -C \"$P\" add .gitmodules &&\n+\tgit -C \"$P\" commit -m x &&\n+\n+\tconfigure_exclusion \"$P\" .gitmodules >h &&\n+\n+\tsane_unset GIT_TEST_SIDEBAND_ALL &&\n+\ttest_must_fail git -c protocol.version=2 -c transfer.fsckobjects=1 \\\n+\t\t-c fetch.uriprotocols=http,https \\\n+\t\tclone \"$HTTPD_URL/smart/http_parent\" http_child 2>err &&\n+\ttest_i18ngrep \"disallowed submodule name\" err\n+'\n+\n # DO NOT add non-httpd-specific tests here, because the last part of this\n # test script is only executed when httpd is available and enabled.\n \n-- \n2.30.0.617.g56c4b15f3c-goog\n\n"},{"id":"417495","messageId":"xmqqft1n7tgn.fsf@gitster.g","threadId":"54994","inReplyTo":"cover.1614021092.git.jonathantanmy@google.com","subject":"Re: [PATCH v2 0/4] Check .gitmodules when using packfile URIs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-22T20:12:08Z","receivedAt":"2021-02-22T20:12:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> Here's v2. I think I've addressed all the review comments, including\n> passing the index-pack args as separate arguments (to avoid the\n> necessity to somehow encode in order to get rid of spaces), and by using\n> a custom error function instead of a specific option in fsck.\n>\n> This applies on master. I mentioned earlier [1] that I was planning to\n> implement this on Ævar's fsck API improvements, but after looking at the\n> latest v2, I see that it omits patch 11 from v1 (which is the one I\n> need), so what I've done is to use a string check in the meantime.\n>\n> [1] https://lore.kernel.org/git/20210219004612.1181920-1-jonathantanmy@google.com/\n\nI only looked at the difference between this round and what is in\n'seen', but everything looked reasonable to me (including the code\nthat is near NEEDSWORK comment, and what the comment said).\n\nWill queue. Thanks.\n"},{"id":"417563","messageId":"874ki29b53.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"57220ceb841056aade08705ca0ac73ccc69f05ab.1614021093.git.jonathantanmy@google.com","subject":"Re: [PATCH v2 2/4] http-fetch: allow custom index-pack args","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-02-23T13:17:12Z","receivedAt":"2021-02-23T13:18:11Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Feb 22 2021, Jonathan Tan wrote:\n\n> diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt\n> index 4deb4893f5..9fa17b60e4 100644\n> --- a/Documentation/git-http-fetch.txt\n> +++ b/Documentation/git-http-fetch.txt\n> @@ -41,11 +41,17 @@ commit-id::\n> \t\t['\\t']\n> \n> --packfile=::\n> -\tInstead of a commit id on the command line (which is not expected in\n> +\tFor internal use only. Instead of a commit id on the command\n> +\tline (which is not expected in\n> \tthis case), 'git http-fetch' fetches the packfile directly at the given\n> \tURL and uses index-pack to generate corresponding .idx and .keep files.\n> \tThe hash is used to determine the name of the temporary file and is\n> -\tarbitrary. The output of index-pack is printed to stdout.\n> +\tarbitrary. The output of index-pack is printed to stdout. Requires\n> +\t--index-pack-args.\n> +\n> +--index-pack-args=::\n> +\tFor internal use only. The command to run on the contents of the\n> +\tdownloaded pack. Arguments are URL-encoded separated by spaces.\n> \n> --recover::\n> \tVerify that everything reachable from target is fetched. Used after\n> diff --git a/fetch-pack.c b/fetch-pack.c\n> index 876f90c759..aeac010b0b 100644\n> --- a/fetch-pack.c\n> +++ b/fetch-pack.c\n> @@ -1645,6 +1645,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n> \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n> \t\t\t (int) the_hash_algo->hexsz,\n> \t\t\t packfile_uris.items[i].string);\n> +\t\tstrvec_push(&cmd.args, \"--index-pack-arg=index-pack\");\n> +\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--stdin\");\n> +\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--keep\");\n\nThe docs say --*-args, but the code checks --*arg, that seems like a\nmistake that should be fixed to make the code/tests use the plural form,\nno?\n"},{"id":"417566","messageId":"20210223165120.1561659-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"874ki29b53.fsf@evledraar.gmail.com","subject":"Re: [PATCH v2 2/4] http-fetch: allow custom index-pack args","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-02-23T16:51:20Z","receivedAt":"2021-02-23T16:52:07Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> > diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt\n> > index 4deb4893f5..9fa17b60e4 100644\n> > --- a/Documentation/git-http-fetch.txt\n> > +++ b/Documentation/git-http-fetch.txt\n> > @@ -41,11 +41,17 @@ commit-id::\n> > \t\t['\\t']\n> > \n> > --packfile=::\n> > -\tInstead of a commit id on the command line (which is not expected in\n> > +\tFor internal use only. Instead of a commit id on the command\n> > +\tline (which is not expected in\n> > \tthis case), 'git http-fetch' fetches the packfile directly at the given\n> > \tURL and uses index-pack to generate corresponding .idx and .keep files.\n> > \tThe hash is used to determine the name of the temporary file and is\n> > -\tarbitrary. The output of index-pack is printed to stdout.\n> > +\tarbitrary. The output of index-pack is printed to stdout. Requires\n> > +\t--index-pack-args.\n> > +\n> > +--index-pack-args=::\n> > +\tFor internal use only. The command to run on the contents of the\n> > +\tdownloaded pack. Arguments are URL-encoded separated by spaces.\n> > \n> > --recover::\n> > \tVerify that everything reachable from target is fetched. Used after\n> > diff --git a/fetch-pack.c b/fetch-pack.c\n> > index 876f90c759..aeac010b0b 100644\n> > --- a/fetch-pack.c\n> > +++ b/fetch-pack.c\n> > @@ -1645,6 +1645,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n> > \t\tstrvec_pushf(&cmd.args, \"--packfile=%.*s\",\n> > \t\t\t (int) the_hash_algo->hexsz,\n> > \t\t\t packfile_uris.items[i].string);\n> > +\t\tstrvec_push(&cmd.args, \"--index-pack-arg=index-pack\");\n> > +\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--stdin\");\n> > +\t\tstrvec_push(&cmd.args, \"--index-pack-arg=--keep\");\n> \n> The docs say --*-args, but the code checks --*arg, that seems like a\n> mistake that should be fixed to make the code/tests use the plural form,\n> no?\n\nThanks for catching that. Originally it was plural since this single\nargument would give multiple arguments to index-pack, but now each\nargument gives only a single argument, so \"arg\" is correct. I'll update\nit in the next version.\n"},{"id":"418277","messageId":"YEF5BfE6eXmhhiAf@google.com","threadId":"54994","inReplyTo":"57220ceb841056aade08705ca0ac73ccc69f05ab.1614021093.git.jonathantanmy@google.com","subject":"Re: [PATCH v2 2/4] http-fetch: allow custom index-pack args","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2021-03-05T00:19:17Z","receivedAt":"2021-03-05T00:19:22Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Jonathan,\n\nJonathan Tan wrote:\n\n> This is the next step in teaching fetch-pack to pass its index-pack\n> arguments when processing packfiles referenced by URIs.\n>\n> The \"--keep\" in fetch-pack.c will be replaced with a full message in a\n> subsequent commit.\n>\n> Signed-off-by: Jonathan Tan \n> Signed-off-by: Junio C Hamano \n> ---\n> Documentation/git-http-fetch.txt | 10 ++++++++--\n> fetch-pack.c | 3 +++\n> http-fetch.c | 20 +++++++++++++++-----\n> t/t5550-http-fetch-dumb.sh | 5 ++++-\n> 4 files changed, 30 insertions(+), 8 deletions(-)\n\nThis is producing an interesting symptom for me:\n\n git init repro\n cd repro\n git config fetch.uriprotocols https\n git config remote.origin.url https://fuchsia.googlesource.com/fuchsia\n git config remote.origin.fetch +refs/heads/*:refs/remotes/origin/*\n git fetch -p origin\n\nExpected result: fetches\n\nActual result:\n\n fatal: pack has bad object at offset 12: unknown object type 5\n fatal: finish_http_pack_request gave result -1\n fatal: fetch-pack: expected keep then TAB at start of http-fetch output\n\nThanks to Nathan Mulcahey (cc-ed) for a clear report.\n\nBisects to b664e9ffa153189dae9b88f32d1c5fedcf85056a, which is part of\n\"next\" and 2.31.0-rc1. Another report of the same is at\nhttps://crbug.com/1184814.\n\nKnown problem?\n\nThanks,\nJonathan\n"},{"id":"418294","messageId":"20210305011621.599649-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"YEF5BfE6eXmhhiAf@google.com","subject":"[PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-03-05T01:16:20Z","receivedAt":"2021-03-05T01:16:27Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"When fetching (as opposed to cloning) from a repository with packfile\nURIs enabled, an error like this may occur:\n\n fatal: pack has bad object at offset 12: unknown object type 5\n fatal: finish_http_pack_request gave result -1\n fatal: fetch-pack: expected keep then TAB at start of http-fetch output\n\nThis bug was introduced in b664e9ffa1 (\"fetch-pack: with packfile URIs,\nuse index-pack arg\", 2021-02-22), when the index-pack args used when\nprocessing the inline packfile of a fetch response and when processing\npackfile URIs were unified.\n\nThis bug happens because fetch, by default, partially reads (and\nconsumes) the header of the inline packfile to determine if it should\nstore the downloaded objects as a packfile or loose objects, and thus\npasses --pack_header=<...> to index-pack to inform it that some bytes\nare missing. However, when it subsequently fetches the additional\npackfiles linked by URIs, it reuses the same index-pack arguments, thus\nwrongly passing --index-pack-arg=--pack_header=<...> when no bytes are\nmissing.\n\nThis does not happen when cloning because \"git clone\" always passes\ndo_keep, which instructs the fetch mechanism to always retain the\npackfile, eliminating the need to read the header.\n\nThere are a few ways to fix this, including filtering out pack_header\narguments when downloading the additional packfiles, but I decided to\nstick to always using index-pack throughout when packfile URIs are\npresent - thus, Git no longer needs to read the bytes, and no longer\nneeds --pack_header here.\n\nSigned-off-by: Jonathan Tan \n---\nHere's a fix for this issue.\n\nThis is on jt/transfer-fsck-across-packs.\n\nOne simplification that we could do is to eliminate the unpack-objects\ncodepath. As far as I understand, the main advantage of writing loose\nobjects is that we have automatic SHA-1 collision detection, but we have\nsuch mitigations when writing packs too, so that might not be as large a\nbenefit as we think. This simplification would have enabled us to avoid\nthis bug, I think.\n---\n fetch-pack.c | 4 ++--\n t/t5702-protocol-v2.sh | 21 +++++++++++++++++++++\n 2 files changed, 23 insertions(+), 2 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex f9def5ac74..e990607742 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -852,7 +852,7 @@ static int get_pack(struct fetch_pack_args *args,\n \telse\n \t\tdemux.out = xd[0];\n \n-\tif (!args->keep_pack && unpack_limit) {\n+\tif (!args->keep_pack && unpack_limit && !index_pack_args) {\n \n \t\tif (read_pack_header(demux.out, &header))\n \t\t\tdie(_(\"protocol error: bad pack header\"));\n@@ -885,7 +885,7 @@ static int get_pack(struct fetch_pack_args *args,\n \t\t\tstrvec_push(&cmd.args, \"-v\");\n \t\tif (args->use_thin_pack)\n \t\t\tstrvec_push(&cmd.args, \"--fix-thin\");\n-\t\tif (do_keep && (args->lock_pack || unpack_limit)) {\n+\t\tif ((do_keep || index_pack_args) && (args->lock_pack || unpack_limit)) {\n \t\t\tchar hostname[HOST_NAME_MAX + 1];\n \t\t\tif (xgethostname(hostname, sizeof(hostname)))\n \t\t\t\txsnprintf(hostname, sizeof(hostname), \"localhost\");\ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex b1bc73a9a9..9df1ec82ca 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -853,6 +853,27 @@ test_expect_success 'part of packfile response provided as URI' '\n \ttest_line_count = 6 filelist\n '\n \n+test_expect_success 'packfile URIs with fetch instead of clone' '\n+\tP=\"$HTTPD_DOCUMENT_ROOT_PATH/http_parent\" &&\n+\trm -rf \"$P\" http_child log &&\n+\n+\tgit init \"$P\" &&\n+\tgit -C \"$P\" config \"uploadpack.allowsidebandall\" \"true\" &&\n+\n+\techo my-blob >\"$P/my-blob\" &&\n+\tgit -C \"$P\" add my-blob &&\n+\tgit -C \"$P\" commit -m x &&\n+\n+\tconfigure_exclusion \"$P\" my-blob >h &&\n+\n+\tgit init http_child &&\n+\n+\tGIT_TEST_SIDEBAND_ALL=1 \\\n+\tgit -C http_child -c protocol.version=2 \\\n+\t\t-c fetch.uriprotocols=http,https \\\n+\t\tfetch \"$HTTPD_URL/smart/http_parent\"\n+'\n+\n test_expect_success 'fetching with valid packfile URI but invalid hash fails' '\n \tP=\"$HTTPD_DOCUMENT_ROOT_PATH/http_parent\" &&\n \trm -rf \"$P\" http_child log &&\n-- \n2.30.1.766.gb4fecdf3b7-goog\n\n"},{"id":"418296","messageId":"xmqqh7lqbc3x.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"20210305011621.599649-1-jonathantanmy@google.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-05T01:52:50Z","receivedAt":"2021-03-05T01:52:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> One simplification that we could do is to eliminate the unpack-objects\n> codepath. As far as I understand, the main advantage of writing loose\n> objects is that we have automatic SHA-1 collision detection, but we have\n> such mitigations when writing packs too, so that might not be as large a\n> benefit as we think. This simplification would have enabled us to avoid\n> this bug, I think.\n\nMy understanding is that the primary advantage of loose objects\ncodepath is to help us avoid having too many little packs (instead,\nwe can accumulate enough objects in the loose form and let GC pack\nthem, at least the ones among them that are still reachable, into a\nsingle pack). Historically, the only mode of operation \"repack\"\noffers that reduces the number of remaining packs has been \"do full\nreachability of the entire history, and pack everything into one\",\nso avoiding creation of little packs and leaving things loose until\nwe accumulate enough used to matter.\n\nWith the geometric rolling repacking, it may not matter as much, and\nkeeping everything packed, even in a small pack, might start to be\noverall win. So I am not opposed to such a simplification; we may\nnot be ready for it right now, but I think it would be a sensible\nfuture direction.\n\n\n\n\n"},{"id":"418332","messageId":"xmqqzgzha0z5.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"20210305011621.599649-1-jonathantanmy@google.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-05T18:50:54Z","receivedAt":"2021-03-05T18:51:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> When fetching (as opposed to cloning) from a repository with packfile\n> URIs enabled, an error like this may occur:\n>\n> fatal: pack has bad object at offset 12: unknown object type 5\n> fatal: finish_http_pack_request gave result -1\n> fatal: fetch-pack: expected keep then TAB at start of http-fetch output\n>\n> This bug was introduced in b664e9ffa1 (\"fetch-pack: with packfile URIs,\n> use index-pack arg\", 2021-02-22), when the index-pack args used when\n> processing the inline packfile of a fetch response and when processing\n> packfile URIs were unified.\n\n> This bug happens because fetch, by default, partially reads (and\n> consumes) the header of the inline packfile to determine if it should\n> store the downloaded objects as a packfile or loose objects, and thus\n> passes --pack_header=<...> to index-pack to inform it that some bytes\n> are missing. \n\n... and what the values in them are.\n\n> However, when it subsequently fetches the additional\n> packfiles linked by URIs, it reuses the same index-pack arguments, thus\n> wrongly passing --index-pack-arg=--pack_header=<...> when no bytes are\n> missing.\n>\n> This does not happen when cloning because \"git clone\" always passes\n> do_keep, which instructs the fetch mechanism to always retain the\n> packfile, eliminating the need to read the header.\n>\n> There are a few ways to fix this, including filtering out pack_header\n> arguments when downloading the additional packfiles, but ...\n\nAvoiding the condition that exhibits the breakage is possible, and I\nthink it is what is done here, but I actually think that the only\nright fix is to pass correct argument to commands we invoke in the\nfirst place. Why are we reusing the same argument array to begin\nwith?\n\n ... goes back and reads the offending commit ...\n\ncommit b664e9ffa153189dae9b88f32d1c5fedcf85056a\nAuthor: Jonathan Tan \nDate: Mon Feb 22 11:20:08 2021 -0800\n\n fetch-pack: with packfile URIs, use index-pack arg\n \n Unify the index-pack arguments used when processing the inline pack and\n when downloading packfiles referenced by URIs. This is done by teaching\n get_pack() to also store the index-pack arguments whenever at least one\n packfile URI is given, and then when processing the packfile URI(s),\n using the stored arguments.\n\nTHis makes it sound like the entire idea of this offending commit\nwas wrong, and before it, the codepath that processed the packfile\nfetched from the packfile URI were using the index-pack correctly\nby using index-pack arguments that are independent from the one that\nis used to process the packfile given in-stream. Why isn't the fix\njust a straight revert of the commit???\n\n> This is on jt/transfer-fsck-across-packs.\n\nOuch. This definitely is an -rc material.\n\n\n> -\tif (!args->keep_pack && unpack_limit) {\n> +\tif (!args->keep_pack && unpack_limit && !index_pack_args) {\n\nThis one makes sense as an \"avoid conditions that reveals how badly\nthe code is broken\" band-aid. When we have index-pack related\narguments, we cannot use the unpack-objects codepath even if we are\nbeing fed a tiny pack, so there is no point peeking at the beginning\nof the pack stream to find out how many objects it has. OK.\n\n> @@ -885,7 +885,7 @@ static int get_pack(struct fetch_pack_args *args,\n> \t\t\tstrvec_push(&cmd.args, \"-v\");\n> \t\tif (args->use_thin_pack)\n> \t\t\tstrvec_push(&cmd.args, \"--fix-thin\");\n> -\t\tif (do_keep && (args->lock_pack || unpack_limit)) {\n> +\t\tif ((do_keep || index_pack_args) && (args->lock_pack || unpack_limit)) {\n> \t\t\tchar hostname[HOST_NAME_MAX + 1];\n> \t\t\tif (xgethostname(hostname, sizeof(hostname)))\n> \t\t\t\txsnprintf(hostname, sizeof(hostname), \"localhost\");\n\nI do not quite get what this hunk is doing. Care to explain?\n\nThanks.\n\n> diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\n> index b1bc73a9a9..9df1ec82ca 100755\n> --- a/t/t5702-protocol-v2.sh\n> +++ b/t/t5702-protocol-v2.sh\n> @@ -853,6 +853,27 @@ test_expect_success 'part of packfile response provided as URI' '\n> \ttest_line_count = 6 filelist\n> '\n> \n> +test_expect_success 'packfile URIs with fetch instead of clone' '\n> +\tP=\"$HTTPD_DOCUMENT_ROOT_PATH/http_parent\" &&\n> +\trm -rf \"$P\" http_child log &&\n> +\n> +\tgit init \"$P\" &&\n> +\tgit -C \"$P\" config \"uploadpack.allowsidebandall\" \"true\" &&\n> +\n> +\techo my-blob >\"$P/my-blob\" &&\n> +\tgit -C \"$P\" add my-blob &&\n> +\tgit -C \"$P\" commit -m x &&\n> +\n> +\tconfigure_exclusion \"$P\" my-blob >h &&\n> +\n> +\tgit init http_child &&\n> +\n> +\tGIT_TEST_SIDEBAND_ALL=1 \\\n> +\tgit -C http_child -c protocol.version=2 \\\n> +\t\t-c fetch.uriprotocols=http,https \\\n> +\t\tfetch \"$HTTPD_URL/smart/http_parent\"\n> +'\n> +\n> test_expect_success 'fetching with valid packfile URI but invalid hash fails' '\n> \tP=\"$HTTPD_DOCUMENT_ROOT_PATH/http_parent\" &&\n> \trm -rf \"$P\" http_child log &&\n"},{"id":"418340","messageId":"xmqqmtvh9ye8.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"xmqqzgzha0z5.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-05T19:46:39Z","receivedAt":"2021-03-05T19:47:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Avoiding the condition that exhibits the breakage is possible, and I\n> think it is what is done here, but I actually think that the only\n> right fix is to pass correct argument to commands we invoke in the\n> first place. Why are we reusing the same argument array to begin\n> with?\n>\n> ... goes back and reads the offending commit ...\n>\n> commit b664e9ffa153189dae9b88f32d1c5fedcf85056a\n> Author: Jonathan Tan \n> Date: Mon Feb 22 11:20:08 2021 -0800\n>\n> fetch-pack: with packfile URIs, use index-pack arg\n> \n> Unify the index-pack arguments used when processing the inline pack and\n> when downloading packfiles referenced by URIs. This is done by teaching\n> get_pack() to also store the index-pack arguments whenever at least one\n> packfile URI is given, and then when processing the packfile URI(s),\n> using the stored arguments.\n>\n> THis makes it sound like the entire idea of this offending commit\n> was wrong, and before it, the codepath that processed the packfile\n> fetched from the packfile URI were using the index-pack correctly\n> by using index-pack arguments that are independent from the one that\n> is used to process the packfile given in-stream. Why isn't the fix\n> just a straight revert of the commit???\n\nBy the way, the band-aid in this patch may be OK for the upcoming\nrelease (purely because it is easy to see that is sufficient for\ntoday's codebase), but I said the above because I worry about the\nhealth of the codebase in the longer term. The \"pass_header\" may\nnot stay to be the only difference between the URI packfile and\nin-stream packfile in the way they make index-pack invocations.\n\n>> This is on jt/transfer-fsck-across-packs.\n>\n> Ouch. This definitely is an -rc material.\n\nThanks.\n"},{"id":"418353","messageId":"20210305225953.739816-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqqzgzha0z5.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-03-05T22:59:53Z","receivedAt":"2021-03-05T23:00:54Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Jonathan Tan writes:\n> \n> > When fetching (as opposed to cloning) from a repository with packfile\n> > URIs enabled, an error like this may occur:\n> >\n> > fatal: pack has bad object at offset 12: unknown object type 5\n> > fatal: finish_http_pack_request gave result -1\n> > fatal: fetch-pack: expected keep then TAB at start of http-fetch output\n> >\n> > This bug was introduced in b664e9ffa1 (\"fetch-pack: with packfile URIs,\n> > use index-pack arg\", 2021-02-22), when the index-pack args used when\n> > processing the inline packfile of a fetch response and when processing\n> > packfile URIs were unified.\n> \n> > This bug happens because fetch, by default, partially reads (and\n> > consumes) the header of the inline packfile to determine if it should\n> > store the downloaded objects as a packfile or loose objects, and thus\n> > passes --pack_header=<...> to index-pack to inform it that some bytes\n> > are missing. \n> \n> ... and what the values in them are.\n\nAh, that's true.\n\n> > However, when it subsequently fetches the additional\n> > packfiles linked by URIs, it reuses the same index-pack arguments, thus\n> > wrongly passing --index-pack-arg=--pack_header=<...> when no bytes are\n> > missing.\n> >\n> > This does not happen when cloning because \"git clone\" always passes\n> > do_keep, which instructs the fetch mechanism to always retain the\n> > packfile, eliminating the need to read the header.\n> >\n> > There are a few ways to fix this, including filtering out pack_header\n> > arguments when downloading the additional packfiles, but ...\n> \n> Avoiding the condition that exhibits the breakage is possible, and I\n> think it is what is done here, but I actually think that the only\n> right fix is to pass correct argument to commands we invoke in the\n> first place. Why are we reusing the same argument array to begin\n> with?\n> \n> ... goes back and reads the offending commit ...\n> \n> commit b664e9ffa153189dae9b88f32d1c5fedcf85056a\n> Author: Jonathan Tan \n> Date: Mon Feb 22 11:20:08 2021 -0800\n> \n> fetch-pack: with packfile URIs, use index-pack arg\n> \n> Unify the index-pack arguments used when processing the inline pack and\n> when downloading packfiles referenced by URIs. This is done by teaching\n> get_pack() to also store the index-pack arguments whenever at least one\n> packfile URI is given, and then when processing the packfile URI(s),\n> using the stored arguments.\n> \n> THis makes it sound like the entire idea of this offending commit\n> was wrong, and before it, the codepath that processed the packfile\n> fetched from the packfile URI were using the index-pack correctly\n> by using index-pack arguments that are independent from the one that\n> is used to process the packfile given in-stream. Why isn't the fix\n> just a straight revert of the commit???\n\nI should probably have written more in the commit message to justify the\nunification, but it is also part of a bug fix (in particular,\n--fsck-objects wasn't being passed to the index-pack that indexed the\npackfiles linked by URI) and for code health purposes (to prevent future\nbugs by eliminating the divergence). So reverting that commit would\nreintroduce another bug.\n\n> > @@ -885,7 +885,7 @@ static int get_pack(struct fetch_pack_args *args,\n> > \t\t\tstrvec_push(&cmd.args, \"-v\");\n> > \t\tif (args->use_thin_pack)\n> > \t\t\tstrvec_push(&cmd.args, \"--fix-thin\");\n> > -\t\tif (do_keep && (args->lock_pack || unpack_limit)) {\n> > +\t\tif ((do_keep || index_pack_args) && (args->lock_pack || unpack_limit)) {\n> > \t\t\tchar hostname[HOST_NAME_MAX + 1];\n> > \t\t\tif (xgethostname(hostname, sizeof(hostname)))\n> > \t\t\t\txsnprintf(hostname, sizeof(hostname), \"localhost\");\n> \n> I do not quite get what this hunk is doing. Care to explain?\n\nThe \"do_keep\" part was unnecessarily restrictive and I used a band-aid\nsolution to loosen it. I think this started from 88e2f9ed8e (\"introduce\nfetch-object: fetch one promisor object\", 2017-12-05) where I might have\nmisunderstood what do_keep was meant to do, and taught fetch-pack to use\n\"index-pack\" if do_keep is true or args->from_promisor is true. What I\nshould have done is to set do_keep to true if args->from_promisor is\ntrue. Future commits continued to do that with fsck_objects and\nindex_pack_args.\n\nMaybe what I can do is to refactor get_pack() so that do_keep retains\nits original meaning of whether to use \"index-pack\" or \"unpack-objects\",\nand then we wouldn't need this line. What do you think (code-wise and\nwhether this fits in with the release schedule, if we want to get this\nin before release)?\n"},{"id":"418354","messageId":"20210305231128.741196-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqqmtvh9ye8.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-03-05T23:11:28Z","receivedAt":"2021-03-05T23:12:32Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> By the way, the band-aid in this patch may be OK for the upcoming\n> release (purely because it is easy to see that is sufficient for\n> today's codebase), but I said the above because I worry about the\n> health of the codebase in the longer term. The \"pass_header\" may\n> not stay to be the only difference between the URI packfile and\n> in-stream packfile in the way they make index-pack invocations.\n\nThat is true, but at the same time, I think it's better to have the\narguments be the same because there are options (e.g. --promisor and\n--fsck-objects) that have to be duplicated, and I think that for the\nmost part, the URI packfiles and the inline packfile will be processed\nidentically.\n"},{"id":"418355","messageId":"xmqqv9a58a18.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"20210305225953.739816-1-jonathantanmy@google.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-05T23:18:11Z","receivedAt":"2021-03-05T23:19:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> I should probably have written more in the commit message to justify the\n> unification, but it is also part of a bug fix (in particular,\n> --fsck-objects wasn't being passed to the index-pack that indexed the\n> packfiles linked by URI) and for code health purposes (to prevent future\n> bugs by eliminating the divergence). So reverting that commit would\n> reintroduce another bug.\n\nNot necessarily. Unifying two that do not inherently have to be\nidentical makes it impossible to pass two different things, and that\nis what we are seeing in the bug this patch is trying to fix (by\nforcing the two to be identical by eliminating the unpack-objects\ncodepath in certain cases). \n\nThe right \"fix\" for the original bug would have been to keep them\nstill separate yet making it easy to pass args that must be used in\nboth of them, no?\n\n>> > -\t\tif (do_keep && (args->lock_pack || unpack_limit)) {\n>> > +\t\tif ((do_keep || index_pack_args) && (args->lock_pack || unpack_limit)) {\n>> > \t\t\tchar hostname[HOST_NAME_MAX + 1];\n>> > \t\t\tif (xgethostname(hostname, sizeof(hostname)))\n>> > \t\t\t\txsnprintf(hostname, sizeof(hostname), \"localhost\");\n>> \n>> I do not quite get what this hunk is doing. Care to explain?\n>\n> The \"do_keep\" part was unnecessarily restrictive and I used a band-aid\n> solution to loosen it. I think this started from 88e2f9ed8e (\"introduce\n> fetch-object: fetch one promisor object\", 2017-12-05) where I might have\n> misunderstood what do_keep was meant to do, and taught fetch-pack to use\n> \"index-pack\" if do_keep is true or args->from_promisor is true. What I\n> should have done is to set do_keep to true if args->from_promisor is\n> true. Future commits continued to do that with fsck_objects and\n> index_pack_args.\n\n> Maybe what I can do is to refactor get_pack() so that do_keep retains\n> its original meaning of whether to use \"index-pack\" or \"unpack-objects\",\n> and then we wouldn't need this line. What do you think (code-wise and\n> whether this fits in with the release schedule, if we want to get this\n> in before release)?\n\nHow bad is the breakage this one is trying to fix? I know it would\nonly affect folks who have to interact with the server that uses\npackfile URI feature, but do they have a workaround, perhaps with a\nconfiguration knob or command line option to ignore the packfile\nURI, and how large is the affected population?\n\nI cannot shake the feeling that we are seeing band-aid on top of\nband-aid forced by having chosen to go in a wrong direction in the\nbeginning X-<, and prefer to see the code drift even further into\nthe same direction; hence my earlier suggestion to go back to the\nroot cause by first reverting the wrong fix that introduced this bug\nand fixing the original bug in a different way.\n\nI dunno how involved the necessary surgery would be, though. If\nthis is easy to work around, perhaps it might be a better option for\nthe overall project to ship the upcoming release with this listed as\na known breakage.\n\nThanks.\n\n\n\n"},{"id":"418357","messageId":"xmqqr1kt89wk.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"xmqqmtvh9ye8.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-05T23:20:59Z","receivedAt":"2021-03-05T23:21:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n>> THis makes it sound like the entire idea of this offending commit\n>> was wrong, and before it, the codepath that processed the packfile\n>> fetched from the packfile URI were using the index-pack correctly\n>> by using index-pack arguments that are independent from the one that\n>> is used to process the packfile given in-stream. Why isn't the fix\n>> just a straight revert of the commit???\n>\n> By the way, the band-aid in this patch may be OK for the upcoming\n> release (purely because it is easy to see that is sufficient for\n> today's codebase), but I said the above because I worry about the\n> health of the codebase in the longer term. The \"pass_header\" may\n> not stay to be the only difference between the URI packfile and\n> in-stream packfile in the way they make index-pack invocations.\n\nFor example, the URI one presumably is a CDN hosted long term one,\nwhich may be a good candidate to --keep, and in-stream one,\nespecially when packfile URI feature is used, can be expected to be\nrecent small leftover bits that it is likely that we do not want to\nkeep (in fact, if they are small enough, we'd prefer to keep them\nloose).\n"},{"id":"418366","messageId":"20210306110439.27694-2-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 01/22] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:18Z","receivedAt":"2021-03-06T11:05:29Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add the object_name member to the initialization macro. This was\nomitted in 7b35efd734e (fsck_walk(): optionally name objects on the\ngo, 2016-07-17) when the field was added.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 733378f1260..2274843ba0c 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -43,8 +43,8 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n-#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n+#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n+#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418367","messageId":"20210306110439.27694-1-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 00/22] fsck: API improvements","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:17Z","receivedAt":"2021-03-06T11:05:29Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Now that jt/transfer-fsck-across-packs has been merged to master\nhere's a re-roll of v1[1]+v2[2] of this series. v2 was slimmed-down +\nhad a trivial typo fix, so I've done the range-diff against v1.\n\nThis makes the recent fetch-pack work use the fsck_msg_id API to\ndistinguish messages, and has other various cleanups and improvements\nto make the fsck API easier to use in the future.\n\nThere's a an easy merge conflict here with other in-flight changes to\nfsck. I figured it was better to send this now than wait for those to\nland.\n\n1. https://lore.kernel.org/git/20210217194246.25342-1-avarab@gmail.com/\n2. https://lore.kernel.org/git/20210218105840.11989-1-avarab@gmail.com/\n\nÆvar Arnfjörð Bjarmason (22):\n fsck.h: update FSCK_OPTIONS_* for object_name\n fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n fsck.h: reduce duplication between FSCK_OPTIONS_{DEFAULT,STRICT}\n fsck.h: add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro\n fsck.h: indent arguments to of fsck_set_msg_type\n fsck.h: use \"enum object_type\" instead of \"int\"\n fsck.c: rename variables in fsck_set_msg_type() for less confusion\n fsck.c: move definition of msg_id into append_msg_id()\n fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"\n fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n fsck.h: re-order and re-assign \"enum fsck_msg_type\"\n fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n fsck.c: undefine temporary STR macro after use\n fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n fsck.c: pass along the fsck_msg_id in the fsck_error callback\n fsck.c: add an fsck_set_msg_type() API that takes enums\n fsck.c: move gitmodules_{found,done} into fsck_options\n fetch-pack: don't needlessly copy fsck_options\n fetch-pack: use file-scope static struct for fsck_options\n fetch-pack: use new fsck API to printing dangling submodules\n\n Makefile | 1 +\n builtin/fsck.c | 7 +-\n builtin/index-pack.c | 30 ++-----\n builtin/mktag.c | 7 +-\n builtin/unpack-objects.c | 3 +-\n fetch-pack.c | 6 +-\n fsck-cb.c | 16 ++++\n fsck.c | 175 ++++++++++++---------------------------\n fsck.h | 132 ++++++++++++++++++++++++++---\n 9 files changed, 211 insertions(+), 166 deletions(-)\n create mode 100644 fsck-cb.c\n\nRange-diff:\n13: 8de91fac068 = 1: 9d809466bd1 fsck.h: update FSCK_OPTIONS_* for object_name\n -: ----------- > 2: 33e8b6d6545 fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n -: ----------- > 3: c23f7ce9e4a fsck.h: reduce duplication between FSCK_OPTIONS_{DEFAULT,STRICT}\n -: ----------- > 4: 5dde68df6c3 fsck.h: add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro\n 1: 88b347b74ed = 5: 7ae35a6e9d2 fsck.h: indent arguments to of fsck_set_msg_type\n 2: 1a60d65d2ca ! 6: dfb5f754b37 fsck.h: use use \"enum object_type\" instead of \"int\"\n @@ Metadata\n Author: Ævar Arnfjörð Bjarmason \n \n ## Commit message ##\n - fsck.h: use use \"enum object_type\" instead of \"int\"\n + fsck.h: use \"enum object_type\" instead of \"int\"\n \n Change the fsck_walk_func to use an \"enum object_type\" instead of an\n \"int\" type. The types are compatible, and ever since this was added in\n 3: 24761f269b7 ! 7: fd58ec73c6b fsck.c: rename variables in fsck_set_msg_type() for less confusion\n @@ Commit message\n It was needlessly confusing that it took a \"msg_type\" argument, but\n then later declared another \"msg_type\" of a different type.\n \n - Let's rename that to \"tmp\", and rename \"id\" to \"msg_id\" and \"msg_id\"\n - to \"msg_id_str\" etc. This will make a follow-up change smaller.\n + Let's rename that to \"severity\", and rename \"id\" to \"msg_id\" and\n + \"msg_id\" to \"msg_id_str\" etc. This will make a follow-up change\n + smaller.\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n @@ fsck.c: int is_valid_msg_type(const char *msg_id, const char *msg_type)\n \t\tint i;\n -\t\tint *msg_type;\n -\t\tALLOC_ARRAY(msg_type, FSCK_MSG_MAX);\n -+\t\tint *tmp;\n -+\t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n ++\t\tint *severity;\n ++\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n -\t\t\tmsg_type[i] = fsck_msg_type(i, options);\n -\t\toptions->msg_type = msg_type;\n -+\t\t\ttmp[i] = fsck_msg_type(i, options);\n -+\t\toptions->msg_type = tmp;\n ++\t\t\tseverity[i] = fsck_msg_type(i, options);\n ++\t\toptions->msg_type = severity;\n \t}\n \n -\toptions->msg_type[id] = type;\n 4: fb4c66f9305 = 8: 48cb4d3bb70 fsck.c: move definition of msg_id into append_msg_id()\n 5: a129dbd9964 ! 9: 2c80ad32038 fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n @@ Commit message\n \"msg_id\". This change is relatively small, and is worth the churn for\n a later change where we have different id's in the \"report\" function.\n \n + Signed-off-by: Ævar Arnfjörð Bjarmason \n +\n ## fsck.c ##\n @@ fsck.c: void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n -: ----------- > 10: 92dfbdfb624 fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"\n 6: d9bee41072e ! 11: c1c476af69b fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n @@ Commit message\n - f27d05b1704 (fsck: allow upgrading fsck warnings to errors,\n 2015-06-22)\n \n + The reason these were defined in two different places is because we\n + use FSCK_{IGNORE,INFO,FATAL} only in fsck.c, but FSCK_{ERROR,WARN} are\n + used by external callbacks.\n +\n + Untangling that would take some more work, since we expose the new\n + \"enum fsck_msg_type\" to both. Similar to \"enum object_type\" it's not\n + worth structuring the API in such a way that only those who need\n + FSCK_{ERROR,WARN} pass around a different type.\n +\n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## builtin/fsck.c ##\n @@ builtin/fsck.c: static int objerror(struct object *obj, const char *err)\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\n \n + ## builtin/index-pack.c ##\n +@@ builtin/index-pack.c: static void show_pack_info(int stat_only)\n + static int print_dangling_gitmodules(struct fsck_options *o,\n + \t\t\t\t const struct object_id *oid,\n + \t\t\t\t enum object_type object_type,\n +-\t\t\t\t int msg_type, const char *message)\n ++\t\t\t\t enum fsck_msg_type msg_type,\n ++\t\t\t\t const char *message)\n + {\n + \t/*\n + \t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\n +\n ## builtin/mktag.c ##\n @@ builtin/mktag.c: static int mktag_config(const char *var, const char *value, void *cb)\n static int mktag_fsck_error_func(struct fsck_options *o,\n @@ fsck.c: void list_config_fsck_msg_ids(struct string_list *list, const char *pref\n +static enum fsck_msg_type fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n --\tint msg_type;\n -+\tenum fsck_msg_type msg_type;\n - \n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n + \tif (!options->msg_type) {\n +-\t\tint msg_type = msg_id_info[msg_id].msg_type;\n ++\t\tenum fsck_msg_type msg_type = msg_id_info[msg_id].msg_type;\n + \n + \t\tif (options->strict && msg_type == FSCK_WARN)\n + \t\t\tmsg_type = FSCK_ERROR;\n @@ fsck.c: static int fsck_msg_type(enum fsck_msg_id msg_id,\n - \treturn msg_type;\n + \treturn options->msg_type[msg_id];\n }\n \n -static int parse_msg_type(const char *str)\n @@ fsck.c: void fsck_set_msg_type(struct fsck_options *options,\n \n \tif (!options->msg_type) {\n \t\tint i;\n --\t\tint *tmp;\n -+\t\tenum fsck_msg_type *tmp;\n - \t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n +-\t\tint *severity;\n ++\t\tenum fsck_msg_type *severity;\n + \t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n - \t\t\ttmp[i] = fsck_msg_type(i, options);\n + \t\t\tseverity[i] = fsck_msg_type(i, options);\n @@ fsck.c: static int report(struct fsck_options *options,\n {\n \tva_list ap;\n @@ fsck.h\n -#define FSCK_ERROR 1\n -#define FSCK_WARN 2\n -#define FSCK_IGNORE 3\n --\n +enum fsck_msg_type {\n -+\tFSCK_INFO = -2,\n ++\tFSCK_INFO = -2,\n +\tFSCK_FATAL = -1,\n +\tFSCK_ERROR = 1,\n +\tFSCK_WARN,\n +\tFSCK_IGNORE\n +};\n + \n struct fsck_options;\n struct object;\n - \n @@ fsck.h: typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n -: ----------- > 12: d55587719a5 fsck.h: re-order and re-assign \"enum fsck_msg_type\"\n 7: 423568026c3 = 13: 32828d1c78c fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n 8: cb43e832738 = 14: 5c62066235c fsck.c: undefine temporary STR macro after use\n 9: 2cd14cb4e2a = 15: f8e50fbf7d3 fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n10: 1ada154ef23 ! 16: cd74dee8769 fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n @@ fsck.c\n ## fsck.h ##\n @@ fsck.h: enum fsck_msg_type {\n \tFSCK_WARN,\n - \tFSCK_IGNORE\n };\n -+\n + \n +#define FOREACH_FSCK_MSG_ID(FUNC) \\\n +\t/* fatal errors */ \\\n +\tFUNC(NUL_IN_HEADER, FATAL) \\\n11: c4179445f22 ! 17: 234e287d081 fsck.c: pass along the fsck_msg_id in the fsck_error callback\n @@ builtin/fsck.c: static int objerror(struct object *obj, const char *err)\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\n \n + ## builtin/index-pack.c ##\n +@@ builtin/index-pack.c: static int print_dangling_gitmodules(struct fsck_options *o,\n + \t\t\t\t const struct object_id *oid,\n + \t\t\t\t enum object_type object_type,\n + \t\t\t\t enum fsck_msg_type msg_type,\n ++\t\t\t\t enum fsck_msg_id msg_id,\n + \t\t\t\t const char *message)\n + {\n + \t/*\n +@@ builtin/index-pack.c: static int print_dangling_gitmodules(struct fsck_options *o,\n + \t\tprintf(\"%s\\n\", oid_to_hex(oid));\n + \t\treturn 0;\n + \t}\n +-\treturn fsck_error_function(o, oid, object_type, msg_type, message);\n ++\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n + }\n + \n + int cmd_index_pack(int argc, const char **argv, const char *prefix)\n +\n ## builtin/mktag.c ##\n @@ builtin/mktag.c: static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n12: c1fc724f0e8 ! 18: 8049dc07391 fsck.c: add an fsck_set_msg_type() API that takes enums\n @@ fsck.c: int is_valid_msg_type(const char *msg_id, const char *msg_type)\n +{\n +\tif (!options->msg_type) {\n +\t\tint i;\n -+\t\tenum fsck_msg_type *tmp;\n -+\t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n ++\t\tenum fsck_msg_type *severity;\n ++\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n +\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n -+\t\t\ttmp[i] = fsck_msg_type(i, options);\n -+\t\toptions->msg_type = tmp;\n ++\t\t\tseverity[i] = fsck_msg_type(i, options);\n ++\t\toptions->msg_type = severity;\n +\t}\n +\n +\toptions->msg_type[msg_id] = msg_type;\n @@ fsck.c: void fsck_set_msg_type(struct fsck_options *options,\n \n -\tif (!options->msg_type) {\n -\t\tint i;\n --\t\tenum fsck_msg_type *tmp;\n --\t\tALLOC_ARRAY(tmp, FSCK_MSG_MAX);\n +-\t\tenum fsck_msg_type *severity;\n +-\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n -\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n --\t\t\ttmp[i] = fsck_msg_type(i, options);\n --\t\toptions->msg_type = tmp;\n +-\t\t\tseverity[i] = fsck_msg_type(i, options);\n +-\t\toptions->msg_type = severity;\n -\t}\n -\n -\toptions->msg_type[msg_id] = msg_type;\n14: 29ff97856ff ! 19: 4224a29d15c fsck.c: move gitmodules_{found,done} into fsck_options\n @@ Commit message\n fsck_options struct. It makes sense to keep all the context in the\n same place.\n \n + This requires changing the recently added register_found_gitmodules()\n + function added in 5476e1efde (fetch-pack: print and use dangling\n + .gitmodules, 2021-02-22) to take fsck_options. That function will be\n + removed in a subsequent commit, but as it'll require the new\n + gitmodules_found attribute of \"fsck_options\" we need this intermediate\n + step first.\n +\n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n + ## fetch-pack.c ##\n +@@ fetch-pack.c: static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n + \n + \toidset_iter_init(gitmodules_oids, &iter);\n + \twhile ((oid = oidset_iter_next(&iter)))\n +-\t\tregister_found_gitmodules(oid);\n ++\t\tregister_found_gitmodules(&fo, oid);\n + \tif (fsck_finish(&fo))\n + \t\tdie(\"fsck failed\");\n + }\n +\n ## fsck.c ##\n @@\n #include \"credential.h\"\n @@ fsck.c: static int fsck_blob(const struct object_id *oid, const char *buf,\n \n \tif (object_on_skiplist(options, oid))\n \t\treturn 0;\n +@@ fsck.c: int fsck_error_function(struct fsck_options *o,\n + \treturn 1;\n + }\n + \n +-void register_found_gitmodules(const struct object_id *oid)\n ++void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n + {\n +-\toidset_insert(&gitmodules_found, oid);\n ++\toidset_insert(&options->gitmodules_found, oid);\n + }\n + \n + int fsck_finish(struct fsck_options *options)\n @@ fsck.c: int fsck_finish(struct fsck_options *options)\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n @@ fsck.h: struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n --#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n --#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n -+#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, OIDSET_INIT, OIDSET_INIT, NULL }\n -+#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, OIDSET_INIT, OIDSET_INIT, NULL }\n +@@ fsck.h: struct fsck_options {\n + \t.walk = NULL, \\\n + \t.msg_type = NULL, \\\n + \t.skiplist = OIDSET_INIT, \\\n ++\t.gitmodules_found = OIDSET_INIT, \\\n ++\t.gitmodules_done = OIDSET_INIT, \\\n + \t.object_names = NULL,\n + #define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n + \tFSCK_OPTIONS_COMMON \\\n +@@ fsck.h: int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n + int fsck_object(struct object *obj, void *data, unsigned long size,\n + \tstruct fsck_options *options);\n + \n +-void register_found_gitmodules(const struct object_id *oid);\n ++void register_found_gitmodules(struct fsck_options *options,\n ++\t\t\t const struct object_id *oid);\n \n - /* descend in all linked child objects\n - * the return value is:\n + /*\n + * fsck a tag, and pass info about it back to the caller. This is\n -: ----------- > 20: 40b13468129 fetch-pack: don't needlessly copy fsck_options\n -: ----------- > 21: 8e418abfbd7 fetch-pack: use file-scope static struct for fsck_options\n -: ----------- > 22: 113de190f7d fetch-pack: use new fsck API to printing dangling submodules\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418368","messageId":"20210306110439.27694-3-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 02/22] fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:19Z","receivedAt":"2021-03-06T11:06:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Signed-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 18 ++++++++++++++++--\n 1 file changed, 16 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 2274843ba0c..40f3cb3f645 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -43,8 +43,22 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n-#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n+#define FSCK_OPTIONS_DEFAULT { \\\n+\t.walk = NULL, \\\n+\t.error_func = fsck_error_function, \\\n+\t.strict = 0, \\\n+\t.msg_type = NULL, \\\n+\t.skiplist = OIDSET_INIT, \\\n+\t.object_names = NULL, \\\n+}\n+#define FSCK_OPTIONS_STRICT { \\\n+\t.walk = NULL, \\\n+\t.error_func = fsck_error_function, \\\n+\t.strict = 1, \\\n+\t.msg_type = NULL, \\\n+\t.skiplist = OIDSET_INIT, \\\n+\t.object_names = NULL, \\\n+}\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418369","messageId":"20210306110439.27694-5-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 04/22] fsck.h: add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:21Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro for those that would like\nto use FSCK_OPTIONS_COMMON in their own initialization, but supply\ntheir own error functions.\n\nNothing is being changed to use this yet, but in some subsequent\ncommits we'll make use of this macro.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 9 ++++++---\n 1 file changed, 6 insertions(+), 3 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex ea3a907ec3b..dc35924cbf5 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -45,12 +45,15 @@ struct fsck_options {\n \n #define FSCK_OPTIONS_COMMON \\\n \t.walk = NULL, \\\n-\t.error_func = fsck_error_function, \\\n \t.msg_type = NULL, \\\n \t.skiplist = OIDSET_INIT, \\\n \t.object_names = NULL,\n-#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON }\n-#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON }\n+#define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n+\tFSCK_OPTIONS_COMMON \\\n+\t.error_func = fsck_error_function\n+\n+#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n+#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418370","messageId":"20210306110439.27694-4-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 03/22] fsck.h: reduce duplication between FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:20Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Use a temporary macro to define what FSCK_OPTIONS_{DEFAULT,STRICT}\nhave in common, and define the two in terms of that macro.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 16 ++++------------\n 1 file changed, 4 insertions(+), 12 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 40f3cb3f645..ea3a907ec3b 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -43,22 +43,14 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { \\\n+#define FSCK_OPTIONS_COMMON \\\n \t.walk = NULL, \\\n \t.error_func = fsck_error_function, \\\n-\t.strict = 0, \\\n \t.msg_type = NULL, \\\n \t.skiplist = OIDSET_INIT, \\\n-\t.object_names = NULL, \\\n-}\n-#define FSCK_OPTIONS_STRICT { \\\n-\t.walk = NULL, \\\n-\t.error_func = fsck_error_function, \\\n-\t.strict = 1, \\\n-\t.msg_type = NULL, \\\n-\t.skiplist = OIDSET_INIT, \\\n-\t.object_names = NULL, \\\n-}\n+\t.object_names = NULL,\n+#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON }\n+#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418371","messageId":"20210306110439.27694-7-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 06/22] fsck.h: use \"enum object_type\" instead of \"int\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:23Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_walk_func to use an \"enum object_type\" instead of an\n\"int\" type. The types are compatible, and ever since this was added in\n355885d5315 (add generic, type aware object chain walker, 2008-02-25)\nwe've used entries from object_type (OBJ_BLOB etc.).\n\nSo this doesn't really change anything as far as the generated code is\nconcerned, it just gives the compiler more information and makes this\neasier to read.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 3 ++-\n builtin/index-pack.c | 3 ++-\n builtin/unpack-objects.c | 3 ++-\n fsck.h | 3 ++-\n 4 files changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 821e7798c70..68f0329e69e 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -197,7 +197,8 @@ static int traverse_reachable(void)\n \treturn !!result;\n }\n \n-static int mark_used(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_used(struct object *obj, enum object_type object_type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn 1;\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex bad57488079..69f24fe9f76 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -212,7 +212,8 @@ static void cleanup_thread(void)\n \tfree(thread_data);\n }\n \n-static int mark_link(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_link(struct object *obj, enum object_type type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn -1;\ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex dd4a75e030d..ca54fd16688 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -187,7 +187,8 @@ static void write_cached_object(struct object *obj, struct obj_buffer *obj_buf)\n * that have reachability requirements and calls this function.\n * Verify its reachability and validity recursively and write it out.\n */\n-static int check_object(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int check_object(struct object *obj, enum object_type type,\n+\t\t\tvoid *data, struct fsck_options *options)\n {\n \tstruct obj_buffer *obj_buf;\n \ndiff --git a/fsck.h b/fsck.h\nindex 5e488cef6b3..f67edd8f1f9 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -23,7 +23,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type);\n * <0\terror signaled and abort\n * >0\terror signaled and do not abort\n */\n-typedef int (*fsck_walk_func)(struct object *obj, int type, void *data, struct fsck_options *options);\n+typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n+\t\t\t void *data, struct fsck_options *options);\n \n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418372","messageId":"20210306110439.27694-9-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 08/22] fsck.c: move definition of msg_id into append_msg_id()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:25Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor code added in 71ab8fa840f (fsck: report the ID of the\nerror/warning, 2015-06-22) to resolve the msg_id to a string in the\nfunction that wants it, instead of doing it in report().\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 0a9ac9ca070..b977493f57a 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,8 +264,9 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, const char *msg_id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n {\n+\tconst char *msg_id = msg_id_info[id].id_string;\n \tfor (;;) {\n \t\tchar c = *(msg_id)++;\n \n@@ -308,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, msg_id_info[id].id_string);\n+\tappend_msg_id(&sb, id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418373","messageId":"20210306110439.27694-6-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 05/22] fsck.h: indent arguments to of fsck_set_msg_type","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:22Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Signed-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex dc35924cbf5..5e488cef6b3 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -11,7 +11,7 @@ struct fsck_options;\n struct object;\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type);\n+\t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n int is_valid_msg_type(const char *msg_id, const char *msg_type);\n \n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418374","messageId":"20210306110439.27694-11-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 10/22] fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:27Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor \"if options->msg_type\" and other code added in\n0282f4dced0 (fsck: offer a function to demote fsck errors to warnings,\n2015-06-22) to reduce the scope of the \"int msg_type\" variable.\n\nThis is in preparation for changing its type in a subsequent commit,\nonly using it in the \"!options->msg_type\" scope makes that change\n\nThis also brings the code in line with the fsck_set_msg_type()\nfunction (also added in 0282f4dced0), which does a similar check for\n\"!options->msg_type\". Another minor benefit is getting rid of the\nstyle violation of not having braces for the body of the \"if\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 12 +++++-------\n 1 file changed, 5 insertions(+), 7 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 6b72ddaa51d..0988ab65792 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -167,19 +167,17 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n static int fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n-\tint msg_type;\n-\n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n-\tif (options->msg_type)\n-\t\tmsg_type = options->msg_type[msg_id];\n-\telse {\n-\t\tmsg_type = msg_id_info[msg_id].msg_type;\n+\tif (!options->msg_type) {\n+\t\tint msg_type = msg_id_info[msg_id].msg_type;\n+\n \t\tif (options->strict && msg_type == FSCK_WARN)\n \t\t\tmsg_type = FSCK_ERROR;\n+\t\treturn msg_type;\n \t}\n \n-\treturn msg_type;\n+\treturn options->msg_type[msg_id];\n }\n \n static int parse_msg_type(const char *str)\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418375","messageId":"20210306110439.27694-14-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 13/22] fsck.c: call parse_msg_type() early in fsck_set_msg_type()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:30Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"There's no reason to defer the calling of parse_msg_type() until after\nwe've checked if the \"id < 0\". This is not a hot codepath, and\nparse_msg_type() itself may die on invalid input.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex fb7d071bbf9..2ccf1a2f0fd 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -201,11 +201,10 @@ void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n \tint msg_id = parse_msg_id(msg_id_str);\n-\tenum fsck_msg_type msg_type;\n+\tenum fsck_msg_type msg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n-\tmsg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418376","messageId":"20210306110439.27694-16-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 15/22] fsck.c: give \"FOREACH_MSG_ID\" a more specific name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:32Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the FOREACH_MSG_ID macro to FOREACH_FSCK_MSG_ID in preparation\nfor moving it over to fsck.h. It's good convention to name macros\nin *.h files in such a way as to clearly not clash with any other\nnames in other files.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex f4c924ed044..6fbc56e9faa 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,7 +22,7 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_MSG_ID(FUNC) \\\n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n \tFUNC(UNTERMINATED_HEADER, FATAL) \\\n@@ -83,7 +83,7 @@ static struct oidset gitmodules_done = OIDSET_INIT;\n \n #define MSG_ID(id, msg_type) FSCK_MSG_##id,\n enum fsck_msg_id {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \tFSCK_MSG_MAX\n };\n #undef MSG_ID\n@@ -96,7 +96,7 @@ static struct {\n \tconst char *camelcased;\n \tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418377","messageId":"20210306110439.27694-10-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 09/22] fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:26Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the remaining variables of type fsck_msg_id from \"id\" to\n\"msg_id\". This change is relatively small, and is worth the churn for\na later change where we have different id's in the \"report\" function.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex b977493f57a..6b72ddaa51d 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,19 +264,19 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id msg_id)\n {\n-\tconst char *msg_id = msg_id_info[id].id_string;\n+\tconst char *msg_id_str = msg_id_info[msg_id].id_string;\n \tfor (;;) {\n-\t\tchar c = *(msg_id)++;\n+\t\tchar c = *(msg_id_str)++;\n \n \t\tif (!c)\n \t\t\tbreak;\n \t\tif (c != '_')\n \t\t\tstrbuf_addch(sb, tolower(c));\n \t\telse {\n-\t\t\tassert(*msg_id);\n-\t\t\tstrbuf_addch(sb, *(msg_id)++);\n+\t\t\tassert(*msg_id_str);\n+\t\t\tstrbuf_addch(sb, *(msg_id_str)++);\n \t\t}\n \t}\n \n@@ -292,11 +292,11 @@ static int object_on_skiplist(struct fsck_options *opts,\n __attribute__((format (printf, 5, 6)))\n static int report(struct fsck_options *options,\n \t\t const struct object_id *oid, enum object_type object_type,\n-\t\t enum fsck_msg_id id, const char *fmt, ...)\n+\t\t enum fsck_msg_id msg_id, const char *fmt, ...)\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(id, options), result;\n+\tint msg_type = fsck_msg_type(msg_id, options), result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -309,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, id);\n+\tappend_msg_id(&sb, msg_id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418378","messageId":"20210306110439.27694-12-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 11/22] fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:28Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} defines into a new\nfsck_msg_type enum.\n\nThese defines were originally introduced in:\n\n - ba002f3b28a (builtin-fsck: move common object checking code to\n fsck.c, 2008-02-25)\n - f50c4407305 (fsck: disallow demoting grave fsck errors to warnings,\n 2015-06-22)\n - efaba7cc77f (fsck: optionally ignore specific fsck issues\n completely, 2015-06-22)\n - f27d05b1704 (fsck: allow upgrading fsck warnings to errors,\n 2015-06-22)\n\nThe reason these were defined in two different places is because we\nuse FSCK_{IGNORE,INFO,FATAL} only in fsck.c, but FSCK_{ERROR,WARN} are\nused by external callbacks.\n\nUntangling that would take some more work, since we expose the new\n\"enum fsck_msg_type\" to both. Similar to \"enum object_type\" it's not\nworth structuring the API in such a way that only those who need\nFSCK_{ERROR,WARN} pass around a different type.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 2 +-\n builtin/index-pack.c | 3 ++-\n builtin/mktag.c | 3 ++-\n fsck.c | 21 ++++++++++-----------\n fsck.h | 16 ++++++++++------\n 5 files changed, 25 insertions(+), 20 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 68f0329e69e..d6d745dc702 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -89,7 +89,7 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t int msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type, const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 69f24fe9f76..56b8efaa89b 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1716,7 +1716,8 @@ static void show_pack_info(int stat_only)\n static int print_dangling_gitmodules(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \t/*\n \t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 41a399a69e4..1834394a9b6 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -22,7 +22,8 @@ static int mktag_config(const char *var, const char *value, void *cb)\n static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/fsck.c b/fsck.c\nindex 0988ab65792..fb7d071bbf9 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,9 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FSCK_FATAL -1\n-#define FSCK_INFO -2\n-\n #define FOREACH_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n@@ -97,7 +94,7 @@ static struct {\n \tconst char *id_string;\n \tconst char *downcased;\n \tconst char *camelcased;\n-\tint msg_type;\n+\tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n \tFOREACH_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n@@ -164,13 +161,13 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n \t\tlist_config_item(list, prefix, msg_id_info[i].camelcased);\n }\n \n-static int fsck_msg_type(enum fsck_msg_id msg_id,\n+static enum fsck_msg_type fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n \tif (!options->msg_type) {\n-\t\tint msg_type = msg_id_info[msg_id].msg_type;\n+\t\tenum fsck_msg_type msg_type = msg_id_info[msg_id].msg_type;\n \n \t\tif (options->strict && msg_type == FSCK_WARN)\n \t\t\tmsg_type = FSCK_ERROR;\n@@ -180,7 +177,7 @@ static int fsck_msg_type(enum fsck_msg_id msg_id,\n \treturn options->msg_type[msg_id];\n }\n \n-static int parse_msg_type(const char *str)\n+static enum fsck_msg_type parse_msg_type(const char *str)\n {\n \tif (!strcmp(str, \"error\"))\n \t\treturn FSCK_ERROR;\n@@ -203,7 +200,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n-\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n+\tint msg_id = parse_msg_id(msg_id_str);\n+\tenum fsck_msg_type msg_type;\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n@@ -214,7 +212,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *severity;\n+\t\tenum fsck_msg_type *severity;\n \t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n \t\t\tseverity[i] = fsck_msg_type(i, options);\n@@ -294,7 +292,8 @@ static int report(struct fsck_options *options,\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(msg_id, options), result;\n+\tenum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);\n+\tint result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -1265,7 +1264,7 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tint msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type, const char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex f67edd8f1f9..2ecc15eee77 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -3,9 +3,13 @@\n \n #include \"oidset.h\"\n \n-#define FSCK_ERROR 1\n-#define FSCK_WARN 2\n-#define FSCK_IGNORE 3\n+enum fsck_msg_type {\n+\tFSCK_INFO = -2,\n+\tFSCK_FATAL = -1,\n+\tFSCK_ERROR = 1,\n+\tFSCK_WARN,\n+\tFSCK_IGNORE\n+};\n \n struct fsck_options;\n struct object;\n@@ -29,17 +33,17 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t int msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tint msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, const char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n \tfsck_error error_func;\n \tunsigned strict:1;\n-\tint *msg_type;\n+\tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n \tkh_oid_map_t *object_names;\n };\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418388","messageId":"20210306110439.27694-8-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 07/22] fsck.c: rename variables in fsck_set_msg_type() for less confusion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:24Z","receivedAt":"2021-03-06T11:06:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename variables in a function added in 0282f4dced0 (fsck: offer a\nfunction to demote fsck errors to warnings, 2015-06-22).\n\nIt was needlessly confusing that it took a \"msg_type\" argument, but\nthen later declared another \"msg_type\" of a different type.\n\nLet's rename that to \"severity\", and rename \"id\" to \"msg_id\" and\n\"msg_id\" to \"msg_id_str\" etc. This will make a follow-up change\nsmaller.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 24 ++++++++++++------------\n 1 file changed, 12 insertions(+), 12 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex e3030f3b358..0a9ac9ca070 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -203,27 +203,27 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n }\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type)\n+\t\tconst char *msg_id_str, const char *msg_type_str)\n {\n-\tint id = parse_msg_id(msg_id), type;\n+\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n \n-\tif (id < 0)\n-\t\tdie(\"Unhandled message id: %s\", msg_id);\n-\ttype = parse_msg_type(msg_type);\n+\tif (msg_id < 0)\n+\t\tdie(\"Unhandled message id: %s\", msg_id_str);\n+\tmsg_type = parse_msg_type(msg_type_str);\n \n-\tif (type != FSCK_ERROR && msg_id_info[id].msg_type == FSCK_FATAL)\n-\t\tdie(\"Cannot demote %s to %s\", msg_id, msg_type);\n+\tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n+\t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *msg_type;\n-\t\tALLOC_ARRAY(msg_type, FSCK_MSG_MAX);\n+\t\tint *severity;\n+\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tmsg_type[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = msg_type;\n+\t\t\tseverity[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = severity;\n \t}\n \n-\toptions->msg_type[id] = type;\n+\toptions->msg_type[msg_id] = msg_type;\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418379","messageId":"20210306110439.27694-20-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 19/22] fsck.c: move gitmodules_{found,done} into fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:36Z","receivedAt":"2021-03-06T11:06:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the gitmodules_{found,done} static variables added in\n159e7b080bf (fsck: detect gitmodules files, 2018-05-02) into the\nfsck_options struct. It makes sense to keep all the context in the\nsame place.\n\nThis requires changing the recently added register_found_gitmodules()\nfunction added in 5476e1efde (fetch-pack: print and use dangling\n.gitmodules, 2021-02-22) to take fsck_options. That function will be\nremoved in a subsequent commit, but as it'll require the new\ngitmodules_found attribute of \"fsck_options\" we need this intermediate\nstep first.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fetch-pack.c | 2 +-\n fsck.c | 23 ++++++++++-------------\n fsck.h | 7 ++++++-\n 3 files changed, 17 insertions(+), 15 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 0cb59acc486..53d7ef00856 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -998,7 +998,7 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(oid);\n+\t\tregister_found_gitmodules(&fo, oid);\n \tif (fsck_finish(&fo))\n \t\tdie(\"fsck failed\");\n }\ndiff --git a/fsck.c b/fsck.c\nindex f26f47b2a10..565274a946c 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -19,9 +19,6 @@\n #include \"credential.h\"\n #include \"help.h\"\n \n-static struct oidset gitmodules_found = OIDSET_INIT;\n-static struct oidset gitmodules_done = OIDSET_INIT;\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\n@@ -624,7 +621,7 @@ static int fsck_tree(const struct object_id *oid,\n \n \t\tif (is_hfs_dotgitmodules(name) || is_ntfs_dotgitmodules(name)) {\n \t\t\tif (!S_ISLNK(mode))\n-\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\telse\n \t\t\t\tretval += report(options,\n \t\t\t\t\t\t oid, OBJ_TREE,\n@@ -638,7 +635,7 @@ static int fsck_tree(const struct object_id *oid,\n \t\t\t\thas_dotgit |= is_ntfs_dotgit(backslash);\n \t\t\t\tif (is_ntfs_dotgitmodules(backslash)) {\n \t\t\t\t\tif (!S_ISLNK(mode))\n-\t\t\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\t\t\telse\n \t\t\t\t\t\tretval += report(options, oid, OBJ_TREE,\n \t\t\t\t\t\t\t\t FSCK_MSG_GITMODULES_SYMLINK,\n@@ -1150,9 +1147,9 @@ static int fsck_blob(const struct object_id *oid, const char *buf,\n \tstruct fsck_gitmodules_data data;\n \tstruct config_options config_opts = { 0 };\n \n-\tif (!oidset_contains(&gitmodules_found, oid))\n+\tif (!oidset_contains(&options->gitmodules_found, oid))\n \t\treturn 0;\n-\toidset_insert(&gitmodules_done, oid);\n+\toidset_insert(&options->gitmodules_done, oid);\n \n \tif (object_on_skiplist(options, oid))\n \t\treturn 0;\n@@ -1217,9 +1214,9 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n-void register_found_gitmodules(const struct object_id *oid)\n+void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n {\n-\toidset_insert(&gitmodules_found, oid);\n+\toidset_insert(&options->gitmodules_found, oid);\n }\n \n int fsck_finish(struct fsck_options *options)\n@@ -1228,13 +1225,13 @@ int fsck_finish(struct fsck_options *options)\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n \n-\toidset_iter_init(&gitmodules_found, &iter);\n+\toidset_iter_init(&options->gitmodules_found, &iter);\n \twhile ((oid = oidset_iter_next(&iter))) {\n \t\tenum object_type type;\n \t\tunsigned long size;\n \t\tchar *buf;\n \n-\t\tif (oidset_contains(&gitmodules_done, oid))\n+\t\tif (oidset_contains(&options->gitmodules_done, oid))\n \t\t\tcontinue;\n \n \t\tbuf = read_object_file(oid, &type, &size);\n@@ -1259,8 +1256,8 @@ int fsck_finish(struct fsck_options *options)\n \t}\n \n \n-\toidset_clear(&gitmodules_found);\n-\toidset_clear(&gitmodules_done);\n+\toidset_clear(&options->gitmodules_found);\n+\toidset_clear(&options->gitmodules_done);\n \treturn ret;\n }\n \ndiff --git a/fsck.h b/fsck.h\nindex 6c2fd9c5cc0..bb59ef05b68 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -118,6 +118,8 @@ struct fsck_options {\n \tunsigned strict:1;\n \tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n+\tstruct oidset gitmodules_found;\n+\tstruct oidset gitmodules_done;\n \tkh_oid_map_t *object_names;\n };\n \n@@ -125,6 +127,8 @@ struct fsck_options {\n \t.walk = NULL, \\\n \t.msg_type = NULL, \\\n \t.skiplist = OIDSET_INIT, \\\n+\t.gitmodules_found = OIDSET_INIT, \\\n+\t.gitmodules_done = OIDSET_INIT, \\\n \t.object_names = NULL,\n #define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n \tFSCK_OPTIONS_COMMON \\\n@@ -149,7 +153,8 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n-void register_found_gitmodules(const struct object_id *oid);\n+void register_found_gitmodules(struct fsck_options *options,\n+\t\t\t const struct object_id *oid);\n \n /*\n * fsck a tag, and pass info about it back to the caller. This is\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418380","messageId":"20210306110439.27694-19-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 18/22] fsck.c: add an fsck_set_msg_type() API that takes enums","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:35Z","receivedAt":"2021-03-06T11:06:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code I added in acf9de4c94e (mktag: use fsck instead of custom\nverify_tag(), 2021-01-05) to make use of a new API function that takes\nthe fsck_msg_{id,type} types, instead of arbitrary strings that\nwe'll (hopefully) parse into those types.\n\nAt the time that the fsck_set_msg_type() API was introduced in\n0282f4dced0 (fsck: offer a function to demote fsck errors to warnings,\n2015-06-22) it was only intended to be used to parse user-supplied\ndata.\n\nFor things that are purely internal to the C code it makes sense to\nhave the compiler check these arguments, and to skip the sanity\nchecking of the data in fsck_set_msg_type() which is redundant to\nchecks we get from the compiler.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/mktag.c | 3 ++-\n fsck.c | 27 +++++++++++++++++----------\n fsck.h | 3 +++\n 3 files changed, 22 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex dc989c356f5..de67a94f24e 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -93,7 +93,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)\n \t\tdie_errno(_(\"could not read from stdin\"));\n \n \tfsck_options.error_func = mktag_fsck_error_func;\n-\tfsck_set_msg_type(&fsck_options, \"extraheaderentry\", \"warn\");\n+\tfsck_set_msg_type_from_ids(&fsck_options, FSCK_MSG_EXTRA_HEADER_ENTRY,\n+\t\t\t\t FSCK_WARN);\n \t/* config might set fsck.extraHeaderEntry=* again */\n \tgit_config(mktag_config, NULL);\n \tif (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,\ndiff --git a/fsck.c b/fsck.c\nindex 5a040eb4fd5..f26f47b2a10 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -132,6 +132,22 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n \treturn 1;\n }\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type)\n+{\n+\tif (!options->msg_type) {\n+\t\tint i;\n+\t\tenum fsck_msg_type *severity;\n+\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n+\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n+\t\t\tseverity[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = severity;\n+\t}\n+\n+\toptions->msg_type[msg_id] = msg_type;\n+}\n+\n void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n@@ -144,16 +160,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n-\tif (!options->msg_type) {\n-\t\tint i;\n-\t\tenum fsck_msg_type *severity;\n-\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n-\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tseverity[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = severity;\n-\t}\n-\n-\toptions->msg_type[msg_id] = msg_type;\n+\tfsck_set_msg_type_from_ids(options, msg_id, msg_type);\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\ndiff --git a/fsck.h b/fsck.h\nindex 33ecf3f3f16..6c2fd9c5cc0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -82,6 +82,9 @@ enum fsck_msg_id {\n struct fsck_options;\n struct object;\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type);\n void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418381","messageId":"20210306110439.27694-13-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 12/22] fsck.h: re-order and re-assign \"enum fsck_msg_type\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:29Z","receivedAt":"2021-03-06T11:06:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the values in the \"enum fsck_msg_type\" from being manually\nassigned to using default C enum values.\n\nThis means we end up with a FSCK_IGNORE=0, which was previously\ndefined as \"2\".\n\nI'm confident that nothing relies on these values, we always compare\nthem explicitly. Let's not omit \"0\" so it won't be assumed that we're\nusing these as a boolean somewhere.\n\nThis also allows us to re-structure the fields to mark which are\n\"private\" v.s. \"public\". See the preceding commit for a rationale for\nnot simply splitting these into two enums, namely that this is used\nfor both the private and public fsck API.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 10 ++++++----\n 1 file changed, 6 insertions(+), 4 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 2ecc15eee77..fce9981a0cb 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -4,11 +4,13 @@\n #include \"oidset.h\"\n \n enum fsck_msg_type {\n-\tFSCK_INFO = -2,\n-\tFSCK_FATAL = -1,\n-\tFSCK_ERROR = 1,\n+\t/* for internal use only */\n+\tFSCK_IGNORE,\n+\tFSCK_INFO,\n+\tFSCK_FATAL,\n+\t/* \"public\", fed to e.g. error_func callbacks */\n+\tFSCK_ERROR,\n \tFSCK_WARN,\n-\tFSCK_IGNORE\n };\n \n struct fsck_options;\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418382","messageId":"20210306110439.27694-18-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 17/22] fsck.c: pass along the fsck_msg_id in the fsck_error callback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:34Z","receivedAt":"2021-03-06T11:06:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_error callback to also pass along the\nfsck_msg_id. Before this change the only way to get the message id was\nto parse it back out of the \"message\".\n\nLet's pass it down explicitly for the benefit of callers that might\nwant to use it, as discussed in [1].\n\nPassing the msg_type is now redundant, as you can always get it back\nfrom the msg_id, but I'm not changing that convention. It's really\ncommon to need the msg_type, and the report() function itself (which\ncalls \"fsck_error\") needs to call fsck_msg_type() to discover\nit. Let's not needlessly re-do that work in the user callback.\n\n1. https://lore.kernel.org/git/87blcja2ha.fsf@evledraar.gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 4 +++-\n builtin/index-pack.c | 3 ++-\n builtin/mktag.c | 1 +\n fsck.c | 6 ++++--\n fsck.h | 6 ++++--\n 5 files changed, 14 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex d6d745dc702..b71fac4ceca 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -89,7 +89,9 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 56b8efaa89b..2b2266a4b7d 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1717,6 +1717,7 @@ static int print_dangling_gitmodules(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n \t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n \t\t\t\t const char *message)\n {\n \t/*\n@@ -1727,7 +1728,7 @@ static int print_dangling_gitmodules(struct fsck_options *o,\n \t\tprintf(\"%s\\n\", oid_to_hex(oid));\n \t\treturn 0;\n \t}\n-\treturn fsck_error_function(o, oid, object_type, msg_type, message);\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n }\n \n int cmd_index_pack(int argc, const char **argv, const char *prefix)\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 1834394a9b6..dc989c356f5 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -23,6 +23,7 @@ static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n \t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n \t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\ndiff --git a/fsck.c b/fsck.c\nindex 8a66168e516..5a040eb4fd5 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -245,7 +245,7 @@ static int report(struct fsck_options *options,\n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n \tresult = options->error_func(options, oid, object_type,\n-\t\t\t\t msg_type, sb.buf);\n+\t\t\t\t msg_type, msg_id, sb.buf);\n \tstrbuf_release(&sb);\n \tva_end(ap);\n \n@@ -1198,7 +1198,9 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type,\n+\t\t\tenum fsck_msg_id msg_id,\n+\t\t\tconst char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex c3d3b47b88b..33ecf3f3f16 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -101,11 +101,13 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\t const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\tconst char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418383","messageId":"20210306110439.27694-22-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 21/22] fetch-pack: use file-scope static struct for fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:38Z","receivedAt":"2021-03-06T11:06:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code added in 5476e1efde (fetch-pack: print and use dangling\n.gitmodules, 2021-02-22) so that we use a file-scoped \"static struct\nfsck_options\" instead of defining one in the \"fsck_gitmodules_oids()\"\nfunction.\n\nWe use this pattern in all of\nbuiltin/{fsck,index-pack,mktag,unpack-objects}.c. It's odd to see\nfetch-pack be the odd one out. One might think that we're using other\nfsck_options structs in fetch-pack, or doing on fsck twice there, but\nwe're not.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fetch-pack.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 53d7ef00856..f961c3067cd 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -38,6 +38,7 @@ static int server_supports_filtering;\n static int advertise_sid;\n static struct shallow_lock shallow_lock;\n static const char *alternate_shallow_file;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n \n@@ -991,15 +992,14 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n {\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n-\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n \n \tif (!oidset_size(gitmodules_oids))\n \t\treturn;\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(&fo, oid);\n-\tif (fsck_finish(&fo))\n+\t\tregister_found_gitmodules(&fsck_options, oid);\n+\tif (fsck_finish(&fsck_options))\n \t\tdie(\"fsck failed\");\n }\n \n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418384","messageId":"20210306110439.27694-15-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 14/22] fsck.c: undefine temporary STR macro after use","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:31Z","receivedAt":"2021-03-06T11:06:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In f417eed8cde (fsck: provide a function to parse fsck message IDs,\n2015-06-22) the \"STR\" macro was introduced, but that short macro name\nwas not undefined after use as was done earlier in the same series for\nthe MSG_ID macro in c99ba492f1c (fsck: introduce identifiers for fsck\nmessages, 2015-06-22).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/fsck.c b/fsck.c\nindex 2ccf1a2f0fd..f4c924ed044 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -100,6 +100,7 @@ static struct {\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n+#undef STR\n \n static void prepare_msg_ids(void)\n {\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418385","messageId":"20210306110439.27694-23-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 22/22] fetch-pack: use new fsck API to printing dangling submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:39Z","receivedAt":"2021-03-06T11:06:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor the check added in 5476e1efde (fetch-pack: print and use\ndangling .gitmodules, 2021-02-22) to make use of us now passing the\n\"msg_id\" to the user defined \"error_func\". We can now compare against\nthe FSCK_MSG_GITMODULES_MISSING instead of parsing the generated\nmessage.\n\nLet's also replace register_found_gitmodules() with directly\nmanipulating the \"gitmodules_found\" member. A recent commit moved it\ninto \"fsck_options\" so we could do this here.\n\nAdd a fsck-cb.c file similar to parse-options-cb.c, the alternative\nwould be to either define this directly in fsck.c as a public API, or\nto create some library shared by fetch-pack.c ad builtin/index-pack.\n\nI expect that there won't be many of these fsck utility functions in\nthe future, so just having a single fsck-cb.c makes sense.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 1 +\n builtin/index-pack.c | 21 +--------------------\n fetch-pack.c | 4 ++--\n fsck-cb.c | 16 ++++++++++++++++\n fsck.c | 5 -----\n fsck.h | 22 +++++++++++++++++++---\n 6 files changed, 39 insertions(+), 30 deletions(-)\n create mode 100644 fsck-cb.c\n\ndiff --git a/Makefile b/Makefile\nindex dd08b4ced01..5bf128c5d2c 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -879,6 +879,7 @@ LIB_OBJS += fetch-negotiator.o\n LIB_OBJS += fetch-pack.o\n LIB_OBJS += fmt-merge-msg.o\n LIB_OBJS += fsck.o\n+LIB_OBJS += fsck-cb.o\n LIB_OBJS += fsmonitor.o\n LIB_OBJS += gettext.o\n LIB_OBJS += gpg-interface.o\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 5ad80b85b47..11f0fafd33b 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -120,7 +120,7 @@ static int nr_threads;\n static int from_stdin;\n static int strict;\n static int do_fsck_object;\n-static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_MISSING_GITMODULES;\n static int verbose;\n static int show_resolving_progress;\n static int show_stat;\n@@ -1713,24 +1713,6 @@ static void show_pack_info(int stat_only)\n \t}\n }\n \n-static int print_dangling_gitmodules(struct fsck_options *o,\n-\t\t\t\t const struct object_id *oid,\n-\t\t\t\t enum object_type object_type,\n-\t\t\t\t enum fsck_msg_type msg_type,\n-\t\t\t\t enum fsck_msg_id msg_id,\n-\t\t\t\t const char *message)\n-{\n-\t/*\n-\t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\n-\t * instead of relying on this string check.\n-\t */\n-\tif (starts_with(message, \"gitmodulesMissing\")) {\n-\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n-\t\treturn 0;\n-\t}\n-\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n-}\n-\n int cmd_index_pack(int argc, const char **argv, const char *prefix)\n {\n \tint i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;\n@@ -1761,7 +1743,6 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \n \tread_replace_refs = 0;\n \tfsck_options.walk = mark_link;\n-\tfsck_options.error_func = print_dangling_gitmodules;\n \n \treset_pack_idx_option(&opts);\n \tgit_config(git_index_pack_config, &opts);\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex f961c3067cd..7fc305b65c4 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -38,7 +38,7 @@ static int server_supports_filtering;\n static int advertise_sid;\n static struct shallow_lock shallow_lock;\n static const char *alternate_shallow_file;\n-static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_MISSING_GITMODULES;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n \n@@ -998,7 +998,7 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(&fsck_options, oid);\n+\t\toidset_insert(&fsck_options.gitmodules_found, oid);\n \tif (fsck_finish(&fsck_options))\n \t\tdie(\"fsck failed\");\n }\ndiff --git a/fsck-cb.c b/fsck-cb.c\nnew file mode 100644\nindex 00000000000..465a49235ac\n--- /dev/null\n+++ b/fsck-cb.c\n@@ -0,0 +1,16 @@\n+#include \"git-compat-util.h\"\n+#include \"fsck.h\"\n+\n+int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n+\t\t\t\t\t const struct object_id *oid,\n+\t\t\t\t\t enum object_type object_type,\n+\t\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t\t const char *message)\n+{\n+\tif (msg_id == FSCK_MSG_GITMODULES_MISSING) {\n+\t\tputs(oid_to_hex(oid));\n+\t\treturn 0;\n+\t}\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n+}\ndiff --git a/fsck.c b/fsck.c\nindex 565274a946c..b0089844db9 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1214,11 +1214,6 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n-void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n-{\n-\toidset_insert(&options->gitmodules_found, oid);\n-}\n-\n int fsck_finish(struct fsck_options *options)\n {\n \tint ret = 0;\ndiff --git a/fsck.h b/fsck.h\nindex bb59ef05b68..ae3107638ab 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -153,9 +153,6 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n-void register_found_gitmodules(struct fsck_options *options,\n-\t\t\t const struct object_id *oid);\n-\n /*\n * fsck a tag, and pass info about it back to the caller. This is\n * exposed fsck_object() internals for git-mktag(1).\n@@ -204,4 +201,23 @@ const char *fsck_describe_object(struct fsck_options *options,\n int fsck_config_internal(const char *var, const char *value, void *cb,\n \t\t\t struct fsck_options *options);\n \n+/*\n+ * Initializations for callbacks in fsck-cb.c\n+ */\n+#define FSCK_OPTIONS_MISSING_GITMODULES { \\\n+\t.strict = 1, \\\n+\t.error_func = fsck_error_cb_print_missing_gitmodules, \\\n+\tFSCK_OPTIONS_COMMON \\\n+}\n+\n+/*\n+ * Error callbacks in fsck-cb.c\n+ */\n+int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n+\t\t\t\t\t const struct object_id *oid,\n+\t\t\t\t\t enum object_type object_type,\n+\t\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t\t const char *message);\n+\n #endif\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418386","messageId":"20210306110439.27694-21-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 20/22] fetch-pack: don't needlessly copy fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:37Z","receivedAt":"2021-03-06T11:06:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the behavior of the .gitmodules validation added in\n5476e1efde (fetch-pack: print and use dangling .gitmodules,\n2021-02-22) so we're using one \"fsck_options\".\n\nI found that code confusing to read. One might think that not setting\nup the error_func earlier means that we're relying on the \"error_func\"\nnot being set in some code in between the two hunks being modified\nhere.\n\nBut we're not, all we're doing in the rest of \"cmd_index_pack()\" is\nfurther setup by calling fsck_set_msg_types(), and assigning to\ndo_fsck_object.\n\nSo there was no reason in 5476e1efde to make a shallow copy of the\nfsck_options struct before setting error_func. Let's just do this\nsetup at the top of the function, along with the \"walk\" assignment.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/index-pack.c | 10 +++-------\n 1 file changed, 3 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 2b2266a4b7d..5ad80b85b47 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1761,6 +1761,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \n \tread_replace_refs = 0;\n \tfsck_options.walk = mark_link;\n+\tfsck_options.error_func = print_dangling_gitmodules;\n \n \treset_pack_idx_option(&opts);\n \tgit_config(git_index_pack_config, &opts);\n@@ -1951,13 +1952,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \telse\n \t\tclose(input_fd);\n \n-\tif (do_fsck_object) {\n-\t\tstruct fsck_options fo = fsck_options;\n-\n-\t\tfo.error_func = print_dangling_gitmodules;\n-\t\tif (fsck_finish(&fo))\n-\t\t\tdie(_(\"fsck error in pack objects\"));\n-\t}\n+\tif (do_fsck_object && fsck_finish(&fsck_options))\n+\t\tdie(_(\"fsck error in pack objects\"));\n \n \tfree(objects);\n \tstrbuf_release(&index_name_buf);\n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418387","messageId":"20210306110439.27694-17-avarab@gmail.com","threadId":"54994","inReplyTo":"20210218105840.11989-1-avarab@gmail.com","subject":"[PATCH v3 16/22] fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-06T11:04:33Z","receivedAt":"2021-03-06T11:06:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FOREACH_FSCK_MSG_ID macro and the fsck_msg_id enum it helps\ndefine from fsck.c to fsck.h. This is in preparation for having\nnon-static functions take the fsck_msg_id as an argument.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 66 ----------------------------------------------------------\n fsck.h | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 66 insertions(+), 66 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 6fbc56e9faa..8a66168e516 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,72 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_FSCK_MSG_ID(FUNC) \\\n-\t/* fatal errors */ \\\n-\tFUNC(NUL_IN_HEADER, FATAL) \\\n-\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n-\t/* errors */ \\\n-\tFUNC(BAD_DATE, ERROR) \\\n-\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n-\tFUNC(BAD_EMAIL, ERROR) \\\n-\tFUNC(BAD_NAME, ERROR) \\\n-\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n-\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n-\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n-\tFUNC(BAD_TIMEZONE, ERROR) \\\n-\tFUNC(BAD_TREE, ERROR) \\\n-\tFUNC(BAD_TREE_SHA1, ERROR) \\\n-\tFUNC(BAD_TYPE, ERROR) \\\n-\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n-\tFUNC(MISSING_AUTHOR, ERROR) \\\n-\tFUNC(MISSING_COMMITTER, ERROR) \\\n-\tFUNC(MISSING_EMAIL, ERROR) \\\n-\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_OBJECT, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_TAG, ERROR) \\\n-\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n-\tFUNC(MISSING_TREE, ERROR) \\\n-\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n-\tFUNC(MISSING_TYPE, ERROR) \\\n-\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n-\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n-\tFUNC(TREE_NOT_SORTED, ERROR) \\\n-\tFUNC(UNKNOWN_TYPE, ERROR) \\\n-\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n-\tFUNC(GITMODULES_MISSING, ERROR) \\\n-\tFUNC(GITMODULES_BLOB, ERROR) \\\n-\tFUNC(GITMODULES_LARGE, ERROR) \\\n-\tFUNC(GITMODULES_NAME, ERROR) \\\n-\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n-\tFUNC(GITMODULES_URL, ERROR) \\\n-\tFUNC(GITMODULES_PATH, ERROR) \\\n-\tFUNC(GITMODULES_UPDATE, ERROR) \\\n-\t/* warnings */ \\\n-\tFUNC(BAD_FILEMODE, WARN) \\\n-\tFUNC(EMPTY_NAME, WARN) \\\n-\tFUNC(FULL_PATHNAME, WARN) \\\n-\tFUNC(HAS_DOT, WARN) \\\n-\tFUNC(HAS_DOTDOT, WARN) \\\n-\tFUNC(HAS_DOTGIT, WARN) \\\n-\tFUNC(NULL_SHA1, WARN) \\\n-\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n-\tFUNC(NUL_IN_COMMIT, WARN) \\\n-\t/* infos (reported as warnings, but ignored by default) */ \\\n-\tFUNC(GITMODULES_PARSE, INFO) \\\n-\tFUNC(BAD_TAG_NAME, INFO) \\\n-\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n-\t/* ignored (elevated when requested) */ \\\n-\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n-\n-#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n-enum fsck_msg_id {\n-\tFOREACH_FSCK_MSG_ID(MSG_ID)\n-\tFSCK_MSG_MAX\n-};\n-#undef MSG_ID\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\ndiff --git a/fsck.h b/fsck.h\nindex fce9981a0cb..c3d3b47b88b 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -13,6 +13,72 @@ enum fsck_msg_type {\n \tFSCK_WARN,\n };\n \n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n+\t/* fatal errors */ \\\n+\tFUNC(NUL_IN_HEADER, FATAL) \\\n+\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n+\t/* errors */ \\\n+\tFUNC(BAD_DATE, ERROR) \\\n+\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n+\tFUNC(BAD_EMAIL, ERROR) \\\n+\tFUNC(BAD_NAME, ERROR) \\\n+\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n+\tFUNC(BAD_TIMEZONE, ERROR) \\\n+\tFUNC(BAD_TREE, ERROR) \\\n+\tFUNC(BAD_TREE_SHA1, ERROR) \\\n+\tFUNC(BAD_TYPE, ERROR) \\\n+\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n+\tFUNC(MISSING_AUTHOR, ERROR) \\\n+\tFUNC(MISSING_COMMITTER, ERROR) \\\n+\tFUNC(MISSING_EMAIL, ERROR) \\\n+\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_OBJECT, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_TAG, ERROR) \\\n+\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n+\tFUNC(MISSING_TREE, ERROR) \\\n+\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n+\tFUNC(MISSING_TYPE, ERROR) \\\n+\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n+\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(TREE_NOT_SORTED, ERROR) \\\n+\tFUNC(UNKNOWN_TYPE, ERROR) \\\n+\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n+\tFUNC(GITMODULES_MISSING, ERROR) \\\n+\tFUNC(GITMODULES_BLOB, ERROR) \\\n+\tFUNC(GITMODULES_LARGE, ERROR) \\\n+\tFUNC(GITMODULES_NAME, ERROR) \\\n+\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n+\tFUNC(GITMODULES_URL, ERROR) \\\n+\tFUNC(GITMODULES_PATH, ERROR) \\\n+\tFUNC(GITMODULES_UPDATE, ERROR) \\\n+\t/* warnings */ \\\n+\tFUNC(BAD_FILEMODE, WARN) \\\n+\tFUNC(EMPTY_NAME, WARN) \\\n+\tFUNC(FULL_PATHNAME, WARN) \\\n+\tFUNC(HAS_DOT, WARN) \\\n+\tFUNC(HAS_DOTDOT, WARN) \\\n+\tFUNC(HAS_DOTGIT, WARN) \\\n+\tFUNC(NULL_SHA1, WARN) \\\n+\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n+\tFUNC(NUL_IN_COMMIT, WARN) \\\n+\t/* infos (reported as warnings, but ignored by default) */ \\\n+\tFUNC(GITMODULES_PARSE, INFO) \\\n+\tFUNC(BAD_TAG_NAME, INFO) \\\n+\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\t/* ignored (elevated when requested) */ \\\n+\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n+\n+#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n+enum fsck_msg_id {\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n+\tFSCK_MSG_MAX\n+};\n+#undef MSG_ID\n+\n struct fsck_options;\n struct object;\n \n-- \n2.31.0.rc0.126.g04f22c5b82\n\n"},{"id":"418425","messageId":"xmqqtupm7ehd.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"Re: [PATCH v3 00/22] fsck: API improvements","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-07T23:04:14Z","receivedAt":"2021-03-07T23:05:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Now that jt/transfer-fsck-across-packs has been merged to master\n> here's a re-roll of v1[1]+v2[2] of this series.\n\nIt unfortunately is not a good time to review or helping any work on\nthis series, as the base topic introduced an unpleasant regression\nand needs to either probably gain a band-aid (or reverted in the\nworst case); of course, it would be appreciated to help resolve the\nissues on that topic ;-)\n\nThanks.\n"},{"id":"418433","messageId":"87im62j98n.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"xmqqtupm7ehd.fsf@gitster.c.googlers.com","subject":"Re: [PATCH v3 00/22] fsck: API improvements","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-08T09:16:40Z","receivedAt":"2021-03-08T09:17:30Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Mar 08 2021, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> Now that jt/transfer-fsck-across-packs has been merged to master\n>> here's a re-roll of v1[1]+v2[2] of this series.\n>\n> It unfortunately is not a good time to review or helping any work on\n> this series, as the base topic introduced an unpleasant regression\n> and needs to either probably gain a band-aid (or reverted in the\n> worst case); of course, it would be appreciated to help resolve the\n> issues on that topic ;-)\n\nI should have mentioned: I saw the bug & proposed fix thread for that.\nI see that 2aec3bc4b64 (fetch-pack: do not mix --pack_header and\npackfile uri, 2021-03-04) down into next is now merged down to next.\n\nMy reading of that thread is that the reported bug is solved, but\nperhaps we're not 100% happy with the solution?\n\nIn any case, that patch does not conflict with this series, and all\ntests pass with/without the two merged together.\n\nI don't forese an issue with the two stepping on each other's toes,\nsince I'm just modifying the rather low-level fsck interface of spewing\nout .gitmodules entries, not touching the logic of what's then done with\nthat information...\n\n\n"},{"id":"418508","messageId":"20210308191424.12149-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqqv9a58a18.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-03-08T19:14:24Z","receivedAt":"2021-03-08T19:15:29Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Jonathan Tan writes:\n> \n> > I should probably have written more in the commit message to justify the\n> > unification, but it is also part of a bug fix (in particular,\n> > --fsck-objects wasn't being passed to the index-pack that indexed the\n> > packfiles linked by URI) and for code health purposes (to prevent future\n> > bugs by eliminating the divergence). So reverting that commit would\n> > reintroduce another bug.\n> \n> Not necessarily. Unifying two that do not inherently have to be\n> identical makes it impossible to pass two different things, and that\n> is what we are seeing in the bug this patch is trying to fix (by\n> forcing the two to be identical by eliminating the unpack-objects\n> codepath in certain cases). \n> \n> The right \"fix\" for the original bug would have been to keep them\n> still separate yet making it easy to pass args that must be used in\n> both of them, no?\n\nOK - I'll do this.\n\n> >> > -\t\tif (do_keep && (args->lock_pack || unpack_limit)) {\n> >> > +\t\tif ((do_keep || index_pack_args) && (args->lock_pack || unpack_limit)) {\n> >> > \t\t\tchar hostname[HOST_NAME_MAX + 1];\n> >> > \t\t\tif (xgethostname(hostname, sizeof(hostname)))\n> >> > \t\t\t\txsnprintf(hostname, sizeof(hostname), \"localhost\");\n> >> \n> >> I do not quite get what this hunk is doing. Care to explain?\n> >\n> > The \"do_keep\" part was unnecessarily restrictive and I used a band-aid\n> > solution to loosen it. I think this started from 88e2f9ed8e (\"introduce\n> > fetch-object: fetch one promisor object\", 2017-12-05) where I might have\n> > misunderstood what do_keep was meant to do, and taught fetch-pack to use\n> > \"index-pack\" if do_keep is true or args->from_promisor is true. What I\n> > should have done is to set do_keep to true if args->from_promisor is\n> > true. Future commits continued to do that with fsck_objects and\n> > index_pack_args.\n> \n> > Maybe what I can do is to refactor get_pack() so that do_keep retains\n> > its original meaning of whether to use \"index-pack\" or \"unpack-objects\",\n> > and then we wouldn't need this line. What do you think (code-wise and\n> > whether this fits in with the release schedule, if we want to get this\n> > in before release)?\n> \n> How bad is the breakage this one is trying to fix? I know it would\n> only affect folks who have to interact with the server that uses\n> packfile URI feature, but do they have a workaround, perhaps with a\n> configuration knob or command line option to ignore the packfile\n> URI,\n\nYes, there's a workaround (to disable packfile URIs from the client side\nusing a config variable).\n\n> and how large is the affected population?\n\nThe only issues I've seen are within $DAYJOB, and there, we can carry\nour own patch to fix this issue. So the affected population (right now)\nis probably not much (if it even exists).\n\n> I cannot shake the feeling that we are seeing band-aid on top of\n> band-aid forced by having chosen to go in a wrong direction in the\n> beginning X-<, and prefer to see the code drift even further into\n> the same direction; hence my earlier suggestion to go back to the\n> root cause by first reverting the wrong fix that introduced this bug\n> and fixing the original bug in a different way.\n> \n> I dunno how involved the necessary surgery would be, though. If\n> this is easy to work around, perhaps it might be a better option for\n> the overall project to ship the upcoming release with this listed as\n> a known breakage.\n\nI don't think it's too difficult - I think we'll only need to filter out\nthe --pack_header when we figure out the arguments to pass for the\npackfiles given by URI. I'll take a look.\n"},{"id":"418513","messageId":"xmqqft155tiu.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"20210308191424.12149-1-jonathantanmy@google.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-08T19:34:33Z","receivedAt":"2021-03-08T19:35:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n>> I dunno how involved the necessary surgery would be, though. If\n>> this is easy to work around, perhaps it might be a better option for\n>> the overall project to ship the upcoming release with this listed as\n>> a known breakage.\n>\n> I don't think it's too difficult - I think we'll only need to filter out\n> the --pack_header when we figure out the arguments to pass for the\n> packfiles given by URI. I'll take a look.\n\nWhat you sent earlier is a much better band-aid than \"keep the\nsingle args array but filter an element out in only one codepath\"\nband-aid, I would think.\n\nAny change that is more involved than a single-liner trivial bugfix\nwould be too late for this cycle, as we'd be cutting -rc2 by the end\nof tomorrow.\n\nThanks.\n"},{"id":"418634","messageId":"xmqqwnugywax.fsf@gitster.c.googlers.com","threadId":"54994","inReplyTo":"xmqqft155tiu.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-09T19:13:58Z","receivedAt":"2021-03-09T19:15:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Jonathan Tan writes:\n>\n>>> I dunno how involved the necessary surgery would be, though. If\n>>> this is easy to work around, perhaps it might be a better option for\n>>> the overall project to ship the upcoming release with this listed as\n>>> a known breakage.\n>>\n>> I don't think it's too difficult - I think we'll only need to filter out\n>> the --pack_header when we figure out the arguments to pass for the\n>> packfiles given by URI. I'll take a look.\n>\n> What you sent earlier is a much better band-aid than \"keep the\n> single args array but filter an element out in only one codepath\"\n> band-aid, I would think.\n>\n> Any change that is more involved than a single-liner trivial bugfix\n> would be too late for this cycle, as we'd be cutting -rc2 by the end\n> of tomorrow.\n\nI was looking at the index_pack_args vs pass_header codepath in\nfetch-pack.c again after finishing the -rc2 stuff, and noticed\nsomething curious.\n\nBefore running the command to process in-stream packdata, we have\nthis bit:\n\n\tif (index_pack_args) {\n\t\tint i;\n\n\t\tfor (i = 0; i < cmd.args.nr; i++)\n\t\t\tstrvec_push(index_pack_args, cmd.args.v[i]);\n\t}\n\nwhere cmd.args is what the original code (before the \"we need to\nprepare the index pack arguments for the offline HTTP transfer\"\nlogic was bolted onto this codepath), so it could of course have\nthings like \"--fix-thin\", \"--promisor\", when we are processing an\nin-stream packfile that has sufficiently large number of objects and\nchoose \"index-pack\" to process it. None of them should be given to\nthe \"index-pack\" that processes the offline packfile that is given\nvia the packfile URI mechanism.\n\nAlso, because this loop copies everything in cmd.args, if our\nin-stream packdata is small, cmd.args.v[0] would be \"unpack-objects\",\nand we end up asking the command to explode the (presumably large\nenough to be worth pre-generating and serving via CDN) packfile that\nis given via the packfile URI mechanism.\n\nWhat I think I am seeing in the code is that there are many things\nother than \"pass_header\" that fundamentally cannot be reused between\nthe processing of the in-stream packdata and the offline packfile\ngiven by the packfile URI (e.g. the in-stream one may want to use\n\"unpack-objects\" to avoid accumulating too many tiny packs, so there\nis nothing to be shared with \"index-pack\" that will always be used\nfor the offline one), and any attempt to \"reuse\" cmd.args while\n\"filtering out\" inappropriate bits is fragile and unfruitful.\n\nInstead, I think we should not touch index_pack in the earlier part\nof the function at all (both reading, writing, or even checking for\nNULL-ness), and use the \"if (index_pack_args)\" block we already have\n(i.e. the one before we call start_command() to process the\nin-stream packdata) to decide what the command line to process the\noffline pack should look like. That way, we won't ever risk such a\nconfusion like running \"unpack-objects\" instead of \"index-pack\" (but\nwe can choose to do so deliberately, of course---the important point\nis to recognise that the in-stream pack and the offline one are\nindependant and we should decide how to cook them separately).\n\n"},{"id":"418679","messageId":"xmqqr1knmvi5.fsf@gitster.g","threadId":"54994","inReplyTo":"xmqqwnugywax.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-10T05:24:18Z","receivedAt":"2021-03-10T05:25:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Instead, I think we should not touch index_pack in the earlier part\n> of the function at all (both reading, writing, or even checking for\n> NULL-ness), ...\n\nI have to take the \"NULL-ness\" part back. As the NULL-ness of the\nvariable is also used to convey that URI packfile is in use, which\nin turn means we have to tell \"index-pack\" we are going to use for\nprocessing in-stream packfile that the objects in the pack may be\npointing at objects that are not yet available. So we do need to\ncheck for the NULL-ness in order to decide what command line to use\nto process the in-stream packdata.\n\n"},{"id":"418690","messageId":"20210310165716.2014235-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqqwnugywax.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-03-10T16:57:16Z","receivedAt":"2021-03-10T16:58:04Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> I was looking at the index_pack_args vs pass_header codepath in\n> fetch-pack.c again after finishing the -rc2 stuff, and noticed\n> something curious.\n> \n> Before running the command to process in-stream packdata, we have\n> this bit:\n> \n> \tif (index_pack_args) {\n> \t\tint i;\n> \n> \t\tfor (i = 0; i < cmd.args.nr; i++)\n> \t\t\tstrvec_push(index_pack_args, cmd.args.v[i]);\n> \t}\n> \n> where cmd.args is what the original code (before the \"we need to\n> prepare the index pack arguments for the offline HTTP transfer\"\n> logic was bolted onto this codepath), so it could of course have\n> things like \"--fix-thin\", \"--promisor\", when we are processing an\n> in-stream packfile that has sufficiently large number of objects and\n> choose \"index-pack\" to process it. None of them should be given to\n> the \"index-pack\" that processes the offline packfile that is given\n> via the packfile URI mechanism.\n\nThanks for continuing to take a look at this. My thinking is that all\npackfiles (inline or through URI) should be processed in as similar a\nmanner as possible. Looking at the potential arguments passed to\nindex-pack:\n\n 1. --shallow-file (before \"index-pack\", that is, an argument passed to\n \"git\" itself and not the subcommand)\n 2. index-pack\n 3. --stdin\n 4. -v\n 5. --fix-thin\n 6. --keep\n 7. [--check-self-contained-and-connected is guarded by !index_pack_args\n so we won't be passing it]\n 8. --promisor\n 9. --pack_header\n 10. --fsck_objects\n 11. [--strict appears in an \"else\" block opposite index_pack_args so we\n won't be passing it]\n\nYou mentioned --fix-thin (5) and --promisor (8). Why do you think that\nnone of these should be given to the \"index-pack\" that processes the\npackfiles given by URI? Perhaps it could be argued that these extra\npackfiles don't need --fix-thin (but I would say that I think servers\nshould be allowed to serve thin packfiles through URI too), but I think\nthat --promisor is necessary (so that a server could, for example,\noffload all trees and commits to a packfile in a CDN, and offload all\nblobs to a separate packfile in a CDN).\n\nLooking at this list, I think that all the arguments (except 9, which\nhas been fixed) are necessary (or at least useful) for indexing a\npackfile given by URI.\n\n> Also, because this loop copies everything in cmd.args, if our\n> in-stream packdata is small, cmd.args.v[0] would be \"unpack-objects\",\n> and we end up asking the command to explode the (presumably large\n> enough to be worth pre-generating and serving via CDN) packfile that\n> is given via the packfile URI mechanism.\n\nI specifically guard against this through the \"if (do_keep ||\nargs->from_promisor || index_pack_args || fsck_objects) {\" line (which\nis a complicated line, unfortunately).\n\n> What I think I am seeing in the code is that there are many things\n> other than \"pass_header\" that fundamentally cannot be reused between\n> the processing of the in-stream packdata and the offline packfile\n> given by the packfile URI (e.g. the in-stream one may want to use\n> \"unpack-objects\" to avoid accumulating too many tiny packs, so there\n> is nothing to be shared with \"index-pack\" that will always be used\n> for the offline one), and any attempt to \"reuse\" cmd.args while\n> \"filtering out\" inappropriate bits is fragile and unfruitful.\n>\n> Instead, I think we should not touch index_pack in the earlier part\n> of the function at all (both reading, writing, or even checking for\n> NULL-ness), and use the \"if (index_pack_args)\" block we already have\n> (i.e. the one before we call start_command() to process the\n> in-stream packdata) to decide what the command line to process the\n> offline pack should look like. That way, we won't ever risk such a\n> confusion like running \"unpack-objects\" instead of \"index-pack\" (but\n> we can choose to do so deliberately, of course---the important point\n> is to recognise that the in-stream pack and the offline one are\n> independant and we should decide how to cook them separately).\n\nWe could do that, although I'm concerned that we would be repeating\nlogic a lot (deciding whether or not to pass an argument). One other\napproach is for each \"strvec.push?(&cmd.args\" to also have another line\nthat pushes to index_pack_args if it's relevant. But as I said earlier,\nI think that all or nearly all arguments will be relevant to both.\n"},{"id":"418699","messageId":"xmqqa6rakgj3.fsf@gitster.g","threadId":"54994","inReplyTo":"20210310165716.2014235-1-jonathantanmy@google.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-10T18:30:40Z","receivedAt":"2021-03-10T18:31:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> You mentioned --fix-thin (5) and --promisor (8). Why do you think that\n> none of these should be given to the \"index-pack\" that processes the\n> packfiles given by URI?\n\nActually, --fix-thin is probably even worse than that.\n\nAs the code processes the in-stream packdata before processing or\neven downloading the pregenerated URI packfile, the objects\nnecessary to fix a \"thin\" in-stream packdata are likely to be\nunavailable (it is exactly the same problem as the one that made us\nto delay the fsckobjects done in index-pack when URI packfile is\ninvolved, isn't it?). Even if the client asks --thin, the server\nside shouldn't produce a thin pack for in-stream packdata, no?\n\n> Perhaps it could be argued that these extra\n> packfiles don't need --fix-thin (but I would say that I think servers\n> should be allowed to serve thin packfiles through URI too), \n\nI agree that URI packfile could be thin; after all, the server end\nchooses, based on what the client claims to have, which pregenerated\npackfile to hand out, so it is perfectly fine to hand out a\npregenerated packfile that is thin if the client asks for a thin\npack and says it has base objects missing from that packfile. And\nbecause it is (assumed to be) pregenerated, we can make a requirement\nthat no URI packfile should depend on objects that are created later\nthat that (which means it won't depend on in-stream packdata).\n\nBut we cannot process a thin in-stream packdata, if we are to\nprocess it first, right?\n\n> but I think\n> that --promisor is necessary (so that a server could, for example,\n> offload all trees and commits to a packfile in a CDN, and offload all\n> blobs to a separate packfile in a CDN).\n\nYes, both packfiles conceptually are given by that same server who\npromises to be always available to feed us everything we'd need\nlater, so both packfiles should be marked to have come from the same\npromisor. So this is one example that happens to be sharable\nbetween the two.\n\nBut I do not see it as an indication that the two packs inherently\nmust be processed with the same options.\n\n> Looking at this list, I think that all the arguments (except 9, which\n> has been fixed) are necessary (or at least useful) for indexing a\n> packfile given by URI.\n\nI have to say that this is focusing too much on the current need by\ngoing through how the current code handles two packs. Of course, if\nwe start from \"two must be the same\" viewpoint, and restrict what\nthe code can do by \"guarding\" bits that require the two to be\ndifferent out based on \"if (index_pack_args)\", then the resulting\ncode would invoke two index-pack the same way.\n\nI am more worried about the longer term code health, so \"currently\nmostly the same\" does not make a convincing argument for the future\nwhy the two must be processed the same way.\n\n>> Also, because this loop copies everything in cmd.args, if our\n>> in-stream packdata is small, cmd.args.v[0] would be \"unpack-objects\",\n>> and we end up asking the command to explode the (presumably large\n>> enough to be worth pre-generating and serving via CDN) packfile that\n>> is given via the packfile URI mechanism.\n>\n> I specifically guard against this through the \"if (do_keep ||\n> args->from_promisor || index_pack_args || fsck_objects) {\" line (which\n> is a complicated line, unfortunately).\n\nI am aware of that line that forbids the in-stream packdata from\ngetting unpacked into loose objects. But unless we were told to\nkeep the resulting pack, or run fsck-objects via the index-pack, I\ndo not see an inherent reason why the \"most recent leftover bits\nthat are not in the pregenerated pack offloaded to CDN\" objects must\nbe kept in a separate packfile, especially if the number of objects\nin it is smaller than the unpack limit threshold. In other words,\nI view that \"guard\" as one of the things that blinds us into thinking\nthat the two packs should be handled the same way. It is the other\nway around---the guard is there only because the code wanted to handle\nthe two packs the same way.\n\nWhen cloning from a server that offers bulk of old history in a URI\npackfile and an in-stream packfile, shouldn't the result be like\ncloning from the server back when it had only the objects in the URI\npackfile, and then fetching from it again when it acquired objects\nthat came in the in-stream packfile? The objects that come during\nthe second fetch would be left loose if there aren't that many, so\nthat the third and subsequent fetches and local activity can\naccumulate enough loose objects to be packed into a single new pack,\navoiding accumulation of too many tiny packs. And the \"guard\"\nbreaks that only because this codepath wants to reuse cmd.args that\nis unrelated to populate index_pack_args. Isn't that an artificial\nlimitation that we may want to eventually fix?\n\nWhen we want to fix that, the \"options are mostly the same when we\nuse the index-pack command for both packdata, so let's copy the\nentire command line\" would come back and haunt us. The person who\nis doing the fix may be somebody other than you, so it may not\nmatter to you today, but it will hurt somebody tomorrow.\n\nI already said that I think 2aec3bc4 (fetch-pack: do not mix\n--pack_header and packfile uri, 2021-03-04) is OK as a short-term\nfix for the upcoming release, but it does not change the fact that\nit is piling technical debt on top of existing technical debt.\n\nAnd that is why I am reacting against your earlier mention of\n\"filering out\" rather strongly. The approach continues the \"keep\nthe single args array in the belief that two must be mostly the\nsame\", which I view as a misguided starting point that must be\nrethought.\n\nThanks.\n\n"},{"id":"418734","messageId":"xmqq5z1ykckc.fsf@gitster.g","threadId":"54994","inReplyTo":"xmqqa6rakgj3.fsf@gitster.g","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-10T19:56:19Z","receivedAt":"2021-03-10T19:57:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> I already said that I think 2aec3bc4 (fetch-pack: do not mix\n> --pack_header and packfile uri, 2021-03-04) is OK as a short-term\n> fix for the upcoming release, but it does not change the fact that\n> it is piling technical debt on top of existing technical debt.\n>\n> And that is why I am reacting against your earlier mention of\n> \"filering out\" rather strongly. The approach continues the \"keep\n> the single args array in the belief that two must be mostly the\n> same\", which I view as a misguided starting point that must be\n> rethought.\n\nAnother way to think about the codepath is this.\n\nCan the bulk of get_pack() that deals with a single incoming\npackfile (from the part that makes the decision to use either\nindex-pack or unpack-objects and chooses what options to pass to the\ncommand, to the part that actually calls run_command() and feeds the\npackdata to the command) be made into a helper function that handles\none packdata stream and nothing else? Such a helper would most\nlikely take as its parameters\n\n - a stream to read the packdata from (for in-stream packfile that\n is handled by get_pack(), we already have it available)\n\n - fetch_pack_args and other options that are meant to affect the\n operation of fetch-pack, among which are two bits that are of\n interest in this topic: if we want to run fsck-objects and if the\n entire fetch-pack is dealing with more than one packfile\n (currently, the only source of need to process multiple packfiles\n is packfile URI mechanism, but that does not have to stay that\n way).\n\nThen get_pack() can move a lot of code out of it to this helper and\njust call it. The processing the other packfile obtained by the\npackfile URI mechanism out of band can open the packstream and call\nthe helper the same way. When packfile URI mechanism is in use, both\ninvocations of the helper would get \"you are not alone so fsck may\nhit missing objects\" bit, if fsck-objects are asked for.\n\nThat would avoid the \"duplicated logic\" and still allow the code to\nchoose the best disposition of the incoming packdata per packfile.\n\nIn an extreme case, it is not hard to imagine that somebody prepares\na very small base packfile and feed it via packfile URI mechanism,\nbut have accumulated so many objects that are not yet rolled into an\nupdated base packfile---cloning from such a repository may result in\nrunning unpack-objects for the packfile that came out of band, while\nprocessing the in-stream packfile with index-pack.\n\nHmm?\n\n\n"},{"id":"418751","messageId":"20210310232906.2135256-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqq5z1ykckc.fsf@gitster.g","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-03-10T23:29:06Z","receivedAt":"2021-03-10T23:29:59Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Junio C Hamano writes:\n> \n> > I already said that I think 2aec3bc4 (fetch-pack: do not mix\n> > --pack_header and packfile uri, 2021-03-04) is OK as a short-term\n> > fix for the upcoming release, but it does not change the fact that\n> > it is piling technical debt on top of existing technical debt.\n> >\n> > And that is why I am reacting against your earlier mention of\n> > \"filering out\" rather strongly. The approach continues the \"keep\n> > the single args array in the belief that two must be mostly the\n> > same\", which I view as a misguided starting point that must be\n> > rethought.\n> \n> Another way to think about the codepath is this.\n> \n> Can the bulk of get_pack() that deals with a single incoming\n> packfile (from the part that makes the decision to use either\n> index-pack or unpack-objects and chooses what options to pass to the\n> command, to the part that actually calls run_command() and feeds the\n> packdata to the command) be made into a helper function that handles\n> one packdata stream and nothing else? Such a helper would most\n> likely take as its parameters\n> \n> - a stream to read the packdata from (for in-stream packfile that\n> is handled by get_pack(), we already have it available)\n> \n> - fetch_pack_args and other options that are meant to affect the\n> operation of fetch-pack, among which are two bits that are of\n> interest in this topic: if we want to run fsck-objects and if the\n> entire fetch-pack is dealing with more than one packfile\n> (currently, the only source of need to process multiple packfiles\n> is packfile URI mechanism, but that does not have to stay that\n> way).\n\nThis probably means that fetch-pack.c itself (instead of\nfinish_http_pack_request(), currently being called from a separate\nhttp_fetch process) should call index-pack for the out-of-band\npackfiles, which is conceptually reasonable. This means that\nfinish_http_pack_request() will need to be able to refrain from running\nindex-pack itself and instead just return where the pack was downloaded.\n\n> Then get_pack() can move a lot of code out of it to this helper and\n> just call it. The processing the other packfile obtained by the\n> packfile URI mechanism out of band can open the packstream and call\n> the helper the same way. When packfile URI mechanism is in use, both\n> invocations of the helper would get \"you are not alone so fsck may\n> hit missing objects\" bit, if fsck-objects are asked for.\n> \n> That would avoid the \"duplicated logic\" and still allow the code to\n> choose the best disposition of the incoming packdata per packfile.\n> \n> In an extreme case, it is not hard to imagine that somebody prepares\n> a very small base packfile and feed it via packfile URI mechanism,\n> but have accumulated so many objects that are not yet rolled into an\n> updated base packfile---cloning from such a repository may result in\n> running unpack-objects for the packfile that came out of band, while\n> processing the in-stream packfile with index-pack.\n> \n> Hmm?\n\nYour suggestion (as opposed to the current situation, in which we're\nlocked into using index-pack for the out-of-band packfiles) would make\nthis possible, yes.\n"},{"id":"418773","messageId":"xmqqlfauh5ds.fsf@gitster.g","threadId":"54994","inReplyTo":"20210310232906.2135256-1-jonathantanmy@google.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-11T00:59:43Z","receivedAt":"2021-03-11T01:00:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n>> Then get_pack() can move a lot of code out of it to this helper and\n>> just call it. The processing the other packfile obtained by the\n>> packfile URI mechanism out of band can open the packstream and call\n>> the helper the same way. When packfile URI mechanism is in use, both\n>> invocations of the helper would get \"you are not alone so fsck may\n>> hit missing objects\" bit, if fsck-objects are asked for.\n>> \n>> That would avoid the \"duplicated logic\" and still allow the code to\n>> choose the best disposition of the incoming packdata per packfile.\n>> \n>> In an extreme case, it is not hard to imagine that somebody prepares\n>> a very small base packfile and feed it via packfile URI mechanism,\n>> but have accumulated so many objects that are not yet rolled into an\n>> updated base packfile---cloning from such a repository may result in\n>> running unpack-objects for the packfile that came out of band, while\n>> processing the in-stream packfile with index-pack.\n>> \n>> Hmm?\n>\n> Your suggestion (as opposed to the current situation, in which we're\n> locked into using index-pack for the out-of-band packfiles) would make\n> this possible, yes.\n\nJust to make sure, I am not interested in running unpack-objects on\noob packfiles, as they are expected to be \"so old, big and not\nchanging that it is worth pre-generating\" packfiles, so \"yes the\napproach would make that useless thing possible\" is not a useful\ncriteria to judge how good the alternative approach would be. If\nthe approach results in a cleaner design that gives us more\nflexibility without risking unnecessary code duplication, it would\nbe a good sign that the approach is more sound than the direction we\ntook so far, though.\n\nThanks.\n"},{"id":"418777","messageId":"xmqq4khih3gj.fsf@gitster.g","threadId":"54994","inReplyTo":"20210310232906.2135256-1-jonathantanmy@google.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-11T01:41:16Z","receivedAt":"2021-03-11T01:42:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> This probably means that fetch-pack.c itself (instead of\n> finish_http_pack_request(), currently being called from a separate\n> http_fetch process) should call index-pack for the out-of-band\n> packfiles, which is conceptually reasonable. This means that\n> finish_http_pack_request() will need to be able to refrain from running\n> index-pack itself and instead just return where the pack was downloaded.\n\nThe HTTP downloading for packfile specified via the packfile URI\nmechansim is so different from the rest of the HTTP codepaths in\nnature, isn't it? It is a straight \"download a static file over the\nweb, and we could even afford to resume, or send multiple requests\nto gain throughput\" usecase, which does not exist anywhere else in\nGit (eh, other than the dumb HTTP protocol nobody sane should be\nusing anymore).\n\nSince we are not in the business of writing a performant HTTP\ndownloader, if we can update the codepath not to rely on our http.c\ncode, and instead spawn one of the command line tools written\nspecifically for the \"download a single large file over HTTP\"\nusecase (like curl, wget or aria2c), wait for it to do its thing and\nthen concentrate on the processing specific to Git (like running\nindex-pack with various options), it would take us closer to the\n\"make clone resumable\" dream, wouldn't it?\n\nThanks.\n"},{"id":"418881","messageId":"20210311172251.2307015-1-jonathantanmy@google.com","threadId":"54994","inReplyTo":"xmqq4khih3gj.fsf@gitster.g","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-03-11T17:22:51Z","receivedAt":"2021-03-11T17:23:41Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> Jonathan Tan writes:\n> \n> > This probably means that fetch-pack.c itself (instead of\n> > finish_http_pack_request(), currently being called from a separate\n> > http_fetch process) should call index-pack for the out-of-band\n> > packfiles, which is conceptually reasonable. This means that\n> > finish_http_pack_request() will need to be able to refrain from running\n> > index-pack itself and instead just return where the pack was downloaded.\n> \n> The HTTP downloading for packfile specified via the packfile URI\n> mechansim is so different from the rest of the HTTP codepaths in\n> nature, isn't it? It is a straight \"download a static file over the\n> web, and we could even afford to resume, or send multiple requests\n> to gain throughput\" usecase, which does not exist anywhere else in\n> Git (eh, other than the dumb HTTP protocol nobody sane should be\n> using anymore).\n\nYes - and I also noticed that finish_http_pack_request() is also used in\nhttp-push.c, but I'm not familiar with that.\n\n> Since we are not in the business of writing a performant HTTP\n> downloader, if we can update the codepath not to rely on our http.c\n> code, and instead spawn one of the command line tools written\n> specifically for the \"download a single large file over HTTP\"\n> usecase (like curl, wget or aria2c), wait for it to do its thing and\n> then concentrate on the processing specific to Git (like running\n> index-pack with various options), it would take us closer to the\n> \"make clone resumable\" dream, wouldn't it?\n> \n> Thanks.\n\nWe would have to figure out how to communicate any Git HTTP config\nvariables to curl/wget etc. (and also declare a dependency on such a\ntool), but that could be done.\n"},{"id":"418907","messageId":"xmqqzgz9bd3x.fsf@gitster.g","threadId":"54994","inReplyTo":"20210311172251.2307015-1-jonathantanmy@google.com","subject":"Re: [PATCH] fetch-pack: do not mix --pack_header and packfile uri","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-11T21:21:38Z","receivedAt":"2021-03-11T21:22:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n>> Since we are not in the business of writing a performant HTTP\n>> downloader, if we can update the codepath not to rely on our http.c\n>> code, and instead spawn one of the command line tools written\n>> specifically for the \"download a single large file over HTTP\"\n>> usecase (like curl, wget or aria2c), wait for it to do its thing and\n>> then concentrate on the processing specific to Git (like running\n>> index-pack with various options), it would take us closer to the\n>> \"make clone resumable\" dream, wouldn't it?\n>> \n>> Thanks.\n>\n> We would have to figure out how to communicate any Git HTTP config\n> variables to curl/wget etc. (and also declare a dependency on such a\n> tool), but that could be done.\n\nSure, and we do not have to go all the way there in a single step.\n\nWe'd likely need to ship with a basic \"download from this URL and\nstore it in this specified temporary file\" (or \"to this fd\") and use\nit as the default downloader. We just need to design the interface\nto that downloader (i.e. which we want to make replaceable) to be\nnot too intimate with the details of the side that spawns the\ndownloader (i.e. git and git-fetch), and other people can write\nreplacement as a thin wrapper around curl/wget etc. to contribute to\nus.\n\nThanks.\n"},{"id":"419351","messageId":"20210316161738.30254-2-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 01/22] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:17Z","receivedAt":"2021-03-16T16:18:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add the object_name member to the initialization macro. This was\nomitted in 7b35efd734e (fsck_walk(): optionally name objects on the\ngo, 2016-07-17) when the field was added.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 733378f126..2274843ba0 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -43,8 +43,8 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n-#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n+#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n+#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419352","messageId":"20210316161738.30254-5-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 04/22] fsck.h: add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:20Z","receivedAt":"2021-03-16T16:18:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro for those that would like\nto use FSCK_OPTIONS_COMMON in their own initialization, but supply\ntheir own error functions.\n\nNothing is being changed to use this yet, but in some subsequent\ncommits we'll make use of this macro.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 9 ++++++---\n 1 file changed, 6 insertions(+), 3 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex ea3a907ec3..dc35924cbf 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -45,12 +45,15 @@ struct fsck_options {\n \n #define FSCK_OPTIONS_COMMON \\\n \t.walk = NULL, \\\n-\t.error_func = fsck_error_function, \\\n \t.msg_type = NULL, \\\n \t.skiplist = OIDSET_INIT, \\\n \t.object_names = NULL,\n-#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON }\n-#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON }\n+#define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n+\tFSCK_OPTIONS_COMMON \\\n+\t.error_func = fsck_error_function\n+\n+#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n+#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419353","messageId":"20210316161738.30254-4-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 03/22] fsck.h: reduce duplication between FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:19Z","receivedAt":"2021-03-16T16:18:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Use a temporary macro to define what FSCK_OPTIONS_{DEFAULT,STRICT}\nhave in common, and define the two in terms of that macro.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 16 ++++------------\n 1 file changed, 4 insertions(+), 12 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 40f3cb3f64..ea3a907ec3 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -43,22 +43,14 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { \\\n+#define FSCK_OPTIONS_COMMON \\\n \t.walk = NULL, \\\n \t.error_func = fsck_error_function, \\\n-\t.strict = 0, \\\n \t.msg_type = NULL, \\\n \t.skiplist = OIDSET_INIT, \\\n-\t.object_names = NULL, \\\n-}\n-#define FSCK_OPTIONS_STRICT { \\\n-\t.walk = NULL, \\\n-\t.error_func = fsck_error_function, \\\n-\t.strict = 1, \\\n-\t.msg_type = NULL, \\\n-\t.skiplist = OIDSET_INIT, \\\n-\t.object_names = NULL, \\\n-}\n+\t.object_names = NULL,\n+#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON }\n+#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419354","messageId":"20210316161738.30254-1-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 00/22] fsck: API improvements","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:16Z","receivedAt":"2021-03-16T16:18:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"A re-send of a rebased v3, which I sent at:\nhttp://lore.kernel.org/git/20210306110439.27694-1-avarab@gmail.com as\nseen in the range-diff there are no changes since v3. I'm just sending\nthis as a post-release bump of this, per\nhttps://lore.kernel.org/git/xmqqy2etczqi.fsf@gitster.g/\n\nÆvar Arnfjörð Bjarmason (22):\n fsck.h: update FSCK_OPTIONS_* for object_name\n fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n fsck.h: reduce duplication between FSCK_OPTIONS_{DEFAULT,STRICT}\n fsck.h: add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro\n fsck.h: indent arguments to of fsck_set_msg_type\n fsck.h: use \"enum object_type\" instead of \"int\"\n fsck.c: rename variables in fsck_set_msg_type() for less confusion\n fsck.c: move definition of msg_id into append_msg_id()\n fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"\n fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n fsck.h: re-order and re-assign \"enum fsck_msg_type\"\n fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n fsck.c: undefine temporary STR macro after use\n fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n fsck.c: pass along the fsck_msg_id in the fsck_error callback\n fsck.c: add an fsck_set_msg_type() API that takes enums\n fsck.c: move gitmodules_{found,done} into fsck_options\n fetch-pack: don't needlessly copy fsck_options\n fetch-pack: use file-scope static struct for fsck_options\n fetch-pack: use new fsck API to printing dangling submodules\n\n Makefile | 1 +\n builtin/fsck.c | 7 +-\n builtin/index-pack.c | 30 ++-----\n builtin/mktag.c | 7 +-\n builtin/unpack-objects.c | 3 +-\n fetch-pack.c | 6 +-\n fsck-cb.c | 16 ++++\n fsck.c | 175 ++++++++++++---------------------------\n fsck.h | 132 ++++++++++++++++++++++++++---\n 9 files changed, 211 insertions(+), 166 deletions(-)\n create mode 100644 fsck-cb.c\n\nRange-diff:\n 1: 9d809466bd = 1: 9cd942b526 fsck.h: update FSCK_OPTIONS_* for object_name\n 2: 33e8b6d654 = 2: d67966b838 fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n 3: c23f7ce9e4 = 3: 211472e0c5 fsck.h: reduce duplication between FSCK_OPTIONS_{DEFAULT,STRICT}\n 4: 5dde68df6c = 4: 70afee988d fsck.h: add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro\n 5: 7ae35a6e9d = 5: 1337d53352 fsck.h: indent arguments to of fsck_set_msg_type\n 6: dfb5f754b3 = 6: e4ef107bb4 fsck.h: use \"enum object_type\" instead of \"int\"\n 7: fd58ec73c6 = 7: 20bac3207e fsck.c: rename variables in fsck_set_msg_type() for less confusion\n 8: 48cb4d3bb7 = 8: 09c3bba9e9 fsck.c: move definition of msg_id into append_msg_id()\n 9: 2c80ad3203 = 9: 8067df53a2 fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n10: 92dfbdfb62 = 10: bdf5e13f3d fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"\n11: c1c476af69 = 11: b03caa237f fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n12: d55587719a = 12: 7b1d13b4cc fsck.h: re-order and re-assign \"enum fsck_msg_type\"\n13: 32828d1c78 = 13: a8e4ca7b19 fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n14: 5c62066235 = 14: 214c375a20 fsck.c: undefine temporary STR macro after use\n15: f8e50fbf7d = 15: 19a2499a80 fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n16: cd74dee876 = 16: 6e1a7b6274 fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n17: 234e287d08 = 17: 42af4e164c fsck.c: pass along the fsck_msg_id in the fsck_error callback\n18: 8049dc0739 = 18: fa47f473a8 fsck.c: add an fsck_set_msg_type() API that takes enums\n19: 4224a29d15 = 19: 4cc3880cc4 fsck.c: move gitmodules_{found,done} into fsck_options\n20: 40b1346812 = 20: fd219d318a fetch-pack: don't needlessly copy fsck_options\n21: 8e418abfbd = 21: e4cd8c250e fetch-pack: use file-scope static struct for fsck_options\n22: 113de190f7 = 22: fdbc3c304c fetch-pack: use new fsck API to printing dangling submodules\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419355","messageId":"20210316161738.30254-3-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 02/22] fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:18Z","receivedAt":"2021-03-16T16:18:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Signed-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 18 ++++++++++++++++--\n 1 file changed, 16 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 2274843ba0..40f3cb3f64 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -43,8 +43,22 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n-#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n+#define FSCK_OPTIONS_DEFAULT { \\\n+\t.walk = NULL, \\\n+\t.error_func = fsck_error_function, \\\n+\t.strict = 0, \\\n+\t.msg_type = NULL, \\\n+\t.skiplist = OIDSET_INIT, \\\n+\t.object_names = NULL, \\\n+}\n+#define FSCK_OPTIONS_STRICT { \\\n+\t.walk = NULL, \\\n+\t.error_func = fsck_error_function, \\\n+\t.strict = 1, \\\n+\t.msg_type = NULL, \\\n+\t.skiplist = OIDSET_INIT, \\\n+\t.object_names = NULL, \\\n+}\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419356","messageId":"20210316161738.30254-6-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 05/22] fsck.h: indent arguments to of fsck_set_msg_type","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:21Z","receivedAt":"2021-03-16T16:18:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Signed-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex dc35924cbf..5e488cef6b 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -11,7 +11,7 @@ struct fsck_options;\n struct object;\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type);\n+\t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n int is_valid_msg_type(const char *msg_id, const char *msg_type);\n \n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419357","messageId":"20210316161738.30254-7-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 06/22] fsck.h: use \"enum object_type\" instead of \"int\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:22Z","receivedAt":"2021-03-16T16:18:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_walk_func to use an \"enum object_type\" instead of an\n\"int\" type. The types are compatible, and ever since this was added in\n355885d5315 (add generic, type aware object chain walker, 2008-02-25)\nwe've used entries from object_type (OBJ_BLOB etc.).\n\nSo this doesn't really change anything as far as the generated code is\nconcerned, it just gives the compiler more information and makes this\neasier to read.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 3 ++-\n builtin/index-pack.c | 3 ++-\n builtin/unpack-objects.c | 3 ++-\n fsck.h | 3 ++-\n 4 files changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 821e7798c7..68f0329e69 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -197,7 +197,8 @@ static int traverse_reachable(void)\n \treturn !!result;\n }\n \n-static int mark_used(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_used(struct object *obj, enum object_type object_type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn 1;\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex bad5748807..69f24fe9f7 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -212,7 +212,8 @@ static void cleanup_thread(void)\n \tfree(thread_data);\n }\n \n-static int mark_link(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_link(struct object *obj, enum object_type type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn -1;\ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex dd4a75e030..ca54fd1668 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -187,7 +187,8 @@ static void write_cached_object(struct object *obj, struct obj_buffer *obj_buf)\n * that have reachability requirements and calls this function.\n * Verify its reachability and validity recursively and write it out.\n */\n-static int check_object(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int check_object(struct object *obj, enum object_type type,\n+\t\t\tvoid *data, struct fsck_options *options)\n {\n \tstruct obj_buffer *obj_buf;\n \ndiff --git a/fsck.h b/fsck.h\nindex 5e488cef6b..f67edd8f1f 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -23,7 +23,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type);\n * <0\terror signaled and abort\n * >0\terror signaled and do not abort\n */\n-typedef int (*fsck_walk_func)(struct object *obj, int type, void *data, struct fsck_options *options);\n+typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n+\t\t\t void *data, struct fsck_options *options);\n \n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419358","messageId":"20210316161738.30254-9-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 08/22] fsck.c: move definition of msg_id into append_msg_id()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:24Z","receivedAt":"2021-03-16T16:18:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor code added in 71ab8fa840f (fsck: report the ID of the\nerror/warning, 2015-06-22) to resolve the msg_id to a string in the\nfunction that wants it, instead of doing it in report().\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 0a9ac9ca07..b977493f57 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,8 +264,9 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, const char *msg_id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n {\n+\tconst char *msg_id = msg_id_info[id].id_string;\n \tfor (;;) {\n \t\tchar c = *(msg_id)++;\n \n@@ -308,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, msg_id_info[id].id_string);\n+\tappend_msg_id(&sb, id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419359","messageId":"20210316161738.30254-8-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 07/22] fsck.c: rename variables in fsck_set_msg_type() for less confusion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:23Z","receivedAt":"2021-03-16T16:18:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename variables in a function added in 0282f4dced0 (fsck: offer a\nfunction to demote fsck errors to warnings, 2015-06-22).\n\nIt was needlessly confusing that it took a \"msg_type\" argument, but\nthen later declared another \"msg_type\" of a different type.\n\nLet's rename that to \"severity\", and rename \"id\" to \"msg_id\" and\n\"msg_id\" to \"msg_id_str\" etc. This will make a follow-up change\nsmaller.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 24 ++++++++++++------------\n 1 file changed, 12 insertions(+), 12 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex e3030f3b35..0a9ac9ca07 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -203,27 +203,27 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n }\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type)\n+\t\tconst char *msg_id_str, const char *msg_type_str)\n {\n-\tint id = parse_msg_id(msg_id), type;\n+\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n \n-\tif (id < 0)\n-\t\tdie(\"Unhandled message id: %s\", msg_id);\n-\ttype = parse_msg_type(msg_type);\n+\tif (msg_id < 0)\n+\t\tdie(\"Unhandled message id: %s\", msg_id_str);\n+\tmsg_type = parse_msg_type(msg_type_str);\n \n-\tif (type != FSCK_ERROR && msg_id_info[id].msg_type == FSCK_FATAL)\n-\t\tdie(\"Cannot demote %s to %s\", msg_id, msg_type);\n+\tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n+\t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *msg_type;\n-\t\tALLOC_ARRAY(msg_type, FSCK_MSG_MAX);\n+\t\tint *severity;\n+\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tmsg_type[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = msg_type;\n+\t\t\tseverity[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = severity;\n \t}\n \n-\toptions->msg_type[id] = type;\n+\toptions->msg_type[msg_id] = msg_type;\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419360","messageId":"20210316161738.30254-11-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 10/22] fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:26Z","receivedAt":"2021-03-16T16:18:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor \"if options->msg_type\" and other code added in\n0282f4dced0 (fsck: offer a function to demote fsck errors to warnings,\n2015-06-22) to reduce the scope of the \"int msg_type\" variable.\n\nThis is in preparation for changing its type in a subsequent commit,\nonly using it in the \"!options->msg_type\" scope makes that change\n\nThis also brings the code in line with the fsck_set_msg_type()\nfunction (also added in 0282f4dced0), which does a similar check for\n\"!options->msg_type\". Another minor benefit is getting rid of the\nstyle violation of not having braces for the body of the \"if\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 12 +++++-------\n 1 file changed, 5 insertions(+), 7 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 6b72ddaa51..0988ab6579 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -167,19 +167,17 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n static int fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n-\tint msg_type;\n-\n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n-\tif (options->msg_type)\n-\t\tmsg_type = options->msg_type[msg_id];\n-\telse {\n-\t\tmsg_type = msg_id_info[msg_id].msg_type;\n+\tif (!options->msg_type) {\n+\t\tint msg_type = msg_id_info[msg_id].msg_type;\n+\n \t\tif (options->strict && msg_type == FSCK_WARN)\n \t\t\tmsg_type = FSCK_ERROR;\n+\t\treturn msg_type;\n \t}\n \n-\treturn msg_type;\n+\treturn options->msg_type[msg_id];\n }\n \n static int parse_msg_type(const char *str)\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419361","messageId":"20210316161738.30254-10-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 09/22] fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:25Z","receivedAt":"2021-03-16T16:18:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the remaining variables of type fsck_msg_id from \"id\" to\n\"msg_id\". This change is relatively small, and is worth the churn for\na later change where we have different id's in the \"report\" function.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex b977493f57..6b72ddaa51 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,19 +264,19 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id msg_id)\n {\n-\tconst char *msg_id = msg_id_info[id].id_string;\n+\tconst char *msg_id_str = msg_id_info[msg_id].id_string;\n \tfor (;;) {\n-\t\tchar c = *(msg_id)++;\n+\t\tchar c = *(msg_id_str)++;\n \n \t\tif (!c)\n \t\t\tbreak;\n \t\tif (c != '_')\n \t\t\tstrbuf_addch(sb, tolower(c));\n \t\telse {\n-\t\t\tassert(*msg_id);\n-\t\t\tstrbuf_addch(sb, *(msg_id)++);\n+\t\t\tassert(*msg_id_str);\n+\t\t\tstrbuf_addch(sb, *(msg_id_str)++);\n \t\t}\n \t}\n \n@@ -292,11 +292,11 @@ static int object_on_skiplist(struct fsck_options *opts,\n __attribute__((format (printf, 5, 6)))\n static int report(struct fsck_options *options,\n \t\t const struct object_id *oid, enum object_type object_type,\n-\t\t enum fsck_msg_id id, const char *fmt, ...)\n+\t\t enum fsck_msg_id msg_id, const char *fmt, ...)\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(id, options), result;\n+\tint msg_type = fsck_msg_type(msg_id, options), result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -309,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, id);\n+\tappend_msg_id(&sb, msg_id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419362","messageId":"20210316161738.30254-12-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 11/22] fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:27Z","receivedAt":"2021-03-16T16:18:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} defines into a new\nfsck_msg_type enum.\n\nThese defines were originally introduced in:\n\n - ba002f3b28a (builtin-fsck: move common object checking code to\n fsck.c, 2008-02-25)\n - f50c4407305 (fsck: disallow demoting grave fsck errors to warnings,\n 2015-06-22)\n - efaba7cc77f (fsck: optionally ignore specific fsck issues\n completely, 2015-06-22)\n - f27d05b1704 (fsck: allow upgrading fsck warnings to errors,\n 2015-06-22)\n\nThe reason these were defined in two different places is because we\nuse FSCK_{IGNORE,INFO,FATAL} only in fsck.c, but FSCK_{ERROR,WARN} are\nused by external callbacks.\n\nUntangling that would take some more work, since we expose the new\n\"enum fsck_msg_type\" to both. Similar to \"enum object_type\" it's not\nworth structuring the API in such a way that only those who need\nFSCK_{ERROR,WARN} pass around a different type.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 2 +-\n builtin/index-pack.c | 3 ++-\n builtin/mktag.c | 3 ++-\n fsck.c | 21 ++++++++++-----------\n fsck.h | 16 ++++++++++------\n 5 files changed, 25 insertions(+), 20 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 68f0329e69..d6d745dc70 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -89,7 +89,7 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t int msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type, const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 69f24fe9f7..56b8efaa89 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1716,7 +1716,8 @@ static void show_pack_info(int stat_only)\n static int print_dangling_gitmodules(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \t/*\n \t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 41a399a69e..1834394a9b 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -22,7 +22,8 @@ static int mktag_config(const char *var, const char *value, void *cb)\n static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/fsck.c b/fsck.c\nindex 0988ab6579..fb7d071bbf 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,9 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FSCK_FATAL -1\n-#define FSCK_INFO -2\n-\n #define FOREACH_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n@@ -97,7 +94,7 @@ static struct {\n \tconst char *id_string;\n \tconst char *downcased;\n \tconst char *camelcased;\n-\tint msg_type;\n+\tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n \tFOREACH_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n@@ -164,13 +161,13 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n \t\tlist_config_item(list, prefix, msg_id_info[i].camelcased);\n }\n \n-static int fsck_msg_type(enum fsck_msg_id msg_id,\n+static enum fsck_msg_type fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n \tif (!options->msg_type) {\n-\t\tint msg_type = msg_id_info[msg_id].msg_type;\n+\t\tenum fsck_msg_type msg_type = msg_id_info[msg_id].msg_type;\n \n \t\tif (options->strict && msg_type == FSCK_WARN)\n \t\t\tmsg_type = FSCK_ERROR;\n@@ -180,7 +177,7 @@ static int fsck_msg_type(enum fsck_msg_id msg_id,\n \treturn options->msg_type[msg_id];\n }\n \n-static int parse_msg_type(const char *str)\n+static enum fsck_msg_type parse_msg_type(const char *str)\n {\n \tif (!strcmp(str, \"error\"))\n \t\treturn FSCK_ERROR;\n@@ -203,7 +200,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n-\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n+\tint msg_id = parse_msg_id(msg_id_str);\n+\tenum fsck_msg_type msg_type;\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n@@ -214,7 +212,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *severity;\n+\t\tenum fsck_msg_type *severity;\n \t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n \t\t\tseverity[i] = fsck_msg_type(i, options);\n@@ -294,7 +292,8 @@ static int report(struct fsck_options *options,\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(msg_id, options), result;\n+\tenum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);\n+\tint result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -1265,7 +1264,7 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tint msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type, const char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex f67edd8f1f..2ecc15eee7 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -3,9 +3,13 @@\n \n #include \"oidset.h\"\n \n-#define FSCK_ERROR 1\n-#define FSCK_WARN 2\n-#define FSCK_IGNORE 3\n+enum fsck_msg_type {\n+\tFSCK_INFO = -2,\n+\tFSCK_FATAL = -1,\n+\tFSCK_ERROR = 1,\n+\tFSCK_WARN,\n+\tFSCK_IGNORE\n+};\n \n struct fsck_options;\n struct object;\n@@ -29,17 +33,17 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t int msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tint msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, const char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n \tfsck_error error_func;\n \tunsigned strict:1;\n-\tint *msg_type;\n+\tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n \tkh_oid_map_t *object_names;\n };\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419363","messageId":"20210316161738.30254-14-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 13/22] fsck.c: call parse_msg_type() early in fsck_set_msg_type()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:29Z","receivedAt":"2021-03-16T16:18:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"There's no reason to defer the calling of parse_msg_type() until after\nwe've checked if the \"id < 0\". This is not a hot codepath, and\nparse_msg_type() itself may die on invalid input.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex fb7d071bbf..2ccf1a2f0f 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -201,11 +201,10 @@ void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n \tint msg_id = parse_msg_id(msg_id_str);\n-\tenum fsck_msg_type msg_type;\n+\tenum fsck_msg_type msg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n-\tmsg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419364","messageId":"20210316161738.30254-13-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 12/22] fsck.h: re-order and re-assign \"enum fsck_msg_type\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:28Z","receivedAt":"2021-03-16T16:19:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the values in the \"enum fsck_msg_type\" from being manually\nassigned to using default C enum values.\n\nThis means we end up with a FSCK_IGNORE=0, which was previously\ndefined as \"2\".\n\nI'm confident that nothing relies on these values, we always compare\nthem explicitly. Let's not omit \"0\" so it won't be assumed that we're\nusing these as a boolean somewhere.\n\nThis also allows us to re-structure the fields to mark which are\n\"private\" v.s. \"public\". See the preceding commit for a rationale for\nnot simply splitting these into two enums, namely that this is used\nfor both the private and public fsck API.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 10 ++++++----\n 1 file changed, 6 insertions(+), 4 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 2ecc15eee7..fce9981a0c 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -4,11 +4,13 @@\n #include \"oidset.h\"\n \n enum fsck_msg_type {\n-\tFSCK_INFO = -2,\n-\tFSCK_FATAL = -1,\n-\tFSCK_ERROR = 1,\n+\t/* for internal use only */\n+\tFSCK_IGNORE,\n+\tFSCK_INFO,\n+\tFSCK_FATAL,\n+\t/* \"public\", fed to e.g. error_func callbacks */\n+\tFSCK_ERROR,\n \tFSCK_WARN,\n-\tFSCK_IGNORE\n };\n \n struct fsck_options;\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419365","messageId":"20210316161738.30254-16-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 15/22] fsck.c: give \"FOREACH_MSG_ID\" a more specific name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:31Z","receivedAt":"2021-03-16T16:19:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the FOREACH_MSG_ID macro to FOREACH_FSCK_MSG_ID in preparation\nfor moving it over to fsck.h. It's good convention to name macros\nin *.h files in such a way as to clearly not clash with any other\nnames in other files.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex f4c924ed04..6fbc56e9fa 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,7 +22,7 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_MSG_ID(FUNC) \\\n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n \tFUNC(UNTERMINATED_HEADER, FATAL) \\\n@@ -83,7 +83,7 @@ static struct oidset gitmodules_done = OIDSET_INIT;\n \n #define MSG_ID(id, msg_type) FSCK_MSG_##id,\n enum fsck_msg_id {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \tFSCK_MSG_MAX\n };\n #undef MSG_ID\n@@ -96,7 +96,7 @@ static struct {\n \tconst char *camelcased;\n \tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419366","messageId":"20210316161738.30254-15-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 14/22] fsck.c: undefine temporary STR macro after use","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:30Z","receivedAt":"2021-03-16T16:19:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In f417eed8cde (fsck: provide a function to parse fsck message IDs,\n2015-06-22) the \"STR\" macro was introduced, but that short macro name\nwas not undefined after use as was done earlier in the same series for\nthe MSG_ID macro in c99ba492f1c (fsck: introduce identifiers for fsck\nmessages, 2015-06-22).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/fsck.c b/fsck.c\nindex 2ccf1a2f0f..f4c924ed04 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -100,6 +100,7 @@ static struct {\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n+#undef STR\n \n static void prepare_msg_ids(void)\n {\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419367","messageId":"20210316161738.30254-21-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 20/22] fetch-pack: don't needlessly copy fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:36Z","receivedAt":"2021-03-16T16:19:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the behavior of the .gitmodules validation added in\n5476e1efde (fetch-pack: print and use dangling .gitmodules,\n2021-02-22) so we're using one \"fsck_options\".\n\nI found that code confusing to read. One might think that not setting\nup the error_func earlier means that we're relying on the \"error_func\"\nnot being set in some code in between the two hunks being modified\nhere.\n\nBut we're not, all we're doing in the rest of \"cmd_index_pack()\" is\nfurther setup by calling fsck_set_msg_types(), and assigning to\ndo_fsck_object.\n\nSo there was no reason in 5476e1efde to make a shallow copy of the\nfsck_options struct before setting error_func. Let's just do this\nsetup at the top of the function, along with the \"walk\" assignment.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/index-pack.c | 10 +++-------\n 1 file changed, 3 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 2b2266a4b7..5ad80b85b4 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1761,6 +1761,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \n \tread_replace_refs = 0;\n \tfsck_options.walk = mark_link;\n+\tfsck_options.error_func = print_dangling_gitmodules;\n \n \treset_pack_idx_option(&opts);\n \tgit_config(git_index_pack_config, &opts);\n@@ -1951,13 +1952,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \telse\n \t\tclose(input_fd);\n \n-\tif (do_fsck_object) {\n-\t\tstruct fsck_options fo = fsck_options;\n-\n-\t\tfo.error_func = print_dangling_gitmodules;\n-\t\tif (fsck_finish(&fo))\n-\t\t\tdie(_(\"fsck error in pack objects\"));\n-\t}\n+\tif (do_fsck_object && fsck_finish(&fsck_options))\n+\t\tdie(_(\"fsck error in pack objects\"));\n \n \tfree(objects);\n \tstrbuf_release(&index_name_buf);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419368","messageId":"20210316161738.30254-17-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 16/22] fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:32Z","receivedAt":"2021-03-16T16:19:19Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FOREACH_FSCK_MSG_ID macro and the fsck_msg_id enum it helps\ndefine from fsck.c to fsck.h. This is in preparation for having\nnon-static functions take the fsck_msg_id as an argument.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 66 ----------------------------------------------------------\n fsck.h | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 66 insertions(+), 66 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 6fbc56e9fa..8a66168e51 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,72 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_FSCK_MSG_ID(FUNC) \\\n-\t/* fatal errors */ \\\n-\tFUNC(NUL_IN_HEADER, FATAL) \\\n-\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n-\t/* errors */ \\\n-\tFUNC(BAD_DATE, ERROR) \\\n-\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n-\tFUNC(BAD_EMAIL, ERROR) \\\n-\tFUNC(BAD_NAME, ERROR) \\\n-\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n-\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n-\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n-\tFUNC(BAD_TIMEZONE, ERROR) \\\n-\tFUNC(BAD_TREE, ERROR) \\\n-\tFUNC(BAD_TREE_SHA1, ERROR) \\\n-\tFUNC(BAD_TYPE, ERROR) \\\n-\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n-\tFUNC(MISSING_AUTHOR, ERROR) \\\n-\tFUNC(MISSING_COMMITTER, ERROR) \\\n-\tFUNC(MISSING_EMAIL, ERROR) \\\n-\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_OBJECT, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_TAG, ERROR) \\\n-\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n-\tFUNC(MISSING_TREE, ERROR) \\\n-\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n-\tFUNC(MISSING_TYPE, ERROR) \\\n-\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n-\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n-\tFUNC(TREE_NOT_SORTED, ERROR) \\\n-\tFUNC(UNKNOWN_TYPE, ERROR) \\\n-\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n-\tFUNC(GITMODULES_MISSING, ERROR) \\\n-\tFUNC(GITMODULES_BLOB, ERROR) \\\n-\tFUNC(GITMODULES_LARGE, ERROR) \\\n-\tFUNC(GITMODULES_NAME, ERROR) \\\n-\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n-\tFUNC(GITMODULES_URL, ERROR) \\\n-\tFUNC(GITMODULES_PATH, ERROR) \\\n-\tFUNC(GITMODULES_UPDATE, ERROR) \\\n-\t/* warnings */ \\\n-\tFUNC(BAD_FILEMODE, WARN) \\\n-\tFUNC(EMPTY_NAME, WARN) \\\n-\tFUNC(FULL_PATHNAME, WARN) \\\n-\tFUNC(HAS_DOT, WARN) \\\n-\tFUNC(HAS_DOTDOT, WARN) \\\n-\tFUNC(HAS_DOTGIT, WARN) \\\n-\tFUNC(NULL_SHA1, WARN) \\\n-\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n-\tFUNC(NUL_IN_COMMIT, WARN) \\\n-\t/* infos (reported as warnings, but ignored by default) */ \\\n-\tFUNC(GITMODULES_PARSE, INFO) \\\n-\tFUNC(BAD_TAG_NAME, INFO) \\\n-\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n-\t/* ignored (elevated when requested) */ \\\n-\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n-\n-#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n-enum fsck_msg_id {\n-\tFOREACH_FSCK_MSG_ID(MSG_ID)\n-\tFSCK_MSG_MAX\n-};\n-#undef MSG_ID\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\ndiff --git a/fsck.h b/fsck.h\nindex fce9981a0c..c3d3b47b88 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -13,6 +13,72 @@ enum fsck_msg_type {\n \tFSCK_WARN,\n };\n \n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n+\t/* fatal errors */ \\\n+\tFUNC(NUL_IN_HEADER, FATAL) \\\n+\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n+\t/* errors */ \\\n+\tFUNC(BAD_DATE, ERROR) \\\n+\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n+\tFUNC(BAD_EMAIL, ERROR) \\\n+\tFUNC(BAD_NAME, ERROR) \\\n+\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n+\tFUNC(BAD_TIMEZONE, ERROR) \\\n+\tFUNC(BAD_TREE, ERROR) \\\n+\tFUNC(BAD_TREE_SHA1, ERROR) \\\n+\tFUNC(BAD_TYPE, ERROR) \\\n+\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n+\tFUNC(MISSING_AUTHOR, ERROR) \\\n+\tFUNC(MISSING_COMMITTER, ERROR) \\\n+\tFUNC(MISSING_EMAIL, ERROR) \\\n+\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_OBJECT, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_TAG, ERROR) \\\n+\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n+\tFUNC(MISSING_TREE, ERROR) \\\n+\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n+\tFUNC(MISSING_TYPE, ERROR) \\\n+\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n+\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(TREE_NOT_SORTED, ERROR) \\\n+\tFUNC(UNKNOWN_TYPE, ERROR) \\\n+\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n+\tFUNC(GITMODULES_MISSING, ERROR) \\\n+\tFUNC(GITMODULES_BLOB, ERROR) \\\n+\tFUNC(GITMODULES_LARGE, ERROR) \\\n+\tFUNC(GITMODULES_NAME, ERROR) \\\n+\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n+\tFUNC(GITMODULES_URL, ERROR) \\\n+\tFUNC(GITMODULES_PATH, ERROR) \\\n+\tFUNC(GITMODULES_UPDATE, ERROR) \\\n+\t/* warnings */ \\\n+\tFUNC(BAD_FILEMODE, WARN) \\\n+\tFUNC(EMPTY_NAME, WARN) \\\n+\tFUNC(FULL_PATHNAME, WARN) \\\n+\tFUNC(HAS_DOT, WARN) \\\n+\tFUNC(HAS_DOTDOT, WARN) \\\n+\tFUNC(HAS_DOTGIT, WARN) \\\n+\tFUNC(NULL_SHA1, WARN) \\\n+\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n+\tFUNC(NUL_IN_COMMIT, WARN) \\\n+\t/* infos (reported as warnings, but ignored by default) */ \\\n+\tFUNC(GITMODULES_PARSE, INFO) \\\n+\tFUNC(BAD_TAG_NAME, INFO) \\\n+\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\t/* ignored (elevated when requested) */ \\\n+\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n+\n+#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n+enum fsck_msg_id {\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n+\tFSCK_MSG_MAX\n+};\n+#undef MSG_ID\n+\n struct fsck_options;\n struct object;\n \n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419369","messageId":"20210316161738.30254-20-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 19/22] fsck.c: move gitmodules_{found,done} into fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:35Z","receivedAt":"2021-03-16T16:19:19Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the gitmodules_{found,done} static variables added in\n159e7b080bf (fsck: detect gitmodules files, 2018-05-02) into the\nfsck_options struct. It makes sense to keep all the context in the\nsame place.\n\nThis requires changing the recently added register_found_gitmodules()\nfunction added in 5476e1efde (fetch-pack: print and use dangling\n.gitmodules, 2021-02-22) to take fsck_options. That function will be\nremoved in a subsequent commit, but as it'll require the new\ngitmodules_found attribute of \"fsck_options\" we need this intermediate\nstep first.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fetch-pack.c | 2 +-\n fsck.c | 23 ++++++++++-------------\n fsck.h | 7 ++++++-\n 3 files changed, 17 insertions(+), 15 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 6a61a46428..82c3c2c043 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -998,7 +998,7 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(oid);\n+\t\tregister_found_gitmodules(&fo, oid);\n \tif (fsck_finish(&fo))\n \t\tdie(\"fsck failed\");\n }\ndiff --git a/fsck.c b/fsck.c\nindex f26f47b2a1..565274a946 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -19,9 +19,6 @@\n #include \"credential.h\"\n #include \"help.h\"\n \n-static struct oidset gitmodules_found = OIDSET_INIT;\n-static struct oidset gitmodules_done = OIDSET_INIT;\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\n@@ -624,7 +621,7 @@ static int fsck_tree(const struct object_id *oid,\n \n \t\tif (is_hfs_dotgitmodules(name) || is_ntfs_dotgitmodules(name)) {\n \t\t\tif (!S_ISLNK(mode))\n-\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\telse\n \t\t\t\tretval += report(options,\n \t\t\t\t\t\t oid, OBJ_TREE,\n@@ -638,7 +635,7 @@ static int fsck_tree(const struct object_id *oid,\n \t\t\t\thas_dotgit |= is_ntfs_dotgit(backslash);\n \t\t\t\tif (is_ntfs_dotgitmodules(backslash)) {\n \t\t\t\t\tif (!S_ISLNK(mode))\n-\t\t\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\t\t\telse\n \t\t\t\t\t\tretval += report(options, oid, OBJ_TREE,\n \t\t\t\t\t\t\t\t FSCK_MSG_GITMODULES_SYMLINK,\n@@ -1150,9 +1147,9 @@ static int fsck_blob(const struct object_id *oid, const char *buf,\n \tstruct fsck_gitmodules_data data;\n \tstruct config_options config_opts = { 0 };\n \n-\tif (!oidset_contains(&gitmodules_found, oid))\n+\tif (!oidset_contains(&options->gitmodules_found, oid))\n \t\treturn 0;\n-\toidset_insert(&gitmodules_done, oid);\n+\toidset_insert(&options->gitmodules_done, oid);\n \n \tif (object_on_skiplist(options, oid))\n \t\treturn 0;\n@@ -1217,9 +1214,9 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n-void register_found_gitmodules(const struct object_id *oid)\n+void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n {\n-\toidset_insert(&gitmodules_found, oid);\n+\toidset_insert(&options->gitmodules_found, oid);\n }\n \n int fsck_finish(struct fsck_options *options)\n@@ -1228,13 +1225,13 @@ int fsck_finish(struct fsck_options *options)\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n \n-\toidset_iter_init(&gitmodules_found, &iter);\n+\toidset_iter_init(&options->gitmodules_found, &iter);\n \twhile ((oid = oidset_iter_next(&iter))) {\n \t\tenum object_type type;\n \t\tunsigned long size;\n \t\tchar *buf;\n \n-\t\tif (oidset_contains(&gitmodules_done, oid))\n+\t\tif (oidset_contains(&options->gitmodules_done, oid))\n \t\t\tcontinue;\n \n \t\tbuf = read_object_file(oid, &type, &size);\n@@ -1259,8 +1256,8 @@ int fsck_finish(struct fsck_options *options)\n \t}\n \n \n-\toidset_clear(&gitmodules_found);\n-\toidset_clear(&gitmodules_done);\n+\toidset_clear(&options->gitmodules_found);\n+\toidset_clear(&options->gitmodules_done);\n \treturn ret;\n }\n \ndiff --git a/fsck.h b/fsck.h\nindex 6c2fd9c5cc..bb59ef05b6 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -118,6 +118,8 @@ struct fsck_options {\n \tunsigned strict:1;\n \tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n+\tstruct oidset gitmodules_found;\n+\tstruct oidset gitmodules_done;\n \tkh_oid_map_t *object_names;\n };\n \n@@ -125,6 +127,8 @@ struct fsck_options {\n \t.walk = NULL, \\\n \t.msg_type = NULL, \\\n \t.skiplist = OIDSET_INIT, \\\n+\t.gitmodules_found = OIDSET_INIT, \\\n+\t.gitmodules_done = OIDSET_INIT, \\\n \t.object_names = NULL,\n #define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n \tFSCK_OPTIONS_COMMON \\\n@@ -149,7 +153,8 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n-void register_found_gitmodules(const struct object_id *oid);\n+void register_found_gitmodules(struct fsck_options *options,\n+\t\t\t const struct object_id *oid);\n \n /*\n * fsck a tag, and pass info about it back to the caller. This is\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419370","messageId":"20210316161738.30254-19-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 18/22] fsck.c: add an fsck_set_msg_type() API that takes enums","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:34Z","receivedAt":"2021-03-16T16:19:19Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code I added in acf9de4c94e (mktag: use fsck instead of custom\nverify_tag(), 2021-01-05) to make use of a new API function that takes\nthe fsck_msg_{id,type} types, instead of arbitrary strings that\nwe'll (hopefully) parse into those types.\n\nAt the time that the fsck_set_msg_type() API was introduced in\n0282f4dced0 (fsck: offer a function to demote fsck errors to warnings,\n2015-06-22) it was only intended to be used to parse user-supplied\ndata.\n\nFor things that are purely internal to the C code it makes sense to\nhave the compiler check these arguments, and to skip the sanity\nchecking of the data in fsck_set_msg_type() which is redundant to\nchecks we get from the compiler.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/mktag.c | 3 ++-\n fsck.c | 27 +++++++++++++++++----------\n fsck.h | 3 +++\n 3 files changed, 22 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex dc989c356f..de67a94f24 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -93,7 +93,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)\n \t\tdie_errno(_(\"could not read from stdin\"));\n \n \tfsck_options.error_func = mktag_fsck_error_func;\n-\tfsck_set_msg_type(&fsck_options, \"extraheaderentry\", \"warn\");\n+\tfsck_set_msg_type_from_ids(&fsck_options, FSCK_MSG_EXTRA_HEADER_ENTRY,\n+\t\t\t\t FSCK_WARN);\n \t/* config might set fsck.extraHeaderEntry=* again */\n \tgit_config(mktag_config, NULL);\n \tif (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,\ndiff --git a/fsck.c b/fsck.c\nindex 5a040eb4fd..f26f47b2a1 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -132,6 +132,22 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n \treturn 1;\n }\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type)\n+{\n+\tif (!options->msg_type) {\n+\t\tint i;\n+\t\tenum fsck_msg_type *severity;\n+\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n+\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n+\t\t\tseverity[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = severity;\n+\t}\n+\n+\toptions->msg_type[msg_id] = msg_type;\n+}\n+\n void fsck_set_msg_type(struct fsck_options *options,\n \t\tconst char *msg_id_str, const char *msg_type_str)\n {\n@@ -144,16 +160,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n-\tif (!options->msg_type) {\n-\t\tint i;\n-\t\tenum fsck_msg_type *severity;\n-\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n-\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tseverity[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = severity;\n-\t}\n-\n-\toptions->msg_type[msg_id] = msg_type;\n+\tfsck_set_msg_type_from_ids(options, msg_id, msg_type);\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\ndiff --git a/fsck.h b/fsck.h\nindex 33ecf3f3f1..6c2fd9c5cc 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -82,6 +82,9 @@ enum fsck_msg_id {\n struct fsck_options;\n struct object;\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type);\n void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419371","messageId":"20210316161738.30254-18-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 17/22] fsck.c: pass along the fsck_msg_id in the fsck_error callback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:33Z","receivedAt":"2021-03-16T16:19:19Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_error callback to also pass along the\nfsck_msg_id. Before this change the only way to get the message id was\nto parse it back out of the \"message\".\n\nLet's pass it down explicitly for the benefit of callers that might\nwant to use it, as discussed in [1].\n\nPassing the msg_type is now redundant, as you can always get it back\nfrom the msg_id, but I'm not changing that convention. It's really\ncommon to need the msg_type, and the report() function itself (which\ncalls \"fsck_error\") needs to call fsck_msg_type() to discover\nit. Let's not needlessly re-do that work in the user callback.\n\n1. https://lore.kernel.org/git/87blcja2ha.fsf@evledraar.gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 4 +++-\n builtin/index-pack.c | 3 ++-\n builtin/mktag.c | 1 +\n fsck.c | 6 ++++--\n fsck.h | 6 ++++--\n 5 files changed, 14 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex d6d745dc70..b71fac4cec 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -89,7 +89,9 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 56b8efaa89..2b2266a4b7 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1717,6 +1717,7 @@ static int print_dangling_gitmodules(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n \t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n \t\t\t\t const char *message)\n {\n \t/*\n@@ -1727,7 +1728,7 @@ static int print_dangling_gitmodules(struct fsck_options *o,\n \t\tprintf(\"%s\\n\", oid_to_hex(oid));\n \t\treturn 0;\n \t}\n-\treturn fsck_error_function(o, oid, object_type, msg_type, message);\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n }\n \n int cmd_index_pack(int argc, const char **argv, const char *prefix)\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 1834394a9b..dc989c356f 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -23,6 +23,7 @@ static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n \t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n \t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\ndiff --git a/fsck.c b/fsck.c\nindex 8a66168e51..5a040eb4fd 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -245,7 +245,7 @@ static int report(struct fsck_options *options,\n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n \tresult = options->error_func(options, oid, object_type,\n-\t\t\t\t msg_type, sb.buf);\n+\t\t\t\t msg_type, msg_id, sb.buf);\n \tstrbuf_release(&sb);\n \tva_end(ap);\n \n@@ -1198,7 +1198,9 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type,\n+\t\t\tenum fsck_msg_id msg_id,\n+\t\t\tconst char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex c3d3b47b88..33ecf3f3f1 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -101,11 +101,13 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\t const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\tconst char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419372","messageId":"20210316161738.30254-23-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 22/22] fetch-pack: use new fsck API to printing dangling submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:38Z","receivedAt":"2021-03-16T16:19:19Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor the check added in 5476e1efde (fetch-pack: print and use\ndangling .gitmodules, 2021-02-22) to make use of us now passing the\n\"msg_id\" to the user defined \"error_func\". We can now compare against\nthe FSCK_MSG_GITMODULES_MISSING instead of parsing the generated\nmessage.\n\nLet's also replace register_found_gitmodules() with directly\nmanipulating the \"gitmodules_found\" member. A recent commit moved it\ninto \"fsck_options\" so we could do this here.\n\nAdd a fsck-cb.c file similar to parse-options-cb.c, the alternative\nwould be to either define this directly in fsck.c as a public API, or\nto create some library shared by fetch-pack.c ad builtin/index-pack.\n\nI expect that there won't be many of these fsck utility functions in\nthe future, so just having a single fsck-cb.c makes sense.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 1 +\n builtin/index-pack.c | 21 +--------------------\n fetch-pack.c | 4 ++--\n fsck-cb.c | 16 ++++++++++++++++\n fsck.c | 5 -----\n fsck.h | 22 +++++++++++++++++++---\n 6 files changed, 39 insertions(+), 30 deletions(-)\n create mode 100644 fsck-cb.c\n\ndiff --git a/Makefile b/Makefile\nindex dfb0f1000f..3faa8bd0d3 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -882,6 +882,7 @@ LIB_OBJS += fetch-negotiator.o\n LIB_OBJS += fetch-pack.o\n LIB_OBJS += fmt-merge-msg.o\n LIB_OBJS += fsck.o\n+LIB_OBJS += fsck-cb.o\n LIB_OBJS += fsmonitor.o\n LIB_OBJS += gettext.o\n LIB_OBJS += gpg-interface.o\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 5ad80b85b4..11f0fafd33 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -120,7 +120,7 @@ static int nr_threads;\n static int from_stdin;\n static int strict;\n static int do_fsck_object;\n-static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_MISSING_GITMODULES;\n static int verbose;\n static int show_resolving_progress;\n static int show_stat;\n@@ -1713,24 +1713,6 @@ static void show_pack_info(int stat_only)\n \t}\n }\n \n-static int print_dangling_gitmodules(struct fsck_options *o,\n-\t\t\t\t const struct object_id *oid,\n-\t\t\t\t enum object_type object_type,\n-\t\t\t\t enum fsck_msg_type msg_type,\n-\t\t\t\t enum fsck_msg_id msg_id,\n-\t\t\t\t const char *message)\n-{\n-\t/*\n-\t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\n-\t * instead of relying on this string check.\n-\t */\n-\tif (starts_with(message, \"gitmodulesMissing\")) {\n-\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n-\t\treturn 0;\n-\t}\n-\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n-}\n-\n int cmd_index_pack(int argc, const char **argv, const char *prefix)\n {\n \tint i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;\n@@ -1761,7 +1743,6 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \n \tread_replace_refs = 0;\n \tfsck_options.walk = mark_link;\n-\tfsck_options.error_func = print_dangling_gitmodules;\n \n \treset_pack_idx_option(&opts);\n \tgit_config(git_index_pack_config, &opts);\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 229fd8e2c2..008a3facd4 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -38,7 +38,7 @@ static int server_supports_filtering;\n static int advertise_sid;\n static struct shallow_lock shallow_lock;\n static const char *alternate_shallow_file;\n-static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_MISSING_GITMODULES;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n \n@@ -998,7 +998,7 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(&fsck_options, oid);\n+\t\toidset_insert(&fsck_options.gitmodules_found, oid);\n \tif (fsck_finish(&fsck_options))\n \t\tdie(\"fsck failed\");\n }\ndiff --git a/fsck-cb.c b/fsck-cb.c\nnew file mode 100644\nindex 0000000000..465a49235a\n--- /dev/null\n+++ b/fsck-cb.c\n@@ -0,0 +1,16 @@\n+#include \"git-compat-util.h\"\n+#include \"fsck.h\"\n+\n+int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n+\t\t\t\t\t const struct object_id *oid,\n+\t\t\t\t\t enum object_type object_type,\n+\t\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t\t const char *message)\n+{\n+\tif (msg_id == FSCK_MSG_GITMODULES_MISSING) {\n+\t\tputs(oid_to_hex(oid));\n+\t\treturn 0;\n+\t}\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n+}\ndiff --git a/fsck.c b/fsck.c\nindex 565274a946..b0089844db 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1214,11 +1214,6 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n-void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n-{\n-\toidset_insert(&options->gitmodules_found, oid);\n-}\n-\n int fsck_finish(struct fsck_options *options)\n {\n \tint ret = 0;\ndiff --git a/fsck.h b/fsck.h\nindex bb59ef05b6..ae3107638a 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -153,9 +153,6 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n-void register_found_gitmodules(struct fsck_options *options,\n-\t\t\t const struct object_id *oid);\n-\n /*\n * fsck a tag, and pass info about it back to the caller. This is\n * exposed fsck_object() internals for git-mktag(1).\n@@ -204,4 +201,23 @@ const char *fsck_describe_object(struct fsck_options *options,\n int fsck_config_internal(const char *var, const char *value, void *cb,\n \t\t\t struct fsck_options *options);\n \n+/*\n+ * Initializations for callbacks in fsck-cb.c\n+ */\n+#define FSCK_OPTIONS_MISSING_GITMODULES { \\\n+\t.strict = 1, \\\n+\t.error_func = fsck_error_cb_print_missing_gitmodules, \\\n+\tFSCK_OPTIONS_COMMON \\\n+}\n+\n+/*\n+ * Error callbacks in fsck-cb.c\n+ */\n+int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n+\t\t\t\t\t const struct object_id *oid,\n+\t\t\t\t\t enum object_type object_type,\n+\t\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t\t const char *message);\n+\n #endif\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419373","messageId":"20210316161738.30254-22-avarab@gmail.com","threadId":"54994","inReplyTo":"20210306110439.27694-1-avarab@gmail.com","subject":"[PATCH v4 21/22] fetch-pack: use file-scope static struct for fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-16T16:17:37Z","receivedAt":"2021-03-16T16:19:19Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code added in 5476e1efde (fetch-pack: print and use dangling\n.gitmodules, 2021-02-22) so that we use a file-scoped \"static struct\nfsck_options\" instead of defining one in the \"fsck_gitmodules_oids()\"\nfunction.\n\nWe use this pattern in all of\nbuiltin/{fsck,index-pack,mktag,unpack-objects}.c. It's odd to see\nfetch-pack be the odd one out. One might think that we're using other\nfsck_options structs in fetch-pack, or doing on fsck twice there, but\nwe're not.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fetch-pack.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 82c3c2c043..229fd8e2c2 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -38,6 +38,7 @@ static int server_supports_filtering;\n static int advertise_sid;\n static struct shallow_lock shallow_lock;\n static const char *alternate_shallow_file;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n \n@@ -991,15 +992,14 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n {\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n-\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n \n \tif (!oidset_size(gitmodules_oids))\n \t\treturn;\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(&fo, oid);\n-\tif (fsck_finish(&fo))\n+\t\tregister_found_gitmodules(&fsck_options, oid);\n+\tif (fsck_finish(&fsck_options))\n \t\tdie(\"fsck failed\");\n }\n \n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419402","messageId":"eba92c82-1a19-f1ea-1bad-a639b9eb98bd@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-3-avarab@gmail.com","subject":"Re: [PATCH v4 02/22] fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-03-16T18:59:38Z","receivedAt":"2021-03-16T19:00:40Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 3/16/2021 12:17 PM, Ævar Arnfjörð Bjarmason wrote:\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.h | 18 ++++++++++++++++--\n> 1 file changed, 16 insertions(+), 2 deletions(-)\n> \n> diff --git a/fsck.h b/fsck.h\n> index 2274843ba0..40f3cb3f64 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -43,8 +43,22 @@ struct fsck_options {\n> \tkh_oid_map_t *object_names;\n> };\n> \n> -#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n> -#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n\nYou just edited these lines in the previous patch. Seems unnecesary\nto split them. You can point out that the object_names portion was\npreviously excluded in the message here.\n\n> +#define FSCK_OPTIONS_DEFAULT { \\\n> +\t.walk = NULL, \\\n> +\t.error_func = fsck_error_function, \\\n> +\t.strict = 0, \\\n> +\t.msg_type = NULL, \\\n> +\t.skiplist = OIDSET_INIT, \\\n> +\t.object_names = NULL, \\\n> +}\n> +#define FSCK_OPTIONS_STRICT { \\\n> +\t.walk = NULL, \\\n> +\t.error_func = fsck_error_function, \\\n> +\t.strict = 1, \\\n> +\t.msg_type = NULL, \\\n> +\t.skiplist = OIDSET_INIT, \\\n> +\t.object_names = NULL, \\\n> +}\n\nThis explicit definition is better.\n\nThanks,\n-Stolee\n"},{"id":"419403","messageId":"b0d72d72-6e3f-0022-417a-2dc253b7dac5@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-5-avarab@gmail.com","subject":"Re: [PATCH v4 04/22] fsck.h: add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-03-16T19:06:30Z","receivedAt":"2021-03-16T19:07:45Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 3/16/2021 12:17 PM, Ævar Arnfjörð Bjarmason wrote:\n> Add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro for those that would like\n> to use FSCK_OPTIONS_COMMON in their own initialization, but supply\n> their own error functions.\n> \n> Nothing is being changed to use this yet, but in some subsequent\n> commits we'll make use of this macro.\n> \n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.h | 9 ++++++---\n> 1 file changed, 6 insertions(+), 3 deletions(-)\n> \n> diff --git a/fsck.h b/fsck.h\n> index ea3a907ec3..dc35924cbf 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -45,12 +45,15 @@ struct fsck_options {\n> \n> #define FSCK_OPTIONS_COMMON \\\n> \t.walk = NULL, \\\n> -\t.error_func = fsck_error_function, \\\n> \t.msg_type = NULL, \\\n> \t.skiplist = OIDSET_INIT, \\\n> \t.object_names = NULL,\n> -#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON }\n> -#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON }\n> +#define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n> +\tFSCK_OPTIONS_COMMON \\\n> +\t.error_func = fsck_error_function\n> +\n> +#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n> +#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n\nOK. It seems like you are converging on your final definitions\nfor these macros. At first glance, this seems like unnecessary\nsplit to demonstrate the tiny changes between, but it could\njust be done with one change and a description of why you want\nthe four different entry points as macros.\n\nThanks,\n-Stolee\n"},{"id":"419404","messageId":"72b6e263-1e53-cc00-9545-cddd11820fe4@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-23-avarab@gmail.com","subject":"Re: [PATCH v4 22/22] fetch-pack: use new fsck API to printing dangling submodules","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-03-16T19:32:13Z","receivedAt":"2021-03-16T19:32:57Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 3/16/2021 12:17 PM, Ævar Arnfjörð Bjarmason wrote:\n> Refactor the check added in 5476e1efde (fetch-pack: print and use\n> dangling .gitmodules, 2021-02-22) to make use of us now passing the\n> \"msg_id\" to the user defined \"error_func\". We can now compare against\n> the FSCK_MSG_GITMODULES_MISSING instead of parsing the generated\n> message.\n> \n> Let's also replace register_found_gitmodules() with directly\n> manipulating the \"gitmodules_found\" member. A recent commit moved it\n> into \"fsck_options\" so we could do this here.\n> \n> Add a fsck-cb.c file similar to parse-options-cb.c, the alternative\n> would be to either define this directly in fsck.c as a public API, or\n> to create some library shared by fetch-pack.c ad builtin/index-pack.\n> \n> I expect that there won't be many of these fsck utility functions in\n> the future, so just having a single fsck-cb.c makes sense.\n\nI'm not convinced that having a single cb function merits its\nown file. But, if you expect this pattern to be expanded a\ncouple more times, then I would say it is worth it. Do you have\nsuch plans?\n\nThanks,\n-Stolee\n"},{"id":"419405","messageId":"b561cdc4-7dad-d923-5403-40f898c577e0@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"Re: [PATCH v4 00/22] fsck: API improvements","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-03-16T19:35:20Z","receivedAt":"2021-03-16T19:36:09Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 3/16/2021 12:17 PM, Ævar Arnfjörð Bjarmason wrote:\n> A re-send of a rebased v3, which I sent at:\n> http://lore.kernel.org/git/20210306110439.27694-1-avarab@gmail.com as\n> seen in the range-diff there are no changes since v3. I'm just sending\n> this as a post-release bump of this, per\n> https://lore.kernel.org/git/xmqqy2etczqi.fsf@gitster.g/\n> \n> Ævar Arnfjörð Bjarmason (22):\n> fsck.h: update FSCK_OPTIONS_* for object_name\n> fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n> fsck.h: reduce duplication between FSCK_OPTIONS_{DEFAULT,STRICT}\n> fsck.h: add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro\n> fsck.h: indent arguments to of fsck_set_msg_type\n> fsck.h: use \"enum object_type\" instead of \"int\"\n> fsck.c: rename variables in fsck_set_msg_type() for less confusion\n> fsck.c: move definition of msg_id into append_msg_id()\n> fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n> fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"\n> fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n> fsck.h: re-order and re-assign \"enum fsck_msg_type\"\n> fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n> fsck.c: undefine temporary STR macro after use\n> fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n> fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n> fsck.c: pass along the fsck_msg_id in the fsck_error callback\n> fsck.c: add an fsck_set_msg_type() API that takes enums\n> fsck.c: move gitmodules_{found,done} into fsck_options\n> fetch-pack: don't needlessly copy fsck_options\n> fetch-pack: use file-scope static struct for fsck_options\n> fetch-pack: use new fsck API to printing dangling submodules\n\nThis series is carefully organized and motivated. It was quite\neasy to read.\n\nMy complaints were minor. One was that patches 1-4 seemed to be\nunnecessarily granular. I'm not sure that having four patches\nlike that will be more helpful for inspecting the history in\nthe future. But, I don't care enough to say this should be\nre-rolled.\n\nFinally, the last issue is that fsck-cb.c is loosely justified\nwith only one method inside. If you have plans in the near\nfuture to add similar methods there, then I think that is fine.\nOtherwise, it would be simpler to avoid the extra file and\ncode move.\n\nThanks,\n-Stolee\n"},{"id":"419470","messageId":"87tup9zybl.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"72b6e263-1e53-cc00-9545-cddd11820fe4@gmail.com","subject":"Re: [PATCH v4 22/22] fetch-pack: use new fsck API to printing dangling submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T13:47:42Z","receivedAt":"2021-03-17T13:48:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Mar 16 2021, Derrick Stolee wrote:\n\n> On 3/16/2021 12:17 PM, Ævar Arnfjörð Bjarmason wrote:\n>> Refactor the check added in 5476e1efde (fetch-pack: print and use\n>> dangling .gitmodules, 2021-02-22) to make use of us now passing the\n>> \"msg_id\" to the user defined \"error_func\". We can now compare against\n>> the FSCK_MSG_GITMODULES_MISSING instead of parsing the generated\n>> message.\n>> \n>> Let's also replace register_found_gitmodules() with directly\n>> manipulating the \"gitmodules_found\" member. A recent commit moved it\n>> into \"fsck_options\" so we could do this here.\n>> \n>> Add a fsck-cb.c file similar to parse-options-cb.c, the alternative\n>> would be to either define this directly in fsck.c as a public API, or\n>> to create some library shared by fetch-pack.c ad builtin/index-pack.\n>> \n>> I expect that there won't be many of these fsck utility functions in\n>> the future, so just having a single fsck-cb.c makes sense.\n>\n> I'm not convinced that having a single cb function merits its\n> own file. But, if you expect this pattern to be expanded a\n> couple more times, then I would say it is worth it. Do you have\n> such plans?\n\nNot really, well. Vague ones, but nothing I have even local patches for.\n\nIt just seemed odd to stick random callback functions shared by related\nprograms into fsck.h's interface, but I guess with\nFSCK_OPTIONS_MISSING_GITMODULES I already did that.\n\nDo you suggest just putting it into fsck.c?\n"},{"id":"419502","messageId":"20210317182054.5986-2-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 01/19] fsck.c: refactor and rename common config callback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:36Z","receivedAt":"2021-03-17T18:21:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor code I recently changed in 1f3299fda9 (fsck: make\nfsck_config() re-usable, 2021-01-05) so that I could use fsck's config\ncallback in mktag in 1f3299fda9 (fsck: make fsck_config() re-usable,\n2021-01-05).\n\nI don't know what I was thinking in structuring the code this way, but\nit clearly makes no sense to have an fsck_config_internal() at all\njust so it can get a fsck_options when git_config() already supports\npassing along some void* data.\n\nLet's just make use of that instead, which gets us rid of the two\nwrapper functions, and brings fsck's common config callback in line\nwith other such reusable config callbacks.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 7 +------\n builtin/mktag.c | 7 +------\n fsck.c | 4 ++--\n fsck.h | 3 +--\n 4 files changed, 5 insertions(+), 16 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 821e7798c7..a56a2d0513 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -71,11 +71,6 @@ static const char *printable_type(const struct object_id *oid,\n \treturn ret;\n }\n \n-static int fsck_config(const char *var, const char *value, void *cb)\n-{\n-\treturn fsck_config_internal(var, value, cb, &fsck_obj_options);\n-}\n-\n static int objerror(struct object *obj, const char *err)\n {\n \terrors_found |= ERROR_OBJECT;\n@@ -803,7 +798,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \tif (name_objects)\n \t\tfsck_enable_object_names(&fsck_walk_options);\n \n-\tgit_config(fsck_config, NULL);\n+\tgit_config(git_fsck_config, &fsck_obj_options);\n \n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 41a399a69e..23c4b8763f 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -14,11 +14,6 @@ static int option_strict = 1;\n \n static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n \n-static int mktag_config(const char *var, const char *value, void *cb)\n-{\n-\treturn fsck_config_internal(var, value, cb, &fsck_options);\n-}\n-\n static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n@@ -93,7 +88,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)\n \tfsck_options.error_func = mktag_fsck_error_func;\n \tfsck_set_msg_type(&fsck_options, \"extraheaderentry\", \"warn\");\n \t/* config might set fsck.extraHeaderEntry=* again */\n-\tgit_config(mktag_config, NULL);\n+\tgit_config(git_fsck_config, &fsck_options);\n \tif (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,\n \t\t\t\t&tagged_oid, &tagged_type))\n \t\tdie(_(\"tag on stdin did not pass our strict fsck check\"));\ndiff --git a/fsck.c b/fsck.c\nindex e3030f3b35..5dfb99665a 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1323,9 +1323,9 @@ int fsck_finish(struct fsck_options *options)\n \treturn ret;\n }\n \n-int fsck_config_internal(const char *var, const char *value, void *cb,\n-\t\t\t struct fsck_options *options)\n+int git_fsck_config(const char *var, const char *value, void *cb)\n {\n+\tstruct fsck_options *options = cb;\n \tif (strcmp(var, \"fsck.skiplist\") == 0) {\n \t\tconst char *path;\n \t\tstruct strbuf sb = STRBUF_INIT;\ndiff --git a/fsck.h b/fsck.h\nindex 733378f126..f70d11c559 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -109,7 +109,6 @@ const char *fsck_describe_object(struct fsck_options *options,\n * git_config() callback for use by fsck-y tools that want to support\n * fsck. fsck.skipList etc.\n */\n-int fsck_config_internal(const char *var, const char *value, void *cb,\n-\t\t\t struct fsck_options *options);\n+int git_fsck_config(const char *var, const char *value, void *cb);\n \n #endif\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419503","messageId":"20210317182054.5986-3-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 02/19] fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:37Z","receivedAt":"2021-03-17T18:21:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor the definitions of FSCK_OPTIONS_{DEFAULT,STRICT} to use\ndesignated initializers.\n\nWhile I'm at it add the \"object_names\" member to the\ninitialization. This was omitted in 7b35efd734e (fsck_walk():\noptionally name objects on the go, 2016-07-17) when the field was\nadded.\n\nI'm using a new FSCK_OPTIONS_COMMON and FSCK_OPTIONS_COMMON_ERROR_FUNC\nhelper macros to define what FSCK_OPTIONS_{DEFAULT,STRICT} have in\ncommon, and define the two in terms of those macro.\n\nThe FSCK_OPTIONS_COMMON macro will be used in a subsequent commit to\ndefine other variants of common fsck initialization that wants to use\na custom error function, but share the rest of the defaults.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 13 +++++++++++--\n 1 file changed, 11 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex f70d11c559..15e12f292f 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -43,8 +43,17 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n-#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n+#define FSCK_OPTIONS_COMMON \\\n+\t.walk = NULL, \\\n+\t.msg_type = NULL, \\\n+\t.skiplist = OIDSET_INIT, \\\n+\t.object_names = NULL,\n+#define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n+\tFSCK_OPTIONS_COMMON \\\n+\t.error_func = fsck_error_function\n+\n+#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n+#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419504","messageId":"20210317182054.5986-4-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 03/19] fsck.h: use \"enum object_type\" instead of \"int\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:38Z","receivedAt":"2021-03-17T18:21:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_walk_func to use an \"enum object_type\" instead of an\n\"int\" type. The types are compatible, and ever since this was added in\n355885d5315 (add generic, type aware object chain walker, 2008-02-25)\nwe've used entries from object_type (OBJ_BLOB etc.).\n\nSo this doesn't really change anything as far as the generated code is\nconcerned, it just gives the compiler more information and makes this\neasier to read.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 3 ++-\n builtin/index-pack.c | 3 ++-\n builtin/unpack-objects.c | 3 ++-\n fsck.h | 3 ++-\n 4 files changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex a56a2d0513..ed5f2af6b5 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -192,7 +192,8 @@ static int traverse_reachable(void)\n \treturn !!result;\n }\n \n-static int mark_used(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_used(struct object *obj, enum object_type object_type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn 1;\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex bad5748807..69f24fe9f7 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -212,7 +212,8 @@ static void cleanup_thread(void)\n \tfree(thread_data);\n }\n \n-static int mark_link(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_link(struct object *obj, enum object_type type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn -1;\ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex dd4a75e030..ca54fd1668 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -187,7 +187,8 @@ static void write_cached_object(struct object *obj, struct obj_buffer *obj_buf)\n * that have reachability requirements and calls this function.\n * Verify its reachability and validity recursively and write it out.\n */\n-static int check_object(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int check_object(struct object *obj, enum object_type type,\n+\t\t\tvoid *data, struct fsck_options *options)\n {\n \tstruct obj_buffer *obj_buf;\n \ndiff --git a/fsck.h b/fsck.h\nindex 15e12f292f..e3edaff8e7 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -23,7 +23,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type);\n * <0\terror signaled and abort\n * >0\terror signaled and do not abort\n */\n-typedef int (*fsck_walk_func)(struct object *obj, int type, void *data, struct fsck_options *options);\n+typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n+\t\t\t void *data, struct fsck_options *options);\n \n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419505","messageId":"20210317182054.5986-1-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 00/19] fsck: API improvements","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:35Z","receivedAt":"2021-03-17T18:21:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"A v5 with changes suggested by Derrick Stolee. Link to v4:\nhttps://lore.kernel.org/git/20210316161738.30254-1-avarab@gmail.com/\n\nChanges:\n\n * 1/19 is new, it's a simple refactoring of some git_config() code in\n fsck.c code I changed recently.\n\n * Squashed the first 4x patches of incrementally redefining two\n macros into one.\n\n * Squashed a whitespace-only change into another patch that changed\n the same code.\n\n * Got rid of fsck-cb.c, that one function just lives at the bottom of\n fsck.c now.\n\nÆvar Arnfjörð Bjarmason (19):\n fsck.c: refactor and rename common config callback\n fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n fsck.h: use \"enum object_type\" instead of \"int\"\n fsck.c: rename variables in fsck_set_msg_type() for less confusion\n fsck.c: move definition of msg_id into append_msg_id()\n fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"\n fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n fsck.h: re-order and re-assign \"enum fsck_msg_type\"\n fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n fsck.c: undefine temporary STR macro after use\n fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n fsck.c: pass along the fsck_msg_id in the fsck_error callback\n fsck.c: add an fsck_set_msg_type() API that takes enums\n fsck.c: move gitmodules_{found,done} into fsck_options\n fetch-pack: don't needlessly copy fsck_options\n fetch-pack: use file-scope static struct for fsck_options\n fetch-pack: use new fsck API to printing dangling submodules\n\n builtin/fsck.c | 14 ++-\n builtin/index-pack.c | 30 +-----\n builtin/mktag.c | 14 ++-\n builtin/unpack-objects.c | 3 +-\n fetch-pack.c | 6 +-\n fsck.c | 197 +++++++++++++++------------------------\n fsck.h | 131 +++++++++++++++++++++++---\n 7 files changed, 213 insertions(+), 182 deletions(-)\n\nRange-diff:\n 1: 9cd942b526 < -: ---------- fsck.h: update FSCK_OPTIONS_* for object_name\n 2: d67966b838 < -: ---------- fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n 3: 211472e0c5 < -: ---------- fsck.h: reduce duplication between FSCK_OPTIONS_{DEFAULT,STRICT}\n 4: 70afee988d < -: ---------- fsck.h: add a FSCK_OPTIONS_COMMON_ERROR_FUNC macro\n 5: 1337d53352 < -: ---------- fsck.h: indent arguments to of fsck_set_msg_type\n -: ---------- > 1: fe33015e0d fsck.c: refactor and rename common config callback\n -: ---------- > 2: 72f2e53afa fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n 6: e4ef107bb4 = 3: 237a280686 fsck.h: use \"enum object_type\" instead of \"int\"\n 7: 20bac3207e ! 4: 13b76c73dd fsck.c: rename variables in fsck_set_msg_type() for less confusion\n @@ Commit message\n \"msg_id\" to \"msg_id_str\" etc. This will make a follow-up change\n smaller.\n \n + While I'm at it properly indent the fsck_set_msg_type() argument list.\n +\n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## fsck.c ##\n @@ fsck.c: int is_valid_msg_type(const char *msg_id, const char *msg_type)\n \n void fsck_set_msg_type(struct fsck_options *options,\n -\t\tconst char *msg_id, const char *msg_type)\n -+\t\tconst char *msg_id_str, const char *msg_type_str)\n ++\t\t const char *msg_id_str, const char *msg_type_str)\n {\n -\tint id = parse_msg_id(msg_id), type;\n +\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n @@ fsck.c: int is_valid_msg_type(const char *msg_id, const char *msg_type)\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\n +\n + ## fsck.h ##\n +@@ fsck.h: struct fsck_options;\n + struct object;\n + \n + void fsck_set_msg_type(struct fsck_options *options,\n +-\t\tconst char *msg_id, const char *msg_type);\n ++\t\t const char *msg_id, const char *msg_type);\n + void fsck_set_msg_types(struct fsck_options *options, const char *values);\n + int is_valid_msg_type(const char *msg_id, const char *msg_type);\n + \n 8: 09c3bba9e9 = 5: 4ae83403b7 fsck.c: move definition of msg_id into append_msg_id()\n 9: 8067df53a2 = 6: 82107f1dac fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n10: bdf5e13f3d = 7: 796096bf73 fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"\n11: b03caa237f ! 8: 3664abb23d fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n @@ builtin/index-pack.c: static void show_pack_info(int stat_only)\n \t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\n \n ## builtin/mktag.c ##\n -@@ builtin/mktag.c: static int mktag_config(const char *var, const char *value, void *cb)\n +@@ builtin/mktag.c: static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n @@ fsck.c: static int fsck_msg_type(enum fsck_msg_id msg_id,\n \t\treturn FSCK_ERROR;\n @@ fsck.c: int is_valid_msg_type(const char *msg_id, const char *msg_type)\n void fsck_set_msg_type(struct fsck_options *options,\n - \t\tconst char *msg_id_str, const char *msg_type_str)\n + \t\t const char *msg_id_str, const char *msg_type_str)\n {\n -\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n +\tint msg_id = parse_msg_id(msg_id_str);\n12: 7b1d13b4cc = 9: 81e6d7ab45 fsck.h: re-order and re-assign \"enum fsck_msg_type\"\n13: a8e4ca7b19 ! 10: 5c2e8e7b84 fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n @@ Commit message\n \n ## fsck.c ##\n @@ fsck.c: void fsck_set_msg_type(struct fsck_options *options,\n - \t\tconst char *msg_id_str, const char *msg_type_str)\n + \t\t const char *msg_id_str, const char *msg_type_str)\n {\n \tint msg_id = parse_msg_id(msg_id_str);\n -\tenum fsck_msg_type msg_type;\n14: 214c375a20 = 11: 7ffbf9af3f fsck.c: undefine temporary STR macro after use\n15: 19a2499a80 = 12: 12ff0f75eb fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n16: 6e1a7b6274 = 13: 0c49dd5164 fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n17: 42af4e164c = 14: 900263f503 fsck.c: pass along the fsck_msg_id in the fsck_error callback\n18: fa47f473a8 ! 15: 5f270e88a0 fsck.c: add an fsck_set_msg_type() API that takes enums\n @@ builtin/mktag.c: int cmd_mktag(int argc, const char **argv, const char *prefix)\n +\tfsck_set_msg_type_from_ids(&fsck_options, FSCK_MSG_EXTRA_HEADER_ENTRY,\n +\t\t\t\t FSCK_WARN);\n \t/* config might set fsck.extraHeaderEntry=* again */\n - \tgit_config(mktag_config, NULL);\n + \tgit_config(git_fsck_config, &fsck_options);\n \tif (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,\n \n ## fsck.c ##\n @@ fsck.c: int is_valid_msg_type(const char *msg_id, const char *msg_type)\n +}\n +\n void fsck_set_msg_type(struct fsck_options *options,\n - \t\tconst char *msg_id_str, const char *msg_type_str)\n + \t\t const char *msg_id_str, const char *msg_type_str)\n {\n @@ fsck.c: void fsck_set_msg_type(struct fsck_options *options,\n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n19: 4cc3880cc4 = 16: 539d019712 fsck.c: move gitmodules_{found,done} into fsck_options\n20: fd219d318a = 17: 1acf744236 fetch-pack: don't needlessly copy fsck_options\n21: e4cd8c250e = 18: b47c3d5ac6 fetch-pack: use file-scope static struct for fsck_options\n22: fdbc3c304c ! 19: f05fa5c3ec fetch-pack: use new fsck API to printing dangling submodules\n @@ Commit message\n manipulating the \"gitmodules_found\" member. A recent commit moved it\n into \"fsck_options\" so we could do this here.\n \n - Add a fsck-cb.c file similar to parse-options-cb.c, the alternative\n - would be to either define this directly in fsck.c as a public API, or\n - to create some library shared by fetch-pack.c ad builtin/index-pack.\n + I'm sticking this callback in fsck.c. Perhaps in the future we'd like\n + to accumulate such callbacks into another file (maybe fsck-cb.c,\n + similar to parse-options-cb.c?), but while we've got just the one\n + let's just put it into fsck.c.\n \n - I expect that there won't be many of these fsck utility functions in\n - the future, so just having a single fsck-cb.c makes sense.\n + A better alternative in this case would be some library some more\n + obvious library shared by fetch-pack.c ad builtin/index-pack.c, but\n + there isn't such a thing.\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n - ## Makefile ##\n -@@ Makefile: LIB_OBJS += fetch-negotiator.o\n - LIB_OBJS += fetch-pack.o\n - LIB_OBJS += fmt-merge-msg.o\n - LIB_OBJS += fsck.o\n -+LIB_OBJS += fsck-cb.o\n - LIB_OBJS += fsmonitor.o\n - LIB_OBJS += gettext.o\n - LIB_OBJS += gpg-interface.o\n -\n ## builtin/index-pack.c ##\n @@ builtin/index-pack.c: static int nr_threads;\n static int from_stdin;\n @@ fetch-pack.c: static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n \t\tdie(\"fsck failed\");\n }\n \n - ## fsck-cb.c (new) ##\n -@@\n -+#include \"git-compat-util.h\"\n -+#include \"fsck.h\"\n + ## fsck.c ##\n +@@ fsck.c: int fsck_error_function(struct fsck_options *o,\n + \treturn 1;\n + }\n + \n +-void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n +-{\n +-\toidset_insert(&options->gitmodules_found, oid);\n +-}\n +-\n + int fsck_finish(struct fsck_options *options)\n + {\n + \tint ret = 0;\n +@@ fsck.c: int git_fsck_config(const char *var, const char *value, void *cb)\n + \n + \treturn git_default_config(var, value, cb);\n + }\n ++\n ++/*\n ++ * Custom error callbacks that are used in more than one place.\n ++ */\n +\n +int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n +\t\t\t\t\t const struct object_id *oid,\n @@ fsck-cb.c (new)\n +\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n +}\n \n - ## fsck.c ##\n -@@ fsck.c: int fsck_error_function(struct fsck_options *o,\n - \treturn 1;\n - }\n - \n --void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n --{\n --\toidset_insert(&options->gitmodules_found, oid);\n --}\n --\n - int fsck_finish(struct fsck_options *options)\n - {\n - \tint ret = 0;\n -\n ## fsck.h ##\n @@ fsck.h: int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n @@ fsck.h: int fsck_walk(struct object *obj, void *data, struct fsck_options *optio\n * fsck a tag, and pass info about it back to the caller. This is\n * exposed fsck_object() internals for git-mktag(1).\n @@ fsck.h: const char *fsck_describe_object(struct fsck_options *options,\n - int fsck_config_internal(const char *var, const char *value, void *cb,\n - \t\t\t struct fsck_options *options);\n + */\n + int git_fsck_config(const char *var, const char *value, void *cb);\n \n +/*\n -+ * Initializations for callbacks in fsck-cb.c\n ++ * Custom error callbacks that are used in more than one place.\n + */\n +#define FSCK_OPTIONS_MISSING_GITMODULES { \\\n +\t.strict = 1, \\\n +\t.error_func = fsck_error_cb_print_missing_gitmodules, \\\n +\tFSCK_OPTIONS_COMMON \\\n +}\n -+\n -+/*\n -+ * Error callbacks in fsck-cb.c\n -+ */\n +int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n +\t\t\t\t\t const struct object_id *oid,\n +\t\t\t\t\t enum object_type object_type,\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419506","messageId":"20210317182054.5986-5-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 04/19] fsck.c: rename variables in fsck_set_msg_type() for less confusion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:39Z","receivedAt":"2021-03-17T18:21:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename variables in a function added in 0282f4dced0 (fsck: offer a\nfunction to demote fsck errors to warnings, 2015-06-22).\n\nIt was needlessly confusing that it took a \"msg_type\" argument, but\nthen later declared another \"msg_type\" of a different type.\n\nLet's rename that to \"severity\", and rename \"id\" to \"msg_id\" and\n\"msg_id\" to \"msg_id_str\" etc. This will make a follow-up change\nsmaller.\n\nWhile I'm at it properly indent the fsck_set_msg_type() argument list.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 24 ++++++++++++------------\n fsck.h | 2 +-\n 2 files changed, 13 insertions(+), 13 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 5dfb99665a..7cc722a25c 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -203,27 +203,27 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n }\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type)\n+\t\t const char *msg_id_str, const char *msg_type_str)\n {\n-\tint id = parse_msg_id(msg_id), type;\n+\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n \n-\tif (id < 0)\n-\t\tdie(\"Unhandled message id: %s\", msg_id);\n-\ttype = parse_msg_type(msg_type);\n+\tif (msg_id < 0)\n+\t\tdie(\"Unhandled message id: %s\", msg_id_str);\n+\tmsg_type = parse_msg_type(msg_type_str);\n \n-\tif (type != FSCK_ERROR && msg_id_info[id].msg_type == FSCK_FATAL)\n-\t\tdie(\"Cannot demote %s to %s\", msg_id, msg_type);\n+\tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n+\t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *msg_type;\n-\t\tALLOC_ARRAY(msg_type, FSCK_MSG_MAX);\n+\t\tint *severity;\n+\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tmsg_type[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = msg_type;\n+\t\t\tseverity[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = severity;\n \t}\n \n-\toptions->msg_type[id] = type;\n+\toptions->msg_type[msg_id] = msg_type;\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\ndiff --git a/fsck.h b/fsck.h\nindex e3edaff8e7..12ff99b56e 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -11,7 +11,7 @@ struct fsck_options;\n struct object;\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type);\n+\t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n int is_valid_msg_type(const char *msg_id, const char *msg_type);\n \n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419507","messageId":"20210317182054.5986-9-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 08/19] fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:43Z","receivedAt":"2021-03-17T18:21:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} defines into a new\nfsck_msg_type enum.\n\nThese defines were originally introduced in:\n\n - ba002f3b28a (builtin-fsck: move common object checking code to\n fsck.c, 2008-02-25)\n - f50c4407305 (fsck: disallow demoting grave fsck errors to warnings,\n 2015-06-22)\n - efaba7cc77f (fsck: optionally ignore specific fsck issues\n completely, 2015-06-22)\n - f27d05b1704 (fsck: allow upgrading fsck warnings to errors,\n 2015-06-22)\n\nThe reason these were defined in two different places is because we\nuse FSCK_{IGNORE,INFO,FATAL} only in fsck.c, but FSCK_{ERROR,WARN} are\nused by external callbacks.\n\nUntangling that would take some more work, since we expose the new\n\"enum fsck_msg_type\" to both. Similar to \"enum object_type\" it's not\nworth structuring the API in such a way that only those who need\nFSCK_{ERROR,WARN} pass around a different type.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 2 +-\n builtin/index-pack.c | 3 ++-\n builtin/mktag.c | 3 ++-\n fsck.c | 21 ++++++++++-----------\n fsck.h | 16 ++++++++++------\n 5 files changed, 25 insertions(+), 20 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex ed5f2af6b5..17940a4e24 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -84,7 +84,7 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t int msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type, const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 69f24fe9f7..56b8efaa89 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1716,7 +1716,8 @@ static void show_pack_info(int stat_only)\n static int print_dangling_gitmodules(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \t/*\n \t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 23c4b8763f..052a510ad7 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -17,7 +17,8 @@ static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/fsck.c b/fsck.c\nindex 2f23255f99..e1e942821d 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,9 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FSCK_FATAL -1\n-#define FSCK_INFO -2\n-\n #define FOREACH_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n@@ -97,7 +94,7 @@ static struct {\n \tconst char *id_string;\n \tconst char *downcased;\n \tconst char *camelcased;\n-\tint msg_type;\n+\tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n \tFOREACH_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n@@ -164,13 +161,13 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n \t\tlist_config_item(list, prefix, msg_id_info[i].camelcased);\n }\n \n-static int fsck_msg_type(enum fsck_msg_id msg_id,\n+static enum fsck_msg_type fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n \tif (!options->msg_type) {\n-\t\tint msg_type = msg_id_info[msg_id].msg_type;\n+\t\tenum fsck_msg_type msg_type = msg_id_info[msg_id].msg_type;\n \n \t\tif (options->strict && msg_type == FSCK_WARN)\n \t\t\tmsg_type = FSCK_ERROR;\n@@ -180,7 +177,7 @@ static int fsck_msg_type(enum fsck_msg_id msg_id,\n \treturn options->msg_type[msg_id];\n }\n \n-static int parse_msg_type(const char *str)\n+static enum fsck_msg_type parse_msg_type(const char *str)\n {\n \tif (!strcmp(str, \"error\"))\n \t\treturn FSCK_ERROR;\n@@ -203,7 +200,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id_str, const char *msg_type_str)\n {\n-\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n+\tint msg_id = parse_msg_id(msg_id_str);\n+\tenum fsck_msg_type msg_type;\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n@@ -214,7 +212,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *severity;\n+\t\tenum fsck_msg_type *severity;\n \t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n \t\t\tseverity[i] = fsck_msg_type(i, options);\n@@ -294,7 +292,8 @@ static int report(struct fsck_options *options,\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(msg_id, options), result;\n+\tenum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);\n+\tint result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -1265,7 +1264,7 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tint msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type, const char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex 12ff99b56e..0fff04373e 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -3,9 +3,13 @@\n \n #include \"oidset.h\"\n \n-#define FSCK_ERROR 1\n-#define FSCK_WARN 2\n-#define FSCK_IGNORE 3\n+enum fsck_msg_type {\n+\tFSCK_INFO = -2,\n+\tFSCK_FATAL = -1,\n+\tFSCK_ERROR = 1,\n+\tFSCK_WARN,\n+\tFSCK_IGNORE\n+};\n \n struct fsck_options;\n struct object;\n@@ -29,17 +33,17 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t int msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tint msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, const char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n \tfsck_error error_func;\n \tunsigned strict:1;\n-\tint *msg_type;\n+\tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n \tkh_oid_map_t *object_names;\n };\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419508","messageId":"20210317182054.5986-7-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 06/19] fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:41Z","receivedAt":"2021-03-17T18:21:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the remaining variables of type fsck_msg_id from \"id\" to\n\"msg_id\". This change is relatively small, and is worth the churn for\na later change where we have different id's in the \"report\" function.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex ffb9115ddb..a9a8783aeb 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,19 +264,19 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id msg_id)\n {\n-\tconst char *msg_id = msg_id_info[id].id_string;\n+\tconst char *msg_id_str = msg_id_info[msg_id].id_string;\n \tfor (;;) {\n-\t\tchar c = *(msg_id)++;\n+\t\tchar c = *(msg_id_str)++;\n \n \t\tif (!c)\n \t\t\tbreak;\n \t\tif (c != '_')\n \t\t\tstrbuf_addch(sb, tolower(c));\n \t\telse {\n-\t\t\tassert(*msg_id);\n-\t\t\tstrbuf_addch(sb, *(msg_id)++);\n+\t\t\tassert(*msg_id_str);\n+\t\t\tstrbuf_addch(sb, *(msg_id_str)++);\n \t\t}\n \t}\n \n@@ -292,11 +292,11 @@ static int object_on_skiplist(struct fsck_options *opts,\n __attribute__((format (printf, 5, 6)))\n static int report(struct fsck_options *options,\n \t\t const struct object_id *oid, enum object_type object_type,\n-\t\t enum fsck_msg_id id, const char *fmt, ...)\n+\t\t enum fsck_msg_id msg_id, const char *fmt, ...)\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(id, options), result;\n+\tint msg_type = fsck_msg_type(msg_id, options), result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -309,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, id);\n+\tappend_msg_id(&sb, msg_id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419509","messageId":"20210317182054.5986-6-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 05/19] fsck.c: move definition of msg_id into append_msg_id()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:40Z","receivedAt":"2021-03-17T18:21:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor code added in 71ab8fa840f (fsck: report the ID of the\nerror/warning, 2015-06-22) to resolve the msg_id to a string in the\nfunction that wants it, instead of doing it in report().\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 7cc722a25c..ffb9115ddb 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,8 +264,9 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, const char *msg_id)\n+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n {\n+\tconst char *msg_id = msg_id_info[id].id_string;\n \tfor (;;) {\n \t\tchar c = *(msg_id)++;\n \n@@ -308,7 +309,7 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, msg_id_info[id].id_string);\n+\tappend_msg_id(&sb, id);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419510","messageId":"20210317182054.5986-10-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 09/19] fsck.h: re-order and re-assign \"enum fsck_msg_type\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:44Z","receivedAt":"2021-03-17T18:21:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the values in the \"enum fsck_msg_type\" from being manually\nassigned to using default C enum values.\n\nThis means we end up with a FSCK_IGNORE=0, which was previously\ndefined as \"2\".\n\nI'm confident that nothing relies on these values, we always compare\nthem explicitly. Let's not omit \"0\" so it won't be assumed that we're\nusing these as a boolean somewhere.\n\nThis also allows us to re-structure the fields to mark which are\n\"private\" v.s. \"public\". See the preceding commit for a rationale for\nnot simply splitting these into two enums, namely that this is used\nfor both the private and public fsck API.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 10 ++++++----\n 1 file changed, 6 insertions(+), 4 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex 0fff04373e..25c456bbd3 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -4,11 +4,13 @@\n #include \"oidset.h\"\n \n enum fsck_msg_type {\n-\tFSCK_INFO = -2,\n-\tFSCK_FATAL = -1,\n-\tFSCK_ERROR = 1,\n+\t/* for internal use only */\n+\tFSCK_IGNORE,\n+\tFSCK_INFO,\n+\tFSCK_FATAL,\n+\t/* \"public\", fed to e.g. error_func callbacks */\n+\tFSCK_ERROR,\n \tFSCK_WARN,\n-\tFSCK_IGNORE\n };\n \n struct fsck_options;\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419511","messageId":"20210317182054.5986-8-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 07/19] fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:42Z","receivedAt":"2021-03-17T18:21:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor \"if options->msg_type\" and other code added in\n0282f4dced0 (fsck: offer a function to demote fsck errors to warnings,\n2015-06-22) to reduce the scope of the \"int msg_type\" variable.\n\nThis is in preparation for changing its type in a subsequent commit,\nonly using it in the \"!options->msg_type\" scope makes that change\n\nThis also brings the code in line with the fsck_set_msg_type()\nfunction (also added in 0282f4dced0), which does a similar check for\n\"!options->msg_type\". Another minor benefit is getting rid of the\nstyle violation of not having braces for the body of the \"if\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 12 +++++-------\n 1 file changed, 5 insertions(+), 7 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex a9a8783aeb..2f23255f99 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -167,19 +167,17 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n static int fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n-\tint msg_type;\n-\n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n-\tif (options->msg_type)\n-\t\tmsg_type = options->msg_type[msg_id];\n-\telse {\n-\t\tmsg_type = msg_id_info[msg_id].msg_type;\n+\tif (!options->msg_type) {\n+\t\tint msg_type = msg_id_info[msg_id].msg_type;\n+\n \t\tif (options->strict && msg_type == FSCK_WARN)\n \t\t\tmsg_type = FSCK_ERROR;\n+\t\treturn msg_type;\n \t}\n \n-\treturn msg_type;\n+\treturn options->msg_type[msg_id];\n }\n \n static int parse_msg_type(const char *str)\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419512","messageId":"20210317182054.5986-11-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 10/19] fsck.c: call parse_msg_type() early in fsck_set_msg_type()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:45Z","receivedAt":"2021-03-17T18:21:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"There's no reason to defer the calling of parse_msg_type() until after\nwe've checked if the \"id < 0\". This is not a hot codepath, and\nparse_msg_type() itself may die on invalid input.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex e1e942821d..341c482fed 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -201,11 +201,10 @@ void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id_str, const char *msg_type_str)\n {\n \tint msg_id = parse_msg_id(msg_id_str);\n-\tenum fsck_msg_type msg_type;\n+\tenum fsck_msg_type msg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n-\tmsg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419513","messageId":"20210317182054.5986-12-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 11/19] fsck.c: undefine temporary STR macro after use","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:46Z","receivedAt":"2021-03-17T18:21:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In f417eed8cde (fsck: provide a function to parse fsck message IDs,\n2015-06-22) the \"STR\" macro was introduced, but that short macro name\nwas not undefined after use as was done earlier in the same series for\nthe MSG_ID macro in c99ba492f1c (fsck: introduce identifiers for fsck\nmessages, 2015-06-22).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/fsck.c b/fsck.c\nindex 341c482fed..e657636a6f 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -100,6 +100,7 @@ static struct {\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n+#undef STR\n \n static void prepare_msg_ids(void)\n {\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419514","messageId":"20210317182054.5986-13-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 12/19] fsck.c: give \"FOREACH_MSG_ID\" a more specific name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:47Z","receivedAt":"2021-03-17T18:21:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the FOREACH_MSG_ID macro to FOREACH_FSCK_MSG_ID in preparation\nfor moving it over to fsck.h. It's good convention to name macros\nin *.h files in such a way as to clearly not clash with any other\nnames in other files.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex e657636a6f..b64526ea35 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,7 +22,7 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_MSG_ID(FUNC) \\\n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n \tFUNC(UNTERMINATED_HEADER, FATAL) \\\n@@ -83,7 +83,7 @@ static struct oidset gitmodules_done = OIDSET_INIT;\n \n #define MSG_ID(id, msg_type) FSCK_MSG_##id,\n enum fsck_msg_id {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \tFSCK_MSG_MAX\n };\n #undef MSG_ID\n@@ -96,7 +96,7 @@ static struct {\n \tconst char *camelcased;\n \tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419515","messageId":"20210317182054.5986-14-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 13/19] fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:48Z","receivedAt":"2021-03-17T18:22:26Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FOREACH_FSCK_MSG_ID macro and the fsck_msg_id enum it helps\ndefine from fsck.c to fsck.h. This is in preparation for having\nnon-static functions take the fsck_msg_id as an argument.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 66 ----------------------------------------------------------\n fsck.h | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 66 insertions(+), 66 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex b64526ea35..49208ec636 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,72 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_FSCK_MSG_ID(FUNC) \\\n-\t/* fatal errors */ \\\n-\tFUNC(NUL_IN_HEADER, FATAL) \\\n-\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n-\t/* errors */ \\\n-\tFUNC(BAD_DATE, ERROR) \\\n-\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n-\tFUNC(BAD_EMAIL, ERROR) \\\n-\tFUNC(BAD_NAME, ERROR) \\\n-\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n-\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n-\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n-\tFUNC(BAD_TIMEZONE, ERROR) \\\n-\tFUNC(BAD_TREE, ERROR) \\\n-\tFUNC(BAD_TREE_SHA1, ERROR) \\\n-\tFUNC(BAD_TYPE, ERROR) \\\n-\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n-\tFUNC(MISSING_AUTHOR, ERROR) \\\n-\tFUNC(MISSING_COMMITTER, ERROR) \\\n-\tFUNC(MISSING_EMAIL, ERROR) \\\n-\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_OBJECT, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_TAG, ERROR) \\\n-\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n-\tFUNC(MISSING_TREE, ERROR) \\\n-\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n-\tFUNC(MISSING_TYPE, ERROR) \\\n-\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n-\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n-\tFUNC(TREE_NOT_SORTED, ERROR) \\\n-\tFUNC(UNKNOWN_TYPE, ERROR) \\\n-\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n-\tFUNC(GITMODULES_MISSING, ERROR) \\\n-\tFUNC(GITMODULES_BLOB, ERROR) \\\n-\tFUNC(GITMODULES_LARGE, ERROR) \\\n-\tFUNC(GITMODULES_NAME, ERROR) \\\n-\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n-\tFUNC(GITMODULES_URL, ERROR) \\\n-\tFUNC(GITMODULES_PATH, ERROR) \\\n-\tFUNC(GITMODULES_UPDATE, ERROR) \\\n-\t/* warnings */ \\\n-\tFUNC(BAD_FILEMODE, WARN) \\\n-\tFUNC(EMPTY_NAME, WARN) \\\n-\tFUNC(FULL_PATHNAME, WARN) \\\n-\tFUNC(HAS_DOT, WARN) \\\n-\tFUNC(HAS_DOTDOT, WARN) \\\n-\tFUNC(HAS_DOTGIT, WARN) \\\n-\tFUNC(NULL_SHA1, WARN) \\\n-\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n-\tFUNC(NUL_IN_COMMIT, WARN) \\\n-\t/* infos (reported as warnings, but ignored by default) */ \\\n-\tFUNC(GITMODULES_PARSE, INFO) \\\n-\tFUNC(BAD_TAG_NAME, INFO) \\\n-\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n-\t/* ignored (elevated when requested) */ \\\n-\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n-\n-#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n-enum fsck_msg_id {\n-\tFOREACH_FSCK_MSG_ID(MSG_ID)\n-\tFSCK_MSG_MAX\n-};\n-#undef MSG_ID\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\ndiff --git a/fsck.h b/fsck.h\nindex 25c456bbd3..7c868410eb 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -13,6 +13,72 @@ enum fsck_msg_type {\n \tFSCK_WARN,\n };\n \n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n+\t/* fatal errors */ \\\n+\tFUNC(NUL_IN_HEADER, FATAL) \\\n+\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n+\t/* errors */ \\\n+\tFUNC(BAD_DATE, ERROR) \\\n+\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n+\tFUNC(BAD_EMAIL, ERROR) \\\n+\tFUNC(BAD_NAME, ERROR) \\\n+\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n+\tFUNC(BAD_TIMEZONE, ERROR) \\\n+\tFUNC(BAD_TREE, ERROR) \\\n+\tFUNC(BAD_TREE_SHA1, ERROR) \\\n+\tFUNC(BAD_TYPE, ERROR) \\\n+\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n+\tFUNC(MISSING_AUTHOR, ERROR) \\\n+\tFUNC(MISSING_COMMITTER, ERROR) \\\n+\tFUNC(MISSING_EMAIL, ERROR) \\\n+\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_OBJECT, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_TAG, ERROR) \\\n+\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n+\tFUNC(MISSING_TREE, ERROR) \\\n+\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n+\tFUNC(MISSING_TYPE, ERROR) \\\n+\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n+\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(TREE_NOT_SORTED, ERROR) \\\n+\tFUNC(UNKNOWN_TYPE, ERROR) \\\n+\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n+\tFUNC(GITMODULES_MISSING, ERROR) \\\n+\tFUNC(GITMODULES_BLOB, ERROR) \\\n+\tFUNC(GITMODULES_LARGE, ERROR) \\\n+\tFUNC(GITMODULES_NAME, ERROR) \\\n+\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n+\tFUNC(GITMODULES_URL, ERROR) \\\n+\tFUNC(GITMODULES_PATH, ERROR) \\\n+\tFUNC(GITMODULES_UPDATE, ERROR) \\\n+\t/* warnings */ \\\n+\tFUNC(BAD_FILEMODE, WARN) \\\n+\tFUNC(EMPTY_NAME, WARN) \\\n+\tFUNC(FULL_PATHNAME, WARN) \\\n+\tFUNC(HAS_DOT, WARN) \\\n+\tFUNC(HAS_DOTDOT, WARN) \\\n+\tFUNC(HAS_DOTGIT, WARN) \\\n+\tFUNC(NULL_SHA1, WARN) \\\n+\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n+\tFUNC(NUL_IN_COMMIT, WARN) \\\n+\t/* infos (reported as warnings, but ignored by default) */ \\\n+\tFUNC(GITMODULES_PARSE, INFO) \\\n+\tFUNC(BAD_TAG_NAME, INFO) \\\n+\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\t/* ignored (elevated when requested) */ \\\n+\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n+\n+#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n+enum fsck_msg_id {\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n+\tFSCK_MSG_MAX\n+};\n+#undef MSG_ID\n+\n struct fsck_options;\n struct object;\n \n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419516","messageId":"20210317182054.5986-17-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 16/19] fsck.c: move gitmodules_{found,done} into fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:51Z","receivedAt":"2021-03-17T18:22:26Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the gitmodules_{found,done} static variables added in\n159e7b080bf (fsck: detect gitmodules files, 2018-05-02) into the\nfsck_options struct. It makes sense to keep all the context in the\nsame place.\n\nThis requires changing the recently added register_found_gitmodules()\nfunction added in 5476e1efde (fetch-pack: print and use dangling\n.gitmodules, 2021-02-22) to take fsck_options. That function will be\nremoved in a subsequent commit, but as it'll require the new\ngitmodules_found attribute of \"fsck_options\" we need this intermediate\nstep first.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fetch-pack.c | 2 +-\n fsck.c | 23 ++++++++++-------------\n fsck.h | 7 ++++++-\n 3 files changed, 17 insertions(+), 15 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 6a61a46428..82c3c2c043 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -998,7 +998,7 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(oid);\n+\t\tregister_found_gitmodules(&fo, oid);\n \tif (fsck_finish(&fo))\n \t\tdie(\"fsck failed\");\n }\ndiff --git a/fsck.c b/fsck.c\nindex 307d454d92..00760b1f42 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -19,9 +19,6 @@\n #include \"credential.h\"\n #include \"help.h\"\n \n-static struct oidset gitmodules_found = OIDSET_INIT;\n-static struct oidset gitmodules_done = OIDSET_INIT;\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\n@@ -624,7 +621,7 @@ static int fsck_tree(const struct object_id *oid,\n \n \t\tif (is_hfs_dotgitmodules(name) || is_ntfs_dotgitmodules(name)) {\n \t\t\tif (!S_ISLNK(mode))\n-\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\telse\n \t\t\t\tretval += report(options,\n \t\t\t\t\t\t oid, OBJ_TREE,\n@@ -638,7 +635,7 @@ static int fsck_tree(const struct object_id *oid,\n \t\t\t\thas_dotgit |= is_ntfs_dotgit(backslash);\n \t\t\t\tif (is_ntfs_dotgitmodules(backslash)) {\n \t\t\t\t\tif (!S_ISLNK(mode))\n-\t\t\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\t\t\telse\n \t\t\t\t\t\tretval += report(options, oid, OBJ_TREE,\n \t\t\t\t\t\t\t\t FSCK_MSG_GITMODULES_SYMLINK,\n@@ -1150,9 +1147,9 @@ static int fsck_blob(const struct object_id *oid, const char *buf,\n \tstruct fsck_gitmodules_data data;\n \tstruct config_options config_opts = { 0 };\n \n-\tif (!oidset_contains(&gitmodules_found, oid))\n+\tif (!oidset_contains(&options->gitmodules_found, oid))\n \t\treturn 0;\n-\toidset_insert(&gitmodules_done, oid);\n+\toidset_insert(&options->gitmodules_done, oid);\n \n \tif (object_on_skiplist(options, oid))\n \t\treturn 0;\n@@ -1217,9 +1214,9 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n-void register_found_gitmodules(const struct object_id *oid)\n+void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n {\n-\toidset_insert(&gitmodules_found, oid);\n+\toidset_insert(&options->gitmodules_found, oid);\n }\n \n int fsck_finish(struct fsck_options *options)\n@@ -1228,13 +1225,13 @@ int fsck_finish(struct fsck_options *options)\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n \n-\toidset_iter_init(&gitmodules_found, &iter);\n+\toidset_iter_init(&options->gitmodules_found, &iter);\n \twhile ((oid = oidset_iter_next(&iter))) {\n \t\tenum object_type type;\n \t\tunsigned long size;\n \t\tchar *buf;\n \n-\t\tif (oidset_contains(&gitmodules_done, oid))\n+\t\tif (oidset_contains(&options->gitmodules_done, oid))\n \t\t\tcontinue;\n \n \t\tbuf = read_object_file(oid, &type, &size);\n@@ -1259,8 +1256,8 @@ int fsck_finish(struct fsck_options *options)\n \t}\n \n \n-\toidset_clear(&gitmodules_found);\n-\toidset_clear(&gitmodules_done);\n+\toidset_clear(&options->gitmodules_found);\n+\toidset_clear(&options->gitmodules_done);\n \treturn ret;\n }\n \ndiff --git a/fsck.h b/fsck.h\nindex 344c3ddc74..b25ae9d8b9 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -118,6 +118,8 @@ struct fsck_options {\n \tunsigned strict:1;\n \tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n+\tstruct oidset gitmodules_found;\n+\tstruct oidset gitmodules_done;\n \tkh_oid_map_t *object_names;\n };\n \n@@ -125,6 +127,8 @@ struct fsck_options {\n \t.walk = NULL, \\\n \t.msg_type = NULL, \\\n \t.skiplist = OIDSET_INIT, \\\n+\t.gitmodules_found = OIDSET_INIT, \\\n+\t.gitmodules_done = OIDSET_INIT, \\\n \t.object_names = NULL,\n #define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n \tFSCK_OPTIONS_COMMON \\\n@@ -149,7 +153,8 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n-void register_found_gitmodules(const struct object_id *oid);\n+void register_found_gitmodules(struct fsck_options *options,\n+\t\t\t const struct object_id *oid);\n \n /*\n * fsck a tag, and pass info about it back to the caller. This is\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419519","messageId":"20210317182054.5986-15-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 14/19] fsck.c: pass along the fsck_msg_id in the fsck_error callback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:49Z","receivedAt":"2021-03-17T18:22:26Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_error callback to also pass along the\nfsck_msg_id. Before this change the only way to get the message id was\nto parse it back out of the \"message\".\n\nLet's pass it down explicitly for the benefit of callers that might\nwant to use it, as discussed in [1].\n\nPassing the msg_type is now redundant, as you can always get it back\nfrom the msg_id, but I'm not changing that convention. It's really\ncommon to need the msg_type, and the report() function itself (which\ncalls \"fsck_error\") needs to call fsck_msg_type() to discover\nit. Let's not needlessly re-do that work in the user callback.\n\n1. https://lore.kernel.org/git/87blcja2ha.fsf@evledraar.gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 4 +++-\n builtin/index-pack.c | 3 ++-\n builtin/mktag.c | 1 +\n fsck.c | 6 ++++--\n fsck.h | 6 ++++--\n 5 files changed, 14 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 17940a4e24..70ff95837a 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -84,7 +84,9 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 56b8efaa89..2b2266a4b7 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1717,6 +1717,7 @@ static int print_dangling_gitmodules(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n \t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n \t\t\t\t const char *message)\n {\n \t/*\n@@ -1727,7 +1728,7 @@ static int print_dangling_gitmodules(struct fsck_options *o,\n \t\tprintf(\"%s\\n\", oid_to_hex(oid));\n \t\treturn 0;\n \t}\n-\treturn fsck_error_function(o, oid, object_type, msg_type, message);\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n }\n \n int cmd_index_pack(int argc, const char **argv, const char *prefix)\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 052a510ad7..96e63bc772 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -18,6 +18,7 @@ static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n \t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n \t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\ndiff --git a/fsck.c b/fsck.c\nindex 49208ec636..01b2724ac0 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -245,7 +245,7 @@ static int report(struct fsck_options *options,\n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n \tresult = options->error_func(options, oid, object_type,\n-\t\t\t\t msg_type, sb.buf);\n+\t\t\t\t msg_type, msg_id, sb.buf);\n \tstrbuf_release(&sb);\n \tva_end(ap);\n \n@@ -1198,7 +1198,9 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type,\n+\t\t\tenum fsck_msg_id msg_id,\n+\t\t\tconst char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex 7c868410eb..80b1984f34 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -101,11 +101,13 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\t const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\tconst char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419517","messageId":"20210317182054.5986-18-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 17/19] fetch-pack: don't needlessly copy fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:52Z","receivedAt":"2021-03-17T18:22:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the behavior of the .gitmodules validation added in\n5476e1efde (fetch-pack: print and use dangling .gitmodules,\n2021-02-22) so we're using one \"fsck_options\".\n\nI found that code confusing to read. One might think that not setting\nup the error_func earlier means that we're relying on the \"error_func\"\nnot being set in some code in between the two hunks being modified\nhere.\n\nBut we're not, all we're doing in the rest of \"cmd_index_pack()\" is\nfurther setup by calling fsck_set_msg_types(), and assigning to\ndo_fsck_object.\n\nSo there was no reason in 5476e1efde to make a shallow copy of the\nfsck_options struct before setting error_func. Let's just do this\nsetup at the top of the function, along with the \"walk\" assignment.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/index-pack.c | 10 +++-------\n 1 file changed, 3 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 2b2266a4b7..5ad80b85b4 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1761,6 +1761,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \n \tread_replace_refs = 0;\n \tfsck_options.walk = mark_link;\n+\tfsck_options.error_func = print_dangling_gitmodules;\n \n \treset_pack_idx_option(&opts);\n \tgit_config(git_index_pack_config, &opts);\n@@ -1951,13 +1952,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \telse\n \t\tclose(input_fd);\n \n-\tif (do_fsck_object) {\n-\t\tstruct fsck_options fo = fsck_options;\n-\n-\t\tfo.error_func = print_dangling_gitmodules;\n-\t\tif (fsck_finish(&fo))\n-\t\t\tdie(_(\"fsck error in pack objects\"));\n-\t}\n+\tif (do_fsck_object && fsck_finish(&fsck_options))\n+\t\tdie(_(\"fsck error in pack objects\"));\n \n \tfree(objects);\n \tstrbuf_release(&index_name_buf);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419518","messageId":"20210317182054.5986-16-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 15/19] fsck.c: add an fsck_set_msg_type() API that takes enums","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:50Z","receivedAt":"2021-03-17T18:22:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code I added in acf9de4c94e (mktag: use fsck instead of custom\nverify_tag(), 2021-01-05) to make use of a new API function that takes\nthe fsck_msg_{id,type} types, instead of arbitrary strings that\nwe'll (hopefully) parse into those types.\n\nAt the time that the fsck_set_msg_type() API was introduced in\n0282f4dced0 (fsck: offer a function to demote fsck errors to warnings,\n2015-06-22) it was only intended to be used to parse user-supplied\ndata.\n\nFor things that are purely internal to the C code it makes sense to\nhave the compiler check these arguments, and to skip the sanity\nchecking of the data in fsck_set_msg_type() which is redundant to\nchecks we get from the compiler.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/mktag.c | 3 ++-\n fsck.c | 27 +++++++++++++++++----------\n fsck.h | 3 +++\n 3 files changed, 22 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 96e63bc772..dddcccdd36 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -88,7 +88,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)\n \t\tdie_errno(_(\"could not read from stdin\"));\n \n \tfsck_options.error_func = mktag_fsck_error_func;\n-\tfsck_set_msg_type(&fsck_options, \"extraheaderentry\", \"warn\");\n+\tfsck_set_msg_type_from_ids(&fsck_options, FSCK_MSG_EXTRA_HEADER_ENTRY,\n+\t\t\t\t FSCK_WARN);\n \t/* config might set fsck.extraHeaderEntry=* again */\n \tgit_config(git_fsck_config, &fsck_options);\n \tif (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,\ndiff --git a/fsck.c b/fsck.c\nindex 01b2724ac0..307d454d92 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -132,6 +132,22 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n \treturn 1;\n }\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type)\n+{\n+\tif (!options->msg_type) {\n+\t\tint i;\n+\t\tenum fsck_msg_type *severity;\n+\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n+\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n+\t\t\tseverity[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = severity;\n+\t}\n+\n+\toptions->msg_type[msg_id] = msg_type;\n+}\n+\n void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id_str, const char *msg_type_str)\n {\n@@ -144,16 +160,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n-\tif (!options->msg_type) {\n-\t\tint i;\n-\t\tenum fsck_msg_type *severity;\n-\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n-\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tseverity[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = severity;\n-\t}\n-\n-\toptions->msg_type[msg_id] = msg_type;\n+\tfsck_set_msg_type_from_ids(options, msg_id, msg_type);\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\ndiff --git a/fsck.h b/fsck.h\nindex 80b1984f34..344c3ddc74 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -82,6 +82,9 @@ enum fsck_msg_id {\n struct fsck_options;\n struct object;\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type);\n void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419520","messageId":"20210317182054.5986-19-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 18/19] fetch-pack: use file-scope static struct for fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:53Z","receivedAt":"2021-03-17T18:22:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code added in 5476e1efde (fetch-pack: print and use dangling\n.gitmodules, 2021-02-22) so that we use a file-scoped \"static struct\nfsck_options\" instead of defining one in the \"fsck_gitmodules_oids()\"\nfunction.\n\nWe use this pattern in all of\nbuiltin/{fsck,index-pack,mktag,unpack-objects}.c. It's odd to see\nfetch-pack be the odd one out. One might think that we're using other\nfsck_options structs in fetch-pack, or doing on fsck twice there, but\nwe're not.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fetch-pack.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 82c3c2c043..229fd8e2c2 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -38,6 +38,7 @@ static int server_supports_filtering;\n static int advertise_sid;\n static struct shallow_lock shallow_lock;\n static const char *alternate_shallow_file;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n \n@@ -991,15 +992,14 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n {\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n-\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n \n \tif (!oidset_size(gitmodules_oids))\n \t\treturn;\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(&fo, oid);\n-\tif (fsck_finish(&fo))\n+\t\tregister_found_gitmodules(&fsck_options, oid);\n+\tif (fsck_finish(&fsck_options))\n \t\tdie(\"fsck failed\");\n }\n \n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419521","messageId":"20210317182054.5986-20-avarab@gmail.com","threadId":"54994","inReplyTo":"20210316161738.30254-1-avarab@gmail.com","subject":"[PATCH v5 19/19] fetch-pack: use new fsck API to printing dangling submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-17T18:20:54Z","receivedAt":"2021-03-17T18:22:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor the check added in 5476e1efde (fetch-pack: print and use\ndangling .gitmodules, 2021-02-22) to make use of us now passing the\n\"msg_id\" to the user defined \"error_func\". We can now compare against\nthe FSCK_MSG_GITMODULES_MISSING instead of parsing the generated\nmessage.\n\nLet's also replace register_found_gitmodules() with directly\nmanipulating the \"gitmodules_found\" member. A recent commit moved it\ninto \"fsck_options\" so we could do this here.\n\nI'm sticking this callback in fsck.c. Perhaps in the future we'd like\nto accumulate such callbacks into another file (maybe fsck-cb.c,\nsimilar to parse-options-cb.c?), but while we've got just the one\nlet's just put it into fsck.c.\n\nA better alternative in this case would be some library some more\nobvious library shared by fetch-pack.c ad builtin/index-pack.c, but\nthere isn't such a thing.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/index-pack.c | 21 +--------------------\n fetch-pack.c | 4 ++--\n fsck.c | 23 ++++++++++++++++++-----\n fsck.h | 18 +++++++++++++++---\n 4 files changed, 36 insertions(+), 30 deletions(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 5ad80b85b4..11f0fafd33 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -120,7 +120,7 @@ static int nr_threads;\n static int from_stdin;\n static int strict;\n static int do_fsck_object;\n-static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_MISSING_GITMODULES;\n static int verbose;\n static int show_resolving_progress;\n static int show_stat;\n@@ -1713,24 +1713,6 @@ static void show_pack_info(int stat_only)\n \t}\n }\n \n-static int print_dangling_gitmodules(struct fsck_options *o,\n-\t\t\t\t const struct object_id *oid,\n-\t\t\t\t enum object_type object_type,\n-\t\t\t\t enum fsck_msg_type msg_type,\n-\t\t\t\t enum fsck_msg_id msg_id,\n-\t\t\t\t const char *message)\n-{\n-\t/*\n-\t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\n-\t * instead of relying on this string check.\n-\t */\n-\tif (starts_with(message, \"gitmodulesMissing\")) {\n-\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n-\t\treturn 0;\n-\t}\n-\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n-}\n-\n int cmd_index_pack(int argc, const char **argv, const char *prefix)\n {\n \tint i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;\n@@ -1761,7 +1743,6 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \n \tread_replace_refs = 0;\n \tfsck_options.walk = mark_link;\n-\tfsck_options.error_func = print_dangling_gitmodules;\n \n \treset_pack_idx_option(&opts);\n \tgit_config(git_index_pack_config, &opts);\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 229fd8e2c2..008a3facd4 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -38,7 +38,7 @@ static int server_supports_filtering;\n static int advertise_sid;\n static struct shallow_lock shallow_lock;\n static const char *alternate_shallow_file;\n-static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_MISSING_GITMODULES;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n \n@@ -998,7 +998,7 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(&fsck_options, oid);\n+\t\toidset_insert(&fsck_options.gitmodules_found, oid);\n \tif (fsck_finish(&fsck_options))\n \t\tdie(\"fsck failed\");\n }\ndiff --git a/fsck.c b/fsck.c\nindex 00760b1f42..048cf81937 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1214,11 +1214,6 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n-void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n-{\n-\toidset_insert(&options->gitmodules_found, oid);\n-}\n-\n int fsck_finish(struct fsck_options *options)\n {\n \tint ret = 0;\n@@ -1284,3 +1279,21 @@ int git_fsck_config(const char *var, const char *value, void *cb)\n \n \treturn git_default_config(var, value, cb);\n }\n+\n+/*\n+ * Custom error callbacks that are used in more than one place.\n+ */\n+\n+int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n+\t\t\t\t\t const struct object_id *oid,\n+\t\t\t\t\t enum object_type object_type,\n+\t\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t\t const char *message)\n+{\n+\tif (msg_id == FSCK_MSG_GITMODULES_MISSING) {\n+\t\tputs(oid_to_hex(oid));\n+\t\treturn 0;\n+\t}\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n+}\ndiff --git a/fsck.h b/fsck.h\nindex b25ae9d8b9..da58f585d7 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -153,9 +153,6 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n-void register_found_gitmodules(struct fsck_options *options,\n-\t\t\t const struct object_id *oid);\n-\n /*\n * fsck a tag, and pass info about it back to the caller. This is\n * exposed fsck_object() internals for git-mktag(1).\n@@ -203,4 +200,19 @@ const char *fsck_describe_object(struct fsck_options *options,\n */\n int git_fsck_config(const char *var, const char *value, void *cb);\n \n+/*\n+ * Custom error callbacks that are used in more than one place.\n+ */\n+#define FSCK_OPTIONS_MISSING_GITMODULES { \\\n+\t.strict = 1, \\\n+\t.error_func = fsck_error_cb_print_missing_gitmodules, \\\n+\tFSCK_OPTIONS_COMMON \\\n+}\n+int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n+\t\t\t\t\t const struct object_id *oid,\n+\t\t\t\t\t enum object_type object_type,\n+\t\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t\t const char *message);\n+\n #endif\n-- \n2.31.0.260.g719c683c1d\n\n"},{"id":"419525","messageId":"xmqq7dm5wrvc.fsf@gitster.g","threadId":"54994","inReplyTo":"20210316161738.30254-2-avarab@gmail.com","subject":"Re: [PATCH v4 01/22] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T18:35:19Z","receivedAt":"2021-03-17T18:36:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Add the object_name member to the initialization macro. This was\n> omitted in 7b35efd734e (fsck_walk(): optionally name objects on the\n> go, 2016-07-17) when the field was added.\n\nWhile this does not hurt, as the missing one was and is at the end\nof the struct members, this has no effect. As you'll be rewriting\neverything into designated initializers anyway, does it matter, I\nhave to wonder (it would affect your commit count karma, but you\nalready have enough of them ;-)?\n\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.h | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/fsck.h b/fsck.h\n> index 733378f126..2274843ba0 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -43,8 +43,8 @@ struct fsck_options {\n> \tkh_oid_map_t *object_names;\n> };\n> \n> -#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n> -#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n> +#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n> +#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n> \n> /* descend in all linked child objects\n> * the return value is:\n"},{"id":"419526","messageId":"xmqq35wtwrq9.fsf@gitster.g","threadId":"54994","inReplyTo":"20210316161738.30254-3-avarab@gmail.com","subject":"Re: [PATCH v4 02/22] fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T18:38:22Z","receivedAt":"2021-03-17T18:39:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.h | 18 ++++++++++++++++--\n> 1 file changed, 16 insertions(+), 2 deletions(-)\n>\n> diff --git a/fsck.h b/fsck.h\n> index 2274843ba0..40f3cb3f64 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -43,8 +43,22 @@ struct fsck_options {\n> \tkh_oid_map_t *object_names;\n> };\n> \n> -#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n> -#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n> +#define FSCK_OPTIONS_DEFAULT { \\\n> +\t.walk = NULL, \\\n> +\t.error_func = fsck_error_function, \\\n> +\t.strict = 0, \\\n> +\t.msg_type = NULL, \\\n> +\t.skiplist = OIDSET_INIT, \\\n> +\t.object_names = NULL, \\\n> +}\n> +#define FSCK_OPTIONS_STRICT { \\\n> +\t.walk = NULL, \\\n> +\t.error_func = fsck_error_function, \\\n> +\t.strict = 1, \\\n> +\t.msg_type = NULL, \\\n> +\t.skiplist = OIDSET_INIT, \\\n> +\t.object_names = NULL, \\\n> +}\n\nBeing explicit is good, but spelling out zero initialization sounds\nmore like cluttering than clarifying. I do not mind .strict = 0 in\nthe DEFAULT one only because it contrasts well with .strict = 1 on\nthe STRICT side, but it would be easier to read to omit these zero\ninitilization of the .walk, .msg_type and .object_names members.\n\n> /* descend in all linked child objects\n> * the return value is:\n"},{"id":"419528","messageId":"xmqqsg4tvcup.fsf@gitster.g","threadId":"54994","inReplyTo":"20210316161738.30254-9-avarab@gmail.com","subject":"Re: [PATCH v4 08/22] fsck.c: move definition of msg_id into append_msg_id()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T18:45:02Z","receivedAt":"2021-03-17T18:46:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Refactor code added in 71ab8fa840f (fsck: report the ID of the\n> error/warning, 2015-06-22) to resolve the msg_id to a string in the\n> function that wants it, instead of doing it in report().\n\nThis reintroduces the same confusion 07/22 tried to get rid of,\nunless msg_id variable is renamed to msg_id_str in this step,\ninstead of being left to the next step, no?\n\n\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.c | 5 +++--\n> 1 file changed, 3 insertions(+), 2 deletions(-)\n>\n> diff --git a/fsck.c b/fsck.c\n> index 0a9ac9ca07..b977493f57 100644\n> --- a/fsck.c\n> +++ b/fsck.c\n> @@ -264,8 +264,9 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n> \tfree(to_free);\n> }\n> \n> -static void append_msg_id(struct strbuf *sb, const char *msg_id)\n> +static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n> {\n> +\tconst char *msg_id = msg_id_info[id].id_string;\n> \tfor (;;) {\n> \t\tchar c = *(msg_id)++;\n> \n> @@ -308,7 +309,7 @@ static int report(struct fsck_options *options,\n> \telse if (msg_type == FSCK_INFO)\n> \t\tmsg_type = FSCK_WARN;\n> \n> -\tappend_msg_id(&sb, msg_id_info[id].id_string);\n> +\tappend_msg_id(&sb, id);\n> \n> \tva_start(ap, fmt);\n> \tstrbuf_vaddf(&sb, fmt, ap);\n"},{"id":"419530","messageId":"xmqqo8fhvcpk.fsf@gitster.g","threadId":"54994","inReplyTo":"20210316161738.30254-12-avarab@gmail.com","subject":"Re: [PATCH v4 11/22] fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T18:48:07Z","receivedAt":"2021-03-17T18:48:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Move the FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} defines into a new\n> fsck_msg_type enum.\n\nNice.\n\n> Untangling that would take some more work, since we expose the new\n> \"enum fsck_msg_type\" to both. Similar to \"enum object_type\" it's not\n> worth structuring the API in such a way that only those who need\n> FSCK_{ERROR,WARN} pass around a different type.\n"},{"id":"419531","messageId":"xmqqk0q5vcl9.fsf@gitster.g","threadId":"54994","inReplyTo":"20210316161738.30254-13-avarab@gmail.com","subject":"Re: [PATCH v4 12/22] fsck.h: re-order and re-assign \"enum fsck_msg_type\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T18:50:42Z","receivedAt":"2021-03-17T18:51:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Change the values in the \"enum fsck_msg_type\" from being manually\n> assigned to using default C enum values.\n>\n> This means we end up with a FSCK_IGNORE=0, which was previously\n> defined as \"2\".\n>\n> I'm confident that nothing relies on these values, we always compare\n> them explicitly. Let's not omit \"0\" so it won't be assumed that we're\n> using these as a boolean somewhere.\n\nDo you mean by \"compare them explicitly\", we always compare for\nequality? \n\nIf the code had depended on constructs like \"if (msg < FSCK_ERROR)\",\nthis change would break badly.\n\n> diff --git a/fsck.h b/fsck.h\n> index 2ecc15eee7..fce9981a0c 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -4,11 +4,13 @@\n> #include \"oidset.h\"\n> \n> enum fsck_msg_type {\n> -\tFSCK_INFO = -2,\n> -\tFSCK_FATAL = -1,\n> -\tFSCK_ERROR = 1,\n> +\t/* for internal use only */\n> +\tFSCK_IGNORE,\n> +\tFSCK_INFO,\n> +\tFSCK_FATAL,\n> +\t/* \"public\", fed to e.g. error_func callbacks */\n> +\tFSCK_ERROR,\n> \tFSCK_WARN,\n> -\tFSCK_IGNORE\n> };\n> \n> struct fsck_options;\n"},{"id":"419532","messageId":"xmqqft0tvc9q.fsf@gitster.g","threadId":"54994","inReplyTo":"20210316161738.30254-15-avarab@gmail.com","subject":"Re: [PATCH v4 14/22] fsck.c: undefine temporary STR macro after use","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T18:57:37Z","receivedAt":"2021-03-17T18:58:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> In f417eed8cde (fsck: provide a function to parse fsck message IDs,\n> 2015-06-22) the \"STR\" macro was introduced, but that short macro name\n> was not undefined after use as was done earlier in the same series for\n> the MSG_ID macro in c99ba492f1c (fsck: introduce identifiers for fsck\n> messages, 2015-06-22).\n\nMakes sense. Thanks.\n\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.c | 1 +\n> 1 file changed, 1 insertion(+)\n>\n> diff --git a/fsck.c b/fsck.c\n> index 2ccf1a2f0f..f4c924ed04 100644\n> --- a/fsck.c\n> +++ b/fsck.c\n> @@ -100,6 +100,7 @@ static struct {\n> \t{ NULL, NULL, NULL, -1 }\n> };\n> #undef MSG_ID\n> +#undef STR\n> \n> static void prepare_msg_ids(void)\n> {\n"},{"id":"419533","messageId":"xmqqblbhvc42.fsf@gitster.g","threadId":"54994","inReplyTo":"20210316161738.30254-18-avarab@gmail.com","subject":"Re: [PATCH v4 17/22] fsck.c: pass along the fsck_msg_id in the fsck_error callback","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T19:01:01Z","receivedAt":"2021-03-17T19:02:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Change the fsck_error callback to also pass along the\n> fsck_msg_id. Before this change the only way to get the message id was\n> to parse it back out of the \"message\".\n\nNice.\n\n> Let's pass it down explicitly for the benefit of callers that might\n> want to use it, as discussed in [1].\n>\n> Passing the msg_type is now redundant, as you can always get it back\n> from the msg_id, but I'm not changing that convention. It's really\n> common to need the msg_type, and the report() function itself (which\n> calls \"fsck_error\") needs to call fsck_msg_type() to discover\n> it. Let's not needlessly re-do that work in the user callback.\n"},{"id":"419535","messageId":"xmqq5z1pvblg.fsf@gitster.g","threadId":"54994","inReplyTo":"20210316161738.30254-23-avarab@gmail.com","subject":"Re: [PATCH v4 22/22] fetch-pack: use new fsck API to printing dangling submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T19:12:11Z","receivedAt":"2021-03-17T19:13:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index 5ad80b85b4..11f0fafd33 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -120,7 +120,7 @@ static int nr_threads;\n> static int from_stdin;\n> static int strict;\n> static int do_fsck_object;\n> -static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n> +static struct fsck_options fsck_options = FSCK_OPTIONS_MISSING_GITMODULES;\n\nHmph, I do not think this is a good way to go. Specifically,\nfsck-cb.c with the definition of what this thing is, and in fsck.h\nfile the normal \"options\" initializers being defined quite far away\nfrom where this is defined, it is hard to see what is different\nbetween the normal strict one and MISSING_GITMODULES one.\n\nRather, it may be far simpler to keep only DEFAULT and STRICT, and\noverride .error_func at runtime in the codepath(s) that needs to,\nwhich would make it more clear what is going on. That way, we do\nnot need the split initializers with _ERROR_FUNC, which is another\nreason why the approach taken by this series is not a good idea (it\ndoes not scale---error-func may seem so special to deserve having\ntwo sets of macros that use the default one and leave the member\nunspecified, but it won't stay to be special forever).\n\nIOW,\n\n> @@ -1761,7 +1743,6 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n> \n> \tread_replace_refs = 0;\n> \tfsck_options.walk = mark_link;\n> -\tfsck_options.error_func = print_dangling_gitmodules;\n\nI doubt this hunk is an improvement.\n\n> diff --git a/fsck-cb.c b/fsck-cb.c\n> new file mode 100644\n> index 0000000000..465a49235a\n> --- /dev/null\n> +++ b/fsck-cb.c\n> @@ -0,0 +1,16 @@\n> +#include \"git-compat-util.h\"\n> +#include \"fsck.h\"\n> +\n> +int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n> +\t\t\t\t\t const struct object_id *oid,\n> +\t\t\t\t\t enum object_type object_type,\n> +\t\t\t\t\t enum fsck_msg_type msg_type,\n> +\t\t\t\t\t enum fsck_msg_id msg_id,\n> +\t\t\t\t\t const char *message)\n> +{\n> +\tif (msg_id == FSCK_MSG_GITMODULES_MISSING) {\n> +\t\tputs(oid_to_hex(oid));\n> +\t\treturn 0;\n> +\t}\n> +\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n> +}\n\nAs Derrick noticed, I do not know if we want to have a separate file\nfor this single function. Shouldn't it be part of builtin/index-pack.c,\nor do we want other places to do the same kind of checks?\n\nThanks.\n"},{"id":"419549","messageId":"9424d5c1-68d4-f6dc-7fb1-3b5617d3ad48@gmail.com","threadId":"54994","inReplyTo":"87tup9zybl.fsf@evledraar.gmail.com","subject":"Re: [PATCH v4 22/22] fetch-pack: use new fsck API to printing dangling submodules","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-03-17T20:27:06Z","receivedAt":"2021-03-17T20:28:04Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 3/17/2021 9:47 AM, Ævar Arnfjörð Bjarmason wrote:\n> \n> On Tue, Mar 16 2021, Derrick Stolee wrote:\n> \n>> On 3/16/2021 12:17 PM, Ævar Arnfjörð Bjarmason wrote:\n>>> I expect that there won't be many of these fsck utility functions in\n>>> the future, so just having a single fsck-cb.c makes sense.\n>>\n>> I'm not convinced that having a single cb function merits its\n>> own file. But, if you expect this pattern to be expanded a\n>> couple more times, then I would say it is worth it. Do you have\n>> such plans?\n> \n> Not really, well. Vague ones, but nothing I have even local patches for.\n> \n> It just seemed odd to stick random callback functions shared by related\n> programs into fsck.h's interface, but I guess with\n> FSCK_OPTIONS_MISSING_GITMODULES I already did that.\n> \n> Do you suggest just putting it into fsck.c?\n\nYeah, if it is frequently paired with fsck operations, I think it\nmakes the most sense there.\n\nAnd looking at it again, I'm not sure parse-options-cb.c has a\ngood excuse for being separate from parse-options.c, but that's\nthe current state so I wouldn't change it now.\n\nThanks,\n-Stolee\n"},{"id":"419550","messageId":"4150ab33-3fde-6f4d-80e6-ec697a264a6e@gmail.com","threadId":"54994","inReplyTo":"20210317182054.5986-1-avarab@gmail.com","subject":"Re: [PATCH v5 00/19] fsck: API improvements","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-03-17T20:30:49Z","receivedAt":"2021-03-17T20:31:34Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 3/17/2021 2:20 PM, Ævar Arnfjörð Bjarmason wrote:\n> A v5 with changes suggested by Derrick Stolee. Link to v4:\n> https://lore.kernel.org/git/20210316161738.30254-1-avarab@gmail.com/\n> \n> Changes:\n> \n> * 1/19 is new, it's a simple refactoring of some git_config() code in\n> fsck.c code I changed recently.\n\nThis new patch is simple, as advertised.\n \n> * Squashed the first 4x patches of incrementally redefining two\n> macros into one.\n\nThanks.\n\n> * Squashed a whitespace-only change into another patch that changed\n> the same code.\n> \n> * Got rid of fsck-cb.c, that one function just lives at the bottom of\n> fsck.c now.\n\nI was late in giving you confirmation that fsck.c is a good place,\nbut you got there, anyway. Thanks!\n\nThis version LGTM.\n\n-Stolee\n"},{"id":"419571","messageId":"xmqqk0q5trpo.fsf@gitster.g","threadId":"54994","inReplyTo":"20210317182054.5986-1-avarab@gmail.com","subject":"Re: [PATCH v5 00/19] fsck: API improvements","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-17T21:06:59Z","receivedAt":"2021-03-17T21:08:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> A v5 with changes suggested by Derrick Stolee. Link to v4:\n> https://lore.kernel.org/git/20210316161738.30254-1-avarab@gmail.com/\n>\n> Changes:\n>\n> * 1/19 is new, it's a simple refactoring of some git_config() code in\n> fsck.c code I changed recently.\n\nThe new step makes sense. I think the series and my comment e-mails\ncrossed, and all of my comments on the previous round still applies\n(including the fsck-cb.c thing, which I think should be added to its\nsole user index-pack.c, not to fsck.c).\n\nThanks, will queue.\n"},{"id":"419709","messageId":"nycvar.QRO.7.76.6.2103191540330.57@tvgsbejvaqbjf.bet","threadId":"54994","inReplyTo":"20210316161738.30254-2-avarab@gmail.com","subject":"Re: [PATCH v4 01/22] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-03-19T14:43:10Z","receivedAt":"2021-03-19T14:44:31Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\njust a general note: this patch, which is the first of v4, is marked as\nreplying to the cover letter of v3. That feels quite odd. If you use\nthreading, why not let it reply to the cover letter of the same patch\nseries iteration?\n\nIn other words, would you mind using the `--thread=shallow` option in the\nfuture, for better structuring on the mailing list?\n\nThanks,\nJohannes\n\nOn Tue, 16 Mar 2021, Ævar Arnfjörð Bjarmason wrote:\n\n> Add the object_name member to the initialization macro. This was\n> omitted in 7b35efd734e (fsck_walk(): optionally name objects on the\n> go, 2016-07-17) when the field was added.\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> fsck.h | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/fsck.h b/fsck.h\n> index 733378f126..2274843ba0 100644\n> --- a/fsck.h\n> +++ b/fsck.h\n> @@ -43,8 +43,8 @@ struct fsck_options {\n> \tkh_oid_map_t *object_names;\n> };\n>\n> -#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n> -#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n> +#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n> +#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n>\n> /* descend in all linked child objects\n> * the return value is:\n> --\n> 2.31.0.260.g719c683c1d\n>\n>\n"},{"id":"419766","messageId":"87czvuyyk6.fsf@evledraar.gmail.com","threadId":"54994","inReplyTo":"nycvar.QRO.7.76.6.2103191540330.57@tvgsbejvaqbjf.bet","subject":"Re: [PATCH v4 01/22] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-20T09:16:57Z","receivedAt":"2021-03-20T11:14:37Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Mar 19 2021, Johannes Schindelin wrote:\n\n> Hi Ævar,\n>\n> just a general note: this patch, which is the first of v4, is marked as\n> replying to the cover letter of v3. That feels quite odd. If you use\n> threading, why not let it reply to the cover letter of the same patch\n> series iteration?\n>\n> In other words, would you mind using the `--thread=shallow` option in the\n> future, for better structuring on the mailing list?\n\nNot at all, I've set it in my config now.\n\nI've just been using the default configuration of format-patch\n--in-reply-to --cover-letter && send-email *.patch all this time.\n\nLooking around at other patch submissions (aside from GGG) this seems to\nbe the norm though, but isn't documented in SubmittingPatches\netc. AFAICT.\n\nSo I wonder if I'm using some different process from the norm, or if\nmost everyone else is just looking carefully at Message-ID/In-Reply-To\nnorms before sending...\n\n> On Tue, 16 Mar 2021, Ævar Arnfjörð Bjarmason wrote:\n>\n>> Add the object_name member to the initialization macro. This was\n>> omitted in 7b35efd734e (fsck_walk(): optionally name objects on the\n>> go, 2016-07-17) when the field was added.\n>>\n>> Signed-off-by: Ævar Arnfjörð Bjarmason \n>> ---\n>> fsck.h | 4 ++--\n>> 1 file changed, 2 insertions(+), 2 deletions(-)\n>>\n>> diff --git a/fsck.h b/fsck.h\n>> index 733378f126..2274843ba0 100644\n>> --- a/fsck.h\n>> +++ b/fsck.h\n>> @@ -43,8 +43,8 @@ struct fsck_options {\n>> \tkh_oid_map_t *object_names;\n>> };\n>>\n>> -#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n>> -#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n>> +#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT, NULL }\n>> +#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT, NULL }\n>>\n>> /* descend in all linked child objects\n>> * the return value is:\n>> --\n>> 2.31.0.260.g719c683c1d\n>>\n>>\n\n"},{"id":"419778","messageId":"xmqqr1k9k2w7.fsf@gitster.g","threadId":"54994","inReplyTo":"87czvuyyk6.fsf@evledraar.gmail.com","subject":"Re: [PATCH v4 01/22] fsck.h: update FSCK_OPTIONS_* for object_name","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-20T20:04:40Z","receivedAt":"2021-03-20T20:05:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n>> In other words, would you mind using the `--thread=shallow` option in the\n>> future, for better structuring on the mailing list?\n>\n> Not at all, I've set it in my config now.\n>\n> I've just been using the default configuration of format-patch\n> --in-reply-to --cover-letter && send-email *.patch all this time.\n> ...\n> So I wonder if I'm using some different process from the norm, or if\n> most everyone else is just looking carefully at Message-ID/In-Reply-To\n> norms before sending...\n\nInteresting. I always let send-email assign the message IDs and\nhaven't used --thread= option at all. In other words, my\nformat-patch output files have no message IDs in them or In-reply-to\nheader fields. That in turn means that in-reply-to is decided not\nwhen format-patch is run, but when send-email sends things out, it\ngives them the ids and structures the in-reply-to chains.\n\nI guess we have too much flexibility in our tooling X-<.\n"},{"id":"420388","messageId":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"20210317182054.5986-1-avarab@gmail.com","subject":"[PATCH v6 00/19] fsck: API improvements","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:32Z","receivedAt":"2021-03-28T13:16:39Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"To recap on the goals in v1[1] this series gets rid of the need to\nhave the rececently added \"print_dangling_gitmodules\" function in\nfavor of a better fsck API to get at that information.\n\nChanges since v5[2]:\n\n * Addressed all outstanding feedback AFAICT\n * The fields we init to 0/NULL in the new designated initializer are\n gone\n * There were comments on the refactoring of append_msg_id(), It turns\n out that we can entirely remove that function. So a new commit go\n added + one ejected to do that.\n * Clarifications in commit messages.\n * I'd still left behind a remnant of the old\n \"print_dangling_gitmodules\" code in v5's last commit. I.e. we had\n code that was accumulating its own list of gitmodules OIDs and then\n injecting into the fsck state, now that the fsck state tracks those\n itself we can that list directly instead.\n\n1. https://lore.kernel.org/git/20210217194246.25342-1-avarab@gmail.com/\n2. https://lore.kernel.org/git/20210317182054.5986-1-avarab@gmail.com/\n\nÆvar Arnfjörð Bjarmason (19):\n fsck.c: refactor and rename common config callback\n fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n fsck.h: use \"enum object_type\" instead of \"int\"\n fsck.c: rename variables in fsck_set_msg_type() for less confusion\n fsck.c: remove (mostly) redundant append_msg_id() function\n fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"\n fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n fsck.h: re-order and re-assign \"enum fsck_msg_type\"\n fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n fsck.c: undefine temporary STR macro after use\n fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n fsck.c: pass along the fsck_msg_id in the fsck_error callback\n fsck.c: add an fsck_set_msg_type() API that takes enums\n fsck.c: move gitmodules_{found,done} into fsck_options\n fetch-pack: don't needlessly copy fsck_options\n fetch-pack: use file-scope static struct for fsck_options\n fetch-pack: use new fsck API to printing dangling submodules\n\n builtin/fsck.c | 14 ++-\n builtin/index-pack.c | 30 +-----\n builtin/mktag.c | 14 ++-\n builtin/unpack-objects.c | 3 +-\n fetch-pack.c | 31 ++----\n fsck.c | 207 +++++++++++++--------------------------\n fsck.h | 127 +++++++++++++++++++++---\n 7 files changed, 210 insertions(+), 216 deletions(-)\n\nRange-diff:\n 1: fe33015e0d9 = 1: 579af32ab3e fsck.c: refactor and rename common config callback\n 2: 72f2e53afac ! 2: b17c982293e fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n @@ Commit message\n fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}\n \n Refactor the definitions of FSCK_OPTIONS_{DEFAULT,STRICT} to use\n - designated initializers.\n -\n - While I'm at it add the \"object_names\" member to the\n - initialization. This was omitted in 7b35efd734e (fsck_walk():\n - optionally name objects on the go, 2016-07-17) when the field was\n - added.\n -\n - I'm using a new FSCK_OPTIONS_COMMON and FSCK_OPTIONS_COMMON_ERROR_FUNC\n - helper macros to define what FSCK_OPTIONS_{DEFAULT,STRICT} have in\n - common, and define the two in terms of those macro.\n -\n - The FSCK_OPTIONS_COMMON macro will be used in a subsequent commit to\n - define other variants of common fsck initialization that wants to use\n - a custom error function, but share the rest of the defaults.\n + designated initializers. This allows us to omit those fields that\n + aren't initialized to zero or NULL.\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n @@ fsck.h: struct fsck_options {\n \n -#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n -#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n -+#define FSCK_OPTIONS_COMMON \\\n -+\t.walk = NULL, \\\n -+\t.msg_type = NULL, \\\n ++#define FSCK_OPTIONS_DEFAULT { \\\n +\t.skiplist = OIDSET_INIT, \\\n -+\t.object_names = NULL,\n -+#define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n -+\tFSCK_OPTIONS_COMMON \\\n -+\t.error_func = fsck_error_function\n -+\n -+#define FSCK_OPTIONS_DEFAULT\t{ .strict = 0, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n -+#define FSCK_OPTIONS_STRICT\t{ .strict = 1, FSCK_OPTIONS_COMMON_ERROR_FUNC }\n ++\t.error_func = fsck_error_function \\\n ++}\n ++#define FSCK_OPTIONS_STRICT { \\\n ++\t.strict = 1, \\\n ++\t.error_func = fsck_error_function, \\\n ++}\n \n /* descend in all linked child objects\n * the return value is:\n 3: 237a2806865 = 3: a721c396c50 fsck.h: use \"enum object_type\" instead of \"int\"\n 4: 13b76c73dd7 = 4: fcdba2f8fe8 fsck.c: rename variables in fsck_set_msg_type() for less confusion\n 5: 4ae83403b73 ! 5: b07e8e026ac fsck.c: move definition of msg_id into append_msg_id()\n @@ Metadata\n Author: Ævar Arnfjörð Bjarmason \n \n ## Commit message ##\n - fsck.c: move definition of msg_id into append_msg_id()\n + fsck.c: remove (mostly) redundant append_msg_id() function\n \n - Refactor code added in 71ab8fa840f (fsck: report the ID of the\n - error/warning, 2015-06-22) to resolve the msg_id to a string in the\n - function that wants it, instead of doing it in report().\n + Remove the append_msg_id() function in favor of calling\n + prepare_msg_ids(). We already have code to compute the camel-cased\n + msg_id strings in msg_id_info, let's use it.\n +\n + When the append_msg_id() function was added in 71ab8fa840f (fsck:\n + report the ID of the error/warning, 2015-06-22) the prepare_msg_ids()\n + function didn't exist. When prepare_msg_ids() was added in\n + a46baac61eb (fsck: factor out msg_id_info[] lazy initialization code,\n + 2018-05-26) this code wasn't moved over to lazy initialization.\n +\n + This changes the behavior of the code to initialize all the messages\n + instead of just camel-casing the one we need on the fly. Since the\n + common case is that we're printing just one message this is mostly\n + redundant work.\n +\n + But that's OK in this case, reporting this fsck issue to the user\n + isn't performance-sensitive. If we were somehow doing so in a tight\n + loop (in a hopelessly broken repository?) this would help, since we'd\n + save ourselves from re-doing this work for identical messages, we\n + could just grab the prepared string from msg_id_info after the first\n + invocation.\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n @@ fsck.c: void fsck_set_msg_types(struct fsck_options *options, const char *values\n }\n \n -static void append_msg_id(struct strbuf *sb, const char *msg_id)\n -+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n +-{\n +-\tfor (;;) {\n +-\t\tchar c = *(msg_id)++;\n +-\n +-\t\tif (!c)\n +-\t\t\tbreak;\n +-\t\tif (c != '_')\n +-\t\t\tstrbuf_addch(sb, tolower(c));\n +-\t\telse {\n +-\t\t\tassert(*msg_id);\n +-\t\t\tstrbuf_addch(sb, *(msg_id)++);\n +-\t\t}\n +-\t}\n +-\n +-\tstrbuf_addstr(sb, \": \");\n +-}\n +-\n + static int object_on_skiplist(struct fsck_options *opts,\n + \t\t\t const struct object_id *oid)\n {\n -+\tconst char *msg_id = msg_id_info[id].id_string;\n - \tfor (;;) {\n - \t\tchar c = *(msg_id)++;\n - \n @@ fsck.c: static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n -\tappend_msg_id(&sb, msg_id_info[id].id_string);\n -+\tappend_msg_id(&sb, id);\n ++\tprepare_msg_ids();\n ++\tstrbuf_addf(&sb, \"%s: \", msg_id_info[id].camelcased);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n 6: 82107f1dac0 ! 6: 321b0c652de fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"\n @@ Commit message\n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## fsck.c ##\n -@@ fsck.c: void fsck_set_msg_types(struct fsck_options *options, const char *values)\n - \tfree(to_free);\n - }\n - \n --static void append_msg_id(struct strbuf *sb, enum fsck_msg_id id)\n -+static void append_msg_id(struct strbuf *sb, enum fsck_msg_id msg_id)\n - {\n --\tconst char *msg_id = msg_id_info[id].id_string;\n -+\tconst char *msg_id_str = msg_id_info[msg_id].id_string;\n - \tfor (;;) {\n --\t\tchar c = *(msg_id)++;\n -+\t\tchar c = *(msg_id_str)++;\n - \n - \t\tif (!c)\n - \t\t\tbreak;\n - \t\tif (c != '_')\n - \t\t\tstrbuf_addch(sb, tolower(c));\n - \t\telse {\n --\t\t\tassert(*msg_id);\n --\t\t\tstrbuf_addch(sb, *(msg_id)++);\n -+\t\t\tassert(*msg_id_str);\n -+\t\t\tstrbuf_addch(sb, *(msg_id_str)++);\n - \t\t}\n - \t}\n - \n @@ fsck.c: static int object_on_skiplist(struct fsck_options *opts,\n __attribute__((format (printf, 5, 6)))\n static int report(struct fsck_options *options,\n @@ fsck.c: static int object_on_skiplist(struct fsck_options *opts,\n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n @@ fsck.c: static int report(struct fsck_options *options,\n - \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n --\tappend_msg_id(&sb, id);\n -+\tappend_msg_id(&sb, msg_id);\n + \tprepare_msg_ids();\n +-\tstrbuf_addf(&sb, \"%s: \", msg_id_info[id].camelcased);\n ++\tstrbuf_addf(&sb, \"%s: \", msg_id_info[msg_id].camelcased);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n 7: 796096bf73e = 7: 948689ad5c8 fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"\n 8: 3664abb23de = 8: 8ea468bf4d8 fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum\n 9: 81e6d7ab450 ! 9: 9316b35cd3b fsck.h: re-order and re-assign \"enum fsck_msg_type\"\n @@ Commit message\n defined as \"2\".\n \n I'm confident that nothing relies on these values, we always compare\n - them explicitly. Let's not omit \"0\" so it won't be assumed that we're\n - using these as a boolean somewhere.\n + them for equality. Let's not omit \"0\" so it won't be assumed that\n + we're using these as a boolean somewhere.\n \n This also allows us to re-structure the fields to mark which are\n \"private\" v.s. \"public\". See the preceding commit for a rationale for\n10: 5c2e8e7b842 = 10: d7f1c5d37de fsck.c: call parse_msg_type() early in fsck_set_msg_type()\n11: 7ffbf9af3fa = 11: ae5efd745cf fsck.c: undefine temporary STR macro after use\n12: 12ff0f75ebf = 12: 96995244806 fsck.c: give \"FOREACH_MSG_ID\" a more specific name\n13: 0c49dd5164f = 13: 1b42aea3a64 fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h\n14: 900263f503a = 14: 563e6a0e5e6 fsck.c: pass along the fsck_msg_id in the fsck_error callback\n15: 5f270e88a0a = 15: 5e504f25c51 fsck.c: add an fsck_set_msg_type() API that takes enums\n16: 539d0197129 ! 16: 611631dd779 fsck.c: move gitmodules_{found,done} into fsck_options\n @@ Commit message\n gitmodules_found attribute of \"fsck_options\" we need this intermediate\n step first.\n \n + An earlier version of this patch removed the small amount of\n + duplication we now have between FSCK_OPTIONS_{DEFAULT,STRICT} with a\n + FSCK_OPTIONS_COMMON macro. I don't think such de-duplication is worth\n + it for this amount of copy/pasting.\n +\n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## fetch-pack.c ##\n @@ fsck.h: struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n -@@ fsck.h: struct fsck_options {\n - \t.walk = NULL, \\\n - \t.msg_type = NULL, \\\n + #define FSCK_OPTIONS_DEFAULT { \\\n \t.skiplist = OIDSET_INIT, \\\n +\t.gitmodules_found = OIDSET_INIT, \\\n +\t.gitmodules_done = OIDSET_INIT, \\\n - \t.object_names = NULL,\n - #define FSCK_OPTIONS_COMMON_ERROR_FUNC \\\n - \tFSCK_OPTIONS_COMMON \\\n + \t.error_func = fsck_error_function \\\n + }\n + #define FSCK_OPTIONS_STRICT { \\\n + \t.strict = 1, \\\n ++\t.gitmodules_found = OIDSET_INIT, \\\n ++\t.gitmodules_done = OIDSET_INIT, \\\n + \t.error_func = fsck_error_function, \\\n + }\n + \n @@ fsck.h: int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n17: 1acf7442365 = 17: 03d512c8448 fetch-pack: don't needlessly copy fsck_options\n18: b47c3d5ac6f = 18: 581c87c63c6 fetch-pack: use file-scope static struct for fsck_options\n19: f05fa5c3ec9 ! 19: 6a38cade8c3 fetch-pack: use new fsck API to printing dangling submodules\n @@ fetch-pack.c: static int server_supports_filtering;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n \n -@@ fetch-pack.c: static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n +@@ fetch-pack.c: static int cmp_ref_by_name(const void *a_, const void *b_)\n + \treturn strcmp(a->name, b->name);\n + }\n \n - \toidset_iter_init(gitmodules_oids, &iter);\n - \twhile ((oid = oidset_iter_next(&iter)))\n +-static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n +-{\n +-\tstruct oidset_iter iter;\n +-\tconst struct object_id *oid;\n +-\n +-\tif (!oidset_size(gitmodules_oids))\n +-\t\treturn;\n +-\n +-\toidset_iter_init(gitmodules_oids, &iter);\n +-\twhile ((oid = oidset_iter_next(&iter)))\n -\t\tregister_found_gitmodules(&fsck_options, oid);\n -+\t\toidset_insert(&fsck_options.gitmodules_found, oid);\n - \tif (fsck_finish(&fsck_options))\n - \t\tdie(\"fsck failed\");\n - }\n +-\tif (fsck_finish(&fsck_options))\n +-\t\tdie(\"fsck failed\");\n +-}\n +-\n + static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n + \t\t\t\t int fd[2],\n + \t\t\t\t const struct ref *orig_ref,\n +@@ fetch-pack.c: static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n + \tint agent_len;\n + \tstruct fetch_negotiator negotiator_alloc;\n + \tstruct fetch_negotiator *negotiator;\n +-\tstruct oidset gitmodules_oids = OIDSET_INIT;\n + \n + \tnegotiator = &negotiator_alloc;\n + \tfetch_negotiator_init(r, negotiator);\n +@@ fetch-pack.c: static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n + \telse\n + \t\talternate_shallow_file = NULL;\n + \tif (get_pack(args, fd, pack_lockfiles, NULL, sought, nr_sought,\n +-\t\t &gitmodules_oids))\n ++\t\t &fsck_options.gitmodules_found))\n + \t\tdie(_(\"git fetch-pack: fetch failed.\"));\n +-\tfsck_gitmodules_oids(&gitmodules_oids);\n ++\tif (fsck_finish(&fsck_options))\n ++\t\tdie(\"fsck failed\");\n + \n + all_done:\n + \tif (negotiator)\n +@@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n + \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n + \tint i;\n + \tstruct strvec index_pack_args = STRVEC_INIT;\n +-\tstruct oidset gitmodules_oids = OIDSET_INIT;\n + \n + \tnegotiator = &negotiator_alloc;\n + \tfetch_negotiator_init(r, negotiator);\n +@@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n + \t\t\tprocess_section_header(&reader, \"packfile\", 0);\n + \t\t\tif (get_pack(args, fd, pack_lockfiles,\n + \t\t\t\t packfile_uris.nr ? &index_pack_args : NULL,\n +-\t\t\t\t sought, nr_sought, &gitmodules_oids))\n ++\t\t\t\t sought, nr_sought, &fsck_options.gitmodules_found))\n + \t\t\t\tdie(_(\"git fetch-pack: fetch failed.\"));\n + \t\t\tdo_check_stateless_delimiter(args, &reader);\n + \n +@@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n + \n + \t\tpackname[the_hash_algo->hexsz] = '\\0';\n + \n +-\t\tparse_gitmodules_oids(cmd.out, &gitmodules_oids);\n ++\t\tparse_gitmodules_oids(cmd.out, &fsck_options.gitmodules_found);\n + \n + \t\tclose(cmd.out);\n + \n +@@ fetch-pack.c: static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n + \tstring_list_clear(&packfile_uris, 0);\n + \tstrvec_clear(&index_pack_args);\n + \n +-\tfsck_gitmodules_oids(&gitmodules_oids);\n ++\tif (fsck_finish(&fsck_options))\n ++\t\tdie(\"fsck failed\");\n + \n + \tif (negotiator)\n + \t\tnegotiator->release(negotiator);\n \n ## fsck.c ##\n @@ fsck.c: int fsck_error_function(struct fsck_options *o,\n @@ fsck.c: int git_fsck_config(const char *var, const char *value, void *cb)\n +}\n \n ## fsck.h ##\n +@@ fsck.h: int fsck_error_function(struct fsck_options *o,\n + \t\t\tconst struct object_id *oid, enum object_type object_type,\n + \t\t\tenum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n + \t\t\tconst char *message);\n ++int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n ++\t\t\t\t\t const struct object_id *oid,\n ++\t\t\t\t\t enum object_type object_type,\n ++\t\t\t\t\t enum fsck_msg_type msg_type,\n ++\t\t\t\t\t enum fsck_msg_id msg_id,\n ++\t\t\t\t\t const char *message);\n + \n + struct fsck_options {\n + \tfsck_walk_func walk;\n +@@ fsck.h: struct fsck_options {\n + \t.gitmodules_done = OIDSET_INIT, \\\n + \t.error_func = fsck_error_function, \\\n + }\n ++#define FSCK_OPTIONS_MISSING_GITMODULES { \\\n ++\t.strict = 1, \\\n ++\t.gitmodules_found = OIDSET_INIT, \\\n ++\t.gitmodules_done = OIDSET_INIT, \\\n ++\t.error_func = fsck_error_cb_print_missing_gitmodules, \\\n ++}\n + \n + /* descend in all linked child objects\n + * the return value is:\n @@ fsck.h: int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n @@ fsck.h: int fsck_walk(struct object *obj, void *data, struct fsck_options *optio\n /*\n * fsck a tag, and pass info about it back to the caller. This is\n * exposed fsck_object() internals for git-mktag(1).\n -@@ fsck.h: const char *fsck_describe_object(struct fsck_options *options,\n - */\n - int git_fsck_config(const char *var, const char *value, void *cb);\n - \n -+/*\n -+ * Custom error callbacks that are used in more than one place.\n -+ */\n -+#define FSCK_OPTIONS_MISSING_GITMODULES { \\\n -+\t.strict = 1, \\\n -+\t.error_func = fsck_error_cb_print_missing_gitmodules, \\\n -+\tFSCK_OPTIONS_COMMON \\\n -+}\n -+int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n -+\t\t\t\t\t const struct object_id *oid,\n -+\t\t\t\t\t enum object_type object_type,\n -+\t\t\t\t\t enum fsck_msg_type msg_type,\n -+\t\t\t\t\t enum fsck_msg_id msg_id,\n -+\t\t\t\t\t const char *message);\n -+\n - #endif\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420389","messageId":"patch-02.20-b17c982293e-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 02/19] fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:34Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor the definitions of FSCK_OPTIONS_{DEFAULT,STRICT} to use\ndesignated initializers. This allows us to omit those fields that\naren't initialized to zero or NULL.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 10 ++++++++--\n 1 file changed, 8 insertions(+), 2 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex f70d11c5594..73e8b9f3e4e 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -43,8 +43,14 @@ struct fsck_options {\n \tkh_oid_map_t *object_names;\n };\n \n-#define FSCK_OPTIONS_DEFAULT { NULL, fsck_error_function, 0, NULL, OIDSET_INIT }\n-#define FSCK_OPTIONS_STRICT { NULL, fsck_error_function, 1, NULL, OIDSET_INIT }\n+#define FSCK_OPTIONS_DEFAULT { \\\n+\t.skiplist = OIDSET_INIT, \\\n+\t.error_func = fsck_error_function \\\n+}\n+#define FSCK_OPTIONS_STRICT { \\\n+\t.strict = 1, \\\n+\t.error_func = fsck_error_function, \\\n+}\n \n /* descend in all linked child objects\n * the return value is:\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420390","messageId":"patch-01.20-579af32ab3e-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 01/19] fsck.c: refactor and rename common config callback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:33Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor code I recently changed in 1f3299fda9 (fsck: make\nfsck_config() re-usable, 2021-01-05) so that I could use fsck's config\ncallback in mktag in 1f3299fda9 (fsck: make fsck_config() re-usable,\n2021-01-05).\n\nI don't know what I was thinking in structuring the code this way, but\nit clearly makes no sense to have an fsck_config_internal() at all\njust so it can get a fsck_options when git_config() already supports\npassing along some void* data.\n\nLet's just make use of that instead, which gets us rid of the two\nwrapper functions, and brings fsck's common config callback in line\nwith other such reusable config callbacks.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 7 +------\n builtin/mktag.c | 7 +------\n fsck.c | 4 ++--\n fsck.h | 3 +--\n 4 files changed, 5 insertions(+), 16 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 821e7798c70..a56a2d0513a 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -71,11 +71,6 @@ static const char *printable_type(const struct object_id *oid,\n \treturn ret;\n }\n \n-static int fsck_config(const char *var, const char *value, void *cb)\n-{\n-\treturn fsck_config_internal(var, value, cb, &fsck_obj_options);\n-}\n-\n static int objerror(struct object *obj, const char *err)\n {\n \terrors_found |= ERROR_OBJECT;\n@@ -803,7 +798,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \tif (name_objects)\n \t\tfsck_enable_object_names(&fsck_walk_options);\n \n-\tgit_config(fsck_config, NULL);\n+\tgit_config(git_fsck_config, &fsck_obj_options);\n \n \tif (connectivity_only) {\n \t\tfor_each_loose_object(mark_loose_for_connectivity, NULL, 0);\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 41a399a69e4..23c4b8763fa 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -14,11 +14,6 @@ static int option_strict = 1;\n \n static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n \n-static int mktag_config(const char *var, const char *value, void *cb)\n-{\n-\treturn fsck_config_internal(var, value, cb, &fsck_options);\n-}\n-\n static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n@@ -93,7 +88,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)\n \tfsck_options.error_func = mktag_fsck_error_func;\n \tfsck_set_msg_type(&fsck_options, \"extraheaderentry\", \"warn\");\n \t/* config might set fsck.extraHeaderEntry=* again */\n-\tgit_config(mktag_config, NULL);\n+\tgit_config(git_fsck_config, &fsck_options);\n \tif (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,\n \t\t\t\t&tagged_oid, &tagged_type))\n \t\tdie(_(\"tag on stdin did not pass our strict fsck check\"));\ndiff --git a/fsck.c b/fsck.c\nindex e3030f3b358..5dfb99665ae 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1323,9 +1323,9 @@ int fsck_finish(struct fsck_options *options)\n \treturn ret;\n }\n \n-int fsck_config_internal(const char *var, const char *value, void *cb,\n-\t\t\t struct fsck_options *options)\n+int git_fsck_config(const char *var, const char *value, void *cb)\n {\n+\tstruct fsck_options *options = cb;\n \tif (strcmp(var, \"fsck.skiplist\") == 0) {\n \t\tconst char *path;\n \t\tstruct strbuf sb = STRBUF_INIT;\ndiff --git a/fsck.h b/fsck.h\nindex 733378f1260..f70d11c5594 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -109,7 +109,6 @@ const char *fsck_describe_object(struct fsck_options *options,\n * git_config() callback for use by fsck-y tools that want to support\n * fsck. fsck.skipList etc.\n */\n-int fsck_config_internal(const char *var, const char *value, void *cb,\n-\t\t\t struct fsck_options *options);\n+int git_fsck_config(const char *var, const char *value, void *cb);\n \n #endif\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420391","messageId":"patch-04.20-fcdba2f8fe8-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 04/19] fsck.c: rename variables in fsck_set_msg_type() for less confusion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:36Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename variables in a function added in 0282f4dced0 (fsck: offer a\nfunction to demote fsck errors to warnings, 2015-06-22).\n\nIt was needlessly confusing that it took a \"msg_type\" argument, but\nthen later declared another \"msg_type\" of a different type.\n\nLet's rename that to \"severity\", and rename \"id\" to \"msg_id\" and\n\"msg_id\" to \"msg_id_str\" etc. This will make a follow-up change\nsmaller.\n\nWhile I'm at it properly indent the fsck_set_msg_type() argument list.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 24 ++++++++++++------------\n fsck.h | 2 +-\n 2 files changed, 13 insertions(+), 13 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 5dfb99665ae..7cc722a25cd 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -203,27 +203,27 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n }\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type)\n+\t\t const char *msg_id_str, const char *msg_type_str)\n {\n-\tint id = parse_msg_id(msg_id), type;\n+\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n \n-\tif (id < 0)\n-\t\tdie(\"Unhandled message id: %s\", msg_id);\n-\ttype = parse_msg_type(msg_type);\n+\tif (msg_id < 0)\n+\t\tdie(\"Unhandled message id: %s\", msg_id_str);\n+\tmsg_type = parse_msg_type(msg_type_str);\n \n-\tif (type != FSCK_ERROR && msg_id_info[id].msg_type == FSCK_FATAL)\n-\t\tdie(\"Cannot demote %s to %s\", msg_id, msg_type);\n+\tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n+\t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *msg_type;\n-\t\tALLOC_ARRAY(msg_type, FSCK_MSG_MAX);\n+\t\tint *severity;\n+\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tmsg_type[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = msg_type;\n+\t\t\tseverity[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = severity;\n \t}\n \n-\toptions->msg_type[id] = type;\n+\toptions->msg_type[msg_id] = msg_type;\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\ndiff --git a/fsck.h b/fsck.h\nindex f20f1259e84..30a3acabc50 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -11,7 +11,7 @@ struct fsck_options;\n struct object;\n \n void fsck_set_msg_type(struct fsck_options *options,\n-\t\tconst char *msg_id, const char *msg_type);\n+\t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n int is_valid_msg_type(const char *msg_id, const char *msg_type);\n \n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420392","messageId":"patch-06.20-321b0c652de-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 06/19] fsck.c: rename remaining fsck_msg_id \"id\" to \"msg_id\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:38Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the remaining variables of type fsck_msg_id from \"id\" to\n\"msg_id\". This change is relatively small, and is worth the churn for\na later change where we have different id's in the \"report\" function.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 25c697fa6a2..a0463ea22cc 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -273,11 +273,11 @@ static int object_on_skiplist(struct fsck_options *opts,\n __attribute__((format (printf, 5, 6)))\n static int report(struct fsck_options *options,\n \t\t const struct object_id *oid, enum object_type object_type,\n-\t\t enum fsck_msg_id id, const char *fmt, ...)\n+\t\t enum fsck_msg_id msg_id, const char *fmt, ...)\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(id, options), result;\n+\tint msg_type = fsck_msg_type(msg_id, options), result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -291,7 +291,7 @@ static int report(struct fsck_options *options,\n \t\tmsg_type = FSCK_WARN;\n \n \tprepare_msg_ids();\n-\tstrbuf_addf(&sb, \"%s: \", msg_id_info[id].camelcased);\n+\tstrbuf_addf(&sb, \"%s: \", msg_id_info[msg_id].camelcased);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420393","messageId":"patch-05.20-b07e8e026ac-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 05/19] fsck.c: remove (mostly) redundant append_msg_id() function","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:37Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove the append_msg_id() function in favor of calling\nprepare_msg_ids(). We already have code to compute the camel-cased\nmsg_id strings in msg_id_info, let's use it.\n\nWhen the append_msg_id() function was added in 71ab8fa840f (fsck:\nreport the ID of the error/warning, 2015-06-22) the prepare_msg_ids()\nfunction didn't exist. When prepare_msg_ids() was added in\na46baac61eb (fsck: factor out msg_id_info[] lazy initialization code,\n2018-05-26) this code wasn't moved over to lazy initialization.\n\nThis changes the behavior of the code to initialize all the messages\ninstead of just camel-casing the one we need on the fly. Since the\ncommon case is that we're printing just one message this is mostly\nredundant work.\n\nBut that's OK in this case, reporting this fsck issue to the user\nisn't performance-sensitive. If we were somehow doing so in a tight\nloop (in a hopelessly broken repository?) this would help, since we'd\nsave ourselves from re-doing this work for identical messages, we\ncould just grab the prepared string from msg_id_info after the first\ninvocation.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 21 ++-------------------\n 1 file changed, 2 insertions(+), 19 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 7cc722a25cd..25c697fa6a2 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -264,24 +264,6 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \tfree(to_free);\n }\n \n-static void append_msg_id(struct strbuf *sb, const char *msg_id)\n-{\n-\tfor (;;) {\n-\t\tchar c = *(msg_id)++;\n-\n-\t\tif (!c)\n-\t\t\tbreak;\n-\t\tif (c != '_')\n-\t\t\tstrbuf_addch(sb, tolower(c));\n-\t\telse {\n-\t\t\tassert(*msg_id);\n-\t\t\tstrbuf_addch(sb, *(msg_id)++);\n-\t\t}\n-\t}\n-\n-\tstrbuf_addstr(sb, \": \");\n-}\n-\n static int object_on_skiplist(struct fsck_options *opts,\n \t\t\t const struct object_id *oid)\n {\n@@ -308,7 +290,8 @@ static int report(struct fsck_options *options,\n \telse if (msg_type == FSCK_INFO)\n \t\tmsg_type = FSCK_WARN;\n \n-\tappend_msg_id(&sb, msg_id_info[id].id_string);\n+\tprepare_msg_ids();\n+\tstrbuf_addf(&sb, \"%s: \", msg_id_info[id].camelcased);\n \n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420394","messageId":"patch-03.20-a721c396c50-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 03/19] fsck.h: use \"enum object_type\" instead of \"int\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:35Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_walk_func to use an \"enum object_type\" instead of an\n\"int\" type. The types are compatible, and ever since this was added in\n355885d5315 (add generic, type aware object chain walker, 2008-02-25)\nwe've used entries from object_type (OBJ_BLOB etc.).\n\nSo this doesn't really change anything as far as the generated code is\nconcerned, it just gives the compiler more information and makes this\neasier to read.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 3 ++-\n builtin/index-pack.c | 3 ++-\n builtin/unpack-objects.c | 3 ++-\n fsck.h | 3 ++-\n 4 files changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex a56a2d0513a..ed5f2af6b5c 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -192,7 +192,8 @@ static int traverse_reachable(void)\n \treturn !!result;\n }\n \n-static int mark_used(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_used(struct object *obj, enum object_type object_type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn 1;\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 21899687e2c..f6e1178df90 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -212,7 +212,8 @@ static void cleanup_thread(void)\n \tfree(thread_data);\n }\n \n-static int mark_link(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int mark_link(struct object *obj, enum object_type type,\n+\t\t void *data, struct fsck_options *options)\n {\n \tif (!obj)\n \t\treturn -1;\ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex a4ba2ebac69..4a70b17f8fb 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -187,7 +187,8 @@ static void write_cached_object(struct object *obj, struct obj_buffer *obj_buf)\n * that have reachability requirements and calls this function.\n * Verify its reachability and validity recursively and write it out.\n */\n-static int check_object(struct object *obj, int type, void *data, struct fsck_options *options)\n+static int check_object(struct object *obj, enum object_type type,\n+\t\t\tvoid *data, struct fsck_options *options)\n {\n \tstruct obj_buffer *obj_buf;\n \ndiff --git a/fsck.h b/fsck.h\nindex 73e8b9f3e4e..f20f1259e84 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -23,7 +23,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type);\n * <0\terror signaled and abort\n * >0\terror signaled and do not abort\n */\n-typedef int (*fsck_walk_func)(struct object *obj, int type, void *data, struct fsck_options *options);\n+typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n+\t\t\t void *data, struct fsck_options *options);\n \n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420395","messageId":"patch-08.20-8ea468bf4d8-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 08/19] fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:40Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} defines into a new\nfsck_msg_type enum.\n\nThese defines were originally introduced in:\n\n - ba002f3b28a (builtin-fsck: move common object checking code to\n fsck.c, 2008-02-25)\n - f50c4407305 (fsck: disallow demoting grave fsck errors to warnings,\n 2015-06-22)\n - efaba7cc77f (fsck: optionally ignore specific fsck issues\n completely, 2015-06-22)\n - f27d05b1704 (fsck: allow upgrading fsck warnings to errors,\n 2015-06-22)\n\nThe reason these were defined in two different places is because we\nuse FSCK_{IGNORE,INFO,FATAL} only in fsck.c, but FSCK_{ERROR,WARN} are\nused by external callbacks.\n\nUntangling that would take some more work, since we expose the new\n\"enum fsck_msg_type\" to both. Similar to \"enum object_type\" it's not\nworth structuring the API in such a way that only those who need\nFSCK_{ERROR,WARN} pass around a different type.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 2 +-\n builtin/index-pack.c | 3 ++-\n builtin/mktag.c | 3 ++-\n fsck.c | 21 ++++++++++-----------\n fsck.h | 16 ++++++++++------\n 5 files changed, 25 insertions(+), 20 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex ed5f2af6b5c..17940a4e24a 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -84,7 +84,7 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t int msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type, const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex f6e1178df90..8338b832b63 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1716,7 +1716,8 @@ static void show_pack_info(int stat_only)\n static int print_dangling_gitmodules(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \t/*\n \t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 23c4b8763fa..052a510ad7f 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -17,7 +17,8 @@ static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n-\t\t\t\t int msg_type, const char *message)\n+\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/fsck.c b/fsck.c\nindex 8614ee2c2a0..c5a81e4ff05 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,9 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FSCK_FATAL -1\n-#define FSCK_INFO -2\n-\n #define FOREACH_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n@@ -97,7 +94,7 @@ static struct {\n \tconst char *id_string;\n \tconst char *downcased;\n \tconst char *camelcased;\n-\tint msg_type;\n+\tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n \tFOREACH_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n@@ -164,13 +161,13 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n \t\tlist_config_item(list, prefix, msg_id_info[i].camelcased);\n }\n \n-static int fsck_msg_type(enum fsck_msg_id msg_id,\n+static enum fsck_msg_type fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n \tif (!options->msg_type) {\n-\t\tint msg_type = msg_id_info[msg_id].msg_type;\n+\t\tenum fsck_msg_type msg_type = msg_id_info[msg_id].msg_type;\n \n \t\tif (options->strict && msg_type == FSCK_WARN)\n \t\t\tmsg_type = FSCK_ERROR;\n@@ -180,7 +177,7 @@ static int fsck_msg_type(enum fsck_msg_id msg_id,\n \treturn options->msg_type[msg_id];\n }\n \n-static int parse_msg_type(const char *str)\n+static enum fsck_msg_type parse_msg_type(const char *str)\n {\n \tif (!strcmp(str, \"error\"))\n \t\treturn FSCK_ERROR;\n@@ -203,7 +200,8 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id_str, const char *msg_type_str)\n {\n-\tint msg_id = parse_msg_id(msg_id_str), msg_type;\n+\tint msg_id = parse_msg_id(msg_id_str);\n+\tenum fsck_msg_type msg_type;\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n@@ -214,7 +212,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \n \tif (!options->msg_type) {\n \t\tint i;\n-\t\tint *severity;\n+\t\tenum fsck_msg_type *severity;\n \t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n \t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n \t\t\tseverity[i] = fsck_msg_type(i, options);\n@@ -275,7 +273,8 @@ static int report(struct fsck_options *options,\n {\n \tva_list ap;\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint msg_type = fsck_msg_type(msg_id, options), result;\n+\tenum fsck_msg_type msg_type = fsck_msg_type(msg_id, options);\n+\tint result;\n \n \tif (msg_type == FSCK_IGNORE)\n \t\treturn 0;\n@@ -1247,7 +1246,7 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tint msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type, const char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex 30a3acabc50..baf37620760 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -3,9 +3,13 @@\n \n #include \"oidset.h\"\n \n-#define FSCK_ERROR 1\n-#define FSCK_WARN 2\n-#define FSCK_IGNORE 3\n+enum fsck_msg_type {\n+\tFSCK_INFO = -2,\n+\tFSCK_FATAL = -1,\n+\tFSCK_ERROR = 1,\n+\tFSCK_WARN,\n+\tFSCK_IGNORE\n+};\n \n struct fsck_options;\n struct object;\n@@ -29,17 +33,17 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t int msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tint msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, const char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n \tfsck_error error_func;\n \tunsigned strict:1;\n-\tint *msg_type;\n+\tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n \tkh_oid_map_t *object_names;\n };\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420396","messageId":"patch-09.20-9316b35cd3b-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 09/19] fsck.h: re-order and re-assign \"enum fsck_msg_type\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:41Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the values in the \"enum fsck_msg_type\" from being manually\nassigned to using default C enum values.\n\nThis means we end up with a FSCK_IGNORE=0, which was previously\ndefined as \"2\".\n\nI'm confident that nothing relies on these values, we always compare\nthem for equality. Let's not omit \"0\" so it won't be assumed that\nwe're using these as a boolean somewhere.\n\nThis also allows us to re-structure the fields to mark which are\n\"private\" v.s. \"public\". See the preceding commit for a rationale for\nnot simply splitting these into two enums, namely that this is used\nfor both the private and public fsck API.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.h | 10 ++++++----\n 1 file changed, 6 insertions(+), 4 deletions(-)\n\ndiff --git a/fsck.h b/fsck.h\nindex baf37620760..a7e092d3fb4 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -4,11 +4,13 @@\n #include \"oidset.h\"\n \n enum fsck_msg_type {\n-\tFSCK_INFO = -2,\n-\tFSCK_FATAL = -1,\n-\tFSCK_ERROR = 1,\n+\t/* for internal use only */\n+\tFSCK_IGNORE,\n+\tFSCK_INFO,\n+\tFSCK_FATAL,\n+\t/* \"public\", fed to e.g. error_func callbacks */\n+\tFSCK_ERROR,\n \tFSCK_WARN,\n-\tFSCK_IGNORE\n };\n \n struct fsck_options;\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420397","messageId":"patch-07.20-948689ad5c8-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 07/19] fsck.c: refactor fsck_msg_type() to limit scope of \"int msg_type\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:39Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor \"if options->msg_type\" and other code added in\n0282f4dced0 (fsck: offer a function to demote fsck errors to warnings,\n2015-06-22) to reduce the scope of the \"int msg_type\" variable.\n\nThis is in preparation for changing its type in a subsequent commit,\nonly using it in the \"!options->msg_type\" scope makes that change\n\nThis also brings the code in line with the fsck_set_msg_type()\nfunction (also added in 0282f4dced0), which does a similar check for\n\"!options->msg_type\". Another minor benefit is getting rid of the\nstyle violation of not having braces for the body of the \"if\".\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 12 +++++-------\n 1 file changed, 5 insertions(+), 7 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex a0463ea22cc..8614ee2c2a0 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -167,19 +167,17 @@ void list_config_fsck_msg_ids(struct string_list *list, const char *prefix)\n static int fsck_msg_type(enum fsck_msg_id msg_id,\n \tstruct fsck_options *options)\n {\n-\tint msg_type;\n-\n \tassert(msg_id >= 0 && msg_id < FSCK_MSG_MAX);\n \n-\tif (options->msg_type)\n-\t\tmsg_type = options->msg_type[msg_id];\n-\telse {\n-\t\tmsg_type = msg_id_info[msg_id].msg_type;\n+\tif (!options->msg_type) {\n+\t\tint msg_type = msg_id_info[msg_id].msg_type;\n+\n \t\tif (options->strict && msg_type == FSCK_WARN)\n \t\t\tmsg_type = FSCK_ERROR;\n+\t\treturn msg_type;\n \t}\n \n-\treturn msg_type;\n+\treturn options->msg_type[msg_id];\n }\n \n static int parse_msg_type(const char *str)\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420398","messageId":"patch-11.20-ae5efd745cf-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 11/19] fsck.c: undefine temporary STR macro after use","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:43Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In f417eed8cde (fsck: provide a function to parse fsck message IDs,\n2015-06-22) the \"STR\" macro was introduced, but that short macro name\nwas not undefined after use as was done earlier in the same series for\nthe MSG_ID macro in c99ba492f1c (fsck: introduce identifiers for fsck\nmessages, 2015-06-22).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/fsck.c b/fsck.c\nindex 80365e62842..1b12e824ef6 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -100,6 +100,7 @@ static struct {\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n+#undef STR\n \n static void prepare_msg_ids(void)\n {\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420399","messageId":"patch-10.20-d7f1c5d37de-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 10/19] fsck.c: call parse_msg_type() early in fsck_set_msg_type()","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:42Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"There's no reason to defer the calling of parse_msg_type() until after\nwe've checked if the \"id < 0\". This is not a hot codepath, and\nparse_msg_type() itself may die on invalid input.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex c5a81e4ff05..80365e62842 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -201,11 +201,10 @@ void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id_str, const char *msg_type_str)\n {\n \tint msg_id = parse_msg_id(msg_id_str);\n-\tenum fsck_msg_type msg_type;\n+\tenum fsck_msg_type msg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_id < 0)\n \t\tdie(\"Unhandled message id: %s\", msg_id_str);\n-\tmsg_type = parse_msg_type(msg_type_str);\n \n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420403","messageId":"patch-14.20-563e6a0e5e6-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 14/19] fsck.c: pass along the fsck_msg_id in the fsck_error callback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:46Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the fsck_error callback to also pass along the\nfsck_msg_id. Before this change the only way to get the message id was\nto parse it back out of the \"message\".\n\nLet's pass it down explicitly for the benefit of callers that might\nwant to use it, as discussed in [1].\n\nPassing the msg_type is now redundant, as you can always get it back\nfrom the msg_id, but I'm not changing that convention. It's really\ncommon to need the msg_type, and the report() function itself (which\ncalls \"fsck_error\") needs to call fsck_msg_type() to discover\nit. Let's not needlessly re-do that work in the user callback.\n\n1. https://lore.kernel.org/git/87blcja2ha.fsf@evledraar.gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/fsck.c | 4 +++-\n builtin/index-pack.c | 3 ++-\n builtin/mktag.c | 1 +\n fsck.c | 6 ++++--\n fsck.h | 6 ++++--\n 5 files changed, 14 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 17940a4e24a..70ff95837ae 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -84,7 +84,9 @@ static int objerror(struct object *obj, const char *err)\n static int fsck_error_func(struct fsck_options *o,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message)\n+\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t const char *message)\n {\n \tswitch (msg_type) {\n \tcase FSCK_WARN:\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 8338b832b63..2f93957fb5e 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1717,6 +1717,7 @@ static int print_dangling_gitmodules(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n \t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n \t\t\t\t const char *message)\n {\n \t/*\n@@ -1727,7 +1728,7 @@ static int print_dangling_gitmodules(struct fsck_options *o,\n \t\tprintf(\"%s\\n\", oid_to_hex(oid));\n \t\treturn 0;\n \t}\n-\treturn fsck_error_function(o, oid, object_type, msg_type, message);\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n }\n \n int cmd_index_pack(int argc, const char **argv, const char *prefix)\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 052a510ad7f..96e63bc772a 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -18,6 +18,7 @@ static int mktag_fsck_error_func(struct fsck_options *o,\n \t\t\t\t const struct object_id *oid,\n \t\t\t\t enum object_type object_type,\n \t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t enum fsck_msg_id msg_id,\n \t\t\t\t const char *message)\n {\n \tswitch (msg_type) {\ndiff --git a/fsck.c b/fsck.c\nindex 150fe467e43..23a77fe2e0f 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -227,7 +227,7 @@ static int report(struct fsck_options *options,\n \tva_start(ap, fmt);\n \tstrbuf_vaddf(&sb, fmt, ap);\n \tresult = options->error_func(options, oid, object_type,\n-\t\t\t\t msg_type, sb.buf);\n+\t\t\t\t msg_type, msg_id, sb.buf);\n \tstrbuf_release(&sb);\n \tva_end(ap);\n \n@@ -1180,7 +1180,9 @@ int fsck_object(struct object *obj, void *data, unsigned long size,\n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid,\n \t\t\tenum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message)\n+\t\t\tenum fsck_msg_type msg_type,\n+\t\t\tenum fsck_msg_id msg_id,\n+\t\t\tconst char *message)\n {\n \tif (msg_type == FSCK_WARN) {\n \t\twarning(\"object %s: %s\", fsck_describe_object(o, oid), message);\ndiff --git a/fsck.h b/fsck.h\nindex 66c4a71139a..fa2d4955ab3 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -101,11 +101,13 @@ typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type,\n /* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */\n typedef int (*fsck_error)(struct fsck_options *o,\n \t\t\t const struct object_id *oid, enum object_type object_type,\n-\t\t\t enum fsck_msg_type msg_type, const char *message);\n+\t\t\t enum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\t const char *message);\n \n int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n-\t\t\tenum fsck_msg_type msg_type, const char *message);\n+\t\t\tenum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n+\t\t\tconst char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420405","messageId":"patch-12.20-96995244806-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 12/19] fsck.c: give \"FOREACH_MSG_ID\" a more specific name","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:44Z","receivedAt":"2021-03-28T13:17:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the FOREACH_MSG_ID macro to FOREACH_FSCK_MSG_ID in preparation\nfor moving it over to fsck.h. It's good convention to name macros\nin *.h files in such a way as to clearly not clash with any other\nnames in other files.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 1b12e824ef6..31c9088e3f7 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,7 +22,7 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_MSG_ID(FUNC) \\\n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n \t/* fatal errors */ \\\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n \tFUNC(UNTERMINATED_HEADER, FATAL) \\\n@@ -83,7 +83,7 @@ static struct oidset gitmodules_done = OIDSET_INIT;\n \n #define MSG_ID(id, msg_type) FSCK_MSG_##id,\n enum fsck_msg_id {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \tFSCK_MSG_MAX\n };\n #undef MSG_ID\n@@ -96,7 +96,7 @@ static struct {\n \tconst char *camelcased;\n \tenum fsck_msg_type msg_type;\n } msg_id_info[FSCK_MSG_MAX + 1] = {\n-\tFOREACH_MSG_ID(MSG_ID)\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n \t{ NULL, NULL, NULL, -1 }\n };\n #undef MSG_ID\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420400","messageId":"patch-15.20-5e504f25c51-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 15/19] fsck.c: add an fsck_set_msg_type() API that takes enums","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:47Z","receivedAt":"2021-03-28T13:17:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code I added in acf9de4c94e (mktag: use fsck instead of custom\nverify_tag(), 2021-01-05) to make use of a new API function that takes\nthe fsck_msg_{id,type} types, instead of arbitrary strings that\nwe'll (hopefully) parse into those types.\n\nAt the time that the fsck_set_msg_type() API was introduced in\n0282f4dced0 (fsck: offer a function to demote fsck errors to warnings,\n2015-06-22) it was only intended to be used to parse user-supplied\ndata.\n\nFor things that are purely internal to the C code it makes sense to\nhave the compiler check these arguments, and to skip the sanity\nchecking of the data in fsck_set_msg_type() which is redundant to\nchecks we get from the compiler.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/mktag.c | 3 ++-\n fsck.c | 27 +++++++++++++++++----------\n fsck.h | 3 +++\n 3 files changed, 22 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/mktag.c b/builtin/mktag.c\nindex 96e63bc772a..dddcccdd368 100644\n--- a/builtin/mktag.c\n+++ b/builtin/mktag.c\n@@ -88,7 +88,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)\n \t\tdie_errno(_(\"could not read from stdin\"));\n \n \tfsck_options.error_func = mktag_fsck_error_func;\n-\tfsck_set_msg_type(&fsck_options, \"extraheaderentry\", \"warn\");\n+\tfsck_set_msg_type_from_ids(&fsck_options, FSCK_MSG_EXTRA_HEADER_ENTRY,\n+\t\t\t\t FSCK_WARN);\n \t/* config might set fsck.extraHeaderEntry=* again */\n \tgit_config(git_fsck_config, &fsck_options);\n \tif (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,\ndiff --git a/fsck.c b/fsck.c\nindex 23a77fe2e0f..a59832a1650 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -132,6 +132,22 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type)\n \treturn 1;\n }\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type)\n+{\n+\tif (!options->msg_type) {\n+\t\tint i;\n+\t\tenum fsck_msg_type *severity;\n+\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n+\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n+\t\t\tseverity[i] = fsck_msg_type(i, options);\n+\t\toptions->msg_type = severity;\n+\t}\n+\n+\toptions->msg_type[msg_id] = msg_type;\n+}\n+\n void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id_str, const char *msg_type_str)\n {\n@@ -144,16 +160,7 @@ void fsck_set_msg_type(struct fsck_options *options,\n \tif (msg_type != FSCK_ERROR && msg_id_info[msg_id].msg_type == FSCK_FATAL)\n \t\tdie(\"Cannot demote %s to %s\", msg_id_str, msg_type_str);\n \n-\tif (!options->msg_type) {\n-\t\tint i;\n-\t\tenum fsck_msg_type *severity;\n-\t\tALLOC_ARRAY(severity, FSCK_MSG_MAX);\n-\t\tfor (i = 0; i < FSCK_MSG_MAX; i++)\n-\t\t\tseverity[i] = fsck_msg_type(i, options);\n-\t\toptions->msg_type = severity;\n-\t}\n-\n-\toptions->msg_type[msg_id] = msg_type;\n+\tfsck_set_msg_type_from_ids(options, msg_id, msg_type);\n }\n \n void fsck_set_msg_types(struct fsck_options *options, const char *values)\ndiff --git a/fsck.h b/fsck.h\nindex fa2d4955ab3..d284bac3614 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -82,6 +82,9 @@ enum fsck_msg_id {\n struct fsck_options;\n struct object;\n \n+void fsck_set_msg_type_from_ids(struct fsck_options *options,\n+\t\t\t\tenum fsck_msg_id msg_id,\n+\t\t\t\tenum fsck_msg_type msg_type);\n void fsck_set_msg_type(struct fsck_options *options,\n \t\t const char *msg_id, const char *msg_type);\n void fsck_set_msg_types(struct fsck_options *options, const char *values);\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420401","messageId":"patch-13.20-1b42aea3a64-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 13/19] fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:45Z","receivedAt":"2021-03-28T13:17:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the FOREACH_FSCK_MSG_ID macro and the fsck_msg_id enum it helps\ndefine from fsck.c to fsck.h. This is in preparation for having\nnon-static functions take the fsck_msg_id as an argument.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fsck.c | 66 ----------------------------------------------------------\n fsck.h | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 66 insertions(+), 66 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 31c9088e3f7..150fe467e43 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -22,72 +22,6 @@\n static struct oidset gitmodules_found = OIDSET_INIT;\n static struct oidset gitmodules_done = OIDSET_INIT;\n \n-#define FOREACH_FSCK_MSG_ID(FUNC) \\\n-\t/* fatal errors */ \\\n-\tFUNC(NUL_IN_HEADER, FATAL) \\\n-\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n-\t/* errors */ \\\n-\tFUNC(BAD_DATE, ERROR) \\\n-\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n-\tFUNC(BAD_EMAIL, ERROR) \\\n-\tFUNC(BAD_NAME, ERROR) \\\n-\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n-\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n-\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n-\tFUNC(BAD_TIMEZONE, ERROR) \\\n-\tFUNC(BAD_TREE, ERROR) \\\n-\tFUNC(BAD_TREE_SHA1, ERROR) \\\n-\tFUNC(BAD_TYPE, ERROR) \\\n-\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n-\tFUNC(MISSING_AUTHOR, ERROR) \\\n-\tFUNC(MISSING_COMMITTER, ERROR) \\\n-\tFUNC(MISSING_EMAIL, ERROR) \\\n-\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_OBJECT, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n-\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n-\tFUNC(MISSING_TAG, ERROR) \\\n-\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n-\tFUNC(MISSING_TREE, ERROR) \\\n-\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n-\tFUNC(MISSING_TYPE, ERROR) \\\n-\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n-\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n-\tFUNC(TREE_NOT_SORTED, ERROR) \\\n-\tFUNC(UNKNOWN_TYPE, ERROR) \\\n-\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n-\tFUNC(GITMODULES_MISSING, ERROR) \\\n-\tFUNC(GITMODULES_BLOB, ERROR) \\\n-\tFUNC(GITMODULES_LARGE, ERROR) \\\n-\tFUNC(GITMODULES_NAME, ERROR) \\\n-\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n-\tFUNC(GITMODULES_URL, ERROR) \\\n-\tFUNC(GITMODULES_PATH, ERROR) \\\n-\tFUNC(GITMODULES_UPDATE, ERROR) \\\n-\t/* warnings */ \\\n-\tFUNC(BAD_FILEMODE, WARN) \\\n-\tFUNC(EMPTY_NAME, WARN) \\\n-\tFUNC(FULL_PATHNAME, WARN) \\\n-\tFUNC(HAS_DOT, WARN) \\\n-\tFUNC(HAS_DOTDOT, WARN) \\\n-\tFUNC(HAS_DOTGIT, WARN) \\\n-\tFUNC(NULL_SHA1, WARN) \\\n-\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n-\tFUNC(NUL_IN_COMMIT, WARN) \\\n-\t/* infos (reported as warnings, but ignored by default) */ \\\n-\tFUNC(GITMODULES_PARSE, INFO) \\\n-\tFUNC(BAD_TAG_NAME, INFO) \\\n-\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n-\t/* ignored (elevated when requested) */ \\\n-\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n-\n-#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n-enum fsck_msg_id {\n-\tFOREACH_FSCK_MSG_ID(MSG_ID)\n-\tFSCK_MSG_MAX\n-};\n-#undef MSG_ID\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\ndiff --git a/fsck.h b/fsck.h\nindex a7e092d3fb4..66c4a71139a 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -13,6 +13,72 @@ enum fsck_msg_type {\n \tFSCK_WARN,\n };\n \n+#define FOREACH_FSCK_MSG_ID(FUNC) \\\n+\t/* fatal errors */ \\\n+\tFUNC(NUL_IN_HEADER, FATAL) \\\n+\tFUNC(UNTERMINATED_HEADER, FATAL) \\\n+\t/* errors */ \\\n+\tFUNC(BAD_DATE, ERROR) \\\n+\tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n+\tFUNC(BAD_EMAIL, ERROR) \\\n+\tFUNC(BAD_NAME, ERROR) \\\n+\tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n+\tFUNC(BAD_PARENT_SHA1, ERROR) \\\n+\tFUNC(BAD_TAG_OBJECT, ERROR) \\\n+\tFUNC(BAD_TIMEZONE, ERROR) \\\n+\tFUNC(BAD_TREE, ERROR) \\\n+\tFUNC(BAD_TREE_SHA1, ERROR) \\\n+\tFUNC(BAD_TYPE, ERROR) \\\n+\tFUNC(DUPLICATE_ENTRIES, ERROR) \\\n+\tFUNC(MISSING_AUTHOR, ERROR) \\\n+\tFUNC(MISSING_COMMITTER, ERROR) \\\n+\tFUNC(MISSING_EMAIL, ERROR) \\\n+\tFUNC(MISSING_NAME_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_OBJECT, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_DATE, ERROR) \\\n+\tFUNC(MISSING_SPACE_BEFORE_EMAIL, ERROR) \\\n+\tFUNC(MISSING_TAG, ERROR) \\\n+\tFUNC(MISSING_TAG_ENTRY, ERROR) \\\n+\tFUNC(MISSING_TREE, ERROR) \\\n+\tFUNC(MISSING_TREE_OBJECT, ERROR) \\\n+\tFUNC(MISSING_TYPE, ERROR) \\\n+\tFUNC(MISSING_TYPE_ENTRY, ERROR) \\\n+\tFUNC(MULTIPLE_AUTHORS, ERROR) \\\n+\tFUNC(TREE_NOT_SORTED, ERROR) \\\n+\tFUNC(UNKNOWN_TYPE, ERROR) \\\n+\tFUNC(ZERO_PADDED_DATE, ERROR) \\\n+\tFUNC(GITMODULES_MISSING, ERROR) \\\n+\tFUNC(GITMODULES_BLOB, ERROR) \\\n+\tFUNC(GITMODULES_LARGE, ERROR) \\\n+\tFUNC(GITMODULES_NAME, ERROR) \\\n+\tFUNC(GITMODULES_SYMLINK, ERROR) \\\n+\tFUNC(GITMODULES_URL, ERROR) \\\n+\tFUNC(GITMODULES_PATH, ERROR) \\\n+\tFUNC(GITMODULES_UPDATE, ERROR) \\\n+\t/* warnings */ \\\n+\tFUNC(BAD_FILEMODE, WARN) \\\n+\tFUNC(EMPTY_NAME, WARN) \\\n+\tFUNC(FULL_PATHNAME, WARN) \\\n+\tFUNC(HAS_DOT, WARN) \\\n+\tFUNC(HAS_DOTDOT, WARN) \\\n+\tFUNC(HAS_DOTGIT, WARN) \\\n+\tFUNC(NULL_SHA1, WARN) \\\n+\tFUNC(ZERO_PADDED_FILEMODE, WARN) \\\n+\tFUNC(NUL_IN_COMMIT, WARN) \\\n+\t/* infos (reported as warnings, but ignored by default) */ \\\n+\tFUNC(GITMODULES_PARSE, INFO) \\\n+\tFUNC(BAD_TAG_NAME, INFO) \\\n+\tFUNC(MISSING_TAGGER_ENTRY, INFO) \\\n+\t/* ignored (elevated when requested) */ \\\n+\tFUNC(EXTRA_HEADER_ENTRY, IGNORE)\n+\n+#define MSG_ID(id, msg_type) FSCK_MSG_##id,\n+enum fsck_msg_id {\n+\tFOREACH_FSCK_MSG_ID(MSG_ID)\n+\tFSCK_MSG_MAX\n+};\n+#undef MSG_ID\n+\n struct fsck_options;\n struct object;\n \n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420402","messageId":"patch-16.20-611631dd779-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 16/19] fsck.c: move gitmodules_{found,done} into fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:48Z","receivedAt":"2021-03-28T13:17:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Move the gitmodules_{found,done} static variables added in\n159e7b080bf (fsck: detect gitmodules files, 2018-05-02) into the\nfsck_options struct. It makes sense to keep all the context in the\nsame place.\n\nThis requires changing the recently added register_found_gitmodules()\nfunction added in 5476e1efde (fetch-pack: print and use dangling\n.gitmodules, 2021-02-22) to take fsck_options. That function will be\nremoved in a subsequent commit, but as it'll require the new\ngitmodules_found attribute of \"fsck_options\" we need this intermediate\nstep first.\n\nAn earlier version of this patch removed the small amount of\nduplication we now have between FSCK_OPTIONS_{DEFAULT,STRICT} with a\nFSCK_OPTIONS_COMMON macro. I don't think such de-duplication is worth\nit for this amount of copy/pasting.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fetch-pack.c | 2 +-\n fsck.c | 23 ++++++++++-------------\n fsck.h | 9 ++++++++-\n 3 files changed, 19 insertions(+), 15 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex fb04a76ca26..0f898a5ae14 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -998,7 +998,7 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(oid);\n+\t\tregister_found_gitmodules(&fo, oid);\n \tif (fsck_finish(&fo))\n \t\tdie(\"fsck failed\");\n }\ndiff --git a/fsck.c b/fsck.c\nindex a59832a1650..642bd2ef9da 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -19,9 +19,6 @@\n #include \"credential.h\"\n #include \"help.h\"\n \n-static struct oidset gitmodules_found = OIDSET_INIT;\n-static struct oidset gitmodules_done = OIDSET_INIT;\n-\n #define STR(x) #x\n #define MSG_ID(id, msg_type) { STR(id), NULL, NULL, FSCK_##msg_type },\n static struct {\n@@ -606,7 +603,7 @@ static int fsck_tree(const struct object_id *oid,\n \n \t\tif (is_hfs_dotgitmodules(name) || is_ntfs_dotgitmodules(name)) {\n \t\t\tif (!S_ISLNK(mode))\n-\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\telse\n \t\t\t\tretval += report(options,\n \t\t\t\t\t\t oid, OBJ_TREE,\n@@ -620,7 +617,7 @@ static int fsck_tree(const struct object_id *oid,\n \t\t\t\thas_dotgit |= is_ntfs_dotgit(backslash);\n \t\t\t\tif (is_ntfs_dotgitmodules(backslash)) {\n \t\t\t\t\tif (!S_ISLNK(mode))\n-\t\t\t\t\t\toidset_insert(&gitmodules_found, oid);\n+\t\t\t\t\t\toidset_insert(&options->gitmodules_found, oid);\n \t\t\t\t\telse\n \t\t\t\t\t\tretval += report(options, oid, OBJ_TREE,\n \t\t\t\t\t\t\t\t FSCK_MSG_GITMODULES_SYMLINK,\n@@ -1132,9 +1129,9 @@ static int fsck_blob(const struct object_id *oid, const char *buf,\n \tstruct fsck_gitmodules_data data;\n \tstruct config_options config_opts = { 0 };\n \n-\tif (!oidset_contains(&gitmodules_found, oid))\n+\tif (!oidset_contains(&options->gitmodules_found, oid))\n \t\treturn 0;\n-\toidset_insert(&gitmodules_done, oid);\n+\toidset_insert(&options->gitmodules_done, oid);\n \n \tif (object_on_skiplist(options, oid))\n \t\treturn 0;\n@@ -1199,9 +1196,9 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n-void register_found_gitmodules(const struct object_id *oid)\n+void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n {\n-\toidset_insert(&gitmodules_found, oid);\n+\toidset_insert(&options->gitmodules_found, oid);\n }\n \n int fsck_finish(struct fsck_options *options)\n@@ -1210,13 +1207,13 @@ int fsck_finish(struct fsck_options *options)\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n \n-\toidset_iter_init(&gitmodules_found, &iter);\n+\toidset_iter_init(&options->gitmodules_found, &iter);\n \twhile ((oid = oidset_iter_next(&iter))) {\n \t\tenum object_type type;\n \t\tunsigned long size;\n \t\tchar *buf;\n \n-\t\tif (oidset_contains(&gitmodules_done, oid))\n+\t\tif (oidset_contains(&options->gitmodules_done, oid))\n \t\t\tcontinue;\n \n \t\tbuf = read_object_file(oid, &type, &size);\n@@ -1241,8 +1238,8 @@ int fsck_finish(struct fsck_options *options)\n \t}\n \n \n-\toidset_clear(&gitmodules_found);\n-\toidset_clear(&gitmodules_done);\n+\toidset_clear(&options->gitmodules_found);\n+\toidset_clear(&options->gitmodules_done);\n \treturn ret;\n }\n \ndiff --git a/fsck.h b/fsck.h\nindex d284bac3614..e20f9bcb394 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -118,15 +118,21 @@ struct fsck_options {\n \tunsigned strict:1;\n \tenum fsck_msg_type *msg_type;\n \tstruct oidset skiplist;\n+\tstruct oidset gitmodules_found;\n+\tstruct oidset gitmodules_done;\n \tkh_oid_map_t *object_names;\n };\n \n #define FSCK_OPTIONS_DEFAULT { \\\n \t.skiplist = OIDSET_INIT, \\\n+\t.gitmodules_found = OIDSET_INIT, \\\n+\t.gitmodules_done = OIDSET_INIT, \\\n \t.error_func = fsck_error_function \\\n }\n #define FSCK_OPTIONS_STRICT { \\\n \t.strict = 1, \\\n+\t.gitmodules_found = OIDSET_INIT, \\\n+\t.gitmodules_done = OIDSET_INIT, \\\n \t.error_func = fsck_error_function, \\\n }\n \n@@ -146,7 +152,8 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n-void register_found_gitmodules(const struct object_id *oid);\n+void register_found_gitmodules(struct fsck_options *options,\n+\t\t\t const struct object_id *oid);\n \n /*\n * fsck a tag, and pass info about it back to the caller. This is\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420404","messageId":"patch-17.20-03d512c8448-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 17/19] fetch-pack: don't needlessly copy fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:49Z","receivedAt":"2021-03-28T13:17:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the behavior of the .gitmodules validation added in\n5476e1efde (fetch-pack: print and use dangling .gitmodules,\n2021-02-22) so we're using one \"fsck_options\".\n\nI found that code confusing to read. One might think that not setting\nup the error_func earlier means that we're relying on the \"error_func\"\nnot being set in some code in between the two hunks being modified\nhere.\n\nBut we're not, all we're doing in the rest of \"cmd_index_pack()\" is\nfurther setup by calling fsck_set_msg_types(), and assigning to\ndo_fsck_object.\n\nSo there was no reason in 5476e1efde to make a shallow copy of the\nfsck_options struct before setting error_func. Let's just do this\nsetup at the top of the function, along with the \"walk\" assignment.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/index-pack.c | 10 +++-------\n 1 file changed, 3 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 2f93957fb5e..5b7bc3c8947 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1761,6 +1761,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \n \tread_replace_refs = 0;\n \tfsck_options.walk = mark_link;\n+\tfsck_options.error_func = print_dangling_gitmodules;\n \n \treset_pack_idx_option(&opts);\n \tgit_config(git_index_pack_config, &opts);\n@@ -1951,13 +1952,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \telse\n \t\tclose(input_fd);\n \n-\tif (do_fsck_object) {\n-\t\tstruct fsck_options fo = fsck_options;\n-\n-\t\tfo.error_func = print_dangling_gitmodules;\n-\t\tif (fsck_finish(&fo))\n-\t\t\tdie(_(\"fsck error in pack objects\"));\n-\t}\n+\tif (do_fsck_object && fsck_finish(&fsck_options))\n+\t\tdie(_(\"fsck error in pack objects\"));\n \n \tfree(objects);\n \tstrbuf_release(&index_name_buf);\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420406","messageId":"patch-19.20-6a38cade8c3-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 19/19] fetch-pack: use new fsck API to printing dangling submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:51Z","receivedAt":"2021-03-28T13:17:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Refactor the check added in 5476e1efde (fetch-pack: print and use\ndangling .gitmodules, 2021-02-22) to make use of us now passing the\n\"msg_id\" to the user defined \"error_func\". We can now compare against\nthe FSCK_MSG_GITMODULES_MISSING instead of parsing the generated\nmessage.\n\nLet's also replace register_found_gitmodules() with directly\nmanipulating the \"gitmodules_found\" member. A recent commit moved it\ninto \"fsck_options\" so we could do this here.\n\nI'm sticking this callback in fsck.c. Perhaps in the future we'd like\nto accumulate such callbacks into another file (maybe fsck-cb.c,\nsimilar to parse-options-cb.c?), but while we've got just the one\nlet's just put it into fsck.c.\n\nA better alternative in this case would be some library some more\nobvious library shared by fetch-pack.c ad builtin/index-pack.c, but\nthere isn't such a thing.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/index-pack.c | 21 +--------------------\n fetch-pack.c | 31 ++++++++-----------------------\n fsck.c | 23 ++++++++++++++++++-----\n fsck.h | 15 ++++++++++++---\n 4 files changed, 39 insertions(+), 51 deletions(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 5b7bc3c8947..15507b5cff0 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -120,7 +120,7 @@ static int nr_threads;\n static int from_stdin;\n static int strict;\n static int do_fsck_object;\n-static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_MISSING_GITMODULES;\n static int verbose;\n static int show_resolving_progress;\n static int show_stat;\n@@ -1713,24 +1713,6 @@ static void show_pack_info(int stat_only)\n \t}\n }\n \n-static int print_dangling_gitmodules(struct fsck_options *o,\n-\t\t\t\t const struct object_id *oid,\n-\t\t\t\t enum object_type object_type,\n-\t\t\t\t enum fsck_msg_type msg_type,\n-\t\t\t\t enum fsck_msg_id msg_id,\n-\t\t\t\t const char *message)\n-{\n-\t/*\n-\t * NEEDSWORK: Plumb the MSG_ID (from fsck.c) here and use it\n-\t * instead of relying on this string check.\n-\t */\n-\tif (starts_with(message, \"gitmodulesMissing\")) {\n-\t\tprintf(\"%s\\n\", oid_to_hex(oid));\n-\t\treturn 0;\n-\t}\n-\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n-}\n-\n int cmd_index_pack(int argc, const char **argv, const char *prefix)\n {\n \tint i, fix_thin_pack = 0, verify = 0, stat_only = 0, rev_index;\n@@ -1761,7 +1743,6 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \n \tread_replace_refs = 0;\n \tfsck_options.walk = mark_link;\n-\tfsck_options.error_func = print_dangling_gitmodules;\n \n \treset_pack_idx_option(&opts);\n \tgit_config(git_index_pack_config, &opts);\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 4ec10a15852..c80eaee7694 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -38,7 +38,7 @@ static int server_supports_filtering;\n static int advertise_sid;\n static struct shallow_lock shallow_lock;\n static const char *alternate_shallow_file;\n-static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_MISSING_GITMODULES;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n \n@@ -988,21 +988,6 @@ static int cmp_ref_by_name(const void *a_, const void *b_)\n \treturn strcmp(a->name, b->name);\n }\n \n-static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n-{\n-\tstruct oidset_iter iter;\n-\tconst struct object_id *oid;\n-\n-\tif (!oidset_size(gitmodules_oids))\n-\t\treturn;\n-\n-\toidset_iter_init(gitmodules_oids, &iter);\n-\twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(&fsck_options, oid);\n-\tif (fsck_finish(&fsck_options))\n-\t\tdie(\"fsck failed\");\n-}\n-\n static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \t\t\t\t int fd[2],\n \t\t\t\t const struct ref *orig_ref,\n@@ -1017,7 +1002,6 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \tint agent_len;\n \tstruct fetch_negotiator negotiator_alloc;\n \tstruct fetch_negotiator *negotiator;\n-\tstruct oidset gitmodules_oids = OIDSET_INIT;\n \n \tnegotiator = &negotiator_alloc;\n \tfetch_negotiator_init(r, negotiator);\n@@ -1134,9 +1118,10 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,\n \telse\n \t\talternate_shallow_file = NULL;\n \tif (get_pack(args, fd, pack_lockfiles, NULL, sought, nr_sought,\n-\t\t &gitmodules_oids))\n+\t\t &fsck_options.gitmodules_found))\n \t\tdie(_(\"git fetch-pack: fetch failed.\"));\n-\tfsck_gitmodules_oids(&gitmodules_oids);\n+\tif (fsck_finish(&fsck_options))\n+\t\tdie(\"fsck failed\");\n \n all_done:\n \tif (negotiator)\n@@ -1587,7 +1572,6 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstruct string_list packfile_uris = STRING_LIST_INIT_DUP;\n \tint i;\n \tstruct strvec index_pack_args = STRVEC_INIT;\n-\tstruct oidset gitmodules_oids = OIDSET_INIT;\n \n \tnegotiator = &negotiator_alloc;\n \tfetch_negotiator_init(r, negotiator);\n@@ -1678,7 +1662,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \t\t\tprocess_section_header(&reader, \"packfile\", 0);\n \t\t\tif (get_pack(args, fd, pack_lockfiles,\n \t\t\t\t packfile_uris.nr ? &index_pack_args : NULL,\n-\t\t\t\t sought, nr_sought, &gitmodules_oids))\n+\t\t\t\t sought, nr_sought, &fsck_options.gitmodules_found))\n \t\t\t\tdie(_(\"git fetch-pack: fetch failed.\"));\n \t\t\tdo_check_stateless_delimiter(args, &reader);\n \n@@ -1721,7 +1705,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \n \t\tpackname[the_hash_algo->hexsz] = '\\0';\n \n-\t\tparse_gitmodules_oids(cmd.out, &gitmodules_oids);\n+\t\tparse_gitmodules_oids(cmd.out, &fsck_options.gitmodules_found);\n \n \t\tclose(cmd.out);\n \n@@ -1742,7 +1726,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,\n \tstring_list_clear(&packfile_uris, 0);\n \tstrvec_clear(&index_pack_args);\n \n-\tfsck_gitmodules_oids(&gitmodules_oids);\n+\tif (fsck_finish(&fsck_options))\n+\t\tdie(\"fsck failed\");\n \n \tif (negotiator)\n \t\tnegotiator->release(negotiator);\ndiff --git a/fsck.c b/fsck.c\nindex 642bd2ef9da..f5ed6a26358 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1196,11 +1196,6 @@ int fsck_error_function(struct fsck_options *o,\n \treturn 1;\n }\n \n-void register_found_gitmodules(struct fsck_options *options, const struct object_id *oid)\n-{\n-\toidset_insert(&options->gitmodules_found, oid);\n-}\n-\n int fsck_finish(struct fsck_options *options)\n {\n \tint ret = 0;\n@@ -1266,3 +1261,21 @@ int git_fsck_config(const char *var, const char *value, void *cb)\n \n \treturn git_default_config(var, value, cb);\n }\n+\n+/*\n+ * Custom error callbacks that are used in more than one place.\n+ */\n+\n+int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n+\t\t\t\t\t const struct object_id *oid,\n+\t\t\t\t\t enum object_type object_type,\n+\t\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t\t const char *message)\n+{\n+\tif (msg_id == FSCK_MSG_GITMODULES_MISSING) {\n+\t\tputs(oid_to_hex(oid));\n+\t\treturn 0;\n+\t}\n+\treturn fsck_error_function(o, oid, object_type, msg_type, msg_id, message);\n+}\ndiff --git a/fsck.h b/fsck.h\nindex e20f9bcb394..7202c3c87e8 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -111,6 +111,12 @@ int fsck_error_function(struct fsck_options *o,\n \t\t\tconst struct object_id *oid, enum object_type object_type,\n \t\t\tenum fsck_msg_type msg_type, enum fsck_msg_id msg_id,\n \t\t\tconst char *message);\n+int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o,\n+\t\t\t\t\t const struct object_id *oid,\n+\t\t\t\t\t enum object_type object_type,\n+\t\t\t\t\t enum fsck_msg_type msg_type,\n+\t\t\t\t\t enum fsck_msg_id msg_id,\n+\t\t\t\t\t const char *message);\n \n struct fsck_options {\n \tfsck_walk_func walk;\n@@ -135,6 +141,12 @@ struct fsck_options {\n \t.gitmodules_done = OIDSET_INIT, \\\n \t.error_func = fsck_error_function, \\\n }\n+#define FSCK_OPTIONS_MISSING_GITMODULES { \\\n+\t.strict = 1, \\\n+\t.gitmodules_found = OIDSET_INIT, \\\n+\t.gitmodules_done = OIDSET_INIT, \\\n+\t.error_func = fsck_error_cb_print_missing_gitmodules, \\\n+}\n \n /* descend in all linked child objects\n * the return value is:\n@@ -152,9 +164,6 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);\n int fsck_object(struct object *obj, void *data, unsigned long size,\n \tstruct fsck_options *options);\n \n-void register_found_gitmodules(struct fsck_options *options,\n-\t\t\t const struct object_id *oid);\n-\n /*\n * fsck a tag, and pass info about it back to the caller. This is\n * exposed fsck_object() internals for git-mktag(1).\n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420407","messageId":"patch-18.20-581c87c63c6-20210328T130947Z-avarab@gmail.com","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"[PATCH v6 18/19] fetch-pack: use file-scope static struct for fsck_options","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-03-28T13:15:50Z","receivedAt":"2021-03-28T13:17:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change code added in 5476e1efde (fetch-pack: print and use dangling\n.gitmodules, 2021-02-22) so that we use a file-scoped \"static struct\nfsck_options\" instead of defining one in the \"fsck_gitmodules_oids()\"\nfunction.\n\nWe use this pattern in all of\nbuiltin/{fsck,index-pack,mktag,unpack-objects}.c. It's odd to see\nfetch-pack be the odd one out. One might think that we're using other\nfsck_options structs in fetch-pack, or doing on fsck twice there, but\nwe're not.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n fetch-pack.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 0f898a5ae14..4ec10a15852 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -38,6 +38,7 @@ static int server_supports_filtering;\n static int advertise_sid;\n static struct shallow_lock shallow_lock;\n static const char *alternate_shallow_file;\n+static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;\n static struct strbuf fsck_msg_types = STRBUF_INIT;\n static struct string_list uri_protocols = STRING_LIST_INIT_DUP;\n \n@@ -991,15 +992,14 @@ static void fsck_gitmodules_oids(struct oidset *gitmodules_oids)\n {\n \tstruct oidset_iter iter;\n \tconst struct object_id *oid;\n-\tstruct fsck_options fo = FSCK_OPTIONS_STRICT;\n \n \tif (!oidset_size(gitmodules_oids))\n \t\treturn;\n \n \toidset_iter_init(gitmodules_oids, &iter);\n \twhile ((oid = oidset_iter_next(&iter)))\n-\t\tregister_found_gitmodules(&fo, oid);\n-\tif (fsck_finish(&fo))\n+\t\tregister_found_gitmodules(&fsck_options, oid);\n+\tif (fsck_finish(&fsck_options))\n \t\tdie(\"fsck failed\");\n }\n \n-- \n2.31.1.445.g087790d4945\n\n"},{"id":"420415","messageId":"YGC5pzAOycCePCU6@satellite","threadId":"54994","inReplyTo":"patch-02.20-b17c982293e-20210328T130947Z-avarab@gmail.com","subject":"Re: [PATCH v6 02/19] fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2021-03-28T17:15:19Z","receivedAt":"2021-03-28T17:16:07Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"On Sun, Mar 28, 2021 at 03:15:34PM +0200, Ævar Arnfjörð Bjarmason wrote:\n> Refactor the definitions of FSCK_OPTIONS_{DEFAULT,STRICT} to use\n> designated initializers. This allows us to omit those fields that\n> aren't initialized to zero or NULL.\n\ns/aren't/are/\n\n[I apologize in advance - I am using mutt for the first time to reply\nto a ML post and I don't know if I should be using L-ist-reply or a\ng-roup-reply! :D ]\n\nATB,\nRamsay Jones\n\n"},{"id":"420427","messageId":"xmqqblb2lnq5.fsf@gitster.g","threadId":"54994","inReplyTo":"YGC5pzAOycCePCU6@satellite","subject":"Re: [PATCH v6 02/19] fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT}","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-29T02:04:18Z","receivedAt":"2021-03-29T02:05:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramsay Jones writes:\n\n> On Sun, Mar 28, 2021 at 03:15:34PM +0200, Ævar Arnfjörð Bjarmason wrote:\n>> Refactor the definitions of FSCK_OPTIONS_{DEFAULT,STRICT} to use\n>> designated initializers. This allows us to omit those fields that\n>> aren't initialized to zero or NULL.\n>\n> s/aren't/are/\n\nThanks; tweak applied while queuing.\n\n> [I apologize in advance - I am using mutt for the first time to reply\n> to a ML post and I don't know if I should be using L-ist-reply or a\n> g-roup-reply! :D ]\n\nFWIW, on lore (reading via nntp), the message I am responding to\nlooks just fine.\n\n"},{"id":"420428","messageId":"xmqq5z1alnmu.fsf@gitster.g","threadId":"54994","inReplyTo":"cover-00.20-00000000000-20210328T130947Z-avarab@gmail.com","subject":"Re: [PATCH v6 00/19] fsck: API improvements","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-29T02:06:17Z","receivedAt":"2021-03-29T02:07:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> To recap on the goals in v1[1] this series gets rid of the need to\n> have the rececently added \"print_dangling_gitmodules\" function in\n> favor of a better fsck API to get at that information.\n\nRead the whole series afresh, as well as \"git diff @{1}\" after\nreplacing to see what changed since the previous round. Didn't find\nanything iffy.\n\nUnless somebody finds improvement opportunities in the coming couple\nof days, let's declare it is good enough and merge to 'next',\npolishing incrementally if needed.\n\nThanks.\n"}]}