{"thread":{"id":"51545","subject":"Problem with fsck and invalid submodule path in history","startedAt":"2019-07-29T07:58:56Z","lastAt":"2019-07-29T21:32:07Z","messageCount":8,"participants":["Olivier Bornet","SZEDER Gábor","Ævar Arnfjörð Bjarmason","Junio C Hamano","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"379475","messageId":"9C668779-15AC-4099-AAFA-7FFF519D426F@puck.ch","threadId":"51545","inReplyTo":null,"subject":"Problem with fsck and invalid submodule path in history","fromName":"Olivier Bornet","fromEmail":"olivier.bornet@puck.ch","sentAt":"2019-07-29T07:58:52Z","receivedAt":"2019-07-29T07:58:56Z","isPatch":false,"sender":{"key":"olivier.bornet@puck.ch","avatar":null},"body":"Hello,\n\nI have a git repository with an error in a submodule path in the history.\nThe submodule path is “-f”, which is not allowed. But for some reason, it’s in the history of the git, and I’m trying to find a way to manage it without having to rewrite the full history on the main gitlab (if possible)...\n\nTo reproduce this unwanted history:\n\n mkdir test-bad-history\n cd test-bad-history\n echo \"Test git submodule problems\" > README.md\n git init\n git add README.md\n git commit --message=\"Start test\"\n git submodule add https://github.com/leachim6/hello-world.git\n git commit --message=\"Commit new submodule with correct path\"\n # the bad part...\n git mv hello-world -- -f\n git commit --message=\"Move submodule to an invalid path\"\n # correct it...\n git mv -- -f valid-path\n sed -i.bak 's/-f/valid-path/' .gitmodules\n git add .gitmodules\n git commit --message=\"Back to a valid path\"\n\nAfter that, even if the git is working correctly, we have a “bad” history if we check with fsck:\n\n $ git fsck\n Checking object directories: 100% (256/256), done.\n error in blob 19a97d3b70760c74b780c8134e33f5392292c2e6: gitmodulesPath: disallowed submodule path: -f\n\nIs it possible to correct it? Must git handle this kind of errors?\n\nThanks in advance for any help.\n\n--\nOlivier Bornet\nOlivier.Bornet@puck.ch\n\n"},{"id":"379485","messageId":"20190729093928.GP20404@szeder.dev","threadId":"51545","inReplyTo":"9C668779-15AC-4099-AAFA-7FFF519D426F@puck.ch","subject":"Re: Problem with fsck and invalid submodule path in history","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-07-29T09:39:28Z","receivedAt":"2019-07-29T09:39:34Z","isPatch":false,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Mon, Jul 29, 2019 at 09:58:52AM +0200, Olivier Bornet wrote:\n> I have a git repository with an error in a submodule path in the history.\n> The submodule path is “-f”, which is not allowed. But for some reason, it’s in the history of the git, and I’m trying to find a way to manage it without having to rewrite the full history on the main gitlab (if possible)...\n> \n> To reproduce this unwanted history:\n\n> After that, even if the git is working correctly, we have a “bad” history if we check with fsck:\n> \n> $ git fsck\n> Checking object directories: 100% (256/256), done.\n> error in blob 19a97d3b70760c74b780c8134e33f5392292c2e6: gitmodulesPath: disallowed submodule path: -f\n> \n> Is it possible to correct it? Must git handle this kind of errors?\n\nTo correct without rewriting history, no.\n\nHowever, you can tell 'git fsck' to ignore it using the\n'fsck.skipList' configuration variable (see 'git help config'; for\nsome reason it's not included in 'git fsck's documentation):\n\n $ cat <.git-fsck-skiplist\n > # invalid submodule path\n > 19a97d3b70760c74b780c8134e33f5392292c2e6\n > EOF\n $ git config fsck.skipList .git-fsck-skiplist\n $ git fsck\n Checking object directories: 100% (256/256), done.\n\nIt may or may not be worth committing this file, I'm not quite sure\nwhat the best practice is. By committing it others don't have to\nmaintain such a skiplist file themselves, though they still have to\nset the config variable. OTOH, if anyone sets this config variable\nand attempts to run 'git fsck' while on a branch that doesn't contain\nthis file, then they will get a 'fatal: could not open object name\nlist: .git-fsck-skiplist' error.\n\nAnd it won't help anyone cloning the repository with\n'fetch.fsckObjects' enabled.\n\n"},{"id":"379487","messageId":"20190729095914.30573-1-szeder.dev@gmail.com","threadId":"51545","inReplyTo":"20190729093928.GP20404@szeder.dev","subject":"[PATCH] Documentation/git-fsck.txt: include fsck.* config variables","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-07-29T09:59:14Z","receivedAt":"2019-07-29T09:59:24Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"The 'fsck.skipList' and 'fsck.' config variables might be\neasier to discover when they are documented in 'git fsck's man page.\n\nSigned-off-by: SZEDER Gábor \n---\n Documentation/git-fsck.txt | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt\nindex e0eae642c1..d72d15be5b 100644\n--- a/Documentation/git-fsck.txt\n+++ b/Documentation/git-fsck.txt\n@@ -104,6 +104,11 @@ care about this output and want to speed it up further.\n \tprogress status even if the standard error stream is not\n \tdirected to a terminal.\n \n+CONFIGURATION\n+-------------\n+\n+include::config/fsck.txt[]\n+\n DISCUSSION\n ----------\n \n-- \n2.22.0.924.g7c03687a56\n\n"},{"id":"379502","messageId":"10AB3C94-B14E-4F5F-AFC6-24F42A2FA5C1@puck.ch","threadId":"51545","inReplyTo":"20190729093928.GP20404@szeder.dev","subject":"Re: Problem with fsck and invalid submodule path in history","fromName":"Olivier Bornet","fromEmail":"olivier.bornet@puck.ch","sentAt":"2019-07-29T14:31:07Z","receivedAt":"2019-07-29T14:31:16Z","isPatch":false,"sender":{"key":"olivier.bornet@puck.ch","avatar":null},"body":"Hello,\n> On 29 Jul 2019, at 11:39, SZEDER Gábor wrote:\n> \n> However, you can tell 'git fsck' to ignore it using the\n> 'fsck.skipList' configuration variable (see 'git help config'; for\n> some reason it's not included in 'git fsck's documentation):\n\nThanks for pointing it out. Thanks to it, we have managed to handle\nthe error with our gitlab automatic fsck procedure.\n\nHave a nice day.\n\n--\nOlivier Bornet\nOlivier.Bornet@puck.ch\n\n\n"},{"id":"379505","messageId":"87sgqo6ejk.fsf@evledraar.gmail.com","threadId":"51545","inReplyTo":"20190729095914.30573-1-szeder.dev@gmail.com","subject":"Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-07-29T15:33:51Z","receivedAt":"2019-07-29T15:33:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 29 2019, SZEDER Gábor wrote:\n\n> The 'fsck.skipList' and 'fsck.' config variables might be\n> easier to discover when they are documented in 'git fsck's man page.\n>\n> Signed-off-by: SZEDER Gábor \n> ---\n> Documentation/git-fsck.txt | 5 +++++\n> 1 file changed, 5 insertions(+)\n>\n> diff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt\n> index e0eae642c1..d72d15be5b 100644\n> --- a/Documentation/git-fsck.txt\n> +++ b/Documentation/git-fsck.txt\n> @@ -104,6 +104,11 @@ care about this output and want to speed it up further.\n> \tprogress status even if the standard error stream is not\n> \tdirected to a terminal.\n>\n> +CONFIGURATION\n> +-------------\n> +\n> +include::config/fsck.txt[]\n\nBefore this include let's add:\n\n The below documentation is the same as what’s found in\n git-config(1):\n\nAs I did for a similar change in git-gc in b6a8d09f6d (\"gc docs: include\nthe \"gc.*\" section from \"config\" in \"gc\"\", 2019-04-07). Sometimes we\nrepeat ourselves, it helps the reader to know this isn't some slightly\ndifferent prose than what's in git-config.\n\n> +\n> DISCUSSION\n> ----------\n"},{"id":"379506","messageId":"xmqq36iox2nn.fsf@gitster-ct.c.googlers.com","threadId":"51545","inReplyTo":"87sgqo6ejk.fsf@evledraar.gmail.com","subject":"Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-07-29T15:48:28Z","receivedAt":"2019-07-29T15:48:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Mon, Jul 29 2019, SZEDER Gábor wrote:\n>\n>> The 'fsck.skipList' and 'fsck.' config variables might be\n>> easier to discover when they are documented in 'git fsck's man page.\n>>\n>> Signed-off-by: SZEDER Gábor \n>> ---\n>> Documentation/git-fsck.txt | 5 +++++\n>> 1 file changed, 5 insertions(+)\n>>\n>> diff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt\n>> index e0eae642c1..d72d15be5b 100644\n>> --- a/Documentation/git-fsck.txt\n>> +++ b/Documentation/git-fsck.txt\n>> @@ -104,6 +104,11 @@ care about this output and want to speed it up further.\n>> \tprogress status even if the standard error stream is not\n>> \tdirected to a terminal.\n>>\n>> +CONFIGURATION\n>> +-------------\n>> +\n>> +include::config/fsck.txt[]\n>\n> Before this include let's add:\n>\n> The below documentation is the same as what’s found in\n> git-config(1):\n\nI actually do not think we would want to do that. I am all for the\nkind of 'include' proposed by this patch, and we should strive to\nmake it easier for us to make sure the duplicated text are in sync.\n\nBut that would mean that the readers will have to see the \"is the\nsame as the other one\" over and over. If our documentation set is\nconsistent, they should not have to.\n\nI think we *must* make such a note in a total opposite case,\ni.e. \"here are the summary of the most often used options; for full\nlist, see git-config(1)\".\n\n> As I did for a similar change in git-gc in b6a8d09f6d (\"gc docs: include\n> the \"gc.*\" section from \"config\" in \"gc\"\", 2019-04-07). Sometimes we\n> repeat ourselves, it helps the reader to know this isn't some slightly\n> different prose than what's in git-config.\n\nSo, I think we should revert that part out of b6a8d09f6d, too.\n"},{"id":"379546","messageId":"20190729201236.GF14943@sigill.intra.peff.net","threadId":"51545","inReplyTo":"xmqq36iox2nn.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-07-29T20:12:36Z","receivedAt":"2019-07-29T20:12:40Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 29, 2019 at 08:48:28AM -0700, Junio C Hamano wrote:\n\n> > Before this include let's add:\n> >\n> > The below documentation is the same as what’s found in\n> > git-config(1):\n> \n> I actually do not think we would want to do that. I am all for the\n> kind of 'include' proposed by this patch, and we should strive to\n> make it easier for us to make sure the duplicated text are in sync.\n> \n> But that would mean that the readers will have to see the \"is the\n> same as the other one\" over and over. If our documentation set is\n> consistent, they should not have to.\n> \n> I think we *must* make such a note in a total opposite case,\n> i.e. \"here are the summary of the most often used options; for full\n> list, see git-config(1)\".\n\nI disagree. _We_ know that the content is the same, because we are\nlooking at the source that says \"include\". But as a user, how do I know\nwhen I get to one section or the other that it is something I have\nalready read and can skip over?\n\nPerhaps if Git were entirely consistent here, they could make the\nassumption that \"CONFIG\" sections were always duplicated and know this\nalready. But I think even that is asking a bit much. Unless they are\nintimately familiar with our documentation, they don't know that we are,\nin fact, consistent. And we are in an uphill battle with every other\nthing the user has read, which may not agree with our assumptions of\nconsistency. ;)\n\nSo IMHO it's worth leaving a note that guides the reader, as long as\nit's short (and I think this one is).\n\n\nThat said, I think an even _better_ solution would be to avoid includes,\nand instead make it clear when we are pointing the user to shared\ncontent. Then we get them to the right place _and_ explicitly instruct\nthem that concepts/content are shared. For config, for example, I've\nworked with a previous system that did something like:\n\n - include fsck.* documentation in the git-fsck manpage\n\n - make a master table of config options in git-config.1 with _just_\n the names and the associated manpage where the definition can be\n found. This serves as an index if you don't know where to look.\n\nThis would probably involve creating new concept-based pages for some of\nthe groupings (e.g., where does \"remote.*\" config go?), but I think that\nwould probably help round out our documentation (if there is a concept\nwith related config options but we don't explain it anywhere, that is\nprobably a gap we should fix).\n\nThe biggest downside is that chasing down references in manpages sucks.\nFor the HTML documentation we'd ideally hyperlink from the git-config.1\nindex into each definition, but there's no way to do that with a regular\nmanpage.\n\n-Peff\n\nPS This is an approach I've advocated for a while:\n\n https://public-inbox.org/git/20110120233429.GB9442@sigill.intra.peff.net/\n\n but haven't actually done much about, so perhaps I should be putting\n my money where my mouth is. ;)\n"},{"id":"379552","messageId":"xmqq5znkv86m.fsf@gitster-ct.c.googlers.com","threadId":"51545","inReplyTo":"20190729201236.GF14943@sigill.intra.peff.net","subject":"Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-07-29T21:32:01Z","receivedAt":"2019-07-29T21:32:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Mon, Jul 29, 2019 at 08:48:28AM -0700, Junio C Hamano wrote:\n>\n>> > Before this include let's add:\n>> >\n>> > The below documentation is the same as what’s found in\n>> > git-config(1):\n>> \n>> I actually do not think we would want to do that. I am all for the\n>> kind of 'include' proposed by this patch, and we should strive to\n>> make it easier for us to make sure the duplicated text are in sync.\n>> \n>> But that would mean that the readers will have to see the \"is the\n>> same as the other one\" over and over. If our documentation set is\n>> consistent, they should not have to.\n>> \n>> I think we *must* make such a note in a total opposite case,\n>> i.e. \"here are the summary of the most often used options; for full\n>> list, see git-config(1)\".\n>\n> I disagree. _We_ know that the content is the same, because we are\n> looking at the source that says \"include\". But as a user, how do I know\n> when I get to one section or the other that it is something I have\n> already read and can skip over?\n\nI want to raise the user expectation so that they would expect from\nour documentation, unless we say \"these are different\", we would\nnever say conflicting things in two places.\n\nSo,... I disagree.\n"}]}