# Problem with fsck and invalid submodule path in history 8 messages from 2019-07-29 to 2019-07-29. Participants: Olivier Bornet, SZEDER Gábor, Ævar Arnfjörð Bjarmason, Junio C Hamano, Jeff King. Thread: https://gitlist.dev/t/51545 ## Olivier Bornet, 2019-07-29 07:58 Subject: Problem with fsck and invalid submodule path in history Message-ID: <9C668779-15AC-4099-AAFA-7FFF519D426F@puck.ch> URL: https://gitlist.dev/e/9C668779-15AC-4099-AAFA-7FFF519D426F%40puck.ch ``` Hello, I have a git repository with an error in a submodule path in the history. The submodule path is “-f”, which is not allowed. But for some reason, it’s in the history of the git, and I’m trying to find a way to manage it without having to rewrite the full history on the main gitlab (if possible)... To reproduce this unwanted history: mkdir test-bad-history cd test-bad-history echo "Test git submodule problems" > README.md git init git add README.md git commit --message="Start test" git submodule add https://github.com/leachim6/hello-world.git git commit --message="Commit new submodule with correct path" # the bad part... git mv hello-world -- -f git commit --message="Move submodule to an invalid path" # correct it... git mv -- -f valid-path sed -i.bak 's/-f/valid-path/' .gitmodules git add .gitmodules git commit --message="Back to a valid path" After that, even if the git is working correctly, we have a “bad” history if we check with fsck: $ git fsck Checking object directories: 100% (256/256), done. error in blob 19a97d3b70760c74b780c8134e33f5392292c2e6: gitmodulesPath: disallowed submodule path: -f Is it possible to correct it? Must git handle this kind of errors? Thanks in advance for any help. -- Olivier Bornet Olivier.Bornet@puck.ch ``` ## SZEDER Gábor, 2019-07-29 09:39 Subject: Re: Problem with fsck and invalid submodule path in history Message-ID: <20190729093928.GP20404@szeder.dev> URL: https://gitlist.dev/e/20190729093928.GP20404%40szeder.dev In-Reply-To: <9C668779-15AC-4099-AAFA-7FFF519D426F@puck.ch> ``` On Mon, Jul 29, 2019 at 09:58:52AM +0200, Olivier Bornet wrote: > I have a git repository with an error in a submodule path in the history. > The submodule path is “-f”, which is not allowed. But for some reason, it’s in the history of the git, and I’m trying to find a way to manage it without having to rewrite the full history on the main gitlab (if possible)... > > To reproduce this unwanted history: > After that, even if the git is working correctly, we have a “bad” history if we check with fsck: > > $ git fsck > Checking object directories: 100% (256/256), done. > error in blob 19a97d3b70760c74b780c8134e33f5392292c2e6: gitmodulesPath: disallowed submodule path: -f > > Is it possible to correct it? Must git handle this kind of errors? To correct without rewriting history, no. However, you can tell 'git fsck' to ignore it using the 'fsck.skipList' configuration variable (see 'git help config'; for some reason it's not included in 'git fsck's documentation): $ cat <.git-fsck-skiplist > # invalid submodule path > 19a97d3b70760c74b780c8134e33f5392292c2e6 > EOF $ git config fsck.skipList .git-fsck-skiplist $ git fsck Checking object directories: 100% (256/256), done. It may or may not be worth committing this file, I'm not quite sure what the best practice is. By committing it others don't have to maintain such a skiplist file themselves, though they still have to set the config variable. OTOH, if anyone sets this config variable and attempts to run 'git fsck' while on a branch that doesn't contain this file, then they will get a 'fatal: could not open object name list: .git-fsck-skiplist' error. And it won't help anyone cloning the repository with 'fetch.fsckObjects' enabled. ``` ## SZEDER Gábor, 2019-07-29 09:59 Subject: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables Message-ID: <20190729095914.30573-1-szeder.dev@gmail.com> URL: https://gitlist.dev/e/20190729095914.30573-1-szeder.dev%40gmail.com In-Reply-To: <20190729093928.GP20404@szeder.dev> ``` The 'fsck.skipList' and 'fsck.' config variables might be easier to discover when they are documented in 'git fsck's man page. Signed-off-by: SZEDER Gábor --- Documentation/git-fsck.txt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt index e0eae642c1..d72d15be5b 100644 --- a/Documentation/git-fsck.txt +++ b/Documentation/git-fsck.txt @@ -104,6 +104,11 @@ care about this output and want to speed it up further. progress status even if the standard error stream is not directed to a terminal. +CONFIGURATION +------------- + +include::config/fsck.txt[] + DISCUSSION ---------- -- 2.22.0.924.g7c03687a56 ``` ## Olivier Bornet, 2019-07-29 14:31 Subject: Re: Problem with fsck and invalid submodule path in history Message-ID: <10AB3C94-B14E-4F5F-AFC6-24F42A2FA5C1@puck.ch> URL: https://gitlist.dev/e/10AB3C94-B14E-4F5F-AFC6-24F42A2FA5C1%40puck.ch In-Reply-To: <20190729093928.GP20404@szeder.dev> ``` Hello, > On 29 Jul 2019, at 11:39, SZEDER Gábor wrote: > > However, you can tell 'git fsck' to ignore it using the > 'fsck.skipList' configuration variable (see 'git help config'; for > some reason it's not included in 'git fsck's documentation): Thanks for pointing it out. Thanks to it, we have managed to handle the error with our gitlab automatic fsck procedure. Have a nice day. -- Olivier Bornet Olivier.Bornet@puck.ch ``` ## Ævar Arnfjörð Bjarmason, 2019-07-29 15:33 Subject: Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables Message-ID: <87sgqo6ejk.fsf@evledraar.gmail.com> URL: https://gitlist.dev/e/87sgqo6ejk.fsf%40evledraar.gmail.com In-Reply-To: <20190729095914.30573-1-szeder.dev@gmail.com> ``` On Mon, Jul 29 2019, SZEDER Gábor wrote: > The 'fsck.skipList' and 'fsck.' config variables might be > easier to discover when they are documented in 'git fsck's man page. > > Signed-off-by: SZEDER Gábor > --- > Documentation/git-fsck.txt | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt > index e0eae642c1..d72d15be5b 100644 > --- a/Documentation/git-fsck.txt > +++ b/Documentation/git-fsck.txt > @@ -104,6 +104,11 @@ care about this output and want to speed it up further. > progress status even if the standard error stream is not > directed to a terminal. > > +CONFIGURATION > +------------- > + > +include::config/fsck.txt[] Before this include let's add: The below documentation is the same as what’s found in git-config(1): As I did for a similar change in git-gc in b6a8d09f6d ("gc docs: include the "gc.*" section from "config" in "gc"", 2019-04-07). Sometimes we repeat ourselves, it helps the reader to know this isn't some slightly different prose than what's in git-config. > + > DISCUSSION > ---------- ``` ## Junio C Hamano, 2019-07-29 15:48 Subject: Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables Message-ID: URL: https://gitlist.dev/e/xmqq36iox2nn.fsf%40gitster-ct.c.googlers.com In-Reply-To: <87sgqo6ejk.fsf@evledraar.gmail.com> ``` Ævar Arnfjörð Bjarmason writes: > On Mon, Jul 29 2019, SZEDER Gábor wrote: > >> The 'fsck.skipList' and 'fsck.' config variables might be >> easier to discover when they are documented in 'git fsck's man page. >> >> Signed-off-by: SZEDER Gábor >> --- >> Documentation/git-fsck.txt | 5 +++++ >> 1 file changed, 5 insertions(+) >> >> diff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt >> index e0eae642c1..d72d15be5b 100644 >> --- a/Documentation/git-fsck.txt >> +++ b/Documentation/git-fsck.txt >> @@ -104,6 +104,11 @@ care about this output and want to speed it up further. >> progress status even if the standard error stream is not >> directed to a terminal. >> >> +CONFIGURATION >> +------------- >> + >> +include::config/fsck.txt[] > > Before this include let's add: > > The below documentation is the same as what’s found in > git-config(1): I actually do not think we would want to do that. I am all for the kind of 'include' proposed by this patch, and we should strive to make it easier for us to make sure the duplicated text are in sync. But that would mean that the readers will have to see the "is the same as the other one" over and over. If our documentation set is consistent, they should not have to. I think we *must* make such a note in a total opposite case, i.e. "here are the summary of the most often used options; for full list, see git-config(1)". > As I did for a similar change in git-gc in b6a8d09f6d ("gc docs: include > the "gc.*" section from "config" in "gc"", 2019-04-07). Sometimes we > repeat ourselves, it helps the reader to know this isn't some slightly > different prose than what's in git-config. So, I think we should revert that part out of b6a8d09f6d, too. ``` ## Jeff King, 2019-07-29 20:12 Subject: Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables Message-ID: <20190729201236.GF14943@sigill.intra.peff.net> URL: https://gitlist.dev/e/20190729201236.GF14943%40sigill.intra.peff.net In-Reply-To: ``` On Mon, Jul 29, 2019 at 08:48:28AM -0700, Junio C Hamano wrote: > > Before this include let's add: > > > > The below documentation is the same as what’s found in > > git-config(1): > > I actually do not think we would want to do that. I am all for the > kind of 'include' proposed by this patch, and we should strive to > make it easier for us to make sure the duplicated text are in sync. > > But that would mean that the readers will have to see the "is the > same as the other one" over and over. If our documentation set is > consistent, they should not have to. > > I think we *must* make such a note in a total opposite case, > i.e. "here are the summary of the most often used options; for full > list, see git-config(1)". I disagree. _We_ know that the content is the same, because we are looking at the source that says "include". But as a user, how do I know when I get to one section or the other that it is something I have already read and can skip over? Perhaps if Git were entirely consistent here, they could make the assumption that "CONFIG" sections were always duplicated and know this already. But I think even that is asking a bit much. Unless they are intimately familiar with our documentation, they don't know that we are, in fact, consistent. And we are in an uphill battle with every other thing the user has read, which may not agree with our assumptions of consistency. ;) So IMHO it's worth leaving a note that guides the reader, as long as it's short (and I think this one is). That said, I think an even _better_ solution would be to avoid includes, and instead make it clear when we are pointing the user to shared content. Then we get them to the right place _and_ explicitly instruct them that concepts/content are shared. For config, for example, I've worked with a previous system that did something like: - include fsck.* documentation in the git-fsck manpage - make a master table of config options in git-config.1 with _just_ the names and the associated manpage where the definition can be found. This serves as an index if you don't know where to look. This would probably involve creating new concept-based pages for some of the groupings (e.g., where does "remote.*" config go?), but I think that would probably help round out our documentation (if there is a concept with related config options but we don't explain it anywhere, that is probably a gap we should fix). The biggest downside is that chasing down references in manpages sucks. For the HTML documentation we'd ideally hyperlink from the git-config.1 index into each definition, but there's no way to do that with a regular manpage. -Peff PS This is an approach I've advocated for a while: https://public-inbox.org/git/20110120233429.GB9442@sigill.intra.peff.net/ but haven't actually done much about, so perhaps I should be putting my money where my mouth is. ;) ``` ## Junio C Hamano, 2019-07-29 21:32 Subject: Re: [PATCH] Documentation/git-fsck.txt: include fsck.* config variables Message-ID: URL: https://gitlist.dev/e/xmqq5znkv86m.fsf%40gitster-ct.c.googlers.com In-Reply-To: <20190729201236.GF14943@sigill.intra.peff.net> ``` Jeff King writes: > On Mon, Jul 29, 2019 at 08:48:28AM -0700, Junio C Hamano wrote: > >> > Before this include let's add: >> > >> > The below documentation is the same as what’s found in >> > git-config(1): >> >> I actually do not think we would want to do that. I am all for the >> kind of 'include' proposed by this patch, and we should strive to >> make it easier for us to make sure the duplicated text are in sync. >> >> But that would mean that the readers will have to see the "is the >> same as the other one" over and over. If our documentation set is >> consistent, they should not have to. >> >> I think we *must* make such a note in a total opposite case, >> i.e. "here are the summary of the most often used options; for full >> list, see git-config(1)". > > I disagree. _We_ know that the content is the same, because we are > looking at the source that says "include". But as a user, how do I know > when I get to one section or the other that it is something I have > already read and can skip over? I want to raise the user expectation so that they would expect from our documentation, unless we say "these are different", we would never say conflicting things in two places. So,... I disagree. ```