Re: [PATCH v4 8/8] fsck: support comments & empty lines in skipList

From: René Scharfe <l.s.r@web.de>
Date: Aug 28, 2018, 14:59 UTC
Message-ID: <2b31e12e-20e9-3d08-58bd-977f8b83e0a7@web.de>
In-Reply-To: <20180828095219.23296-9-avarab@gmail.com>

Am 28.08.2018 um 11:52 schrieb Ævar Arnfjörð Bjarmason:

Show 55 quoted lines

> It's annoying not to be able to put comments and empty lines in the
> skipList, when e.g. keeping a big central list of commits to skip in
> /etc/gitconfig, which was my motivation for 1362df0d41 ("fetch:
> implement fetch.fsck.*", 2018-07-27).
> 
> Implement that, and document what version of Git this was changed in,
> since this on-disk format can be expected to be used by multiple
> versions of git.
> 
> There is no notable performance impact from this change, using the
> test setup described a couple of commist back:
> 
>     Test                                             HEAD~             HEAD
>     ----------------------------------------------------------------------------------------
>     1450.3: fsck with 0 skipped bad commits          7.81(7.42+0.39)   7.72(7.34+0.38) -1.2%
>     1450.5: fsck with 1 skipped bad commits          7.75(7.36+0.38)   7.66(7.26+0.39) -1.2%
>     1450.7: fsck with 10 skipped bad commits         7.81(7.43+0.38)   7.70(7.30+0.39) -1.4%
>     1450.9: fsck with 100 skipped bad commits        7.85(7.42+0.42)   7.73(7.31+0.41) -1.5%
>     1450.11: fsck with 1000 skipped bad commits      7.81(7.43+0.38)   7.84(7.46+0.38) +0.4%
>     1450.13: fsck with 10000 skipped bad commits     7.87(7.47+0.40)   7.86(7.46+0.40) -0.1%
>     1450.15: fsck with 100000 skipped bad commits    7.77(7.39+0.38)   7.83(7.48+0.34) +0.8%
>     1450.17: fsck with 1000000 skipped bad commits   7.17(6.92+0.24)   7.11(6.85+0.26) -0.8%
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  Documentation/config.txt        | 4 ++--
>  fsck.c                          | 2 ++
>  t/t5504-fetch-receive-strict.sh | 6 +++---
>  3 files changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index ebaa044689..824634c412 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -1712,8 +1712,8 @@ will only cause git to warn.
>  fsck.skipList::
>  	The path to a list of object names (i.e. one SHA-1 per
>  	line) that are known to be broken in a non-fatal way and should
> -	be ignored. Comments ('#') and empty lines are not supported, and
> -	will error out.
> +	be ignored. On versions of Git 2.20 and later comments ('#') and empty
> +	lines are ignored, but will error out on older versions.
>  +
>  This feature is useful when an established project should be accepted
>  despite early commits containing errors that can be safely ignored
> diff --git a/fsck.c b/fsck.c
> index 4c643f1d40..589548308a 100644
> --- a/fsck.c
> +++ b/fsck.c
> @@ -190,6 +190,8 @@ static void init_skiplist(struct fsck_options *options, const char *path)
>  		die("Could not open skip list: %s", path);
>  	while (!strbuf_getline(&sb, fp)) {
>  		const char *p;
> +		if (!strcmp(sb.buf, "") || starts_with(sb.buf, "#"))
> +			continue;

Checking sb.len == 0 is simpler and might be slightly quicker.

But what is an empty line? Shouldn't whitespace-only lines qualify? And why not allow trailing comments, while we're at it? I.e. what about something like this instead?

		const char *hash = strchr(sb.buf, '#);
		if (hash)
			strbuf_setlen(&sb, hash - sb.buf);
		strbuf_rtrim(&sb);
		if (!sb.len)
			continue;

Too much?

Show 32 quoted lines

>  		if (parse_oid_hex(sb.buf, &oid, &p) || *p != '\0')
>  			die("Invalid SHA-1: %s", sb.buf);
>  		oidset_insert(&options->skiplist, &oid);
> diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
> index c7224db3bb..a1bac164d1 100755
> --- a/t/t5504-fetch-receive-strict.sh
> +++ b/t/t5504-fetch-receive-strict.sh
> @@ -169,20 +169,20 @@ test_expect_success 'fsck with invalid or bogus skipList input' '
>  	test_i18ngrep "Invalid SHA-1: \[core\]" err
>  '
>  
> -test_expect_success 'fsck with invalid or bogus skipList input (comments & empty lines)' '
> +test_expect_success 'fsck with other accepted skipList input (comments & empty lines)' '
>  	cat >SKIP.with-comment <<-EOF &&
>  	# Some bad commit
>  	0000000000000000000000000000000000000001
>  	EOF
>  	test_must_fail git -c fsck.skipList=SKIP.with-comment fsck 2>err-with-comment &&
> -	test_i18ngrep "^fatal: Invalid SHA-1: # Some bad commit$" err-with-comment &&
> +	test_i18ngrep "missingEmail" err-with-comment &&
>  	cat >SKIP.with-empty-line <<-EOF &&
>  	0000000000000000000000000000000000000001
>  
>  	0000000000000000000000000000000000000002
>  	EOF
>  	test_must_fail git -c fsck.skipList=SKIP.with-empty-line fsck 2>err-with-empty-line &&
> -	test_i18ngrep "^fatal: Invalid SHA-1: " err-with-empty-line
> +	test_i18ngrep "missingEmail" err-with-empty-line
>  '
>  
>  test_expect_success 'fsck no garbage output from comments & empty lines errors' '
>