git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 8/8] fsck: support comments & empty lines in skipList

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Aug 28, 2018, 09:52 UTC
Message-ID
<20180828095219.23296-9-avarab@gmail.com>
In-Reply-To
<87lg8refcr.fsf@evledraar.gmail.com>

It's annoying not to be able to put comments and empty lines in the skipList, when e.g. keeping a big central list of commits to skip in /etc/gitconfig, which was my motivation for 1362df0d41 ("fetch: implement fetch.fsck.*", 2018-07-27).

Implement that, and document what version of Git this was changed in, since this on-disk format can be expected to be used by multiple versions of git.

There is no notable performance impact from this change, using the test setup described a couple of commist back:

    Test                                             HEAD~             HEAD
    ----------------------------------------------------------------------------------------
    1450.3: fsck with 0 skipped bad commits          7.81(7.42+0.39)   7.72(7.34+0.38) -1.2%
    1450.5: fsck with 1 skipped bad commits          7.75(7.36+0.38)   7.66(7.26+0.39) -1.2%
    1450.7: fsck with 10 skipped bad commits         7.81(7.43+0.38)   7.70(7.30+0.39) -1.4%
    1450.9: fsck with 100 skipped bad commits        7.85(7.42+0.42)   7.73(7.31+0.41) -1.5%
    1450.11: fsck with 1000 skipped bad commits      7.81(7.43+0.38)   7.84(7.46+0.38) +0.4%
    1450.13: fsck with 10000 skipped bad commits     7.87(7.47+0.40)   7.86(7.46+0.40) -0.1%
    1450.15: fsck with 100000 skipped bad commits    7.77(7.39+0.38)   7.83(7.48+0.34) +0.8%
    1450.17: fsck with 1000000 skipped bad commits   7.17(6.92+0.24)   7.11(6.85+0.26) -0.8%
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 4 ++--
 fsck.c                          | 2 ++
 t/t5504-fetch-receive-strict.sh | 6 +++---
 3 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index ebaa044689..824634c412 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1712,8 +1712,8 @@ will only cause git to warn.
 fsck.skipList::
 	The path to a list of object names (i.e. one SHA-1 per
 	line) that are known to be broken in a non-fatal way and should
-	be ignored. Comments ('#') and empty lines are not supported, and
-	will error out.
+	be ignored. On versions of Git 2.20 and later comments ('#') and empty
+	lines are ignored, but will error out on older versions.
 +
 This feature is useful when an established project should be accepted
 despite early commits containing errors that can be safely ignored
diff --git a/fsck.c b/fsck.c
index 4c643f1d40..589548308a 100644
--- a/fsck.c
+++ b/fsck.c
@@ -190,6 +190,8 @@ static void init_skiplist(struct fsck_options *options, const char *path)
 		die("Could not open skip list: %s", path);
 	while (!strbuf_getline(&sb, fp)) {
 		const char *p;
+		if (!strcmp(sb.buf, "") || starts_with(sb.buf, "#"))
+			continue;
 		if (parse_oid_hex(sb.buf, &oid, &p) || *p != '\0')
 			die("Invalid SHA-1: %s", sb.buf);
 		oidset_insert(&options->skiplist, &oid);
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index c7224db3bb..a1bac164d1 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -169,20 +169,20 @@ test_expect_success 'fsck with invalid or bogus skipList input' '
 	test_i18ngrep "Invalid SHA-1: \[core\]" err
 '
 
-test_expect_success 'fsck with invalid or bogus skipList input (comments & empty lines)' '
+test_expect_success 'fsck with other accepted skipList input (comments & empty lines)' '
 	cat >SKIP.with-comment <<-EOF &&
 	# Some bad commit
 	0000000000000000000000000000000000000001
 	EOF
 	test_must_fail git -c fsck.skipList=SKIP.with-comment fsck 2>err-with-comment &&
-	test_i18ngrep "^fatal: Invalid SHA-1: # Some bad commit$" err-with-comment &&
+	test_i18ngrep "missingEmail" err-with-comment &&
 	cat >SKIP.with-empty-line <<-EOF &&
 	0000000000000000000000000000000000000001
 
 	0000000000000000000000000000000000000002
 	EOF
 	test_must_fail git -c fsck.skipList=SKIP.with-empty-line fsck 2>err-with-empty-line &&
-	test_i18ngrep "^fatal: Invalid SHA-1: " err-with-empty-line
+	test_i18ngrep "missingEmail" err-with-empty-line
 '
 
 test_expect_success 'fsck no garbage output from comments & empty lines errors' '
-- 
2.19.0.rc0.228.g281dcd1b4d0
Previous: Ævar Arnfjörð BjarmasonNext: René Scharfe
Message 17 of 32 in “use oidset for skiplist + docs + tests + comment support”
  1. 0/7 use oidset for skiplist + docs + tests + comment supportÆvar Arnfjörð Bjarmason, Aug 27, 2018
  2. 1/7 fsck tests: setup of bogus commit objectÆvar Arnfjörð Bjarmason, Aug 27, 2018
  3. 2/7 fsck tests: add a test for no skipList inputÆvar Arnfjörð Bjarmason, Aug 27, 2018
  4. 3/7 fsck: document and test sorted skipList inputÆvar Arnfjörð Bjarmason, Aug 27, 2018
  5. 4/7 fsck: document and test commented & empty line skipList inputÆvar Arnfjörð Bjarmason, Aug 27, 2018
  6. 5/7 fsck: use strbuf_getline() to read skiplist fileÆvar Arnfjörð Bjarmason, Aug 27, 2018
  7. 6/7 fsck: use oidset for skiplistÆvar Arnfjörð Bjarmason, Aug 27, 2018
  8. Ævar Arnfjörð BjarmasonAug 27, 2018
  9. 1/8 fsck tests: setup of bogus commit objectÆvar Arnfjörð Bjarmason, Aug 28, 2018
  10. 0/8 use oidset for skiplist + docs + tests + comment supportÆvar Arnfjörð Bjarmason, Aug 28, 2018
  11. 2/8 fsck tests: add a test for no skipList inputÆvar Arnfjörð Bjarmason, Aug 28, 2018
  12. 3/8 fsck: document and test sorted skipList inputÆvar Arnfjörð Bjarmason, Aug 28, 2018
  13. 4/8 fsck: document and test commented & empty line skipList inputÆvar Arnfjörð Bjarmason, Aug 28, 2018
  14. 5/8 fsck: add a performance test for skipListÆvar Arnfjörð Bjarmason, Aug 28, 2018
  15. 6/8 fsck: use strbuf_getline() to read skiplist fileÆvar Arnfjörð Bjarmason, Aug 28, 2018
  16. 7/8 fsck: use oidset instead of oid_array for skipListÆvar Arnfjörð Bjarmason, Aug 28, 2018
  17. 8/8 fsck: support comments & empty lines in skipListÆvar Arnfjörð Bjarmason, Aug 28, 2018
  18. René ScharfeAug 28, 2018
  19. 00/10 use oidset for skiplist + docs + tests + comment supportÆvar Arnfjörð Bjarmason, Sep 3, 2018
  20. 01/10 fsck tests: setup of bogus commit objectÆvar Arnfjörð Bjarmason, Sep 3, 2018
  21. 02/10 fsck tests: add a test for no skipList inputÆvar Arnfjörð Bjarmason, Sep 3, 2018
  22. 03/10 fsck: document and test sorted skipList inputÆvar Arnfjörð Bjarmason, Sep 3, 2018
  23. 04/10 fsck: document and test commented & empty line skipList inputÆvar Arnfjörð Bjarmason, Sep 3, 2018
  24. 05/10 fsck: document that skipList input must be unabbreviatedÆvar Arnfjörð Bjarmason, Sep 3, 2018
  25. 06/10 fsck: add a performance testÆvar Arnfjörð Bjarmason, Sep 3, 2018
  26. 08/10 fsck: use strbuf_getline() to read skiplist fileÆvar Arnfjörð Bjarmason, Sep 3, 2018
  27. 09/10 fsck: use oidset instead of oid_array for skipListÆvar Arnfjörð Bjarmason, Sep 3, 2018
  28. 10/10 fsck: support comments & empty lines in skipListÆvar Arnfjörð Bjarmason, Sep 3, 2018
  29. 07/10 fsck: add a performance test for skipListÆvar Arnfjörð Bjarmason, Sep 3, 2018
  30. René ScharfeAug 28, 2018
  31. 7/7 fsck: support comments & empty lines in skipListÆvar Arnfjörð Bjarmason, Aug 27, 2018
  32. Ævar Arnfjörð BjarmasonAug 27, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.