git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 3/8] fsck: document and test sorted skipList input

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Aug 28, 2018, 09:52 UTC
Message-ID
<20180828095219.23296-4-avarab@gmail.com>
In-Reply-To
<87lg8refcr.fsf@evledraar.gmail.com>

Ever since the skipList support was first added in cd94c6f91 ("fsck: git receive-pack: support excluding objects from fsck'ing", 2015-06-22) the documentation for the format has that the file is a sorted list of object names.

Thus, anyone using the feature would have thought the list needed to be sorted. E.g. I recently in conjunction with my fetch.fsck.* implementation in 1362df0d41 ("fetch: implement fetch.fsck.*", 2018-07-27) wrote some code to ship a skipList, and went out of my way to sort it.

Doing so seems intuitive, since it contains fixed-width records, and has no support for comments, so one might expect it to be binary searched in-place on-disk.

However, as documented here this was never a requirement, so let's change the documentation. Since this is a file format change let's also document what was said about this in the past, so e.g. someone like myself reading the new docs can see this never needed to be sorted ("why do I have all this code to sort this thing...").

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 10 +++++++++-
 t/t5504-fetch-receive-strict.sh | 19 +++++++++++++++++++
 2 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index eb66a11975..b2fdbc6764 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1710,7 +1710,7 @@ doing the same for `receive.fsck.<msg-id>` and `fetch.fsck.<msg-id>`
 will only cause git to warn.
 
 fsck.skipList::
-	The path to a sorted list of object names (i.e. one SHA-1 per
+	The path to a list of object names (i.e. one SHA-1 per
 	line) that are known to be broken in a non-fatal way and should
 	be ignored. This feature is useful when an established project
 	should be accepted despite early commits containing errors that
@@ -1725,6 +1725,14 @@ Unlike variables like `color.ui` and `core.editor` the
 fall back on the `fsck.skipList` configuration if they aren't set. To
 uniformly configure the same fsck settings in different circumstances
 all three of them they must all set to the same values.
++
+Older versions of Git (before 2.20) documented that the object names
+list should be sorted. This was never a requirement, the object names
+can appear in any order, but when reading the list we track whether
+the list is sorted for the purposes of an internal binary search
+implementation, which can save itself some work with an already sorted
+list.  Unless you have a humongous list there's no reason to go out of
+your way to pre-sort the list.
 
 gc.aggressiveDepth::
 	The depth parameter used in the delta compression
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index cbae31f330..fa56052f0f 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -142,6 +142,25 @@ test_expect_success 'fsck with no skipList input' '
 	test_i18ngrep "missingEmail" err
 '
 
+test_expect_success 'setup sorted and unsorted skipLists' '
+	cat >SKIP.unsorted <<-EOF &&
+	0000000000000000000000000000000000000004
+	0000000000000000000000000000000000000002
+	$commit
+	0000000000000000000000000000000000000001
+	0000000000000000000000000000000000000003
+	EOF
+	sort SKIP.unsorted >SKIP.sorted
+'
+
+test_expect_success 'fsck with sorted skipList' '
+	git -c fsck.skipList=SKIP.sorted fsck
+'
+
+test_expect_success 'fsck with unsorted skipList' '
+	git -c fsck.skipList=SKIP.unsorted fsck
+'
+
 test_expect_success 'fsck with invalid or bogus skipList input' '
 	git -c fsck.skipList=/dev/null -c fsck.missingEmail=ignore fsck &&
 	test_must_fail git -c fsck.skipList=does-not-exist -c fsck.missingEmail=ignore fsck 2>err &&
-- 
2.19.0.rc0.228.g281dcd1b4d0
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 12 of 32 in “use oidset for skiplist + docs + tests + comment support”
  1. 0/7 use oidset for skiplist + docs + tests + comment supportÆvar Arnfjörð Bjarmason, Aug 27, 2018
  2. 1/7 fsck tests: setup of bogus commit objectÆvar Arnfjörð Bjarmason, Aug 27, 2018
  3. 2/7 fsck tests: add a test for no skipList inputÆvar Arnfjörð Bjarmason, Aug 27, 2018
  4. 3/7 fsck: document and test sorted skipList inputÆvar Arnfjörð Bjarmason, Aug 27, 2018
  5. 4/7 fsck: document and test commented & empty line skipList inputÆvar Arnfjörð Bjarmason, Aug 27, 2018
  6. 5/7 fsck: use strbuf_getline() to read skiplist fileÆvar Arnfjörð Bjarmason, Aug 27, 2018
  7. 6/7 fsck: use oidset for skiplistÆvar Arnfjörð Bjarmason, Aug 27, 2018
  8. Ævar Arnfjörð BjarmasonAug 27, 2018
  9. 1/8 fsck tests: setup of bogus commit objectÆvar Arnfjörð Bjarmason, Aug 28, 2018
  10. 0/8 use oidset for skiplist + docs + tests + comment supportÆvar Arnfjörð Bjarmason, Aug 28, 2018
  11. 2/8 fsck tests: add a test for no skipList inputÆvar Arnfjörð Bjarmason, Aug 28, 2018
  12. 3/8 fsck: document and test sorted skipList inputÆvar Arnfjörð Bjarmason, Aug 28, 2018
  13. 4/8 fsck: document and test commented & empty line skipList inputÆvar Arnfjörð Bjarmason, Aug 28, 2018
  14. 5/8 fsck: add a performance test for skipListÆvar Arnfjörð Bjarmason, Aug 28, 2018
  15. 6/8 fsck: use strbuf_getline() to read skiplist fileÆvar Arnfjörð Bjarmason, Aug 28, 2018
  16. 7/8 fsck: use oidset instead of oid_array for skipListÆvar Arnfjörð Bjarmason, Aug 28, 2018
  17. 8/8 fsck: support comments & empty lines in skipListÆvar Arnfjörð Bjarmason, Aug 28, 2018
  18. René ScharfeAug 28, 2018
  19. 00/10 use oidset for skiplist + docs + tests + comment supportÆvar Arnfjörð Bjarmason, Sep 3, 2018
  20. 01/10 fsck tests: setup of bogus commit objectÆvar Arnfjörð Bjarmason, Sep 3, 2018
  21. 02/10 fsck tests: add a test for no skipList inputÆvar Arnfjörð Bjarmason, Sep 3, 2018
  22. 03/10 fsck: document and test sorted skipList inputÆvar Arnfjörð Bjarmason, Sep 3, 2018
  23. 04/10 fsck: document and test commented & empty line skipList inputÆvar Arnfjörð Bjarmason, Sep 3, 2018
  24. 05/10 fsck: document that skipList input must be unabbreviatedÆvar Arnfjörð Bjarmason, Sep 3, 2018
  25. 06/10 fsck: add a performance testÆvar Arnfjörð Bjarmason, Sep 3, 2018
  26. 08/10 fsck: use strbuf_getline() to read skiplist fileÆvar Arnfjörð Bjarmason, Sep 3, 2018
  27. 09/10 fsck: use oidset instead of oid_array for skipListÆvar Arnfjörð Bjarmason, Sep 3, 2018
  28. 10/10 fsck: support comments & empty lines in skipListÆvar Arnfjörð Bjarmason, Sep 3, 2018
  29. 07/10 fsck: add a performance test for skipListÆvar Arnfjörð Bjarmason, Sep 3, 2018
  30. René ScharfeAug 28, 2018
  31. 7/7 fsck: support comments & empty lines in skipListÆvar Arnfjörð Bjarmason, Aug 27, 2018
  32. Ævar Arnfjörð BjarmasonAug 27, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.