{"thread":{"id":"64240","subject":"[PATCH v2 1/4] doc: add some missing technical documents","startedAt":"2025-10-02T22:13:13Z","lastAt":"2025-10-23T20:09:39Z","messageCount":25,"participants":["Ramsay Jones","Kristoffer Haugsbakk","Elijah Newren","Patrick Steinhardt","Junio C Hamano"],"isPatch":true,"patchVersion":2,"patchTotal":4},"messages":[{"id":"527834","messageId":"20251002221233.541844-1-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"https://lore.kernel.org/git/bcb3b3a3-bb13-4808-9363-442b5f9be05f@ramsayjones.plus.com/","subject":"[PATCH v2 0/4] technical docs in make build","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-02T22:12:12Z","receivedAt":"2025-10-02T22:13:13Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"OK, so I have recently developed an intense dislike of both asciidoc\nand asciidoctor. :)\n\nChanges in v2:\n\n - Actual commit messages\n - (almost) total re-write of patches #2 and #3\n - removed the RFC from patches #2->#4\n\nI have not included a range-diff, because it doesn't show anything\ninteresting/readable with or without a large --creation-factor!\n\nThere are two issues I am aware of:\n\n - mis-formatting of monospaced text containing an '{' character\n mentioned in the original cover letter below. I have not found\n a fix for this, but there are other examples in patch #3!\n - breakage of two html links representing URLS pointing to emails\n at 'lore.kernel.org'. I don't think it is a coincidence that it\n is only these two references that contain a reserved character;\n a '+' in the first (see known bugs 7) and two (separate) '='\n characters in the second (mail ref [13]). I tried %encoding\n them, but that didn't make any difference.\n\nThere are probably other formatting issues that I am not aware of!\n\n\nOriginal cover letter:\n\nI have been trying to get back to the 'misc build updates (part #3)'\npatches, so that I can send them to the list, but I have not been able\nto find a spare minute for quite some time. :(\n\nHowever, this sub-sequence of patches hangs together as a single theme and\nI need help to finish them up! (asciidoc is not my forte).\n\nThe first patch adds some technical documents to the Makefile build which\nare already part of the meson build. In particular, the following are\nbuilt by meson, but not by the Makefile:\n\n commit-graph.adoc\n directory-rename-detection.adoc\n packfile-uri.adoc\n remembering-renames.adoc\n repository-version.adoc\n rerere.adoc\n sparse-checkout.adoc\n sparse-index.adoc\n\nAlthough I am not convinced that some of these files were ever meant to be\nformatted by asciidoc, I have assumed that is the case for the purposes of\nthis patch series. (otherwise, we should remove them from the meson build\nand rename the files instead).\n\nWhen I attempt to build the html docs, with patch #1 applied, on Linux:\n\n $ make html >out-doc 2>&1\n\n $ grep SyntaxWarning out-doc | head -n1\n :1: SyntaxWarning: invalid escape sequence '\\S'\n $ grep SyntaxWarning out-doc | wc -l\n 524\n $ \n\n $ asciidoc --version\n asciidoc 10.2.0\n $ python3 --version\n Python 3.12.3\n $ \n\nThis is caused by the python version I am using, which was recently changed\n(in version 3.12) to issue the SyntaxWarning when a 'non-raw' string contains\nsome escape sequences (here \\S). [some versions prior to 3.12 used to issue\na deprecation warning].\n\nThis is a known issue, see e.g. [0], which has been addressed by a patch [1],\nand as seen in [2] has been included in a new version 10.2.1 of asciidoc.\n\n [0] https://trac.macports.org/ticket/70039\n [1] 1https://github.com/asciidoc-py/asciidoc-py/pull/267\n [2] https://github.com/asciidoc-py/asciidoc-py/commits/main/\n\n[cygwin does not have this problem, because the phython version is 3.9.16]\n\nSo, ignoring that issue, we still see some warnings from asciidoc:\n\n $ grep WARNING out-doc\n asciidoc: WARNING: remembering-renames.adoc: line 13: list item index: expected 1 got 0\n asciidoc: WARNING: remembering-renames.adoc: line 15: list item index: expected 2 got 1\n asciidoc: WARNING: remembering-renames.adoc: line 17: list item index: expected 3 got 2\n asciidoc: WARNING: remembering-renames.adoc: line 20: list item index: expected 4 got 3\n asciidoc: WARNING: remembering-renames.adoc: line 23: list item index: expected 5 got 4\n asciidoc: WARNING: remembering-renames.adoc: line 25: list item index: expected 6 got 5\n asciidoc: WARNING: remembering-renames.adoc: line 29: list item index: expected 7 got 6\n asciidoc: WARNING: remembering-renames.adoc: line 31: list item index: expected 8 got 7\n asciidoc: WARNING: remembering-renames.adoc: line 33: list item index: expected 9 got 8\n asciidoc: WARNING: remembering-renames.adoc: line 38: section title out of sequence: expected level 1, got level 2\n asciidoc: WARNING: sparse-checkout.adoc: line 17: section title out of sequence: expected level 1, got level 2\n asciidoc: WARNING: sparse-checkout.adoc: line 928: list item index: expected 1 got 0\n asciidoc: WARNING: sparse-checkout.adoc: line 931: list item index: expected 2 got 1\n asciidoc: WARNING: sparse-checkout.adoc: line 951: list item index: expected 3 got 2\n asciidoc: WARNING: sparse-checkout.adoc: line 974: list item index: expected 4 got 3\n asciidoc: WARNING: sparse-checkout.adoc: line 980: list item index: expected 5 got 4\n asciidoc: WARNING: sparse-checkout.adoc: line 1033: list item index: expected 6 got 5\n asciidoc: WARNING: sparse-checkout.adoc: line 1049: list item index: expected 7 got 6\n $ \n\nI also tried asciidoctor, just for fun:\n\n $ asciidoctor --version\n Asciidoctor 2.0.20 [https://asciidoctor.org]\n Runtime Environment (ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [x86_64-linux-gnu]) (lc:UTF-8 fs:UTF-8 in:UTF-8 ex:UTF-8)\n $ \n\n $ make USE_ASCIIDOCTOR=1 html >out-doctor 2>&1\n\n $ grep WARNING out-doctor\n asciidoctor: WARNING: remembering-renames.adoc: line 13: list item index: expected 1, got 0\n asciidoctor: WARNING: remembering-renames.adoc: line 15: list item index: expected 2, got 1\n asciidoctor: WARNING: remembering-renames.adoc: line 17: list item index: expected 3, got 2\n asciidoctor: WARNING: remembering-renames.adoc: line 20: list item index: expected 4, got 3\n asciidoctor: WARNING: remembering-renames.adoc: line 23: list item index: expected 5, got 4\n asciidoctor: WARNING: remembering-renames.adoc: line 25: list item index: expected 6, got 5\n asciidoctor: WARNING: remembering-renames.adoc: line 29: list item index: expected 7, got 6\n asciidoctor: WARNING: remembering-renames.adoc: line 31: list item index: expected 8, got 7\n asciidoctor: WARNING: remembering-renames.adoc: line 33: list item index: expected 9, got 8\n asciidoctor: WARNING: remembering-renames.adoc: line 38: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 94: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 141: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 142: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 184: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 185: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 257: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 288: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 289: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 290: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 397: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 424: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 485: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 486: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 487: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 17: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 95: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 258: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 303: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 316: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 545: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 612: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 752: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 824: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 895: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 923: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 928: list item index: expected 1, got 0\n asciidoctor: WARNING: sparse-checkout.adoc: line 931: list item index: expected 2, got 1\n asciidoctor: WARNING: sparse-checkout.adoc: line 951: list item index: expected 3, got 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 974: list item index: expected 4, got 3\n asciidoctor: WARNING: sparse-checkout.adoc: line 980: list item index: expected 5, got 4\n asciidoctor: WARNING: sparse-checkout.adoc: line 1033: list item index: expected 6, got 5\n asciidoctor: WARNING: sparse-checkout.adoc: line 1049: list item index: expected 7, got 6\n asciidoctor: WARNING: sparse-checkout.adoc: line 1053: section title out of sequence: expected level 1, got level 2\n $ \n\nYou can see that asciidoc only complains about the first 'section title out of\nsequence', whereas asciidoctor complains about them all.\n\n[asciidoctor also reports:\nNote: namesp. cut : stripped namespace before processing Git User Manual]\n\nPatch #2 was a nightmare which I really gave up on! :) An early attempt\ninvolved renumbering the 'outline list' at the top from 0->8 to 1->9\n(I thought there was a way to start numbering at zero, but I lost a lot\nof time trying to do so, without any success). So, of course I 'just'\ntried global search/replace in vim to do the renumbering (backwards).\nThis was a complete disaster (of course), which I 'fixed' many many times.\n(Not everything which is numbered is a section, there are 'cases' as well).\n\nIn the end, I just disabled the 'outline' list, by removing the period\non the numbers (again '0\\. Assumptions' should have worked, but didn't)\nand fixing up the section titles without renumbering them. Note that\nasciidoctor mis-formats the 'ascii branch diagrams', which asciidoc\nformats correctly. I think there are other formatting problems left.\n\nIn patch #3, the formatting changes are confined to the section titles and\nrenumbering the 'known bugs' from 0->6 to 1->7. (I think I noticed some\nsub-sub lists which are not formatted correctly, but I don't seem to be\nable to see them now ...).\n\nIn patch #4, most of the formatting changes relate to section titles, but\nI could not fix some inline text formatting starting at 'File Layouts'\n(within 'Commit-Graph Chains') with text that is monospaced with `` but\nalso contains an '{' character. For example:\n\n `$OBJDIR/info/commit-graphs/graph-{hash}.graph`\n\nis monospaced (blue colour with asciidoc) up until the {hash}.graph which\ndoes not have any formatting. (It is not so noticeable with asciidoctor\nbecause the formatting consists of a *very* subtle gray background to the\ntext which, to my eyes anyway, is almost not visible).\n\nI have tried several suggestions from an on-line asciidoc syntax cheatsheet\nsuch as:\n\n `$OBJDIR/info/commit-graphs/graph-\\{hash\\}.graph`\n `+$OBJDIR/info/commit-graphs/graph-{hash}.graph+`\n\nbut nothing worked. Note that there are many similar instances of this\nproblem (including just `{hash}`).\n\nNote also that asciidoctor did not render the second diagram correctly\n(the one in 'Merging commit-graph files'), but asciidoc was just fine.\n\nThe remaining documents:\n\n directory-rename-detection.adoc\n packfile-uri.adoc\n repository-version.adoc\n rerere.adoc\n sparse-index.adoc\n\nall appear to be formatted correctly.\n\nSo, I really need help with the asciidoc formatting, in patches #2->#4,\nwhich I am marking as RFC. Having said that, these patches represent\nan improvement over the existing documents in terms of formatting\n(just not by much!).\n\nAny help fixing up these patches would be much appreciated. :)\n\nThanks.\n\nATB,\nRamsay Jones\n\nRamsay Jones (4):\n doc: add some missing technical documents\n doc: remembering-renames.adoc: fix asciidoc warnings\n doc: sparse-checkout.adoc: fix asciidoc warnings\n doc: commit-graph.adoc: fix up some formatting\n\n Documentation/Makefile | 8 +\n Documentation/technical/commit-graph.adoc | 29 +-\n .../technical/remembering-renames.adoc | 120 +--\n Documentation/technical/sparse-checkout.adoc | 704 ++++++++++--------\n 4 files changed, 481 insertions(+), 380 deletions(-)\n\n-- \n2.51.0\n\n"},{"id":"527833","messageId":"20251002221233.541844-2-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251002221233.541844-1-ramsay@ramsayjones.plus.com","subject":"[PATCH v2 1/4] doc: add some missing technical documents","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-02T22:12:13Z","receivedAt":"2025-10-02T22:13:14Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Commit bcf7edee09 (\"meson: generate articles\", 2024-12-27) added the\ngeneration of the 'howto' and 'technical' documents to the meson build.\nAt this time those documents had a '*.txt' file extension, but they were\nrenamed with an '*.adoc' extension by commit 1f010d6bdf (\"doc: use .adoc\nextension for AsciiDoc files\", 2025-01-20), for the most part. For the\nmeson build, commit 87eccc3a81 (\"meson: fix building technical and howto\ndocs\", 2025-03-02) fixed the meson.build files, which had not been\nupdated when the files were renamed.\n\nHowever, the 'Documentation/Makefile' has not been updated to include\nall of the recently added technical documents. In particular, the\nfollowing are built by meson, but not by the Makefile:\n\n commit-graph.adoc\n directory-rename-detection.adoc\n packfile-uri.adoc\n remembering-renames.adoc\n repository-version.adoc\n rerere.adoc\n sparse-checkout.adoc\n sparse-index.adoc\n\nIn order to ensure that both build systems format the same technical\ndocuments, add the above documents to the TECH_DOCS variable in the\nDocumentation/Makefile.\n\nSigned-off-by: Ramsay Jones \n---\n Documentation/Makefile | 8 ++++++++\n 1 file changed, 8 insertions(+)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 6fb83d0c6e..a3fbd29744 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -119,18 +119,26 @@ TECH_DOCS += ToolsForGit\n TECH_DOCS += technical/bitmap-format\n TECH_DOCS += technical/build-systems\n TECH_DOCS += technical/bundle-uri\n+TECH_DOCS += technical/commit-graph\n+TECH_DOCS += technical/directory-rename-detection\n TECH_DOCS += technical/hash-function-transition\n TECH_DOCS += technical/long-running-process-protocol\n TECH_DOCS += technical/multi-pack-index\n+TECH_DOCS += technical/packfile-uri\n TECH_DOCS += technical/pack-heuristics\n TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/platform-support\n TECH_DOCS += technical/racy-git\n TECH_DOCS += technical/reftable\n+TECH_DOCS += technical/remembering-renames\n+TECH_DOCS += technical/repository-version\n+TECH_DOCS += technical/rerere\n TECH_DOCS += technical/scalar\n TECH_DOCS += technical/send-pack-pipeline\n TECH_DOCS += technical/shallow\n+TECH_DOCS += technical/sparse-checkout\n+TECH_DOCS += technical/sparse-index\n TECH_DOCS += technical/trivial-merge\n TECH_DOCS += technical/unit-tests\n SP_ARTICLES += $(TECH_DOCS)\n-- \n2.51.0\n\n"},{"id":"527835","messageId":"20251002221233.541844-3-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251002221233.541844-1-ramsay@ramsayjones.plus.com","subject":"[PATCH v2 2/4] doc: remembering-renames.adoc: fix asciidoc warnings","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-02T22:12:14Z","receivedAt":"2025-10-02T22:13:22Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Both asciidoc and ascidoctor issue warnings about 'list item index:\nexpected n got n-1' for n=1->9 on lines 13, 15, 17, 20, 23, 25, 29,\n31 and 33. In asciidoc, numbered lists must start at one, whereas this\nfile has a list starting at zero. Also, asciidoc and asciidoctor warn\nabout 'section title out of sequence: expected level 1, got level 2'\non line 38. (asciidoc only complains about the first instance of this,\nwhile asciidoctor complains about them all, on lines 94, 141, 142,\n184, 185, 257, 288, 289, 290, 397, 424, 485, 486 and 487). These\nwarnings stem from the section titles not being correctly nested within\na document/chapter title.\n\nIn order to address the first set of warnings, simply renumber the list\nfrom one to nine, rather than zero to eight. This also requires altering\nthe text which refers to the section numbers, including other section\ntitles.\n\nIn order to address the second set of warnings, change the section title\nsyntax from '=== title ===' to '== title ==', effectively reducing the\nnesting level of the title by one. Also, some of the titles are given\nover multiple lines (they are very long), with an title '===' prefix\non each line. This leads to them being treated as separate sections\nwith no body text (as you can see from the line numbers given for the\nasciidoctor warnings, above). So, for these titles, turn them into a\nsingle (long) line of text.\n\nIn addition to the warnings, address some other formatting issues:\n\n - the ascii branch diagrams didn't format correctly on asciidoctor\n so include them in a literal block.\n - several blocks of text were intended to be formatted 'as is' but\n were not included in a literal block.\n - in section 8, format the (A)->(D) in the text description as a\n literal with `` marks, since (C) is rendered as a copyright\n symbol in html otherwise.\n - in section 9, a sub-list of two items is not formatted as such.\n change the '*' introducer to '**' to correct the sub-list format.\n\nSigned-off-by: Ramsay Jones \n---\n .../technical/remembering-renames.adoc | 120 ++++++++++++------\n 1 file changed, 78 insertions(+), 42 deletions(-)\n\ndiff --git a/Documentation/technical/remembering-renames.adoc b/Documentation/technical/remembering-renames.adoc\nindex 73f41761e2..6155f36c72 100644\n--- a/Documentation/technical/remembering-renames.adoc\n+++ b/Documentation/technical/remembering-renames.adoc\n@@ -10,32 +10,32 @@ history as an optimization, assuming all merges are automatic and clean\n \n Outline:\n \n- 0. Assumptions\n+ 1. Assumptions\n \n- 1. How rebasing and cherry-picking work\n+ 2. How rebasing and cherry-picking work\n \n- 2. Why the renames on MERGE_SIDE1 in any given pick are *always* a\n+ 3. Why the renames on MERGE_SIDE1 in any given pick are *always* a\n superset of the renames on MERGE_SIDE1 for the next pick.\n \n- 3. Why any rename on MERGE_SIDE1 in any given pick is _almost_ always also\n+ 4. Why any rename on MERGE_SIDE1 in any given pick is _almost_ always also\n a rename on MERGE_SIDE1 for the next pick\n \n- 4. A detailed description of the counter-examples to #3.\n+ 5. A detailed description of the counter-examples to #4.\n \n- 5. Why the special cases in #4 are still fully reasonable to use to pair\n+ 6. Why the special cases in #5 are still fully reasonable to use to pair\n up files for three-way content merging in the merge machinery, and why\n they do not affect the correctness of the merge.\n \n- 6. Interaction with skipping of \"irrelevant\" renames\n+ 7. Interaction with skipping of \"irrelevant\" renames\n \n- 7. Additional items that need to be cached\n+ 8. Additional items that need to be cached\n \n- 8. How directory rename detection interacts with the above and why this\n+ 9. How directory rename detection interacts with the above and why this\n optimization is still safe even if merge.directoryRenames is set to\n \"true\".\n \n \n-=== 0. Assumptions ===\n+== 1. Assumptions ==\n \n There are two assumptions that will hold throughout this document:\n \n@@ -44,8 +44,8 @@ There are two assumptions that will hold throughout this document:\n \n * All merges are fully automatic\n \n-and a third that will hold in sections 2-5 for simplicity, that I'll later\n-address in section 8:\n+and a third that will hold in sections 3-6 for simplicity, that I'll later\n+address in section 9:\n \n * No directory renames occur\n \n@@ -77,9 +77,9 @@ conflicts that the user needs to resolve), the cache of renames is not\n stored on disk, and thus is thrown away as soon as the rebase or cherry\n pick stops for the user to resolve the operation.\n \n-The third assumption makes sections 2-5 simpler, and allows people to\n+The third assumption makes sections 3-6 simpler, and allows people to\n understand the basics of why this optimization is safe and effective, and\n-then I can go back and address the specifics in section 8. It is probably\n+then I can go back and address the specifics in section 9. It is probably\n also worth noting that if directory renames do occur, then the default of\n merge.directoryRenames being set to \"conflict\" means that the operation\n will stop for users to resolve the conflicts and the cache will be thrown\n@@ -88,22 +88,26 @@ reason we need to address directory renames specifically, is that some\n users will have set merge.directoryRenames to \"true\" to allow the merges to\n continue to proceed automatically. The optimization is still safe with\n this config setting, but we have to discuss a few more cases to show why;\n-this discussion is deferred until section 8.\n+this discussion is deferred until section 9.\n \n \n-=== 1. How rebasing and cherry-picking work ===\n+== 2. How rebasing and cherry-picking work ==\n \n Consider the following setup (from the git-rebase manpage):\n \n+------------\n \t\t A---B---C topic\n \t\t /\n \t D---E---F---G main\n+------------\n \n After rebasing or cherry-picking topic onto main, this will appear as:\n \n+------------\n \t\t\t A'--B'--C' topic\n \t\t\t /\n \t D---E---F---G main\n+------------\n \n The way the commits A', B', and C' are created is through a series of\n merges, where rebase or cherry-pick sequentially uses each of the three\n@@ -111,6 +115,7 @@ A-B-C commits in a special merge operation. Let's label the three commits\n in the merge operation as MERGE_BASE, MERGE_SIDE1, and MERGE_SIDE2. For\n this picture, the three commits for each of the three merges would be:\n \n+....\n To create A':\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -125,6 +130,7 @@ To create C':\n MERGE_BASE: B\n MERGE_SIDE1: B'\n MERGE_SIDE2: C\n+....\n \n Sometimes, folks are surprised that these three-way merges are done. It\n can be useful in understanding these three-way merges to view them in a\n@@ -138,8 +144,7 @@ Conceptually the two statements above are the same as a three-way merge of\n B, B', and C, at least the parts before you decide to record a commit.\n \n \n-=== 2. Why the renames on MERGE_SIDE1 in any given pick are always a ===\n-=== superset of the renames on MERGE_SIDE1 for the next pick. ===\n+== 3. Why the renames on MERGE_SIDE1 in any given pick are always a superset of the renames on MERGE_SIDE1 for the next pick. ==\n \n The merge machinery uses the filenames it is fed from MERGE_BASE,\n MERGE_SIDE1, and MERGE_SIDE2. It will only move content to a different\n@@ -156,6 +161,7 @@ filename under one of three conditions:\n First, let's remember what commits are involved in the first and second\n picks of the cherry-pick or rebase sequence:\n \n+....\n To create A':\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -165,6 +171,7 @@ To create B':\n MERGE_BASE: A\n MERGE_SIDE1: A'\n MERGE_SIDE2: B\n+....\n \n So, in particular, we need to show that the renames between E and G are a\n superset of those between A and A'.\n@@ -181,11 +188,11 @@ are a subset of those between E and G. Equivalently, all renames between E\n and G are a superset of those between A and A'.\n \n \n-=== 3. Why any rename on MERGE_SIDE1 in any given pick is _almost_ ===\n-=== always also a rename on MERGE_SIDE1 for the next pick. ===\n+== 4. Why any rename on MERGE_SIDE1 in any given pick is _almost_ always also a rename on MERGE_SIDE1 for the next pick. ==\n \n Let's again look at the first two picks:\n \n+....\n To create A':\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -195,17 +202,25 @@ To create B':\n MERGE_BASE: A\n MERGE_SIDE1: A'\n MERGE_SIDE2: B\n+....\n \n Now let's look at any given rename from MERGE_SIDE1 of the first pick, i.e.\n any given rename from E to G. Let's use the filenames 'oldfile' and\n 'newfile' for demonstration purposes. That first pick will function as\n follows; when the rename is detected, the merge machinery will do a\n three-way content merge of the following:\n+\n+....\n E:oldfile\n G:newfile\n A:oldfile\n+....\n+\n and produce a new result:\n+\n+....\n A':newfile\n+....\n \n Note above that I've assumed that E->A did not rename oldfile. If that\n side did rename, then we most likely have a rename/rename(1to2) conflict\n@@ -254,19 +269,21 @@ were detected as renames, A:oldfile and A':newfile should also be\n detectable as renames almost always.\n \n \n-=== 4. A detailed description of the counter-examples to #3. ===\n+== 5. A detailed description of the counter-examples to #4. ==\n \n-We already noted in section 3 that rename/rename(1to1) (i.e. both sides\n+We already noted in section 4 that rename/rename(1to1) (i.e. both sides\n renaming a file the same way) was one counter-example. The more\n interesting bit, though, is why did we need to use the \"almost\" qualifier\n when stating that A:oldfile and A':newfile are \"almost\" always detectable\n as renames?\n \n-Let's repeat an earlier point that section 3 made:\n+Let's repeat an earlier point that section 4 made:\n \n+....\n A':newfile was created by applying the changes between E:oldfile and\n G:newfile to A:oldfile. The changes between E:oldfile and G:newfile were\n <50% of the size of E:oldfile.\n+....\n \n If those changes that were <50% of the size of E:oldfile are also <50% of\n the size of A:oldfile, then A:oldfile and A':newfile will be detectable as\n@@ -276,18 +293,21 @@ still somehow merge cleanly), then traditional rename detection would not\n detect A:oldfile and A':newfile as renames.\n \n Here's an example where that can happen:\n+\n * E:oldfile had 20 lines\n * G:newfile added 10 new lines at the beginning of the file\n * A:oldfile kept the first 3 lines of the file, and deleted all the rest\n+\n then\n+\n+....\n => A':newfile would have 13 lines, 3 of which matches those in A:oldfile.\n-E:oldfile -> G:newfile would be detected as a rename, but A:oldfile and\n-A':newfile would not be.\n+ E:oldfile -> G:newfile would be detected as a rename, but A:oldfile and\n+ A':newfile would not be.\n+....\n \n \n-=== 5. Why the special cases in #4 are still fully reasonable to use to ===\n-=== pair up files for three-way content merging in the merge machinery, ===\n-=== and why they do not affect the correctness of the merge. ===\n+== 6. Why the special cases in #5 are still fully reasonable to use to pair up files for three-way content merging in the merge machinery, and why they do not affect the correctness of the merge. ==\n \n In the rename/rename(1to1) case, A:newfile and A':newfile are not renames\n since they use the *same* filename. However, files with the same filename\n@@ -295,14 +315,14 @@ are obviously fine to pair up for three-way content merging (the merge\n machinery has never employed break detection). The interesting\n counter-example case is thus not the rename/rename(1to1) case, but the case\n where A did not rename oldfile. That was the case that we spent most of\n-the time discussing in sections 3 and 4. The remainder of this section\n+the time discussing in sections 4 and 5. The remainder of this section\n will be devoted to that case as well.\n \n So, even if A:oldfile and A':newfile aren't detectable as renames, why is\n it still reasonable to pair them up for three-way content merging in the\n merge machinery? There are multiple reasons:\n \n- * As noted in sections 3 and 4, the diff between A:oldfile and A':newfile\n+ * As noted in sections 4 and 5, the diff between A:oldfile and A':newfile\n is *exactly* the same as the diff between E:oldfile and G:newfile. The\n latter pair were detected as renames, so it seems unlikely to surprise\n users for us to treat A:oldfile and A':newfile as renames.\n@@ -394,7 +414,7 @@ cases 1 and 3 seem to provide as good or better behavior with the\n optimization than without.\n \n \n-=== 6. Interaction with skipping of \"irrelevant\" renames ===\n+== 7. Interaction with skipping of \"irrelevant\" renames ==\n \n Previous optimizations involved skipping rename detection for paths\n considered to be \"irrelevant\". See for example the following commits:\n@@ -421,24 +441,27 @@ detection -- though we can limit it to the paths for which we have not\n already detected renames.\n \n \n-=== 7. Additional items that need to be cached ===\n+== 8. Additional items that need to be cached ==\n \n It turns out we have to cache more than just renames; we also cache:\n \n+....\n A) non-renames (i.e. unpaired deletes)\n B) counts of renames within directories\n C) sources that were marked as RELEVANT_LOCATION, but which were\n downgraded to RELEVANT_NO_MORE\n D) the toplevel trees involved in the merge\n+....\n \n These are all stored in struct rename_info, and respectively appear in\n+\n * cached_pairs (along side actual renames, just with a value of NULL)\n * dir_rename_counts\n * cached_irrelevant\n * merge_trees\n \n-The reason for (A) comes from the irrelevant renames skipping\n-optimization discussed in section 6. The fact that irrelevant renames\n+The reason for `(A)` comes from the irrelevant renames skipping\n+optimization discussed in section 7. The fact that irrelevant renames\n are skipped means we only get a subset of the potential renames\n detected and subsequent commits may need to run rename detection on\n the upstream side on a subset of the remaining renames (to get the\n@@ -447,23 +470,24 @@ deletes are involved in rename detection too, we don't want to\n repeatedly check that those paths remain unpaired on the upstream side\n with every commit we are transplanting.\n \n-The reason for (B) is that diffcore_rename_extended() is what\n+The reason for `(B)` is that diffcore_rename_extended() is what\n generates the counts of renames by directory which is needed in\n directory rename detection, and if we don't run\n diffcore_rename_extended() again then we need to have the output from\n it, including dir_rename_counts, from the previous run.\n \n-The reason for (C) is that merge-ort's tree traversal will again think\n+The reason for `(C)` is that merge-ort's tree traversal will again think\n those paths are relevant (marking them as RELEVANT_LOCATION), but the\n fact that they were downgraded to RELEVANT_NO_MORE means that\n dir_rename_counts already has the information we need for directory\n rename detection. (A path which becomes RELEVANT_CONTENT in a\n subsequent commit will be removed from cached_irrelevant.)\n \n-The reason for (D) is that is how we determine whether the remember\n+The reason for `(D)` is that is how we determine whether the remember\n renames optimization can be used. In particular, remembering that our\n sequence of merges looks like:\n \n+....\n Merge 1:\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -475,6 +499,7 @@ sequence of merges looks like:\n MERGE_SIDE1: A'\n MERGE_SIDE2: B\n => Creates B'\n+....\n \n It is the fact that the trees A and A' appear both in Merge 1 and in\n Merge 2, with A as a parent of A' that allows this optimization. So\n@@ -482,12 +507,11 @@ we store the trees to compare with what we are asked to merge next\n time.\n \n \n-=== 8. How directory rename detection interacts with the above and ===\n-=== why this optimization is still safe even if ===\n-=== merge.directoryRenames is set to \"true\". ===\n+== 9. How directory rename detection interacts with the above and why this optimization is still safe even if merge.directoryRenames is set to \"true\". ==\n \n As noted in the assumptions section:\n \n+....\n \"\"\"\n ...if directory renames do occur, then the default of\n merge.directoryRenames being set to \"conflict\" means that the operation\n@@ -497,11 +521,13 @@ As noted in the assumptions section:\n is that some users will have set merge.directoryRenames to \"true\" to\n allow the merges to continue to proceed automatically.\n \"\"\"\n+....\n \n Let's remember that we need to look at how any given pick affects the next\n one. So let's again use the first two picks from the diagram in section\n one:\n \n+....\n First pick does this three-way merge:\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -513,6 +539,7 @@ one:\n MERGE_SIDE1: A'\n MERGE_SIDE2: B\n => creates B'\n+....\n \n Now, directory rename detection exists so that if one side of history\n renames a directory, and the other side adds a new file to the old\n@@ -545,7 +572,7 @@ while considering all of these cases:\n concerned; see the assumptions section). Two interesting sub-notes\n about these counts:\n \n- * If we need to perform rename-detection again on the given side (e.g.\n+ ** If we need to perform rename-detection again on the given side (e.g.\n some paths are relevant for rename detection that weren't before),\n then we clear dir_rename_counts and recompute it, making use of\n cached_pairs. The reason it is important to do this is optimizations\n@@ -556,7 +583,7 @@ while considering all of these cases:\n easiest way to \"fix up\" dir_rename_counts in such cases is to just\n recompute it.\n \n- * If we prune rename/rename(1to1) entries from the cache, then we also\n+ ** If we prune rename/rename(1to1) entries from the cache, then we also\n need to update dir_rename_counts to decrement the counts for the\n involved directory and any relevant parent directories (to undo what\n update_dir_rename_counts() in diffcore-rename.c incremented when the\n@@ -578,6 +605,7 @@ in order:\n \n Case 1: MERGE_SIDE1 renames old dir, MERGE_SIDE2 adds new file to old dir\n \n+....\n This case looks like this:\n \n MERGE_BASE: E, Has olddir/\n@@ -595,10 +623,13 @@ Case 1: MERGE_SIDE1 renames old dir, MERGE_SIDE2 adds new file to old dir\n * MERGE_SIDE1 has cached olddir/newfile -> newdir/newfile\n Given the cached rename noted above, the second merge can proceed as\n expected without needing to perform rename detection from A -> A'.\n+....\n \n Case 2: MERGE_SIDE1 renames old dir, MERGE_SIDE2 renames file into old dir\n \n+....\n This case looks like this:\n+\n MERGE_BASE: E oldfile, olddir/\n MERGE_SIDE1: G oldfile, olddir/ -> newdir/\n MERGE_SIDE2: A oldfile -> olddir/newfile\n@@ -617,9 +648,11 @@ Case 2: MERGE_SIDE1 renames old dir, MERGE_SIDE2 renames file into old dir\n \n Given the cached rename noted above, the second merge can proceed as\n expected without needing to perform rename detection from A -> A'.\n+....\n \n Case 3: MERGE_SIDE1 adds new file to old dir, MERGE_SIDE2 renames old dir\n \n+....\n This case looks like this:\n \n MERGE_BASE: E, Has olddir/\n@@ -635,9 +668,11 @@ Case 3: MERGE_SIDE1 adds new file to old dir, MERGE_SIDE2 renames old dir\n In this case, with the optimization, note that after the first commit there\n were no renames on MERGE_SIDE1, and any renames on MERGE_SIDE2 are tossed.\n But the second merge didn't need any renames so this is fine.\n+....\n \n Case 4: MERGE_SIDE1 renames file into old dir, MERGE_SIDE2 renames old dir\n \n+....\n This case looks like this:\n \n MERGE_BASE: E, Has olddir/\n@@ -658,6 +693,7 @@ Case 4: MERGE_SIDE1 renames file into old dir, MERGE_SIDE2 renames old dir\n \n Given the cached rename noted above, the second merge can proceed as\n expected without needing to perform rename detection from A -> A'.\n+....\n \n Finally, I'll just note here that interactions with the\n skip-irrelevant-renames optimization means we sometimes don't detect\n-- \n2.51.0\n\n"},{"id":"527836","messageId":"20251002221233.541844-4-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251002221233.541844-1-ramsay@ramsayjones.plus.com","subject":"[PATCH v2 3/4] doc: sparse-checkout.adoc: fix asciidoc warnings","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-02T22:12:15Z","receivedAt":"2025-10-02T22:13:28Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Both asciidoc and asciidoctor issue warnings about 'list item index:\nexpected n got n-1' for n=1->7 on lines 928, 931, 951, 974, 980, 1033\nand 1049. In asciidoc, numbered lists must start at one, whereas this\nfile has a list starting at zero. Also, asciidoc and asciidoctor warn\nabout 'section title out of sequence: expected level 1, got level 2'\non line 17. (asciidoc only complains about the first instance of this,\nwhile asciidoctor complains about them all, on lines 95, 258, 303, 316,\n545, 612, 752, 824, 895, 923 and 1053). These warnings stem from the\nsection titles not being correctly nested within a document/chapter\ntitle.\n\nIn order to address the first set of warnings, simply renumber the list\nfrom one to severn, rather than zero to six. Fortunately, this does not\nrequire altering additional text, since the enumeration of 'Known Bugs'\nis not referred to anywhere else in the document.\n\nIn order to address the second set of warnings, change the section title\nsyntax from '=== title ===' to '== title ==', effectively reducing the\nnesting level of the title by one. Also, some apparent (sub-)titles are\nnot marked up with sub-title syntax, so add some '=== ' prefix(s) to the\nrelevant headings.\n\nIn addition to the warnings, address some other formatting issues:\n\n - the use of heavily nested unordered lists is not reflected in the\n output (making the file totally unreadable) because each level of\n nesting requires a different syntax. (i.e. replace '*' with '**'\n for the second level, '*' with '***' for the third level, etc.)\n - make use of literal blocks and manual indentation to get asciidoc\n and asciidoctor to display even remotely similar output.\n - make use of labelled lists, in some places, to get a similar looking\n output to the input, for both asciidoc and asciidoctor.\n - replace the trailing space in: `git grep ${SEARCH_TERM} OLDREV `\n otherwise the entire line in which that appears is removed from\n the output.\n\nSigned-off-by: Ramsay Jones \n---\n Documentation/technical/sparse-checkout.adoc | 704 ++++++++++---------\n 1 file changed, 376 insertions(+), 328 deletions(-)\n\ndiff --git a/Documentation/technical/sparse-checkout.adoc b/Documentation/technical/sparse-checkout.adoc\nindex 0f750ef3e3..3fa8e53655 100644\n--- a/Documentation/technical/sparse-checkout.adoc\n+++ b/Documentation/technical/sparse-checkout.adoc\n@@ -14,37 +14,41 @@ Table of contents:\n * Reference Emails\n \n \n-=== Terminology ===\n+== Terminology ==\n \n-cone mode: one of two modes for specifying the desired subset of files\n+*`cone mode`*::\n+\tone of two modes for specifying the desired subset of files\n \tin a sparse-checkout. In cone-mode, the user specifies\n \tdirectories (getting both everything under that directory as\n \twell as everything in leading directories), while in non-cone\n \tmode, the user specifies gitignore-style patterns. Controlled\n \tby the --[no-]cone option to sparse-checkout init|set.\n \n-SKIP_WORKTREE: When tracked files do not match the sparse specification and\n+*`SKIP_WORKTREE`*::\n+\tWhen tracked files do not match the sparse specification and\n \tare removed from the working tree, the file in the index is marked\n \twith a SKIP_WORKTREE bit. Note that if a tracked file has the\n \tSKIP_WORKTREE bit set but the file is later written by the user to\n \tthe working tree anyway, the SKIP_WORKTREE bit will be cleared at\n \tthe beginning of any subsequent Git operation.\n-\n-\tMost sparse checkout users are unaware of this implementation\n-\tdetail, and the term should generally be avoided in user-facing\n-\tdescriptions and command flags. Unfortunately, prior to the\n-\t`sparse-checkout` subcommand this low-level detail was exposed,\n-\tand as of time of writing, is still exposed in various places.\n-\n-sparse-checkout: a subcommand in git used to reduce the files present in\n++\n+Most sparse checkout users are unaware of this implementation\n+detail, and the term should generally be avoided in user-facing\n+descriptions and command flags. Unfortunately, prior to the\n+`sparse-checkout` subcommand this low-level detail was exposed,\n+and as of time of writing, is still exposed in various places.\n+\n+*`sparse-checkout`*::\n+\ta subcommand in git used to reduce the files present in\n \tthe working tree to a subset of all tracked files. Also, the\n \tname of the file in the $GIT_DIR/info directory used to track\n \tthe sparsity patterns corresponding to the user's desired\n \tsubset.\n \n-sparse cone: see cone mode\n+*`sparse cone`*:: see cone mode\n \n-sparse directory: An entry in the index corresponding to a directory, which\n+*`sparse directory`*::\n+\tAn entry in the index corresponding to a directory, which\n \tappears in the index instead of all the files under that directory\n \tthat would normally appear. See also sparse-index. Something that\n \tcan cause confusion is that the \"sparse directory\" does NOT match\n@@ -52,7 +56,8 @@ sparse directory: An entry in the index corresponding to a directory, which\n \tworking tree. May be renamed in the future (e.g. to \"skipped\n \tdirectory\").\n \n-sparse index: A special mode for sparse-checkout that also makes the\n+*`sparse index`*::\n+\tA special mode for sparse-checkout that also makes the\n \tindex sparse by recording a directory entry in lieu of all the\n \tfiles underneath that directory (thus making that a \"skipped\n \tdirectory\" which unfortunately has also been called a \"sparse\n@@ -60,7 +65,8 @@ sparse index: A special mode for sparse-checkout that also makes the\n \tdirectories. Controlled by the --[no-]sparse-index option to\n \tinit|set|reapply.\n \n-sparsity patterns: patterns from $GIT_DIR/info/sparse-checkout used to\n+*`sparsity patterns`*::\n+\tpatterns from $GIT_DIR/info/sparse-checkout used to\n \tdefine the set of files of interest. A warning: It is easy to\n \tover-use this term (or the shortened \"patterns\" term), for two\n \treasons: (1) users in cone mode specify directories rather than\n@@ -70,7 +76,8 @@ sparsity patterns: patterns from $GIT_DIR/info/sparse-checkout used to\n \ttransiently differ in the working tree or index from the sparsity\n \tpatterns (see \"Sparse specification vs. sparsity patterns\").\n \n-sparse specification: The set of paths in the user's area of focus. This\n+*`sparse specification`*::\n+\tThe set of paths in the user's area of focus. This\n \tis typically just the tracked files that match the sparsity\n \tpatterns, but the sparse specification can temporarily differ and\n \tinclude additional files. (See also \"Sparse specification\n@@ -87,12 +94,13 @@ sparse specification: The set of paths in the user's area of focus. This\n \t* If working with the index and the working copy, the sparse\n \t specification is the union of the paths from above.\n \n-vivifying: When a command restores a tracked file to the working tree (and\n+*`vivifying`*::\n+\tWhen a command restores a tracked file to the working tree (and\n \thopefully also clears the SKIP_WORKTREE bit in the index for that\n \tfile), this is referred to as \"vivifying\" the file.\n \n \n-=== Purpose of sparse-checkouts ===\n+== Purpose of sparse-checkouts ==\n \n sparse-checkouts exist to allow users to work with a subset of their\n files.\n@@ -120,14 +128,12 @@ those usecases, sparse-checkouts can modify different subcommands in over a\n half dozen different ways. Let's start by considering the high level\n usecases:\n \n- A) Users are _only_ interested in the sparse portion of the repo\n-\n- A*) Users are _only_ interested in the sparse portion of the repo\n- that they have downloaded so far\n-\n- B) Users want a sparse working tree, but are working in a larger whole\n-\n- C) sparse-checkout is a behind-the-scenes implementation detail allowing\n+[horizontal]\n+A):: Users are _only_ interested in the sparse portion of the repo\n+A*):: Users are _only_ interested in the sparse portion of the repo\n+ that they have downloaded so far\n+B):: Users want a sparse working tree, but are working in a larger whole\n+C):: sparse-checkout is a behind-the-scenes implementation detail allowing\n Git to work with a specially crafted in-house virtual file system;\n users are actually working with a \"full\" working tree that is\n lazily populated, and sparse-checkout helps with the lazy population\n@@ -136,7 +142,7 @@ usecases:\n It may be worth explaining each of these in a bit more detail:\n \n \n- (Behavior A) Users are _only_ interested in the sparse portion of the repo\n+=== (Behavior A) Users are _only_ interested in the sparse portion of the repo\n \n These folks might know there are other things in the repository, but\n don't care. They are uninterested in other parts of the repository, and\n@@ -163,8 +169,7 @@ side-effects of various other commands (such as the printed diffstat\n after a merge or pull) can lead to worries about local repository size\n growing unnecessarily[10].\n \n- (Behavior A*) Users are _only_ interested in the sparse portion of the repo\n- that they have downloaded so far (a variant on the first usecase)\n+=== (Behavior A*) Users are _only_ interested in the sparse portion of the repo that they have downloaded so far (a variant on the first usecase)\n \n This variant is driven by folks who using partial clones together with\n sparse checkouts and do disconnected development (so far sounding like a\n@@ -173,15 +178,14 @@ reason for yet another variant is that downloading even just the blobs\n through history within their sparse specification may be too much, so they\n only download some. They would still like operations to succeed without\n network connectivity, though, so things like `git log -S${SEARCH_TERM} -p`\n-or `git grep ${SEARCH_TERM} OLDREV ` would need to be prepared to provide\n+or `git grep ${SEARCH_TERM} OLDREV` would need to be prepared to provide\n partial results that depend on what happens to have been downloaded.\n \n This variant could be viewed as Behavior A with the sparse specification\n for history querying operations modified from \"sparsity patterns\" to\n \"sparsity patterns limited to the blobs we have already downloaded\".\n \n- (Behavior B) Users want a sparse working tree, but are working in a\n- larger whole\n+=== (Behavior B) Users want a sparse working tree, but are working in a larger whole\n \n Stolee described this usecase this way[11]:\n \n@@ -229,8 +233,7 @@ those expensive checks when interacting with the working copy, and may\n prefer getting \"unrelated\" results from their history queries over having\n slow commands.\n \n- (Behavior C) sparse-checkout is an implementational detail supporting a\n-\t special VFS.\n+=== (Behavior C) sparse-checkout is an implementational detail supporting a special VFS.\n \n This usecase goes slightly against the traditional definition of\n sparse-checkout in that it actually tries to present a full or dense\n@@ -255,13 +258,13 @@ will perceive the checkout as dense, and commands should thus behave as if\n all files are present.\n \n \n-=== Usecases of primary concern ===\n+== Usecases of primary concern ==\n \n Most of the rest of this document will focus on Behavior A and Behavior\n B. Some notes about the other two cases and why we are not focusing on\n them:\n \n- (Behavior A*)\n+=== (Behavior A*)\n \n Supporting this usecase is estimated to be difficult and a lot of work.\n There are no plans to implement it currently, but it may be a potential\n@@ -275,7 +278,7 @@ valid for this usecase, with the only exception being that it redefines the\n sparse specification to restrict it to already-downloaded blobs. The hard\n part is in making commands capable of respecting that modified definition.\n \n- (Behavior C)\n+=== (Behavior C)\n \n This usecase violates some of the early sparse-checkout documented\n assumptions (since files marked as SKIP_WORKTREE will be displayed to users\n@@ -300,20 +303,20 @@ Behavior C do not assume they are part of the Behavior B camp and propose\n patches that break things for the real Behavior B folks.\n \n \n-=== Oversimplified mental models ===\n+== Oversimplified mental models ==\n \n An oversimplification of the differences in the above behaviors is:\n \n- Behavior A: Restrict worktree and history operations to sparse specification\n- Behavior B: Restrict worktree operations to sparse specification; have any\n-\t history operations work across all files\n- Behavior C: Do not restrict either worktree or history operations to the\n-\t sparse specification...with the exception of branch checkouts or\n-\t switches which avoid writing files that will match the index so\n-\t they can later lazily be populated instead.\n+(Behavior A):: Restrict worktree and history operations to sparse specification\n+(Behavior B):: Restrict worktree operations to sparse specification; have any\n+\t history operations work across all files\n+(Behavior C):: Do not restrict either worktree or history operations to the\n+\t sparse specification...with the exception of branch checkouts or\n+\t switches which avoid writing files that will match the index so\n+\t they can later lazily be populated instead.\n \n \n-=== Desired behavior ===\n+== Desired behavior ==\n \n As noted previously, despite the simple idea of just working with a subset\n of files, there are a range of different behavioral changes that need to be\n@@ -326,37 +329,38 @@ understanding these differences can be beneficial.\n \n * Commands behaving the same regardless of high-level use-case\n \n- * commands that only look at files within the sparsity specification\n+ ** commands that only look at files within the sparsity specification\n \n- * diff (without --cached or REVISION arguments)\n- * grep (without --cached or REVISION arguments)\n- * diff-files\n+ *** diff (without --cached or REVISION arguments)\n+ *** grep (without --cached or REVISION arguments)\n+ *** diff-files\n \n- * commands that restore files to the working tree that match sparsity\n+ ** commands that restore files to the working tree that match sparsity\n patterns, and remove unmodified files that don't match those\n patterns:\n \n- * switch\n- * checkout (the switch-like half)\n- * read-tree\n- * reset --hard\n+ *** switch\n+ *** checkout (the switch-like half)\n+ *** read-tree\n+ *** reset --hard\n \n- * commands that write conflicted files to the working tree, but otherwise\n+ ** commands that write conflicted files to the working tree, but otherwise\n will omit writing files to the working tree that do not match the\n sparsity patterns:\n \n- * merge\n- * rebase\n- * cherry-pick\n- * revert\n+ *** merge\n+ *** rebase\n+ *** cherry-pick\n+ *** revert\n \n- * `am` and `apply --cached` should probably be in this section but\n+ *** `am` and `apply --cached` should probably be in this section but\n \tare buggy (see the \"Known bugs\" section below)\n \n The behavior for these commands somewhat depends upon the merge\n strategy being used:\n- * `ort` behaves as described above\n- * `octopus` and `resolve` will always vivify any file changed in the merge\n+\n+ *** `ort` behaves as described above\n+ *** `octopus` and `resolve` will always vivify any file changed in the merge\n \trelative to the first parent, which is rather suboptimal.\n \n It is also important to note that these commands WILL update the index\n@@ -372,21 +376,21 @@ understanding these differences can be beneficial.\n specification and the sparsity patterns (much like the commands in the\n previous section).\n \n- * commands that always ignore sparsity since commits must be full-tree\n+ ** commands that always ignore sparsity since commits must be full-tree\n \n- * archive\n- * bundle\n- * commit\n- * format-patch\n- * fast-export\n- * fast-import\n- * commit-tree\n+ *** archive\n+ *** bundle\n+ *** commit\n+ *** format-patch\n+ *** fast-export\n+ *** fast-import\n+ *** commit-tree\n \n- * commands that write any modified file to the working tree (conflicted\n+ ** commands that write any modified file to the working tree (conflicted\n or not, and whether those paths match sparsity patterns or not):\n \n- * stash\n- * apply (without `--index` or `--cached`)\n+ *** stash\n+ *** apply (without `--index` or `--cached`)\n \n * Commands that may slightly differ for behavior A vs. behavior B:\n \n@@ -394,19 +398,20 @@ understanding these differences can be beneficial.\n behaviors, but may differ in verbosity and types of warning and error\n messages.\n \n- * commands that make modifications to which files are tracked:\n- * add\n- * rm\n- * mv\n- * update-index\n+ ** commands that make modifications to which files are tracked:\n+\n+ *** add\n+ *** rm\n+ *** mv\n+ *** update-index\n \n The fact that files can move between the 'tracked' and 'untracked'\n categories means some commands will have to treat untracked files\n differently. But if we have to treat untracked files differently,\n then additional commands may also need changes:\n \n- * status\n- * clean\n+ *** status\n+ *** clean\n \n In particular, `status` may need to report any untracked files outside\n the sparsity specification as an erroneous condition (especially to\n@@ -420,9 +425,10 @@ understanding these differences can be beneficial.\n may need to ignore the sparse specification by its nature. Also, its\n current --[no-]ignore-skip-worktree-entries default is totally bogus.\n \n- * commands for manually tweaking paths in both the index and the working tree\n- * `restore`\n- * the restore-like half of `checkout`\n+ ** commands for manually tweaking paths in both the index and the working tree\n+\n+ *** `restore`\n+ *** the restore-like half of `checkout`\n \n These commands should be similar to add/rm/mv in that they should\n only operate on the sparse specification by default, and require a\n@@ -433,18 +439,19 @@ understanding these differences can be beneficial.\n \n * Commands that significantly differ for behavior A vs. behavior B:\n \n- * commands that query history\n- * diff (with --cached or REVISION arguments)\n- * grep (with --cached or REVISION arguments)\n- * show (when given commit arguments)\n- * blame (only matters when one or more -C flags are passed)\n-\t* and annotate\n- * log\n- * whatchanged (may not exist anymore)\n- * ls-files\n- * diff-index\n- * diff-tree\n- * ls-tree\n+ ** commands that query history\n+\n+ *** diff (with --cached or REVISION arguments)\n+ *** grep (with --cached or REVISION arguments)\n+ *** show (when given commit arguments)\n+ *** blame (only matters when one or more -C flags are passed)\n+\t**** and annotate\n+ *** log\n+ *** whatchanged (may not exist anymore)\n+ *** ls-files\n+ *** diff-index\n+ *** diff-tree\n+ *** ls-tree\n \n Note: for log and whatchanged, revision walking logic is unaffected\n but displaying of patches is affected by scoping the command to the\n@@ -458,91 +465,91 @@ understanding these differences can be beneficial.\n \n * Commands I don't know how to classify\n \n- * range-diff\n+ ** range-diff\n \n Is this like `log` or `format-patch`?\n \n- * cherry\n+ ** cherry\n \n See range-diff\n \n * Commands unaffected by sparse-checkouts\n \n- * shortlog\n- * show-branch\n- * rev-list\n- * bisect\n-\n- * branch\n- * describe\n- * fetch\n- * gc\n- * init\n- * maintenance\n- * notes\n- * pull (merge & rebase have the necessary changes)\n- * push\n- * submodule\n- * tag\n-\n- * config\n- * filter-branch (works in separate checkout without sparse-checkout setup)\n- * pack-refs\n- * prune\n- * remote\n- * repack\n- * replace\n-\n- * bugreport\n- * count-objects\n- * fsck\n- * gitweb\n- * help\n- * instaweb\n- * merge-tree (doesn't touch worktree or index, and merges always compute full-tree)\n- * rerere\n- * verify-commit\n- * verify-tag\n-\n- * commit-graph\n- * hash-object\n- * index-pack\n- * mktag\n- * mktree\n- * multi-pack-index\n- * pack-objects\n- * prune-packed\n- * symbolic-ref\n- * unpack-objects\n- * update-ref\n- * write-tree (operates on index, possibly optimized to use sparse dir entries)\n-\n- * for-each-ref\n- * get-tar-commit-id\n- * ls-remote\n- * merge-base (merges are computed full tree, so merge base should be too)\n- * name-rev\n- * pack-redundant\n- * rev-parse\n- * show-index\n- * show-ref\n- * unpack-file\n- * var\n- * verify-pack\n-\n- * \n- * \n- * \n- * \n+ ** shortlog\n+ ** show-branch\n+ ** rev-list\n+ ** bisect\n+\n+ ** branch\n+ ** describe\n+ ** fetch\n+ ** gc\n+ ** init\n+ ** maintenance\n+ ** notes\n+ ** pull (merge & rebase have the necessary changes)\n+ ** push\n+ ** submodule\n+ ** tag\n+\n+ ** config\n+ ** filter-branch (works in separate checkout without sparse-checkout setup)\n+ ** pack-refs\n+ ** prune\n+ ** remote\n+ ** repack\n+ ** replace\n+\n+ ** bugreport\n+ ** count-objects\n+ ** fsck\n+ ** gitweb\n+ ** help\n+ ** instaweb\n+ ** merge-tree (doesn't touch worktree or index, and merges always compute full-tree)\n+ ** rerere\n+ ** verify-commit\n+ ** verify-tag\n+\n+ ** commit-graph\n+ ** hash-object\n+ ** index-pack\n+ ** mktag\n+ ** mktree\n+ ** multi-pack-index\n+ ** pack-objects\n+ ** prune-packed\n+ ** symbolic-ref\n+ ** unpack-objects\n+ ** update-ref\n+ ** write-tree (operates on index, possibly optimized to use sparse dir entries)\n+\n+ ** for-each-ref\n+ ** get-tar-commit-id\n+ ** ls-remote\n+ ** merge-base (merges are computed full tree, so merge base should be too)\n+ ** name-rev\n+ ** pack-redundant\n+ ** rev-parse\n+ ** show-index\n+ ** show-ref\n+ ** unpack-file\n+ ** var\n+ ** verify-pack\n+\n+ ** \n+ ** \n+ ** \n+ ** \n \n * Commands that might be affected, but who cares?\n \n- * merge-file\n- * merge-index\n- * gitk?\n+ ** merge-file\n+ ** merge-index\n+ ** gitk?\n \n \n-=== Behavior classes ===\n+== Behavior classes ==\n \n From the above there are a few classes of behavior:\n \n@@ -573,18 +580,19 @@ From the above there are a few classes of behavior:\n \n Commands in this class generally behave like the \"restrict\" class,\n except that:\n- (1) they will ignore the sparse specification and write files with\n-\t conflicts to the working tree (thus temporarily expanding the\n-\t sparse specification to include such files.)\n- (2) they are grouped with commands which move to a new commit, since\n-\t they often create a commit and then move to it, even though we\n-\t know there are many exceptions to moving to the new commit. (For\n-\t example, the user may rebase a commit that becomes empty, or have\n-\t a cherry-pick which conflicts, or a user could run `merge\n-\t --no-commit`, and we also view `apply --index` kind of like `am\n-\t --no-commit`.) As such, these commands can make changes to index\n-\t files outside the sparse specification, though they'll mark such\n-\t files with SKIP_WORKTREE.\n+\n+\t(1) they will ignore the sparse specification and write files with\n+\t conflicts to the working tree (thus temporarily expanding the\n+\t sparse specification to include such files.)\n+\t(2) they are grouped with commands which move to a new commit, since\n+\t they often create a commit and then move to it, even though we\n+\t know there are many exceptions to moving to the new commit. (For\n+\t example, the user may rebase a commit that becomes empty, or have\n+\t a cherry-pick which conflicts, or a user could run `merge\n+\t --no-commit`, and we also view `apply --index` kind of like `am\n+\t --no-commit`.) As such, these commands can make changes to index\n+\t files outside the sparse specification, though they'll mark such\n+\t files with SKIP_WORKTREE.\n \n * \"restrict also specially applied to untracked files\"\n \n@@ -609,37 +617,39 @@ From the above there are a few classes of behavior:\n specification.\n \n \n-=== Subcommand-dependent defaults ===\n+== Subcommand-dependent defaults ==\n \n Note that we have different defaults depending on the command for the\n desired behavior :\n \n * Commands defaulting to \"restrict\":\n- * diff-files\n- * diff (without --cached or REVISION arguments)\n- * grep (without --cached or REVISION arguments)\n- * switch\n- * checkout (the switch-like half)\n- * reset ()\n-\n- * restore\n- * checkout (the restore-like half)\n- * checkout-index\n- * reset (with pathspec)\n+\n+ ** diff-files\n+ ** diff (without --cached or REVISION arguments)\n+ ** grep (without --cached or REVISION arguments)\n+ ** switch\n+ ** checkout (the switch-like half)\n+ ** reset ()\n+\n+ ** restore\n+ ** checkout (the restore-like half)\n+ ** checkout-index\n+ ** reset (with pathspec)\n \n This behavior makes sense; these interact with the working tree.\n \n * Commands defaulting to \"restrict modulo conflicts\":\n- * merge\n- * rebase\n- * cherry-pick\n- * revert\n \n- * am\n- * apply --index (which is kind of like an `am --no-commit`)\n+ ** merge\n+ ** rebase\n+ ** cherry-pick\n+ ** revert\n+\n+ ** am\n+ ** apply --index (which is kind of like an `am --no-commit`)\n \n- * read-tree (especially with -m or -u; is kind of like a --no-commit merge)\n- * reset (, due to similarity to read-tree)\n+ ** read-tree (especially with -m or -u; is kind of like a --no-commit merge)\n+ ** reset (, due to similarity to read-tree)\n \n These also interact with the working tree, but require slightly\n different behavior either so that (a) conflicts can be resolved or (b)\n@@ -648,16 +658,17 @@ desired behavior :\n (See also the \"Known bugs\" section below regarding `am` and `apply`)\n \n * Commands defaulting to \"no restrict\":\n- * archive\n- * bundle\n- * commit\n- * format-patch\n- * fast-export\n- * fast-import\n- * commit-tree\n \n- * stash\n- * apply (without `--index`)\n+ ** archive\n+ ** bundle\n+ ** commit\n+ ** format-patch\n+ ** fast-export\n+ ** fast-import\n+ ** commit-tree\n+\n+ ** stash\n+ ** apply (without `--index`)\n \n These have completely different defaults and perhaps deserve the most\n detailed explanation:\n@@ -679,53 +690,59 @@ desired behavior :\n sparse specification then we'll lose changes from the user.\n \n * Commands defaulting to \"restrict also specially applied to untracked files\":\n- * add\n- * rm\n- * mv\n- * update-index\n- * status\n- * clean (?)\n-\n- Our original implementation for the first three of these commands was\n- \"no restrict\", but it had some severe usability issues:\n- * `git add ` if honored and outside the sparse\n-\tspecification, can result in the file randomly disappearing later\n-\twhen some subsequent command is run (since various commands\n-\tautomatically clean up unmodified files outside the sparse\n-\tspecification).\n- * `git rm '*.jpg'` could very negatively surprise users if it deletes\n-\tfiles outside the range of the user's interest.\n- * `git mv` has similar surprises when moving into or out of the cone,\n-\tso best to restrict by default\n-\n- So, we switched `add` and `rm` to default to \"restrict\", which made\n- usability problems much less severe and less frequent, but we still got\n- complaints because commands like:\n-\tgit add \n-\tgit rm \n- would silently do nothing. We should instead print an error in those\n- cases to get usability right.\n-\n- update-index needs to be updated to match, and status and maybe clean\n- also need to be updated to specially handle untracked paths.\n-\n- There may be a difference in here between behavior A and behavior B in\n- terms of verboseness of errors or additional warnings.\n+\n+ ** add\n+ ** rm\n+ ** mv\n+ ** update-index\n+ ** status\n+ ** clean (?)\n+\n+....\n+ Our original implementation for the first three of these commands was\n+ \"no restrict\", but it had some severe usability issues:\n+\n+ * `git add ` if honored and outside the sparse\n+\t specification, can result in the file randomly disappearing later\n+\t when some subsequent command is run (since various commands\n+\t automatically clean up unmodified files outside the sparse\n+\t specification).\n+ * `git rm '*.jpg'` could very negatively surprise users if it deletes\n+\t files outside the range of the user's interest.\n+ * `git mv` has similar surprises when moving into or out of the cone,\n+\t so best to restrict by default\n+\n+ So, we switched `add` and `rm` to default to \"restrict\", which made\n+ usability problems much less severe and less frequent, but we still got\n+ complaints because commands like:\n+\n+\t git add \n+\t git rm \n+\n+ would silently do nothing. We should instead print an error in those\n+ cases to get usability right.\n+\n+ update-index needs to be updated to match, and status and maybe clean\n+ also need to be updated to specially handle untracked paths.\n+\n+ There may be a difference in here between behavior A and behavior B in\n+ terms of verboseness of errors or additional warnings.\n+....\n \n * Commands falling under \"restrict or no restrict dependent upon behavior\n A vs. behavior B\"\n \n- * diff (with --cached or REVISION arguments)\n- * grep (with --cached or REVISION arguments)\n- * show (when given commit arguments)\n- * blame (only matters when one or more -C flags passed)\n- * and annotate\n- * log\n- * and variants: shortlog, gitk, show-branch, whatchanged, rev-list\n- * ls-files\n- * diff-index\n- * diff-tree\n- * ls-tree\n+ ** diff (with --cached or REVISION arguments)\n+ ** grep (with --cached or REVISION arguments)\n+ ** show (when given commit arguments)\n+ ** blame (only matters when one or more -C flags passed)\n+ *** and annotate\n+ ** log\n+ *** and variants: shortlog, gitk, show-branch, whatchanged, rev-list\n+ ** ls-files\n+ ** diff-index\n+ ** diff-tree\n+ ** ls-tree\n \n For now, we default to behavior B for these, which want a default of\n \"no restrict\".\n@@ -749,7 +766,7 @@ desired behavior :\n implemented.\n \n \n-=== Sparse specification vs. sparsity patterns ===\n+== Sparse specification vs. sparsity patterns ==\n \n In a well-behaved situation, the sparse specification is given directly\n by the $GIT_DIR/info/sparse-checkout file. However, it can transiently\n@@ -821,45 +838,48 @@ under behavior B index operations are lumped with history and tend to\n operate full-tree.\n \n \n-=== Implementation Questions ===\n-\n- * Do the options --scope={sparse,all} sound good to others? Are there better\n- options?\n- * Names in use, or appearing in patches, or previously suggested:\n- * --sparse/--dense\n- * --ignore-skip-worktree-bits\n- * --ignore-skip-worktree-entries\n- * --ignore-sparsity\n- * --[no-]restrict-to-sparse-paths\n- * --full-tree/--sparse-tree\n- * --[no-]restrict\n- * --scope={sparse,all}\n- * --focus/--unfocus\n- * --limit/--unlimited\n- * Rationale making me lean slightly towards --scope={sparse,all}:\n- * We want a name that works for many commands, so we need a name that\n+== Implementation Questions ==\n+\n+ * Do the options --scope={sparse,all} sound good to others? Are there better options?\n+\n+ ** Names in use, or appearing in patches, or previously suggested:\n+\n+ *** --sparse/--dense\n+ *** --ignore-skip-worktree-bits\n+ *** --ignore-skip-worktree-entries\n+ *** --ignore-sparsity\n+ *** --[no-]restrict-to-sparse-paths\n+ *** --full-tree/--sparse-tree\n+ *** --[no-]restrict\n+ *** --scope={sparse,all}\n+ *** --focus/--unfocus\n+ *** --limit/--unlimited\n+\n+ ** Rationale making me lean slightly towards --scope={sparse,all}:\n+\n+ *** We want a name that works for many commands, so we need a name that\n \tdoes not conflict\n- * We know that we have more than two possible usecases, so it is best\n+ *** We know that we have more than two possible usecases, so it is best\n \tto avoid a flag that appears to be binary.\n- * --scope={sparse,all} isn't overly long and seems relatively\n+ *** --scope={sparse,all} isn't overly long and seems relatively\n \texplanatory\n- * `--sparse`, as used in add/rm/mv, is totally backwards for\n+ *** `--sparse`, as used in add/rm/mv, is totally backwards for\n \tgrep/log/etc. Changing the meaning of `--sparse` for these\n \tcommands would fix the backwardness, but possibly break existing\n \tscripts. Using a new name pairing would allow us to treat\n \t`--sparse` in these commands as a deprecated alias.\n- * There is a different `--sparse`/`--dense` pair for commands using\n+ *** There is a different `--sparse`/`--dense` pair for commands using\n \trevision machinery, so using that naming might cause confusion\n- * There is also a `--sparse` in both pack-objects and show-branch, which\n+ *** There is also a `--sparse` in both pack-objects and show-branch, which\n \tdon't conflict but do suggest that `--sparse` is overloaded\n- * The name --ignore-skip-worktree-bits is a double negative, is\n+ *** The name --ignore-skip-worktree-bits is a double negative, is\n \tquite a mouthful, refers to an implementation detail that many\n \tusers may not be familiar with, and we'd need a negation for it\n \twhich would probably be even more ridiculously long. (But we\n \tcan make --ignore-skip-worktree-bits a deprecated alias for\n \t--no-restrict.)\n \n- * If a config option is added (sparse.scope?) what should the values and\n+ ** If a config option is added (sparse.scope?) what should the values and\n description be? \"sparse\" (behavior A), \"worktree-sparse-history-dense\"\n (behavior B), \"dense\" (behavior C)? There's a risk of confusion,\n because even for Behaviors A and B we want some commands to be\n@@ -868,19 +888,20 @@ operate full-tree.\n the primary difference we are focusing is just the history-querying\n commands (log/diff/grep). Previous config suggestion here: [13]\n \n- * Is `--no-expand` a good alias for ls-files's `--sparse` option?\n+ ** Is `--no-expand` a good alias for ls-files's `--sparse` option?\n (`--sparse` does not map to either `--scope=sparse` or `--scope=all`,\n because in non-cone mode it does nothing and in cone-mode it shows the\n sparse directory entries which are technically outside the sparse\n specification)\n \n- * Under Behavior A:\n- * Does ls-files' `--no-expand` override the default `--scope=all`, or\n+ ** Under Behavior A:\n+\n+ *** Does ls-files' `--no-expand` override the default `--scope=all`, or\n does it need an extra flag?\n- * Does ls-files' `-t` option imply `--scope=all`?\n- * Does update-index's `--[no-]skip-worktree` option imply `--scope=all`?\n+ *** Does ls-files' `-t` option imply `--scope=all`?\n+ *** Does update-index's `--[no-]skip-worktree` option imply `--scope=all`?\n \n- * sparse-checkout: once behavior A is fully implemented, should we take\n+ ** sparse-checkout: once behavior A is fully implemented, should we take\n an interim measure to ease people into switching the default? Namely,\n if folks are not already in a sparse checkout, then require\n `sparse-checkout init/set` to take a\n@@ -892,7 +913,7 @@ operate full-tree.\n is seamless for them.\n \n \n-=== Implementation Goals/Plans ===\n+== Implementation Goals/Plans ==\n \n * Get buy-in on this document in general.\n \n@@ -910,25 +931,26 @@ operate full-tree.\n request that they not trigger this bug.\" flag\n \n * Flags & Config\n- * Make `--sparse` in add/rm/mv a deprecated alias for `--scope=all`\n- * Make `--ignore-skip-worktree-bits` in checkout-index/checkout/restore\n+\n+ ** Make `--sparse` in add/rm/mv a deprecated alias for `--scope=all`\n+ ** Make `--ignore-skip-worktree-bits` in checkout-index/checkout/restore\n a deprecated aliases for `--scope=all`\n- * Create config option (sparse.scope?), tie it to the \"Cliff notes\"\n+ ** Create config option (sparse.scope?), tie it to the \"Cliff notes\"\n overview\n \n- * Add --scope=sparse (and --scope=all) flag to each of the history querying\n+ ** Add --scope=sparse (and --scope=all) flag to each of the history querying\n commands. IMPORTANT: make sure diff machinery changes don't mess with\n format-patch, fast-export, etc.\n \n-=== Known bugs ===\n+== Known bugs ==\n \n This list used to be a lot longer (see e.g. [1,2,3,4,5,6,7,8,9]), but we've\n been working on it.\n \n-0. Behavior A is not well supported in Git. (Behavior B didn't used to\n+1. Behavior A is not well supported in Git. (Behavior B didn't used to\n be either, but was the easier of the two to implement.)\n \n-1. am and apply:\n+2. am and apply:\n \n apply, without `--index` or `--cached`, relies on files being present\n in the working copy, and also writes to them unconditionally. As\n@@ -948,7 +970,7 @@ been working on it.\n files and then complain that those vivified files would be\n overwritten by merge.\n \n-2. reset --hard:\n+3. reset --hard:\n \n reset --hard provides confusing error message (works correctly, but\n misleads the user into believing it didn't):\n@@ -971,13 +993,13 @@ been working on it.\n `git reset --hard` DID remove addme from the index and the working tree, contrary\n to the error message, but in line with how reset --hard should behave.\n \n-3. read-tree\n+4. read-tree\n \n `read-tree` doesn't apply the 'SKIP_WORKTREE' bit to *any* of the\n entries it reads into the index, resulting in all your files suddenly\n appearing to be \"deleted\".\n \n-4. Checkout, restore:\n+5. Checkout, restore:\n \n These command do not handle path & revision arguments appropriately:\n \n@@ -1030,7 +1052,7 @@ been working on it.\n S tracked\n H tracked-but-maybe-skipped\n \n-5. checkout and restore --staged, continued:\n+6. checkout and restore --staged, continued:\n \n These commands do not correctly scope operations to the sparse\n specification, and make it worse by not setting important SKIP_WORKTREE\n@@ -1046,56 +1068,82 @@ been working on it.\n the sparse specification, but then it will be important to set the\n SKIP_WORKTREE bits appropriately.\n \n-6. Performance issues; see:\n- https://lore.kernel.org/git/CABPp-BEkJQoKZsQGCYioyga_uoDQ6iBeW+FKr8JhyuuTMK1RDw@mail.gmail.com/\n+7. Performance issues; see:\n+\n+ https://lore.kernel.org/git/CABPp-BEkJQoKZsQGCYioyga_uoDQ6iBeW+FKr8JhyuuTMK1RDw@mail.gmail.com/\n \n \n-=== Reference Emails ===\n+== Reference Emails ==\n \n Emails that detail various bugs we've had in sparse-checkout:\n \n-[1] (Original descriptions of behavior A & behavior B)\n- https://lore.kernel.org/git/CABPp-BGJ_Nvi5TmgriD9Bh6eNXE2EDq2f8e8QKXAeYG3BxZafA@mail.gmail.com/\n-[2] (Fix stash applications in sparse checkouts; bugs from behavioral differences)\n- https://lore.kernel.org/git/ccfedc7140dbf63ba26a15f93bd3885180b26517.1606861519.git.gitgitgadget@gmail.com/\n-[3] (Present-despite-skipped entries)\n- https://lore.kernel.org/git/11d46a399d26c913787b704d2b7169cafc28d639.1642175983.git.gitgitgadget@gmail.com/\n-[4] (Clone --no-checkout interaction)\n- https://lore.kernel.org/git/pull.801.v2.git.git.1591324899170.gitgitgadget@gmail.com/ (clone --no-checkout)\n-[5] (The need for update_sparsity() and avoiding `read-tree -mu HEAD`)\n- https://lore.kernel.org/git/3a1f084641eb47515b5a41ed4409a36128913309.1585270142.git.gitgitgadget@gmail.com/\n-[6] (SKIP_WORKTREE is advisory, not mandatory)\n- https://lore.kernel.org/git/844306c3e86ef67591cc086decb2b760e7d710a3.1585270142.git.gitgitgadget@gmail.com/\n-[7] (`worktree add` should copy sparsity settings from current worktree)\n- https://lore.kernel.org/git/c51cb3714e7b1d2f8c9370fe87eca9984ff4859f.1644269584.git.gitgitgadget@gmail.com/\n-[8] (Avoid negative surprises in add, rm, and mv)\n- https://lore.kernel.org/git/cover.1617914011.git.matheus.bernardino@usp.br/\n- https://lore.kernel.org/git/pull.1018.v4.git.1632497954.gitgitgadget@gmail.com/\n-[9] (Move from out-of-cone to in-cone)\n- https://lore.kernel.org/git/20220630023737.473690-6-shaoxuan.yuan02@gmail.com/\n- https://lore.kernel.org/git/20220630023737.473690-4-shaoxuan.yuan02@gmail.com/\n-[10] (Unnecessarily downloading objects outside sparse specification)\n- https://lore.kernel.org/git/CAOLTT8QfwOi9yx_qZZgyGa8iL8kHWutEED7ok_jxwTcYT_hf9Q@mail.gmail.com/\n-\n-[11] (Stolee's comments on high-level usecases)\n- https://lore.kernel.org/git/1a1e33f6-3514-9afc-0a28-5a6b85bd8014@gmail.com/\n+[1] (Original descriptions of behavior A & behavior B):\n+\n+https://lore.kernel.org/git/CABPp-BGJ_Nvi5TmgriD9Bh6eNXE2EDq2f8e8QKXAeYG3BxZafA@mail.gmail.com/\n+\n+[2] (Fix stash applications in sparse checkouts; bugs from behavioral differences):\n+\n+https://lore.kernel.org/git/ccfedc7140dbf63ba26a15f93bd3885180b26517.1606861519.git.gitgitgadget@gmail.com/\n+\n+[3] (Present-despite-skipped entries):\n+\n+https://lore.kernel.org/git/11d46a399d26c913787b704d2b7169cafc28d639.1642175983.git.gitgitgadget@gmail.com/\n+\n+[4] (Clone --no-checkout interaction):\n+\n+https://lore.kernel.org/git/pull.801.v2.git.git.1591324899170.gitgitgadget@gmail.com/ (clone --no-checkout)\n+\n+[5] (The need for update_sparsity() and avoiding `read-tree -mu HEAD`):\n+\n+https://lore.kernel.org/git/3a1f084641eb47515b5a41ed4409a36128913309.1585270142.git.gitgitgadget@gmail.com/\n+\n+[6] (SKIP_WORKTREE is advisory, not mandatory):\n+\n+https://lore.kernel.org/git/844306c3e86ef67591cc086decb2b760e7d710a3.1585270142.git.gitgitgadget@gmail.com/\n+\n+[7] (`worktree add` should copy sparsity settings from current worktree):\n+\n+https://lore.kernel.org/git/c51cb3714e7b1d2f8c9370fe87eca9984ff4859f.1644269584.git.gitgitgadget@gmail.com/\n+\n+[8] (Avoid negative surprises in add, rm, and mv):\n+\n+ * https://lore.kernel.org/git/cover.1617914011.git.matheus.bernardino@usp.br/\n+ * https://lore.kernel.org/git/pull.1018.v4.git.1632497954.gitgitgadget@gmail.com/\n+\n+[9] (Move from out-of-cone to in-cone):\n+\n+ * https://lore.kernel.org/git/20220630023737.473690-6-shaoxuan.yuan02@gmail.com/\n+ * https://lore.kernel.org/git/20220630023737.473690-4-shaoxuan.yuan02@gmail.com/\n+\n+[10] (Unnecessarily downloading objects outside sparse specification):\n+\n+https://lore.kernel.org/git/CAOLTT8QfwOi9yx_qZZgyGa8iL8kHWutEED7ok_jxwTcYT_hf9Q@mail.gmail.com/\n+\n+[11] (Stolee's comments on high-level usecases):\n+\n+https://lore.kernel.org/git/1a1e33f6-3514-9afc-0a28-5a6b85bd8014@gmail.com/\n \n [12] Others commenting on eventually switching default to behavior A:\n+\n * https://lore.kernel.org/git/xmqqh719pcoo.fsf@gitster.g/\n * https://lore.kernel.org/git/xmqqzgeqw0sy.fsf@gitster.g/\n * https://lore.kernel.org/git/a86af661-cf58-a4e5-0214-a67d3a794d7e@github.com/\n \n-[13] Previous config name suggestion and description\n- * https://lore.kernel.org/git/CABPp-BE6zW0nJSStcVU=_DoDBnPgLqOR8pkTXK3dW11=T01OhA@mail.gmail.com/\n+[13] Previous config name suggestion and description:\n+\n+ https://lore.kernel.org/git/CABPp-BE6zW0nJSStcVU=_DoDBnPgLqOR8pkTXK3dW11=T01OhA@mail.gmail.com/\n \n [14] Tangential issue: switch to cone mode as default sparse specification mechanism:\n- https://lore.kernel.org/git/a1b68fd6126eb341ef3637bb93fedad4309b36d0.1650594746.git.gitgitgadget@gmail.com/\n+\n+https://lore.kernel.org/git/a1b68fd6126eb341ef3637bb93fedad4309b36d0.1650594746.git.gitgitgadget@gmail.com/\n \n [15] Lengthy email on grep behavior, covering what should be searched:\n- * https://lore.kernel.org/git/CABPp-BGVO3QdbfE84uF_3QDF0-y2iHHh6G5FAFzNRfeRitkuHw@mail.gmail.com/\n+\n+https://lore.kernel.org/git/CABPp-BGVO3QdbfE84uF_3QDF0-y2iHHh6G5FAFzNRfeRitkuHw@mail.gmail.com/\n \n [16] Email explaining sparsity patterns vs. SKIP_WORKTREE and history operations,\n search for the parenthetical comment starting \"We do not check\".\n- https://lore.kernel.org/git/CABPp-BFsCPPNOZ92JQRJeGyNd0e-TCW-LcLyr0i_+VSQJP+GCg@mail.gmail.com/\n+\n+https://lore.kernel.org/git/CABPp-BFsCPPNOZ92JQRJeGyNd0e-TCW-LcLyr0i_+VSQJP+GCg@mail.gmail.com/\n \n [17] https://lore.kernel.org/git/20220207190320.2960362-1-jonathantanmy@google.com/\n-- \n2.51.0\n\n"},{"id":"527837","messageId":"20251002221233.541844-5-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251002221233.541844-1-ramsay@ramsayjones.plus.com","subject":"[PATCH v2 4/4] doc: commit-graph.adoc: fix up some formatting","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-02T22:12:16Z","receivedAt":"2025-10-02T22:13:33Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"The formatting markup syntax used in this document (markdown?) is not\ninterpreted correctly by asciidoc or asciidoctor. The main problem is\nthe use of a '## ' prefix markup for some sub-headings, along with the\nuse of '```' code markup and some missing literal blocks.\n\nIn order to improve the (html) document formatting:\n\n - replace the '## ' prefix sub-title syntax with the '~~' underlining\n syntax for the relevant sub-headings.\n - replace the '```' code markup, which causes asciidoc(tor) to simply\n remove the marked up text, with a literal block '----' markup.\n - the second ascii diagram, in the 'Merging commit-graph files'\n section, is not rendered correctly by asciidoctor (asciidoc is fine)\n so enclose it in a '....' block.\n\nSigned-off-by: Ramsay Jones \n---\n Documentation/technical/commit-graph.adoc | 29 +++++++++++++++--------\n 1 file changed, 19 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/technical/commit-graph.adoc b/Documentation/technical/commit-graph.adoc\nindex 2c26e95e51..a259d1567b 100644\n--- a/Documentation/technical/commit-graph.adoc\n+++ b/Documentation/technical/commit-graph.adoc\n@@ -39,6 +39,7 @@ A consumer may load the following info for a commit from the graph:\n Values 1-4 satisfy the requirements of parse_commit_gently().\n \n There are two definitions of generation number:\n+\n 1. Corrected committer dates (generation number v2)\n 2. Topological levels (generation number v1)\n \n@@ -158,7 +159,8 @@ number of commits in the full history. By creating a \"chain\" of commit-graphs,\n we enable fast writes of new commit data without rewriting the entire commit\n history -- at least, most of the time.\n \n-## File Layout\n+File Layout\n+~~~~~~~~~~~\n \n A commit-graph chain uses multiple files, and we use a fixed naming convention\n to organize these files. Each commit-graph file has a name\n@@ -170,11 +172,11 @@ hashes for the files in order from \"lowest\" to \"highest\".\n \n For example, if the `commit-graph-chain` file contains the lines\n \n-```\n+----\n \t{hash0}\n \t{hash1}\n \t{hash2}\n-```\n+----\n \n then the commit-graph chain looks like the following diagram:\n \n@@ -213,7 +215,8 @@ specifying the hashes of all files in the lower layers. In the above example,\n `graph-{hash1}.graph` contains `{hash0}` while `graph-{hash2}.graph` contains\n `{hash0}` and `{hash1}`.\n \n-## Merging commit-graph files\n+Merging commit-graph files\n+~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n If we only added a new commit-graph file on every write, we would run into a\n linear search problem through many commit-graph files. Instead, we use a merge\n@@ -225,6 +228,7 @@ is determined by the merge strategy that the files should collapse to\n the commits in `graph-{hash1}` should be combined into a new `graph-{hash3}`\n file.\n \n+....\n \t\t\t +---------------------+\n \t\t\t | |\n \t\t\t | (new commits) |\n@@ -250,6 +254,7 @@ file.\n | |\n | |\n +-----------------------+\n+....\n \n During this process, the commits to write are combined, sorted and we write the\n contents to a temporary file, all while holding a `commit-graph-chain.lock`\n@@ -257,14 +262,15 @@ lock-file. When the file is flushed, we rename it to `graph-{hash3}`\n according to the computed `{hash3}`. Finally, we write the new chain data to\n `commit-graph-chain.lock`:\n \n-```\n+----\n \t{hash3}\n \t{hash0}\n-```\n+----\n \n We then close the lock-file.\n \n-## Merge Strategy\n+Merge Strategy\n+~~~~~~~~~~~~~~\n \n When writing a set of commits that do not exist in the commit-graph stack of\n height N, we default to creating a new file at level N + 1. We then decide to\n@@ -289,7 +295,8 @@ The merge strategy values (2 for the size multiple, 64,000 for the maximum\n number of commits) could be extracted into config settings for full\n flexibility.\n \n-## Handling Mixed Generation Number Chains\n+Handling Mixed Generation Number Chains\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n With the introduction of generation number v2 and generation data chunk, the\n following scenario is possible:\n@@ -318,7 +325,8 @@ have corrected commit dates when written by compatible versions of Git. Thus,\n rewriting split commit-graph as a single file (`--split=replace`) creates a\n single layer with corrected commit dates.\n \n-## Deleting graph-{hash} files\n+Deleting graph-\\{hash\\} files\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n After a new tip file is written, some `graph-{hash}` files may no longer\n be part of a chain. It is important to remove these files from disk, eventually.\n@@ -333,7 +341,8 @@ files whose modified times are older than a given expiry window. This window\n defaults to zero, but can be changed using command-line arguments or a config\n setting.\n \n-## Chains across multiple object directories\n+Chains across multiple object directories\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n In a repo with alternates, we look for the `commit-graph-chain` file starting\n in the local object directory and then in each alternate. The first file that\n-- \n2.51.0\n\n"},{"id":"527853","messageId":"1a72434f-7935-4d0c-868f-03bd24601d4d@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251002221233.541844-1-ramsay@ramsayjones.plus.com","subject":"Re: [PATCH v2 0/4] technical docs in make build","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-02T22:38:27Z","receivedAt":"2025-10-02T22:41:37Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 02/10/2025 11:12 pm, Ramsay Jones wrote:\n> OK, so I have recently developed an intense dislike of both asciidoc\n> and asciidoctor. :)\n> \n\nHeh, sorry about this, but I messed up the threading (again). This time, for\nsome unknown reason I pasted the 'lore.kernel.org' URL for the v1 cover letter,\nrather than the message-ID:\n\n \n\nI shouldn't be allowed to operate 'git send-email' after dark! :)\n\nATB,\nRamsay Jones\n\n\n\n\n"},{"id":"528088","messageId":"b771b1ca-96a2-4dc1-8c66-0a3006f18565@app.fastmail.com","threadId":"64240","inReplyTo":"20251002221233.541844-4-ramsay@ramsayjones.plus.com","subject":"Re: [PATCH v2 3/4] doc: sparse-checkout.adoc: fix asciidoc warnings","fromName":"Kristoffer Haugsbakk","fromEmail":"kristofferhaugsbakk@fastmail.com","sentAt":"2025-10-07T12:20:52Z","receivedAt":"2025-10-07T12:21:17Z","isPatch":true,"sender":{"key":"kristofferhaugsbakk@fastmail.com","avatar":null},"body":"On Fri, Oct 3, 2025, at 00:12, Ramsay Jones wrote:\n>[snip]\n>\n> In order to address the first set of warnings, simply renumber the list\n> from one to severn, rather than zero to six. Fortunately, this does not\n\ns/severn/seven/\n\n>[snip]\n"},{"id":"528177","messageId":"436fb507-6764-46f4-abb1-34c82e27b808@ramsayjones.plus.com","threadId":"64240","inReplyTo":"b771b1ca-96a2-4dc1-8c66-0a3006f18565@app.fastmail.com","subject":"Re: [PATCH v2 3/4] doc: sparse-checkout.adoc: fix asciidoc warnings","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-07T22:17:47Z","receivedAt":"2025-10-07T22:20:57Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 07/10/2025 1:20 pm, Kristoffer Haugsbakk wrote:\n> On Fri, Oct 3, 2025, at 00:12, Ramsay Jones wrote:\n>> [snip]\n>>\n>> In order to address the first set of warnings, simply renumber the list\n>> from one to severn, rather than zero to six. Fortunately, this does not\n> \n> s/severn/seven/\n\nThanks. I have updated locally, while (hopefully) waiting for more feedback.\n\nATB,\nRamsay Jones\n\n\n"},{"id":"528191","messageId":"CABPp-BGiziz6-7zyq+Z-f0g+JDPMpGuXanmXNEM=0hV-7jKNsQ@mail.gmail.com","threadId":"64240","inReplyTo":"20251002221233.541844-3-ramsay@ramsayjones.plus.com","subject":"Re: [PATCH v2 2/4] doc: remembering-renames.adoc: fix asciidoc warnings","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-10-08T03:51:38Z","receivedAt":"2025-10-08T03:51:50Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Oct 2, 2025 at 3:13 PM Ramsay Jones wrote:\n>\n> Both asciidoc and ascidoctor issue warnings about 'list item index:\n> expected n got n-1' for n=1->9 on lines 13, 15, 17, 20, 23, 25, 29,\n> 31 and 33. In asciidoc, numbered lists must start at one, whereas this\n> file has a list starting at zero. Also, asciidoc and asciidoctor warn\n> about 'section title out of sequence: expected level 1, got level 2'\n> on line 38. (asciidoc only complains about the first instance of this,\n> while asciidoctor complains about them all, on lines 94, 141, 142,\n> 184, 185, 257, 288, 289, 290, 397, 424, 485, 486 and 487). These\n> warnings stem from the section titles not being correctly nested within\n> a document/chapter title.\n>\n> In order to address the first set of warnings, simply renumber the list\n> from one to nine, rather than zero to eight. This also requires altering\n> the text which refers to the section numbers, including other section\n> titles.\n>\n> In order to address the second set of warnings, change the section title\n> syntax from '=== title ===' to '== title ==', effectively reducing the\n> nesting level of the title by one. Also, some of the titles are given\n> over multiple lines (they are very long), with an title '===' prefix\n> on each line. This leads to them being treated as separate sections\n> with no body text (as you can see from the line numbers given for the\n> asciidoctor warnings, above). So, for these titles, turn them into a\n> single (long) line of text.\n>\n> In addition to the warnings, address some other formatting issues:\n>\n> - the ascii branch diagrams didn't format correctly on asciidoctor\n> so include them in a literal block.\n> - several blocks of text were intended to be formatted 'as is' but\n> were not included in a literal block.\n> - in section 8, format the (A)->(D) in the text description as a\n> literal with `` marks, since (C) is rendered as a copyright\n> symbol in html otherwise.\n> - in section 9, a sub-list of two items is not formatted as such.\n> change the '*' introducer to '**' to correct the sub-list format.\n\nSorry to put you through all this work. I had no idea the stuff under\nDocumentation/technical/ was ever meant to be run through\nasciidoc/asciidoctor. The .txt ending didn't hint at anything like\nthis; I mean, sure lots of other files were put through those, but I\nassumed this directory was just stuff for other Git developers...\n\n> -=== 0. Assumptions ===\n> +== 1. Assumptions ==\n\nIt doesn't like '===' but is fine with '=='? I'm a bit surprised. If\nit was about nesting, wouldn't '==' also complain since there is no\n'=' headers anywhere.\n"},{"id":"528192","messageId":"CABPp-BEYF6MdcaXU1qAYctRBAt754j7PGkE3Tgjmm03bBkBjNQ@mail.gmail.com","threadId":"64240","inReplyTo":"20251002221233.541844-4-ramsay@ramsayjones.plus.com","subject":"Re: [PATCH v2 3/4] doc: sparse-checkout.adoc: fix asciidoc warnings","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-10-08T03:57:41Z","receivedAt":"2025-10-08T03:57:53Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Oct 2, 2025 at 3:13 PM Ramsay Jones wrote:\n>\n> Both asciidoc and asciidoctor issue warnings about 'list item index:\n> expected n got n-1' for n=1->7 on lines 928, 931, 951, 974, 980, 1033\n> and 1049. In asciidoc, numbered lists must start at one, whereas this\n> file has a list starting at zero. Also, asciidoc and asciidoctor warn\n> about 'section title out of sequence: expected level 1, got level 2'\n> on line 17. (asciidoc only complains about the first instance of this,\n> while asciidoctor complains about them all, on lines 95, 258, 303, 316,\n> 545, 612, 752, 824, 895, 923 and 1053). These warnings stem from the\n> section titles not being correctly nested within a document/chapter\n> title.\n>\n> In order to address the first set of warnings, simply renumber the list\n> from one to severn, rather than zero to six. Fortunately, this does not\n> require altering additional text, since the enumeration of 'Known Bugs'\n> is not referred to anywhere else in the document.\n>\n> In order to address the second set of warnings, change the section title\n> syntax from '=== title ===' to '== title ==', effectively reducing the\n> nesting level of the title by one. Also, some apparent (sub-)titles are\n> not marked up with sub-title syntax, so add some '=== ' prefix(s) to the\n> relevant headings.\n\nKinda surprising; if it's complaining about lack of title nesting, I'd\nthink you'd need a '= title =' somewhere before using '== title =='.\nMaybe jumping skipping one nesting level it's fine with, but skipping\ntwo is where the problem starts? No idea.\n\n> In addition to the warnings, address some other formatting issues:\n>\n> - the use of heavily nested unordered lists is not reflected in the\n> output (making the file totally unreadable) because each level of\n> nesting requires a different syntax. (i.e. replace '*' with '**'\n> for the second level, '*' with '***' for the third level, etc.)\n> - make use of literal blocks and manual indentation to get asciidoc\n> and asciidoctor to display even remotely similar output.\n> - make use of labelled lists, in some places, to get a similar looking\n> output to the input, for both asciidoc and asciidoctor.\n> - replace the trailing space in: `git grep ${SEARCH_TERM} OLDREV `\n> otherwise the entire line in which that appears is removed from\n> the output.\n\nAgain, sorry for putting you through all this; I had assumed\nDocumentation/technical/ was stuff meant for other Git developers to\nsee and didn't need to be typeset with asciidoc or asciidoctor and had\nnever attempted to run the documents I added there under either.\nSomeone else renamed them to .adoc...\n\n\nI skimmed through the document, and it all looked like typesetting\nchanges which don't impair the readability of the source text, so\nseems fine to me. (Same with the previous patch)\n"},{"id":"528209","messageId":"aOYImjMXcFkdwar5@pks.im","threadId":"64240","inReplyTo":"20251002221233.541844-2-ramsay@ramsayjones.plus.com","subject":"Re: [PATCH v2 1/4] doc: add some missing technical documents","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-10-08T06:45:46Z","receivedAt":"2025-10-08T06:45:53Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Oct 02, 2025 at 11:12:13PM +0100, Ramsay Jones wrote:\n> Commit bcf7edee09 (\"meson: generate articles\", 2024-12-27) added the\n> generation of the 'howto' and 'technical' documents to the meson build.\n> At this time those documents had a '*.txt' file extension, but they were\n> renamed with an '*.adoc' extension by commit 1f010d6bdf (\"doc: use .adoc\n> extension for AsciiDoc files\", 2025-01-20), for the most part. For the\n> meson build, commit 87eccc3a81 (\"meson: fix building technical and howto\n> docs\", 2025-03-02) fixed the meson.build files, which had not been\n> updated when the files were renamed.\n> \n> However, the 'Documentation/Makefile' has not been updated to include\n> all of the recently added technical documents. In particular, the\n> following are built by meson, but not by the Makefile:\n> \n> commit-graph.adoc\n> directory-rename-detection.adoc\n> packfile-uri.adoc\n> remembering-renames.adoc\n> repository-version.adoc\n> rerere.adoc\n> sparse-checkout.adoc\n> sparse-index.adoc\n> \n> In order to ensure that both build systems format the same technical\n> documents, add the above documents to the TECH_DOCS variable in the\n> Documentation/Makefile.\n\nI was wondering whether we also want to have a change like the\nfollowing:\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 6fb83d0c6e..666b0b6fbd 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -524,15 +524,20 @@ lint-docs-manpages:\n lint-docs-meson:\n \t@# awk acts up when trying to match single quotes, so we use \\047 instead.\n \t@mkdir -p tmp-meson-diff && \\\n-\tawk \"/^manpages = {$$/ {flag=1 ; next } /^}$$/ { flag=0 } flag { gsub(/^ \\047/, \\\"\\\"); gsub(/\\047 : [157],\\$$/, \\\"\\\"); print }\" meson.build | \\\n+\t{ \\\n+\t\tawk \"/^manpages = {$$/ {flag=1 ; next } /^}$$/ { flag=0 } flag { gsub(/^ \\047/, \\\"\\\"); gsub(/\\047 : [157],\\$$/, \\\"\\\"); print }\" meson.build && \\\n+\t\tawk \"/^articles = \\[$$/ {flag=1 ; next } /^\\]$$/ { flag=0 } flag { gsub(/^ \\047/, \\\"\\\"); gsub(/\\047,$$/, \\\"\\\"); print }\" technical/meson.build; \\\n+\t} | \\\n \t\tgrep -v -e '#' -e '^$$' | \\\n \t\tsort >tmp-meson-diff/meson.adoc && \\\n-\tls git*.adoc scalar.adoc | \\\n+\tls git*.adoc scalar.adoc technical/*.adoc | \\\n+\t\txargs -n1 basename | \\\n \t\tgrep -v -e git-bisect-lk2009.adoc \\\n \t\t\t-e git-pack-redundant.adoc \\\n \t\t\t-e git-tools.adoc \\\n \t\t\t-e git-whatchanged.adoc \\\n-\t\t\t>tmp-meson-diff/actual.adoc && \\\n+\t\t\t-e api-.*.adoc | \\\n+\t\t\tsort >tmp-meson-diff/actual.adoc && \\\n \tif ! cmp tmp-meson-diff/meson.adoc tmp-meson-diff/actual.adoc; then \\\n \t\techo \"Meson man pages differ from actual man pages:\"; \\\n \t\tdiff -u tmp-meson-diff/meson.adoc tmp-meson-diff/actual.adoc; \\\n\nThis builds on our existing linting rule and would catch any discrepancy\nin man pages that we have in \"Documentation/technical/\" that isn't\nlisted in Meson.\n\nThis check isn't quite complete, there's two things missing:\n\n - We have an equivalent check in \"Documentation/meson.build\" that we\n might want to extend to also cover articles.\n\n - We don't have a check to ensure that our Makefile and Meson are in\n sync.\n\nBut regardless of that, the above check surfaces one more missing\narticle:\n\n $ make lint-docs-meson\n GEN doc.dep\n make: *** Deleting file 'doc.dep'\n tmp-meson-diff/meson.adoc tmp-meson-diff/actual.adoc differ: byte 3877, line 206\n Meson man pages differ from actual man pages:\n --- tmp-meson-diff/meson.adoc\t2025-10-08 08:42:49.864991169 +0200\n +++ tmp-meson-diff/actual.adoc\t2025-10-08 08:42:50.072988794 +0200\n @@ -203,6 +203,7 @@\n git-worktree.adoc\n git-write-tree.adoc\n hash-function-transition.adoc\n +large-object-promisors.adoc\n long-running-process-protocol.adoc\n multi-pack-index.adoc\n packfile-uri.adoc\n make: *** [Makefile:526: lint-docs-meson] Error 1\n\nPatrick\n"},{"id":"528288","messageId":"xmqqfrbtfcbv.fsf@gitster.g","threadId":"64240","inReplyTo":"aOYImjMXcFkdwar5@pks.im","subject":"Re: [PATCH v2 1/4] doc: add some missing technical documents","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-08T19:00:36Z","receivedAt":"2025-10-08T19:00:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> This builds on our existing linting rule and would catch any discrepancy\n> in man pages that we have in \"Documentation/technical/\" that isn't\n> listed in Meson.\n\nYeah, I remember the existing check helping me spot potential issues\nin a series or two.\n\n> But regardless of that, the above check surfaces one more missing\n> article:\n>\n> $ make lint-docs-meson\n> GEN doc.dep\n> make: *** Deleting file 'doc.dep'\n> tmp-meson-diff/meson.adoc tmp-meson-diff/actual.adoc differ: byte 3877, line 206\n> Meson man pages differ from actual man pages:\n> --- tmp-meson-diff/meson.adoc\t2025-10-08 08:42:49.864991169 +0200\n> +++ tmp-meson-diff/actual.adoc\t2025-10-08 08:42:50.072988794 +0200\n> @@ -203,6 +203,7 @@\n> git-worktree.adoc\n> git-write-tree.adoc\n> hash-function-transition.adoc\n> +large-object-promisors.adoc\n> long-running-process-protocol.adoc\n> multi-pack-index.adoc\n> packfile-uri.adoc\n> make: *** [Makefile:526: lint-docs-meson] Error 1\n\nGood. I'll expect Ramsay will handle this one in v3?\n\nThanks.\n"},{"id":"528331","messageId":"f75779d4-9a92-4681-ae91-83ca7724c655@ramsayjones.plus.com","threadId":"64240","inReplyTo":"CABPp-BGiziz6-7zyq+Z-f0g+JDPMpGuXanmXNEM=0hV-7jKNsQ@mail.gmail.com","subject":"Re: [PATCH v2 2/4] doc: remembering-renames.adoc: fix asciidoc warnings","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-08T21:38:39Z","receivedAt":"2025-10-08T21:41:50Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 08/10/2025 4:51 am, Elijah Newren wrote:\n> On Thu, Oct 2, 2025 at 3:13 PM Ramsay Jones wrote:\n>>\n>> Both asciidoc and ascidoctor issue warnings about 'list item index:\n>> expected n got n-1' for n=1->9 on lines 13, 15, 17, 20, 23, 25, 29,\n>> 31 and 33. In asciidoc, numbered lists must start at one, whereas this\n>> file has a list starting at zero. Also, asciidoc and asciidoctor warn\n>> about 'section title out of sequence: expected level 1, got level 2'\n>> on line 38. (asciidoc only complains about the first instance of this,\n>> while asciidoctor complains about them all, on lines 94, 141, 142,\n>> 184, 185, 257, 288, 289, 290, 397, 424, 485, 486 and 487). These\n>> warnings stem from the section titles not being correctly nested within\n>> a document/chapter title.\n>>\n>> In order to address the first set of warnings, simply renumber the list\n>> from one to nine, rather than zero to eight. This also requires altering\n>> the text which refers to the section numbers, including other section\n>> titles.\n>>\n>> In order to address the second set of warnings, change the section title\n>> syntax from '=== title ===' to '== title ==', effectively reducing the\n>> nesting level of the title by one. Also, some of the titles are given\n>> over multiple lines (they are very long), with an title '===' prefix\n>> on each line. This leads to them being treated as separate sections\n>> with no body text (as you can see from the line numbers given for the\n>> asciidoctor warnings, above). So, for these titles, turn them into a\n>> single (long) line of text.\n>>\n>> In addition to the warnings, address some other formatting issues:\n>>\n>> - the ascii branch diagrams didn't format correctly on asciidoctor\n>> so include them in a literal block.\n>> - several blocks of text were intended to be formatted 'as is' but\n>> were not included in a literal block.\n>> - in section 8, format the (A)->(D) in the text description as a\n>> literal with `` marks, since (C) is rendered as a copyright\n>> symbol in html otherwise.\n>> - in section 9, a sub-list of two items is not formatted as such.\n>> change the '*' introducer to '**' to correct the sub-list format.\n> \n> Sorry to put you through all this work. I had no idea the stuff under\n> Documentation/technical/ was ever meant to be run through\n> asciidoc/asciidoctor. The .txt ending didn't hint at anything like\n> this; I mean, sure lots of other files were put through those, but I\n> assumed this directory was just stuff for other Git developers...\n\nAs I mentioned in my cover letter, I didn't think these documents were\never meant to be submitted to asciidoc(tor) either, but had to assume\nthat the current policy required it; so, I had to show willing ... :)\n\nIf it was not already obvious, until this patch series I had managed to\ncompletely avoid any knowledge of 'asciidoc standard markup' (which appears\nto be anything but standard)!\n\n>> -=== 0. Assumptions ===\n>> +== 1. Assumptions ==\n> \n> It doesn't like '===' but is fine with '=='? I'm a bit surprised. If\n> it was about nesting, wouldn't '==' also complain since there is no\n> '=' headers anywhere.\n> \n\nYep, '=' is a level 0 header, but the asciidoc message said 'expected\nlevel 1, got level 2', so I just dropped it down one level and asciidoc(tor)\nwas happy!\n\nThanks.\n\nATB,\nRamsay Jones\n\n\n"},{"id":"528334","messageId":"05bc7369-af6a-45db-a792-a452d2442dbb@ramsayjones.plus.com","threadId":"64240","inReplyTo":"CABPp-BEYF6MdcaXU1qAYctRBAt754j7PGkE3Tgjmm03bBkBjNQ@mail.gmail.com","subject":"Re: [PATCH v2 3/4] doc: sparse-checkout.adoc: fix asciidoc warnings","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-08T21:54:25Z","receivedAt":"2025-10-08T21:54:28Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 08/10/2025 4:57 am, Elijah Newren wrote:\n> On Thu, Oct 2, 2025 at 3:13 PM Ramsay Jones wrote:\n>>\n>> Both asciidoc and asciidoctor issue warnings about 'list item index:\n>> expected n got n-1' for n=1->7 on lines 928, 931, 951, 974, 980, 1033\n>> and 1049. In asciidoc, numbered lists must start at one, whereas this\n>> file has a list starting at zero. Also, asciidoc and asciidoctor warn\n>> about 'section title out of sequence: expected level 1, got level 2'\n>> on line 17. (asciidoc only complains about the first instance of this,\n>> while asciidoctor complains about them all, on lines 95, 258, 303, 316,\n>> 545, 612, 752, 824, 895, 923 and 1053). These warnings stem from the\n>> section titles not being correctly nested within a document/chapter\n>> title.\n>>\n>> In order to address the first set of warnings, simply renumber the list\n>> from one to severn, rather than zero to six. Fortunately, this does not\n>> require altering additional text, since the enumeration of 'Known Bugs'\n>> is not referred to anywhere else in the document.\n>>\n>> In order to address the second set of warnings, change the section title\n>> syntax from '=== title ===' to '== title ==', effectively reducing the\n>> nesting level of the title by one. Also, some apparent (sub-)titles are\n>> not marked up with sub-title syntax, so add some '=== ' prefix(s) to the\n>> relevant headings.\n> \n> Kinda surprising; if it's complaining about lack of title nesting, I'd\n> think you'd need a '= title =' somewhere before using '== title =='.\n> Maybe jumping skipping one nesting level it's fine with, but skipping\n> two is where the problem starts? No idea.\n\nI have no idea either! see previous email.\n\n> \n>> In addition to the warnings, address some other formatting issues:\n>>\n>> - the use of heavily nested unordered lists is not reflected in the\n>> output (making the file totally unreadable) because each level of\n>> nesting requires a different syntax. (i.e. replace '*' with '**'\n>> for the second level, '*' with '***' for the third level, etc.)\n>> - make use of literal blocks and manual indentation to get asciidoc\n>> and asciidoctor to display even remotely similar output.\n>> - make use of labelled lists, in some places, to get a similar looking\n>> output to the input, for both asciidoc and asciidoctor.\n>> - replace the trailing space in: `git grep ${SEARCH_TERM} OLDREV `\n>> otherwise the entire line in which that appears is removed from\n>> the output.\n> \n> Again, sorry for putting you through all this; I had assumed\n> Documentation/technical/ was stuff meant for other Git developers to\n> see and didn't need to be typeset with asciidoc or asciidoctor and had\n> never attempted to run the documents I added there under either.\n> Someone else renamed them to .adoc...\n\nNo problem. I already floated the idea of renaming these files to .txt\nand removing them from the meson build (in my cover letter), but I had\nto assume that it was now the policy for these docs to be formatted.\n\nI was very conscious of me butchering your documents (and Derrick's) to\nmake an attempt to fix-up the formatting. It was quite frustrating to\nfind that asciidoc and asciidoctor don't agree on how that should be\ndone ... (frequently). :(\n\n[I was hopeful that an asciidoc guru would help me fix the two remaining\nproblems (that I know about) - fingers crossed!]\n\n> I skimmed through the document, and it all looked like typesetting\n> changes which don't impair the readability of the source text, so\n> seems fine to me. (Same with the previous patch)\n\nI hoped that would be the case, but I must say that I think you are\nbeing very generous! ;)\n\nThanks.\n\nATB,\nRamsay Jones\n\n\n\n"},{"id":"528336","messageId":"8e3aba11-90d7-4336-9cd4-b1fb4144bf69@ramsayjones.plus.com","threadId":"64240","inReplyTo":"aOYImjMXcFkdwar5@pks.im","subject":"Re: [PATCH v2 1/4] doc: add some missing technical documents","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-08T21:56:45Z","receivedAt":"2025-10-08T21:56:48Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 08/10/2025 7:45 am, Patrick Steinhardt wrote:\n> On Thu, Oct 02, 2025 at 11:12:13PM +0100, Ramsay Jones wrote:\n[snip]\n\n> This builds on our existing linting rule and would catch any discrepancy\n> in man pages that we have in \"Documentation/technical/\" that isn't\n> listed in Meson.\n> \n> This check isn't quite complete, there's two things missing:\n> \n> - We have an equivalent check in \"Documentation/meson.build\" that we\n> might want to extend to also cover articles.\n> \n> - We don't have a check to ensure that our Makefile and Meson are in\n> sync.\n> \n> But regardless of that, the above check surfaces one more missing\n> article:\n> \n> $ make lint-docs-meson\n> GEN doc.dep\n> make: *** Deleting file 'doc.dep'\n> tmp-meson-diff/meson.adoc tmp-meson-diff/actual.adoc differ: byte 3877, line 206\n> Meson man pages differ from actual man pages:\n> --- tmp-meson-diff/meson.adoc\t2025-10-08 08:42:49.864991169 +0200\n> +++ tmp-meson-diff/actual.adoc\t2025-10-08 08:42:50.072988794 +0200\n> @@ -203,6 +203,7 @@\n> git-worktree.adoc\n> git-write-tree.adoc\n> hash-function-transition.adoc\n> +large-object-promisors.adoc\n> long-running-process-protocol.adoc\n> multi-pack-index.adoc\n> packfile-uri.adoc\n> make: *** [Makefile:526: lint-docs-meson] Error 1\n\nSo, it has already paid for itself!\n\nThanks.\n\nATB,\nRamsay Jones\n\n\n"},{"id":"528339","messageId":"3286707e-8cc0-430e-a2d3-546352d50b6d@ramsayjones.plus.com","threadId":"64240","inReplyTo":"xmqqfrbtfcbv.fsf@gitster.g","subject":"Re: [PATCH v2 1/4] doc: add some missing technical documents","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-08T22:01:18Z","receivedAt":"2025-10-08T22:01:21Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 08/10/2025 8:00 pm, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n>> This builds on our existing linting rule and would catch any discrepancy\n>> in man pages that we have in \"Documentation/technical/\" that isn't\n>> listed in Meson.\n> \n> Yeah, I remember the existing check helping me spot potential issues\n> in a series or two.\n> \n>> But regardless of that, the above check surfaces one more missing\n>> article:\n>>\n>> $ make lint-docs-meson\n>> GEN doc.dep\n>> make: *** Deleting file 'doc.dep'\n>> tmp-meson-diff/meson.adoc tmp-meson-diff/actual.adoc differ: byte 3877, line 206\n>> Meson man pages differ from actual man pages:\n>> --- tmp-meson-diff/meson.adoc\t2025-10-08 08:42:49.864991169 +0200\n>> +++ tmp-meson-diff/actual.adoc\t2025-10-08 08:42:50.072988794 +0200\n>> @@ -203,6 +203,7 @@\n>> git-worktree.adoc\n>> git-write-tree.adoc\n>> hash-function-transition.adoc\n>> +large-object-promisors.adoc\n>> long-running-process-protocol.adoc\n>> multi-pack-index.adoc\n>> packfile-uri.adoc\n>> make: *** [Makefile:526: lint-docs-meson] Error 1\n> \n> Good. I'll expect Ramsay will handle this one in v3?\n\nOK, will do.\n\nSince patch #1 is already in 'next', do I effectively create a new\npatch series out of patches #2->#4, plus this new patch?\n\nATB,\nRamsay Jones\n\n\n"},{"id":"528344","messageId":"xmqqms61dnwo.fsf@gitster.g","threadId":"64240","inReplyTo":"3286707e-8cc0-430e-a2d3-546352d50b6d@ramsayjones.plus.com","subject":"Re: [PATCH v2 1/4] doc: add some missing technical documents","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-08T22:33:27Z","receivedAt":"2025-10-08T22:33:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramsay Jones writes:\n\n> Since patch #1 is already in 'next', do I effectively create a new\n> patch series out of patches #2->#4, plus this new patch?\n\nThat would be great. Your original was one patch with 3 RFC patches\non top, so they are queued separately already on two different topic\nbranches.\n"},{"id":"528997","messageId":"20251016200301.1595204-1-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"1a72434f-7935-4d0c-868f-03bd24601d4d@ramsayjones.plus.com","subject":"[PATCH v3 0/4] technical docs in make build","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-16T20:02:57Z","receivedAt":"2025-10-16T20:06:27Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Changes in v3:\n\n- old patch #1 discarded since it was separated into its own branch\n ('rj/doc-missing-technical-docs' in next)\n- tyop in patch #2 (old patch #3)\n- new patch #4\n\nA range diff against v2 is given below.\n\nNote that the two remaining problems (see v2 below) have not been\naddressed but, even without a solution, these patches represent a\ngood improvement. ;) (I am still hopeful that an asciidoc guru will\nturn up!)\n\nNOTE: this series is based on the v2-version of the patch #1, which\nin turn is based on commit 6ad8021821 (\"The fifth batch\", 2025-08-29).\n\nv2 cover letter:\n\nOK, so I have recently developed an intense dislike of both asciidoc\nand asciidoctor. :)\n\nChanges in v2:\n\n - Actual commit messages\n - (almost) total re-write of patches #2 and #3\n - removed the RFC from patches #2->#4\n\nI have not included a range-diff, because it doesn't show anything\ninteresting/readable with or without a large --creation-factor!\n\nThere are two issues I am aware of:\n\n - mis-formatting of monospaced text containing an '{' character\n mentioned in the original cover letter below. I have not found\n a fix for this, but there are other examples in patch #3!\n - breakage of two html links representing URLS pointing to emails\n at 'lore.kernel.org'. I don't think it is a coincidence that it\n is only these two references that contain a reserved character;\n a '+' in the first (see known bugs 7) and two (separate) '='\n characters in the second (mail ref [13]). I tried %encoding\n them, but that didn't make any difference.\n\nThere are probably other formatting issues that I am not aware of!\n\n\nOriginal cover letter:\n\nI have been trying to get back to the 'misc build updates (part #3)'\npatches, so that I can send them to the list, but I have not been able\nto find a spare minute for quite some time. :(\n\nHowever, this sub-sequence of patches hangs together as a single theme and\nI need help to finish them up! (asciidoc is not my forte).\n\nThe first patch adds some technical documents to the Makefile build which\nare already part of the meson build. In particular, the following are\nbuilt by meson, but not by the Makefile:\n\n commit-graph.adoc\n directory-rename-detection.adoc\n packfile-uri.adoc\n remembering-renames.adoc\n repository-version.adoc\n rerere.adoc\n sparse-checkout.adoc\n sparse-index.adoc\n\nAlthough I am not convinced that some of these files were ever meant to be\nformatted by asciidoc, I have assumed that is the case for the purposes of\nthis patch series. (otherwise, we should remove them from the meson build\nand rename the files instead).\n\nWhen I attempt to build the html docs, with patch #1 applied, on Linux:\n\n $ make html >out-doc 2>&1\n\n $ grep SyntaxWarning out-doc | head -n1\n :1: SyntaxWarning: invalid escape sequence '\\S'\n $ grep SyntaxWarning out-doc | wc -l\n 524\n $ \n\n $ asciidoc --version\n asciidoc 10.2.0\n $ python3 --version\n Python 3.12.3\n $ \n\nThis is caused by the python version I am using, which was recently changed\n(in version 3.12) to issue the SyntaxWarning when a 'non-raw' string contains\nsome escape sequences (here \\S). [some versions prior to 3.12 used to issue\na deprecation warning].\n\nThis is a known issue, see e.g. [0], which has been addressed by a patch [1],\nand as seen in [2] has been included in a new version 10.2.1 of asciidoc.\n\n [0] https://trac.macports.org/ticket/70039\n [1] 1https://github.com/asciidoc-py/asciidoc-py/pull/267\n [2] https://github.com/asciidoc-py/asciidoc-py/commits/main/\n\n[cygwin does not have this problem, because the phython version is 3.9.16]\n\nSo, ignoring that issue, we still see some warnings from asciidoc:\n\n $ grep WARNING out-doc\n asciidoc: WARNING: remembering-renames.adoc: line 13: list item index: expected 1 got 0\n asciidoc: WARNING: remembering-renames.adoc: line 15: list item index: expected 2 got 1\n asciidoc: WARNING: remembering-renames.adoc: line 17: list item index: expected 3 got 2\n asciidoc: WARNING: remembering-renames.adoc: line 20: list item index: expected 4 got 3\n asciidoc: WARNING: remembering-renames.adoc: line 23: list item index: expected 5 got 4\n asciidoc: WARNING: remembering-renames.adoc: line 25: list item index: expected 6 got 5\n asciidoc: WARNING: remembering-renames.adoc: line 29: list item index: expected 7 got 6\n asciidoc: WARNING: remembering-renames.adoc: line 31: list item index: expected 8 got 7\n asciidoc: WARNING: remembering-renames.adoc: line 33: list item index: expected 9 got 8\n asciidoc: WARNING: remembering-renames.adoc: line 38: section title out of sequence: expected level 1, got level 2\n asciidoc: WARNING: sparse-checkout.adoc: line 17: section title out of sequence: expected level 1, got level 2\n asciidoc: WARNING: sparse-checkout.adoc: line 928: list item index: expected 1 got 0\n asciidoc: WARNING: sparse-checkout.adoc: line 931: list item index: expected 2 got 1\n asciidoc: WARNING: sparse-checkout.adoc: line 951: list item index: expected 3 got 2\n asciidoc: WARNING: sparse-checkout.adoc: line 974: list item index: expected 4 got 3\n asciidoc: WARNING: sparse-checkout.adoc: line 980: list item index: expected 5 got 4\n asciidoc: WARNING: sparse-checkout.adoc: line 1033: list item index: expected 6 got 5\n asciidoc: WARNING: sparse-checkout.adoc: line 1049: list item index: expected 7 got 6\n $ \n\nI also tried asciidoctor, just for fun:\n\n $ asciidoctor --version\n Asciidoctor 2.0.20 [https://asciidoctor.org]\n Runtime Environment (ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [x86_64-linux-gnu]) (lc:UTF-8 fs:UTF-8 in:UTF-8 ex:UTF-8)\n $ \n\n $ make USE_ASCIIDOCTOR=1 html >out-doctor 2>&1\n\n $ grep WARNING out-doctor\n asciidoctor: WARNING: remembering-renames.adoc: line 13: list item index: expected 1, got 0\n asciidoctor: WARNING: remembering-renames.adoc: line 15: list item index: expected 2, got 1\n asciidoctor: WARNING: remembering-renames.adoc: line 17: list item index: expected 3, got 2\n asciidoctor: WARNING: remembering-renames.adoc: line 20: list item index: expected 4, got 3\n asciidoctor: WARNING: remembering-renames.adoc: line 23: list item index: expected 5, got 4\n asciidoctor: WARNING: remembering-renames.adoc: line 25: list item index: expected 6, got 5\n asciidoctor: WARNING: remembering-renames.adoc: line 29: list item index: expected 7, got 6\n asciidoctor: WARNING: remembering-renames.adoc: line 31: list item index: expected 8, got 7\n asciidoctor: WARNING: remembering-renames.adoc: line 33: list item index: expected 9, got 8\n asciidoctor: WARNING: remembering-renames.adoc: line 38: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 94: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 141: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 142: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 184: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 185: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 257: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 288: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 289: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 290: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 397: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 424: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 485: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 486: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: remembering-renames.adoc: line 487: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 17: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 95: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 258: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 303: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 316: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 545: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 612: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 752: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 824: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 895: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 923: section title out of sequence: expected level 1, got level 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 928: list item index: expected 1, got 0\n asciidoctor: WARNING: sparse-checkout.adoc: line 931: list item index: expected 2, got 1\n asciidoctor: WARNING: sparse-checkout.adoc: line 951: list item index: expected 3, got 2\n asciidoctor: WARNING: sparse-checkout.adoc: line 974: list item index: expected 4, got 3\n asciidoctor: WARNING: sparse-checkout.adoc: line 980: list item index: expected 5, got 4\n asciidoctor: WARNING: sparse-checkout.adoc: line 1033: list item index: expected 6, got 5\n asciidoctor: WARNING: sparse-checkout.adoc: line 1049: list item index: expected 7, got 6\n asciidoctor: WARNING: sparse-checkout.adoc: line 1053: section title out of sequence: expected level 1, got level 2\n $ \n\nYou can see that asciidoc only complains about the first 'section title out of\nsequence', whereas asciidoctor complains about them all.\n\n[asciidoctor also reports:\nNote: namesp. cut : stripped namespace before processing Git User Manual]\n\nPatch #2 was a nightmare which I really gave up on! :) An early attempt\ninvolved renumbering the 'outline list' at the top from 0->8 to 1->9\n(I thought there was a way to start numbering at zero, but I lost a lot\nof time trying to do so, without any success). So, of course I 'just'\ntried global search/replace in vim to do the renumbering (backwards).\nThis was a complete disaster (of course), which I 'fixed' many many times.\n(Not everything which is numbered is a section, there are 'cases' as well).\n\nIn the end, I just disabled the 'outline' list, by removing the period\non the numbers (again '0\\. Assumptions' should have worked, but didn't)\nand fixing up the section titles without renumbering them. Note that\nasciidoctor mis-formats the 'ascii branch diagrams', which asciidoc\nformats correctly. I think there are other formatting problems left.\n\nIn patch #3, the formatting changes are confined to the section titles and\nrenumbering the 'known bugs' from 0->6 to 1->7. (I think I noticed some\nsub-sub lists which are not formatted correctly, but I don't seem to be\nable to see them now ...).\n\nIn patch #4, most of the formatting changes relate to section titles, but\nI could not fix some inline text formatting starting at 'File Layouts'\n(within 'Commit-Graph Chains') with text that is monospaced with `` but\nalso contains an '{' character. For example:\n\n `$OBJDIR/info/commit-graphs/graph-{hash}.graph`\n\nis monospaced (blue colour with asciidoc) up until the {hash}.graph which\ndoes not have any formatting. (It is not so noticeable with asciidoctor\nbecause the formatting consists of a *very* subtle gray background to the\ntext which, to my eyes anyway, is almost not visible).\n\nI have tried several suggestions from an on-line asciidoc syntax cheatsheet\nsuch as:\n\n `$OBJDIR/info/commit-graphs/graph-\\{hash\\}.graph`\n `+$OBJDIR/info/commit-graphs/graph-{hash}.graph+`\n\nbut nothing worked. Note that there are many similar instances of this\nproblem (including just `{hash}`).\n\nNote also that asciidoctor did not render the second diagram correctly\n(the one in 'Merging commit-graph files'), but asciidoc was just fine.\n\nThe remaining documents:\n\n directory-rename-detection.adoc\n packfile-uri.adoc\n repository-version.adoc\n rerere.adoc\n sparse-index.adoc\n\nall appear to be formatted correctly.\n\nSo, I really need help with the asciidoc formatting, in patches #2->#4,\nwhich I am marking as RFC. Having said that, these patches represent\nan improvement over the existing documents in terms of formatting\n(just not by much!).\n\nAny help fixing up these patches would be much appreciated. :)\n\nThanks.\n\nATB,\nRamsay Jones\n\nRamsay Jones (4):\n doc: remembering-renames.adoc: fix asciidoc warnings\n doc: sparse-checkout.adoc: fix asciidoc warnings\n doc: commit-graph.adoc: fix up some formatting\n doc: add large-object-promisors.adoc to the docs build\n\n Documentation/Makefile | 1 +\n Documentation/technical/commit-graph.adoc | 29 +-\n .../technical/large-object-promisors.adoc | 64 +-\n Documentation/technical/meson.build | 1 +\n .../technical/remembering-renames.adoc | 120 +--\n Documentation/technical/sparse-checkout.adoc | 704 ++++++++++--------\n 6 files changed, 507 insertions(+), 412 deletions(-)\n\nrange-diff against v2:\n\n1: f1e3b36cad < -: ---------- doc: add some missing technical documents\n2: fd923c16fa = 1: d61b4d2958 doc: remembering-renames.adoc: fix asciidoc warnings\n3: 1e5882a2d5 ! 2: 0cd1524c27 doc: sparse-checkout.adoc: fix asciidoc warnings\n @@ Commit message\n title.\n \n In order to address the first set of warnings, simply renumber the list\n - from one to severn, rather than zero to six. Fortunately, this does not\n + from one to seven, rather than zero to six. Fortunately, this does not\n require altering additional text, since the enumeration of 'Known Bugs'\n is not referred to anywhere else in the document.\n \n4: c8e31e35b7 = 3: f29e225263 doc: commit-graph.adoc: fix up some formatting\n-: ---------- > 4: 3c1effbbb6 doc: add large-object-promisors.adoc to the docs build\n\n\n-- \n2.51.0\n\n"},{"id":"528998","messageId":"20251016200301.1595204-2-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251016200301.1595204-1-ramsay@ramsayjones.plus.com","subject":"[PATCH v3 1/4] doc: remembering-renames.adoc: fix asciidoc warnings","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-16T20:02:58Z","receivedAt":"2025-10-16T20:06:33Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Both asciidoc and ascidoctor issue warnings about 'list item index:\nexpected n got n-1' for n=1->9 on lines 13, 15, 17, 20, 23, 25, 29,\n31 and 33. In asciidoc, numbered lists must start at one, whereas this\nfile has a list starting at zero. Also, asciidoc and asciidoctor warn\nabout 'section title out of sequence: expected level 1, got level 2'\non line 38. (asciidoc only complains about the first instance of this,\nwhile asciidoctor complains about them all, on lines 94, 141, 142,\n184, 185, 257, 288, 289, 290, 397, 424, 485, 486 and 487). These\nwarnings stem from the section titles not being correctly nested within\na document/chapter title.\n\nIn order to address the first set of warnings, simply renumber the list\nfrom one to nine, rather than zero to eight. This also requires altering\nthe text which refers to the section numbers, including other section\ntitles.\n\nIn order to address the second set of warnings, change the section title\nsyntax from '=== title ===' to '== title ==', effectively reducing the\nnesting level of the title by one. Also, some of the titles are given\nover multiple lines (they are very long), with an title '===' prefix\non each line. This leads to them being treated as separate sections\nwith no body text (as you can see from the line numbers given for the\nasciidoctor warnings, above). So, for these titles, turn them into a\nsingle (long) line of text.\n\nIn addition to the warnings, address some other formatting issues:\n\n - the ascii branch diagrams didn't format correctly on asciidoctor\n so include them in a literal block.\n - several blocks of text were intended to be formatted 'as is' but\n were not included in a literal block.\n - in section 8, format the (A)->(D) in the text description as a\n literal with `` marks, since (C) is rendered as a copyright\n symbol in html otherwise.\n - in section 9, a sub-list of two items is not formatted as such.\n change the '*' introducer to '**' to correct the sub-list format.\n\nSigned-off-by: Ramsay Jones \n---\n .../technical/remembering-renames.adoc | 120 ++++++++++++------\n 1 file changed, 78 insertions(+), 42 deletions(-)\n\ndiff --git a/Documentation/technical/remembering-renames.adoc b/Documentation/technical/remembering-renames.adoc\nindex 73f41761e2..6155f36c72 100644\n--- a/Documentation/technical/remembering-renames.adoc\n+++ b/Documentation/technical/remembering-renames.adoc\n@@ -10,32 +10,32 @@ history as an optimization, assuming all merges are automatic and clean\n \n Outline:\n \n- 0. Assumptions\n+ 1. Assumptions\n \n- 1. How rebasing and cherry-picking work\n+ 2. How rebasing and cherry-picking work\n \n- 2. Why the renames on MERGE_SIDE1 in any given pick are *always* a\n+ 3. Why the renames on MERGE_SIDE1 in any given pick are *always* a\n superset of the renames on MERGE_SIDE1 for the next pick.\n \n- 3. Why any rename on MERGE_SIDE1 in any given pick is _almost_ always also\n+ 4. Why any rename on MERGE_SIDE1 in any given pick is _almost_ always also\n a rename on MERGE_SIDE1 for the next pick\n \n- 4. A detailed description of the counter-examples to #3.\n+ 5. A detailed description of the counter-examples to #4.\n \n- 5. Why the special cases in #4 are still fully reasonable to use to pair\n+ 6. Why the special cases in #5 are still fully reasonable to use to pair\n up files for three-way content merging in the merge machinery, and why\n they do not affect the correctness of the merge.\n \n- 6. Interaction with skipping of \"irrelevant\" renames\n+ 7. Interaction with skipping of \"irrelevant\" renames\n \n- 7. Additional items that need to be cached\n+ 8. Additional items that need to be cached\n \n- 8. How directory rename detection interacts with the above and why this\n+ 9. How directory rename detection interacts with the above and why this\n optimization is still safe even if merge.directoryRenames is set to\n \"true\".\n \n \n-=== 0. Assumptions ===\n+== 1. Assumptions ==\n \n There are two assumptions that will hold throughout this document:\n \n@@ -44,8 +44,8 @@ There are two assumptions that will hold throughout this document:\n \n * All merges are fully automatic\n \n-and a third that will hold in sections 2-5 for simplicity, that I'll later\n-address in section 8:\n+and a third that will hold in sections 3-6 for simplicity, that I'll later\n+address in section 9:\n \n * No directory renames occur\n \n@@ -77,9 +77,9 @@ conflicts that the user needs to resolve), the cache of renames is not\n stored on disk, and thus is thrown away as soon as the rebase or cherry\n pick stops for the user to resolve the operation.\n \n-The third assumption makes sections 2-5 simpler, and allows people to\n+The third assumption makes sections 3-6 simpler, and allows people to\n understand the basics of why this optimization is safe and effective, and\n-then I can go back and address the specifics in section 8. It is probably\n+then I can go back and address the specifics in section 9. It is probably\n also worth noting that if directory renames do occur, then the default of\n merge.directoryRenames being set to \"conflict\" means that the operation\n will stop for users to resolve the conflicts and the cache will be thrown\n@@ -88,22 +88,26 @@ reason we need to address directory renames specifically, is that some\n users will have set merge.directoryRenames to \"true\" to allow the merges to\n continue to proceed automatically. The optimization is still safe with\n this config setting, but we have to discuss a few more cases to show why;\n-this discussion is deferred until section 8.\n+this discussion is deferred until section 9.\n \n \n-=== 1. How rebasing and cherry-picking work ===\n+== 2. How rebasing and cherry-picking work ==\n \n Consider the following setup (from the git-rebase manpage):\n \n+------------\n \t\t A---B---C topic\n \t\t /\n \t D---E---F---G main\n+------------\n \n After rebasing or cherry-picking topic onto main, this will appear as:\n \n+------------\n \t\t\t A'--B'--C' topic\n \t\t\t /\n \t D---E---F---G main\n+------------\n \n The way the commits A', B', and C' are created is through a series of\n merges, where rebase or cherry-pick sequentially uses each of the three\n@@ -111,6 +115,7 @@ A-B-C commits in a special merge operation. Let's label the three commits\n in the merge operation as MERGE_BASE, MERGE_SIDE1, and MERGE_SIDE2. For\n this picture, the three commits for each of the three merges would be:\n \n+....\n To create A':\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -125,6 +130,7 @@ To create C':\n MERGE_BASE: B\n MERGE_SIDE1: B'\n MERGE_SIDE2: C\n+....\n \n Sometimes, folks are surprised that these three-way merges are done. It\n can be useful in understanding these three-way merges to view them in a\n@@ -138,8 +144,7 @@ Conceptually the two statements above are the same as a three-way merge of\n B, B', and C, at least the parts before you decide to record a commit.\n \n \n-=== 2. Why the renames on MERGE_SIDE1 in any given pick are always a ===\n-=== superset of the renames on MERGE_SIDE1 for the next pick. ===\n+== 3. Why the renames on MERGE_SIDE1 in any given pick are always a superset of the renames on MERGE_SIDE1 for the next pick. ==\n \n The merge machinery uses the filenames it is fed from MERGE_BASE,\n MERGE_SIDE1, and MERGE_SIDE2. It will only move content to a different\n@@ -156,6 +161,7 @@ filename under one of three conditions:\n First, let's remember what commits are involved in the first and second\n picks of the cherry-pick or rebase sequence:\n \n+....\n To create A':\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -165,6 +171,7 @@ To create B':\n MERGE_BASE: A\n MERGE_SIDE1: A'\n MERGE_SIDE2: B\n+....\n \n So, in particular, we need to show that the renames between E and G are a\n superset of those between A and A'.\n@@ -181,11 +188,11 @@ are a subset of those between E and G. Equivalently, all renames between E\n and G are a superset of those between A and A'.\n \n \n-=== 3. Why any rename on MERGE_SIDE1 in any given pick is _almost_ ===\n-=== always also a rename on MERGE_SIDE1 for the next pick. ===\n+== 4. Why any rename on MERGE_SIDE1 in any given pick is _almost_ always also a rename on MERGE_SIDE1 for the next pick. ==\n \n Let's again look at the first two picks:\n \n+....\n To create A':\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -195,17 +202,25 @@ To create B':\n MERGE_BASE: A\n MERGE_SIDE1: A'\n MERGE_SIDE2: B\n+....\n \n Now let's look at any given rename from MERGE_SIDE1 of the first pick, i.e.\n any given rename from E to G. Let's use the filenames 'oldfile' and\n 'newfile' for demonstration purposes. That first pick will function as\n follows; when the rename is detected, the merge machinery will do a\n three-way content merge of the following:\n+\n+....\n E:oldfile\n G:newfile\n A:oldfile\n+....\n+\n and produce a new result:\n+\n+....\n A':newfile\n+....\n \n Note above that I've assumed that E->A did not rename oldfile. If that\n side did rename, then we most likely have a rename/rename(1to2) conflict\n@@ -254,19 +269,21 @@ were detected as renames, A:oldfile and A':newfile should also be\n detectable as renames almost always.\n \n \n-=== 4. A detailed description of the counter-examples to #3. ===\n+== 5. A detailed description of the counter-examples to #4. ==\n \n-We already noted in section 3 that rename/rename(1to1) (i.e. both sides\n+We already noted in section 4 that rename/rename(1to1) (i.e. both sides\n renaming a file the same way) was one counter-example. The more\n interesting bit, though, is why did we need to use the \"almost\" qualifier\n when stating that A:oldfile and A':newfile are \"almost\" always detectable\n as renames?\n \n-Let's repeat an earlier point that section 3 made:\n+Let's repeat an earlier point that section 4 made:\n \n+....\n A':newfile was created by applying the changes between E:oldfile and\n G:newfile to A:oldfile. The changes between E:oldfile and G:newfile were\n <50% of the size of E:oldfile.\n+....\n \n If those changes that were <50% of the size of E:oldfile are also <50% of\n the size of A:oldfile, then A:oldfile and A':newfile will be detectable as\n@@ -276,18 +293,21 @@ still somehow merge cleanly), then traditional rename detection would not\n detect A:oldfile and A':newfile as renames.\n \n Here's an example where that can happen:\n+\n * E:oldfile had 20 lines\n * G:newfile added 10 new lines at the beginning of the file\n * A:oldfile kept the first 3 lines of the file, and deleted all the rest\n+\n then\n+\n+....\n => A':newfile would have 13 lines, 3 of which matches those in A:oldfile.\n-E:oldfile -> G:newfile would be detected as a rename, but A:oldfile and\n-A':newfile would not be.\n+ E:oldfile -> G:newfile would be detected as a rename, but A:oldfile and\n+ A':newfile would not be.\n+....\n \n \n-=== 5. Why the special cases in #4 are still fully reasonable to use to ===\n-=== pair up files for three-way content merging in the merge machinery, ===\n-=== and why they do not affect the correctness of the merge. ===\n+== 6. Why the special cases in #5 are still fully reasonable to use to pair up files for three-way content merging in the merge machinery, and why they do not affect the correctness of the merge. ==\n \n In the rename/rename(1to1) case, A:newfile and A':newfile are not renames\n since they use the *same* filename. However, files with the same filename\n@@ -295,14 +315,14 @@ are obviously fine to pair up for three-way content merging (the merge\n machinery has never employed break detection). The interesting\n counter-example case is thus not the rename/rename(1to1) case, but the case\n where A did not rename oldfile. That was the case that we spent most of\n-the time discussing in sections 3 and 4. The remainder of this section\n+the time discussing in sections 4 and 5. The remainder of this section\n will be devoted to that case as well.\n \n So, even if A:oldfile and A':newfile aren't detectable as renames, why is\n it still reasonable to pair them up for three-way content merging in the\n merge machinery? There are multiple reasons:\n \n- * As noted in sections 3 and 4, the diff between A:oldfile and A':newfile\n+ * As noted in sections 4 and 5, the diff between A:oldfile and A':newfile\n is *exactly* the same as the diff between E:oldfile and G:newfile. The\n latter pair were detected as renames, so it seems unlikely to surprise\n users for us to treat A:oldfile and A':newfile as renames.\n@@ -394,7 +414,7 @@ cases 1 and 3 seem to provide as good or better behavior with the\n optimization than without.\n \n \n-=== 6. Interaction with skipping of \"irrelevant\" renames ===\n+== 7. Interaction with skipping of \"irrelevant\" renames ==\n \n Previous optimizations involved skipping rename detection for paths\n considered to be \"irrelevant\". See for example the following commits:\n@@ -421,24 +441,27 @@ detection -- though we can limit it to the paths for which we have not\n already detected renames.\n \n \n-=== 7. Additional items that need to be cached ===\n+== 8. Additional items that need to be cached ==\n \n It turns out we have to cache more than just renames; we also cache:\n \n+....\n A) non-renames (i.e. unpaired deletes)\n B) counts of renames within directories\n C) sources that were marked as RELEVANT_LOCATION, but which were\n downgraded to RELEVANT_NO_MORE\n D) the toplevel trees involved in the merge\n+....\n \n These are all stored in struct rename_info, and respectively appear in\n+\n * cached_pairs (along side actual renames, just with a value of NULL)\n * dir_rename_counts\n * cached_irrelevant\n * merge_trees\n \n-The reason for (A) comes from the irrelevant renames skipping\n-optimization discussed in section 6. The fact that irrelevant renames\n+The reason for `(A)` comes from the irrelevant renames skipping\n+optimization discussed in section 7. The fact that irrelevant renames\n are skipped means we only get a subset of the potential renames\n detected and subsequent commits may need to run rename detection on\n the upstream side on a subset of the remaining renames (to get the\n@@ -447,23 +470,24 @@ deletes are involved in rename detection too, we don't want to\n repeatedly check that those paths remain unpaired on the upstream side\n with every commit we are transplanting.\n \n-The reason for (B) is that diffcore_rename_extended() is what\n+The reason for `(B)` is that diffcore_rename_extended() is what\n generates the counts of renames by directory which is needed in\n directory rename detection, and if we don't run\n diffcore_rename_extended() again then we need to have the output from\n it, including dir_rename_counts, from the previous run.\n \n-The reason for (C) is that merge-ort's tree traversal will again think\n+The reason for `(C)` is that merge-ort's tree traversal will again think\n those paths are relevant (marking them as RELEVANT_LOCATION), but the\n fact that they were downgraded to RELEVANT_NO_MORE means that\n dir_rename_counts already has the information we need for directory\n rename detection. (A path which becomes RELEVANT_CONTENT in a\n subsequent commit will be removed from cached_irrelevant.)\n \n-The reason for (D) is that is how we determine whether the remember\n+The reason for `(D)` is that is how we determine whether the remember\n renames optimization can be used. In particular, remembering that our\n sequence of merges looks like:\n \n+....\n Merge 1:\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -475,6 +499,7 @@ sequence of merges looks like:\n MERGE_SIDE1: A'\n MERGE_SIDE2: B\n => Creates B'\n+....\n \n It is the fact that the trees A and A' appear both in Merge 1 and in\n Merge 2, with A as a parent of A' that allows this optimization. So\n@@ -482,12 +507,11 @@ we store the trees to compare with what we are asked to merge next\n time.\n \n \n-=== 8. How directory rename detection interacts with the above and ===\n-=== why this optimization is still safe even if ===\n-=== merge.directoryRenames is set to \"true\". ===\n+== 9. How directory rename detection interacts with the above and why this optimization is still safe even if merge.directoryRenames is set to \"true\". ==\n \n As noted in the assumptions section:\n \n+....\n \"\"\"\n ...if directory renames do occur, then the default of\n merge.directoryRenames being set to \"conflict\" means that the operation\n@@ -497,11 +521,13 @@ As noted in the assumptions section:\n is that some users will have set merge.directoryRenames to \"true\" to\n allow the merges to continue to proceed automatically.\n \"\"\"\n+....\n \n Let's remember that we need to look at how any given pick affects the next\n one. So let's again use the first two picks from the diagram in section\n one:\n \n+....\n First pick does this three-way merge:\n MERGE_BASE: E\n MERGE_SIDE1: G\n@@ -513,6 +539,7 @@ one:\n MERGE_SIDE1: A'\n MERGE_SIDE2: B\n => creates B'\n+....\n \n Now, directory rename detection exists so that if one side of history\n renames a directory, and the other side adds a new file to the old\n@@ -545,7 +572,7 @@ while considering all of these cases:\n concerned; see the assumptions section). Two interesting sub-notes\n about these counts:\n \n- * If we need to perform rename-detection again on the given side (e.g.\n+ ** If we need to perform rename-detection again on the given side (e.g.\n some paths are relevant for rename detection that weren't before),\n then we clear dir_rename_counts and recompute it, making use of\n cached_pairs. The reason it is important to do this is optimizations\n@@ -556,7 +583,7 @@ while considering all of these cases:\n easiest way to \"fix up\" dir_rename_counts in such cases is to just\n recompute it.\n \n- * If we prune rename/rename(1to1) entries from the cache, then we also\n+ ** If we prune rename/rename(1to1) entries from the cache, then we also\n need to update dir_rename_counts to decrement the counts for the\n involved directory and any relevant parent directories (to undo what\n update_dir_rename_counts() in diffcore-rename.c incremented when the\n@@ -578,6 +605,7 @@ in order:\n \n Case 1: MERGE_SIDE1 renames old dir, MERGE_SIDE2 adds new file to old dir\n \n+....\n This case looks like this:\n \n MERGE_BASE: E, Has olddir/\n@@ -595,10 +623,13 @@ Case 1: MERGE_SIDE1 renames old dir, MERGE_SIDE2 adds new file to old dir\n * MERGE_SIDE1 has cached olddir/newfile -> newdir/newfile\n Given the cached rename noted above, the second merge can proceed as\n expected without needing to perform rename detection from A -> A'.\n+....\n \n Case 2: MERGE_SIDE1 renames old dir, MERGE_SIDE2 renames file into old dir\n \n+....\n This case looks like this:\n+\n MERGE_BASE: E oldfile, olddir/\n MERGE_SIDE1: G oldfile, olddir/ -> newdir/\n MERGE_SIDE2: A oldfile -> olddir/newfile\n@@ -617,9 +648,11 @@ Case 2: MERGE_SIDE1 renames old dir, MERGE_SIDE2 renames file into old dir\n \n Given the cached rename noted above, the second merge can proceed as\n expected without needing to perform rename detection from A -> A'.\n+....\n \n Case 3: MERGE_SIDE1 adds new file to old dir, MERGE_SIDE2 renames old dir\n \n+....\n This case looks like this:\n \n MERGE_BASE: E, Has olddir/\n@@ -635,9 +668,11 @@ Case 3: MERGE_SIDE1 adds new file to old dir, MERGE_SIDE2 renames old dir\n In this case, with the optimization, note that after the first commit there\n were no renames on MERGE_SIDE1, and any renames on MERGE_SIDE2 are tossed.\n But the second merge didn't need any renames so this is fine.\n+....\n \n Case 4: MERGE_SIDE1 renames file into old dir, MERGE_SIDE2 renames old dir\n \n+....\n This case looks like this:\n \n MERGE_BASE: E, Has olddir/\n@@ -658,6 +693,7 @@ Case 4: MERGE_SIDE1 renames file into old dir, MERGE_SIDE2 renames old dir\n \n Given the cached rename noted above, the second merge can proceed as\n expected without needing to perform rename detection from A -> A'.\n+....\n \n Finally, I'll just note here that interactions with the\n skip-irrelevant-renames optimization means we sometimes don't detect\n-- \n2.51.0\n\n"},{"id":"528999","messageId":"20251016200301.1595204-3-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251016200301.1595204-1-ramsay@ramsayjones.plus.com","subject":"[PATCH v3 2/4] doc: sparse-checkout.adoc: fix asciidoc warnings","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-16T20:02:59Z","receivedAt":"2025-10-16T20:06:33Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Both asciidoc and asciidoctor issue warnings about 'list item index:\nexpected n got n-1' for n=1->7 on lines 928, 931, 951, 974, 980, 1033\nand 1049. In asciidoc, numbered lists must start at one, whereas this\nfile has a list starting at zero. Also, asciidoc and asciidoctor warn\nabout 'section title out of sequence: expected level 1, got level 2'\non line 17. (asciidoc only complains about the first instance of this,\nwhile asciidoctor complains about them all, on lines 95, 258, 303, 316,\n545, 612, 752, 824, 895, 923 and 1053). These warnings stem from the\nsection titles not being correctly nested within a document/chapter\ntitle.\n\nIn order to address the first set of warnings, simply renumber the list\nfrom one to seven, rather than zero to six. Fortunately, this does not\nrequire altering additional text, since the enumeration of 'Known Bugs'\nis not referred to anywhere else in the document.\n\nIn order to address the second set of warnings, change the section title\nsyntax from '=== title ===' to '== title ==', effectively reducing the\nnesting level of the title by one. Also, some apparent (sub-)titles are\nnot marked up with sub-title syntax, so add some '=== ' prefix(s) to the\nrelevant headings.\n\nIn addition to the warnings, address some other formatting issues:\n\n - the use of heavily nested unordered lists is not reflected in the\n output (making the file totally unreadable) because each level of\n nesting requires a different syntax. (i.e. replace '*' with '**'\n for the second level, '*' with '***' for the third level, etc.)\n - make use of literal blocks and manual indentation to get asciidoc\n and asciidoctor to display even remotely similar output.\n - make use of labelled lists, in some places, to get a similar looking\n output to the input, for both asciidoc and asciidoctor.\n - replace the trailing space in: `git grep ${SEARCH_TERM} OLDREV `\n otherwise the entire line in which that appears is removed from\n the output.\n\nSigned-off-by: Ramsay Jones \n---\n Documentation/technical/sparse-checkout.adoc | 704 ++++++++++---------\n 1 file changed, 376 insertions(+), 328 deletions(-)\n\ndiff --git a/Documentation/technical/sparse-checkout.adoc b/Documentation/technical/sparse-checkout.adoc\nindex 0f750ef3e3..3fa8e53655 100644\n--- a/Documentation/technical/sparse-checkout.adoc\n+++ b/Documentation/technical/sparse-checkout.adoc\n@@ -14,37 +14,41 @@ Table of contents:\n * Reference Emails\n \n \n-=== Terminology ===\n+== Terminology ==\n \n-cone mode: one of two modes for specifying the desired subset of files\n+*`cone mode`*::\n+\tone of two modes for specifying the desired subset of files\n \tin a sparse-checkout. In cone-mode, the user specifies\n \tdirectories (getting both everything under that directory as\n \twell as everything in leading directories), while in non-cone\n \tmode, the user specifies gitignore-style patterns. Controlled\n \tby the --[no-]cone option to sparse-checkout init|set.\n \n-SKIP_WORKTREE: When tracked files do not match the sparse specification and\n+*`SKIP_WORKTREE`*::\n+\tWhen tracked files do not match the sparse specification and\n \tare removed from the working tree, the file in the index is marked\n \twith a SKIP_WORKTREE bit. Note that if a tracked file has the\n \tSKIP_WORKTREE bit set but the file is later written by the user to\n \tthe working tree anyway, the SKIP_WORKTREE bit will be cleared at\n \tthe beginning of any subsequent Git operation.\n-\n-\tMost sparse checkout users are unaware of this implementation\n-\tdetail, and the term should generally be avoided in user-facing\n-\tdescriptions and command flags. Unfortunately, prior to the\n-\t`sparse-checkout` subcommand this low-level detail was exposed,\n-\tand as of time of writing, is still exposed in various places.\n-\n-sparse-checkout: a subcommand in git used to reduce the files present in\n++\n+Most sparse checkout users are unaware of this implementation\n+detail, and the term should generally be avoided in user-facing\n+descriptions and command flags. Unfortunately, prior to the\n+`sparse-checkout` subcommand this low-level detail was exposed,\n+and as of time of writing, is still exposed in various places.\n+\n+*`sparse-checkout`*::\n+\ta subcommand in git used to reduce the files present in\n \tthe working tree to a subset of all tracked files. Also, the\n \tname of the file in the $GIT_DIR/info directory used to track\n \tthe sparsity patterns corresponding to the user's desired\n \tsubset.\n \n-sparse cone: see cone mode\n+*`sparse cone`*:: see cone mode\n \n-sparse directory: An entry in the index corresponding to a directory, which\n+*`sparse directory`*::\n+\tAn entry in the index corresponding to a directory, which\n \tappears in the index instead of all the files under that directory\n \tthat would normally appear. See also sparse-index. Something that\n \tcan cause confusion is that the \"sparse directory\" does NOT match\n@@ -52,7 +56,8 @@ sparse directory: An entry in the index corresponding to a directory, which\n \tworking tree. May be renamed in the future (e.g. to \"skipped\n \tdirectory\").\n \n-sparse index: A special mode for sparse-checkout that also makes the\n+*`sparse index`*::\n+\tA special mode for sparse-checkout that also makes the\n \tindex sparse by recording a directory entry in lieu of all the\n \tfiles underneath that directory (thus making that a \"skipped\n \tdirectory\" which unfortunately has also been called a \"sparse\n@@ -60,7 +65,8 @@ sparse index: A special mode for sparse-checkout that also makes the\n \tdirectories. Controlled by the --[no-]sparse-index option to\n \tinit|set|reapply.\n \n-sparsity patterns: patterns from $GIT_DIR/info/sparse-checkout used to\n+*`sparsity patterns`*::\n+\tpatterns from $GIT_DIR/info/sparse-checkout used to\n \tdefine the set of files of interest. A warning: It is easy to\n \tover-use this term (or the shortened \"patterns\" term), for two\n \treasons: (1) users in cone mode specify directories rather than\n@@ -70,7 +76,8 @@ sparsity patterns: patterns from $GIT_DIR/info/sparse-checkout used to\n \ttransiently differ in the working tree or index from the sparsity\n \tpatterns (see \"Sparse specification vs. sparsity patterns\").\n \n-sparse specification: The set of paths in the user's area of focus. This\n+*`sparse specification`*::\n+\tThe set of paths in the user's area of focus. This\n \tis typically just the tracked files that match the sparsity\n \tpatterns, but the sparse specification can temporarily differ and\n \tinclude additional files. (See also \"Sparse specification\n@@ -87,12 +94,13 @@ sparse specification: The set of paths in the user's area of focus. This\n \t* If working with the index and the working copy, the sparse\n \t specification is the union of the paths from above.\n \n-vivifying: When a command restores a tracked file to the working tree (and\n+*`vivifying`*::\n+\tWhen a command restores a tracked file to the working tree (and\n \thopefully also clears the SKIP_WORKTREE bit in the index for that\n \tfile), this is referred to as \"vivifying\" the file.\n \n \n-=== Purpose of sparse-checkouts ===\n+== Purpose of sparse-checkouts ==\n \n sparse-checkouts exist to allow users to work with a subset of their\n files.\n@@ -120,14 +128,12 @@ those usecases, sparse-checkouts can modify different subcommands in over a\n half dozen different ways. Let's start by considering the high level\n usecases:\n \n- A) Users are _only_ interested in the sparse portion of the repo\n-\n- A*) Users are _only_ interested in the sparse portion of the repo\n- that they have downloaded so far\n-\n- B) Users want a sparse working tree, but are working in a larger whole\n-\n- C) sparse-checkout is a behind-the-scenes implementation detail allowing\n+[horizontal]\n+A):: Users are _only_ interested in the sparse portion of the repo\n+A*):: Users are _only_ interested in the sparse portion of the repo\n+ that they have downloaded so far\n+B):: Users want a sparse working tree, but are working in a larger whole\n+C):: sparse-checkout is a behind-the-scenes implementation detail allowing\n Git to work with a specially crafted in-house virtual file system;\n users are actually working with a \"full\" working tree that is\n lazily populated, and sparse-checkout helps with the lazy population\n@@ -136,7 +142,7 @@ usecases:\n It may be worth explaining each of these in a bit more detail:\n \n \n- (Behavior A) Users are _only_ interested in the sparse portion of the repo\n+=== (Behavior A) Users are _only_ interested in the sparse portion of the repo\n \n These folks might know there are other things in the repository, but\n don't care. They are uninterested in other parts of the repository, and\n@@ -163,8 +169,7 @@ side-effects of various other commands (such as the printed diffstat\n after a merge or pull) can lead to worries about local repository size\n growing unnecessarily[10].\n \n- (Behavior A*) Users are _only_ interested in the sparse portion of the repo\n- that they have downloaded so far (a variant on the first usecase)\n+=== (Behavior A*) Users are _only_ interested in the sparse portion of the repo that they have downloaded so far (a variant on the first usecase)\n \n This variant is driven by folks who using partial clones together with\n sparse checkouts and do disconnected development (so far sounding like a\n@@ -173,15 +178,14 @@ reason for yet another variant is that downloading even just the blobs\n through history within their sparse specification may be too much, so they\n only download some. They would still like operations to succeed without\n network connectivity, though, so things like `git log -S${SEARCH_TERM} -p`\n-or `git grep ${SEARCH_TERM} OLDREV ` would need to be prepared to provide\n+or `git grep ${SEARCH_TERM} OLDREV` would need to be prepared to provide\n partial results that depend on what happens to have been downloaded.\n \n This variant could be viewed as Behavior A with the sparse specification\n for history querying operations modified from \"sparsity patterns\" to\n \"sparsity patterns limited to the blobs we have already downloaded\".\n \n- (Behavior B) Users want a sparse working tree, but are working in a\n- larger whole\n+=== (Behavior B) Users want a sparse working tree, but are working in a larger whole\n \n Stolee described this usecase this way[11]:\n \n@@ -229,8 +233,7 @@ those expensive checks when interacting with the working copy, and may\n prefer getting \"unrelated\" results from their history queries over having\n slow commands.\n \n- (Behavior C) sparse-checkout is an implementational detail supporting a\n-\t special VFS.\n+=== (Behavior C) sparse-checkout is an implementational detail supporting a special VFS.\n \n This usecase goes slightly against the traditional definition of\n sparse-checkout in that it actually tries to present a full or dense\n@@ -255,13 +258,13 @@ will perceive the checkout as dense, and commands should thus behave as if\n all files are present.\n \n \n-=== Usecases of primary concern ===\n+== Usecases of primary concern ==\n \n Most of the rest of this document will focus on Behavior A and Behavior\n B. Some notes about the other two cases and why we are not focusing on\n them:\n \n- (Behavior A*)\n+=== (Behavior A*)\n \n Supporting this usecase is estimated to be difficult and a lot of work.\n There are no plans to implement it currently, but it may be a potential\n@@ -275,7 +278,7 @@ valid for this usecase, with the only exception being that it redefines the\n sparse specification to restrict it to already-downloaded blobs. The hard\n part is in making commands capable of respecting that modified definition.\n \n- (Behavior C)\n+=== (Behavior C)\n \n This usecase violates some of the early sparse-checkout documented\n assumptions (since files marked as SKIP_WORKTREE will be displayed to users\n@@ -300,20 +303,20 @@ Behavior C do not assume they are part of the Behavior B camp and propose\n patches that break things for the real Behavior B folks.\n \n \n-=== Oversimplified mental models ===\n+== Oversimplified mental models ==\n \n An oversimplification of the differences in the above behaviors is:\n \n- Behavior A: Restrict worktree and history operations to sparse specification\n- Behavior B: Restrict worktree operations to sparse specification; have any\n-\t history operations work across all files\n- Behavior C: Do not restrict either worktree or history operations to the\n-\t sparse specification...with the exception of branch checkouts or\n-\t switches which avoid writing files that will match the index so\n-\t they can later lazily be populated instead.\n+(Behavior A):: Restrict worktree and history operations to sparse specification\n+(Behavior B):: Restrict worktree operations to sparse specification; have any\n+\t history operations work across all files\n+(Behavior C):: Do not restrict either worktree or history operations to the\n+\t sparse specification...with the exception of branch checkouts or\n+\t switches which avoid writing files that will match the index so\n+\t they can later lazily be populated instead.\n \n \n-=== Desired behavior ===\n+== Desired behavior ==\n \n As noted previously, despite the simple idea of just working with a subset\n of files, there are a range of different behavioral changes that need to be\n@@ -326,37 +329,38 @@ understanding these differences can be beneficial.\n \n * Commands behaving the same regardless of high-level use-case\n \n- * commands that only look at files within the sparsity specification\n+ ** commands that only look at files within the sparsity specification\n \n- * diff (without --cached or REVISION arguments)\n- * grep (without --cached or REVISION arguments)\n- * diff-files\n+ *** diff (without --cached or REVISION arguments)\n+ *** grep (without --cached or REVISION arguments)\n+ *** diff-files\n \n- * commands that restore files to the working tree that match sparsity\n+ ** commands that restore files to the working tree that match sparsity\n patterns, and remove unmodified files that don't match those\n patterns:\n \n- * switch\n- * checkout (the switch-like half)\n- * read-tree\n- * reset --hard\n+ *** switch\n+ *** checkout (the switch-like half)\n+ *** read-tree\n+ *** reset --hard\n \n- * commands that write conflicted files to the working tree, but otherwise\n+ ** commands that write conflicted files to the working tree, but otherwise\n will omit writing files to the working tree that do not match the\n sparsity patterns:\n \n- * merge\n- * rebase\n- * cherry-pick\n- * revert\n+ *** merge\n+ *** rebase\n+ *** cherry-pick\n+ *** revert\n \n- * `am` and `apply --cached` should probably be in this section but\n+ *** `am` and `apply --cached` should probably be in this section but\n \tare buggy (see the \"Known bugs\" section below)\n \n The behavior for these commands somewhat depends upon the merge\n strategy being used:\n- * `ort` behaves as described above\n- * `octopus` and `resolve` will always vivify any file changed in the merge\n+\n+ *** `ort` behaves as described above\n+ *** `octopus` and `resolve` will always vivify any file changed in the merge\n \trelative to the first parent, which is rather suboptimal.\n \n It is also important to note that these commands WILL update the index\n@@ -372,21 +376,21 @@ understanding these differences can be beneficial.\n specification and the sparsity patterns (much like the commands in the\n previous section).\n \n- * commands that always ignore sparsity since commits must be full-tree\n+ ** commands that always ignore sparsity since commits must be full-tree\n \n- * archive\n- * bundle\n- * commit\n- * format-patch\n- * fast-export\n- * fast-import\n- * commit-tree\n+ *** archive\n+ *** bundle\n+ *** commit\n+ *** format-patch\n+ *** fast-export\n+ *** fast-import\n+ *** commit-tree\n \n- * commands that write any modified file to the working tree (conflicted\n+ ** commands that write any modified file to the working tree (conflicted\n or not, and whether those paths match sparsity patterns or not):\n \n- * stash\n- * apply (without `--index` or `--cached`)\n+ *** stash\n+ *** apply (without `--index` or `--cached`)\n \n * Commands that may slightly differ for behavior A vs. behavior B:\n \n@@ -394,19 +398,20 @@ understanding these differences can be beneficial.\n behaviors, but may differ in verbosity and types of warning and error\n messages.\n \n- * commands that make modifications to which files are tracked:\n- * add\n- * rm\n- * mv\n- * update-index\n+ ** commands that make modifications to which files are tracked:\n+\n+ *** add\n+ *** rm\n+ *** mv\n+ *** update-index\n \n The fact that files can move between the 'tracked' and 'untracked'\n categories means some commands will have to treat untracked files\n differently. But if we have to treat untracked files differently,\n then additional commands may also need changes:\n \n- * status\n- * clean\n+ *** status\n+ *** clean\n \n In particular, `status` may need to report any untracked files outside\n the sparsity specification as an erroneous condition (especially to\n@@ -420,9 +425,10 @@ understanding these differences can be beneficial.\n may need to ignore the sparse specification by its nature. Also, its\n current --[no-]ignore-skip-worktree-entries default is totally bogus.\n \n- * commands for manually tweaking paths in both the index and the working tree\n- * `restore`\n- * the restore-like half of `checkout`\n+ ** commands for manually tweaking paths in both the index and the working tree\n+\n+ *** `restore`\n+ *** the restore-like half of `checkout`\n \n These commands should be similar to add/rm/mv in that they should\n only operate on the sparse specification by default, and require a\n@@ -433,18 +439,19 @@ understanding these differences can be beneficial.\n \n * Commands that significantly differ for behavior A vs. behavior B:\n \n- * commands that query history\n- * diff (with --cached or REVISION arguments)\n- * grep (with --cached or REVISION arguments)\n- * show (when given commit arguments)\n- * blame (only matters when one or more -C flags are passed)\n-\t* and annotate\n- * log\n- * whatchanged (may not exist anymore)\n- * ls-files\n- * diff-index\n- * diff-tree\n- * ls-tree\n+ ** commands that query history\n+\n+ *** diff (with --cached or REVISION arguments)\n+ *** grep (with --cached or REVISION arguments)\n+ *** show (when given commit arguments)\n+ *** blame (only matters when one or more -C flags are passed)\n+\t**** and annotate\n+ *** log\n+ *** whatchanged (may not exist anymore)\n+ *** ls-files\n+ *** diff-index\n+ *** diff-tree\n+ *** ls-tree\n \n Note: for log and whatchanged, revision walking logic is unaffected\n but displaying of patches is affected by scoping the command to the\n@@ -458,91 +465,91 @@ understanding these differences can be beneficial.\n \n * Commands I don't know how to classify\n \n- * range-diff\n+ ** range-diff\n \n Is this like `log` or `format-patch`?\n \n- * cherry\n+ ** cherry\n \n See range-diff\n \n * Commands unaffected by sparse-checkouts\n \n- * shortlog\n- * show-branch\n- * rev-list\n- * bisect\n-\n- * branch\n- * describe\n- * fetch\n- * gc\n- * init\n- * maintenance\n- * notes\n- * pull (merge & rebase have the necessary changes)\n- * push\n- * submodule\n- * tag\n-\n- * config\n- * filter-branch (works in separate checkout without sparse-checkout setup)\n- * pack-refs\n- * prune\n- * remote\n- * repack\n- * replace\n-\n- * bugreport\n- * count-objects\n- * fsck\n- * gitweb\n- * help\n- * instaweb\n- * merge-tree (doesn't touch worktree or index, and merges always compute full-tree)\n- * rerere\n- * verify-commit\n- * verify-tag\n-\n- * commit-graph\n- * hash-object\n- * index-pack\n- * mktag\n- * mktree\n- * multi-pack-index\n- * pack-objects\n- * prune-packed\n- * symbolic-ref\n- * unpack-objects\n- * update-ref\n- * write-tree (operates on index, possibly optimized to use sparse dir entries)\n-\n- * for-each-ref\n- * get-tar-commit-id\n- * ls-remote\n- * merge-base (merges are computed full tree, so merge base should be too)\n- * name-rev\n- * pack-redundant\n- * rev-parse\n- * show-index\n- * show-ref\n- * unpack-file\n- * var\n- * verify-pack\n-\n- * \n- * \n- * \n- * \n+ ** shortlog\n+ ** show-branch\n+ ** rev-list\n+ ** bisect\n+\n+ ** branch\n+ ** describe\n+ ** fetch\n+ ** gc\n+ ** init\n+ ** maintenance\n+ ** notes\n+ ** pull (merge & rebase have the necessary changes)\n+ ** push\n+ ** submodule\n+ ** tag\n+\n+ ** config\n+ ** filter-branch (works in separate checkout without sparse-checkout setup)\n+ ** pack-refs\n+ ** prune\n+ ** remote\n+ ** repack\n+ ** replace\n+\n+ ** bugreport\n+ ** count-objects\n+ ** fsck\n+ ** gitweb\n+ ** help\n+ ** instaweb\n+ ** merge-tree (doesn't touch worktree or index, and merges always compute full-tree)\n+ ** rerere\n+ ** verify-commit\n+ ** verify-tag\n+\n+ ** commit-graph\n+ ** hash-object\n+ ** index-pack\n+ ** mktag\n+ ** mktree\n+ ** multi-pack-index\n+ ** pack-objects\n+ ** prune-packed\n+ ** symbolic-ref\n+ ** unpack-objects\n+ ** update-ref\n+ ** write-tree (operates on index, possibly optimized to use sparse dir entries)\n+\n+ ** for-each-ref\n+ ** get-tar-commit-id\n+ ** ls-remote\n+ ** merge-base (merges are computed full tree, so merge base should be too)\n+ ** name-rev\n+ ** pack-redundant\n+ ** rev-parse\n+ ** show-index\n+ ** show-ref\n+ ** unpack-file\n+ ** var\n+ ** verify-pack\n+\n+ ** \n+ ** \n+ ** \n+ ** \n \n * Commands that might be affected, but who cares?\n \n- * merge-file\n- * merge-index\n- * gitk?\n+ ** merge-file\n+ ** merge-index\n+ ** gitk?\n \n \n-=== Behavior classes ===\n+== Behavior classes ==\n \n From the above there are a few classes of behavior:\n \n@@ -573,18 +580,19 @@ From the above there are a few classes of behavior:\n \n Commands in this class generally behave like the \"restrict\" class,\n except that:\n- (1) they will ignore the sparse specification and write files with\n-\t conflicts to the working tree (thus temporarily expanding the\n-\t sparse specification to include such files.)\n- (2) they are grouped with commands which move to a new commit, since\n-\t they often create a commit and then move to it, even though we\n-\t know there are many exceptions to moving to the new commit. (For\n-\t example, the user may rebase a commit that becomes empty, or have\n-\t a cherry-pick which conflicts, or a user could run `merge\n-\t --no-commit`, and we also view `apply --index` kind of like `am\n-\t --no-commit`.) As such, these commands can make changes to index\n-\t files outside the sparse specification, though they'll mark such\n-\t files with SKIP_WORKTREE.\n+\n+\t(1) they will ignore the sparse specification and write files with\n+\t conflicts to the working tree (thus temporarily expanding the\n+\t sparse specification to include such files.)\n+\t(2) they are grouped with commands which move to a new commit, since\n+\t they often create a commit and then move to it, even though we\n+\t know there are many exceptions to moving to the new commit. (For\n+\t example, the user may rebase a commit that becomes empty, or have\n+\t a cherry-pick which conflicts, or a user could run `merge\n+\t --no-commit`, and we also view `apply --index` kind of like `am\n+\t --no-commit`.) As such, these commands can make changes to index\n+\t files outside the sparse specification, though they'll mark such\n+\t files with SKIP_WORKTREE.\n \n * \"restrict also specially applied to untracked files\"\n \n@@ -609,37 +617,39 @@ From the above there are a few classes of behavior:\n specification.\n \n \n-=== Subcommand-dependent defaults ===\n+== Subcommand-dependent defaults ==\n \n Note that we have different defaults depending on the command for the\n desired behavior :\n \n * Commands defaulting to \"restrict\":\n- * diff-files\n- * diff (without --cached or REVISION arguments)\n- * grep (without --cached or REVISION arguments)\n- * switch\n- * checkout (the switch-like half)\n- * reset ()\n-\n- * restore\n- * checkout (the restore-like half)\n- * checkout-index\n- * reset (with pathspec)\n+\n+ ** diff-files\n+ ** diff (without --cached or REVISION arguments)\n+ ** grep (without --cached or REVISION arguments)\n+ ** switch\n+ ** checkout (the switch-like half)\n+ ** reset ()\n+\n+ ** restore\n+ ** checkout (the restore-like half)\n+ ** checkout-index\n+ ** reset (with pathspec)\n \n This behavior makes sense; these interact with the working tree.\n \n * Commands defaulting to \"restrict modulo conflicts\":\n- * merge\n- * rebase\n- * cherry-pick\n- * revert\n \n- * am\n- * apply --index (which is kind of like an `am --no-commit`)\n+ ** merge\n+ ** rebase\n+ ** cherry-pick\n+ ** revert\n+\n+ ** am\n+ ** apply --index (which is kind of like an `am --no-commit`)\n \n- * read-tree (especially with -m or -u; is kind of like a --no-commit merge)\n- * reset (, due to similarity to read-tree)\n+ ** read-tree (especially with -m or -u; is kind of like a --no-commit merge)\n+ ** reset (, due to similarity to read-tree)\n \n These also interact with the working tree, but require slightly\n different behavior either so that (a) conflicts can be resolved or (b)\n@@ -648,16 +658,17 @@ desired behavior :\n (See also the \"Known bugs\" section below regarding `am` and `apply`)\n \n * Commands defaulting to \"no restrict\":\n- * archive\n- * bundle\n- * commit\n- * format-patch\n- * fast-export\n- * fast-import\n- * commit-tree\n \n- * stash\n- * apply (without `--index`)\n+ ** archive\n+ ** bundle\n+ ** commit\n+ ** format-patch\n+ ** fast-export\n+ ** fast-import\n+ ** commit-tree\n+\n+ ** stash\n+ ** apply (without `--index`)\n \n These have completely different defaults and perhaps deserve the most\n detailed explanation:\n@@ -679,53 +690,59 @@ desired behavior :\n sparse specification then we'll lose changes from the user.\n \n * Commands defaulting to \"restrict also specially applied to untracked files\":\n- * add\n- * rm\n- * mv\n- * update-index\n- * status\n- * clean (?)\n-\n- Our original implementation for the first three of these commands was\n- \"no restrict\", but it had some severe usability issues:\n- * `git add ` if honored and outside the sparse\n-\tspecification, can result in the file randomly disappearing later\n-\twhen some subsequent command is run (since various commands\n-\tautomatically clean up unmodified files outside the sparse\n-\tspecification).\n- * `git rm '*.jpg'` could very negatively surprise users if it deletes\n-\tfiles outside the range of the user's interest.\n- * `git mv` has similar surprises when moving into or out of the cone,\n-\tso best to restrict by default\n-\n- So, we switched `add` and `rm` to default to \"restrict\", which made\n- usability problems much less severe and less frequent, but we still got\n- complaints because commands like:\n-\tgit add \n-\tgit rm \n- would silently do nothing. We should instead print an error in those\n- cases to get usability right.\n-\n- update-index needs to be updated to match, and status and maybe clean\n- also need to be updated to specially handle untracked paths.\n-\n- There may be a difference in here between behavior A and behavior B in\n- terms of verboseness of errors or additional warnings.\n+\n+ ** add\n+ ** rm\n+ ** mv\n+ ** update-index\n+ ** status\n+ ** clean (?)\n+\n+....\n+ Our original implementation for the first three of these commands was\n+ \"no restrict\", but it had some severe usability issues:\n+\n+ * `git add ` if honored and outside the sparse\n+\t specification, can result in the file randomly disappearing later\n+\t when some subsequent command is run (since various commands\n+\t automatically clean up unmodified files outside the sparse\n+\t specification).\n+ * `git rm '*.jpg'` could very negatively surprise users if it deletes\n+\t files outside the range of the user's interest.\n+ * `git mv` has similar surprises when moving into or out of the cone,\n+\t so best to restrict by default\n+\n+ So, we switched `add` and `rm` to default to \"restrict\", which made\n+ usability problems much less severe and less frequent, but we still got\n+ complaints because commands like:\n+\n+\t git add \n+\t git rm \n+\n+ would silently do nothing. We should instead print an error in those\n+ cases to get usability right.\n+\n+ update-index needs to be updated to match, and status and maybe clean\n+ also need to be updated to specially handle untracked paths.\n+\n+ There may be a difference in here between behavior A and behavior B in\n+ terms of verboseness of errors or additional warnings.\n+....\n \n * Commands falling under \"restrict or no restrict dependent upon behavior\n A vs. behavior B\"\n \n- * diff (with --cached or REVISION arguments)\n- * grep (with --cached or REVISION arguments)\n- * show (when given commit arguments)\n- * blame (only matters when one or more -C flags passed)\n- * and annotate\n- * log\n- * and variants: shortlog, gitk, show-branch, whatchanged, rev-list\n- * ls-files\n- * diff-index\n- * diff-tree\n- * ls-tree\n+ ** diff (with --cached or REVISION arguments)\n+ ** grep (with --cached or REVISION arguments)\n+ ** show (when given commit arguments)\n+ ** blame (only matters when one or more -C flags passed)\n+ *** and annotate\n+ ** log\n+ *** and variants: shortlog, gitk, show-branch, whatchanged, rev-list\n+ ** ls-files\n+ ** diff-index\n+ ** diff-tree\n+ ** ls-tree\n \n For now, we default to behavior B for these, which want a default of\n \"no restrict\".\n@@ -749,7 +766,7 @@ desired behavior :\n implemented.\n \n \n-=== Sparse specification vs. sparsity patterns ===\n+== Sparse specification vs. sparsity patterns ==\n \n In a well-behaved situation, the sparse specification is given directly\n by the $GIT_DIR/info/sparse-checkout file. However, it can transiently\n@@ -821,45 +838,48 @@ under behavior B index operations are lumped with history and tend to\n operate full-tree.\n \n \n-=== Implementation Questions ===\n-\n- * Do the options --scope={sparse,all} sound good to others? Are there better\n- options?\n- * Names in use, or appearing in patches, or previously suggested:\n- * --sparse/--dense\n- * --ignore-skip-worktree-bits\n- * --ignore-skip-worktree-entries\n- * --ignore-sparsity\n- * --[no-]restrict-to-sparse-paths\n- * --full-tree/--sparse-tree\n- * --[no-]restrict\n- * --scope={sparse,all}\n- * --focus/--unfocus\n- * --limit/--unlimited\n- * Rationale making me lean slightly towards --scope={sparse,all}:\n- * We want a name that works for many commands, so we need a name that\n+== Implementation Questions ==\n+\n+ * Do the options --scope={sparse,all} sound good to others? Are there better options?\n+\n+ ** Names in use, or appearing in patches, or previously suggested:\n+\n+ *** --sparse/--dense\n+ *** --ignore-skip-worktree-bits\n+ *** --ignore-skip-worktree-entries\n+ *** --ignore-sparsity\n+ *** --[no-]restrict-to-sparse-paths\n+ *** --full-tree/--sparse-tree\n+ *** --[no-]restrict\n+ *** --scope={sparse,all}\n+ *** --focus/--unfocus\n+ *** --limit/--unlimited\n+\n+ ** Rationale making me lean slightly towards --scope={sparse,all}:\n+\n+ *** We want a name that works for many commands, so we need a name that\n \tdoes not conflict\n- * We know that we have more than two possible usecases, so it is best\n+ *** We know that we have more than two possible usecases, so it is best\n \tto avoid a flag that appears to be binary.\n- * --scope={sparse,all} isn't overly long and seems relatively\n+ *** --scope={sparse,all} isn't overly long and seems relatively\n \texplanatory\n- * `--sparse`, as used in add/rm/mv, is totally backwards for\n+ *** `--sparse`, as used in add/rm/mv, is totally backwards for\n \tgrep/log/etc. Changing the meaning of `--sparse` for these\n \tcommands would fix the backwardness, but possibly break existing\n \tscripts. Using a new name pairing would allow us to treat\n \t`--sparse` in these commands as a deprecated alias.\n- * There is a different `--sparse`/`--dense` pair for commands using\n+ *** There is a different `--sparse`/`--dense` pair for commands using\n \trevision machinery, so using that naming might cause confusion\n- * There is also a `--sparse` in both pack-objects and show-branch, which\n+ *** There is also a `--sparse` in both pack-objects and show-branch, which\n \tdon't conflict but do suggest that `--sparse` is overloaded\n- * The name --ignore-skip-worktree-bits is a double negative, is\n+ *** The name --ignore-skip-worktree-bits is a double negative, is\n \tquite a mouthful, refers to an implementation detail that many\n \tusers may not be familiar with, and we'd need a negation for it\n \twhich would probably be even more ridiculously long. (But we\n \tcan make --ignore-skip-worktree-bits a deprecated alias for\n \t--no-restrict.)\n \n- * If a config option is added (sparse.scope?) what should the values and\n+ ** If a config option is added (sparse.scope?) what should the values and\n description be? \"sparse\" (behavior A), \"worktree-sparse-history-dense\"\n (behavior B), \"dense\" (behavior C)? There's a risk of confusion,\n because even for Behaviors A and B we want some commands to be\n@@ -868,19 +888,20 @@ operate full-tree.\n the primary difference we are focusing is just the history-querying\n commands (log/diff/grep). Previous config suggestion here: [13]\n \n- * Is `--no-expand` a good alias for ls-files's `--sparse` option?\n+ ** Is `--no-expand` a good alias for ls-files's `--sparse` option?\n (`--sparse` does not map to either `--scope=sparse` or `--scope=all`,\n because in non-cone mode it does nothing and in cone-mode it shows the\n sparse directory entries which are technically outside the sparse\n specification)\n \n- * Under Behavior A:\n- * Does ls-files' `--no-expand` override the default `--scope=all`, or\n+ ** Under Behavior A:\n+\n+ *** Does ls-files' `--no-expand` override the default `--scope=all`, or\n does it need an extra flag?\n- * Does ls-files' `-t` option imply `--scope=all`?\n- * Does update-index's `--[no-]skip-worktree` option imply `--scope=all`?\n+ *** Does ls-files' `-t` option imply `--scope=all`?\n+ *** Does update-index's `--[no-]skip-worktree` option imply `--scope=all`?\n \n- * sparse-checkout: once behavior A is fully implemented, should we take\n+ ** sparse-checkout: once behavior A is fully implemented, should we take\n an interim measure to ease people into switching the default? Namely,\n if folks are not already in a sparse checkout, then require\n `sparse-checkout init/set` to take a\n@@ -892,7 +913,7 @@ operate full-tree.\n is seamless for them.\n \n \n-=== Implementation Goals/Plans ===\n+== Implementation Goals/Plans ==\n \n * Get buy-in on this document in general.\n \n@@ -910,25 +931,26 @@ operate full-tree.\n request that they not trigger this bug.\" flag\n \n * Flags & Config\n- * Make `--sparse` in add/rm/mv a deprecated alias for `--scope=all`\n- * Make `--ignore-skip-worktree-bits` in checkout-index/checkout/restore\n+\n+ ** Make `--sparse` in add/rm/mv a deprecated alias for `--scope=all`\n+ ** Make `--ignore-skip-worktree-bits` in checkout-index/checkout/restore\n a deprecated aliases for `--scope=all`\n- * Create config option (sparse.scope?), tie it to the \"Cliff notes\"\n+ ** Create config option (sparse.scope?), tie it to the \"Cliff notes\"\n overview\n \n- * Add --scope=sparse (and --scope=all) flag to each of the history querying\n+ ** Add --scope=sparse (and --scope=all) flag to each of the history querying\n commands. IMPORTANT: make sure diff machinery changes don't mess with\n format-patch, fast-export, etc.\n \n-=== Known bugs ===\n+== Known bugs ==\n \n This list used to be a lot longer (see e.g. [1,2,3,4,5,6,7,8,9]), but we've\n been working on it.\n \n-0. Behavior A is not well supported in Git. (Behavior B didn't used to\n+1. Behavior A is not well supported in Git. (Behavior B didn't used to\n be either, but was the easier of the two to implement.)\n \n-1. am and apply:\n+2. am and apply:\n \n apply, without `--index` or `--cached`, relies on files being present\n in the working copy, and also writes to them unconditionally. As\n@@ -948,7 +970,7 @@ been working on it.\n files and then complain that those vivified files would be\n overwritten by merge.\n \n-2. reset --hard:\n+3. reset --hard:\n \n reset --hard provides confusing error message (works correctly, but\n misleads the user into believing it didn't):\n@@ -971,13 +993,13 @@ been working on it.\n `git reset --hard` DID remove addme from the index and the working tree, contrary\n to the error message, but in line with how reset --hard should behave.\n \n-3. read-tree\n+4. read-tree\n \n `read-tree` doesn't apply the 'SKIP_WORKTREE' bit to *any* of the\n entries it reads into the index, resulting in all your files suddenly\n appearing to be \"deleted\".\n \n-4. Checkout, restore:\n+5. Checkout, restore:\n \n These command do not handle path & revision arguments appropriately:\n \n@@ -1030,7 +1052,7 @@ been working on it.\n S tracked\n H tracked-but-maybe-skipped\n \n-5. checkout and restore --staged, continued:\n+6. checkout and restore --staged, continued:\n \n These commands do not correctly scope operations to the sparse\n specification, and make it worse by not setting important SKIP_WORKTREE\n@@ -1046,56 +1068,82 @@ been working on it.\n the sparse specification, but then it will be important to set the\n SKIP_WORKTREE bits appropriately.\n \n-6. Performance issues; see:\n- https://lore.kernel.org/git/CABPp-BEkJQoKZsQGCYioyga_uoDQ6iBeW+FKr8JhyuuTMK1RDw@mail.gmail.com/\n+7. Performance issues; see:\n+\n+ https://lore.kernel.org/git/CABPp-BEkJQoKZsQGCYioyga_uoDQ6iBeW+FKr8JhyuuTMK1RDw@mail.gmail.com/\n \n \n-=== Reference Emails ===\n+== Reference Emails ==\n \n Emails that detail various bugs we've had in sparse-checkout:\n \n-[1] (Original descriptions of behavior A & behavior B)\n- https://lore.kernel.org/git/CABPp-BGJ_Nvi5TmgriD9Bh6eNXE2EDq2f8e8QKXAeYG3BxZafA@mail.gmail.com/\n-[2] (Fix stash applications in sparse checkouts; bugs from behavioral differences)\n- https://lore.kernel.org/git/ccfedc7140dbf63ba26a15f93bd3885180b26517.1606861519.git.gitgitgadget@gmail.com/\n-[3] (Present-despite-skipped entries)\n- https://lore.kernel.org/git/11d46a399d26c913787b704d2b7169cafc28d639.1642175983.git.gitgitgadget@gmail.com/\n-[4] (Clone --no-checkout interaction)\n- https://lore.kernel.org/git/pull.801.v2.git.git.1591324899170.gitgitgadget@gmail.com/ (clone --no-checkout)\n-[5] (The need for update_sparsity() and avoiding `read-tree -mu HEAD`)\n- https://lore.kernel.org/git/3a1f084641eb47515b5a41ed4409a36128913309.1585270142.git.gitgitgadget@gmail.com/\n-[6] (SKIP_WORKTREE is advisory, not mandatory)\n- https://lore.kernel.org/git/844306c3e86ef67591cc086decb2b760e7d710a3.1585270142.git.gitgitgadget@gmail.com/\n-[7] (`worktree add` should copy sparsity settings from current worktree)\n- https://lore.kernel.org/git/c51cb3714e7b1d2f8c9370fe87eca9984ff4859f.1644269584.git.gitgitgadget@gmail.com/\n-[8] (Avoid negative surprises in add, rm, and mv)\n- https://lore.kernel.org/git/cover.1617914011.git.matheus.bernardino@usp.br/\n- https://lore.kernel.org/git/pull.1018.v4.git.1632497954.gitgitgadget@gmail.com/\n-[9] (Move from out-of-cone to in-cone)\n- https://lore.kernel.org/git/20220630023737.473690-6-shaoxuan.yuan02@gmail.com/\n- https://lore.kernel.org/git/20220630023737.473690-4-shaoxuan.yuan02@gmail.com/\n-[10] (Unnecessarily downloading objects outside sparse specification)\n- https://lore.kernel.org/git/CAOLTT8QfwOi9yx_qZZgyGa8iL8kHWutEED7ok_jxwTcYT_hf9Q@mail.gmail.com/\n-\n-[11] (Stolee's comments on high-level usecases)\n- https://lore.kernel.org/git/1a1e33f6-3514-9afc-0a28-5a6b85bd8014@gmail.com/\n+[1] (Original descriptions of behavior A & behavior B):\n+\n+https://lore.kernel.org/git/CABPp-BGJ_Nvi5TmgriD9Bh6eNXE2EDq2f8e8QKXAeYG3BxZafA@mail.gmail.com/\n+\n+[2] (Fix stash applications in sparse checkouts; bugs from behavioral differences):\n+\n+https://lore.kernel.org/git/ccfedc7140dbf63ba26a15f93bd3885180b26517.1606861519.git.gitgitgadget@gmail.com/\n+\n+[3] (Present-despite-skipped entries):\n+\n+https://lore.kernel.org/git/11d46a399d26c913787b704d2b7169cafc28d639.1642175983.git.gitgitgadget@gmail.com/\n+\n+[4] (Clone --no-checkout interaction):\n+\n+https://lore.kernel.org/git/pull.801.v2.git.git.1591324899170.gitgitgadget@gmail.com/ (clone --no-checkout)\n+\n+[5] (The need for update_sparsity() and avoiding `read-tree -mu HEAD`):\n+\n+https://lore.kernel.org/git/3a1f084641eb47515b5a41ed4409a36128913309.1585270142.git.gitgitgadget@gmail.com/\n+\n+[6] (SKIP_WORKTREE is advisory, not mandatory):\n+\n+https://lore.kernel.org/git/844306c3e86ef67591cc086decb2b760e7d710a3.1585270142.git.gitgitgadget@gmail.com/\n+\n+[7] (`worktree add` should copy sparsity settings from current worktree):\n+\n+https://lore.kernel.org/git/c51cb3714e7b1d2f8c9370fe87eca9984ff4859f.1644269584.git.gitgitgadget@gmail.com/\n+\n+[8] (Avoid negative surprises in add, rm, and mv):\n+\n+ * https://lore.kernel.org/git/cover.1617914011.git.matheus.bernardino@usp.br/\n+ * https://lore.kernel.org/git/pull.1018.v4.git.1632497954.gitgitgadget@gmail.com/\n+\n+[9] (Move from out-of-cone to in-cone):\n+\n+ * https://lore.kernel.org/git/20220630023737.473690-6-shaoxuan.yuan02@gmail.com/\n+ * https://lore.kernel.org/git/20220630023737.473690-4-shaoxuan.yuan02@gmail.com/\n+\n+[10] (Unnecessarily downloading objects outside sparse specification):\n+\n+https://lore.kernel.org/git/CAOLTT8QfwOi9yx_qZZgyGa8iL8kHWutEED7ok_jxwTcYT_hf9Q@mail.gmail.com/\n+\n+[11] (Stolee's comments on high-level usecases):\n+\n+https://lore.kernel.org/git/1a1e33f6-3514-9afc-0a28-5a6b85bd8014@gmail.com/\n \n [12] Others commenting on eventually switching default to behavior A:\n+\n * https://lore.kernel.org/git/xmqqh719pcoo.fsf@gitster.g/\n * https://lore.kernel.org/git/xmqqzgeqw0sy.fsf@gitster.g/\n * https://lore.kernel.org/git/a86af661-cf58-a4e5-0214-a67d3a794d7e@github.com/\n \n-[13] Previous config name suggestion and description\n- * https://lore.kernel.org/git/CABPp-BE6zW0nJSStcVU=_DoDBnPgLqOR8pkTXK3dW11=T01OhA@mail.gmail.com/\n+[13] Previous config name suggestion and description:\n+\n+ https://lore.kernel.org/git/CABPp-BE6zW0nJSStcVU=_DoDBnPgLqOR8pkTXK3dW11=T01OhA@mail.gmail.com/\n \n [14] Tangential issue: switch to cone mode as default sparse specification mechanism:\n- https://lore.kernel.org/git/a1b68fd6126eb341ef3637bb93fedad4309b36d0.1650594746.git.gitgitgadget@gmail.com/\n+\n+https://lore.kernel.org/git/a1b68fd6126eb341ef3637bb93fedad4309b36d0.1650594746.git.gitgitgadget@gmail.com/\n \n [15] Lengthy email on grep behavior, covering what should be searched:\n- * https://lore.kernel.org/git/CABPp-BGVO3QdbfE84uF_3QDF0-y2iHHh6G5FAFzNRfeRitkuHw@mail.gmail.com/\n+\n+https://lore.kernel.org/git/CABPp-BGVO3QdbfE84uF_3QDF0-y2iHHh6G5FAFzNRfeRitkuHw@mail.gmail.com/\n \n [16] Email explaining sparsity patterns vs. SKIP_WORKTREE and history operations,\n search for the parenthetical comment starting \"We do not check\".\n- https://lore.kernel.org/git/CABPp-BFsCPPNOZ92JQRJeGyNd0e-TCW-LcLyr0i_+VSQJP+GCg@mail.gmail.com/\n+\n+https://lore.kernel.org/git/CABPp-BFsCPPNOZ92JQRJeGyNd0e-TCW-LcLyr0i_+VSQJP+GCg@mail.gmail.com/\n \n [17] https://lore.kernel.org/git/20220207190320.2960362-1-jonathantanmy@google.com/\n-- \n2.51.0\n\n"},{"id":"529000","messageId":"20251016200301.1595204-4-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251016200301.1595204-1-ramsay@ramsayjones.plus.com","subject":"[PATCH v3 3/4] doc: commit-graph.adoc: fix up some formatting","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-16T20:03:00Z","receivedAt":"2025-10-16T20:06:40Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"The formatting markup syntax used in this document (markdown?) is not\ninterpreted correctly by asciidoc or asciidoctor. The main problem is\nthe use of a '## ' prefix markup for some sub-headings, along with the\nuse of '```' code markup and some missing literal blocks.\n\nIn order to improve the (html) document formatting:\n\n - replace the '## ' prefix sub-title syntax with the '~~' underlining\n syntax for the relevant sub-headings.\n - replace the '```' code markup, which causes asciidoc(tor) to simply\n remove the marked up text, with a literal block '----' markup.\n - the second ascii diagram, in the 'Merging commit-graph files'\n section, is not rendered correctly by asciidoctor (asciidoc is fine)\n so enclose it in a '....' block.\n\nSigned-off-by: Ramsay Jones \n---\n Documentation/technical/commit-graph.adoc | 29 +++++++++++++++--------\n 1 file changed, 19 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/technical/commit-graph.adoc b/Documentation/technical/commit-graph.adoc\nindex 2c26e95e51..a259d1567b 100644\n--- a/Documentation/technical/commit-graph.adoc\n+++ b/Documentation/technical/commit-graph.adoc\n@@ -39,6 +39,7 @@ A consumer may load the following info for a commit from the graph:\n Values 1-4 satisfy the requirements of parse_commit_gently().\n \n There are two definitions of generation number:\n+\n 1. Corrected committer dates (generation number v2)\n 2. Topological levels (generation number v1)\n \n@@ -158,7 +159,8 @@ number of commits in the full history. By creating a \"chain\" of commit-graphs,\n we enable fast writes of new commit data without rewriting the entire commit\n history -- at least, most of the time.\n \n-## File Layout\n+File Layout\n+~~~~~~~~~~~\n \n A commit-graph chain uses multiple files, and we use a fixed naming convention\n to organize these files. Each commit-graph file has a name\n@@ -170,11 +172,11 @@ hashes for the files in order from \"lowest\" to \"highest\".\n \n For example, if the `commit-graph-chain` file contains the lines\n \n-```\n+----\n \t{hash0}\n \t{hash1}\n \t{hash2}\n-```\n+----\n \n then the commit-graph chain looks like the following diagram:\n \n@@ -213,7 +215,8 @@ specifying the hashes of all files in the lower layers. In the above example,\n `graph-{hash1}.graph` contains `{hash0}` while `graph-{hash2}.graph` contains\n `{hash0}` and `{hash1}`.\n \n-## Merging commit-graph files\n+Merging commit-graph files\n+~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n If we only added a new commit-graph file on every write, we would run into a\n linear search problem through many commit-graph files. Instead, we use a merge\n@@ -225,6 +228,7 @@ is determined by the merge strategy that the files should collapse to\n the commits in `graph-{hash1}` should be combined into a new `graph-{hash3}`\n file.\n \n+....\n \t\t\t +---------------------+\n \t\t\t | |\n \t\t\t | (new commits) |\n@@ -250,6 +254,7 @@ file.\n | |\n | |\n +-----------------------+\n+....\n \n During this process, the commits to write are combined, sorted and we write the\n contents to a temporary file, all while holding a `commit-graph-chain.lock`\n@@ -257,14 +262,15 @@ lock-file. When the file is flushed, we rename it to `graph-{hash3}`\n according to the computed `{hash3}`. Finally, we write the new chain data to\n `commit-graph-chain.lock`:\n \n-```\n+----\n \t{hash3}\n \t{hash0}\n-```\n+----\n \n We then close the lock-file.\n \n-## Merge Strategy\n+Merge Strategy\n+~~~~~~~~~~~~~~\n \n When writing a set of commits that do not exist in the commit-graph stack of\n height N, we default to creating a new file at level N + 1. We then decide to\n@@ -289,7 +295,8 @@ The merge strategy values (2 for the size multiple, 64,000 for the maximum\n number of commits) could be extracted into config settings for full\n flexibility.\n \n-## Handling Mixed Generation Number Chains\n+Handling Mixed Generation Number Chains\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n With the introduction of generation number v2 and generation data chunk, the\n following scenario is possible:\n@@ -318,7 +325,8 @@ have corrected commit dates when written by compatible versions of Git. Thus,\n rewriting split commit-graph as a single file (`--split=replace`) creates a\n single layer with corrected commit dates.\n \n-## Deleting graph-{hash} files\n+Deleting graph-\\{hash\\} files\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n After a new tip file is written, some `graph-{hash}` files may no longer\n be part of a chain. It is important to remove these files from disk, eventually.\n@@ -333,7 +341,8 @@ files whose modified times are older than a given expiry window. This window\n defaults to zero, but can be changed using command-line arguments or a config\n setting.\n \n-## Chains across multiple object directories\n+Chains across multiple object directories\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n In a repo with alternates, we look for the `commit-graph-chain` file starting\n in the local object directory and then in each alternate. The first file that\n-- \n2.51.0\n\n"},{"id":"529001","messageId":"20251016200301.1595204-5-ramsay@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251016200301.1595204-1-ramsay@ramsayjones.plus.com","subject":"[PATCH v3 4/4] doc: add large-object-promisors.adoc to the docs build","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-16T20:03:01Z","receivedAt":"2025-10-16T20:06:45Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Commit 5040f9f164 (\"doc: add technical design doc for large object\npromisors\", 2025-02-18) added the large object promisors document\nas a technical document (with a '.txt' extension). The merge commit\n2c6fd30198 (\"Merge branch 'cc/lop-remote'\", 2025-03-05) seems to\nhave renamed the file with an '.adoc' extension.\n\nDespite the '.adoc' extension, this document was not being formatted\nby asciidoc(tor) as part of the docs build. In order to do so, add\nthe document to the make and meson build files.\n\nHaving added the document to the build, asciidoc and asciidoctor find\n(slightly different) problems with the syntax of the input document.\n\nThe first set of warnings (only issued by asciidoc) relate to some\n'section title out of sequence: expected level 3, got level 4'. This\ndocument uses 'setext' style of section headers, using a series of\nunderline characters, where the character used denotes the level of\nthe title. From document title to level 5 (see [1]), these characters\nare =, -, ~, ^, +. This does not seem to fit the error message, which\nimplies that those characters denote levels 0 -> 4. Replacing the headings\nunderlined with '+' by the '^' character eliminates these warnings.\n\nThe second set of warnings (only issued by asciidoctor) relate to some\nheadings which seem to use both arabic and roman numerals as part of\na single 'list' sequence. This elicited either 'unterminated listing\nblock' or (for example) 'list item index: expected I, got II' warnings.\nIn order not to mix arabic and roman numerals, remove the numeral from\nthe '0) Non goals' heading. Similarly, the remaining roman numeral\nentries had the ')' removed and turned into regular headings with I, II,\nIII ... at the beginning.\n\n[1] https://asciidoctor.org/docs/asciidoc-recommended-practices/\n\nSigned-off-by: Ramsay Jones \n---\n Documentation/Makefile | 1 +\n .../technical/large-object-promisors.adoc | 64 +++++++++----------\n Documentation/technical/meson.build | 1 +\n 3 files changed, 34 insertions(+), 32 deletions(-)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex a3fbd29744..a3ba25e659 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -122,6 +122,7 @@ TECH_DOCS += technical/bundle-uri\n TECH_DOCS += technical/commit-graph\n TECH_DOCS += technical/directory-rename-detection\n TECH_DOCS += technical/hash-function-transition\n+TECH_DOCS += technical/large-object-promisors\n TECH_DOCS += technical/long-running-process-protocol\n TECH_DOCS += technical/multi-pack-index\n TECH_DOCS += technical/packfile-uri\ndiff --git a/Documentation/technical/large-object-promisors.adoc b/Documentation/technical/large-object-promisors.adoc\nindex dea8dafa66..2aa815e023 100644\n--- a/Documentation/technical/large-object-promisors.adoc\n+++ b/Documentation/technical/large-object-promisors.adoc\n@@ -34,8 +34,8 @@ a new object representation for large blobs as discussed in:\n \n https://lore.kernel.org/git/xmqqbkdometi.fsf@gitster.g/\n \n-0) Non goals\n-------------\n+Non goals\n+---------\n \n - We will not discuss those client side improvements here, as they\n would require changes in different parts of Git than this effort.\n@@ -90,8 +90,8 @@ later in this document:\n even more to host content with larger blobs or more large blobs\n than currently.\n \n-I) Issues with the current situation\n-------------------------------------\n+I Issues with the current situation\n+-----------------------------------\n \n - Some statistics made on GitLab repos have shown that more than 75%\n of the disk space is used by blobs that are larger than 1MB and\n@@ -138,8 +138,8 @@ I) Issues with the current situation\n complaining that these tools require significant effort to set up,\n learn and use correctly.\n \n-II) Main features of the \"Large Object Promisors\" solution\n-----------------------------------------------------------\n+II Main features of the \"Large Object Promisors\" solution\n+---------------------------------------------------------\n \n The main features below should give a rough overview of how the\n solution may work. Details about needed elements can be found in\n@@ -166,7 +166,7 @@ format. They should be used along with main remotes that contain the\n other objects.\n \n Note 1\n-++++++\n+^^^^^^\n \n To clarify, a LOP is a normal promisor remote, except that:\n \n@@ -178,7 +178,7 @@ To clarify, a LOP is a normal promisor remote, except that:\n itself.\n \n Note 2\n-++++++\n+^^^^^^\n \n Git already makes it possible for a main remote to also be a promisor\n remote storing both regular objects and large blobs for a client that\n@@ -186,13 +186,13 @@ clones from it with a filter on blob size. But here we explicitly want\n to avoid that.\n \n Rationale\n-+++++++++\n+^^^^^^^^^\n \n LOPs aim to be good at handling large blobs while main remotes are\n already good at handling other objects.\n \n Implementation\n-++++++++++++++\n+^^^^^^^^^^^^^^\n \n Git already has support for multiple promisor remotes, see\n link:partial-clone.html#using-many-promisor-remotes[the partial clone documentation].\n@@ -213,19 +213,19 @@ remote helper (see linkgit:gitremote-helpers[7]) which makes the\n underlying object storage appear like a remote to Git.\n \n Note\n-++++\n+^^^^\n \n A LOP can be a promisor remote accessed using a remote helper by\n both some clients and the main remote.\n \n Rationale\n-+++++++++\n+^^^^^^^^^\n \n This looks like the simplest way to create LOPs that can cheaply\n handle many large blobs.\n \n Implementation\n-++++++++++++++\n+^^^^^^^^^^^^^^\n \n Remote helpers are quite easy to write as shell scripts, but it might\n be more efficient and maintainable to write them using other languages\n@@ -247,7 +247,7 @@ The underlying object storage that a LOP uses could also serve as\n storage for large files handled by Git LFS.\n \n Rationale\n-+++++++++\n+^^^^^^^^^\n \n This would simplify the server side if it wants to both use a LOP and\n act as a Git LFS server.\n@@ -259,7 +259,7 @@ On the server side, a main remote should have a way to offload to a\n LOP all its blobs with a size over a configurable threshold.\n \n Rationale\n-+++++++++\n+^^^^^^^^^\n \n This makes it easy to set things up and to clean things up. For\n example, an admin could use this to manually convert a repo not using\n@@ -268,7 +268,7 @@ some users would sometimes push large blobs, a cron job could use this\n to regularly make sure the large blobs are moved to the LOP.\n \n Implementation\n-++++++++++++++\n+^^^^^^^^^^^^^^\n \n Using something based on `git repack --filter=...` to separate the\n blobs we want to offload from the other Git objects could be a good\n@@ -284,13 +284,13 @@ should have ways to prevent oversize blobs to be fetched, and also\n perhaps pushed, into it.\n \n Rationale\n-+++++++++\n+^^^^^^^^^\n \n A main remote containing many oversize blobs would defeat the purpose\n of LOPs.\n \n Implementation\n-++++++++++++++\n+^^^^^^^^^^^^^^\n \n The way to offload to a LOP discussed in 4) above can be used to\n regularly offload oversize blobs. About preventing oversize blobs from\n@@ -326,18 +326,18 @@ large blobs directly from the LOP and the server would not need to\n fetch those blobs from the LOP to be able to serve the client.\n \n Note\n-++++\n+^^^^\n \n For fetches instead of clones, a protocol negotiation might not always\n happen, see the \"What about fetches?\" FAQ entry below for details.\n \n Rationale\n-+++++++++\n+^^^^^^^^^\n \n Security, configurability and efficiency of setting things up.\n \n Implementation\n-++++++++++++++\n+^^^^^^^^^^^^^^\n \n A \"promisor-remote\" protocol v2 capability looks like a good way to\n implement this. The way the client and server use this capability\n@@ -356,7 +356,7 @@ the client should be able to offload some large blobs it has fetched,\n but might not need anymore, to the LOP.\n \n Note\n-++++\n+^^^^\n \n It might depend on the context if it should be OK or not for clients\n to offload large blobs they have created, instead of fetched, directly\n@@ -367,13 +367,13 @@ This should be discussed and refined when we get closer to\n implementing this feature.\n \n Rationale\n-+++++++++\n+^^^^^^^^^\n \n On the client, the easiest way to deal with unneeded large blobs is to\n offload them.\n \n Implementation\n-++++++++++++++\n+^^^^^^^^^^^^^^\n \n This is very similar to what 4) above is about, except on the client\n side instead of the server side. So a good solution to 4) could likely\n@@ -385,8 +385,8 @@ when cloning (see 6) above). Also if the large blobs were fetched from\n a LOP, it is likely, and can easily be confirmed, that the LOP still\n has them, so that they can just be removed from the client.\n \n-III) Benefits of using LOPs\n----------------------------\n+III Benefits of using LOPs\n+--------------------------\n \n Many benefits are related to the issues discussed in \"I) Issues with\n the current situation\" above:\n@@ -406,8 +406,8 @@ the current situation\" above:\n \n - Reduced storage needs on the client side.\n \n-IV) FAQ\n--------\n+IV FAQ\n+------\n \n What about using multiple LOPs on the server and client side?\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n@@ -533,7 +533,7 @@ some objects it already knows about but doesn't have because they are\n on a promisor remote.\n \n Regular fetch\n-+++++++++++++\n+^^^^^^^^^^^^^\n \n In a regular fetch, the client will contact the main remote and a\n protocol negotiation will happen between them. It's a good thing that\n@@ -551,7 +551,7 @@ new fetch will happen in the same way as the previous clone or fetch,\n using, or not using, the same LOP(s) as last time.\n \n \"Backfill\" or \"lazy\" fetch\n-++++++++++++++++++++++++++\n+^^^^^^^^^^^^^^^^^^^^^^^^^^\n \n When there is a backfill fetch, the client doesn't necessarily contact\n the main remote first. It will try to fetch from its promisor remotes\n@@ -576,8 +576,8 @@ from the client when it fetches from them. The client could get the\n token when performing a protocol negotiation with the main remote (see\n section II.6 above).\n \n-V) Future improvements\n-----------------------\n+V Future improvements\n+---------------------\n \n It is expected that at the beginning using LOPs will be mostly worth\n it either in a corporate context where the Git version that clients\ndiff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build\nindex a13aafcfbb..34b5ebe5c3 100644\n--- a/Documentation/technical/meson.build\n+++ b/Documentation/technical/meson.build\n@@ -13,6 +13,7 @@ articles = [\n 'commit-graph.adoc',\n 'directory-rename-detection.adoc',\n 'hash-function-transition.adoc',\n+ 'large-object-promisors.adoc',\n 'long-running-process-protocol.adoc',\n 'multi-pack-index.adoc',\n 'packfile-uri.adoc',\n-- \n2.51.0\n\n"},{"id":"529083","messageId":"e009f8c7-9809-4ea7-ba6a-8351282de8ca@ramsayjones.plus.com","threadId":"64240","inReplyTo":"20251016200301.1595204-5-ramsay@ramsayjones.plus.com","subject":"Re: [PATCH v3 4/4] doc: add large-object-promisors.adoc to the docs build","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-17T16:37:41Z","receivedAt":"2025-10-17T16:40:51Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Sorry, I meant to add Christian on CC:, since he wrote this document\nin commit 5040f9f164, but I totally forgot. :( Sorry about that.\n\nOn 16/10/2025 9:03 pm, Ramsay Jones wrote:\n> Commit 5040f9f164 (\"doc: add technical design doc for large object\n> promisors\", 2025-02-18) added the large object promisors document\n> as a technical document (with a '.txt' extension). The merge commit\n> 2c6fd30198 (\"Merge branch 'cc/lop-remote'\", 2025-03-05) seems to\n> have renamed the file with an '.adoc' extension.\n> \n> Despite the '.adoc' extension, this document was not being formatted\n> by asciidoc(tor) as part of the docs build. In order to do so, add\n> the document to the make and meson build files.\n> \n> Having added the document to the build, asciidoc and asciidoctor find\n> (slightly different) problems with the syntax of the input document.\n> \n> The first set of warnings (only issued by asciidoc) relate to some\n> 'section title out of sequence: expected level 3, got level 4'. This\n> document uses 'setext' style of section headers, using a series of\n> underline characters, where the character used denotes the level of\n> the title. From document title to level 5 (see [1]), these characters\n> are =, -, ~, ^, +. This does not seem to fit the error message, which\n> implies that those characters denote levels 0 -> 4. Replacing the headings\n> underlined with '+' by the '^' character eliminates these warnings.\n> \n> The second set of warnings (only issued by asciidoctor) relate to some\n> headings which seem to use both arabic and roman numerals as part of\n> a single 'list' sequence. This elicited either 'unterminated listing\n> block' or (for example) 'list item index: expected I, got II' warnings.\n> In order not to mix arabic and roman numerals, remove the numeral from\n> the '0) Non goals' heading. Similarly, the remaining roman numeral\n> entries had the ')' removed and turned into regular headings with I, II,\n> III ... at the beginning.\n> \n> [1] https://asciidoctor.org/docs/asciidoc-recommended-practices/\n> \n> Signed-off-by: Ramsay Jones \n> ---\n> Documentation/Makefile | 1 +\n> .../technical/large-object-promisors.adoc | 64 +++++++++----------\n> Documentation/technical/meson.build | 1 +\n> 3 files changed, 34 insertions(+), 32 deletions(-)\n> \n> diff --git a/Documentation/Makefile b/Documentation/Makefile\n> index a3fbd29744..a3ba25e659 100644\n> --- a/Documentation/Makefile\n> +++ b/Documentation/Makefile\n> @@ -122,6 +122,7 @@ TECH_DOCS += technical/bundle-uri\n> TECH_DOCS += technical/commit-graph\n> TECH_DOCS += technical/directory-rename-detection\n> TECH_DOCS += technical/hash-function-transition\n> +TECH_DOCS += technical/large-object-promisors\n> TECH_DOCS += technical/long-running-process-protocol\n> TECH_DOCS += technical/multi-pack-index\n> TECH_DOCS += technical/packfile-uri\n> diff --git a/Documentation/technical/large-object-promisors.adoc b/Documentation/technical/large-object-promisors.adoc\n> index dea8dafa66..2aa815e023 100644\n> --- a/Documentation/technical/large-object-promisors.adoc\n> +++ b/Documentation/technical/large-object-promisors.adoc\n> @@ -34,8 +34,8 @@ a new object representation for large blobs as discussed in:\n> \n> https://lore.kernel.org/git/xmqqbkdometi.fsf@gitster.g/\n> \n> -0) Non goals\n> -------------\n> +Non goals\n> +---------\n> \n> - We will not discuss those client side improvements here, as they\n> would require changes in different parts of Git than this effort.\n> @@ -90,8 +90,8 @@ later in this document:\n> even more to host content with larger blobs or more large blobs\n> than currently.\n> \n> -I) Issues with the current situation\n> -------------------------------------\n> +I Issues with the current situation\n> +-----------------------------------\n> \n> - Some statistics made on GitLab repos have shown that more than 75%\n> of the disk space is used by blobs that are larger than 1MB and\n> @@ -138,8 +138,8 @@ I) Issues with the current situation\n> complaining that these tools require significant effort to set up,\n> learn and use correctly.\n> \n> -II) Main features of the \"Large Object Promisors\" solution\n> -----------------------------------------------------------\n> +II Main features of the \"Large Object Promisors\" solution\n> +---------------------------------------------------------\n> \n> The main features below should give a rough overview of how the\n> solution may work. Details about needed elements can be found in\n> @@ -166,7 +166,7 @@ format. They should be used along with main remotes that contain the\n> other objects.\n> \n> Note 1\n> -++++++\n> +^^^^^^\n> \n> To clarify, a LOP is a normal promisor remote, except that:\n> \n> @@ -178,7 +178,7 @@ To clarify, a LOP is a normal promisor remote, except that:\n> itself.\n> \n> Note 2\n> -++++++\n> +^^^^^^\n> \n> Git already makes it possible for a main remote to also be a promisor\n> remote storing both regular objects and large blobs for a client that\n> @@ -186,13 +186,13 @@ clones from it with a filter on blob size. But here we explicitly want\n> to avoid that.\n> \n> Rationale\n> -+++++++++\n> +^^^^^^^^^\n> \n> LOPs aim to be good at handling large blobs while main remotes are\n> already good at handling other objects.\n> \n> Implementation\n> -++++++++++++++\n> +^^^^^^^^^^^^^^\n> \n> Git already has support for multiple promisor remotes, see\n> link:partial-clone.html#using-many-promisor-remotes[the partial clone documentation].\n> @@ -213,19 +213,19 @@ remote helper (see linkgit:gitremote-helpers[7]) which makes the\n> underlying object storage appear like a remote to Git.\n> \n> Note\n> -++++\n> +^^^^\n> \n> A LOP can be a promisor remote accessed using a remote helper by\n> both some clients and the main remote.\n> \n> Rationale\n> -+++++++++\n> +^^^^^^^^^\n> \n> This looks like the simplest way to create LOPs that can cheaply\n> handle many large blobs.\n> \n> Implementation\n> -++++++++++++++\n> +^^^^^^^^^^^^^^\n> \n> Remote helpers are quite easy to write as shell scripts, but it might\n> be more efficient and maintainable to write them using other languages\n> @@ -247,7 +247,7 @@ The underlying object storage that a LOP uses could also serve as\n> storage for large files handled by Git LFS.\n> \n> Rationale\n> -+++++++++\n> +^^^^^^^^^\n> \n> This would simplify the server side if it wants to both use a LOP and\n> act as a Git LFS server.\n> @@ -259,7 +259,7 @@ On the server side, a main remote should have a way to offload to a\n> LOP all its blobs with a size over a configurable threshold.\n> \n> Rationale\n> -+++++++++\n> +^^^^^^^^^\n> \n> This makes it easy to set things up and to clean things up. For\n> example, an admin could use this to manually convert a repo not using\n> @@ -268,7 +268,7 @@ some users would sometimes push large blobs, a cron job could use this\n> to regularly make sure the large blobs are moved to the LOP.\n> \n> Implementation\n> -++++++++++++++\n> +^^^^^^^^^^^^^^\n> \n> Using something based on `git repack --filter=...` to separate the\n> blobs we want to offload from the other Git objects could be a good\n> @@ -284,13 +284,13 @@ should have ways to prevent oversize blobs to be fetched, and also\n> perhaps pushed, into it.\n> \n> Rationale\n> -+++++++++\n> +^^^^^^^^^\n> \n> A main remote containing many oversize blobs would defeat the purpose\n> of LOPs.\n> \n> Implementation\n> -++++++++++++++\n> +^^^^^^^^^^^^^^\n> \n> The way to offload to a LOP discussed in 4) above can be used to\n> regularly offload oversize blobs. About preventing oversize blobs from\n> @@ -326,18 +326,18 @@ large blobs directly from the LOP and the server would not need to\n> fetch those blobs from the LOP to be able to serve the client.\n> \n> Note\n> -++++\n> +^^^^\n> \n> For fetches instead of clones, a protocol negotiation might not always\n> happen, see the \"What about fetches?\" FAQ entry below for details.\n> \n> Rationale\n> -+++++++++\n> +^^^^^^^^^\n> \n> Security, configurability and efficiency of setting things up.\n> \n> Implementation\n> -++++++++++++++\n> +^^^^^^^^^^^^^^\n> \n> A \"promisor-remote\" protocol v2 capability looks like a good way to\n> implement this. The way the client and server use this capability\n> @@ -356,7 +356,7 @@ the client should be able to offload some large blobs it has fetched,\n> but might not need anymore, to the LOP.\n> \n> Note\n> -++++\n> +^^^^\n> \n> It might depend on the context if it should be OK or not for clients\n> to offload large blobs they have created, instead of fetched, directly\n> @@ -367,13 +367,13 @@ This should be discussed and refined when we get closer to\n> implementing this feature.\n> \n> Rationale\n> -+++++++++\n> +^^^^^^^^^\n> \n> On the client, the easiest way to deal with unneeded large blobs is to\n> offload them.\n> \n> Implementation\n> -++++++++++++++\n> +^^^^^^^^^^^^^^\n> \n> This is very similar to what 4) above is about, except on the client\n> side instead of the server side. So a good solution to 4) could likely\n> @@ -385,8 +385,8 @@ when cloning (see 6) above). Also if the large blobs were fetched from\n> a LOP, it is likely, and can easily be confirmed, that the LOP still\n> has them, so that they can just be removed from the client.\n> \n> -III) Benefits of using LOPs\n> ----------------------------\n> +III Benefits of using LOPs\n> +--------------------------\n> \n> Many benefits are related to the issues discussed in \"I) Issues with\n> the current situation\" above:\n> @@ -406,8 +406,8 @@ the current situation\" above:\n> \n> - Reduced storage needs on the client side.\n> \n> -IV) FAQ\n> --------\n> +IV FAQ\n> +------\n> \n> What about using multiple LOPs on the server and client side?\n> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n> @@ -533,7 +533,7 @@ some objects it already knows about but doesn't have because they are\n> on a promisor remote.\n> \n> Regular fetch\n> -+++++++++++++\n> +^^^^^^^^^^^^^\n> \n> In a regular fetch, the client will contact the main remote and a\n> protocol negotiation will happen between them. It's a good thing that\n> @@ -551,7 +551,7 @@ new fetch will happen in the same way as the previous clone or fetch,\n> using, or not using, the same LOP(s) as last time.\n> \n> \"Backfill\" or \"lazy\" fetch\n> -++++++++++++++++++++++++++\n> +^^^^^^^^^^^^^^^^^^^^^^^^^^\n> \n> When there is a backfill fetch, the client doesn't necessarily contact\n> the main remote first. It will try to fetch from its promisor remotes\n> @@ -576,8 +576,8 @@ from the client when it fetches from them. The client could get the\n> token when performing a protocol negotiation with the main remote (see\n> section II.6 above).\n> \n> -V) Future improvements\n> -----------------------\n> +V Future improvements\n> +---------------------\n> \n> It is expected that at the beginning using LOPs will be mostly worth\n> it either in a corporate context where the Git version that clients\n> diff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build\n> index a13aafcfbb..34b5ebe5c3 100644\n> --- a/Documentation/technical/meson.build\n> +++ b/Documentation/technical/meson.build\n> @@ -13,6 +13,7 @@ articles = [\n> 'commit-graph.adoc',\n> 'directory-rename-detection.adoc',\n> 'hash-function-transition.adoc',\n> + 'large-object-promisors.adoc',\n> 'long-running-process-protocol.adoc',\n> 'multi-pack-index.adoc',\n> 'packfile-uri.adoc',\n\n"},{"id":"529521","messageId":"xmqq3479v2e9.fsf@gitster.g","threadId":"64240","inReplyTo":"20251016200301.1595204-1-ramsay@ramsayjones.plus.com","subject":"Re: [PATCH v3 0/4] technical docs in make build","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-23T19:33:50Z","receivedAt":"2025-10-23T19:33:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramsay Jones writes:\n\n> Changes in v3:\n>\n> - old patch #1 discarded since it was separated into its own branch\n> ('rj/doc-missing-technical-docs' in next)\n> - tyop in patch #2 (old patch #3)\n> - new patch #4\n>\n> A range diff against v2 is given below.\n>\n> Note that the two remaining problems (see v2 below) have not been\n> addressed but, even without a solution, these patches represent a\n> good improvement. ;) (I am still hopeful that an asciidoc guru will\n> turn up!)\n>\n> NOTE: this series is based on the v2-version of the patch #1, which\n> in turn is based on commit 6ad8021821 (\"The fifth batch\", 2025-08-29).\n\nLet's merge this iteration down and if there are things that still\nneed working, do them on top.\n\nThanks.\n"},{"id":"529524","messageId":"fb18d753-75f8-49a6-a92b-4f4e810bc408@ramsayjones.plus.com","threadId":"64240","inReplyTo":"xmqq3479v2e9.fsf@gitster.g","subject":"Re: [PATCH v3 0/4] technical docs in make build","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-10-23T20:06:29Z","receivedAt":"2025-10-23T20:09:39Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 23/10/2025 8:33 pm, Junio C Hamano wrote:\n> Ramsay Jones writes:\n> \n>> Changes in v3:\n>>\n>> - old patch #1 discarded since it was separated into its own branch\n>> ('rj/doc-missing-technical-docs' in next)\n>> - tyop in patch #2 (old patch #3)\n>> - new patch #4\n>>\n>> A range diff against v2 is given below.\n>>\n>> Note that the two remaining problems (see v2 below) have not been\n>> addressed but, even without a solution, these patches represent a\n>> good improvement. ;) (I am still hopeful that an asciidoc guru will\n>> turn up!)\n>>\n>> NOTE: this series is based on the v2-version of the patch #1, which\n>> in turn is based on commit 6ad8021821 (\"The fifth batch\", 2025-08-29).\n> \n> Let's merge this iteration down and if there are things that still\n> need working, do them on top.\n> \n> Thanks.\n\nThat sounds good to me.\n\nThanks!\n\nATB,\nRamsay Jones\n\n\n\n"}]}