{"thread":{"id":"57946","subject":"[PATCH 0/2] bitmap-format.txt: fix some formatting issues and include checksum info","startedAt":"2022-06-02T13:52:52Z","lastAt":"2022-06-16T21:19:03Z","messageCount":37,"participants":["Abhradeep Chakraborty via GitGitGadget","Junio C Hamano","Abhradeep Chakraborty","Taylor Blau"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"456511","messageId":"pull.1246.git.1654177966.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":null,"subject":"[PATCH 0/2] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-02T13:52:44Z","receivedAt":"2022-06-02T13:52:52Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"There are some issues in the bitmap-format html page. For example, some\nnested lists are shown as top-level lists (e.g. [1]- Here\nBITMAP_OPT_FULL_DAG (0x1) and BITMAP_OPT_HASH_CACHE (0x4) are shown as\ntop-level list).\n\nThe first commit fix those.\n\nThe second commit is about including the info of trailing checksum in the\nbitmap-format documentation.\n\n[1] https://git-scm.com/docs/bitmap-format#_on_disk_format\n\nAbhradeep Chakraborty (2):\n bitmap-format.txt: fix some formatting issues\n bitmap-format.txt: add information for trailing checksum\n\n Documentation/technical/bitmap-format.txt | 100 +++++++++++-----------\n 1 file changed, 49 insertions(+), 51 deletions(-)\n\n\nbase-commit: 2668e3608e47494f2f10ef2b6e69f08a84816bcb\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1246%2FAbhra303%2Ffix-doc-formatting-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1246/Abhra303/fix-doc-formatting-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1246\n-- \ngitgitgadget\n"},{"id":"456512","messageId":"ba534b5d4868b4451b377e74d1b554fb1d2e8ad2.1654177966.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.git.1654177966.gitgitgadget@gmail.com","subject":"[PATCH 2/2] bitmap-format.txt: add information for trailing checksum","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-02T13:52:46Z","receivedAt":"2022-06-02T13:52:54Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nBitmap file has a trailing checksum at the end of the file. However\nthere is no information in the bitmap-format documentation about it.\n\nAdd a trailer section to include the trailing checksum info in the\n`Documentation/technical/bitmap-format.txt` file.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/technical/bitmap-format.txt | 4 ++++\n 1 file changed, 4 insertions(+)\n\ndiff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\nindex 110d7ddf8ed..6846e7221a7 100644\n--- a/Documentation/technical/bitmap-format.txt\n+++ b/Documentation/technical/bitmap-format.txt\n@@ -125,6 +125,10 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \n \t\t\t** The compressed bitmap itself, see Appendix A.\n \n+\t* TRAILER:\n+\n+\t\tIndex checksum of the above contents.\n+\n == Appendix A: Serialization format for an EWAH bitmap\n \n Ewah bitmaps are serialized in the same protocol as the JAVAEWAH\n-- \ngitgitgadget\n"},{"id":"456513","messageId":"976361e624a3dd58c8f291358d42f4e4c66eb266.1654177966.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.git.1654177966.gitgitgadget@gmail.com","subject":"[PATCH 1/2] bitmap-format.txt: fix some formatting issues","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-02T13:52:45Z","receivedAt":"2022-06-02T13:53:02Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nThe asciidoc generated html for `Documentation/technical/bitmap-\nformat.txt` is broken. This is mainly because `-` is used for nested\nlists (which is not allowed in asciidoc) instead of `*`.\n\nFix these and also reformat it (e.g. removing some blank lines) for\nbetter readability of the html page.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/technical/bitmap-format.txt | 96 +++++++++++------------\n 1 file changed, 45 insertions(+), 51 deletions(-)\n\ndiff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\nindex 04b3ec21785..110d7ddf8ed 100644\n--- a/Documentation/technical/bitmap-format.txt\n+++ b/Documentation/technical/bitmap-format.txt\n@@ -39,7 +39,7 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \n == On-disk format\n \n-\t- A header appears at the beginning:\n+\t* A header appears at the beginning:\n \n \t\t4-byte signature: {'B', 'I', 'T', 'M'}\n \n@@ -48,35 +48,30 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \t\t\tof the bitmap index (the same one as JGit).\n \n \t\t2-byte flags (network byte order)\n-\n \t\t\tThe following flags are supported:\n-\n-\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n-\t\t\tThis flag must always be present. It implies that the\n-\t\t\tbitmap index has been generated for a packfile or\n-\t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n-\t\t\tevery single object in the packfile/MIDX can find its\n-\t\t\tparent links inside the same packfile/MIDX). This is a\n-\t\t\trequirement for the bitmap index format, also present in\n-\t\t\tJGit, that greatly reduces the complexity of the\n-\t\t\timplementation.\n-\n-\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n-\t\t\tIf present, the end of the bitmap file contains\n-\t\t\t`N` 32-bit name-hash values, one per object in the\n-\t\t\tpack/MIDX. The format and meaning of the name-hash is\n-\t\t\tdescribed below.\n+\t\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n+\t\t\t\tThis flag must always be present. It implies that the\n+\t\t\t\tbitmap index has been generated for a packfile or\n+\t\t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n+\t\t\t\tevery single object in the packfile/MIDX can find its\n+\t\t\t\tparent links inside the same packfile/MIDX). This is a\n+\t\t\t\trequirement for the bitmap index format, also present in\n+\t\t\t\tJGit, that greatly reduces the complexity of the\n+\t\t\t\timplementation.\n+\t\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n+\t\t\t\tIf present, the end of the bitmap file contains\n+\t\t\t\t`N` 32-bit name-hash values, one per object in the\n+\t\t\t\tpack/MIDX. The format and meaning of the name-hash is\n+\t\t\t\tdescribed below.\n \n \t\t4-byte entry count (network byte order)\n-\n \t\t\tThe total count of entries (bitmapped commits) in this bitmap index.\n \n \t\t20-byte checksum\n-\n \t\t\tThe SHA1 checksum of the pack/MIDX this bitmap index\n \t\t\tbelongs to.\n \n-\t- 4 EWAH bitmaps that act as type indexes\n+\t* 4 EWAH bitmaps that act as type indexes\n \n \t\tType indexes are serialized after the hash cache in the shape\n \t\tof four EWAH bitmaps stored consecutively (see Appendix A for\n@@ -84,7 +79,6 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \n \t\tThere is a bitmap for each Git object type, stored in the following\n \t\torder:\n-\n \t\t\t- Commits\n \t\t\t- Trees\n \t\t\t- Blobs\n@@ -97,39 +91,39 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \t\tin a full set (all bits set), and the AND of all 4 bitmaps will\n \t\tresult in an empty bitmap (no bits set).\n \n-\t- N entries with compressed bitmaps, one for each indexed commit\n+\t* N entries with compressed bitmaps, one for each indexed commit\n \n \t\tWhere `N` is the total amount of entries in this bitmap index.\n \t\tEach entry contains the following:\n \n-\t\t- 4-byte object position (network byte order)\n-\t\t\tThe position **in the index for the packfile or\n-\t\t\tmulti-pack index** where the bitmap for this commit is\n-\t\t\tfound.\n-\n-\t\t- 1-byte XOR-offset\n-\t\t\tThe xor offset used to compress this bitmap. For an entry\n-\t\t\tin position `x`, a XOR offset of `y` means that the actual\n-\t\t\tbitmap representing this commit is composed by XORing the\n-\t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n-\t\t\tthe bitmap `y` entries before this one).\n-\n-\t\t\tNote that this compression can be recursive. In order to\n-\t\t\tXOR this entry with a previous one, the previous entry needs\n-\t\t\tto be decompressed first, and so on.\n-\n-\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n-\t\t\txor'ed against one of the 160 entries preceding it). This\n-\t\t\tnumber is always positive, and hence entries are always xor'ed\n-\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n-\t\t\tin the index.\n-\n-\t\t- 1-byte flags for this bitmap\n-\t\t\tAt the moment the only available flag is `0x1`, which hints\n-\t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n-\t\t\tfor the repository.\n-\n-\t\t- The compressed bitmap itself, see Appendix A.\n+\t\t\t** 4-byte object position (network byte order)\n+\t\t\t\tThe position **in the index for the packfile or\n+\t\t\t\tmulti-pack index** where the bitmap for this commit is\n+\t\t\t\tfound.\n+\n+\t\t\t** 1-byte XOR-offset\n+\t\t\t\tThe xor offset used to compress this bitmap. For an entry\n+\t\t\t\tin position `x`, a XOR offset of `y` means that the actual\n+\t\t\t\tbitmap representing this commit is composed by XORing the\n+\t\t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n+\t\t\t\tthe bitmap `y` entries before this one).\n+\n+\t\t\t\tNote that this compression can be recursive. In order to\n+\t\t\t\tXOR this entry with a previous one, the previous entry needs\n+\t\t\t\tto be decompressed first, and so on.\n+\n+\t\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n+\t\t\t\txor'ed against one of the 160 entries preceding it). This\n+\t\t\t\tnumber is always positive, and hence entries are always xor'ed\n+\t\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n+\t\t\t\tin the index.\n+\n+\t\t\t** 1-byte flags for this bitmap\n+\t\t\t\tAt the moment the only available flag is `0x1`, which hints\n+\t\t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n+\t\t\t\tfor the repository.\n+\n+\t\t\t** The compressed bitmap itself, see Appendix A.\n \n == Appendix A: Serialization format for an EWAH bitmap\n \n-- \ngitgitgadget\n\n"},{"id":"456718","messageId":"xmqqsfohbxbp.fsf@gitster.g","threadId":"57946","inReplyTo":"976361e624a3dd58c8f291358d42f4e4c66eb266.1654177966.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/2] bitmap-format.txt: fix some formatting issues","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-06T15:55:54Z","receivedAt":"2022-06-06T15:56:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Abhradeep Chakraborty via GitGitGadget\" \nwrites:\n\n> From: Abhradeep Chakraborty \n> Cc: git@vger.kernel.org, Abhradeep Chakraborty \n\nIdentify those who may have input with \"git log --no-merges\" and add\nthem here, perhaps?\n\n> The asciidoc generated html for `Documentation/technical/bitmap-\n> format.txt` is broken. This is mainly because `-` is used for nested\n> lists (which is not allowed in asciidoc) instead of `*`.\n\nAre we missing another step that must come much earlier than this\npatch? It seems to me that Documentation/Makefile does not even\nconsider that we should feed this file to AsciiDoc.\n\n> Fix these and also reformat it (e.g. removing some blank lines) for\n> better readability of the html page.\n\nDo these blank lines hurt very badly how the end-result is formatted\nin HTML? Does the extra indentation between the line with \"The\nfollowing flags are supported\" on it and the two bullet items in the\nheader make the output better in significant way?\n\nThese changes make the input text much harder to read, and are not\nvery welcome, so unless they are part of \"fixing generated HTML is\nbroken\", please omit them. As evidenced by the lack of HTML output\nin the build system, a lot more folks read this document in text than\nin HTML, and readability of the source matters.\n\nThanks.\n\n> Signed-off-by: Abhradeep Chakraborty \n> ---\n> Documentation/technical/bitmap-format.txt | 96 +++++++++++------------\n> 1 file changed, 45 insertions(+), 51 deletions(-)\n>\n> diff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\n> index 04b3ec21785..110d7ddf8ed 100644\n> --- a/Documentation/technical/bitmap-format.txt\n> +++ b/Documentation/technical/bitmap-format.txt\n> @@ -39,7 +39,7 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n> \n> == On-disk format\n> \n> -\t- A header appears at the beginning:\n> +\t* A header appears at the beginning:\n> \n> \t\t4-byte signature: {'B', 'I', 'T', 'M'}\n> \n> @@ -48,35 +48,30 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n> \t\t\tof the bitmap index (the same one as JGit).\n> \n> \t\t2-byte flags (network byte order)\n> -\n> \t\t\tThe following flags are supported:\n> -\n> -\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n> -\t\t\tThis flag must always be present. It implies that the\n> -\t\t\tbitmap index has been generated for a packfile or\n> -\t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n> -\t\t\tevery single object in the packfile/MIDX can find its\n> -\t\t\tparent links inside the same packfile/MIDX). This is a\n> -\t\t\trequirement for the bitmap index format, also present in\n> -\t\t\tJGit, that greatly reduces the complexity of the\n> -\t\t\timplementation.\n> -\n> -\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n> -\t\t\tIf present, the end of the bitmap file contains\n> -\t\t\t`N` 32-bit name-hash values, one per object in the\n> -\t\t\tpack/MIDX. The format and meaning of the name-hash is\n> -\t\t\tdescribed below.\n> +\t\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n> +\t\t\t\tThis flag must always be present. It implies that the\n> +\t\t\t\tbitmap index has been generated for a packfile or\n> +\t\t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n> +\t\t\t\tevery single object in the packfile/MIDX can find its\n> +\t\t\t\tparent links inside the same packfile/MIDX). This is a\n> +\t\t\t\trequirement for the bitmap index format, also present in\n> +\t\t\t\tJGit, that greatly reduces the complexity of the\n> +\t\t\t\timplementation.\n> +\t\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n> +\t\t\t\tIf present, the end of the bitmap file contains\n> +\t\t\t\t`N` 32-bit name-hash values, one per object in the\n> +\t\t\t\tpack/MIDX. The format and meaning of the name-hash is\n> +\t\t\t\tdescribed below.\n> \n> \t\t4-byte entry count (network byte order)\n> -\n> \t\t\tThe total count of entries (bitmapped commits) in this bitmap index.\n> \n> \t\t20-byte checksum\n> -\n> \t\t\tThe SHA1 checksum of the pack/MIDX this bitmap index\n> \t\t\tbelongs to.\n> \n> -\t- 4 EWAH bitmaps that act as type indexes\n> +\t* 4 EWAH bitmaps that act as type indexes\n> \n> \t\tType indexes are serialized after the hash cache in the shape\n> \t\tof four EWAH bitmaps stored consecutively (see Appendix A for\n> @@ -84,7 +79,6 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n> \n> \t\tThere is a bitmap for each Git object type, stored in the following\n> \t\torder:\n> -\n> \t\t\t- Commits\n> \t\t\t- Trees\n> \t\t\t- Blobs\n> @@ -97,39 +91,39 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n> \t\tin a full set (all bits set), and the AND of all 4 bitmaps will\n> \t\tresult in an empty bitmap (no bits set).\n> \n> -\t- N entries with compressed bitmaps, one for each indexed commit\n> +\t* N entries with compressed bitmaps, one for each indexed commit\n> \n> \t\tWhere `N` is the total amount of entries in this bitmap index.\n> \t\tEach entry contains the following:\n> \n> -\t\t- 4-byte object position (network byte order)\n> -\t\t\tThe position **in the index for the packfile or\n> -\t\t\tmulti-pack index** where the bitmap for this commit is\n> -\t\t\tfound.\n> -\n> -\t\t- 1-byte XOR-offset\n> -\t\t\tThe xor offset used to compress this bitmap. For an entry\n> -\t\t\tin position `x`, a XOR offset of `y` means that the actual\n> -\t\t\tbitmap representing this commit is composed by XORing the\n> -\t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n> -\t\t\tthe bitmap `y` entries before this one).\n> -\n> -\t\t\tNote that this compression can be recursive. In order to\n> -\t\t\tXOR this entry with a previous one, the previous entry needs\n> -\t\t\tto be decompressed first, and so on.\n> -\n> -\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n> -\t\t\txor'ed against one of the 160 entries preceding it). This\n> -\t\t\tnumber is always positive, and hence entries are always xor'ed\n> -\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n> -\t\t\tin the index.\n> -\n> -\t\t- 1-byte flags for this bitmap\n> -\t\t\tAt the moment the only available flag is `0x1`, which hints\n> -\t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n> -\t\t\tfor the repository.\n> -\n> -\t\t- The compressed bitmap itself, see Appendix A.\n> +\t\t\t** 4-byte object position (network byte order)\n> +\t\t\t\tThe position **in the index for the packfile or\n> +\t\t\t\tmulti-pack index** where the bitmap for this commit is\n> +\t\t\t\tfound.\n> +\n> +\t\t\t** 1-byte XOR-offset\n> +\t\t\t\tThe xor offset used to compress this bitmap. For an entry\n> +\t\t\t\tin position `x`, a XOR offset of `y` means that the actual\n> +\t\t\t\tbitmap representing this commit is composed by XORing the\n> +\t\t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n> +\t\t\t\tthe bitmap `y` entries before this one).\n> +\n> +\t\t\t\tNote that this compression can be recursive. In order to\n> +\t\t\t\tXOR this entry with a previous one, the previous entry needs\n> +\t\t\t\tto be decompressed first, and so on.\n> +\n> +\t\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n> +\t\t\t\txor'ed against one of the 160 entries preceding it). This\n> +\t\t\t\tnumber is always positive, and hence entries are always xor'ed\n> +\t\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n> +\t\t\t\tin the index.\n> +\n> +\t\t\t** 1-byte flags for this bitmap\n> +\t\t\t\tAt the moment the only available flag is `0x1`, which hints\n> +\t\t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n> +\t\t\t\tfor the repository.\n> +\n> +\t\t\t** The compressed bitmap itself, see Appendix A.\n> \n> == Appendix A: Serialization format for an EWAH bitmap\n"},{"id":"456782","messageId":"20220607102504.2243-1-chakrabortyabhradeep79@gmail.com","threadId":"57946","inReplyTo":"xmqqsfohbxbp.fsf@gitster.g","subject":"Re: [PATCH 1/2] bitmap-format.txt: fix some formatting issues","fromName":"Abhradeep Chakraborty","fromEmail":"chakrabortyabhradeep79@gmail.com","sentAt":"2022-06-07T10:25:04Z","receivedAt":"2022-06-07T10:27:38Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"Junio C Hamano wrote:\n\n> Identify those who may have input with \"git log --no-merges\" and add\n> them here, perhaps?\n\nThanks, I hopefully cc'd all the people who can give some input about the\npatch except Peff. I got to know that he took a break so I decided not to\ncc him (will surely do if you say). I would love to hear from other people\nwho has knowledge on asciidoc.\n\nI previously informed Taylor and Kaartic about the patch but forgot to\ncc them :P\n\nAnother thing to note that the checksum that I included in the last\ncommit is suggested by Taylor himself. I was having problem to understand\nsome portion of `load_bitmap_header()` (because I wasn't aware of the\ntrailing checksum) when he cleared my doubt by saying that a trailer\nchecksum exists and also suggested to make a PR addressing that -\n\n> I'm glad that it was helpful! If you think others may be confused by the same, feel free to write a patch modifying Documentation/technical/bitmap-format.txt to point out the trailing checksum.\n\nJunio wrote -\n\n> Are we missing another step that must come much earlier than this\n> patch? It seems to me that Documentation/Makefile does not even\n> consider that we should feed this file to AsciiDoc.\n\nI also think the same. At first, I thought this is intentional. When\nI ran `make doc` (to test the resulting html file), it didn't generate\nany html file for bitmap-format.txt. But thankfully there is an online\nasciidoc editor[1] where you can check the resulting html file. You also\ncan check the resulting html by copy-pasting the content[2] of my github\nbranch bitmap-format file to that editor.\n\nWill write a patch for it.\n\nThe current broken page can be found at - https://git-scm.com/docs/bitmap-format\n\n> Do these blank lines hurt very badly how the end-result is formatted\n> in HTML? Does the extra indentation between the line with \"The\n> following flags are supported\" on it and the two bullet items in the\n> header make the output better in significant way?\n\nAnswering to the first question - yes, those are necessary to improve\nthe html readability (you can verify that by including and removing the\nblank lines in the editor and obsering the changes). This ensures that\nall the related paragraphes are contained in the same block.\n\nThe extra identations are not necessary. I add those because I thought\nthat these would be visually better for html page readers. If you think\nit does the opposite, I can remove those.\n\nI tried to use two bullets as less as possible ( In most cases, nested\nlists came under 
 blocks, so I didn't have to use two bullets).\nBut in one case, I had to use it for nested lists (Try the editor to\nsee the rendered output).\n\n> These changes make the input text much harder to read, and are not\n> very welcome, so unless they are part of \"fixing generated HTML is\n> broken\", please omit them.  As evidenced by the lack of HTML output\n> in the build system, a lot more folks read this document in text than\n> in HTML, and readability of the source matters.\n\nOkay, I will then remove those extra indentations. But besides that, all\nare necessary.\n\nI admit that readability of source matters but I think html pages are\nalso important (even more important)  for people who don't have the\nsource codes and want to know the git internals.\n\nThanks :)\n\n[1] https://asciidoclive.com/edit/scratch/1\n[2] https://github.com/Abhra303/git/blob/fix-doc-formatting/Documentation/technical/bitmap-format.txt\n"},{"id":"456802","messageId":"2171d31fb2b783371bdc31ba54856dea8224de65.1654623814.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v2.git.1654623814.gitgitgadget@gmail.com","subject":"[PATCH v2 3/3] bitmap-format.txt: add information for trailing checksum","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-07T17:43:34Z","receivedAt":"2022-06-07T18:02:57Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nBitmap file has a trailing checksum at the end of the file. However\nthere is no information in the bitmap-format documentation about it.\n\nAdd a trailer section to include the trailing checksum info in the\n`Documentation/technical/bitmap-format.txt` file.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/technical/bitmap-format.txt | 4 ++++\n 1 file changed, 4 insertions(+)\n\ndiff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\nindex f22669b5916..a43d2fe2bbf 100644\n--- a/Documentation/technical/bitmap-format.txt\n+++ b/Documentation/technical/bitmap-format.txt\n@@ -125,6 +125,10 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \n \t\t** The compressed bitmap itself, see Appendix A.\n \n+\t* TRAILER:\n+\n+\t\tIndex checksum of the above contents. It is a 20-byte SHA1 checksum.\n+\n == Appendix A: Serialization format for an EWAH bitmap\n \n Ewah bitmaps are serialized in the same protocol as the JAVAEWAH\n-- \ngitgitgadget\n"},{"id":"456803","messageId":"cb919513c14d426b51051ee5c16badec37538032.1654623814.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v2.git.1654623814.gitgitgadget@gmail.com","subject":"[PATCH v2 2/3] bitmap-format.txt: fix some formatting issues","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-07T17:43:33Z","receivedAt":"2022-06-07T18:03:00Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nThe asciidoc generated html for `Documentation/technical/bitmap-\nformat.txt` is broken. This is mainly because `-` is used for nested\nlists (which is not allowed in asciidoc) instead of `*`.\n\nFix these and also reformat it (e.g. removing some blank lines) for\nbetter readability of the html page.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/technical/bitmap-format.txt | 20 +++++++-------------\n 1 file changed, 7 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\nindex 04b3ec21785..f22669b5916 100644\n--- a/Documentation/technical/bitmap-format.txt\n+++ b/Documentation/technical/bitmap-format.txt\n@@ -39,7 +39,7 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \n == On-disk format\n \n-\t- A header appears at the beginning:\n+\t* A header appears at the beginning:\n \n \t\t4-byte signature: {'B', 'I', 'T', 'M'}\n \n@@ -48,9 +48,7 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \t\t\tof the bitmap index (the same one as JGit).\n \n \t\t2-byte flags (network byte order)\n-\n \t\t\tThe following flags are supported:\n-\n \t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n \t\t\tThis flag must always be present. It implies that the\n \t\t\tbitmap index has been generated for a packfile or\n@@ -60,7 +58,6 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \t\t\trequirement for the bitmap index format, also present in\n \t\t\tJGit, that greatly reduces the complexity of the\n \t\t\timplementation.\n-\n \t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n \t\t\tIf present, the end of the bitmap file contains\n \t\t\t`N` 32-bit name-hash values, one per object in the\n@@ -68,15 +65,13 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \t\t\tdescribed below.\n \n \t\t4-byte entry count (network byte order)\n-\n \t\t\tThe total count of entries (bitmapped commits) in this bitmap index.\n \n \t\t20-byte checksum\n-\n \t\t\tThe SHA1 checksum of the pack/MIDX this bitmap index\n \t\t\tbelongs to.\n \n-\t- 4 EWAH bitmaps that act as type indexes\n+\t* 4 EWAH bitmaps that act as type indexes\n \n \t\tType indexes are serialized after the hash cache in the shape\n \t\tof four EWAH bitmaps stored consecutively (see Appendix A for\n@@ -84,7 +79,6 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \n \t\tThere is a bitmap for each Git object type, stored in the following\n \t\torder:\n-\n \t\t\t- Commits\n \t\t\t- Trees\n \t\t\t- Blobs\n@@ -97,17 +91,17 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \t\tin a full set (all bits set), and the AND of all 4 bitmaps will\n \t\tresult in an empty bitmap (no bits set).\n \n-\t- N entries with compressed bitmaps, one for each indexed commit\n+\t* N entries with compressed bitmaps, one for each indexed commit\n \n \t\tWhere `N` is the total amount of entries in this bitmap index.\n \t\tEach entry contains the following:\n \n-\t\t- 4-byte object position (network byte order)\n+\t\t** 4-byte object position (network byte order)\n \t\t\tThe position **in the index for the packfile or\n \t\t\tmulti-pack index** where the bitmap for this commit is\n \t\t\tfound.\n \n-\t\t- 1-byte XOR-offset\n+\t\t** 1-byte XOR-offset\n \t\t\tThe xor offset used to compress this bitmap. For an entry\n \t\t\tin position `x`, a XOR offset of `y` means that the actual\n \t\t\tbitmap representing this commit is composed by XORing the\n@@ -124,12 +118,12 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n \t\t\tin the index.\n \n-\t\t- 1-byte flags for this bitmap\n+\t\t** 1-byte flags for this bitmap\n \t\t\tAt the moment the only available flag is `0x1`, which hints\n \t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n \t\t\tfor the repository.\n \n-\t\t- The compressed bitmap itself, see Appendix A.\n+\t\t** The compressed bitmap itself, see Appendix A.\n \n == Appendix A: Serialization format for an EWAH bitmap\n \n-- \ngitgitgadget\n\n"},{"id":"456804","messageId":"a1b9bd9af90df88b7ce14de60a9626d2a1f2d3e8.1654623814.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v2.git.1654623814.gitgitgadget@gmail.com","subject":"[PATCH v2 1/3] bitmap-format.txt: feed the file to asciidoc to generate html","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-07T17:43:32Z","receivedAt":"2022-06-07T18:07:05Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nDocumentation/Makefile does not include bitmap-format.txt to generate\na html page using asciidoc.\n\nTeach Documentation/Makefile to also generate a html page for\nDocumentation/technical/bitmap-format.txt file.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/Makefile | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex d3f043f50d2..8d405a14330 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -94,6 +94,7 @@ TECH_DOCS += MyFirstContribution\n TECH_DOCS += MyFirstObjectWalk\n TECH_DOCS += SubmittingPatches\n TECH_DOCS += ToolsForGit\n+TECH_DOCS += technical/bitmap-format\n TECH_DOCS += technical/bundle-format\n TECH_DOCS += technical/hash-function-transition\n TECH_DOCS += technical/http-protocol\n-- \ngitgitgadget\n\n"},{"id":"456805","messageId":"pull.1246.v2.git.1654623814.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.git.1654177966.gitgitgadget@gmail.com","subject":"[PATCH v2 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-07T17:43:31Z","receivedAt":"2022-06-07T18:07:07Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"There are some issues in the bitmap-format html page. For example, some\nnested lists are shown as top-level lists (e.g. [1]- Here\nBITMAP_OPT_FULL_DAG (0x1) and BITMAP_OPT_HASH_CACHE (0x4) are shown as\ntop-level list). There is also a need of adding info about trailing checksum\nin the docs.\n\nChanges since v1:\n\n * a new commit addressing bitmap-format.txt html page generation is added\n * Remove extra indentation from the previous change\n * elaborate more about the trailing checksum (as suggested by Kaartic)\n\ninitial version:\n\n * first commit fixes some formatting issues\n * information about trailing checksum in the bitmap file is added in the\n   bitmap-format doc.\n\n[1] https://git-scm.com/docs/bitmap-format#_on_disk_format\n\nAbhradeep Chakraborty (3):\n  bitmap-format.txt: feed the file to asciidoc to generate html\n  bitmap-format.txt: fix some formatting issues\n  bitmap-format.txt: add information for trailing checksum\n\n Documentation/Makefile                    |  1 +\n Documentation/technical/bitmap-format.txt | 24 +++++++++++------------\n 2 files changed, 12 insertions(+), 13 deletions(-)\n\n\nbase-commit: 2668e3608e47494f2f10ef2b6e69f08a84816bcb\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1246%2FAbhra303%2Ffix-doc-formatting-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1246/Abhra303/fix-doc-formatting-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/1246\n\nRange-diff vs v1:\n\n -:  ----------- > 1:  a1b9bd9af90 bitmap-format.txt: feed the file to asciidoc to generate html\n 1:  976361e624a ! 2:  cb919513c14 bitmap-format.txt: fix some formatting issues\n     @@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cac\n      -\n       \t\t\tThe following flags are supported:\n      -\n     --\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n     --\t\t\tThis flag must always be present. It implies that the\n     --\t\t\tbitmap index has been generated for a packfile or\n     --\t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n     --\t\t\tevery single object in the packfile/MIDX can find its\n     --\t\t\tparent links inside the same packfile/MIDX). This is a\n     --\t\t\trequirement for the bitmap index format, also present in\n     --\t\t\tJGit, that greatly reduces the complexity of the\n     --\t\t\timplementation.\n     + \t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n     + \t\t\tThis flag must always be present. It implies that the\n     + \t\t\tbitmap index has been generated for a packfile or\n     +@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     + \t\t\trequirement for the bitmap index format, also present in\n     + \t\t\tJGit, that greatly reduces the complexity of the\n     + \t\t\timplementation.\n      -\n     --\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n     --\t\t\tIf present, the end of the bitmap file contains\n     --\t\t\t`N` 32-bit name-hash values, one per object in the\n     --\t\t\tpack/MIDX. The format and meaning of the name-hash is\n     --\t\t\tdescribed below.\n     -+\t\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n     -+\t\t\t\tThis flag must always be present. It implies that the\n     -+\t\t\t\tbitmap index has been generated for a packfile or\n     -+\t\t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n     -+\t\t\t\tevery single object in the packfile/MIDX can find its\n     -+\t\t\t\tparent links inside the same packfile/MIDX). This is a\n     -+\t\t\t\trequirement for the bitmap index format, also present in\n     -+\t\t\t\tJGit, that greatly reduces the complexity of the\n     -+\t\t\t\timplementation.\n     -+\t\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n     -+\t\t\t\tIf present, the end of the bitmap file contains\n     -+\t\t\t\t`N` 32-bit name-hash values, one per object in the\n     -+\t\t\t\tpack/MIDX. The format and meaning of the name-hash is\n     -+\t\t\t\tdescribed below.\n     + \t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n     + \t\t\tIf present, the end of the bitmap file contains\n     + \t\t\t`N` 32-bit name-hash values, one per object in the\n     +@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     + \t\t\tdescribed below.\n       \n       \t\t4-byte entry count (network byte order)\n      -\n     @@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cac\n       \t\tEach entry contains the following:\n       \n      -\t\t- 4-byte object position (network byte order)\n     --\t\t\tThe position **in the index for the packfile or\n     --\t\t\tmulti-pack index** where the bitmap for this commit is\n     --\t\t\tfound.\n     --\n     ++\t\t** 4-byte object position (network byte order)\n     + \t\t\tThe position **in the index for the packfile or\n     + \t\t\tmulti-pack index** where the bitmap for this commit is\n     + \t\t\tfound.\n     + \n      -\t\t- 1-byte XOR-offset\n     --\t\t\tThe xor offset used to compress this bitmap. For an entry\n     --\t\t\tin position `x`, a XOR offset of `y` means that the actual\n     --\t\t\tbitmap representing this commit is composed by XORing the\n     --\t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n     --\t\t\tthe bitmap `y` entries before this one).\n     --\n     --\t\t\tNote that this compression can be recursive. In order to\n     --\t\t\tXOR this entry with a previous one, the previous entry needs\n     --\t\t\tto be decompressed first, and so on.\n     --\n     --\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n     --\t\t\txor'ed against one of the 160 entries preceding it). This\n     --\t\t\tnumber is always positive, and hence entries are always xor'ed\n     --\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n     --\t\t\tin the index.\n     --\n     ++\t\t** 1-byte XOR-offset\n     + \t\t\tThe xor offset used to compress this bitmap. For an entry\n     + \t\t\tin position `x`, a XOR offset of `y` means that the actual\n     + \t\t\tbitmap representing this commit is composed by XORing the\n     +@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     + \t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n     + \t\t\tin the index.\n     + \n      -\t\t- 1-byte flags for this bitmap\n     --\t\t\tAt the moment the only available flag is `0x1`, which hints\n     --\t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n     --\t\t\tfor the repository.\n     --\n     ++\t\t** 1-byte flags for this bitmap\n     + \t\t\tAt the moment the only available flag is `0x1`, which hints\n     + \t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n     + \t\t\tfor the repository.\n     + \n      -\t\t- The compressed bitmap itself, see Appendix A.\n     -+\t\t\t** 4-byte object position (network byte order)\n     -+\t\t\t\tThe position **in the index for the packfile or\n     -+\t\t\t\tmulti-pack index** where the bitmap for this commit is\n     -+\t\t\t\tfound.\n     -+\n     -+\t\t\t** 1-byte XOR-offset\n     -+\t\t\t\tThe xor offset used to compress this bitmap. For an entry\n     -+\t\t\t\tin position `x`, a XOR offset of `y` means that the actual\n     -+\t\t\t\tbitmap representing this commit is composed by XORing the\n     -+\t\t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n     -+\t\t\t\tthe bitmap `y` entries before this one).\n     -+\n     -+\t\t\t\tNote that this compression can be recursive. In order to\n     -+\t\t\t\tXOR this entry with a previous one, the previous entry needs\n     -+\t\t\t\tto be decompressed first, and so on.\n     -+\n     -+\t\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n     -+\t\t\t\txor'ed against one of the 160 entries preceding it). This\n     -+\t\t\t\tnumber is always positive, and hence entries are always xor'ed\n     -+\t\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n     -+\t\t\t\tin the index.\n     -+\n     -+\t\t\t** 1-byte flags for this bitmap\n     -+\t\t\t\tAt the moment the only available flag is `0x1`, which hints\n     -+\t\t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n     -+\t\t\t\tfor the repository.\n     -+\n     -+\t\t\t** The compressed bitmap itself, see Appendix A.\n     ++\t\t** The compressed bitmap itself, see Appendix A.\n       \n       == Appendix A: Serialization format for an EWAH bitmap\n       \n 2:  ba534b5d486 ! 3:  2171d31fb2b bitmap-format.txt: add information for trailing checksum\n     @@ Commit message\n       ## Documentation/technical/bitmap-format.txt ##\n      @@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n       \n     - \t\t\t** The compressed bitmap itself, see Appendix A.\n     + \t\t** The compressed bitmap itself, see Appendix A.\n       \n      +\t* TRAILER:\n      +\n     -+\t\tIndex checksum of the above contents.\n     ++\t\tIndex checksum of the above contents. It is a 20-byte SHA1 checksum.\n      +\n       == Appendix A: Serialization format for an EWAH bitmap\n       \n\n-- \ngitgitgadget\n"},{"id":"456809","messageId":"xmqqfskgwcou.fsf@gitster.g","threadId":"57946","inReplyTo":"pull.1246.v2.git.1654623814.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-07T18:28:17Z","receivedAt":"2022-06-07T20:17:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Abhradeep Chakraborty via GitGitGadget\" \nwrites:\n\n> There are some issues in the bitmap-format html page.\n\n\"First, it does not even exist!\" before anything else ;-)\n\n> For example, some\n> nested lists are shown as top-level lists (e.g. [1]- Here\n> BITMAP_OPT_FULL_DAG (0x1) and BITMAP_OPT_HASH_CACHE (0x4) are shown as\n> top-level list). There is also a need of adding info about trailing checksum\n> in the docs.\n>\n> Changes since v1:\n>\n>  * a new commit addressing bitmap-format.txt html page generation is added\n\nGood.\n\n>  * Remove extra indentation from the previous change\n\nGood.\n\n>  * elaborate more about the trailing checksum (as suggested by Kaartic)\n\nGood.\n\nWill take a look (and audiences are requested to do so, too).\n\nThanks.\n"},{"id":"456810","messageId":"xmqqa6aowc6r.fsf@gitster.g","threadId":"57946","inReplyTo":"a1b9bd9af90df88b7ce14de60a9626d2a1f2d3e8.1654623814.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 1/3] bitmap-format.txt: feed the file to asciidoc to generate html","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-07T18:39:08Z","receivedAt":"2022-06-07T20:49:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Abhradeep Chakraborty via GitGitGadget\" \nwrites:\n\n> From: Abhradeep Chakraborty \n>\n> Documentation/Makefile does not include bitmap-format.txt to generate\n> a html page using asciidoc.\n>\n> Teach Documentation/Makefile to also generate a html page for\n> Documentation/technical/bitmap-format.txt file.\n>\n> Signed-off-by: Abhradeep Chakraborty \n> ---\n>  Documentation/Makefile | 1 +\n>  1 file changed, 1 insertion(+)\n\nThe change itself is obviously correct (assuming that it is worth\npassing the document to AsciiDoc, instead of reading it in text,\nthat is).\n\n> diff --git a/Documentation/Makefile b/Documentation/Makefile\n> index d3f043f50d2..8d405a14330 100644\n> --- a/Documentation/Makefile\n> +++ b/Documentation/Makefile\n> @@ -94,6 +94,7 @@ TECH_DOCS += MyFirstContribution\n>  TECH_DOCS += MyFirstObjectWalk\n>  TECH_DOCS += SubmittingPatches\n>  TECH_DOCS += ToolsForGit\n> +TECH_DOCS += technical/bitmap-format\n>  TECH_DOCS += technical/bundle-format\n>  TECH_DOCS += technical/hash-function-transition\n>  TECH_DOCS += technical/http-protocol\n\nIs bitmap-format the only one that is not fed to AsciiDoc, by the\nway?  Are there other 'text-only' document that is worth converting\nto AsciiDoc? \n\nIt is outside the scope of this series, of course, to actually\nadjusting them, but since you are already doing the homework, I\nthought you might already know the answer, which may become a source\nof inspriation for others to find something to work on.\n\nThanks.\n\n\n\n"},{"id":"456812","messageId":"Yp+7+QV4HDT3eY53@nand.local","threadId":"57946","inReplyTo":"xmqqfskgwcou.fsf@gitster.g","subject":"Re: [PATCH v2 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-06-07T20:58:33Z","receivedAt":"2022-06-08T00:26:54Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Jun 07, 2022 at 11:28:17AM -0700, Junio C Hamano wrote:\n> Will take a look (and audiences are requested to do so, too).\n\nI think this is on a good track. The rendered HTML still has much of its\ncontent inside of 
 elements, but that may be an acceptable\ntrade-off to maintain readability of the source material.\n\nIf there's a way to make the rendered page more appealing without\ncompromising on the readability of the source, I'd be in favor of that.\nBut I trust Abhradeep's judgement here, so if there isn't, I'd be happy\nwith the series (mostly) as-is.\n\nI left a textual suggestion on the third patch, which I'd like to adopt\nbefore picking this up (this will also give Abhradeep a chance to\ninvestigate the formatting improvements on patch 2/3).\n\nIn the meantime, it's probably safe to drop Vicent Martí from the CC\nlist, since he is no longer working on Git (though I miss him very\nmuch!).\n\nThanks,\nTaylor\n"},{"id":"456815","messageId":"xmqqo7z4ur2i.fsf@gitster.g","threadId":"57946","inReplyTo":"xmqqfskgwcou.fsf@gitster.g","subject":"Re: [PATCH v2 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-07T21:00:37Z","receivedAt":"2022-06-08T00:27:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano  writes:\n\n> \"Abhradeep Chakraborty via GitGitGadget\" \n> writes:\n>\n>> There are some issues in the bitmap-format html page.\n>\n> \"First, it does not even exist!\" before anything else ;-)\n>\n>> For example, some\n>> nested lists are shown as top-level lists (e.g. [1]- Here\n>> BITMAP_OPT_FULL_DAG (0x1) and BITMAP_OPT_HASH_CACHE (0x4) are shown as\n>> top-level list). There is also a need of adding info about trailing checksum\n>> in the docs.\n> ...\n\nNo, this is not quite ready for production.\n\nAlmost all the \"indented\" material are shown in fixed-width\ntypewriter format in the resulting HTML output.\n\nLook how ugly the output from it is.  Not your fault; it is mostly\nbecause when the original text was written, it was not even meant to\nbe given to AsciiDoc.\n\n  https://twitter.com/jch2355/status/1534276427607986178/photo/1\n  https://pbs.twimg.com/media/FUrYP2nakAAnRaH?format=png\n\nAnd as I already said, removal of the blank lines made it harder to\nsee what is going on in the source, and because the output is pretty\nmuch straight copy of the source in the fixed-font, just like reading\nthe source in the terminal, the output here is equally hard to read.\n\n  https://twitter.com/jch2355/status/1534277664441511937/photo/1\n  https://pbs.twimg.com/media/FUrZZXUUsAEmEeT?format=png\n\nIf we really want to give it to AsciiDoc, we'd need to reformat it\nmore extensively, not just tweak it on the surface and making an\nequivalent of 
...
 slightly easier to read, which is what\nthis patch does.\n\nThanks.\n"},{"id":"456821","messageId":"xmqq1qw0uo7l.fsf@gitster.g","threadId":"57946","inReplyTo":"Yp+6WU+k2OwHDB1b@nand.local","subject":"Re: [PATCH v2 2/3] bitmap-format.txt: fix some formatting issues","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-07T22:02:22Z","receivedAt":"2022-06-08T00:28:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau  writes:\n\n> Similarly, everything below the \"A header appears at the beginning\"\n> list item appears in a 
 element, so the rendered HTML looks more\n> like plaintext to me.\n\nTrue.  Unless we are going to revamp the text in some major way so\nthat we produce \"true\" HTML, not just the text source enclosed in a\n
 pair, I would think we are better off keeping it not\npassed to AsciiDoc and leaving it in text format.  After all, modern\nbrowsers, which I presume those who want HTML output files would\nread them with, can display plain text files just fine, don't they?\n\n> This isn't new from your patch, but I wonder if now is a good\n> opportunity to make some light use of the formatting options that\n> ASCIIDoc gives us to make the page read a little bit more easily when\n> rendered as HTML.\n\nThere was some talk about asking those who are adept at website\nengineering to work on git-scm.com; it may be a good starting point\nto look at these text files that weren't originally written to be\ngiven to AsciiDoc and convert them to be true AsciiDoc sources.\n\nThanks.\n"},{"id":"456822","messageId":"Yp+zQvbo/e7XDsDf@nand.local","threadId":"57946","inReplyTo":"a1b9bd9af90df88b7ce14de60a9626d2a1f2d3e8.1654623814.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 1/3] bitmap-format.txt: feed the file to asciidoc to generate html","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-06-07T20:21:22Z","receivedAt":"2022-06-08T00:33:06Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Jun 07, 2022 at 05:43:32PM +0000, Abhradeep Chakraborty via GitGitGadget wrote:\n> From: Abhradeep Chakraborty \n>\n> Documentation/Makefile does not include bitmap-format.txt to generate\n> a html page using asciidoc.\n>\n> Teach Documentation/Makefile to also generate a html page for\n> Documentation/technical/bitmap-format.txt file.\n\nI am glad to see us finally getting around to this ;). I proposed this\nback in:\n\n    https://lore.kernel.org/git/b0bb2e8051f19ec47140fda6500e092e37c6bea8.1624314293.git.me@ttaylorr.com/\n\nbut I dropped it from later versions of that series, due in large part\nto some of the formatting issues that your series here fixes.\n\nThanks,\nTaylor\n"},{"id":"456840","messageId":"Yp+7aXdaCX3Fh9SE@nand.local","threadId":"57946","inReplyTo":"2171d31fb2b783371bdc31ba54856dea8224de65.1654623814.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 3/3] bitmap-format.txt: add information for trailing checksum","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-06-07T20:56:09Z","receivedAt":"2022-06-08T01:21:13Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Jun 07, 2022 at 05:43:34PM +0000, Abhradeep Chakraborty via GitGitGadget wrote:\n> From: Abhradeep Chakraborty \n>\n> Bitmap file has a trailing checksum at the end of the file. However\n> there is no information in the bitmap-format documentation about it.\n>\n> Add a trailer section to include the trailing checksum info in the\n> `Documentation/technical/bitmap-format.txt` file.\n>\n> Signed-off-by: Abhradeep Chakraborty \n> ---\n>  Documentation/technical/bitmap-format.txt | 4 ++++\n>  1 file changed, 4 insertions(+)\n>\n> diff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\n> index f22669b5916..a43d2fe2bbf 100644\n> --- a/Documentation/technical/bitmap-format.txt\n> +++ b/Documentation/technical/bitmap-format.txt\n> @@ -125,6 +125,10 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n>\n>  \t\t** The compressed bitmap itself, see Appendix A.\n>\n> +\t* TRAILER:\n> +\n> +\t\tIndex checksum of the above contents. It is a 20-byte SHA1 checksum.\n> +\n\nI assume by \"Index checksum\" you are referring to a checksum of the\nbitmap _index_'s contents. That term is used a little throughout\npack-format.txt, but it's foreign to me. Assuming that's how you meant\nit, a more conventional term (I think) would be just \"trailing\nchecksum\".\n\nIt is also not guaranteed to be a SHA-1 checksum, if the repository\nwhich wrote the bitmap is in SHA-256 mode. So I would suggest that this\naddition just read:\n\n    * TRAILER:\n\n      Trailing checksum of the preceding contents.\n\nThanks,\nTaylor\n"},{"id":"456851","messageId":"Yp+6WU+k2OwHDB1b@nand.local","threadId":"57946","inReplyTo":"cb919513c14d426b51051ee5c16badec37538032.1654623814.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 2/3] bitmap-format.txt: fix some formatting issues","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-06-07T20:51:37Z","receivedAt":"2022-06-08T03:03:23Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Jun 07, 2022 at 05:43:33PM +0000, Abhradeep Chakraborty via GitGitGadget wrote:\n> From: Abhradeep Chakraborty \n>\n> The asciidoc generated html for `Documentation/technical/bitmap-\n> format.txt` is broken. This is mainly because `-` is used for nested\n> lists (which is not allowed in asciidoc) instead of `*`.\n>\n> Fix these and also reformat it (e.g. removing some blank lines) for\n> better readability of the html page.\n\nHmm. When I render the HTML for this page and view it in my browser, the\nremoved blank lines makes the contents of the section \"2-byte flags\n(network byte order)\" run together, and I think it hurts readability\nIMHO.\n\nIs there a way to keep those line breaks without significantly\nreformatting the source of this file?\n\n> Signed-off-by: Abhradeep Chakraborty \n> ---\n>  Documentation/technical/bitmap-format.txt | 20 +++++++-------------\n>  1 file changed, 7 insertions(+), 13 deletions(-)\n>\n> diff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\n> index 04b3ec21785..f22669b5916 100644\n> --- a/Documentation/technical/bitmap-format.txt\n> +++ b/Documentation/technical/bitmap-format.txt\n> @@ -39,7 +39,7 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n>\n>  == On-disk format\n>\n> -\t- A header appears at the beginning:\n> +\t* A header appears at the beginning:\n>\n>  \t\t4-byte signature: {'B', 'I', 'T', 'M'}\n\nSimilarly, everything below the \"A header appears at the beginning\"\nlist item appears in a 
 element, so the rendered HTML looks more\nlike plaintext to me.\n\nThis isn't new from your patch, but I wonder if now is a good\nopportunity to make some light use of the formatting options that\nASCIIDoc gives us to make the page read a little bit more easily when\nrendered as HTML.\n\nI don't want to compromise too much on the readability of the .txt file,\nthough, so if there isn't a good way to strike this balance, then I\ntrust you and think we should leave it as you have modified things here.\n\nThanks,\nTaylor\n"},{"id":"456865","messageId":"20220608150211.9831-1-chakrabortyabhradeep79@gmail.com","threadId":"57946","inReplyTo":"xmqqa6aowc6r.fsf@gitster.g","subject":"Re: [PATCH v2 1/3] bitmap-format.txt: feed the file to asciidoc to generate html","fromName":"Abhradeep Chakraborty","fromEmail":"chakrabortyabhradeep79@gmail.com","sentAt":"2022-06-08T15:02:11Z","receivedAt":"2022-06-08T15:11:16Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"Junio C Hamano  wrote:\n\n> Is bitmap-format the only one that is not fed to AsciiDoc, by the\n> way?  Are there other 'text-only' document that is worth converting\n> to AsciiDoc? \n>\n> It is outside the scope of this series, of course, to actually\n> adjusting them, but since you are already doing the homework, I\n> thought you might already know the answer, which may become a source\n> of inspriation for others to find something to work on.\n\nNo, bitmap-format is not the only one. There are more text-only files.\nSome of them which I found till now are - technical/chunk-format.txt,\ntechnical/commit-graph.txt etc. There are more but I don't know if they\nactually need html conversion. These two texts (which I mentioned) is I\nthink worth having html files.\n\nI was thinking of adding those in my commit but later I thought it would\ndivert the patch series.\n\nThanks :)\n"},{"id":"456868","messageId":"20220608154050.10278-1-chakrabortyabhradeep79@gmail.com","threadId":"57946","inReplyTo":"Yp+6WU+k2OwHDB1b@nand.local","subject":"Re: [PATCH v2 2/3] bitmap-format.txt: fix some formatting issues","fromName":"Abhradeep Chakraborty","fromEmail":"chakrabortyabhradeep79@gmail.com","sentAt":"2022-06-08T15:40:50Z","receivedAt":"2022-06-08T15:41:06Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"Taylor Blau  wrote:\n\n> Hmm. When I render the HTML for this page and view it in my browser, the\n> removed blank lines makes the contents of the section \"2-byte flags\n> (network byte order)\" run together, and I think it hurts readability\n> IMHO.\n\nHonestly I agree with you. I also felt the same but then I thought it\nis still better than the currently broken page.\n\n> Is there a way to keep those line breaks without significantly\n> reformatting the source of this file?\n\nI have a limited knowledge on asciidoc. I removed those blank lines\nonly because it generates weird html output. I didn't find any other\nway to fix that (with minimum source code changes).\n\n> This isn't new from your patch, but I wonder if now is a good\n> opportunity to make some light use of the formatting options that\n> ASCIIDoc gives us to make the page read a little bit more easily when\n> rendered as HTML.\n\nYeah, quite sensible. I will surely look for better way.\n\n> I don't want to compromise too much on the readability of the .txt file,\n> though, so if there isn't a good way to strike this balance, then I\n> trust you and think we should leave it as you have modified things here.\n\nThis is one of the main reason why I removed those blank lines and other\nstuff. It is the minimum change to fix the html doc. But I will look more\ninto it.\n\nThanks :)\n"},{"id":"456870","messageId":"20220608160626.10332-1-chakrabortyabhradeep79@gmail.com","threadId":"57946","inReplyTo":"xmqq1qw0uo7l.fsf@gitster.g","subject":"Re: [PATCH v2 2/3] bitmap-format.txt: fix some formatting issues","fromName":"Abhradeep Chakraborty","fromEmail":"chakrabortyabhradeep79@gmail.com","sentAt":"2022-06-08T16:06:26Z","receivedAt":"2022-06-08T16:07:01Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"Junio C Hamano  wrote:\n\n> True.  Unless we are going to revamp the text in some major way so\n> that we produce \"true\" HTML, not just the text source enclosed in a\n> 
 pair, I would think we are better off keeping it not\n> passed to AsciiDoc and leaving it in text format.  After all, modern\n> browsers, which I presume those who want HTML output files would\n> read them with, can display plain text files just fine, don't they?\n\nI am not sure whether that's a good idea or not. As I come from web\ndev background, I know that people get bored if they need to read\na plain-text long article. SEO optimisation also need some beautiful\ndesigning of articles so that people can spend more time with visual\nease.\n\nOf course, git doesn't need any SEO optimisation as it is very much\npopular. But readers want some visual satisfaction while reading\nDocs. That's why some people complain about GNU sites (git's site is\nbeautiful by the way).\n\nObviously, here I am using `people` to refer non git developers who are\ncurious about git internals.\n\nThanks :)\n"},{"id":"456873","messageId":"20220608161523.10359-1-chakrabortyabhradeep79@gmail.com","threadId":"57946","inReplyTo":"Yp+7aXdaCX3Fh9SE@nand.local","subject":"Re: [PATCH v2 3/3] bitmap-format.txt: add information for trailing checksum","fromName":"Abhradeep Chakraborty","fromEmail":"chakrabortyabhradeep79@gmail.com","sentAt":"2022-06-08T16:15:23Z","receivedAt":"2022-06-08T16:15:46Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"Taylor Blau  wrote:\n\n> I assume by \"Index checksum\" you are referring to a checksum of the\n> bitmap _index_'s contents. \n\nYeah, I meant a checksum of the bitmap file's content.\n\n> That term is used a little throughout\n> pack-format.txt, but it's foreign to me. Assuming that's how you meant\n> it, a more conventional term (I think) would be just \"trailing\n> checksum\".\n\nActually, I copy-paste it from the pack-format.txt file ;). Will surely\nfollow your suggestions.\n\n> It is also not guaranteed to be a SHA-1 checksum, if the repository\n> which wrote the bitmap is in SHA-256 mode. So I would suggest that this\n> addition just read:\n>\n>     * TRAILER:\n>\n>       Trailing checksum of the preceding contents.\n\nGot it. Thanks !\n\n"},{"id":"456875","messageId":"20220608171257.10455-1-chakrabortyabhradeep79@gmail.com","threadId":"57946","inReplyTo":"xmqqo7z4ur2i.fsf@gitster.g","subject":"Re: [PATCH v2 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Abhradeep Chakraborty","fromEmail":"chakrabortyabhradeep79@gmail.com","sentAt":"2022-06-08T17:12:57Z","receivedAt":"2022-06-08T17:22:34Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"Junio C Hamano  wrote:\n\n> No, this is not quite ready for production.\n>\n> Almost all the \"indented\" material are shown in fixed-width\n> typewriter format in the resulting HTML output.\n>\n> Look how ugly the output from it is.  Not your fault; it is mostly\n> because when the original text was written, it was not even meant to\n> be given to AsciiDoc.\n\nActually, I am wondering how git-scm.com is able to produce a html page\nfor bitmap-format.txt (if it is not passing to asciidoc). The design of\nasciidoc generated html pages in `make docs` are not same as the design\nof production html page designs. Probably, production uses some extra\ncss code to beautify the asciidoc generated html files.\n\nSo, the generated html file (production version) is not as bad as the\nlocally built generated html. I need some understanding of the working\nof git-scm though (to verify it).\n\nIf you see other locally built html pages - they would look similar to\nthe bitmap-format html page. But in production, they are beautiful enough.\n\nBy the way, I forgot to inform that https://git-scm.com/docs/pack-format#_original_version_1_pack_idx_files_have_the_following_format also has\nsome weird formatting issues. See the 
 block after the pack-idx structure\ndrawing. There are other issues also which you can find (like having\nunnecessary indentations e.g. here[1] the second block under the \"The header\nis followed by number of object entries....\").\n\nThanks :)\n\n[1] https://git-scm.com/docs/pack-format#_pack_pack_files_have_the_following_format\n"},{"id":"457012","messageId":"pull.1246.v3.git.1654858481.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v2.git.1654623814.gitgitgadget@gmail.com","subject":"[PATCH v3 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-10T10:54:38Z","receivedAt":"2022-06-10T10:57:25Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"There are some issues in the bitmap-format html page. For example, some\nnested lists are shown as top-level lists (e.g. [1]- Here\nBITMAP_OPT_FULL_DAG (0x1) and BITMAP_OPT_HASH_CACHE (0x4) are shown as\ntop-level list). There is also a need of adding info about trailing checksum\nin the docs.\n\nChanges since v2: The last two commits are updated to address the\nsuggestions. These changes are -\n\n * previously omitted blank lines are re-added. In the updated commit, use\n   of 
 blocks are decreased. Description lists and + are used instead\n   to add more than one paragraphs under lists. Readability of the source\n   text might decrease due to the use of +. But other documentation files\n   (e.g. git-add.txt) also use it to connect two paragraphs. So, I hope this\n   is acceptable.\n\n * Information about trailing checksum is updated (as suggested by Taylor)\n\nChanges since v1:\n\n * a new commit addressing bitmap-format.txt html page generation is added\n * Remove extra indentation from the previous change\n * elaborate more about the trailing checksum (as suggested by Kaartic)\n\ninitial version:\n\n * first commit fixes some formatting issues\n * information about trailing checksum in the bitmap file is added in the\n   bitmap-format doc.\n\n[1] https://git-scm.com/docs/bitmap-format#_on_disk_format\n\nAbhradeep Chakraborty (3):\n  bitmap-format.txt: feed the file to asciidoc to generate html\n  bitmap-format.txt: fix some formatting issues\n  bitmap-format.txt: add information for trailing checksum\n\n Documentation/Makefile                    |   1 +\n Documentation/technical/bitmap-format.txt | 113 ++++++++++++----------\n 2 files changed, 63 insertions(+), 51 deletions(-)\n\n\nbase-commit: 2668e3608e47494f2f10ef2b6e69f08a84816bcb\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1246%2FAbhra303%2Ffix-doc-formatting-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1246/Abhra303/fix-doc-formatting-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/1246\n\nRange-diff vs v2:\n\n 1:  a1b9bd9af90 = 1:  a1b9bd9af90 bitmap-format.txt: feed the file to asciidoc to generate html\n 2:  cb919513c14 ! 2:  c74b9a52c2a bitmap-format.txt: fix some formatting issues\n     @@ Commit message\n          format.txt` is broken. This is mainly because `-` is used for nested\n          lists (which is not allowed in asciidoc) instead of `*`.\n      \n     -    Fix these and also reformat it (e.g. removing some blank lines) for\n     -    better readability of the html page.\n     +    Fix these and also reformat it for better readability of the html page.\n      \n          Signed-off-by: Abhradeep Chakraborty \n      \n     @@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cac\n      -\t- A header appears at the beginning:\n      +\t* A header appears at the beginning:\n       \n     - \t\t4-byte signature: {'B', 'I', 'T', 'M'}\n     +-\t\t4-byte signature: {'B', 'I', 'T', 'M'}\n     ++\t\t4-byte signature: :: {'B', 'I', 'T', 'M'}\n     ++\n     ++\t\t2-byte version number (network byte order): ::\n       \n     -@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     +-\t\t2-byte version number (network byte order)\n     + \t\t\tThe current implementation only supports version 1\n       \t\t\tof the bitmap index (the same one as JGit).\n       \n     - \t\t2-byte flags (network byte order)\n     --\n     +-\t\t2-byte flags (network byte order)\n     ++\t\t2-byte flags (network byte order): ::\n     + \n       \t\t\tThe following flags are supported:\n     --\n     - \t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n     + \n     +-\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n     ++\t\t\t** {empty}\n     ++\t\t\tBITMAP_OPT_FULL_DAG (0x1) REQUIRED: :::\n     ++\n       \t\t\tThis flag must always be present. It implies that the\n       \t\t\tbitmap index has been generated for a packfile or\n     + \t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n      @@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     - \t\t\trequirement for the bitmap index format, also present in\n       \t\t\tJGit, that greatly reduces the complexity of the\n       \t\t\timplementation.\n     --\n     - \t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n     + \n     +-\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n     ++\t\t\t** {empty}\n     ++\t\t\tBITMAP_OPT_HASH_CACHE (0x4): :::\n     ++\n       \t\t\tIf present, the end of the bitmap file contains\n       \t\t\t`N` 32-bit name-hash values, one per object in the\n     -@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     + \t\t\tpack/MIDX. The format and meaning of the name-hash is\n       \t\t\tdescribed below.\n       \n     - \t\t4-byte entry count (network byte order)\n     +-\t\t4-byte entry count (network byte order)\n      -\n     ++\t\t4-byte entry count (network byte order): ::\n       \t\t\tThe total count of entries (bitmapped commits) in this bitmap index.\n       \n     - \t\t20-byte checksum\n     +-\t\t20-byte checksum\n      -\n     ++\t\t20-byte checksum: ::\n       \t\t\tThe SHA1 checksum of the pack/MIDX this bitmap index\n       \t\t\tbelongs to.\n       \n      -\t- 4 EWAH bitmaps that act as type indexes\n     -+\t* 4 EWAH bitmaps that act as type indexes\n     - \n     - \t\tType indexes are serialized after the hash cache in the shape\n     - \t\tof four EWAH bitmaps stored consecutively (see Appendix A for\n     -@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     - \n     - \t\tThere is a bitmap for each Git object type, stored in the following\n     - \t\torder:\n      -\n     - \t\t\t- Commits\n     - \t\t\t- Trees\n     - \t\t\t- Blobs\n     -@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     - \t\tin a full set (all bits set), and the AND of all 4 bitmaps will\n     - \t\tresult in an empty bitmap (no bits set).\n     - \n     +-\t\tType indexes are serialized after the hash cache in the shape\n     +-\t\tof four EWAH bitmaps stored consecutively (see Appendix A for\n     +-\t\tthe serialization format of an EWAH bitmap).\n     +-\n     +-\t\tThere is a bitmap for each Git object type, stored in the following\n     +-\t\torder:\n     +-\n     +-\t\t\t- Commits\n     +-\t\t\t- Trees\n     +-\t\t\t- Blobs\n     +-\t\t\t- Tags\n     +-\n     +-\t\tIn each bitmap, the `n`th bit is set to true if the `n`th object\n     +-\t\tin the packfile or multi-pack index is of that type.\n     +-\n     +-\t\tThe obvious consequence is that the OR of all 4 bitmaps will result\n     +-\t\tin a full set (all bits set), and the AND of all 4 bitmaps will\n     +-\t\tresult in an empty bitmap (no bits set).\n     +-\n      -\t- N entries with compressed bitmaps, one for each indexed commit\n     -+\t* N entries with compressed bitmaps, one for each indexed commit\n     - \n     - \t\tWhere `N` is the total amount of entries in this bitmap index.\n     - \t\tEach entry contains the following:\n     - \n     +-\n     +-\t\tWhere `N` is the total amount of entries in this bitmap index.\n     +-\t\tEach entry contains the following:\n     +-\n      -\t\t- 4-byte object position (network byte order)\n     -+\t\t** 4-byte object position (network byte order)\n     ++\t* 4 EWAH bitmaps that act as type indexes\n     +++\n     ++Type indexes are serialized after the hash cache in the shape\n     ++of four EWAH bitmaps stored consecutively (see Appendix A for\n     ++the serialization format of an EWAH bitmap).\n     +++\n     ++There is a bitmap for each Git object type, stored in the following\n     ++order:\n     +++\n     ++\t- Commits\n     ++\t- Trees\n     ++\t- Blobs\n     ++\t- Tags\n     ++\n     +++\n     ++In each bitmap, the `n`th bit is set to true if the `n`th object\n     ++in the packfile or multi-pack index is of that type.\n     ++\n     ++    The obvious consequence is that the OR of all 4 bitmaps will result\n     ++    in a full set (all bits set), and the AND of all 4 bitmaps will\n     ++    result in an empty bitmap (no bits set).\n     ++\n     ++\t* N entries with compressed bitmaps, one for each indexed commit\n     +++\n     ++Where `N` is the total amount of entries in this bitmap index.\n     ++Each entry contains the following:\n     ++\n     ++\t\t** {empty}\n     ++\t\t4-byte object position (network byte order): ::\n       \t\t\tThe position **in the index for the packfile or\n       \t\t\tmulti-pack index** where the bitmap for this commit is\n       \t\t\tfound.\n       \n      -\t\t- 1-byte XOR-offset\n     -+\t\t** 1-byte XOR-offset\n     ++\t\t** {empty}\n     ++\t\t1-byte XOR-offset: ::\n       \t\t\tThe xor offset used to compress this bitmap. For an entry\n       \t\t\tin position `x`, a XOR offset of `y` means that the actual\n       \t\t\tbitmap representing this commit is composed by XORing the\n     -@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     - \t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n     - \t\t\tin the index.\n     - \n     + \t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n     + \t\t\tthe bitmap `y` entries before this one).\n     +-\n     +-\t\t\tNote that this compression can be recursive. In order to\n     +-\t\t\tXOR this entry with a previous one, the previous entry needs\n     +-\t\t\tto be decompressed first, and so on.\n     +-\n     +-\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n     +-\t\t\txor'ed against one of the 160 entries preceding it). This\n     +-\t\t\tnumber is always positive, and hence entries are always xor'ed\n     +-\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n     +-\t\t\tin the index.\n     +-\n      -\t\t- 1-byte flags for this bitmap\n     -+\t\t** 1-byte flags for this bitmap\n     +++\n     ++NOTE: This compression can be recursive. In order to\n     ++XOR this entry with a previous one, the previous entry needs\n     ++to be decompressed first, and so on.\n     +++\n     ++The hard-limit for this offset is 160 (an entry can only be\n     ++xor'ed against one of the 160 entries preceding it). This\n     ++number is always positive, and hence entries are always xor'ed\n     ++with **previous** bitmaps, not bitmaps that will come afterwards\n     ++in the index.\n     ++\n     ++\t\t** {empty}\n     ++\t\t1-byte flags for this bitmap: ::\n       \t\t\tAt the moment the only available flag is `0x1`, which hints\n       \t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n       \t\t\tfor the repository.\n 3:  2171d31fb2b ! 3:  b971558e1cb bitmap-format.txt: add information for trailing checksum\n     @@ Commit message\n          Signed-off-by: Abhradeep Chakraborty \n      \n       ## Documentation/technical/bitmap-format.txt ##\n     -@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     +@@ Documentation/technical/bitmap-format.txt: in the index.\n       \n       \t\t** The compressed bitmap itself, see Appendix A.\n       \n     -+\t* TRAILER:\n     -+\n     -+\t\tIndex checksum of the above contents. It is a 20-byte SHA1 checksum.\n     ++\t* {empty}\n     ++\tTRAILER: ::\n     ++\t\tTrailing checksum of the preceding contents.\n      +\n       == Appendix A: Serialization format for an EWAH bitmap\n       \n\n-- \ngitgitgadget\n"},{"id":"457013","messageId":"a1b9bd9af90df88b7ce14de60a9626d2a1f2d3e8.1654858481.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v3.git.1654858481.gitgitgadget@gmail.com","subject":"[PATCH v3 1/3] bitmap-format.txt: feed the file to asciidoc to generate html","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-10T10:54:39Z","receivedAt":"2022-06-10T10:57:28Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nDocumentation/Makefile does not include bitmap-format.txt to generate\na html page using asciidoc.\n\nTeach Documentation/Makefile to also generate a html page for\nDocumentation/technical/bitmap-format.txt file.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/Makefile | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex d3f043f50d2..8d405a14330 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -94,6 +94,7 @@ TECH_DOCS += MyFirstContribution\n TECH_DOCS += MyFirstObjectWalk\n TECH_DOCS += SubmittingPatches\n TECH_DOCS += ToolsForGit\n+TECH_DOCS += technical/bitmap-format\n TECH_DOCS += technical/bundle-format\n TECH_DOCS += technical/hash-function-transition\n TECH_DOCS += technical/http-protocol\n-- \ngitgitgadget\n\n"},{"id":"457014","messageId":"c74b9a52c2a7b5f3ebbfaca08c8de42aac7f7eac.1654858481.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v3.git.1654858481.gitgitgadget@gmail.com","subject":"[PATCH v3 2/3] bitmap-format.txt: fix some formatting issues","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-10T10:54:40Z","receivedAt":"2022-06-10T10:57:34Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nThe asciidoc generated html for `Documentation/technical/bitmap-\nformat.txt` is broken. This is mainly because `-` is used for nested\nlists (which is not allowed in asciidoc) instead of `*`.\n\nFix these and also reformat it for better readability of the html page.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/technical/bitmap-format.txt | 109 ++++++++++++----------\n 1 file changed, 58 insertions(+), 51 deletions(-)\n\ndiff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\nindex 04b3ec21785..cd621379f42 100644\n--- a/Documentation/technical/bitmap-format.txt\n+++ b/Documentation/technical/bitmap-format.txt\n@@ -39,19 +39,22 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \n == On-disk format\n \n-\t- A header appears at the beginning:\n+\t* A header appears at the beginning:\n \n-\t\t4-byte signature: {'B', 'I', 'T', 'M'}\n+\t\t4-byte signature: :: {'B', 'I', 'T', 'M'}\n+\n+\t\t2-byte version number (network byte order): ::\n \n-\t\t2-byte version number (network byte order)\n \t\t\tThe current implementation only supports version 1\n \t\t\tof the bitmap index (the same one as JGit).\n \n-\t\t2-byte flags (network byte order)\n+\t\t2-byte flags (network byte order): ::\n \n \t\t\tThe following flags are supported:\n \n-\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n+\t\t\t** {empty}\n+\t\t\tBITMAP_OPT_FULL_DAG (0x1) REQUIRED: :::\n+\n \t\t\tThis flag must always be present. It implies that the\n \t\t\tbitmap index has been generated for a packfile or\n \t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n@@ -61,75 +64,79 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \t\t\tJGit, that greatly reduces the complexity of the\n \t\t\timplementation.\n \n-\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n+\t\t\t** {empty}\n+\t\t\tBITMAP_OPT_HASH_CACHE (0x4): :::\n+\n \t\t\tIf present, the end of the bitmap file contains\n \t\t\t`N` 32-bit name-hash values, one per object in the\n \t\t\tpack/MIDX. The format and meaning of the name-hash is\n \t\t\tdescribed below.\n \n-\t\t4-byte entry count (network byte order)\n-\n+\t\t4-byte entry count (network byte order): ::\n \t\t\tThe total count of entries (bitmapped commits) in this bitmap index.\n \n-\t\t20-byte checksum\n-\n+\t\t20-byte checksum: ::\n \t\t\tThe SHA1 checksum of the pack/MIDX this bitmap index\n \t\t\tbelongs to.\n \n-\t- 4 EWAH bitmaps that act as type indexes\n-\n-\t\tType indexes are serialized after the hash cache in the shape\n-\t\tof four EWAH bitmaps stored consecutively (see Appendix A for\n-\t\tthe serialization format of an EWAH bitmap).\n-\n-\t\tThere is a bitmap for each Git object type, stored in the following\n-\t\torder:\n-\n-\t\t\t- Commits\n-\t\t\t- Trees\n-\t\t\t- Blobs\n-\t\t\t- Tags\n-\n-\t\tIn each bitmap, the `n`th bit is set to true if the `n`th object\n-\t\tin the packfile or multi-pack index is of that type.\n-\n-\t\tThe obvious consequence is that the OR of all 4 bitmaps will result\n-\t\tin a full set (all bits set), and the AND of all 4 bitmaps will\n-\t\tresult in an empty bitmap (no bits set).\n-\n-\t- N entries with compressed bitmaps, one for each indexed commit\n-\n-\t\tWhere `N` is the total amount of entries in this bitmap index.\n-\t\tEach entry contains the following:\n-\n-\t\t- 4-byte object position (network byte order)\n+\t* 4 EWAH bitmaps that act as type indexes\n++\n+Type indexes are serialized after the hash cache in the shape\n+of four EWAH bitmaps stored consecutively (see Appendix A for\n+the serialization format of an EWAH bitmap).\n++\n+There is a bitmap for each Git object type, stored in the following\n+order:\n++\n+\t- Commits\n+\t- Trees\n+\t- Blobs\n+\t- Tags\n+\n++\n+In each bitmap, the `n`th bit is set to true if the `n`th object\n+in the packfile or multi-pack index is of that type.\n+\n+    The obvious consequence is that the OR of all 4 bitmaps will result\n+    in a full set (all bits set), and the AND of all 4 bitmaps will\n+    result in an empty bitmap (no bits set).\n+\n+\t* N entries with compressed bitmaps, one for each indexed commit\n++\n+Where `N` is the total amount of entries in this bitmap index.\n+Each entry contains the following:\n+\n+\t\t** {empty}\n+\t\t4-byte object position (network byte order): ::\n \t\t\tThe position **in the index for the packfile or\n \t\t\tmulti-pack index** where the bitmap for this commit is\n \t\t\tfound.\n \n-\t\t- 1-byte XOR-offset\n+\t\t** {empty}\n+\t\t1-byte XOR-offset: ::\n \t\t\tThe xor offset used to compress this bitmap. For an entry\n \t\t\tin position `x`, a XOR offset of `y` means that the actual\n \t\t\tbitmap representing this commit is composed by XORing the\n \t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n \t\t\tthe bitmap `y` entries before this one).\n-\n-\t\t\tNote that this compression can be recursive. In order to\n-\t\t\tXOR this entry with a previous one, the previous entry needs\n-\t\t\tto be decompressed first, and so on.\n-\n-\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n-\t\t\txor'ed against one of the 160 entries preceding it). This\n-\t\t\tnumber is always positive, and hence entries are always xor'ed\n-\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n-\t\t\tin the index.\n-\n-\t\t- 1-byte flags for this bitmap\n++\n+NOTE: This compression can be recursive. In order to\n+XOR this entry with a previous one, the previous entry needs\n+to be decompressed first, and so on.\n++\n+The hard-limit for this offset is 160 (an entry can only be\n+xor'ed against one of the 160 entries preceding it). This\n+number is always positive, and hence entries are always xor'ed\n+with **previous** bitmaps, not bitmaps that will come afterwards\n+in the index.\n+\n+\t\t** {empty}\n+\t\t1-byte flags for this bitmap: ::\n \t\t\tAt the moment the only available flag is `0x1`, which hints\n \t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n \t\t\tfor the repository.\n \n-\t\t- The compressed bitmap itself, see Appendix A.\n+\t\t** The compressed bitmap itself, see Appendix A.\n \n == Appendix A: Serialization format for an EWAH bitmap\n \n-- \ngitgitgadget\n\n"},{"id":"457015","messageId":"b971558e1cba0a40b5adf20f53dfd3822dc7a42c.1654858481.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v3.git.1654858481.gitgitgadget@gmail.com","subject":"[PATCH v3 3/3] bitmap-format.txt: add information for trailing checksum","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-10T10:54:41Z","receivedAt":"2022-06-10T10:57:38Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nBitmap file has a trailing checksum at the end of the file. However\nthere is no information in the bitmap-format documentation about it.\n\nAdd a trailer section to include the trailing checksum info in the\n`Documentation/technical/bitmap-format.txt` file.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/technical/bitmap-format.txt | 4 ++++\n 1 file changed, 4 insertions(+)\n\ndiff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\nindex cd621379f42..3f8cdd0ed91 100644\n--- a/Documentation/technical/bitmap-format.txt\n+++ b/Documentation/technical/bitmap-format.txt\n@@ -138,6 +138,10 @@ in the index.\n \n \t\t** The compressed bitmap itself, see Appendix A.\n \n+\t* {empty}\n+\tTRAILER: ::\n+\t\tTrailing checksum of the preceding contents.\n+\n == Appendix A: Serialization format for an EWAH bitmap\n \n Ewah bitmaps are serialized in the same protocol as the JAVAEWAH\n-- \ngitgitgadget\n"},{"id":"457034","messageId":"xmqq8rq4fo6p.fsf@gitster.g","threadId":"57946","inReplyTo":"pull.1246.v3.git.1654858481.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-10T17:01:02Z","receivedAt":"2022-06-10T17:01:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Abhradeep Chakraborty via GitGitGadget\" \nwrites:\n\n> There are some issues in the bitmap-format html page. For example, some\n> nested lists are shown as top-level lists (e.g. [1]- Here\n> BITMAP_OPT_FULL_DAG (0x1) and BITMAP_OPT_HASH_CACHE (0x4) are shown as\n> top-level list). There is also a need of adding info about trailing checksum\n> in the docs.\n\nQuite honestly, I am not sure if a piecemeal \"let's make\n
...
 a bit prettier\" is worth our time.  Especially\nrelative to the importance of adding missing information to the\ndocumentation.\n\nSo, if this round (I haven't looked at the formatting changes at all\nyet) turns out to be still not doing the HTML properly, I'd suggest\nshuffling the patches around, add missing information so that readers\ncan get the corrections in text regardless of the rest of HTMLify\neffort.  We'll see.\n\nThanks.\n"},{"id":"457242","messageId":"YqlDlYHR1HBJRiDZ@nand.local","threadId":"57946","inReplyTo":"c74b9a52c2a7b5f3ebbfaca08c8de42aac7f7eac.1654858481.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 2/3] bitmap-format.txt: fix some formatting issues","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-06-15T02:27:33Z","receivedAt":"2022-06-15T02:27:39Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi Abhradeep,\n\nOn Fri, Jun 10, 2022 at 10:54:40AM +0000, Abhradeep Chakraborty via GitGitGadget wrote:\n> ++\n> +In each bitmap, the `n`th bit is set to true if the `n`th object\n> +in the packfile or multi-pack index is of that type.\n> +\n> +    The obvious consequence is that the OR of all 4 bitmaps will result\n> +    in a full set (all bits set), and the AND of all 4 bitmaps will\n> +    result in an empty bitmap (no bits set).\n> +\n> +\t* N entries with compressed bitmaps, one for each indexed commit\n> ++\n> +Where `N` is the total amount of entries in this bitmap index.\n> +Each entry contains the following:\n\nThe new formatting looks terrific; it's much easier to read this in my\nbrowser after generating the HTML version of these docs. Two questions:\n\n- Are the hard-tabs added in this file required for ASCIIDoc to treat it\n  correctly? They are a slight impediment to reading the source in my\n  editor, but it's not a huge deal. It would just be nice if we could\n  replace \"\\t\" characters with two or four spaces or something.\n\n- The above hunk is the only one which rendered slightly oddly to me; it\n  looks like the paragraph beginning with \"The obvious consequence ...\"\n  is surrounded by a 
 element, when it should be a continuation of\n  the above paragraph (\"In each bitmap ...\").\n\nOtherwise, this series is looking great. Let me know what you think!\n\nThanks,\nTaylor\n"},{"id":"457243","messageId":"YqlD2qtwqmIKG9lo@nand.local","threadId":"57946","inReplyTo":"xmqq8rq4fo6p.fsf@gitster.g","subject":"Re: [PATCH v3 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-06-15T02:28:42Z","receivedAt":"2022-06-15T02:28:47Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Jun 10, 2022 at 10:01:02AM -0700, Junio C Hamano wrote:\n> \"Abhradeep Chakraborty via GitGitGadget\" \n> writes:\n>\n> > There are some issues in the bitmap-format html page. For example, some\n> > nested lists are shown as top-level lists (e.g. [1]- Here\n> > BITMAP_OPT_FULL_DAG (0x1) and BITMAP_OPT_HASH_CACHE (0x4) are shown as\n> > top-level list). There is also a need of adding info about trailing checksum\n> > in the docs.\n>\n> Quite honestly, I am not sure if a piecemeal \"let's make\n> 
...
 a bit prettier\" is worth our time.  Especially\n> relative to the importance of adding missing information to the\n> documentation.\n>\n> So, if this round (I haven't looked at the formatting changes at all\n> yet) turns out to be still not doing the HTML properly, I'd suggest\n> shuffling the patches around, add missing information so that readers\n> can get the corrections in text regardless of the rest of HTMLify\n> effort.  We'll see.\n\nThis version of the series significantly improves the readability of the\ngenerated HTML, and I only had a minor comment or two.\n\nSo I think that the improvement is worthwhile, though if others disagree\nstrongly, the third patch should get picked up regardless, since it\naddresses a legitimate gap in our documentation.\n\nThanks,\nTaylor\n"},{"id":"457285","messageId":"20220615142804.69543-1-chakrabortyabhradeep79@gmail.com","threadId":"57946","inReplyTo":"YqlDlYHR1HBJRiDZ@nand.local","subject":"Re: [PATCH v3 2/3] bitmap-format.txt: fix some formatting issues","fromName":"Abhradeep Chakraborty","fromEmail":"chakrabortyabhradeep79@gmail.com","sentAt":"2022-06-15T14:28:04Z","receivedAt":"2022-06-15T14:28:57Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"Taylor Blau  wrote:\n\n> - Are the hard-tabs added in this file required for ASCIIDoc to treat it\n>   correctly? They are a slight impediment to reading the source in my\n>   editor, but it's not a huge deal. It would just be nice if we could\n>   replace \"\\t\" characters with two or four spaces or something.\n\n\nNo, it is not required for Asciidoc. But `git diff --check` was complaining\nagainst it. Don't know if that is related to my git configuration settings.\nMoreover other parts of the file didn't seem to use spaces. For these reasons,\nI used tabs. But can remove it if you say.\n\n> - The above hunk is the only one which rendered slightly oddly to me; it\n>   looks like the paragraph beginning with \"The obvious consequence ...\"\n>   is surrounded by a 
 element, when it should be a continuation of\n>   the above paragraph (\"In each bitmap ...\").\n\nThanks for pointing out. Don't know how it was missed. Correcting it.\n\nThanks :)\n"},{"id":"457323","messageId":"xmqqtu8ly2fk.fsf@gitster.g","threadId":"57946","inReplyTo":"YqlD2qtwqmIKG9lo@nand.local","subject":"Re: [PATCH v3 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-15T22:41:51Z","receivedAt":"2022-06-15T22:41:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau  writes:\n\n> This version of the series significantly improves the readability of the\n> generated HTML, and I only had a minor comment or two.\n\nYeah, I looked at the output and it is improved so much to the point\nthat the remaining paragraph or two that are still typeset in the fixed\nfont incorrectly start to look even irritating ;-)\n\nI've tentatively queued it in my tree.  I doubt that the topic is\nultra-urgent so if the remaining mark-up issues can be fixed before\nthe topic hits 'next', that would be great.\n\nThanks, both.\n"},{"id":"457355","messageId":"pull.1246.v4.git.1655355834.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v3.git.1654858481.gitgitgadget@gmail.com","subject":"[PATCH v4 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-16T05:03:51Z","receivedAt":"2022-06-16T05:04:03Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"There are some issues in the bitmap-format html page. For example, some\nnested lists are shown as top-level lists (e.g. [1]- Here\nBITMAP_OPT_FULL_DAG (0x1) and BITMAP_OPT_HASH_CACHE (0x4) are shown as\ntop-level list). There is also a need of adding info about trailing checksum\nin the docs.\n\nChanges since v3:\n\n * spaces are used instead of tabs\n * fixed remaining 
 blocks\n\nChanges since v2: The last two commits are updated to address the\nsuggestions. These changes are -\n\n * previously omitted blank lines are re-added. In the updated commit, use\n   of 
 blocks are decreased. Description lists and + are used instead\n   to add more than one paragraphs under lists. Readability of the source\n   text might decrease due to the use of +. But other documentation files\n   (e.g. git-add.txt) also use it to connect two paragraphs. So, I hope this\n   is acceptable.\n\n * Information about trailing checksum is updated (as suggested by Taylor)\n\nChanges since v1:\n\n * a new commit addressing bitmap-format.txt html page generation is added\n * Remove extra indentation from the previous change\n * elaborate more about the trailing checksum (as suggested by Kaartic)\n\ninitial version:\n\n * first commit fixes some formatting issues\n * information about trailing checksum in the bitmap file is added in the\n   bitmap-format doc.\n\n[1] https://git-scm.com/docs/bitmap-format#_on_disk_format\n\nAbhradeep Chakraborty (3):\n  bitmap-format.txt: feed the file to asciidoc to generate html\n  bitmap-format.txt: fix some formatting issues\n  bitmap-format.txt: add information for trailing checksum\n\n Documentation/Makefile                    |   1 +\n Documentation/technical/bitmap-format.txt | 203 ++++++++++++----------\n 2 files changed, 108 insertions(+), 96 deletions(-)\n\n\nbase-commit: 5699ec1b0aec51b9e9ba5a2785f65970c5a95d84\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1246%2FAbhra303%2Ffix-doc-formatting-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1246/Abhra303/fix-doc-formatting-v4\nPull-Request: https://github.com/gitgitgadget/git/pull/1246\n\nRange-diff vs v3:\n\n 1:  a1b9bd9af90 ! 1:  494c1c1bd52 bitmap-format.txt: feed the file to asciidoc to generate html\n     @@ Documentation/Makefile: TECH_DOCS += MyFirstContribution\n       TECH_DOCS += ToolsForGit\n      +TECH_DOCS += technical/bitmap-format\n       TECH_DOCS += technical/bundle-format\n     + TECH_DOCS += technical/cruft-packs\n       TECH_DOCS += technical/hash-function-transition\n     - TECH_DOCS += technical/http-protocol\n 2:  c74b9a52c2a ! 2:  25512aa9c5b bitmap-format.txt: fix some formatting issues\n     @@ Commit message\n          Signed-off-by: Abhradeep Chakraborty \n      \n       ## Documentation/technical/bitmap-format.txt ##\n     +@@ Documentation/technical/bitmap-format.txt: An object is uniquely described by its bit position within a bitmap:\n     + \tis defined as follows:\n     + \n     + \t\to1 <= o2 <==> pack(o1) <= pack(o2) /\\ offset(o1) <= offset(o2)\n     +-\n     +-\tThe ordering between packs is done according to the MIDX's .rev file.\n     +-\tNotably, the preferred pack sorts ahead of all other packs.\n     +++\n     ++The ordering between packs is done according to the MIDX's .rev file.\n     ++Notably, the preferred pack sorts ahead of all other packs.\n     + \n     + The on-disk representation (described below) of a bitmap is the same regardless\n     + of whether or not that bitmap belongs to a packfile or a MIDX. The only\n      @@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n       \n       == On-disk format\n       \n      -\t- A header appears at the beginning:\n     -+\t* A header appears at the beginning:\n     - \n     +-\n      -\t\t4-byte signature: {'B', 'I', 'T', 'M'}\n     -+\t\t4-byte signature: :: {'B', 'I', 'T', 'M'}\n     -+\n     -+\t\t2-byte version number (network byte order): ::\n     - \n     +-\n      -\t\t2-byte version number (network byte order)\n     - \t\t\tThe current implementation only supports version 1\n     - \t\t\tof the bitmap index (the same one as JGit).\n     - \n     +-\t\t\tThe current implementation only supports version 1\n     +-\t\t\tof the bitmap index (the same one as JGit).\n     +-\n      -\t\t2-byte flags (network byte order)\n     -+\t\t2-byte flags (network byte order): ::\n     - \n     - \t\t\tThe following flags are supported:\n     - \n     +-\n     +-\t\t\tThe following flags are supported:\n     +-\n      -\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n     -+\t\t\t** {empty}\n     -+\t\t\tBITMAP_OPT_FULL_DAG (0x1) REQUIRED: :::\n     -+\n     - \t\t\tThis flag must always be present. It implies that the\n     - \t\t\tbitmap index has been generated for a packfile or\n     - \t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n     -@@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cache extensions are required.\n     - \t\t\tJGit, that greatly reduces the complexity of the\n     - \t\t\timplementation.\n     - \n     +-\t\t\tThis flag must always be present. It implies that the\n     +-\t\t\tbitmap index has been generated for a packfile or\n     +-\t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n     +-\t\t\tevery single object in the packfile/MIDX can find its\n     +-\t\t\tparent links inside the same packfile/MIDX). This is a\n     +-\t\t\trequirement for the bitmap index format, also present in\n     +-\t\t\tJGit, that greatly reduces the complexity of the\n     +-\t\t\timplementation.\n     +-\n      -\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n     -+\t\t\t** {empty}\n     -+\t\t\tBITMAP_OPT_HASH_CACHE (0x4): :::\n     -+\n     - \t\t\tIf present, the end of the bitmap file contains\n     - \t\t\t`N` 32-bit name-hash values, one per object in the\n     - \t\t\tpack/MIDX. The format and meaning of the name-hash is\n     - \t\t\tdescribed below.\n     - \n     +-\t\t\tIf present, the end of the bitmap file contains\n     +-\t\t\t`N` 32-bit name-hash values, one per object in the\n     +-\t\t\tpack/MIDX. The format and meaning of the name-hash is\n     +-\t\t\tdescribed below.\n     +-\n      -\t\t4-byte entry count (network byte order)\n      -\n     -+\t\t4-byte entry count (network byte order): ::\n     - \t\t\tThe total count of entries (bitmapped commits) in this bitmap index.\n     - \n     +-\t\t\tThe total count of entries (bitmapped commits) in this bitmap index.\n     +-\n      -\t\t20-byte checksum\n      -\n     -+\t\t20-byte checksum: ::\n     - \t\t\tThe SHA1 checksum of the pack/MIDX this bitmap index\n     - \t\t\tbelongs to.\n     - \n     +-\t\t\tThe SHA1 checksum of the pack/MIDX this bitmap index\n     +-\t\t\tbelongs to.\n     +-\n      -\t- 4 EWAH bitmaps that act as type indexes\n      -\n      -\t\tType indexes are serialized after the hash cache in the shape\n     @@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cac\n      -\t\tEach entry contains the following:\n      -\n      -\t\t- 4-byte object position (network byte order)\n     -+\t* 4 EWAH bitmaps that act as type indexes\n     +-\t\t\tThe position **in the index for the packfile or\n     +-\t\t\tmulti-pack index** where the bitmap for this commit is\n     +-\t\t\tfound.\n     +-\n     +-\t\t- 1-byte XOR-offset\n     +-\t\t\tThe xor offset used to compress this bitmap. For an entry\n     +-\t\t\tin position `x`, a XOR offset of `y` means that the actual\n     +-\t\t\tbitmap representing this commit is composed by XORing the\n     +-\t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n     +-\t\t\tthe bitmap `y` entries before this one).\n     +-\n     +-\t\t\tNote that this compression can be recursive. In order to\n     +-\t\t\tXOR this entry with a previous one, the previous entry needs\n     +-\t\t\tto be decompressed first, and so on.\n     +-\n     +-\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n     +-\t\t\txor'ed against one of the 160 entries preceding it). This\n     +-\t\t\tnumber is always positive, and hence entries are always xor'ed\n     +-\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n     +-\t\t\tin the index.\n     +-\n     +-\t\t- 1-byte flags for this bitmap\n     +-\t\t\tAt the moment the only available flag is `0x1`, which hints\n     +-\t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n     +-\t\t\tfor the repository.\n     +-\n     +-\t\t- The compressed bitmap itself, see Appendix A.\n     ++    * A header appears at the beginning:\n     ++\n     ++        4-byte signature: :: {'B', 'I', 'T', 'M'}\n     ++\n     ++        2-byte version number (network byte order): ::\n     ++\n     ++            The current implementation only supports version 1\n     ++            of the bitmap index (the same one as JGit).\n     ++\n     ++        2-byte flags (network byte order): ::\n     ++\n     ++            The following flags are supported:\n     ++\n     ++            ** {empty}\n     ++            BITMAP_OPT_FULL_DAG (0x1) REQUIRED: :::\n     ++\n     ++            This flag must always be present. It implies that the\n     ++            bitmap index has been generated for a packfile or\n     ++            multi-pack index (MIDX) with full closure (i.e. where\n     ++            every single object in the packfile/MIDX can find its\n     ++            parent links inside the same packfile/MIDX). This is a\n     ++            requirement for the bitmap index format, also present in\n     ++            JGit, that greatly reduces the complexity of the\n     ++            implementation.\n     ++\n     ++            ** {empty}\n     ++            BITMAP_OPT_HASH_CACHE (0x4): :::\n     ++\n     ++            If present, the end of the bitmap file contains\n     ++            `N` 32-bit name-hash values, one per object in the\n     ++            pack/MIDX. The format and meaning of the name-hash is\n     ++            described below.\n     ++\n     ++        4-byte entry count (network byte order): ::\n     ++            The total count of entries (bitmapped commits) in this bitmap index.\n     ++\n     ++        20-byte checksum: ::\n     ++            The SHA1 checksum of the pack/MIDX this bitmap index\n     ++            belongs to.\n     ++\n     ++    * 4 EWAH bitmaps that act as type indexes\n      ++\n      +Type indexes are serialized after the hash cache in the shape\n      +of four EWAH bitmaps stored consecutively (see Appendix A for\n     @@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cac\n      +There is a bitmap for each Git object type, stored in the following\n      +order:\n      ++\n     -+\t- Commits\n     -+\t- Trees\n     -+\t- Blobs\n     -+\t- Tags\n     ++    - Commits\n     ++    - Trees\n     ++    - Blobs\n     ++    - Tags\n      +\n      ++\n      +In each bitmap, the `n`th bit is set to true if the `n`th object\n      +in the packfile or multi-pack index is of that type.\n     +++\n     ++The obvious consequence is that the OR of all 4 bitmaps will result\n     ++in a full set (all bits set), and the AND of all 4 bitmaps will\n     ++result in an empty bitmap (no bits set).\n      +\n     -+    The obvious consequence is that the OR of all 4 bitmaps will result\n     -+    in a full set (all bits set), and the AND of all 4 bitmaps will\n     -+    result in an empty bitmap (no bits set).\n     -+\n     -+\t* N entries with compressed bitmaps, one for each indexed commit\n     ++    * N entries with compressed bitmaps, one for each indexed commit\n      ++\n      +Where `N` is the total amount of entries in this bitmap index.\n      +Each entry contains the following:\n      +\n     -+\t\t** {empty}\n     -+\t\t4-byte object position (network byte order): ::\n     - \t\t\tThe position **in the index for the packfile or\n     - \t\t\tmulti-pack index** where the bitmap for this commit is\n     - \t\t\tfound.\n     - \n     --\t\t- 1-byte XOR-offset\n     -+\t\t** {empty}\n     -+\t\t1-byte XOR-offset: ::\n     - \t\t\tThe xor offset used to compress this bitmap. For an entry\n     - \t\t\tin position `x`, a XOR offset of `y` means that the actual\n     - \t\t\tbitmap representing this commit is composed by XORing the\n     - \t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n     - \t\t\tthe bitmap `y` entries before this one).\n     --\n     --\t\t\tNote that this compression can be recursive. In order to\n     --\t\t\tXOR this entry with a previous one, the previous entry needs\n     --\t\t\tto be decompressed first, and so on.\n     --\n     --\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n     --\t\t\txor'ed against one of the 160 entries preceding it). This\n     --\t\t\tnumber is always positive, and hence entries are always xor'ed\n     --\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n     --\t\t\tin the index.\n     --\n     --\t\t- 1-byte flags for this bitmap\n     ++        ** {empty}\n     ++        4-byte object position (network byte order): ::\n     ++            The position **in the index for the packfile or\n     ++            multi-pack index** where the bitmap for this commit is\n     ++            found.\n     ++\n     ++        ** {empty}\n     ++        1-byte XOR-offset: ::\n     ++            The xor offset used to compress this bitmap. For an entry\n     ++            in position `x`, a XOR offset of `y` means that the actual\n     ++            bitmap representing this commit is composed by XORing the\n     ++            bitmap for this entry with the bitmap in entry `x-y` (i.e.\n     ++            the bitmap `y` entries before this one).\n      ++\n      +NOTE: This compression can be recursive. In order to\n      +XOR this entry with a previous one, the previous entry needs\n     @@ Documentation/technical/bitmap-format.txt: MIDXs, both the bit-cache and rev-cac\n      +with **previous** bitmaps, not bitmaps that will come afterwards\n      +in the index.\n      +\n     -+\t\t** {empty}\n     -+\t\t1-byte flags for this bitmap: ::\n     - \t\t\tAt the moment the only available flag is `0x1`, which hints\n     - \t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n     - \t\t\tfor the repository.\n     - \n     --\t\t- The compressed bitmap itself, see Appendix A.\n     -+\t\t** The compressed bitmap itself, see Appendix A.\n     ++        ** {empty}\n     ++        1-byte flags for this bitmap: ::\n     ++            At the moment the only available flag is `0x1`, which hints\n     ++            that this bitmap can be re-used when rebuilding bitmap indexes\n     ++            for the repository.\n     ++\n     ++        ** The compressed bitmap itself, see Appendix A.\n       \n       == Appendix A: Serialization format for an EWAH bitmap\n       \n     +@@ Documentation/technical/bitmap-format.txt: implementation:\n     + \t- 4-byte number of words of the COMPRESSED bitmap, when stored\n     + \n     + \t- N x 8-byte words, as specified by the previous field\n     +-\n     +-\t\tThis is the actual content of the compressed bitmap.\n     +++\n     ++This is the actual content of the compressed bitmap.\n     + \n     + \t- 4-byte position of the current RLW for the compressed\n     + \t\tbitmap\n 3:  b971558e1cb ! 3:  dbb86dca205 bitmap-format.txt: add information for trailing checksum\n     @@ Commit message\n       ## Documentation/technical/bitmap-format.txt ##\n      @@ Documentation/technical/bitmap-format.txt: in the index.\n       \n     - \t\t** The compressed bitmap itself, see Appendix A.\n     +         ** The compressed bitmap itself, see Appendix A.\n       \n      +\t* {empty}\n      +\tTRAILER: ::\n\n-- \ngitgitgadget\n"},{"id":"457356","messageId":"494c1c1bd522a6de1d6bc811b86f579ca6507013.1655355834.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v4.git.1655355834.gitgitgadget@gmail.com","subject":"[PATCH v4 1/3] bitmap-format.txt: feed the file to asciidoc to generate html","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-16T05:03:52Z","receivedAt":"2022-06-16T05:04:05Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nDocumentation/Makefile does not include bitmap-format.txt to generate\na html page using asciidoc.\n\nTeach Documentation/Makefile to also generate a html page for\nDocumentation/technical/bitmap-format.txt file.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/Makefile | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex f2e7fc1daa5..4f801f4e4c9 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -94,6 +94,7 @@ TECH_DOCS += MyFirstContribution\n TECH_DOCS += MyFirstObjectWalk\n TECH_DOCS += SubmittingPatches\n TECH_DOCS += ToolsForGit\n+TECH_DOCS += technical/bitmap-format\n TECH_DOCS += technical/bundle-format\n TECH_DOCS += technical/cruft-packs\n TECH_DOCS += technical/hash-function-transition\n-- \ngitgitgadget\n\n"},{"id":"457357","messageId":"25512aa9c5b6d6df0c20c0400a0cac11afb64842.1655355834.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v4.git.1655355834.gitgitgadget@gmail.com","subject":"[PATCH v4 2/3] bitmap-format.txt: fix some formatting issues","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-16T05:03:53Z","receivedAt":"2022-06-16T05:04:06Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nThe asciidoc generated html for `Documentation/technical/bitmap-\nformat.txt` is broken. This is mainly because `-` is used for nested\nlists (which is not allowed in asciidoc) instead of `*`.\n\nFix these and also reformat it for better readability of the html page.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/technical/bitmap-format.txt | 199 +++++++++++-----------\n 1 file changed, 103 insertions(+), 96 deletions(-)\n\ndiff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\nindex 04b3ec21785..49c8e819804 100644\n--- a/Documentation/technical/bitmap-format.txt\n+++ b/Documentation/technical/bitmap-format.txt\n@@ -25,9 +25,9 @@ An object is uniquely described by its bit position within a bitmap:\n \tis defined as follows:\n \n \t\to1 <= o2 <==> pack(o1) <= pack(o2) /\\ offset(o1) <= offset(o2)\n-\n-\tThe ordering between packs is done according to the MIDX's .rev file.\n-\tNotably, the preferred pack sorts ahead of all other packs.\n++\n+The ordering between packs is done according to the MIDX's .rev file.\n+Notably, the preferred pack sorts ahead of all other packs.\n \n The on-disk representation (described below) of a bitmap is the same regardless\n of whether or not that bitmap belongs to a packfile or a MIDX. The only\n@@ -39,97 +39,104 @@ MIDXs, both the bit-cache and rev-cache extensions are required.\n \n == On-disk format\n \n-\t- A header appears at the beginning:\n-\n-\t\t4-byte signature: {'B', 'I', 'T', 'M'}\n-\n-\t\t2-byte version number (network byte order)\n-\t\t\tThe current implementation only supports version 1\n-\t\t\tof the bitmap index (the same one as JGit).\n-\n-\t\t2-byte flags (network byte order)\n-\n-\t\t\tThe following flags are supported:\n-\n-\t\t\t- BITMAP_OPT_FULL_DAG (0x1) REQUIRED\n-\t\t\tThis flag must always be present. It implies that the\n-\t\t\tbitmap index has been generated for a packfile or\n-\t\t\tmulti-pack index (MIDX) with full closure (i.e. where\n-\t\t\tevery single object in the packfile/MIDX can find its\n-\t\t\tparent links inside the same packfile/MIDX). This is a\n-\t\t\trequirement for the bitmap index format, also present in\n-\t\t\tJGit, that greatly reduces the complexity of the\n-\t\t\timplementation.\n-\n-\t\t\t- BITMAP_OPT_HASH_CACHE (0x4)\n-\t\t\tIf present, the end of the bitmap file contains\n-\t\t\t`N` 32-bit name-hash values, one per object in the\n-\t\t\tpack/MIDX. The format and meaning of the name-hash is\n-\t\t\tdescribed below.\n-\n-\t\t4-byte entry count (network byte order)\n-\n-\t\t\tThe total count of entries (bitmapped commits) in this bitmap index.\n-\n-\t\t20-byte checksum\n-\n-\t\t\tThe SHA1 checksum of the pack/MIDX this bitmap index\n-\t\t\tbelongs to.\n-\n-\t- 4 EWAH bitmaps that act as type indexes\n-\n-\t\tType indexes are serialized after the hash cache in the shape\n-\t\tof four EWAH bitmaps stored consecutively (see Appendix A for\n-\t\tthe serialization format of an EWAH bitmap).\n-\n-\t\tThere is a bitmap for each Git object type, stored in the following\n-\t\torder:\n-\n-\t\t\t- Commits\n-\t\t\t- Trees\n-\t\t\t- Blobs\n-\t\t\t- Tags\n-\n-\t\tIn each bitmap, the `n`th bit is set to true if the `n`th object\n-\t\tin the packfile or multi-pack index is of that type.\n-\n-\t\tThe obvious consequence is that the OR of all 4 bitmaps will result\n-\t\tin a full set (all bits set), and the AND of all 4 bitmaps will\n-\t\tresult in an empty bitmap (no bits set).\n-\n-\t- N entries with compressed bitmaps, one for each indexed commit\n-\n-\t\tWhere `N` is the total amount of entries in this bitmap index.\n-\t\tEach entry contains the following:\n-\n-\t\t- 4-byte object position (network byte order)\n-\t\t\tThe position **in the index for the packfile or\n-\t\t\tmulti-pack index** where the bitmap for this commit is\n-\t\t\tfound.\n-\n-\t\t- 1-byte XOR-offset\n-\t\t\tThe xor offset used to compress this bitmap. For an entry\n-\t\t\tin position `x`, a XOR offset of `y` means that the actual\n-\t\t\tbitmap representing this commit is composed by XORing the\n-\t\t\tbitmap for this entry with the bitmap in entry `x-y` (i.e.\n-\t\t\tthe bitmap `y` entries before this one).\n-\n-\t\t\tNote that this compression can be recursive. In order to\n-\t\t\tXOR this entry with a previous one, the previous entry needs\n-\t\t\tto be decompressed first, and so on.\n-\n-\t\t\tThe hard-limit for this offset is 160 (an entry can only be\n-\t\t\txor'ed against one of the 160 entries preceding it). This\n-\t\t\tnumber is always positive, and hence entries are always xor'ed\n-\t\t\twith **previous** bitmaps, not bitmaps that will come afterwards\n-\t\t\tin the index.\n-\n-\t\t- 1-byte flags for this bitmap\n-\t\t\tAt the moment the only available flag is `0x1`, which hints\n-\t\t\tthat this bitmap can be re-used when rebuilding bitmap indexes\n-\t\t\tfor the repository.\n-\n-\t\t- The compressed bitmap itself, see Appendix A.\n+    * A header appears at the beginning:\n+\n+        4-byte signature: :: {'B', 'I', 'T', 'M'}\n+\n+        2-byte version number (network byte order): ::\n+\n+            The current implementation only supports version 1\n+            of the bitmap index (the same one as JGit).\n+\n+        2-byte flags (network byte order): ::\n+\n+            The following flags are supported:\n+\n+            ** {empty}\n+            BITMAP_OPT_FULL_DAG (0x1) REQUIRED: :::\n+\n+            This flag must always be present. It implies that the\n+            bitmap index has been generated for a packfile or\n+            multi-pack index (MIDX) with full closure (i.e. where\n+            every single object in the packfile/MIDX can find its\n+            parent links inside the same packfile/MIDX). This is a\n+            requirement for the bitmap index format, also present in\n+            JGit, that greatly reduces the complexity of the\n+            implementation.\n+\n+            ** {empty}\n+            BITMAP_OPT_HASH_CACHE (0x4): :::\n+\n+            If present, the end of the bitmap file contains\n+            `N` 32-bit name-hash values, one per object in the\n+            pack/MIDX. The format and meaning of the name-hash is\n+            described below.\n+\n+        4-byte entry count (network byte order): ::\n+            The total count of entries (bitmapped commits) in this bitmap index.\n+\n+        20-byte checksum: ::\n+            The SHA1 checksum of the pack/MIDX this bitmap index\n+            belongs to.\n+\n+    * 4 EWAH bitmaps that act as type indexes\n++\n+Type indexes are serialized after the hash cache in the shape\n+of four EWAH bitmaps stored consecutively (see Appendix A for\n+the serialization format of an EWAH bitmap).\n++\n+There is a bitmap for each Git object type, stored in the following\n+order:\n++\n+    - Commits\n+    - Trees\n+    - Blobs\n+    - Tags\n+\n++\n+In each bitmap, the `n`th bit is set to true if the `n`th object\n+in the packfile or multi-pack index is of that type.\n++\n+The obvious consequence is that the OR of all 4 bitmaps will result\n+in a full set (all bits set), and the AND of all 4 bitmaps will\n+result in an empty bitmap (no bits set).\n+\n+    * N entries with compressed bitmaps, one for each indexed commit\n++\n+Where `N` is the total amount of entries in this bitmap index.\n+Each entry contains the following:\n+\n+        ** {empty}\n+        4-byte object position (network byte order): ::\n+            The position **in the index for the packfile or\n+            multi-pack index** where the bitmap for this commit is\n+            found.\n+\n+        ** {empty}\n+        1-byte XOR-offset: ::\n+            The xor offset used to compress this bitmap. For an entry\n+            in position `x`, a XOR offset of `y` means that the actual\n+            bitmap representing this commit is composed by XORing the\n+            bitmap for this entry with the bitmap in entry `x-y` (i.e.\n+            the bitmap `y` entries before this one).\n++\n+NOTE: This compression can be recursive. In order to\n+XOR this entry with a previous one, the previous entry needs\n+to be decompressed first, and so on.\n++\n+The hard-limit for this offset is 160 (an entry can only be\n+xor'ed against one of the 160 entries preceding it). This\n+number is always positive, and hence entries are always xor'ed\n+with **previous** bitmaps, not bitmaps that will come afterwards\n+in the index.\n+\n+        ** {empty}\n+        1-byte flags for this bitmap: ::\n+            At the moment the only available flag is `0x1`, which hints\n+            that this bitmap can be re-used when rebuilding bitmap indexes\n+            for the repository.\n+\n+        ** The compressed bitmap itself, see Appendix A.\n \n == Appendix A: Serialization format for an EWAH bitmap\n \n@@ -142,8 +149,8 @@ implementation:\n \t- 4-byte number of words of the COMPRESSED bitmap, when stored\n \n \t- N x 8-byte words, as specified by the previous field\n-\n-\t\tThis is the actual content of the compressed bitmap.\n++\n+This is the actual content of the compressed bitmap.\n \n \t- 4-byte position of the current RLW for the compressed\n \t\tbitmap\n-- \ngitgitgadget\n\n"},{"id":"457358","messageId":"dbb86dca20574eef0cb783be597dbe05677f1efb.1655355834.git.gitgitgadget@gmail.com","threadId":"57946","inReplyTo":"pull.1246.v4.git.1655355834.gitgitgadget@gmail.com","subject":"[PATCH v4 3/3] bitmap-format.txt: add information for trailing checksum","fromName":"Abhradeep Chakraborty via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-16T05:03:54Z","receivedAt":"2022-06-16T05:04:12Z","isPatch":true,"sender":{"key":"chakrabortyabhradeep79@gmail.com","avatar":"https://avatars.githubusercontent.com/u/75240995?v=4"},"body":"From: Abhradeep Chakraborty \n\nBitmap file has a trailing checksum at the end of the file. However\nthere is no information in the bitmap-format documentation about it.\n\nAdd a trailer section to include the trailing checksum info in the\n`Documentation/technical/bitmap-format.txt` file.\n\nSigned-off-by: Abhradeep Chakraborty \n---\n Documentation/technical/bitmap-format.txt | 4 ++++\n 1 file changed, 4 insertions(+)\n\ndiff --git a/Documentation/technical/bitmap-format.txt b/Documentation/technical/bitmap-format.txt\nindex 49c8e819804..7be5f2318ba 100644\n--- a/Documentation/technical/bitmap-format.txt\n+++ b/Documentation/technical/bitmap-format.txt\n@@ -138,6 +138,10 @@ in the index.\n \n         ** The compressed bitmap itself, see Appendix A.\n \n+\t* {empty}\n+\tTRAILER: ::\n+\t\tTrailing checksum of the preceding contents.\n+\n == Appendix A: Serialization format for an EWAH bitmap\n \n Ewah bitmaps are serialized in the same protocol as the JAVAEWAH\n-- \ngitgitgadget\n"},{"id":"457397","messageId":"xmqq35g4tp7c.fsf@gitster.g","threadId":"57946","inReplyTo":"pull.1246.v4.git.1655355834.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-16T18:53:27Z","receivedAt":"2022-06-16T18:53:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This version looks good and seems to format well.  Well done.\n\nThanks.  Will queue.\n"},{"id":"457421","messageId":"YqueOVZcv8/zYWUF@nand.local","threadId":"57946","inReplyTo":"xmqq35g4tp7c.fsf@gitster.g","subject":"Re: [PATCH v4 0/3] bitmap-format.txt: fix some formatting issues and include checksum info","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-06-16T21:18:49Z","receivedAt":"2022-06-16T21:19:03Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jun 16, 2022 at 11:53:27AM -0700, Junio C Hamano wrote:\n> This version looks good and seems to format well.  Well done.\n\nAgreed. Nice work, Abhradeep!\n\nThanks,\nTaylor\n"}]}