git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v8 5/9] pack-bitmap-write: rebuild using new bitmap when remapping

From
Christian Couder <christian.couder@gmail.com>
Date
Oct 2, 2023, 16:55 UTC
Message-ID
<20231002165504.1325153-6-christian.couder@gmail.com>
In-Reply-To
<20231002165504.1325153-1-christian.couder@gmail.com>

`git repack` is about to learn a new `--filter=<filter-spec>` option and we will want to check that this option is incompatible with `--write-bitmap-index`.

Unfortunately it appears that a test like:
test_expect_success '--filter fails with --write-bitmap-index' '
       test_must_fail \
               env GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=0 \
               git -C bare.git repack -a -d --write-bitmap-index --filter=blob:none
'

sometimes fail because when rebuilding bitmaps, it appears that we are reusing existing bitmap information. So instead of detecting that some objects are missing and erroring out as it should, the `git repack --write-bitmap-index --filter=...` command succeeds.

Let's fix that by making sure we rebuild bitmaps using new bitmaps instead of existing ones.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 pack-bitmap-write.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c
index f6757c3cbf..f4ecdf8b0e 100644
--- a/pack-bitmap-write.c
+++ b/pack-bitmap-write.c
@@ -413,15 +413,19 @@ static int fill_bitmap_commit(struct bb_commit *ent,
 
 		if (old_bitmap && mapping) {
 			struct ewah_bitmap *old = bitmap_for_commit(old_bitmap, c);
+			struct bitmap *remapped = bitmap_new();
 			/*
 			 * If this commit has an old bitmap, then translate that
 			 * bitmap and add its bits to this one. No need to walk
 			 * parents or the tree for this commit.
 			 */
-			if (old && !rebuild_bitmap(mapping, old, ent->bitmap)) {
+			if (old && !rebuild_bitmap(mapping, old, remapped)) {
+				bitmap_or(ent->bitmap, remapped);
+				bitmap_free(remapped);
 				reused_bitmaps_nr++;
 				continue;
 			}
+			bitmap_free(remapped);
 		}
 
 		/*
-- 
2.42.0.305.g5bfd918c90
Previous: Christian CouderNext: Christian Couder
Message 156 of 161 in “Repack objects into separate packfiles based on a filter”
  1. 0/9 Repack objects into separate packfiles based on a filterChristian Couder, Jun 14, 2023
  2. 1/9 pack-objects: allow `--filter` without `--stdout`Christian Couder, Jun 14, 2023
  3. Taylor BlauJun 21, 2023
  4. Christian CouderJul 5, 2023
  5. 2/9 pack-objects: add `--print-filtered` to print omitted objectsChristian Couder, Jun 14, 2023
  6. Junio C HamanoJun 15, 2023
  7. Taylor BlauJun 21, 2023
  8. Christian CouderJun 21, 2023
  9. Taylor BlauJun 21, 2023
  10. 3/9 t/helper: add 'find-pack' test-toolChristian Couder, Jun 14, 2023
  11. Junio C HamanoJun 15, 2023
  12. Christian CouderJun 21, 2023
  13. Taylor BlauJun 21, 2023
  14. 5/9 repack: refactor finishing pack-objects commandChristian Couder, Jun 14, 2023
  15. Junio C HamanoJun 16, 2023
  16. Taylor BlauJun 21, 2023
  17. Christian CouderJun 21, 2023
  18. Taylor BlauJun 21, 2023
  19. 4/9 repack: refactor piping an oid to a commandChristian Couder, Jun 14, 2023
  20. Junio C HamanoJun 15, 2023
  21. Taylor BlauJun 21, 2023
  22. Christian CouderJun 21, 2023
  23. 6/9 repack: add `--filter=<filter-spec>` optionChristian Couder, Jun 14, 2023
  24. Junio C HamanoJun 16, 2023
  25. Taylor BlauJun 21, 2023
  26. Christian CouderJun 21, 2023
  27. Taylor BlauJun 22, 2023
  28. Christian CouderJun 21, 2023
  29. Junio C HamanoJun 21, 2023
  30. Christian CouderJun 22, 2023
  31. Junio C HamanoJun 22, 2023
  32. Taylor BlauJun 21, 2023
  33. Christian CouderJul 5, 2023
  34. 8/9 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Jun 14, 2023
  35. Junio C HamanoJun 16, 2023
  36. Taylor BlauJun 21, 2023
  37. Christian CouderJun 21, 2023
  38. Taylor BlauJun 21, 2023
  39. Junio C HamanoJun 21, 2023
  40. Christian CouderJul 5, 2023
  41. 7/9 gc: add `gc.repackFilter` config optionChristian Couder, Jun 14, 2023
  42. 9/9 gc: add `gc.repackFilterTo` config optionChristian Couder, Jun 14, 2023
  43. Junio C HamanoJun 16, 2023
  44. Junio C HamanoJun 14, 2023
  45. Junio C HamanoJun 16, 2023
  46. 0/8 Repack objects into separate packfiles based on a filterChristian Couder, Jul 5, 2023
  47. 1/8 pack-objects: allow `--filter` without `--stdout`Christian Couder, Jul 5, 2023
  48. 2/8 t/helper: add 'find-pack' test-toolChristian Couder, Jul 5, 2023
  49. 3/8 repack: refactor finishing pack-objects commandChristian Couder, Jul 5, 2023
  50. 4/8 repack: refactor finding pack prefixChristian Couder, Jul 5, 2023
  51. 5/8 repack: add `--filter=<filter-spec>` optionChristian Couder, Jul 5, 2023
  52. Junio C HamanoJul 5, 2023
  53. Christian CouderJul 24, 2023
  54. Junio C HamanoJul 24, 2023
  55. Christian CouderJul 25, 2023
  56. Junio C HamanoJul 25, 2023
  57. Junio C HamanoJul 25, 2023
  58. Christian CouderAug 8, 2023
  59. Taylor BlauAug 9, 2023
  60. Junio C HamanoAug 9, 2023
  61. Junio C HamanoAug 9, 2023
  62. Jeff KingAug 10, 2023
  63. Junio C HamanoJul 5, 2023
  64. Christian CouderJul 24, 2023
  65. 6/8 gc: add `gc.repackFilter` config optionChristian Couder, Jul 5, 2023
  66. 8/8 gc: add `gc.repackFilterTo` config optionChristian Couder, Jul 5, 2023
  67. 7/8 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Jul 5, 2023
  68. Junio C HamanoJul 5, 2023
  69. Christian CouderJul 24, 2023
  70. Junio C HamanoJul 24, 2023
  71. Robert CoupJul 25, 2023
  72. Junio C HamanoJul 25, 2023
  73. Christian CouderJul 25, 2023
  74. 0/8 Repack objects into separate packfiles based on a filterChristian Couder, Jul 24, 2023
  75. 1/8 pack-objects: allow `--filter` without `--stdout`Christian Couder, Jul 24, 2023
  76. Taylor BlauJul 25, 2023
  77. Junio C HamanoJul 25, 2023
  78. 3/8 repack: refactor finishing pack-objects commandChristian Couder, Jul 24, 2023
  79. Taylor BlauJul 25, 2023
  80. 4/8 repack: refactor finding pack prefixChristian Couder, Jul 24, 2023
  81. Taylor BlauJul 25, 2023
  82. Christian CouderAug 8, 2023
  83. 2/8 t/helper: add 'find-pack' test-toolChristian Couder, Jul 24, 2023
  84. Taylor BlauJul 25, 2023
  85. Christian CouderAug 8, 2023
  86. 5/8 repack: add `--filter=<filter-spec>` optionChristian Couder, Jul 24, 2023
  87. Taylor BlauJul 25, 2023
  88. Christian CouderAug 8, 2023
  89. Taylor BlauAug 9, 2023
  90. 6/8 gc: add `gc.repackFilter` config optionChristian Couder, Jul 24, 2023
  91. Taylor BlauJul 25, 2023
  92. Christian CouderAug 8, 2023
  93. Taylor BlauAug 9, 2023
  94. 8/8 gc: add `gc.repackFilterTo` config optionChristian Couder, Jul 24, 2023
  95. 7/8 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Jul 24, 2023
  96. Taylor BlauJul 25, 2023
  97. 0/8 Repack objects into separate packfiles based on a filterChristian Couder, Aug 8, 2023
  98. 7/8 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Aug 8, 2023
  99. 5/8 repack: add `--filter=<filter-spec>` optionChristian Couder, Aug 8, 2023
  100. Taylor BlauAug 9, 2023
  101. 4/8 repack: refactor finding pack prefixChristian Couder, Aug 8, 2023
  102. Taylor BlauAug 9, 2023
  103. 3/8 repack: refactor finishing pack-objects commandChristian Couder, Aug 8, 2023
  104. 6/8 gc: add `gc.repackFilter` config optionChristian Couder, Aug 8, 2023
  105. 8/8 gc: add `gc.repackFilterTo` config optionChristian Couder, Aug 8, 2023
  106. 1/8 pack-objects: allow `--filter` without `--stdout`Christian Couder, Aug 8, 2023
  107. 2/8 t/helper: add 'find-pack' test-toolChristian Couder, Aug 8, 2023
  108. Taylor BlauAug 9, 2023
  109. Taylor BlauAug 9, 2023
  110. Junio C HamanoAug 9, 2023
  111. Christian CouderAug 12, 2023
  112. 0/8 Repack objects into separate packfiles based on a filterChristian Couder, Aug 12, 2023
  113. 1/8 pack-objects: allow `--filter` without `--stdout`Christian Couder, Aug 12, 2023
  114. 2/8 t/helper: add 'find-pack' test-toolChristian Couder, Aug 12, 2023
  115. 3/8 repack: refactor finishing pack-objects commandChristian Couder, Aug 12, 2023
  116. 4/8 repack: refactor finding pack prefixChristian Couder, Aug 12, 2023
  117. 5/8 repack: add `--filter=<filter-spec>` optionChristian Couder, Aug 12, 2023
  118. 6/8 gc: add `gc.repackFilter` config optionChristian Couder, Aug 12, 2023
  119. 7/8 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Aug 12, 2023
  120. 8/8 gc: add `gc.repackFilterTo` config optionChristian Couder, Aug 12, 2023
  121. Junio C HamanoAug 15, 2023
  122. Taylor BlauAug 15, 2023
  123. Junio C HamanoAug 15, 2023
  124. Taylor BlauAug 15, 2023
  125. Junio C HamanoAug 15, 2023
  126. Taylor BlauAug 16, 2023
  127. Junio C HamanoAug 16, 2023
  128. Christian CouderSep 11, 2023
  129. 0/9 Repack objects into separate packfiles based on a filterChristian Couder, Sep 11, 2023
  130. 4/9 repack: refactor finding pack prefixChristian Couder, Sep 11, 2023
  131. 5/9 pack-bitmap-write: rebuild using new bitmap when remappingChristian Couder, Sep 11, 2023
  132. 2/9 t/helper: add 'find-pack' test-toolChristian Couder, Sep 11, 2023
  133. 6/9 repack: add `--filter=<filter-spec>` optionChristian Couder, Sep 11, 2023
  134. 1/9 pack-objects: allow `--filter` without `--stdout`Christian Couder, Sep 11, 2023
  135. 8/9 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Sep 11, 2023
  136. 7/9 gc: add `gc.repackFilter` config optionChristian Couder, Sep 11, 2023
  137. 3/9 repack: refactor finishing pack-objects commandChristian Couder, Sep 11, 2023
  138. 9/9 gc: add `gc.repackFilterTo` config optionChristian Couder, Sep 11, 2023
  139. 0/9 Repack objects into separate packfiles based on a filterChristian Couder, Sep 25, 2023
  140. 1/9 pack-objects: allow `--filter` without `--stdout`Christian Couder, Sep 25, 2023
  141. 2/9 t/helper: add 'find-pack' test-toolChristian Couder, Sep 25, 2023
  142. 3/9 repack: refactor finishing pack-objects commandChristian Couder, Sep 25, 2023
  143. 4/9 repack: refactor finding pack prefixChristian Couder, Sep 25, 2023
  144. 5/9 pack-bitmap-write: rebuild using new bitmap when remappingChristian Couder, Sep 25, 2023
  145. 7/9 gc: add `gc.repackFilter` config optionChristian Couder, Sep 25, 2023
  146. 6/9 repack: add `--filter=<filter-spec>` optionChristian Couder, Sep 25, 2023
  147. 8/9 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Sep 25, 2023
  148. 9/9 gc: add `gc.repackFilterTo` config optionChristian Couder, Sep 25, 2023
  149. Junio C HamanoSep 25, 2023
  150. Taylor BlauSep 25, 2023
  151. 0/9 Repack objects into separate packfiles based on a filterChristian Couder, Oct 2, 2023
  152. 1/9 pack-objects: allow `--filter` without `--stdout`Christian Couder, Oct 2, 2023
  153. 2/9 t/helper: add 'find-pack' test-toolChristian Couder, Oct 2, 2023
  154. 3/9 repack: refactor finishing pack-objects commandChristian Couder, Oct 2, 2023
  155. 4/9 repack: refactor finding pack prefixChristian Couder, Oct 2, 2023
  156. 5/9 pack-bitmap-write: rebuild using new bitmap when remappingChristian Couder, Oct 2, 2023
  157. 7/9 gc: add `gc.repackFilter` config optionChristian Couder, Oct 2, 2023
  158. 9/9 gc: add `gc.repackFilterTo` config optionChristian Couder, Oct 2, 2023
  159. 8/9 repack: implement `--filter-to` for storing filtered out objectsChristian Couder, Oct 2, 2023
  160. 6/9 repack: add `--filter=<filter-spec>` optionChristian Couder, Oct 2, 2023
  161. Taylor BlauOct 2, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.