git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 17/19] repack: consider bitmaps when performing repacks

From
Jeff King <peff@peff.net>
Date
Oct 24, 2013, 18:08 UTC
Message-ID
<20131024180811.GQ24180@sigill.intra.peff.net>
In-Reply-To
<20131024175915.GA23398@sigill.intra.peff.net>
From: Vicent Marti <tanoku@gmail.com>

Since `pack-objects` will write a `.bitmap` file next to the `.pack` and `.idx` files, this commit teaches `git-repack` to consider the new bitmap indexes (if they exist) when performing repack operations.

This implies moving old bitmap indexes out of the way if we are repacking a repository that already has them, and moving the newly generated bitmap indexes into the `objects/pack` directory, next to their corresponding packfiles.

Since `git repack` is now capable of handling these `.bitmap` files, a normal `git gc` run on a repository that has `pack.writebitmaps` set to true in its config file will generate bitmap indexes as part of the garbage collection process.

Alternatively, `git repack` can be called with the `-b` switch to explicitly generate bitmap indexes if you are experimenting and don't want them on all the time.

Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
---
 builtin/repack.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/builtin/repack.c b/builtin/repack.c
index 8b7dfd0..239f278 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -94,7 +94,7 @@ static void get_non_kept_pack_filenames(struct string_list *fname_list)
 
 static void remove_redundant_pack(const char *dir_name, const char *base_name)
 {
-	const char *exts[] = {".pack", ".idx", ".keep"};
+	const char *exts[] = {".pack", ".idx", ".keep", ".bitmap"};
 	int i;
 	struct strbuf buf = STRBUF_INIT;
 	size_t plen;
@@ -121,6 +121,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 	} exts[] = {
 		{".pack"},
 		{".idx"},
+		{".bitmap", 1},
 	};
 	struct child_process cmd;
 	struct string_list_item *item;
@@ -143,6 +144,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 	int no_update_server_info = 0;
 	int quiet = 0;
 	int local = 0;
+	int write_bitmap = -1;
 
 	struct option builtin_repack_options[] = {
 		OPT_BIT('a', NULL, &pack_everything,
@@ -161,6 +163,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 		OPT__QUIET(&quiet, N_("be quiet")),
 		OPT_BOOL('l', "local", &local,
 				N_("pass --local to git-pack-objects")),
+		OPT_BOOL('b', "write-bitmap-index", &write_bitmap,
+				N_("write bitmap index")),
 		OPT_STRING(0, "unpack-unreachable", &unpack_unreachable, N_("approxidate"),
 				N_("with -A, do not loosen objects older than this")),
 		OPT_INTEGER(0, "window", &window,
@@ -202,6 +206,9 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 		argv_array_pushf(&cmd_args, "--no-reuse-delta");
 	if (no_reuse_object)
 		argv_array_pushf(&cmd_args, "--no-reuse-object");
+	if (write_bitmap >= 0)
+		argv_array_pushf(&cmd_args, "--%swrite-bitmap-index",
+				 write_bitmap ? "" : "no-");
 
 	if (pack_everything & ALL_INTO_ONE) {
 		get_non_kept_pack_filenames(&existing_packs);
-- 
1.8.4.1.898.g8bf8a41.dirty
Previous: Jeff KingNext: Jeff King
Message 46 of 87 in “pack bitmaps”
  1. 0/19 pack bitmapsJeff King, Oct 24, 2013
  2. 01/19 sha1write: make buffer const-correctJeff King, Oct 24, 2013
  3. 02/19 revindex: Export new APIsJeff King, Oct 24, 2013
  4. 03/19 pack-objects: Refactor the packing listJeff King, Oct 24, 2013
  5. 04/19 pack-objects: factor out name_hashJeff King, Oct 24, 2013
  6. 05/19 revision: allow setting custom limiter functionJeff King, Oct 24, 2013
  7. 06/19 sha1_file: export `git_open_noatime`Jeff King, Oct 24, 2013
  8. 07/19 compat: add endianness helpersJeff King, Oct 24, 2013
  9. Thomas RastOct 26, 2013
  10. Jeff KingOct 30, 2013
  11. Vicent MartíOct 30, 2013
  12. 08/19 ewah: compressed bitmap implementationJeff King, Oct 24, 2013
  13. Junio C HamanoOct 24, 2013
  14. Jeff KingOct 25, 2013
  15. Thomas RastOct 26, 2013
  16. 09/19 documentation: add documentation for the bitmap formatJeff King, Oct 24, 2013
  17. Duy NguyenOct 25, 2013
  18. Jeff KingOct 25, 2013
  19. Duy NguyenOct 25, 2013
  20. Shawn PearceOct 25, 2013
  21. Jeff KingOct 30, 2013
  22. Shawn PearceOct 30, 2013
  23. Vicent MartiOct 30, 2013
  24. Vicent MartiOct 30, 2013
  25. 10/19 pack-bitmap: add support for bitmap indexesJeff King, Oct 24, 2013
  26. Shawn PearceOct 25, 2013
  27. Jeff KingOct 30, 2013
  28. Shawn PearceOct 30, 2013
  29. Vicent MartiOct 30, 2013
  30. Shawn PearceOct 30, 2013
  31. Jeff KingOct 30, 2013
  32. 11/19 pack-objects: use bitmaps when packing objectsJeff King, Oct 24, 2013
  33. Shawn PearceOct 25, 2013
  34. Jeff KingOct 30, 2013
  35. Shawn PearceOct 30, 2013
  36. Vicent MartiOct 30, 2013
  37. 12/19 rev-list: add bitmap mode to speed up object listsJeff King, Oct 24, 2013
  38. Shawn PearceOct 25, 2013
  39. Jeff KingOct 30, 2013
  40. 13/19 pack-objects: implement bitmap writingJeff King, Oct 24, 2013
  41. Duy NguyenOct 25, 2013
  42. Jeff KingOct 25, 2013
  43. 14/19 repack: stop using magic number for ARRAY_SIZE(exts)Jeff King, Oct 24, 2013
  44. 15/19 repack: turn exts array into array-of-structJeff King, Oct 24, 2013
  45. 16/19 repack: handle optional files created by pack-objectsJeff King, Oct 24, 2013
  46. 17/19 repack: consider bitmaps when performing repacksJeff King, Oct 24, 2013
  47. 18/19 t: add basic bitmap functionality testsJeff King, Oct 24, 2013
  48. 19/19 pack-bitmap: implement optional name_hash cacheJeff King, Oct 24, 2013
  49. Junio C HamanoOct 24, 2013
  50. Junio C HamanoOct 25, 2013
  51. 0/19 pack bitmapsJeff King, Oct 25, 2013
  52. 01/19 sha1write: make buffer const-correctJeff King, Oct 25, 2013
  53. 02/19 revindex: Export new APIsJeff King, Oct 25, 2013
  54. 03/19 pack-objects: Refactor the packing listJeff King, Oct 25, 2013
  55. 04/19 pack-objects: factor out name_hashJeff King, Oct 25, 2013
  56. 05/19 revision: allow setting custom limiter functionJeff King, Oct 25, 2013
  57. 06/19 sha1_file: export `git_open_noatime`Jeff King, Oct 25, 2013
  58. 07/19 compat: add endianness helpersJeff King, Oct 25, 2013
  59. 08/19 ewah: compressed bitmap implementationJeff King, Oct 25, 2013
  60. 09/19 documentation: add documentation for the bitmap formatJeff King, Oct 25, 2013
  61. 10/19 pack-bitmap: add support for bitmap indexesJeff King, Oct 25, 2013
  62. Junio C HamanoOct 25, 2013
  63. Jeff KingOct 26, 2013
  64. Jeff KingOct 26, 2013
  65. Junio C HamanoOct 28, 2013
  66. Jeff KingOct 30, 2013
  67. Duy NguyenOct 26, 2013
  68. Jeff KingOct 30, 2013
  69. 11/19 pack-objects: use bitmaps when packing objectsJeff King, Oct 25, 2013
  70. Duy NguyenOct 26, 2013
  71. Jeff KingOct 30, 2013
  72. Duy NguyenOct 30, 2013
  73. Jeff KingOct 30, 2013
  74. Duy NguyenOct 31, 2013
  75. 12/19 rev-list: add bitmap mode to speed up object listsJeff King, Oct 25, 2013
  76. 13/19 pack-objects: implement bitmap writingJeff King, Oct 25, 2013
  77. 14/19 repack: stop using magic number for ARRAY_SIZE(exts)Jeff King, Oct 25, 2013
  78. 15/19 repack: turn exts array into array-of-structJeff King, Oct 25, 2013
  79. 16/19 repack: handle optional files created by pack-objectsJeff King, Oct 25, 2013
  80. 17/19 repack: consider bitmaps when performing repacksJeff King, Oct 25, 2013
  81. 18/19 t: add basic bitmap functionality testsJeff King, Oct 25, 2013
  82. SZEDER GáborOct 28, 2013
  83. Jeff KingOct 30, 2013
  84. 19/19 pack-bitmap: implement optional name_hash cacheJeff King, Oct 25, 2013
  85. 20/19 count-objects: consider .bitmap without .pack/.idx pair garbageNguyễn Thái Ngọc Duy, Oct 26, 2013
  86. Jeff KingOct 30, 2013
  87. Junio C HamanoOct 30, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.