git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 7/9] multi-pack-index: prepare 'repack' subcommand

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Jan 23, 2019, 22:38 UTC
Message-ID
<20190123223845.147378-1-jonathantanmy@google.com>
In-Reply-To
<b39f90ad09265614efd466fee5089354a193ae8c.1547047269.git.gitgitgadget@gmail.com>
Show 18 quoted lines
> diff --git a/Documentation/git-multi-pack-index.txt b/Documentation/git-multi-pack-index.txt
> index 6186c4c936..cc63531cc0 100644
> --- a/Documentation/git-multi-pack-index.txt
> +++ b/Documentation/git-multi-pack-index.txt
> @@ -36,6 +36,17 @@ expire::
>  	have no objects referenced by the MIDX. Rewrite the MIDX file
>  	afterward to remove all references to these pack-files.
>  
> +repack::
> +	Collect a batch of pack-files whose size are all at most the
> +	size given by --batch-size, but whose sizes sum to larger
> +	than --batch-size. The batch is selected by greedily adding
> +	small pack-files starting with the oldest pack-files that fit
> +	the size. Create a new pack-file containing the objects the
> +	multi-pack-index indexes into those pack-files, and rewrite
> +	the multi-pack-index to contain that pack-file. A later run
> +	of 'git multi-pack-index expire' will delete the pack-files
> +	that were part of this batch.

I see in the subsequent patch that you stop once the batch size is matched or exceeded - I see that you mention "whose sizes sum to larger than --batch-size", but this leads me to think that if the total so happens to not exceed the batch size, don't do anything, but otherwise repack *all* the small packs together.

I would write this as:
  Create a new packfile containing the objects in the N least-sized
  packfiles referenced by the multi-pack-index, where N is the smallest
  number such that the total size of the packfiles equals or exceeds the
  given batch size. Rewrite the multi-pack-index to reference the new
  packfile instead of the N packfiles. A later run of ...
Previous: SZEDER GáborNext: Derrick Stolee
Message 42 of 92 in “Create 'expire' and 'repack' verbs for git-multi-pack-index”
  1. 0/5 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee via GitGitGadget, Dec 10, 2018
  2. 1/5 multi-pack-index: prepare for 'expire' verbDerrick Stolee via GitGitGadget, Dec 10, 2018
  3. Stefan BellerDec 11, 2018
  4. SZEDER GáborDec 11, 2018
  5. Derrick StoleeDec 11, 2018
  6. 3/5 multi-pack-index: implement 'expire' verbDerrick Stolee via GitGitGadget, Dec 10, 2018
  7. 2/5 midx: refactor permutation logicDerrick Stolee via GitGitGadget, Dec 10, 2018
  8. 4/5 multi-pack-index: prepare 'repack' verbDerrick Stolee via GitGitGadget, Dec 10, 2018
  9. Stefan BellerDec 11, 2018
  10. Derrick StoleeDec 11, 2018
  11. 5/5 midx: implement midx_repack()Derrick Stolee via GitGitGadget, Dec 10, 2018
  12. Stefan BellerDec 11, 2018
  13. Derrick StoleeDec 11, 2018
  14. Junio C HamanoDec 12, 2018
  15. Junio C HamanoDec 13, 2018
  16. 0/7 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee via GitGitGadget, Dec 21, 2018
  17. 1/7 repack: refactor pack deletion for future useDerrick Stolee via GitGitGadget, Dec 21, 2018
  18. 2/7 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee via GitGitGadget, Dec 21, 2018
  19. 3/7 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee via GitGitGadget, Dec 21, 2018
  20. 6/7 multi-pack-index: prepare 'repack' subcommandDerrick Stolee via GitGitGadget, Dec 21, 2018
  21. 5/7 multi-pack-index: implement 'expire' verbDerrick Stolee via GitGitGadget, Dec 21, 2018
  22. 7/7 midx: implement midx_repack()Derrick Stolee via GitGitGadget, Dec 21, 2018
  23. 4/7 midx: refactor permutation logicDerrick Stolee via GitGitGadget, Dec 21, 2018
  24. 0/9 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee via GitGitGadget, Jan 9, 2019
  25. 1/9 repack: refactor pack deletion for future useDerrick Stolee via GitGitGadget, Jan 9, 2019
  26. 3/9 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee via GitGitGadget, Jan 9, 2019
  27. 4/9 midx: simplify computation of pack name lengthsDerrick Stolee via GitGitGadget, Jan 9, 2019
  28. 6/9 multi-pack-index: implement 'expire' verbDerrick Stolee via GitGitGadget, Jan 9, 2019
  29. SZEDER GáborJan 9, 2019
  30. Junio C HamanoJan 10, 2019
  31. Jonathan TanJan 23, 2019
  32. Derrick StoleeJan 24, 2019
  33. 5/9 midx: refactor permutation logic and pack sortingDerrick Stolee via GitGitGadget, Jan 9, 2019
  34. Jonathan TanJan 23, 2019
  35. Derrick StoleeJan 24, 2019
  36. Derrick StoleeJan 24, 2019
  37. 8/9 midx: implement midx_repack()Derrick Stolee via GitGitGadget, Jan 9, 2019
  38. Jonathan TanJan 23, 2019
  39. 9/9 multi-pack-index: test expire while adding packsDerrick Stolee via GitGitGadget, Jan 9, 2019
  40. 7/9 multi-pack-index: prepare 'repack' subcommandDerrick Stolee via GitGitGadget, Jan 9, 2019
  41. SZEDER GáborJan 9, 2019
  42. Jonathan TanJan 23, 2019
  43. Derrick StoleeJan 24, 2019
  44. Jonathan TanJan 24, 2019
  45. 2/9 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee via GitGitGadget, Jan 9, 2019
  46. Derrick StoleeJan 17, 2019
  47. Jonathan TanJan 23, 2019
  48. 00/10 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee via GitGitGadget, Jan 24, 2019
  49. 01/10 repack: refactor pack deletion for future useDerrick Stolee via GitGitGadget, Jan 24, 2019
  50. 02/10 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee via GitGitGadget, Jan 24, 2019
  51. 04/10 midx: simplify computation of pack name lengthsDerrick Stolee via GitGitGadget, Jan 24, 2019
  52. 05/10 midx: refactor permutation logic and pack sortingDerrick Stolee via GitGitGadget, Jan 24, 2019
  53. 06/10 multi-pack-index: implement 'expire' subcommandDerrick Stolee via GitGitGadget, Jan 24, 2019
  54. 03/10 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee via GitGitGadget, Jan 24, 2019
  55. 08/10 midx: implement midx_repack()Derrick Stolee via GitGitGadget, Jan 24, 2019
  56. Derrick StoleeJan 26, 2019
  57. Junio C HamanoJan 27, 2019
  58. 09/10 multi-pack-index: test expire while adding packsDerrick Stolee via GitGitGadget, Jan 24, 2019
  59. 10/10 midx: add test that 'expire' respects .keep filesDerrick Stolee via GitGitGadget, Jan 24, 2019
  60. 07/10 multi-pack-index: prepare 'repack' subcommandDerrick Stolee via GitGitGadget, Jan 24, 2019
  61. Josh SteadmonJan 25, 2019
  62. Jonathan TanJan 24, 2019
  63. Josh SteadmonJan 25, 2019
  64. 00/11 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee, Apr 24, 2019
  65. 01/11 repack: refactor pack deletion for future useDerrick Stolee, Apr 24, 2019
  66. 03/11 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee, Apr 24, 2019
  67. 02/11 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee, Apr 24, 2019
  68. 04/11 midx: simplify computation of pack name lengthsDerrick Stolee, Apr 24, 2019
  69. 07/11 multi-pack-index: prepare 'repack' subcommandDerrick Stolee, Apr 24, 2019
  70. 08/11 midx: implement midx_repack()Derrick Stolee, Apr 24, 2019
  71. 09/11 multi-pack-index: test expire while adding packsDerrick Stolee, Apr 24, 2019
  72. 10/11 midx: add test that 'expire' respects .keep filesDerrick Stolee, Apr 24, 2019
  73. 11/11 t5319-multi-pack-index.sh: test batch size zeroDerrick Stolee, Apr 24, 2019
  74. 06/11 multi-pack-index: implement 'expire' subcommandDerrick Stolee, Apr 24, 2019
  75. 05/11 midx: refactor permutation logic and pack sortingDerrick Stolee, Apr 24, 2019
  76. Junio C HamanoApr 25, 2019
  77. Derrick StoleeApr 25, 2019
  78. 00/11 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee, May 14, 2019
  79. 01/11 repack: refactor pack deletion for future useDerrick Stolee, May 14, 2019
  80. 02/11 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee, May 14, 2019
  81. 05/11 midx: refactor permutation logic and pack sortingDerrick Stolee, May 14, 2019
  82. 06/11 multi-pack-index: implement 'expire' subcommandDerrick Stolee, May 14, 2019
  83. 03/11 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee, May 14, 2019
  84. 07/11 multi-pack-index: prepare 'repack' subcommandDerrick Stolee, May 14, 2019
  85. 09/11 multi-pack-index: test expire while adding packsDerrick Stolee, May 14, 2019
  86. 10/11 midx: add test that 'expire' respects .keep filesDerrick Stolee, May 14, 2019
  87. 11/11 t5319-multi-pack-index.sh: test batch size zeroDerrick Stolee, May 14, 2019
  88. 08/11 midx: implement midx_repack()Derrick Stolee, May 14, 2019
  89. 04/11 midx: simplify computation of pack name lengthsDerrick Stolee, May 14, 2019
  90. Derrick StoleeJun 10, 2019
  91. Junio C HamanoJun 10, 2019
  92. Derrick StoleeJun 10, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.