git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 8/9] midx: implement midx_repack()

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Jan 23, 2019, 22:33 UTC
Message-ID
<20190123223318.146120-1-jonathantanmy@google.com>
In-Reply-To
<a4c2d5a8e16f888f9a3d1090fa92d986919f0162.1547047269.git.gitgitgadget@gmail.com>
Show 7 quoted lines
> From: Derrick Stolee <dstolee@microsoft.com>
> 
> To repack using a multi-pack-index, first sort all pack-files by
> their modified time. Second, walk those pack-files from oldest
> to newest, adding the packs to a list if they are smaller than the
> given pack-size. Finally, collect the objects from the multi-pack-
> index that are in those packs and send them to 'git pack-objects'.
Also mention that we stop once the total is at least the batch size.
Show 12 quoted lines
> +test_expect_success 'repack creates a new pack' '
> +	(
> +		cd dup &&
> +		SECOND_SMALLEST_SIZE=$(ls -l .git/objects/pack/*pack | awk "{print \$5;}" | sort -n | head -n 2 | tail -n 1) &&
> +		BATCH_SIZE=$(($SECOND_SMALLEST_SIZE + 1)) &&
> +		git multi-pack-index repack --batch-size=$BATCH_SIZE &&
> +		ls .git/objects/pack/*idx >idx-list &&
> +		test_line_count = 5 idx-list &&
> +		test-tool read-midx .git/objects | grep idx >midx-list &&
> +		test_line_count = 5 midx-list
> +	)
> +'

Can there be a test_line_count of the output of ls at the beginning, so that we do not have to look at the previous test to indeed see that the 5 is greater than before?

Also add a test to check what happens when we have 3 packs below the batch size, but taking 2 of them together is sufficient to exceed the batch size.

Show 11 quoted lines
> +test_expect_success 'expire removes repacked packs' '
> +	(
> +		cd dup &&
> +		ls -S .git/objects/pack/*pack | head -n 3 >expect &&
> +		git multi-pack-index expire &&
> +		ls -S .git/objects/pack/*pack >actual &&
> +		test_cmp expect actual &&
> +		test-tool read-midx .git/objects | grep idx >midx-list &&
> +		test_line_count = 3 midx-list
> +	)
> +'
Same comment as above.
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 38 of 92 in “Create 'expire' and 'repack' verbs for git-multi-pack-index”
  1. 0/5 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee via GitGitGadget, Dec 10, 2018
  2. 1/5 multi-pack-index: prepare for 'expire' verbDerrick Stolee via GitGitGadget, Dec 10, 2018
  3. Stefan BellerDec 11, 2018
  4. SZEDER GáborDec 11, 2018
  5. Derrick StoleeDec 11, 2018
  6. 3/5 multi-pack-index: implement 'expire' verbDerrick Stolee via GitGitGadget, Dec 10, 2018
  7. 2/5 midx: refactor permutation logicDerrick Stolee via GitGitGadget, Dec 10, 2018
  8. 4/5 multi-pack-index: prepare 'repack' verbDerrick Stolee via GitGitGadget, Dec 10, 2018
  9. Stefan BellerDec 11, 2018
  10. Derrick StoleeDec 11, 2018
  11. 5/5 midx: implement midx_repack()Derrick Stolee via GitGitGadget, Dec 10, 2018
  12. Stefan BellerDec 11, 2018
  13. Derrick StoleeDec 11, 2018
  14. Junio C HamanoDec 12, 2018
  15. Junio C HamanoDec 13, 2018
  16. 0/7 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee via GitGitGadget, Dec 21, 2018
  17. 1/7 repack: refactor pack deletion for future useDerrick Stolee via GitGitGadget, Dec 21, 2018
  18. 2/7 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee via GitGitGadget, Dec 21, 2018
  19. 3/7 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee via GitGitGadget, Dec 21, 2018
  20. 6/7 multi-pack-index: prepare 'repack' subcommandDerrick Stolee via GitGitGadget, Dec 21, 2018
  21. 5/7 multi-pack-index: implement 'expire' verbDerrick Stolee via GitGitGadget, Dec 21, 2018
  22. 7/7 midx: implement midx_repack()Derrick Stolee via GitGitGadget, Dec 21, 2018
  23. 4/7 midx: refactor permutation logicDerrick Stolee via GitGitGadget, Dec 21, 2018
  24. 0/9 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee via GitGitGadget, Jan 9, 2019
  25. 1/9 repack: refactor pack deletion for future useDerrick Stolee via GitGitGadget, Jan 9, 2019
  26. 3/9 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee via GitGitGadget, Jan 9, 2019
  27. 4/9 midx: simplify computation of pack name lengthsDerrick Stolee via GitGitGadget, Jan 9, 2019
  28. 6/9 multi-pack-index: implement 'expire' verbDerrick Stolee via GitGitGadget, Jan 9, 2019
  29. SZEDER GáborJan 9, 2019
  30. Junio C HamanoJan 10, 2019
  31. Jonathan TanJan 23, 2019
  32. Derrick StoleeJan 24, 2019
  33. 5/9 midx: refactor permutation logic and pack sortingDerrick Stolee via GitGitGadget, Jan 9, 2019
  34. Jonathan TanJan 23, 2019
  35. Derrick StoleeJan 24, 2019
  36. Derrick StoleeJan 24, 2019
  37. 8/9 midx: implement midx_repack()Derrick Stolee via GitGitGadget, Jan 9, 2019
  38. Jonathan TanJan 23, 2019
  39. 9/9 multi-pack-index: test expire while adding packsDerrick Stolee via GitGitGadget, Jan 9, 2019
  40. 7/9 multi-pack-index: prepare 'repack' subcommandDerrick Stolee via GitGitGadget, Jan 9, 2019
  41. SZEDER GáborJan 9, 2019
  42. Jonathan TanJan 23, 2019
  43. Derrick StoleeJan 24, 2019
  44. Jonathan TanJan 24, 2019
  45. 2/9 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee via GitGitGadget, Jan 9, 2019
  46. Derrick StoleeJan 17, 2019
  47. Jonathan TanJan 23, 2019
  48. 00/10 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee via GitGitGadget, Jan 24, 2019
  49. 01/10 repack: refactor pack deletion for future useDerrick Stolee via GitGitGadget, Jan 24, 2019
  50. 02/10 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee via GitGitGadget, Jan 24, 2019
  51. 04/10 midx: simplify computation of pack name lengthsDerrick Stolee via GitGitGadget, Jan 24, 2019
  52. 05/10 midx: refactor permutation logic and pack sortingDerrick Stolee via GitGitGadget, Jan 24, 2019
  53. 06/10 multi-pack-index: implement 'expire' subcommandDerrick Stolee via GitGitGadget, Jan 24, 2019
  54. 03/10 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee via GitGitGadget, Jan 24, 2019
  55. 08/10 midx: implement midx_repack()Derrick Stolee via GitGitGadget, Jan 24, 2019
  56. Derrick StoleeJan 26, 2019
  57. Junio C HamanoJan 27, 2019
  58. 09/10 multi-pack-index: test expire while adding packsDerrick Stolee via GitGitGadget, Jan 24, 2019
  59. 10/10 midx: add test that 'expire' respects .keep filesDerrick Stolee via GitGitGadget, Jan 24, 2019
  60. 07/10 multi-pack-index: prepare 'repack' subcommandDerrick Stolee via GitGitGadget, Jan 24, 2019
  61. Josh SteadmonJan 25, 2019
  62. Jonathan TanJan 24, 2019
  63. Josh SteadmonJan 25, 2019
  64. 00/11 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee, Apr 24, 2019
  65. 01/11 repack: refactor pack deletion for future useDerrick Stolee, Apr 24, 2019
  66. 03/11 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee, Apr 24, 2019
  67. 02/11 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee, Apr 24, 2019
  68. 04/11 midx: simplify computation of pack name lengthsDerrick Stolee, Apr 24, 2019
  69. 07/11 multi-pack-index: prepare 'repack' subcommandDerrick Stolee, Apr 24, 2019
  70. 08/11 midx: implement midx_repack()Derrick Stolee, Apr 24, 2019
  71. 09/11 multi-pack-index: test expire while adding packsDerrick Stolee, Apr 24, 2019
  72. 10/11 midx: add test that 'expire' respects .keep filesDerrick Stolee, Apr 24, 2019
  73. 11/11 t5319-multi-pack-index.sh: test batch size zeroDerrick Stolee, Apr 24, 2019
  74. 06/11 multi-pack-index: implement 'expire' subcommandDerrick Stolee, Apr 24, 2019
  75. 05/11 midx: refactor permutation logic and pack sortingDerrick Stolee, Apr 24, 2019
  76. Junio C HamanoApr 25, 2019
  77. Derrick StoleeApr 25, 2019
  78. 00/11 Create 'expire' and 'repack' verbs for git-multi-pack-indexDerrick Stolee, May 14, 2019
  79. 01/11 repack: refactor pack deletion for future useDerrick Stolee, May 14, 2019
  80. 02/11 Docs: rearrange subcommands for multi-pack-indexDerrick Stolee, May 14, 2019
  81. 05/11 midx: refactor permutation logic and pack sortingDerrick Stolee, May 14, 2019
  82. 06/11 multi-pack-index: implement 'expire' subcommandDerrick Stolee, May 14, 2019
  83. 03/11 multi-pack-index: prepare for 'expire' subcommandDerrick Stolee, May 14, 2019
  84. 07/11 multi-pack-index: prepare 'repack' subcommandDerrick Stolee, May 14, 2019
  85. 09/11 multi-pack-index: test expire while adding packsDerrick Stolee, May 14, 2019
  86. 10/11 midx: add test that 'expire' respects .keep filesDerrick Stolee, May 14, 2019
  87. 11/11 t5319-multi-pack-index.sh: test batch size zeroDerrick Stolee, May 14, 2019
  88. 08/11 midx: implement midx_repack()Derrick Stolee, May 14, 2019
  89. 04/11 midx: simplify computation of pack name lengthsDerrick Stolee, May 14, 2019
  90. Derrick StoleeJun 10, 2019
  91. Junio C HamanoJun 10, 2019
  92. Derrick StoleeJun 10, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.