git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 16/24] config: create core.multiPackIndex setting

From
Derrick Stolee <stolee@gmail.com>
Date
Jul 12, 2018, 13:19 UTC
Message-ID
<2270ec32-f5e3-e23a-5605-f5f0e7a44c46@gmail.com>
In-Reply-To
<CAPig+cTJuvOjyzkmAj5axAkS3q1UyeV03jQk6-oB==KyzbEK_w@mail.gmail.com>
On 7/6/2018 1:39 AM, Eric Sunshine wrote:
Show 31 quoted lines
> On Thu, Jul 5, 2018 at 8:54 PM Derrick Stolee <stolee@gmail.com> wrote:
>> The core.multiPackIndex config setting controls the multi-pack-
>> index (MIDX) feature. If false, the setting will disable all reads
>> from the multi-pack-index file.
>>
>> Add comparison commands in t5319-multi-pack-index.sh to check
>> typical Git behavior remains the same as the config setting is turned
>> on and off. This currently includes 'git rev-list' and 'git log'
>> commands to trigger several object database reads.
>>
>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>> diff --git a/cache.h b/cache.h
>> @@ -814,6 +814,7 @@ extern char *git_replace_ref_base;
>> +extern int core_multi_pack_index;
>> diff --git a/config.c b/config.c
>> @@ -1313,6 +1313,11 @@ static int git_default_core_config(const char *var, const char *value)
>> +       if (!strcmp(var, "core.multipackindex")) {
>> +               core_multi_pack_index = git_config_bool(var, value);
>> +               return 0;
>> +       }
> This is a rather unusual commit. This new configuration is assigned,
> but it's never actually consulted, which means that it's impossible
> for it to have any impact on functionality, yet tests are being added
> to check whether it _did_ have any impact on functionality. Confusing.
>
> Patch 17 does consult 'core_multi_pack_index', so it's only at that
> point that it could have any impact. This situation would be less
> confusing if you swapped patches 16 and 17 (and, of course, move the
> declaration of 'core_multi_pack_index' to patch 17 with a reasonable
> default value).

You're right that this commit is a bit too aware of the future, but I disagree with the recommendation to change it.

Yes, in this commit there is no possible way that these tests could fail. The point is that patches 17-23 all change behavior if this setting is on, and we want to make sure we do not break at any point along that journey (or in future iterations of the multi-pack-index feature).

With this in mind, I don't think there is a better commit to place these tests.

Show 40 quoted lines
>> diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
>> @@ -61,12 +63,42 @@ test_expect_success 'write midx with one v1 pack' '
>> +midx_git_two_modes() {
>> +       git -c core.multiPackIndex=false $1 >expect &&
>> +       git -c core.multiPackIndex=true $1 >actual &&
>> +       test_cmp expect actual
>> +}
>> +
>> +compare_results_with_midx() {
>> +       MSG=$1
>> +       test_expect_success "check normal git operations: $MSG" '
>> +               midx_git_two_modes "rev-list --objects --all" &&
>> +               midx_git_two_modes "log --raw"
>> +       '
>> +}
> Here, you define midx_git_two_modes() and compare_results_with_midx()...
>
>>   test_expect_success 'write midx with one v2 pack' '
>> -       git pack-objects --index-version=2,0x40 pack/test <obj-list &&
>> -       git multi-pack-index --object-dir=. write &&
>> -       midx_read_expect 1 17 4 .
>> +       git pack-objects --index-version=2,0x40 $objdir/pack/test <obj-list &&
>> +       git multi-pack-index --object-dir=$objdir write &&
>> +       midx_read_expect 1 17 4 $objdir
>>   '
>>
>> +midx_git_two_modes() {
>> +       git -c core.multiPackIndex=false $1 >expect &&
>> +       git -c core.multiPackIndex=true $1 >actual &&
>> +       test_cmp expect actual
>> +}
>> +
>> +compare_results_with_midx() {
>> +       MSG=$1
>> +       test_expect_success "check normal git operations: $MSG" '
>> +               midx_git_two_modes "rev-list --objects --all" &&
>> +               midx_git_two_modes "log --raw"
>> +       '
>> +}
> ... and here you define both functions again.
This was a mistake. Thanks for catching it.
Thanks,
-Stolee
Previous: Eric SunshineNext: Derrick Stolee
Message 145 of 192 in “Multi-pack-index (MIDX)”
  1. 00/23 Multi-pack-index (MIDX)Derrick Stolee, Jun 7, 2018
  2. 01/23 midx: add design documentDerrick Stolee, Jun 7, 2018
  3. Stefan BellerJun 11, 2018
  4. Derrick StoleeJun 18, 2018
  5. 03/23 midx: add midx builtinDerrick Stolee, Jun 7, 2018
  6. Duy NguyenJun 7, 2018
  7. Derrick StoleeJun 18, 2018
  8. Stefan BellerJun 11, 2018
  9. Derrick StoleeJun 18, 2018
  10. Stefan BellerJun 18, 2018
  11. Derrick StoleeJun 18, 2018
  12. 04/23 midx: add 'write' subcommand and basic wiringDerrick Stolee, Jun 7, 2018
  13. Duy NguyenJun 7, 2018
  14. 09/23 midx: write pack names in chunkDerrick Stolee, Jun 7, 2018
  15. Duy NguyenJun 7, 2018
  16. Derrick StoleeJun 21, 2018
  17. Junio C HamanoJun 21, 2018
  18. Derrick StoleeJun 22, 2018
  19. Junio C HamanoJun 22, 2018
  20. Derrick StoleeJun 22, 2018
  21. 12/23 midx: write object ids in a chunkDerrick Stolee, Jun 7, 2018
  22. Duy NguyenJun 9, 2018
  23. 10/23 midx: write a lookup into the pack names chunkDerrick Stolee, Jun 7, 2018
  24. Duy NguyenJun 9, 2018
  25. Derrick StoleeJun 21, 2018
  26. 16/23 midx: prepare midxed_git structDerrick Stolee, Jun 7, 2018
  27. Duy NguyenJun 9, 2018
  28. 11/23 midx: sort and deduplicate objects from packfilesDerrick Stolee, Jun 7, 2018
  29. Duy NguyenJun 9, 2018
  30. Derrick StoleeJun 21, 2018
  31. 21/23 midx: prevent duplicate packfile loadsDerrick Stolee, Jun 7, 2018
  32. Duy NguyenJun 9, 2018
  33. 23/23 midx: clear midx on repackDerrick Stolee, Jun 7, 2018
  34. Duy NguyenJun 9, 2018
  35. Derrick StoleeJun 22, 2018
  36. 22/23 midx: use midx to find ref-deltasDerrick Stolee, Jun 7, 2018
  37. 13/23 midx: write object id fanout chunkDerrick Stolee, Jun 7, 2018
  38. Duy NguyenJun 9, 2018
  39. Derrick StoleeJun 21, 2018
  40. 14/23 midx: write object offsetsDerrick Stolee, Jun 7, 2018
  41. Duy NguyenJun 9, 2018
  42. 20/23 midx: use midx in approximate_object_countDerrick Stolee, Jun 7, 2018
  43. Duy NguyenJun 9, 2018
  44. Derrick StoleeJun 22, 2018
  45. 19/23 midx: use existing midx when writing new oneDerrick Stolee, Jun 7, 2018
  46. 15/23 midx: create core.midx config settingDerrick Stolee, Jun 7, 2018
  47. 18/23 midx: use midx in abbreviation calculationsDerrick Stolee, Jun 7, 2018
  48. Duy NguyenJun 9, 2018
  49. Derrick StoleeJun 22, 2018
  50. 17/23 midx: read objects from multi-pack-indexDerrick Stolee, Jun 7, 2018
  51. Duy NguyenJun 9, 2018
  52. Derrick StoleeJun 21, 2018
  53. 05/23 midx: write header information to lockfileDerrick Stolee, Jun 7, 2018
  54. Duy NguyenJun 7, 2018
  55. Duy NguyenJun 12, 2018
  56. Derrick StoleeJun 19, 2018
  57. Duy NguyenJun 19, 2018
  58. Derrick StoleeJun 19, 2018
  59. 08/23 midx: read packfiles from pack directoryDerrick Stolee, Jun 7, 2018
  60. Duy NguyenJun 7, 2018
  61. packfile: generalize pack directory listDerrick Stolee, Jun 20, 2018
  62. 07/23 midx: expand test dataDerrick Stolee, Jun 7, 2018
  63. 06/23 midx: struct midxed_git and 'read' subcommandDerrick Stolee, Jun 7, 2018
  64. Duy NguyenJun 7, 2018
  65. Derrick StoleeJun 20, 2018
  66. Duy NguyenJun 7, 2018
  67. Derrick StoleeJun 20, 2018
  68. Duy NguyenJun 20, 2018
  69. Derrick StoleeJun 20, 2018
  70. 02/23 midx: add midx format details to pack-format.txtDerrick Stolee, Jun 7, 2018
  71. Stefan BellerJun 11, 2018
  72. Derrick StoleeJun 18, 2018
  73. Stefan BellerJun 18, 2018
  74. Derrick StoleeJun 7, 2018
  75. Ævar Arnfjörð BjarmasonJun 7, 2018
  76. Derrick StoleeJun 7, 2018
  77. 00/24 Multi-pack-index (MIDX)Derrick Stolee, Jun 25, 2018
  78. 01/24 multi-pack-index: add design documentDerrick Stolee, Jun 25, 2018
  79. 02/24 multi-pack-index: add format detailsDerrick Stolee, Jun 25, 2018
  80. 04/24 multi-pack-index: add 'write' verbDerrick Stolee, Jun 25, 2018
  81. 03/24 multi-pack-index: add builtinDerrick Stolee, Jun 25, 2018
  82. Junio C HamanoJun 25, 2018
  83. 05/24 midx: write header information to lockfileDerrick Stolee, Jun 25, 2018
  84. Junio C HamanoJun 25, 2018
  85. Derrick StoleeJul 5, 2018
  86. 08/24 packfile: generalize pack directory listDerrick Stolee, Jun 25, 2018
  87. Junio C HamanoJun 25, 2018
  88. 09/24 multi-pack-index: read packfile listDerrick Stolee, Jun 25, 2018
  89. 06/24 multi-pack-index: load into memoryDerrick Stolee, Jun 25, 2018
  90. Junio C HamanoJun 25, 2018
  91. Derrick StoleeJul 5, 2018
  92. Eric SunshineJul 5, 2018
  93. Junio C HamanoJul 6, 2018
  94. 11/24 midx: read pack names into arrayDerrick Stolee, Jun 25, 2018
  95. Eric SunshineJun 25, 2018
  96. 12/24 midx: sort and deduplicate objects from packfilesDerrick Stolee, Jun 25, 2018
  97. 13/24 midx: write object ids in a chunkDerrick Stolee, Jun 25, 2018
  98. 14/24 midx: write object id fanout chunkDerrick Stolee, Jun 25, 2018
  99. 15/24 midx: write object offsetsDerrick Stolee, Jun 25, 2018
  100. 16/24 config: create core.multiPackIndex settingDerrick Stolee, Jun 25, 2018
  101. 18/24 midx: read objects from multi-pack-indexDerrick Stolee, Jun 25, 2018
  102. 10/24 multi-pack-index: write pack names in chunkDerrick Stolee, Jun 25, 2018
  103. 19/24 midx: use midx in abbreviation calculationsDerrick Stolee, Jun 25, 2018
  104. 17/24 midx: prepare midxed_git structDerrick Stolee, Jun 25, 2018
  105. 23/24 packfile: skip loading index if in multi-pack-indexDerrick Stolee, Jun 25, 2018
  106. 24/24 midx: clear midx on repackDerrick Stolee, Jun 25, 2018
  107. 20/24 midx: use existing midx when writing new oneDerrick Stolee, Jun 25, 2018
  108. 22/24 midx: prevent duplicate packfile loadsDerrick Stolee, Jun 25, 2018
  109. 21/24 midx: use midx in approximate_object_countDerrick Stolee, Jun 25, 2018
  110. 07/24 multi-pack-index: expand test dataDerrick Stolee, Jun 25, 2018
  111. Junio C HamanoJun 25, 2018
  112. 00/24 Multi-pack-index (MIDX)Derrick Stolee, Jul 6, 2018
  113. 01/24 multi-pack-index: add design documentDerrick Stolee, Jul 6, 2018
  114. 02/24 multi-pack-index: add format detailsDerrick Stolee, Jul 6, 2018
  115. 04/24 multi-pack-index: add 'write' verbDerrick Stolee, Jul 6, 2018
  116. Eric SunshineJul 6, 2018
  117. 03/24 multi-pack-index: add builtinDerrick Stolee, Jul 6, 2018
  118. Eric SunshineJul 6, 2018
  119. 05/24 midx: write header information to lockfileDerrick Stolee, Jul 6, 2018
  120. 08/24 packfile: generalize pack directory listDerrick Stolee, Jul 6, 2018
  121. 06/24 multi-pack-index: load into memoryDerrick Stolee, Jul 6, 2018
  122. Eric SunshineJul 6, 2018
  123. Eric SunshineJul 6, 2018
  124. Junio C HamanoJul 9, 2018
  125. Derrick StoleeJul 12, 2018
  126. 09/24 multi-pack-index: read packfile listDerrick Stolee, Jul 6, 2018
  127. 10/24 multi-pack-index: write pack names in chunkDerrick Stolee, Jul 6, 2018
  128. 13/24 midx: write object ids in a chunkDerrick Stolee, Jul 6, 2018
  129. Eric SunshineJul 6, 2018
  130. 12/24 midx: sort and deduplicate objects from packfilesDerrick Stolee, Jul 6, 2018
  131. 14/24 midx: write object id fanout chunkDerrick Stolee, Jul 6, 2018
  132. 15/24 midx: write object offsetsDerrick Stolee, Jul 6, 2018
  133. Eric SunshineJul 6, 2018
  134. Derrick StoleeJul 12, 2018
  135. 18/24 midx: read objects from multi-pack-indexDerrick Stolee, Jul 6, 2018
  136. 19/24 midx: use midx in abbreviation calculationsDerrick Stolee, Jul 6, 2018
  137. 20/24 midx: use existing midx when writing new oneDerrick Stolee, Jul 6, 2018
  138. 24/24 midx: clear midx on repackDerrick Stolee, Jul 6, 2018
  139. Eric SunshineJul 6, 2018
  140. 21/24 midx: use midx in approximate_object_countDerrick Stolee, Jul 6, 2018
  141. 22/24 midx: prevent duplicate packfile loadsDerrick Stolee, Jul 6, 2018
  142. 23/24 packfile: skip loading index if in multi-pack-indexDerrick Stolee, Jul 6, 2018
  143. 16/24 config: create core.multiPackIndex settingDerrick Stolee, Jul 6, 2018
  144. Eric SunshineJul 6, 2018
  145. Derrick StoleeJul 12, 2018
  146. Derrick StoleeJul 12, 2018
  147. SZEDER GáborJul 11, 2018
  148. Derrick StoleeJul 12, 2018
  149. SZEDER GáborJul 12, 2018
  150. Derrick StoleeJul 12, 2018
  151. Junio C HamanoJul 12, 2018
  152. 17/24 midx: prepare midxed_git structDerrick Stolee, Jul 6, 2018
  153. Eric SunshineJul 6, 2018
  154. 07/24 multi-pack-index: expand test dataDerrick Stolee, Jul 6, 2018
  155. Eric SunshineJul 6, 2018
  156. Eric SunshineJul 6, 2018
  157. Derrick StoleeJul 12, 2018
  158. Eric SunshineJul 12, 2018
  159. Derrick StoleeJul 12, 2018
  160. 11/24 midx: read pack names into arrayDerrick Stolee, Jul 6, 2018
  161. Eric SunshineJul 6, 2018
  162. 00/23 Multi-pack-index (MIDX)Derrick Stolee, Jul 12, 2018
  163. 02/23 multi-pack-index: add format detailsDerrick Stolee, Jul 12, 2018
  164. 01/23 multi-pack-index: add design documentDerrick Stolee, Jul 12, 2018
  165. 03/23 multi-pack-index: add builtinDerrick Stolee, Jul 12, 2018
  166. Junio C HamanoJul 20, 2018
  167. brian m. carlsonJul 20, 2018
  168. Junio C HamanoJul 20, 2018
  169. 04/23 multi-pack-index: add 'write' verbDerrick Stolee, Jul 12, 2018
  170. Eric SunshineJul 12, 2018
  171. 06/23 multi-pack-index: load into memoryDerrick Stolee, Jul 12, 2018
  172. 05/23 midx: write header information to lockfileDerrick Stolee, Jul 12, 2018
  173. 07/23 t5319: expand test dataDerrick Stolee, Jul 12, 2018
  174. 08/23 packfile: generalize pack directory listDerrick Stolee, Jul 12, 2018
  175. 09/23 multi-pack-index: read packfile listDerrick Stolee, Jul 12, 2018
  176. 11/23 midx: read pack names into arrayDerrick Stolee, Jul 12, 2018
  177. 10/23 multi-pack-index: write pack names in chunkDerrick Stolee, Jul 12, 2018
  178. 12/23 midx: sort and deduplicate objects from packfilesDerrick Stolee, Jul 12, 2018
  179. 13/23 midx: write object ids in a chunkDerrick Stolee, Jul 12, 2018
  180. 14/23 midx: write object id fanout chunkDerrick Stolee, Jul 12, 2018
  181. 15/23 midx: write object offsetsDerrick Stolee, Jul 12, 2018
  182. 16/23 config: create core.multiPackIndex settingDerrick Stolee, Jul 12, 2018
  183. Junio C HamanoJul 12, 2018
  184. Derrick StoleeJul 13, 2018
  185. 17/23 midx: read objects from multi-pack-indexDerrick Stolee, Jul 12, 2018
  186. 18/23 midx: use midx in abbreviation calculationsDerrick Stolee, Jul 12, 2018
  187. 19/23 midx: use existing midx when writing new oneDerrick Stolee, Jul 12, 2018
  188. 20/23 midx: use midx in approximate_object_countDerrick Stolee, Jul 12, 2018
  189. 21/23 midx: prevent duplicate packfile loadsDerrick Stolee, Jul 12, 2018
  190. 23/23 midx: clear midx on repackDerrick Stolee, Jul 12, 2018
  191. 22/23 packfile: skip loading index if in multi-pack-indexDerrick Stolee, Jul 12, 2018
  192. Junio C HamanoJul 12, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.