git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 09/18] maintenance: add loose-objects task

From
Junio C Hamano <gitster@pobox.com>
Date
Jul 24, 2020, 19:57 UTC
Message-ID
<xmqqk0yslm82.fsf@gitster.c.googlers.com>
In-Reply-To
<b861244c-af0d-581f-b1c8-6c6cb667eed8@gmail.com>
Derrick Stolee <stolee@gmail.com> writes:
Show 11 quoted lines
>  * In a partial clone, it is likely that we get loose objects only
>    due to a command like "git log -p" that downloads blobs
>    one-by-one. In such a case, this step coming in later and picking
>    up those blobs _will_ find good deltas because they are present
>    in the same batch.
>
>  * (I know this case isn't important to core Git, but please indulge
>    me) In a VFS for Git repo, the loose objects correspond to blobs
>    that were faulted in by a virtual filesystem read. In this case,
>    the blobs are usually from a single commit in history, so good
>    deltas between the blobs don't actually exist!

Let me stop here by saying that I am now starting to worry about overfitting the repacking strategy to lazy clone repositories. I am perfectly fine with the plan to start with just one strategy overfit for partially cloned repositories, as long as we make sure that we can be extended it to suit other access/object acquisition patterns.

Previous: Derrick StoleeNext: Emily Shaffer
Message 110 of 164 in “Maintenance builtin, allowing 'gc --auto' customization”
  1. 00/21 Maintenance builtin, allowing 'gc --auto' customizationDerrick Stolee via GitGitGadget, Jul 7, 2020
  2. 01/21 gc: use the_repository less oftenDerrick Stolee via GitGitGadget, Jul 7, 2020
  3. 02/21 gc: use repository in too_many_loose_objects()Derrick Stolee via GitGitGadget, Jul 7, 2020
  4. 03/21 gc: use repo configDerrick Stolee via GitGitGadget, Jul 7, 2020
  5. 05/21 maintenance: create basic maintenance runnerDerrick Stolee via GitGitGadget, Jul 7, 2020
  6. 06/21 maintenance: add --quiet optionDerrick Stolee via GitGitGadget, Jul 7, 2020
  7. 08/21 maintenance: initialize task array and hashmapDerrick Stolee via GitGitGadget, Jul 7, 2020
  8. Jonathan TanJul 9, 2020
  9. Derrick StoleeJul 9, 2020
  10. Junio C HamanoJul 9, 2020
  11. 07/21 maintenance: replace run_auto_gc()Derrick Stolee via GitGitGadget, Jul 7, 2020
  12. 09/21 maintenance: add commit-graph taskDerrick Stolee via GitGitGadget, Jul 7, 2020
  13. Jonathan TanJul 9, 2020
  14. Derrick StoleeJul 9, 2020
  15. Jeff KingJul 9, 2020
  16. Derrick StoleeJul 9, 2020
  17. 10/21 maintenance: add --task optionDerrick Stolee via GitGitGadget, Jul 7, 2020
  18. 11/21 maintenance: take a lock on the objects directoryDerrick Stolee via GitGitGadget, Jul 7, 2020
  19. 17/21 maintenance: use pointers to check --autoDerrick Stolee via GitGitGadget, Jul 7, 2020
  20. 19/21 maintenance: create auto condition for loose-objectsDerrick Stolee via GitGitGadget, Jul 7, 2020
  21. 20/21 maintenance: add pack-files auto conditionDerrick Stolee via GitGitGadget, Jul 7, 2020
  22. 21/21 midx: use start_delayed_progress()Derrick Stolee via GitGitGadget, Jul 7, 2020
  23. 16/21 maintenance: create maintenance.<task>.enabled configDerrick Stolee via GitGitGadget, Jul 7, 2020
  24. 18/21 maintenance: add auto condition for commit-graph taskDerrick Stolee via GitGitGadget, Jul 7, 2020
  25. 13/21 maintenance: add loose-objects taskDerrick Stolee via GitGitGadget, Jul 7, 2020
  26. 15/21 maintenance: auto-size pack-files batchDerrick Stolee via GitGitGadget, Jul 7, 2020
  27. 14/21 maintenance: add pack-files taskDerrick Stolee via GitGitGadget, Jul 7, 2020
  28. 04/21 gc: drop the_repository in log locationDerrick Stolee via GitGitGadget, Jul 7, 2020
  29. Jonathan TanJul 9, 2020
  30. Derrick StoleeJul 9, 2020
  31. 12/21 maintenance: add fetch taskDerrick Stolee via GitGitGadget, Jul 7, 2020
  32. Jonathan TanJul 9, 2020
  33. Emily ShafferJul 8, 2020
  34. Derrick StoleeJul 9, 2020
  35. Derrick StoleeJul 9, 2020
  36. Jeff KingJul 9, 2020
  37. Derrick StoleeJul 9, 2020
  38. Emily ShafferJul 10, 2020
  39. Son Luong NgocJul 10, 2020
  40. Junio C HamanoJul 9, 2020
  41. Derrick StoleeJul 9, 2020
  42. Junio C HamanoJul 9, 2020
  43. Derrick StoleeJul 9, 2020
  44. 00/18 Maintenance builtin, allowing 'gc --auto' customizationDerrick Stolee via GitGitGadget, Jul 23, 2020
  45. 01/18 maintenance: create basic maintenance runnerDerrick Stolee via GitGitGadget, Jul 23, 2020
  46. Taylor BlauJul 25, 2020
  47. Đoàn Trần Công DanhJul 25, 2020
  48. Jonathan NiederJul 29, 2020
  49. Derrick StoleeJul 30, 2020
  50. Jonathan NiederJul 31, 2020
  51. Derrick StoleeAug 3, 2020
  52. Jonathan NiederAug 3, 2020
  53. Taylor BlauAug 3, 2020
  54. Jonathan NiederAug 3, 2020
  55. Taylor BlauAug 3, 2020
  56. Jonathan NiederAug 3, 2020
  57. Junio C HamanoAug 4, 2020
  58. Derrick StoleeAug 4, 2020
  59. Jonathan NiederAug 4, 2020
  60. Derrick StoleeAug 4, 2020
  61. Jonathan NiederAug 4, 2020
  62. Derrick StoleeAug 4, 2020
  63. Derrick StoleeAug 5, 2020
  64. Jonathan NiederJul 31, 2020
  65. Jonathan NiederAug 3, 2020
  66. 02/18 maintenance: add --quiet optionDerrick Stolee via GitGitGadget, Jul 23, 2020
  67. 03/18 maintenance: replace run_auto_gc()Derrick Stolee via GitGitGadget, Jul 23, 2020
  68. Junio C HamanoJul 23, 2020
  69. Taylor BlauJul 25, 2020
  70. Derrick StoleeJul 30, 2020
  71. Derrick StoleeJul 30, 2020
  72. Eric SunshineJul 30, 2020
  73. Derrick StoleeJul 30, 2020
  74. 04/18 maintenance: initialize task arrayDerrick Stolee via GitGitGadget, Jul 23, 2020
  75. Junio C HamanoJul 23, 2020
  76. Derrick StoleeJul 24, 2020
  77. Derrick StoleeJul 24, 2020
  78. Junio C HamanoJul 24, 2020
  79. Taylor BlauJul 25, 2020
  80. Emily ShafferJul 29, 2020
  81. 05/18 maintenance: add commit-graph taskDerrick Stolee via GitGitGadget, Jul 23, 2020
  82. Junio C HamanoJul 23, 2020
  83. Derrick StoleeJul 24, 2020
  84. Junio C HamanoJul 24, 2020
  85. Taylor BlauJul 25, 2020
  86. Derrick StoleeJul 30, 2020
  87. Jeff KingJul 29, 2020
  88. 07/18 maintenance: take a lock on the objects directoryDerrick Stolee via GitGitGadget, Jul 23, 2020
  89. 06/18 maintenance: add --task optionDerrick Stolee via GitGitGadget, Jul 23, 2020
  90. Junio C HamanoJul 23, 2020
  91. Junio C HamanoJul 23, 2020
  92. Derrick StoleeJul 24, 2020
  93. Junio C HamanoJul 24, 2020
  94. 08/18 maintenance: add prefetch taskDerrick Stolee via GitGitGadget, Jul 23, 2020
  95. Junio C HamanoJul 23, 2020
  96. Derrick StoleeJul 24, 2020
  97. Junio C HamanoJul 24, 2020
  98. Đoàn Trần Công DanhJul 25, 2020
  99. Junio C HamanoJul 25, 2020
  100. Derrick StoleeJul 27, 2020
  101. Junio C HamanoJul 27, 2020
  102. Derrick StoleeJul 27, 2020
  103. fetch: optionally allow disabling FETCH_HEAD updateJunio C Hamano, Jul 28, 2020
  104. Phillip WoodJul 29, 2020
  105. Phillip WoodJul 29, 2020
  106. Derrick StoleeJul 30, 2020
  107. 09/18 maintenance: add loose-objects taskDerrick Stolee via GitGitGadget, Jul 23, 2020
  108. Junio C HamanoJul 23, 2020
  109. Derrick StoleeJul 24, 2020
  110. Junio C HamanoJul 24, 2020
  111. Emily ShafferJul 29, 2020
  112. Derrick StoleeJul 30, 2020
  113. 10/18 maintenance: add incremental-repack taskDerrick Stolee via GitGitGadget, Jul 23, 2020
  114. Junio C HamanoJul 23, 2020
  115. Derrick StoleeJul 24, 2020
  116. Emily ShafferJul 29, 2020
  117. 11/18 maintenance: auto-size incremental-repack batchDerrick Stolee via GitGitGadget, Jul 23, 2020
  118. Junio C HamanoJul 23, 2020
  119. Eric SunshineJul 23, 2020
  120. Junio C HamanoJul 23, 2020
  121. Derrick StoleeJul 24, 2020
  122. Derrick StoleeJul 24, 2020
  123. Junio C HamanoJul 24, 2020
  124. Emily ShafferJul 29, 2020
  125. Derrick StoleeJul 30, 2020
  126. Derrick StoleeJul 30, 2020
  127. Chris TorekJul 30, 2020
  128. Đoàn Trần Công DanhAug 5, 2020
  129. Derrick StoleeAug 6, 2020
  130. 13/18 maintenance: use pointers to check --autoDerrick Stolee via GitGitGadget, Jul 23, 2020
  131. 14/18 maintenance: add auto condition for commit-graph taskDerrick Stolee via GitGitGadget, Jul 23, 2020
  132. 12/18 maintenance: create maintenance.<task>.enabled configDerrick Stolee via GitGitGadget, Jul 23, 2020
  133. 18/18 maintenance: add trace2 regions for task executionDerrick Stolee via GitGitGadget, Jul 23, 2020
  134. 17/18 midx: use start_delayed_progress()Derrick Stolee via GitGitGadget, Jul 23, 2020
  135. 16/18 maintenance: add incremental-repack auto conditionDerrick Stolee via GitGitGadget, Jul 23, 2020
  136. 15/18 maintenance: create auto condition for loose-objectsDerrick Stolee via GitGitGadget, Jul 23, 2020
  137. Emily ShafferJul 29, 2020
  138. 00/20 Maintenance builtin, allowing 'gc --auto' customizationDerrick Stolee via GitGitGadget, Jul 30, 2020
  139. 01/20 maintenance: create basic maintenance runnerDerrick Stolee via GitGitGadget, Jul 30, 2020
  140. 02/20 maintenance: add --quiet optionDerrick Stolee via GitGitGadget, Jul 30, 2020
  141. 04/20 maintenance: initialize task arrayDerrick Stolee via GitGitGadget, Jul 30, 2020
  142. 03/20 maintenance: replace run_auto_gc()Derrick Stolee via GitGitGadget, Jul 30, 2020
  143. 05/20 maintenance: add commit-graph taskDerrick Stolee via GitGitGadget, Jul 30, 2020
  144. 06/20 maintenance: add --task optionDerrick Stolee via GitGitGadget, Jul 30, 2020
  145. 08/20 fetch: optionally allow disabling FETCH_HEAD updateJunio C Hamano via GitGitGadget, Jul 30, 2020
  146. 07/20 maintenance: take a lock on the objects directoryDerrick Stolee via GitGitGadget, Jul 30, 2020
  147. 10/20 maintenance: add loose-objects taskDerrick Stolee via GitGitGadget, Jul 30, 2020
  148. 09/20 maintenance: add prefetch taskDerrick Stolee via GitGitGadget, Jul 30, 2020
  149. 11/20 midx: enable core.multiPackIndex by defaultDerrick Stolee via GitGitGadget, Jul 30, 2020
  150. 14/20 maintenance: create maintenance.<task>.enabled configDerrick Stolee via GitGitGadget, Jul 30, 2020
  151. 15/20 maintenance: use pointers to check --autoDerrick Stolee via GitGitGadget, Jul 30, 2020
  152. 13/20 maintenance: auto-size incremental-repack batchDerrick Stolee via GitGitGadget, Jul 30, 2020
  153. Chris TorekJul 30, 2020
  154. Derrick StoleeAug 3, 2020
  155. 12/20 maintenance: add incremental-repack taskDerrick Stolee via GitGitGadget, Jul 30, 2020
  156. 20/20 maintenance: add trace2 regions for task executionDerrick Stolee via GitGitGadget, Jul 30, 2020
  157. 18/20 maintenance: add incremental-repack auto conditionDerrick Stolee via GitGitGadget, Jul 30, 2020
  158. 17/20 maintenance: create auto condition for loose-objectsDerrick Stolee via GitGitGadget, Jul 30, 2020
  159. 19/20 midx: use start_delayed_progress()Derrick Stolee via GitGitGadget, Jul 30, 2020
  160. 16/20 maintenance: add auto condition for commit-graph taskDerrick Stolee via GitGitGadget, Jul 30, 2020
  161. Junio C HamanoJul 30, 2020
  162. Junio C HamanoJul 30, 2020
  163. Junio C HamanoJul 31, 2020
  164. Derrick StoleeAug 6, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.