git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 01/17] Documentation/technical: add cruft-packs.txt

From
Taylor Blau <me@ttaylorr.com>
Date
Mar 30, 2022, 02:23 UTC
Message-ID
<YkO/P3xgGYmhAz2O@nand.local>
In-Reply-To
<xmqqa6d8yckj.fsf@gitster.g>
On Tue, Mar 29, 2022 at 08:59:24AM -0700, Junio C Hamano wrote:
Show 25 quoted lines
> Taylor Blau <me@ttaylorr.com> writes:
>
> > I'm not sure... if we did:
> >
> > --- 8< ---
> >
> > diff --git a/setup.c b/setup.c
> > index 04ce33cdcd..fa54c9baa4 100644
> > --- a/setup.c
> > +++ b/setup.c
> > @@ -565,2 +565,4 @@ static enum extension_result handle_extension(const char *var,
> >  		return EXTENSION_OK;
> > +	} else if (!strcmp(ext, "cruftpacks")) {
> > +		return EXTENSION_OK;
> >  	}
> >
> > --- >8 ---
> >
> > but nothing more, then a hypothetical `extensions.cruftPacks` could be
> > used to prevent older writers in a mixed version environment. But if you
> > don't have or care about older versions of Git, you can avoid setting it
> > altogether.
>
> Smells like "unsafe by default, but you can opt into safety", which
> is backwards, isn't it?

I see it a little differently. The default (not writing cruft packs at all) is safe, even in a mixed-version environment. If a user (a) wants to use cruft packs, and (b) has older versions of Git also gc'ing the repository, and (c) can't get rid of them, _then_ an opt-in extension would make it impossible for those older versions to interact with the repository.

I still can't shake the feeling that this is a pretty fringe and timing-dependent scenario, which at worst keeps too many unreachable objects around.

But I think this in conjunction with the already opt-in nature of cruft packs would be a nice way to create safeguards for the situation Jonathan described. There may be a simpler way, but I'm not sure I see it (i.e., if you control whether or not `--cruft` is passed when doing maintenance with newer versions of Git, but not whether older versions are running around doing their own maintenance, then an extension would be necessary to lock the old versions out).

Show 19 quoted lines
> > But cruft packs are already guarded in two places:
> >
> >   - `git repack` won't write a cruft pack unless given the `--cruft`
> >     flag (i.e., `git repack -A` doesn't suddenly start generating cruft
> >     packs upon upgrade).
> >
> >   - `git gc` won't write cruft packs unless the `gc.cruftPacks`
> >     configuration is set, or `--cruft` is given as a flag.
>
> Hmph, OK.  So individuals can sort-of protect from hurting
> themselves by refraining from running these with --cruft or writing
> --cruft in their maintenance scripts.  An organization that wants to
> let the more adventurous types to early opt-in can prepare two
> versions of the maintenance scripts they distribute to their users,
> one with and the other without --cruft, and use the mechanism they
> use for gradual rollouts to control the population.  Perhaps that
> would make sufficient protection?  I dunno.
>
> Jonathan, what do you think?

I'm confused: if newer versions of Git are writing cruft packs, then having the older versions gc'ing in the same repository runs into the same scenario Jonathan originally describes.

The thing I think Jonathan seeks to prevent is older versions of Git gc'ing a repo that has cruft packs. I think I may need you to clarify a little, sorry :-(.

Show 13 quoted lines
> > I'd be curious what Jonathan and others think of that approach (which,
> > to be clear, is what this series already implements). We could make it
> > clear to say:
> >
> >     If you have mixed versions of Git which both repack a repository
> >     (either manually or by auto-GC / background maintenance), consider
> >     leaving `gc.cruftPacks` unset and avoiding passing `--cruft` as a
> >     command-line argument to `git repack` and `git gc`, since doing so
> >     can lead to [...]
>
> That message is (depending on what comes in [...]) much more helpful
> than just throwing a word "mtime" out and letting the reader figure
> out the rest ;-)
Yes, totally agreed.

Thanks, Taylor

Previous: Junio C HamanoNext: Junio C Hamano
Message 86 of 201 in “cruft packs”
  1. 00/17 cruft packsTaylor Blau, Nov 29, 2021
  2. 01/17 Documentation/technical: add cruft-packs.txtTaylor Blau, Nov 29, 2021
  3. Derrick StoleeDec 2, 2021
  4. Taylor BlauDec 3, 2021
  5. Elijah NewrenDec 4, 2021
  6. Taylor BlauDec 4, 2021
  7. 02/17 pack-mtimes: support reading .mtimes filesTaylor Blau, Nov 29, 2021
  8. Derrick StoleeDec 2, 2021
  9. brian m. carlsonDec 2, 2021
  10. Taylor BlauDec 3, 2021
  11. Taylor BlauJan 7, 2022
  12. 04/17 chunk-format.h: extract oid_version()Taylor Blau, Nov 29, 2021
  13. Derrick StoleeDec 2, 2021
  14. Taylor BlauDec 3, 2021
  15. Derrick StoleeDec 6, 2021
  16. 03/17 pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles'Taylor Blau, Nov 29, 2021
  17. 05/17 pack-mtimes: support writing pack .mtimes filesTaylor Blau, Nov 29, 2021
  18. Derrick StoleeDec 2, 2021
  19. Taylor BlauDec 3, 2021
  20. 09/17 reachable: add options to add_unseen_recent_objects_to_traversalTaylor Blau, Nov 29, 2021
  21. 13/17 builtin/repack.c: allow configuring cruft pack generationTaylor Blau, Nov 29, 2021
  22. 06/17 t/helper: add 'pack-mtimes' test-toolTaylor Blau, Nov 29, 2021
  23. Derrick StoleeDec 6, 2021
  24. Taylor BlauFeb 23, 2022
  25. 11/17 builtin/pack-objects.c: --cruft with expirationTaylor Blau, Nov 29, 2021
  26. Derrick StoleeDec 7, 2021
  27. Taylor BlauFeb 23, 2022
  28. 10/17 reachable: report precise timestamps from objects in cruft packsTaylor Blau, Nov 29, 2021
  29. 07/17 builtin/pack-objects.c: return from create_object_entry()Taylor Blau, Nov 29, 2021
  30. 08/17 builtin/pack-objects.c: --cruft without expirationTaylor Blau, Nov 29, 2021
  31. Derrick StoleeDec 6, 2021
  32. Taylor BlauMar 1, 2022
  33. Derrick StoleeDec 7, 2021
  34. Taylor BlauFeb 23, 2022
  35. 12/17 builtin/repack.c: support generating a cruft packTaylor Blau, Nov 29, 2021
  36. Junio C HamanoDec 5, 2021
  37. Taylor BlauMar 1, 2022
  38. Derrick StoleeDec 7, 2021
  39. Taylor BlauFeb 23, 2022
  40. 15/17 builtin/repack.c: add cruft packs to MIDX during geometric repackTaylor Blau, Nov 29, 2021
  41. 16/17 builtin/gc.c: conditionally avoid pruning objects via looseTaylor Blau, Nov 29, 2021
  42. 17/17 sha1-file.c: don't freshen cruft packsTaylor Blau, Nov 29, 2021
  43. 14/17 builtin/repack.c: use named flags for existing_packsTaylor Blau, Nov 29, 2021
  44. Junio C HamanoDec 3, 2021
  45. Taylor BlauDec 3, 2021
  46. Taylor BlauDec 3, 2021
  47. 00/17 cruft packsTaylor Blau, Mar 2, 2022
  48. 01/17 Documentation/technical: add cruft-packs.txtTaylor Blau, Mar 2, 2022
  49. 03/17 pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles'Taylor Blau, Mar 2, 2022
  50. 02/17 pack-mtimes: support reading .mtimes filesTaylor Blau, Mar 2, 2022
  51. Derrick StoleeMar 2, 2022
  52. Taylor BlauMar 2, 2022
  53. 04/17 chunk-format.h: extract oid_version()Taylor Blau, Mar 2, 2022
  54. 06/17 t/helper: add 'pack-mtimes' test-toolTaylor Blau, Mar 2, 2022
  55. 05/17 pack-mtimes: support writing pack .mtimes filesTaylor Blau, Mar 2, 2022
  56. 07/17 builtin/pack-objects.c: return from create_object_entry()Taylor Blau, Mar 2, 2022
  57. 08/17 builtin/pack-objects.c: --cruft without expirationTaylor Blau, Mar 2, 2022
  58. 09/17 reachable: add options to add_unseen_recent_objects_to_traversalTaylor Blau, Mar 2, 2022
  59. Derrick StoleeMar 2, 2022
  60. Taylor BlauMar 2, 2022
  61. 11/17 builtin/pack-objects.c: --cruft with expirationTaylor Blau, Mar 2, 2022
  62. Junio C HamanoMar 2, 2022
  63. Taylor BlauMar 2, 2022
  64. Derrick StoleeMar 2, 2022
  65. 10/17 reachable: report precise timestamps from objects in cruft packsTaylor Blau, Mar 2, 2022
  66. 13/17 builtin/repack.c: allow configuring cruft pack generationTaylor Blau, Mar 2, 2022
  67. 12/17 builtin/repack.c: support generating a cruft packTaylor Blau, Mar 2, 2022
  68. 14/17 builtin/repack.c: use named flags for existing_packsTaylor Blau, Mar 2, 2022
  69. 15/17 builtin/repack.c: add cruft packs to MIDX during geometric repackTaylor Blau, Mar 2, 2022
  70. 16/17 builtin/gc.c: conditionally avoid pruning objects via looseTaylor Blau, Mar 2, 2022
  71. 17/17 sha1-file.c: don't freshen cruft packsTaylor Blau, Mar 2, 2022
  72. Derrick StoleeMar 2, 2022
  73. Taylor BlauMar 2, 2022
  74. 00/17 cruft packsTaylor Blau, Mar 3, 2022
  75. 01/17 Documentation/technical: add cruft-packs.txtTaylor Blau, Mar 3, 2022
  76. Jonathan NiederMar 7, 2022
  77. Taylor BlauMar 22, 2022
  78. Jonathan NiederMar 22, 2022
  79. Taylor BlauMar 22, 2022
  80. Jonathan NiederMar 22, 2022
  81. Taylor BlauMar 23, 2022
  82. Taylor BlauMar 28, 2022
  83. Junio C HamanoMar 28, 2022
  84. Taylor BlauMar 28, 2022
  85. Junio C HamanoMar 29, 2022
  86. Taylor BlauMar 30, 2022
  87. Junio C HamanoMar 30, 2022
  88. Taylor BlauMar 30, 2022
  89. 03/17 pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles'Taylor Blau, Mar 3, 2022
  90. 02/17 pack-mtimes: support reading .mtimes filesTaylor Blau, Mar 3, 2022
  91. 04/17 chunk-format.h: extract oid_version()Taylor Blau, Mar 3, 2022
  92. Ævar Arnfjörð BjarmasonMar 3, 2022
  93. Taylor BlauMar 3, 2022
  94. Junio C HamanoMar 4, 2022
  95. 05/17 pack-mtimes: support writing pack .mtimes filesTaylor Blau, Mar 3, 2022
  96. Ævar Arnfjörð BjarmasonMar 3, 2022
  97. Taylor BlauMar 3, 2022
  98. Ævar Arnfjörð BjarmasonMar 4, 2022
  99. 06/17 t/helper: add 'pack-mtimes' test-toolTaylor Blau, Mar 3, 2022
  100. 07/17 builtin/pack-objects.c: return from create_object_entry()Taylor Blau, Mar 3, 2022
  101. 08/17 builtin/pack-objects.c: --cruft without expirationTaylor Blau, Mar 3, 2022
  102. 09/17 reachable: add options to add_unseen_recent_objects_to_traversalTaylor Blau, Mar 3, 2022
  103. 10/17 reachable: report precise timestamps from objects in cruft packsTaylor Blau, Mar 3, 2022
  104. 11/17 builtin/pack-objects.c: --cruft with expirationTaylor Blau, Mar 3, 2022
  105. 12/17 builtin/repack.c: support generating a cruft packTaylor Blau, Mar 3, 2022
  106. 13/17 builtin/repack.c: allow configuring cruft pack generationTaylor Blau, Mar 3, 2022
  107. 14/17 builtin/repack.c: use named flags for existing_packsTaylor Blau, Mar 3, 2022
  108. 16/17 builtin/gc.c: conditionally avoid pruning objects via looseTaylor Blau, Mar 3, 2022
  109. 17/17 sha1-file.c: don't freshen cruft packsTaylor Blau, Mar 3, 2022
  110. 15/17 builtin/repack.c: add cruft packs to MIDX during geometric repackTaylor Blau, Mar 3, 2022
  111. Derrick StoleeMar 3, 2022
  112. 00/17 cruft packsTaylor Blau, May 18, 2022
  113. 01/17 Documentation/technical: add cruft-packs.txtTaylor Blau, May 18, 2022
  114. Junio C HamanoMay 19, 2022
  115. 02/17 pack-mtimes: support reading .mtimes filesTaylor Blau, May 18, 2022
  116. Ævar Arnfjörð BjarmasonMay 19, 2022
  117. Junio C HamanoMay 19, 2022
  118. Ævar Arnfjörð BjarmasonMay 20, 2022
  119. Taylor BlauMay 20, 2022
  120. 03/17 pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles'Taylor Blau, May 18, 2022
  121. 06/17 t/helper: add 'pack-mtimes' test-toolTaylor Blau, May 18, 2022
  122. 07/17 builtin/pack-objects.c: return from create_object_entry()Taylor Blau, May 18, 2022
  123. 10/17 reachable: report precise timestamps from objects in cruft packsTaylor Blau, May 18, 2022
  124. 09/17 reachable: add options to add_unseen_recent_objects_to_traversalTaylor Blau, May 18, 2022
  125. 08/17 builtin/pack-objects.c: --cruft without expirationTaylor Blau, May 18, 2022
  126. Junio C HamanoMay 19, 2022
  127. Junio C HamanoMay 19, 2022
  128. Taylor BlauMay 20, 2022
  129. 11/17 builtin/pack-objects.c: --cruft with expirationTaylor Blau, May 18, 2022
  130. 12/17 builtin/repack.c: support generating a cruft packTaylor Blau, May 18, 2022
  131. Ævar Arnfjörð BjarmasonMay 19, 2022
  132. Taylor BlauMay 20, 2022
  133. 04/17 chunk-format.h: extract oid_version()Taylor Blau, May 18, 2022
  134. Ævar Arnfjörð BjarmasonMay 19, 2022
  135. 05/17 pack-mtimes: support writing pack .mtimes filesTaylor Blau, May 18, 2022
  136. 13/17 builtin/repack.c: allow configuring cruft pack generationTaylor Blau, May 18, 2022
  137. 14/17 builtin/repack.c: use named flags for existing_packsTaylor Blau, May 18, 2022
  138. 15/17 builtin/repack.c: add cruft packs to MIDX during geometric repackTaylor Blau, May 18, 2022
  139. Ævar Arnfjörð BjarmasonMay 19, 2022
  140. Taylor BlauMay 20, 2022
  141. 16/17 builtin/gc.c: conditionally avoid pruning objects via looseTaylor Blau, May 18, 2022
  142. 17/17 sha1-file.c: don't freshen cruft packsTaylor Blau, May 18, 2022
  143. Derrick StoleeMay 18, 2022
  144. Junio C HamanoMay 20, 2022
  145. Taylor BlauMay 20, 2022
  146. 0/2 Utility functions for duplicated pack(write) codeÆvar Arnfjörð Bjarmason, May 19, 2022
  147. 1/2 packfile API: add and use a pack_name_to_ext() utility functionÆvar Arnfjörð Bjarmason, May 19, 2022
  148. Junio C HamanoMay 19, 2022
  149. 2/2 hash API: add and use a hash_short_id_by_algo() functionÆvar Arnfjörð Bjarmason, May 19, 2022
  150. Junio C HamanoMay 19, 2022
  151. Ævar Arnfjörð BjarmasonMay 19, 2022
  152. Junio C HamanoMay 19, 2022
  153. Ævar Arnfjörð BjarmasonMay 19, 2022
  154. 00/17 cruft packsTaylor Blau, May 20, 2022
  155. 01/17 Documentation/technical: add cruft-packs.txtTaylor Blau, May 20, 2022
  156. 05/17 pack-mtimes: support writing pack .mtimes filesTaylor Blau, May 20, 2022
  157. 04/17 chunk-format.h: extract oid_version()Taylor Blau, May 20, 2022
  158. 03/17 pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles'Taylor Blau, May 20, 2022
  159. 06/17 t/helper: add 'pack-mtimes' test-toolTaylor Blau, May 20, 2022
  160. 07/17 builtin/pack-objects.c: return from create_object_entry()Taylor Blau, May 20, 2022
  161. 02/17 pack-mtimes: support reading .mtimes filesTaylor Blau, May 20, 2022
  162. Jonathan NiederMay 24, 2022
  163. rsbecker@nexbridge.comMay 24, 2022
  164. Taylor BlauMay 24, 2022
  165. rsbecker@nexbridge.comMay 24, 2022
  166. Taylor BlauMay 25, 2022
  167. rsbecker@nexbridge.comMay 25, 2022
  168. adding new 32-bit on-disk (unsigned) timestamp formats (was: [PATCH v5 02/17] pack-mtimes: support reading .mtimes files)Ævar Arnfjörð Bjarmason, May 25, 2022
  169. Derrick StoleeMay 25, 2022
  170. Taylor BlauMay 25, 2022
  171. Ævar Arnfjörð BjarmasonMay 26, 2022
  172. Taylor BlauMay 26, 2022
  173. Taylor BlauMay 24, 2022
  174. Jonathan NiederMay 25, 2022
  175. Taylor BlauMay 25, 2022
  176. rsbecker@nexbridge.comMay 25, 2022
  177. Taylor BlauMay 25, 2022
  178. Taylor BlauMay 25, 2022
  179. Junio C HamanoMay 26, 2022
  180. Andreas SchwabJun 1, 2023
  181. 08/17 builtin/pack-objects.c: --cruft without expirationTaylor Blau, May 20, 2022
  182. 10/17 reachable: report precise timestamps from objects in cruft packsTaylor Blau, May 20, 2022
  183. 09/17 reachable: add options to add_unseen_recent_objects_to_traversalTaylor Blau, May 20, 2022
  184. 11/17 builtin/pack-objects.c: --cruft with expirationTaylor Blau, May 20, 2022
  185. 12/17 builtin/repack.c: support generating a cruft packTaylor Blau, May 20, 2022
  186. 13/17 builtin/repack.c: allow configuring cruft pack generationTaylor Blau, May 20, 2022
  187. 15/17 builtin/repack.c: add cruft packs to MIDX during geometric repackTaylor Blau, May 20, 2022
  188. 14/17 builtin/repack.c: use named flags for existing_packsTaylor Blau, May 20, 2022
  189. 17/17 sha1-file.c: don't freshen cruft packsTaylor Blau, May 20, 2022
  190. 16/17 builtin/gc.c: conditionally avoid pruning objects via looseTaylor Blau, May 20, 2022
  191. René ScharfeJun 19, 2022
  192. Junio C HamanoJun 21, 2022
  193. Ævar Arnfjörð BjarmasonMay 21, 2022
  194. Jonathan NiederMay 24, 2022
  195. Taylor BlauMay 24, 2022
  196. Ævar Arnfjörð BjarmasonMay 24, 2022
  197. Taylor BlauMay 24, 2022
  198. Jonathan NiederMay 25, 2022
  199. Derrick StoleeMay 25, 2022
  200. Taylor BlauMay 25, 2022
  201. Ævar Arnfjörð BjarmasonMay 26, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.