git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 01/10] packfile: introduce 'find_kept_pack_entry()'

From
Taylor Blau <me@ttaylorr.com>
Date
Jan 29, 2021, 18:38 UTC
Message-ID
<YBRWPnTxeFPBft7y@nand.local>
In-Reply-To
<xmqqwnvwtqu1.fsf@gitster.c.googlers.com>
On Thu, Jan 28, 2021 at 06:33:10PM -0800, Junio C Hamano wrote:
Show 19 quoted lines
> Taylor Blau <me@ttaylorr.com> writes:
>
> > Future callers will want a function to fill a 'struct pack_entry' for a
> > given object id but _only_ from its position in any kept pack(s). They
> > could accomplish this by calling 'find_pack_entry()' and checking
> > whether the found pack is kept or not, but this is insufficient, since
> > there may be duplicate objects (and the mru cache makes it unpredictable
> > which variant we'll get).
>
> I wonder if we eventually need a callback interface to walk _all_
> pack entries for a given object, so that "I am only interested in
> instances in kept packs" will be under total control of the callers.
> As it stands, it is "just grab any one that is in a kept pack, any
> one of them is fine", which is almost just of as narrow utility as
> the original's "just grab the first one---any one of them is fine",
> the latter of which is "insufficient" as the log message says.
>
> But this (in the context of the remainder of the series) might be
> sufficient, at least for now.

As you note, it's more about "can I find this object in any kept pack (of a certain kind)" versus, "show me this object in a pack" (and hope that if it appears in a kept pack, that that's the copy that is picked).

Show 7 quoted lines
> > Teach this new function to treat the two different kinds of kept packs
> > (on disk ones with .keep files, as well as in-core ones which are set by
> > manually poking the 'pack_keep_in_core' bit) separately. This will
> > become important for callers that only want to respect a certain kind of
> > kept pack.
>
> Or maybe not ;-)

:-). The difference here is that we will only want to stop the traversal at packs which are considered to be stable from the perspective of a geometric repack.

We mark those packs as "stable" by setting their in-core kept bit, but we don't write ".keep" files (which would make them on-disk kept). The latter is up to the user, not us.

Show 7 quoted lines
> If there are notable relationship between on-disk and in-core kept
> packs (e.g. "the set of on-disk kept packs is a subset of in-core
> kept packs", "usually on-disk kept packs get in-core kept bit upon
> their packed_git instances are populated, but we can drop the bit at
> runtime, so on-disk and in-core are pretty much independent and
> there is no notable relationship"), it must be explained upfront to
> help the reader form a sensible world view.

Unfortunately, I don't think that there is a sensible world-view here to be formed. Honestly, the distinction between .keep packs and in-core kept packs is incredibly narrow, and I find our separate handling of them awkward and error-prone.

But, it is sort of what you'd want here (i.e., a way to mark all objects in a pack as ignored without actually writing the physical file that says "ignore all objects in this pack").

Thanks, Taylor

Previous: Junio C HamanoNext: Jeff King
Message 9 of 120 in “repack: support repacking into a geometric sequence”
  1. 00/10 repack: support repacking into a geometric sequenceTaylor Blau, Jan 19, 2021
  2. 02/10 revision: learn '--no-kept-objects'Taylor Blau, Jan 19, 2021
  3. Junio C HamanoJan 29, 2021
  4. Taylor BlauJan 29, 2021
  5. 01/10 packfile: introduce 'find_kept_pack_entry()'Taylor Blau, Jan 19, 2021
  6. Derrick StoleeJan 20, 2021
  7. Taylor BlauJan 20, 2021
  8. Junio C HamanoJan 29, 2021
  9. Taylor BlauJan 29, 2021
  10. Jeff KingJan 29, 2021
  11. Junio C HamanoJan 29, 2021
  12. 07/10 packfile: add kept-pack cache for find_kept_pack_entry()Taylor Blau, Jan 19, 2021
  13. 06/10 pack-objects: rewrite honor-pack-keep logicTaylor Blau, Jan 19, 2021
  14. 05/10 p5303: measure time to repack with keepTaylor Blau, Jan 19, 2021
  15. Junio C HamanoJan 29, 2021
  16. Jeff KingJan 29, 2021
  17. p5303: avoid sed GNU-ismJeff King, Jan 29, 2021
  18. Eric SunshineJan 29, 2021
  19. Jeff KingJan 29, 2021
  20. Eric SunshineJan 29, 2021
  21. Taylor BlauJan 29, 2021
  22. Junio C HamanoJan 29, 2021
  23. Jeff KingJan 29, 2021
  24. Junio C HamanoJan 29, 2021
  25. 08/10 builtin/pack-objects.c: teach '--keep-pack-stdin'Taylor Blau, Jan 19, 2021
  26. 10/10 builtin/repack.c: add '--geometric' optionTaylor Blau, Jan 19, 2021
  27. 03/10 builtin/pack-objects.c: learn '--assume-kept-packs-closed'Taylor Blau, Jan 19, 2021
  28. Junio C HamanoJan 29, 2021
  29. Jeff KingJan 29, 2021
  30. Taylor BlauJan 29, 2021
  31. Jeff KingJan 29, 2021
  32. Taylor BlauJan 29, 2021
  33. Jeff KingJan 29, 2021
  34. Junio C HamanoJan 29, 2021
  35. Taylor BlauJan 29, 2021
  36. Taylor BlauFeb 2, 2021
  37. Jeff KingJan 29, 2021
  38. Junio C HamanoJan 29, 2021
  39. Junio C HamanoJan 29, 2021
  40. Jeff KingJan 29, 2021
  41. Taylor BlauJan 29, 2021
  42. Jeff KingJan 29, 2021
  43. Junio C HamanoJan 29, 2021
  44. 09/10 builtin/repack.c: extract loose object handlingTaylor Blau, Jan 19, 2021
  45. Derrick StoleeJan 20, 2021
  46. Taylor BlauJan 20, 2021
  47. Derrick StoleeJan 20, 2021
  48. Junio C HamanoJan 21, 2021
  49. 04/10 p5303: add missing &&-chainsTaylor Blau, Jan 19, 2021
  50. Derrick StoleeJan 20, 2021
  51. 0/8 repack: support repacking into a geometric sequenceTaylor Blau, Feb 4, 2021
  52. 4/8 p5303: add missing &&-chainsTaylor Blau, Feb 4, 2021
  53. 3/8 builtin/pack-objects.c: add '--stdin-packs' optionTaylor Blau, Feb 4, 2021
  54. Jeff KingFeb 16, 2021
  55. Taylor BlauFeb 17, 2021
  56. Jeff KingFeb 17, 2021
  57. 2/8 revision: learn '--no-kept-objects'Taylor Blau, Feb 4, 2021
  58. Jeff KingFeb 16, 2021
  59. Taylor BlauFeb 17, 2021
  60. 1/8 packfile: introduce 'find_kept_pack_entry()'Taylor Blau, Feb 4, 2021
  61. Jeff KingFeb 16, 2021
  62. Taylor BlauFeb 16, 2021
  63. 5/8 p5303: measure time to repack with keepTaylor Blau, Feb 4, 2021
  64. Jeff KingFeb 16, 2021
  65. Jeff KingFeb 17, 2021
  66. Taylor BlauFeb 17, 2021
  67. Jeff KingFeb 17, 2021
  68. 7/8 packfile: add kept-pack cache for find_kept_pack_entry()Taylor Blau, Feb 4, 2021
  69. Jeff KingFeb 17, 2021
  70. Taylor BlauFeb 17, 2021
  71. Jeff KingFeb 17, 2021
  72. Taylor BlauFeb 17, 2021
  73. Jeff KingFeb 17, 2021
  74. 8/8 builtin/repack.c: add '--geometric' optionTaylor Blau, Feb 4, 2021
  75. Jeff KingFeb 17, 2021
  76. Taylor BlauFeb 17, 2021
  77. 6/8 builtin/pack-objects.c: rewrite honor-pack-keep logicTaylor Blau, Feb 4, 2021
  78. Jeff KingFeb 17, 2021
  79. Taylor BlauFeb 17, 2021
  80. Jeff KingFeb 17, 2021
  81. Jeff KingFeb 17, 2021
  82. Jeff KingFeb 17, 2021
  83. 0/8 repack: support repacking into a geometric sequenceTaylor Blau, Feb 18, 2021
  84. 1/8 packfile: introduce 'find_kept_pack_entry()'Taylor Blau, Feb 18, 2021
  85. 2/8 revision: learn '--no-kept-objects'Taylor Blau, Feb 18, 2021
  86. 4/8 p5303: add missing &&-chainsTaylor Blau, Feb 18, 2021
  87. 3/8 builtin/pack-objects.c: add '--stdin-packs' optionTaylor Blau, Feb 18, 2021
  88. 6/8 builtin/pack-objects.c: rewrite honor-pack-keep logicTaylor Blau, Feb 18, 2021
  89. 5/8 p5303: measure time to repack with keepTaylor Blau, Feb 18, 2021
  90. 7/8 packfile: add kept-pack cache for find_kept_pack_entry()Taylor Blau, Feb 18, 2021
  91. 8/8 builtin/repack.c: add '--geometric' optionTaylor Blau, Feb 18, 2021
  92. Jeff KingFeb 23, 2021
  93. Taylor BlauFeb 23, 2021
  94. Jeff KingFeb 23, 2021
  95. 0/8 repack: support repacking into a geometric sequenceTaylor Blau, Feb 23, 2021
  96. 1/8 packfile: introduce 'find_kept_pack_entry()'Taylor Blau, Feb 23, 2021
  97. 2/8 revision: learn '--no-kept-objects'Taylor Blau, Feb 23, 2021
  98. 3/8 builtin/pack-objects.c: add '--stdin-packs' optionTaylor Blau, Feb 23, 2021
  99. Junio C HamanoFeb 23, 2021
  100. Jeff KingFeb 23, 2021
  101. 4/8 p5303: add missing &&-chainsTaylor Blau, Feb 23, 2021
  102. 6/8 builtin/pack-objects.c: rewrite honor-pack-keep logicTaylor Blau, Feb 23, 2021
  103. 5/8 p5303: measure time to repack with keepTaylor Blau, Feb 23, 2021
  104. 8/8 builtin/repack.c: add '--geometric' optionTaylor Blau, Feb 23, 2021
  105. Junio C HamanoFeb 24, 2021
  106. Junio C HamanoFeb 24, 2021
  107. Taylor BlauMar 4, 2021
  108. Taylor BlauMar 4, 2021
  109. 7/8 packfile: add kept-pack cache for find_kept_pack_entry()Taylor Blau, Feb 23, 2021
  110. Jeff KingFeb 23, 2021
  111. Junio C HamanoFeb 23, 2021
  112. Jeff KingFeb 23, 2021
  113. Martin FickFeb 23, 2021
  114. Taylor BlauFeb 23, 2021
  115. Martin FickFeb 23, 2021
  116. Jeff KingFeb 23, 2021
  117. Martin FickFeb 23, 2021
  118. Jeff KingFeb 23, 2021
  119. Martin FickFeb 24, 2021
  120. Jeff KingFeb 26, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.