git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: tb/cruft-packs (was Re: What's cooking in git.git (Mar 2022, #01; Thu, 3))

From
Derrick Stolee <derrickstolee@github.com>
Date
Mar 7, 2022, 20:51 UTC
Message-ID
<332f9b54-edfb-50e7-46a7-2e46d3b574e4@github.com>
In-Reply-To
<YiZoojeTodMr+Ypw@google.com>
On 3/7/2022 3:18 PM, Jonathan Nieder wrote:
Show 30 quoted lines
> Derrick Stolee wrote:
>> On 3/7/2022 1:18 PM, Taylor Blau wrote:
>>> On Mon, Mar 07, 2022 at 10:06:00AM -0800, Jonathan Nieder wrote:
> 
>>>>  2. Marking this as a repository format extension so it doesn't interact
>>>>     poorly with Git implementations (including older versions of Git
>>>>     itself) that are not aware of the new feature
>>>
>>> The design of cruft packs was done intentionally to avoid needing a
>>> format extension. The cruft pack is "just a pack" to any older version
>>> of Git. The only thing an older version of Git wouldn't understand is
>>> how to interpret the .mtimes file. But that's no different than the
>>> current behavior without cruft packs, where any unreachable object
>>> inherits the mtime of its containing pack.
>>>
>>> So an older version of Git might prune a different set of objects than a
>>> version that understands cruft packs depending on the contents of the
>>> .mtimes file, the mtime of the cruft pack, and the width of the grace
>>> period. But I think by downgrading you are more or less buying into the
>>> existing behavior. So I don't think there is a compelling reason to
>>> introduce a format extension here.
>>
>> In particular, older versions would first explode unreachable objects
>> out of the cruft pack and into loose objects before expiring any of
>> them based on the loose object mtime. There is no risk here of causing
>> problems with older versions of Git and does not need an extension.
> 
> Surely when older and versions are acting on the same repository, they
> would fight by exploding out unreachable objects, packing them back
> into a cruft pack, etc, no?

You are referring to a situation where there are multiple possible versions responsible for maintaining a repository. Git does not support parallel writers doing significant updates like full repacks and GCs and instead relies on the user to control the concurrency there. The standard we keep to is that parallel readers can still access the repo during this time.

If someone was running a case where they had these parallel maintenance processes, then they would already be risking failure with existing features (though actually in the case of the old versions breaking the new ones): what if the new/old versions differ in their understanding of the commit-graph? The old one could remove commits but not update the commit-graph, leaving extra commits in that file that the new one would fail to verify. How about the multi-pack-index? The new version would try loading objects from missing pack-files since the old version deleted those packs without updating the multi-pack-index.

At least in the cruft packs the worst case is that no objects are ever expired because they are toggling between loose objects and cruft packs.

Thanks, -Stolee

Previous: Jonathan NiederNext: Junio C Hamano
Message 14 of 22 in “What's cooking in git.git (Mar 2022, #01; Thu, 3)”
  1. Junio C HamanoMar 4, 2022
  2. ab/plug-random-leaks (was Re: What's cooking in git.git (Mar 2022, #01; Thu, 3))Derrick Stolee, Mar 4, 2022
  3. Ævar Arnfjörð BjarmasonMar 4, 2022
  4. tests: test show --word-diff --color-movedMichael J Gruber, Mar 17, 2022
  5. 0/2 diff.c: fix a recent memory leak regressionÆvar Arnfjörð Bjarmason, Mar 17, 2022
  6. 2/2 diff.c: fix a double-free regression in a18d66cefbÆvar Arnfjörð Bjarmason, Mar 17, 2022
  7. 1/2 tests: demonstrate "show --word-diff --color-moved" regressionÆvar Arnfjörð Bjarmason, Mar 17, 2022
  8. Junio C HamanoMar 17, 2022
  9. tb/cruft-packs (was Re: What's cooking in git.git (Mar 2022, #01; Thu, 3))Derrick Stolee, Mar 4, 2022
  10. Jonathan NiederMar 7, 2022
  11. Taylor BlauMar 7, 2022
  12. Derrick StoleeMar 7, 2022
  13. Jonathan NiederMar 7, 2022
  14. Derrick StoleeMar 7, 2022
  15. Junio C HamanoMar 7, 2022
  16. Taylor BlauMar 8, 2022
  17. Junio C HamanoMar 8, 2022
  18. Taylor BlauMar 8, 2022
  19. jc/stash-drop (was: What's cooking in git.git (Mar 2022, #01; Thu, 3))Ævar Arnfjörð Bjarmason, Mar 5, 2022
  20. Junio C HamanoMar 7, 2022
  21. ds/commit-graph-gen-v2-fixes (was Re: What's cooking in git.git (Mar 2022, #01; Thu, 3))Derrick Stolee, Mar 7, 2022
  22. Junio C HamanoMar 7, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.