git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/7] midx.c: prevent `expire` from removing the cruft pack

From
Derrick Stolee <derrickstolee@github.com>
Date
Sep 22, 2022, 14:08 UTC
Message-ID
<37ffe1a6-bed9-70ce-2c66-b7a2d8c45ba9@github.com>
In-Reply-To
<3ae9903d2df491e291ab975c56ec78aa13d95655.1663638929.git.me@ttaylorr.com>
On 9/19/2022 9:55 PM, Taylor Blau wrote:
> The `expire` sub-command unlinks any packs that are (a) contained in the
> MIDX, but (b) have no objects referenced by the MIDX.
It is important to note that this can only happen if all objects in
the pack have duplicates in other pack-files.
Show 7 quoted lines
> This sub-command ignores `.keep` packs, which remain on-disk even if
> they have no objects referenced by the MIDX. Cruft packs, however,
> aren't given the same treatment: if none of the objects contained in the
> cruft pack are selected from the cruft pack by the MIDX, then the cruft
> pack is eligible to be expired.
> 
> This is less than desireable, since the cruft pack has important

s/desireable/desirable/ (according to my spell-checker)

Show 9 quoted lines
> metadata about the individual object mtimes, which is useful to
> determine how quickly an object should age out of the repository when
> pruning.
>
> Ordinarily, we wouldn't expect the contents of a cruft pack to
> duplicated across non-cruft packs (and we'd expect to see the MIDX
> select all cruft objects from other sources even less often). But
> nonetheless, it is still possible to trick the `expire` sub-command into
> removing the `.mtimes` file in this circumstance.

I was initially unconvinced that this scenario was super-critical to keeping the .mtimes file, but I was able to think of cases where objects are duplicated out of the cruft pack due to de-thinning or otherwise inefficiently packing unreachable objects into these other pack-files.

Thanks, -Stolee

Previous: Taylor BlauNext: Taylor Blau
Message 5 of 11 in “midx: ignore cruft pack with `repack`, `expire`”
  1. 0/7 midx: ignore cruft pack with `repack`, `expire`Taylor Blau, Sep 20, 2022
  2. 1/7 Documentation/git-multi-pack-index.txt: fix typoTaylor Blau, Sep 20, 2022
  3. 2/7 Documentation/git-multi-pack-index.txt: clarify expire behaviorTaylor Blau, Sep 20, 2022
  4. 3/7 midx.c: prevent `expire` from removing the cruft packTaylor Blau, Sep 20, 2022
  5. Derrick StoleeSep 22, 2022
  6. 4/7 midx.c: avoid cruft packs with `repack --batch-size=0`Taylor Blau, Sep 20, 2022
  7. 5/7 midx.c: replace `xcalloc()` with `CALLOC_ARRAY()`Taylor Blau, Sep 20, 2022
  8. 6/7 midx.c: remove unnecessary loop conditionTaylor Blau, Sep 20, 2022
  9. 7/7 midx.c: avoid cruft packs with non-zero `repack --batch-size`Taylor Blau, Sep 20, 2022
  10. Christian CouderSep 20, 2022
  11. Derrick StoleeSep 22, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.