git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Why repository grows after "git gc"? / Purpose of *.keep files?

From
Junio C Hamano <gitster@pobox.com>
Date
May 12, 2008, 23:53 UTC
Message-ID
<7vod7bw03a.fsf@gitster.siamese.dyndns.org>
In-Reply-To
<20080512234906.GX29038@spearce.org>
"Shawn O. Pearce" <spearce@spearce.org> writes:
Show 46 quoted lines
> David Tweed <david.tweed@gmail.com> wrote:
>> On Mon, May 12, 2008 at 4:52 PM, Teemu Likonen <tlikonen@iki.fi> wrote:
>> > Teemu Likonen wrote (2008-05-12 15:29 +0300):
>> > Probably a crazy idea: What if "gc --aggressive" first removed *.keep
>> > files and after packing and garbage-collecting and whatever it does it
>> > would add a .keep file for the newly created pack?
>> 
>> My understanding is that the repacking with -a redoes the computation
>> to repack ALL the objects in every pack and loose objects,
>
> No.  -a means repack all objects in all packs which do not have a
> .keep on them.  Without -a we only repack loose objects.
>
>> whereas
>> what would be preferred is to try to delta new objects (loose and
>> packed) against the existing .keep pack (extending it with the new
>> objects) but not trying to re-deltify objects in the .keep pack.
>
> We cannot do that.  Deltas in pack A may not reference base objects
> in pack B.  This is a simplification rule that prevents us from
> needing to worry about damaging a pack when we repack and delete
> another pack.
>
>> This
>> is because .keep files are primarily for those who are cloning onto a
>> machine that isn't powerful (maybe even a laptop/palmtop) but who are
>> cloning from a powerful server, so that you wouldn't necessarily want
>> to apply your strategy unconditionally.
>
> Yes, sort of.  We use .keep for two reasons:
>
>   - As a "lock file" to prevent a pack that was just created by a
>     git-fetch or git-recieve-pack from being deleted by a concurrent
>     git-repack before the objects it contains are linked into the
>     refs space and thus considered reachable;
>
>   - As a way to avoid _huge_ packs (say >1G) that would take a lot
>     of disk IO just to copy with 100% delta reuse from an old pack
>     to a new pack each time the user runs git-gc.
>
> I think git-clone marking a 150M linux-2.6 pack with .keep is wrong;
> most users working with the linux-2.6 sources have sufficient
> hardware to deal with the disk IO required to copy that with 100%
> delta reuse.  But I have a repository at day-job with a 600M pack,
> that's starting to head into the realm where git-gc while running
> on battery on a laptop would prefer to have that .keep.

Perhaps clone can decide to keep the .keep file depending on the size of the pack then?

Previous: Shawn O. PearceNext: Shawn O. Pearce
Message 29 of 35 in “Why repository grows after "git gc"? / Purpose of *.keep files?”
  1. Teemu LikonenMay 12, 2008
  2. Teemu LikonenMay 12, 2008
  3. Johannes SchindelinMay 12, 2008
  4. Teemu LikonenMay 12, 2008
  5. Nicolas PitreMay 12, 2008
  6. Teemu LikonenMay 12, 2008
  7. Nicolas PitreMay 12, 2008
  8. Govind SalinasMay 12, 2008
  9. Nicolas PitreMay 12, 2008
  10. Govind SalinasMay 12, 2008
  11. Teemu LikonenMay 12, 2008
  12. Mike HommeyMay 12, 2008
  13. Mike HommeyMay 12, 2008
  14. Shawn O. PearceMay 13, 2008
  15. Mike HommeyMay 13, 2008
  16. Nicolas PitreMay 14, 2008
  17. Junio C HamanoMay 14, 2008
  18. Juergen RuehleMay 14, 2008
  19. Nicolas PitreMay 14, 2008
  20. Junio C HamanoMay 14, 2008
  21. Linus TorvaldsMay 14, 2008
  22. Linus TorvaldsMay 14, 2008
  23. Nicolas PitreMay 14, 2008
  24. Linus TorvaldsMay 14, 2008
  25. A Large Angry SCMMay 14, 2008
  26. Nicolas PitreMay 12, 2008
  27. David TweedMay 12, 2008
  28. Shawn O. PearceMay 12, 2008
  29. Junio C HamanoMay 12, 2008
  30. Shawn O. PearceMay 13, 2008
  31. Paolo BonziniMay 13, 2008
  32. Shawn O. PearceMay 13, 2008
  33. Teemu LikonenMay 13, 2008
  34. Stephen R. van den BergMay 13, 2008
  35. Teemu LikonenMay 14, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.