git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git pull & git gc

From
Mike Hommey <mh@glandium.org>
Date
Mar 19, 2015, 04:26 UTC
Message-ID
<20150319042617.GA1072@glandium.org>
In-Reply-To
<20150319041453.GB29437@peff.net>
On Thu, Mar 19, 2015 at 12:14:53AM -0400, Jeff King wrote:
Show 24 quoted lines
> On Thu, Mar 19, 2015 at 11:01:17AM +0900, Mike Hommey wrote:
> 
> > > I don't think packing the unreachables is a good plan. They just end up
> > > accumulating then, and they never expire, because we keep refreshing
> > > their mtime at each pack (unless you pack them once and then leave them
> > > to expire, but then you end up with a large number of packs).
> > 
> > Note, sometimes I wish unreachables were packed. Recently, I ended up in
> > a situation where running gc created something like 3GB of data as per
> > du, because I suddenly had something like 600K unreachable objects, each
> > of them, as a loose object, taking at least 4K on disk. This made my
> > .git take 5GB instead of 2GB. That surely didn't feel like garbage
> > collection.
> 
> That's definitely a thing that happens, but it is a bit of a corner
> case. It's unusual to have such a large number of unreferenced objects
> all at once.
> 
> I don't suppose you happen to remember the details, but would a lower
> expiration time (e.g., 1 day or 1 hour) have made all of those objects
> go away? Or were they really from some extremely recent event (of
> course, "event" here might just have been "I did a full repack right
> before rewriting history" which would freshen the mtimes on everything
> in the pack).

Unfortunately, I don't know the exact details. But yes, I guess a lower expiration time might have helped.

Show 6 quoted lines
> Certainly the "loosening" behavior for unreachable objects has corner
> cases like this, and they suck when you hit one. Leaving the objects
> packed would be better, but IMHO is not a viable alternative unless
> somebody comes up with a plan for segregating the "old" objects in a way
> that they actually expire eventually, and don't just keep getting
> repacked and freshened over and over.

It sure is a corner case, otoh, when it happens, every single git operation calls git gc --auto, which happily spends 5 minutes sucking CPU to end up doing nothing in practice. And add more salt on the injury if you are on battery

6700 loose objects seems easy to reach on a repo with 6M objects...
Mike
Previous: Jeff KingNext: Junio C Hamano
Message 12 of 21 in “git pull & git gc”
  1. Дилян ПалаузовMar 18, 2015
  2. Duy NguyenMar 18, 2015
  3. Дилян ПалаузовMar 18, 2015
  4. Duy NguyenMar 18, 2015
  5. Duy NguyenMar 18, 2015
  6. John KeepingMar 18, 2015
  7. Jeff KingMar 18, 2015
  8. Duy NguyenMar 19, 2015
  9. Jeff KingMar 19, 2015
  10. Mike HommeyMar 19, 2015
  11. Jeff KingMar 19, 2015
  12. Mike HommeyMar 19, 2015
  13. Junio C HamanoMar 19, 2015
  14. Jeff KingMar 19, 2015
  15. Duy NguyenMar 19, 2015
  16. Jeff KingMar 19, 2015
  17. Duy NguyenMar 19, 2015
  18. Jeff KingMar 19, 2015
  19. Duy NguyenMar 19, 2015
  20. Дилян ПалаузовMar 18, 2015
  21. Jeff KingMar 18, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.