git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git pull & git gc

From
Duy Nguyen <pclouds@gmail.com>
Date
Mar 19, 2015, 00:31 UTC
Message-ID
<CACsJy8AJfx8id+oHmhsLODBJeNLgRqD+5B295TtS+c-hrC5raQ@mail.gmail.com>
In-Reply-To
<20150318210458.GA3376@peff.net>
On Thu, Mar 19, 2015 at 4:04 AM, Jeff King <peff@peff.net> wrote:
Show 36 quoted lines
> On Wed, Mar 18, 2015 at 02:58:15PM +0000, John Keeping wrote:
>
>> On Wed, Mar 18, 2015 at 09:41:59PM +0700, Duy Nguyen wrote:
>> > On Wed, Mar 18, 2015 at 9:33 PM, Duy Nguyen <pclouds@gmail.com> wrote:
>> > > If not, I made some mistake in analyzing this and we'll start again.
>> >
>> > I did make one mistake, the first "gc" should have reduced the number
>> > of loose objects to zero. Why didn't it.?  I'll come back to this
>> > tomorrow if nobody finds out first :)
>>
>> Most likely they are not referenced by anything but are younger than 2
>> weeks.
>>
>> I saw a similar issue with automatic gc triggering after every operation
>> when I did something equivalent to:
>>
>>       git add <lots of files>
>>       git commit
>>       git reset --hard HEAD^
>>
>> which creates a log of unreachable objects which are not old enough to
>> be pruned.
>
> Yes, this is almost certainly the problem. Though to be pedantic, the
> command above will still have a reflog entry, so the objects will be
> reachable (and packed). But there are other variants that don't leave
> the objects reachable from even reflogs.
>
> I don't know if there is an easy way around this. Auto-gc's object count
> is making the assumption that running the gc will reduce the number of
> objects, but obviously it does not always do so. We could do a more
> thorough check and find the number of actual packable and prune-able
> objects. The "prune-able" part of that is easy; just omit objects from
> the count that are newer than 2 weeks. But "packable" is expensive. You
> would have to compute reachability by walking from the tips. That can
> take tens of seconds on a large repo.

Or we could count/estimate the number of loose objects again after repack/prune. Then we can maybe have a way to prevent the next gc that we know will not improve the situation anyway. One option is pack unreachable objects in the second pack. This would stop the next gc, but that would screw prune up because st_mtime info is gone.. Maybe we just save a file to tell gc to ignore the number of loose objects until after a specific date.

-- 
Duy
Previous: Jeff KingNext: Jeff King
Message 8 of 21 in “git pull & git gc”
  1. Дилян ПалаузовMar 18, 2015
  2. Duy NguyenMar 18, 2015
  3. Дилян ПалаузовMar 18, 2015
  4. Duy NguyenMar 18, 2015
  5. Duy NguyenMar 18, 2015
  6. John KeepingMar 18, 2015
  7. Jeff KingMar 18, 2015
  8. Duy NguyenMar 19, 2015
  9. Jeff KingMar 19, 2015
  10. Mike HommeyMar 19, 2015
  11. Jeff KingMar 19, 2015
  12. Mike HommeyMar 19, 2015
  13. Junio C HamanoMar 19, 2015
  14. Jeff KingMar 19, 2015
  15. Duy NguyenMar 19, 2015
  16. Jeff KingMar 19, 2015
  17. Duy NguyenMar 19, 2015
  18. Jeff KingMar 19, 2015
  19. Duy NguyenMar 19, 2015
  20. Дилян ПалаузовMar 18, 2015
  21. Jeff KingMar 18, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.