git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git gc --aggressive led to about 40 times slower "git log --raw"

From
David Kastrup <dak@gnu.org>
Date
Feb 20, 2014, 18:07 UTC
Message-ID
<87ppmhr72d.fsf@fencepost.gnu.org>
In-Reply-To
<87y515r9wb.fsf@fencepost.gnu.org>
David Kastrup <dak@gnu.org> writes:
Show 20 quoted lines
> David Kastrup <dak@gnu.org> writes:
>
>> Duy Nguyen <pclouds@gmail.com> writes:
>>
>> Something's _really_ fishy about that cache behavior.  Note that the
>> _system_ time goes up considerably, not just user time.  Since the
>> packs are zlib-packed, it's reasonable that more I/O time is also
>> associated with more user time and it is well possible that the user
>> time increase is entirely explainable by the larger amount of
>> compressed data to access.
>>
>> But this stinks.
>
> And an obvious contender for the stinking is that the "LRU" scheme used
> here is _strictly_ freeing memory based on which cache entry has been
> _created_ the longest time ago, not which cache entry has been
> _accessed_ the longest time ago.  Which means a pure round-robin
> strategy for freeing memory rather than LRU.
>
> Let's see what happens when changing this.

Not much. With any cache size, using a "true" LRU scheme does not buy more than 2%. On the other hand, increasing core.deltaBaseCacheLimit from its default of 16m to 128m in the config file results in the following difference (with default #define MAX_DELTA_CACHE (256)):

dak@lola:/usr/local/tmp/emacs$ time ../git/git blame src/xdisp.c >/dev/null

real 1m17.446s user 0m30.696s sys 0m46.332s dak@lola:/usr/local/tmp/emacs$ time ../git/git blame src/xdisp.c >/dev/null

real 0m27.519s user 0m20.248s sys 0m7.156s

So it would seem that the default available cache slots are not utilized anyway when operating on this file (about 1MB in size) with the default of core.deltaBaseCacheLimit.

It is still irritating that the performance drops quite a bit with a considerably larger number of cache slots.

-- 
David Kastrup
Previous: David KastrupNext: Junio C Hamano
Message 17 of 31 in “git gc --aggressive led to about 40 times slower "git log --raw"”
  1. Christian JaegerFeb 18, 2014
  2. David KastrupFeb 18, 2014
  3. Duy NguyenFeb 18, 2014
  4. David KastrupFeb 18, 2014
  5. Jonathan NiederFeb 18, 2014
  6. Junio C HamanoFeb 18, 2014
  7. Duy NguyenFeb 18, 2014
  8. Junio C HamanoFeb 19, 2014
  9. Duy NguyenFeb 19, 2014
  10. Philippe VaucherFeb 19, 2014
  11. David KastrupFeb 19, 2014
  12. Duy NguyenFeb 19, 2014
  13. Duy NguyenFeb 19, 2014
  14. Christian JaegerFeb 20, 2014
  15. David KastrupFeb 20, 2014
  16. David KastrupFeb 20, 2014
  17. David KastrupFeb 20, 2014
  18. Junio C HamanoFeb 19, 2014
  19. Duy NguyenFeb 20, 2014
  20. Christian JaegerFeb 21, 2014
  21. Junio C HamanoFeb 21, 2014
  22. Duy NguyenFeb 21, 2014
  23. Junio C HamanoFeb 21, 2014
  24. Philippe VaucherFeb 24, 2014
  25. Duy NguyenFeb 22, 2014
  26. David KastrupFeb 22, 2014
  27. David KastrupFeb 22, 2014
  28. Duy NguyenFeb 22, 2014
  29. Duy NguyenFeb 22, 2014
  30. Andreas SchwabFeb 22, 2014
  31. Christian JaegerFeb 18, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.