git/list[1] front-page[2] threads[3] people[4] search[5] about
 

git gc --aggressive led to about 40 times slower "git log --raw"

From
CJChristian Jaeger <chrjae@gmail.com>
Date
Feb 18, 2014, 07:25 UTC
Message-ID
<CAEjYwfU==yYtQBDzZzEPdvbqz1N=gZtbMr5ccRaC_U7NfViQLA@mail.gmail.com>
Hi

I've got a repository where "git log --raw > _somefile" took a few seconds in the past, but after an attempt at merging some commits that were collected in a clone of the same repo that was created about a year ago, I noticed that this command was now taking 3 minutes 7 seconds. "git gc", "git fsck", "git clone file:///the/repo/.git" also now each took between ~4-10 minutes, also "git log --raw somefile" got equally unusably slow. With the help of the people on the IRC, I tracked it down to my recent use of "git gc --aggressive" in this repo. Running "git repack -a -d -f" solved it, now it's again taking 4-5 seconds. After running "git gc --aggressive" again for confirmation, "git log --raw > _somefile" was again slowed down, although now 'only' to 1 minute 34 seconds; did perhaps my "git remote add -f other-repo", which I remember was also running rather slowly, exacerbate the problem (to the > 3 minutes I was seeing)?

The repo has about 6000 commits, about 12'000 files in the current HEAD, and about 43 MB packed .git contents. The files are (almost) all plain text, about half of them are about 42 bytes long, the rest up to about 2 MB although most of them are just around 5-50 KB. Most files mostly grow at the end. The biggest files (500KB-2MB) are quite long-lived and don't stop growing, again mostly at the end. Also, about 2*5K files are each in the same directory, meaning that the tree objects representing those 2 directories are big but changing only in a few places.

I've now learned to avoid "git gc --aggressive". Perhaps there are some other conclusions to be drawn, I don't know.

Christian.
Next: David Kastrup
Message 1 of 31 in “git gc --aggressive led to about 40 times slower "git log --raw"”
  1. Christian JaegerFeb 18, 2014
  2. David KastrupFeb 18, 2014
  3. Duy NguyenFeb 18, 2014
  4. David KastrupFeb 18, 2014
  5. Jonathan NiederFeb 18, 2014
  6. Junio C HamanoFeb 18, 2014
  7. Duy NguyenFeb 18, 2014
  8. Junio C HamanoFeb 19, 2014
  9. Duy NguyenFeb 19, 2014
  10. Philippe VaucherFeb 19, 2014
  11. David KastrupFeb 19, 2014
  12. Duy NguyenFeb 19, 2014
  13. Duy NguyenFeb 19, 2014
  14. Christian JaegerFeb 20, 2014
  15. David KastrupFeb 20, 2014
  16. David KastrupFeb 20, 2014
  17. David KastrupFeb 20, 2014
  18. Junio C HamanoFeb 19, 2014
  19. Duy NguyenFeb 20, 2014
  20. Christian JaegerFeb 21, 2014
  21. Junio C HamanoFeb 21, 2014
  22. Duy NguyenFeb 21, 2014
  23. Junio C HamanoFeb 21, 2014
  24. Philippe VaucherFeb 24, 2014
  25. Duy NguyenFeb 22, 2014
  26. David KastrupFeb 22, 2014
  27. David KastrupFeb 22, 2014
  28. Duy NguyenFeb 22, 2014
  29. Duy NguyenFeb 22, 2014
  30. Andreas SchwabFeb 22, 2014
  31. Christian JaegerFeb 18, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.