git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Question: .idx without .pack causes performance issues?

From
Doug Kelly <dougk.ff7@gmail.com>
Date
Jul 21, 2015, 18:41 UTC
Message-ID
<CAEtYS8QWCg5_DtrJw-e+c50vcG0OpciR6LWon-3GgyngGn+0pQ@mail.gmail.com>
Hi all,

I just wanted to relay an issue we've seen before at my day job (and it just recently cropped up again). When moving users from Git for Windows 1.8.3 to 1.9.5, we found a few users started having operations take an excruciatingly long amount of time. At some point, we traced the issue to a number of .pack files had been deleted (possibly garbage collected?) -- but their associated .idx files were still present. Upon removing the "orphaned" idx files, we found performance returned to normal. Otherwise, git fsck reported no issues with the repositories.

Other users have noted that using git gc would sometimes correct the issue for them, but not always.

Anyway, has anyone else experienced this performance degradation? I have some feeling that it's an issue that may be exclusive to Windows (or at least, only slow enough to matter on Windows), but I have no proof, and I've never heard of an issue like this outside work. (One idea that came to mind was even the .idx files were locked, and thus not deleted.) Something tells me deleting the orphaned .idx files isn't the "nicest" solution, either.

Thanks!
--Doug

P.S. In addition to running the Git for Windows/msysgit builds, we have a handful of users running Git Extensions as well, and also have been seeing an increase in use of Visual Studio 2013 -- which of course has libgit2 integrated. So, I think the chance that any one of these might be using the repo or holding files open is very high.

Next: Junio C Hamano
Message 1 of 34 in “Question: .idx without .pack causes performance issues?”
  1. Doug KellyJul 21, 2015
  2. Junio C HamanoJul 21, 2015
  3. Junio C HamanoJul 21, 2015
  4. Junio C HamanoJul 21, 2015
  5. Doug KellyJul 21, 2015
  6. Doug KellyAug 3, 2015
  7. Junio C HamanoAug 4, 2015
  8. Doug KellyAug 7, 2015
  9. Junio C HamanoAug 7, 2015
  10. 1/2 prepare_packed_git(): refactor garbage reporting in pack directoryDoug Kelly, Aug 13, 2015
  11. 2/2 gc: Remove garbage .idx files from pack dirDoug Kelly, Aug 13, 2015
  12. Junio C HamanoAug 17, 2015
  13. Junio C HamanoAug 17, 2015
  14. Eric SunshineAug 13, 2015
  15. Junio C HamanoAug 17, 2015
  16. Junio C HamanoOct 28, 2015
  17. Doug KellyOct 28, 2015
  18. 1/3 prepare_packed_git(): refactor garbage reporting in pack directoryDoug Kelly, Nov 4, 2015
  19. 2/3 t5304: Add test for cleaning pack garbageDoug Kelly, Nov 4, 2015
  20. 3/3 gc: Remove garbage .idx files from pack dirDoug Kelly, Nov 4, 2015
  21. Doug KellyNov 4, 2015
  22. Junio C HamanoNov 4, 2015
  23. Doug KellyNov 4, 2015
  24. Jeff KingNov 4, 2015
  25. Doug KellyNov 4, 2015
  26. Jeff KingNov 4, 2015
  27. Jeff KingDec 30, 2015
  28. Doug KellyJan 13, 2016
  29. Junio C HamanoJan 13, 2016
  30. Doug KellyJan 13, 2016
  31. Jeff KingJan 13, 2016
  32. Jeff KingNov 4, 2015
  33. Doug KellyJul 21, 2015
  34. Fwd: Question: .idx without .pack causes performance issues?Thomas Berg, Nov 11, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.