git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git and larger trees, not so fast?

From
Linus Torvalds <torvalds@linux-foundation.org>
Date
Aug 9, 2007, 18:06 UTC
Message-ID
<alpine.LFD.0.999.0708091056180.25146@woody.linux-foundation.org>
In-Reply-To
<alpine.LFD.0.999.0708091015500.25146@woody.linux-foundation.org>
On Thu, 9 Aug 2007, Linus Torvalds wrote:
> 
> Gaah. This shouldn't be *that* hard to fix, but I'm not entirely sure I'll 
> have time today.
In fact, I'm almost sure I will *not* have time today.

Anyway, the really trivial (and ugly) fix is to handle the cases of adding _independent_ stages to the index (which is the case for both "git diff-index" and "git read-tree -m") differently: instead of using the standard "add_index_entry()", which does all the complex sorting and checks that there aren't duplicates, we could do a much simpler one that just unconditionally appends to the end of the index.

This works, because when the stages are independent, there can be no index clashes (by definition).

Then, after adding all the stages, we could just do a "qsort()" on the result, and rather than having an expensive O(n**2) thing, we'd have a much nicer and well-behaved (with a smaller constant too) O(n*logn) thing.

I bet it's just ~50 lines of code, it really shouldn't be that hard to do. I just won't be able to do it and test it until late tonight or tomorrow, I suspect.

Sadly, this is an area that is almost exclusively mine and Junio's. I'd love for somebody else to get their feet wet, but doing a

	gitk read-cache.c

shows that few enough people have done anythign really fundamental in this file..

			Linus
Previous: Junio C HamanoNext: Junio C Hamano
Message 5 of 37 in “git and larger trees, not so fast?”
  1. moeAug 9, 2007
  2. Linus TorvaldsAug 9, 2007
  3. Linus TorvaldsAug 9, 2007
  4. Junio C HamanoAug 9, 2007
  5. Linus TorvaldsAug 9, 2007
  6. Junio C HamanoAug 9, 2007
  7. Junio C HamanoAug 9, 2007
  8. SeanAug 9, 2007
  9. Junio C HamanoAug 9, 2007
  10. Linus TorvaldsAug 9, 2007
  11. Linus TorvaldsAug 9, 2007
  12. Junio C HamanoAug 9, 2007
  13. Junio C HamanoAug 9, 2007
  14. Junio C HamanoAug 10, 2007
  15. Linus TorvaldsAug 10, 2007
  16. Junio C HamanoAug 10, 2007
  17. Linus TorvaldsAug 10, 2007
  18. Junio C HamanoAug 10, 2007
  19. Junio C HamanoAug 10, 2007
  20. Linus TorvaldsAug 10, 2007
  21. Linus TorvaldsAug 10, 2007
  22. Fix "git commit directory/" performance anomalyLinus Torvalds, Aug 10, 2007
  23. Linus TorvaldsAug 10, 2007
  24. Junio C HamanoAug 10, 2007
  25. Linus TorvaldsAug 10, 2007
  26. Daniel BarkalowAug 10, 2007
  27. Linus TorvaldsAug 9, 2007
  28. David KastrupAug 9, 2007
  29. Linus TorvaldsAug 10, 2007
  30. Linus TorvaldsAug 11, 2007
  31. Fernando J. PeredaAug 11, 2007
  32. Linus TorvaldsAug 11, 2007
  33. Fernando J. PeredaAug 11, 2007
  34. Linus TorvaldsAug 11, 2007
  35. David KastrupAug 11, 2007
  36. moeAug 11, 2007
  37. moeAug 23, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.