git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: remove_duplicates() in builtin/fetch-pack.c is O(N^2)

From
Jeff King <peff@peff.net>
Date
May 21, 2012, 19:41 UTC
Message-ID
<20120521194114.GA28358@sigill.intra.peff.net>
In-Reply-To
<201205211215.14455.mfick@codeaurora.org>
On Mon, May 21, 2012 at 12:15:13PM -0600, Martin Fick wrote:
Show 11 quoted lines
> Of course, we use Gerrit, so features tend to be called 
> changes and each change may get many revisions (patchsets), 
> so all of these get refs, but I think that it might be wrong 
> to consider that out of the ordinary anymore.  After all, 
> should a version control system such as git not support 100K 
> revisions of features developed independently on separate 
> branches (within Gerrit or not)?  100K is not really that 
> many when you consider a large project.  Even without 
> Gerrit, if someone wanted to track that many features 
> (likely over a few years), they will probably use up tons of 
> refs.

I think the more compelling line of argument is not "is this reasonable?", but rather that git has been designed from the ground-up to be efficient, and these are not fundamental design issues with git at all. They are just silly little spots where we used a quick-to-write quadratic algorithm instead of something more complex with better asymptotic behavior. And if we can fix these silly spots easily, then there's no reason not to. It helps the small-N case a tiny bit, and it makes the big-N case feasible.

So far, the only quadratic case I have seen that is not easy to fix is replacing "struct commit_list" with a priority queue or similar. But we managed to hack around that long ago with fce87ae (Fix quadratic performance in rewrite_one., 2008-07-12), and I don't think it's generally a problem in practice.

Anyway, my point is that we don't even have to talk about "reasonable" or "absurd". Git should be fast even on absurd cases, because 99% of the work has already been done, and the last 1% is easy.

-Peff
Previous: Martin FickNext: Junio C Hamano
Message 39 of 46 in “remove_duplicates() in builtin/fetch-pack.c is O(N^2)”
  1. Michael HaggertyMay 21, 2012
  2. Junio C HamanoMay 21, 2012
  3. demerphqMay 21, 2012
  4. Jeff KingMay 21, 2012
  5. Jeff KingMay 21, 2012
  6. 1/5 fetch-pack: sort incoming headsJeff King, May 21, 2012
  7. Junio C HamanoMay 22, 2012
  8. Jeff KingMay 22, 2012
  9. Jeff KingMay 24, 2012
  10. 2/5 fetch-pack: avoid quadratic behavior in remove_duplicatesJeff King, May 21, 2012
  11. 3/5 add sorting infrastructure for list refsJeff King, May 21, 2012
  12. 4/5 fetch-pack: sort the list of incoming refsJeff King, May 21, 2012
  13. 5/5 fetch-pack: avoid quadratic loop in filter_refsJeff King, May 21, 2012
  14. Junio C HamanoMay 22, 2012
  15. Jeff KingMay 21, 2012
  16. Jeff KingMay 22, 2012
  17. Michael HaggertyMay 22, 2012
  18. Jeff KingMay 22, 2012
  19. Michael HaggertyMay 22, 2012
  20. Jeff KingMay 22, 2012
  21. Michael HaggertyMay 22, 2012
  22. Jeff KingMay 22, 2012
  23. Michael HaggertyMay 24, 2012
  24. Martin FickMay 25, 2012
  25. Jeff KingMay 25, 2012
  26. Martin FickMay 25, 2012
  27. Jeff KingMay 25, 2012
  28. Martin FickMay 25, 2012
  29. Jeff KingMay 25, 2012
  30. Nguyen Thai Ngoc DuyMay 22, 2012
  31. Michael HaggertyMay 22, 2012
  32. Jeff KingMay 22, 2012
  33. Junio C HamanoMay 22, 2012
  34. Jeff KingMay 22, 2012
  35. Michael HaggertyMay 24, 2012
  36. Nguyen Thai Ngoc DuyMay 23, 2012
  37. Junio C HamanoMay 22, 2012
  38. Martin FickMay 21, 2012
  39. Jeff KingMay 21, 2012
  40. Junio C HamanoMay 21, 2012
  41. Jeff KingMay 21, 2012
  42. Martin FickMay 22, 2012
  43. Jeff KingMay 22, 2012
  44. Martin FickMay 22, 2012
  45. Junio C HamanoMay 22, 2012
  46. Martin FickMay 23, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.