git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Optimize common case of git-rev-list (was Re: gitweb.cgi)

From
Linus Torvalds <torvalds@osdl.org>
Date
Oct 19, 2005, 00:53 UTC
Message-ID
<Pine.LNX.4.64.0510181742070.3369@g5.osdl.org>
In-Reply-To
<43559399.2030903@zytor.com>
On Tue, 18 Oct 2005, H. Peter Anvin wrote:
Show 8 quoted lines
> 
> > Considering that apparently the load is enough that it takes 45 seconds to
> > generate (scary in itself), is should clearly be cached for more than one
> > minute. More like ten minutes or half an hour, especially since mirroring
> > any content changes takes longer than that anyway.
> 
> The latency for an I/O operation on the kernel.org servers is positively
> scary.

I took a look at webgit, and it looks like at least for the "projects" page, the most common operation ends up being basically

	git-rev-list --header --parents --max-count=1 HEAD

Now, the thing is, the way "git-rev-list" works, it always keeps on popping the parents and parsing them in order to build the list of parents, and it turns out that even though we just want a single commit, git-rev-list will invariably look up _three_ generations of commits.

It will parse:
 - the commit we want (it obviously needs this)
 - it's parent(s) as part of the "pop_most_recent_commit()" logic
 - it will then pop one of the parents before it notices that it doesn't 
   need any more
 - and as part of popping the parent, it will parse the grandparent (again 
   due to "pop_most_recent_commit()".

Now, I've strace'd it, and it really is pretty efficient on the whole, but if things aren't nicely cached, and with long-latency IO, doing those two extra objects (at a minimum - if the parent is a merge it will be more) is just wasted time, and potentially a lot of it.

So here's a quick special-case for the trivial case of "just one commit, and no date-limits or other special rules".

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---

I've actually tried to test it (but hey, "exhaustive" is hard with all the different options), and I tried to be very careful to only do the special-case when really nothing can go wrong, and this all looks obvious.

But buyer beware.

Btw, doing an "strace" on git-rev-list shows that most of the system calls by far are the dynamic loader. Now, those accesses _should_ all be cached, so they should be fast and low-latency, but it's entirely possible that for a server configuration you might want to actually link things statically. Or not. Just a thought.

diff --git a/rev-list.c b/rev-list.c
index c60aa72..d4da1bd 100644
--- a/rev-list.c
+++ b/rev-list.c
@@ -624,6 +624,10 @@ int main(int argc, char **argv)
 
 	if (!merge_order) {		
 		sort_by_date(&list);
+		if (list && !limited && max_count == 1) {
+			show_commit(list->item);
+			return 0;
+		}
 	        if (limited)
 			list = limit_list(list);
 		if (topo_order)
Previous: H. Peter AnvinNext: H. Peter Anvin
Message 8 of 15 in “gitweb.cgi”
  1. H. Peter AnvinOct 18, 2005
  2. Kay SieversOct 18, 2005
  3. H. Peter AnvinOct 18, 2005
  4. H. Peter AnvinOct 18, 2005
  5. Brian GerstOct 18, 2005
  6. Linus TorvaldsOct 19, 2005
  7. H. Peter AnvinOct 19, 2005
  8. Optimize common case of git-rev-list (was Re: gitweb.cgi)Linus Torvalds, Oct 19, 2005
  9. H. Peter AnvinOct 19, 2005
  10. Linus TorvaldsOct 19, 2005
  11. H. Peter AnvinOct 19, 2005
  12. Linus TorvaldsOct 19, 2005
  13. H. Peter AnvinOct 19, 2005
  14. Kay SieversOct 19, 2005
  15. Kay SieversOct 19, 2005

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.