git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 00/10] reroll of ap/log-mailmap

From
Junio C Hamano <gitster@pobox.com>
Date
Jan 8, 2013, 08:02 UTC
Message-ID
<7vzk0kf6fq.fsf@alter.siamese.dyndns.org>
In-Reply-To
<7v4nisgm3l.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> writes:
Show 8 quoted lines
> Antoine Pelisse <apelisse@gmail.com> writes:
>
>> Have you been able to measure a speed increase due to less copies ?
>
> No.
>
> This topic was not strictly my itch, but I did the rewrite because I
> couldn't stand staring at that *_extended() function ;-)

Now I have. The overall numbers are larger primarily because the kernel history has more commits than back when I did the previous commit (thanks for reminding me of that one, by the way), but the basic trend is the same.

The patch with updated numbers in the log is attached.

I am not absolutely sure about the correctness of the third patch in the series; a review on the logic that uses the insertion point search to implement the prefix match is very much appreciated.

Thanks.
-- >8 --
Subject: [PATCH] log --use-mailmap: optimize for cases without --author/--committer search

When we taught the commit_match() mechanism to pay attention to the new --use-mailmap option, we started to unconditionally copy the commit object to a temporary buffer, just in case we need the author and committer lines updated via the mailmap mechanism, and rewrite author and committer using the mailmap.

It turns out that this has a rather unpleasant performance implications. In the linux kernel repository, running

  $ git log --author='Junio C Hamano' --pretty=short >/dev/null

under /usr/bin/time, with and without --use-mailmap (the .mailmap file is 118 entries long, the particular author does not appear in it), cost (with warm cache):

  [without --use-mailmap]
  5.42user 0.26system 0:05.70elapsed 99%CPU (0avgtext+0avgdata 2005936maxresident)k
  0inputs+0outputs (0major+137669minor)pagefaults 0swaps
  [with --use-mailmap]
  6.47user 0.30system 0:06.78elapsed 99%CPU (0avgtext+0avgdata 2006288maxresident)k
  0inputs+0outputs (0major+137692minor)pagefaults 0swaps

which incurs about 20% overhead. The command is doing extra work, so the extra cost may be justified.

But it is inexcusable to pay the cost when we do not need author/committer match. In the same repository,

  $ git log --grep='fix menuconfig on debian lenny' --pretty=short >/dev/null
shows very similar numbers as the above:
  [without --use-mailmap]
  5.32user 0.30system 0:05.63elapsed 99%CPU (0avgtext+0avgdata 2005984maxresident)k
  0inputs+0outputs (0major+137672minor)pagefaults 0swaps
  [with --use-mailmap]
  6.64user 0.24system 0:06.89elapsed 99%CPU (0avgtext+0avgdata 2006320maxresident)k
  0inputs+0outputs (0major+137694minor)pagefaults 0swaps

The latter case is an unnecessary performance regression. We may want to _show_ the result with mailmap applied, but we do not have to copy and rewrite the author/committer of all commits we try to match if we do not query for these fields.

Trivially optimize this performace regression by limiting the rewrites for only when we are matching with author/committer fields.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 revision.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/revision.c b/revision.c
index 2cce85a..d7562ee 100644
--- a/revision.c
+++ b/revision.c
@@ -2283,7 +2283,7 @@ static int commit_match(struct commit *commit, struct rev_info *opt)
 	if (buf.len)
 		strbuf_addstr(&buf, commit->buffer);
 
-	if (opt->mailmap) {
+	if (opt->grep_filter.header_list && opt->mailmap) {
 		if (!buf.len)
 			strbuf_addstr(&buf, commit->buffer);
 
-- 
1.8.1.304.gf036638
Previous: Junio C Hamano
Message 18 of 18 in “reroll of ap/log-mailmap”
  1. 00/10 reroll of ap/log-mailmapJunio C Hamano, Jan 8, 2013
  2. 01/10 string-list: allow case-insensitive string listJunio C Hamano, Jan 8, 2013
  3. René ScharfeJan 10, 2013
  4. 02/10 Use split_ident_line to parse author and committerJunio C Hamano, Jan 8, 2013
  5. 03/10 mailmap: remove email copy and length limitationJunio C Hamano, Jan 8, 2013
  6. Antoine PelisseJan 9, 2013
  7. Junio C HamanoJan 9, 2013
  8. Junio C HamanoJan 9, 2013
  9. 04/10 mailmap: simplify map_user() interfaceJunio C Hamano, Jan 8, 2013
  10. 05/10 mailmap: add mailmap structure to rev_info and ppJunio C Hamano, Jan 8, 2013
  11. 06/10 pretty: use mailmap to display username and emailJunio C Hamano, Jan 8, 2013
  12. 07/10 log: add --use-mailmap optionJunio C Hamano, Jan 8, 2013
  13. 08/10 test: add test for --use-mailmap optionJunio C Hamano, Jan 8, 2013
  14. 09/10 log: grep author/committer using mailmapJunio C Hamano, Jan 8, 2013
  15. 10/10 log: add log.mailmap configuration optionJunio C Hamano, Jan 8, 2013
  16. Antoine PelisseJan 8, 2013
  17. Junio C HamanoJan 8, 2013
  18. Junio C HamanoJan 8, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.