threads / discuss / 3455

git-annotate efficiency

Subject: git-annotate efficiency

## tl;dr

4 messages between Feb 24, 2006 and Feb 24, 2006.

replies: 3people: 3as markdown or json

Morten Welinder· Feb 24, 2006, 15:37 UTC · lore
So I wanted to give git-annotate a spin and typed...
    git annotate Makefile

Bad idea. It's been ten minutes and no output yet. While the script only appears to use ~20% of cpu according to top, an strace shows that it spins off a huge number of very short-lived subprocesses.

Morten

... rt_sigaction(SIGQUIT, {SIG_IGN}, {SIG_DFL}, 8) = 0 waitpid(1539, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0) = 1539 --- SIGCHLD (Child exited) @ 0 (0) --- rt_sigaction(SIGHUP, {SIG_DFL}, NULL, 8) = 0 rt_sigaction(SIGINT, {SIG_DFL}, NULL, 8) = 0 rt_sigaction(SIGQUIT, {SIG_DFL}, NULL, 8) = 0 pipe([3, 4]) = 0 pipe([5, 6]) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x401db708) = 1540 close(6) = 0 close(4) = 0 read(5, "", 4) = 0 close(5) = 0 ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfffc718) = -1 EINVAL (Invalid argument) _llseek(3, 0, 0xbfffc760, SEEK_CUR) = -1 ESPIPE (Illegal seek) fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 read(3, "diff --git a/Makefile b/Makefile"..., 4096) = 63 read(3, "--- a/Makefile\n+++ b/Makefile\n@@"..., 4096) = 203 --- SIGCHLD (Child exited) @ 0 (0) --- read(3, "", 4096) = 0 close(3) = 0 rt_sigaction(SIGHUP, {SIG_IGN}, {SIG_DFL}, 8) = 0 rt_sigaction(SIGINT, {SIG_IGN}, {SIG_DFL}, 8) = 0 ...

Morten Welinder· Feb 24, 2006, 18:00 UTC · re: Morten Welinder · lore

Re: git-annotate efficiency

It looks like handle_rev is seeing the same revisions over and over again. I don't know why that would be, but the following patch just skips dups. I have no idea if it is right, though.

Morten
diff --git a/git-annotate.perl b/git-annotate.perl
index 3800c46..a5e2d86 100755
--- a/git-annotate.perl
+++ b/git-annotate.perl
@@ -117,7 +117,10 @@ sub init_claim {

 sub handle_rev {
        my $i = 0;
+       my %seen = ();
        while (my $rev = shift @revqueue) {
+               next if $seen{$rev};
+               $seen{$rev} = 1;

                my %revinfo = git_commit_info($rev);
Randal L. Schwartz· Feb 24, 2006, 19:06 UTC · re: Morten Welinder · lore

Re: git-annotate efficiency

>>>>> "Morten" == Morten Welinder <mwelinder@gmail.com> writes:

Morten> It looks like handle_rev is seeing the same revisions over and over again. Morten> I don't know why that would be, but the following patch just skips dups. Morten> I have no idea if it is right, though.

Morten> Morten

Morten> diff --git a/git-annotate.perl b/git-annotate.perl Morten> index 3800c46..a5e2d86 100755 Morten> --- a/git-annotate.perl Morten> +++ b/git-annotate.perl Morten> @@ -117,7 +117,10 @@ sub init_claim {

Morten> sub handle_rev { Morten> my $i = 0; Morten> + my %seen = (); Morten> while (my $rev = shift @revqueue) { Morten> + next if $seen{$rev}; Morten> + $seen{$rev} = 1;

Morten>                 my %revinfo = git_commit_info($rev);
The traditional idiom for that is
        next if $seen{$rev}++;
-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Ryan Anderson· Feb 24, 2006, 19:42 UTC · re: Morten Welinder · lore

Re: git-annotate efficiency

On Fri, Feb 24, 2006 at 01:00:24PM -0500, Morten Welinder wrote:
> It looks like handle_rev is seeing the same revisions over and over again.
> I don't know why that would be, but the following patch just skips dups.
> I have no idea if it is right, though.
Merges.
a--b--c--d--f
   \-g--h--/
It would do f,d,c,b,a + f,h,g,b,a

So yes, this fix is correct, and Junio, I'll be doing some changes this weekend and send it along with a few other things.

(On a medium-sized test tree at work with 3500 commits in the tree, 37 on the main Makefile (6 merges), this cuts the annotate time from 10s to a little over 2, so it's, umm, very worthwhile.)

Show 23 quoted lines
> 
> Morten
> 
> 
> diff --git a/git-annotate.perl b/git-annotate.perl
> index 3800c46..a5e2d86 100755
> --- a/git-annotate.perl
> +++ b/git-annotate.perl
> @@ -117,7 +117,10 @@ sub init_claim {
> 
>  sub handle_rev {
>         my $i = 0;
> +       my %seen = ();
>         while (my $rev = shift @revqueue) {
> +               next if $seen{$rev};
> +               $seen{$rev} = 1;
> 
>                 my %revinfo = git_commit_info($rev);
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
-- 
Ryan Anderson
  sometimes Pug Majere

← back to recent threads