git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] git gc: Speed it up by 18% via faster hash comparisons

From
IMIngo Molnar <mingo@elte.hu>
Date
Apr 28, 2011, 15:14 UTC
Message-ID
<20110428151409.GA32025@elte.hu>
In-Reply-To
<20110428133708.GA31383@elte.hu>
* Ingo Molnar <mingo@elte.hu> wrote:
Show 38 quoted lines
> * Jonathan Nieder <jrnieder@gmail.com> wrote:
> 
> > Hi,
> > 
> > A side note for amusement.
> > 
> > Erik Faye-Lund wrote:
> > 
> > > --- a/cache.h
> > > +++ b/cache.h
> > > @@ -681,13 +681,17 @@ extern char *sha1_pack_name(const unsigned char *sha1);
> > >  extern char *sha1_pack_index_name(const unsigned char *sha1);
> > >  extern const char *find_unique_abbrev(const unsigned char *sha1, int);
> > >  extern const unsigned char null_sha1[20];
> > > -static inline int is_null_sha1(const unsigned char *sha1)
> > > +static inline int hashcmp(const unsigned char *sha1, const unsigned char *sha2)
> > >  {
> > > -	return !memcmp(sha1, null_sha1, 20);
> > > +	/* early out for fast mis-match */
> > > +	if (*sha1 != *sha2)
> > > +		return *sha1 - *sha2;
> > > +
> > > +	return memcmp(sha1 + 1, sha2 + 1, 19);
> > >  }
> > 
> > On the off-chance that sha1 and sha2 are nicely aligned, a more
> > redundant
> > 
> > 	if (*sha1 != *sha2)
> > 		return *sha1 - *sha2;
> > 
> > 	return memcmp(sha1, sha2, 20);
> > 
> > would take advantage of that (yes, this is just superstition, but it
> > somehow seems comforting anyway).
> 
> Your variant also makes the code slightly more compact as the sha1+1 and sha2+1 
> addresses do not have to be computed. I'll re-test and resend this variant.
Seems to perform measurably worse:
 #
 # Open-coded loop:
 #
 Performance counter stats for './git gc' (10 runs):
       2358.560100 task-clock               #    0.763 CPUs utilized            ( +-  0.06% )
             1,870 context-switches         #    0.001 M/sec                    ( +-  3.09% )
               170 CPU-migrations           #    0.000 M/sec                    ( +-  3.54% )
            38,230 page-faults              #    0.016 M/sec                    ( +-  0.03% )
     7,513,529,543 cycles                   #    3.186 GHz                      ( +-  0.06% )
     1,634,103,128 stalled-cycles           #   21.75% of all cycles are idle   ( +-  0.28% )
    11,068,971,207 instructions             #    1.47  insns per cycle        
                                            #    0.15  stalled cycles per insn  ( +-  0.04% )
     2,487,656,519 branches                 # 1054.735 M/sec                    ( +-  0.03% )
        59,233,604 branch-misses            #    2.38% of all branches          ( +-  0.09% )
        3.092183093  seconds time elapsed  ( +-  3.49% )
 #
 # Front test + memcmp:
 #
 Performance counter stats for './git gc' (10 runs):
       2723.468639 task-clock               #    0.833 CPUs utilized            ( +-  0.22% )
             1,751 context-switches         #    0.001 M/sec                    ( +-  2.02% )
               167 CPU-migrations           #    0.000 M/sec                    ( +-  1.23% )
            38,230 page-faults              #    0.014 M/sec                    ( +-  0.03% )
     8,684,682,538 cycles                   #    3.189 GHz                      ( +-  0.21% )
     2,062,906,208 stalled-cycles           #   23.75% of all cycles are idle   ( +-  0.60% )
     9,019,624,641 instructions             #    1.04  insns per cycle        
                                            #    0.23  stalled cycles per insn  ( +-  0.04% )
     1,771,179,402 branches                 #  650.340 M/sec                    ( +-  0.04% )
        75,026,810 branch-misses            #    4.24% of all branches          ( +-  0.04% )
        3.271415104  seconds time elapsed  ( +-  1.97% )
So i think the open-coded loop variant i posted is faster.
The key observation is that there's two cases that matter to performance:
 - the hashes are different: in this case the front test catches 99% of the cases
 - the hashes are *equal*: in this case the open-coded loop performs better than the memcmp
My patch addresses both cases.
Thanks,
	Ingo
Previous: Ingo MolnarNext: Erik Faye-Lund
Message 27 of 45 in “git gc: Speed it up by 18% via faster hash comparisons”
  1. git gc: Speed it up by 18% via faster hash comparisonsIngo Molnar, Apr 27, 2011
  2. Ingo MolnarApr 27, 2011
  3. Jonathan NiederApr 27, 2011
  4. Ingo MolnarApr 28, 2011
  5. Jonathan NiederApr 28, 2011
  6. Ingo MolnarApr 28, 2011
  7. Dmitry PotapovApr 28, 2011
  8. Junio C HamanoApr 27, 2011
  9. Ralf BaechleApr 28, 2011
  10. Bernhard R. LinkApr 28, 2011
  11. Andreas EricssonApr 28, 2011
  12. Erik Faye-LundApr 28, 2011
  13. H. Peter AnvinApr 28, 2011
  14. Ingo MolnarApr 28, 2011
  15. Erik Faye-LundApr 28, 2011
  16. Ingo MolnarApr 28, 2011
  17. Ingo MolnarApr 28, 2011
  18. Erik Faye-LundApr 28, 2011
  19. Pekka EnbergApr 28, 2011
  20. Erik Faye-LundApr 28, 2011
  21. Pekka EnbergApr 28, 2011
  22. Erik Faye-LundApr 28, 2011
  23. Pekka EnbergApr 28, 2011
  24. Jonathan NiederApr 28, 2011
  25. Erik Faye-LundApr 28, 2011
  26. Ingo MolnarApr 28, 2011
  27. Ingo MolnarApr 28, 2011
  28. Erik Faye-LundApr 28, 2011
  29. Ingo MolnarApr 28, 2011
  30. Alex RiesenApr 29, 2011
  31. H. Peter AnvinApr 29, 2011
  32. Tor ArntsenApr 28, 2011
  33. H. Peter AnvinApr 28, 2011
  34. Andreas EricssonApr 28, 2011
  35. Erik Faye-LundApr 28, 2011
  36. Ingo MolnarApr 28, 2011
  37. Nguyen Thai Ngoc DuyApr 28, 2011
  38. Erik Faye-LundApr 28, 2011
  39. Junio C HamanoApr 28, 2011
  40. Dmitry PotapovApr 28, 2011
  41. Dmitry PotapovApr 28, 2011
  42. Ingo MolnarApr 28, 2011
  43. Dmitry PotapovApr 28, 2011
  44. Ingo MolnarApr 28, 2011
  45. Ingo MolnarApr 28, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.