git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v6 0/3] read-cache: speed up add_index_entry

From
Jeff King <peff@peff.net>
Date
Apr 8, 2017, 10:43 UTC
Message-ID
<20170408104328.3iuvedkewygkovnb@sigill.intra.peff.net>
In-Reply-To
<6f31ee65-517e-419c-b0c1-3ccdd3f95b37@jeffhostetler.com>
On Fri, Apr 07, 2017 at 02:27:24PM -0400, Jeff Hostetler wrote:
Show 22 quoted lines
> > Just thinking about this algorithmically for a moment. You're saving the
> > binary search when the input is given in sorted order. But in other
> > cases you're adding an extra strcmp() before the binary search begins.
> > So it's a tradeoff.
> > 
> > How often is the input sorted?  You save O(log n) strcmps for a "hit"
> > with your patch, and one for a "miss". So it's a net win if we expect at
> > least 1/log(n) of additions to be sorted (I'm talking about individual
> > calls, but it should scale linearly either way over a set of n calls).
> > 
> > I have no clue if that's a reasonable assumption or not.
> 
> I was seeing checkout call merge_working_tree to iterate over the
> source index/trees and call add_index_entry() for each.  For example,
> in a "checkout -b" like operation where both sides are the same, this
> calls keep_entry() which appends the entry to the new index array.
> The append path should always be taken because the iteration is being
> driven from a sorted list.
> 
> I would think calls to add/stage individual files arrive in random
> order, so I'm not suggesting replacing the code -- just checking the
> end first.

Right, what I was wondering is how much this costs in those random-order cases. We _know_ it speeds up the cases you care about, but I want to make sure that it is not making some other case worse. How often do the random-order cases come up, and how much are they slowed?

I suspect in practice that calls here fall into one of two camps: feeding a small-ish (compared to the total number of entries) set of paths, or feeding _every_ path. And if you are feeding every path, you are likely to do so in sorted order, rather than a random jumble. So it helps in the big cases, and the small cases are presumably small enough that we don't care much.

At least that seems like a plausible line of reasoning to me. ;)
-Peff
Previous: Jeff Hostetler
Message 9 of 9 in “read-cache: speed up add_index_entry”
  1. 0/3 read-cache: speed up add_index_entrygit@jeffhostetler.com, Apr 6, 2017
  2. 1/3 read-cache: add strcmp_offset functiongit@jeffhostetler.com, Apr 6, 2017
  3. René ScharfeApr 6, 2017
  4. Jeff HostetlerApr 7, 2017
  5. 2/3 p0004-read-tree: perf test to time read-treegit@jeffhostetler.com, Apr 6, 2017
  6. 3/3 read-cache: speed up add_index_entry during checkoutgit@jeffhostetler.com, Apr 6, 2017
  7. Jeff KingApr 7, 2017
  8. Jeff HostetlerApr 7, 2017
  9. Jeff KingApr 8, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.