git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 5/5] sparse-index: improve lstat caching of sparse paths

From
Derrick Stolee <stolee@gmail.com>
Date
Jun 28, 2024, 01:56 UTC
Message-ID
<90731058-8079-45af-97e5-bbee4ec85378@gmail.com>
In-Reply-To
<xmqqy16qrsup.fsf@gitster.g>
On 6/27/24 5:14 PM, Junio C Hamano wrote:
Show 14 quoted lines
> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> +/**
>> + * Return the length of the largest common substring that ends in a
> 
> "largest" here is understandable (it means longest).
> 
>> + * slash ('/') to indicate the largest common parent directory. Returns
> 
> here I find it a bit confusing.  It is the deepest common parent
> directory between the two (or "the common parent directory with the
> longest path"), but my initial reaction was "largest common parent
> directory?  wouldn't the root level by definition the 'largest'
> (having the largest number of paths underneath) directory that is
> common?)".

I'm not sure why my brain got stuck on "largest" here when "longest" would be a better choice.

Show 23 quoted lines
>> + * zero if no common directory exists.
>> + */
>> +static size_t max_common_dir_prefix(const char *path1, const char *path2)
>> +{
>> +	size_t common_prefix = 0;
>> +	for (size_t i = 0; path1[i] && path2[i]; i++) {
>> +		if (path1[i] != path2[i])
>> +			break;
>> +
>> +		/*
>> +		 * If they agree at a directory separator, then add one
>> +		 * to make sure it is included in the common prefix string.
>> +		 */
>> +		if (path1[i] == '/')
>> +			common_prefix = i + 1;
>> +	}
>> +
>> +	return common_prefix;
>> +}
> 
> Looking good.  I assume that these two paths are relative to the
> top-level of the working tree (in other words, they do not begin
> with a slash)?

Yes, they do not begin with a slash. In this specific application, they are paths from cache entries in the index. If this were generalized for use elsewhere then paths beginning with a slash should be considered.

Show 22 quoted lines
>>   static int path_found(const char *path, struct path_found_data *data)
>>   {
>> ...
>> +	 * At this point, we know that 'path' doesn't exist, and we know that
>> +	 * the parent directory of 'data->dir' does exist. Let's set 'data->dir'
>> +	 * to be the top-most non-existing directory of 'path'. If the first
>> +	 * parent of 'path' exists, then we will act as though 'path'
>> +	 * corresponds to a directory (by adding a slash).
>>   	 */
>> -	newdir = strrchr(path, '/');
>> -	if (!newdir)
>> -		return 0; /* Didn't find a parent dir; just return 0 now. */
>> +	common_prefix = max_common_dir_prefix(path, data->dir.buf);
>> ...
>> +	strbuf_setlen(&data->dir, common_prefix);
>> +	while (1) {
> 
> Oooh, nice.  So you learned /a/b/c/d did not exist, so check /a/b/c,
> and then /a/b/ and stop, because you know /a does exist already.
> With luck, our next query is for /a/b/c/e or for /a/b/f, and knowing
> that /a/b/ does not exist would allow us to say "no, they do not
> exist" without having to lstat().  OK.

I probably should have led with an example like this, as it makes the point more clearly than my abstract description.

Thanks, -Stolee

Previous: Junio C HamanoNext: Junio C Hamano
Message 33 of 44 in “sparse-index: improve clear_skip_worktree_from_present_files()”
  1. 0/5 sparse-index: improve clear_skip_worktree_from_present_files()Derrick Stolee via GitGitGadget, Jun 20, 2024
  2. 1/5 sparse-index: refactor skip worktree retry logicDerrick Stolee via GitGitGadget, Jun 20, 2024
  3. Elijah NewrenJun 24, 2024
  4. Derrick StoleeJun 26, 2024
  5. 2/5 sparse-index: refactor path_found()Derrick Stolee via GitGitGadget, Jun 20, 2024
  6. Elijah NewrenJun 24, 2024
  7. Derrick StoleeJun 26, 2024
  8. 3/5 sparse-index: use strbuf in path_found()Derrick Stolee via GitGitGadget, Jun 20, 2024
  9. Elijah NewrenJun 24, 2024
  10. 4/5 sparse-index: count lstat() callsDerrick Stolee via GitGitGadget, Jun 20, 2024
  11. Elijah NewrenJun 24, 2024
  12. 5/5 sparse-index: improve lstat caching of sparse pathsDerrick Stolee via GitGitGadget, Jun 20, 2024
  13. Elijah NewrenJun 24, 2024
  14. Junio C HamanoJun 25, 2024
  15. Derrick StoleeJun 26, 2024
  16. Elijah NewrenJun 28, 2024
  17. Junio C HamanoJun 20, 2024
  18. Derrick StoleeJun 20, 2024
  19. Junio C HamanoJun 20, 2024
  20. 0/5 sparse-index: improve clear_skip_worktree_from_present_files()Derrick Stolee via GitGitGadget, Jun 26, 2024
  21. 1/5 sparse-checkout: refactor skip worktree retry logicDerrick Stolee via GitGitGadget, Jun 26, 2024
  22. Junio C HamanoJun 27, 2024
  23. Elijah NewrenJun 28, 2024
  24. Derrick StoleeJun 28, 2024
  25. Junio C HamanoJun 28, 2024
  26. Elijah NewrenJun 28, 2024
  27. Derrick StoleeJun 28, 2024
  28. 2/5 sparse-index: refactor path_found()Derrick Stolee via GitGitGadget, Jun 26, 2024
  29. 3/5 sparse-index: use strbuf in path_found()Derrick Stolee via GitGitGadget, Jun 26, 2024
  30. 4/5 sparse-index: count lstat() callsDerrick Stolee via GitGitGadget, Jun 26, 2024
  31. 5/5 sparse-index: improve lstat caching of sparse pathsDerrick Stolee via GitGitGadget, Jun 26, 2024
  32. Junio C HamanoJun 27, 2024
  33. Derrick StoleeJun 28, 2024
  34. Junio C HamanoJun 27, 2024
  35. Elijah NewrenJun 28, 2024
  36. Derrick StoleeJun 28, 2024
  37. 0/5 sparse-index: improve clear_skip_worktree_from_present_files()Derrick Stolee via GitGitGadget, Jun 28, 2024
  38. 1/5 sparse-checkout: refactor skip worktree retry logicDerrick Stolee via GitGitGadget, Jun 28, 2024
  39. 2/5 sparse-index: refactor path_found()Derrick Stolee via GitGitGadget, Jun 28, 2024
  40. 3/5 sparse-index: use strbuf in path_found()Derrick Stolee via GitGitGadget, Jun 28, 2024
  41. 4/5 sparse-index: count lstat() callsDerrick Stolee via GitGitGadget, Jun 28, 2024
  42. 5/5 sparse-index: improve lstat caching of sparse pathsDerrick Stolee via GitGitGadget, Jun 28, 2024
  43. Elijah NewrenJun 28, 2024
  44. Junio C HamanoJun 28, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.