git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4 6/8] dir: fix checks on common prefix directory

From
Junio C Hamano <gitster@pobox.com>
Date
Dec 18, 2019, 21:29 UTC
Message-ID
<xmqq1rt1e30q.fsf@gitster-ct.c.googlers.com>
In-Reply-To
<1f3978aa461929923eeb5037e69be6569f0ba331.1576697386.git.gitgitgadget@gmail.com>
"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
Show 55 quoted lines
> ...
> Fix most these problems by making treat_leading_path() not only loop
> over each leading path component, but calling treat_path() directly on
> each.  To do so, we have to create a synthetic dir_entry, but that only
> takes a few lines.  Then, pay attention to the path_treatment result we
> get from treat_path() and don't treat path_excluded, path_untracked, and
> path_recurse all the same as path_recurse.
>
> This leaves one remaining problem, the new inconsistency from commit
> df5bcdf83ae.  That will be addressed in a subsequent commit.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  dir.c                                         | 57 +++++++++++++++----
>  ...common-prefixes-and-directory-traversal.sh |  6 +-
>  2 files changed, 49 insertions(+), 14 deletions(-)
>
> diff --git a/dir.c b/dir.c
> index 645b44ea64..1de5d7ad33 100644
> --- a/dir.c
> +++ b/dir.c
> @@ -2102,37 +2102,72 @@ static int treat_leading_path(struct dir_struct *dir,
>  			      const struct pathspec *pathspec)
>  {
>  	struct strbuf sb = STRBUF_INIT;
> -	int baselen, rc = 0;
> +	int prevlen, baselen;
>  	const char *cp;
> +	struct cached_dir cdir;
> +	struct dirent *de;
> +	enum path_treatment state = path_none;
> +
> +	/*
> +	 * For each directory component of path, we are going to check whether
> +	 * that path is relevant given the pathspec.  For example, if path is
> +	 *    foo/bar/baz/
> +	 * then we will ask treat_path() whether we should go into foo, then
> +	 * whether we should go into bar, then whether baz is relevant.
> +	 * Checking each is important because e.g. if path is
> +	 *    .git/info/
> +	 * then we need to check .git to know we shouldn't traverse it.
> +	 * If the return from treat_path() is:
> +	 *    * path_none, for any path, we return false.
> +	 *    * path_recurse, for all path components, we return true
> +	 *    * <anything else> for some intermediate component, we make sure
> +	 *        to add that path to the relevant list but return false
> +	 *        signifying that we shouldn't recurse into it.
> +	 */
>  
>  	while (len && path[len - 1] == '/')
>  		len--;
>  	if (!len)
>  		return 1;
> +
> +	de = xcalloc(1, sizeof(struct dirent)+len+1);

That "+len+1" may deserve a comment? If we wanted to shoot for the minimum memory consumption (and we do not), we would probably allocate

	(sizeof(struct dirent) - sizeof(de->d_name)) +
		max(sizeof(de->d_name), len + 1)

bytes, but unconditionally adding len+1 is simpler and easier to understand. Either way, we *are* relying on the assumption that either:

 (1) the "struct dirent" would have d_name[] array at the end of the
     struct, and by over-allocating, we can safely fit and carry a
     name that is much longer than sizeof(.d_name[]); OR
 (2) the "struct dirent" has d_name[] that is large enough to hold len+1
     bytes, if the assumption (1) does not hold.
is true.
Show 21 quoted lines
> +	memset(&cdir, 0, sizeof(cdir));
> +	cdir.de = de;
> +#if defined(DT_UNKNOWN) && !defined(NO_D_TYPE_IN_DIRENT)
> +	de->d_type = DT_DIR;
> +#endif
>  	baselen = 0;
> +	prevlen = 0;
>  	while (1) {
> -		cp = path + baselen + !!baselen;
> +		prevlen = baselen + !!baselen;
> +		cp = path + prevlen;
>  		cp = memchr(cp, '/', path + len - cp);
>  		if (!cp)
>  			baselen = len;
>  		else
>  			baselen = cp - path;
> -		strbuf_setlen(&sb, 0);
> +		strbuf_reset(&sb);
>  		strbuf_add(&sb, path, baselen);
>  		if (!is_directory(sb.buf))
>  			break;
Show 10 quoted lines
> -		if (simplify_away(sb.buf, sb.len, pathspec))
> -			break;
> -		if (treat_one_path(dir, NULL, istate, &sb, baselen, pathspec,
> -				   DT_DIR, NULL) == path_none)
> +		strbuf_reset(&sb);
> +		strbuf_add(&sb, path, prevlen);
> +		memcpy(de->d_name, path+prevlen, baselen-prevlen);
> +		de->d_name[baselen-prevlen] = '\0';
> +		state = treat_path(dir, NULL, &cdir, istate, &sb, prevlen,
> +				    pathspec);

So this is the crux fo the fix---instead of doing a (poor) imitation of what treat_path() does by calling simplify_away() and treat_one_path() ourselves, we make a call to the real thing.

Looking good.  Thanks.
Previous: Elijah Newren via GitGitGadgetNext: Elijah Newren
Message 52 of 69 in “Directory traversal bugs”
  1. 0/8 Directory traversal bugsElijah Newren via GitGitGadget, Dec 9, 2019
  2. 2/8 Revert "dir.c: make 'git-status --ignored' work within leading directories"Elijah Newren via GitGitGadget, Dec 9, 2019
  3. Denton LiuDec 9, 2019
  4. Elijah NewrenDec 9, 2019
  5. Eric SunshineDec 9, 2019
  6. 3/8 dir: remove stray quote character in commentElijah Newren via GitGitGadget, Dec 9, 2019
  7. 5/8 dir: break part of read_directory_recursive() out for reuseElijah Newren via GitGitGadget, Dec 9, 2019
  8. 7/8 dir: synchronize treat_leading_path() and read_directory_recursive()Elijah Newren via GitGitGadget, Dec 9, 2019
  9. 8/8 dir: consolidate similar code in treat_directory()Elijah Newren via GitGitGadget, Dec 9, 2019
  10. 6/8 dir: fix checks on common prefix directoryElijah Newren via GitGitGadget, Dec 9, 2019
  11. 4/8 dir: exit before wildcard fall-through if there is no wildcardElijah Newren via GitGitGadget, Dec 9, 2019
  12. 1/8 t3011: demonstrate directory traversal failuresElijah Newren via GitGitGadget, Dec 9, 2019
  13. Denton LiuDec 9, 2019
  14. 0/8 Directory traversal bugsElijah Newren via GitGitGadget, Dec 10, 2019
  15. 2/8 Revert "dir.c: make 'git-status --ignored' work within leading directories"Elijah Newren via GitGitGadget, Dec 10, 2019
  16. 1/8 t3011: demonstrate directory traversal failuresElijah Newren via GitGitGadget, Dec 10, 2019
  17. 6/8 dir: fix checks on common prefix directoryElijah Newren via GitGitGadget, Dec 10, 2019
  18. Johannes SchindelinDec 15, 2019
  19. Elijah NewrenDec 16, 2019
  20. Elijah NewrenDec 16, 2019
  21. Junio C HamanoDec 16, 2019
  22. Elijah NewrenDec 16, 2019
  23. Junio C HamanoDec 16, 2019
  24. Elijah NewrenDec 16, 2019
  25. Johannes SchindelinDec 17, 2019
  26. Junio C HamanoDec 17, 2019
  27. Johannes SchindelinDec 17, 2019
  28. Junio C HamanoDec 17, 2019
  29. Johannes SchindelinDec 17, 2019
  30. Elijah NewrenDec 17, 2019
  31. Johannes SchindelinDec 17, 2019
  32. Elijah NewrenDec 17, 2019
  33. 3/8 dir: remove stray quote character in commentElijah Newren via GitGitGadget, Dec 10, 2019
  34. 8/8 dir: consolidate similar code in treat_directory()Elijah Newren via GitGitGadget, Dec 10, 2019
  35. 5/8 dir: break part of read_directory_recursive() out for reuseElijah Newren via GitGitGadget, Dec 10, 2019
  36. 4/8 dir: exit before wildcard fall-through if there is no wildcardElijah Newren via GitGitGadget, Dec 10, 2019
  37. 7/8 dir: synchronize treat_leading_path() and read_directory_recursive()Elijah Newren via GitGitGadget, Dec 10, 2019
  38. 0/3 Directory traversal bugsElijah Newren via GitGitGadget, Dec 17, 2019
  39. 2/3 dir: remove stray quote character in commentElijah Newren via GitGitGadget, Dec 17, 2019
  40. 3/3 dir: exit before wildcard fall-through if there is no wildcardElijah Newren via GitGitGadget, Dec 17, 2019
  41. 1/3 t3011: demonstrate directory traversal failuresElijah Newren via GitGitGadget, Dec 17, 2019
  42. Johannes SchindelinDec 17, 2019
  43. Junio C HamanoDec 17, 2019
  44. Johannes SchindelinDec 21, 2019
  45. 0/8 Directory traversal bugsElijah Newren via GitGitGadget, Dec 18, 2019
  46. 2/8 Revert "dir.c: make 'git-status --ignored' work within leading directories"Elijah Newren via GitGitGadget, Dec 18, 2019
  47. 1/8 t3011: demonstrate directory traversal failuresElijah Newren via GitGitGadget, Dec 18, 2019
  48. 3/8 dir: remove stray quote character in commentElijah Newren via GitGitGadget, Dec 18, 2019
  49. 4/8 dir: exit before wildcard fall-through if there is no wildcardElijah Newren via GitGitGadget, Dec 18, 2019
  50. 5/8 dir: break part of read_directory_recursive() out for reuseElijah Newren via GitGitGadget, Dec 18, 2019
  51. 6/8 dir: fix checks on common prefix directoryElijah Newren via GitGitGadget, Dec 18, 2019
  52. Junio C HamanoDec 18, 2019
  53. Elijah NewrenDec 19, 2019
  54. Jeff KingDec 19, 2019
  55. Elijah NewrenDec 20, 2019
  56. Jeff KingDec 20, 2019
  57. Junio C HamanoDec 20, 2019
  58. Jeff KingDec 20, 2019
  59. 8/8 dir: consolidate similar code in treat_directory()Elijah Newren via GitGitGadget, Dec 18, 2019
  60. 7/8 dir: synchronize treat_leading_path() and read_directory_recursive()Elijah Newren via GitGitGadget, Dec 18, 2019
  61. 0/8 Directory traversal bugsElijah Newren via GitGitGadget, Dec 19, 2019
  62. 1/8 t3011: demonstrate directory traversal failuresElijah Newren via GitGitGadget, Dec 19, 2019
  63. 5/8 dir: break part of read_directory_recursive() out for reuseElijah Newren via GitGitGadget, Dec 19, 2019
  64. 4/8 dir: exit before wildcard fall-through if there is no wildcardElijah Newren via GitGitGadget, Dec 19, 2019
  65. 3/8 dir: remove stray quote character in commentElijah Newren via GitGitGadget, Dec 19, 2019
  66. 2/8 Revert "dir.c: make 'git-status --ignored' work within leading directories"Elijah Newren via GitGitGadget, Dec 19, 2019
  67. 6/8 dir: fix checks on common prefix directoryElijah Newren via GitGitGadget, Dec 19, 2019
  68. 8/8 dir: consolidate similar code in treat_directory()Elijah Newren via GitGitGadget, Dec 19, 2019
  69. 7/8 dir: synchronize treat_leading_path() and read_directory_recursive()Elijah Newren via GitGitGadget, Dec 19, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.