git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4] clone: report duplicate entries on case-insensitive filesystems

From
Torsten Bögershausen <tboegi@web.de>
Date
Aug 15, 2018, 19:08 UTC
Message-ID
<20180815190816.GA26521@tor.lan>
In-Reply-To
<20180812090714.19060-1-pclouds@gmail.com>
On Sun, Aug 12, 2018 at 11:07:14AM +0200, Nguyễn Thái Ngọc Duy wrote:
Show 79 quoted lines
> Paths that only differ in case work fine in a case-sensitive
> filesystems, but if those repos are cloned in a case-insensitive one,
> you'll get problems. The first thing to notice is "git status" will
> never be clean with no indication what exactly is "dirty".
> 
> This patch helps the situation a bit by pointing out the problem at
> clone time. Even though this patch talks about case sensitivity, the
> patch makes no assumption about folding rules by the filesystem. It
> simply observes that if an entry has been already checked out at clone
> time when we're about to write a new path, some folding rules are
> behind this.
> 
> This patch is tested with vim-colorschemes repository on a JFS partition
> with case insensitive support on Linux. This repository has two files
> darkBlue.vim and darkblue.vim.
> 
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  v4 removes nr_duplicates (and fixes that false warning Szeder
>  reported). It also hints about case insensitivity as a cause of
>  problem because it's most likely the case when this warning shows up.
> 
>  builtin/clone.c  |  1 +
>  cache.h          |  1 +
>  entry.c          | 28 ++++++++++++++++++++++++++++
>  t/t5601-clone.sh |  8 +++++++-
>  unpack-trees.c   | 28 ++++++++++++++++++++++++++++
>  unpack-trees.h   |  1 +
>  6 files changed, 66 insertions(+), 1 deletion(-)
> 
> diff --git a/builtin/clone.c b/builtin/clone.c
> index 5c439f1394..0702b0e9d0 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -747,6 +747,7 @@ static int checkout(int submodule_progress)
>  	memset(&opts, 0, sizeof opts);
>  	opts.update = 1;
>  	opts.merge = 1;
> +	opts.clone = 1;
>  	opts.fn = oneway_merge;
>  	opts.verbose_update = (option_verbosity >= 0);
>  	opts.src_index = &the_index;
> diff --git a/cache.h b/cache.h
> index 8b447652a7..6d6138f4f1 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -1455,6 +1455,7 @@ struct checkout {
>  	unsigned force:1,
>  		 quiet:1,
>  		 not_new:1,
> +		 clone:1,
>  		 refresh_cache:1;
>  };
>  #define CHECKOUT_INIT { NULL, "" }
> diff --git a/entry.c b/entry.c
> index b5d1d3cf23..c70340df8e 100644
> --- a/entry.c
> +++ b/entry.c
> @@ -399,6 +399,31 @@ static int check_path(const char *path, int len, struct stat *st, int skiplen)
>  	return lstat(path, st);
>  }
>  
> +static void mark_colliding_entries(const struct checkout *state,
> +				   struct cache_entry *ce, struct stat *st)
> +{
> +	int i;
> +
> +	ce->ce_flags |= CE_MATCHED;
> +
> +#if !defined(GIT_WINDOWS_NATIVE) /* inode is always zero on Windows */
> +	for (i = 0; i < state->istate->cache_nr; i++) {
> +		struct cache_entry *dup = state->istate->cache[i];
> +
> +		if (dup == ce)
> +			break;
> +
> +		if (dup->ce_flags & (CE_MATCHED | CE_VALID | CE_SKIP_WORKTREE))
> +			continue;
> +
Should the following be protected by core.checkstat ? 
	if (check_stat) {
Show 6 quoted lines
> +		if (dup->ce_stat_data.sd_ino == st->st_ino) {
> +			dup->ce_flags |= CE_MATCHED;
> +			break;
> +		}
> +	}
> +#endif

Another thing is that we switch of the ASCII case-folding-detection-logic off for Windows users, even if we otherwise rely on icase. I think we can use fspathcmp() as a fallback. when inodes fail, because we may be on a network file system. (I don't have a test setup at the moment, but what happens with inodes when a Windows machine exports a share to Linux or Mac ?)

Is there a chance to get the fspathcmp() back, like this ?
static void mark_colliding_entries(const struct checkout *state,
				   struct cache_entry *ce, struct stat *st)
{
	int i;
	ce->ce_flags |= CE_MATCHED;
	for (i = 0; i < state->istate->cache_nr; i++) {
		struct cache_entry *dup = state->istate->cache[i];
		int folded = 0;
		if (dup == ce)
			break;
		if (dup->ce_flags & (CE_MATCHED | CE_VALID | CE_SKIP_WORKTREE))
			continue;
		if (!fspathcmp(dup->name, ce->name))
			folded = 1;
#if !defined(GIT_WINDOWS_NATIVE) /* inode is always zero on Windows */
		if (check_stat && (dup->ce_stat_data.sd_ino == st->st_ino))
			folded = 1;
#endif
		if (folded) {
			dup->ce_flags |= CE_MATCHED;
			break;
		}
	}
}
Previous: Junio C HamanoNext: Duy Nguyen
Message 60 of 96 in “Git clone and case sensitivity”
  1. Paweł ParuzelJul 27, 2018
  2. brian m. carlsonJul 27, 2018
  3. Duy NguyenJul 28, 2018
  4. Duy NguyenJul 28, 2018
  5. Jeff KingJul 28, 2018
  6. Duy NguyenJul 28, 2018
  7. Simon RuderichJul 28, 2018
  8. Jeff KingJul 28, 2018
  9. brian m. carlsonJul 28, 2018
  10. Duy NguyenJul 29, 2018
  11. Jeff KingJul 29, 2018
  12. clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Jul 30, 2018
  13. Torsten BögershausenJul 31, 2018
  14. Duy NguyenAug 1, 2018
  15. Elijah NewrenJul 31, 2018
  16. Junio C HamanoJul 31, 2018
  17. Jeff KingJul 31, 2018
  18. Junio C HamanoJul 31, 2018
  19. Jeff KingJul 31, 2018
  20. Junio C HamanoJul 31, 2018
  21. Junio C HamanoAug 1, 2018
  22. Duy NguyenAug 2, 2018
  23. Junio C HamanoAug 2, 2018
  24. Jeff KingAug 2, 2018
  25. Junio C HamanoAug 2, 2018
  26. Jeff KingAug 2, 2018
  27. Jeff HostetlerAug 3, 2018
  28. Junio C HamanoAug 3, 2018
  29. Jeff KingAug 3, 2018
  30. Jeff HostetlerAug 5, 2018
  31. Torsten BögershausenAug 3, 2018
  32. Duy NguyenAug 1, 2018
  33. Junio C HamanoJul 31, 2018
  34. Duy NguyenAug 1, 2018
  35. clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Aug 7, 2018
  36. Junio C HamanoAug 7, 2018
  37. Jeff HostetlerAug 8, 2018
  38. Jeff KingAug 8, 2018
  39. Junio C HamanoAug 9, 2018
  40. Jeff KingAug 9, 2018
  41. Jeff HostetlerAug 9, 2018
  42. Jeff KingAug 9, 2018
  43. Elijah NewrenAug 9, 2018
  44. Jeff KingAug 9, 2018
  45. Elijah NewrenAug 9, 2018
  46. Jeff KingAug 9, 2018
  47. Elijah NewrenAug 9, 2018
  48. Junio C HamanoAug 9, 2018
  49. 0/1 clone: warn on colidding entries on checkoutNguyễn Thái Ngọc Duy, Aug 10, 2018
  50. 1/1 clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Aug 10, 2018
  51. Junio C HamanoAug 10, 2018
  52. SZEDER GáborAug 11, 2018
  53. Duy NguyenAug 11, 2018
  54. Junio C HamanoAug 13, 2018
  55. Duy NguyenAug 13, 2018
  56. Junio C HamanoAug 10, 2018
  57. clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Aug 12, 2018
  58. Jeff HostetlerAug 13, 2018
  59. Junio C HamanoAug 13, 2018
  60. Torsten BögershausenAug 15, 2018
  61. Duy NguyenAug 15, 2018
  62. config.txt: clarify core.checkStat = minimalNguyễn Thái Ngọc Duy, Aug 16, 2018
  63. Junio C HamanoAug 16, 2018
  64. Duy NguyenAug 16, 2018
  65. Junio C HamanoAug 16, 2018
  66. Junio C HamanoAug 17, 2018
  67. Duy NguyenAug 17, 2018
  68. Junio C HamanoAug 15, 2018
  69. Torsten BögershausenAug 16, 2018
  70. Duy NguyenAug 16, 2018
  71. Junio C HamanoAug 16, 2018
  72. clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Aug 17, 2018
  73. Junio C HamanoAug 17, 2018
  74. Duy NguyenAug 17, 2018
  75. Torsten BögershausenAug 17, 2018
  76. clone: report duplicate entries on case-insensitive filesystemsCarlo Marcelo Arenas Belón, Nov 19, 2018
  77. Torsten BögershausenNov 19, 2018
  78. Carlo ArenasNov 19, 2018
  79. Duy NguyenNov 19, 2018
  80. Duy NguyenNov 19, 2018
  81. Duy NguyenNov 19, 2018
  82. Duy NguyenNov 19, 2018
  83. Ramsay JonesNov 19, 2018
  84. Ramsay JonesNov 19, 2018
  85. Carlo ArenasNov 20, 2018
  86. Junio C HamanoNov 20, 2018
  87. clone: fix colliding file detection on APFSNguyễn Thái Ngọc Duy, Nov 20, 2018
  88. Ramsay JonesNov 20, 2018
  89. Carlo ArenasNov 20, 2018
  90. Duy NguyenNov 20, 2018
  91. 1/1 t5601-99: Enable colliding file detection for MINGWtboegi@web.de, Nov 22, 2018
  92. 1/1 t5601-99: Enable colliding file detection for MINGWCarlo Marcelo Arenas Belón, Nov 22, 2018
  93. Johannes SchindelinNov 23, 2018
  94. Ramsay JonesNov 19, 2018
  95. Carlo ArenasNov 19, 2018
  96. Jeff HostetlerJul 31, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.