Re: Git clone and case sensitivity

From: Duy Nguyen <pclouds@gmail.com>
Date: Jul 28, 2018, 04:36 UTC
Message-ID: <20180728043559.GA29185@duynguyen.home>
In-Reply-To: <20180727205909.GC376343@genre.crustytoothpaste.net>

On Fri, Jul 27, 2018 at 08:59:09PM +0000, brian m. carlson wrote:

Show 15 quoted lines

> On Fri, Jul 27, 2018 at 11:59:33AM +0200, Paweł Paruzel wrote:
> > Hi,
> > 
> > Lately, I have been wondering why my test files in repo are modified
> > after I clone it. It turned out to be two files: boolStyle_t_f and
> > boolStyle_T_F.
> > The system that pushed those files was case sensitive while my mac
> > after High Sierra update had APFS which is by default
> > case-insensitive. I highly suggest that git clone threw an exception
> > when files are case sensitive and being cloned to a case insensitive
> > system. This has caused problems with overriding files for test cases
> > without any warning.
> 
> If we did what you proposed, it would be impossible to clone such a
> repository on a case-insensitive system.

I agree throwing a real exception would be bad. But how about detecting the problem and trying our best to keep the repo in somewhat usable state like this?

This patch uses sparse checkout to hide all those paths that we fail to checkout, so you can still have a clean worktree to do things, as long as you don't touch those paths.

-- 8< --

diff --git a/builtin/clone.c b/builtin/clone.c
index 1d939af9d8..a6b5e2c948 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -711,6 +711,30 @@ static void update_head(const struct ref *our, const struct ref *remote,
 	}
 }
 
+static int enable_sparse_checkout_on_icase_fs(struct index_state *istate)
+{
+	int i;
+	int skip_count = 0;
+	FILE *fp = fopen(git_path("info/sparse"), "a+");
+
+	for (i = 0; i < istate->cache_nr; i++) {
+		struct cache_entry *ce = istate->cache[i];
+		if (!ce_skip_worktree(ce))
+			continue;
+		if (!skip_count) {
+			git_config_set_multivar_gently("core.sparseCheckout",
+						       "true",
+						       CONFIG_REGEX_NONE, 0);
+			fprintf(fp, "# List of paths hidden by 'git clone'\n");
+		}
+		fprintf(fp, "/%s\n", ce->name);
+		skip_count++;
+	}
+	fclose(fp);
+
+	return skip_count;
+}
+
 static int checkout(int submodule_progress)
 {
 	struct object_id oid;
@@ -751,6 +775,7 @@ static int checkout(int submodule_progress)
 	opts.verbose_update = (option_verbosity >= 0);
 	opts.src_index = &the_index;
 	opts.dst_index = &the_index;
+	opts.clone_checkout = 1;
 
 	tree = parse_tree_indirect(&oid);
 	parse_tree(tree);
@@ -761,6 +786,12 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
+	if (enable_sparse_checkout_on_icase_fs(&the_index))
+		warning("Paths that differ only in case are detected "
+			"and will not work correctly on this case-insensitive "
+			"filesystem. Sparse checkout has been enabled to hide "
+			"these paths.");
+
 	err |= run_hook_le(NULL, "post-checkout", sha1_to_hex(null_sha1),
 			   oid_to_hex(&oid), "1", NULL);
 
diff --git a/cache.h b/cache.h
index 8b447652a7..9ecf7ad952 100644
--- a/cache.h
+++ b/cache.h
@@ -1455,6 +1455,7 @@ struct checkout {
 	unsigned force:1,
 		 quiet:1,
 		 not_new:1,
+		 set_skipworktree_on_updated:1,
 		 refresh_cache:1;
 };
 #define CHECKOUT_INIT { NULL, "" }
diff --git a/entry.c b/entry.c
index b5d1d3cf23..ba21db63e7 100644
--- a/entry.c
+++ b/entry.c
@@ -447,6 +447,11 @@ int checkout_entry(struct cache_entry *ce,
 
 		if (!changed)
 			return 0;
+		if (state->set_skipworktree_on_updated) {
+			ce->ce_flags |= CE_SKIP_WORKTREE;
+			state->istate->cache_changed |= CE_ENTRY_CHANGED;
+			return 0;
+		}
 		if (!state->force) {
 			if (!state->quiet)
 				fprintf(stderr,
diff --git a/unpack-trees.c b/unpack-trees.c
index 66741130ae..a8a24e0b13 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -358,6 +358,7 @@ static int check_updates(struct unpack_trees_options *o)
 	state.quiet = 1;
 	state.refresh_cache = 1;
 	state.istate = index;
+	state.set_skipworktree_on_updated = o->clone_checkout;
 
 	progress = get_progress(o);
 
diff --git a/unpack-trees.h b/unpack-trees.h
index c2b434c606..8ebe2e2ec5 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -49,6 +49,7 @@ struct unpack_trees_options {
 		     aggressive,
 		     skip_unmerged,
 		     initial_checkout,
+		     clone_checkout,
 		     diff_index_cached,
 		     debug_unpack,
 		     skip_sparse_checkout,
-- 8< --

> While this might be fine for a closed system such as inside a
> company, this would make many open source repositories unusable,
> even when the files differing in case are nonfunctional (like README
> and readme).
> 
> This is actually one of a few ways people can make repositories that
> will show as modified on Windows or macOS systems, due to limitations in
> those OSes.  If you want to be sure that your repository is unmodified
> after clone, you can ensure that the output of git status --porcelain is
> empty, such as by checking for a zero exit from
> "test $(git status --porcelain | wc -l) -eq 0".
> -- 
> brian m. carlson: Houston, Texas, US
> OpenPGP: https://keybase.io/bk2204