git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/2] refs/files: use correct error type when locking fails

From
Karthik Nayak <karthik.188@gmail.com>
Date
Sep 2, 2025, 08:34 UTC
Message-ID
<20250902-587-git-fetch-1-fails-fetches-on-case-insensitive-repositories-v1-1-35e69bbb507d@gmail.com>
In-Reply-To
<20250902-587-git-fetch-1-fails-fetches-on-case-insensitive-repositories-v1-0-35e69bbb507d@gmail.com>

During the 'prepare' phase of reference transaction in the files backend, we create the lock files for references to be created. When using batched updates on case-insensitive filesystems, the transactions would be aborted if there are conflicting names such as:

  refs/heads/Foo
  refs/heads/foo

This affects all commands which were migrated to use batched updates in Git 2.51, including 'git-fetch(1)' and 'git-receive-pack(1)'. Before that, references updates would be applied serially with one transaction used per update. When users fetched multiple references on case-insensitive systems, subsequent references would simply overwrite any earlier references. So when fetching:

  refs/heads/foo: 5f34ec0bfeac225b1c854340257a65b106f70ea6
  refs/heads/Foo: ec3053b0977e83d9b67fc32c4527a117953994f3
  refs/heads/sample: 2eefd1150e06d8fca1ddfa684dec016f36bf4e56
The user would simply end up with:
  refs/heads/foo: ec3053b0977e83d9b67fc32c4527a117953994f3
  refs/heads/sample: 2eefd1150e06d8fca1ddfa684dec016f36bf4e56

This is buggy behavior since the user is never intimated about the overrides performed and missing references. Nevertheless, the user is left with a working repository with a subset of the references. Since Git 2.51, in such situations fetches would simply fail without applying any references. Which is also buggy behavior and worse off since the user is left without any references.

The error is triggered in `lock_raw_ref()` where the files backend attempts to create a lock file. When a lock file already exists the function returns a 'REF_TRANSACTION_ERROR_GENERIC'. Change this to return 'REF_TRANSACTION_ERROR_CREATE_EXISTS' instead to aid the batched update mechanism to simply reject such errors.

This bubbles the error type up to `files_transaction_prepare()` which tries to lock each reference update. So if the locking fails, we check if the rejection type can be ignored, which is done by calling `ref_transaction_maybe_set_rejected()`.

As the error type is now 'REF_TRANSACTION_ERROR_CREATE_EXISTS', the specific reference update would simply be rejected, while other updates in the transaction would continue to be applied. This allows partial application of references in case-insensitive filesystems when fetching colliding references.

While the earlier implementation allowed the last reference to be applied overriding the initial references, this change would allow the first reference to be applied while rejecting consequent collisions. This should be an OKAY compromise since with the files backend, there is no scenario possible where we would retain all colliding references.

The change only affects batched updates since batched updates will reject individual updates with non-generic errors. So specifically this would only affect:

    1. git fetch
    2. git receive-pack
    3. git update-ref --batch-updates

Let's also be more pro-active and notify users on case-insensitive filesystems about such problems by providing a brief about the issue while also recommending using the reftable backend, which doesn't have the same issue.

Reported-by: Joe Drew <joe.drew@indexexchange.com>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 builtin/fetch.c       | 21 ++++++++++++++++++---
 refs/files-backend.c  |  2 ++
 t/t1400-update-ref.sh | 24 ++++++++++++++++++++++++
 t/t5510-fetch.sh      | 22 +++++++++++++++++++++-
 4 files changed, 65 insertions(+), 4 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 24645c4653..9563abbe12 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1643,7 +1643,8 @@ static int set_head(const struct ref *remote_refs, struct remote *remote)
 
 struct ref_rejection_data {
 	int *retcode;
-	int conflict_msg_shown;
+	bool conflict_msg_shown;
+	bool case_sensitive_msg_shown;
 	const char *remote_name;
 };
 
@@ -1657,11 +1658,25 @@ static void ref_transaction_rejection_handler(const char *refname,
 {
 	struct ref_rejection_data *data = cb_data;
 
-	if (err == REF_TRANSACTION_ERROR_NAME_CONFLICT && !data->conflict_msg_shown) {
+	if (err == REF_TRANSACTION_ERROR_CREATE_EXISTS && ignore_case &&
+	    !data->case_sensitive_msg_shown) {
+		error(_("You're on a case-insensitive filesystem, and the remote you are\n"
+			"trying to fetch from has references that only differ in casing. It\n"
+			"is impossible to store such references with the 'files' backend. You\n"
+			"can either accept this as-is, in which case you won't be able to\n"
+			"store all remote references on disk. Or you can alternatively\n"
+			"migrate your repository to use the 'reftable' backend with the\n"
+			"following command:\n\n    git refs migrate --ref-format=reftable\n\n"
+			"Please keep in mind that not all implementations of Git support this\n"
+			"new format yet. So if you use tools other than Git to access this\n"
+			"repository it may not be an option to migrate to reftables.\n"));
+		data->case_sensitive_msg_shown = true;
+	} else if (err == REF_TRANSACTION_ERROR_NAME_CONFLICT &&
+		   !data->conflict_msg_shown) {
 		error(_("some local refs could not be updated; try running\n"
 			" 'git remote prune %s' to remove any old, conflicting "
 			"branches"), data->remote_name);
-		data->conflict_msg_shown = 1;
+		data->conflict_msg_shown = true;
 	} else {
 		const char *reason = ref_transaction_error_msg(err);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 088b52c740..9f58ea4858 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -776,6 +776,8 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
 			goto retry;
 		} else {
 			unable_to_lock_message(ref_file.buf, myerr, err);
+			if (myerr == EEXIST)
+				ret = REF_TRANSACTION_ERROR_CREATE_EXISTS;
 			goto error_return;
 		}
 	}
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 96648a6e5d..e37a5d83e8 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -2294,6 +2294,30 @@ do
 		)
 	'
 
+	test_expect_success CASE_INSENSITIVE_FS,REFFILES "stdin $type batch-updates existing reference" '
+		git init repo &&
+		test_when_finished "rm -fr repo" &&
+		(
+			cd repo &&
+			test_commit one &&
+			old_head=$(git rev-parse HEAD) &&
+			test_commit two &&
+			head=$(git rev-parse HEAD) &&
+
+			format_command $type "create refs/heads/foo" "$head" >stdin &&
+			format_command $type "create refs/heads/ref" "$old_head" >>stdin &&
+			format_command $type "create refs/heads/Foo" "$old_head" >>stdin &&
+			git update-ref $type --stdin --batch-updates <stdin >stdout &&
+
+			echo $head >expect &&
+			git rev-parse refs/heads/foo >actual &&
+			echo $old_head >expect &&
+			git rev-parse refs/heads/ref >actual &&
+			test_cmp expect actual &&
+			test_grep -q "reference already exists" stdout
+		)
+	'
+
 	test_expect_success "stdin $type batch-updates delete incorrect symbolic ref" '
 		git init repo &&
 		test_when_finished "rm -fr repo" &&
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index ebc696546b..57f60da81b 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -47,7 +47,13 @@ test_expect_success "clone and setup child repos" '
 		git config set branch.main.merge refs/heads/one
 	) &&
 	git clone . bundle &&
-	git clone . seven
+	git clone . seven &&
+	git clone --ref-format=reftable . case_sensitive &&
+	(
+		cd case_sensitive &&
+		git branch branch1 &&
+		git branch bRanch1
+	)
 '
 
 test_expect_success "fetch test" '
@@ -1526,6 +1532,20 @@ test_expect_success SYMLINKS 'clone does not get confused by a D/F conflict' '
 	test_path_is_missing whoops
 '
 
+test_expect_success CASE_INSENSITIVE_FS,REFFILES 'existing references in a case insensitive filesystem' '
+	test_when_finished rm -rf case_insensitive &&
+	(
+		git init --bare case_insensitive &&
+		cd case_insensitive &&
+		git remote add origin -- ../case_sensitive &&
+		test_must_fail git fetch -f origin "refs/heads/*:refs/heads/*" 2>err &&
+		test_grep "You${SQ}re on a case-insensitive filesystem" err &&
+		git rev-parse refs/heads/main >expect &&
+		git rev-parse refs/heads/branch1 >actual &&
+		test_cmp expect actual
+	)
+'
+
 . "$TEST_DIRECTORY"/lib-httpd.sh
 start_httpd
 

-- 
2.50.1
Previous: Karthik NayakNext: Chris Torek
Message 2 of 58 in “refs/files: fix issues with git-fetch on case-insensitive FS”
  1. 0/2 refs/files: fix issues with git-fetch on case-insensitive FSKarthik Nayak, Sep 2, 2025
  2. 1/2 refs/files: use correct error type when locking failsKarthik Nayak, Sep 2, 2025
  3. Chris TorekSep 2, 2025
  4. Karthik NayakSep 4, 2025
  5. Patrick SteinhardtSep 3, 2025
  6. Karthik NayakSep 3, 2025
  7. Patrick SteinhardtSep 3, 2025
  8. Junio C HamanoSep 3, 2025
  9. 2/2 refs/files: handle F/D conflicts in case-insensitive FSKarthik Nayak, Sep 2, 2025
  10. Patrick SteinhardtSep 3, 2025
  11. Karthik NayakSep 8, 2025
  12. Junio C HamanoSep 3, 2025
  13. Junio C HamanoSep 4, 2025
  14. Karthik NayakSep 8, 2025
  15. Karthik NayakSep 8, 2025
  16. 0/4 refs/files: fix issues with git-fetch on case-insensitive FSKarthik Nayak, Sep 8, 2025
  17. 1/4 refs/files: catch conflicts on case-insensitive file-systemsKarthik Nayak, Sep 8, 2025
  18. Patrick SteinhardtSep 9, 2025
  19. Karthik NayakSep 11, 2025
  20. 2/4 refs/files: use correct error type when lock existsKarthik Nayak, Sep 8, 2025
  21. Patrick SteinhardtSep 9, 2025
  22. Karthik NayakSep 11, 2025
  23. 3/4 refs/files: handle F/D conflicts in case-insensitive FSKarthik Nayak, Sep 8, 2025
  24. 4/4 refs/files: handle D/F conflicts during lockingKarthik Nayak, Sep 8, 2025
  25. Patrick SteinhardtSep 9, 2025
  26. Karthik NayakSep 12, 2025
  27. 0/4 refs/files: fix issues with git-fetch on case-insensitive FSKarthik Nayak, Sep 13, 2025
  28. 1/4 refs/files: catch conflicts on case-insensitive file-systemsKarthik Nayak, Sep 13, 2025
  29. Justin ToblerSep 16, 2025
  30. Karthik NayakSep 17, 2025
  31. Karthik NayakSep 17, 2025
  32. Justin ToblerSep 17, 2025
  33. 2/4 refs/files: use correct error type when lock existsKarthik Nayak, Sep 13, 2025
  34. Patrick SteinhardtSep 16, 2025
  35. Karthik NayakSep 17, 2025
  36. Justin ToblerSep 16, 2025
  37. Karthik NayakSep 17, 2025
  38. 3/4 refs/files: handle F/D conflicts in case-insensitive FSKarthik Nayak, Sep 13, 2025
  39. Justin ToblerSep 16, 2025
  40. Karthik NayakSep 17, 2025
  41. 4/4 refs/files: handle D/F conflicts during lockingKarthik Nayak, Sep 13, 2025
  42. Junio C HamanoSep 16, 2025
  43. Karthik NayakSep 17, 2025
  44. Toon ClaesSep 17, 2025
  45. Karthik NayakSep 17, 2025
  46. Junio C HamanoSep 17, 2025
  47. Karthik NayakSep 17, 2025
  48. Junio C HamanoSep 17, 2025
  49. Karthik NayakSep 17, 2025
  50. 0/4 refs/files: fix issues with git-fetch on case-insensitive FSKarthik Nayak, Sep 17, 2025
  51. 1/4 refs/files: catch conflicts on case-insensitive file-systemsKarthik Nayak, Sep 17, 2025
  52. 2/4 refs/files: use correct error type when lock existsKarthik Nayak, Sep 17, 2025
  53. 3/4 refs/files: handle F/D conflicts in case-insensitive FSKarthik Nayak, Sep 17, 2025
  54. 4/4 refs/files: handle D/F conflicts during lockingKarthik Nayak, Sep 17, 2025
  55. Justin ToblerSep 17, 2025
  56. Toon ClaesSep 17, 2025
  57. Junio C HamanoSep 17, 2025
  58. Junio C HamanoSep 17, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.