git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/2] refs/files: handle F/D conflicts in case-insensitive FS

From
Patrick Steinhardt <ps@pks.im>
Date
Sep 3, 2025, 07:40 UTC
Message-ID
<aLfw-peLY8NEKSZd@pks.im>
In-Reply-To
<20250902-587-git-fetch-1-fails-fetches-on-case-insensitive-repositories-v1-2-35e69bbb507d@gmail.com>
On Tue, Sep 02, 2025 at 10:34:26AM +0200, Karthik Nayak wrote:
Show 6 quoted lines
> Similar to the previous commit, when using the files-backend on
> case-insensitive filesystems, there is possibility of hitting F/D
> conflicts when creating references within a single transaction, such as:
> 
>   - 'refs/heads/foo'
>   - 'refs/heads/Foo/bar'
Great, I wanted to ask about this scenario.
Show 21 quoted lines
> Ideally such conflicts are caught in `refs_verify_refnames_available()`
> which is responsible for checking F/D conflicts within a given
> transaction. This utility function is shared across the reference
> backends. As such, it doesn't consider the issues of using a
> case-insensitive, which only affects the files-backend.
> 
> While one solution would be to make the function aware of such issues.
> This feels like leaking implementation details of file-backend specific
> issues into the utility function. So opt for the more simpler option, of
> lowercasing all references sent to this function when on a
> case-insensitive filesystem and operating on the files-backend.
> 
> To do this, simply use a `struct strbuf` to convert the refname to a
> lower case and append it to the list of refnames to be checked. Since we
> use a `struct strbuf` and the memory is cleared right after, make sure
> that the string list duplicates all provided string.
> 
> Without this change, the user would simply be left with a repository
> with '.lock' files which were created in the 'prepare' phase of the
> transaction, as the 'commit' phase would simply abort and not do the
> necessary cleanup.

Oh, that's a clever hack. Does this also work for the case where we have preexisting refs already that differ only in casing? I guess it should given that the lookups we perform should yield those refs regardless of their casing.

In any case, if we don't already have such a test it would be great to also verify that this works as expected.

Show 22 quoted lines
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 9f58ea4858..466cdfe121 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -869,8 +869,23 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs,
>  		 * If the ref did not exist and we are creating it, we have to
>  		 * make sure there is no existing packed ref that conflicts
>  		 * with refname. This check is deferred so that we can batch it.
> +		 *
> +		 * For case-insensitive filesystems, we should also check for F/D
> +		 * conflicts between 'foo' and 'Foo/bar'. So let's lowercase
> +		 * the refname.
>  		 */
> -		item = string_list_append(refnames_to_check, refname);
> +		if (ignore_case) {
> +			struct strbuf lower = STRBUF_INIT;
> +
> +			strbuf_addstr(&lower, refname);
> +			strbuf_tolower(&lower);
> +
> +			item = string_list_append(refnames_to_check, lower.buf);
> +			strbuf_release(&lower);

Can we use `string_list_append_nodup()` together with `strbuf_detach()` here to avoid one memory allocation?

Show 24 quoted lines
> +		} else {
> +			item = string_list_append(refnames_to_check, refname);
> +		}
> +
>  		item->util = xmalloc(sizeof(update_idx));
>  		memcpy(item->util, &update_idx, sizeof(update_idx));
>  	}
> diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
> index 57f60da81b..84dc68e5f3 100755
> --- a/t/t5510-fetch.sh
> +++ b/t/t5510-fetch.sh
> @@ -53,6 +53,12 @@ test_expect_success "clone and setup child repos" '
>  		cd case_sensitive &&
>  		git branch branch1 &&
>  		git branch bRanch1
> +	) &&
> +	git clone --ref-format=reftable . case_sensitive_fd &&
> +	(
> +		cd case_sensitive_fd &&
> +		git branch foo/bar &&
> +		git branch Foo
>  	)
>  '
>  
Nice idea to use the reftable format here.
Show 17 quoted lines
> @@ -1546,6 +1552,20 @@ test_expect_success CASE_INSENSITIVE_FS,REFFILES 'existing references in a case
>  	)
>  '
>  
> +test_expect_success CASE_INSENSITIVE_FS,REFFILES 'F/D conflict on case insensitive filesystem' '
> +	test_when_finished rm -rf case_insensitive &&
> +	(
> +		git init --bare case_insensitive &&
> +		cd case_insensitive &&
> +		git remote add origin -- ../case_sensitive_fd &&
> +		test_must_fail git fetch -f origin "refs/heads/*:refs/heads/*" 2>err &&
> +		test_grep "failed: refname conflict" err &&
> +		git rev-parse refs/heads/main >expect &&
> +		git rev-parse refs/heads/foo/bar >actual &&
> +		test_cmp expect actual
> +	)
> +'

Okay, so we again only end up with one of these references, which is the best we can do.

atrick
Previous: Karthik NayakNext: Karthik Nayak
Message 10 of 58 in “refs/files: fix issues with git-fetch on case-insensitive FS”
  1. 0/2 refs/files: fix issues with git-fetch on case-insensitive FSKarthik Nayak, Sep 2, 2025
  2. 1/2 refs/files: use correct error type when locking failsKarthik Nayak, Sep 2, 2025
  3. Chris TorekSep 2, 2025
  4. Karthik NayakSep 4, 2025
  5. Patrick SteinhardtSep 3, 2025
  6. Karthik NayakSep 3, 2025
  7. Patrick SteinhardtSep 3, 2025
  8. Junio C HamanoSep 3, 2025
  9. 2/2 refs/files: handle F/D conflicts in case-insensitive FSKarthik Nayak, Sep 2, 2025
  10. Patrick SteinhardtSep 3, 2025
  11. Karthik NayakSep 8, 2025
  12. Junio C HamanoSep 3, 2025
  13. Junio C HamanoSep 4, 2025
  14. Karthik NayakSep 8, 2025
  15. Karthik NayakSep 8, 2025
  16. 0/4 refs/files: fix issues with git-fetch on case-insensitive FSKarthik Nayak, Sep 8, 2025
  17. 1/4 refs/files: catch conflicts on case-insensitive file-systemsKarthik Nayak, Sep 8, 2025
  18. Patrick SteinhardtSep 9, 2025
  19. Karthik NayakSep 11, 2025
  20. 2/4 refs/files: use correct error type when lock existsKarthik Nayak, Sep 8, 2025
  21. Patrick SteinhardtSep 9, 2025
  22. Karthik NayakSep 11, 2025
  23. 3/4 refs/files: handle F/D conflicts in case-insensitive FSKarthik Nayak, Sep 8, 2025
  24. 4/4 refs/files: handle D/F conflicts during lockingKarthik Nayak, Sep 8, 2025
  25. Patrick SteinhardtSep 9, 2025
  26. Karthik NayakSep 12, 2025
  27. 0/4 refs/files: fix issues with git-fetch on case-insensitive FSKarthik Nayak, Sep 13, 2025
  28. 1/4 refs/files: catch conflicts on case-insensitive file-systemsKarthik Nayak, Sep 13, 2025
  29. Justin ToblerSep 16, 2025
  30. Karthik NayakSep 17, 2025
  31. Karthik NayakSep 17, 2025
  32. Justin ToblerSep 17, 2025
  33. 2/4 refs/files: use correct error type when lock existsKarthik Nayak, Sep 13, 2025
  34. Patrick SteinhardtSep 16, 2025
  35. Karthik NayakSep 17, 2025
  36. Justin ToblerSep 16, 2025
  37. Karthik NayakSep 17, 2025
  38. 3/4 refs/files: handle F/D conflicts in case-insensitive FSKarthik Nayak, Sep 13, 2025
  39. Justin ToblerSep 16, 2025
  40. Karthik NayakSep 17, 2025
  41. 4/4 refs/files: handle D/F conflicts during lockingKarthik Nayak, Sep 13, 2025
  42. Junio C HamanoSep 16, 2025
  43. Karthik NayakSep 17, 2025
  44. Toon ClaesSep 17, 2025
  45. Karthik NayakSep 17, 2025
  46. Junio C HamanoSep 17, 2025
  47. Karthik NayakSep 17, 2025
  48. Junio C HamanoSep 17, 2025
  49. Karthik NayakSep 17, 2025
  50. 0/4 refs/files: fix issues with git-fetch on case-insensitive FSKarthik Nayak, Sep 17, 2025
  51. 1/4 refs/files: catch conflicts on case-insensitive file-systemsKarthik Nayak, Sep 17, 2025
  52. 2/4 refs/files: use correct error type when lock existsKarthik Nayak, Sep 17, 2025
  53. 3/4 refs/files: handle F/D conflicts in case-insensitive FSKarthik Nayak, Sep 17, 2025
  54. 4/4 refs/files: handle D/F conflicts during lockingKarthik Nayak, Sep 17, 2025
  55. Justin ToblerSep 17, 2025
  56. Toon ClaesSep 17, 2025
  57. Junio C HamanoSep 17, 2025
  58. Junio C HamanoSep 17, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.