threads / patch / 30850

patch, 2 partssubmodule: Demonstrate failure to add with auto/safecrlf

Subject: [PATCH 1/2] submodule: Demonstrate failure to add with auto/safecrlf

## tl;dr

12 messages between Jun 20, 2012 and Jun 21, 2012. Diffs are folded; open one to read it.

replies: 11people: 4as markdown or json

Brad King· Jun 20, 2012, 14:43 UTC · lore

[PATCH 0/2] submodule add + autocrlf + safecrlf

Hi Folks,

When 'git submodule add' uses 'git config' to create a '.gitmodules' file it gets LF newlines that the subsequent 'git add --force .gitmodules' rejects if autocrlf and safecrlf are both enabled. This series adds a test and proposes a fix that simply uses '-c core.safecrlf=false' to disable safecrlf when adding '.gitmodules'.

I'm not excited by allowing a LF file in work tree that has clearly been configured to prefer CRLF, but avoiding that for .gitmodules is probably a separate issue.

-Brad
Brad King (2):
  submodule: Demonstrate failure to add with auto/safecrlf
  submodule: Tolerate auto/safecrlf when adding .gitmodules
 git-submodule.sh           |    2 +-
 t/t7400-submodule-basic.sh |   13 +++++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)
-- 
1.7.10
Brad King· Jun 20, 2012, 14:43 UTC · re: Brad King · lore

If 'core.autocrlf' and 'core.safecrlf' are both enabled then 'git submodule add' fails with an error such as

 fatal: LF would be replaced by CRLF in .gitmodules
 Failed to register submodule 'submod'

because it generates a '.gitmodules' file with LF newlines that are rejected by 'git add' under this configuration. Demonstrate this known breakage with a new test in t7400-submodule-basic covering the case.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 t/t7400-submodule-basic.sh |   13 +++++++++++++
 1 file changed, 13 insertions(+)
Show changes to t/t7400-submodule-basic.sh +13 −0
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 81827e6..5eaeb04 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -43,6 +43,7 @@ test_expect_success 'setup - hide init subdirectory' '
 
 test_expect_success 'setup - repository to add submodules to' '
 	git init addtest &&
+	git init addtest-crlf &&
 	git init addtest-ignore
 '
 
@@ -98,6 +99,18 @@ test_expect_success 'submodule add' '
 	test_cmp empty untracked
 '
 
+test_expect_failure 'submodule add with core.autocrlf and core.safecrlf' '
+	(
+		cd addtest-crlf &&
+		git config core.autocrlf true &&
+		git config core.safecrlf true &&
+		git submodule add "$submodurl" submod &&
+		echo ".gitmodules" >expect &&
+		git ls-files -- .gitmodules >actual &&
+		test_cmp expect actual
+	)
+'
+
 test_expect_success 'submodule add to .gitignored path fails' '
 	(
 		cd addtest-ignore &&
-- 
1.7.10
Brad King· Jun 20, 2012, 14:43 UTC · re: Brad King · lore

[PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

Temporarily disable 'core.safecrlf' to add '.gitmodules' so that 'git add' does not reject the LF newlines we write to the file even if both 'core.autocrlf' and 'core.safecrlf' are enabled. This fixes known breakage tested in t7400-submodule-basic.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 git-submodule.sh           |    2 +-
 t/t7400-submodule-basic.sh |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
Show changes to 2 files +2 −2

git-submodule.sh, t/t7400-submodule-basic.sh

diff --git a/git-submodule.sh b/git-submodule.sh
index 5c61ae2..ed9a54a 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -303,7 +303,7 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
-	git add --force .gitmodules ||
+	git -c core.safecrlf=false add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
 
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5eaeb04..9a4da9b 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -99,7 +99,7 @@ test_expect_success 'submodule add' '
 	test_cmp empty untracked
 '
 
-test_expect_failure 'submodule add with core.autocrlf and core.safecrlf' '
+test_expect_success 'submodule add with core.autocrlf and core.safecrlf' '
 	(
 		cd addtest-crlf &&
 		git config core.autocrlf true &&
-- 
1.7.10
Jens Lehmann· Jun 20, 2012, 17:52 UTC · re: Brad King · lore

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

Am 20.06.2012 16:43, schrieb Brad King:
> Temporarily disable 'core.safecrlf' to add '.gitmodules' so that
> 'git add' does not reject the LF newlines we write to the file
> even if both 'core.autocrlf' and 'core.safecrlf' are enabled.
> This fixes known breakage tested in t7400-submodule-basic.

Hmm, I have no objections against the intention of the patch. But as I understand it this message will reoccur when the user e.g. edits the .gitmodules file later with any editor who just writes lfs and adds it.

I don't know terribly much about crlf support but maybe flagging the .gitmodules file in .gitattributes would be a better solution here? Opinions?

Show 33 quoted lines
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  git-submodule.sh           |    2 +-
>  t/t7400-submodule-basic.sh |    2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 5c61ae2..ed9a54a 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -303,7 +303,7 @@ Use -f if you really want to add it." >&2
>  
>  	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
>  	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
> -	git add --force .gitmodules ||
> +	git -c core.safecrlf=false add --force .gitmodules ||
>  	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
>  }
>  
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index 5eaeb04..9a4da9b 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -99,7 +99,7 @@ test_expect_success 'submodule add' '
>  	test_cmp empty untracked
>  '
>  
> -test_expect_failure 'submodule add with core.autocrlf and core.safecrlf' '
> +test_expect_success 'submodule add with core.autocrlf and core.safecrlf' '
>  	(
>  		cd addtest-crlf &&
>  		git config core.autocrlf true &&
> 
Brad King· Jun 20, 2012, 18:06 UTC · re: Jens Lehmann · lore

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

On 06/20/2012 01:52 PM, Jens Lehmann wrote:
Show 14 quoted lines
> Am 20.06.2012 16:43, schrieb Brad King:
>> Temporarily disable 'core.safecrlf' to add '.gitmodules' so that
>> 'git add' does not reject the LF newlines we write to the file
>> even if both 'core.autocrlf' and 'core.safecrlf' are enabled.
>> This fixes known breakage tested in t7400-submodule-basic.
> 
> Hmm, I have no objections against the intention of the patch. But
> as I understand it this message will reoccur when the user e.g.
> edits the .gitmodules file later with any editor who just writes
> lfs and adds it.
> 
> I don't know terribly much about crlf support but maybe flagging
> the .gitmodules file in .gitattributes would be a better solution
> here? Opinions?

Once a user edits the file with an outside tool it is his/her responsibility to add .gitattributes for the file. In the reported case Git is creating the file and already knows the crlf mode when creating it.

I think Junio's proposal to teach "git config" to respect crlf configuration is a more general solution.

-Brad
Jens Lehmann· Jun 20, 2012, 18:21 UTC · re: Brad King · lore

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

Am 20.06.2012 20:06, schrieb Brad King:
Show 23 quoted lines
> On 06/20/2012 01:52 PM, Jens Lehmann wrote:
>> Am 20.06.2012 16:43, schrieb Brad King:
>>> Temporarily disable 'core.safecrlf' to add '.gitmodules' so that
>>> 'git add' does not reject the LF newlines we write to the file
>>> even if both 'core.autocrlf' and 'core.safecrlf' are enabled.
>>> This fixes known breakage tested in t7400-submodule-basic.
>>
>> Hmm, I have no objections against the intention of the patch. But
>> as I understand it this message will reoccur when the user e.g.
>> edits the .gitmodules file later with any editor who just writes
>> lfs and adds it.
>>
>> I don't know terribly much about crlf support but maybe flagging
>> the .gitmodules file in .gitattributes would be a better solution
>> here? Opinions?
> 
> Once a user edits the file with an outside tool it is his/her
> responsibility to add .gitattributes for the file.  In the reported
> case Git is creating the file and already knows the crlf mode when
> creating it.
> 
> I think Junio's proposal to teach "git config" to respect crlf
> configuration is a more general solution.
Yep.
Jeff King· Jun 20, 2012, 19:11 UTC · re: Brad King · lore

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

On Wed, Jun 20, 2012 at 02:06:43PM -0400, Brad King wrote:
Show 16 quoted lines
> > Hmm, I have no objections against the intention of the patch. But
> > as I understand it this message will reoccur when the user e.g.
> > edits the .gitmodules file later with any editor who just writes
> > lfs and adds it.
> > 
> > I don't know terribly much about crlf support but maybe flagging
> > the .gitmodules file in .gitattributes would be a better solution
> > here? Opinions?
> 
> Once a user edits the file with an outside tool it is his/her
> responsibility to add .gitattributes for the file.  In the reported
> case Git is creating the file and already knows the crlf mode when
> creating it.
> 
> I think Junio's proposal to teach "git config" to respect crlf
> configuration is a more general solution.

I don't think so. It should not be an issue at all that .gitmodules has CRLF or even mixed line endings; the config parser knows that its input is a text file and ignores the CRs already.

The only problem is when adding the file, because git is not aware that .gitmodules is, by definition, a text file. The point of safecrlf was to prevent accidents with files which are really binary, or for which mixed line endings must be preserved; .gitmodules is not such a file. We know that, but we never tell git.

We can paper over the problem by normalizing line endings when config writes it out, but that does not cover the case of somebody editing it manually and introducing mixed line endings. Nor does it help if somebody checks in a .gitmodules file with CRLF, which we then checkout on a LF-based system. Or if we do a merge between two versions with different line endings.

The only sane thing is to have a canonical in-repo representation. Fortunately we already have the infrastructure for that, and in theory it should be as easy as adding ".gitmodules text" to our built-in gitattributes (you could even do "eol=lf", but I don't see a reason not to respect the native line endings in the working tree, given that git can handle the CRLFs just fine).

I say "in theory" there because I am not sure whether specifying a file as definitely text via attributes will actually suppress the safecrlf check or not. IMHO, it should, since safecrlf is really about preventing false positives via autocrlf or text=auto.

I don't see any reason for each individual repo to have to add these attributes manually. This is a git-specific file, and the format is dictated by git. We know that it's a text file, so why not help out the user? We should possibly do the same thing for .gitattributes and .gitignore.

-Peff
Junio C Hamano· Jun 20, 2012, 19:53 UTC · re: Jeff King · lore

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

Jeff King <peff@peff.net> writes:
> This is a git-specific file, and the format is
> dictated by git. We know that it's a text file, so why not help out the
> user? We should possibly do the same thing for .gitattributes and
> .gitignore.
OK.
Jens Lehmann· Jun 21, 2012, 19:06 UTC · re: Jeff King · lore

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

Am 20.06.2012 21:11, schrieb Jeff King:
Show 11 quoted lines
> The only sane thing is to have a canonical in-repo representation.
> Fortunately we already have the infrastructure for that, and in theory
> it should be as easy as adding ".gitmodules text" to our built-in
> gitattributes (you could even do "eol=lf", but I don't see a reason not
> to respect the native line endings in the working tree, given that git
> can handle the CRLFs just fine).
> 
> I say "in theory" there because I am not sure whether specifying a file
> as definitely text via attributes will actually suppress the safecrlf
> check or not. IMHO, it should, since safecrlf is really about preventing
> false positives via autocrlf or text=auto.

A quick test shows that unfortunately theory differs from practice here. Adding ".gitmodules text" to the built-in gitattributes lets the test Brad wrote still fail. You have to use ".gitmodules eol=lf" to make it pass. I stopped digging deeper at this point.

Show 5 quoted lines
> I don't see any reason for each individual repo to have to add these
> attributes manually. This is a git-specific file, and the format is
> dictated by git. We know that it's a text file, so why not help out the
> user? We should possibly do the same thing for .gitattributes and
> .gitignore.

I really like this approach. (And in the long run would like to see a ini-file aware merge driver being used for the .gitmodules file too, which would just merge submodules added in different branches instead of producing a conflict)

Junio C Hamano· Jun 20, 2012, 17:49 UTC · re: Brad King · lore

Re: [PATCH 0/2] submodule add + autocrlf + safecrlf

Brad King <brad.king@kitware.com> writes:
Show 10 quoted lines
> When 'git submodule add' uses 'git config' to create a
> '.gitmodules' file it gets LF newlines that the subsequent
> 'git add --force .gitmodules' rejects if autocrlf and
> safecrlf are both enabled.  This series adds a test and
> proposes a fix that simply uses '-c core.safecrlf=false'
> to disable safecrlf when adding '.gitmodules'.
>
> I'm not excited by allowing a LF file in work tree that
> has clearly been configured to prefer CRLF, but avoiding
> that for .gitmodules is probably a separate issue.

I have a suspicion that "git config" should be taught about this kind of thing instead.

Shoudn't your .git/config file that is outside the revision control also end with CRLF if your platform and project prefer CRLF over LF?

Brad King· Jun 20, 2012, 18:09 UTC · re: Junio C Hamano · lore

Re: [PATCH 0/2] submodule add + autocrlf + safecrlf

On 06/20/2012 01:49 PM, Junio C Hamano wrote:
Show 5 quoted lines
> I have a suspicion that "git config" should be taught about this
> kind of thing instead.
> 
> Shoudn't your .git/config file that is outside the revision control
> also end with CRLF if your platform and project prefer CRLF over LF?

That would be reasonable, but is beyond the scope I'm willing to tackle myself.

I don't actually have a project like this so I have no strong opinion on this issue. I discovered the problem by accident and have already worked around it in the obscure case it matters for me.

Perhaps only the first patch in the series is worth inclusion. It can become the beginning of a series if someone wants to address handling crlf in config files. Note that in the case I discovered this the crlf configuration was in ~/.gitconfig so the project knew nothing about it and had no .gitattributes.

-Brad
Junio C Hamano· Jun 20, 2012, 19:24 UTC · re: Brad King · lore

Re: [PATCH 0/2] submodule add + autocrlf + safecrlf

Brad King <brad.king@kitware.com> writes:
Show 12 quoted lines
> On 06/20/2012 01:49 PM, Junio C Hamano wrote:
>> I have a suspicion that "git config" should be taught about this
>> kind of thing instead.
>> 
>> Shoudn't your .git/config file that is outside the revision control
>> also end with CRLF if your platform and project prefer CRLF over LF?
>
> That would be reasonable, but is beyond the scope I'm willing to
> tackle myself.
>
> I don't actually have a project like this so I have no strong
> opinion on this issue.

If that is the case, my preference would be to wait until that "git config" change happens (provided that it is a reasonably way forward---it may turn out to be a bad idea) and do nothing to clutter "git submodule" at all. In the meantime, if there are real projects that are hurt by this, we can tell them to explicitly specify that .gitmodules is a LF text file in their .gitattributes.

← back to recent threads