threads / discuss / 24588

noob user, want checkins to all be forced to LF terminated lines

Subject: noob user, want checkins to all be forced to LF terminated lines

## tl;dr

16 messages between Jul 31, 2010 and Aug 4, 2010.

replies: 15people: 7as markdown or json

Walter Bright· Jul 31, 2010, 04:23 UTC · lore
I've just started with git. Exactly what do I put in $HOME/.gitconfig ?

I find the text at http://www.kernel.org/pub/software/scm/git/docs/gitattributes.html#_checking_out_and_checking_in

to be confusing. Which of the text, eol, crlf, or autocrlf attributes do I set, and exactly what text do I put in the config file?

Thanks for any help!
Jonathan Nieder· Jul 31, 2010, 04:49 UTC · re: Walter Bright · lore

Re: noob user, want checkins to all be forced to LF terminated lines

Walter Bright wrote:
> I've just started with git.
I thought I saw you here years ago. :)
> Exactly what do I put in $HOME/.gitconfig ?
Well, naturally it depends on what you want to happen.

If you just want to make sure any new files you commit are tracked with simple LF line endings, you can use

	[core]
		autocrlf = input

With this setting, Git will not do any munging to files in the work tree in any way (unless there is a .gitattributes file requesting to do so).

That is an _altruistic_ setting to use. It ensures you do not pollute history with some alternative line-ending, but your own work tree may not necessarily match the cleaned up versions you are checking in; so if you try to "git add" and then "touch" a file with CRLF line endings with this setting enabled, you may be surprised at the result! (Though a simple "git checkout file.c" afterwards should fix up the line endings in the work tree.)

If you want to make sure text files in the work tree use LF line endings and you are using a recent version of Git, use the above setting or

	[core]
		eol = lf

On Unix-y systems, you do not have to do that, since it is the default. On Windows, the "[core] autocrlf" setting is set up by default in /etc/gitconfig so you would probably want to override that with

	[core]
		autocrlf = false
if you are not setting it to input.

Which files are text files? you may ask. By default (unless autocrlf is enabled), Git treats files as raw data; to get it to futz with line endings, you have to declare your text files in a file named .gitattributes in the tracked tree.

	* crlf
	*.jpg -crlf
	*.png -crlf

The keyword crlf here means “apply line-ending conversions” and nothing more. In particular, it does not represent the preferred line ending.

If everyone for which you want these setting to take effect uses a recent version of git, you can write “text” instead of “crlf” if you prefer.

Hope that helps, Jonathan

Walter Bright· Jul 31, 2010, 05:14 UTC · re: Jonathan Nieder · lore

Re: noob user, want checkins to all be forced to LF terminated lines

Jonathan Nieder wrote:
Show 5 quoted lines
> Walter Bright wrote:
> 
>> I've just started with git.
> 
> I thought I saw you here years ago. :)
I've been around, just not in git.
Show 13 quoted lines
>> Exactly what do I put in $HOME/.gitconfig ?
> 
> Well, naturally it depends on what you want to happen.
> 
> If you just want to make sure any new files you commit are tracked
> with simple LF line endings, you can use
> 
> 	[core]
> 		autocrlf = input
> 
> With this setting, Git will not do any munging to files in the work
> tree in any way (unless there is a .gitattributes file requesting to
> do so).

git is installed under Ubuntu, but I'll be checking in files that I edit on both Windows and Ubuntu, so the line endings will vary depending on which platform I last editted the file on. Hence, I want to force them all to be LF upon checkin.

Show 7 quoted lines
> That is an _altruistic_ setting to use.  It ensures you do not pollute
> history with some alternative line-ending, but your own work tree may
> not necessarily match the cleaned up versions you are checking in; so
> if you try to "git add" and then "touch" a file with CRLF line endings
> with this setting enabled, you may be surprised at the result!
> (Though a simple "git checkout file.c" afterwards should fix up the
> line endings in the work tree.)
Show 6 quoted lines
> If you want to make sure text files in the work tree use LF line
> endings and you are using a recent version of Git, use the above
> setting or
> 
> 	[core]
> 		eol = lf

So this changes the file in the repository to lf only, but not in the worktree? That's what I want.

Show 18 quoted lines
> On Unix-y systems, you do not have to do that, since it is the
> default.  On Windows, the "[core] autocrlf" setting is set up
> by default in /etc/gitconfig so you would probably want to
> override that with
> 
> 	[core]
> 		autocrlf = false
> 
> if you are not setting it to input.
> 
> Which files are text files? you may ask.  By default (unless
> autocrlf is enabled), Git treats files as raw data; to get it
> to futz with line endings, you have to declare your text files
> in a file named .gitattributes in the tracked tree.
> 
> 	* crlf
> 	*.jpg -crlf
> 	*.png -crlf
In the tracked tree? The documentation:
http://www.kernel.org/pub/software/scm/git/docs/gitattributes.html#_checking_out_and_checking_in
says it goes in:
  $GIT_DIR/info/attributes, .gitattributes

so I'm confused again. Does .gitattributes go in $GIT_DIR, or in $GIT_DIR/info ? And what if both of those files are there, which one 'wins' ?

Show 7 quoted lines
> The keyword crlf here means “apply line-ending conversions” and
> nothing more.  In particular, it does not represent the preferred
> line ending.
> 
> If everyone for which you want these setting to take effect uses a
> recent version of git, you can write “text” instead of “crlf” if
> you prefer.
git --version says I'm using 1.5.6.3
> Hope that helps,
Yes, it does, thank you!

A final question: where does the repository actually go (so I can back it up)? This is a local thing, I'm not trying to set up a networked or remote repository, so it'll be the default location.

Ilari Liusvaara· Jul 31, 2010, 05:39 UTC · re: Walter Bright · lore

Re: noob user, want checkins to all be forced to LF terminated lines

On Fri, Jul 30, 2010 at 10:14:40PM -0700, Walter Bright wrote:
> 
> A final question: where does the repository actually go (so I can
> back it up)? This is a local thing, I'm not trying to set up a
> networked or remote repository, so it'll be the default location.

.git directory in root of working copy. One way to backup is just to do recursive backup of entiere working copy[1].

[1] But upon restore, the working copy cache will be wrong and needs to be rebuilt (git update-index --refresh).

-Ilari
Walter Bright· Jul 31, 2010, 05:45 UTC · re: Ilari Liusvaara · lore

Re: noob user, want checkins to all be forced to LF terminated lines

Ilari Liusvaara wrote:
Show 9 quoted lines
> On Fri, Jul 30, 2010 at 10:14:40PM -0700, Walter Bright wrote:
>> A final question: where does the repository actually go (so I can
>> back it up)? This is a local thing, I'm not trying to set up a
>> networked or remote repository, so it'll be the default location.
> 
> .git directory in root of working copy. One way to backup is just
> to do recursive backup of entiere working copy[1].
> 
> [1] But upon restore, the working copy cache will be wrong
Why? Is it someplace else?
> and needs to be rebuilt (git update-index --refresh).
Ilari Liusvaara· Jul 31, 2010, 05:57 UTC · re: Walter Bright · lore

Re: noob user, want checkins to all be forced to LF terminated lines

On Fri, Jul 30, 2010 at 10:45:36PM -0700, Walter Bright wrote:
> >[1] But upon restore, the working copy cache will be wrong
> 
> Why? Is it someplace else?

It is inside .git directory and will be caught by full recursive backup. Unfortunately, it contains i-node numbers, which aren't preserved through backup and restore.

The wrong i-node numbers would then confuse git (false positives
in modification detection). Fortunately, this data can be rebuilt
with single command (see below).
 
> >and needs to be rebuilt (git update-index --refresh).
-Ilari
Walter Bright· Jul 31, 2010, 06:24 UTC · re: Ilari Liusvaara · lore

Re: noob user, want checkins to all be forced to LF terminated lines

Ilari Liusvaara wrote:
Show 12 quoted lines
> On Fri, Jul 30, 2010 at 10:45:36PM -0700, Walter Bright wrote:
> 
>>> [1] But upon restore, the working copy cache will be wrong
>> Why? Is it someplace else?
> 
> It is inside .git directory and will be caught by full recursive
> backup. Unfortunately, it contains i-node numbers, which aren't
> preserved through backup and restore.
> 
> The wrong i-node numbers would then confuse git (false positives
> in modification detection). Fortunately, this data can be rebuilt
> with single command (see below).

Hmm. One thing I wanted to get away from from Windows was the inability to restore files by just copying the tree.

>>> and needs to be rebuilt (git update-index --refresh).
That's crucial to know. Thanks!
David Aguilar· Jul 31, 2010, 15:12 UTC · re: Walter Bright · lore

Re: noob user, want checkins to all be forced to LF terminated lines

On Fri, Jul 30, 2010 at 11:24:34PM -0700, Walter Bright wrote:
Show 16 quoted lines
> Ilari Liusvaara wrote:
> >On Fri, Jul 30, 2010 at 10:45:36PM -0700, Walter Bright wrote:
> >
> >>>[1] But upon restore, the working copy cache will be wrong
> >>Why? Is it someplace else?
> >
> >It is inside .git directory and will be caught by full recursive
> >backup. Unfortunately, it contains i-node numbers, which aren't
> >preserved through backup and restore.
> >
> >The wrong i-node numbers would then confuse git (false positives
> >in modification detection). Fortunately, this data can be rebuilt
> >with single command (see below).
> 
> Hmm. One thing I wanted to get away from from Windows was the
> inability to restore files by just copying the tree.

I think we gave you too much information. Just copy the tree, that's all there is to see here...

> >>>and needs to be rebuilt (git update-index --refresh).
> 
> That's crucial to know. Thanks!
Actually, I think that's too much to know.

Casual git users can just say 'git status' and git will DTRT updating the index itself.

Sorry if the information in this thread made you think that there's "something else" that you have to worry about backing up. Nope, it's all just .git.

-- 
		David
Jonathan Nieder· Jul 31, 2010, 05:44 UTC · re: Walter Bright · lore

Re: noob user, want checkins to all be forced to LF terminated lines

Hi again,
Some clarifications.
Walter Bright wrote:
> git is installed under Ubuntu, but I'll be checking in files that I
> edit on both Windows and Ubuntu, so the line endings will vary
> depending on which platform I last editted the file on. Hence, I
> want to force them all to be LF upon checkin.

"[core] autocrlf = input" would work. With this setting, the work tree is considered sacred (i.e., not touched in any magical way at all) but content checked in that looks like text is converted to use LF.

Using .gitattributes you can override the autodetection (see convert.c::is_binary) of text files.

Show 5 quoted lines
>>	[core]
>>		eol = lf
>
> So this changes the file in the repository to lf only, but not in
> the worktree? That's what I want.

The opposite. This makes the file in the worktree use lf on checkout, if it is known to be a text file.

On Linux it is a no-op. For files known to be text files, the version checked in _always_ uses LF anyway. The setting "[core] eol = lf" is just a way to turn off "[core] eol = crlf".

Show 11 quoted lines
> In the tracked tree? The documentation:
> 
> http://www.kernel.org/pub/software/scm/git/docs/gitattributes.html#_checking_out_and_checking_in
> 
> says it goes in:
> 
>  $GIT_DIR/info/attributes, .gitattributes
> 
> so I'm confused again. Does .gitattributes go in $GIT_DIR, or in
> $GIT_DIR/info ? And what if both of those files are there, which one
> 'wins' ?

Though I said "in the tracked tree", it is generally the file in the worktree that counts. There can be .gitattributes files in any subdirectory of the toplevel of the work tree.

.git/info/attributes is a place to put local attribute settings that should not be tracked. It has higher precedence than the .gitattributes files. As the gitattributes(5) page says:

	git consults $GIT_DIR/info/attributes file (which has the
	highest precedence), .gitattributes file in the same
	directory as the path in question, and its parent
	directories up to the toplevel of the work tree (the
	further the directory that contains .gitattributes is
	from the path in question, the lower its precedence).
Show 5 quoted lines
>> If everyone for which you want these setting to take effect uses a
>> recent version of git, you can write “text” instead of “crlf” if
>> you prefer.
>
> git --version says I'm using 1.5.6.3
Not recent enough. :)

Actually versions before 1.7.2 do not have the "[core] eol" configuration, either, so there is one less thing to worry about.

> A final question: where does the repository actually go (so I can
> back it up)?
The subdirectory .git of the top level of the worktree.

You can back up with "git clone" or "git bundle", but copying the .git directory also works fine.

Regards, Jonathan

Walter Bright· Jul 31, 2010, 06:32 UTC · re: Jonathan Nieder · lore

Re: noob user, want checkins to all be forced to LF terminated lines

Jonathan Nieder wrote:
Show 30 quoted lines
> Hi again,
> 
> Some clarifications.
> 
> Walter Bright wrote:
> 
>> git is installed under Ubuntu, but I'll be checking in files that I
>> edit on both Windows and Ubuntu, so the line endings will vary
>> depending on which platform I last editted the file on. Hence, I
>> want to force them all to be LF upon checkin.
> 
> "[core] autocrlf = input" would work.  With this setting, the work
> tree is considered sacred (i.e., not touched in any magical way at
> all) but content checked in that looks like text is converted to
> use LF.
> 
> Using .gitattributes you can override the autodetection (see
> convert.c::is_binary) of text files.
> 
>>> 	[core]
>>> 		eol = lf
>> So this changes the file in the repository to lf only, but not in
>> the worktree? That's what I want.
> 
> The opposite.  This makes the file in the worktree use lf on
> checkout, if it is known to be a text file.
> 
> On Linux it is a no-op.  For files known to be text files, the version
> checked in _always_ uses LF anyway.  The setting "[core] eol = lf" is
> just a way to turn off "[core] eol = crlf".

So I'm lost again. If the version in the repository is always converted to LF, then why do I need to set autocrlf=input ?

Show 27 quoted lines
> 
>> In the tracked tree? The documentation:
>>
>> http://www.kernel.org/pub/software/scm/git/docs/gitattributes.html#_checking_out_and_checking_in
>>
>> says it goes in:
>>
>>  $GIT_DIR/info/attributes, .gitattributes
>>
>> so I'm confused again. Does .gitattributes go in $GIT_DIR, or in
>> $GIT_DIR/info ? And what if both of those files are there, which one
>> 'wins' ?
> 
> Though I said "in the tracked tree", it is generally the file in the
> worktree that counts.  There can be .gitattributes files in any
> subdirectory of the toplevel of the work tree.
> 
> .git/info/attributes is a place to put local attribute settings that
> should not be tracked.  It has higher precedence than the
> .gitattributes files.  As the gitattributes(5) page says:
> 
> 	git consults $GIT_DIR/info/attributes file (which has the
> 	highest precedence), .gitattributes file in the same
> 	directory as the path in question, and its parent
> 	directories up to the toplevel of the work tree (the
> 	further the directory that contains .gitattributes is
> 	from the path in question, the lower its precedence).
Ok, got it!
Show 7 quoted lines
> 
>>> If everyone for which you want these setting to take effect uses a
>>> recent version of git, you can write “text” instead of “crlf” if
>>> you prefer.
>> git --version says I'm using 1.5.6.3
> 
> Not recent enough. :)
Eh, it's what ubuntu's apt-get gave me.
Show 10 quoted lines
> Actually versions before 1.7.2 do not have the "[core] eol"
> configuration, either, so there is one less thing to worry about.
> 
>> A final question: where does the repository actually go (so I can
>> back it up)?
> 
> The subdirectory .git of the top level of the worktree.
> 
> You can back up with "git clone" or "git bundle", but copying the
> .git directory also works fine.

Why would "git clone" even exist if copying the directory works? Is it the embedded inode problem that Ilari mentioned?

Ilari Liusvaara· Jul 31, 2010, 06:41 UTC · re: Walter Bright · lore

Re: noob user, want checkins to all be forced to LF terminated lines

On Fri, Jul 30, 2010 at 11:32:53PM -0700, Walter Bright wrote:
> 
> Why would "git clone" even exist if copying the directory works? Is
> it the embedded inode problem that Ilari mentioned?

Cloning remote repositories. But there are URLs for local repositories too, so those can be cloned as well.

-Ilari
Jonathan Nieder· Jul 31, 2010, 20:38 UTC · re: Walter Bright · lore

Re: noob user, want checkins to all be forced to LF terminated lines

Walter Bright wrote:
> Why would "git clone" even exist if copying the directory works?

As Ilari said, it works remotely. It’s even faster than just copying the directory (since it has to do less reading), and it only copies the repository, not the work tree or .git/config, which may or may not be what you want.

More importantly, the reason I mentioned it was this: if you clone to make your first backup, you are more likely (when appropriate) to use "git fetch" instead of rsync to update the backup afterwards. And for that operation, "git fetch" is much more efficient.

Anyway, feel free to ignore this advice. "cp -a" works fine already and is sometimes just the right thing to do.

Jonathan
Walter Bright· Jul 31, 2010, 20:51 UTC · re: Jonathan Nieder · lore

Re: noob user, want checkins to all be forced to LF terminated lines

Jonathan Nieder wrote:
> Anyway, feel free to ignore this advice.  "cp -a" works fine already
> and is sometimes just the right thing to do.

Thanks for the info. I am so thoroughly fed up with applications that store their data in ways that cannot be backed up / restored with straightforward file tree copying. I don't know why anyone else puts up with that, either.

Jay Soffian· Aug 3, 2010, 23:56 UTC · re: Walter Bright · lore

Re: noob user, want checkins to all be forced to LF terminated lines

On Sat, Jul 31, 2010 at 2:32 AM, Walter Bright <boost@digitalmars.com> wrote:
> So I'm lost again. If the version in the repository is always converted to
> LF, then why do I need to set autocrlf=input ?
Let's start over. :-)

Before git-1.7.2, EOL conversion was rather insane. It's fixed in 1.7.2, so I'm going to start by explaining what happens with that version and later.

Option 1 (text/eol attributes):
- Normally git will perform no EOL conversion. Files are committed
into the repo exactly as you see them in your checkout.
- To have git perform EOL conversion, you need to either explicitly
tell it which files are text, or let it autodetect. You normally do
this via the .gitattributes file using:
  <pattern> text
or
  * text=auto

In the former case, you're telling git explicitly which files are text. In the latter case, you're telling git to do its best to detect which files are text.

Files which are explicitly tagged as text, or auto-detected as text, will have their EOLs converted to LF on check-in, and converted to core.eol on check-out. core.eol defaults to "native" which means LF on Unix and CRLF on Cygwin, but you can explicitly set it to "lf" or "crlf" if you desire.

Certain files you may wish to specify their EOL in the working copy explicitly. You do this with the eol attribute. e.g.:

  <pattern> eol=crlf
  <pattern> eol=lf

Files which are tagged with eol={crlf,lf} are implicitly text, and will have their EOLs converted to LF on check-in, and converted to the specified EOL on check-out.

Okay, so that's how you ensure that certain files have LFs in the repo, and the desired EOL in the working-copy.

Option 2 (core.autocrlf):

With core.autocrlf=true, any files in the repo that git detects as text and which already have LF EOLs will have their EOLs converted to CRLF on check-out. Also, any additions to these files, or any new files that git detects as text, will have their EOLs converted to LF on check-in.

In this way, core.autocrlf=true is similar to "* text=auto", however it does not affect any files in the repo which already have CRLF EOLs.

AFAIK, there's no reason to set core.autocrlf=true on Unix. You'd typically only set it under Windows.

I'm going to stop here as I think these are the only options that make sense with 1.7.2 and above. If you want me to explain your options using earlier versions of git, I can try, but it's even more confusing.

j.
Will Palmer· Aug 4, 2010, 05:29 UTC · re: Jay Soffian · lore

Re: noob user, want checkins to all be forced to LF terminated lines

On Tue, 2010-08-03 at 19:56 -0400, Jay Soffian wrote:
Show 68 quoted lines
> On Sat, Jul 31, 2010 at 2:32 AM, Walter Bright <boost@digitalmars.com> wrote:
> > So I'm lost again. If the version in the repository is always converted to
> > LF, then why do I need to set autocrlf=input ?
> 
> Let's start over. :-)
> 
> Before git-1.7.2, EOL conversion was rather insane. It's fixed in
> 1.7.2, so I'm going to start by explaining what happens with that
> version and later.
> 
> Option 1 (text/eol attributes):
> 
> - Normally git will perform no EOL conversion. Files are committed
> into the repo exactly as you see them in your checkout.
> 
> - To have git perform EOL conversion, you need to either explicitly
> tell it which files are text, or let it autodetect. You normally do
> this via the .gitattributes file using:
> 
>   <pattern> text
> 
> or
> 
>   * text=auto
> 
> In the former case, you're telling git explicitly which files are
> text. In the latter case, you're telling git to do its best to detect
> which files are text.
> 
> Files which are explicitly tagged as text, or auto-detected as text,
> will have their EOLs converted to LF on check-in, and converted to
> core.eol on check-out. core.eol defaults to "native" which means LF on
> Unix and CRLF on Cygwin, but you can explicitly set it to "lf" or
> "crlf" if you desire.
> 
> Certain files you may wish to specify their EOL in the working copy
> explicitly. You do this with the eol attribute. e.g.:
> 
>   <pattern> eol=crlf
>   <pattern> eol=lf
> 
> Files which are tagged with eol={crlf,lf} are implicitly text, and
> will have their EOLs converted to LF on check-in, and converted to the
> specified EOL on check-out.
> 
> Okay, so that's how you ensure that certain files have LFs in the
> repo, and the desired EOL in the working-copy.
> 
> Option 2 (core.autocrlf):
> 
> With core.autocrlf=true, any files in the repo that git detects as
> text and which already have LF EOLs will have their EOLs converted to
> CRLF on check-out. Also, any additions to these files, or any new
> files that git detects as text, will have their EOLs converted to LF
> on check-in.
> 
> In this way, core.autocrlf=true is similar to "* text=auto", however
> it does not affect any files in the repo which already have CRLF EOLs.
> 
> AFAIK, there's no reason to set core.autocrlf=true on Unix. You'd
> typically only set it under Windows.
> 
> I'm going to stop here as I think these are the only options that make
> sense with 1.7.2 and above. If you want me to explain your options
> using earlier versions of git, I can try, but it's even more
> confusing.
> 
> j.

If it is accurate (and I have no way of knowing, since I've never understood the ins-and-outs before), then that was the best, most straightforward, most complete, least confusing description of core.autocrlf, eol={crlf,lf}, and text=auto that I have ever read. I want this post re-formatted and included in the docs :)

For completeness, would you mind explaining the "old way" too, and include a note on oddities such as "git says my entire working copy has changed, but reset --hard does nothing"? I just feel like I'm closer than I've ever been to understanding the whole mess.

> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 
-- Will
Miles Bader· Jul 31, 2010, 05:41 UTC · re: Walter Bright · lore

Re: noob user, want checkins to all be forced to LF terminated lines

Walter Bright <boost@digitalmars.com> writes:
> I've just started with git. Exactly what do I put in $HOME/.gitconfig ?
+ your personal info (user.*)
+ various global personal preferences for tool operation
  (diff.renames, branch.autosetupmerge, apply.whitespace)
+ core.excludesfile = ~/.gitignore
  ... then in ~/.gitignore, put filenames to ignore that are related to
  your personal habits or tool preferences rather than any particular
  project (e.g., "*~" if you use emacs; I also put ",*" because I use
  that pattern for scratch files)
+ command aliases (alias.*)
-miles
-- 
A zen-buddhist walked into a pizza shop and
said, "Make me one with everything."

← back to recent threads