threads / bug / 63500

[BUG] Hard links to large files cause unexpected refresh

Subject: [BUG] Hard links to large files cause unexpected refresh

## tl;dr

2 messages between May 22, 2025 and May 22, 2025.

replies: 1people: 2as markdown or json

Roel Sengers· May 22, 2025, 14:45 UTC · lore

Thank you for filling out a Git bug report! Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

In my workflow there is a step which hard-links about 80GB of such files to locations where a 3rd party application can find them. However, after this hard-linking step, git status (or other git commands) hang for a long time. Note that making a copy of the file does not cause such slowdowns, which was surprising to me.

I am using Git with git-lfs for these large files, however I was able to reproduce this issue without LFS enabled.

The script below reproduces the environment which triggers the issue:

mkdir git-hardlink-test cd git-hardlink-test

git init echo ignore.bin > .gitignore git add .gitignore git commit -m 'Initial commit'

dd if=/dev/urandom of=file.bin bs=1M count=1000 git add file.bin git commit -m 'Add file.bin'

# $ cp file.bin ignore.bin; time git status # On branch main # nothing to commit, working tree clean # # real 0m0,002s # user 0m0,000s # sys 0m0,002s

# $ ln file.bin ignore.bin; time git status # Refresh index: 100% (2/2), done. # On branch main # nothing to commit, working tree clean # # real 0m16,100s # user 0m15,700s # sys 0m0,255s

What did you expect to happen? (Expected behavior)

After creating a hard link to a checked-in object, I expected the final git status to finish in a time that is barely noticeable.

The file itself is large, so having Git taking its time to refresh the state of the working directory would not be suspicious were it not for the fact that creating a copy of the same file does not suffer from the same performance penalty.

I expected Git to be as fast for both copied and hard-linked files.
What happened instead? (Actual behavior)

git status takes an unusual amount of time to complete when an uncommitted hard link is created to a prior committed file.

What's different between what you expected and what actually happened?

Timing git status tells me that it took 2 milliseconds to complete after a copy was made.

Running git status after creating a hard link took 16 seconds instead, I expected these times to be similar as the file contents do not differ.

The target file is also listed in .gitignore, so I would not expect Git to read it at all.

Anything else you want to add:

Making copies of these files instead is not desirable as this would double the storage capacity needed and copying itself would also take a long time.

Please review the rest of the bug report below. You can delete any lines you don't wish to share.

[System Info]
git version:
git version 2.49.0
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /nix/store/xy4jjgw87sbgwylm5kn047d9gkbhsr9x-bash-5.2p37/bin/bash
libcurl: 8.13.0
OpenSSL: OpenSSL 3.4.1 11 Feb 2025
zlib-ng: 2.2.4
uname: Linux 6.12.28 #1-NixOS SMP PREEMPT_DYNAMIC Fri May  9 07:50:53 
UTC 2025 x86_64
compiler info: gnuc: 14.2
libc info: glibc: 2.40
$SHELL (typically, interactive shell): /run/current-system/sw/bin/bash
[Enabled Hooks]

With kind regards, Roel Sengers

Justin Tobler· May 22, 2025, 16:40 UTC · re: Roel Sengers · lore

Re: [BUG] Hard links to large files cause unexpected refresh

On 25/05/22 04:45PM, Roel Sengers wrote:
Show 55 quoted lines
> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
> 
> What did you do before the bug happened? (Steps to reproduce your issue)
> 
> In my workflow there is a step which hard-links about 80GB of such files to
> locations where a 3rd party application can find them. However, after this
> hard-linking step, git status (or other git commands) hang for a long time.
> Note that making a copy of the file does not cause such slowdowns, which was
> surprising to me.
> 
> I am using Git with git-lfs for these large files, however I was able to
> reproduce this issue without LFS enabled.
> 
> The script below reproduces the environment which triggers the issue:
> 
> mkdir git-hardlink-test
> cd git-hardlink-test
> 
> git init
> echo ignore.bin > .gitignore
> git add .gitignore
> git commit -m 'Initial commit'
> 
> dd if=/dev/urandom of=file.bin bs=1M count=1000
> git add file.bin
> git commit -m 'Add file.bin'
> 
> # $ cp file.bin ignore.bin; time git status
> # On branch main
> # nothing to commit, working tree clean
> #
> # real	0m0,002s
> # user	0m0,000s
> # sys	0m0,002s
> 
> # $ ln file.bin ignore.bin; time git status
> # Refresh index: 100% (2/2), done.
> # On branch main
> # nothing to commit, working tree clean
> #
> # real	0m16,100s
> # user	0m15,700s
> # sys	0m0,255s
> 
> 
> What did you expect to happen? (Expected behavior)
> 
> After creating a hard link to a checked-in object, I expected the final git
> status to finish in a time that is barely noticeable.
> 
> The file itself is large, so having Git taking its time to refresh the state
> of the working directory would not be suspicious were it not for the fact
> that creating a copy of the same file does not suffer from the same
> performance penalty.

When git-status(1) is run, the index state is checked to see if any changes have occurred. From the provided example, while the contents of "file.bin" remain unchanged, the act of creating the hardlink does update metadata such as the number of links and the file ctime. You can see this when you stat(1) "file.bin" before and after creating "ignore.bin".

Git sees these changes and then refreshes the index entry. When the file is copied, no such modification happens to "file.bin" and thus that entry does not need to be refreshed.

To workaround this problem, you could set `core.trustCTime=false` which tells Git to ignore ctime differences between the index and working tree.

-Justin

← back to recent threads