# bug: git pull may delete untracked files in submodule without notice 3 messages from 2019-05-03 to 2019-05-03. Participants: Christian Spanier, Duy Nguyen, Phillip Wood. Thread: https://gitlist.dev/t/51026 ## Christian Spanier, 2019-05-03 08:02 Subject: bug: git pull may delete untracked files in submodule without notice Message-ID: <2e321dbe-42a3-7516-52aa-4bc50a3c403e@boxie.eu> URL: https://gitlist.dev/e/2e321dbe-42a3-7516-52aa-4bc50a3c403e%40boxie.eu ``` Hi, I found a bug where Git may delete untracked files without notice in certain situations. This bug effects Git 2.21.0 both on Linux and Windows. In summary this happens when git pull merges a commit that replaces a submodule folder with a symlink. Any files within the folder are deleted without notice. Check out the script below for details. This happend on some developer's machine and deleted a repository containing about 200GiB of files and tons of uncommited local scripts, log files and whatever, just because some other dev accidentally commited a temporary change. Greetings, Christian Spanier ##### PREPARATION ##### # New empty repository #1 mkdir rep1 cd rep1 git init --bare . cd .. # New empty repository #2 mkdir rep2 cd rep2 git init --bare . cd .. # Clone repository #1 and create initial commit git clone rep1 clone_rep1_user1 cd clone_rep1_user1 touch README git add README git commit -m "initial commit" git push cd .. # Clone repository #2 and create initial commit git clone rep2 clone_rep2 cd clone_rep2 touch README git add README git commit -m "initial commit" git push cd .. # Add repository #2 as a submodule to repository #1 cd clone_rep1_user1 git submodule add ../rep2 git commit -m "add submodule" git push cd .. # User 2 also clones repository #1 and #2 recursively git clone --recursive rep1 clone_rep1_user2 # User 2 starts working in his folder and adds an important local file which is # not yet committed inside the submodule folder. cd clone_rep1_user2/rep2 echo "important work" > uncommitted_file cd ../../ # Meanwhile, user 1 temporarily switch out folder /clone_rep1_user1/rep2 with a # symbolic link to a different folder (for whatever reason, maybe a copy of an # older version or anything). mkdir rep2_alternative cd clone_rep1_user1 mv rep2 ../rep2_backup ln -s ../rep2_alternative rep2 # On Windows this can be done with 'mklink /D rep2 ../rep2_alternative', # which requires admin privileges. The bug is not reproducible when # using a directory junction with 'mklink /D /J ...'. # He does some work on rep1 but then accidently adds the symbolic link to his # next commit and pushes the changes. Notice the typechange of rep2. echo "some" > work git status # On branch master # Your branch is up to date with 'origin/master'. # # Changes not staged for commit: # (use "git add ..." to update what will be committed) # (use "git checkout -- ..." to discard changes in working directory) # # typechange: rep2 # # Untracked files: # (use "git add ..." to include in what will be committed) # # work # # no changes added to commit (use "git add" and/or "git commit -a") git add . git commit -m "do some work" git push cd .. # NOW THE BUG: # User 2 pulls the changes and loses his important work in # rep2/uncommitted_file because Git replaces the folder with a symlink # without checking for modified or uncommited files! # He should get an error in this case! cd clone_rep1_user2 git pull cat rep2/uncommitted_file # cat: rep2/uncommitted_file: Not a directory # "important work" in rep2/uncommitted_file is gone :( ``` ## Duy Nguyen, 2019-05-03 10:28 Subject: Re: bug: git pull may delete untracked files in submodule without notice Message-ID: URL: https://gitlist.dev/e/CACsJy8ArJ9Lc1pA7SbJ1OE6hK%2BrprXpZViLDPUKPMcWG7Bre3A%40mail.gmail.com In-Reply-To: <2e321dbe-42a3-7516-52aa-4bc50a3c403e@boxie.eu> ``` On Fri, May 3, 2019 at 5:25 PM Christian Spanier wrote: > > Hi, > > I found a bug where Git may delete untracked files without notice in > certain situations. This bug effects Git 2.21.0 both on Linux and Windows. > In summary this happens when git pull merges a commit that replaces a > submodule folder with a symlink. Any files within the folder are deleted > without notice. Maybe it's not the same, maybe it is. But Phillip recently did some work protecting untracked files and I think he touched a test case about submodule. Adding him so he can check, if he has time. > Check out the script below for details. > > This happend on some developer's machine and deleted a repository > containing about 200GiB of files and tons of uncommited local scripts, > log files and whatever, just because some other dev accidentally > commited a temporary change. > > Greetings, > Christian Spanier > > ##### PREPARATION ##### > > # New empty repository #1 > mkdir rep1 > cd rep1 > git init --bare . > cd .. > > # New empty repository #2 > mkdir rep2 > cd rep2 > git init --bare . > cd .. > > # Clone repository #1 and create initial commit > git clone rep1 clone_rep1_user1 > cd clone_rep1_user1 > touch README > git add README > git commit -m "initial commit" > git push > cd .. > > # Clone repository #2 and create initial commit > git clone rep2 clone_rep2 > cd clone_rep2 > touch README > git add README > git commit -m "initial commit" > git push > cd .. > > # Add repository #2 as a submodule to repository #1 > cd clone_rep1_user1 > git submodule add ../rep2 > git commit -m "add submodule" > git push > cd .. > > # User 2 also clones repository #1 and #2 recursively > git clone --recursive rep1 clone_rep1_user2 > > # User 2 starts working in his folder and adds an important local file > which is > # not yet committed inside the submodule folder. > cd clone_rep1_user2/rep2 > echo "important work" > uncommitted_file > cd ../../ > > # Meanwhile, user 1 temporarily switch out folder /clone_rep1_user1/rep2 > with a > # symbolic link to a different folder (for whatever reason, maybe a copy > of an > # older version or anything). > mkdir rep2_alternative > cd clone_rep1_user1 > mv rep2 ../rep2_backup > ln -s ../rep2_alternative rep2 > # On Windows this can be done with 'mklink /D rep2 ../rep2_alternative', > # which requires admin privileges. The bug is not reproducible when > # using a directory junction with 'mklink /D /J ...'. > > # He does some work on rep1 but then accidently adds the symbolic link > to his > # next commit and pushes the changes. Notice the typechange of rep2. > echo "some" > work > git status > # On branch master > # Your branch is up to date with 'origin/master'. > # > # Changes not staged for commit: > # (use "git add ..." to update what will be committed) > # (use "git checkout -- ..." to discard changes in working > directory) > # > # typechange: rep2 > # > # Untracked files: > # (use "git add ..." to include in what will be committed) > # > # work > # > # no changes added to commit (use "git add" and/or "git commit -a") > git add . > git commit -m "do some work" > git push > cd .. > > # NOW THE BUG: > > # User 2 pulls the changes and loses his important work in > # rep2/uncommitted_file because Git replaces the folder with a symlink > # without checking for modified or uncommited files! > # He should get an error in this case! > cd clone_rep1_user2 > git pull > cat rep2/uncommitted_file > # cat: rep2/uncommitted_file: Not a directory > # "important work" in rep2/uncommitted_file is gone :( -- Duy ``` ## Phillip Wood, 2019-05-03 15:10 Subject: Re: bug: git pull may delete untracked files in submodule without notice Message-ID: <6d985c61-d06f-b794-8533-c7f29444b148@gmail.com> URL: https://gitlist.dev/e/6d985c61-d06f-b794-8533-c7f29444b148%40gmail.com In-Reply-To: ``` On 03/05/2019 11:28, Duy Nguyen wrote: > On Fri, May 3, 2019 at 5:25 PM Christian Spanier wrote: >> >> Hi, >> >> I found a bug where Git may delete untracked files without notice in >> certain situations. This bug effects Git 2.21.0 both on Linux and Windows. >> In summary this happens when git pull merges a commit that replaces a >> submodule folder with a symlink. Any files within the folder are deleted >> without notice. > > Maybe it's not the same, maybe it is. But Phillip recently did some > work protecting untracked files and I think he touched a test case > about submodule. Adding him so he can check, if he has time. That was a different case. I've just tried adding an untracked file to the submodule directory in the test that replaces a submodule with a file (patch below hopefully thunderbird does not break it) and it still passes when running t1013-read-tree-submodule.sh (it's test 8) - so I think there is something wrong with the detection of untracked files in a submodule directory when we replace it with a file. Normally I'd ping Stefan about submodules but he's not around at the moment. Best Wishes Phillip diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh index 5b56b23166..618176ae54 100755 --- a/t/lib-submodule-update.sh +++ b/t/lib-submodule-update.sh @@ -735,6 +735,7 @@ test_submodule_recursing_with_args_common() { reset_work_tree_to_interested add_sub1 && ( cd submodule_update && + echo untracked>sub1/untracked && git branch -t replace_sub1_with_file origin/replace_sub1_with_file && $command replace_sub1_with_file && test_superproject_content origin/replace_sub1_with_file && > >> Check out the script below for details. >> >> This happend on some developer's machine and deleted a repository >> containing about 200GiB of files and tons of uncommited local scripts, >> log files and whatever, just because some other dev accidentally >> commited a temporary change. >> >> Greetings, >> Christian Spanier >> >> ##### PREPARATION ##### >> >> # New empty repository #1 >> mkdir rep1 >> cd rep1 >> git init --bare . >> cd .. >> >> # New empty repository #2 >> mkdir rep2 >> cd rep2 >> git init --bare . >> cd .. >> >> # Clone repository #1 and create initial commit >> git clone rep1 clone_rep1_user1 >> cd clone_rep1_user1 >> touch README >> git add README >> git commit -m "initial commit" >> git push >> cd .. >> >> # Clone repository #2 and create initial commit >> git clone rep2 clone_rep2 >> cd clone_rep2 >> touch README >> git add README >> git commit -m "initial commit" >> git push >> cd .. >> >> # Add repository #2 as a submodule to repository #1 >> cd clone_rep1_user1 >> git submodule add ../rep2 >> git commit -m "add submodule" >> git push >> cd .. >> >> # User 2 also clones repository #1 and #2 recursively >> git clone --recursive rep1 clone_rep1_user2 >> >> # User 2 starts working in his folder and adds an important local file >> which is >> # not yet committed inside the submodule folder. >> cd clone_rep1_user2/rep2 >> echo "important work" > uncommitted_file >> cd ../../ >> >> # Meanwhile, user 1 temporarily switch out folder /clone_rep1_user1/rep2 >> with a >> # symbolic link to a different folder (for whatever reason, maybe a copy >> of an >> # older version or anything). >> mkdir rep2_alternative >> cd clone_rep1_user1 >> mv rep2 ../rep2_backup >> ln -s ../rep2_alternative rep2 >> # On Windows this can be done with 'mklink /D rep2 ../rep2_alternative', >> # which requires admin privileges. The bug is not reproducible when >> # using a directory junction with 'mklink /D /J ...'. >> >> # He does some work on rep1 but then accidently adds the symbolic link >> to his >> # next commit and pushes the changes. Notice the typechange of rep2. >> echo "some" > work >> git status >> # On branch master >> # Your branch is up to date with 'origin/master'. >> # >> # Changes not staged for commit: >> # (use "git add ..." to update what will be committed) >> # (use "git checkout -- ..." to discard changes in working >> directory) >> # >> # typechange: rep2 >> # >> # Untracked files: >> # (use "git add ..." to include in what will be committed) >> # >> # work >> # >> # no changes added to commit (use "git add" and/or "git commit -a") >> git add . >> git commit -m "do some work" >> git push >> cd .. >> >> # NOW THE BUG: >> >> # User 2 pulls the changes and loses his important work in >> # rep2/uncommitted_file because Git replaces the folder with a symlink >> # without checking for modified or uncommited files! >> # He should get an error in this case! >> cd clone_rep1_user2 >> git pull >> cat rep2/uncommitted_file >> # cat: rep2/uncommitted_file: Not a directory >> # "important work" in rep2/uncommitted_file is gone :( > > > ```