threads / discuss / 41075

git repository modified after migration

Subject: git repository modified after migration

## tl;dr

6 messages between Dec 25, 2015 and Jan 12, 2016.

replies: 5people: 3as markdown or json

Yang Yu· Dec 25, 2015, 20:49 UTC · lore

I migrated a 11G git repository converted from svn on a host with Debian 8.2, reiserfs, git 2.1.4 to a host with Ubuntu 12.04.5 LTS, xfs, git 2.6.4. After the migration, `git status` showing a good amount of files modified.

I did the transfer with
1) `rsync -azP`, after noticing the modified files I ran `rsync -avH
--delete` but it did not correct the problem
2) tar zcf, then on the destination download the tar.gz (served by
nginx) with wget
Both had the same result. But the original repository was still clean.

I did some comparison between "modified" and original files a) same hash (md5sum, shasum) b) same permission (-rw-r--r-- 1 ) c) same encoding and line termination (UTF-8 Unicode (with BOM) text, with CRLF line terminators) d) no git attributes for either

On the destination host, I ran `git checkout` on each of those modified files. After one pass I got less modified files. Repeating `git checkout` on remaining files for a few more times, finally I got a clean repository on the destination host.

What could have caused git to consider those files as modified? And why multiple `git checkout` again the same file was necessary?

Thanks.
Yang
Michael J Gruber· Jan 11, 2016, 15:04 UTC · re: Yang Yu · lore

Re: git repository modified after migration

Yang Yu venit, vidit, dixit 25.12.2015 21:49:
Show 33 quoted lines
> I migrated a 11G git repository converted from svn on a host with
> Debian 8.2, reiserfs, git 2.1.4 to a host with Ubuntu 12.04.5 LTS,
> xfs, git 2.6.4. After the migration, `git status` showing a good
> amount of files modified.
> 
> I did the transfer with
> 1) `rsync -azP`, after noticing the modified files I ran `rsync -avH
> --delete` but it did not correct the problem
> 2) tar zcf, then on the destination download the tar.gz (served by
> nginx) with wget
> 
> Both had the same result. But the original repository was still clean.
> 
> I did some comparison between "modified" and original files
> a) same hash (md5sum, shasum)
> b) same permission (-rw-r--r-- 1 )
> c) same encoding and line termination (UTF-8 Unicode (with BOM) text,
> with CRLF line terminators)
> d) no git attributes for either
> 
> 
> On the destination host, I ran `git checkout` on each of those
> modified files. After one pass I got less modified files. Repeating
> `git checkout` on remaining files for a few more times, finally I got
> a clean repository on the destination host.
> 
> What could have caused git to consider those files as modified? And
> why multiple `git checkout` again the same file was necessary?
> 
> Thanks.
> 
> 
> Yang

This happens whenever the "stat" information changes, e.g. due to changed device numbering and such. "git reset --hard" would have been the quickiest way to reset the stat cache/index - after git diff, of course ;)

Michael
Yang Yu· Jan 11, 2016, 17:48 UTC · re: Michael J Gruber · lore

Re: git repository modified after migration

On Mon, Jan 11, 2016 at 9:04 AM, Michael J Gruber <git@drmicha.warpmail.net> wrote:

Show 5 quoted lines
> This happens whenever the "stat" information changes, e.g. due to
> changed device numbering and such. "git reset --hard" would have been
> the quickiest way to reset the stat cache/index - after git diff, of
> course ;)
>

Can you go into a bit more details about this? Why does it only affect certain files (always these filed modified) on every host I tried? Thanks.

Junio C Hamano· Jan 11, 2016, 18:19 UTC · re: Michael J Gruber · lore

Re: git repository modified after migration

Michael J Gruber <git@drmicha.warpmail.net> writes:
> This happens whenever the "stat" information changes, e.g. due to
> changed device numbering and such. "git reset --hard" would have been
> the quickiest way to reset the stat cache/index - after git diff, of
> course ;)

That does not quite explain why 'git status' reported modified files in the first place. It would have refreshed the cached stat info in the index as the first thing to do. "git status" I think is the recommended way these days ("update-index --refresh" for us old timers) to nondestructively correct the cached stat information discrepancy caused by "cp -R".

If you need to resort to "reset --hard", then there is something else going on.

Michael J Gruber· Jan 12, 2016, 07:01 UTC · re: Junio C Hamano · lore

Re: git repository modified after migration

Junio C Hamano venit, vidit, dixit 11.01.2016 19:19:
Show 16 quoted lines
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
>> This happens whenever the "stat" information changes, e.g. due to
>> changed device numbering and such. "git reset --hard" would have been
>> the quickiest way to reset the stat cache/index - after git diff, of
>> course ;)
> 
> That does not quite explain why 'git status' reported modified files
> in the first place.  It would have refreshed the cached stat info in
> the index as the first thing to do.  "git status" I think is the
> recommended way these days ("update-index --refresh" for us old
> timers) to nondestructively correct the cached stat information
> discrepancy caused by "cp -R".
> 
> If you need to resort to "reset --hard", then there is something
> else going on.

Back than when I had the same problem with git repos on removable file systems (if I remember correctly) git status did not correct that information. It may be different now.

Michael
Junio C Hamano· Jan 12, 2016, 18:07 UTC · re: Michael J Gruber · lore

Re: git repository modified after migration

Michael J Gruber <git@drmicha.warpmail.net> writes:
Show 21 quoted lines
> Junio C Hamano venit, vidit, dixit 11.01.2016 19:19:
>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>> 
>>> This happens whenever the "stat" information changes, e.g. due to
>>> changed device numbering and such. "git reset --hard" would have been
>>> the quickiest way to reset the stat cache/index - after git diff, of
>>> course ;)
>> 
>> That does not quite explain why 'git status' reported modified files
>> in the first place.  It would have refreshed the cached stat info in
>> the index as the first thing to do.  "git status" I think is the
>> recommended way these days ("update-index --refresh" for us old
>> timers) to nondestructively correct the cached stat information
>> discrepancy caused by "cp -R".
>> 
>> If you need to resort to "reset --hard", then there is something
>> else going on.
>
> Back than when I had the same problem with git repos on removable file
> systems (if I remember correctly) git status did not correct that
> information. It may be different now.

I do not recall we did anything specific to help removable devices, so if the report is coming from a filesystem on a removable device we may be seeing the same symptom.

I somehow doubt that is the case, though.

A not-entirely-inplausible theory is that the index in the original repository somehow marks modified entries as clean, fooling "status" that is run in the original repository into reporting that nothing changed. Because "cp -R" into a different location forces the content-level checking, "status" run in the copy notices that they are indeed different and tells the true story. If that is the case, it would be interesting to see how the index got into that state.

← back to recent threads