files are disappearing in git

Hello!

I've the problem that some files (a directory with 3 files) is simply 'away':

- We added it once
- In current tree it's away
- pasky aided me in irc to find the commit where it is gone with git bisect
--> very nice tool
- the commit, after which the directory was gone did NOT modify this directory
- though the directory is gone

What should I do know to find out what's the reason git 'forgot' that directory?

Nico

P.S.: I cannot put the .git directory online, it's a closed source project
      for a special customer with customer data in it.

On Wed, 23 Nov 2005, Nico -telmich- Schottelius wrote:

I bet somebody just messed up the index before that commit. So the commit probably _did_ modify the directory, though some incorrect patching or some scripting bug.

If you can guess at all what went on at around that commit (what else happened there?) that might help. Was it a merge? If those three files existed in the base commit of the merge but not in one of the branches, then the merge would have assumed that the branch just deleted it.

Is the tree public so that we can look at it and perhaps make a guess from what happened around it?

		Linus

Linus Torvalds [Wed, Nov 23, 2005 at 09:20:28AM -0800]:

What would be the best things to corrupt the index? Our developers here do the following each day:

- cg-update
- <work>
- cg-commit
- cg-update + merge if there are changes
- cg-push origin
--> this goes to NFS mounted /home (in which developers also work)
[9:34] klapperwachstum:walderlift% cat .git/branches/origin 
/home/server/git/walderlift.git

Hrm, perhaps this is true, though I am not aware _how_ they (two people besides me) could even manage to do that. Our developers are Kylix developers under Linux, who are raelly limited to the above cg-commands and using gitweb for seeing changes.

In this commit where specified three files changed. The commit text is not related to the directory which is missing.

Files changed are

Code/Components/Utilities/LW1Calendar.pas Code/lw1/Client/Terminierung/FeinTerminierungForm.pas Code/lw1/Client/Terminierung/FeinTerminierungForm.xfm

and files missing after this release are

Code/Spikes/Statistik/

If I select "history" of one of the files at the point it still existed in the tree, it has only one entry, where it was added.

No, we also assumed that first, but git bisect showed that it is not.

I am sorry it is not. Perhaps I can convince my boss to allow access to it for some git developers, so someone could debug it. Thought, every information found in the tree would have to be treated confidental.

Do you have some other hints on how to debug that? Perhaps some conistency checking tool?

Or perhaps should I put that git directory under cvs? ;-)

Nico

Nico -telmich- Schottelius <nico-linux-git@schottelius.org> writes:

Was any of the CG:F line changed/removed by the developer during cg-commit?

Does any of the files under Code/Spikes/Statistik/ have funny characters in their pathnames?

Junio C Hamano [Thu, Nov 24, 2005 at 01:15:13AM -0800]:

He said it could be possible. In fact the situation that we delete those lines are not so seldem.

No:

Code/Spikes/Statistik/Project.dpr [new file with mode: 0644] blob Code/Spikes/Statistik/Statistik.pas [new file with mode: 0644] blob Code/Spikes/Statistik/Statistik.xfm [new file with mode: 0644] blob

Nico

Nico -telmich- Schottelius wrote:

 >>Is the tree public so that we can look at it and perhaps make a guess
from

Is there anything in the directory structure that would be confidential?

Can you maybe provide the output of "git-whatchanged" (with no parameters) or maybe

	git-whatchanged $commit1..$commit2 path/

Where $commit1 is a few commits before the problem, and $commit2 is a few after it, and path/ is a path above the problem path?

You might add -m to that command line, too.

If the output of one of those commands, which will be commit objects + tree differences, is less problematic, perhaps that would be easier to share?

Ryan Anderson [Thu, Nov 24, 2005 at 08:54:34PM -0500]:

The problem is that this is a project developed for a single company in switzerland which uses it to manage elevators. We are right before to release it and switch the customer away from MS Access to our program. It is currently not clear, which license applies after the customer uses it, because he is interested that some people should not use the software.

As soon as this is clearified the software may even become GPLed, but currently I may not publish it public.

If I do it as you described, I get the usage of git-rev-list:

[11:01] srsyg01:walderlift% git-whatchanged 4cc7ff0cb20df228e4db467114c1c67b0933592a..cfd54ffd2aa8e44ff5c341030e9c5f22fbbc7f9f Code/Spikes/ usage: git-rev-list [OPTION] <commit-id>... [ -- paths... ]

I junh did git-whatchanged and stripped it manually. I included the initial commit until the commit when it is gone. See attached file.

I think this could be done.

Nico

Ok, 

  Nico gave me private access to the tree, so I quickly cloned it and 
started bisecting it to figure out where the problem was. I haven't looked 
at the source code, and all the commit messages seem to be in German 
(which I can kind of understand if I work at it, but not very well), but 
it definitely turns out none of that matters.

The problem is a bad merge. And in fact, that merge lost _more_ than just the three files under Code/Spikes/Statistik/, it also lost a file called Code/lw1/Client/Pics/icon/lw1-icon.png.

I don't quite see _how_ it lost them. The merge in question is a totally trivial in-index merge, and when I re-do it, I don't lose those files. In this case, all the lost files were from the "other branch" of the merge, and they were new to that branch. IOW, in git-merge-one-file parlance, it is that trivial "added in one" case.

Pasky - do you know of any historical cogito problems like this?

Nico: the files were added in commit

 - 9e9b91166bfca448a8f1ddeb4580d73ac8ea0986:
	Neuer Spike: Statistik ...

and they stayed around in that branch until commit (which is still good):

 - dcd70e89dc7b2280260628fd699aa906c319c68f:
	Bug 359 - Defektmeldungen ...

but they disappeared when the that branch was merged with commit

 - 056c65efea28066b7b241240f0d8421d3204b624
	Bug 195 - Tagesarbeitsplan: ...

and the result is the bad merge commit:

 - 228b94dd0a7aa1516eb867674cdf8c7c7b2bfd4c:
	Merge with /home/server/git/walderlift.git

After that merge, those new files are gone - they were never added by the merge.

Quite frankly, this is clearly a bug, and it looks very much like a really serious bug in cg-merge. It's not in the core git: even old versions of git should never have been able to generate that broken merge.

But when I try to do that same merge with current cogito, I can't make it break _either_. In all cases, regardless of how I do the merge, when I do (bad-merge is the merge in your tree, "cg-test" in this case is the merge I did using cogito):

	git-diff-tree -r bad-merge cg-test

I get

	:000000 100644 0000000000000000000000000000000000000000 5c8c47d855ee0f78a7fb81c4a49224b1607a88b7 A      Code/Spikes/Statistik/Project.dpr
	:000000 100644 0000000000000000000000000000000000000000 7bd2de9b7142999bc6bbadde06e73267d1b5ee0e A      Code/Spikes/Statistik/Statistik.pas
	:000000 100644 0000000000000000000000000000000000000000 a0fb45e2b906bdac7929cfd8174053015ab629b4 A      Code/Spikes/Statistik/Statistik.xfm
	:000000 100644 0000000000000000000000000000000000000000 24f8254c06ea5efc89944a3a9f52d2c79c841c0b A      Code/lw1/Client/Pics/icon/lw1-icon.png

ie the "bad merge" thing (commit 228b94dd0a7aa1516eb867674cdf8c7c7b2bfd4c in your tree) will not have those four files, and the good merge (whether I use the git "recursive" or git "resolve" merge, or use "cg-merge" to do it) will always have those four files.

So I do not see how that bad commit happened, especially since it's even a fairly recent commit (the date of the merge is 2005-11-17).

I wonder if there's a really old and broken version of cogito somewhere around. If so, it's on "srsyg03".

I also went through every single merge in your tree, and apart from two merges that had merge conflicts and were fixed up by hand (and that I thus didn't check), all other merges looked ok. So as far as I can tell it's only that 228b94dd0a7aa1516eb867674cdf8c7c7b2bfd4c merge that is bad.

(And the two commits that needed manual merging _look_ fine. No lost files at least, except for one of the merges losing "bitte_bitte_loesch_mich", which judging by it's name _should_ be lost ;)

Finally: Nico, I've deleted the trees on my machine, and you can remove my 
ssh key. I don't think I can tell you anything more.

			Linus

Dear diary, on Fri, Nov 25, 2005 at 08:12:00PM CET, I got a letter where Linus Torvalds <torvalds@osdl.org> said that...

Puzzling. In the past, cg-merge might get confused by merging on top of tree with some uncommitted changes (either deleting them, or possibly intermixing them with the merge - not sure about the latter now), but a quick scan of Cogito changes over the last few months shows no other bugs, and I can't remember any major bugfixes besides this either.

Any idea what Cogito version (roughly) could have been used to perform the merge, Nico?

On Fri, Nov 25, 2005 at 11:12:00AM -0800, Linus Torvalds wrote:

Can something like this sequence do it?

	git-init-db
	git add file1 ; git commit -m "1"
	(cd .. ; git clone tree1 tree2 )
	(cd .. ; git clone tree1 tree3 )
	git add file2 ; git commit -m "2"
	git push ../tree2/
	git add file3 ; git commit -m "3"

	cd ../tree3/
	git add file4
	git commit -m "4"

	cd ../tree2/
	git pull ../tree3/

The key point being that the merge is done in a tree that has its index out of sync with its HEAD (git push ../tree2/ .... git pull ../tree3/ )

I think that's the situation where I've personally managed to lose and/or revert some changes.

Ryan Anderson <ryan@michonline.com> writes:

The moral of the story is not to push into the checked-out branch of a non-naked repository, or you need to do what you are doing if you choose to do so. It might have made sense if index file recorded which tree object its contents came from, but I am not sure what tools is responsible for recording that if we choose to extend index file to store that information. Probably read-tree is, but it should not do that unconditionally.

In any case, the workflow presented by Nick is that individual developers use cg-* commands in CVS like workflow where the only place you push into is the central repository and everybody pulls from there, so I do not think the above example applies.

On Fri, 25 Nov 2005, Ryan Anderson wrote:

Nope, I don't think that should matter. Also, that doesn't seem to match what Nico & co are doing, but that's hard to tell..

A merge result should be totally independent of the index file(s) involved.

A dirty index file can cause a merge to _fail_, in that git may refuse to do the merge at all because of the index not matching the original branch, but a successful automated merge should never have any dependencies on what happens to be in the index at the time the merge was done.

So you can think of a merge as being totally specified by the trees involved, unless we have some bug, of course. I can't think of any.

Now, what _can_ happen (I think) is that if a merge is a failure (and there, a dirty index can certainly be the _cause_ of that failure), then when you fix it up and commit, there's some mix-up at _that_ stage.

For example, let's say that you had a dirty tree or something, and then the merge failed, and you didn't see anything wrong, so you just end up doing a "git commit". At _that_ point, what you had in the index matters very much, of course, since the index is what will be committed.

Hmm.. Can you elaborate?

(Side note: all my commentary is purely about the "raw git" interfaces. I don't know what cogito may do on top of it).

		Linus

Dear diary, on Fri, Nov 25, 2005 at 11:11:22PM CET, I got a letter where Linus Torvalds <torvalds@osdl.org> said that...

Note that Cogito is now (v0.15.1 or later) supposed to handle merges on top of trees with local changes fine - it will either error out, or in case the merge is done on unrelated files it will temporarily ignore your local changes and cg-commit won't mix them up (even if you do some conflict fixups).

Good evening,

Linus Torvalds [Fri, Nov 25, 2005 at 11:12:00AM -0800]:

Well, there's still a broken username, but that's our problem with iso8859-1 in passwd.

Good to know, we didn't that we lost it until now :)

For information:

[22:07] srsyg03:packages% ls -l /usr/packages insgesamt 28 drwxr-xr-x 4 root root 4096 Nov 10 15:52 cogito-/ drwxr-xr-x 4 root root 4096 Okt 14 15:15 cogito-0.15/ drwxr-xr-x 4 root root 4096 Nov 23 13:31 cogito-73874dddeec2d0a8e5cd343eec762d98314def63/ drwxr-xr-x 4 root root 4096 Okt 14 15:15 cvsps-2.1/ drwxr-xr-x 4 root root 4096 Nov 10 15:52 git-/ drwxr-xr-x 4 root root 4096 Okt 17 14:09 git-20051016.git/ drwxr-xr-x 4 root root 4096 Nov 23 13:31 git-c61642185d411e5e3350566a68483e358ca392b9/

At the time of 2005-11-17 we'll have used a cogito and git version, which was from 2005-11-10.

Above you see the versions of which were/are installed. I always link the latest binaries to /usr/local/bin/, so our developers do not need to care about what version we have.

Those were some lessons in our house. Our developers had to test if everything works, including deleting :)

Ok. Thanks for your help! I think we'll add those four files back to git on monday and continue to work with them.

Nico

Dear diary, on Fri, Nov 25, 2005 at 10:28:46PM CET, I got a letter where Nico -telmich- Schottelius <nico-linux-git@schottelius.org> said that...

That's really weird - I can't see anything since then that could influence it. The only possibility is that you were working on those files before, left them modified but uncommitted, then did a merge which would touch them, the merge would fail because of local changes, then you would delete your local instances of the files and try the merge again.

That's something I fixed only very recently and the fix is not even in any public release yet; I didn't realize that it could have such dangerous consequences, but apparently it does. I guess I will release 0.16 final tomorrow.