git fsck messages - what to do about it?

When running git fsck I get some messages:

-> git-fsck-cache --unreachable $(cat .git/HEAD .git/refs/heads/*) unreachable commit 08cf4121ad704faa5ae7c7ae7b3dca4476ef2770 unreachable tree 29a4c18afd958cc6cee938f0fe7fa59764418097 unreachable tree 3b722a7278e02b272ddc8aefbc7be22703d8a94f unreachable tree 4ebc50d240d99b4236952d13d5044691f9078e3c unreachable tree 63af5944e130b449338cc907ee77c51f4a69e339 unreachable tree 79c9a6bb02a80df15c7d829bdac0901f88c5e301 unreachable blob 85dcf4b5c4c0f6833cd63fa1d0fd29ef250162af unreachable commit c1b779fbce762c970e3ace4332bb6ddc0de65e61 unreachable blob d70d1d44f12db7b2a3976c2a5313c90568a62c25 unreachable commit e1bcd2a5d53bceef2679737474487e32d87e7f2c

I have to admit that I don't know what to do next...

a) I understand that these  messages  are  indications  of  problems,
   right? What can I do to clean this up?

b) Is there any way to find out what I did wrong in the first place?

In case it matters: the tree in question is available at {rsync,http}://source.denx.net/git/u-boot.git

Thanks in advance,

Best regards,

Wolfgang Denk

Wolfgang Denk <wd@denx.de> wrote:

Did you use StGIT on your tree? StGIT can leave unreachable objects. Otherwise, git-prune should remove them.

Note that if you use StGIT, git-prune will also remove the commit objects for the patches which are not pushed onto the stack. I will add a 'stg prune' commands which takes care of these.

In message <tnx3bo59tm8.fsf@arm.com> you wrote:

No, I did not. I *think* these unreachable commits resulted from "git resolve"s which resulted in merge conflicts, which I resolved manually. After this, I ran "git commit" manually, but probably I did something wrong. I just don't understand what this might have been...

Ummm.... I hesitate to do this before understanding where these messages come from and what sort of data is affected.

Best regards,

Wolfgang Denk

Wolfgang Denk <wd@denx.de> writes:

Does git-fsck-cache without --unreachable report dangling commits? If so I would do something like this (this is my 'git-lost-found' script):

        #!/bin/sh

        . git-sh-setup-script || die "Not a git archive."

        laf="$GIT_DIR/refs/lost-found"
        mkdir -p "$laf" || exit

        git fsck-cache |
        while read dangling type sha1
        do
                case "$dangling" in
                dangling)
                        echo "$sha1" >"$laf/$sha1"
                        case "$type" in
                        commit)
                                git-show-branch "$sha1" ;;
                        esac
                        ;;
                esac
        done

and run 'gitk --all' to see what these dangling commits are, and where in your commit ancestry mesh they fit in.

After you understand what they are, do not forget to "rm -f $GIT_DIR/refs/lost-found/*"; you made them reachable just to make inspecting them easier, but you probably do not want them.

I can sort of understand dangling blobs can come from manually resolved merges, but I do not think that would create dangling commits.

Have you rewound and rebased your history? 'git reset --hard' obviously throws the tip of the current branch away and 'git rebase' does it in order to recreate an alternate history. If that is the case, and if you are satisfied with the history your repository currently have, then obviously these can be safely pruned.

Another possibility is what Pasky suggested -- do you have a symlink .git/objects in another repository that points at the .git/objects directory of this repository? These dangling things may be objects the other repository needs. If that is the case you obviously cannot prune them -- you would corrupt the other repository.

Dear diary, on Fri, Sep 16, 2005 at 12:11:38PM CEST, I got a letter where Wolfgang Denk <wd@denx.de> told me that...

In case you were using Cogito, cg-admin-uncommit will leave unreachable objects in the database - it's safe to prune them (unless you'll decide later that you after all wanted the uncommitted stuff, of course).

But note that IT IS NOT SAFE to prune repositories cloned with -l or _being_ cloned with -l. Perhaps one solution to this would be having .git/objects/info/repousers/ or something, which would contain symlinks back to all the repositories using the given objects database. Opinions?

On Fri, 16 Sep 2005, Wolfgang Denk wrote:

Btw, you shouldn't need to list the heads explicitly any more: if you don't give any heads, git-fsck-cache by default will pick up all of them. In fact, it will generally do a better job of it that you did, since it will look at _all_ the refs in .git/refs/.

The above is very old-fashioned ;)

First off, not necessarily. Especially if you have mixed rsync and the "native git pack" transports, you may have packs that refer to unpacked objects, and then since git-fsck-cache by default doesn't look into packs (it assumes that they are fine and self-sufficient), you can get this error.

The same is true if you have "alternates" directories: git-fsck-cache won't do the reachability in the other directories by default.

If you add the "--full" flag, git-fsck-cache will also look at all those things.

Now, I assume that this isn't the case, and what you have is actually real unreachable objects. And that's perfectly normal too: it will happen if you ever "drop" a commit (undo) by resetting a branch to its parent. For example, if I notice I've made a mistake after a commit, I just do

	git reset HEAD^

which effectively undoes the commit. I can then fix things up and re-commit. And this is all _normal_.

And what it results in is that the commit objects that I undid are obviously not reachable from anything else. git-fsck-cache will say that they are unreachable, and "git prune" will get rid of them.

The same thing happens if you do "git-update-cache" on a file, but instead of committing that blob object, you realize that you need to edit it a bit more, and do _another_ git-update-cache and commit only the final blob. The first blob will be unreachable - nothing ends up pointing to it.

So an unreachable object is not necessarily an error at all. Git - very much on purpose - will leave the stale objects around, because it's too expensive to try to be exact, so it's much better to just let them be, and do an occasional "cleanup" with "git prune".

There are other reasons to have "unreachable" objects, even more benign. 

Any time you use "rsync" as the fetch protocol, you'll get _all_ the objects from the other end. Regardless of whether you actually fetch all the _refs_ from the other end. In particular, if the other end has multiple branches, and you only pull _one_ of them, you'll still get the objects for all the others. And git-fsck-cache will correctly say that those objects are unreachable.

(And _yet_ another reason for unreachable objects: you share the object directory with some other project - those objects will be unreachable from _your_ project, but maybe they're reachable from another one?)

Well, it probably isn't even an error, but there are nice tools to check it out. For example, you can just list the commits it lists to gitk, and you'll get a nice graphical view of where they are in the tree. That probably makes you go "Duh, yes, I did a 'git reset' to undo those".

Or maybe you recognize that the commit was clearly part of a chain that you hadn't even committed, but came from another tree that you had rsync'ed with.

And if that is the case, just ignore them. Or do "git prune" to keep your tree nice and clean, and to make "git-fsck-cache --unreachable" happy.

		Linus

On Fri, 16 Sep 2005, Linus Torvalds wrote:

Doing exactly that:

	gitk 08cf4121ad704faa5ae7c7ae7b3dca4476ef2770 \
		8493c04f1b8c9da0c5dc83091cabbcd0b98954ed \
		e1bcd2a5d53bceef2679737474487e32d87e7f2c \
		c1b779fbce762c970e3ace4332bb6ddc0de65e61

says that three of the commits were the same one, redone:

	Added support for mtddevnum and mtddevname variables.

in one form or another (quick look at the diffs says that one was apparently re-done because you forgot the changelog entry, I didn't look any closer).

So I suspect that's you just undoing a commit, and redoing it. I assume that somewhere in the _reachable_ tree is one final commit that actually did that, but I didn't bother to search for it (but that's easy too if you want to:just go "gitk --all" and then search for that "Added support for mtddevnum"...)

The fourth commit is 8493c

	Rebased: Rewrite of NAND code based on what is in 2.6.12 Linux kernel

and considering that you seem to have other undo's in the tree, I'll just assume this is one more, but you would know.

		Linus

On Fri, 16 Sep 2005, Linus Torvalds wrote:

Oh, btw, one final note on this problem.

The "--unreachable" flag is meant to show a list of all objects that aren't reachable from the refs (either the explicit ones, or the ones git-fsck-objects finds on its own). That's very useful for when you actually want to remove them, ie "git prune" uses that flag.

However, it's often not very interesting to see the full list of unreachable objects, because you are probably _not_ interested in the trees that are reachable from the unreachable commit - you're really interested in the "tips of unreachability".

And that's what git-fsck-object actually reports _without_ any flags, as "dangling" objects.

So let's say that you have one commit that you've undone.

If you don't give any flags at all, git-fsck-objects will report

	dangling commit ...

but if you use "--unreachable" then you'll get the full list of objects, so you'll get

	unreachable tree ...
	unreachable blob ...
	unreachable commit ...
	unreachable blob ...

which is usually _not_ what you want.

Similarly, let's say that you undid a series of three commits. There is only one "dangling" commit (the "tip" of the series), but there are three unreachable commits (and any number of unreachable trees/blobs associated with those commits). Again, what a human tends to be interested in is the "dangling tip", not the whole collection of unreachable objects that can be reached through that tip.

So using "--unreachable" gives much more raw data, but it usually tends to hide the real _information_, and the end result is thus much harder to understand.

			Linus

Linus Torvalds <torvalds@osdl.org> writes:

Yup. That's exactly why the 'git-lost-found' sample script given in <7vll1xndxi.fsf@assigned-by-dhcp.cox.net> runs git-fsck-objects output without --unreachable.