git object-count differs between clones

Hello,

I am using git 2.7.0 on Ubuntu 14.04. I recently tried pushing a large (90,000+ commits) repository to a gogs server (https://gogs.io/) and then cloning the repository back to my machine. After running "git count-objects -v", I see a discrepancy in the number of objects:

Original Repository: count: 0 size: 0 in-pack: 1258300 packs: 1 size-pack: 593889 prune-packable: 0 garbage: 0 size-garbage: 0

Clone from gogs: count: 0 size: 0 in-pack: 1258270 packs: 1 size-pack: 593884 prune-packable: 0 garbage: 0 size-garbage: 0

I ran "git fsck" on both, which reported no problems. Moreover, I ran "git gc" and made sure there were no objects pending garbage collection, but still I cannot account for this difference. Can someone explain why these numbers differ, and if this is a problem or not?

Thanks,

Andrew

Andrew Martin <amartin@xes-inc.com> writes:

It's not sufficient: you may have objects reachable from your reflog, hence not candidate for garbage collection. Since the reflog is not propagated, pushing + cloning will not transfer these objects if the reflog is the only way to reach them.

You may try expiring your reflog and "git gc" again.

----- Original Message -----

Matthieu,

Thanks, I found some commits that are not referenced in any branch. How can I remove these from the reflog? I tried running "git reflog expire --expire=now --expire-unreachable=now --all" followed by "git gc" but still the same number of objects remain.

Thanks,

Andrew

On Tue, Feb 02, 2016 at 10:21:17AM -0600, Andrew Martin wrote:

Are the objects now loose, or still in packs? Git has a grace period for pruning objects, so that we do not delete objects for an in-progress operation. The life cycle of an unreferenced object should be something like:

  - reachable by reflogs, which are pruned after 30 days (or
    gc.reflogExpireUnreachable config). Objects will be repacked as
    normal during this time. Override with "reflog expire" as you did
    above.

  - after the reflog expires, the objects are now unreachable. During
    the next repack, they'll be ejected from the pack into loose
    objects, and their mtimes set to match the pack they came from
    (which is probably quite recent if you just repacked!).

  - after 2 weeks (or gc.pruneExpire), unreachable loose objects are
    dropped by "git prune", which is called as part of "git gc". This is
    based on the object mtime.

    You can accelerate this with "git gc --prune=now" (or
    "--prune=5.minutes.ago").

-Peff

----- Original Message -----

Jeff,

Thanks for the clarification. I now ran "git repack -A" followed by "git gc --prune=now", however I am still seeing the same number of objects. What else can I try to successfully mark these and unreachable and garbage collect them?

Thanks,

Andrew

On Tue, Feb 02, 2016 at 11:22:08AM -0600, Andrew Martin wrote:

That should clear out any unreachable objects. Are we sure that the objects in question are, in fact, unreachable?

Try:

  git rev-list --objects --all --reflog | wc -l

which should give a count of reachable objects. I'd expect that to line up with that "git count-objects -v" reports after having run your gc above.

In your original email, the discrepancy was between your "original" repository and the one that had round-tripped to a clone. Is it possible there are refs in the original that did not get pushed? Try comparing "git for-each-ref" in each repository.

We also consider objects in the index to be reachable for packing. Could your original perhaps have some uncommitted objects mentioned in the index?

-Peff

----- Original Message -----

Jeff,

This did it - those commits were still referenced by some remotes: $ git for-each-ref 945c3a60dfb4d9ab774708d19f7aa74dd545db90 commit refs/heads/master 945c3a60dfb4d9ab774708d19f7aa74dd545db90 commit refs/remotes/origin/brancha 8b331e4bb42f6291c33eb0847c4481407e3d753c commit refs/remotes/origin/branchb

I removed them: $ git update-ref -d refs/remotes/origin/brancha $ git update-ref -d refs/remotes/origin/branchb

And then I was able to garbage collect and get the expected object count: $ git reflog expire --expire=now --expire-unreachable=now --all $ git gc --prune=now

Thanks for the help!

Andrew