Corrupt repo with force push when using a reference repo

Thank you for filling out a Git bug report! Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

With 3 clones of a repo, one on branch-a one a mirror,  another using the mirror as a reference repo on branch-b, do the following

# in the repo on branch-a, make a change, commit and push cd bad-object-test echo $(( RANDOM % 1000 + 1 )) > file.txt git ci -am "change a file" git push

# do a git fetch in the reference repo cd ../bad-object-test.git/ git fetch

# do a git pull in the repo on branch-b cd ../bad-object-test-with-reference-repo/ git pull

# All good so far

# in the repo on branch-a, discard HEAD and force push cd ../bad-object-test git reset --hard HEAD~1 git push --force

# do a git fetch in the reference repo and then prune loose objects cd ../bad-object-test.git/ git fetch git prune -v --expire=now

# do a git pull in the repo on branch-b cd ../bad-object-test-with-reference-repo git pull

What did you expect to happen? (Expected behavior)

the repo on branch-b does a force update of romote tracking branch branch-a

What happened instead? (Actual behavior) xxx> git pull fatal: bad orefs/remotes/origin/branch-a error: https://github.com/xxxxx/yyyyyyy.git did not send all necessary objects

What's different between what you expected and what actually happened?

I expected the force push to succeed, but insted I get a corrupt repo

Anything else you want to add:

git rev-parse refs/remotes/origin/branch-a will return the hash of the commit that was discarded by the git reset command

Please review the rest of the bug report below. You can delete any lines you don't wish to share.

[System Info] git version: git version 2.41.0 cpu: x86_64 no commit associated with this build sizeof-long: 8 sizeof-size_t: 8 shell-path: /bin/sh uname: Linux 5.15.0-79-generic #86-Ubuntu SMP Mon Jul 10 16:07:21 UTC 2023 x86_64 compiler info: gnuc: 11.3 libc info: glibc: 2.35 $SHELL (typically, interactive shell): /bin/bash

[Enabled Hooks]

On 2023-08-15 at 04:05:34, Grant Singleton wrote:

When you clone using --shared or --reference, you implicitly depend on the objects in the referenced repository existing without being pruned for the integrity of the repository.

The manual page for git clone says this:

  NOTE: this is a possibly dangerous operation; do not use it unless you
  understand what it does. If you clone your repository using this
  option and then delete branches (or use any other Git command that
  makes any existing commit unreferenced) in the source repository, some
  objects may become unreferenced (or dangling). These objects may be
  removed by normal Git operations (such as git commit) which
  automatically call git maintenance run --auto. (See
  git‐maintenance(1).) If these objects are removed and were referenced
  by the cloned repository, then the cloned repository will become
  corrupt.

If you want to change the references in any non-fast-forward way or do any sort of GC or pruning in the reference repository, you need to run `git repack -a` in the non-reference repository to repack all of the objects in that repository and dissociate it from the reference repository.

In general, unless you are _very_ sure you know what you're doing, you usually want to use `--dissociate` with `--reference` such that you detach immediately.

On Wed, Aug 16, 2023 at 01:21:11AM +0000, brian m. carlson wrote:

Ok, sorry and thanks, I should have read more of the documentation.

For some context, we're doing this on a bunch of Jenkins agents for our CI/CD pipeline on a somewhat bloating monolithic repo. We have 30-40 clones on each machine, so the reference repos save us a lot of network traffic, time to fetch and disk space.

So IIUC I can do a `git repack -kad` in the reference repo to pack loose objects and keep unreachable commits that may be referenced by other repos using the reference repo.

Grant Singleton Christchurch, New Zealand