Re: git clone --reference=<repository> (without --dissociate), forgets to copy/get_pack_of objects from <repository> when "--no-local"/"file://" is present
- From
Han Jiang <jhcarl0814@gmail.com>
- Date
- Aug 14, 2024, 04:56 UTC
- Message-ID
- <CANrWfmQw4pG6Dz3kCPqc-P4wLKm_VqyHmb+4MwKgfCUUaN5bzw@mail.gmail.com>
- In-Reply-To
- <20240813131614.GF968816@coredump.intra.peff.net>
Thank you for pointing out the error! My commands' use of git is incorrect, so the results don't make sense. I will post the corrected commands here in case anyone needs them.
1. git clone --reference=<repository>, then the referenced repository is updated (e.g. amend/rebase), how to keep up to date:
cd '/'; cd '/'; rm --force --recursive -- './test_git'; mkdir "$_"; cd "$_"; mkdir --parents -- './server' './client';
git -C './server' init --bare './repo.git' git --git-dir='./server/repo.git' --work-tree='.' commit --allow-empty -m "$((++number))" find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --shared '../server/repo.git' './repo1' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --shared --local '../server/repo.git' './repo2' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --shared --no-local '../server/repo.git' './repo3' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --shared 'file://'"$(realpath './server/repo.git')" './repo4' find './server/repo.git/objects' -printf '%p %n\n'
cp --recursive --no-target-directory -- './server/repo.git' './client/mirror.git/'
git -C './client' clone --reference='./mirror.git' '../server/repo.git' './repo5' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --local '../server/repo.git' './repo6' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --no-local '../server/repo.git' './repo7' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' 'file://'"$(realpath './server/repo.git')" './repo8' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --dissociate '../server/repo.git' './repo9' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --dissociate --local '../server/repo.git' './repo10' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --dissociate --no-local '../server/repo.git' './repo11' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --dissociate 'file://'"$(realpath './server/repo.git')" './repo12' find './server/repo.git/objects' -printf '%p %n\n'
tree './client/repo1/.git/objects' tree './client/repo2/.git/objects' tree './client/repo3/.git/objects' tree './client/repo4/.git/objects' tree './client/repo5/.git/objects' tree './client/repo6/.git/objects' tree './client/repo7/.git/objects' tree './client/repo8/.git/objects' tree './client/repo9/.git/objects' tree './client/repo10/.git/objects' tree './client/repo11/.git/objects' tree './client/repo12/.git/objects'
git -C './client/repo1' log git -C './client/repo2' log git -C './client/repo3' log git -C './client/repo4' log git -C './client/repo5' log git -C './client/repo6' log git -C './client/repo7' log git -C './client/repo8' log git -C './client/repo9' log git -C './client/repo10' log git -C './client/repo11' log git -C './client/repo12' log
git -C './server/repo.git' cat-file --batch-check --batch-all-objects git --git-dir='./server/repo.git' --work-tree='.' commit --amend --allow-empty -m "$((++number))" git -C './server/repo.git' cat-file --batch-check --batch-all-objects git -C './server/repo.git' -c 'gc.pruneExpire=now' -c 'gc.reflogExpireUnreachable=now' gc git -C './server/repo.git' cat-file --batch-check --batch-all-objects git -C './client/repo1' log git -C './client/repo2' log git -C './client/repo3' log git -C './client/repo4' log git -C './client/repo5' log git -C './client/repo6' log git -C './client/repo7' log git -C './client/repo8' log git -C './client/repo9' log git -C './client/repo10' log git -C './client/repo11' log git -C './client/repo12' log
rm --force --recursive -- './client/mirror.git/' cp --recursive --no-target-directory -- './server/repo.git' './client/mirror.git/' git -C './client/repo1' log git -C './client/repo2' log git -C './client/repo3' log git -C './client/repo4' log git -C './client/repo5' log git -C './client/repo6' log git -C './client/repo7' log git -C './client/repo8' log git -C './client/repo9' log git -C './client/repo10' log git -C './client/repo11' log git -C './client/repo12' log
git -C './client/repo7' branch --all --verbose --verbose git -C './client/repo7' branch --all --verbose git -C './client/repo7' branch --all git -C './client/repo7' update-ref -d 'refs/heads/main' git -C './client/repo7' update-ref -d 'refs/remotes/origin/main' git -C './client/repo7' pull git -C './client/repo7' branch --all --verbose --verbose tree './client/repo7/.git/objects'
git -C './client/repo8' branch --all --verbose --verbose git -C './client/repo8' branch --all --verbose git -C './client/repo8' branch --all git -C './client/repo8' update-ref -d 'refs/heads/main' git -C './client/repo8' update-ref -d 'refs/remotes/origin/main' git -C './client/repo8' pull git -C './client/repo8' branch --all --verbose --verbose tree './client/repo8/.git/objects'
2. git clone --reference=<repository>, then the referenced repository is removed (e.g. rm), how to switch back to not-referencing-the-repository mode:
cd '/'; cd '/'; rm --force --recursive -- './test_git'; mkdir "$_"; cd "$_"; mkdir --parents -- './server' './client';
git -C './server' init --bare './repo.git' git --git-dir='./server/repo.git' --work-tree='.' commit --allow-empty -m "$((++number))" find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --shared '../server/repo.git' './repo1' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --shared --local '../server/repo.git' './repo2' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --shared --no-local '../server/repo.git' './repo3' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --shared 'file://'"$(realpath './server/repo.git')" './repo4' find './server/repo.git/objects' -printf '%p %n\n'
cp --recursive --no-target-directory -- './server/repo.git' './client/mirror.git/'
git -C './client' clone --reference='./mirror.git' '../server/repo.git' './repo5' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --local '../server/repo.git' './repo6' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --no-local '../server/repo.git' './repo7' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' 'file://'"$(realpath './server/repo.git')" './repo8' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --dissociate '../server/repo.git' './repo9' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --dissociate --local '../server/repo.git' './repo10' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --dissociate --no-local '../server/repo.git' './repo11' find './server/repo.git/objects' -printf '%p %n\n'
git -C './client' clone --reference='./mirror.git' --dissociate 'file://'"$(realpath './server/repo.git')" './repo12' find './server/repo.git/objects' -printf '%p %n\n'
tree './client/repo1/.git/objects' tree './client/repo2/.git/objects' tree './client/repo3/.git/objects' tree './client/repo4/.git/objects' tree './client/repo5/.git/objects' tree './client/repo6/.git/objects' tree './client/repo7/.git/objects' tree './client/repo8/.git/objects' tree './client/repo9/.git/objects' tree './client/repo10/.git/objects' tree './client/repo11/.git/objects' tree './client/repo12/.git/objects'
git -C './client/repo1' log git -C './client/repo2' log git -C './client/repo3' log git -C './client/repo4' log git -C './client/repo5' log git -C './client/repo6' log git -C './client/repo7' log git -C './client/repo8' log git -C './client/repo9' log git -C './client/repo10' log git -C './client/repo11' log git -C './client/repo12' log
rm --force --recursive -- './client/mirror.git/' git -C './client/repo1' log git -C './client/repo2' log git -C './client/repo3' log git -C './client/repo4' log git -C './client/repo5' log git -C './client/repo6' log git -C './client/repo7' log git -C './client/repo8' log git -C './client/repo9' log git -C './client/repo10' log git -C './client/repo11' log git -C './client/repo12' log
git -C './client/repo7' branch --all --verbose --verbose git -C './client/repo7' branch --all --verbose git -C './client/repo7' branch --all rm './client/repo7/.git/objects/info/alternates' git -C './client/repo7' branch --all --verbose --verbose git -C './client/repo7' branch --all --verbose git -C './client/repo7' branch --all git -C './client/repo7' update-ref -d 'refs/heads/main' git -C './client/repo7' pull git -C './client/repo7' branch --all --verbose --verbose tree './client/repo7/.git/objects'
git -C './client/repo8' branch --all --verbose --verbose git -C './client/repo8' branch --all --verbose git -C './client/repo8' branch --all rm './client/repo8/.git/objects/info/alternates' git -C './client/repo8' branch --all --verbose --verbose git -C './client/repo8' branch --all --verbose git -C './client/repo8' branch --all git -C './client/repo8' update-ref -d 'refs/heads/main' git -C './client/repo8' pull git -C './client/repo8' branch --all --verbose --verbose tree './client/repo8/.git/objects'
On Wed, Aug 14, 2024 at 1:16 AM Jeff King <peff@peff.net> wrote:
Show 42 quoted lines
> > On Tue, Aug 06, 2024 at 09:37:33AM +1200, Han Jiang wrote: > > > What did you do before the bug happened? (Steps to reproduce your issue) > > > > (repo7 and repo8 should behave similar to repo5 and repo6, i.e. setup > > ".git/objects/info/alternates" && copy/get_pack_of objects;) > > (repo7 and repo8 shouldn't: forget to copy/get_pack_of objects.) > > (The problem only surfaces after "./client/mirror.git/" is removed: > > repo7 and repo8 should have the original objects (so git log shouldn't > > give error) but they don't have.) > > I don't think this expectation is correct. Using "--reference" without > "-dissociate" creates a dependency on the mirror repository. It's not > safe to delete it. This is discussed in the "--shared" description of > git-clone(1), and the "--reference" description refers to it. > > So in your example: > > > git -C './client' clone --reference='./mirror.git' --no-local '../server/repo.git' './repo7' > > I would not expect repo7 to have any objects. Its needs were satisfied > by the mirror, and when it told the upload-pack process running in > server/repo.git what it had, that process knew that it did not need to > send any objects. > > It's actually your repo5 and repo6 that are the oddballs: > > > git -C './client' clone --reference='./mirror.git' '../server/repo.git' './repo5' > > Here since we are cloning a local-file repo, we take a shortcut and do > not do any object negotiation at all! Instead, we just take all of the > objects from server/repo.git as hardlinks, since doing so is less work > than figuring out what we might need. > > It makes us robust to mirror.git being removed (since we ended up with > duplicates of all of its objects anyway). But it's still not safe in the > long run. If the mirror gets new objects, we'd rely on them for > subsequent fetches (rather than getting new copies), and the same > dependency is created. > > -Peff