When Git fetches in a superproject with --recurse-submodules, it appears to try to fetch the corresponding submodule repository commits for every new or updated superproject branch. Presumably this is so that everthing is ready to switch to one of those branches without further fetching.

Developers may create commits that contain submodules that reference commits in the submodule repository, but those commits may not be pushed to the submodule's remote repository. When the superproject commits are pushed to a personal remote branch anyone else's Git fetch cannot find the corresponding submodule commit and fails. For example:

 $ git fetch
 remote: Enumerating objects: 4, done.
 remote: Counting objects: 100% (4/4), done.
 remote: Compressing objects: 100% (2/2), done.
 remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
 Unpacking objects: 100% (3/3), 355 bytes | 355.00 KiB/s, done.
 From ssh://localhost/home/mac/git/git/repro/repositories/super
  * [new branch]      repro-branch -> origin/repro-branch
 Fetching submodule submodule
 error: Server does not allow request for unadvertised object f6b0ccce6e2085cf03c3fd924730f5c9f91e3db1
 Errors during submodule fetch:
         submodule

(when fetching via ssh)

or:

 fatal: remote error: want c1f59d10bd6f24adbc96fee6a5041e9f3dc94b7c not valid
 fatal: internal server error
 fatal: remote error: want f91af98469911e79c2a27329d8e115dfc59f31c0 not valid
 fatal: internal server error
 Errors during submodule fetch:
 	sources/repo-1
 	sources/repo-2

(when using a JGit server configured to allow any SHA-1 to be fetched.)

These are hard errors that cause Git to exit with a non-zero exit status. Repeating the operation succeeds because no there is no update to the remote branch to trigger the submodule fetch again.

I've had a couple of goes at bisecting this but I always end up failing on unrelated commits due to the master/main default branch change and my attempts to work around that produce inconsistent results.

I don't believe that developers who have the ability to create personal branches should be able to force anyone else cloning or fetching from the repository to suffer such a failure. This is particularly a problem for CI systems but it confuses users too.

Potential mitigations:

1. Use --no-recurse-submodules. This disables all submodule processing
   though, which is not desirable.

2. Use `git fetch || git fetch` to repeat the fetch if it fails the first
   time. This will work around the problem almost all of the time but is
   racy since there's a small risk that the second fetch will encounter a
   new branch with the same problem.

I've added a script which can be used to reproduce the problem to the end of this message.

I'm not really sure what a good solution to this is:

1. The recursive fetch could only look for submodule commits to fetch on
   the current tracking branch.

2. Treat any failure to fetch submodules as non-fatal. Hacking
   fetch_submodules() to always return zero does solve this problem but at
   the cost of not failing for more-serious ones! Any attempt to check out
   the unfetchable commit would fail at that point though.

Thanks.

Mike.

--8<-- #!/bin/bash set -xe temp=$(pwd)/repro

rm -rf ${temp} mkdir ${temp} repositories=${temp}/repositories

# Work around protocol.file.allow = "user" by default in new # repositories without affecting the user's global Git configuration # by fetching over ssh from localhost. remote=ssh://localhost${repositories}

# Create the "remote" repositories mkdir -p ${repositories}/{super,sub} git -C ${repositories}/super init --bare git -C ${repositories}/sub init --bare

# Create original submodule repository contents mkdir -p ${temp}/sub pushd ${temp}/sub git init . date > submodule-file git add submodule-file git commit -m "Initial submodule commit" git remote add origin ${remote}/sub git push -u origin popd rm -rf ${temp}/sub

# Create original super repository contents workspace=${temp}/workspace mkdir -p ${workspace} pushd ${workspace} git clone ${remote}/super orig pushd orig date > file git add file git commit -m "Initial supermodule commit" git submodule add ${remote}/sub submodule git commit -m "Add initial submodule" git push popd

# Create a new clone of the super and submodules git clone --recurse-submodules ${remote}/super clone

# Now create a dangling submodule commit in the original supermodule date >> ${workspace}/orig/submodule/submodule-file git -C ${workspace}/orig/submodule add submodule-file git -C ${workspace}/orig/submodule commit -m "Modify submodule" git -C ${workspace}/orig checkout -b repro-branch git -C ${workspace}/orig add submodule git -C ${workspace}/orig commit -m "Bump submodule" git -C ${workspace}/orig push origin repro-branch

# Now update the second clone to show an error even though the # missing submodule commit is on a completely different branch. git -C ${workspace}/clone fetch