# [BUG] fetching all remote branches results in failed multiple updates 6 messages from 2019-03-07 to 2019-03-14. Participants: Alexander Huynh, SZEDER Gábor, Junio C Hamano. Thread: https://gitlist.dev/t/50687 ## Alexander Huynh, 2019-03-07 21:44 Subject: [BUG] fetching all remote branches results in failed multiple updates Message-ID: <20190307214447.GA4909@chabuduo> URL: https://gitlist.dev/e/20190307214447.GA4909%40chabuduo ``` Hello all, When running the latest release of git, I receive an error when attempting to fetch all remote branches from a repo: fatal: multiple updates for ref 'refs/remotes/origin/maint' not allowed The specific ref that it fails on changes depending on the repository, but the end result is the repo isn't cloned. The specific configuration that causes this bug is the `remote.origin.fetch` option, specifically: [remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/* These settings are listed as an example in "CONFIGURED REMOTE-TRACKING BRANCHES" of git-fetch(1), as well as expanded upon in https://stackoverflow.com/a/40739835. I'm running git version 2.21.0. Full reproduction steps are below: [root@chabuduo ~]# useradd -m git-test -s /bin/bash [root@chabuduo ~]# sudo --preserve-env=SSH_AUTH_SOCK -u git-test -i [git-test@chabuduo ~]$ git --version git version 2.21.0 [git-test@chabuduo ~]$ cat > ~/.gitconfig [remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/* [git-test@chabuduo ~]$ cat ~/.gitconfig [remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/* [git-test@chabuduo ~]$ git clone git@github.com:git/git.git Cloning into 'git'... Warning: Permanently added the RSA host key for IP address '192.30.255.112' to the list of known hosts. remote: Enumerating objects: 265113, done. remote: Total 265113 (delta 0), reused 0 (delta 0), pack-reused 265113 Receiving objects: 100% (265113/265113), 113.20 MiB | 5.49 MiB/s, done. Resolving deltas: 100% (196542/196542), done. fatal: multiple updates for ref 'refs/remotes/origin/maint' not allowed ``` ## SZEDER Gábor, 2019-03-07 23:40 Subject: Re: [BUG] fetching all remote branches results in failed multiple updates Message-ID: <20190307234015.GD28939@szeder.dev> URL: https://gitlist.dev/e/20190307234015.GD28939%40szeder.dev In-Reply-To: <20190307214447.GA4909@chabuduo> ``` On Thu, Mar 07, 2019 at 09:44:47PM +0000, Alexander Huynh wrote: > Hello all, > > When running the latest release of git, I receive an error when attempting to > fetch all remote branches from a repo: > > fatal: multiple updates for ref 'refs/remotes/origin/maint' not allowed > > The specific ref that it fails on changes depending on the repository, but the > end result is the repo isn't cloned. > > The specific configuration that causes this bug is the `remote.origin.fetch` > option, specifically: > > [remote "origin"] > fetch = +refs/heads/*:refs/remotes/origin/* > > These settings are listed as an example in "CONFIGURED REMOTE-TRACKING > BRANCHES" of git-fetch(1), as well as expanded upon in > https://stackoverflow.com/a/40739835. > > I'm running git version 2.21.0. > > Full reproduction steps are below: > > [root@chabuduo ~]# useradd -m git-test -s /bin/bash > [root@chabuduo ~]# sudo --preserve-env=SSH_AUTH_SOCK -u git-test -i > [git-test@chabuduo ~]$ git --version > git version 2.21.0 > [git-test@chabuduo ~]$ cat > ~/.gitconfig > [remote "origin"] > fetch = +refs/heads/*:refs/remotes/origin/* Why do you do this? First, this is the default refspec configuration that clone will fetch and set up in the clone's config file anyway. You don't have to set it. Second, setting repository-specific stuff like the default fetch refspec in the global configuration is not such a good idea. So the simplest workaround is: rm ~/.gitconfig; git clone git@github.com:git/git.git > [git-test@chabuduo ~]$ cat ~/.gitconfig > [remote "origin"] > fetch = +refs/heads/*:refs/remotes/origin/* > [git-test@chabuduo ~]$ git clone git@github.com:git/git.git > Cloning into 'git'... > Warning: Permanently added the RSA host key for IP address '192.30.255.112' to the list of known hosts. > remote: Enumerating objects: 265113, done. > remote: Total 265113 (delta 0), reused 0 (delta 0), pack-reused 265113 > Receiving objects: 100% (265113/265113), 113.20 MiB | 5.49 MiB/s, done. > Resolving deltas: 100% (196542/196542), done. > fatal: multiple updates for ref 'refs/remotes/origin/maint' not allowed Anyway, this is a side-effect of 515be83382 (clone: respect additional configured fetch refspecs during initial fetch, 2018-11-14). Since then the same refspec comes up twice in remote->fetch, once from the configuration and once the explicitly added default refspec. And since 'git clone' doesn't run a fully-fledged 'git fetch' and has its own simple refs-to-refspec matching logic, it ends up trying to write each ref twice, once for each of the two refspecs, in a single ref transaction. ``` ## Alexander Huynh, 2019-03-08 00:08 Subject: Re: [BUG] fetching all remote branches results in failed multiple updates Message-ID: <20190308000810.GA8044@chabuduo> URL: https://gitlist.dev/e/20190308000810.GA8044%40chabuduo In-Reply-To: <20190307234015.GD28939@szeder.dev> ``` On Fri, Mar 08, 2019 at 12:40:15AM +0100, SZEDER Gábor wrote: > On Thu, Mar 07, 2019 at 09:44:47PM +0000, Alexander Huynh wrote: > > [git-test@chabuduo ~]$ cat > ~/.gitconfig > > [remote "origin"] > > fetch = +refs/heads/*:refs/remotes/origin/* > > Why do you do this? > > First, this is the default refspec configuration that clone will fetch > and set up in the clone's config file anyway. You don't have to set > it. My mistake. I was under the assumption that this configuration variable was needed for git to fetch all remote refs, and that this behaviour wasn't default. > Anyway, this is a side-effect of 515be83382 (clone: respect additional > configured fetch refspecs during initial fetch, 2018-11-14). Since > then the same refspec comes up twice in remote->fetch, once from the > configuration and once the explicitly added default refspec. And > since 'git clone' doesn't run a fully-fledged 'git fetch' and has its > own simple refs-to-refspec matching logic, it ends up trying to write > each ref twice, once for each of the two refspecs, in a single ref > transaction. Thanks for the clarification! One question: should we update the documentation at https://git-scm.com/docs/git-fetch#CRTB to not list this as an example to follow? Thanks again, Alex ``` ## Junio C Hamano, 2019-03-08 01:09 Subject: Re: [BUG] fetching all remote branches results in failed multiple updates Message-ID: URL: https://gitlist.dev/e/xmqqef7ikw9f.fsf%40gitster-ct.c.googlers.com In-Reply-To: <20190308000810.GA8044@chabuduo> ``` Alexander Huynh writes: > Thanks for the clarification! One question: should we update the documentation > at https://git-scm.com/docs/git-fetch#CRTB to not list this as an example to > follow? The example itself is valuable. The way it is labeled might be misleading to some people (although personally I do not find it so). It says "Typically such a variable may look like this" and shows the fetch refspec that is given to a repository cloned from anywhere by default. I think it might help to update the text to clearly state that the sample one is what you get in a clone without doing anything special. The example is primarily there to show you how each part of the fetch refspec is used in the operations described in the following paragraphs,and it is not there to tell you that you have to add duplicate yourself to make these things described in the following paragraphs to happen. ``` ## Alexander Huynh, 2019-03-13 23:45 Subject: Re: [BUG] fetching all remote branches results in failed multiple updates Message-ID: <20190313234524.GA3053@chabuduo> URL: https://gitlist.dev/e/20190313234524.GA3053%40chabuduo In-Reply-To: ``` Hi all, I appreciate the help offered in identifying what I was doing incorrectly. Upon further investigation, I remembered why I added the original `origin.fetch` parameter: I was compensating for shallow clones not fetching all remote refs. Here's the use case that I have: [root@chabuduo ~]# useradd -m git-test -s /bin/bash [root@chabuduo ~]# sudo --preserve-env=SSH_AUTH_SOCK -u git-test -i [git-test@chabuduo ~]$ git clone --depth=1 git@github.com:git/git.git Cloning into 'git'... The authenticity of host 'github.com (192.30.255.112)' can't be established. RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'github.com,192.30.255.112' (RSA) to the list of known hosts. remote: Enumerating objects: 3664, done. remote: Counting objects: 100% (3664/3664), done. remote: Compressing objects: 100% (3282/3282), done. remote: Total 3664 (delta 276), reused 1921 (delta 232), pack-reused 0 Receiving objects: 100% (3664/3664), 8.31 MiB | 9.61 MiB/s, done. Resolving deltas: 100% (276/276), done. [git-test@chabuduo ~]$ cd git/ [git-test@chabuduo git]$ git branch -a * master remotes/origin/HEAD -> origin/master remotes/origin/master Running a shallow clone results in only the remote `HEAD` ref being fetched, as observed by the last `git branch -a`. No additional refs are fetched, even when promoting the shallow repository to a full one: [git-test@chabuduo git]$ git pull --unshallow remote: Enumerating objects: 255783, done. remote: Counting objects: 100% (255777/255777), done. remote: Compressing objects: 100% (62079/62079), done. remote: Total 253423 (delta 193725), reused 249056 (delta 189413), pack-reused 0 Receiving objects: 100% (253423/253423), 94.28 MiB | 33.96 MiB/s, done. Resolving deltas: 100% (193725/193725), completed with 2155 local objects. remote: Enumerating objects: 709, done. remote: Total 709 (delta 0), reused 0 (delta 0), pack-reused 709 Receiving objects: 100% (709/709), 363.70 KiB | 12.12 MiB/s, done. From github.com:git/git * [new tag] gitgui-0.10.0 -> gitgui-0.10.0 * [new tag] gitgui-0.10.1 -> gitgui-0.10.1 ... Already up to date. [git-test@chabuduo git]$ git branch -a * master remotes/origin/HEAD -> origin/master remotes/origin/master How I've managed to work around the issue is by force fetching all refs, as explained in the example at https://git-scm.com/docs/git-fetch#CRTB: [git-test@chabuduo git]$ git fetch origin '+refs/heads/*:refs/remotes/origin/*' remote: Enumerating objects: 1218, done. remote: Counting objects: 100% (1218/1218), done. remote: Total 7699 (delta 1218), reused 1218 (delta 1218), pack-reused 6481 Receiving objects: 100% (7699/7699), 4.31 MiB | 17.22 MiB/s, done. Resolving deltas: 100% (4881/4881), completed with 435 local objects. From github.com:git/git * [new branch] maint -> origin/maint * [new branch] next -> origin/next * [new branch] pu -> origin/pu * [new branch] todo -> origin/todo This populates the missing references from a shallow clone: [git-test@chabuduo git]$ git branch -a * master remotes/origin/HEAD -> origin/master remotes/origin/maint remotes/origin/master remotes/origin/next remotes/origin/pu remotes/origin/todo I've created an alias that allows for filling out refs for future shallow clones: % git config --global alias.fetcha '!f() { remote="${1:-origin}"; git fetch "$remote" "+refs/heads/*:refs/remotes/$remote/*"; }; f' I'm also interested in more efficient ways to obtain all refs from shallow clones, if anyone has better ideas. Thanks for taking the time to help me understand, Alex ``` ## Alexander Huynh, 2019-03-14 17:20 Subject: Re: [BUG] fetching all remote branches results in failed multiple updates Message-ID: <20190314172032.GA10461@chabuduo> URL: https://gitlist.dev/e/20190314172032.GA10461%40chabuduo In-Reply-To: <20190313234524.GA3053@chabuduo> ``` Hi all, Sorry to bother one last time. After a bit more digging, I found that the problem was with my workflow. Shallow clones set `remote.origin.fetch` to one branch, so what I was doing in the past was overriding the respository-specific `remote.origin.fetch` with my global config. I've now changed my workflow to perform a shallow clone at first, `cd` into the repository, and locally set `remote.origin.fetch` to `+refs/heads/*:refs/remotes/origin/*`. Thanks again, Alex ```