{"thread":{"id":"50687","subject":"[BUG] fetching all remote branches results in failed multiple updates","startedAt":"2019-03-07T21:44:58Z","lastAt":"2019-03-14T17:20:42Z","messageCount":6,"participants":["Alexander Huynh","SZEDER Gábor","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"370927","messageId":"20190307214447.GA4909@chabuduo","threadId":"50687","inReplyTo":null,"subject":"[BUG] fetching all remote branches results in failed multiple updates","fromName":"Alexander Huynh","fromEmail":"alex@grande.coffee","sentAt":"2019-03-07T21:44:47Z","receivedAt":"2019-03-07T21:44:58Z","isPatch":false,"sender":{"key":"alex@grande.coffee","avatar":null},"body":"Hello all,\n\nWhen running the latest release of git, I receive an error when attempting to\nfetch all remote branches from a repo:\n\n    fatal: multiple updates for ref 'refs/remotes/origin/maint' not allowed\n\nThe specific ref that it fails on changes depending on the repository, but the\nend result is the repo isn't cloned.\n\nThe specific configuration that causes this bug is the `remote.origin.fetch`\noption, specifically:\n\n    [remote \"origin\"]\n    \tfetch = +refs/heads/*:refs/remotes/origin/*\n\nThese settings are listed as an example in \"CONFIGURED REMOTE-TRACKING\nBRANCHES\" of git-fetch(1), as well as expanded upon in\nhttps://stackoverflow.com/a/40739835.\n\nI'm running git version 2.21.0.\n\nFull reproduction steps are below:\n\n    [root@chabuduo ~]# useradd -m git-test -s /bin/bash\n    [root@chabuduo ~]# sudo --preserve-env=SSH_AUTH_SOCK -u git-test -i\n    [git-test@chabuduo ~]$ git --version\n    git version 2.21.0\n    [git-test@chabuduo ~]$ cat > ~/.gitconfig\n    [remote \"origin\"]\n    \tfetch = +refs/heads/*:refs/remotes/origin/*\n    [git-test@chabuduo ~]$ cat ~/.gitconfig\n    [remote \"origin\"]\n    \tfetch = +refs/heads/*:refs/remotes/origin/*\n    [git-test@chabuduo ~]$ git clone git@github.com:git/git.git\n    Cloning into 'git'...\n    Warning: Permanently added the RSA host key for IP address '192.30.255.112' to the list of known hosts.\n    remote: Enumerating objects: 265113, done.\n    remote: Total 265113 (delta 0), reused 0 (delta 0), pack-reused 265113\n    Receiving objects: 100% (265113/265113), 113.20 MiB | 5.49 MiB/s, done.\n    Resolving deltas: 100% (196542/196542), done.\n    fatal: multiple updates for ref 'refs/remotes/origin/maint' not allowed\n"},{"id":"370929","messageId":"20190307234015.GD28939@szeder.dev","threadId":"50687","inReplyTo":"20190307214447.GA4909@chabuduo","subject":"Re: [BUG] fetching all remote branches results in failed multiple updates","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-03-07T23:40:15Z","receivedAt":"2019-03-07T23:40:22Z","isPatch":false,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Thu, Mar 07, 2019 at 09:44:47PM +0000, Alexander Huynh wrote:\n> Hello all,\n> \n> When running the latest release of git, I receive an error when attempting to\n> fetch all remote branches from a repo:\n> \n>     fatal: multiple updates for ref 'refs/remotes/origin/maint' not allowed\n> \n> The specific ref that it fails on changes depending on the repository, but the\n> end result is the repo isn't cloned.\n> \n> The specific configuration that causes this bug is the `remote.origin.fetch`\n> option, specifically:\n> \n>     [remote \"origin\"]\n>     \tfetch = +refs/heads/*:refs/remotes/origin/*\n> \n> These settings are listed as an example in \"CONFIGURED REMOTE-TRACKING\n> BRANCHES\" of git-fetch(1), as well as expanded upon in\n> https://stackoverflow.com/a/40739835.\n> \n> I'm running git version 2.21.0.\n> \n> Full reproduction steps are below:\n> \n>     [root@chabuduo ~]# useradd -m git-test -s /bin/bash\n>     [root@chabuduo ~]# sudo --preserve-env=SSH_AUTH_SOCK -u git-test -i\n>     [git-test@chabuduo ~]$ git --version\n>     git version 2.21.0\n>     [git-test@chabuduo ~]$ cat > ~/.gitconfig\n>     [remote \"origin\"]\n>     \tfetch = +refs/heads/*:refs/remotes/origin/*\n\nWhy do you do this?\n\nFirst, this is the default refspec configuration that clone will fetch\nand set up in the clone's config file anyway.  You don't have to set\nit.\n\nSecond, setting repository-specific stuff like the default fetch\nrefspec in the global configuration is not such a good idea.\n\nSo the simplest workaround is:\n\n  rm ~/.gitconfig; git clone git@github.com:git/git.git\n\n>     [git-test@chabuduo ~]$ cat ~/.gitconfig\n>     [remote \"origin\"]\n>     \tfetch = +refs/heads/*:refs/remotes/origin/*\n>     [git-test@chabuduo ~]$ git clone git@github.com:git/git.git\n>     Cloning into 'git'...\n>     Warning: Permanently added the RSA host key for IP address '192.30.255.112' to the list of known hosts.\n>     remote: Enumerating objects: 265113, done.\n>     remote: Total 265113 (delta 0), reused 0 (delta 0), pack-reused 265113\n>     Receiving objects: 100% (265113/265113), 113.20 MiB | 5.49 MiB/s, done.\n>     Resolving deltas: 100% (196542/196542), done.\n>     fatal: multiple updates for ref 'refs/remotes/origin/maint' not allowed\n\nAnyway, this is a side-effect of 515be83382 (clone: respect additional\nconfigured fetch refspecs during initial fetch, 2018-11-14).  Since\nthen the same refspec comes up twice in remote->fetch, once from the\nconfiguration and once the explicitly added default refspec.  And\nsince 'git clone' doesn't run a fully-fledged 'git fetch' and has its\nown simple refs-to-refspec matching logic, it ends up trying to write\neach ref twice, once for each of the two refspecs, in a single ref\ntransaction.\n\n\n\n"},{"id":"370931","messageId":"20190308000810.GA8044@chabuduo","threadId":"50687","inReplyTo":"20190307234015.GD28939@szeder.dev","subject":"Re: [BUG] fetching all remote branches results in failed multiple updates","fromName":"Alexander Huynh","fromEmail":"alex@grande.coffee","sentAt":"2019-03-08T00:08:10Z","receivedAt":"2019-03-08T00:08:28Z","isPatch":false,"sender":{"key":"alex@grande.coffee","avatar":null},"body":"On Fri, Mar 08, 2019 at 12:40:15AM +0100, SZEDER Gábor wrote:\n> On Thu, Mar 07, 2019 at 09:44:47PM +0000, Alexander Huynh wrote:\n> >     [git-test@chabuduo ~]$ cat > ~/.gitconfig\n> >     [remote \"origin\"]\n> >     \tfetch = +refs/heads/*:refs/remotes/origin/*\n>\n> Why do you do this?\n>\n> First, this is the default refspec configuration that clone will fetch\n> and set up in the clone's config file anyway.  You don't have to set\n> it.\n\nMy mistake. I was under the assumption that this configuration variable was\nneeded for git to fetch all remote refs, and that this behaviour wasn't\ndefault.\n\n> Anyway, this is a side-effect of 515be83382 (clone: respect additional\n> configured fetch refspecs during initial fetch, 2018-11-14).  Since\n> then the same refspec comes up twice in remote->fetch, once from the\n> configuration and once the explicitly added default refspec.  And\n> since 'git clone' doesn't run a fully-fledged 'git fetch' and has its\n> own simple refs-to-refspec matching logic, it ends up trying to write\n> each ref twice, once for each of the two refspecs, in a single ref\n> transaction.\n\nThanks for the clarification! One question: should we update the documentation\nat https://git-scm.com/docs/git-fetch#CRTB to not list this as an example to\nfollow?\n\nThanks again,\nAlex\n"},{"id":"370934","messageId":"xmqqef7ikw9f.fsf@gitster-ct.c.googlers.com","threadId":"50687","inReplyTo":"20190308000810.GA8044@chabuduo","subject":"Re: [BUG] fetching all remote branches results in failed multiple updates","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-03-08T01:09:00Z","receivedAt":"2019-03-08T01:09:07Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Alexander Huynh <alex@grande.coffee> writes:\n\n> Thanks for the clarification! One question: should we update the documentation\n> at https://git-scm.com/docs/git-fetch#CRTB to not list this as an example to\n> follow?\n\nThe example itself is valuable.  The way it is labeled might be\nmisleading to some people (although personally I do not find it\nso).  It says \"Typically such a variable may look like this\" and\nshows the fetch refspec that is given to a repository cloned from\nanywhere by default.\n\nI think it might help to update the text to clearly state that the\nsample one is what you get in a clone without doing anything\nspecial.  The example is primarily there to show you how each part\nof the fetch refspec is used in the operations described in the\nfollowing paragraphs,and it is not there to tell you that you have\nto add duplicate yourself to make these things described in the\nfollowing paragraphs to happen.\n\n"},{"id":"371420","messageId":"20190313234524.GA3053@chabuduo","threadId":"50687","inReplyTo":"xmqqef7ikw9f.fsf@gitster-ct.c.googlers.com","subject":"Re: [BUG] fetching all remote branches results in failed multiple updates","fromName":"Alexander Huynh","fromEmail":"alex@grande.coffee","sentAt":"2019-03-13T23:45:24Z","receivedAt":"2019-03-13T23:45:37Z","isPatch":false,"sender":{"key":"alex@grande.coffee","avatar":null},"body":"Hi all,\n\nI appreciate the help offered in identifying what I was doing incorrectly.\n\nUpon further investigation, I remembered why I added the original\n`origin.fetch` parameter: I was compensating for shallow clones not fetching\nall remote refs.\n\nHere's the use case that I have:\n\n    [root@chabuduo ~]# useradd -m git-test -s /bin/bash\n    [root@chabuduo ~]# sudo --preserve-env=SSH_AUTH_SOCK -u git-test -i\n    [git-test@chabuduo ~]$ git clone --depth=1 git@github.com:git/git.git\n    Cloning into 'git'...\n    The authenticity of host 'github.com (192.30.255.112)' can't be established.\n    RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.\n    Are you sure you want to continue connecting (yes/no)? yes\n    Warning: Permanently added 'github.com,192.30.255.112' (RSA) to the list of known hosts.\n    remote: Enumerating objects: 3664, done.\n    remote: Counting objects: 100% (3664/3664), done.\n    remote: Compressing objects: 100% (3282/3282), done.\n    remote: Total 3664 (delta 276), reused 1921 (delta 232), pack-reused 0\n    Receiving objects: 100% (3664/3664), 8.31 MiB | 9.61 MiB/s, done.\n    Resolving deltas: 100% (276/276), done.\n    [git-test@chabuduo ~]$ cd git/\n    [git-test@chabuduo git]$ git branch -a\n    * master\n      remotes/origin/HEAD -> origin/master\n      remotes/origin/master\n\nRunning a shallow clone results in only the remote `HEAD` ref being fetched,\nas observed by the last `git branch -a`.\n\nNo additional refs are fetched, even when promoting the shallow repository to\na full one:\n\n    [git-test@chabuduo git]$ git pull --unshallow\n    remote: Enumerating objects: 255783, done.\n    remote: Counting objects: 100% (255777/255777), done.\n    remote: Compressing objects: 100% (62079/62079), done.\n    remote: Total 253423 (delta 193725), reused 249056 (delta 189413), pack-reused 0\n    Receiving objects: 100% (253423/253423), 94.28 MiB | 33.96 MiB/s, done.\n    Resolving deltas: 100% (193725/193725), completed with 2155 local objects.\n    remote: Enumerating objects: 709, done.\n    remote: Total 709 (delta 0), reused 0 (delta 0), pack-reused 709\n    Receiving objects: 100% (709/709), 363.70 KiB | 12.12 MiB/s, done.\n    From github.com:git/git\n     * [new tag]             gitgui-0.10.0    -> gitgui-0.10.0\n     * [new tag]             gitgui-0.10.1    -> gitgui-0.10.1\n    ...\n    Already up to date.\n    [git-test@chabuduo git]$ git branch -a\n    * master\n      remotes/origin/HEAD -> origin/master\n      remotes/origin/master\n\nHow I've managed to work around the issue is by force fetching all refs, as\nexplained in the example at https://git-scm.com/docs/git-fetch#CRTB:\n\n    [git-test@chabuduo git]$ git fetch origin '+refs/heads/*:refs/remotes/origin/*'\n    remote: Enumerating objects: 1218, done.\n    remote: Counting objects: 100% (1218/1218), done.\n    remote: Total 7699 (delta 1218), reused 1218 (delta 1218), pack-reused 6481\n    Receiving objects: 100% (7699/7699), 4.31 MiB | 17.22 MiB/s, done.\n    Resolving deltas: 100% (4881/4881), completed with 435 local objects.\n    From github.com:git/git\n     * [new branch]            maint      -> origin/maint\n     * [new branch]            next       -> origin/next\n     * [new branch]            pu         -> origin/pu\n     * [new branch]            todo       -> origin/todo\n\nThis populates the missing references from a shallow clone:\n\n    [git-test@chabuduo git]$ git branch -a\n    * master\n      remotes/origin/HEAD -> origin/master\n      remotes/origin/maint\n      remotes/origin/master\n      remotes/origin/next\n      remotes/origin/pu\n      remotes/origin/todo\n\nI've created an alias that allows for filling out refs for future shallow\nclones:\n\n    % git config --global alias.fetcha '!f() { remote=\"${1:-origin}\"; git fetch \"$remote\" \"+refs/heads/*:refs/remotes/$remote/*\"; }; f'\n\nI'm also interested in more efficient ways to obtain all refs from shallow\nclones, if anyone has better ideas.\n\nThanks for taking the time to help me understand,\nAlex\n"},{"id":"371526","messageId":"20190314172032.GA10461@chabuduo","threadId":"50687","inReplyTo":"20190313234524.GA3053@chabuduo","subject":"Re: [BUG] fetching all remote branches results in failed multiple updates","fromName":"Alexander Huynh","fromEmail":"alex@grande.coffee","sentAt":"2019-03-14T17:20:32Z","receivedAt":"2019-03-14T17:20:42Z","isPatch":false,"sender":{"key":"alex@grande.coffee","avatar":null},"body":"Hi all,\n\nSorry to bother one last time. After a bit more digging, I found that the\nproblem was with my workflow.\n\nShallow clones set `remote.origin.fetch` to one branch, so what I was doing in\nthe past was overriding the respository-specific `remote.origin.fetch` with my\nglobal config.\n\nI've now changed my workflow to perform a shallow clone at first, `cd` into\nthe repository, and locally set `remote.origin.fetch` to\n`+refs/heads/*:refs/remotes/origin/*`.\n\nThanks again,\nAlex\n"}]}