# Git multiple remotes push stop at first failed connection 5 messages from 2020-06-01 to 2020-06-02. Participants: John Siu, Jeff King, Junio C Hamano. Thread: https://gitlist.dev/t/53588 ## John Siu, 2020-06-01 00:28 Subject: Git multiple remotes push stop at first failed connection Message-ID: URL: https://gitlist.dev/e/CAGKX4vGhTbEqZS9%2BiYA2wZWRRaddQC6O4KV%2BzLaNYKkZgN36Xg%40mail.gmail.com ``` Hi, git version 2.26.2 OS: MacOS 10.15.5 Install via brew Let say my project has following remotes: $ git remote -v git.all "server A git url" (fetch) git.all "server A git url" (push) git.all "server B git url" (push) git.all "server C git ur" (push) When all serverA/B/C are online, "git push" works. However "git push" will stop at the first server it failed to connect. So if git cannot connect to server A, it will not continue with server B/C. In the past I have server C turn off from time to time, so failing the last push is expected. However recently server A went offline completely and we notice git is not pushing to the remaining 2 remotes. Not sure if this is intended behavior or can be improved. John Siu ``` ## Jeff King, 2020-06-01 21:40 Subject: Re: Git multiple remotes push stop at first failed connection Message-ID: <20200601214003.GA3309882@coredump.intra.peff.net> URL: https://gitlist.dev/e/20200601214003.GA3309882%40coredump.intra.peff.net In-Reply-To: ``` On Sun, May 31, 2020 at 08:28:38PM -0400, John Siu wrote: > Let say my project has following remotes: > > $ git remote -v > git.all "server A git url" (fetch) > git.all "server A git url" (push) > git.all "server B git url" (push) > git.all "server C git ur" (push) > > When all serverA/B/C are online, "git push" works. A slight nomenclature nit, but that's _one_ remote that has several push urls. > However "git push" will stop at the first server it failed to connect. > So if git cannot connect to server A, it will not continue with server > B/C. > > In the past I have server C turn off from time to time, so failing the > last push is expected. However recently server A went offline > completely and we notice git is not pushing to the remaining 2 > remotes. > > Not sure if this is intended behavior or can be improved. I don't think we've ever documented the error-handling semantics. Looking at the relevant code in builtin/push.c:do_push(): url_nr = push_url_of_remote(remote, &url); if (url_nr) { for (i = 0; i < url_nr; i++) { struct transport *transport = transport_get(remote, url[i]); if (flags & TRANSPORT_PUSH_OPTIONS) transport->push_options = push_options; if (push_with_options(transport, push_refspec, flags)) errs++; } } else { struct transport *transport = transport_get(remote, NULL); if (flags & TRANSPORT_PUSH_OPTIONS) transport->push_options = push_options; if (push_with_options(transport, push_refspec, flags)) errs++; } return !!errs; it does seem to try each one and collect the errors. But the underlying transport code is so ready to die() on errors, taking down the whole process, that I suspect it rarely manages to do so. You're probably much better off defining a separate remote for each push destination, then running your own shell loop: err=0 for dst in serverA serverB serverC; do git push $dst || err=1 done exit $err There's really no benefit to doing it all in a single Git process, as we'd connect to each independently, run a separate independent pack-objects for each, etc. I'd even suggest that Git implement such a loop itself, as we did for "git fetch --all", but sadly "push --all" is already taken for a different meaning (but it might still be worth doing under a different option name). -Peff ``` ## John Siu, 2020-06-02 01:32 Subject: Re: Git multiple remotes push stop at first failed connection Message-ID: URL: https://gitlist.dev/e/CAGKX4vH42x7nhCgtJ80fiNeaj1ROVxc-R8gO6xV21-6D3GbQSg%40mail.gmail.com In-Reply-To: <20200601214003.GA3309882@coredump.intra.peff.net> ``` Thank Jeff for replying. Yes, one route we are looking into is use fetch -all, push -all and wrap push tag with a loop, or may do what you suggest and wrap all 3 in a loop for consistency. On Mon, Jun 1, 2020 at 5:40 PM Jeff King wrote: > > On Sun, May 31, 2020 at 08:28:38PM -0400, John Siu wrote: > > > Let say my project has following remotes: > > > > $ git remote -v > > git.all "server A git url" (fetch) > > git.all "server A git url" (push) > > git.all "server B git url" (push) > > git.all "server C git ur" (push) > > > > When all serverA/B/C are online, "git push" works. > > A slight nomenclature nit, but that's _one_ remote that has several > push urls. > > > However "git push" will stop at the first server it failed to connect. > > So if git cannot connect to server A, it will not continue with server > > B/C. > > > > In the past I have server C turn off from time to time, so failing the > > last push is expected. However recently server A went offline > > completely and we notice git is not pushing to the remaining 2 > > remotes. > > > > Not sure if this is intended behavior or can be improved. > > I don't think we've ever documented the error-handling semantics. > Looking at the relevant code in builtin/push.c:do_push(): > > url_nr = push_url_of_remote(remote, &url); > if (url_nr) { > for (i = 0; i < url_nr; i++) { > struct transport *transport = > transport_get(remote, url[i]); > if (flags & TRANSPORT_PUSH_OPTIONS) > transport->push_options = push_options; > if (push_with_options(transport, push_refspec, flags)) > errs++; > } > } else { > struct transport *transport = > transport_get(remote, NULL); > if (flags & TRANSPORT_PUSH_OPTIONS) > transport->push_options = push_options; > if (push_with_options(transport, push_refspec, flags)) > errs++; > } > return !!errs; > > it does seem to try each one and collect the errors. But the underlying > transport code is so ready to die() on errors, taking down the whole > process, that I suspect it rarely manages to do so. You're probably much > better off defining a separate remote for each push destination, then > running your own shell loop: > > err=0 > for dst in serverA serverB serverC; do > git push $dst || err=1 > done > exit $err > > There's really no benefit to doing it all in a single Git process, as > we'd connect to each independently, run a separate independent > pack-objects for each, etc. > > I'd even suggest that Git implement such a loop itself, as we did for > "git fetch --all", but sadly "push --all" is already taken for a > different meaning (but it might still be worth doing under a different > option name). > > -Peff ``` ## Junio C Hamano, 2020-06-02 16:26 Subject: Re: Git multiple remotes push stop at first failed connection Message-ID: URL: https://gitlist.dev/e/xmqqr1uxtow4.fsf%40gitster.c.googlers.com In-Reply-To: <20200601214003.GA3309882@coredump.intra.peff.net> ``` Jeff King writes: > There's really no benefit to doing it all in a single Git process, as > we'd connect to each independently, run a separate independent > pack-objects for each, etc. > > I'd even suggest that Git implement such a loop itself, as we did for > "git fetch --all", but sadly "push --all" is already taken for a > different meaning (but it might still be worth doing under a different > option name). I wonder if it is possible to update the implementation to do so without changing the UI at all, though. The presence of the "--all" option in "fetch" command is tied closely to the fact that it makes no sense to have multiple URLs that are used to download from at the same time under a single remote name (e.g. what should "remotes/origin/master" point at if two URLs say different things if such an arrangement were allowed?). On the other hand, the pushURL for a single remote can be multiple places for redundancy (a possible #leftoverbits here is that we should probably disable the "pretend that we immediately turned around and fetched from them after pushing" optimization when pushing to a remote that has multiple pushURLs defined) does not need an extra option. If the way we currently push is suboptimal and it is better to spawn a separate "git push" instance via the run_command() API, that can safely be done as a bugfix without affecting any UI elements, no? ``` ## John Siu, 2020-06-02 16:54 Subject: Re: Git multiple remotes push stop at first failed connection Message-ID: URL: https://gitlist.dev/e/CAGKX4vFcqQ_0XFb5qOku9wAxF3%2Bfj-fByrm%2BzmSXHr3k60yjKw%40mail.gmail.com In-Reply-To: ``` On Tue, Jun 2, 2020 at 12:26 PM Junio C Hamano wrote: > > Jeff King writes: > > > There's really no benefit to doing it all in a single Git process, as > > we'd connect to each independently, run a separate independent > > pack-objects for each, etc. > > > > I'd even suggest that Git implement such a loop itself, as we did for > > "git fetch --all", but sadly "push --all" is already taken for a > > different meaning (but it might still be worth doing under a different > > option name). > Yes. We notice the fetch/push --all is for branches. > I wonder if it is possible to update the implementation to do so > without changing the UI at all, though. > > The presence of the "--all" option in "fetch" command is tied > closely to the fact that it makes no sense to have multiple URLs > that are used to download from at the same time under a single > remote name (e.g. what should "remotes/origin/master" point at if > two URLs say different things if such an arrangement were allowed?). > > On the other hand, the pushURL for a single remote can be multiple > places for redundancy (a possible #leftoverbits here is that we > should probably disable the "pretend that we immediately turned > around and fetched from them after pushing" optimization when > pushing to a remote that has multiple pushURLs defined) does not > need an extra option. If the way we currently push is suboptimal > and it is better to spawn a separate "git push" instance via the > run_command() API, that can safely be done as a bugfix without > affecting any UI elements, no? > I agree a "bugfix" for push only is good enough and safe. As the current behavior is already pushing to all pushURLs of a single remote. We are not trying to change behavior or do anything extra. ```