git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Concurrent fetch commands

From
DSDragan Simic <dsimic@manjaro.org>
Date
Dec 31, 2023, 13:48 UTC
Message-ID
<0a98597e270276c67a7aafae20c6d073@manjaro.org>
In-Reply-To
<c11ca0b3-aaf4-4a8d-80a1-3832954aa7aa@haller-berlin.de>
On 2023-12-31 14:30, Stefan Haller wrote:
Show 23 quoted lines
> Currently, git doesn't seem to be very good at handling two concurrent
> invocations of git fetch (or git fetch and git pull). This is a problem
> because it is common for git clients to run git fetch periodically in
> the background. In that case, when you happen to invoke git pull while
> such a background fetch is running, an error occurs ("Cannot rebase 
> onto
> multiple branches").
> 
> I can reliably reproduce this by doing
> 
>    $ git fetch&; sleep 0.1; git pull
>    [1] 42160
>    [1]  + done       git fetch
>    fatal: Cannot rebase onto multiple branches.
> 
> The reason for this failure seems to be that both the first fetch and
> the fetch that runs as part of the pull append their information to
> .git/FETCH_HEAD, so that the information for the current branch ends up
> twice in the file.
> 
> Do you think git fetch should be made more robust against scenarios 
> like
> this?

I believe a similar issue has been already raised recently, so perhaps introducing some kind of file-based locking within git itself could be justified. It would make the things a bit more robust, and would also improve the overall user experience.

Show 29 quoted lines
> More context: the git client that I'm contributing to (lazygit) used to
> guard against this for its own background fetch with a global mutex 
> that
> allowed only one single fetch, pull, or push at a time. This solved the
> problem nicely for lazygit's own operations (at the expense of some 
> lag,
> occasionally); and I'm not aware of any reports about failures because
> some other git client's background fetch got in the way, so maybe we
> don't have to worry about that too much.
> 
> However, we now removed that mutex to allow certain parallel fetch
> operations to run at the same time, most notably fetching (and 
> updating)
> a branch that is not checked out (by doing "git fetch origin
> branch:branch"). It is useful to be able to trigger this for multiple
> branches concurrently, and actually this works fine.
> 
> But now we have the problem described above, where a pull of the
> checked-out branch runs at the same time as a background fetch; this is
> not so unlikely, because lazygit triggers the first background fetch at
> startup, so invoking the pull command right after starting lazygit is
> very likely to fail.
> 
> We could re-introduce a mutex and just make it a little less global;
> e.g. protect only pull and parameter-less fetch. But fixing it in git
> itself seems preferable to me.
> 
> Sorry for the wall of text, but I figured giving more context could be
> useful.
Previous: Stefan HallerNext: Konstantin Tokarev
Message 2 of 20 in “Concurrent fetch commands”
  1. Stefan HallerDec 31, 2023
  2. Dragan SimicDec 31, 2023
  3. Konstantin TokarevDec 31, 2023
  4. Dragan SimicDec 31, 2023
  5. Stefan HallerJan 1, 2024
  6. Federico KircheisJan 1, 2024
  7. Junio C HamanoDec 31, 2023
  8. Dragan SimicDec 31, 2023
  9. Stefan HallerJan 1, 2024
  10. Stefan HallerJan 1, 2024
  11. Patrick SteinhardtJan 3, 2024
  12. Patrick SteinhardtJan 3, 2024
  13. Patrick SteinhardtJan 3, 2024
  14. Taylor BlauJan 3, 2024
  15. Junio C HamanoJan 3, 2024
  16. Stefan HallerJan 4, 2024
  17. Mike HommeyJan 4, 2024
  18. Junio C HamanoJan 4, 2024
  19. Mike HommeyJan 4, 2024
  20. Taylor BlauJan 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.