git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Concurrent fetch commands

From
SHStefan Haller <lists@haller-berlin.de>
Date
Dec 31, 2023, 13:30 UTC
Message-ID
<c11ca0b3-aaf4-4a8d-80a1-3832954aa7aa@haller-berlin.de>

Currently, git doesn't seem to be very good at handling two concurrent invocations of git fetch (or git fetch and git pull). This is a problem because it is common for git clients to run git fetch periodically in the background. In that case, when you happen to invoke git pull while such a background fetch is running, an error occurs ("Cannot rebase onto multiple branches").

I can reliably reproduce this by doing
   $ git fetch&; sleep 0.1; git pull
   [1] 42160
   [1]  + done       git fetch
   fatal: Cannot rebase onto multiple branches.

The reason for this failure seems to be that both the first fetch and the fetch that runs as part of the pull append their information to .git/FETCH_HEAD, so that the information for the current branch ends up twice in the file.

Do you think git fetch should be made more robust against scenarios like this?

More context: the git client that I'm contributing to (lazygit) used to guard against this for its own background fetch with a global mutex that allowed only one single fetch, pull, or push at a time. This solved the problem nicely for lazygit's own operations (at the expense of some lag, occasionally); and I'm not aware of any reports about failures because some other git client's background fetch got in the way, so maybe we don't have to worry about that too much.

However, we now removed that mutex to allow certain parallel fetch operations to run at the same time, most notably fetching (and updating) a branch that is not checked out (by doing "git fetch origin branch:branch"). It is useful to be able to trigger this for multiple branches concurrently, and actually this works fine.

But now we have the problem described above, where a pull of the checked-out branch runs at the same time as a background fetch; this is not so unlikely, because lazygit triggers the first background fetch at startup, so invoking the pull command right after starting lazygit is very likely to fail.

We could re-introduce a mutex and just make it a little less global; e.g. protect only pull and parameter-less fetch. But fixing it in git itself seems preferable to me.

Sorry for the wall of text, but I figured giving more context could be useful.

Thanks, Stefan

Next: Dragan Simic
Message 1 of 20 in “Concurrent fetch commands”
  1. Stefan HallerDec 31, 2023
  2. Dragan SimicDec 31, 2023
  3. Konstantin TokarevDec 31, 2023
  4. Dragan SimicDec 31, 2023
  5. Stefan HallerJan 1, 2024
  6. Federico KircheisJan 1, 2024
  7. Junio C HamanoDec 31, 2023
  8. Dragan SimicDec 31, 2023
  9. Stefan HallerJan 1, 2024
  10. Stefan HallerJan 1, 2024
  11. Patrick SteinhardtJan 3, 2024
  12. Patrick SteinhardtJan 3, 2024
  13. Patrick SteinhardtJan 3, 2024
  14. Taylor BlauJan 3, 2024
  15. Junio C HamanoJan 3, 2024
  16. Stefan HallerJan 4, 2024
  17. Mike HommeyJan 4, 2024
  18. Junio C HamanoJan 4, 2024
  19. Mike HommeyJan 4, 2024
  20. Taylor BlauJan 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.