git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Concurrent fetch commands

From
DSDragan Simic <dsimic@manjaro.org>
Date
Dec 31, 2023, 17:41 UTC
Message-ID
<f7d4f092ea8955113c2d6c0346932b70@manjaro.org>
In-Reply-To
<xmqqy1daffk8.fsf@gitster.g>
On 2023-12-31 18:27, Junio C Hamano wrote:
Show 47 quoted lines
> Stefan Haller <lists@haller-berlin.de> writes:
> 
>> I can reliably reproduce this by doing
>> 
>>    $ git fetch&; sleep 0.1; git pull
>>    [1] 42160
>>    [1]  + done       git fetch
>>    fatal: Cannot rebase onto multiple branches.
> 
> I see a bug here.
> 
> How this _ought_ to work is
> 
>  - The first "git fetch" wants to report what it fetched by writing
>    into the $GIT_DIR/FETCH_HEAD file ("git merge FETCH_HEAD" after
>    the fetch finishes can consume its contents).
> 
>  - The second "git pull" runs "git fetch" under the hood.  Because
>    it also wants to write to $GIT_DIR/FETCH_HEAD, and because there
>    is already somebody writing to the file, it should notice and
>    barf, saying "fatal: a 'git fetch' is already working" or
>    something.
> 
> But because there is no "Do not overwrite FETCH_HEAD somebody else
> is using" protection, "git merge" or "git rebase" that is run as the
> second half of the "git pull" ends up working on the contents of
> FETCH_HEAD that is undefined, and GIGO result follows.
> 
> The "bug" that the second "git fetch" does not notice an already
> running one (who is in possession of FETCH_HEAD) and refrain from
> starting is not easy to design a fix for---we cannot just abort by
> opening it with O_CREAT|O_EXCL because it is a normal thing for
> $GIT_DIR/FETCH_HEAD to exist after the "last" fetch.  We truncate
> its contents before starting to avoid getting affected by contents
> leftover by the last fetch, but when there is a "git fetch" that is
> actively running, and it finishes _after_ the second one starts and
> truncates the file, the second one will end up seeing the contents
> the first one left.  We have the "--no-write-fetch-head" option for
> users to explicitly tell which invocation of "git fetch" should not
> write FETCH_HEAD.
> 
> Running "background/priming" fetches (the one before "sleep 0.1" you
> have) is not a crime by itself, but it is a crime to run them
> without the "--no-fetch-head" option.  Since you have *NO* intention
> of using its contents to feed a "git merge" (or equivalent)
> yourself, you are breaking your "git pull" step in your example
> reproduction yourself.
Thank you very much for this highly detailed explanation.
Previous: Junio C HamanoNext: Stefan Haller
Message 8 of 20 in “Concurrent fetch commands”
  1. Stefan HallerDec 31, 2023
  2. Dragan SimicDec 31, 2023
  3. Konstantin TokarevDec 31, 2023
  4. Dragan SimicDec 31, 2023
  5. Stefan HallerJan 1, 2024
  6. Federico KircheisJan 1, 2024
  7. Junio C HamanoDec 31, 2023
  8. Dragan SimicDec 31, 2023
  9. Stefan HallerJan 1, 2024
  10. Stefan HallerJan 1, 2024
  11. Patrick SteinhardtJan 3, 2024
  12. Patrick SteinhardtJan 3, 2024
  13. Patrick SteinhardtJan 3, 2024
  14. Taylor BlauJan 3, 2024
  15. Junio C HamanoJan 3, 2024
  16. Stefan HallerJan 4, 2024
  17. Mike HommeyJan 4, 2024
  18. Junio C HamanoJan 4, 2024
  19. Mike HommeyJan 4, 2024
  20. Taylor BlauJan 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.