Re: git fetch recursion problem

From: Glen Choo <chooglen@google.com>
Date: Jun 1, 2023, 22:40 UTC
Message-ID: <kl6lzg5iztzw.fsf@chooglen-macbookpro.roam.corp.google.com>
In-Reply-To: <CA+3o5aPQ3BkbAn3u6qSMCkAfNgTcpjT+_ikdqAkkc69NBGybpw@mail.gmail.com>

Fraser Hanson <fraser.hanson@gmail.com> writes:

Show 221 quoted lines

> The git trace output looks like this (some server names redacted):
>     $ export GIT_TRACE=1
>     $ git fetch
>     07:41:19.325652 git.c:439               trace: built-in: git fetch
>     07:41:19.330118 run-command.c:655       trace: run_command:
> GIT_DIR=.git git remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:19.337765 git.c:725               trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:19.338220 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:20.664527 run-command.c:655       trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
>     07:41:20.679814 git.c:439               trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
>     07:41:20.681735 run-command.c:655       trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:20.692494 git.c:725               trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:20.693132 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:21.695677 run-command.c:655       trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
>     07:41:21.715621 git.c:439               trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
>     07:41:21.717230 run-command.c:655       trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:21.727616 git.c:725               trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:21.728337 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:22.801117 run-command.c:655       trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
>     07:41:22.816026 git.c:439               trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
>     07:41:22.817972 run-command.c:655       trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:22.828436 git.c:725               trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:22.829219 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:24.242220 run-command.c:655       trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
>     07:41:24.257211 git.c:439               trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
>     07:41:24.259264 run-command.c:655       trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:24.269718 git.c:725               trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:24.270612 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:25.306129 run-command.c:655       trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
>     07:41:25.320842 git.c:439               trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
>     07:41:25.322801 run-command.c:655       trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:25.332775 git.c:725               trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:25.333522 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:26.332580 run-command.c:655       trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
>     07:41:26.348349 git.c:439               trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
>     07:41:26.350225 run-command.c:655       trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:26.361171 git.c:725               trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:26.361969 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:27.585641 run-command.c:655       trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
>     07:41:27.600316 git.c:439               trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
>     07:41:27.602370 run-command.c:655       trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:27.611823 git.c:725               trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:27.612607 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:28.666477 run-command.c:655       trace: run_command: git -c
> fetch.negotiationAlgorithm=noop fetch origin --no-tags
> --no-write-fetch-head --recurse-submodules=no --filter=blob:none
> --stdin
>     07:41:28.678956 git.c:439               trace: built-in: git fetch
> origin --no-tags --no-write-fetch-head --recurse-submodules=no
> --filter=blob:none --stdin
>     07:41:28.680665 run-command.c:655       trace: run_command: git
> remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:28.690534 git.c:725               trace: exec:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:41:28.691277 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://repomirror.eng.corporate.com/github-neovim/nvim-treesitter/nvim-treesitter-textobjects.git
>     ^C
>
>
> Random clues and experiments:
>
> A `git clone` from our corporate mirrors always succeeds.  Immediately
> after a `git clone`, I am always able to `git fetch`, so far.
> Something else must cause the repository to enter a broken state,
> possibly the addition of upstream commits.  I can't verify that since
> I can't complete `git fetch.`
>
> I can fix a broken repository by deleting these settings from `.git/config`:
>         promisor = true
>         partialclonefilter = blob:none
> After this, `git fetch` succeeds.
> Deleting just one of these settings is not sufficient, it must be both.
>
> I copied one such broken repository out of our secure environment onto
> a mac laptop that has access to both our mirror sites and to the
> internet.
> I verified that `git fetch` from our mirror is still broken on the mac.
> Then I modified .git/config to point to the https://github.com/ url
> instead of our mirror site, leaving the 'promisor' and
> 'partialclonefilter' settings in place.
> Running `git fetch` in this state succeeded.
>
> This suggests that our internal mirror site is returning something
> different from github itself.
>
>
> Here is the trace from successfully fetching from github.com:
>     $ git fetch
>     07:43:30.901275 git.c:439               trace: built-in: git fetch
>     07:43:30.904188 run-command.c:655       trace: run_command:
> GIT_DIR=.git git remote-https origin
> https://github.com/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:43:30.917345 git.c:725               trace: exec:
> git-remote-https origin
> https://github.com/nvim-treesitter/nvim-treesitter-textobjects.git
>     07:43:30.918560 run-command.c:655       trace: run_command:
> git-remote-https origin
> https://github.com/nvim-treesitter/nvim-treesitter-textobjects.git
>     remote: Enumerating objects: 8, done.
>     remote: Counting objects: 100% (8/8), done.
>     remote: Compressing objects: 100% (5/5), done.
>     07:43:31.714337 run-command.c:655       trace: run_command: git
> index-pack --stdin -v --fix-thin --promisor --pack_header=2,8
>     remote: Total 8 (delta 2), reused 5 (delta 1), pack-reused 0
>     07:43:31.729793 git.c:439               trace: built-in: git
> index-pack --stdin -v --fix-thin --promisor --pack_header=2,8
>     Receiving objects: 100% (8/8), 3.36 KiB | 3.36 MiB/s, done.
>     Resolving deltas: 100% (2/2), completed with 1 local object.
>     07:43:31.738815 run-command.c:655       trace: run_command: git
> rev-list --objects --stdin --exclude-promisor-objects --not --all
> --quiet --alternate-refs
>     07:43:31.748286 git.c:439               trace: built-in: git
> rev-list --objects --stdin --exclude-promisor-objects --not --all
> --quiet --alternate-refs
>     From https://github.com/nvim-treesitter/nvim-treesitter-textobjects
>        9c5237b..95b76b9  master        -> origin/master
>        9c5237b..11ae4c7  update-readme -> origin/update-readme
>     07:43:31.757504 run-command.c:1524      run_processes_parallel:
> preparing to run up to 1 tasks
>     07:43:31.757515 run-command.c:1551      run_processes_parallel: done
>     07:43:31.757522 run-command.c:655       trace: run_command: git
> maintenance run --auto --no-quiet
>     07:43:31.765282 git.c:439               trace: built-in: git
> maintenance run --auto --no-quiet
>
> The difference starts just a few lines in, my failed fetches never see this:
>     remote: Counting objects: 100% (8/8), done.
>     remote: Compressing objects: 100% (5/
>
> Next I enabled both GIT_TRACE and GIT_CURL_VERBOSE to identify
> differences in the HTTP conversation between a failed fetch versus a
> successful one.
> The successful run was made by deleting the `promisor` and
> `partialclonefilter` settings from .git/config.
> I'm having some some trouble attaching these to this email, I'll have
> to include them in a follow up.

This looks a lot like the partial clone fetch is recursing infinitely. This could happen if Git encountered a missing object, initiated a partial clone fetch to get the missing object, then in the process of that fetch it encountered a missing object, initiated a partial clone fetch to get the missing object, then (etc etc etc).

I don't have good clues as to why that might be the case though.

Wild speculation: Does this reproduce if you clone without submodules, or if you pass --no-recurse-submodules to "git fetch"? In a non-partial clone fetch, "git fetch" may read .gitmodules to get submodule information. Perhaps something similar is happening in the partial clone fetch.