git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2] fetch: limit shared symref check only for local branches

From
Orgad Shaneh <orgads@gmail.com>
Date
May 17, 2022, 06:05 UTC
Message-ID
<CAGHpTBJDeOMCfv36Sey1tGadQThS8mGR00YiK4C16BbV==W8XQ@mail.gmail.com>
In-Reply-To
<xmqqv8u54gcm.fsf@gitster.g>
On Mon, May 16, 2022 at 7:01 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 29 quoted lines
>
> "Orgad Shaneh via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Orgad Shaneh <orgads@gmail.com>
> >
> > This check was introduced in 8ee5d73137f (Fix fetch/pull when run without
> > --update-head-ok, 2008-10-13) in order to protect against replacing the ref
> > of the active branch by mistake, for example by running git fetch origin
> > master:master.
> >
> > It was later extended in 8bc1f39f411 (fetch: protect branches checked out
> > in all worktrees, 2021-12-01) to scan all worktrees.
> >
> > This operation is very expensive (takes about 30s in my repository) when
> > there are many tags or branches, and it is executed on every fetch, even if
> > no local heads are updated at all.
> >
> > Limit it to protect only refs/heads/* to improve fetch performance.
>
> The point of the check is to prevent the index+working tree in the
> worktrees to go out of sync with HEAD, and HEAD by definition can
> point only into refs/heads/*, this change should be OK.
>
> It is surprising find_shared_symref() is so expensive, though.  If
> you have a dozen worktrees linked to the current repository, there
> are at most a dozen HEAD that point at various refs in refs/heads/
> namespace.  Even if you need to check a thousand ref_map elements,
> it should cost almost nothing if you build a hashmap to find matches
> with these dozen HEADs upfront, no?

I also had this idea, but I'm not familiar enough with the codebase to implement it. I see you already started that.

Show 6 quoted lines
> Another thing that is surprising is that you say this loop is
> expensive when there are many tags or branches.  Do you mean it is
> expensive when there are many tags and branches that are updated, or
> it is expensive to merely have thousands of dormant tags and
> branches?  If the latter, I wonder if it is sensible to limit the
> check only to the refs that are going to be updated.

It's expensive even when *nothing* is updated. I have a repo with 44K tags, 13K of the tags are annotated, 134 remote branches and 4 worktrees (except the main repo) with 33 local branches.

I counted the calls to find_shared_symref - it was called 35755 times, and refs_read_raw_ref was called 357585 times.

- Orgad
Previous: Junio C HamanoNext: Junio C Hamano
Message 5 of 8 in “fetch: limit shared symref check only for local branches”
  1. fetch: limit shared symref check only for local branchesOrgad Shaneh via GitGitGadget, May 16, 2022
  2. fetch: limit shared symref check only for local branchesOrgad Shaneh via GitGitGadget, May 16, 2022
  3. Junio C HamanoMay 16, 2022
  4. Junio C HamanoMay 16, 2022
  5. Orgad ShanehMay 17, 2022
  6. Junio C HamanoMay 17, 2022
  7. Orgad ShanehMay 17, 2022
  8. Junio C HamanoMay 18, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.