git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Pathological performance with git remote rename and many tracking refs

From
brian m. carlson <sandals@crustytoothpaste.net>
Date
Apr 13, 2022, 22:32 UTC
Message-ID
<YldPmUskbU+bOU2n@camp.crustytoothpaste.net>

In my day-to-day work, I have the occasion to use GitHub Codespaces on a repository with about 20,000 refs on the server. The environment is set up to pre-clone the repository, but I use a different default remote name than "origin" ("def", to be particular), and thus, one of the things I do when I set up that environment is to run "git remote rename origin def".

This process takes 35 minutes, which is extremely pathological. I believe what's happening is that all of the refs are packed, and renaming the ref causes a loose ref to be created and the old ref to be deleted (necessitating a rewrite of the packed-refs file). This is essentially O(N^2) in the order of refs.

We recently added a --progress option, but I think this performance is bad enough that that's not going to suffice here, and we should try to do better.

I found that using "git for-each-ref" and "git update-ref --stdin" in a pipeline to create and delete the refs as a single transaction takes a little over 2 seconds. This is greater than a 99.9% improvement and is much more along the line of what I'd expect.

I thought about porting this code to use a ref transaction, but I realized that we don't rename reflogs in that situation, which might be a problem for some people. In my case, since it's a freshly cloned repo and the reflogs aren't interesting, I don't care.

I think a possible way forward may be to either teach ref transactions about ref renames, or simply to add a --no-reflogs option, which omits the reflogs in case the user doesn't care. I'm interested to hear ideas from others, though, about the best way forward.

-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA
Next: Ævar Arnfjörð Bjarmason
Message 1 of 6 in “Pathological performance with git remote rename and many tracking refs”
  1. brian m. carlsonApr 13, 2022
  2. Ævar Arnfjörð BjarmasonApr 14, 2022
  3. brian m. carlsonApr 15, 2022
  4. Ævar Arnfjörð BjarmasonApr 15, 2022
  5. Junio C HamanoApr 15, 2022
  6. Ævar Arnfjörð BjarmasonApr 16, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.