git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Why is "git fetch --prune" so much slower than "git remote prune"?

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Mar 6, 2015, 16:48 UTC
Message-ID
<CACBZZX5n5tTCSa-_A5gQzbzboF_v8a3_oVUjdjyFtKHHe8h-NA@mail.gmail.com>

The --prune option to fetch added in v1.6.5-8-gf360d84 seems to be around 20-30x slower than the equivalent operation with git remote prune. I'm wondering if I'm missing something and fetch does something more, but it doesn't seem so.

To test this clone git.git, create 1000 branches it in, create two local clones of that clone and then delete the 1000 branches in the original. I have a script to do this at https://gist.github.com/avar/497c8c8fbd641fb756ef

Then in each of the clones:
    $ git branch -a|wc -l; time (~/g/git/git-fetch --prune origin
>/dev/null 2>&1); git branch -a | wc -l
1003
    real    0m3.337s
    user    0m2.996s
    sys     0m0.336s
    3
    $ git branch -a|wc -l; time (~/g/git/git-remote prune origin
>/dev/null 2>&1); git branch -a | wc -l
    1003
    real    0m0.067s
    user    0m0.020s
    sys     0m0.040s
    3

Both of these ends up doing a "git fetch", so it's not that. I'm quite rusty in C profiling but here's a gprof of the git-fetch command:

$ gprof ~/g/git/git-fetch|head -n 20 Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total
 time   seconds   seconds    calls   s/call   s/call  name
 26.42      0.33     0.33  1584583     0.00     0.00  strbuf_getwholeline
 14.63      0.51     0.18 90601347     0.00     0.00  strbuf_grow
 13.82      0.68     0.17  1045676     0.00     0.00  find_pack_entry_one
  8.13      0.78     0.10  1050062     0.00     0.00  check_refname_format
  6.50      0.86     0.08  1584675     0.00     0.00  get_sha1_hex
  5.69      0.93     0.07  2100529     0.00     0.00  starts_with
  3.25      0.97     0.04  1044043     0.00     0.00  refname_is_safe
  3.25      1.01     0.04     8007     0.00     0.00  get_packed_ref_cache
  2.44      1.04     0.03  2605595     0.00     0.00  search_ref_dir
  2.44      1.07     0.03  1040500     0.00     0.00  peel_entry
  1.63      1.09     0.02  2632661     0.00     0.00  get_ref_dir
  1.63      1.11     0.02  1044043     0.00     0.00  create_ref_entry
  1.63      1.13     0.02     8024     0.00     0.00  do_for_each_entry_in_dir
  0.81      1.14     0.01  2155105     0.00     0.00  memory_limit_check
  0.81      1.15     0.01  1580503     0.00     0.00  sha1_to_hex
And of the git-remote command:

$ gprof ~/g/git/git-remote|head -n 20 Flat profile:

Each sample counts as 0.01 seconds.
 no time accumulated
  %   cumulative   self              self     total
 time   seconds   seconds    calls  Ts/call  Ts/call  name
  0.00      0.00     0.00   197475     0.00     0.00  strbuf_grow
  0.00      0.00     0.00    24214     0.00     0.00  sort_ref_dir
  0.00      0.00     0.00    24190     0.00     0.00  search_ref_dir
  0.00      0.00     0.00    21661     0.00     0.00  memory_limit_check
  0.00      0.00     0.00    20236     0.00     0.00  get_ref_dir
  0.00      0.00     0.00     9187     0.00     0.00  xrealloc
  0.00      0.00     0.00     7048     0.00     0.00  strbuf_add
  0.00      0.00     0.00     6348     0.00     0.00  do_xmalloc
  0.00      0.00     0.00     6126     0.00     0.00  xcalloc
  0.00      0.00     0.00     6056     0.00     0.00  cleanup_path
  0.00      0.00     0.00     6050     0.00     0.00  get_git_dir
  0.00      0.00     0.00     6050     0.00     0.00  vsnpath
  0.00      0.00     0.00     5554     0.00     0.00  config_file_fgetc

Aside from the slowness of git-fetch it seems git-remote can be sped up quite a bit by more aggressively allocating a larger string buffer from the get-go.

Next: Jeff King
Message 1 of 8 in “Why is "git fetch --prune" so much slower than "git remote prune"?”
  1. Ævar Arnfjörð BjarmasonMar 6, 2015
  2. Jeff KingMar 6, 2015
  3. Michael HaggertyMar 19, 2015
  4. Jeff KingMar 19, 2015
  5. Junio C HamanoMar 19, 2015
  6. Jeff KingMar 19, 2015
  7. Michael HaggertyMar 20, 2015
  8. Junio C HamanoMar 20, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.