git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Help troubleshoot performance regression cloning with depth: git 2.44 vs git 2.42

From
Dhruva Krishnamurthy <dhruvakm@gmail.com>
Date
May 1, 2024, 05:26 UTC
Message-ID
<CAKOHPAn1btewYTdLYWpW+fOaXMY+JQZsLCQxUSwoUqnnFN_ohA@mail.gmail.com>

Hello, Cloning by specifying depth exhibits performance regression in pack-objects (~20x). The repository I am cloning is on NFS (mounted with NFSv3 & positive lookup cache enabled).

Ran under 'perf' command to capture profiling information to see if something really stands out. There is a significant overhead in calls to file open/open64, fstat64 & mmap/munmap in git 2.44 compared to git 2.42. Not sure if there is an increase in the number of calls or something more is done.

Could someone please guide me on how to troubleshoot this better?

--- Details of the test environment and the clone commands with output --- # There are 10 loose objects under objects/08 $ git count-objects -vH count: 3627 size: 25.84 MiB in-pack: 1108374 packs: 2 size-pack: 303.38 MiB prune-packable: 0 garbage: 0 size-garbage: 0 bytes

# Simple driver script to enable performance tracking for upload-pack only $ cat trace-git-upload-pack #!/usr/bin/env bash export GIT_TRACE_PERFORMANCE=true exec git-upload-pack "$@"

# git clone with 2.42: pack objects take 17s
$ /opt/git/bin/git clone --no-checkout --no-local --depth=500
--upload-pack=$(pwd)/trace-git-upload-pack .. prod
Cloning into 'prod'...
remote: Enumerating objects: 669941, done.
remote: Counting objects: 100% (669941/669941), done.
remote: Compressing objects: 100% (154988/154988), done.
Receiving objects: 100% (669941/669941), 144.54 MiB | 25.78 MiB/s, done.
remote: Total 669941 (delta 533745), reused 645666 (delta 512193), pack-reused 0
remote: 05:35:40.654828 trace.c:414             performance:
17.098198597 s: git command: git --shallow-file '' pack-objects --revs
--thin --stdout --shallow --progress --delta-base-offset --include-tag
        05:35:40.708162 trace.c:414             performance:
24.764812264 s: git command: git-upload-pack /large_repo/perf/..
Resolving deltas: 100% (533745/533745), done.
Checking connectivity: 669940, done.
# git clone with 2.44: pack objects take 325s
$ /opt/gitn/bin/git clone --no-checkout --no-local --depth=500
--upload-pack=$(pwd)/trace-git-upload-pack .. dev
Cloning into 'dev'...
remote: Enumerating objects: 669941, done.
remote: Counting objects: 100% (669941/669941), done.
remote: Compressing objects: 100% (154988/154988), done.
Receiving objects: 100% (669941/669941), 144.66 MiB | 29.08 MiB/s, done.
remote: Total 669941 (delta 533742), reused 645666 (delta 512193),
pack-reused 0 (from 0)
remote: 05:42:01.017156 trace.c:414             performance:
325.552424902 s: git command: git --shallow-file '' pack-objects
--revs --thin --stdout --shallow --progress --delta-base-offset
--include-tag
        05:42:01.063013 trace.c:414             performance:
330.965731114 s: git command: git-upload-pack /large_repo/perf/..
Resolving deltas: 100% (533742/533742), done.
Checking connectivity: 669940, done.

Best regards, Dhruva

Next: Jeff King
Message 1 of 20 in “Help troubleshoot performance regression cloning with depth: git 2.44 vs git 2.42”
  1. Dhruva KrishnamurthyMay 1, 2024
  2. using tree as attribute source is slow, was Re: Help troubleshoot performance regression cloning with depth: git 2.44 vs git 2.42Jeff King, May 1, 2024
  3. rsbecker@nexbridge.comMay 1, 2024
  4. Junio C HamanoMay 1, 2024
  5. Taylor BlauMay 2, 2024
  6. Taylor BlauMay 2, 2024
  7. Junio C HamanoMay 2, 2024
  8. Taylor BlauMay 2, 2024
  9. Karthik NayakMay 2, 2024
  10. Junio C HamanoMay 2, 2024
  11. Dhruva KrishnamurthyMay 3, 2024
  12. Re* using tree as attribute source is slow, was Re: Help troubleshoot performance regression cloning with depth: git 2.44 vs git 2.42Junio C Hamano, May 3, 2024
  13. Jeff KingMay 3, 2024
  14. Taylor BlauMay 6, 2024
  15. John CaiMay 13, 2024
  16. attr.tree: HEAD:.gitattributes is no longer the default in a bare repoJunio C Hamano, Jun 5, 2024
  17. Jeff KingJun 6, 2024
  18. Junio C HamanoJun 6, 2024
  19. Dhruva KrishnamurthyMay 2, 2024
  20. Dhruva KrishnamurthyMay 2, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.