git/list[1] front-page[2] threads[3] people[4] search[5] about
 

RE: Troubleshoot clone issue to NFS.

From
Ssteve.norman@thomsonreuters.com <steve.norman@thomsonreuters.com>
Date
May 21, 2015, 15:53 UTC
Message-ID
<7FAE15F0A93C0144AD8B5FBD584E1C5519759641@C111KXTEMBX51.ERF.thomson.com>
In-Reply-To
<CACsJy8Cs6GcRQ-kgnSqwxP4MPHfds9qiir1_O1hc5cZ+0QP-EA@mail.gmail.com>
On Thu, May 21, 2015a at 9:31 PM, Duy Nguyen [mailto:pclouds@gmail.com], did scribble:
Show 8 quoted lines
> > In case an object is not found pack directory is re-read again, which
> > might cause some increased load on nfs.
> > has_sha1_file() not finding the object should not happen often..
> 
> That last statement is probably very wrong, but I have no time to test this
> now. In index-pack, there is a has_sha1_file() for file collision test. That call
> on a fresh clone would fail for _every_ object in the
> (new) pack and the cost of reprepare pack files could be sky high...
Confirmed with bisect that it is that commit:
~/git $ git bisect bad
45e8a7487339c0f0ea28244ef06851308d07387c is the first bad commit
commit 45e8a7487339c0f0ea28244ef06851308d07387c
Author: Jeff King <peff@peff.net>
Date:   Fri Aug 30 15:14:13 2013 -0400
I have an strace for that build but it is 153 megabytes so I probably shouldn't attach, but the call summary is below:
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 93.71   39.670084         103    386835      2171 futex
  3.16    1.338572           7    190550       181 open
  1.56    0.658786          18     37450         3 read
  0.62    0.262740           2    141390           pread
  0.41    0.171526           5     37313         9 write
  0.18    0.076166           0    188859    188835 access
  0.11    0.047941           0    374712           getdents
  0.11    0.045174          11      4067      3910 lstat
  0.06    0.023630           0    190425           close
  0.04    0.017995           6      2975         1 fstat
  0.02    0.007668        1917         4           wait4
  0.01    0.004150           1      5065           madvise
  0.01    0.003548           0     16090         8 recvfrom
  0.00    0.001872           0      8048           select
  0.00    0.001870          11       173         1 mkdir
  0.00    0.000872           0      8055           poll
  0.00    0.000262          22        12        12 readlink
  0.00    0.000185           0      1217      1146 stat
  0.00    0.000158           0       457           mprotect
  0.00    0.000074           0       298           mmap
  0.00    0.000069           1       109         8 rt_sigreturn
  0.00    0.000047           0       159           brk
  0.00    0.000021           1        17           getcwd
  0.00    0.000000           0        42         3 lseek
  0.00    0.000000           0        92           munmap
  0.00    0.000000           0        35           rt_sigaction
  0.00    0.000000           0         9           rt_sigprocmask
  0.00    0.000000           0         8         3 ioctl
  0.00    0.000000           0        11           pipe
  0.00    0.000000           0         3           dup
  0.00    0.000000           0         8           dup2
  0.00    0.000000           0         6           setitimer
  0.00    0.000000           0        11         1 socket
  0.00    0.000000           0         8         7 connect
  0.00    0.000000           0         8           sendto
  0.00    0.000000           0         2           recvmsg
  0.00    0.000000           0         1           bind
  0.00    0.000000           0         1           getsockname
  0.00    0.000000           0         3         1 getpeername
  0.00    0.000000           0         2           setsockopt
  0.00    0.000000           0         2           getsockopt
  0.00    0.000000           0         8           clone
  0.00    0.000000           0         5           execve
  0.00    0.000000           0         3           uname
  0.00    0.000000           0       100           fcntl
  0.00    0.000000           0         2           fsync
  0.00    0.000000           0        13           chdir
  0.00    0.000000           0        14           rename
  0.00    0.000000           0         2           link
  0.00    0.000000           0         5           unlink
  0.00    0.000000           0         2           symlink
  0.00    0.000000           0         9           chmod
  0.00    0.000000           0         6           getrlimit
  0.00    0.000000           0         2           sysinfo
  0.00    0.000000           0         8           getuid
  0.00    0.000000           0         1           statfs
  0.00    0.000000           0         5           arch_prctl
  0.00    0.000000           0         1           gettid
  0.00    0.000000           0         5           set_tid_address
  0.00    0.000000           0        13           set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00   42.333410               1594736    196300 total
Is there anything else I can provide or test?
Thanks for the help,
Steve
Previous: Duy NguyenNext: Duy Nguyen
Message 5 of 32 in “Troubleshoot clone issue to NFS.”
  1. steve.norman@thomsonreuters.comMay 21, 2015
  2. Christian CouderMay 21, 2015
  3. Duy NguyenMay 21, 2015
  4. Duy NguyenMay 21, 2015
  5. steve.norman@thomsonreuters.comMay 21, 2015
  6. Duy NguyenMay 22, 2015
  7. Jeff KingMay 22, 2015
  8. steve.norman@thomsonreuters.comMay 22, 2015
  9. Duy NguyenMay 22, 2015
  10. Junio C HamanoMay 22, 2015
  11. steve.norman@thomsonreuters.comMay 22, 2015
  12. 0/3 using stat() to avoid re-scanning pack dirJeff King, May 22, 2015
  13. 1/3 stat_validity: handle non-regular filesJeff King, May 22, 2015
  14. Michael HaggertyMay 23, 2015
  15. Jeff KingMay 24, 2015
  16. 2/3 cache.h: move stat_validity definition upJeff King, May 22, 2015
  17. 3/3 prepare_packed_git: use stat_validity to avoid re-reading packsJeff King, May 22, 2015
  18. Duy NguyenMay 23, 2015
  19. Duy NguyenMay 23, 2015
  20. Jeff KingMay 24, 2015
  21. Duy NguyenMay 24, 2015
  22. steve.norman@thomsonreuters.comJun 5, 2015
  23. Jeff KingJun 5, 2015
  24. index-pack: avoid excessive re-reading of pack directoryJeff King, Jun 5, 2015
  25. Jeff KingJun 9, 2015
  26. Jeff KingJun 9, 2015
  27. Shawn PearceJun 10, 2015
  28. Jeff KingJun 10, 2015
  29. Duy NguyenJun 10, 2015
  30. Shawn PearceJun 10, 2015
  31. steve.norman@thomsonreuters.comJun 5, 2015
  32. Jeff KingJun 16, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.