Re: oprofile on svn import

Stats after 18 hours into git-svnimport. Process is now stuck in the kernel 64% of the time. All of the kernel time is in page management. Perl svnimport process is 290MB now.

My top candidates for causing the problem are the fork in the perl code or the execing of a million tiny git processes.

The key low level git functions could be made into a library to avoid the need to exec them continuously. The svn functions are libraries and they hardly show up.

   606218  2.4143 /usr/local/bin/git-update-index
   127170  0.5065 /usr/local/bin/git-write-tree
    81153  0.3232 /usr/local/bin/git-read-tree
    13065  0.0520 /usr/local/bin/git-ls-files
     2624  0.0105 /usr/local/bin/git-hash-object
      754  0.0030 /usr/local/bin/git-commit-tree
      462  0.0018 /usr/local/bin/git-ls-tree
      398  0.0016 /usr/local/bin/git-rev-parse

versus

   102784  0.3641 /usr/lib/libsvn_subr-1.so.0.0.0
    70235  0.2488 /usr/lib/libsvn_fs_fs-1.so.0.0.0
    67081  0.2376 /usr/lib/libsvn_delta-1.so.0.0.0
      848  0.0030 /usr/lib/libsvn_swig_perl-1.so.0.0.0
      512  0.0018 /usr/lib/libsvn_ra_local-1.so.0.0.0
      350  0.0012 /usr/lib/libsvn_fs-1.so.0.0.0
      222 7.9e-04 /usr/lib/libsvn_repos-1.so.0.0.0
      124 4.4e-04 /usr/lib/libsvn_ra-1.so.0.0.0

------------------------------------------------------------------------------------------------------------

  4093890 64.3711 /home/good/vmlinux
   906014 14.2459 /lib/libcrypto.so.0.9.8a
   435744  6.8515 /lib/libc-2.4.so
   158325  2.4895 /usr/lib/libz.so.1.2.3
   139995  2.2012 /usr/local/bin/git-update-index
    75322  1.1843 /nvidia
    64349  1.0118 /usr/bin/oprofiled
    52825  0.8306 /usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
    51930  0.8165 /usr/lib/libapr-1.so.0.2.2
    42771  0.6725 /usr/local/bin/git-read-tree
    37774  0.5939 /lib/ld-2.4.so
    34761  0.5466 /usr/local/bin/git-write-tree
    29560  0.4648 /usr/lib/libsvn_subr-1.so.0.0.0
    28210  0.4436 /usr/lib/libaprutil-1.so.0.2.2

-----------------------------------------------------------------------------------------------------------------

2471826 32.8741 copy_page_range 375260 18.2903 unmap_vmas 574208 7.6367 release_pages 572189 7.6098 page_remove_rmap 233367 3.1037 free_pages_and_swap_cache 191051 2.5409 get_page_from_freelist 169058 2.2484 unlock_page 162027 2.1549 vm_normal_page 155691 2.0706 swap_info_get 136324 1.8130 swap_duplicate 119227 1.5857 page_fault 99729 1.3263 page_waitqueue 49288 0.6555 remove_exclusive_swap_page 39611 0.5268 do_wp_page 39142 0.5206 __wake_up_bit 34384 0.4573 __copy_from_user_ll 31111 0.4138 __handle_mm_fault 29990 0.3989 find_get_page 29682 0.3948 do_page_fault