git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Anomaly with the new code - Re: git-svn performance

From
HLHin-Tak Leung <htl10@users.sourceforge.net>
Date
Oct 24, 2014, 07:06 UTC
Message-ID
<1414134386.28852.YahooMailBasic@web172306.mail.ir2.yahoo.com>
In-Reply-To
<1413999510.36832.YahooMailBasic@web172305.mail.ir2.yahoo.com>

I keep tabs of a particular svn repository over many years and run "git svn fetch --all" every few days. So that's the old clone. Since this discussion started, I made a new one with git 2.1.0 patched with the first two patches below, a couple of weeks ago. And I ran 'git svn fetch --all' on both every few days since.

I have added a few more patches, so the whole list is the 6 below against 2.1.0. The latest fetch is really strange - the fetch against the new clone took almost twice as long and uses almost twice as much memory, vs against the old. 17 min, 800 MB vs 10 min 400MB. Details below. Maybe this is a performance issue about how the clones were made?

0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch 0003-git-svn-reduce-check_cherry_pick-cache-overhead.patch 0004-git-svn-cache-only-mergeinfo-revisions.patch 0006-git-svn-clear-global-SVN-pool-between-get_log-invoca.patch 0007-git-svn-remove-mergeinfo-rev-caching.patch

(I dropped #5 because it doesn't seem interesting?)
<---
$ /usr/bin/time -v git svn fetch --all
...
	Command being timed: "git svn fetch --all"
	User time (seconds): 622.20
	System time (seconds): 12.52
	Percent of CPU this job got: 98%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 10:42.21
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 399588
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 320
	Minor (reclaiming a frame) page faults: 383987
	Voluntary context switches: 2088
	Involuntary context switches: 68304
	Swaps: 0
	File system inputs: 168288
	File system outputs: 148960
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
[Hin-Tak@localhost R]$ cd ../R-2/
[Hin-Tak@localhost R-2]$ /usr/bin/time -v git svn fetch --all
	M	src/library/stats/R/hclust.R
	M	src/library/stats/R/dendrogram.R
r66853 = 7c18b2e4084529d5912cf789c045f2eab7d4083c (refs/remotes/trunk)
	M	doc/manual/R-exts.texi
r66854 = bc7b131e34eaf04859fede1ecedb796c0a33be02 (refs/remotes/trunk)
	M	doc/manual/R-exts.texi
Checking svn:mergeinfo changes since r66844: 6 sources, 1 changed
W:svn cherry-pick ignored (/trunk:66824,66854) - missing 1084 commit(s) (eg 6453a2d844e27f2963ba87142028b023c50385ef)
r66855 = de5daf8db948732fa96c3d5b32077d8057e2a7e7 (refs/remotes/R-3-1-branch)
	M	src/modules/internet/internet.c
r66856 = a1e9300c6dd49ec4c3dd11f861bca0dbe3ca65b4 (refs/remotes/trunk)
	M	doc/manual/R-admin.texi
r66857 = eb5f3175e67a806482c39def71246f5d18bf8660 (refs/remotes/trunk)
	M	doc/manual/R-admin.texi
Checking svn:mergeinfo changes since r66855: 6 sources, 1 changed
W:svn cherry-pick ignored (/trunk:66854,66857) - missing 1086 commit(s) (eg e8cc0c31ddeeea3f8fa1ad47105d09a2c19e1a98)
r66858 = 10c8013f103d57c8a717b738e2a51c8d397c88f0 (refs/remotes/R-3-1-branch)
	M	VERSION
r66859 = 0f865f247da3191431bb17bcc3c307e8735dbd97 (refs/remotes/R-3-1-branch)
	Command being timed: "git svn fetch --all"
	User time (seconds): 1023.06
	System time (seconds): 15.30
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 17:27.65
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 785332
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 884
	Minor (reclaiming a frame) page faults: 527668
	Voluntary context switches: 2792
	Involuntary context switches: 107718
	Swaps: 0
	File system inputs: 194704
	File system outputs: 170032
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
--->
Previous: Hin-Tak LeungNext: Eric Wong
Message 2 of 4 in “Re: git-svn performance”
  1. Hin-Tak LeungOct 22, 2014
  2. Anomaly with the new code - Re: git-svn performanceHin-Tak Leung, Oct 24, 2014
  3. Eric WongOct 24, 2014
  4. Eric WongOct 25, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.