{"thread":{"id":"43458","subject":"Re: git-svn: File added multiple times?","startedAt":"2006-12-02T21:38:00Z","lastAt":"2006-12-03T16:42:01Z","messageCount":6,"participants":["Florian Weimer","Eric Wong"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"295286","messageId":"87psb2ou6f.fsf@mid.deneb.enyo.de","threadId":"43458","inReplyTo":null,"subject":"git-svn: File added multiple times?","fromName":"Florian Weimer","fromEmail":"fw@deneb.enyo.de","sentAt":"2006-12-02T21:38:00Z","receivedAt":"2006-12-02T21:38:00Z","isPatch":false,"sender":{"key":"fw@deneb.enyo.de","avatar":null},"body":"Is this expected?\n\n$ sort /tmp/git-svn-output | uniq -c | sort -rn | head\n 4 A mlton/trunk/doc/web/papers/index.html\n 4 A mlton/trunk/doc/web/papers/01-icfp.ps.gz\n 4 A mlton/trunk/doc/web/papers/00-esop.ps.gz\n 4 A mlton/trunk/doc/examples/save-world/save-world.sml\n 4 A mlton/trunk/doc/examples/save-world/Makefile\n 4 A mlton/trunk/doc/examples/profiling/profiling.sml\n 4 A mlton/trunk/doc/examples/profiling/Makefile\n 4 A mlton/trunk/doc/examples/ffi/Makefile\n 4 A mlton/trunk/doc/examples/ffi/main.sml\n 4 A mlton/trunk/doc/examples/ffi/ffi.h\n$ \n\nIt's somewhat counter-intuitive, at least. This is with Debian's\n"},{"id":"298010","messageId":"20061202223419.GA7057@localdomain","threadId":"43458","inReplyTo":"87psb2ou6f.fsf@mid.deneb.enyo.de","subject":"Re: git-svn: File added multiple times?","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2006-12-02T22:34:19Z","receivedAt":"2006-12-02T22:34:19Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Florian Weimer wrote:\n> Is this expected?\n> \n> $ sort /tmp/git-svn-output | uniq -c | sort -rn | head\n> 4 A mlton/trunk/doc/web/papers/index.html\n> 4 A mlton/trunk/doc/web/papers/01-icfp.ps.gz\n> 4 A mlton/trunk/doc/web/papers/00-esop.ps.gz\n> 4 A mlton/trunk/doc/examples/save-world/save-world.sml\n> 4 A mlton/trunk/doc/examples/save-world/Makefile\n> 4 A mlton/trunk/doc/examples/profiling/profiling.sml\n> 4 A mlton/trunk/doc/examples/profiling/Makefile\n> 4 A mlton/trunk/doc/examples/ffi/Makefile\n> 4 A mlton/trunk/doc/examples/ffi/main.sml\n> 4 A mlton/trunk/doc/examples/ffi/ffi.h\n> $ \n> \n> It's somewhat counter-intuitive, at least. This is with Debian's\n> git-core 1.4.4.1-1 package, and the SVN:: Perl modules are installed.\n\nNo it's not expected. Is this on a public SVN repo I can look at?\nThanks.\n\ngit-svn 1.4.4.1 always cat-ed the entire file (this code was stolen from\ngit-svnimport, the version in master can transfer deltas).\n\nThis is (or only seems to be) a UI reporting error and the actual data\nimported should be correct.\n\n-- \n"},{"id":"294938","messageId":"87u00dord9.fsf@mid.deneb.enyo.de","threadId":"43458","inReplyTo":"20061202223419.GA7057@localdomain","subject":"Re: git-svn: File added multiple times?","fromName":"Florian Weimer","fromEmail":"fw@deneb.enyo.de","sentAt":"2006-12-02T22:38:42Z","receivedAt":"2006-12-02T22:38:42Z","isPatch":false,"sender":{"key":"fw@deneb.enyo.de","avatar":null},"body":"* Eric Wong:\n\n>> Is this expected?\n>> It's somewhat counter-intuitive, at least. This is with Debian's\n>> git-core 1.4.4.1-1 package, and the SVN:: Perl modules are installed.\n>\n> No it's not expected. Is this on a public SVN repo I can look at?\n> Thanks.\n\nThis is the svn://mlton.org/mlton/trunk repository. The second commit\nshows this behavior, but it's a bit large.\n\n> This is (or only seems to be) a UI reporting error and the actual data\n> imported should be correct.\n\nI think it might download the data multiple times as well (at least\nthe timing suggests that). The generated repository seems to be fine,\nthough. The import is still running, so I haven't done any\n"},{"id":"294879","messageId":"87psb1or8j.fsf@mid.deneb.enyo.de","threadId":"43458","inReplyTo":"87u00dord9.fsf@mid.deneb.enyo.de","subject":"Re: git-svn: File added multiple times?","fromName":"Florian Weimer","fromEmail":"fw@deneb.enyo.de","sentAt":"2006-12-02T22:41:32Z","receivedAt":"2006-12-02T22:41:32Z","isPatch":false,"sender":{"key":"fw@deneb.enyo.de","avatar":null},"body":"* Florian Weimer:\n\n> * Eric Wong:\n>\n>>> Is this expected?\n>>> It's somewhat counter-intuitive, at least. This is with Debian's\n>>> git-core 1.4.4.1-1 package, and the SVN:: Perl modules are installed.\n>>\n>> No it's not expected. Is this on a public SVN repo I can look at?\n>> Thanks.\n>\n> This is the svn://mlton.org/mlton/trunk repository. The second commit\n> shows this behavior, but it's a bit large.\n\nIt also occurs with r2048, which is smaller:\n\n[...]\n A mlton/trunk/doc/examples/finalizable/cons.c\n A mlton/trunk/doc/examples/finalizable/cons.c\n[...]\n"},{"id":"297792","messageId":"20061203001930.GA468@soma","threadId":"43458","inReplyTo":"20061202223419.GA7057@localdomain","subject":"[PATCH] git-svn: avoid fetching files twice in the same revision","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2006-12-03T00:19:31Z","receivedAt":"2006-12-03T00:19:31Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"SVN is not entirely consistent in returning log information and\nsometimes returns file information when adding subdirectories,\nand sometimes it does not (only returning information about the\ndirectory that was added). This caused git-svn to occasionally\nadd a file to the list of files to be fetched twice. Now we\nchange the data structure to be hash to avoid repeated fetches.\n\nAs of now (in master), this only affects repositories fetched\nwithout deltas enabled (file://, and when manually overriden\nwith GIT_SVN_DELTA_FETCH=0); so this bug mainly affects users of\n1.4.4.1 and maint.\n\nThanks to Florian Weimer for reporting this bug.\n\nSigned-off-by: Eric Wong \n---\n git-svn.perl | 10 +++++-----\n 1 files changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/git-svn.perl b/git-svn.perl\nindex 3891122..d0bd0bd 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -2982,7 +2982,7 @@ sub libsvn_fetch_delta {\n sub libsvn_fetch_full {\n \tmy ($last_commit, $paths, $rev, $author, $date, $msg) = @_;\n \topen my $gui, '| git-update-index -z --index-info' or croak $!;\n-\tmy @amr;\n+\tmy %amr;\n \tmy $p = $SVN->{svn_path};\n \tforeach my $f (keys %$paths) {\n \t\tmy $m = $paths->{$f}->action();\n@@ -3001,7 +3001,7 @@ sub libsvn_fetch_full {\n \t\tmy $t = $SVN->check_path($f, $rev, $pool);\n \t\tif ($t == $SVN::Node::file) {\n \t\t\tif ($m =~ /^[AMR]$/) {\n-\t\t\t\tpush @amr, [ $m, $f ];\n+\t\t\t\t$amr{$f} = $m;\n \t\t\t} else {\n \t\t\t\tdie \"Unrecognized action: $m, ($f r$rev)\\n\";\n \t\t\t}\n@@ -3009,13 +3009,13 @@ sub libsvn_fetch_full {\n \t\t\tmy @traversed = ();\n \t\t\tlibsvn_traverse($gui, '', $f, $rev, \\@traversed);\n \t\t\tforeach (@traversed) {\n-\t\t\t\tpush @amr, [ $m, $_ ]\n+\t\t\t\t$amr{$_} = $m;\n \t\t\t}\n \t\t}\n \t\t$pool->clear;\n \t}\n-\tforeach (@amr) {\n-\t\tlibsvn_get_file($gui, $_->[1], $rev, $_->[0]);\n+\tforeach (keys %amr) {\n+\t\tlibsvn_get_file($gui, $_, $rev, $amr{$_});\n \t}\n \tclose $gui or croak $?;\n \treturn libsvn_log_entry($rev, $author, $date, $msg, [$last_commit]);\n-- \n1.4.4.1.gdf6b\n"},{"id":"298311","messageId":"87fybxgcdi.fsf@mid.deneb.enyo.de","threadId":"43458","inReplyTo":"20061203001930.GA468@soma","subject":"Re: [PATCH] git-svn: avoid fetching files twice in the same revision","fromName":"Florian Weimer","fromEmail":"fw@deneb.enyo.de","sentAt":"2006-12-03T16:42:01Z","receivedAt":"2006-12-03T16:42:01Z","isPatch":true,"sender":{"key":"fw@deneb.enyo.de","avatar":null},"body":"* Eric Wong:\n\n> +\tforeach (keys %amr) {\n> +\t\tlibsvn_get_file($gui, $_, $rev, $amr{$_});\n\nYou could throw in a \"sort\". Perhaps the improved locality helps the\nserver a bit.\n\n"}]}