{"thread":{"id":"4167","subject":"git-svn vs. $Id$","startedAt":"2006-05-16T17:33:03Z","lastAt":"2006-05-24T02:23:41Z","messageCount":8,"participants":["Tommi Virtanen","Linus Torvalds","Eric Wong"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"20072","messageId":"446A0CCF.2060903@inoi.fi","threadId":"4167","inReplyTo":null,"subject":"git-svn vs. $Id$","fromName":"Tommi Virtanen","fromEmail":"tv@inoi.fi","sentAt":"2006-05-16T17:33:03Z","receivedAt":"2006-05-16T17:33:03Z","isPatch":false,"sender":{"key":"tv@debian.org","avatar":null},"body":"Hi. I just ran into trouble with git-svn, related to a file\ncontaining $Id$. Yes, I know $Id$ sucks and should be avoided,\nand I'll be removing them shortly, but that doesn't change the\nfact that the history contains files with them.\n\nJust wanted to let you know of a workaround:\nmanually edit the relevant file in .git/git-svn/tree/ to\nundo the $Id$ change, and git-svn fetch works again.\n\n$ git-svn fetch\nTree mismatch, Got: c242bb60d78c1dfce133e0bbaca7f13895de00b2, Expected:\n07d35ac911cc56aabea86f4467cafc1d92b724c4\n at /home/tv/bin/git-svn line 426\n        main::assert_tree('a5890d459de08dc8adbbe34cdfb4b1f44f377ad8')\ncalled at /home/tv/bin/git-svn line 392\n        main::assert_svn_wc_clean(2039,\n'a5890d459de08dc8adbbe34cdfb4b1f44f377ad8') called at\n/home/tv/bin/git-svn line 262\n        main::fetch() called at /home/tv/bin/git-svn line 105\n\n$ git diff-tree -p 07d35ac911cc56aabea86f4467cafc1d92b724c4 \\\n  c242bb60d78c1dfce133e0bbaca7f13895de00b2\ndiff --git a/anonymized b/anonymized\nindex 16b3988..f43782a 100644\n--- a/anonymized\n+++ b/anonymized\n@@ -1,4 +1,4 @@\n-## $Id: anonymized 1775 2006-04-20 09:25:22Z tv $\n+## $Id: anonymized 2025 2006-05-16 07:25:24Z tv $\n\n blah\n blah\n\n\nSo editing .git/git-svn/tree/anonymized and replacing\n\"1775 2006-04-20 09:25:22Z tv\" with \"2025 2006-05-16 07:25:24Z tv\"\nmake git-svn happy again.\n\n-- \nInoi Oy, Tykistökatu 4 D (4. krs), FI-20520 Turku, Finland\nhttp://www.inoi.fi/\nMobile +358 40 762 5656\n"},{"id":"20073","messageId":"Pine.LNX.4.64.0605161037220.3866@g5.osdl.org","threadId":"4167","inReplyTo":"446A0CCF.2060903@inoi.fi","subject":"Re: git-svn vs. $Id$","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-05-16T17:48:03Z","receivedAt":"2006-05-16T17:48:03Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 16 May 2006, Tommi Virtanen wrote:\n> \n> Just wanted to let you know of a workaround:\n> manually edit the relevant file in .git/git-svn/tree/ to\n> undo the $Id$ change, and git-svn fetch works again.\n\nIsn't there some flag to svn to avoid keyword expansion, like \"-ko\" to \nCVS?\n\nAny import script definitely should avoid keyword expansion (and that's \ntrue whether you end up wanting to use keywords or not).\n\n(And yes, CVS is probably a bad example. Those \"substitution modes\" are \nconfusing as hell, and I don't know which one is the right one. Is it \n\"-ko\" or \"-kk\"? Don't ask me, I'm CVS-illiterate. I don't know why the \ncurrent cvsimport uses -kk, and only does it conditionally. Whatever.)\n\n\t\tLinus\n"},{"id":"20075","messageId":"446A15F8.3040106@inoi.fi","threadId":"4167","inReplyTo":"Pine.LNX.4.64.0605161037220.3866@g5.osdl.org","subject":"Re: git-svn vs. $Id$","fromName":"Tommi Virtanen","fromEmail":"tv@inoi.fi","sentAt":"2006-05-16T18:12:08Z","receivedAt":"2006-05-16T18:12:08Z","isPatch":false,"sender":{"key":"tv@debian.org","avatar":null},"body":"Linus Torvalds wrote:\n> Isn't there some flag to svn to avoid keyword expansion, like \"-ko\" to \n> CVS?\n> \n> Any import script definitely should avoid keyword expansion (and that's \n> true whether you end up wanting to use keywords or not).\n\nWell, yes, I agree. But, at least git-svn.txt says this:\n\nBUGS\n----\n...\nsvn:keywords can't be ignored in Subversion (at least I don't know of\na way to ignore them).\n\nI guess one might be able to reach that information through the svn API.\n\nOr just propget svn:keywords and sed s/\\$Id\\(:[^$]*\\)\\$/$Id$/ all files\nwith keywords, for all relevant keywords. Eww.\n\n-- \nInoi Oy, Tykistökatu 4 D (4. krs), FI-20520 Turku, Finland\nhttp://www.inoi.fi/\nMobile +358 40 762 5656\n"},{"id":"20361","messageId":"20060521011204.GA18886@hand.yhbt.net","threadId":"4167","inReplyTo":"446A15F8.3040106@inoi.fi","subject":"Re: git-svn vs. $Id$","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2006-05-21T01:12:06Z","receivedAt":"2006-05-21T01:12:06Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Tommi Virtanen <tv@inoi.fi> wrote:\n> Linus Torvalds wrote:\n> > Isn't there some flag to svn to avoid keyword expansion, like \"-ko\" to \n> > CVS?\n> > \n> > Any import script definitely should avoid keyword expansion (and that's \n> > true whether you end up wanting to use keywords or not).\n> \n> Well, yes, I agree. But, at least git-svn.txt says this:\n> \n> BUGS\n> ----\n> ...\n> svn:keywords can't be ignored in Subversion (at least I don't know of\n> a way to ignore them).\n> \n> I guess one might be able to reach that information through the svn API.\n> \n> Or just propget svn:keywords and sed s/\\$Id\\(:[^$]*\\)\\$/$Id$/ all files\n> with keywords, for all relevant keywords. Eww.\n\nI'm working on a solution to this (using files in .svn/text-base).\n\nkeyword expansion behavior seems inconsistent on some SVN repos and I\ncan't reproduce it on my local repositories, so I think I will have to\nweaken some integrity checks[1] in git-svn to work around it...\n\n1 - I don't think these integrity checks were ever tripped in the first\nplace.\n\n-- \nEric Wong\n"},{"id":"20372","messageId":"20060521030330.GA15812@soma","threadId":"4167","inReplyTo":"446A0CCF.2060903@inoi.fi","subject":"[PATCH] git-svn: ignore expansion of svn:keywords","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2006-05-21T03:03:30Z","receivedAt":"2006-05-21T03:03:30Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Can you try this patch?\n\nThis weakens an integrity in order to work on certain\nrepositories (see comments).  It's probably safe to use,\nthough...  More testing and feedback would be nice.\n\nI've split out the test for this feature to make things easier\nto manage and test, too.\n\nAlso changed assert_svn_wc_clean() to only work on svn, and\nrequire a separate call to assert_tree() to check wc integrity\nagainst git in preparation for another change I'm planning.\n\nSigned-off-by: Eric Wong <normalperson@yhbt.net>\n\n---\n\n @Junio: please don't apply this to git.git just yet, thanks.\n\n contrib/git-svn/Makefile                         |    1 \n contrib/git-svn/git-svn.perl                     |   79 +++++++++++++++++-----\n contrib/git-svn/t/lib-git-svn.sh                 |   39 +++++++++++\n contrib/git-svn/t/t0000-contrib-git-svn.sh       |   43 +-----------\n contrib/git-svn/t/t0001-contrib-git-svn-props.sh |   51 ++++++++++++++\n 5 files changed, 153 insertions(+), 60 deletions(-)\n create mode 100644 contrib/git-svn/t/lib-git-svn.sh\n create mode 100644 contrib/git-svn/t/t0001-contrib-git-svn-props.sh\n\neb8f17257c7d15ec6815baf18208af06f72b3cdb\ndiff --git a/contrib/git-svn/Makefile b/contrib/git-svn/Makefile\nindex acedf73..48f60b3 100644\n--- a/contrib/git-svn/Makefile\n+++ b/contrib/git-svn/Makefile\n@@ -30,6 +30,7 @@ git-svn.html : git-svn.txt\n \t\t-f ../../Documentation/asciidoc.conf $<\n test: git-svn\n \tcd t && $(SHELL) ./t0000-contrib-git-svn.sh\n+\tcd t && $(SHELL) ./t0001-contrib-git-svn-props.sh\n \n clean:\n \trm -f git-svn *.xml *.html *.1\ndiff --git a/contrib/git-svn/git-svn.perl b/contrib/git-svn/git-svn.perl\nindex de13a96..86b687a 100755\n--- a/contrib/git-svn/git-svn.perl\n+++ b/contrib/git-svn/git-svn.perl\n@@ -26,6 +26,7 @@ use Carp qw/croak/;\n use IO::File qw//;\n use File::Basename qw/dirname basename/;\n use File::Path qw/mkpath/;\n+use File::Copy qw/cp/;\n use Getopt::Long qw/:config gnu_getopt no_ignore_case auto_abbrev/;\n use File::Spec qw//;\n use POSIX qw/strftime/;\n@@ -207,7 +208,7 @@ sub rebuild {\n \t\tpush @svn_up, '--ignore-externals' unless $_no_ignore_ext;\n \t\tsys(@svn_up,\"-r$newest_rev\");\n \t\t$ENV{GIT_INDEX_FILE} = $GIT_SVN_INDEX;\n-\t\tgit_addremove();\n+\t\tindex_changes();\n \t\texec('git-write-tree');\n \t}\n \twaitpid $pid, 0;\n@@ -249,7 +250,7 @@ sub fetch {\n \t\tchdir $SVN_WC or croak $!;\n \t\tread_uuid();\n \t\t$last_commit = git_commit($base, @parents);\n-\t\tassert_svn_wc_clean($base->{revision}, $last_commit);\n+\t\tassert_tree($last_commit);\n \t} else {\n \t\tchdir $SVN_WC or croak $!;\n \t\tread_uuid();\n@@ -259,7 +260,11 @@ sub fetch {\n \tpush @svn_up, '--ignore-externals' unless $_no_ignore_ext;\n \tmy $last = $base;\n \twhile (my $log_msg = next_log_entry($svn_log)) {\n-\t\tassert_svn_wc_clean($last->{revision}, $last_commit);\n+\t\t# this assertion is commented out because it breaks keywords\n+\t\t# on https://svn.musicpd.org/Jamming/trunk (r166:167), but\n+\t\t# I can't seem to reproduce something like that on a test...\n+\t\t# assert_svn_wc_clean($last->{revision});\n+\t\tassert_tree($last_commit);\n \t\tif ($last->{revision} >= $log_msg->{revision}) {\n \t\t\tcroak \"Out of order: last >= current: \",\n \t\t\t\t\"$last->{revision} >= $log_msg->{revision}\\n\";\n@@ -268,7 +273,8 @@ sub fetch {\n \t\t$last_commit = git_commit($log_msg, $last_commit, @parents);\n \t\t$last = $log_msg;\n \t}\n-\tassert_svn_wc_clean($last->{revision}, $last_commit);\n+\tassert_svn_wc_clean($last->{revision});\n+\tassert_tree($last_commit);\n \tunless (-e \"$GIT_DIR/refs/heads/master\") {\n \t\tsys(qw(git-update-ref refs/heads/master),$last_commit);\n \t}\n@@ -314,7 +320,6 @@ sub commit {\n \t\t$svn_current_rev = svn_commit_tree($svn_current_rev, $c);\n \t}\n \tprint \"Done committing \",scalar @revs,\" revisions to SVN\\n\";\n-\n }\n \n sub show_ignore {\n@@ -367,13 +372,11 @@ sub setup_git_svn {\n }\n \n sub assert_svn_wc_clean {\n-\tmy ($svn_rev, $treeish) = @_;\n+\tmy ($svn_rev) = @_;\n \tcroak \"$svn_rev is not an integer!\\n\" unless ($svn_rev =~ /^\\d+$/);\n-\tcroak \"$treeish is not a sha1!\\n\" unless ($treeish =~ /^$sha1$/o);\n \tmy $lcr = svn_info('.')->{'Last Changed Rev'};\n \tif ($svn_rev != $lcr) {\n \t\tprint STDERR \"Checking for copy-tree ... \";\n-\t\t# use\n \t\tmy @diff = grep(/^Index: /,(safe_qx(qw(svn diff),\n \t\t\t\t\t\t\"-r$lcr:$svn_rev\")));\n \t\tif (@diff) {\n@@ -389,7 +392,6 @@ sub assert_svn_wc_clean {\n \t\tprint STDERR $_ foreach @status;\n \t\tcroak;\n \t}\n-\tassert_tree($treeish);\n }\n \n sub assert_tree {\n@@ -416,7 +418,7 @@ sub assert_tree {\n \t\tunlink $tmpindex or croak $!;\n \t}\n \t$ENV{GIT_INDEX_FILE} = $tmpindex;\n-\tgit_addremove();\n+\tindex_changes(1);\n \tchomp(my $tree = `git-write-tree`);\n \tif ($old_index) {\n \t\t$ENV{GIT_INDEX_FILE} = $old_index;\n@@ -426,6 +428,7 @@ sub assert_tree {\n \tif ($tree ne $expected) {\n \t\tcroak \"Tree mismatch, Got: $tree, Expected: $expected\\n\";\n \t}\n+\tunlink $tmpindex;\n }\n \n sub parse_diff_tree {\n@@ -562,7 +565,8 @@ sub precommit_check {\n sub svn_checkout_tree {\n \tmy ($svn_rev, $treeish) = @_;\n \tmy $from = file_to_s(\"$REV_DIR/$svn_rev\");\n-\tassert_svn_wc_clean($svn_rev,$from);\n+\tassert_svn_wc_clean($svn_rev);\n+\tassert_tree($from);\n \tprint \"diff-tree $from $treeish\\n\";\n \tmy $pid = open my $diff_fh, '-|';\n \tdefined $pid or croak $!;\n@@ -852,13 +856,50 @@ sub svn_info {\n \n sub sys { system(@_) == 0 or croak $? }\n \n-sub git_addremove {\n-\tsystem( \"git-diff-files --name-only -z \".\n-\t\t\t\t\" | git-update-index --remove -z --stdin && \".\n-\t\t\"git-ls-files -z --others \".\n-\t\t\t\"'--exclude-from=$GIT_DIR/$GIT_SVN/info/exclude'\".\n-\t\t\t\t\" | git-update-index --add -z --stdin\"\n-\t\t) == 0 or croak $?\n+sub do_update_index {\n+\tmy ($z_cmd, $cmd, $no_text_base) = @_;\n+\n+\tmy $z = open my $p, '-|';\n+\tdefined $z or croak $!;\n+\tunless ($z) { exec @$z_cmd or croak $! }\n+\n+\tmy $pid = open my $ui, '|-';\n+\tdefined $pid or croak $!;\n+\tunless ($pid) {\n+\t\texec('git-update-index',\"--$cmd\",'-z','--stdin') or croak $!;\n+\t}\n+\tlocal $/ = \"\\0\";\n+\twhile (my $x = <$p>) {\n+\t\tchomp $x;\n+\t\tif (!$no_text_base && lstat $x && ! -l _) {\n+\t\t\tmy $mode = -x _ ? 0755 : 0644;\n+\t\t\tmy ($v,$d,$f) = File::Spec->splitpath($x);\n+\t\t\tmy $tb = File::Spec->catfile($d, '.svn', 'tmp',\n+\t\t\t\t\t\t'text-base',\"$f.svn-base\");\n+\t\t\t$tb =~ s#^/##;\n+\t\t\tunless (-f $tb) {\n+\t\t\t\t$tb = File::Spec->catfile($d, '.svn',\n+\t\t\t\t\t\t'text-base',\"$f.svn-base\");\n+\t\t\t\t$tb =~ s#^/##;\n+\t\t\t}\n+\t\t\tunlink $x or croak $!;\n+\t\t\tcp($tb, $x) or croak $!;\n+\t\t\tchmod(($mode &~ umask), $x) or croak $!;\n+\t\t}\n+\t\tprint $ui $x,\"\\0\";\n+\t}\n+\tclose $ui or croak $!;\n+}\n+\n+sub index_changes {\n+\tmy $no_text_base = shift;\n+\tdo_update_index([qw/git-diff-files --name-only -z/],\n+\t\t\t'remove',\n+\t\t\t$no_text_base);\n+\tdo_update_index([qw/git-ls-files -z --others/,\n+\t\t\t      \"--exclude-from=$GIT_DIR/$GIT_SVN/info/exclude\"],\n+\t\t\t'add',\n+\t\t\t$no_text_base);\n }\n \n sub s_to_file {\n@@ -936,7 +977,7 @@ sub git_commit {\n \tdefined $pid or croak $!;\n \tif ($pid == 0) {\n \t\t$ENV{GIT_INDEX_FILE} = $GIT_SVN_INDEX;\n-\t\tgit_addremove();\n+\t\tindex_changes();\n \t\tchomp(my $tree = `git-write-tree`);\n \t\tcroak if $?;\n \t\tif (exists $tree_map{$tree}) {\ndiff --git a/contrib/git-svn/t/lib-git-svn.sh b/contrib/git-svn/t/lib-git-svn.sh\nnew file mode 100644\nindex 0000000..a98e9d1\n--- /dev/null\n+++ b/contrib/git-svn/t/lib-git-svn.sh\n@@ -0,0 +1,39 @@\n+PATH=$PWD/../:$PATH\n+if test -d ../../../t\n+then\n+    cd ../../../t\n+else\n+    echo \"Must be run in contrib/git-svn/t\" >&2\n+    exit 1\n+fi\n+\n+. ./test-lib.sh\n+\n+GIT_DIR=$PWD/.git\n+GIT_SVN_DIR=$GIT_DIR/git-svn\n+SVN_TREE=$GIT_SVN_DIR/tree\n+\n+svnadmin >/dev/null 2>&1\n+if test $? != 1\n+then\n+    test_expect_success 'skipping contrib/git-svn test' :\n+    test_done\n+    exit\n+fi\n+\n+svn >/dev/null 2>&1\n+if test $? != 1\n+then\n+    test_expect_success 'skipping contrib/git-svn test' :\n+    test_done\n+    exit\n+fi\n+\n+svnrepo=$PWD/svnrepo\n+\n+set -e\n+\n+svnadmin create $svnrepo\n+svnrepo=\"file://$svnrepo/test-git-svn\"\n+\n+\ndiff --git a/contrib/git-svn/t/t0000-contrib-git-svn.sh b/contrib/git-svn/t/t0000-contrib-git-svn.sh\nindex 80ad357..8b3a0d9 100644\n--- a/contrib/git-svn/t/t0000-contrib-git-svn.sh\n+++ b/contrib/git-svn/t/t0000-contrib-git-svn.sh\n@@ -3,48 +3,10 @@ #\n # Copyright (c) 2006 Eric Wong\n #\n \n-\n-PATH=$PWD/../:$PATH\n test_description='git-svn tests'\n-if test -d ../../../t\n-then\n-    cd ../../../t\n-else\n-    echo \"Must be run in contrib/git-svn/t\" >&2\n-    exit 1\n-fi\n-\n-. ./test-lib.sh\n-\n-GIT_DIR=$PWD/.git\n-GIT_SVN_DIR=$GIT_DIR/git-svn\n-SVN_TREE=$GIT_SVN_DIR/tree\n-\n-svnadmin >/dev/null 2>&1\n-if test $? != 1\n-then\n-    test_expect_success 'skipping contrib/git-svn test' :\n-    test_done\n-    exit\n-fi\n-\n-svn >/dev/null 2>&1\n-if test $? != 1\n-then\n-    test_expect_success 'skipping contrib/git-svn test' :\n-    test_done\n-    exit\n-fi\n-\n-svnrepo=$PWD/svnrepo\n-\n-set -e\n-\n-svnadmin create $svnrepo\n-svnrepo=\"file://$svnrepo/test-git-svn\"\n+. ./lib-git-svn.sh\n \n mkdir import\n-\n cd import\n \n echo foo > foo\n@@ -55,10 +17,9 @@ mkdir -p bar\n echo 'zzz' > bar/zzz\n echo '#!/bin/sh' > exec.sh\n chmod +x exec.sh\n-svn import -m 'import for git-svn' . $svnrepo >/dev/null\n+svn import -m 'import for git-svn' . \"$svnrepo\" >/dev/null\n \n cd ..\n-\n rm -rf import\n \n test_expect_success \\\ndiff --git a/contrib/git-svn/t/t0001-contrib-git-svn-props.sh b/contrib/git-svn/t/t0001-contrib-git-svn-props.sh\nnew file mode 100644\nindex 0000000..20c5c4e\n--- /dev/null\n+++ b/contrib/git-svn/t/t0001-contrib-git-svn-props.sh\n@@ -0,0 +1,51 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2006 Eric Wong\n+#\n+\n+test_description='git-svn property tests'\n+. ./lib-git-svn.sh\n+\n+mkdir import\n+\n+cd import\n+\tcat >> kw.c <<''\n+/* Make it look like somebody copied a file from CVS into SVN: */\n+/* $Id: kw.c,v 1.1.1.1 1994/03/06 00:00:00 eric Exp $ */\n+\n+\tsvn import -m 'import for git-svn' . \"$svnrepo\" >/dev/null\n+cd ..\n+\n+rm -rf import\n+svn co \"$svnrepo\" test_wc\n+\n+cd test_wc\n+\techo 'Greetings' >> kw.c\n+\tsvn commit -m 'Not yet an $Id$'\n+\tsvn up\n+\n+\techo 'Hello world' >> kw.c\n+\tsvn commit -m 'Modified file, but still not yet an $Id$'\n+\tsvn up\n+\n+\tsvn propset svn:keywords Id kw.c\n+\tsvn commit -m 'Propset $Id$'\n+\tsvn up\n+cd ..\n+\n+git-svn init \"$svnrepo\"\n+git-svn fetch\n+\n+git checkout -b mybranch remotes/git-svn\n+echo 'Hi again' >> kw.c\n+name='test svn:keywords ignoring'\n+\n+git commit -a -m \"$name\"\n+git-svn commit remotes/git-svn..mybranch\n+git pull . remotes/git-svn\n+\n+expect='/* $Id$ */'\n+got=\"`sed -ne 2p kw.c`\"\n+test_expect_success 'raw $Id$ found in kw.c' \"test '$expect' = '$got'\"\n+\n+test_done\n-- \n1.3.2.g7d11\n"},{"id":"20602","messageId":"11484374214073-git-send-email-normalperson@yhbt.net","threadId":"4167","inReplyTo":"20060521030330.GA15812@soma","subject":"Re: git-svn vs. $Id$","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2006-05-24T02:23:39Z","receivedAt":"2006-05-24T02:23:39Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Tommi: I'm reasonably confident about this patch, but I won't be\ngetting much real-world usage since I rarely see repos that use\nsvn:keywords, and I'm afflicted with RSI which prevents me from\ndoing very much in front of a computer these days.  So\nfeedback/testing from you or any other git-svn guine^Wusers\nwould be greatly appreciated.\n\nJunio: This is probably safe enough for next or even master, the\none in pu is completely broken wrt svn:eol-style.\n"},{"id":"20603","messageId":"11484374222795-git-send-email-normalperson@yhbt.net","threadId":"4167","inReplyTo":"11484374214073-git-send-email-normalperson@yhbt.net","subject":"[PATCH] git-svn: starting a 1.1.0-pre development version","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2006-05-24T02:23:40Z","receivedAt":"2006-05-24T02:23:40Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Some not-very-well-tested changes coming...\n\nSigned-off-by: Eric Wong <normalperson@yhbt.net>\n\n---\n\n contrib/git-svn/git-svn.perl |    2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\n49967a4b7bc2c9938b0ebf0670cd47d4ccfe85a3\ndiff --git a/contrib/git-svn/git-svn.perl b/contrib/git-svn/git-svn.perl\nindex de13a96..39471b5 100755\n--- a/contrib/git-svn/git-svn.perl\n+++ b/contrib/git-svn/git-svn.perl\n@@ -8,7 +8,7 @@ use vars qw/\t$AUTHOR $VERSION\n \t\t$GIT_SVN_INDEX $GIT_SVN\n \t\t$GIT_DIR $REV_DIR/;\n $AUTHOR = 'Eric Wong <normalperson@yhbt.net>';\n-$VERSION = '1.0.0';\n+$VERSION = '1.1.0-pre';\n \n use Cwd qw/abs_path/;\n $GIT_DIR = abs_path($ENV{GIT_DIR} || '.git');\n-- \n1.3.2.g7d11\n"},{"id":"20604","messageId":"11484374243815-git-send-email-normalperson@yhbt.net","threadId":"4167","inReplyTo":"11484374222795-git-send-email-normalperson@yhbt.net","subject":"[PATCH] git-svn: ignore expansion of svn:keywords","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2006-05-24T02:23:41Z","receivedAt":"2006-05-24T02:23:41Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Unlike my earlier test patch, this also checks svn:eol-style and\nmakes sure it's applied to working copy updates.  This is\ndefinitely more correct than my original attempt at killing\nkeyword expansions, but I still haven't tested it enough to\nknow.  Feedback would be much appreciated.\n\nAlso changed assert_svn_wc_clean() to only work on the svn\nworking copy.  This requires a separate call to assert_tree() to\ncheck wc integrity against git in preparation for another change\nI'm planning.\n\nSigned-off-by: Eric Wong <normalperson@yhbt.net>\n\n---\n\n contrib/git-svn/Makefile                         |    1 \n contrib/git-svn/git-svn.perl                     |  106 +++++++++++++++----\n contrib/git-svn/t/lib-git-svn.sh                 |   39 +++++++\n contrib/git-svn/t/t0000-contrib-git-svn.sh       |   43 --------\n contrib/git-svn/t/t0001-contrib-git-svn-props.sh |  125 ++++++++++++++++++++++\n 5 files changed, 254 insertions(+), 60 deletions(-)\n create mode 100644 contrib/git-svn/t/lib-git-svn.sh\n create mode 100644 contrib/git-svn/t/t0001-contrib-git-svn-props.sh\n\nb9ba79157fc4f92e14aac89d9a5f838539d6eede\ndiff --git a/contrib/git-svn/Makefile b/contrib/git-svn/Makefile\nindex acedf73..48f60b3 100644\n--- a/contrib/git-svn/Makefile\n+++ b/contrib/git-svn/Makefile\n@@ -30,6 +30,7 @@ git-svn.html : git-svn.txt\n \t\t-f ../../Documentation/asciidoc.conf $<\n test: git-svn\n \tcd t && $(SHELL) ./t0000-contrib-git-svn.sh\n+\tcd t && $(SHELL) ./t0001-contrib-git-svn-props.sh\n \n clean:\n \trm -f git-svn *.xml *.html *.1\ndiff --git a/contrib/git-svn/git-svn.perl b/contrib/git-svn/git-svn.perl\nindex 39471b5..fc9b7ef 100755\n--- a/contrib/git-svn/git-svn.perl\n+++ b/contrib/git-svn/git-svn.perl\n@@ -39,6 +39,10 @@ my $_svn_co_url_revs;\n my %fc_opts = ( 'no-ignore-externals' => \\$_no_ignore_ext,\n \t\t'branch|b=s' => \\@_branch_from,\n \t\t'authors-file|A=s' => \\$_authors );\n+\n+# yes, 'native' sets \"\\n\".  Patches to fix this for non-*nix systems welcome:\n+my %EOL = ( CR => \"\\015\", LF => \"\\012\", CRLF => \"\\015\\012\", native => \"\\012\" );\n+\n my %cmd = (\n \tfetch => [ \\&fetch, \"Download new revisions from SVN\",\n \t\t\t{ 'revision|r=s' => \\$_revision, %fc_opts } ],\n@@ -207,7 +211,7 @@ sub rebuild {\n \t\tpush @svn_up, '--ignore-externals' unless $_no_ignore_ext;\n \t\tsys(@svn_up,\"-r$newest_rev\");\n \t\t$ENV{GIT_INDEX_FILE} = $GIT_SVN_INDEX;\n-\t\tgit_addremove();\n+\t\tindex_changes();\n \t\texec('git-write-tree');\n \t}\n \twaitpid $pid, 0;\n@@ -249,7 +253,7 @@ sub fetch {\n \t\tchdir $SVN_WC or croak $!;\n \t\tread_uuid();\n \t\t$last_commit = git_commit($base, @parents);\n-\t\tassert_svn_wc_clean($base->{revision}, $last_commit);\n+\t\tassert_tree($last_commit);\n \t} else {\n \t\tchdir $SVN_WC or croak $!;\n \t\tread_uuid();\n@@ -259,16 +263,20 @@ sub fetch {\n \tpush @svn_up, '--ignore-externals' unless $_no_ignore_ext;\n \tmy $last = $base;\n \twhile (my $log_msg = next_log_entry($svn_log)) {\n-\t\tassert_svn_wc_clean($last->{revision}, $last_commit);\n+\t\tassert_tree($last_commit);\n \t\tif ($last->{revision} >= $log_msg->{revision}) {\n \t\t\tcroak \"Out of order: last >= current: \",\n \t\t\t\t\"$last->{revision} >= $log_msg->{revision}\\n\";\n \t\t}\n+\t\t# Revert is needed for cases like:\n+\t\t# https://svn.musicpd.org/Jamming/trunk (r166:167), but\n+\t\t# I can't seem to reproduce something like that on a test...\n+\t\tsys(qw/svn revert -R ./);\n+\t\tassert_svn_wc_clean($last->{revision});\n \t\tsys(@svn_up,\"-r$log_msg->{revision}\");\n \t\t$last_commit = git_commit($log_msg, $last_commit, @parents);\n \t\t$last = $log_msg;\n \t}\n-\tassert_svn_wc_clean($last->{revision}, $last_commit);\n \tunless (-e \"$GIT_DIR/refs/heads/master\") {\n \t\tsys(qw(git-update-ref refs/heads/master),$last_commit);\n \t}\n@@ -314,7 +322,6 @@ sub commit {\n \t\t$svn_current_rev = svn_commit_tree($svn_current_rev, $c);\n \t}\n \tprint \"Done committing \",scalar @revs,\" revisions to SVN\\n\";\n-\n }\n \n sub show_ignore {\n@@ -367,13 +374,11 @@ sub setup_git_svn {\n }\n \n sub assert_svn_wc_clean {\n-\tmy ($svn_rev, $treeish) = @_;\n+\tmy ($svn_rev) = @_;\n \tcroak \"$svn_rev is not an integer!\\n\" unless ($svn_rev =~ /^\\d+$/);\n-\tcroak \"$treeish is not a sha1!\\n\" unless ($treeish =~ /^$sha1$/o);\n \tmy $lcr = svn_info('.')->{'Last Changed Rev'};\n \tif ($svn_rev != $lcr) {\n \t\tprint STDERR \"Checking for copy-tree ... \";\n-\t\t# use\n \t\tmy @diff = grep(/^Index: /,(safe_qx(qw(svn diff),\n \t\t\t\t\t\t\"-r$lcr:$svn_rev\")));\n \t\tif (@diff) {\n@@ -389,7 +394,6 @@ sub assert_svn_wc_clean {\n \t\tprint STDERR $_ foreach @status;\n \t\tcroak;\n \t}\n-\tassert_tree($treeish);\n }\n \n sub assert_tree {\n@@ -416,7 +420,7 @@ sub assert_tree {\n \t\tunlink $tmpindex or croak $!;\n \t}\n \t$ENV{GIT_INDEX_FILE} = $tmpindex;\n-\tgit_addremove();\n+\tindex_changes(1);\n \tchomp(my $tree = `git-write-tree`);\n \tif ($old_index) {\n \t\t$ENV{GIT_INDEX_FILE} = $old_index;\n@@ -426,6 +430,7 @@ sub assert_tree {\n \tif ($tree ne $expected) {\n \t\tcroak \"Tree mismatch, Got: $tree, Expected: $expected\\n\";\n \t}\n+\tunlink $tmpindex;\n }\n \n sub parse_diff_tree {\n@@ -562,7 +567,8 @@ sub precommit_check {\n sub svn_checkout_tree {\n \tmy ($svn_rev, $treeish) = @_;\n \tmy $from = file_to_s(\"$REV_DIR/$svn_rev\");\n-\tassert_svn_wc_clean($svn_rev,$from);\n+\tassert_svn_wc_clean($svn_rev);\n+\tassert_tree($from);\n \tprint \"diff-tree $from $treeish\\n\";\n \tmy $pid = open my $diff_fh, '-|';\n \tdefined $pid or croak $!;\n@@ -852,13 +858,75 @@ sub svn_info {\n \n sub sys { system(@_) == 0 or croak $? }\n \n-sub git_addremove {\n-\tsystem( \"git-diff-files --name-only -z \".\n-\t\t\t\t\" | git-update-index --remove -z --stdin && \".\n-\t\t\"git-ls-files -z --others \".\n-\t\t\t\"'--exclude-from=$GIT_DIR/$GIT_SVN/info/exclude'\".\n-\t\t\t\t\" | git-update-index --add -z --stdin\"\n-\t\t) == 0 or croak $?\n+sub eol_cp {\n+\tmy ($from, $to) = @_;\n+\tmy $es = safe_qx(qw/svn propget svn:eol-style/, $to);\n+\topen my $rfd, '<', $from or croak $!;\n+\tbinmode $rfd or croak $!;\n+\topen my $wfd, '>', $to or croak $!;\n+\tbinmode $wfd or croak $!;\n+\n+\tmy $eol = $EOL{$es} or undef;\n+\tif ($eol) {\n+\t\tprint  \"$eol: $from => $to\\n\";\n+\t}\n+\tmy $buf;\n+\twhile (1) {\n+\t\tmy ($r, $w, $t);\n+\t\tdefined($r = sysread($rfd, $buf, 4096)) or croak $!;\n+\t\treturn unless $r;\n+\t\t$buf =~ s/(?:\\015|\\012|\\015\\012)/$eol/gs if $eol;\n+\t\tfor ($w = 0; $w < $r; $w += $t) {\n+\t\t\t$t = syswrite($wfd, $buf, $r - $w, $w) or croak $!;\n+\t\t}\n+\t}\n+}\n+\n+sub do_update_index {\n+\tmy ($z_cmd, $cmd, $no_text_base) = @_;\n+\n+\tmy $z = open my $p, '-|';\n+\tdefined $z or croak $!;\n+\tunless ($z) { exec @$z_cmd or croak $! }\n+\n+\tmy $pid = open my $ui, '|-';\n+\tdefined $pid or croak $!;\n+\tunless ($pid) {\n+\t\texec('git-update-index',\"--$cmd\",'-z','--stdin') or croak $!;\n+\t}\n+\tlocal $/ = \"\\0\";\n+\twhile (my $x = <$p>) {\n+\t\tchomp $x;\n+\t\tif (!$no_text_base && lstat $x && ! -l _ &&\n+\t\t\t\tsafe_qx(qw/svn propget svn:keywords/,$x)) {\n+\t\t\tmy $mode = -x _ ? 0755 : 0644;\n+\t\t\tmy ($v,$d,$f) = File::Spec->splitpath($x);\n+\t\t\tmy $tb = File::Spec->catfile($d, '.svn', 'tmp',\n+\t\t\t\t\t\t'text-base',\"$f.svn-base\");\n+\t\t\t$tb =~ s#^/##;\n+\t\t\tunless (-f $tb) {\n+\t\t\t\t$tb = File::Spec->catfile($d, '.svn',\n+\t\t\t\t\t\t'text-base',\"$f.svn-base\");\n+\t\t\t\t$tb =~ s#^/##;\n+\t\t\t}\n+\t\t\tunlink $x or croak $!;\n+\t\t\teol_cp($tb, $x);\n+\t\t\tchmod(($mode &~ umask), $x) or croak $!;\n+\t\t}\n+\t\tprint $ui $x,\"\\0\";\n+\t}\n+\tclose $ui or croak $!;\n+}\n+\n+sub index_changes {\n+\tmy $no_text_base = shift;\n+\tdo_update_index([qw/git-diff-files --name-only -z/],\n+\t\t\t'remove',\n+\t\t\t$no_text_base);\n+\tdo_update_index([qw/git-ls-files -z --others/,\n+\t\t\t      \"--exclude-from=$GIT_DIR/$GIT_SVN/info/exclude\"],\n+\t\t\t'add',\n+\t\t\t$no_text_base);\n }\n \n sub s_to_file {\n@@ -936,7 +1004,7 @@ sub git_commit {\n \tdefined $pid or croak $!;\n \tif ($pid == 0) {\n \t\t$ENV{GIT_INDEX_FILE} = $GIT_SVN_INDEX;\n-\t\tgit_addremove();\n+\t\tindex_changes();\n \t\tchomp(my $tree = `git-write-tree`);\n \t\tcroak if $?;\n \t\tif (exists $tree_map{$tree}) {\ndiff --git a/contrib/git-svn/t/lib-git-svn.sh b/contrib/git-svn/t/lib-git-svn.sh\nnew file mode 100644\nindex 0000000..a98e9d1\n--- /dev/null\n+++ b/contrib/git-svn/t/lib-git-svn.sh\n@@ -0,0 +1,39 @@\n+PATH=$PWD/../:$PATH\n+if test -d ../../../t\n+then\n+    cd ../../../t\n+else\n+    echo \"Must be run in contrib/git-svn/t\" >&2\n+    exit 1\n+fi\n+\n+. ./test-lib.sh\n+\n+GIT_DIR=$PWD/.git\n+GIT_SVN_DIR=$GIT_DIR/git-svn\n+SVN_TREE=$GIT_SVN_DIR/tree\n+\n+svnadmin >/dev/null 2>&1\n+if test $? != 1\n+then\n+    test_expect_success 'skipping contrib/git-svn test' :\n+    test_done\n+    exit\n+fi\n+\n+svn >/dev/null 2>&1\n+if test $? != 1\n+then\n+    test_expect_success 'skipping contrib/git-svn test' :\n+    test_done\n+    exit\n+fi\n+\n+svnrepo=$PWD/svnrepo\n+\n+set -e\n+\n+svnadmin create $svnrepo\n+svnrepo=\"file://$svnrepo/test-git-svn\"\n+\n+\ndiff --git a/contrib/git-svn/t/t0000-contrib-git-svn.sh b/contrib/git-svn/t/t0000-contrib-git-svn.sh\nindex f400202..a07fbad 100644\n--- a/contrib/git-svn/t/t0000-contrib-git-svn.sh\n+++ b/contrib/git-svn/t/t0000-contrib-git-svn.sh\n@@ -3,48 +3,10 @@ #\n # Copyright (c) 2006 Eric Wong\n #\n \n-\n-PATH=$PWD/../:$PATH\n test_description='git-svn tests'\n-if test -d ../../../t\n-then\n-    cd ../../../t\n-else\n-    echo \"Must be run in contrib/git-svn/t\" >&2\n-    exit 1\n-fi\n-\n-. ./test-lib.sh\n-\n-GIT_DIR=$PWD/.git\n-GIT_SVN_DIR=$GIT_DIR/git-svn\n-SVN_TREE=$GIT_SVN_DIR/tree\n-\n-svnadmin >/dev/null 2>&1\n-if test $? != 1\n-then\n-    test_expect_success 'skipping contrib/git-svn test' :\n-    test_done\n-    exit\n-fi\n-\n-svn >/dev/null 2>&1\n-if test $? != 1\n-then\n-    test_expect_success 'skipping contrib/git-svn test' :\n-    test_done\n-    exit\n-fi\n-\n-svnrepo=$PWD/svnrepo\n-\n-set -e\n-\n-svnadmin create $svnrepo\n-svnrepo=\"file://$svnrepo/test-git-svn\"\n+. ./lib-git-svn.sh\n \n mkdir import\n-\n cd import\n \n echo foo > foo\n@@ -55,10 +17,9 @@ mkdir -p bar\n echo 'zzz' > bar/zzz\n echo '#!/bin/sh' > exec.sh\n chmod +x exec.sh\n-svn import -m 'import for git-svn' . $svnrepo >/dev/null\n+svn import -m 'import for git-svn' . \"$svnrepo\" >/dev/null\n \n cd ..\n-\n rm -rf import\n \n test_expect_success \\\ndiff --git a/contrib/git-svn/t/t0001-contrib-git-svn-props.sh b/contrib/git-svn/t/t0001-contrib-git-svn-props.sh\nnew file mode 100644\nindex 0000000..6fa7889\n--- /dev/null\n+++ b/contrib/git-svn/t/t0001-contrib-git-svn-props.sh\n@@ -0,0 +1,125 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2006 Eric Wong\n+#\n+\n+test_description='git-svn property tests'\n+. ./lib-git-svn.sh\n+\n+mkdir import\n+\n+a_crlf=\n+a_lf=\n+a_cr=\n+a_ne_crlf=\n+a_ne_lf=\n+a_ne_cr=\n+a_empty=\n+a_empty_lf=\n+a_empty_cr=\n+a_empty_crlf=\n+\n+cd import\n+\tcat >> kw.c <<''\n+/* Make it look like somebody copied a file from CVS into SVN: */\n+/* $Id: kw.c,v 1.1.1.1 1994/03/06 00:00:00 eric Exp $ */\n+\n+\tprintf \"Hello\\r\\nWorld\\r\\n\" > crlf\n+\ta_crlf=`git-hash-object -w crlf`\n+\tprintf \"Hello\\rWorld\\r\" > cr\n+\ta_cr=`git-hash-object -w cr`\n+\tprintf \"Hello\\nWorld\\n\" > lf\n+\ta_lf=`git-hash-object -w lf`\n+\n+\tprintf \"Hello\\r\\nWorld\" > ne_crlf\n+\ta_ne_crlf=`git-hash-object -w ne_crlf`\n+\tprintf \"Hello\\nWorld\" > ne_lf\n+\ta_ne_lf=`git-hash-object -w ne_lf`\n+\tprintf \"Hello\\rWorld\" > ne_cr\n+\ta_ne_cr=`git-hash-object -w ne_cr`\n+\n+\ttouch empty\n+\ta_empty=`git-hash-object -w empty`\n+\tprintf \"\\n\" > empty_lf\n+\ta_empty_lf=`git-hash-object -w empty_lf`\n+\tprintf \"\\r\" > empty_cr\n+\ta_empty_cr=`git-hash-object -w empty_cr`\n+\tprintf \"\\r\\n\" > empty_crlf\n+\ta_empty_crlf=`git-hash-object -w empty_crlf`\n+\n+\tsvn import -m 'import for git-svn' . \"$svnrepo\" >/dev/null\n+cd ..\n+\n+rm -rf import\n+svn co \"$svnrepo\" test_wc\n+\n+cd test_wc\n+\techo 'Greetings' >> kw.c\n+\tsvn commit -m 'Not yet an $Id$'\n+\tsvn up\n+\n+\techo 'Hello world' >> kw.c\n+\tsvn commit -m 'Modified file, but still not yet an $Id$'\n+\tsvn up\n+\n+\tsvn propset svn:keywords Id kw.c\n+\tsvn commit -m 'Propset $Id$'\n+\tsvn up\n+cd ..\n+\n+git-svn init \"$svnrepo\"\n+git-svn fetch\n+\n+git checkout -b mybranch remotes/git-svn\n+echo 'Hi again' >> kw.c\n+name='test svn:keywords ignoring'\n+\n+git commit -a -m \"$name\"\n+git-svn commit remotes/git-svn..mybranch\n+git pull . remotes/git-svn\n+\n+expect='/* $Id$ */'\n+got=\"`sed -ne 2p kw.c`\"\n+test_expect_success 'raw $Id$ found in kw.c' \"test '$expect' = '$got'\"\n+\n+cd test_wc\n+\tsvn propset svn:eol-style CR empty\n+\tsvn propset svn:eol-style CR crlf\n+\tsvn propset svn:eol-style CR ne_crlf\n+\tsvn commit -m 'propset CR on crlf files'\n+\tsvn up\n+cd ..\n+\n+git-svn fetch\n+git pull . remotes/git-svn\n+\n+svn co \"$svnrepo\" new_wc\n+for i in crlf ne_crlf lf ne_lf cr ne_cr empty_cr empty_lf empty empty_crlf\n+do\n+\ttest_expect_success \"Comparing $i\" \"cmp $i new_wc/$i\"\n+done\n+\n+\n+cd test_wc\n+\tprintf '$Id$\\rHello\\rWorld\\r' > cr\n+\tprintf '$Id$\\rHello\\rWorld' > ne_cr\n+\ta_cr=`printf '$Id$\\r\\nHello\\r\\nWorld\\r\\n' | git-hash-object --stdin`\n+\ta_ne_cr=`printf '$Id$\\r\\nHello\\r\\nWorld' | git-hash-object --stdin`\n+\tsvn propset svn:eol-style CRLF cr\n+\tsvn propset svn:eol-style CRLF ne_cr\n+\tsvn propset svn:keywords Id cr\n+\tsvn propset svn:keywords Id ne_cr\n+\tsvn commit -m 'propset CRLF on cr files'\n+\tsvn up\n+cd ..\n+\n+git-svn fetch\n+git pull . remotes/git-svn\n+\n+b_cr=\"`git-hash-object cr`\"\n+b_ne_cr=\"`git-hash-object ne_cr`\"\n+\n+test_expect_success 'CRLF + $Id$' \"test '$a_cr' = '$b_cr'\"\n+test_expect_success 'CRLF + $Id$ (no newline)' \"test '$a_ne_cr' = '$b_ne_cr'\"\n+\n+test_done\n-- \n1.3.2.g7d11\n"}]}