{"thread":{"id":"21977","subject":"git svn mkdirs ignores compressed unhandled.log files","startedAt":"2009-12-17T17:10:03Z","lastAt":"2009-12-23T04:12:02Z","messageCount":7,"participants":["Robert Zeh","Eric Wong","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"130037","messageId":"8BB233FB-4269-4B14-8703-A4FF1E25FB0D@gmail.com","threadId":"21977","inReplyTo":null,"subject":"git svn mkdirs ignores compressed unhandled.log files","fromName":"Robert Zeh","fromEmail":"robert.a.zeh@gmail.com","sentAt":"2009-12-17T17:10:03Z","receivedAt":"2009-12-17T17:10:03Z","isPatch":false,"sender":{"key":"robert.a.zeh@gmail.com","avatar":"https://avatars.githubusercontent.com/u/174737?v=4"},"body":"It looks like there is a conflict between git svn gc and git svn mkdirs. The git svn mkdirs command only looks at unhandled.log files. Shouldn't it also look at any compressed unhandled.log files too?\n\nRobert\n"},{"id":"130040","messageId":"20091217200852.GA5797@dcvr.yhbt.net","threadId":"21977","inReplyTo":"8BB233FB-4269-4B14-8703-A4FF1E25FB0D@gmail.com","subject":"Re: git svn mkdirs ignores compressed unhandled.log files","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-12-17T20:08:52Z","receivedAt":"2009-12-17T20:08:52Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Robert Zeh wrote:\n> It looks like there is a conflict between git svn gc and git svn\n> mkdirs. The git svn mkdirs command only looks at unhandled.log files.\n> Shouldn't it also look at any compressed unhandled.log files too?\n\nHi Robert,\n\nYes, an oversight. Does this patch work for you? (Highly untested)\n\nWould you mind writing a test case, been a bit busy with other stuff.\nThanks.\n\ndiff --git a/git-svn.perl b/git-svn.perl\nindex a4b052c..d362de7 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -2740,21 +2740,44 @@ sub do_fetch {\n \n sub mkemptydirs {\n \tmy ($self, $r) = @_;\n+\n+\tsub scan {\n+\t\tmy ($r, $empty_dirs, $line) = @_;\n+\t\tif (defined $r && $line =~ /^r(\\d+)$/) {\n+\t\t\treturn 0 if $1 > $r;\n+\t\t} elsif ($line =~ /^ \\+empty_dir: (.+)$/) {\n+\t\t\t$empty_dirs->{$1} = 1;\n+\t\t} elsif ($line =~ /^ \\-empty_dir: (.+)$/) {\n+\t\t\tmy @d = grep {m[^\\Q$1\\E(/|$)]} (keys %$empty_dirs);\n+\t\t\tdelete @$empty_dirs{@d};\n+\t\t}\n+\t\t1; # continue\n+\t};\n+\n \tmy %empty_dirs = ();\n+\tmy $gz_file = \"$self->{dir}/unhandled.log.gz\";\n+\tif (-f $gz_file) {\n+\t\tif (!$can_compress) {\n+\t\t\twarn \"Compress::Zlib could not be found; \",\n+\t\t\t \"empty directories in $gz_file will not be read\\n\";\n+\t\t} else {\n+\t\t\tmy $gz = Compress::Zlib::gzopen($gz_file, \"rb\") or\n+\t\t\t\tdie \"Unable to open $gz_file: $!\\n\";\n+\t\t\tmy $line;\n+\t\t\twhile ($gz->gzreadline($line) > 0) {\n+\t\t\t\tscan($r, \\%empty_dirs, $line) or last;\n+\t\t\t}\n+\t\t\t$gz->gzclose;\n+\t\t}\n+\t}\n \n-\topen my $fh, '<', \"$self->{dir}/unhandled.log\" or return;\n-\tbinmode $fh or croak \"binmode: $!\";\n-\twhile (<$fh>) {\n-\t\tif (defined $r && /^r(\\d+)$/) {\n-\t\t\tlast if $1 > $r;\n-\t\t} elsif (/^ \\+empty_dir: (.+)$/) {\n-\t\t\t$empty_dirs{$1} = 1;\n-\t\t} elsif (/^ \\-empty_dir: (.+)$/) {\n-\t\t\tmy @d = grep {m[^\\Q$1\\E(/|$)]} (keys %empty_dirs);\n-\t\t\tdelete @empty_dirs{@d};\n+\tif (open my $fh, '<', \"$self->{dir}/unhandled.log\") {\n+\t\tbinmode $fh or croak \"binmode: $!\";\n+\t\twhile (<$fh>) {\n+\t\t\tscan($r, \\%empty_dirs, $_) or last;\n \t\t}\n+\t\tclose $fh;\n \t}\n-\tclose $fh;\n \n \tmy $strip = qr/\\A\\Q$self->{path}\\E(?:\\/|$)/;\n \tforeach my $d (sort keys %empty_dirs) {\n-- \nEric Wong\n"},{"id":"130164","messageId":"20091219222738.GA20331@dcvr.yhbt.net","threadId":"21977","inReplyTo":"20091217200852.GA5797@dcvr.yhbt.net","subject":"[PATCH] git svn: make empty directory creation gc-aware","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-12-19T22:27:38Z","receivedAt":"2009-12-19T22:27:38Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"The \"git svn gc\" command creates and appends to unhandled.log.gz\nfiles which should be parsed before the uncompressed\nunhandled.log files.\n\nReported-by: Robert Zeh\nSigned-off-by: Eric Wong \n---\n Eric Wong wrote:\n > Robert Zeh wrote:\n > > It looks like there is a conflict between git svn gc and git svn\n > > mkdirs. The git svn mkdirs command only looks at unhandled.log files.\n > > Shouldn't it also look at any compressed unhandled.log files too?\n > \n > Hi Robert,\n > \n > Yes, an oversight. Does this patch work for you? (Highly untested)\n\n Test case included and pushed out to git://git.bogomips.org/git-svn\n\n More pushes hopefully coming as Sam and Andrew work out the mergeinfo\n performance problems and I look into crossing svn-remote boundaries\n for parent lookups.\n\n git-svn.perl | 45 +++++++++++++++++++++++++++++++----------\n t/t9146-git-svn-empty-dirs.sh | 24 +++++++++++++++++++++\n 2 files changed, 58 insertions(+), 11 deletions(-)\n\ndiff --git a/git-svn.perl b/git-svn.perl\nindex a4b052c..d362de7 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -2740,21 +2740,44 @@ sub do_fetch {\n \n sub mkemptydirs {\n \tmy ($self, $r) = @_;\n+\n+\tsub scan {\n+\t\tmy ($r, $empty_dirs, $line) = @_;\n+\t\tif (defined $r && $line =~ /^r(\\d+)$/) {\n+\t\t\treturn 0 if $1 > $r;\n+\t\t} elsif ($line =~ /^ \\+empty_dir: (.+)$/) {\n+\t\t\t$empty_dirs->{$1} = 1;\n+\t\t} elsif ($line =~ /^ \\-empty_dir: (.+)$/) {\n+\t\t\tmy @d = grep {m[^\\Q$1\\E(/|$)]} (keys %$empty_dirs);\n+\t\t\tdelete @$empty_dirs{@d};\n+\t\t}\n+\t\t1; # continue\n+\t};\n+\n \tmy %empty_dirs = ();\n+\tmy $gz_file = \"$self->{dir}/unhandled.log.gz\";\n+\tif (-f $gz_file) {\n+\t\tif (!$can_compress) {\n+\t\t\twarn \"Compress::Zlib could not be found; \",\n+\t\t\t \"empty directories in $gz_file will not be read\\n\";\n+\t\t} else {\n+\t\t\tmy $gz = Compress::Zlib::gzopen($gz_file, \"rb\") or\n+\t\t\t\tdie \"Unable to open $gz_file: $!\\n\";\n+\t\t\tmy $line;\n+\t\t\twhile ($gz->gzreadline($line) > 0) {\n+\t\t\t\tscan($r, \\%empty_dirs, $line) or last;\n+\t\t\t}\n+\t\t\t$gz->gzclose;\n+\t\t}\n+\t}\n \n-\topen my $fh, '<', \"$self->{dir}/unhandled.log\" or return;\n-\tbinmode $fh or croak \"binmode: $!\";\n-\twhile (<$fh>) {\n-\t\tif (defined $r && /^r(\\d+)$/) {\n-\t\t\tlast if $1 > $r;\n-\t\t} elsif (/^ \\+empty_dir: (.+)$/) {\n-\t\t\t$empty_dirs{$1} = 1;\n-\t\t} elsif (/^ \\-empty_dir: (.+)$/) {\n-\t\t\tmy @d = grep {m[^\\Q$1\\E(/|$)]} (keys %empty_dirs);\n-\t\t\tdelete @empty_dirs{@d};\n+\tif (open my $fh, '<', \"$self->{dir}/unhandled.log\") {\n+\t\tbinmode $fh or croak \"binmode: $!\";\n+\t\twhile (<$fh>) {\n+\t\t\tscan($r, \\%empty_dirs, $_) or last;\n \t\t}\n+\t\tclose $fh;\n \t}\n-\tclose $fh;\n \n \tmy $strip = qr/\\A\\Q$self->{path}\\E(?:\\/|$)/;\n \tforeach my $d (sort keys %empty_dirs) {\ndiff --git a/t/t9146-git-svn-empty-dirs.sh b/t/t9146-git-svn-empty-dirs.sh\nindex 9b8d046..3f2d719 100755\n--- a/t/t9146-git-svn-empty-dirs.sh\n+++ b/t/t9146-git-svn-empty-dirs.sh\n@@ -114,5 +114,29 @@ test_expect_success 'removed top-level directory does not exist' '\n \ttest ! -e removed/d\n \n '\n+unhandled=.git/svn/refs/remotes/git-svn/unhandled.log\n+test_expect_success 'git svn gc-ed files work' '\n+\t(\n+\t\tcd removed &&\n+\t\tgit svn gc &&\n+\t\t: Compress::Zlib may not be available &&\n+\t\tif test -f \"$unhandled\".gz\n+\t\tthen\n+\t\t\tsvn mkdir -m gz \"$svnrepo\"/gz &&\n+\t\t\tgit reset --hard $(git rev-list HEAD | tail -1) &&\n+\t\t\tgit svn rebase &&\n+\t\t\ttest -f \"$unhandled\".gz &&\n+\t\t\ttest -f \"$unhandled\" &&\n+\t\t\tfor i in a b c \"weird file name\" gz \"! !\"\n+\t\t\tdo\n+\t\t\t\tif ! test -d \"$i\"\n+\t\t\t\tthen\n+\t\t\t\t\techo >&2 \"$i does not exist\"\n+\t\t\t\t\texit 1\n+\t\t\t\tfi\n+\t\t\tdone\n+\t\tfi\n+\t)\n+'\n \n test_done\n-- \nEric Wong\n"},{"id":"130178","messageId":"7vzl5e3yx3.fsf@alter.siamese.dyndns.org","threadId":"21977","inReplyTo":"20091219222738.GA20331@dcvr.yhbt.net","subject":"Re: [PATCH] git svn: make empty directory creation gc-aware","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-12-20T07:08:40Z","receivedAt":"2009-12-20T07:08:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong writes:\n\n> The \"git svn gc\" command creates and appends to unhandled.log.gz\n> files which should be parsed before the uncompressed\n> unhandled.log files.\n>\n> Reported-by: Robert Zeh\n> Signed-off-by: Eric Wong \n> ---\n> Eric Wong wrote:\n> > Robert Zeh wrote:\n> > > It looks like there is a conflict between git svn gc and git svn\n> > > mkdirs. The git svn mkdirs command only looks at unhandled.log files.\n> > > Shouldn't it also look at any compressed unhandled.log files too?\n> > \n> > Hi Robert,\n> > \n> > Yes, an oversight. Does this patch work for you? (Highly untested)\n>\n> Test case included and pushed out to git://git.bogomips.org/git-svn\n>\n> More pushes hopefully coming as Sam and Andrew work out the mergeinfo\n> performance problems and I look into crossing svn-remote boundaries\n> for parent lookups.\n\nThanks.\n\nThis particular patch should be in 1.6.6 final, because mkdirs first\nappeared in 1.6.6-rc0 at 6111b93 (git svn: attempt to create empty dirs on\nclone+rebase, 2009-11-15), and 1.6.5.X series does not have the command,\nso this seems like a new feature that never existed in any tagged release,\nand if we shipped 1.6.6 without this patch, we will be shipping it with a\nknow breakage, while if we shipped it with this, even if this patch\nsomehow had an unintended side effect, at worst we'd be exchanging a bug\nwith some other bug, so it wouldn't be worse.\n\nIs mkdirs the only \"noteworthy\" feature that should be mentioned in the\nRelease Notes in your area? It would be really nice if you can give a\npatch to Documentation/RelNotes-1.6.6.txt in a few days to turn a single\nliner I have there to something more helpful. The current shortlog since\n1.6.5 indicates there weren't that much activity during this release.\n\n\n\nAlex Vandiver (3):\n git-svn: sort svk merge tickets to account for minimal parents\n git-svn: Set svn.authorsfile to an absolute path when cloning\n git-svn: set svn.authorsfile earlier when cloning\n\nEric Wong (7):\n git svn: fix fetch where glob is on the top-level URL\n git svn: read global+system config for clone+init\n git svn: attempt to create empty dirs on clone+rebase\n git svn: always reuse existing remotes on fetch\n git svn: strip leading path when making empty dirs\n git svn: log removals of empty directories\n git svn: make empty directory creation gc-aware\n\nGreg Price (1):\n git svn: Don't create empty directories whose parents were deleted\n\nJonathan Nieder (2):\n add -i, send-email, svn, p4, etc: use \"git var GIT_EDITOR\"\n am -i, git-svn: use \"git var GIT_PAGER\"\n\nSam Vilain (2):\n git-svn: convert SVK merge tickets to extra parents\n git-svn: convert SVN 1.5+ / svnmerge.py svn:mergeinfo props to parents\n\nThomas Rast (1):\n Document git-svn's first-parent rule\n\nToby Allsopp (1):\n git svn: handle SVN merges from revisions past the tip of the branch\n"},{"id":"130179","messageId":"20091220070959.GA8141@dcvr.yhbt.net","threadId":"21977","inReplyTo":"20091219222738.GA20331@dcvr.yhbt.net","subject":"[PATCH 2/1] t9146: use 'svn_cmd' wrapper","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-12-20T07:09:59Z","receivedAt":"2009-12-20T07:09:59Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Using 'svn' directly may not work for all users.\n\nSigned-off-by: Eric Wong \n---\n > Test case included and pushed out to git://git.bogomips.org/git-svn\n\n Junio: Not sure if you've merged yet, but feel free to squash this\n with the other one. Thanks.\n\n t/t9146-git-svn-empty-dirs.sh | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/t/t9146-git-svn-empty-dirs.sh b/t/t9146-git-svn-empty-dirs.sh\nindex 3f2d719..565365c 100755\n--- a/t/t9146-git-svn-empty-dirs.sh\n+++ b/t/t9146-git-svn-empty-dirs.sh\n@@ -122,7 +122,7 @@ test_expect_success 'git svn gc-ed files work' '\n \t\t: Compress::Zlib may not be available &&\n \t\tif test -f \"$unhandled\".gz\n \t\tthen\n-\t\t\tsvn mkdir -m gz \"$svnrepo\"/gz &&\n+\t\t\tsvn_cmd mkdir -m gz \"$svnrepo\"/gz &&\n \t\t\tgit reset --hard $(git rev-list HEAD | tail -1) &&\n \t\t\tgit svn rebase &&\n \t\t\ttest -f \"$unhandled\".gz &&\n-- \nEric Wong\n"},{"id":"130180","messageId":"20091220072015.GB8141@dcvr.yhbt.net","threadId":"21977","inReplyTo":"7vzl5e3yx3.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] git svn: make empty directory creation gc-aware","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-12-20T07:21:42Z","receivedAt":"2009-12-20T07:21:42Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano wrote:\n> > More pushes hopefully coming as Sam and Andrew work out the mergeinfo\n> > performance problems and I look into crossing svn-remote boundaries\n> > for parent lookups.\n> \n> Thanks.\n> \n> This particular patch should be in 1.6.6 final, because mkdirs first\n> appeared in 1.6.6-rc0 at 6111b93 (git svn: attempt to create empty dirs on\n> clone+rebase, 2009-11-15), and 1.6.5.X series does not have the command,\n> so this seems like a new feature that never existed in any tagged release,\n> and if we shipped 1.6.6 without this patch, we will be shipping it with a\n> know breakage, while if we shipped it with this, even if this patch\n> somehow had an unintended side effect, at worst we'd be exchanging a bug\n> with some other bug, so it wouldn't be worse.\n\nI agree completely.\n\n> Is mkdirs the only \"noteworthy\" feature that should be mentioned in the\n> Release Notes in your area? It would be really nice if you can give a\n> patch to Documentation/RelNotes-1.6.6.txt in a few days to turn a single\n> liner I have there to something more helpful. The current shortlog since\n> 1.6.5 indicates there weren't that much activity during this release.\n\nSam's merge handling work is definitely noteworthy, but it's already in\nthe release notes, hopefully the performance regression there is worked\nout. I'll definitely send you a patch to the release notes after I get\na chance to figure out the other issue with multiple svn-remotes\ntonight/tomorrow.\n\n-- \nEric Wong\n"},{"id":"130270","messageId":"661FC27C-6C5D-4703-93BE-48B1812C3800@gmail.com","threadId":"21977","inReplyTo":"20091217200852.GA5797@dcvr.yhbt.net","subject":"Re: git svn mkdirs ignores compressed unhandled.log files","fromName":"Robert Zeh","fromEmail":"robert.a.zeh@gmail.com","sentAt":"2009-12-23T04:12:02Z","receivedAt":"2009-12-23T04:12:02Z","isPatch":false,"sender":{"key":"robert.a.zeh@gmail.com","avatar":"https://avatars.githubusercontent.com/u/174737?v=4"},"body":"On Dec 17, 2009, at 2:08 PM, Eric Wong wrote:\n\n> Robert Zeh wrote:\n>> It looks like there is a conflict between git svn gc and git svn\n>> mkdirs. The git svn mkdirs command only looks at unhandled.log files.\n>> Shouldn't it also look at any compressed unhandled.log files too?\n> \n> Hi Robert,\n> \n> Yes, an oversight. Does this patch work for you? (Highly untested)\n> \n> Would you mind writing a test case, been a bit busy with other stuff.\n> Thanks.\n\nEric,\n\nYour patch works for the existing t9146-git-svn-empty-dirs.sh test, and the test\nI've sent as a patch in another email.\n\nRobert"}]}