Robert ZehIt looks like there is a conflict between git svn gc and git svn mkdirs. The git svn mkdirs command only looks at unhandled.log files. Shouldn't it also look at any compressed unhandled.log files too?
Robert
threads / discuss / 21977
git svn mkdirs ignores compressed unhandled.log files
Subject: git svn mkdirs ignores compressed unhandled.log files
Robert ZehRe: git svn mkdirs ignores compressed unhandled.log files
Robert Zeh <robert.a.zeh@gmail.com> wrote:
> It looks like there is a conflict between git svn gc and git svn > mkdirs. The git svn mkdirs command only looks at unhandled.log files. > Shouldn't it also look at any compressed unhandled.log files too?
Hi Robert,
Yes, an oversight. Does this patch work for you? (Highly untested)
Would you mind writing a test case, been a bit busy with other stuff. Thanks.
diff --git a/git-svn.perl b/git-svn.perl index a4b052c..d362de7 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -2740,21 +2740,44 @@ sub do_fetch { sub mkemptydirs { my ($self, $r) = @_; + + sub scan { + my ($r, $empty_dirs, $line) = @_; + if (defined $r && $line =~ /^r(\d+)$/) { + return 0 if $1 > $r; + } elsif ($line =~ /^ \+empty_dir: (.+)$/) { + $empty_dirs->{$1} = 1; + } elsif ($line =~ /^ \-empty_dir: (.+)$/) { + my @d = grep {m[^\Q$1\E(/|$)]} (keys %$empty_dirs); + delete @$empty_dirs{@d}; + } + 1; # continue + }; + my %empty_dirs = (); + my $gz_file = "$self->{dir}/unhandled.log.gz"; + if (-f $gz_file) { + if (!$can_compress) { + warn "Compress::Zlib could not be found; ", + "empty directories in $gz_file will not be read\n"; + } else { + my $gz = Compress::Zlib::gzopen($gz_file, "rb") or + die "Unable to open $gz_file: $!\n"; + my $line; + while ($gz->gzreadline($line) > 0) { + scan($r, \%empty_dirs, $line) or last; + } + $gz->gzclose; + } + } - open my $fh, '<', "$self->{dir}/unhandled.log" or return; - binmode $fh or croak "binmode: $!"; - while (<$fh>) { - if (defined $r && /^r(\d+)$/) { - last if $1 > $r; - } elsif (/^ \+empty_dir: (.+)$/) { - $empty_dirs{$1} = 1; - } elsif (/^ \-empty_dir: (.+)$/) { - my @d = grep {m[^\Q$1\E(/|$)]} (keys %empty_dirs); - delete @empty_dirs{@d}; + if (open my $fh, '<', "$self->{dir}/unhandled.log") { + binmode $fh or croak "binmode: $!"; + while (<$fh>) { + scan($r, \%empty_dirs, $_) or last; } + close $fh; } - close $fh; my $strip = qr/\A\Q$self->{path}\E(?:\/|$)/; foreach my $d (sort keys %empty_dirs) {
-- Eric Wong
[PATCH] git svn: make empty directory creation gc-aware
The "git svn gc" command creates and appends to unhandled.log.gz files which should be parsed before the uncompressed unhandled.log files.
Reported-by: Robert Zeh Signed-off-by: Eric Wong <normalperson@yhbt.net> --- Eric Wong <normalperson@yhbt.net> wrote: > Robert Zeh <robert.a.zeh@gmail.com> wrote: > > It looks like there is a conflict between git svn gc and git svn > > mkdirs. The git svn mkdirs command only looks at unhandled.log files. > > Shouldn't it also look at any compressed unhandled.log files too? > > Hi Robert, > > Yes, an oversight. Does this patch work for you? (Highly untested)
Test case included and pushed out to git://git.bogomips.org/git-svn
More pushes hopefully coming as Sam and Andrew work out the mergeinfo performance problems and I look into crossing svn-remote boundaries for parent lookups.
git-svn.perl | 45 +++++++++++++++++++++++++++++++---------- t/t9146-git-svn-empty-dirs.sh | 24 +++++++++++++++++++++ 2 files changed, 58 insertions(+), 11 deletions(-)
diff --git a/git-svn.perl b/git-svn.perl index a4b052c..d362de7 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -2740,21 +2740,44 @@ sub do_fetch { sub mkemptydirs { my ($self, $r) = @_; + + sub scan { + my ($r, $empty_dirs, $line) = @_; + if (defined $r && $line =~ /^r(\d+)$/) { + return 0 if $1 > $r; + } elsif ($line =~ /^ \+empty_dir: (.+)$/) { + $empty_dirs->{$1} = 1; + } elsif ($line =~ /^ \-empty_dir: (.+)$/) { + my @d = grep {m[^\Q$1\E(/|$)]} (keys %$empty_dirs); + delete @$empty_dirs{@d}; + } + 1; # continue + }; + my %empty_dirs = (); + my $gz_file = "$self->{dir}/unhandled.log.gz"; + if (-f $gz_file) { + if (!$can_compress) { + warn "Compress::Zlib could not be found; ", + "empty directories in $gz_file will not be read\n"; + } else { + my $gz = Compress::Zlib::gzopen($gz_file, "rb") or + die "Unable to open $gz_file: $!\n"; + my $line; + while ($gz->gzreadline($line) > 0) { + scan($r, \%empty_dirs, $line) or last; + } + $gz->gzclose; + } + } - open my $fh, '<', "$self->{dir}/unhandled.log" or return; - binmode $fh or croak "binmode: $!"; - while (<$fh>) { - if (defined $r && /^r(\d+)$/) { - last if $1 > $r; - } elsif (/^ \+empty_dir: (.+)$/) { - $empty_dirs{$1} = 1; - } elsif (/^ \-empty_dir: (.+)$/) { - my @d = grep {m[^\Q$1\E(/|$)]} (keys %empty_dirs); - delete @empty_dirs{@d}; + if (open my $fh, '<', "$self->{dir}/unhandled.log") { + binmode $fh or croak "binmode: $!"; + while (<$fh>) { + scan($r, \%empty_dirs, $_) or last; } + close $fh; } - close $fh; my $strip = qr/\A\Q$self->{path}\E(?:\/|$)/; foreach my $d (sort keys %empty_dirs) { diff --git a/t/t9146-git-svn-empty-dirs.sh b/t/t9146-git-svn-empty-dirs.sh index 9b8d046..3f2d719 100755 --- a/t/t9146-git-svn-empty-dirs.sh +++ b/t/t9146-git-svn-empty-dirs.sh @@ -114,5 +114,29 @@ test_expect_success 'removed top-level directory does not exist' ' test ! -e removed/d ' +unhandled=.git/svn/refs/remotes/git-svn/unhandled.log +test_expect_success 'git svn gc-ed files work' ' + ( + cd removed && + git svn gc && + : Compress::Zlib may not be available && + if test -f "$unhandled".gz + then + svn mkdir -m gz "$svnrepo"/gz && + git reset --hard $(git rev-list HEAD | tail -1) && + git svn rebase && + test -f "$unhandled".gz && + test -f "$unhandled" && + for i in a b c "weird file name" gz "! !" + do + if ! test -d "$i" + then + echo >&2 "$i does not exist" + exit 1 + fi + done + fi + ) +' test_done
-- Eric Wong
Junio C HamanoRe: [PATCH] git svn: make empty directory creation gc-aware
Eric Wong <normalperson@yhbt.net> writes:
Show 22 quoted lines
> The "git svn gc" command creates and appends to unhandled.log.gz > files which should be parsed before the uncompressed > unhandled.log files. > > Reported-by: Robert Zeh > Signed-off-by: Eric Wong <normalperson@yhbt.net> > --- > Eric Wong <normalperson@yhbt.net> wrote: > > Robert Zeh <robert.a.zeh@gmail.com> wrote: > > > It looks like there is a conflict between git svn gc and git svn > > > mkdirs. The git svn mkdirs command only looks at unhandled.log files. > > > Shouldn't it also look at any compressed unhandled.log files too? > > > > Hi Robert, > > > > Yes, an oversight. Does this patch work for you? (Highly untested) > > Test case included and pushed out to git://git.bogomips.org/git-svn > > More pushes hopefully coming as Sam and Andrew work out the mergeinfo > performance problems and I look into crossing svn-remote boundaries > for parent lookups.
Thanks.
This particular patch should be in 1.6.6 final, because mkdirs first appeared in 1.6.6-rc0 at 6111b93 (git svn: attempt to create empty dirs on clone+rebase, 2009-11-15), and 1.6.5.X series does not have the command, so this seems like a new feature that never existed in any tagged release, and if we shipped 1.6.6 without this patch, we will be shipping it with a know breakage, while if we shipped it with this, even if this patch somehow had an unintended side effect, at worst we'd be exchanging a bug with some other bug, so it wouldn't be worse.
Is mkdirs the only "noteworthy" feature that should be mentioned in the Release Notes in your area? It would be really nice if you can give a patch to Documentation/RelNotes-1.6.6.txt in a few days to turn a single liner I have there to something more helpful. The current shortlog since 1.6.5 indicates there weren't that much activity during this release.
Alex Vandiver (3):
git-svn: sort svk merge tickets to account for minimal parents
git-svn: Set svn.authorsfile to an absolute path when cloning
git-svn: set svn.authorsfile earlier when cloningEric Wong (7):
git svn: fix fetch where glob is on the top-level URL
git svn: read global+system config for clone+init
git svn: attempt to create empty dirs on clone+rebase
git svn: always reuse existing remotes on fetch
git svn: strip leading path when making empty dirs
git svn: log removals of empty directories
git svn: make empty directory creation gc-awareGreg Price (1):
git svn: Don't create empty directories whose parents were deletedJonathan Nieder (2):
add -i, send-email, svn, p4, etc: use "git var GIT_EDITOR"
am -i, git-svn: use "git var GIT_PAGER"Sam Vilain (2):
git-svn: convert SVK merge tickets to extra parents
git-svn: convert SVN 1.5+ / svnmerge.py svn:mergeinfo props to parentsThomas Rast (1):
Document git-svn's first-parent ruleToby Allsopp (1):
git svn: handle SVN merges from revisions past the tip of the branchRe: [PATCH] git svn: make empty directory creation gc-aware
Junio C Hamano <gitster@pobox.com> wrote:
Show 14 quoted lines
> > More pushes hopefully coming as Sam and Andrew work out the mergeinfo > > performance problems and I look into crossing svn-remote boundaries > > for parent lookups. > > Thanks. > > This particular patch should be in 1.6.6 final, because mkdirs first > appeared in 1.6.6-rc0 at 6111b93 (git svn: attempt to create empty dirs on > clone+rebase, 2009-11-15), and 1.6.5.X series does not have the command, > so this seems like a new feature that never existed in any tagged release, > and if we shipped 1.6.6 without this patch, we will be shipping it with a > know breakage, while if we shipped it with this, even if this patch > somehow had an unintended side effect, at worst we'd be exchanging a bug > with some other bug, so it wouldn't be worse.
I agree completely.
Show 5 quoted lines
> Is mkdirs the only "noteworthy" feature that should be mentioned in the > Release Notes in your area? It would be really nice if you can give a > patch to Documentation/RelNotes-1.6.6.txt in a few days to turn a single > liner I have there to something more helpful. The current shortlog since > 1.6.5 indicates there weren't that much activity during this release.
Sam's merge handling work is definitely noteworthy, but it's already in the release notes, hopefully the performance regression there is worked out. I'll definitely send you a patch to the release notes after I get a chance to figure out the other issue with multiple svn-remotes tonight/tomorrow.
-- Eric Wong
[PATCH 2/1] t9146: use 'svn_cmd' wrapper
Using 'svn' directly may not work for all users.
Signed-off-by: Eric Wong <normalperson@yhbt.net> --- > Test case included and pushed out to git://git.bogomips.org/git-svn
Junio: Not sure if you've merged yet, but feel free to squash this with the other one. Thanks.
t/t9146-git-svn-empty-dirs.sh | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/t/t9146-git-svn-empty-dirs.sh b/t/t9146-git-svn-empty-dirs.sh index 3f2d719..565365c 100755 --- a/t/t9146-git-svn-empty-dirs.sh +++ b/t/t9146-git-svn-empty-dirs.sh @@ -122,7 +122,7 @@ test_expect_success 'git svn gc-ed files work' ' : Compress::Zlib may not be available && if test -f "$unhandled".gz then - svn mkdir -m gz "$svnrepo"/gz && + svn_cmd mkdir -m gz "$svnrepo"/gz && git reset --hard $(git rev-list HEAD | tail -1) && git svn rebase && test -f "$unhandled".gz &&
-- Eric Wong
Robert ZehRe: git svn mkdirs ignores compressed unhandled.log files
On Dec 17, 2009, at 2:08 PM, Eric Wong wrote:
Show 11 quoted lines
> Robert Zeh <robert.a.zeh@gmail.com> wrote: >> It looks like there is a conflict between git svn gc and git svn >> mkdirs. The git svn mkdirs command only looks at unhandled.log files. >> Shouldn't it also look at any compressed unhandled.log files too? > > Hi Robert, > > Yes, an oversight. Does this patch work for you? (Highly untested) > > Would you mind writing a test case, been a bit busy with other stuff. > Thanks.
Eric,
Your patch works for the existing t9146-git-svn-empty-dirs.sh test, and the test I've sent as a patch in another email.
Robert