git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 3/8] Improve our URL canonicalization to be more like SVN 1.7's.

From
Michael G. Schwern <schwern@pobox.com>
Date
Jul 28, 2012, 09:47 UTC
Message-ID
<1343468872-72133-4-git-send-email-schwern@pobox.com>
In-Reply-To
<1343468872-72133-1-git-send-email-schwern@pobox.com>
From: "Michael G. Schwern" <schwern@pobox.com>

Previously, our URL canonicalization didn't do much of anything. Now it actually escapes and collapses slashes. This is mostly a cut & paste of escape_url from git-svn.

This is closer to how SVN 1.7's canonicalization behaves. Doing it with 1.6 lets us chase down some problems caused by more effective canonicalization without having to deal with all the other 1.7 issues on top of that.

* Remote URLs have to be canonicalized otherwise Git::SVN->find_existing_remote
  will think they're different.
* The SVN remote is now written to the git config canonicalized.  That
  should be ok.  Adjust a test to account for that.
---
 perl/Git/SVN.pm                    |  4 ++--
 perl/Git/SVN/Utils.pm              | 19 +++++++++++++++++--
 t/Git-SVN/Utils/canonicalize_url.t | 26 ++++++++++++++++++++++++++
 t/t9107-git-svn-migrate.sh         |  4 +++-
 4 files changed, 48 insertions(+), 5 deletions(-)
 create mode 100644 t/Git-SVN/Utils/canonicalize_url.t
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index 798f6c4..cb6d83a 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -201,9 +201,9 @@ sub read_all_remotes {
 		} elsif (m!^(.+)\.usesvmprops=\s*(.*)\s*$!) {
 			$r->{$1}->{svm} = {};
 		} elsif (m!^(.+)\.url=\s*(.*)\s*$!) {
-			$r->{$1}->{url} = $2;
+			$r->{$1}->{url} = canonicalize_url($2);
 		} elsif (m!^(.+)\.pushurl=\s*(.*)\s*$!) {
-			$r->{$1}->{pushurl} = $2;
+			$r->{$1}->{pushurl} = canonicalize_url($2);
 		} elsif (m!^(.+)\.ignore-refs=\s*(.*)\s*$!) {
 			$r->{$1}->{ignore_refs_regex} = $2;
 		} elsif (m!^(.+)\.(branches|tags)=$svn_refspec$!) {
diff --git a/perl/Git/SVN/Utils.pm b/perl/Git/SVN/Utils.pm
index 7ae6fac..dab6e4d 100644
--- a/perl/Git/SVN/Utils.pm
+++ b/perl/Git/SVN/Utils.pm
@@ -147,10 +147,25 @@ sub canonicalize_url {
 }
 
 
+sub _canonicalize_url_path {
+	my ($uri_path) = @_;
+
+	my @parts;
+	foreach my $part (split m{/+}, $uri_path) {
+		$part =~ s/([^~\w.%+-]|%(?![a-fA-F0-9]{2}))/sprintf("%%%02X",ord($1))/eg;
+		push @parts, $part;
+	}
+
+	return join('/', @parts);
+}
+
 sub _canonicalize_url_ourselves {
 	my ($url) = @_;
-	$url =~ s#^([^:]+://[^/]*/)(.*)$#$1 . canonicalize_path($2)#e;
-	return $url;
+	if ($url =~ m#^([^:]+)://([^/]*)(.*)$#) {
+		my ($scheme, $domain, $uri) = ($1, $2, _canonicalize_url_path(canonicalize_path($3)));
+		$url = "$scheme://$domain$uri";
+	}
+	$url;
 }
 
 
diff --git a/t/Git-SVN/Utils/canonicalize_url.t b/t/Git-SVN/Utils/canonicalize_url.t
new file mode 100644
index 0000000..05795ab
--- /dev/null
+++ b/t/Git-SVN/Utils/canonicalize_url.t
@@ -0,0 +1,26 @@
+#!/usr/bin/env perl
+
+# Test our own home rolled URL canonicalizer.  Test the private one
+# directly because we can't predict what the SVN API is doing to do.
+
+use strict;
+use warnings;
+
+use Test::More 'no_plan';
+
+use Git::SVN::Utils;
+my $canonicalize_url = \&Git::SVN::Utils::_canonicalize_url_ourselves;
+
+my %tests = (
+	"http://x.com"			=> "http://x.com",
+	"http://x.com/"			=> "http://x.com",
+	"http://x.com/foo/bar"		=> "http://x.com/foo/bar",
+	"http://x.com//foo//bar//"	=> "http://x.com/foo/bar",
+	"http://x.com/  /%/"		=> "http://x.com/%20%20/%25",
+);
+
+for my $arg (keys %tests) {
+	my $want = $tests{$arg};
+
+	is $canonicalize_url->($arg), $want, "canonicalize_url('$arg') => $want";
+}
diff --git a/t/t9107-git-svn-migrate.sh b/t/t9107-git-svn-migrate.sh
index cfb4453..ee73013 100755
--- a/t/t9107-git-svn-migrate.sh
+++ b/t/t9107-git-svn-migrate.sh
@@ -27,6 +27,8 @@ test_expect_success 'setup old-looking metadata' '
 head=`git rev-parse --verify refs/heads/git-svn-HEAD^0`
 test_expect_success 'git-svn-HEAD is a real HEAD' "test -n '$head'"
 
+svnrepo_escaped=`echo $svnrepo | sed 's/ /%20/'`
+
 test_expect_success 'initialize old-style (v0) git svn layout' '
 	mkdir -p "$GIT_DIR"/git-svn/info "$GIT_DIR"/svn/info &&
 	echo "$svnrepo" > "$GIT_DIR"/git-svn/info/url &&
@@ -35,7 +37,7 @@ test_expect_success 'initialize old-style (v0) git svn layout' '
 	! test -d "$GIT_DIR"/git-svn &&
 	git rev-parse --verify refs/${remotes_git_svn}^0 &&
 	git rev-parse --verify refs/remotes/svn^0 &&
-	test "$(git config --get svn-remote.svn.url)" = "$svnrepo" &&
+	test "$(git config --get svn-remote.svn.url)" = "$svnrepo_escaped" &&
 	test `git config --get svn-remote.svn.fetch` = \
              ":refs/${remotes_git_svn}"
 	'
-- 
1.7.11.3
Previous: Michael G. SchwernNext: Michael G. Schwern
Message 15 of 50 in “Fix git-svn for SVN 1.7”
  1. Michael G. SchwernJul 28, 2012
  2. 1/8 SVN 1.7 will truncate "not-a%40{0}" to just "not-a".Michael G. Schwern, Jul 28, 2012
  3. Jonathan NiederJul 28, 2012
  4. Michael G SchwernJul 28, 2012
  5. svn test: escape peg revision separator using empty peg revJonathan Nieder, Oct 9, 2012
  6. Michael J GruberOct 9, 2012
  7. Jonathan NiederOct 9, 2012
  8. Eric WongOct 10, 2012
  9. Jonathan NiederOct 10, 2012
  10. Eric WongOct 10, 2012
  11. Jonathan NiederOct 10, 2012
  12. Eric WongOct 10, 2012
  13. Junio C HamanoOct 10, 2012
  14. 2/8 Fix typo in testMichael G. Schwern, Jul 28, 2012
  15. 3/8 Improve our URL canonicalization to be more like SVN 1.7's.Michael G. Schwern, Jul 28, 2012
  16. 4/8 Replace hand rolled URL escapes with canonicalizationMichael G. Schwern, Jul 28, 2012
  17. 5/8 Canonicalize earlier in a couple spots.Michael G. Schwern, Jul 28, 2012
  18. 6/8 Add function to append a path to a URL.Michael G. Schwern, Jul 28, 2012
  19. 7/8 Turn on canonicalization on newly minted URLs.Michael G. Schwern, Jul 28, 2012
  20. test: work around SVN 1.7 mishandling of svn:special changesJonathan Nieder, Oct 6, 2012
  21. git svn: work around SVN 1.7 mishandling of svn:special changesJonathan Nieder, Oct 9, 2012
  22. Eric WongOct 10, 2012
  23. git svn: work around SVN 1.7 mishandling of svn:special changesJonathan Nieder, Oct 10, 2012
  24. 8/8 Remove some ad hoc canonicalizations.Michael G. Schwern, Jul 28, 2012
  25. Eric WongJul 30, 2012
  26. Michael G SchwernJul 30, 2012
  27. Eric WongJul 30, 2012
  28. Michael G SchwernJul 31, 2012
  29. Eric WongJul 31, 2012
  30. Michael G SchwernJul 31, 2012
  31. Junio C HamanoJul 31, 2012
  32. Michael G SchwernJul 31, 2012
  33. Eric WongJul 31, 2012
  34. Junio C HamanoJul 31, 2012
  35. Michael G SchwernJul 31, 2012
  36. Michael G SchwernJul 31, 2012
  37. Eric WongAug 1, 2012
  38. Eric WongAug 2, 2012
  39. Jonathan NiederAug 2, 2012
  40. Junio C HamanoAug 2, 2012
  41. Robin H. JohnsonAug 2, 2012
  42. Eric WongAug 2, 2012
  43. Junio C HamanoAug 21, 2012
  44. Eric WongAug 21, 2012
  45. Junio C HamanoAug 21, 2012
  46. Eric WongAug 2, 2012
  47. Junio C HamanoAug 2, 2012
  48. Eric WongAug 2, 2012
  49. Eric WongAug 2, 2012
  50. Junio C HamanoAug 2, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.