threads / patch / 20863

patch, 2 partsAdd url.<base>.pushInsteadOf: URL rewriting for push only

Subject: [PATCH 0/2] Add url.<base>.pushInsteadOf: URL rewriting for push only

## tl;dr

5 messages between Sep 7, 2009 and Sep 7, 2009. Diffs are folded; open one to read it.

replies: 4people: 2as markdown or json

Josh Triplett· Sep 7, 2009, 06:59 UTC · lore

Many sites host repositories via both git:// for fetch-only anonymous access and ssh:// for push-capable access. The "insteadOf" mechanism makes it straightforward to substitute the push-capable URLs for the pull-only URLs, which proves convenient when the site hosts many repositories using the same URL scheme. However, if you use such a substitution and you cannot use the ssh:// URLs (either because you don't have SSH access or you don't have permission to a particular repository), you cannot clone or fetch either, even though you could do so via the git:// URLs. A situation like this arises when sharing git configuration files between systems, of which only a few have SSH access to repositories.

"pushurl" provides a way to specify URLs used only for push, but this requires configuring a pushurl for each such repository. As in the rationale for insteadOf, it makes sense to configure this for all repositories hosted on a given system at once.

This patch series adds a new "pushInsteadOf" option to go with "insteadOf". pushInsteadOf allows systematically rewriting fetch-only URLs to push-capable URLs when used with push. For instance:

[url "ssh://example.org/"]
    pushInsteadOf = "git://example.org/"

This will allow clones of "git://example.org/path/to/repo" to subsequently push to "ssh://example.org/path/to/repo", without manually configuring pushurl for that remote.

Includes documentation for the new option, bash completion updates, and test cases (both that pushInsteadOf applies to push and that it does *not* apply to fetch).

Josh Triplett (2):
  Wrap rewrite globals in a struct in preparation for adding another set
  Add url.<base>.pushInsteadOf: URL rewriting for push only
 Documentation/config.txt               |   12 +++++
 Documentation/urls.txt                 |   18 +++++++
 contrib/completion/git-completion.bash |    2 +-
 remote.c                               |   80 +++++++++++++++++++------------
 t/t5516-fetch-push.sh                  |   31 ++++++++++++
 5 files changed, 111 insertions(+), 32 deletions(-)
Josh Triplett· Sep 7, 2009, 07:00 UTC · re: Josh Triplett · lore

[PATCH 1/2] Wrap rewrite globals in a struct in preparation for adding another set

remote.c has a global set of URL rewrites, accessed by alias_url and make_rewrite. Wrap them in a new "struct rewrites", passed to alias_url and make_rewrite. This allows adding other sets of rewrites.

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
---
 remote.c |   53 ++++++++++++++++++++++++++++-------------------------
 1 files changed, 28 insertions(+), 25 deletions(-)
Show changes to remote.c +28 −25
diff --git a/remote.c b/remote.c
index 4b5b905..ff8e71f 100644
--- a/remote.c
+++ b/remote.c
@@ -28,6 +28,11 @@ struct rewrite {
 	int instead_of_nr;
 	int instead_of_alloc;
 };
+struct rewrites {
+	struct rewrite **rewrite;
+	int rewrite_alloc;
+	int rewrite_nr;
+};
 
 static struct remote **remotes;
 static int remotes_alloc;
@@ -41,14 +46,12 @@ static struct branch *current_branch;
 static const char *default_remote_name;
 static int explicit_default_remote_name;
 
-static struct rewrite **rewrite;
-static int rewrite_alloc;
-static int rewrite_nr;
+static struct rewrites rewrites;
 
 #define BUF_SIZE (2048)
 static char buffer[BUF_SIZE];
 
-static const char *alias_url(const char *url)
+static const char *alias_url(const char *url, struct rewrites *r)
 {
 	int i, j;
 	char *ret;
@@ -57,14 +60,14 @@ static const char *alias_url(const char *url)
 
 	longest = NULL;
 	longest_i = -1;
-	for (i = 0; i < rewrite_nr; i++) {
-		if (!rewrite[i])
+	for (i = 0; i < r->rewrite_nr; i++) {
+		if (!r->rewrite[i])
 			continue;
-		for (j = 0; j < rewrite[i]->instead_of_nr; j++) {
-			if (!prefixcmp(url, rewrite[i]->instead_of[j].s) &&
+		for (j = 0; j < r->rewrite[i]->instead_of_nr; j++) {
+			if (!prefixcmp(url, r->rewrite[i]->instead_of[j].s) &&
 			    (!longest ||
-			     longest->len < rewrite[i]->instead_of[j].len)) {
-				longest = &(rewrite[i]->instead_of[j]);
+			     longest->len < r->rewrite[i]->instead_of[j].len)) {
+				longest = &(r->rewrite[i]->instead_of[j]);
 				longest_i = i;
 			}
 		}
@@ -72,10 +75,10 @@ static const char *alias_url(const char *url)
 	if (!longest)
 		return url;
 
-	ret = xmalloc(rewrite[longest_i]->baselen +
+	ret = xmalloc(r->rewrite[longest_i]->baselen +
 		     (strlen(url) - longest->len) + 1);
-	strcpy(ret, rewrite[longest_i]->base);
-	strcpy(ret + rewrite[longest_i]->baselen, url + longest->len);
+	strcpy(ret, r->rewrite[longest_i]->base);
+	strcpy(ret + r->rewrite[longest_i]->baselen, url + longest->len);
 	return ret;
 }
 
@@ -103,7 +106,7 @@ static void add_url(struct remote *remote, const char *url)
 
 static void add_url_alias(struct remote *remote, const char *url)
 {
-	add_url(remote, alias_url(url));
+	add_url(remote, alias_url(url, &rewrites));
 }
 
 static void add_pushurl(struct remote *remote, const char *pushurl)
@@ -169,22 +172,22 @@ static struct branch *make_branch(const char *name, int len)
 	return ret;
 }
 
-static struct rewrite *make_rewrite(const char *base, int len)
+static struct rewrite *make_rewrite(struct rewrites *r, const char *base, int len)
 {
 	struct rewrite *ret;
 	int i;
 
-	for (i = 0; i < rewrite_nr; i++) {
+	for (i = 0; i < r->rewrite_nr; i++) {
 		if (len
-		    ? (len == rewrite[i]->baselen &&
-		       !strncmp(base, rewrite[i]->base, len))
-		    : !strcmp(base, rewrite[i]->base))
-			return rewrite[i];
+		    ? (len == r->rewrite[i]->baselen &&
+		       !strncmp(base, r->rewrite[i]->base, len))
+		    : !strcmp(base, r->rewrite[i]->base))
+			return r->rewrite[i];
 	}
 
-	ALLOC_GROW(rewrite, rewrite_nr + 1, rewrite_alloc);
+	ALLOC_GROW(r->rewrite, r->rewrite_nr + 1, r->rewrite_alloc);
 	ret = xcalloc(1, sizeof(struct rewrite));
-	rewrite[rewrite_nr++] = ret;
+	r->rewrite[r->rewrite_nr++] = ret;
 	if (len) {
 		ret->base = xstrndup(base, len);
 		ret->baselen = len;
@@ -355,7 +358,7 @@ static int handle_config(const char *key, const char *value, void *cb)
 		subkey = strrchr(name, '.');
 		if (!subkey)
 			return 0;
-		rewrite = make_rewrite(name, subkey - name);
+		rewrite = make_rewrite(&rewrites, name, subkey - name);
 		if (!strcmp(subkey, ".insteadof")) {
 			if (!value)
 				return config_error_nonbool(key);
@@ -433,10 +436,10 @@ static void alias_all_urls(void)
 		if (!remotes[i])
 			continue;
 		for (j = 0; j < remotes[i]->url_nr; j++) {
-			remotes[i]->url[j] = alias_url(remotes[i]->url[j]);
+			remotes[i]->url[j] = alias_url(remotes[i]->url[j], &rewrites);
 		}
 		for (j = 0; j < remotes[i]->pushurl_nr; j++) {
-			remotes[i]->pushurl[j] = alias_url(remotes[i]->pushurl[j]);
+			remotes[i]->pushurl[j] = alias_url(remotes[i]->pushurl[j], &rewrites);
 		}
 	}
 }
-- 
1.6.3.3
Josh Triplett· Sep 7, 2009, 07:00 UTC · re: Josh Triplett · lore

[PATCH 2/2] Add url.<base>.pushInsteadOf: URL rewriting for push only

This configuration option allows systematically rewriting fetch-only URLs to push-capable URLs when used with push. For instance:

[url "ssh://example.org/"]
    pushInsteadOf = "git://example.org/"

This will allow clones of "git://example.org/path/to/repo" to subsequently push to "ssh://example.org/path/to/repo", without manually configuring pushurl for that remote.

Includes documentation for the new option, bash completion updates, and test cases (both that pushInsteadOf applies to push and that it does *not* apply to fetch).

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
---
 Documentation/config.txt               |   12 +++++++++++
 Documentation/urls.txt                 |   18 +++++++++++++++++
 contrib/completion/git-completion.bash |    2 +-
 remote.c                               |   33 +++++++++++++++++++++++--------
 t/t5516-fetch-push.sh                  |   31 ++++++++++++++++++++++++++++++
 5 files changed, 86 insertions(+), 10 deletions(-)
Show changes to 5 files +86 −10

Documentation/config.txt, Documentation/urls.txt, contrib/completion/git-completion.bash, remote.c, t/t5516-fetch-push.sh

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 5256c7f..726aa89 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1500,6 +1500,18 @@ url.<base>.insteadOf::
 	never-before-seen repository on the site.  When more than one
 	insteadOf strings match a given URL, the longest match is used.
 
+url.<base>.pushInsteadOf::
+	Any URL that starts with this value will not be pushed to;
+	instead, it will be rewritten to start with <base>, and the
+	resulting URL will be pushed to. In cases where some site serves
+	a large number of repositories, and serves them with multiple
+	access methods, some of which do not allow push, this feature
+	allows people to specify a pull-only URL and have git
+	automatically use an appropriate URL to push, even for a
+	never-before-seen repository on the site.  When more than one
+	pushInsteadOf strings match a given URL, the longest match is
+	used.
+
 user.email::
 	Your email address to be recorded in any newly created commits.
 	Can be overridden by the 'GIT_AUTHOR_EMAIL', 'GIT_COMMITTER_EMAIL', and
diff --git a/Documentation/urls.txt b/Documentation/urls.txt
index 5355ebc..d813ceb 100644
--- a/Documentation/urls.txt
+++ b/Documentation/urls.txt
@@ -67,3 +67,21 @@ For example, with this:
 a URL like "work:repo.git" or like "host.xz:/path/to/repo.git" will be
 rewritten in any context that takes a URL to be "git://git.host.xz/repo.git".
 
+If you want to rewrite URLs for push only, you can create a
+configuration section of the form:
+
+------------
+	[url "<actual url base>"]
+		pushInsteadOf = <other url base>
+------------
+
+For example, with this:
+
+------------
+	[url "ssh://example.org/"]
+		pushInsteadOf = git://example.org/
+------------
+
+a URL like "git://example.org/path/to/repo.git" will be rewritten to
+"ssh://example.org/path/to/repo.git" for pushes, but pulls will still
+use the original URL.
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index bf688e1..9859204 100755
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1532,7 +1532,7 @@ _git_config ()
 	url.*.*)
 		local pfx="${cur%.*}."
 		cur="${cur##*.}"
-		__gitcomp "insteadof" "$pfx" "$cur"
+		__gitcomp "insteadOf pushInsteadOf" "$pfx" "$cur"
 		return
 		;;
 	esac
diff --git a/remote.c b/remote.c
index ff8e71f..6789786 100644
--- a/remote.c
+++ b/remote.c
@@ -47,6 +47,7 @@ static const char *default_remote_name;
 static int explicit_default_remote_name;
 
 static struct rewrites rewrites;
+static struct rewrites rewrites_push;
 
 #define BUF_SIZE (2048)
 static char buffer[BUF_SIZE];
@@ -104,17 +105,25 @@ static void add_url(struct remote *remote, const char *url)
 	remote->url[remote->url_nr++] = url;
 }
 
-static void add_url_alias(struct remote *remote, const char *url)
-{
-	add_url(remote, alias_url(url, &rewrites));
-}
-
 static void add_pushurl(struct remote *remote, const char *pushurl)
 {
 	ALLOC_GROW(remote->pushurl, remote->pushurl_nr + 1, remote->pushurl_alloc);
 	remote->pushurl[remote->pushurl_nr++] = pushurl;
 }
 
+static void add_pushurl_alias(struct remote *remote, const char *url)
+{
+	const char *pushurl = alias_url(url, &rewrites_push);
+	if (pushurl != url)
+		add_pushurl(remote, pushurl);
+}
+
+static void add_url_alias(struct remote *remote, const char *url)
+{
+	add_url(remote, alias_url(url, &rewrites));
+	add_pushurl_alias(remote, url);
+}
+
 static struct remote *make_remote(const char *name, int len)
 {
 	struct remote *ret;
@@ -358,8 +367,13 @@ static int handle_config(const char *key, const char *value, void *cb)
 		subkey = strrchr(name, '.');
 		if (!subkey)
 			return 0;
-		rewrite = make_rewrite(&rewrites, name, subkey - name);
 		if (!strcmp(subkey, ".insteadof")) {
+			rewrite = make_rewrite(&rewrites, name, subkey - name);
+			if (!value)
+				return config_error_nonbool(key);
+			add_instead_of(rewrite, xstrdup(value));
+		} else if (!strcmp(subkey, ".pushinsteadof")) {
+			rewrite = make_rewrite(&rewrites_push, name, subkey - name);
 			if (!value)
 				return config_error_nonbool(key);
 			add_instead_of(rewrite, xstrdup(value));
@@ -435,12 +449,13 @@ static void alias_all_urls(void)
 	for (i = 0; i < remotes_nr; i++) {
 		if (!remotes[i])
 			continue;
-		for (j = 0; j < remotes[i]->url_nr; j++) {
-			remotes[i]->url[j] = alias_url(remotes[i]->url[j], &rewrites);
-		}
 		for (j = 0; j < remotes[i]->pushurl_nr; j++) {
 			remotes[i]->pushurl[j] = alias_url(remotes[i]->pushurl[j], &rewrites);
 		}
+		for (j = 0; j < remotes[i]->url_nr; j++) {
+			remotes[i]->url[j] = alias_url(remotes[i]->url[j], &rewrites);
+			add_pushurl_alias(remotes[i], remotes[i]->url[j]);
+		}
 	}
 }
 
diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh
index 2d2633f..8f455c7 100755
--- a/t/t5516-fetch-push.sh
+++ b/t/t5516-fetch-push.sh
@@ -122,6 +122,23 @@ test_expect_success 'fetch with insteadOf' '
 	)
 '
 
+test_expect_success 'fetch with pushInsteadOf (should not rewrite)' '
+	mk_empty &&
+	(
+		TRASH=$(pwd)/ &&
+		cd testrepo &&
+		git config "url.trash/.pushInsteadOf" "$TRASH" &&
+		git config remote.up.url "$TRASH." &&
+		git config remote.up.fetch "refs/heads/*:refs/remotes/origin/*" &&
+		git fetch up &&
+
+		r=$(git show-ref -s --verify refs/remotes/origin/master) &&
+		test "z$r" = "z$the_commit" &&
+
+		test 1 = $(git for-each-ref refs/remotes/origin | wc -l)
+	)
+'
+
 test_expect_success 'push without wildcard' '
 	mk_empty &&
 
@@ -162,6 +179,20 @@ test_expect_success 'push with insteadOf' '
 	)
 '
 
+test_expect_success 'push with pushInsteadOf' '
+	mk_empty &&
+	TRASH="$(pwd)/" &&
+	git config "url.$TRASH.pushInsteadOf" trash/ &&
+	git push trash/testrepo refs/heads/master:refs/remotes/origin/master &&
+	(
+		cd testrepo &&
+		r=$(git show-ref -s --verify refs/remotes/origin/master) &&
+		test "z$r" = "z$the_commit" &&
+
+		test 1 = $(git for-each-ref refs/remotes/origin | wc -l)
+	)
+'
+
 test_expect_success 'push with matching heads' '
 
 	mk_test heads/master &&
-- 
1.6.3.3
Junio C Hamano· Sep 7, 2009, 07:53 UTC · re: Josh Triplett · lore

Re: [PATCH 2/2] Add url.<base>.pushInsteadOf: URL rewriting for push only

Josh Triplett <josh@joshtriplett.org> writes:
Show 9 quoted lines
> This configuration option allows systematically rewriting fetch-only
> URLs to push-capable URLs when used with push.  For instance:
>
> [url "ssh://example.org/"]
>     pushInsteadOf = "git://example.org/"
>
> This will allow clones of "git://example.org/path/to/repo" to
> subsequently push to "ssh://example.org/path/to/repo", without manually
> configuring pushurl for that remote.
Nice.
Show 14 quoted lines
> @@ -435,12 +449,13 @@ static void alias_all_urls(void)
>  	for (i = 0; i < remotes_nr; i++) {
>  		if (!remotes[i])
>  			continue;
> -		for (j = 0; j < remotes[i]->url_nr; j++) {
> -			remotes[i]->url[j] = alias_url(remotes[i]->url[j], &rewrites);
> -		}
>  		for (j = 0; j < remotes[i]->pushurl_nr; j++) {
>  			remotes[i]->pushurl[j] = alias_url(remotes[i]->pushurl[j], &rewrites);
>  		}
> +		for (j = 0; j < remotes[i]->url_nr; j++) {
> +			remotes[i]->url[j] = alias_url(remotes[i]->url[j], &rewrites);
> +			add_pushurl_alias(remotes[i], remotes[i]->url[j]);
> +		}

Even if you have URL but not pushURL, now you get a corresponding pushURL for free by just adding pushinsteadof mapping that covers the URL without having to configue pushURL for each of them.

What happens if you already had a pair of concrete url and pushurl defined for one of your repositories (say git://git.kernel.org/pub/scm/git/git.git for fetch, ssh://x.kernel.org/pub/scm/git/git.git for push) at a site, and then upon seeing this new feature, added a pushinsteadof pattern that also covers the URL side of that pair (e.g. everything in git://git.kernel.org/ is mapped to x.kernel.org:/ namespsace)?

Do you end up pushing to both (e.g. ssh://x.kernel.org/pub/scm/git/git.git and x.kernel.org:/pub/scm/git/git.git), or in such a case, the pushURL you gave explicitly prevents the pushinsteadof to give unexpected duplicates?

Josh Triplett· Sep 7, 2009, 08:19 UTC · re: Junio C Hamano · lore

Re: [PATCH 2/2] Add url.<base>.pushInsteadOf: URL rewriting for push only

On Mon, Sep 07, 2009 at 12:53:18AM -0700, Junio C Hamano wrote:
Show 12 quoted lines
> Josh Triplett <josh@joshtriplett.org> writes:
> > This configuration option allows systematically rewriting fetch-only
> > URLs to push-capable URLs when used with push.  For instance:
> >
> > [url "ssh://example.org/"]
> >     pushInsteadOf = "git://example.org/"
> >
> > This will allow clones of "git://example.org/path/to/repo" to
> > subsequently push to "ssh://example.org/path/to/repo", without manually
> > configuring pushurl for that remote.
> 
> Nice.
Thanks.
Show 18 quoted lines
> > @@ -435,12 +449,13 @@ static void alias_all_urls(void)
> >  	for (i = 0; i < remotes_nr; i++) {
> >  		if (!remotes[i])
> >  			continue;
> > -		for (j = 0; j < remotes[i]->url_nr; j++) {
> > -			remotes[i]->url[j] = alias_url(remotes[i]->url[j], &rewrites);
> > -		}
> >  		for (j = 0; j < remotes[i]->pushurl_nr; j++) {
> >  			remotes[i]->pushurl[j] = alias_url(remotes[i]->pushurl[j], &rewrites);
> >  		}
> > +		for (j = 0; j < remotes[i]->url_nr; j++) {
> > +			remotes[i]->url[j] = alias_url(remotes[i]->url[j], &rewrites);
> > +			add_pushurl_alias(remotes[i], remotes[i]->url[j]);
> > +		}
> 
> Even if you have URL but not pushURL, now you get a corresponding pushURL
> for free by just adding pushinsteadof mapping that covers the URL without
> having to configue pushURL for each of them.
Yes, exactly.
Show 10 quoted lines
> What happens if you already had a pair of concrete url and pushurl defined
> for one of your repositories (say git://git.kernel.org/pub/scm/git/git.git
> for fetch, ssh://x.kernel.org/pub/scm/git/git.git for push) at a site, and
> then upon seeing this new feature, added a pushinsteadof pattern that also
> covers the URL side of that pair (e.g. everything in git://git.kernel.org/
> is mapped to x.kernel.org:/ namespsace)?
> 
> Do you end up pushing to both (e.g. ssh://x.kernel.org/pub/scm/git/git.git
> and x.kernel.org:/pub/scm/git/git.git), or in such a case, the pushURL you
> gave explicitly prevents the pushinsteadof to give unexpected duplicates?
You get a duplicate:
~$ grep -B1 steadOf .gitconfig
[url "ssh://joshtriplett.org/"]
        pushInsteadOf="git://joshtriplett.org/"
~$ grep -B1 url .git/config
[remote "origin"]
        url = git://joshtriplett.org/git/home.git
        pushurl = ssh://joshtriplett.org/git/home.git
~$ ~/src/git/git push
Everything up-to-date
Everything up-to-date

Initially, that behavior seemed pretty reasonable to me; nothing else in the remotes handling attempts to remove duplicates, and they seem harmless enough and easily resolved by removing one or the other. Now that I think about it, though, an explicit pushurl should definitely disable pushInsteadOf's implicit pushurls. If you explicitly configure a *different* pushurl for a remote, you may not *want* the default pushurl that corresponds to your url. For instance, consider what would happen if you configure url to point to the main public repository and pushurl to point to a private repository. In this case, you definitely don't want "git push" to helpfully push to the public repository as well; if you do, you can easily enough add a second pushurl for that.

I can easily change the patch to make an explicit pushurl disable pushInsteadOf. Expect v2 shortly.

- Josh Triplett

← back to recent threads