threads / patch / 12481

v2, 3 partsTeach git-fetch to exploit server side automatic tag following

Subject: [PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

## tl;dr

5 messages between Mar 4, 2008 and Mar 6, 2008. Diffs are folded; open one to read it.

replies: 4people: 2as markdown or json

Shawn O. Pearce· Mar 4, 2008, 03:27 UTC · lore

If the remote peer upload-pack process supports the auto-include-tag protocol extension then we can avoid running a second fetch cycle on the client side by letting the server send us the annotated tags along with the objects it is packing for us. In the following graph we can now fetch both "tag1" and "tag2" on the same connection that we fetched "master" from the remote when we only have L available on the local side:

         T - tag1          S - tag2
        /                 /
   L - o ------ o ------ B
    \                     \
     \                     \
      origin/master         master

The objects for "tag1" are implicitly downloaded without our direct knowledge. The existing "quickfetch" optimization within git-fetch discovers that tag1 is complete after the first connection and does not open a second connection.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 builtin-fetch.c      |    3 +++
 t/t5503-tagfollow.sh |   26 ++++++++++++++++++++++++++
 transport.c          |    5 +++++
 transport.h          |    3 +++
 4 files changed, 37 insertions(+), 0 deletions(-)
Show changes to 4 files +37 −0

builtin-fetch.c, t/t5503-tagfollow.sh, transport.c, transport.h

diff --git a/builtin-fetch.c b/builtin-fetch.c
index 26c3d74..55f611e 100644
--- a/builtin-fetch.c
+++ b/builtin-fetch.c
@@ -555,6 +555,8 @@ static int do_fetch(struct transport *transport,
 			read_ref(rm->peer_ref->name, rm->peer_ref->old_sha1);
 	}
 
+	if (tags == TAGS_DEFAULT && autotags)
+		transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 	if (fetch_refs(transport, ref_map)) {
 		free_refs(ref_map);
 		return 1;
@@ -568,6 +570,7 @@ static int do_fetch(struct transport *transport,
 		ref_map = NULL;
 		find_non_local_tags(transport, &ref_map, &tail);
 		if (ref_map) {
+			transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, NULL);
 			transport_set_option(transport, TRANS_OPT_DEPTH, "0");
 			fetch_refs(transport, ref_map);
 		}
diff --git a/t/t5503-tagfollow.sh b/t/t5503-tagfollow.sh
index 45ff982..86e5b9b 100755
--- a/t/t5503-tagfollow.sh
+++ b/t/t5503-tagfollow.sh
@@ -121,4 +121,30 @@ test_expect_success 'fetch B, S (commit and tag : 1 connection)' '
 	git diff expect actual
 '
 
+cat - <<EOF >expect
+#S
+want $B
+want $S
+#E
+EOF
+test_expect_success 'new clone fetch master and tags' '
+	git branch -D cat
+	rm -f $U
+	(
+		mkdir clone2 &&
+		cd clone2 &&
+		git init &&
+		git remote add origin .. &&
+		GIT_DEBUG_SEND_PACK=3 git fetch 3>../$U &&
+		test $B = $(git rev-parse --verify origin/master) &&
+		test $S = $(git rev-parse --verify tag2) &&
+		test $B = $(git rev-parse --verify tag2^0) &&
+		test $T = $(git rev-parse --verify tag1) &&
+		test $A = $(git rev-parse --verify tag1^0)
+	) &&
+	test -s $U &&
+	cut -d" " -f1,2 $U >actual &&
+	git diff expect actual
+'
+
 test_done
diff --git a/transport.c b/transport.c
index 166c1d1..12873db 100644
--- a/transport.c
+++ b/transport.c
@@ -560,6 +560,7 @@ static int close_bundle(struct transport *transport)
 struct git_transport_data {
 	unsigned thin : 1;
 	unsigned keep : 1;
+	unsigned followtags : 1;
 	int depth;
 	struct child_process *conn;
 	int fd[2];
@@ -580,6 +581,9 @@ static int set_git_option(struct transport *connection,
 	} else if (!strcmp(name, TRANS_OPT_THIN)) {
 		data->thin = !!value;
 		return 0;
+	} else if (!strcmp(name, TRANS_OPT_FOLLOWTAGS)) {
+		data->followtags = !!value;
+		return 0;
 	} else if (!strcmp(name, TRANS_OPT_KEEP)) {
 		data->keep = !!value;
 		return 0;
@@ -628,6 +632,7 @@ static int fetch_refs_via_pack(struct transport *transport,
 	args.keep_pack = data->keep;
 	args.lock_pack = 1;
 	args.use_thin_pack = data->thin;
+	args.auto_include_tag = data->followtags;
 	args.verbose = transport->verbose > 0;
 	args.depth = data->depth;
 
diff --git a/transport.h b/transport.h
index 6fb4526..8abfc0a 100644
--- a/transport.h
+++ b/transport.h
@@ -53,6 +53,9 @@ struct transport *transport_get(struct remote *, const char *);
 /* Limit the depth of the fetch if not null */
 #define TRANS_OPT_DEPTH "depth"
 
+/* Aggressively fetch annotated tags if possible */
+#define TRANS_OPT_FOLLOWTAGS "followtags"
+
 /**
  * Returns 0 if the option was used, non-zero otherwise. Prints a
  * message to stderr if the option is not used.
-- 
1.5.4.3.529.gb25fb
Shawn O. Pearce· Mar 5, 2008, 05:56 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> just two comments from me for this patch series:
> 
> - it's a beaut, and
Thanks.
 
> - should followtags not be the default?
No.  Absolutely not.

The client may not want tag objects from this remote. It might not want those tags for all sorts of reasons. Maybe they are doing a one-shot pull. Maybe they don't trust this remote. Maybe they trust this remote but this remote is famous for 192M tags containing PDFs of photo images of build logs printed out on paper, then photographed on a wood table and finally scanned in at 600 dpi.

So it should only be enabled *if* we were willing to open a second connection to this remote to followtags from it. And that's not a change from prior behavior either, its just faster.

-- 
Shawn.
Johannes Schindelin· Mar 5, 2008, 11:21 UTC · re: Shawn O. Pearce · lore

Re: [PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

Hi,
On Wed, 5 Mar 2008, Shawn O. Pearce wrote:
Show 12 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>  
> > - should followtags not be the default?
> 
> No.  Absolutely not.
> 
> The client may not want tag objects from this remote.  It might not want 
> those tags for all sorts of reasons.  Maybe they are doing a one-shot 
> pull.  Maybe they don't trust this remote.  Maybe they trust this remote 
> but this remote is famous for 192M tags containing PDFs of photo images 
> of build logs printed out on paper, then photographed on a wood table 
> and finally scanned in at 600 dpi.

Sorry, I meant to say: "should the followtags feature not be on by default in the circumstances where we would follow tags anyway"...

Maybe you do that, but I did not see it.
> So it should only be enabled *if* we were willing to open a second 
> connection to this remote to followtags from it.  And that's not a 
> change from prior behavior either, its just faster.

Thanks, Dscho

Shawn O. Pearce· Mar 6, 2008, 04:53 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
Show 18 quoted lines
> On Wed, 5 Mar 2008, Shawn O. Pearce wrote:
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> >  
> > > - should followtags not be the default?
> > 
> > No.  Absolutely not.
> > 
> > The client may not want tag objects from this remote.  It might not want 
> > those tags for all sorts of reasons.  Maybe they are doing a one-shot 
> > pull.  Maybe they don't trust this remote.  Maybe they trust this remote 
> > but this remote is famous for 192M tags containing PDFs of photo images 
> > of build logs printed out on paper, then photographed on a wood table 
> > and finally scanned in at 600 dpi.
> 
> Sorry, I meant to say: "should the followtags feature not be on by default 
> in the circumstances where we would follow tags anyway"...
> 
> Maybe you do that, but I did not see it.

Nope. followtags should be on if the client wants tags, and off if the client does not want tags. Its that simple. There's no reason for the client to disable followtags if the server will actually support it, as disabling it when you do have tags to follow will only cost you extra net latency.

-- 
Shawn.

← back to recent threads