threads / patch / 33530

v2, 6 partstransport-helper: some clarifications and a fix

Subject: [PATCH v2 0/6] transport-helper: some clarifications and a fix

## tl;dr

26 messages between Apr 18, 2013 and Apr 19, 2013. Diffs are folded; open one to read it.

replies: 25people: 4as markdown or json

Felipe Contreras· Apr 18, 2013, 04:14 UTC · lore
Hi,
Same as before, but with a small bug fix, and minor test improvements.

It seems the workings of transport-helper are anything but clear, so let's try to clarify them a bit, and after that, hopefully it would become clearer why the last patch is actually a good fix.

Felipe Contreras (6):
  transport-helper: clarify *:* refspec
  transport-helper: update refspec documentation
  transport-helper: clarify pushing without refspecs
  transport-helper: warn when refspec is not used
  transport-helper: trivial code shuffle
  transport-helper: update remote helper namespace
 Documentation/gitremote-helpers.txt | 12 +++++------
 t/t5801-remote-helpers.sh           | 39 ++++++++++++++++++------------------
 transport-helper.c                  | 40 ++++++++++++++++++++++++++-----------
 3 files changed, 53 insertions(+), 38 deletions(-)
-- 
1.8.2.1.679.g509521a
Felipe Contreras· Apr 18, 2013, 04:14 UTC · re: Felipe Contreras · lore

[PATCH v2 1/6] transport-helper: clarify *:* refspec

The *:* refspec doesn't work, and never has, clarify the code and documentation to reflect that. This in effect reverts commit 9e7673e (gitremote-helpers(1): clarify refspec behaviour).

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 Documentation/gitremote-helpers.txt |  4 ++--
 t/t5801-remote-helpers.sh           | 15 ---------------
 transport-helper.c                  |  2 +-
 3 files changed, 3 insertions(+), 18 deletions(-)
Show changes to 3 files +3 −18

Documentation/gitremote-helpers.txt, t/t5801-remote-helpers.sh, transport-helper.c

diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
index f506031..0c91aba 100644
--- a/Documentation/gitremote-helpers.txt
+++ b/Documentation/gitremote-helpers.txt
@@ -174,8 +174,8 @@ ref.
 This capability can be advertised multiple times.  The first
 applicable refspec takes precedence.  The left-hand of refspecs
 advertised with this capability must cover all refs reported by
-the list command.  If a helper does not need a specific 'refspec'
-capability then it should advertise `refspec *:*`.
+the list command.  If no 'refspec' capability is advertised,
+there is an implied `refspec *:*`.
 
 'bidi-import'::
 	This modifies the 'import' capability.
diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
index f387027..cd1873c 100755
--- a/t/t5801-remote-helpers.sh
+++ b/t/t5801-remote-helpers.sh
@@ -120,21 +120,6 @@ test_expect_failure 'pushing without refspecs' '
 	compare_refs local2 HEAD server HEAD
 '
 
-test_expect_success 'pulling with straight refspec' '
-	(cd local2 &&
-	GIT_REMOTE_TESTGIT_REFSPEC="*:*" git pull) &&
-	compare_refs local2 HEAD server HEAD
-'
-
-test_expect_failure 'pushing with straight refspec' '
-	test_when_finished "(cd local2 && git reset --hard origin)" &&
-	(cd local2 &&
-	echo content >>file &&
-	git commit -a -m eleven &&
-	GIT_REMOTE_TESTGIT_REFSPEC="*:*" git push) &&
-	compare_refs local2 HEAD server HEAD
-'
-
 test_expect_success 'pulling without marks' '
 	(cd local2 &&
 	GIT_REMOTE_TESTGIT_NO_MARKS=1 git pull) &&
diff --git a/transport-helper.c b/transport-helper.c
index dcd8d97..cea787c 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -469,7 +469,7 @@ static int fetch_with_import(struct transport *transport,
 	 * were fetching.
 	 *
 	 * (If no "refspec" capability was specified, for historical
-	 * reasons we default to *:*.)
+	 * reasons we default to the equivalent of *:*.)
 	 *
 	 * Store the result in to_fetch[i].old_sha1.  Callers such
 	 * as "git fetch" can use the value to write feedback to the
-- 
1.8.2.1.679.g509521a
John Keeping· Apr 18, 2013, 08:24 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 1/6] transport-helper: clarify *:* refspec

On Wed, Apr 17, 2013 at 11:14:28PM -0500, Felipe Contreras wrote:
> The *:* refspec doesn't work, and never has, clarify the code and
> documentation to reflect that. This in effect reverts commit 9e7673e
> (gitremote-helpers(1): clarify refspec behaviour).

In what way doesn't it work? If I specify that refspec then I do get output that appears sensible.

Show 19 quoted lines
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> ---
>  Documentation/gitremote-helpers.txt |  4 ++--
>  t/t5801-remote-helpers.sh           | 15 ---------------
>  transport-helper.c                  |  2 +-
>  3 files changed, 3 insertions(+), 18 deletions(-)
> 
> diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
> index f506031..0c91aba 100644
> --- a/Documentation/gitremote-helpers.txt
> +++ b/Documentation/gitremote-helpers.txt
> @@ -174,8 +174,8 @@ ref.
>  This capability can be advertised multiple times.  The first
>  applicable refspec takes precedence.  The left-hand of refspecs
>  advertised with this capability must cover all refs reported by
> -the list command.  If a helper does not need a specific 'refspec'
> -capability then it should advertise `refspec *:*`.
> +the list command.  If no 'refspec' capability is advertised,
> +there is an implied `refspec *:*`.

This is wrong. As your later patch makes clearer, there is no implied refspec for push - it only works for fetch. I found the wording you've reverted to extremely misleading. How about something like this:

    For historical reasons, 'import' treats the absence of a 'refspec'
    line as equivalent to `refspec *:*`; remote helpers should always
    specify an explicit refspec.
?
Show 43 quoted lines
>  'bidi-import'::
>  	This modifies the 'import' capability.
> diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
> index f387027..cd1873c 100755
> --- a/t/t5801-remote-helpers.sh
> +++ b/t/t5801-remote-helpers.sh
> @@ -120,21 +120,6 @@ test_expect_failure 'pushing without refspecs' '
>  	compare_refs local2 HEAD server HEAD
>  '
>  
> -test_expect_success 'pulling with straight refspec' '
> -	(cd local2 &&
> -	GIT_REMOTE_TESTGIT_REFSPEC="*:*" git pull) &&
> -	compare_refs local2 HEAD server HEAD
> -'
> -
> -test_expect_failure 'pushing with straight refspec' '
> -	test_when_finished "(cd local2 && git reset --hard origin)" &&
> -	(cd local2 &&
> -	echo content >>file &&
> -	git commit -a -m eleven &&
> -	GIT_REMOTE_TESTGIT_REFSPEC="*:*" git push) &&
> -	compare_refs local2 HEAD server HEAD
> -'
> -
>  test_expect_success 'pulling without marks' '
>  	(cd local2 &&
>  	GIT_REMOTE_TESTGIT_NO_MARKS=1 git pull) &&
> diff --git a/transport-helper.c b/transport-helper.c
> index dcd8d97..cea787c 100644
> --- a/transport-helper.c
> +++ b/transport-helper.c
> @@ -469,7 +469,7 @@ static int fetch_with_import(struct transport *transport,
>  	 * were fetching.
>  	 *
>  	 * (If no "refspec" capability was specified, for historical
> -	 * reasons we default to *:*.)
> +	 * reasons we default to the equivalent of *:*.)
>  	 *
>  	 * Store the result in to_fetch[i].old_sha1.  Callers such
>  	 * as "git fetch" can use the value to write feedback to the
> -- 
> 1.8.2.1.679.g509521a
Felipe Contreras· Apr 18, 2013, 09:27 UTC · re: John Keeping · lore

Re: [PATCH v2 1/6] transport-helper: clarify *:* refspec

On Thu, Apr 18, 2013 at 3:24 AM, John Keeping <john@keeping.me.uk> wrote:
Show 7 quoted lines
> On Wed, Apr 17, 2013 at 11:14:28PM -0500, Felipe Contreras wrote:
>> The *:* refspec doesn't work, and never has, clarify the code and
>> documentation to reflect that. This in effect reverts commit 9e7673e
>> (gitremote-helpers(1): clarify refspec behaviour).
>
> In what way doesn't it work?  If I specify that refspec then I do get
> output that appears sensible.

% git checkout origin/master % make -C t t5801-remote-helpers.sh

not ok 15 - pushing with straight refspec # TODO known breakage
It fails for me here.
Show 19 quoted lines
>>  Documentation/gitremote-helpers.txt |  4 ++--
>>  t/t5801-remote-helpers.sh           | 15 ---------------
>>  transport-helper.c                  |  2 +-
>>  3 files changed, 3 insertions(+), 18 deletions(-)
>>
>> diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
>> index f506031..0c91aba 100644
>> --- a/Documentation/gitremote-helpers.txt
>> +++ b/Documentation/gitremote-helpers.txt
>> @@ -174,8 +174,8 @@ ref.
>>  This capability can be advertised multiple times.  The first
>>  applicable refspec takes precedence.  The left-hand of refspecs
>>  advertised with this capability must cover all refs reported by
>> -the list command.  If a helper does not need a specific 'refspec'
>> -capability then it should advertise `refspec *:*`.
>> +the list command.  If no 'refspec' capability is advertised,
>> +there is an implied `refspec *:*`.
>
> This is wrong.
It has been like that since v1.7.0.
Show 7 quoted lines
> As your later patch makes clearer, there is no implied
> refspec for push - it only works for fetch.  I found the wording you've
> reverted to extremely misleading.  How about something like this:
>
>     For historical reasons, 'import' treats the absence of a 'refspec'
>     line as equivalent to `refspec *:*`; remote helpers should always
>     specify an explicit refspec.

Maybe my version was "misleading" because it didn't mention it was for historical reasons, but yours is plain wrong; remote helpers might not be using 'import' or 'export' at all, so not all remote helpers should always specify an explicit refspec. And if the previous text "It is recommended that all importers providing the 'import' capability use this. It's mandatory for 'export'." does not convey the idea these remote helpers should always specify an explicit refspec, I don't know what it does.

-- 
Felipe Contreras
John Keeping· Apr 18, 2013, 09:45 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 1/6] transport-helper: clarify *:* refspec

On Thu, Apr 18, 2013 at 04:27:57AM -0500, Felipe Contreras wrote:
Show 18 quoted lines
> On Thu, Apr 18, 2013 at 3:24 AM, John Keeping <john@keeping.me.uk> wrote:
> > On Wed, Apr 17, 2013 at 11:14:28PM -0500, Felipe Contreras wrote:
> >> diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
> >> index f506031..0c91aba 100644
> >> --- a/Documentation/gitremote-helpers.txt
> >> +++ b/Documentation/gitremote-helpers.txt
> >> @@ -174,8 +174,8 @@ ref.
> >>  This capability can be advertised multiple times.  The first
> >>  applicable refspec takes precedence.  The left-hand of refspecs
> >>  advertised with this capability must cover all refs reported by
> >> -the list command.  If a helper does not need a specific 'refspec'
> >> -capability then it should advertise `refspec *:*`.
> >> +the list command.  If no 'refspec' capability is advertised,
> >> +there is an implied `refspec *:*`.
> >
> > This is wrong.
> 
> It has been like that since v1.7.0.

That doesn't mean it's right. I suspect that it was originally correct, but then 'export' was added at which point this became stale.

Show 16 quoted lines
> > As your later patch makes clearer, there is no implied
> > refspec for push - it only works for fetch.  I found the wording you've
> > reverted to extremely misleading.  How about something like this:
> >
> >     For historical reasons, 'import' treats the absence of a 'refspec'
> >     line as equivalent to `refspec *:*`; remote helpers should always
> >     specify an explicit refspec.
> 
> Maybe my version was "misleading" because it didn't mention it was for
> historical reasons, but yours is plain wrong; remote helpers might not
> be using 'import' or 'export' at all, so not all remote helpers should
> always specify an explicit refspec. And if the previous text "It is
> recommended that all importers providing the 'import' capability use
> this. It's mandatory for 'export'." does not convey the idea these
> remote helpers should always specify an explicit refspec, I don't know
> what it does.

I didn't say mine was correct, but there was a reason that I wanted to change the existing text. Just going back to what was there before is not a good way to make things better.

In my copy of pu I don't see the text "It's mandatory for 'export'", it just stops after "It is recommended that all importers providing the 'import' capability use this". That paragraph also starts with "This modifies the 'import' capability" making no mention of export.

Perhaps we need a slightly more extensive re-write of the documentation for refspec to make it very clear where it applies and when it is needed.

Felipe Contreras· Apr 18, 2013, 10:02 UTC · re: John Keeping · lore

Re: [PATCH v2 1/6] transport-helper: clarify *:* refspec

On Thu, Apr 18, 2013 at 4:45 AM, John Keeping <john@keeping.me.uk> wrote:
> On Thu, Apr 18, 2013 at 04:27:57AM -0500, Felipe Contreras wrote:
Show 12 quoted lines
>> Maybe my version was "misleading" because it didn't mention it was for
>> historical reasons, but yours is plain wrong; remote helpers might not
>> be using 'import' or 'export' at all, so not all remote helpers should
>> always specify an explicit refspec. And if the previous text "It is
>> recommended that all importers providing the 'import' capability use
>> this. It's mandatory for 'export'." does not convey the idea these
>> remote helpers should always specify an explicit refspec, I don't know
>> what it does.
>
> I didn't say mine was correct, but there was a reason that I wanted to
> change the existing text.  Just going back to what was there before is
> not a good way to make things better.

And just because it was that way before doesn't mean it was worst. Your patch essentially switched from "it is implied", to "it should be explicit", which is wrong. Going back to the previous text is the simplest change that restores a reasonable explanation. The next patches in this series deal with the rest of the issues in this text.

> In my copy of pu I don't see the text "It's mandatory for 'export'", it
> just stops after "It is recommended that all importers providing the
> 'import' capability use this".  That paragraph also starts with "This
> modifies the 'import' capability" making no mention of export.
This is patch 1 of 6, keep going.
-- 
Felipe Contreras
John Keeping· Apr 18, 2013, 10:06 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 1/6] transport-helper: clarify *:* refspec

On Thu, Apr 18, 2013 at 05:02:11AM -0500, Felipe Contreras wrote:
Show 7 quoted lines
> On Thu, Apr 18, 2013 at 4:45 AM, John Keeping <john@keeping.me.uk> wrote:
> > In my copy of pu I don't see the text "It's mandatory for 'export'", it
> > just stops after "It is recommended that all importers providing the
> > 'import' capability use this".  That paragraph also starts with "This
> > modifies the 'import' capability" making no mention of export.
> 
> This is patch 1 of 6, keep going.

Ah, I see now. I had read all of the patches initially, but then focused on this one for discussion.

The end result after this series does look good, but I find it slightly strange to take a step backwards on the way.

Junio C Hamano· Apr 18, 2013, 17:27 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 1/6] transport-helper: clarify *:* refspec

Felipe Contreras <felipe.contreras@gmail.com> writes:
Show 10 quoted lines
> The *:* refspec doesn't work, and never has, clarify the code and
> documentation to reflect that. This in effect reverts commit 9e7673e
> (gitremote-helpers(1): clarify refspec behaviour).
> ...
>  applicable refspec takes precedence.  The left-hand of refspecs
>  advertised with this capability must cover all refs reported by
> -the list command.  If a helper does not need a specific 'refspec'
> -capability then it should advertise `refspec *:*`.
> +the list command.  If no 'refspec' capability is advertised,
> +there is an implied `refspec *:*`.
I presume
    s/.$/, but `*:*` does not work so do not use it./
is implied?
Show 13 quoted lines
> diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
> index f387027..cd1873c 100755
> --- a/t/t5801-remote-helpers.sh
> +++ b/t/t5801-remote-helpers.sh
> @@ -120,21 +120,6 @@ test_expect_failure 'pushing without refspecs' '
>  	compare_refs local2 HEAD server HEAD
>  '
>  
> -test_expect_success 'pulling with straight refspec' '
> -	(cd local2 &&
> -	GIT_REMOTE_TESTGIT_REFSPEC="*:*" git pull) &&
> -	compare_refs local2 HEAD server HEAD
> -'
This one made me raise my eyebrows, first.

The only reason this test "passes" is because this particular "pull", due to what local2 and server happens to have before this test runs, is an "Already up-to-date" and a no-op. You are removing this because it is not really testing anything useful, and if you change it to test something real, e.g. by rewinding local2, it will reveal the breakage.

Am I reading you correctly?
Show 13 quoted lines
> diff --git a/transport-helper.c b/transport-helper.c
> index dcd8d97..cea787c 100644
> --- a/transport-helper.c
> +++ b/transport-helper.c
> @@ -469,7 +469,7 @@ static int fetch_with_import(struct transport *transport,
>  	 * were fetching.
>  	 *
>  	 * (If no "refspec" capability was specified, for historical
> -	 * reasons we default to *:*.)
> +	 * reasons we default to the equivalent of *:*.)
>  	 *
>  	 * Store the result in to_fetch[i].old_sha1.  Callers such
>  	 * as "git fetch" can use the value to write feedback to the
Felipe Contreras· Apr 18, 2013, 19:07 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 1/6] transport-helper: clarify *:* refspec

On Thu, Apr 18, 2013 at 12:27 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 18 quoted lines
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> The *:* refspec doesn't work, and never has, clarify the code and
>> documentation to reflect that. This in effect reverts commit 9e7673e
>> (gitremote-helpers(1): clarify refspec behaviour).
>> ...
>>  applicable refspec takes precedence.  The left-hand of refspecs
>>  advertised with this capability must cover all refs reported by
>> -the list command.  If a helper does not need a specific 'refspec'
>> -capability then it should advertise `refspec *:*`.
>> +the list command.  If no 'refspec' capability is advertised,
>> +there is an implied `refspec *:*`.
>
> I presume
>
>     s/.$/, but `*:*` does not work so do not use it./
>
> is implied?
I believe
  s/.$/, but you should specify an apropriate one as described above./
Would be better.
Show 24 quoted lines
>> diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
>> index f387027..cd1873c 100755
>> --- a/t/t5801-remote-helpers.sh
>> +++ b/t/t5801-remote-helpers.sh
>> @@ -120,21 +120,6 @@ test_expect_failure 'pushing without refspecs' '
>>       compare_refs local2 HEAD server HEAD
>>  '
>>
>> -test_expect_success 'pulling with straight refspec' '
>> -     (cd local2 &&
>> -     GIT_REMOTE_TESTGIT_REFSPEC="*:*" git pull) &&
>> -     compare_refs local2 HEAD server HEAD
>> -'
>
> This one made me raise my eyebrows, first.
>
> The only reason this test "passes" is because this particular
> "pull", due to what local2 and server happens to have before this
> test runs, is an "Already up-to-date" and a no-op.  You are removing
> this because it is not really testing anything useful, and if you
> change it to test something real, e.g. by rewinding local2, it will
> reveal the breakage.
>
> Am I reading you correctly?

Not really. This particular fetch does work, and it's tricky to explain all the reasons why, even if you update the server ref before fetching. The reason it's written this way is that it comes from from a branch of mine that has a hack to make the push above works, and since the transport-helper's push doesn't update the ref in this version, the test did actually do something and was working.

I am removing the test because we don't care if it works or not, remote-helpers should not be doing that. The fact that the test wasn't actually doing anything is icing on the cake.

Cheers.
-- 
Felipe Contreras
Felipe Contreras· Apr 18, 2013, 04:14 UTC · re: Felipe Contreras · lore

[PATCH v2 2/6] transport-helper: update refspec documentation

The refspec capability is not only used by 'import', also by 'export', and it's recommend in both.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 Documentation/gitremote-helpers.txt | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
Show changes to Documentation/gitremote-helpers.txt +5 −5
diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
index 0c91aba..ba7240c 100644
--- a/Documentation/gitremote-helpers.txt
+++ b/Documentation/gitremote-helpers.txt
@@ -159,11 +159,11 @@ Miscellaneous capabilities
 	carried out.
 
 'refspec' <refspec>::
-	This modifies the 'import' capability, allowing the produced
-	fast-import stream to modify refs in a private namespace
-	instead of writing to refs/heads or refs/remotes directly.
-	It is recommended that all importers providing the 'import'
-	capability use this.
+	For remote helpers that implement 'import' or 'export', this capability
+	allows the refs to be constrained to a private namespace, instead of
+	writing to refs/heads or refs/remotes directly.
+	It is recommended that all importers providing the 'import' or 'export'
+	capabilities use this.
 +
 A helper advertising the capability
 `refspec refs/heads/*:refs/svn/origin/branches/*`
-- 
1.8.2.1.679.g509521a
Felipe Contreras· Apr 18, 2013, 04:14 UTC · re: Felipe Contreras · lore

[PATCH v2 3/6] transport-helper: clarify pushing without refspecs

This has never worked, since it's inception the code simply skips all the refs, essentially telling fast-export to do nothing.

Let's at least tell the user what's going on.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 Documentation/gitremote-helpers.txt | 4 ++--
 t/t5801-remote-helpers.sh           | 6 +++---
 transport-helper.c                  | 5 +++--
 3 files changed, 8 insertions(+), 7 deletions(-)
Show changes to 3 files +8 −7

Documentation/gitremote-helpers.txt, t/t5801-remote-helpers.sh, transport-helper.c

diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
index ba7240c..4d26e37 100644
--- a/Documentation/gitremote-helpers.txt
+++ b/Documentation/gitremote-helpers.txt
@@ -162,8 +162,8 @@ Miscellaneous capabilities
 	For remote helpers that implement 'import' or 'export', this capability
 	allows the refs to be constrained to a private namespace, instead of
 	writing to refs/heads or refs/remotes directly.
-	It is recommended that all importers providing the 'import' or 'export'
-	capabilities use this.
+	It is recommended that all importers providing the 'import'
+	capability use this. It's mandatory for 'export'.
 +
 A helper advertising the capability
 `refspec refs/heads/*:refs/svn/origin/branches/*`
diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
index cd1873c..3eeb309 100755
--- a/t/t5801-remote-helpers.sh
+++ b/t/t5801-remote-helpers.sh
@@ -111,13 +111,13 @@ test_expect_success 'pulling without refspecs' '
 	compare_refs local2 HEAD server HEAD
 '
 
-test_expect_failure 'pushing without refspecs' '
+test_expect_success 'pushing without refspecs' '
 	test_when_finished "(cd local2 && git reset --hard origin)" &&
 	(cd local2 &&
 	echo content >>file &&
 	git commit -a -m ten &&
-	GIT_REMOTE_TESTGIT_REFSPEC="" git push) &&
-	compare_refs local2 HEAD server HEAD
+	GIT_REMOTE_TESTGIT_REFSPEC="" test_must_fail git push 2> ../error) &&
+	grep "remote-helper doesn.t support push; refspec needed" error
 '
 
 test_expect_success 'pulling without marks' '
diff --git a/transport-helper.c b/transport-helper.c
index cea787c..4d98567 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -785,6 +785,9 @@ static int push_refs_with_export(struct transport *transport,
 	struct string_list revlist_args = STRING_LIST_INIT_NODUP;
 	struct strbuf buf = STRBUF_INIT;
 
+	if (!data->refspecs)
+		die("remote-helper doesn't support push; refspec needed");
+
 	helper = get_helper(transport);
 
 	write_constant(helper->in, "export\n");
@@ -795,8 +798,6 @@ static int push_refs_with_export(struct transport *transport,
 		char *private;
 		unsigned char sha1[20];
 
-		if (!data->refspecs)
-			continue;
 		private = apply_refspecs(data->refspecs, data->refspec_nr, ref->name);
 		if (private && !get_sha1(private, sha1)) {
 			strbuf_addf(&buf, "^%s", private);
-- 
1.8.2.1.679.g509521a
John Keeping· Apr 18, 2013, 10:11 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 3/6] transport-helper: clarify pushing without refspecs

On Wed, Apr 17, 2013 at 11:14:30PM -0500, Felipe Contreras wrote:
Show 24 quoted lines
> This has never worked, since it's inception the code simply skips all
> the refs, essentially telling fast-export to do nothing.
> 
> Let's at least tell the user what's going on.
> 
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> ---
>  Documentation/gitremote-helpers.txt | 4 ++--
>  t/t5801-remote-helpers.sh           | 6 +++---
>  transport-helper.c                  | 5 +++--
>  3 files changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
> index ba7240c..4d26e37 100644
> --- a/Documentation/gitremote-helpers.txt
> +++ b/Documentation/gitremote-helpers.txt
> @@ -162,8 +162,8 @@ Miscellaneous capabilities
>  	For remote helpers that implement 'import' or 'export', this capability
>  	allows the refs to be constrained to a private namespace, instead of
>  	writing to refs/heads or refs/remotes directly.
> -	It is recommended that all importers providing the 'import' or 'export'
> -	capabilities use this.
> +	It is recommended that all importers providing the 'import'
> +	capability use this. It's mandatory for 'export'.
s/It's/It is/
Show 34 quoted lines
>  +
>  A helper advertising the capability
>  `refspec refs/heads/*:refs/svn/origin/branches/*`
> diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
> index cd1873c..3eeb309 100755
> --- a/t/t5801-remote-helpers.sh
> +++ b/t/t5801-remote-helpers.sh
> @@ -111,13 +111,13 @@ test_expect_success 'pulling without refspecs' '
>  	compare_refs local2 HEAD server HEAD
>  '
>  
> -test_expect_failure 'pushing without refspecs' '
> +test_expect_success 'pushing without refspecs' '
>  	test_when_finished "(cd local2 && git reset --hard origin)" &&
>  	(cd local2 &&
>  	echo content >>file &&
>  	git commit -a -m ten &&
> -	GIT_REMOTE_TESTGIT_REFSPEC="" git push) &&
> -	compare_refs local2 HEAD server HEAD
> +	GIT_REMOTE_TESTGIT_REFSPEC="" test_must_fail git push 2> ../error) &&
> +	grep "remote-helper doesn.t support push; refspec needed" error
>  '
>  
>  test_expect_success 'pulling without marks' '
> diff --git a/transport-helper.c b/transport-helper.c
> index cea787c..4d98567 100644
> --- a/transport-helper.c
> +++ b/transport-helper.c
> @@ -785,6 +785,9 @@ static int push_refs_with_export(struct transport *transport,
>  	struct string_list revlist_args = STRING_LIST_INIT_NODUP;
>  	struct strbuf buf = STRBUF_INIT;
>  
> +	if (!data->refspecs)
> +		die("remote-helper doesn't support push; refspec needed");

I think the "refspec needed" text is likely to be confusing if an end-user ever sees this message. I'm not sure how we can provide useful feedback for both remote helper authors and end-users though.

Show 15 quoted lines
> +
>  	helper = get_helper(transport);
>  
>  	write_constant(helper->in, "export\n");
> @@ -795,8 +798,6 @@ static int push_refs_with_export(struct transport *transport,
>  		char *private;
>  		unsigned char sha1[20];
>  
> -		if (!data->refspecs)
> -			continue;
>  		private = apply_refspecs(data->refspecs, data->refspec_nr, ref->name);
>  		if (private && !get_sha1(private, sha1)) {
>  			strbuf_addf(&buf, "^%s", private);
> -- 
> 1.8.2.1.679.g509521a
Felipe Contreras· Apr 18, 2013, 10:14 UTC · re: John Keeping · lore

Re: [PATCH v2 3/6] transport-helper: clarify pushing without refspecs

On Thu, Apr 18, 2013 at 5:11 AM, John Keeping <john@keeping.me.uk> wrote:
Show 27 quoted lines
> On Wed, Apr 17, 2013 at 11:14:30PM -0500, Felipe Contreras wrote:
>> This has never worked, since it's inception the code simply skips all
>> the refs, essentially telling fast-export to do nothing.
>>
>> Let's at least tell the user what's going on.
>>
>> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
>> ---
>>  Documentation/gitremote-helpers.txt | 4 ++--
>>  t/t5801-remote-helpers.sh           | 6 +++---
>>  transport-helper.c                  | 5 +++--
>>  3 files changed, 8 insertions(+), 7 deletions(-)
>>
>> diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
>> index ba7240c..4d26e37 100644
>> --- a/Documentation/gitremote-helpers.txt
>> +++ b/Documentation/gitremote-helpers.txt
>> @@ -162,8 +162,8 @@ Miscellaneous capabilities
>>       For remote helpers that implement 'import' or 'export', this capability
>>       allows the refs to be constrained to a private namespace, instead of
>>       writing to refs/heads or refs/remotes directly.
>> -     It is recommended that all importers providing the 'import' or 'export'
>> -     capabilities use this.
>> +     It is recommended that all importers providing the 'import'
>> +     capability use this. It's mandatory for 'export'.
>
> s/It's/It is/
What's the difference?
Show 12 quoted lines
>> --- a/transport-helper.c
>> +++ b/transport-helper.c
>> @@ -785,6 +785,9 @@ static int push_refs_with_export(struct transport *transport,
>>       struct string_list revlist_args = STRING_LIST_INIT_NODUP;
>>       struct strbuf buf = STRBUF_INIT;
>>
>> +     if (!data->refspecs)
>> +             die("remote-helper doesn't support push; refspec needed");
>
> I think the "refspec needed" text is likely to be confusing if an
> end-user ever sees this message.  I'm not sure how we can provide useful
> feedback for both remote helper authors and end-users though.

It doesn't have to be. They'll google it, or they'll post a bug report with this warning verbatim, or they'll just ignore it. I don't think there's much more we can do to help in either of these cases.

-- 
Felipe Contreras
John Keeping· Apr 18, 2013, 10:24 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 3/6] transport-helper: clarify pushing without refspecs

On Thu, Apr 18, 2013 at 05:14:18AM -0500, Felipe Contreras wrote:
Show 30 quoted lines
> On Thu, Apr 18, 2013 at 5:11 AM, John Keeping <john@keeping.me.uk> wrote:
> > On Wed, Apr 17, 2013 at 11:14:30PM -0500, Felipe Contreras wrote:
> >> This has never worked, since it's inception the code simply skips all
> >> the refs, essentially telling fast-export to do nothing.
> >>
> >> Let's at least tell the user what's going on.
> >>
> >> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> >> ---
> >>  Documentation/gitremote-helpers.txt | 4 ++--
> >>  t/t5801-remote-helpers.sh           | 6 +++---
> >>  transport-helper.c                  | 5 +++--
> >>  3 files changed, 8 insertions(+), 7 deletions(-)
> >>
> >> diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
> >> index ba7240c..4d26e37 100644
> >> --- a/Documentation/gitremote-helpers.txt
> >> +++ b/Documentation/gitremote-helpers.txt
> >> @@ -162,8 +162,8 @@ Miscellaneous capabilities
> >>       For remote helpers that implement 'import' or 'export', this capability
> >>       allows the refs to be constrained to a private namespace, instead of
> >>       writing to refs/heads or refs/remotes directly.
> >> -     It is recommended that all importers providing the 'import' or 'export'
> >> -     capabilities use this.
> >> +     It is recommended that all importers providing the 'import'
> >> +     capability use this. It's mandatory for 'export'.
> >
> > s/It's/It is/
> 
> What's the difference?

"It's" is considered informal, while we do adopt that tone on the mailing list and sometimes in the documentation, in general I think we should avoid contractions in the formal documentation.

In this case it is particularly jarring because it immediately follows a sentence where we use the full "It is" form.

Junio C Hamano· Apr 18, 2013, 17:29 UTC · re: John Keeping · lore

Re: [PATCH v2 3/6] transport-helper: clarify pushing without refspecs

John Keeping <john@keeping.me.uk> writes:
>> +	It is recommended that all importers providing the 'import'
>> +	capability use this. It's mandatory for 'export'.
>
> s/It's/It is/

I personally do not care _too_ deeply either way, but I agree our documentation tends to use the latter more and being consistent would be good.

Show 14 quoted lines
>> diff --git a/transport-helper.c b/transport-helper.c
>> index cea787c..4d98567 100644
>> --- a/transport-helper.c
>> +++ b/transport-helper.c
>> @@ -785,6 +785,9 @@ static int push_refs_with_export(struct transport *transport,
>>  	struct string_list revlist_args = STRING_LIST_INIT_NODUP;
>>  	struct strbuf buf = STRBUF_INIT;
>>  
>> +	if (!data->refspecs)
>> +		die("remote-helper doesn't support push; refspec needed");
>
> I think the "refspec needed" text is likely to be confusing if an
> end-user ever sees this message.  I'm not sure how we can provide useful
> feedback for both remote helper authors and end-users though.

This "refspecs" only come via the helper and not directly from the end user, no?

If that is the case, I do not think "confusing" is much of an issue. Not _("localizing") is also the right thing to do. We may want to say "BUG: " at front to clarify it is not the end-user's fault, but a problem they may want to report. If we at this point know what helper attempted export without giving refspecs, it may help to show it here, so that developers will know with what helper the user had problems with.

John Keeping· Apr 18, 2013, 18:35 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 3/6] transport-helper: clarify pushing without refspecs

On Thu, Apr 18, 2013 at 10:29:30AM -0700, Junio C Hamano wrote:
Show 26 quoted lines
> John Keeping <john@keeping.me.uk> writes:
> >> diff --git a/transport-helper.c b/transport-helper.c
> >> index cea787c..4d98567 100644
> >> --- a/transport-helper.c
> >> +++ b/transport-helper.c
> >> @@ -785,6 +785,9 @@ static int push_refs_with_export(struct transport *transport,
> >>  	struct string_list revlist_args = STRING_LIST_INIT_NODUP;
> >>  	struct strbuf buf = STRBUF_INIT;
> >>  
> >> +	if (!data->refspecs)
> >> +		die("remote-helper doesn't support push; refspec needed");
> >
> > I think the "refspec needed" text is likely to be confusing if an
> > end-user ever sees this message.  I'm not sure how we can provide useful
> > feedback for both remote helper authors and end-users though.
> 
> This "refspecs" only come via the helper and not directly from the
> end user, no?
> 
> If that is the case, I do not think "confusing" is much of an issue.
> Not _("localizing") is also the right thing to do.  We may want to
> say "BUG: " at front to clarify it is not the end-user's fault, but
> a problem they may want to report.  If we at this point know what
> helper attempted export without giving refspecs, it may help to show
> it here, so that developers will know with what helper the user
> had problems with.

I like this idea. Perhaps we should say "Bug in remote helper; refspec needed with export", so that it clearly indicates to both end-users and remote helper authors that the error is in the remote helper.

Junio C Hamano· Apr 18, 2013, 17:28 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 3/6] transport-helper: clarify pushing without refspecs

Felipe Contreras <felipe.contreras@gmail.com> writes:
> This has never worked, since it's inception the code simply skips all
> the refs, essentially telling fast-export to do nothing.
Good description.
Show 22 quoted lines
>
> Let's at least tell the user what's going on.
>
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> ---
>  Documentation/gitremote-helpers.txt | 4 ++--
>  t/t5801-remote-helpers.sh           | 6 +++---
>  transport-helper.c                  | 5 +++--
>  3 files changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
> index ba7240c..4d26e37 100644
> --- a/Documentation/gitremote-helpers.txt
> +++ b/Documentation/gitremote-helpers.txt
> @@ -162,8 +162,8 @@ Miscellaneous capabilities
>  	For remote helpers that implement 'import' or 'export', this capability
>  	allows the refs to be constrained to a private namespace, instead of
>  	writing to refs/heads or refs/remotes directly.
> -	It is recommended that all importers providing the 'import' or 'export'
> -	capabilities use this.
> +	It is recommended that all importers providing the 'import'
> +	capability use this. It's mandatory for 'export'.

As [1/6] said "*:*" does not work and has never worked, and especially on the push side the patch below makes it clear it was a glorified no-op, it may be better to say a bit more than "use this. It's mandatory". It is not like any value makes sense, right?

Perhaps the documentation will be further updated in a later step in the series to clarify it. Let's read on til the end of the series...

Show 19 quoted lines
> diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
> index cd1873c..3eeb309 100755
> --- a/t/t5801-remote-helpers.sh
> +++ b/t/t5801-remote-helpers.sh
> @@ -111,13 +111,13 @@ test_expect_success 'pulling without refspecs' '
>  	compare_refs local2 HEAD server HEAD
>  '
>  
> -test_expect_failure 'pushing without refspecs' '
> +test_expect_success 'pushing without refspecs' '
>  	test_when_finished "(cd local2 && git reset --hard origin)" &&
>  	(cd local2 &&
>  	echo content >>file &&
>  	git commit -a -m ten &&
> -	GIT_REMOTE_TESTGIT_REFSPEC="" git push) &&
> -	compare_refs local2 HEAD server HEAD
> +	GIT_REMOTE_TESTGIT_REFSPEC="" test_must_fail git push 2> ../error) &&
> +	grep "remote-helper doesn.t support push; refspec needed" error
>  '

I somehow like this change that turns a _failure into a _success by expecting test_must_fail, especially because it is clear why the tested "push" should fail when you look at the rest of the patch ;-)

Fun.
Show 23 quoted lines
> diff --git a/transport-helper.c b/transport-helper.c
> index cea787c..4d98567 100644
> --- a/transport-helper.c
> +++ b/transport-helper.c
> @@ -785,6 +785,9 @@ static int push_refs_with_export(struct transport *transport,
>  	struct string_list revlist_args = STRING_LIST_INIT_NODUP;
>  	struct strbuf buf = STRBUF_INIT;
>  
> +	if (!data->refspecs)
> +		die("remote-helper doesn't support push; refspec needed");
> +
>  	helper = get_helper(transport);
>  
>  	write_constant(helper->in, "export\n");
> @@ -795,8 +798,6 @@ static int push_refs_with_export(struct transport *transport,
>  		char *private;
>  		unsigned char sha1[20];
>  
> -		if (!data->refspecs)
> -			continue;
>  		private = apply_refspecs(data->refspecs, data->refspec_nr, ref->name);
>  		if (private && !get_sha1(private, sha1)) {
>  			strbuf_addf(&buf, "^%s", private);
Eric Sunshine· Apr 19, 2013, 00:27 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 3/6] transport-helper: clarify pushing without refspecs

On Thu, Apr 18, 2013 at 12:14 AM, Felipe Contreras <felipe.contreras@gmail.com> wrote:

Show 18 quoted lines
> diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
> index cd1873c..3eeb309 100755
> --- a/t/t5801-remote-helpers.sh
> +++ b/t/t5801-remote-helpers.sh
> @@ -111,13 +111,13 @@ test_expect_success 'pulling without refspecs' '
>         compare_refs local2 HEAD server HEAD
>  '
>
> -test_expect_failure 'pushing without refspecs' '
> +test_expect_success 'pushing without refspecs' '
>         test_when_finished "(cd local2 && git reset --hard origin)" &&
>         (cd local2 &&
>         echo content >>file &&
>         git commit -a -m ten &&
> -       GIT_REMOTE_TESTGIT_REFSPEC="" git push) &&
> -       compare_refs local2 HEAD server HEAD
> +       GIT_REMOTE_TESTGIT_REFSPEC="" test_must_fail git push 2> ../error) &&
> +       grep "remote-helper doesn.t support push; refspec needed" error

Is "doesn.t" intentional? It certainly works by accident in grep, but did you mean s/doesn.t/doesn't/ ?

>  '
>
>  test_expect_success 'pulling without marks' '
Felipe Contreras· Apr 19, 2013, 00:30 UTC · re: Eric Sunshine · lore

Re: [PATCH v2 3/6] transport-helper: clarify pushing without refspecs

On Thu, Apr 18, 2013 at 7:27 PM, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 23 quoted lines
> On Thu, Apr 18, 2013 at 12:14 AM, Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
>> diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
>> index cd1873c..3eeb309 100755
>> --- a/t/t5801-remote-helpers.sh
>> +++ b/t/t5801-remote-helpers.sh
>> @@ -111,13 +111,13 @@ test_expect_success 'pulling without refspecs' '
>>         compare_refs local2 HEAD server HEAD
>>  '
>>
>> -test_expect_failure 'pushing without refspecs' '
>> +test_expect_success 'pushing without refspecs' '
>>         test_when_finished "(cd local2 && git reset --hard origin)" &&
>>         (cd local2 &&
>>         echo content >>file &&
>>         git commit -a -m ten &&
>> -       GIT_REMOTE_TESTGIT_REFSPEC="" git push) &&
>> -       compare_refs local2 HEAD server HEAD
>> +       GIT_REMOTE_TESTGIT_REFSPEC="" test_must_fail git push 2> ../error) &&
>> +       grep "remote-helper doesn.t support push; refspec needed" error
>
> Is "doesn.t" intentional? It certainly works by accident in grep, but
> did you mean s/doesn.t/doesn't/ ?
In all git test cases we are already inside single quotes; can't do that.
-- 
Felipe Contreras
Junio C Hamano· Apr 19, 2013, 03:41 UTC · re: Eric Sunshine · lore

Re: [PATCH v2 3/6] transport-helper: clarify pushing without refspecs

Eric Sunshine <sunshine@sunshineco.com> writes:
>> +       grep "remote-helper doesn.t support push; refspec needed" error
>
> Is "doesn.t" intentional? It certainly works by accident in grep, but
> did you mean s/doesn.t/doesn't/ ?

The pattern matching the expected string is not by accident, but by design.

It of course can be made more strict to reject "doesnot" and require "doesn't" by doing something like this:

       grep "remote-helper doesn'\''t support push; refspec needed" error

but at some point, it simply stops being worth it to tighten the pattern.

For that matter, it could be as loose as
	grep "support push; refspec needed" error
if you know the string is unique enough.
Felipe Contreras· Apr 18, 2013, 04:14 UTC · re: Felipe Contreras · lore

[PATCH v2 4/6] transport-helper: warn when refspec is not used

For the modes that need it. In the future we should probably error out, instead of providing half-assed support.

The reason we want to do this is because if it's not present, the remote helper might be updating refs/heads/*, or refs/remotes/origin/*, directly, and in the process fetch will get confused trying to update refs that are already updated, or older than what they should be. We shouldn't be messing with the rest of git.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 t/t5801-remote-helpers.sh | 6 ++++--
 transport-helper.c        | 2 ++
 2 files changed, 6 insertions(+), 2 deletions(-)
Show changes to 2 files +6 −2

t/t5801-remote-helpers.sh, transport-helper.c

diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
index 3eeb309..1bb7529 100755
--- a/t/t5801-remote-helpers.sh
+++ b/t/t5801-remote-helpers.sh
@@ -100,14 +100,16 @@ test_expect_failure 'push new branch with old:new refspec' '
 
 test_expect_success 'cloning without refspec' '
 	GIT_REMOTE_TESTGIT_REFSPEC="" \
-	git clone "testgit::${PWD}/server" local2 &&
+	git clone "testgit::${PWD}/server" local2 2> error &&
+	grep "This remote helper should implement refspec capability" error &&
 	compare_refs local2 HEAD server HEAD
 '
 
 test_expect_success 'pulling without refspecs' '
 	(cd local2 &&
 	git reset --hard &&
-	GIT_REMOTE_TESTGIT_REFSPEC="" git pull) &&
+	GIT_REMOTE_TESTGIT_REFSPEC="" git pull 2> ../error) &&
+	grep "This remote helper should implement refspec capability" error &&
 	compare_refs local2 HEAD server HEAD
 '
 
diff --git a/transport-helper.c b/transport-helper.c
index 4d98567..573eaf7 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -215,6 +215,8 @@ static struct child_process *get_helper(struct transport *transport)
 			free((char *)refspecs[i]);
 		}
 		free(refspecs);
+	} else if (data->import || data->bidi_import || data->export) {
+		warning("This remote helper should implement refspec capability.");
 	}
 	strbuf_release(&buf);
 	if (debug)
-- 
1.8.2.1.679.g509521a
Junio C Hamano· Apr 18, 2013, 17:30 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 4/6] transport-helper: warn when refspec is not used

Felipe Contreras <felipe.contreras@gmail.com> writes:
Show 8 quoted lines
> For the modes that need it. In the future we should probably error out,
> instead of providing half-assed support.
>
> The reason we want to do this is because if it's not present, the remote
> helper might be updating refs/heads/*, or refs/remotes/origin/*,
> directly, and in the process fetch will get confused trying to update
> refs that are already updated, or older than what they should be. We
> shouldn't be messing with the rest of git.

So that answers my question in the response to an earlier one in this series. We expect the ref updates to be done by the fetch or push that drives the helper, and do not want the helper to interfere with its ref updates.

So it is not just 'refspec' _allows_ the refs to be constrained to a private namespace, like the earlier updates made the documentation say; it _is_ mandatory to use refspecs to constrain them to avoid touching refs/heads and refs/remotes namespace.

Am I reading you correctly?
Assuming I am, the patch in this message looks reasonable.

It makes the documentation updates a few patches ago look a bit wanting, though.

Thanks.
Show 42 quoted lines
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> ---
>  t/t5801-remote-helpers.sh | 6 ++++--
>  transport-helper.c        | 2 ++
>  2 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
> index 3eeb309..1bb7529 100755
> --- a/t/t5801-remote-helpers.sh
> +++ b/t/t5801-remote-helpers.sh
> @@ -100,14 +100,16 @@ test_expect_failure 'push new branch with old:new refspec' '
>  
>  test_expect_success 'cloning without refspec' '
>  	GIT_REMOTE_TESTGIT_REFSPEC="" \
> -	git clone "testgit::${PWD}/server" local2 &&
> +	git clone "testgit::${PWD}/server" local2 2> error &&
> +	grep "This remote helper should implement refspec capability" error &&
>  	compare_refs local2 HEAD server HEAD
>  '
>  
>  test_expect_success 'pulling without refspecs' '
>  	(cd local2 &&
>  	git reset --hard &&
> -	GIT_REMOTE_TESTGIT_REFSPEC="" git pull) &&
> +	GIT_REMOTE_TESTGIT_REFSPEC="" git pull 2> ../error) &&
> +	grep "This remote helper should implement refspec capability" error &&
>  	compare_refs local2 HEAD server HEAD
>  '
>  
> diff --git a/transport-helper.c b/transport-helper.c
> index 4d98567..573eaf7 100644
> --- a/transport-helper.c
> +++ b/transport-helper.c
> @@ -215,6 +215,8 @@ static struct child_process *get_helper(struct transport *transport)
>  			free((char *)refspecs[i]);
>  		}
>  		free(refspecs);
> +	} else if (data->import || data->bidi_import || data->export) {
> +		warning("This remote helper should implement refspec capability.");
>  	}
>  	strbuf_release(&buf);
>  	if (debug)
Felipe Contreras· Apr 18, 2013, 19:12 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 4/6] transport-helper: warn when refspec is not used

On Thu, Apr 18, 2013 at 12:30 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 20 quoted lines
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> For the modes that need it. In the future we should probably error out,
>> instead of providing half-assed support.
>>
>> The reason we want to do this is because if it's not present, the remote
>> helper might be updating refs/heads/*, or refs/remotes/origin/*,
>> directly, and in the process fetch will get confused trying to update
>> refs that are already updated, or older than what they should be. We
>> shouldn't be messing with the rest of git.
>
> So that answers my question in the response to an earlier one in
> this series.  We expect the ref updates to be done by the fetch or
> push that drives the helper, and do not want the helper to interfere
> with its ref updates.
>
> So it is not just 'refspec' _allows_ the refs to be constrained to a
> private namespace, like the earlier updates made the documentation
> say; it _is_ mandatory to use refspecs to constrain them to avoid
> touching refs/heads and refs/remotes namespace.

Yeah, it was not stated that it was mandatory, but I think it's in everybody's best interest that it is.

-- 
Felipe Contreras
Felipe Contreras· Apr 18, 2013, 04:14 UTC · re: Felipe Contreras · lore

[PATCH v2 5/6] transport-helper: trivial code shuffle

Just shuffle the die() part to make it more explicit, and cleanup the code-style.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 transport-helper.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
Show changes to transport-helper.c +3 −5
diff --git a/transport-helper.c b/transport-helper.c
index 573eaf7..9d31f2d 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -800,6 +800,9 @@ static int push_refs_with_export(struct transport *transport,
 		char *private;
 		unsigned char sha1[20];
 
+		if (ref->deletion)
+			die("remote-helpers do not support ref deletion");
+
 		private = apply_refspecs(data->refspecs, data->refspec_nr, ref->name);
 		if (private && !get_sha1(private, sha1)) {
 			strbuf_addf(&buf, "^%s", private);
@@ -807,13 +810,8 @@ static int push_refs_with_export(struct transport *transport,
 		}
 		free(private);
 
-		if (ref->deletion) {
-			die("remote-helpers do not support ref deletion");
-		}
-
 		if (ref->peer_ref)
 			string_list_append(&revlist_args, ref->peer_ref->name);
-
 	}
 
 	if (get_exporter(transport, &exporter, &revlist_args))
-- 
1.8.2.1.679.g509521a
Felipe Contreras· Apr 18, 2013, 04:14 UTC · re: Felipe Contreras · lore

[PATCH v2 6/6] transport-helper: update remote helper namespace

When pushing, the remote namespace is updated correctly (e.g. refs/origin/master), but not the remote helper's (e.g. refs/testgit/origin/master), which currently is only updated while fetching.

Since the remote namespace is used to tell fast-export which commits to avoid (because they were already imported/exported), it makes sense to have them in sync so they don't get generated twice. If the remote helper was implemented properly, they would be ignored, if not, they probably would end up repeated (probably).

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 t/t5801-remote-helpers.sh | 12 ++++++++++++
 transport-helper.c        | 23 +++++++++++++++++++----
 2 files changed, 31 insertions(+), 4 deletions(-)
Show changes to 2 files +31 −4

t/t5801-remote-helpers.sh, transport-helper.c

diff --git a/t/t5801-remote-helpers.sh b/t/t5801-remote-helpers.sh
index 1bb7529..5466969 100755
--- a/t/t5801-remote-helpers.sh
+++ b/t/t5801-remote-helpers.sh
@@ -153,4 +153,16 @@ test_expect_success 'push ref with existing object' '
 	compare_refs local dup server dup
 '
 
+test_expect_success 'push update refs' '
+	(cd local &&
+	git checkout -b update master &&
+	echo update >>file &&
+	git commit -a -m update &&
+	git push origin update
+	git rev-parse --verify remotes/origin/update >expect &&
+	git rev-parse --verify testgit/origin/heads/update >actual &&
+	test_cmp expect actual
+	)
+'
+
 test_done
diff --git a/transport-helper.c b/transport-helper.c
index 9d31f2d..da16393 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -11,6 +11,7 @@
 #include "thread-utils.h"
 #include "sigchain.h"
 #include "argv-array.h"
+#include "refs.h"
 
 static int debug;
 
@@ -620,7 +621,7 @@ static int fetch(struct transport *transport,
 	return -1;
 }
 
-static void push_update_ref_status(struct strbuf *buf,
+static int push_update_ref_status(struct strbuf *buf,
 				   struct ref **ref,
 				   struct ref *remote_refs)
 {
@@ -686,7 +687,7 @@ static void push_update_ref_status(struct strbuf *buf,
 		*ref = find_ref_by_name(remote_refs, refname);
 	if (!*ref) {
 		warning("helper reported unexpected status of %s", refname);
-		return;
+		return 1;
 	}
 
 	if ((*ref)->status != REF_STATUS_NONE) {
@@ -695,11 +696,12 @@ static void push_update_ref_status(struct strbuf *buf,
 		 * status reported by the remote helper if the latter is 'no match'.
 		 */
 		if (status == REF_STATUS_NONE)
-			return;
+			return 1;
 	}
 
 	(*ref)->status = status;
 	(*ref)->remote_status = msg;
+	return 0;
 }
 
 static void push_update_refs_status(struct helper_data *data,
@@ -708,11 +710,24 @@ static void push_update_refs_status(struct helper_data *data,
 	struct strbuf buf = STRBUF_INIT;
 	struct ref *ref = remote_refs;
 	for (;;) {
+		char *private;
+
 		recvline(data, &buf);
 		if (!buf.len)
 			break;
 
-		push_update_ref_status(&buf, &ref, remote_refs);
+		if (push_update_ref_status(&buf, &ref, remote_refs))
+			continue;
+
+		if (!data->refspecs)
+			continue;
+
+		/* propagate back the update to the remote namespace */
+		private = apply_refspecs(data->refspecs, data->refspec_nr, ref->name);
+		if (!private)
+			continue;
+		update_ref("update by helper", private, ref->new_sha1, NULL, 0, 0);
+		free(private);
 	}
 	strbuf_release(&buf);
 }
-- 
1.8.2.1.679.g509521a
Junio C Hamano· Apr 18, 2013, 17:30 UTC · re: Felipe Contreras · lore

Re: [PATCH v2 6/6] transport-helper: update remote helper namespace

Felipe Contreras <felipe.contreras@gmail.com> writes:
Show 13 quoted lines
> When pushing, the remote namespace is updated correctly
> (e.g. refs/origin/master), but not the remote helper's
> (e.g. refs/testgit/origin/master), which currently is only updated while
> fetching.
>
> Since the remote namespace is used to tell fast-export which commits to
> avoid (because they were already imported/exported), it makes sense to
> have them in sync so they don't get generated twice. If the remote
> helper was implemented properly, they would be ignored, if not, they
> probably would end up repeated (probably).
>
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> ---
This one looks suspiciously familiar ;-).

I'll drop the last one I queued and pushed out a few days ago and replace it with these 6 patches.

← back to recent threads