threads / patch / 33701

v2upload-pack: ignore 'shallow' lines with unknown obj-ids

Subject: [PATCH v2] upload-pack: ignore 'shallow' lines with unknown obj-ids

## tl;dr

3 messages between May 2, 2013 and May 2, 2013. Diffs are folded; open one to read it.

replies: 2people: 2as markdown or json

Michael Heemskerk· May 2, 2013, 14:56 UTC · lore

When the client sends a 'shallow' line for an object that the server does not have, the server currently dies with the error: "did not find object for shallow <obj-id>". The client may have truncated the history at the commit by fetching shallowly from a different server, or the commit may have been garbage collected by the server. In either case, this unknown commit is not relevant for calculating the pack that is to be sent and can be safely ignored, and it is not used when recomputing where the updated history of the client is cauterised.

The documentation in technical/pack-protocol.txt has been updated to remove the restriction that "Clients MUST NOT mention an obj-id which it does not know exists on the server". This requirement is not realistic because clients cannot know whether an object has been garbage collected by the server.

Signed-off-by: Michael Heemskerk <mheemskerk@atlassian.com>
Reviewed-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/technical/pack-protocol.txt |  3 +--
 t/t5500-fetch-pack.sh                     | 13 +++++++++++++
 upload-pack.c                             |  2 +-
 3 files changed, 15 insertions(+), 3 deletions(-)
Show changes to 3 files +15 −3

Documentation/technical/pack-protocol.txt, t/t5500-fetch-pack.sh, upload-pack.c

diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt
index f1a51ed..b898e97 100644
--- a/Documentation/technical/pack-protocol.txt
+++ b/Documentation/technical/pack-protocol.txt
@@ -228,8 +228,7 @@ obtained through ref discovery.
 The client MUST write all obj-ids which it only has shallow copies
 of (meaning that it does not have the parents of a commit) as
 'shallow' lines so that the server is aware of the limitations of
-the client's history. Clients MUST NOT mention an obj-id which
-it does not know exists on the server.
+the client's history.
 
 The client now sends the maximum commit history depth it wants for
 this transaction, which is the number of commits it wants from the
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index d574085..3f0ca10 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -373,6 +373,19 @@ test_expect_success 'clone shallow with packed refs' '
 	test_cmp count8.expected count8.actual
 '
 
+test_expect_success 'fetch in shallow repo unreachable shallow objects' '
+	(
+		git clone --bare --branch B --single-branch "file://$(pwd)/." no-reflog &&
+		git clone --depth 1 "file://$(pwd)/no-reflog" shallow9 &&
+		cd no-reflog &&
+		git tag -d TAGB1 TAGB2 &&
+		git update-ref refs/heads/B B~~ &&
+		git gc --prune=now &&
+		cd ../shallow9 &&
+		git fetch origin
+	)
+'
+
 test_expect_success 'setup tests for the --stdin parameter' '
 	for head in C D E F
 	do
diff --git a/upload-pack.c b/upload-pack.c
index bfa6279..127e59a 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -592,7 +592,7 @@ static void receive_needs(void)
 				die("invalid shallow line: %s", line);
 			object = parse_object(sha1);
 			if (!object)
-				die("did not find object for %s", line);
+				continue;
 			if (object->type != OBJ_COMMIT)
 				die("invalid shallow object %s", sha1_to_hex(sha1));
 			if (!(object->flags & CLIENT_SHALLOW)) {
-- 
1.8.0.2
Junio C Hamano· May 2, 2013, 16:54 UTC · re: Michael Heemskerk · lore

Re: [PATCH v2] upload-pack: ignore 'shallow' lines with unknown obj-ids

Michael Heemskerk <mheemskerk@atlassian.com> writes:
Show 7 quoted lines
> When the client sends a 'shallow' line for an object that the server does
> not have, the server currently dies with the error: "did not find object
> ...
>  Documentation/technical/pack-protocol.txt |  3 +--
>  t/t5500-fetch-pack.sh                     | 13 +++++++++++++
>  upload-pack.c                             |  2 +-
>  3 files changed, 15 insertions(+), 3 deletions(-)
Thanks.

The previous one (without the test) is already in 'next' so I'll pick only the test part and queue it as an update to the series.

Show 24 quoted lines
> diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
> index d574085..3f0ca10 100755
> --- a/t/t5500-fetch-pack.sh
> +++ b/t/t5500-fetch-pack.sh
> @@ -373,6 +373,19 @@ test_expect_success 'clone shallow with packed refs' '
>  	test_cmp count8.expected count8.actual
>  '
>  
> +test_expect_success 'fetch in shallow repo unreachable shallow objects' '
> +	(
> +		git clone --bare --branch B --single-branch "file://$(pwd)/." no-reflog &&
> +		git clone --depth 1 "file://$(pwd)/no-reflog" shallow9 &&
> +		cd no-reflog &&
> +		git tag -d TAGB1 TAGB2 &&
> +		git update-ref refs/heads/B B~~ &&
> +		git gc --prune=now &&
> +		cd ../shallow9 &&
> +		git fetch origin
> +	)
> +'
> +
>  test_expect_success 'setup tests for the --stdin parameter' '
>  	for head in C D E F
>  	do
Junio C Hamano· May 2, 2013, 17:05 UTC · re: Junio C Hamano · lore

Re: [PATCH v2] upload-pack: ignore 'shallow' lines with unknown obj-ids

Junio C Hamano <gitster@pobox.com> writes:
Show 35 quoted lines
> Michael Heemskerk <mheemskerk@atlassian.com> writes:
>
>> When the client sends a 'shallow' line for an object that the server does
>> not have, the server currently dies with the error: "did not find object
>> ...
>>  Documentation/technical/pack-protocol.txt |  3 +--
>>  t/t5500-fetch-pack.sh                     | 13 +++++++++++++
>>  upload-pack.c                             |  2 +-
>>  3 files changed, 15 insertions(+), 3 deletions(-)
>
> Thanks.
>
> The previous one (without the test) is already in 'next' so I'll
> pick only the test part and queue it as an update to the series.
>
>> diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
>> index d574085..3f0ca10 100755
>> --- a/t/t5500-fetch-pack.sh
>> +++ b/t/t5500-fetch-pack.sh
>> @@ -373,6 +373,19 @@ test_expect_success 'clone shallow with packed refs' '
>>  	test_cmp count8.expected count8.actual
>>  '
>>  
>> +test_expect_success 'fetch in shallow repo unreachable shallow objects' '
>> +	(
>> +		git clone --bare --branch B --single-branch "file://$(pwd)/." no-reflog &&
>> +		git clone --depth 1 "file://$(pwd)/no-reflog" shallow9 &&
>> +		cd no-reflog &&
>> +		git tag -d TAGB1 TAGB2 &&
>> +		git update-ref refs/heads/B B~~ &&
>> +		git gc --prune=now &&
>> +		cd ../shallow9 &&
>> +		git fetch origin
>> +	)
>> +'

It would also be good to make sure that shallow9 does not lose the shallow history boundary that the origin does not know about after this fetch, so I'll further squash this in.

 t/t5500-fetch-pack.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Show changes to t/t5500-fetch-pack.sh +2 −1
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index 3f0ca10..6133d9e 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -382,7 +382,8 @@ test_expect_success 'fetch in shallow repo unreachable shallow objects' '
 		git update-ref refs/heads/B B~~ &&
 		git gc --prune=now &&
 		cd ../shallow9 &&
-		git fetch origin
+		git fetch origin &&
+		git fsck --no-dangling
 	)
 '
 

← back to recent threads