threads / patch / 6322

patchDon't die in git-http-fetch when fetching packs.

Subject: [PATCH] Don't die in git-http-fetch when fetching packs.

## tl;dr

3 messages between Jan 10, 2007 and Jan 10, 2007. Diffs are folded; open one to read it.

replies: 2people: 2as markdown or json

Shawn O. Pearce· Jan 10, 2007, 01:04 UTC · lore

My sp/mmap changes to pack-check.c modified the function such that it expects packed_git.pack_size to be populated with the total bytecount of the packfile by the caller.

But that isn't the case for packs obtained by git-http-fetch as pack_size was not initialized before being accessed. This caused verify_pack to think it had 2^32-21 bytes available when the downloaded pack perhaps was only 305 bytes in length. The use_pack function then later dies with "offset beyond end of packfile" when computing the overall file checksum.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 For the tip of my sp/mmap topic.  Without it HTTP fetch and push
 support is severely broken as it cannot deal with packs.
 http-fetch.c |    1 +
 http-push.c  |    1 +
 2 files changed, 2 insertions(+), 0 deletions(-)
Show changes to 2 files +2 −0

http-fetch.c, http-push.c

diff --git a/http-fetch.c b/http-fetch.c
index fe8cd7b..67dfb0a 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -809,6 +809,7 @@ static int fetch_pack(struct alt_base *repo, unsigned char *sha1)
 		return error("Unable to start request");
 	}
 
+	target->pack_size = ftell(packfile);
 	fclose(packfile);
 
 	ret = move_temp_to_file(tmpfile, filename);
diff --git a/http-push.c b/http-push.c
index 7e73eac..8eb448e 100644
--- a/http-push.c
+++ b/http-push.c
@@ -770,6 +770,7 @@ static void finish_request(struct transfer_request *request)
 				request->url, curl_errorstr);
 			remote->can_update_info_refs = 0;
 		} else {
+			target->pack_size = ftell(request->local_stream);
 			fclose(request->local_stream);
 			request->local_stream = NULL;
 			if (!move_temp_to_file(request->tmpfile,
-- 
1.4.4.4.gf198
Junio C Hamano· Jan 10, 2007, 01:10 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] Don't die in git-http-fetch when fetching packs.

"Shawn O. Pearce" <spearce@spearce.org> writes:
Show 16 quoted lines
> My sp/mmap changes to pack-check.c modified the function such that
> it expects packed_git.pack_size to be populated with the total
> bytecount of the packfile by the caller.
>
> But that isn't the case for packs obtained by git-http-fetch as
> pack_size was not initialized before being accessed.  This caused
> verify_pack to think it had 2^32-21 bytes available when the
> downloaded pack perhaps was only 305 bytes in length.  The use_pack
> function then later dies with "offset beyond end of packfile"
> when computing the overall file checksum.
>
> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
> ---
>
>  For the tip of my sp/mmap topic.  Without it HTTP fetch and push
>  support is severely broken as it cannot deal with packs.

Ah, thanks. I've seen this today and a repeated fetch after that fixed it and thought it was a mirroring lag.

Shawn O. Pearce· Jan 10, 2007, 01:28 UTC · re: Junio C Hamano · lore

Re: [PATCH] Don't die in git-http-fetch when fetching packs.

Junio C Hamano <junkio@cox.net> wrote:
Show 6 quoted lines
> "Shawn O. Pearce" <spearce@spearce.org> writes:
> >  For the tip of my sp/mmap topic.  Without it HTTP fetch and push
> >  support is severely broken as it cannot deal with packs.
> 
> Ah, thanks.  I've seen this today and a repeated fetch after
> that fixed it and thought it was a mirroring lag.

Me too. I have actually been fighting with it on and off since we got sp/mmap reasonably stable. I just always kept attributing it to kernel.org mirroring lag until I realized it was just happening way too often.

-- 
Shawn.

← back to recent threads