git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/5] archive-tar: stream large blobs to tar file

From
René Scharfe <rene.scharfe@lsrfire.ath.cx>
Date
Apr 30, 2012, 21:08 UTC
Message-ID
<4F9EFF39.4010804@lsrfire.ath.cx>
In-Reply-To
<1335761837-12482-5-git-send-email-pclouds@gmail.com>
Am 30.04.2012 06:57, schrieb Nguyễn Thái Ngọc Duy:
Show 45 quoted lines
> 
> Signed-off-by: Nguyễn Thái Ngọc Duy<pclouds@gmail.com>
> ---
>   archive-tar.c    |   38 +++++++++++++++++++++++++++++++++++---
>   t/t1050-large.sh |    4 ++++
>   2 files changed, 39 insertions(+), 3 deletions(-)
> 
> diff --git a/archive-tar.c b/archive-tar.c
> index 61821f4..865ef6d 100644
> --- a/archive-tar.c
> +++ b/archive-tar.c
> @@ -4,6 +4,7 @@
>   #include "cache.h"
>   #include "tar.h"
>   #include "archive.h"
> +#include "streaming.h"
>   #include "run-command.h"
> 
>   #define RECORDSIZE	(512)
> @@ -62,6 +63,29 @@ static void write_blocked(const void *data, unsigned long size)
>   	write_if_needed();
>   }
> 
> +static int stream_blob_to_file(const unsigned char *sha1)
> +{
> +	struct git_istream *st;
> +	enum object_type type;
> +	unsigned long sz;
> +
> +	st = open_istream(sha1,&type,&sz, NULL);
> +	if (!st)
> +		return error("cannot stream blob %s", sha1_to_hex(sha1));
> +	for (;;) {
> +		char buf[BLOCKSIZE];
> +		ssize_t readlen;
> +
> +		readlen = read_istream(st, buf, sizeof(buf));
> +
> +		if (readlen <= 0)
> +			return readlen;
> +		write_blocked(buf, readlen);
> +	}
> +	close_istream(st);
> +	return 0;
> +}
The stream is never closed.  Perhaps squash this in?
diff --git a/archive-tar.c b/archive-tar.c
index 506c8cb..6109fd3 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -66,6 +66,7 @@ static void write_blocked(const void *data, unsigned long size)
 static int stream_blob_to_file(const unsigned char *sha1)
 {
 	struct git_istream *st;
+	ssize_t readlen;
 	enum object_type type;
 	unsigned long sz;
 
@@ -74,16 +75,15 @@ static int stream_blob_to_file(const unsigned char *sha1)
 		return error("cannot stream blob %s", sha1_to_hex(sha1));
 	for (;;) {
 		char buf[BLOCKSIZE];
-		ssize_t readlen;
 
 		readlen = read_istream(st, buf, sizeof(buf));
 
 		if (readlen <= 0)
-			return readlen;
+			break;
 		write_blocked(buf, readlen);
 	}
 	close_istream(st);
-	return 0;
+	return readlen;
 }
 
 /*
Previous: Junio C HamanoNext: Junio C Hamano
Message 10 of 23 in “Large file support for git-archive”
  1. 0/5 Large file support for git-archiveNguyễn Thái Ngọc Duy, Apr 30, 2012
  2. 1/5 archive-tar: turn write_tar_entry into blob-writing onlyNguyễn Thái Ngọc Duy, Apr 30, 2012
  3. Junio C HamanoApr 30, 2012
  4. René ScharfeApr 30, 2012
  5. 2/5 archive-tar: unindent write_tar_entry by one levelNguyễn Thái Ngọc Duy, Apr 30, 2012
  6. 3/5 archive: delegate blob reading to backendNguyễn Thái Ngọc Duy, Apr 30, 2012
  7. René ScharfeApr 30, 2012
  8. 4/5 archive-tar: stream large blobs to tar fileNguyễn Thái Ngọc Duy, Apr 30, 2012
  9. Junio C HamanoApr 30, 2012
  10. René ScharfeApr 30, 2012
  11. Junio C HamanoApr 30, 2012
  12. René ScharfeApr 30, 2012
  13. 5/5 archive-zip: stream large blobs into zip fileNguyễn Thái Ngọc Duy, Apr 30, 2012
  14. Junio C HamanoApr 30, 2012
  15. René ScharfeApr 30, 2012
  16. streaming: void pointer instead of char pointerRené Scharfe, Apr 30, 2012
  17. archive-zip: remove uncompressed_sizeRené Scharfe, Apr 30, 2012
  18. archive-zip: factor out helpers for writing sizes and CRCRené Scharfe, Apr 30, 2012
  19. archive-zip: streaming for stored filesRené Scharfe, Apr 30, 2012
  20. archive-zip: streaming for deflated filesRené Scharfe, Apr 30, 2012
  21. Junio C HamanoApr 30, 2012
  22. René ScharfeApr 30, 2012
  23. Nguyen Thai Ngoc DuyMay 1, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.