git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2] pack-objects: use streaming interface for reading large loose blobs

From
Junio C Hamano <gitster@pobox.com>
Date
May 14, 2012, 19:43 UTC
Message-ID
<7v62byh7l4.fsf@alter.siamese.dyndns.org>
In-Reply-To
<1336883862-9013-1-git-send-email-pclouds@gmail.com>
Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
Show 13 quoted lines
> git usually streams large blobs directly to packs. But there are cases
> diff --git a/t/t1050-large.sh b/t/t1050-large.sh
> index 55ed955..7fbd2e1 100755
> --- a/t/t1050-large.sh
> +++ b/t/t1050-large.sh
> @@ -134,6 +134,22 @@ test_expect_success 'repack' '
>  	git repack -ad
>  '
>  
> +test_expect_success 'pack-objects with large loose object' '
> +	echo Z | dd of=large4 bs=1k seek=2000 &&
> +	OBJ=9f36d94e145816ec642592c09cc8e601d83af157 &&
> +	P=.git/objects/9f/36d94e145816ec642592c09cc8e601d83af157 &&

I do not think you need these hardcoded constants; you will run hash-object later, no?

Also, relying on $P to exist after hash-object -w returns is somewhat flaky, no?

> +	(
> +	unset GIT_ALLOC_LIMIT &&
sane_unset?
Show 5 quoted lines
> +	cat large4 | git hash-object -w --stdin &&
> +	git cat-file blob $OBJ >actual &&
> +	cmp large4 actual
> +	) &&
> +	echo $OBJ | git pack-objects .git/objects/pack/pack &&

If you do not write directly into this directory, and then create another _empty_ repository and deposit the packfile and its associated .idx file there, then you do not have to care how the earlier hash-object wrote its object. You are interested only in pack-objects producing a usable pack, and testing it like so will be the most direct way to test it, no?

> +	rm $P &&
> +	git cat-file blob $OBJ >actual &&
> +	cmp large4 actual
> +'

In any case, the patch when applied on top of cd07cc5 (Update draft release notes to 1.7.11 (11th batch), 2012-05-11) does not pass this part of the test on my box.

>  test_expect_success 'tar achiving' '
>  	git archive --format=tar HEAD >/dev/null
>  '
Previous: Junio C HamanoNext: Nguyen Thai Ngoc Duy
Message 5 of 16 in “pack-objects: use streaming interface for reading large loose blobs”
  1. pack-objects: use streaming interface for reading large loose blobsNguyễn Thái Ngọc Duy, May 12, 2012
  2. Nicolas PitreMay 12, 2012
  3. pack-objects: use streaming interface for reading large loose blobsNguyễn Thái Ngọc Duy, May 13, 2012
  4. Junio C HamanoMay 14, 2012
  5. Junio C HamanoMay 14, 2012
  6. Nguyen Thai Ngoc DuyMay 15, 2012
  7. Junio C HamanoMay 15, 2012
  8. Nguyen Thai Ngoc DuyMay 16, 2012
  9. 1/4 streaming: allow to call close_istream(NULL);Nguyễn Thái Ngọc Duy, May 16, 2012
  10. 2/4 pack-objects, streaming: turn "xx >= big_file_threshold" to ".. > .."Nguyễn Thái Ngọc Duy, May 16, 2012
  11. Junio C HamanoMay 18, 2012
  12. 3/4 pack-objects: refactor write_object()Nguyễn Thái Ngọc Duy, May 16, 2012
  13. Junio C HamanoMay 18, 2012
  14. Nicolas PitreMay 19, 2012
  15. 4/4 pack-objects: use streaming interface for reading large loose blobsNguyễn Thái Ngọc Duy, May 16, 2012
  16. Junio C HamanoMay 18, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.