git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 07/10] object_info: content_limit only applies to blobs

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 26, 2024, 22:02 UTC
Message-ID
<xmqqwmk3eydh.fsf@gitster.g>
In-Reply-To
<20240823224630.1180772-8-e@80x24.org>
Eric Wong <e@80x24.org> writes:
Show 28 quoted lines
> Streaming is only supported for blobs, so we'd end up having to
> slurp all the other object types into memory regardless.  So
> slurp all the non-blob types up front when requesting content
> since we always handle them in-core, anyways.
>
> Signed-off-by: Eric Wong <e@80x24.org>
> ---
>  builtin/cat-file.c  | 21 +++++++++++++++++++--
>  object-file.c       |  3 ++-
>  packfile.c          |  8 +++++---
>  t/t1006-cat-file.sh | 19 ++++++++++++++++---
>  4 files changed, 42 insertions(+), 9 deletions(-)
>
> diff --git a/builtin/cat-file.c b/builtin/cat-file.c
> index 8debcdca3e..2aedd62324 100644
> --- a/builtin/cat-file.c
> +++ b/builtin/cat-file.c
> @@ -385,7 +385,24 @@ static void print_object_or_die(struct batch_options *opt, struct expand_data *d
>  	assert(data->info.typep);
>  
>  	if (data->content) {
> -		batch_write(opt, data->content, data->size);
> +		void *content = data->content;
> +		unsigned long size = data->size;
> +
> +		data->content = NULL;
> +		if (use_mailmap && (data->type == OBJ_COMMIT ||
> +					data->type == OBJ_TAG)) {
Line-wrap at higher level in the parse tree, i.e.
		if (use_mailmap &&
		    (data->type == OBJ_COMMIT || data->type == OBJ_TAG)) {
Show 9 quoted lines
> +			size_t s = size;
> +
> +			if (data->info.whence == OI_DBCACHED) {
> +				content = xmemdupz(content, s);
> +				data->info.whence = OI_PACKED;
> +			}
> +
> +			content = replace_idents_using_mailmap(content, &s);
> +			size = cast_size_t_to_ulong(s);

This piece of code does look good, but it makes me wonder where the need to duplicate the code comes from. Before this patch, if somebody asked use_mailmap, we must have been making the replace_idents_using_mailmap() call and showing the result elsewhere in the existing code, and it is curious why that code path is not removed, or if we can share more common code with that code paths here.

Show 12 quoted lines
> +		}
> +
> +		batch_write(opt, content, size);
>  		switch (data->info.whence) {
>  		case OI_CACHED:
>  			/*
> @@ -395,7 +412,7 @@ static void print_object_or_die(struct batch_options *opt, struct expand_data *d
>  			BUG("TODO OI_CACHED support not done");
>  		case OI_LOOSE:
>  		case OI_PACKED:
> -			FREE_AND_NULL(data->content);
> +			free(content);

data->content has been nuked earlier, and content that holds the new copy with replaced one is freed (and the original data->content was freed by replace_idents_using_mailmap(), so there is no leak or double-free here. Good.

Show 14 quoted lines
> diff --git a/object-file.c b/object-file.c
> index 19100e823d..59842cfe1b 100644
> --- a/object-file.c
> +++ b/object-file.c
> @@ -1492,7 +1492,8 @@ static int loose_object_info(struct repository *r,
>  
>  		if (!oi->contentp)
>  			break;
> -		if (oi->content_limit && *oi->sizep > oi->content_limit) {
> +		if (oi->content_limit && *oi->typep == OBJ_BLOB &&
> +				*oi->sizep > oi->content_limit) {
>  			git_inflate_end(&stream);
>  			oi->contentp = NULL;
>  			goto cleanup;

OK, so non-BLOB objects we would fall through this if/else cascade and inflate them fully.

Previous: Eric WongNext: Eric Wong
Message 44 of 51 in “cat-file speedups”
  1. 00/10 cat-file speedupsEric Wong, Jul 15, 2024
  2. 01/10 packfile: move sizep computationEric Wong, Jul 15, 2024
  3. Patrick SteinhardtJul 24, 2024
  4. 02/10 packfile: allow content-limit for cat-fileEric Wong, Jul 15, 2024
  5. Patrick SteinhardtJul 24, 2024
  6. Eric WongJul 26, 2024
  7. 03/10 packfile: fix off-by-one in content_limit comparisonEric Wong, Jul 15, 2024
  8. Patrick SteinhardtJul 24, 2024
  9. Eric WongJul 26, 2024
  10. 04/10 packfile: inline cache_or_unpack_entryEric Wong, Jul 15, 2024
  11. 05/10 cat-file: use delta_base_cache entries directlyEric Wong, Jul 15, 2024
  12. Patrick SteinhardtJul 24, 2024
  13. Eric WongJul 26, 2024
  14. assert vs BUG [was: [PATCH v1 05/10] cat-file: use delta_base_cache entries directly]Eric Wong, Aug 18, 2024
  15. Junio C HamanoAug 19, 2024
  16. 06/10 packfile: packed_object_info avoids packed_to_object_typeEric Wong, Jul 15, 2024
  17. Patrick SteinhardtJul 24, 2024
  18. Eric WongJul 26, 2024
  19. 07/10 object_info: content_limit only applies to blobsEric Wong, Jul 15, 2024
  20. 08/10 cat-file: batch-command uses content_limitEric Wong, Jul 15, 2024
  21. 09/10 cat-file: batch_write: use size_t for lengthEric Wong, Jul 15, 2024
  22. 10/10 cat-file: use writev(2) if availableEric Wong, Jul 15, 2024
  23. Patrick SteinhardtJul 24, 2024
  24. 00/10 cat-file speedupsEric Wong, Aug 23, 2024
  25. 01/10 packfile: move sizep computationEric Wong, Aug 23, 2024
  26. Taylor BlauSep 17, 2024
  27. 02/10 packfile: allow content-limit for cat-fileEric Wong, Aug 23, 2024
  28. Junio C HamanoAug 26, 2024
  29. Eric WongAug 27, 2024
  30. Taylor BlauSep 17, 2024
  31. Junio C HamanoSep 17, 2024
  32. 03/10 packfile: fix off-by-one in content_limit comparisonEric Wong, Aug 23, 2024
  33. Junio C HamanoAug 26, 2024
  34. Taylor BlauSep 17, 2024
  35. 04/10 packfile: inline cache_or_unpack_entryEric Wong, Aug 23, 2024
  36. Junio C HamanoAug 26, 2024
  37. Eric WongOct 6, 2024
  38. 05/10 cat-file: use delta_base_cache entries directlyEric Wong, Aug 23, 2024
  39. Junio C HamanoAug 26, 2024
  40. Junio C HamanoAug 26, 2024
  41. 06/10 packfile: packed_object_info avoids packed_to_object_typeEric Wong, Aug 23, 2024
  42. Junio C HamanoAug 26, 2024
  43. 07/10 object_info: content_limit only applies to blobsEric Wong, Aug 23, 2024
  44. Junio C HamanoAug 26, 2024
  45. 08/10 cat-file: batch-command uses content_limitEric Wong, Aug 23, 2024
  46. Junio C HamanoAug 26, 2024
  47. 09/10 cat-file: batch_write: use size_t for lengthEric Wong, Aug 23, 2024
  48. Junio C HamanoAug 27, 2024
  49. 10/10 cat-file: use writev(2) if availableEric Wong, Aug 23, 2024
  50. Junio C HamanoAug 27, 2024
  51. Junio C HamanoAug 27, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.