git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/2] blob.c: remove buffer & size arguments to parse_blob_buffer()

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Apr 10, 2021, 13:01 UTC
Message-ID
<87v98ufg3i.fsf@evledraar.gmail.com>
In-Reply-To
<87y2dqfgbb.fsf@evledraar.gmail.com>
On Sat, Apr 10 2021, Ævar Arnfjörð Bjarmason wrote:
Show 73 quoted lines
> On Fri, Apr 09 2021, Jeff King wrote:
>
>> On Fri, Apr 09, 2021 at 10:07:27AM +0200, Ævar Arnfjörð Bjarmason wrote:
>>
>>> As noted in the comment introduced in 837d395a5c0 (Replace
>>> parse_blob() with an explanatory comment, 2010-01-18) the old
>>> parse_blob() function and the current parse_blob_buffer() exist merely
>>> to provide consistency in the API.
>>> 
>>> We're not going to parse blobs like we "parse" commits, trees or
>>> tags. So let's not have the parse_blob_buffer() take arguments that
>>> pretends that we do. Its only use is to set the "parsed" flag.
>>> 
>>> See bd2c39f58f9 ([PATCH] don't load and decompress objects twice with
>>> parse_object(), 2005-05-06) for the introduction of parse_blob_buffer().
>>
>> OK. Calling it parse_blob_buffer() is a little silly since it doesn't
>> even take a buffer anymore. But I guess parse_blob() might imply that it
>> actually loads the contents from disk to check them (which the other
>> parse_foo() functions do), so that's not a good name.
>>
>> So this might be the least bad thing. Given that there are only two
>> callers, just setting blob->object.parsed might not be unreasonable,
>> either. But I don't think it's worth spending too much time on.
>>
>>> @@ -266,7 +266,7 @@ struct object *parse_object(struct repository *r, const struct object_id *oid)
>>>  			error(_("hash mismatch %s"), oid_to_hex(oid));
>>>  			return NULL;
>>>  		}
>>> -		parse_blob_buffer(lookup_blob(r, oid), NULL, 0);
>>> +		parse_blob_buffer(lookup_blob(r, oid));
>>>  		return lookup_object(r, oid);
>>
>> Not new in your patch, but I wondered if this could cause a segfault
>> when lookup_blob() returns NULL. I _think_ the answer is "no". We'd hit
>> this code path when either:
>>
>>   - lookup_object() returns an object with type OBJ_BLOB, in which case
>>     lookup_blob() would return that same object
>>
>>   - lookup_object() returned NULL, in which case lookup_blob() will call
>>     it again, get NULL again, and then auto-create the blob and return
>>     it
>>
>> So I think it is OK. But there are a bunch of duplicate hash lookups in
>> this code. It would be clearer and more efficient as:
>>
>> diff --git a/object.c b/object.c
>> index 2c32691dc4..2dfa038f13 100644
>> --- a/object.c
>> +++ b/object.c
>> @@ -262,12 +262,14 @@ struct object *parse_object(struct repository *r, const struct object_id *oid)
>>  	if ((obj && obj->type == OBJ_BLOB && repo_has_object_file(r, oid)) ||
>>  	    (!obj && repo_has_object_file(r, oid) &&
>>  	     oid_object_info(r, oid, NULL) == OBJ_BLOB)) {
>> +		if (!obj)
>> +			obj = create_object(r, oid, alloc_blob_node(r));
>>  		if (check_object_signature(r, repl, NULL, 0, NULL) < 0) {
>>  			error(_("hash mismatch %s"), oid_to_hex(oid));
>>  			return NULL;
>>  		}
>> -		parse_blob_buffer(lookup_blob(r, oid), NULL, 0);
>> -		return lookup_object(r, oid);
>> +		parse_blob_buffer(obj, NULL, 0);
>> +		return obj;
>>  	}
>>  
>>  	buffer = repo_read_object_file(r, oid, &type, &size);
>>
>> but I doubt the efficiency matters much in practice. Those hash lookups
>> will be lost in the noise of computing the hash of the blob contents.
>
> I was trying to keep the changes smaller, but what about just doing this?:
Sent a bit too soon...:
Show 78 quoted lines
> diff --git a/blob.c b/blob.c
> index 182718aba9..69293e7d8e 100644
> --- a/blob.c
> +++ b/blob.c
> @@ -5,16 +5,16 @@
>  
>  const char *blob_type = "blob";
>  
> +struct blob *create_blob(struct repository *r, const struct object_id *oid)
> +{
> +	return create_object(r, oid, alloc_blob_node(r));
> +}
> +
>  struct blob *lookup_blob(struct repository *r, const struct object_id *oid)
>  {
>  	struct object *obj = lookup_object(r, oid);
>  	if (!obj)
> -		return create_object(r, oid, alloc_blob_node(r));
> -	return object_as_type(obj, OBJ_BLOB, 0);
> -}
> +		return create_blob(r, oid);
>  
> -int parse_blob_buffer(struct blob *item, void *buffer, unsigned long size)
> -{
> -	item->object.parsed = 1;
> -	return 0;
> +	return object_as_type(obj, OBJ_BLOB, 0);
>  }
> diff --git a/blob.h b/blob.h
> index 1664872055..ad34f0e9cc 100644
> --- a/blob.h
> +++ b/blob.h
> @@ -9,10 +9,9 @@ struct blob {
>  	struct object object;
>  };
>  
> +struct blob *create_blob(struct repository *r, const struct object_id *oid);
>  struct blob *lookup_blob(struct repository *r, const struct object_id *oid);
>  
> -int parse_blob_buffer(struct blob *item, void *buffer, unsigned long size);
> -
>  /**
>   * Blobs do not contain references to other objects and do not have
>   * structured data that needs parsing. However, code may use the
> diff --git a/object.c b/object.c
> index 78343781ae..2699431404 100644
> --- a/object.c
> +++ b/object.c
> @@ -195,8 +195,7 @@ struct object *parse_object_buffer(struct repository *r, const struct object_id
>  	if (type == OBJ_BLOB) {
>  		struct blob *blob = lookup_blob(r, oid);
>  		if (blob) {
> -			if (parse_blob_buffer(blob, buffer, size))
> -				return NULL;
> +			blob->object.parsed = 1;
>  			obj = &blob->object;
>  		}
>  	} else if (type == OBJ_TREE) {
> @@ -262,12 +261,16 @@ struct object *parse_object(struct repository *r, const struct object_id *oid)
>  	if ((obj && obj->type == OBJ_BLOB && repo_has_object_file(r, oid)) ||
>  	    (!obj && repo_has_object_file(r, oid) &&
>  	     oid_object_info(r, oid, NULL) == OBJ_BLOB)) {
> +		struct blob *blob;
> +		if (!obj)
> +			blob = create_blob(r, oid);
>
>  		if (check_object_signature(r, repl, NULL, 0, NULL) < 0) {
>  			error(_("hash mismatch %s"), oid_to_hex(oid));
>  			return NULL;
>  		}
> -		parse_blob_buffer(lookup_blob(r, oid), NULL, 0);
> -		return lookup_object(r, oid);
> +		obj = &blob->object;
> +		obj->parsed = 1;
> +		return obj;
>  	}
>  
>  	buffer = repo_read_object_file(r, oid, &type, &size);

Well, aside from this segfault think-o introduced while experimenting. Needs to be:

@@ -262,12 +261,16 @@ struct object *parse_object(struct repository *r, const struct object_id *oid)
 	if ((obj && obj->type == OBJ_BLOB && repo_has_object_file(r, oid)) ||
 	    (!obj && repo_has_object_file(r, oid) &&
 	     oid_object_info(r, oid, NULL) == OBJ_BLOB)) {
+		if (!obj) {
+			struct blob *blob = create_blob(r, oid);
+			obj = &blob->object;
+		}
 		if (check_object_signature(r, repl, NULL, 0, NULL) < 0) {
 			error(_("hash mismatch %s"), oid_to_hex(oid));
 			return NULL;
 		}
-		parse_blob_buffer(lookup_blob(r, oid), NULL, 0);
-		return lookup_object(r, oid);
+		obj->parsed = 1;
+		return obj;
 	}
 
 	buffer = repo_read_object_file(r, oid, &type, &size);
Previous: Ævar Arnfjörð BjarmasonNext: Jeff King
Message 51 of 142 in “Pull objects of various types”
  1. 0/2 Pull objects of various typesDaniel Barkalow, Jun 22, 2005
  2. 1/2 Parse tags for absent objectsDaniel Barkalow, Jun 22, 2005
  3. 2/7 object.c: make type_from_string() return "enum object_type"Ævar Arnfjörð Bjarmason, Mar 8, 2021
  4. Taylor BlauMar 8, 2021
  5. Junio C HamanoMar 8, 2021
  6. 0/7 improve reporting of unexpected objectsÆvar Arnfjörð Bjarmason, Mar 8, 2021
  7. 00/10 improve reporting of unexpected objectsÆvar Arnfjörð Bjarmason, Mar 28, 2021
  8. 03/10 object.c: make type_from_string() return "enum object_type"Ævar Arnfjörð Bjarmason, Mar 28, 2021
  9. 01/10 object.c: stop supporting len == -1 in type_from_string_gently()Ævar Arnfjörð Bjarmason, Mar 28, 2021
  10. Junio C HamanoMar 28, 2021
  11. Ævar Arnfjörð BjarmasonMar 28, 2021
  12. Junio C HamanoMar 28, 2021
  13. Felipe ContrerasApr 22, 2021
  14. 04/10 object-file.c: make oid_object_info() return "enum object_type"Ævar Arnfjörð Bjarmason, Mar 28, 2021
  15. 06/10 tree.c: fix misindentation in parse_tree_gently()Ævar Arnfjörð Bjarmason, Mar 28, 2021
  16. 05/10 object-name.c: make dependency on object_type order more obviousÆvar Arnfjörð Bjarmason, Mar 28, 2021
  17. 07/10 object.c: add a utility function for "expected type X, got Y"Ævar Arnfjörð Bjarmason, Mar 28, 2021
  18. 02/10 object.c: refactor type_from_string_gently()Ævar Arnfjörð Bjarmason, Mar 28, 2021
  19. 08/10 object.c: add and use oid_is_type_or_die_msg() functionÆvar Arnfjörð Bjarmason, Mar 28, 2021
  20. 10/10 tag: don't misreport type of tagged objects in errorsÆvar Arnfjörð Bjarmason, Mar 28, 2021
  21. Junio C HamanoMar 30, 2021
  22. Jeff KingMar 31, 2021
  23. Junio C HamanoMar 31, 2021
  24. Ævar Arnfjörð BjarmasonMar 31, 2021
  25. Jeff KingMar 31, 2021
  26. Ævar Arnfjörð BjarmasonMar 31, 2021
  27. Jeff KingApr 1, 2021
  28. ref-filter: fix NULL check for parse object failureJeff King, Apr 1, 2021
  29. 0/5 mktag tests & fix for-each-ref segfaultÆvar Arnfjörð Bjarmason, Apr 1, 2021
  30. 5/5 ref-filter: fix NULL check for parse object failureÆvar Arnfjörð Bjarmason, Apr 1, 2021
  31. Ramsay JonesApr 1, 2021
  32. 4/5 mktag tests: test for maybe segfaulting for-each-refÆvar Arnfjörð Bjarmason, Apr 1, 2021
  33. 2/5 mktag tests: invert --no-strict testÆvar Arnfjörð Bjarmason, Apr 1, 2021
  34. 1/5 mktag tests: parse out options in helperÆvar Arnfjörð Bjarmason, Apr 1, 2021
  35. 3/5 mktag tests: do fsck on failureÆvar Arnfjörð Bjarmason, Apr 1, 2021
  36. Junio C HamanoApr 1, 2021
  37. Ævar Arnfjörð BjarmasonApr 2, 2021
  38. Junio C HamanoApr 2, 2021
  39. Junio C HamanoApr 1, 2021
  40. Junio C HamanoMar 31, 2021
  41. Jeff KingMar 31, 2021
  42. 09/10 object tests: add test for unexpected objects in tagsÆvar Arnfjörð Bjarmason, Mar 28, 2021
  43. Jeff KingMar 28, 2021
  44. Ævar Arnfjörð BjarmasonMar 29, 2021
  45. Jeff KingMar 31, 2021
  46. 0/2 blob/object.c: trivial readability improvementsÆvar Arnfjörð Bjarmason, Apr 9, 2021
  47. 1/2 blob.c: remove buffer & size arguments to parse_blob_buffer()Ævar Arnfjörð Bjarmason, Apr 9, 2021
  48. Jeff KingApr 9, 2021
  49. Junio C HamanoApr 9, 2021
  50. Ævar Arnfjörð BjarmasonApr 10, 2021
  51. Ævar Arnfjörð BjarmasonApr 10, 2021
  52. Jeff KingApr 13, 2021
  53. 2/2 object.c: initialize automatic variable in lookup_object()Ævar Arnfjörð Bjarmason, Apr 9, 2021
  54. Jeff KingApr 9, 2021
  55. Junio C HamanoApr 9, 2021
  56. 0/6 {tag,object}*.c: refactorings + prep for a larger changeÆvar Arnfjörð Bjarmason, Apr 9, 2021
  57. 1/6 object.c: stop supporting len == -1 in type_from_string_gently()Ævar Arnfjörð Bjarmason, Apr 9, 2021
  58. Jeff KingApr 9, 2021
  59. Jeff KingApr 9, 2021
  60. 3/6 object.c: make type_from_string() return "enum object_type"Ævar Arnfjörð Bjarmason, Apr 9, 2021
  61. Jeff KingApr 9, 2021
  62. Ævar Arnfjörð BjarmasonApr 9, 2021
  63. Jeff KingApr 9, 2021
  64. 2/6 object.c: remove "gently" argument to type_from_string_gently()Ævar Arnfjörð Bjarmason, Apr 9, 2021
  65. Jeff KingApr 9, 2021
  66. 4/6 object-file.c: make oid_object_info() return "enum object_type"Ævar Arnfjörð Bjarmason, Apr 9, 2021
  67. Jeff KingApr 9, 2021
  68. 6/6 tag.c: use type_from_string_gently() when parsing tagsÆvar Arnfjörð Bjarmason, Apr 9, 2021
  69. Jeff KingApr 9, 2021
  70. 5/6 object-name.c: make dependency on object_type order more obviousÆvar Arnfjörð Bjarmason, Apr 9, 2021
  71. Jeff KingApr 9, 2021
  72. 0/7 object.c: add and use "is expected" utility function + object_as_type() useÆvar Arnfjörð Bjarmason, Apr 9, 2021
  73. 2/7 object.c: add a utility function for "expected type X, got Y"Ævar Arnfjörð Bjarmason, Apr 9, 2021
  74. 1/7 tree.c: fix misindentation in parse_tree_gently()Ævar Arnfjörð Bjarmason, Apr 9, 2021
  75. 3/7 object.c: add and use oid_is_type_or_die_msg() functionÆvar Arnfjörð Bjarmason, Apr 9, 2021
  76. 4/7 commit-graph: use obj->type, not object_as_type()Ævar Arnfjörð Bjarmason, Apr 9, 2021
  77. 5/7 commit.c: don't use deref_tag() -> object_as_type()Ævar Arnfjörð Bjarmason, Apr 9, 2021
  78. 6/7 object.c: normalize brace style in object_as_type()Ævar Arnfjörð Bjarmason, Apr 9, 2021
  79. 7/7 object.c: remove "quiet" parameter from object_as_type()Ævar Arnfjörð Bjarmason, Apr 9, 2021
  80. 0/8 object.c: add and use "is expected" utility function + object_as_type() useÆvar Arnfjörð Bjarmason, Apr 20, 2021
  81. 1/8 tree.c: fix misindentation in parse_tree_gently()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  82. 2/8 object.c: add a utility function for "expected type X, got Y"Ævar Arnfjörð Bjarmason, Apr 20, 2021
  83. Jonathan TanApr 21, 2021
  84. Ævar Arnfjörð BjarmasonApr 22, 2021
  85. 3/8 object.c: add and use oid_is_type_or_die_msg() functionÆvar Arnfjörð Bjarmason, Apr 20, 2021
  86. Jonathan TanApr 21, 2021
  87. Josh SteadmonApr 21, 2021
  88. Junio C HamanoApr 28, 2021
  89. 4/8 commit-graph: use obj->type, not object_as_type()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  90. 5/8 branch tests: assert lookup_commit_reference_gently() errorÆvar Arnfjörð Bjarmason, Apr 20, 2021
  91. 6/8 commit.c: don't use deref_tag() -> object_as_type()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  92. Jonathan TanApr 21, 2021
  93. 7/8 object.c: normalize brace style in object_as_type()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  94. 8/8 object.c: remove "quiet" parameter from object_as_type()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  95. 00/10 {tag,object}*.c: refactorings + prep for a larger changeÆvar Arnfjörð Bjarmason, Apr 20, 2021
  96. 01/10 object.c: stop supporting len == -1 in type_from_string_gently()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  97. 02/10 object.c: remove "gently" argument to type_from_string_gently()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  98. 03/10 object.c: make type_from_string() return "enum object_type"Ævar Arnfjörð Bjarmason, Apr 20, 2021
  99. 04/10 object-file.c: make oid_object_info() return "enum object_type"Ævar Arnfjörð Bjarmason, Apr 20, 2021
  100. 05/10 object-name.c: make dependency on object_type order more obviousÆvar Arnfjörð Bjarmason, Apr 20, 2021
  101. 06/10 tag.c: use type_from_string_gently() when parsing tagsÆvar Arnfjörð Bjarmason, Apr 20, 2021
  102. 07/10 hash-object: pass along type length to object.cÆvar Arnfjörð Bjarmason, Apr 20, 2021
  103. 09/10 hash-object: show usage on invalid --typeÆvar Arnfjörð Bjarmason, Apr 20, 2021
  104. 08/10 hash-object: refactor nested else/if/if into else if/else ifÆvar Arnfjörð Bjarmason, Apr 20, 2021
  105. 10/10 object.c: move type_from_string() code to its last userÆvar Arnfjörð Bjarmason, Apr 20, 2021
  106. 00/10 object.c et al: tests, small bug fixes etc.Ævar Arnfjörð Bjarmason, Apr 20, 2021
  107. 01/10 cat-file tests: test for bogus type name handlingÆvar Arnfjörð Bjarmason, Apr 20, 2021
  108. Junio C HamanoApr 29, 2021
  109. 03/10 mktree tests: add test for invalid object typeÆvar Arnfjörð Bjarmason, Apr 20, 2021
  110. 04/10 object-file.c: take type id, not string, in read_object_with_reference()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  111. Junio C HamanoApr 29, 2021
  112. 05/10 {commit,tree,blob,tag}.c: add a create_{commit,tree,blob,tag}()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  113. Junio C HamanoApr 29, 2021
  114. Ævar Arnfjörð BjarmasonApr 29, 2021
  115. 02/10 hash-object tests: more detailed test for invalid typeÆvar Arnfjörð Bjarmason, Apr 20, 2021
  116. 06/10 blob.c: remove parse_blob_buffer()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  117. Junio C HamanoApr 29, 2021
  118. 08/10 object.c: don't go past "len" under die() in type_from_string_gently()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  119. Junio C HamanoApr 29, 2021
  120. 07/10 object.c: simplify return semantic of parse_object_buffer()Ævar Arnfjörð Bjarmason, Apr 20, 2021
  121. 09/10 mktree: stop setting *ntr++ to NILÆvar Arnfjörð Bjarmason, Apr 20, 2021
  122. Junio C HamanoApr 29, 2021
  123. 10/10 mktree: emit a more detailed error when the <type> is invalidÆvar Arnfjörð Bjarmason, Apr 20, 2021
  124. 3/7 oid_object_info(): return "enum object_type"Ævar Arnfjörð Bjarmason, Mar 8, 2021
  125. Junio C HamanoMar 8, 2021
  126. Junio C HamanoMar 8, 2021
  127. Jeff KingMar 9, 2021
  128. 4/7 tree.c: fix misindentation in parse_tree_gently()Ævar Arnfjörð Bjarmason, Mar 8, 2021
  129. 5/7 object.c: add a utility function for "expected type X, got Y"Ævar Arnfjörð Bjarmason, Mar 8, 2021
  130. Taylor BlauMar 8, 2021
  131. Junio C HamanoMar 8, 2021
  132. 1/7 object.c: refactor type_from_string_gently()Ævar Arnfjörð Bjarmason, Mar 8, 2021
  133. Taylor BlauMar 8, 2021
  134. Jeff KingMar 9, 2021
  135. 6/7 object tests: add test for unexpected objects in tagsÆvar Arnfjörð Bjarmason, Mar 8, 2021
  136. Jeff KingMar 9, 2021
  137. Ævar Arnfjörð BjarmasonMar 28, 2021
  138. Jeff KingMar 28, 2021
  139. Ævar Arnfjörð BjarmasonMar 28, 2021
  140. Jeff KingMar 29, 2021
  141. 7/7 tag: don't misreport type of tagged objects in errorsÆvar Arnfjörð Bjarmason, Mar 8, 2021
  142. 2/2 Pull misc objectsDaniel Barkalow, Jun 22, 2005

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.