git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/3] upload-pack: skip parse-object re-hashing of "want" objects

From
Jeff King <peff@peff.net>
Date
Sep 7, 2022, 20:50 UTC
Message-ID
<YxkEAx/qq5Rmn/Yx@coredump.intra.peff.net>
In-Reply-To
<3f901613-72c6-b644-079a-f74e3024a8fe@github.com>
On Wed, Sep 07, 2022 at 10:45:39AM -0400, Derrick Stolee wrote:
Show 22 quoted lines
> After writing this, it was bothering me that 'rev-list --verify' should
> be checking for corruption throughout the history, not just at the tips.
> 
> This is done via the condition in builtin/rev-list.c:finish_object():
> 
> static int finish_object(struct object *obj, const char *name, void *cb_data)
> {
> 	struct rev_list_info *info = cb_data;
> 	if (oid_object_info_extended(the_repository, &obj->oid, NULL, 0) < 0) {
> 		finish_object__ma(obj);
> 		return 1;
> 	}
> 	if (info->revs->verify_objects && !obj->parsed && obj->type != OBJ_COMMIT)
> 		parse_object(the_repository, &obj->oid);
> 	return 0;
> }
> 
> So this call is the one that is used most-often by the rev-list command,
> but isn't necessary to update for the purpose of our desired speedup. This
> is another place where we would want to use read_object_file(). (It may even
> be the _only_ place, since finish_object() should be called even for the
> tip objects.)

Yeah, I think this is the only place. At least, if you remove the hash check entirely, then this parse_object() is the only spots that causes a problem (via the existing test in t1450).

> In case you think it is valuable to ensure we test both cases ("tip" and
> "not tip") I modified your test to have a third commit and test two rev-list
> calls:

I don't think it's important to test here, as we know we just touched the tip code here. But also, that existing "rev-list --verify-objects" test in t1450 is covering that case: it corrupts a blob that is reachable from an otherwise good commit/tree.

We should also have tip and non-tip tests for commits, but I posted those in a separate series. ;)

-Peff
Previous: Derrick StoleeNext: Junio C Hamano
Message 25 of 40 in “Partial-clone cause big performance impact on server”
  1. 程洋Aug 11, 2022
  2. Jonathan TanAug 11, 2022
  3. 回复: [External Mail]Re: Partial-clone cause big performance impact on server程洋, Aug 13, 2022
  4. 回复: [External Mail]Re: Partial-clone cause big performance impact on server程洋, Aug 13, 2022
  5. ZheNing HuAug 15, 2022
  6. 程洋Aug 15, 2022
  7. Derrick StoleeAug 12, 2022
  8. Jeff KingAug 14, 2022
  9. Derrick StoleeAug 15, 2022
  10. 程洋Aug 15, 2022
  11. 程洋Aug 17, 2022
  12. Derrick StoleeAug 17, 2022
  13. Jeff KingAug 18, 2022
  14. 程洋Sep 1, 2022
  15. Jeff KingSep 1, 2022
  16. 程洋Sep 5, 2022
  17. Jeff KingSep 6, 2022
  18. 0/3 speeding up on-demand fetch for blobs in partial cloneJeff King, Sep 6, 2022
  19. 1/3 parse_object(): allow skipping hash checkJeff King, Sep 6, 2022
  20. Derrick StoleeSep 7, 2022
  21. Jeff KingSep 7, 2022
  22. 2/3 upload-pack: skip parse-object re-hashing of "want" objectsJeff King, Sep 6, 2022
  23. Derrick StoleeSep 7, 2022
  24. Derrick StoleeSep 7, 2022
  25. Jeff KingSep 7, 2022
  26. Junio C HamanoSep 7, 2022
  27. Jeff KingSep 7, 2022
  28. [BUG] t1800: Fails for error text comparisonrsbecker@nexbridge.com, Sep 7, 2022
  29. Junio C HamanoSep 7, 2022
  30. rsbecker@nexbridge.comSep 7, 2022
  31. Jeff KingSep 7, 2022
  32. Junio C HamanoSep 7, 2022
  33. Jeff KingSep 8, 2022
  34. Junio C HamanoSep 8, 2022
  35. 3/3 parse_object(): check commit-graph when skip_hash setJeff King, Sep 6, 2022
  36. Derrick StoleeSep 7, 2022
  37. Junio C HamanoSep 7, 2022
  38. 程洋Sep 8, 2022
  39. Jeff KingSep 8, 2022
  40. Derrick StoleeSep 7, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.