git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 4/4] revision: avoid hitting packfiles when commits are in commit-graph

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 2, 2021, 20:01 UTC
Message-ID
<xmqqwnp3vcow.fsf@gitster.g>
In-Reply-To
<f6fc2a5e6d94befa915fb59b6296ce3153820c13.1627896460.git.ps@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 7 quoted lines
> diff --git a/commit-graph.c b/commit-graph.c
> index 3860a0d847..a81d5cebc0 100644
> --- a/commit-graph.c
> +++ b/commit-graph.c
> @@ -864,6 +864,48 @@ static int fill_commit_in_graph(struct repository *r,
>  	return 1;
>  }
Describe return-value here.  0 for not-found, !0 for found?
Show 15 quoted lines
> +static int find_object_id_in_graph(const struct object_id *id, struct commit_graph *g, uint32_t *pos)
> +{
> +	struct commit_graph *cur_g = g;
> +	uint32_t lex_index;
> +
> +	while (cur_g && !bsearch_graph(cur_g, (struct object_id *)id, &lex_index))
> +		cur_g = cur_g->base_graph;
> +
> +	if (cur_g) {
> +		*pos = lex_index + cur_g->num_commits_in_base;
> +		return 1;
> +	}
> +
> +	return 0;
> +}

Likewise, or as this is public, perhaps in commit-graph.h next to its declaration.

Show 9 quoted lines
> +int find_object_in_graph(struct repository *repo, struct object *object)
> +{
> +	struct commit *commit;
> +	uint32_t pos;
> +
> +	if (object->parsed) {
> +		if (object->type != OBJ_COMMIT)
> +			return -1;
> +		return 0;

This is puzzling---at least it is not consistent with what the function name says ("please say if you find _this_ object in the commit-graph file"---if that is not what this function does, it needs a comment before the implementation).

The caller had object and we has already been parsed. If the function were "with help from commit-graph, please tell me if you can positively say this is a commit", the above is understandable. If we know positively that it is not commit, we say "no, it is not a commit" (which may be suboptimal---if the caller falls back to another codepath, the object will still not be a commit) and if we know it is a commit, we can say "yes, it definitely is a commit" and the caller can stop there.

I guess my only problem with this function is that its name and what it does does not align. If the caller uses it for the real purpose of the function I guessed, then the logic itself may be OK.

> +	}
> +
> +	if (!repo->objects->commit_graph)
> +		return -1;

There is no commit-graph, then we decline to make a decision, which makes sense.

> +	if (!find_object_id_in_graph(&object->oid, repo->objects->commit_graph, &pos))
> +		return -1;
If it does not exist in the graph, we cannot tell, either.
Show 29 quoted lines
> +	commit = object_as_type(object, OBJ_COMMIT, 1);
> +	if (!commit)
> +		return -1;
> +	if (!fill_commit_in_graph(repo, commit, repo->objects->commit_graph, pos))
> +		return -1;
> +
> +	return 0;
> +}
> +
>  static int find_commit_in_graph(struct commit *item, struct commit_graph *g, uint32_t *pos)
>  {
>  	uint32_t graph_pos = commit_graph_position(item);
> @@ -871,18 +913,7 @@ static int find_commit_in_graph(struct commit *item, struct commit_graph *g, uin
>  		*pos = graph_pos;
>  		return 1;
>  	} else {
> -		struct commit_graph *cur_g = g;
> -		uint32_t lex_index;
> -
> -		while (cur_g && !bsearch_graph(cur_g, &(item->object.oid), &lex_index))
> -			cur_g = cur_g->base_graph;
> -
> -		if (cur_g) {
> -			*pos = lex_index + cur_g->num_commits_in_base;
> -			return 1;
> -		}
> -
> -		return 0;
> +		return find_object_id_in_graph(&item->object.oid, g, pos);

And I think this one is a op-op refactoring that does not change the behaviour of find_commit_in_graph()? It might be easier if done in a separate preparatory step, but it is small enough.

Show 33 quoted lines
> diff --git a/commit-graph.h b/commit-graph.h
> index 96c24fb577..f373fab4c0 100644
> --- a/commit-graph.h
> +++ b/commit-graph.h
> @@ -139,6 +139,8 @@ int write_commit_graph(struct object_directory *odb,
>  		       enum commit_graph_write_flags flags,
>  		       const struct commit_graph_opts *opts);
>  
> +int find_object_in_graph(struct repository *repo, struct object *object);
> +
>  #define COMMIT_GRAPH_VERIFY_SHALLOW	(1 << 0)
>  
>  int verify_commit_graph(struct repository *r, struct commit_graph *g, int flags);
> diff --git a/revision.c b/revision.c
> index 671b6d6513..c3f9cf2998 100644
> --- a/revision.c
> +++ b/revision.c
> @@ -362,10 +362,12 @@ static struct object *get_reference(struct rev_info *revs, const char *name,
>  	struct object *object = lookup_unknown_object(revs->repo, oid);
>  
>  	if (object->type == OBJ_NONE) {
> -		int type = oid_object_info(revs->repo, oid, NULL);
> -		if (type < 0 || !object_as_type(object, type, 1)) {
> -			object = NULL;
> -			goto out;
> +		if (find_object_in_graph(revs->repo, object) < 0) {
> +			int type = oid_object_info(revs->repo, oid, NULL);
> +			if (type < 0 || !object_as_type(object, type, 1)) {
> +				object = NULL;
> +				goto out;
> +			}
>  		}
>  	}
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 40 of 64 in “Speed up connectivity checks via bitmaps”
  1. 0/3 Speed up connectivity checks via bitmapsPatrick Steinhardt, Jun 28, 2021
  2. 1/3 p5400: add perf tests for git-receive-pack(1)Patrick Steinhardt, Jun 28, 2021
  3. Ævar Arnfjörð BjarmasonJun 28, 2021
  4. Patrick SteinhardtJun 29, 2021
  5. Ævar Arnfjörð BjarmasonJun 29, 2021
  6. 2/3 receive-pack: skip connectivity checks on delete-only commandsPatrick Steinhardt, Jun 28, 2021
  7. Ævar Arnfjörð BjarmasonJun 28, 2021
  8. Ævar Arnfjörð BjarmasonJun 28, 2021
  9. Patrick SteinhardtJun 29, 2021
  10. Jeff KingJun 30, 2021
  11. Jeff KingJun 30, 2021
  12. Patrick SteinhardtJun 30, 2021
  13. 3/3 connected: implement connectivity check using bitmapsPatrick Steinhardt, Jun 28, 2021
  14. Taylor BlauJun 28, 2021
  15. Taylor BlauJun 29, 2021
  16. Jeff KingJun 30, 2021
  17. Taylor BlauJun 30, 2021
  18. Jeff KingJun 30, 2021
  19. Taylor BlauJul 2, 2021
  20. Jeff KingJul 2, 2021
  21. Jeff KingJun 30, 2021
  22. Patrick SteinhardtJul 20, 2021
  23. 0/4 Speed up connectivity checksPatrick Steinhardt, Aug 2, 2021
  24. 1/4 connected: do not sort input revisionsPatrick Steinhardt, Aug 2, 2021
  25. Ævar Arnfjörð BjarmasonAug 2, 2021
  26. Patrick SteinhardtAug 3, 2021
  27. Ævar Arnfjörð BjarmasonAug 4, 2021
  28. Junio C HamanoAug 2, 2021
  29. Patrick SteinhardtAug 3, 2021
  30. Junio C HamanoAug 3, 2021
  31. 2/4 revision: stop retrieving reference twicePatrick Steinhardt, Aug 2, 2021
  32. Ævar Arnfjörð BjarmasonAug 2, 2021
  33. 3/4 revision: avoid loading object headers multiple timesPatrick Steinhardt, Aug 2, 2021
  34. Ævar Arnfjörð BjarmasonAug 2, 2021
  35. Patrick SteinhardtAug 5, 2021
  36. Junio C HamanoAug 2, 2021
  37. Patrick SteinhardtAug 3, 2021
  38. Patrick SteinhardtAug 6, 2021
  39. 4/4 revision: avoid hitting packfiles when commits are in commit-graphPatrick Steinhardt, Aug 2, 2021
  40. Junio C HamanoAug 2, 2021
  41. Patrick SteinhardtAug 3, 2021
  42. Junio C HamanoAug 3, 2021
  43. Patrick SteinhardtAug 5, 2021
  44. Junio C HamanoAug 5, 2021
  45. Ævar Arnfjörð BjarmasonAug 4, 2021
  46. 0/6 Speed up connectivity checksPatrick Steinhardt, Aug 5, 2021
  47. 1/6 revision: separate walk and unsorted flagsPatrick Steinhardt, Aug 5, 2021
  48. Junio C HamanoAug 5, 2021
  49. 2/6 connected: do not sort input revisionsPatrick Steinhardt, Aug 5, 2021
  50. Junio C HamanoAug 5, 2021
  51. Patrick SteinhardtAug 6, 2021
  52. Junio C HamanoAug 6, 2021
  53. 3/6 revision: stop retrieving reference twicePatrick Steinhardt, Aug 5, 2021
  54. 4/6 revision: avoid loading object headers multiple timesPatrick Steinhardt, Aug 5, 2021
  55. 5/6 commit-graph: split out function to search commit positionPatrick Steinhardt, Aug 5, 2021
  56. 6/6 revision: avoid hitting packfiles when commits are in commit-graphPatrick Steinhardt, Aug 5, 2021
  57. 0/5 Speed up connectivity checksPatrick Steinhardt, Aug 9, 2021
  58. Patrick SteinhardtAug 9, 2021
  59. 0/5 Speed up connectivity checksPatrick Steinhardt, Aug 9, 2021
  60. 1/5 revision: separate walk and unsorted flagsPatrick Steinhardt, Aug 9, 2021
  61. 2/5 connected: do not sort input revisionsPatrick Steinhardt, Aug 9, 2021
  62. 3/5 revision: stop retrieving reference twicePatrick Steinhardt, Aug 9, 2021
  63. 4/5 commit-graph: split out function to search commit positionPatrick Steinhardt, Aug 9, 2021
  64. 5/5 revision: avoid hitting packfiles when commits are in commit-graphPatrick Steinhardt, Aug 9, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.