git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/5] upload-pack: look up "want" lines via commit-graph

From
Derrick Stolee <derrickstolee@github.com>
Date
Feb 23, 2022, 14:13 UTC
Message-ID
<ddb91a1d-6ddf-ba25-f1af-ba3f4c18726e@github.com>
In-Reply-To
<ca5e136cca495c7d927e99f5ae8a672d93823eea.1645619224.git.ps@pks.im>
On 2/23/2022 7:35 AM, Patrick Steinhardt wrote:
Show 15 quoted lines
> The following benchmark is executed in a repository with a huge number
> of references. It uses cached request from git-fetch(1) as input and
> contains about 876,000 "want" lines:
> 
>     Benchmark 1: git-upload-pack (HEAD~)
>       Time (mean ± σ):      7.113 s ±  0.028 s    [User: 6.900 s, System: 0.662 s]
>       Range (min … max):    7.072 s …  7.168 s    10 runs
> 
>     Benchmark 2: git-upload-pack (HEAD)
>       Time (mean ± σ):      6.622 s ±  0.061 s    [User: 6.452 s, System: 0.650 s]
>       Range (min … max):    6.535 s …  6.727 s    10 runs
> 
>     Summary
>       'git-upload-pack (HEAD)' ran
>         1.07 ± 0.01 times faster than 'git-upload-pack (HEAD~)'
Nice!
Show 7 quoted lines
> -		o = parse_object(the_repository, &oid);
> +		commit = lookup_commit_in_graph(the_repository, &oid);
> +		if (commit)
> +			o = &commit->object;
> +		else
> +			o = parse_object(the_repository, &oid);
> +

This is a neat trick. I see that we've also done this trick in revision.c:get_reference(). Perhaps it is worth creating a helper, maybe named parse_probably_commit()?

Show 26 quoted lines
>  		if (!o) {
>  			packet_writer_error(writer,
>  					    "upload-pack: not our ref %s",
> @@ -1434,7 +1440,7 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
>  	if (skip_prefix(line, "want-ref ", &refname_nons)) {
>  		struct object_id oid;
>  		struct string_list_item *item;
> -		struct object *o;
> +		struct object *o = NULL;
>  		struct strbuf refname = STRBUF_INIT;
>  
>  		strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons);
> @@ -1448,7 +1454,15 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
>  		item = string_list_append(wanted_refs, refname_nons);
>  		item->util = oiddup(&oid);
>  
> -		o = parse_object_or_die(&oid, refname_nons);
> +		if (!starts_with(refname_nons, "refs/tags/")) {
> +			struct commit *commit = lookup_commit_in_graph(the_repository, &oid);
> +			if (commit)
> +				o = &commit->object;
> +		}
> +
> +		if (!o)
> +			o = parse_object_or_die(&oid, refname_nons);
> +

Even here, we _could_ use a parse_probably_commit() helper inside the if (!starts_with(...)) block, even though we would still need the if (!o) check later.

Thanks, -Stolee

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 6 of 20 in “fetch: more optimizations for mirror fetches”
  1. 0/5 fetch: more optimizations for mirror fetchesPatrick Steinhardt, Feb 23, 2022
  2. 2/5 fetch: avoid lookup of commits when not appending to FETCH_HEADPatrick Steinhardt, Feb 23, 2022
  3. Derrick StoleeFeb 23, 2022
  4. Patrick SteinhardtMar 1, 2022
  5. 1/5 upload-pack: look up "want" lines via commit-graphPatrick Steinhardt, Feb 23, 2022
  6. Derrick StoleeFeb 23, 2022
  7. Patrick SteinhardtMar 1, 2022
  8. Patrick SteinhardtMar 1, 2022
  9. Derrick StoleeMar 2, 2022
  10. 3/5 refs: add ability for backends to special-case reading of symbolic refsPatrick Steinhardt, Feb 23, 2022
  11. 4/5 remote: read symbolic refs via `refs_read_symbolic_ref()`Patrick Steinhardt, Feb 23, 2022
  12. 5/5 refs/files-backend: optimize reading of symbolic refsPatrick Steinhardt, Feb 23, 2022
  13. 0/5 fetch: more optimizations for mirror fetchesPatrick Steinhardt, Mar 1, 2022
  14. 1/5 upload-pack: look up "want" lines via commit-graphPatrick Steinhardt, Mar 1, 2022
  15. 2/5 fetch: avoid lookup of commits when not appending to FETCH_HEADPatrick Steinhardt, Mar 1, 2022
  16. 3/5 refs: add ability for backends to special-case reading of symbolic refsPatrick Steinhardt, Mar 1, 2022
  17. 4/5 remote: read symbolic refs via `refs_read_symbolic_ref()`Patrick Steinhardt, Mar 1, 2022
  18. 5/5 refs/files-backend: optimize reading of symbolic refsPatrick Steinhardt, Mar 1, 2022
  19. Junio C HamanoMar 1, 2022
  20. Derrick StoleeMar 2, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.