git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/5] upload-pack: look up "want" lines via commit-graph

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 1, 2022, 08:43 UTC
Message-ID
<Yh3cprbfc/UQ01fo@ncase>
In-Reply-To
<ddb91a1d-6ddf-ba25-f1af-ba3f4c18726e@github.com>
On Wed, Feb 23, 2022 at 09:13:53AM -0500, Derrick Stolee wrote:
Show 31 quoted lines
> On 2/23/2022 7:35 AM, Patrick Steinhardt wrote:
> 
> > The following benchmark is executed in a repository with a huge number
> > of references. It uses cached request from git-fetch(1) as input and
> > contains about 876,000 "want" lines:
> > 
> >     Benchmark 1: git-upload-pack (HEAD~)
> >       Time (mean ± σ):      7.113 s ±  0.028 s    [User: 6.900 s, System: 0.662 s]
> >       Range (min … max):    7.072 s …  7.168 s    10 runs
> > 
> >     Benchmark 2: git-upload-pack (HEAD)
> >       Time (mean ± σ):      6.622 s ±  0.061 s    [User: 6.452 s, System: 0.650 s]
> >       Range (min … max):    6.535 s …  6.727 s    10 runs
> > 
> >     Summary
> >       'git-upload-pack (HEAD)' ran
> >         1.07 ± 0.01 times faster than 'git-upload-pack (HEAD~)'
> 
> Nice!
> 
> > -		o = parse_object(the_repository, &oid);
> > +		commit = lookup_commit_in_graph(the_repository, &oid);
> > +		if (commit)
> > +			o = &commit->object;
> > +		else
> > +			o = parse_object(the_repository, &oid);
> > +
> 
> This is a neat trick. I see that we've also done this trick in
> revision.c:get_reference(). Perhaps it is worth creating a helper,
> maybe named parse_probably_commit()?

That might be a good idea, thanks. I'll have a look at what the end result would look like.

Patrick
Show 33 quoted lines
> >  		if (!o) {
> >  			packet_writer_error(writer,
> >  					    "upload-pack: not our ref %s",
> > @@ -1434,7 +1440,7 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
> >  	if (skip_prefix(line, "want-ref ", &refname_nons)) {
> >  		struct object_id oid;
> >  		struct string_list_item *item;
> > -		struct object *o;
> > +		struct object *o = NULL;
> >  		struct strbuf refname = STRBUF_INIT;
> >  
> >  		strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons);
> > @@ -1448,7 +1454,15 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
> >  		item = string_list_append(wanted_refs, refname_nons);
> >  		item->util = oiddup(&oid);
> >  
> > -		o = parse_object_or_die(&oid, refname_nons);
> > +		if (!starts_with(refname_nons, "refs/tags/")) {
> > +			struct commit *commit = lookup_commit_in_graph(the_repository, &oid);
> > +			if (commit)
> > +				o = &commit->object;
> > +		}
> > +
> > +		if (!o)
> > +			o = parse_object_or_die(&oid, refname_nons);
> > +
> 
> Even here, we _could_ use a parse_probably_commit() helper
> inside the if (!starts_with(...)) block, even though we would
> still need the if (!o) check later.
> 
> Thanks,
> -Stolee
Previous: Derrick StoleeNext: Patrick Steinhardt
Message 7 of 20 in “fetch: more optimizations for mirror fetches”
  1. 0/5 fetch: more optimizations for mirror fetchesPatrick Steinhardt, Feb 23, 2022
  2. 2/5 fetch: avoid lookup of commits when not appending to FETCH_HEADPatrick Steinhardt, Feb 23, 2022
  3. Derrick StoleeFeb 23, 2022
  4. Patrick SteinhardtMar 1, 2022
  5. 1/5 upload-pack: look up "want" lines via commit-graphPatrick Steinhardt, Feb 23, 2022
  6. Derrick StoleeFeb 23, 2022
  7. Patrick SteinhardtMar 1, 2022
  8. Patrick SteinhardtMar 1, 2022
  9. Derrick StoleeMar 2, 2022
  10. 3/5 refs: add ability for backends to special-case reading of symbolic refsPatrick Steinhardt, Feb 23, 2022
  11. 4/5 remote: read symbolic refs via `refs_read_symbolic_ref()`Patrick Steinhardt, Feb 23, 2022
  12. 5/5 refs/files-backend: optimize reading of symbolic refsPatrick Steinhardt, Feb 23, 2022
  13. 0/5 fetch: more optimizations for mirror fetchesPatrick Steinhardt, Mar 1, 2022
  14. 1/5 upload-pack: look up "want" lines via commit-graphPatrick Steinhardt, Mar 1, 2022
  15. 2/5 fetch: avoid lookup of commits when not appending to FETCH_HEADPatrick Steinhardt, Mar 1, 2022
  16. 3/5 refs: add ability for backends to special-case reading of symbolic refsPatrick Steinhardt, Mar 1, 2022
  17. 4/5 remote: read symbolic refs via `refs_read_symbolic_ref()`Patrick Steinhardt, Mar 1, 2022
  18. 5/5 refs/files-backend: optimize reading of symbolic refsPatrick Steinhardt, Mar 1, 2022
  19. Junio C HamanoMar 1, 2022
  20. Derrick StoleeMar 2, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.