git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 5/5] index-pack: repack local links into promisor packs

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Nov 1, 2024, 20:14 UTC
Message-ID
<20241101201448.1695516-1-jonathantanmy@google.com>
In-Reply-To
<radxsrv6sjemdzl2mw5zzkieyim6xfikrevwggjmzi774g2sob@4nx7fwcjfk32>
Josh Steadmon <steadmon@google.com> writes:
Show 45 quoted lines
> > @@ -1719,6 +1772,57 @@ static void show_pack_info(int stat_only)
> >  	free(chain_histogram);
> >  }
> >  
> > +static void repack_local_links(void)
> > +{
> > +	struct child_process cmd = CHILD_PROCESS_INIT;
> > +	FILE *out;
> > +	struct strbuf line = STRBUF_INIT;
> > +	struct oidset_iter iter;
> > +	struct object_id *oid;
> > +	char *base_name;
> > +
> > +	if (!oidset_size(&local_links))
> > +		return;
> > +
> > +	base_name = mkpathdup("%s/pack/pack", repo_get_object_directory(the_repository));
> > +
> > +	strvec_push(&cmd.args, "pack-objects");
> > +	strvec_push(&cmd.args, "--exclude-promisor-objects-best-effort");
> > +	strvec_push(&cmd.args, base_name);
> > +	cmd.git_cmd = 1;
> > +	cmd.in = -1;
> > +	cmd.out = -1;
> > +	if (start_command(&cmd))
> > +		die(_("could not start pack-objects to repack local links"));
> > +
> > +	oidset_iter_init(&local_links, &iter);
> > +	while ((oid = oidset_iter_next(&iter))) {
> > +		if (write_in_full(cmd.in, oid_to_hex(oid), the_hash_algo->hexsz) < 0 ||
> > +		    write_in_full(cmd.in, "\n", 1) < 0)
> > +			die(_("failed to feed local object to pack-objects"));
> > +	}
> > +	close(cmd.in);
> > +
> > +	out = xfdopen(cmd.out, "r");
> > +	while (strbuf_getline_lf(&line, out) != EOF) {
> > +		unsigned char binary[GIT_MAX_RAWSZ];
> > +		if (line.len != the_hash_algo->hexsz ||
> > +		    !hex_to_bytes(binary, line.buf, line.len))
> > +			die(_("index-pack: Expecting full hex object ID lines only from pack-objects."));
> 
> I'm not sure why we check the pack-objects output here, is this just to
> detect errors? Could we instead just check the exit status of
> pack-objects, and discard the output?

The output is a hex object ID that tells us what we need to name the .promisor file. (Later in this function, "binary" is used.) So we can't discard it.

Show 22 quoted lines
> > diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> > index e15fbaeb21..a565ab9b40 100644
> > --- a/builtin/pack-objects.c
> > +++ b/builtin/pack-objects.c
> > @@ -4310,6 +4310,18 @@ static int option_parse_cruft_expiration(const struct option *opt UNUSED,
> >  	return 0;
> >  }
> >  
> > +static int should_include_obj(struct object *obj, void *data UNUSED)
> > +{
> > +	struct object_info info = OBJECT_INFO_INIT;
> > +	if (oid_object_info_extended(the_repository, &obj->oid, &info, 0))
> > +		BUG("should_include_obj should only be called on existing objects");
> > +	return info.whence != OI_PACKED || !info.u.packed.pack->pack_promisor;
> > +}
> > +
> > +static int should_include(struct commit *commit, void *data) {
> > +	return should_include_obj((struct object *) commit, data);
> > +}
> > +
> 
> Nit: these two functions could be named a bit more descriptively.
OK, done.
Previous: Josh SteadmonNext: Han Young
Message 12 of 37 in “When fetching from a promisor remote, repack local objects referenced”
  1. 0/5 When fetching from a promisor remote, repack local objects referencedJonathan Tan, Oct 24, 2024
  2. 1/5 pack-objects: make variable non-staticJonathan Tan, Oct 24, 2024
  3. Taylor BlauOct 28, 2024
  4. Jonathan TanOct 28, 2024
  5. Taylor BlauOct 28, 2024
  6. Jonathan TanOct 28, 2024
  7. 2/5 t0410: make test description clearerJonathan Tan, Oct 24, 2024
  8. 3/5 t0410: use from-scratch serverJonathan Tan, Oct 24, 2024
  9. 4/5 t5300: move --window clamp test next to unclampedJonathan Tan, Oct 24, 2024
  10. 5/5 index-pack: repack local links into promisor packsJonathan Tan, Oct 24, 2024
  11. Josh SteadmonOct 30, 2024
  12. Jonathan TanNov 1, 2024
  13. Han YoungOct 25, 2024
  14. Taylor BlauOct 25, 2024
  15. Junio C HamanoNov 2, 2024
  16. Taylor BlauOct 25, 2024
  17. 0/4 When fetching from a promisor remote, repack local objects referencedJonathan Tan, Nov 1, 2024
  18. 1/4 t0410: make test description clearerJonathan Tan, Nov 1, 2024
  19. 2/4 t0410: use from-scratch serverJonathan Tan, Nov 1, 2024
  20. 3/4 t5300: move --window clamp test next to unclampedJonathan Tan, Nov 1, 2024
  21. Jeff KingNov 13, 2024
  22. Jonathan TanNov 13, 2024
  23. Jeff KingNov 14, 2024
  24. Junio C HamanoNov 14, 2024
  25. Jeff KingNov 15, 2024
  26. Jonathan TanNov 15, 2024
  27. Jeff KingNov 16, 2024
  28. index-pack: teach --promisor to require --stdinJonathan Tan, Nov 18, 2024
  29. Junio C HamanoNov 19, 2024
  30. Jeff KingNov 19, 2024
  31. Junio C HamanoNov 20, 2024
  32. index-pack: teach --promisor to forbid pack nameJonathan Tan, Nov 19, 2024
  33. Jeff KingNov 20, 2024
  34. Jeff KingNov 14, 2024
  35. 4/4 index-pack: repack local links into promisor packsJonathan Tan, Nov 1, 2024
  36. Junio C HamanoNov 4, 2024
  37. Junio C HamanoNov 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.