git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/4] repack: populate extension bits incrementally

From
Taylor Blau <me@ttaylorr.com>
Date
Oct 21, 2022, 23:20 UTC
Message-ID
<Y1MpUHBQtq8uP5Uy@nand.local>
In-Reply-To
<Y1MSkubbDBh0Ma1G@coredump.intra.peff.net>
On Fri, Oct 21, 2022 at 05:43:46PM -0400, Jeff King wrote:
Show 8 quoted lines
> There are two small problems with this:
>
>   - repack_promisor_objects() may have added entries to "names", and
>     already called populate_pack_exts() for them. This is mostly just
>     wasteful, as we'll stat() the filename with each possible extension,
>     get the same result, and just overwrite our bits. But it makes the
>     code flow confusing, and it will become a problem if we try to make
>     populate_pack_exts() do more things.

Hmm. I agree with you that repack_promisor_objects() calling populate_pack_exts() itself is at best weird, and at worst wasteful.

>
>   - it would be nice to record the generated filenames as soon as
>     possible. We don't currently use them for cleaning up from a failed
>     operation, but a future patch will do so.

That said, I think that it's worth having a single spot within cmd_repack() that is responsible for populating the generated pack extensions, since it protects against a caller forgetting to do so (and then tricking repack into thinking that we didn't generate *any* of the corresponding extensions).

But I'm sure future patch you're referring to cares about knowing these as soon as possible, since that's the point of this series ;-).

I think a reasonable middle ground here is to do something like the following on top of this patch:

--- >8 ---
diff --git a/builtin/repack.c b/builtin/repack.c
index b5bd9e5fed..16a941f48b 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1002,6 +1002,12 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
 			return ret;
 	}

+	for_each_string_list_item(item, &names) {
+		if (!item->util)
+			BUG("missing generated_pack_data for pack %s",
+			    item->string);
+	}
+
 	string_list_sort(&names);

 	close_object_store(the_repository->objects);
--- 8< ---

which still lets you eagerly keep track of the generated pack extensions
while also protecting against forgetful callers. Obviously we're relying
on a runtime check which is going to be somewhat weaker. But I think

The patch itself looks good.

Thanks,
Taylor
Previous: Jeff KingNext: Jeff King
Message 11 of 34 in “repack tempfile-cleanup signal deadlock”
  1. 0/4 repack tempfile-cleanup signal deadlockJeff King, Oct 21, 2022
  2. 1/4 repack: convert "names" util bitfield to arrayJeff King, Oct 21, 2022
  3. Junio C HamanoOct 21, 2022
  4. Taylor BlauOct 21, 2022
  5. Jeff KingOct 21, 2022
  6. Junio C HamanoOct 21, 2022
  7. Jeff KingOct 21, 2022
  8. Junio C HamanoOct 21, 2022
  9. Jeff KingOct 22, 2022
  10. 2/4 repack: populate extension bits incrementallyJeff King, Oct 21, 2022
  11. Taylor BlauOct 21, 2022
  12. Jeff KingOct 21, 2022
  13. Taylor BlauOct 21, 2022
  14. Jeff KingOct 21, 2022
  15. 3/4 repack: use tempfiles for signal cleanupJeff King, Oct 21, 2022
  16. Junio C HamanoOct 21, 2022
  17. Jeff KingOct 21, 2022
  18. Taylor BlauOct 21, 2022
  19. Jeff KingOct 22, 2022
  20. Jeff KingOct 22, 2022
  21. 4/4 repack: drop remove_temporary_files()Jeff King, Oct 21, 2022
  22. Taylor BlauOct 21, 2022
  23. 0/5 repack tempfile-cleanup signal deadlockJeff King, Oct 22, 2022
  24. 1/5 repack: convert "names" util bitfield to arrayJeff King, Oct 22, 2022
  25. 2/5 repack: populate extension bits incrementallyJeff King, Oct 22, 2022
  26. 3/5 repack: expand error message for missing pack filesJeff King, Oct 22, 2022
  27. 4/5 repack: use tempfiles for signal cleanupJeff King, Oct 22, 2022
  28. Jeff KingOct 22, 2022
  29. Junio C HamanoOct 23, 2022
  30. Jeff KingOct 23, 2022
  31. Junio C HamanoOct 23, 2022
  32. Jeff KingOct 23, 2022
  33. Junio C HamanoOct 23, 2022
  34. 5/5 repack: drop remove_temporary_files()Jeff King, Oct 22, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.