git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 2/2] builtin/pack-objects.c: introduce `pack.recentObjectsHook`

From
Jeff King <peff@peff.net>
Date
May 12, 2023, 22:01 UTC
Message-ID
<20230512220117.GA2971903@coredump.intra.peff.net>
In-Reply-To
<20230512214542.GB2495860@coredump.intra.peff.net>
On Fri, May 12, 2023 at 05:45:43PM -0400, Jeff King wrote:
Show 23 quoted lines
> The thing that confused me when looking at the code earlier is that
> git-prune itself checks the mtime of the objects, too, even if they were
> not mentioned in the recent-but-reachable walk. That seems redundant to
> me, but somehow it isn't. If I do:
> 
> diff --git a/builtin/prune.c b/builtin/prune.c
> index 5dc9b20720..22b7ce4b10 100644
> --- a/builtin/prune.c
> +++ b/builtin/prune.c
> @@ -92,7 +92,7 @@ static int prune_object(const struct object_id *oid, const char *fullpath,
>  		return 0;
>  	}
>  	if (st.st_mtime > expire)
> -		return 0;
> +		BUG("object should have been saved via is_object_reachable!");
>  	if (show_only || verbose) {
>  		enum object_type type = oid_object_info(the_repository, oid,
>  							NULL);
> 
> then t5304 shows some failures. I'm not quite sure what is going on, but
> _if_ that mtime check in prune_object() is important, then it probably
> should also respect the hook mechanism. So there may be a (3) to add to
> your patch there.

Ah, I see. The problem is that "git prune --expire=never" relies on this check. In mark_reachable_objects(), we take a "0" expiration as "do not bother adding these objects to the traversal", so it's here that we catch them and say "ah, we are not expiring anything, so keep this".

If my hack above is switched to:
diff --git a/builtin/prune.c b/builtin/prune.c
index 5dc9b20720..108305cd26 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -91,8 +91,11 @@ static int prune_object(const struct object_id *oid, const char *fullpath,
 		error("Could not stat '%s'", fullpath);
 		return 0;
 	}
-	if (st.st_mtime > expire)
-		return 0;
+	if (st.st_mtime > expire) {
+		if (!expire)
+			return 0; /* we are not expiring anything! */
+		BUG("object should have been saved via is_object_reachable!");
+	}
 	if (show_only || verbose) {
 		enum object_type type = oid_object_info(the_repository, oid,
 							NULL);

then all of the tests pass. I do suspect this code could trigger racily
in the real world (we do our traversal, and then a new object is added,
which is of course recent). So we would not want to drop the check.

Is it important there for your patch to kick in and say "we were
specifically asked to consider this object recent, so do so"? I think it
shouldn't matter, because any such object is by definition recent,
having an mtime greater than or equal to when we started (it cannot even
get racily old while we traverse, because we decide the cutoff time as
an absolute value even before starting the traversal). The strict
greater-than in the check makes it feel like an off-by-one problem, but
unless you are saying "now" it must be at least 1 second in the past
anyway.

It does make me wonder why "git prune --expire=never" does not simply
exit immediately. Isn't there by definition nothing useful to do?

-Peff
Previous: Jeff KingNext: Junio C Hamano
Message 10 of 28 in “pack-objects: introduce `pack.extraRecentObjectsHook`”
  1. 0/2 pack-objects: introduce `pack.extraRecentObjectsHook`Taylor Blau, May 11, 2023
  2. 1/2 reachable.c: extract `obj_is_recent()`Taylor Blau, May 11, 2023
  3. 2/2 builtin/pack-objects.c: introduce `pack.recentObjectsHook`Taylor Blau, May 11, 2023
  4. Jeff KingMay 12, 2023
  5. Taylor BlauMay 15, 2023
  6. Jeff KingMay 12, 2023
  7. Taylor BlauMay 12, 2023
  8. Jeff KingMay 12, 2023
  9. Jeff KingMay 12, 2023
  10. Jeff KingMay 12, 2023
  11. Junio C HamanoMay 12, 2023
  12. Jeff KingMay 13, 2023
  13. Jeff KingMay 13, 2023
  14. Taylor BlauMay 15, 2023
  15. Taylor BlauMay 15, 2023
  16. Taylor BlauMay 11, 2023
  17. Junio C HamanoMay 11, 2023
  18. Taylor BlauMay 11, 2023
  19. 0/2 gc: introduce `gc.recentObjectsHook`Taylor Blau, May 16, 2023
  20. 1/2 reachable.c: extract `obj_is_recent()`Taylor Blau, May 16, 2023
  21. 2/2 gc: introduce `gc.recentObjectsHook`Taylor Blau, May 16, 2023
  22. Glen ChooMay 24, 2023
  23. Taylor BlauJun 7, 2023
  24. 0/2 gc: introduce `gc.recentObjectsHook`Taylor Blau, Jun 7, 2023
  25. 1/2 reachable.c: extract `obj_is_recent()`Taylor Blau, Jun 7, 2023
  26. 2/2 gc: introduce `gc.recentObjectsHook`Taylor Blau, Jun 7, 2023
  27. Glen ChooJun 9, 2023
  28. Junio C HamanoJun 12, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.