git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 11/13] object-file: refactor freshening of objects

From
Patrick Steinhardt <ps@pks.im>
Date
Oct 24, 2025, 09:56 UTC
Message-ID
<20251024-b4-pks-odb-loose-backend-v1-11-1a4202273c38@pks.im>
In-Reply-To
<20251024-b4-pks-odb-loose-backend-v1-0-1a4202273c38@pks.im>

When writing an object that already exists in our object database we skip the write and instead only update mtimes of the object, either in its packed or loose object format. This logic is wholly contained in "object-file.c", but that file is really only concerned with loose objects. So it does not really make sense that it also contains the logic to freshen a packed object.

Introduce a new `odb_freshen_object()` function that sits on the object database level and two functions `packfile_store_freshen_object()` and `odb_loose_source_freshen_object()`. Like this, the format-specific functions can be part of their respective subsystems, while the backend agnostic function to freshen an object sits at the object database layer.

Note that this change also moves the logic that iterates through object sources from the object source layer into the object database layer. This change is intentional: object sources should ideally only have to worry about themselves, and coordination of different sources should be handled on the object database level.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 object-file.c | 33 +++++----------------------------
 object-file.h |  3 +++
 odb.c         | 16 ++++++++++++++++
 odb.h         |  3 +++
 packfile.c    | 16 ++++++++++++++++
 packfile.h    |  3 +++
 6 files changed, 46 insertions(+), 28 deletions(-)
diff --git a/object-file.c b/object-file.c
index 979aee32de0..2ca90adc2c7 100644
--- a/object-file.c
+++ b/object-file.c
@@ -968,30 +968,10 @@ static int write_loose_object(struct odb_source *source,
 					  FOF_SKIP_COLLISION_CHECK);
 }
 
-static int freshen_loose_object(struct object_database *odb,
-				const struct object_id *oid)
+int odb_loose_source_freshen_object(struct odb_source *source,
+				    const struct object_id *oid)
 {
-	odb_prepare_alternates(odb);
-	for (struct odb_source *source = odb->sources; source; source = source->next)
-		if (check_and_freshen_source(source, oid, 1))
-			return 1;
-	return 0;
-}
-
-static int freshen_packed_object(struct object_database *odb,
-				 const struct object_id *oid)
-{
-	struct pack_entry e;
-	if (!find_pack_entry(odb->repo, oid, &e))
-		return 0;
-	if (e.p->is_cruft)
-		return 0;
-	if (e.p->freshened)
-		return 1;
-	if (!freshen_file(e.p->pack_name))
-		return 0;
-	e.p->freshened = 1;
-	return 1;
+	return !!check_and_freshen_source(source, oid, 1);
 }
 
 int stream_loose_object(struct odb_source *source,
@@ -1073,12 +1053,10 @@ int stream_loose_object(struct odb_source *source,
 		die(_("deflateEnd on stream object failed (%d)"), ret);
 	close_loose_object(source, fd, tmp_file.buf);
 
-	if (freshen_packed_object(source->odb, oid) ||
-	    freshen_loose_object(source->odb, oid)) {
+	if (odb_freshen_object(source->odb, oid)) {
 		unlink_or_warn(tmp_file.buf);
 		goto cleanup;
 	}
-
 	odb_loose_path(source, &filename, oid);
 
 	/* We finally know the object path, and create the missing dir. */
@@ -1137,8 +1115,7 @@ int write_object_file(struct odb_source *source,
 	 * it out into .git/objects/??/?{38} file.
 	 */
 	write_object_file_prepare(algo, buf, len, type, oid, hdr, &hdrlen);
-	if (freshen_packed_object(source->odb, oid) ||
-	    freshen_loose_object(source->odb, oid))
+	if (odb_freshen_object(source->odb, oid))
 		return 0;
 	if (write_loose_object(source, oid, hdr, hdrlen, buf, len, 0, flags))
 		return -1;
diff --git a/object-file.h b/object-file.h
index 8e0f38d413f..b27c08380d8 100644
--- a/object-file.h
+++ b/object-file.h
@@ -59,6 +59,9 @@ void *odb_loose_source_map_object(struct odb_source *source,
 int odb_loose_source_has_object(struct odb_source *source,
 				const struct object_id *oid);
 
+int odb_loose_source_freshen_object(struct odb_source *source,
+				    const struct object_id *oid);
+
 /*
  * Populate and return the loose object cache array corresponding to the
  * given object ID.
diff --git a/odb.c b/odb.c
index 5dc1e2c7eb0..6f8f665351b 100644
--- a/odb.c
+++ b/odb.c
@@ -987,6 +987,22 @@ int odb_has_object(struct object_database *odb, const struct object_id *oid,
 	return odb_read_object_info_extended(odb, oid, NULL, object_info_flags) >= 0;
 }
 
+int odb_freshen_object(struct object_database *odb,
+		       const struct object_id *oid)
+{
+	struct odb_source *source;
+
+	if (packfile_store_freshen_object(odb->packfiles, oid))
+		return 1;
+
+	odb_prepare_alternates(odb);
+	for (source = odb->sources; source; source = source->next)
+		if (odb_loose_source_freshen_object(source, oid))
+			return 1;
+
+	return 0;
+}
+
 void odb_assert_oid_type(struct object_database *odb,
 			 const struct object_id *oid, enum object_type expect)
 {
diff --git a/odb.h b/odb.h
index 25fbcd7d951..8681b7782b4 100644
--- a/odb.h
+++ b/odb.h
@@ -396,6 +396,9 @@ int odb_has_object(struct object_database *odb,
 		   const struct object_id *oid,
 		   unsigned flags);
 
+int odb_freshen_object(struct object_database *odb,
+		       const struct object_id *oid);
+
 void odb_assert_oid_type(struct object_database *odb,
 			 const struct object_id *oid, enum object_type expect);
 
diff --git a/packfile.c b/packfile.c
index 5a7caec2925..2ab49a0beb1 100644
--- a/packfile.c
+++ b/packfile.c
@@ -819,6 +819,22 @@ struct packed_git *packfile_store_load_pack(struct packfile_store *store,
 	return p;
 }
 
+int packfile_store_freshen_object(struct packfile_store *store,
+				  const struct object_id *oid)
+{
+	struct pack_entry e;
+	if (!find_pack_entry(store->odb->repo, oid, &e))
+		return 0;
+	if (e.p->is_cruft)
+		return 0;
+	if (e.p->freshened)
+		return 1;
+	if (utime(e.p->pack_name, NULL))
+		return 0;
+	e.p->freshened = 1;
+	return 1;
+}
+
 void (*report_garbage)(unsigned seen_bits, const char *path);
 
 static void report_helper(const struct string_list *list,
diff --git a/packfile.h b/packfile.h
index e7a5792b6cf..0ad080046f1 100644
--- a/packfile.h
+++ b/packfile.h
@@ -161,6 +161,9 @@ struct list_head *packfile_store_get_packs_mru(struct packfile_store *store);
 struct packed_git *packfile_store_load_pack(struct packfile_store *store,
 					    const char *idx_path, int local);
 
+int packfile_store_freshen_object(struct packfile_store *store,
+				  const struct object_id *oid);
+
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
-- 
2.51.1.930.gacf6e81ea2.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 26 of 58 in “Carve out loose object source”
  1. 00/13 Carve out loose object sourcePatrick Steinhardt, Oct 24, 2025
  2. 01/13 odb: fix subtle logic to check whether an alternate is usablePatrick Steinhardt, Oct 24, 2025
  3. Junio C HamanoOct 24, 2025
  4. Karthik NayakOct 30, 2025
  5. 02/13 odb: introduce `odb_source_new()`Patrick Steinhardt, Oct 24, 2025
  6. Junio C HamanoOct 24, 2025
  7. Patrick SteinhardtOct 27, 2025
  8. 03/13 odb: adjust naming to free object sourcesPatrick Steinhardt, Oct 24, 2025
  9. Karthik NayakOct 30, 2025
  10. 04/13 object-file: move `fetch_if_missing`Patrick Steinhardt, Oct 24, 2025
  11. 05/13 object-file: introduce `struct odb_loose_source`Patrick Steinhardt, Oct 24, 2025
  12. Karthik NayakOct 30, 2025
  13. Patrick SteinhardtOct 30, 2025
  14. Patrick SteinhardtOct 31, 2025
  15. Junio C HamanoOct 31, 2025
  16. Patrick SteinhardtNov 3, 2025
  17. Junio C HamanoOct 31, 2025
  18. 06/13 object-file: move loose object cache into loose sourcePatrick Steinhardt, Oct 24, 2025
  19. Junio C HamanoOct 24, 2025
  20. Patrick SteinhardtOct 27, 2025
  21. 07/13 object-file: hide internals when we need to reprepare loose sourcesPatrick Steinhardt, Oct 24, 2025
  22. 08/13 object-file: move loose object map into loose sourcePatrick Steinhardt, Oct 24, 2025
  23. 09/13 object-file: read objects via the loose object sourcePatrick Steinhardt, Oct 24, 2025
  24. Karthik NayakOct 30, 2025
  25. 10/13 object-file: rename `has_loose_object()`Patrick Steinhardt, Oct 24, 2025
  26. 11/13 object-file: refactor freshening of objectsPatrick Steinhardt, Oct 24, 2025
  27. 12/13 object-file: rename `write_object_file()`Patrick Steinhardt, Oct 24, 2025
  28. 13/13 object-file: refactor writing objects via a streamPatrick Steinhardt, Oct 24, 2025
  29. Karthik NayakOct 30, 2025
  30. 00/13 Carve out loose object sourcePatrick Steinhardt, Oct 31, 2025
  31. 01/13 odb: fix subtle logic to check whether an alternate is usablePatrick Steinhardt, Oct 31, 2025
  32. 02/13 odb: introduce `odb_source_new()`Patrick Steinhardt, Oct 31, 2025
  33. 03/13 odb: adjust naming to free object sourcesPatrick Steinhardt, Oct 31, 2025
  34. 04/13 object-file: move `fetch_if_missing`Patrick Steinhardt, Oct 31, 2025
  35. 05/13 object-file: introduce `struct odb_loose_source`Patrick Steinhardt, Oct 31, 2025
  36. 06/13 object-file: move loose object cache into loose sourcePatrick Steinhardt, Oct 31, 2025
  37. 07/13 object-file: hide internals when we need to reprepare loose sourcesPatrick Steinhardt, Oct 31, 2025
  38. 08/13 object-file: move loose object map into loose sourcePatrick Steinhardt, Oct 31, 2025
  39. 09/13 object-file: read objects via the loose object sourcePatrick Steinhardt, Oct 31, 2025
  40. 10/13 object-file: rename `has_loose_object()`Patrick Steinhardt, Oct 31, 2025
  41. 11/13 object-file: refactor freshening of objectsPatrick Steinhardt, Oct 31, 2025
  42. 12/13 object-file: rename `write_object_file()`Patrick Steinhardt, Oct 31, 2025
  43. 13/13 object-file: refactor writing objects via a streamPatrick Steinhardt, Oct 31, 2025
  44. 00/13 Carve out loose object sourcePatrick Steinhardt, Nov 3, 2025
  45. 01/13 odb: fix subtle logic to check whether an alternate is usablePatrick Steinhardt, Nov 3, 2025
  46. 02/13 odb: introduce `odb_source_new()`Patrick Steinhardt, Nov 3, 2025
  47. 03/13 odb: adjust naming to free object sourcesPatrick Steinhardt, Nov 3, 2025
  48. 04/13 object-file: move `fetch_if_missing`Patrick Steinhardt, Nov 3, 2025
  49. 05/13 object-file: introduce `struct odb_source_loose`Patrick Steinhardt, Nov 3, 2025
  50. 06/13 object-file: move loose object cache into loose sourcePatrick Steinhardt, Nov 3, 2025
  51. 07/13 object-file: hide internals when we need to reprepare loose sourcesPatrick Steinhardt, Nov 3, 2025
  52. 08/13 object-file: move loose object map into loose sourcePatrick Steinhardt, Nov 3, 2025
  53. 09/13 object-file: read objects via the loose object sourcePatrick Steinhardt, Nov 3, 2025
  54. 10/13 object-file: rename `has_loose_object()`Patrick Steinhardt, Nov 3, 2025
  55. 11/13 object-file: refactor freshening of objectsPatrick Steinhardt, Nov 3, 2025
  56. 12/13 object-file: rename `write_object_file()`Patrick Steinhardt, Nov 3, 2025
  57. 13/13 object-file: refactor writing objects via a streamPatrick Steinhardt, Nov 3, 2025
  58. Karthik NayakNov 3, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.