git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[WIP v2 1/4] packfile: split promisor objects oidset into two

From
Han Young <hanyang.tony@bytedance.com>
Date
Aug 23, 2024, 12:43 UTC
Message-ID
<20240823124354.12982-2-hanyang.tony@bytedance.com>
In-Reply-To
<20240823124354.12982-1-hanyang.tony@bytedance.com>

split promisor objects oidset into two, one is objects in promisor packfile, and other set is objects referenced in promisor packfile. This enable us to check if an object is in promisor packfile.

---
 packfile.c | 25 ++++++++++++++++---------
 packfile.h |  7 ++++++-
 2 files changed, 22 insertions(+), 10 deletions(-)
diff --git a/packfile.c b/packfile.c
index cf12a539ea..1cf69a17be 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2234,12 +2234,17 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	return r ? r : pack_errors;
 }
 
+struct promisor_objects {
+	struct oidset promisor_pack_objects;
+	struct oidset promisor_pack_referenced_objects;
+};
+
 static int add_promisor_object(const struct object_id *oid,
 			       struct packed_git *pack UNUSED,
 			       uint32_t pos UNUSED,
 			       void *set_)
 {
-	struct oidset *set = set_;
+	struct promisor_objects *set = set_;
 	struct object *obj;
 	int we_parsed_object;
 
@@ -2254,7 +2259,7 @@ static int add_promisor_object(const struct object_id *oid,
 	if (!obj)
 		return 1;
 
-	oidset_insert(set, oid);
+	oidset_insert(&set->promisor_pack_objects, oid);
 
 	/*
 	 * If this is a tree, commit, or tag, the objects it refers
@@ -2272,26 +2277,26 @@ static int add_promisor_object(const struct object_id *oid,
 			 */
 			return 0;
 		while (tree_entry_gently(&desc, &entry))
-			oidset_insert(set, &entry.oid);
+			oidset_insert(&set->promisor_pack_referenced_objects, &entry.oid);
 		if (we_parsed_object)
 			free_tree_buffer(tree);
 	} else if (obj->type == OBJ_COMMIT) {
 		struct commit *commit = (struct commit *) obj;
 		struct commit_list *parents = commit->parents;
 
-		oidset_insert(set, get_commit_tree_oid(commit));
+		oidset_insert(&set->promisor_pack_referenced_objects, get_commit_tree_oid(commit));
 		for (; parents; parents = parents->next)
-			oidset_insert(set, &parents->item->object.oid);
+			oidset_insert(&set->promisor_pack_referenced_objects, &parents->item->object.oid);
 	} else if (obj->type == OBJ_TAG) {
 		struct tag *tag = (struct tag *) obj;
-		oidset_insert(set, get_tagged_oid(tag));
+		oidset_insert(&set->promisor_pack_referenced_objects, get_tagged_oid(tag));
 	}
 	return 0;
 }
 
-int is_promisor_object(const struct object_id *oid)
+int is_in_promisor_pack(const struct object_id *oid, int referenced)
 {
-	static struct oidset promisor_objects;
+	static struct promisor_objects promisor_objects;
 	static int promisor_objects_prepared;
 
 	if (!promisor_objects_prepared) {
@@ -2303,5 +2308,7 @@ int is_promisor_object(const struct object_id *oid)
 		}
 		promisor_objects_prepared = 1;
 	}
-	return oidset_contains(&promisor_objects, oid);
+	
+	return oidset_contains(&promisor_objects.promisor_pack_objects, oid) ||
+		(referenced && oidset_contains(&promisor_objects.promisor_pack_referenced_objects, oid));
 }
diff --git a/packfile.h b/packfile.h
index 0f78658229..13a349e223 100644
--- a/packfile.h
+++ b/packfile.h
@@ -195,11 +195,16 @@ int has_object_kept_pack(const struct object_id *oid, unsigned flags);
 
 int has_pack_index(const unsigned char *sha1);
 
+int is_in_promisor_pack(const struct object_id *oid, int referenced);
+
 /*
  * Return 1 if an object in a promisor packfile is or refers to the given
  * object, 0 otherwise.
  */
-int is_promisor_object(const struct object_id *oid);
+static inline int is_promisor_object(const struct object_id *oid)
+{
+	return is_in_promisor_pack(oid, 1);
+}
 
 /*
  * Expose a function for fuzz testing.
-- 
2.45.2
Previous: Han YoungNext: Han Young
Message 12 of 65 in “revision: fix reachable objects being gc'ed in no blob clone repo”
  1. 0/1 revision: fix reachable objects being gc'ed in no blob clone repoHan Young, Aug 2, 2024
  2. 1/1 revision: don't set parents as uninteresting if exclude promisor objectsHan Young, Aug 2, 2024
  3. Junio C HamanoAug 2, 2024
  4. 韩仰Aug 12, 2024
  5. Junio C HamanoAug 12, 2024
  6. 韩仰Aug 22, 2024
  7. Jonathan TanAug 13, 2024
  8. Jonathan TanAug 13, 2024
  9. Junio C HamanoAug 14, 2024
  10. Jonathan TanAug 14, 2024
  11. 0/4 revision: fix reachable objects being gc'ed in no blob clone repoHan Young, Aug 23, 2024
  12. 1/4 packfile: split promisor objects oidset into twoHan Young, Aug 23, 2024
  13. 2/4 revision: add exclude-promisor-pack-objects optionHan Young, Aug 23, 2024
  14. 3/4 revision: don't mark commit as UNINTERESTING if --exclude-promisor-objects is setHan Young, Aug 23, 2024
  15. 4/4 repack: use new exclude promisor pack objects optionHan Young, Aug 23, 2024
  16. 0/2 revision: fix reachable commits being gc'ed in partial repoCalvin Wan, Sep 19, 2024
  17. 1/2 packfile: split promisor objects oidset into twoCalvin Wan, Sep 19, 2024
  18. Junio C HamanoSep 22, 2024
  19. 2/2 fetch-pack.c: do not declare local commits as "have" in partial reposCalvin Wan, Sep 19, 2024
  20. Junio C HamanoSep 22, 2024
  21. Junio C HamanoSep 22, 2024
  22. 韩仰Sep 23, 2024
  23. Junio C HamanoSep 23, 2024
  24. Calvin WanOct 2, 2024
  25. 0/2 repack: pack everything into promisor packfile in partial reposHan Young, Sep 25, 2024
  26. 1/2 repack: pack everything into packfileHan Young, Sep 25, 2024
  27. 2/2 t0410: adapt tests to repack changesHan Young, Sep 25, 2024
  28. Phillip WoodSep 25, 2024
  29. Junio C HamanoSep 25, 2024
  30. Junio C HamanoSep 25, 2024
  31. Missing Promisor Objects in Partial Repo Design DocCalvin Wan, Oct 1, 2024
  32. Junio C HamanoOct 1, 2024
  33. Junio C HamanoOct 2, 2024
  34. Han YoungOct 2, 2024
  35. Calvin WanOct 8, 2024
  36. Han YoungOct 9, 2024
  37. Jonathan TanOct 9, 2024
  38. Jonathan TanOct 12, 2024
  39. Han YoungOct 12, 2024
  40. Jonathan TanOct 14, 2024
  41. Jonathan TanOct 9, 2024
  42. 0/3 repack: pack everything into promisor packfile in partial reposHan Young, Oct 8, 2024
  43. 1/3 repack: pack everything into packfileHan Young, Oct 8, 2024
  44. Calvin WanOct 8, 2024
  45. 2/3 t0410: adapt tests to repack changesHan Young, Oct 8, 2024
  46. 3/3 partial-clone: update docHan Young, Oct 8, 2024
  47. Junio C HamanoOct 8, 2024
  48. Junio C HamanoOct 8, 2024
  49. Han YoungOct 9, 2024
  50. 0/3 repack: pack everything into promisor packfile in partial reposHan Young, Oct 11, 2024
  51. 1/3 repack: pack everything into packfileHan Young, Oct 11, 2024
  52. 2/3 repack: adapt tests to repack changesHan Young, Oct 11, 2024
  53. 3/3 partial-clone: update docHan Young, Oct 11, 2024
  54. Junio C HamanoOct 11, 2024
  55. Junio C HamanoOct 11, 2024
  56. 0/3 repack: pack everything into promisor packfile in partial reposHan Young, Oct 14, 2024
  57. 1/3 repack: pack everything into packfileHan Young, Oct 14, 2024
  58. 2/3 t0410: adapt tests to repack changesHan Young, Oct 14, 2024
  59. 3/3 partial-clone: update docHan Young, Oct 14, 2024
  60. 0/3 Repack on fetchJonathan Tan, Oct 21, 2024
  61. 1/3 move variableJonathan Tan, Oct 21, 2024
  62. 2/3 pack-objectsJonathan Tan, Oct 21, 2024
  63. 3/3 record local links and call pack-objectsJonathan Tan, Oct 21, 2024
  64. Han YoungOct 23, 2024
  65. Jonathan TanOct 23, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.