git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 3/3] rev-list: add commit object support in `--missing` option

From
Karthik Nayak <karthik.188@gmail.com>
Date
Oct 16, 2023, 10:38 UTC
Message-ID
<20231016103830.56486-4-karthik.188@gmail.com>
In-Reply-To
<20231016103830.56486-1-karthik.188@gmail.com>

The `--missing` object option in rev-list currently works only with missing blobs/trees. For missing commits the revision walker fails with a fatal error.

Let's extend the functionality of `--missing` option to also support commit objects. This is done by adding a new `MISSING` flag that the revision walker sets whenever it encounters a missing tree/commit. The revision walker will now continue the traversal and call `show_commit()` even for missing commits. In rev-list we can then check for this flag and call the existing code for parsing `--missing` objects.

A scenario where this option would be used is to find the boundary objects between different object directories. Consider a repository with a main object directory (GIT_OBJECT_DIRECTORY) and one or more alternate object directories (GIT_ALTERNATE_OBJECT_DIRECTORIES). In such a repository, using the `--missing=print` option while disabling the alternate object directory allows us to find the boundary objects between the main and alternate object directory.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 builtin/rev-list.c          |  6 +++
 list-objects.c              |  2 +-
 object.h                    |  2 +-
 revision.c                  | 11 ++++--
 revision.h                  |  3 ++
 t/t6022-rev-list-missing.sh | 74 +++++++++++++++++++++++++++++++++++++
 6 files changed, 93 insertions(+), 5 deletions(-)
 create mode 100755 t/t6022-rev-list-missing.sh
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 98542e8b3c..604ae01d0c 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -149,6 +149,12 @@ static void show_commit(struct commit *commit, void *data)
 
 	display_progress(progress, ++progress_counter);
 
+	if (revs->do_not_die_on_missing_objects &&
+	    commit->object.flags & MISSING) {
+		finish_object__ma(&commit->object);
+		return;
+	}
+
 	if (show_disk_usage)
 		total_disk_usage += get_object_disk_usage(&commit->object);
 
diff --git a/list-objects.c b/list-objects.c
index 47296dff2f..60c5533721 100644
--- a/list-objects.c
+++ b/list-objects.c
@@ -387,7 +387,7 @@ static void do_traverse(struct traversal_context *ctx)
 		 * an uninteresting boundary commit may not have its tree
 		 * parsed yet, but we are not going to show them anyway
 		 */
-		if (!ctx->revs->tree_objects)
+		if (!ctx->revs->tree_objects || commit->object.flags & MISSING)
 			; /* do not bother loading tree */
 		else if (repo_get_commit_tree(the_repository, commit)) {
 			struct tree *tree = repo_get_commit_tree(the_repository,
diff --git a/object.h b/object.h
index 114d45954d..12913e6116 100644
--- a/object.h
+++ b/object.h
@@ -62,7 +62,7 @@ void object_array_init(struct object_array *array);
 
 /*
  * object flag allocation:
- * revision.h:               0---------10         15             23------27
+ * revision.h:               0---------10         15             22------27
  * fetch-pack.c:             01    67
  * negotiator/default.c:       2--5
  * walker.c:                 0-2
diff --git a/revision.c b/revision.c
index e789834dd1..d147e0f611 100644
--- a/revision.c
+++ b/revision.c
@@ -1110,7 +1110,7 @@ static int process_parents(struct rev_info *revs, struct commit *commit,
 	struct commit_list *parent = commit->parents;
 	unsigned pass_flags;
 
-	if (commit->object.flags & ADDED)
+	if (commit->object.flags & ADDED || commit->object.flags & MISSING)
 		return 0;
 	commit->object.flags |= ADDED;
 
@@ -1168,7 +1168,8 @@ static int process_parents(struct rev_info *revs, struct commit *commit,
 	for (parent = commit->parents; parent; parent = parent->next) {
 		struct commit *p = parent->item;
 		int gently = revs->ignore_missing_links ||
-			     revs->exclude_promisor_objects;
+			     revs->exclude_promisor_objects ||
+			     revs->do_not_die_on_missing_objects;
 		if (repo_parse_commit_gently(revs->repo, p, gently) < 0) {
 			if (revs->exclude_promisor_objects &&
 			    is_promisor_object(&p->object.oid)) {
@@ -1176,7 +1177,11 @@ static int process_parents(struct rev_info *revs, struct commit *commit,
 					break;
 				continue;
 			}
-			return -1;
+
+			if (!revs->do_not_die_on_missing_objects)
+				return -1;
+			else
+				p->object.flags |= MISSING;
 		}
 		if (revs->sources) {
 			char **slot = revision_sources_at(revs->sources, p);
diff --git a/revision.h b/revision.h
index c73c92ef40..07906bc3bc 100644
--- a/revision.h
+++ b/revision.h
@@ -41,6 +41,9 @@
 /* WARNING: This is also used as REACHABLE in commit-graph.c. */
 #define PULL_MERGE	(1u<<15)
 
+/* WARNING: This is also used as NEEDS_BITMAP in pack-bitmap.c. */
+#define MISSING         (1u<<22)
+
 #define TOPO_WALK_EXPLORED	(1u<<23)
 #define TOPO_WALK_INDEGREE	(1u<<24)
 
diff --git a/t/t6022-rev-list-missing.sh b/t/t6022-rev-list-missing.sh
new file mode 100755
index 0000000000..40265a4f66
--- /dev/null
+++ b/t/t6022-rev-list-missing.sh
@@ -0,0 +1,74 @@
+#!/bin/sh
+
+test_description='handling of missing objects in rev-list'
+
+TEST_PASSES_SANITIZE_LEAK=true
+. ./test-lib.sh
+
+# We setup the repository with two commits, this way HEAD is always
+# available and we can hide commit 1.
+test_expect_success 'create repository and alternate directory' '
+	test_commit 1 &&
+	test_commit 2 &&
+	test_commit 3
+'
+
+for obj in "HEAD~1" "HEAD~1^{tree}" "HEAD:1.t"
+do
+	test_expect_success "rev-list --missing=error fails with missing object $obj" '
+		oid="$(git rev-parse $obj)" &&
+		path=".git/objects/$(test_oid_to_path $oid)" &&
+
+		mv "$path" "$path.hidden" &&
+		test_when_finished "mv $path.hidden $path" &&
+
+		test_must_fail git rev-list --missing=error --objects \
+			--no-object-names HEAD
+	'
+done
+
+for obj in "HEAD~1" "HEAD~1^{tree}" "HEAD:1.t"
+do
+	for action in "allow-any" "print"
+	do
+		test_expect_success "rev-list --missing=$action with missing $obj" '
+			oid="$(git rev-parse $obj)" &&
+			path=".git/objects/$(test_oid_to_path $oid)" &&
+
+			# Before the object is made missing, we use rev-list to
+			# get the expected oids.
+			git rev-list --objects --no-object-names \
+				HEAD ^$obj >expect.raw &&
+
+			# Blobs are shared by all commits, so evethough a commit/tree
+			# might be skipped, its blob must be accounted for.
+			if [ $obj != "HEAD:1.t" ]; then
+				echo $(git rev-parse HEAD:1.t) >>expect.raw &&
+				echo $(git rev-parse HEAD:2.t) >>expect.raw
+			fi &&
+
+			mv "$path" "$path.hidden" &&
+			test_when_finished "mv $path.hidden $path" &&
+
+			git rev-list --missing=$action --objects --no-object-names \
+				HEAD >actual.raw &&
+
+			# When the action is to print, we should also add the missing
+			# oid to the expect list.
+			case $action in
+			allow-any)
+				;;
+			print)
+				grep ?$oid actual.raw &&
+				echo ?$oid >>expect.raw
+				;;
+			esac &&
+
+			sort actual.raw >actual &&
+			sort expect.raw >expect &&
+			test_cmp expect actual
+		'
+	done
+done
+
+test_done
-- 
2.41.0
Previous: Karthik NayakNext: Junio C Hamano
Message 31 of 61 in “rev-list: add support for commits in `--missing`”
  1. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 9, 2023
  2. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 9, 2023
  3. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 9, 2023
  4. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 9, 2023
  5. Junio C HamanoOct 9, 2023
  6. Patrick SteinhardtOct 10, 2023
  7. Junio C HamanoOct 10, 2023
  8. Karthik NayakOct 11, 2023
  9. Junio C HamanoOct 11, 2023
  10. Karthik NayakOct 12, 2023
  11. Patrick SteinhardtOct 12, 2023
  12. Karthik NayakOct 12, 2023
  13. Junio C HamanoOct 12, 2023
  14. Patrick SteinhardtOct 13, 2023
  15. Patrick SteinhardtOct 13, 2023
  16. commit: detect commits that exist in commit-graph but not in the ODBPatrick Steinhardt, Oct 13, 2023
  17. Junio C HamanoOct 13, 2023
  18. Patrick SteinhardtOct 17, 2023
  19. Junio C HamanoOct 17, 2023
  20. Patrick SteinhardtOct 19, 2023
  21. Patrick SteinhardtOct 19, 2023
  22. Junio C HamanoOct 19, 2023
  23. Jeff KingOct 20, 2023
  24. Junio C HamanoOct 20, 2023
  25. Patrick SteinhardtOct 23, 2023
  26. Junio C HamanoOct 13, 2023
  27. Junio C HamanoOct 12, 2023
  28. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 16, 2023
  29. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 16, 2023
  30. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 16, 2023
  31. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 16, 2023
  32. Junio C HamanoOct 16, 2023
  33. Karthik NayakOct 16, 2023
  34. Junio C HamanoOct 16, 2023
  35. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 19, 2023
  36. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 19, 2023
  37. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 19, 2023
  38. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 19, 2023
  39. Junio C HamanoOct 19, 2023
  40. Junio C HamanoOct 19, 2023
  41. Karthik NayakOct 20, 2023
  42. Karthik NayakOct 20, 2023
  43. Junio C HamanoOct 20, 2023
  44. Junio C HamanoOct 20, 2023
  45. Karthik NayakOct 24, 2023
  46. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 24, 2023
  47. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 24, 2023
  48. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 24, 2023
  49. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 24, 2023
  50. Junio C HamanoOct 24, 2023
  51. Junio C HamanoOct 25, 2023
  52. Karthik NayakOct 25, 2023
  53. Patrick SteinhardtOct 25, 2023
  54. Junio C HamanoOct 26, 2023
  55. 0/3 rev-list: add support for commits in `--missing`Karthik Nayak, Oct 26, 2023
  56. 1/3 revision: rename bit to `do_not_die_on_missing_objects`Karthik Nayak, Oct 26, 2023
  57. 2/3 rev-list: move `show_commit()` to the bottomKarthik Nayak, Oct 26, 2023
  58. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 26, 2023
  59. Patrick SteinhardtOct 27, 2023
  60. Karthik NayakOct 27, 2023
  61. 3/3 rev-list: add commit object support in `--missing` optionKarthik Nayak, Oct 27, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.