git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 09/12] sequencer: rewrite update-refs as user edits todo list

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Jul 19, 2022, 18:33 UTC
Message-ID
<95e2bbcedb1740f240a0a470822549d63075cc46.1658255624.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1247.v5.git.1658255624.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

An interactive rebase provides opportunities for the user to edit the todo list. The --update-refs option initializes the list with some 'update-ref <ref>' steps, but the user could add these manually. Further, the user could add or remove these steps during pauses in the interactive rebase.

Add a new method, todo_list_filter_update_refs(), that scans a todo_list and compares it to the stored update-refs file. There are two actions that can happen at this point:

1. If a '<ref>/<before>/<after>' triple in the update-refs file does not
   have a matching 'update-ref <ref>' command in the todo-list _and_ the
   <after> value is the null OID, then remove that triple. Here, the
   user removed the 'update-ref <ref>' command before it was executed,
   since if it was executed then the <after> value would store the
   commit at that position.
2. If a 'update-ref <ref>' command in the todo-list does not have a
   matching '<ref>/<before>/<after>' triple in the update-refs file,
   then insert a new one. Store the <before> value to be the current
   OID pointed at by <ref>. This is handled inside of the
   init_update_ref_record() helper method.

We can test that this works by rewriting the todo-list several times in the course of a rebase. Check that each ref is locked or unlocked for updates after each todo-list update. We can also verify that the ref update fails if a concurrent process updates one of the refs after the rebase process records the "locked" ref location.

To help these tests, add a new 'set_replace_editor' helper that will replace the todo-list with an exact file.

Reported-by: Phillip Wood <phillip.wood123@gmail.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 rebase-interactive.c          |   6 ++
 sequencer.c                   |  96 +++++++++++++++++++++++
 sequencer.h                   |  12 +++
 t/lib-rebase.sh               |  15 ++++
 t/t3404-rebase-interactive.sh | 139 ++++++++++++++++++++++++++++++++++
 5 files changed, 268 insertions(+)
diff --git a/rebase-interactive.c b/rebase-interactive.c
index 1ff07647af3..7407c593191 100644
--- a/rebase-interactive.c
+++ b/rebase-interactive.c
@@ -146,6 +146,12 @@ int edit_todo_list(struct repository *r, struct todo_list *todo_list,
 		return -4;
 	}
 
+	/*
+	 * See if branches need to be added or removed from the update-refs
+	 * file based on the new todo list.
+	 */
+	todo_list_filter_update_refs(r, new_todo);
+
 	return 0;
 }
 
diff --git a/sequencer.c b/sequencer.c
index ce8c7d8cd3a..67812c0294f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -4143,6 +4143,102 @@ cleanup:
 	return result;
 }
 
+/*
+ * Parse the update-refs file for the current rebase, then remove the
+ * refs that do not appear in the todo_list (and have not had updated
+ * values stored) and add refs that are in the todo_list but not
+ * represented in the update-refs file.
+ *
+ * If there are changes to the update-refs list, then write the new state
+ * to disk.
+ */
+void todo_list_filter_update_refs(struct repository *r,
+				  struct todo_list *todo_list)
+{
+	int i;
+	int updated = 0;
+	struct string_list update_refs = STRING_LIST_INIT_DUP;
+
+	sequencer_get_update_refs_state(r->gitdir, &update_refs);
+
+	/*
+	 * For each item in the update_refs list, if it has no updated
+	 * value and does not appear in the todo_list, then remove it
+	 * from the update_refs list.
+	 */
+	for (i = 0; i < update_refs.nr; i++) {
+		int j;
+		int found = 0;
+		const char *ref = update_refs.items[i].string;
+		size_t reflen = strlen(ref);
+		struct update_ref_record *rec = update_refs.items[i].util;
+
+		/* OID already stored as updated. */
+		if (!is_null_oid(&rec->after))
+			continue;
+
+		for (j = 0; !found && j < todo_list->total_nr; j++) {
+			struct todo_item *item = &todo_list->items[j];
+			const char *arg = todo_list->buf.buf + item->arg_offset;
+
+			if (item->command != TODO_UPDATE_REF)
+				continue;
+
+			if (item->arg_len != reflen ||
+			    strncmp(arg, ref, reflen))
+				continue;
+
+			found = 1;
+		}
+
+		if (!found) {
+			free(update_refs.items[i].string);
+			free(update_refs.items[i].util);
+
+			update_refs.nr--;
+			MOVE_ARRAY(update_refs.items + i, update_refs.items + i + 1, update_refs.nr - i);
+
+			updated = 1;
+			i--;
+		}
+	}
+
+	/*
+	 * For each todo_item, check if its ref is in the update_refs list.
+	 * If not, then add it as an un-updated ref.
+	 */
+	for (i = 0; i < todo_list->total_nr; i++) {
+		struct todo_item *item = &todo_list->items[i];
+		const char *arg = todo_list->buf.buf + item->arg_offset;
+		int j, found = 0;
+
+		if (item->command != TODO_UPDATE_REF)
+			continue;
+
+		for (j = 0; !found && j < update_refs.nr; j++) {
+			const char *ref = update_refs.items[j].string;
+
+			found = strlen(ref) == item->arg_len &&
+				!strncmp(ref, arg, item->arg_len);
+		}
+
+		if (!found) {
+			struct string_list_item *inserted;
+			struct strbuf argref = STRBUF_INIT;
+
+			strbuf_add(&argref, arg, item->arg_len);
+			inserted = string_list_insert(&update_refs, argref.buf);
+			inserted->util = init_update_ref_record(argref.buf);
+			strbuf_release(&argref);
+			updated = 1;
+		}
+	}
+
+	if (updated)
+		write_update_refs_state(&update_refs);
+	string_list_clear(&update_refs, 1);
+}
+
 static int do_update_ref(struct repository *r, const char *refname)
 {
 	struct string_list_item *item;
diff --git a/sequencer.h b/sequencer.h
index e671d7e0743..98f3bcc1658 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -132,6 +132,18 @@ void todo_list_release(struct todo_list *todo_list);
 const char *todo_item_get_arg(struct todo_list *todo_list,
 			      struct todo_item *item);
 
+/*
+ * Parse the update-refs file for the current rebase, then remove the
+ * refs that do not appear in the todo_list (and have not had updated
+ * values stored) and add refs that are in the todo_list but not
+ * represented in the update-refs file.
+ *
+ * If there are changes to the update-refs list, then write the new state
+ * to disk.
+ */
+void todo_list_filter_update_refs(struct repository *r,
+				  struct todo_list *todo_list);
+
 /* Call this to setup defaults before parsing command line options */
 void sequencer_init_config(struct replay_opts *opts);
 int sequencer_pick_revisions(struct repository *repo,
diff --git a/t/lib-rebase.sh b/t/lib-rebase.sh
index ec6b9b107da..b57541356bd 100644
--- a/t/lib-rebase.sh
+++ b/t/lib-rebase.sh
@@ -207,3 +207,18 @@ check_reworded_commits () {
 		>reword-log &&
 	test_cmp reword-expected reword-log
 }
+
+# usage: set_replace_editor <file>
+#
+# Replace the todo file with the exact contents of the given file.
+set_replace_editor () {
+	cat >script <<-\EOF &&
+	cat FILENAME >"$1"
+
+	echo 'rebase -i script after editing:'
+	cat "$1"
+	EOF
+
+	sed -e "s/FILENAME/$1/g" <script | write_script fake-editor.sh &&
+	test_set_editor "$(pwd)/fake-editor.sh"
+}
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 2fffcf5d5fb..bebf9b4def7 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -1834,6 +1834,145 @@ test_expect_success '--update-refs updates refs correctly' '
 	test_cmp_rev HEAD refs/heads/no-conflict-branch
 '
 
+test_expect_success 'respect user edits to update-ref steps' '
+	git checkout -B update-refs-break no-conflict-branch &&
+	git branch -f base HEAD~4 &&
+	git branch -f first HEAD~3 &&
+	git branch -f second HEAD~3 &&
+	git branch -f third HEAD~1 &&
+	git branch -f unseen base &&
+
+	# First, we will add breaks to the expected todo file
+	cat >fake-todo-1 <<-EOF &&
+	pick $(git rev-parse HEAD~3)
+	break
+	update-ref refs/heads/second
+	update-ref refs/heads/first
+
+	pick $(git rev-parse HEAD~2)
+	pick $(git rev-parse HEAD~1)
+	update-ref refs/heads/third
+
+	pick $(git rev-parse HEAD)
+	update-ref refs/heads/no-conflict-branch
+	EOF
+
+	# Second, we will drop some update-refs commands (and move one)
+	cat >fake-todo-2 <<-EOF &&
+	update-ref refs/heads/second
+
+	pick $(git rev-parse HEAD~2)
+	update-ref refs/heads/third
+	pick $(git rev-parse HEAD~1)
+	break
+
+	pick $(git rev-parse HEAD)
+	EOF
+
+	# Third, we will:
+	# * insert a new one (new-branch),
+	# * re-add an old one (first), and
+	# * add a second instance of a previously-stored one (second)
+	cat >fake-todo-3 <<-EOF &&
+	update-ref refs/heads/unseen
+	update-ref refs/heads/new-branch
+	pick $(git rev-parse HEAD)
+	update-ref refs/heads/first
+	update-ref refs/heads/second
+	EOF
+
+	(
+		set_replace_editor fake-todo-1 &&
+		git rebase -i --update-refs primary &&
+
+		# These branches are currently locked.
+		for b in first second third no-conflict-branch
+		do
+			test_must_fail git branch -f $b base || return 1
+		done &&
+
+		set_replace_editor fake-todo-2 &&
+		git rebase --edit-todo &&
+
+		# These branches are currently locked.
+		for b in second third
+		do
+			test_must_fail git branch -f $b base || return 1
+		done &&
+
+		# These branches are currently unlocked for checkout.
+		for b in first no-conflict-branch
+		do
+			git worktree add wt-$b $b &&
+			git worktree remove wt-$b || return 1
+		done &&
+
+		git rebase --continue &&
+
+		set_replace_editor fake-todo-3 &&
+		git rebase --edit-todo &&
+
+		# These branches are currently locked.
+		for b in second third first unseen
+		do
+			test_must_fail git branch -f $b base || return 1
+		done &&
+
+		# These branches are currently unlocked for checkout.
+		for b in no-conflict-branch
+		do
+			git worktree add wt-$b $b &&
+			git worktree remove wt-$b || return 1
+		done &&
+
+		git rebase --continue
+	) &&
+
+	test_cmp_rev HEAD~2 refs/heads/third &&
+	test_cmp_rev HEAD~1 refs/heads/unseen &&
+	test_cmp_rev HEAD~1 refs/heads/new-branch &&
+	test_cmp_rev HEAD refs/heads/first &&
+	test_cmp_rev HEAD refs/heads/second &&
+	test_cmp_rev HEAD refs/heads/no-conflict-branch
+'
+
+test_expect_success '--update-refs: check failed ref update' '
+	git checkout -B update-refs-error no-conflict-branch &&
+	git branch -f base HEAD~4 &&
+	git branch -f first HEAD~3 &&
+	git branch -f second HEAD~2 &&
+	git branch -f third HEAD~1 &&
+
+	cat >fake-todo <<-EOF &&
+	pick $(git rev-parse HEAD~3)
+	break
+	update-ref refs/heads/first
+
+	pick $(git rev-parse HEAD~2)
+	update-ref refs/heads/second
+
+	pick $(git rev-parse HEAD~1)
+	update-ref refs/heads/third
+
+	pick $(git rev-parse HEAD)
+	update-ref refs/heads/no-conflict-branch
+	EOF
+
+	(
+		set_replace_editor fake-todo &&
+		git rebase -i --update-refs base
+	) &&
+
+	# At this point, the values of first, second, and third are
+	# recorded in the update-refs file. We will force-update the
+	# "second" ref, but "git branch -f" will not work because of
+	# the lock in the update-refs file.
+	git rev-parse third >.git/refs/heads/second &&
+
+	git rebase --continue 2>err &&
+	grep "update_ref failed for ref '\''refs/heads/second'\''" err
+'
+
 # This must be the last test in this file
 test_expect_success '$EDITOR and friends are unchanged' '
 	test_editor_unchanged
-- 
gitgitgadget
Previous: Phillip WoodNext: Phillip Wood
Message 135 of 144 in “rebase: update branches in multi-part topic”
  1. 0/4 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jun 3, 2022
  2. 2/4 branch: add branch_checked_out() helperDerrick Stolee via GitGitGadget, Jun 3, 2022
  3. Junio C HamanoJun 3, 2022
  4. 1/4 log-tree: create for_each_decoration()Derrick Stolee via GitGitGadget, Jun 3, 2022
  5. Junio C HamanoJun 3, 2022
  6. Derrick StoleeJun 3, 2022
  7. Junio C HamanoJun 3, 2022
  8. 3/4 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jun 3, 2022
  9. Phillip WoodJun 7, 2022
  10. 4/4 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jun 3, 2022
  11. Junio C HamanoJun 3, 2022
  12. Taylor BlauJun 3, 2022
  13. Junio C HamanoJun 3, 2022
  14. Jeff HostetlerJun 3, 2022
  15. Taylor BlauJun 3, 2022
  16. Junio C HamanoJun 3, 2022
  17. Phillip WoodJun 4, 2022
  18. Derrick StoleeJun 6, 2022
  19. Phillip WoodJun 7, 2022
  20. Derrick StoleeJun 7, 2022
  21. Junio C HamanoJun 8, 2022
  22. Junio C HamanoJun 6, 2022
  23. Elijah NewrenJun 7, 2022
  24. 0/7 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jun 7, 2022
  25. 1/7 log-tree: create for_each_decoration()Derrick Stolee via GitGitGadget, Jun 7, 2022
  26. 3/7 sequencer: define array with enum valuesDerrick Stolee via GitGitGadget, Jun 7, 2022
  27. Junio C HamanoJun 7, 2022
  28. 2/7 branch: add branch_checked_out() helperDerrick Stolee via GitGitGadget, Jun 7, 2022
  29. Junio C HamanoJun 7, 2022
  30. Derrick StoleeJun 8, 2022
  31. Derrick StoleeJun 8, 2022
  32. Junio C HamanoJun 8, 2022
  33. 4/7 sequencer: add update-refs commandDerrick Stolee via GitGitGadget, Jun 7, 2022
  34. 6/7 sequencer: implement 'update-refs' commandDerrick Stolee via GitGitGadget, Jun 7, 2022
  35. Junio C HamanoJun 7, 2022
  36. 5/7 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jun 7, 2022
  37. 7/7 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jun 7, 2022
  38. Phillip WoodJun 8, 2022
  39. Derrick StoleeJun 8, 2022
  40. Phillip WoodJun 9, 2022
  41. 0/8 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jun 28, 2022
  42. 1/8 t2407: test branches currently using apply backendDerrick Stolee via GitGitGadget, Jun 28, 2022
  43. Junio C HamanoJun 28, 2022
  44. Derrick StoleeJun 29, 2022
  45. Junio C HamanoJun 30, 2022
  46. Derrick StoleeJun 30, 2022
  47. 8/8 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jun 28, 2022
  48. 6/8 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jun 28, 2022
  49. Junio C HamanoJun 28, 2022
  50. Derrick StoleeJun 29, 2022
  51. Phillip WoodJul 1, 2022
  52. Elijah NewrenJul 1, 2022
  53. Derrick StoleeJul 4, 2022
  54. Elijah NewrenJul 4, 2022
  55. Derrick StoleeJul 5, 2022
  56. Elijah NewrenJul 8, 2022
  57. 7/8 rebase: update refs from 'update-ref' commandsDerrick Stolee via GitGitGadget, Jun 28, 2022
  58. Junio C HamanoJun 28, 2022
  59. Derrick StoleeJun 29, 2022
  60. Junio C HamanoJun 30, 2022
  61. Derrick StoleeJun 29, 2022
  62. Phillip WoodJul 1, 2022
  63. Junio C HamanoJul 1, 2022
  64. Elijah NewrenJul 1, 2022
  65. Derrick StoleeJul 4, 2022
  66. 5/8 sequencer: add update-ref commandDerrick Stolee via GitGitGadget, Jun 28, 2022
  67. Phillip WoodJun 30, 2022
  68. 4/8 sequencer: define array with enum valuesDerrick Stolee via GitGitGadget, Jun 28, 2022
  69. Junio C HamanoJun 28, 2022
  70. 2/8 branch: consider refs under 'update-refs'Derrick Stolee via GitGitGadget, Jun 28, 2022
  71. Junio C HamanoJun 28, 2022
  72. Derrick StoleeJun 29, 2022
  73. Phillip WoodJun 30, 2022
  74. Junio C HamanoJun 30, 2022
  75. Junio C HamanoJun 30, 2022
  76. Phillip WoodJun 30, 2022
  77. Derrick StoleeJun 30, 2022
  78. Phillip WoodJul 1, 2022
  79. 3/8 rebase-interactive: update 'merge' descriptionDerrick Stolee via GitGitGadget, Jun 28, 2022
  80. Junio C HamanoJun 28, 2022
  81. Derrick StoleeJun 29, 2022
  82. Junio C HamanoJun 30, 2022
  83. Phillip WoodJun 30, 2022
  84. Junio C HamanoJun 28, 2022
  85. Phillip WoodJul 1, 2022
  86. 00/12 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jul 12, 2022
  87. 01/12 t2407: test bisect and rebase as black-boxesDerrick Stolee via GitGitGadget, Jul 12, 2022
  88. 02/12 t2407: test branches currently using apply backendDerrick Stolee via GitGitGadget, Jul 12, 2022
  89. 03/12 branch: consider refs under 'update-refs'Derrick Stolee via GitGitGadget, Jul 12, 2022
  90. Phillip WoodJul 15, 2022
  91. 05/12 sequencer: define array with enum valuesDerrick Stolee via GitGitGadget, Jul 12, 2022
  92. 04/12 rebase-interactive: update 'merge' descriptionDerrick Stolee via GitGitGadget, Jul 12, 2022
  93. 06/12 sequencer: add update-ref commandDerrick Stolee via GitGitGadget, Jul 12, 2022
  94. 07/12 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jul 12, 2022
  95. Elijah NewrenJul 16, 2022
  96. Derrick StoleeJul 19, 2022
  97. SZEDER GáborJul 18, 2022
  98. Derrick StoleeJul 18, 2022
  99. Junio C HamanoJul 18, 2022
  100. Derrick StoleeJul 19, 2022
  101. Junio C HamanoJul 19, 2022
  102. Derrick StoleeJul 19, 2022
  103. 08/12 rebase: update refs from 'update-ref' commandsDerrick Stolee via GitGitGadget, Jul 12, 2022
  104. Phillip WoodJul 15, 2022
  105. Derrick StoleeJul 19, 2022
  106. 09/12 sequencer: rewrite update-refs as user edits todo listDerrick Stolee via GitGitGadget, Jul 12, 2022
  107. Phillip WoodJul 15, 2022
  108. Derrick StoleeJul 15, 2022
  109. Phillip WoodJul 18, 2022
  110. Elijah NewrenJul 16, 2022
  111. 11/12 sequencer: ignore HEAD ref under --update-refsDerrick Stolee via GitGitGadget, Jul 12, 2022
  112. 10/12 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jul 12, 2022
  113. 12/12 sequencer: notify user of --update-refs activityDerrick Stolee via GitGitGadget, Jul 12, 2022
  114. Phillip WoodJul 15, 2022
  115. Derrick StoleeJul 15, 2022
  116. Elijah NewrenJul 16, 2022
  117. Elijah NewrenJul 16, 2022
  118. Derrick StoleeJul 19, 2022
  119. Junio C HamanoJul 12, 2022
  120. Derrick StoleeJul 14, 2022
  121. Junio C HamanoJul 14, 2022
  122. Elijah NewrenJul 16, 2022
  123. Elijah NewrenJul 16, 2022
  124. Phillip WoodJul 15, 2022
  125. 00/12 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jul 19, 2022
  126. 01/12 t2407: test bisect and rebase as black-boxesDerrick Stolee via GitGitGadget, Jul 19, 2022
  127. 02/12 t2407: test branches currently using apply backendDerrick Stolee via GitGitGadget, Jul 19, 2022
  128. 03/12 branch: consider refs under 'update-refs'Derrick Stolee via GitGitGadget, Jul 19, 2022
  129. 04/12 rebase-interactive: update 'merge' descriptionDerrick Stolee via GitGitGadget, Jul 19, 2022
  130. 05/12 sequencer: define array with enum valuesDerrick Stolee via GitGitGadget, Jul 19, 2022
  131. 06/12 sequencer: add update-ref commandDerrick Stolee via GitGitGadget, Jul 19, 2022
  132. 07/12 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jul 19, 2022
  133. 08/12 rebase: update refs from 'update-ref' commandsDerrick Stolee via GitGitGadget, Jul 19, 2022
  134. Phillip WoodJul 21, 2022
  135. 09/12 sequencer: rewrite update-refs as user edits todo listDerrick Stolee via GitGitGadget, Jul 19, 2022
  136. Phillip WoodJul 21, 2022
  137. 10/12 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jul 19, 2022
  138. 12/12 sequencer: notify user of --update-refs activityDerrick Stolee via GitGitGadget, Jul 19, 2022
  139. 11/12 sequencer: ignore HEAD ref under --update-refsDerrick Stolee via GitGitGadget, Jul 19, 2022
  140. Elijah NewrenJul 21, 2022
  141. Derrick StoleeJul 21, 2022
  142. Elijah NewrenJul 21, 2022
  143. Derrick StoleeJul 21, 2022
  144. Phillip WoodJul 21, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.