git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 6/8] rebase: add --update-refs option

From
PWPhillip Wood <phillip.wood123@gmail.com>
Date
Jul 1, 2022, 09:20 UTC
Message-ID
<f410f5fd-aee9-0ec1-d5a7-639a4eb8fdb0@gmail.com>
In-Reply-To
<918b398d6a2264a99acadd319f780c796bbafc18.1656422759.git.gitgitgadget@gmail.com>
On 28/06/2022 14:25, Derrick Stolee via GitGitGadget wrote:
Show 38 quoted lines
> From: Derrick Stolee <derrickstolee@github.com>
> 
> When working on a large feature, it can be helpful to break that feature
> into multiple smaller parts that become reviewed in sequence. During
> development or during review, a change to one part of the feature could
> affect multiple of these parts. An interactive rebase can help adjust
> the multi-part "story" of the branch.
> 
> However, if there are branches tracking the different parts of the
> feature, then rebasing the entire list of commits can create commits not
> reachable from those "sub branches". It can take a manual step to update
> those branches.
> 
> Add a new --update-refs option to 'git rebase -i' that adds 'update-ref
> <ref>' steps to the todo file whenever a commit that is being rebased is
> decorated with that <ref>. At the very end, the rebase process updates
> all of the listed refs to the values stored during the rebase operation.
> 
> Be sure to iterate after any squashing or fixups are placed. Update the
> branch only after those squashes and fixups are complete. This allows a
> --fixup commit at the tip of the feature to apply correctly to the sub
> branch, even if it is fixing up the most-recent commit in that part.
> 
> One potential problem here is that refs decorating commits that are
> already marked as "fixup!" or "squash!" will not be included in this
> list. Generally, the reordering of the "fixup!" and "squash!" is likely
> to change the relative order of these refs, so it is not recommended.
> The workflow here is intended to allow these kinds of commits at the tip
> of the rebased branch while the other sub branches come along for the
> ride without intervention.
> 
> This change update the documentation and builtin to accept the
> --update-refs option as well as updating the todo file with the
> 'update-ref' commands. Tests are added to ensure that these todo
> commands are added in the correct locations.
> 
> A future change will update the behavior to actually update the refs
> at the end of the rebase sequence.

This looks quite good to me, my main concern is that we don't check what the user did when they edited the todo list which leaves a potentially long window when another process could update a branch that is being rebased. I've left some comments below, they're mostly me talking to myself while reading the code.

Show 38 quoted lines
> +struct todo_add_branch_context {
> +	struct todo_item *items;
> +	size_t items_nr;
> +	size_t items_alloc;
> +	struct strbuf *buf;
> +	struct commit *commit;
> +	struct string_list refs_to_oids;
> +};
> +
> +static int add_decorations_to_list(const struct commit *commit,
> +				   struct todo_add_branch_context *ctx)
> +{
> +	const struct name_decoration *decoration = get_name_decoration(&commit->object);
> +
> +	while (decoration) {
> +		struct todo_item *item;
> +		const char *path;
> +		size_t base_offset = ctx->buf->len;
> +
> +		ALLOC_GROW(ctx->items,
> +			ctx->items_nr + 1,
> +			ctx->items_alloc);
> +		item = &ctx->items[ctx->items_nr];
> +		memset(item, 0, sizeof(*item));
> +
> +		/* If the branch is checked out, then leave a comment instead. */
> +		if ((path = branch_checked_out(decoration->name))) {
> +			item->command = TODO_COMMENT;
> +			strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n",
> +				    decoration->name, path);
> +		} else {
> +			struct string_list_item *sti;
> +			item->command = TODO_UPDATE_REF;
> +			strbuf_addf(ctx->buf, "%s\n", decoration->name);
> +
> +			sti = string_list_append(&ctx->refs_to_oids,
> +						 decoration->name);
> +			sti->util = oiddup(the_hash_algo->null_oid);

We don't record where the branch is pointing, but we do stop checkout from checking out the branch. There is a small race window where another process can checkout the branch (or start rebasing a branch with 'git rebase <upstream> <branch>') after we've added it to this list but have not written it to disk.

Show 34 quoted lines
> +		}
> +
> +		item->offset_in_buf = base_offset;
> +		item->arg_offset = base_offset;
> +		item->arg_len = ctx->buf->len - base_offset;
> +		ctx->items_nr++;
> +
> +		decoration = decoration->next;
> +	}
> +
> +	return 0;
> +}
> +
> +/*
> + * For each 'pick' command, find out if the commit has a decoration in
> + * refs/heads/. If so, then add a 'label for-update-refs/' command.
> + */
> +static int todo_list_add_update_ref_commands(struct todo_list *todo_list)
> +{
> +	int i;
> +	static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP;
> +	static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP;
> +	static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP;
> +	struct decoration_filter decoration_filter = {
> +		.include_ref_pattern = &decorate_refs_include,
> +		.exclude_ref_pattern = &decorate_refs_exclude,
> +		.exclude_ref_config_pattern = &decorate_refs_exclude_config,
> +	};
> +	struct todo_add_branch_context ctx = {
> +		.buf = &todo_list->buf,
> +		.refs_to_oids = STRING_LIST_INIT_DUP,
> +	};
> +
> +	ctx.items_alloc = 2 * todo_list->nr + 1;

We pre-allocate enough space for each commit to have an update-refs command added after it ...

Show 12 quoted lines
> +	ALLOC_ARRAY(ctx.items, ctx.items_alloc);
> +
> +	string_list_append(&decorate_refs_include, "refs/heads/");
> +	load_ref_decorations(&decoration_filter, 0);
> +
> +	for (i = 0; i < todo_list->nr; ) {
> +		struct todo_item *item = &todo_list->items[i];
> +
> +		/* insert ith item into new list */
> +		ALLOC_GROW(ctx.items,
> +			   ctx.items_nr + 1,
> +			   ctx.items_alloc);
But we are careful to grow the array if that space is not enough.
Show 32 quoted lines
> +		ctx.items[ctx.items_nr++] = todo_list->items[i++];
> +
> +		if (item->commit) {
> +			ctx.commit = item->commit;
> +			add_decorations_to_list(item->commit, &ctx);
> +		}
> +	}
> +
> +	string_list_clear(&ctx.refs_to_oids, 1);
> +	free(todo_list->items);
> +	todo_list->items = ctx.items;
> +	todo_list->nr = ctx.items_nr;
> +	todo_list->alloc = ctx.items_alloc;
> +
> +	return 0;
> +}
> +
>   int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags,
>   		    const char *shortrevisions, const char *onto_name,
>   		    struct commit *onto, const struct object_id *orig_head,
>   		    struct string_list *commands, unsigned autosquash,
> +		    unsigned update_refs,
>   		    struct todo_list *todo_list)
>   {
>   	char shortonto[GIT_MAX_HEXSZ + 1];
> @@ -5637,6 +5741,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla
>   		item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0;
>   	}
>   
> +	if (update_refs && todo_list_add_update_ref_commands(todo_list))
> +		return -1;
> +

We add the update-ref commands before the user edits the todo list but do not check if they have added or removed any update-ref commands when they edit it. This means we may lock a branch unnecessarily if they deleted an update-ref command or fail to lock a branch in a timely manner if they add an update-ref command. It will eventually be locked when we process the update-ref command during the rebase but that could be quite a bit later if there are conflicts.

If we were to add the update-ref commands before the user edited the list and then write the update-refs file after it has been edited that would avoid some of these problems but leaves a bigger race window when two process think they can safely update a branch. That could be addressed by checking the other worktrees again once we have written the file which we really need to do in order to check the user hasn't added an update-ref command for a branch that is already being rebased in another worktree.

Best Wishes
Phillip
Previous: Derrick StoleeNext: Elijah Newren
Message 51 of 144 in “rebase: update branches in multi-part topic”
  1. 0/4 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jun 3, 2022
  2. 2/4 branch: add branch_checked_out() helperDerrick Stolee via GitGitGadget, Jun 3, 2022
  3. Junio C HamanoJun 3, 2022
  4. 1/4 log-tree: create for_each_decoration()Derrick Stolee via GitGitGadget, Jun 3, 2022
  5. Junio C HamanoJun 3, 2022
  6. Derrick StoleeJun 3, 2022
  7. Junio C HamanoJun 3, 2022
  8. 3/4 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jun 3, 2022
  9. Phillip WoodJun 7, 2022
  10. 4/4 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jun 3, 2022
  11. Junio C HamanoJun 3, 2022
  12. Taylor BlauJun 3, 2022
  13. Junio C HamanoJun 3, 2022
  14. Jeff HostetlerJun 3, 2022
  15. Taylor BlauJun 3, 2022
  16. Junio C HamanoJun 3, 2022
  17. Phillip WoodJun 4, 2022
  18. Derrick StoleeJun 6, 2022
  19. Phillip WoodJun 7, 2022
  20. Derrick StoleeJun 7, 2022
  21. Junio C HamanoJun 8, 2022
  22. Junio C HamanoJun 6, 2022
  23. Elijah NewrenJun 7, 2022
  24. 0/7 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jun 7, 2022
  25. 1/7 log-tree: create for_each_decoration()Derrick Stolee via GitGitGadget, Jun 7, 2022
  26. 3/7 sequencer: define array with enum valuesDerrick Stolee via GitGitGadget, Jun 7, 2022
  27. Junio C HamanoJun 7, 2022
  28. 2/7 branch: add branch_checked_out() helperDerrick Stolee via GitGitGadget, Jun 7, 2022
  29. Junio C HamanoJun 7, 2022
  30. Derrick StoleeJun 8, 2022
  31. Derrick StoleeJun 8, 2022
  32. Junio C HamanoJun 8, 2022
  33. 4/7 sequencer: add update-refs commandDerrick Stolee via GitGitGadget, Jun 7, 2022
  34. 6/7 sequencer: implement 'update-refs' commandDerrick Stolee via GitGitGadget, Jun 7, 2022
  35. Junio C HamanoJun 7, 2022
  36. 5/7 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jun 7, 2022
  37. 7/7 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jun 7, 2022
  38. Phillip WoodJun 8, 2022
  39. Derrick StoleeJun 8, 2022
  40. Phillip WoodJun 9, 2022
  41. 0/8 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jun 28, 2022
  42. 1/8 t2407: test branches currently using apply backendDerrick Stolee via GitGitGadget, Jun 28, 2022
  43. Junio C HamanoJun 28, 2022
  44. Derrick StoleeJun 29, 2022
  45. Junio C HamanoJun 30, 2022
  46. Derrick StoleeJun 30, 2022
  47. 8/8 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jun 28, 2022
  48. 6/8 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jun 28, 2022
  49. Junio C HamanoJun 28, 2022
  50. Derrick StoleeJun 29, 2022
  51. Phillip WoodJul 1, 2022
  52. Elijah NewrenJul 1, 2022
  53. Derrick StoleeJul 4, 2022
  54. Elijah NewrenJul 4, 2022
  55. Derrick StoleeJul 5, 2022
  56. Elijah NewrenJul 8, 2022
  57. 7/8 rebase: update refs from 'update-ref' commandsDerrick Stolee via GitGitGadget, Jun 28, 2022
  58. Junio C HamanoJun 28, 2022
  59. Derrick StoleeJun 29, 2022
  60. Junio C HamanoJun 30, 2022
  61. Derrick StoleeJun 29, 2022
  62. Phillip WoodJul 1, 2022
  63. Junio C HamanoJul 1, 2022
  64. Elijah NewrenJul 1, 2022
  65. Derrick StoleeJul 4, 2022
  66. 5/8 sequencer: add update-ref commandDerrick Stolee via GitGitGadget, Jun 28, 2022
  67. Phillip WoodJun 30, 2022
  68. 4/8 sequencer: define array with enum valuesDerrick Stolee via GitGitGadget, Jun 28, 2022
  69. Junio C HamanoJun 28, 2022
  70. 2/8 branch: consider refs under 'update-refs'Derrick Stolee via GitGitGadget, Jun 28, 2022
  71. Junio C HamanoJun 28, 2022
  72. Derrick StoleeJun 29, 2022
  73. Phillip WoodJun 30, 2022
  74. Junio C HamanoJun 30, 2022
  75. Junio C HamanoJun 30, 2022
  76. Phillip WoodJun 30, 2022
  77. Derrick StoleeJun 30, 2022
  78. Phillip WoodJul 1, 2022
  79. 3/8 rebase-interactive: update 'merge' descriptionDerrick Stolee via GitGitGadget, Jun 28, 2022
  80. Junio C HamanoJun 28, 2022
  81. Derrick StoleeJun 29, 2022
  82. Junio C HamanoJun 30, 2022
  83. Phillip WoodJun 30, 2022
  84. Junio C HamanoJun 28, 2022
  85. Phillip WoodJul 1, 2022
  86. 00/12 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jul 12, 2022
  87. 01/12 t2407: test bisect and rebase as black-boxesDerrick Stolee via GitGitGadget, Jul 12, 2022
  88. 02/12 t2407: test branches currently using apply backendDerrick Stolee via GitGitGadget, Jul 12, 2022
  89. 03/12 branch: consider refs under 'update-refs'Derrick Stolee via GitGitGadget, Jul 12, 2022
  90. Phillip WoodJul 15, 2022
  91. 05/12 sequencer: define array with enum valuesDerrick Stolee via GitGitGadget, Jul 12, 2022
  92. 04/12 rebase-interactive: update 'merge' descriptionDerrick Stolee via GitGitGadget, Jul 12, 2022
  93. 06/12 sequencer: add update-ref commandDerrick Stolee via GitGitGadget, Jul 12, 2022
  94. 07/12 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jul 12, 2022
  95. Elijah NewrenJul 16, 2022
  96. Derrick StoleeJul 19, 2022
  97. SZEDER GáborJul 18, 2022
  98. Derrick StoleeJul 18, 2022
  99. Junio C HamanoJul 18, 2022
  100. Derrick StoleeJul 19, 2022
  101. Junio C HamanoJul 19, 2022
  102. Derrick StoleeJul 19, 2022
  103. 08/12 rebase: update refs from 'update-ref' commandsDerrick Stolee via GitGitGadget, Jul 12, 2022
  104. Phillip WoodJul 15, 2022
  105. Derrick StoleeJul 19, 2022
  106. 09/12 sequencer: rewrite update-refs as user edits todo listDerrick Stolee via GitGitGadget, Jul 12, 2022
  107. Phillip WoodJul 15, 2022
  108. Derrick StoleeJul 15, 2022
  109. Phillip WoodJul 18, 2022
  110. Elijah NewrenJul 16, 2022
  111. 11/12 sequencer: ignore HEAD ref under --update-refsDerrick Stolee via GitGitGadget, Jul 12, 2022
  112. 10/12 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jul 12, 2022
  113. 12/12 sequencer: notify user of --update-refs activityDerrick Stolee via GitGitGadget, Jul 12, 2022
  114. Phillip WoodJul 15, 2022
  115. Derrick StoleeJul 15, 2022
  116. Elijah NewrenJul 16, 2022
  117. Elijah NewrenJul 16, 2022
  118. Derrick StoleeJul 19, 2022
  119. Junio C HamanoJul 12, 2022
  120. Derrick StoleeJul 14, 2022
  121. Junio C HamanoJul 14, 2022
  122. Elijah NewrenJul 16, 2022
  123. Elijah NewrenJul 16, 2022
  124. Phillip WoodJul 15, 2022
  125. 00/12 rebase: update branches in multi-part topicDerrick Stolee via GitGitGadget, Jul 19, 2022
  126. 01/12 t2407: test bisect and rebase as black-boxesDerrick Stolee via GitGitGadget, Jul 19, 2022
  127. 02/12 t2407: test branches currently using apply backendDerrick Stolee via GitGitGadget, Jul 19, 2022
  128. 03/12 branch: consider refs under 'update-refs'Derrick Stolee via GitGitGadget, Jul 19, 2022
  129. 04/12 rebase-interactive: update 'merge' descriptionDerrick Stolee via GitGitGadget, Jul 19, 2022
  130. 05/12 sequencer: define array with enum valuesDerrick Stolee via GitGitGadget, Jul 19, 2022
  131. 06/12 sequencer: add update-ref commandDerrick Stolee via GitGitGadget, Jul 19, 2022
  132. 07/12 rebase: add --update-refs optionDerrick Stolee via GitGitGadget, Jul 19, 2022
  133. 08/12 rebase: update refs from 'update-ref' commandsDerrick Stolee via GitGitGadget, Jul 19, 2022
  134. Phillip WoodJul 21, 2022
  135. 09/12 sequencer: rewrite update-refs as user edits todo listDerrick Stolee via GitGitGadget, Jul 19, 2022
  136. Phillip WoodJul 21, 2022
  137. 10/12 rebase: add rebase.updateRefs config optionDerrick Stolee via GitGitGadget, Jul 19, 2022
  138. 12/12 sequencer: notify user of --update-refs activityDerrick Stolee via GitGitGadget, Jul 19, 2022
  139. 11/12 sequencer: ignore HEAD ref under --update-refsDerrick Stolee via GitGitGadget, Jul 19, 2022
  140. Elijah NewrenJul 21, 2022
  141. Derrick StoleeJul 21, 2022
  142. Elijah NewrenJul 21, 2022
  143. Derrick StoleeJul 21, 2022
  144. Phillip WoodJul 21, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.