Well, yes and no.
No, because we are dealing with regression in v2.24.0 here, so the simpler the fix the better it is for maint. I don't think a fix can get any simpler than my patch, with or without the suggestions from Phillip.
Yes, we should definitely consider updating the stat data after the sequencer writes the todo list, or any other options with which the sequencer could notice a modified todo list file with less subtlety. Alas, there is a big can of worms in that direction, see the patch below, and we have to be very careful going that way, so I think it's only viable in the long term, but less suitable as a regression fix for maint.
(Hrm, perhaps I spent too many words on the all zeroed out stat data, and managed to sidetrack you a bit...)
--- >8 ---
Subject: [PATCH] t3429: try to protect against a potential racy todo file
problem
During an interactive rebase session the user can run basically any commands with an 'exec' instruction, including commands that modify the todo list of the ongoing rebase. Similarly, the user might choose to modify the todo list while editing a commit message for a 'reword' or a 'squash' instruction. The ongoing interactive rebase process should be able to notice the modified todo file and re-read it to follow the updated instructions. This has been working well for the 'exec' instruction for some time, but for 'reword' and 'squash' it has been fixed only recently in a47ba3c777 (rebase -i: check for updated todo after squash and reword, 2019-08-19).
To notice a changed todo file the sequencer stores the file's stat data in its 'struct todo_list' instance, and compares it with the file's current stat data after 'reword', 'squash' and 'exec' instructions. If the two stat data doesn't match, it re-reads the todo file.
Sounds simple, but there are some subtleties going on here:
- The 'struct todo_list' holds the stat data from the time when the
todo file was last read. - This stat data in 'struct todo_list' is not updated when the
sequencer itself writes the todo file. - Before executing each instruction during an interactive rebase,
the sequencer always updates the todo file by removing the
just-about-to-be-executed instruction. This changes the file's
size and inode [1].Consequently, when the sequencer looks at the stat data after a 'reword', 'squash' or 'exec' instruction, it most likely finds that they differ, even when the user didn't modify the todo list at all! This is not an issue in practice, it just wastes a few cycles on re-reading the todo list that matches what the sequencer already has in memory anyway.
However, an unsuspecting Git developer might try to "fix" it by simply updating the stat data each time the sequencer writes the todo list for an interactive rebase. On first sight it looks quite sensible and straightforward, but we have to be very careful when doing that, because potential racy problems lie that way.
It is possible to overwrite the todo list file without modifying either its inode or size, and if such an overwrite were to happen in the same second when the file was last read (our stat data has one second granularity by default), then the actual stat data on the file system would match the stat data that the sequencer has in memory. Consequently, such a modification to the todo list file would go unnoticed.
Add a test to 't3429-rebase-edit-todo.sh' that modifies the todo list during an interactive rebase in a way that the file's stat data very likely doesn't change. Since we have no control over ctime repeat the test a few times, just like we do in the racy git and racy split index tests (t0010 and t1701). This test succeeds right now because of all the above, but it does fail most of the time if the stat data is naively updated when writing the todo file [2]. Hopefully this test will help prevent us from introducing such racy problems if we ever change how the sequencer looks out for modified todo list files.
This new test only checks what it's supposed to when it doesn't change the size of the todo file while modifying it, thus it critically depends on the todo file format. And the todo file format did change in the past: early on it stored abbreviated commit object ids, but since 75c6976655 (rebase -i: fix short SHA-1 collision, 2013-08-23) it stores full object ids. Perhaps it will change in the future as well, so be extra defensive about this, and check that the file size before and after the edit actually match, and complain with a "bug in the test script" if they don't.
[1] The todo file is written using the lockfile API, i.e. first to the
lockfile, which is then moved to place, so the new file can't
possibly have the same inode as the file it replaces. Note,
however, that the file system might reuse inodes, so it is
possible that the new todo file ends up with the same inode as is
recorded in the 'struct todo_list' from the last time the file was
read.[2] To trigger failure in the new test updating the stat data when
writing the todo file like the diff below does. No other test in
our test suite seems to fail with this change applied. diff --git a/sequencer.c b/sequencer.c
index 3b05d0277d..b2acb7d536 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2730,6 +2730,7 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
struct lock_file todo_lock = LOCK_INIT;
const char *todo_path = get_todo_path(opts);
int next = todo_list->current, offset, fd;
+ struct stat st; /*
* rebase -i writes "git-rebase-todo" without the currently executing
@@ -2748,6 +2749,10 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
if (commit_lock_file(&todo_lock) < 0)
return error(_("failed to finalize '%s'"), todo_path); + if (stat(todo_path, &st) < 0)
+ return error(_("could not stat '%s'"), todo_path);
+ fill_stat_data(&todo_list->stat, &st);
+
if (is_rebase_i(opts) && next > 0) {
const char *done = rebase_path_done();
int fd = open(done, O_CREAT | O_WRONLY | O_APPEND, 0666);Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
t/t3429-rebase-edit-todo.sh | 59 +++++++++++++++++++++++++++++++++++++
1 file changed, 59 insertions(+)
