{"thread":{"id":"66549","subject":"[PATCH] cache-tree: do not die when the index is locked","startedAt":"2026-10-15T21:42:42Z","lastAt":"2026-10-19T07:01:00Z","messageCount":4,"participants":["Szymon Wilczek","Patrick Steinhardt"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"555199","messageId":"20261015214242.1196722-1-swilczek.lx@gmail.com","threadId":"66549","inReplyTo":null,"subject":"[PATCH] cache-tree: do not die when the index is locked","fromName":"Szymon Wilczek","fromEmail":"swilczek.lx@gmail.com","sentAt":"2026-10-15T21:42:42Z","receivedAt":"2026-10-15T21:42:42Z","isPatch":true,"sender":{"key":"swilczek.lx@gmail.com","avatar":null},"body":"`write_index_as_tree()` locks the index with `LOCK_DIE_ON_ERROR`, even\nthough we only need the lock for the optional write-back of the updated\ncache-tree. As the comment next to `write_locked_index()` already notes,\nfailing to write the index here is fine.\n\nSince 356ee4659b (sequencer: try to commit without forking 'git commit',\n2017-11-24), `try_to_commit()` calls `write_index_as_tree()` in-process\nright after the pick has updated the index and written\n`CHERRY_PICK_HEAD`. Before that commit the `die()` happened in a `git\ncommit` child process, and the sequencer stopped cleanly with its state\nsaved. Now, if a background `git status` holds `index.lock` at that\nmoment, we die before writing `rebase-merge/message`, and a subsequent\n`git rebase --continue` refuses to run:\n\n    error: you have staged changes in your working tree\n\nDrop `LOCK_DIE_ON_ERROR` so that we take the lock gently, just like\nother callers that only refresh the index, and skip writing the index\nwhen the lock is held by someone else. This also lets `git write-tree`\nand `git am` proceed while another process holds the lock.\n\nSigned-off-by: Szymon Wilczek <swilczek.lx@gmail.com>\n---\nI ran into this while rebasing a ~120-commit branch with Magit open,\nwhich runs `git status` in the background. To recover from the\nhalf-picked state, I had to commit the pick manually with\n`git commit -C CHERRY_PICK_HEAD` before running `git rebase --continue`.\n\nThis can be reproduced within a few tries by rebasing a long branch\nwhile running `git status --porcelain` in a loop, or triggered every\ntime in gdb:\n\n    break try_to_commit\n    run\n    shell touch .git/index.lock\n    continue\n\nIf the lock is taken slightly earlier, during `do_recursive_merge()`,\nthe pick still fails, but that path reports an error and reschedules the\npick, so `git rebase --continue` works there.\n\n cache-tree.c          | 10 +++++++---\n t/t0090-cache-tree.sh | 13 +++++++++++++\n 2 files changed, 20 insertions(+), 3 deletions(-)\n\ndiff --git a/cache-tree.c b/cache-tree.c\nindex b8cbb5da22..1bea880912 100644\n--- a/cache-tree.c\n+++ b/cache-tree.c\n@@ -809,11 +809,15 @@ struct tree *write_in_core_index_as_tree(struct repository *repo,\n \n int write_index_as_tree(struct object_id *oid, struct index_state *index_state, const char *index_path, int flags, const char *prefix)\n {\n-\tint entries, was_valid;\n+\tint entries, was_valid, fd;\n \tstruct lock_file lock_file = LOCK_INIT;\n \tint ret;\n \n-\thold_lock_file_for_update(&lock_file, index_path, LOCK_DIE_ON_ERROR);\n+\t/*\n+\t * The lock only protects the cache-tree write-back below; if somebody\n+\t * else holds it, compute the tree without updating the index.\n+\t */\n+\tfd = hold_lock_file_for_update(&lock_file, index_path, 0);\n \n \tentries = read_index_from(index_state, index_path,\n \t\t\t\t  repo_get_git_dir(index_state->repo));\n@@ -828,7 +832,7 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,\n \n \tret = write_index_as_tree_internal(oid, index_state, was_valid, flags,\n \t\t\t\t\t   prefix);\n-\tif (!ret && !was_valid) {\n+\tif (!ret && !was_valid && 0 <= fd) {\n \t\twrite_locked_index(index_state, &lock_file, COMMIT_LOCK);\n \t\t/* Not being able to write is fine -- we are only interested\n \t\t * in updating the cache-tree part, and if the next caller\ndiff --git a/t/t0090-cache-tree.sh b/t/t0090-cache-tree.sh\nindex 0964718d7f..4b7547e108 100755\n--- a/t/t0090-cache-tree.sh\n+++ b/t/t0090-cache-tree.sh\n@@ -286,4 +286,17 @@ test_expect_success 'cache-tree is used by write-tree when valid' '\n \ttest_grep ! region_enter.*cache_tree.*update trace.output\n '\n \n+test_expect_success 'write-tree succeeds while the index is locked' '\n+\ttest_when_finished \"rm -f .git/index.lock\" &&\n+\ttest_commit locked &&\n+\t>uncommitted &&\n+\tgit add uncommitted &&\n+\t>.git/index.lock &&\n+\tgit write-tree >actual &&\n+\ttest_path_is_file .git/index.lock &&\n+\trm .git/index.lock &&\n+\tgit write-tree >expect &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\n\nbase-commit: f57c9a73884400380bddd172acc88b569834f909\n-- \n2.56.0\n\n\n"},{"id":"555209","messageId":"atG8oVH1otZRVtf8@pks.im","threadId":"66549","inReplyTo":"20261015214242.1196722-1-swilczek.lx@gmail.com","subject":"Re: [PATCH] cache-tree: do not die when the index is locked","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-16T05:56:49Z","receivedAt":"2026-10-16T05:56:49Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Oct 15, 2026 at 11:42:42PM +0200, Szymon Wilczek wrote:\n> `write_index_as_tree()` locks the index with `LOCK_DIE_ON_ERROR`, even\n> though we only need the lock for the optional write-back of the updated\n> cache-tree. As the comment next to `write_locked_index()` already notes,\n> failing to write the index here is fine.\n\nGiving a bit of context here would help the reader to figure out what\nthe function even does. Namely, it's two different things:\n\n  - It is responsible for turning the index into a tree object.\n\n  - And when that succeeds, it will also rewrite the index to populate\n    the cache tree extension.\n\nThe first part is what must succeed, but the second part is allowed to\nfail without impacting the result.\n\n> Since 356ee4659b (sequencer: try to commit without forking 'git commit',\n> 2017-11-24), `try_to_commit()` calls `write_index_as_tree()` in-process\n> right after the pick has updated the index and written\n> `CHERRY_PICK_HEAD`. Before that commit the `die()` happened in a `git\n> commit` child process, and the sequencer stopped cleanly with its state\n> saved. Now, if a background `git status` holds `index.lock` at that\n> moment, we die before writing `rebase-merge/message`, and a subsequent\n> `git rebase --continue` refuses to run:\n> \n>     error: you have staged changes in your working tree\n\nThis is the symptom that you're trying to fix and should come first in\nyour message. Commit messages typically describe the problem first\n(this), then explain cause of  and context around that problem (your\nabove information about `write_index_as_tree()`) and then finally how\nyou solve the problem (the next paragraph).\n\n> diff --git a/cache-tree.c b/cache-tree.c\n> index b8cbb5da22..1bea880912 100644\n> --- a/cache-tree.c\n> +++ b/cache-tree.c\n> @@ -809,11 +809,15 @@ struct tree *write_in_core_index_as_tree(struct repository *repo,\n>  \n>  int write_index_as_tree(struct object_id *oid, struct index_state *index_state, const char *index_path, int flags, const char *prefix)\n>  {\n> -\tint entries, was_valid;\n> +\tint entries, was_valid, fd;\n>  \tstruct lock_file lock_file = LOCK_INIT;\n>  \tint ret;\n>  \n> -\thold_lock_file_for_update(&lock_file, index_path, LOCK_DIE_ON_ERROR);\n> +\t/*\n> +\t * The lock only protects the cache-tree write-back below; if somebody\n> +\t * else holds it, compute the tree without updating the index.\n> +\t */\n> +\tfd = hold_lock_file_for_update(&lock_file, index_path, 0);\n\nAn alternative would be to supply a locking timeout, but given that\nwriting the cache tree extension is entirely optional I agree that this\nhere is the better fix.\n\n> @@ -828,7 +832,7 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,\n>  \n>  \tret = write_index_as_tree_internal(oid, index_state, was_valid, flags,\n>  \t\t\t\t\t   prefix);\n> -\tif (!ret && !was_valid) {\n> +\tif (!ret && !was_valid && 0 <= fd) {\n\nNit: This may just be me, but this reverse notation always confuses me,\nand I think writing this as `fd > 0` feels more natural to many. Anyway,\nI don't feel strongly about this as this may very well be subjective.\n\n>  \t\twrite_locked_index(index_state, &lock_file, COMMIT_LOCK);\n>  \t\t/* Not being able to write is fine -- we are only interested\n>  \t\t * in updating the cache-tree part, and if the next caller\n\nAnd here we explain that it's fine if we fail writing the cache tree\nextension. Makes sense.\n\n> diff --git a/t/t0090-cache-tree.sh b/t/t0090-cache-tree.sh\n> index 0964718d7f..4b7547e108 100755\n> --- a/t/t0090-cache-tree.sh\n> +++ b/t/t0090-cache-tree.sh\n> @@ -286,4 +286,17 @@ test_expect_success 'cache-tree is used by write-tree when valid' '\n>  \ttest_grep ! region_enter.*cache_tree.*update trace.output\n>  '\n>  \n> +test_expect_success 'write-tree succeeds while the index is locked' '\n> +\ttest_when_finished \"rm -f .git/index.lock\" &&\n> +\ttest_commit locked &&\n> +\t>uncommitted &&\n> +\tgit add uncommitted &&\n> +\t>.git/index.lock &&\n> +\tgit write-tree >actual &&\n> +\ttest_path_is_file .git/index.lock &&\n> +\trm .git/index.lock &&\n> +\tgit write-tree >expect &&\n> +\ttest_cmp expect actual\n> +'\n\nThe test looks obviously good to me. Nice that the issue is this easy to\nreproduce.\n\nThanks!\n\nPatrick\n\n"},{"id":"555225","messageId":"20261016091647.1332692-1-swilczek.lx@gmail.com","threadId":"66549","inReplyTo":"20261015214242.1196722-1-swilczek.lx@gmail.com","subject":"[PATCH v2] cache-tree: do not die when the index is locked","fromName":"Szymon Wilczek","fromEmail":"swilczek.lx@gmail.com","sentAt":"2026-10-16T09:16:47Z","receivedAt":"2026-10-16T09:16:47Z","isPatch":true,"sender":{"key":"swilczek.lx@gmail.com","avatar":null},"body":"When a background process such as `git status` holds `index.lock` while\n`git rebase` picks a commit, the rebase can die in the middle of the\npick and leave the sequencer in a state from which `git rebase\n--continue` refuses to run:\n\n    error: you have staged changes in your working tree\n\nThis happens because `try_to_commit()` calls `write_index_as_tree()`\nright after the pick has updated the index and written\n`CHERRY_PICK_HEAD`, and `write_index_as_tree()` dies when it cannot\ntake the index lock. We then die before writing `rebase-merge/message`.\nSince 356ee4659b (sequencer: try to commit without forking 'git commit',\n2017-11-24), this call happens in-process. Before that commit the\n`die()` happened in a `git commit` child process, and the sequencer\nstopped cleanly with its state saved.\n\n`write_index_as_tree()` does two different things:\n\n  - It turns the index into a tree object. This part must succeed.\n\n  - When that succeeds, it also rewrites the index to populate the\n    cache-tree extension. This part is allowed to fail without\n    impacting the result; as the comment next to\n    `write_locked_index()` already notes, failing to write the index\n    here is fine.\n\nYet the function locks the index with `LOCK_DIE_ON_ERROR`, even though\nwe only need the lock for the optional second part.\n\nDrop `LOCK_DIE_ON_ERROR` so that we take the lock gently, just like\nother callers that only refresh the index, and skip writing the index\nwhen the lock is held by someone else. This also lets `git write-tree`\nand `git am` proceed while another process holds the lock.\n\nSigned-off-by: Szymon Wilczek <swilczek.lx@gmail.com>\n---\nChanges in v2:\n\n  - Reordered the commit message to describe the symptom first, then\n    its cause, and the fix last, and explained the two things that\n    `write_index_as_tree()` does (thanks, Patrick).\n\n  - Spelled the lock check as `fd >= 0` instead of `0 <= fd`.\n\nNotes from v1:\n\nI ran into this while rebasing a ~120-commit branch with Magit open,\nwhich runs `git status` in the background. To recover from the\nhalf-picked state, I had to commit the pick manually with\n`git commit -C CHERRY_PICK_HEAD` before running `git rebase --continue`.\n\nThis can be reproduced within a few tries by rebasing a long branch\nwhile running `git status --porcelain` in a loop, or triggered every\ntime in gdb:\n\n    break try_to_commit\n    run\n    shell touch .git/index.lock\n    continue\n\nIf the lock is taken slightly earlier, during `do_recursive_merge()`,\nthe pick still fails, but that path reports an error and reschedules the\npick, so `git rebase --continue` works there.\n\n cache-tree.c          | 10 +++++++---\n t/t0090-cache-tree.sh | 13 +++++++++++++\n 2 files changed, 20 insertions(+), 3 deletions(-)\n\ndiff --git a/cache-tree.c b/cache-tree.c\nindex b8cbb5da22..671d4165bf 100644\n--- a/cache-tree.c\n+++ b/cache-tree.c\n@@ -809,11 +809,15 @@ struct tree *write_in_core_index_as_tree(struct repository *repo,\n \n int write_index_as_tree(struct object_id *oid, struct index_state *index_state, const char *index_path, int flags, const char *prefix)\n {\n-\tint entries, was_valid;\n+\tint entries, was_valid, fd;\n \tstruct lock_file lock_file = LOCK_INIT;\n \tint ret;\n \n-\thold_lock_file_for_update(&lock_file, index_path, LOCK_DIE_ON_ERROR);\n+\t/*\n+\t * The lock only protects the cache-tree write-back below; if somebody\n+\t * else holds it, compute the tree without updating the index.\n+\t */\n+\tfd = hold_lock_file_for_update(&lock_file, index_path, 0);\n \n \tentries = read_index_from(index_state, index_path,\n \t\t\t\t  repo_get_git_dir(index_state->repo));\n@@ -828,7 +832,7 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,\n \n \tret = write_index_as_tree_internal(oid, index_state, was_valid, flags,\n \t\t\t\t\t   prefix);\n-\tif (!ret && !was_valid) {\n+\tif (!ret && !was_valid && fd >= 0) {\n \t\twrite_locked_index(index_state, &lock_file, COMMIT_LOCK);\n \t\t/* Not being able to write is fine -- we are only interested\n \t\t * in updating the cache-tree part, and if the next caller\ndiff --git a/t/t0090-cache-tree.sh b/t/t0090-cache-tree.sh\nindex 0964718d7f..4b7547e108 100755\n--- a/t/t0090-cache-tree.sh\n+++ b/t/t0090-cache-tree.sh\n@@ -286,4 +286,17 @@ test_expect_success 'cache-tree is used by write-tree when valid' '\n \ttest_grep ! region_enter.*cache_tree.*update trace.output\n '\n \n+test_expect_success 'write-tree succeeds while the index is locked' '\n+\ttest_when_finished \"rm -f .git/index.lock\" &&\n+\ttest_commit locked &&\n+\t>uncommitted &&\n+\tgit add uncommitted &&\n+\t>.git/index.lock &&\n+\tgit write-tree >actual &&\n+\ttest_path_is_file .git/index.lock &&\n+\trm .git/index.lock &&\n+\tgit write-tree >expect &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\n\nRange-diff against v1:\n1:  d16173cf1c ! 1:  cdfdd81c3f cache-tree: do not die when the index is locked\n    @@ Metadata\n      ## Commit message ##\n         cache-tree: do not die when the index is locked\n     \n    -    `write_index_as_tree()` locks the index with `LOCK_DIE_ON_ERROR`, even\n    -    though we only need the lock for the optional write-back of the updated\n    -    cache-tree. As the comment next to `write_locked_index()` already notes,\n    -    failing to write the index here is fine.\n    +    When a background process such as `git status` holds `index.lock` while\n    +    `git rebase` picks a commit, the rebase can die in the middle of the\n    +    pick and leave the sequencer in a state from which `git rebase\n    +    --continue` refuses to run:\n     \n    -    Since 356ee4659b (sequencer: try to commit without forking 'git commit',\n    -    2017-11-24), `try_to_commit()` calls `write_index_as_tree()` in-process\n    +        error: you have staged changes in your working tree\n    +\n    +    This happens because `try_to_commit()` calls `write_index_as_tree()`\n         right after the pick has updated the index and written\n    -    `CHERRY_PICK_HEAD`. Before that commit the `die()` happened in a `git\n    -    commit` child process, and the sequencer stopped cleanly with its state\n    -    saved. Now, if a background `git status` holds `index.lock` at that\n    -    moment, we die before writing `rebase-merge/message`, and a subsequent\n    -    `git rebase --continue` refuses to run:\n    +    `CHERRY_PICK_HEAD`, and `write_index_as_tree()` dies when it cannot\n    +    take the index lock. We then die before writing `rebase-merge/message`.\n    +    Since 356ee4659b (sequencer: try to commit without forking 'git commit',\n    +    2017-11-24), this call happens in-process. Before that commit the\n    +    `die()` happened in a `git commit` child process, and the sequencer\n    +    stopped cleanly with its state saved.\n     \n    -        error: you have staged changes in your working tree\n    +    `write_index_as_tree()` does two different things:\n    +\n    +      - It turns the index into a tree object. This part must succeed.\n    +\n    +      - When that succeeds, it also rewrites the index to populate the\n    +        cache-tree extension. This part is allowed to fail without\n    +        impacting the result; as the comment next to\n    +        `write_locked_index()` already notes, failing to write the index\n    +        here is fine.\n    +\n    +    Yet the function locks the index with `LOCK_DIE_ON_ERROR`, even though\n    +    we only need the lock for the optional second part.\n     \n         Drop `LOCK_DIE_ON_ERROR` so that we take the lock gently, just like\n         other callers that only refresh the index, and skip writing the index\n    @@ cache-tree.c: int write_index_as_tree(struct object_id *oid, struct index_state\n      \tret = write_index_as_tree_internal(oid, index_state, was_valid, flags,\n      \t\t\t\t\t   prefix);\n     -\tif (!ret && !was_valid) {\n    -+\tif (!ret && !was_valid && 0 <= fd) {\n    ++\tif (!ret && !was_valid && fd >= 0) {\n      \t\twrite_locked_index(index_state, &lock_file, COMMIT_LOCK);\n      \t\t/* Not being able to write is fine -- we are only interested\n      \t\t * in updating the cache-tree part, and if the next caller\n\nbase-commit: f57c9a73884400380bddd172acc88b569834f909\n-- \n2.56.0\n\n\n"},{"id":"555390","messageId":"atXALEuiGk0TrdMN@pks.im","threadId":"66549","inReplyTo":"20261016091647.1332692-1-swilczek.lx@gmail.com","subject":"Re: [PATCH v2] cache-tree: do not die when the index is locked","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-19T07:01:00Z","receivedAt":"2026-10-19T07:01:00Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Oct 16, 2026 at 11:16:47AM +0200, Szymon Wilczek wrote:\n> Changes in v2:\n> \n>   - Reordered the commit message to describe the symptom first, then\n>     its cause, and the fix last, and explained the two things that\n>     `write_index_as_tree()` does (thanks, Patrick).\n> \n>   - Spelled the lock check as `fd >= 0` instead of `0 <= fd`.\n\nThanks, I'm happy with this version.\n\nPatrick\n\n"}]}