{"thread":{"id":"64395","subject":"[PATCH] refs: support migration with worktrees","startedAt":"2025-10-27T18:26:24Z","lastAt":"2025-10-30T06:37:31Z","messageCount":8,"participants":["Sam Bostock via GitGitGadget","Patrick Steinhardt","Junio C Hamano","Kristoffer Haugsbakk","Ben Knoble"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"529749","messageId":"pull.2077.git.git.1761589580028.gitgitgadget@gmail.com","threadId":"64395","inReplyTo":null,"subject":"[PATCH] refs: support migration with worktrees","fromName":"Sam Bostock via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-10-27T18:26:20Z","receivedAt":"2025-10-27T18:26:24Z","isPatch":true,"sender":{"key":"sam.bostock@shopify.com","avatar":"https://avatars.githubusercontent.com/u/8219340?v=4"},"body":"From: Sam Bostock <sam.bostock@shopify.com>\n\nRemove the worktree limitation from `git refs migrate` by implementing\nmigration support for repositories with linked worktrees.\n\nPreviously, attempting to migrate a repository with worktrees would fail\nwith \"migrating repositories with worktrees is not supported yet\". This\nlimitation existed because each worktree has its own ref storage that\nneeded to be migrated separately.\n\nMigration now uses a multi-phase approach to safely handle multiple\nworktrees:\n\n1. Phase 1: Iterate through all worktrees and create temporary new ref\n   storage for each in a staging directory.\n\n2. Phase 2: For each worktree, backup the existing ref storage, then\n   move the new storage into place.\n\n3. Phase 3: Update the repository format config, clear cached ref stores,\n   and delete all backups. On failure, restore from backups and report\n   where the migrated refs can be found for manual recovery.\n\nThis approach ensures that if migration fails partway through, the\nrepository can be restored to its original state.\n\nKey implementation details:\n\n- For files backend: Create a commondir file in temp directories for\n  linked worktrees so the files backend knows where the common git\n  directory is located.\n\n- For linked worktrees: Use non-INITIAL transactions to avoid creating\n  packed-refs files (linked worktrees should never have packed-refs).\n\n- Filter refs during iteration: Linked worktrees only migrate their\n  per-worktree refs (refs/bisect/*, refs/rewritten/*, refs/worktree/*).\n  Shared refs are migrated once in the main worktree.\n\n- Write per-worktree refs as loose files: The files backend's\n  transaction_finish_initial() optimization writes most refs to\n  packed-refs, but per-worktree refs must be stored as loose files\n  to maintain proper worktree isolation.\n\n- Backup root refs: During Phase 2, backup all root refs (HEAD,\n  ORIG_HEAD, etc.) by iterating files in the git directory and using\n  is_root_ref() to identify them. This ensures safe rollback if\n  migration fails.\n\nTests are updated to expect migration with worktrees to succeed, and\nnew tests verify:\n- Basic worktree migration in both directions (files ↔ reftable)\n- Migration with multiple worktrees\n- Dry-run mode with worktrees\n- Physical separation of per-worktree refs\n- Bare repository with worktrees\n\nDocumentation is updated to reflect the worktree support and note that\nmigration must be run from the main worktree.\n\nAs the author's familiarity with git internals and C is limited, this\nchange was made with the assistance of Claude Code. However, the author\nhas carefully reviewed and iterated on the work to ensure quality to the\nbest of their ability.\n\nSigned-off-by: Sam Bostock <sam.bostock@shopify.com>\nCo-Authored-By: Claude <noreply@anthropic.com>\n---\n    Teach git refs migrate to support worktrees\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2077%2Fsambostock%2Frefs-migrate-worktree-support-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2077/sambostock/refs-migrate-worktree-support-v1\nPull-Request: https://github.com/git/git/pull/2077\n\n Documentation/git-refs.adoc |   5 +-\n refs.c                      | 726 ++++++++++++++++++++++++++++--------\n refs/files-backend.c        |   6 +-\n t/t1460-refs-migrate.sh     | 446 +++++++++++++++++++++-\n 4 files changed, 1023 insertions(+), 160 deletions(-)\n\ndiff --git a/Documentation/git-refs.adoc b/Documentation/git-refs.adoc\nindex fa33680cc7..f6a3bf4f03 100644\n--- a/Documentation/git-refs.adoc\n+++ b/Documentation/git-refs.adoc\n@@ -30,7 +30,8 @@ COMMANDS\n --------\n \n `migrate`::\n-\tMigrate ref store between different formats.\n+\tMigrate ref store between different formats. Supports repositories\n+\twith worktrees; migration must be run from the main worktree.\n \n `verify`::\n \tVerify reference database consistency.\n@@ -95,7 +96,7 @@ KNOWN LIMITATIONS\n \n The ref format migration has several known limitations in its current form:\n \n-* It is not possible to migrate repositories that have worktrees.\n+* Migration must be run from the main worktree.\n \n * There is no way to block concurrent writes to the repository during an\n   ongoing migration. Concurrent writes can lead to an inconsistent migrated\ndiff --git a/refs.c b/refs.c\nindex 965381367e..0834329e5a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -5,6 +5,7 @@\n #define USE_THE_REPOSITORY_VARIABLE\n \n #include \"git-compat-util.h\"\n+#include \"abspath.h\"\n #include \"advice.h\"\n #include \"config.h\"\n #include \"environment.h\"\n@@ -28,6 +29,7 @@\n #include \"setup.h\"\n #include \"sigchain.h\"\n #include \"date.h\"\n+#include \"dir.h\"\n #include \"commit.h\"\n #include \"wildmatch.h\"\n #include \"ident.h\"\n@@ -2974,8 +2976,296 @@ struct migration_data {\n \tstruct strbuf *errbuf;\n \tstruct strbuf sb, name, mail;\n \tuint64_t index;\n+\tint is_main_worktree;\n };\n \n+/*\n+ * Holds the state for migrating a single worktree's ref storage.\n+ */\n+struct worktree_migration_data {\n+\tstruct worktree *worktree;\n+\tstruct ref_store *old_refs;\n+\tstruct ref_store *new_refs;\n+\tstruct strbuf new_dir;\n+\tstruct strbuf backup_dir;\n+\tint is_main;\n+};\n+\n+static void worktree_migration_data_release(struct worktree_migration_data *wt_data)\n+{\n+\tif (wt_data->new_refs) {\n+\t\tref_store_release(wt_data->new_refs);\n+\t\tFREE_AND_NULL(wt_data->new_refs);\n+\t}\n+\tstrbuf_release(&wt_data->new_dir);\n+\tstrbuf_release(&wt_data->backup_dir);\n+}\n+\n+static void worktree_migration_data_array_release(struct worktree_migration_data *wt_data,\n+\t\t\t\t\t\t   size_t nr)\n+{\n+\tfor (size_t i = 0; i < nr; i++)\n+\t\tworktree_migration_data_release(&wt_data[i]);\n+\tfree(wt_data);\n+}\n+\n+/*\n+ * Create a commondir file in the temporary migration directory for a linked\n+ * worktree. The files backend needs this to locate the common git directory.\n+ * Returns 0 on success, -1 on failure.\n+ */\n+static int create_commondir_file(const char *new_dir, const char *worktree_path,\n+\t\t\t\t  struct strbuf *errbuf)\n+{\n+\tstruct strbuf commondir_path = STRBUF_INIT;\n+\tstruct strbuf commondir_content = STRBUF_INIT;\n+\tint fd = -1;\n+\tint ret = 0;\n+\n+\tstrbuf_addf(&commondir_path, \"%s/commondir\", new_dir);\n+\tstrbuf_addstr(&commondir_content, \"../..\\n\");\n+\n+\tfd = open(commondir_path.buf, O_WRONLY | O_CREAT | O_EXCL, 0666);\n+\tif (fd < 0) {\n+\t\tstrbuf_addf(errbuf, _(\"cannot create commondir file for worktree '%s': %s\"),\n+\t\t\t    worktree_path, strerror(errno));\n+\t\tret = -1;\n+\t\tgoto done;\n+\t}\n+\n+\tif (write_in_full(fd, commondir_content.buf, commondir_content.len) < 0) {\n+\t\tstrbuf_addf(errbuf, _(\"cannot write commondir file for worktree '%s': %s\"),\n+\t\t\t    worktree_path, strerror(errno));\n+\t\tret = -1;\n+\t\tgoto done;\n+\t}\n+\n+done:\n+\tif (fd >= 0)\n+\t\tclose(fd);\n+\tstrbuf_release(&commondir_path);\n+\tstrbuf_release(&commondir_content);\n+\treturn ret;\n+}\n+\n+/*\n+ * Returns the list of ref storage items to backup/restore for a worktree.\n+ * Main worktrees include packed-refs, linked worktrees do not.\n+ */\n+static const char **get_ref_storage_items(int is_main_worktree)\n+{\n+\tstatic const char *main_items[] = {\"refs\", \"logs\", \"reftable\", \"packed-refs\", NULL};\n+\tstatic const char *linked_items[] = {\"refs\", \"logs\", \"reftable\", NULL};\n+\n+\treturn is_main_worktree ? main_items : linked_items;\n+}\n+\n+/*\n+ * Move root ref files from one directory to another. Root refs are individual\n+ * files in the git directory like HEAD, ORIG_HEAD, etc. If remove_dest is set,\n+ * unlink the destination file before moving.\n+ *\n+ * Returns -1 on fatal error (cannot open source directory), 0 on success,\n+ * or positive count of files that failed to move. Detailed error messages\n+ * are appended to errbuf if provided.\n+ */\n+static int move_root_refs(const char *from_dir, const char *to_dir,\n+\t\t\t  int remove_dest, struct strbuf *errbuf)\n+{\n+\tstruct strbuf from = STRBUF_INIT, to = STRBUF_INIT;\n+\tDIR *dir = opendir(from_dir);\n+\tstruct dirent *e;\n+\tint ret = 0;\n+\tint failed_moves = 0;\n+\n+\tif (!dir) {\n+\t\tret = -1;\n+\t\tgoto done;\n+\t}\n+\n+\twhile ((e = readdir(dir))) {\n+\t\tstruct stat st;\n+\n+\t\tif (!strcmp(e->d_name, \".\") || !strcmp(e->d_name, \"..\"))\n+\t\t\tcontinue;\n+\n+\t\t/* Only process files that are root refs */\n+\t\tif (!is_root_ref(e->d_name))\n+\t\t\tcontinue;\n+\n+\t\tstrbuf_reset(&from);\n+\t\tstrbuf_addf(&from, \"%s/%s\", from_dir, e->d_name);\n+\n+\t\tif (stat(from.buf, &st) < 0) {\n+\t\t\tif (errno != ENOENT && errbuf) {\n+\t\t\t\tstrbuf_addf(errbuf, _(\"could not stat '%s': %s; \"),\n+\t\t\t\t\t    from.buf, strerror(errno));\n+\t\t\t\tfailed_moves++;\n+\t\t\t}\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (!S_ISREG(st.st_mode))\n+\t\t\tcontinue; /* skip non-files */\n+\n+\t\tstrbuf_reset(&to);\n+\t\tstrbuf_addf(&to, \"%s/%s\", to_dir, e->d_name);\n+\n+\t\tif (remove_dest) {\n+\t\t\tif (unlink(to.buf) < 0 && errno != ENOENT && errbuf) {\n+\t\t\t\tstrbuf_addf(errbuf, _(\"could not unlink '%s': %s; \"),\n+\t\t\t\t\t    to.buf, strerror(errno));\n+\t\t\t\tfailed_moves++;\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (rename(from.buf, to.buf) < 0) {\n+\t\t\tif (errbuf) {\n+\t\t\t\tstrbuf_addf(errbuf, _(\"could not move '%s' to '%s': %s; \"),\n+\t\t\t\t\t    from.buf, to.buf, strerror(errno));\n+\t\t\t\tfailed_moves++;\n+\t\t\t}\n+\t\t}\n+\t}\n+\tclosedir(dir);\n+\tret = failed_moves;\n+\n+done:\n+\tstrbuf_release(&from);\n+\tstrbuf_release(&to);\n+\treturn ret;\n+}\n+\n+/*\n+ * Backup ref storage by moving ref-related files/directories to a backup\n+ * location. Returns 0 on success, -1 on failure.\n+ */\n+static int backup_ref_storage(const char *gitdir, const char *backup_dir,\n+\t\t\t       int is_main_worktree, struct strbuf *errbuf)\n+{\n+\tstruct strbuf from = STRBUF_INIT, to = STRBUF_INIT;\n+\tconst char **items = get_ref_storage_items(is_main_worktree);\n+\tsize_t i;\n+\tint ret = 0;\n+\n+\t/*\n+\t * Move ref-related files and directories. Not all will exist depending\n+\t * on the backend, which is fine.\n+\t */\n+\tfor (i = 0; items[i]; i++) {\n+\t\tconst char *item;\n+\t\tstruct stat st;\n+\n+\t\titem = items[i];\n+\t\tstrbuf_reset(&from);\n+\t\tstrbuf_addf(&from, \"%s/%s\", gitdir, item);\n+\n+\t\tif (stat(from.buf, &st) < 0) {\n+\t\t\tif (errno == ENOENT)\n+\t\t\t\tcontinue; /* doesn't exist, skip */\n+\t\t\tstrbuf_addf(errbuf, _(\"could not stat '%s': %s\"),\n+\t\t\t\t    from.buf, strerror(errno));\n+\t\t\tret = -1;\n+\t\t\tgoto done;\n+\t\t}\n+\n+\t\tstrbuf_reset(&to);\n+\t\tstrbuf_addf(&to, \"%s/%s\", backup_dir, item);\n+\n+\t\tif (rename(from.buf, to.buf) < 0) {\n+\t\t\tstrbuf_addf(errbuf, _(\"could not move '%s' to '%s': %s\"),\n+\t\t\t\t    from.buf, to.buf, strerror(errno));\n+\t\t\tret = -1;\n+\t\t\tgoto done;\n+\t\t}\n+\t}\n+\n+\t/* Backup root refs (HEAD, ORIG_HEAD, etc.) */\n+\tret = move_root_refs(gitdir, backup_dir, 0, errbuf);\n+\tif (ret < 0) {\n+\t\tstrbuf_addf(errbuf, _(\"could not open directory '%s' to backup root refs: %s\"),\n+\t\t\t    gitdir, strerror(errno));\n+\t\tret = -1;\n+\t\tgoto done;\n+\t} else if (ret > 0) {\n+\t\t/* Some root refs failed to backup - this is fatal */\n+\t\tstrbuf_addstr(errbuf, _(\"failed to backup some root refs\"));\n+\t\tret = -1;\n+\t\tgoto done;\n+\t}\n+\tret = 0;\n+\n+done:\n+\tstrbuf_release(&from);\n+\tstrbuf_release(&to);\n+\treturn ret;\n+}\n+\n+/*\n+ * Restore ref storage from backup by moving files back.\n+ */\n+static void restore_ref_storage_from_backup(const char *gitdir,\n+\t\t\t\t\t     const char *backup_dir,\n+\t\t\t\t\t     int is_main_worktree)\n+{\n+\tstruct strbuf from = STRBUF_INIT, to = STRBUF_INIT;\n+\tconst char **items = get_ref_storage_items(is_main_worktree);\n+\tsize_t i;\n+\n+\tfor (i = 0; items[i]; i++) {\n+\t\tconst char *item;\n+\t\tstruct stat st;\n+\n+\t\titem = items[i];\n+\t\tstrbuf_reset(&from);\n+\t\tstrbuf_addf(&from, \"%s/%s\", backup_dir, item);\n+\n+\t\tif (stat(from.buf, &st) < 0)\n+\t\t\tcontinue; /* doesn't exist in backup */\n+\n+\t\tstrbuf_reset(&to);\n+\t\tstrbuf_addf(&to, \"%s/%s\", gitdir, item);\n+\n+\t\t/* Remove what's currently there (new storage that failed) */\n+\t\tif (stat(to.buf, &st) == 0) {\n+\t\t\tif (S_ISDIR(st.st_mode))\n+\t\t\t\tremove_dir_recursively(&to, 0);\n+\t\t\telse\n+\t\t\t\tunlink(to.buf);\n+\t\t}\n+\n+\t\trename(from.buf, to.buf);\n+\t}\n+\n+\t/* Restore root refs from backup */\n+\t{\n+\t\tstruct strbuf restore_err = STRBUF_INIT;\n+\t\tint restore_ret = move_root_refs(backup_dir, gitdir, 1, &restore_err);\n+\t\tif (restore_ret < 0) {\n+\t\t\twarning_errno(_(\"could not open directory '%s' to restore root refs\"), backup_dir);\n+\t\t} else if (restore_ret > 0) {\n+\t\t\twarning(_(\"failed to restore some root refs: %s\"), restore_err.buf);\n+\t\t}\n+\t\tstrbuf_release(&restore_err);\n+\t}\n+\n+\tstrbuf_release(&from);\n+\tstrbuf_release(&to);\n+}\n+\n+/*\n+ * Delete backup directory and its contents.\n+ */\n+static void delete_backup(const char *backup_dir)\n+{\n+\tstruct strbuf path = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&path, backup_dir);\n+\tremove_dir_recursively(&path, 0);\n+\tstrbuf_release(&path);\n+}\n+\n static int migrate_one_ref(const char *refname, const char *referent UNUSED, const struct object_id *oid,\n \t\t\t   int flags, void *cb_data)\n {\n@@ -2983,6 +3273,16 @@ static int migrate_one_ref(const char *refname, const char *referent UNUSED, con\n \tstruct strbuf symref_target = STRBUF_INIT;\n \tint ret;\n \n+\t/*\n+\t * For linked worktrees, only migrate per-worktree refs. Shared refs\n+\t * are migrated once in the main worktree.\n+\t */\n+\tif (!data->is_main_worktree) {\n+\t\tenum ref_worktree_type type = parse_worktree_ref(refname, NULL, NULL, NULL);\n+\t\tif (type != REF_WORKTREE_CURRENT)\n+\t\t\treturn 0;\n+\t}\n+\n \tif (flags & REF_ISSYMREF) {\n \t\tret = refs_read_symbolic_ref(data->old_refs, refname, &symref_target);\n \t\tif (ret < 0)\n@@ -3052,7 +3352,7 @@ static int move_files(const char *from_path, const char *to_path, struct strbuf\n \n \tfrom_dir = opendir(from_path);\n \tif (!from_dir) {\n-\t\tstrbuf_addf(errbuf, \"could not open source directory '%s': %s\",\n+\t\tstrbuf_addf(errbuf, _(\"could not open source directory '%s': %s\"),\n \t\t\t    from_path, strerror(errno));\n \t\tret = -1;\n \t\tgoto done;\n@@ -3086,14 +3386,14 @@ static int move_files(const char *from_path, const char *to_path, struct strbuf\n \n \t\tret = rename(from_buf.buf, to_buf.buf);\n \t\tif (ret < 0) {\n-\t\t\tstrbuf_addf(errbuf, \"could not link file '%s' to '%s': %s\",\n+\t\t\tstrbuf_addf(errbuf, _(\"could not link file '%s' to '%s': %s\"),\n \t\t\t\t    from_buf.buf, to_buf.buf, strerror(errno));\n \t\t\tgoto done;\n \t\t}\n \t}\n \n \tif (errno) {\n-\t\tstrbuf_addf(errbuf, \"could not read entry from directory '%s': %s\",\n+\t\tstrbuf_addf(errbuf, _(\"could not read entry from directory '%s': %s\"),\n \t\t\t    from_path, strerror(errno));\n \t\tret = -1;\n \t\tgoto done;\n@@ -3109,211 +3409,339 @@ done:\n \treturn ret;\n }\n \n-static int has_worktrees(void)\n-{\n-\tstruct worktree **worktrees = get_worktrees();\n-\tint ret = 0;\n-\tsize_t i;\n-\n-\tfor (i = 0; worktrees[i]; i++) {\n-\t\tif (is_main_worktree(worktrees[i]))\n-\t\t\tcontinue;\n-\t\tret = 1;\n-\t}\n-\n-\tfree_worktrees(worktrees);\n-\treturn ret;\n-}\n-\n int repo_migrate_ref_storage_format(struct repository *repo,\n \t\t\t\t    enum ref_storage_format format,\n \t\t\t\t    unsigned int flags,\n \t\t\t\t    struct strbuf *errbuf)\n {\n-\tstruct ref_store *old_refs = NULL, *new_refs = NULL;\n-\tstruct ref_transaction *transaction = NULL;\n-\tstruct strbuf new_gitdir = STRBUF_INIT;\n-\tstruct migration_data data = {\n-\t\t.sb = STRBUF_INIT,\n-\t\t.name = STRBUF_INIT,\n-\t\t.mail = STRBUF_INIT,\n-\t};\n-\tint did_migrate_refs = 0;\n+\tstruct worktree **worktrees = NULL;\n+\tstruct worktree_migration_data *wt_migrations = NULL;\n+\tsize_t nr_worktrees = 0;\n \tint ret;\n \n \tif (repo->ref_storage_format == format) {\n-\t\tstrbuf_addstr(errbuf, \"current and new ref storage format are equal\");\n+\t\tstrbuf_addstr(errbuf, _(\"current and new ref storage format are equal\"));\n \t\tret = -1;\n \t\tgoto done;\n \t}\n \n-\told_refs = get_main_ref_store(repo);\n+\t/*\n+\t * Enumerate all worktrees. We use the variant that doesn't try to read\n+\t * HEAD both because we don't need it (we'll migrate all refs including\n+\t * HEAD anyway) and to avoid failures if the ref storage is already\n+\t * inconsistent (e.g., from a previous interrupted migration or corruption).\n+\t */\n+\tworktrees = get_worktrees_without_reading_head();\n+\tfor (nr_worktrees = 0; worktrees[nr_worktrees]; nr_worktrees++)\n+\t\t; /* count worktrees */\n \n \t/*\n-\t * Worktrees complicate the migration because every worktree has a\n-\t * separate ref storage. While it should be feasible to implement, this\n-\t * is pushed out to a future iteration.\n-\t *\n-\t * TODO: we should really be passing the caller-provided repository to\n-\t * `has_worktrees()`, but our worktree subsystem doesn't yet support\n-\t * that.\n+\t * Migration must be run from the main worktree. When running from a\n+\t * linked worktree, the_repository context points to the worktree's\n+\t * gitdir, causing the migration logic to operate on the wrong\n+\t * directory structure.\n \t */\n-\tif (has_worktrees()) {\n-\t\tstrbuf_addstr(errbuf, \"migrating repositories with worktrees is not supported yet\");\n-\t\tret = -1;\n-\t\tgoto done;\n+\tfor (size_t i = 0; i < nr_worktrees; i++) {\n+\t\tif (worktrees[i]->is_current && !is_main_worktree(worktrees[i])) {\n+\t\t\tstrbuf_addf(errbuf, _(\"migration must be run from the main worktree at %s\"),\n+\t\t\t\t    worktrees[0]->path);\n+\t\t\tret = -1;\n+\t\t\tgoto done;\n+\t\t}\n \t}\n \n+\tCALLOC_ARRAY(wt_migrations, nr_worktrees);\n+\n \t/*\n \t * The overall logic looks like this:\n \t *\n-\t *   1. Set up a new temporary directory and initialize it with the new\n-\t *      format. This is where all refs will be migrated into.\n+\t *   1. For each worktree, set up a new temporary directory and\n+\t *      initialize it with the new format. This is where all refs for\n+\t *      that worktree will be migrated into.\n \t *\n-\t *   2. Enumerate all refs and write them into the new ref storage.\n-\t *      This operation is safe as we do not yet modify the main\n-\t *      repository.\n+\t *   2. For each worktree, enumerate all refs and write them into the\n+\t *      new ref storage. This operation is safe as we do not yet modify\n+\t *      the main repository.\n \t *\n-\t *   3. Enumerate all reflogs and write them into the new ref storage.\n-\t *      This operation is safe as we do not yet modify the main\n-\t *      repository.\n+\t *   3. For each worktree, enumerate all reflogs and write them into\n+\t *      the new ref storage. This operation is safe as we do not yet\n+\t *      modify the main repository.\n \t *\n \t *   4. If we're in dry-run mode then we are done and can hand over the\n-\t *      directory to the caller for inspection. If not, we now start\n+\t *      directories to the caller for inspection. If not, we now start\n \t *      with the destructive part.\n \t *\n-\t *   5. Delete the old ref storage from disk. As we have a copy of refs\n-\t *      in the new ref storage it's okay(ish) if we now get interrupted\n-\t *      as there is an equivalent copy of all refs available.\n+\t *   5. For each worktree, create a backup of the old ref storage by\n+\t *      moving it to a backup location.\n+\t *\n+\t *   6. For each worktree, move the new ref storage files into place.\n+\t *      As we have a backup it's okay if we now get interrupted as the\n+\t *      repository can be restored to its original state.\n \t *\n-\t *   6. Move the new ref storage files into place.\n+\t *   7. Change the repository format to the new ref format. Clear any\n+\t *      cached ref stores so they get reloaded with the new format.\n \t *\n-\t *  7. Change the repository format to the new ref format.\n+\t *   8. Delete the backup directories.\n+\t *\n+\t * All worktrees are processed sequentially. Parallelization would add\n+\t * complexity for minimal benefit since most repos have few worktrees\n+\t * and migration is a one-time operation.\n \t */\n-\tstrbuf_addf(&new_gitdir, \"%s/%s\", old_refs->gitdir, \"ref_migration.XXXXXX\");\n-\tif (!mkdtemp(new_gitdir.buf)) {\n-\t\tstrbuf_addf(errbuf, \"cannot create migration directory: %s\",\n-\t\t\t    strerror(errno));\n-\t\tret = -1;\n-\t\tgoto done;\n-\t}\n+\tfor (size_t i = 0; i < nr_worktrees; i++) {\n+\t\tstruct worktree_migration_data *wt_data = &wt_migrations[i];\n+\t\tstruct ref_transaction *transaction = NULL;\n+\t\tstruct migration_data data = {\n+\t\t\t.sb = STRBUF_INIT,\n+\t\t\t.name = STRBUF_INIT,\n+\t\t\t.mail = STRBUF_INIT,\n+\t\t};\n+\t\tint create_flags = 0;\n+\n+\t\twt_data->worktree = worktrees[i];\n+\t\twt_data->is_main = is_main_worktree(worktrees[i]);\n+\t\twt_data->old_refs = get_worktree_ref_store(worktrees[i]);\n+\t\tstrbuf_init(&wt_data->new_dir, 0);\n+\t\tstrbuf_init(&wt_data->backup_dir, 0);\n+\n+\t\t/* Create temporary directory for new ref storage */\n+\t\tstrbuf_addf(&wt_data->new_dir, \"%s/ref_migration.XXXXXX\",\n+\t\t\t    wt_data->old_refs->gitdir);\n+\n+\t\tif (!mkdtemp(wt_data->new_dir.buf)) {\n+\t\t\tstrbuf_addf(errbuf, _(\"cannot create migration directory for worktree '%s': %s\"),\n+\t\t\t\t    worktrees[i]->path, strerror(errno));\n+\t\t\tret = -1;\n+\t\t\tgoto done;\n+\t\t}\n \n-\tnew_refs = ref_store_init(repo, format, new_gitdir.buf,\n-\t\t\t\t  REF_STORE_ALL_CAPS);\n-\tret = ref_store_create_on_disk(new_refs, 0, errbuf);\n-\tif (ret < 0)\n-\t\tgoto done;\n+\t\t/*\n+\t\t * For linked worktrees migrating to files format, create a commondir\n+\t\t * file in the temp directory so the files backend knows where the\n+\t\t * common git directory is. Reftable doesn't use commondir files.\n+\t\t */\n+\t\tif (!wt_data->is_main && format == REF_STORAGE_FORMAT_FILES) {\n+\t\t\tif (create_commondir_file(wt_data->new_dir.buf,\n+\t\t\t\t\t\t  worktrees[i]->path, errbuf) < 0) {\n+\t\t\t\tret = -1;\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t}\n \n-\ttransaction = ref_store_transaction_begin(new_refs, REF_TRANSACTION_FLAG_INITIAL,\n-\t\t\t\t\t\t  errbuf);\n-\tif (!transaction)\n-\t\tgoto done;\n+\t\t/* Initialize new ref store */\n+\t\twt_data->new_refs = ref_store_init(repo, format, wt_data->new_dir.buf,\n+\t\t\t\t\t\t   REF_STORE_ALL_CAPS);\n \n-\tdata.old_refs = old_refs;\n-\tdata.transaction = transaction;\n-\tdata.errbuf = errbuf;\n+\t\t/* For linked worktrees, we only need to create the worktree-specific structure */\n+\t\tif (!wt_data->is_main)\n+\t\t\tcreate_flags = REF_STORE_CREATE_ON_DISK_IS_WORKTREE;\n \n-\t/*\n-\t * We need to use the internal `do_for_each_ref()` here so that we can\n-\t * also include broken refs and symrefs. These would otherwise be\n-\t * skipped silently.\n-\t *\n-\t * Ideally, we would do this call while locking the old ref storage\n-\t * such that there cannot be any concurrent modifications. We do not\n-\t * have the infra for that though, and the \"files\" backend does not\n-\t * allow for a central lock due to its design. It's thus on the user to\n-\t * ensure that there are no concurrent writes.\n-\t */\n-\tret = do_for_each_ref(old_refs, \"\", NULL, migrate_one_ref, 0,\n-\t\t\t      DO_FOR_EACH_INCLUDE_ROOT_REFS | DO_FOR_EACH_INCLUDE_BROKEN,\n-\t\t\t      &data);\n-\tif (ret < 0)\n-\t\tgoto done;\n+\t\tret = ref_store_create_on_disk(wt_data->new_refs, create_flags, errbuf);\n+\t\tif (ret < 0)\n+\t\t\tgoto done;\n+\n+\t\t/*\n+\t\t * Begin transaction for migrating refs. For linked worktrees,\n+\t\t * we don't use REF_TRANSACTION_FLAG_INITIAL because that flag\n+\t\t * causes refs to be written to packed-refs, which should not\n+\t\t * exist in linked worktree directories.\n+\t\t */\n+\t\ttransaction = ref_store_transaction_begin(wt_data->new_refs,\n+\t\t\t\t\t\t\t  wt_data->is_main ? REF_TRANSACTION_FLAG_INITIAL : 0,\n+\t\t\t\t\t\t\t  errbuf);\n+\t\tif (!transaction) {\n+\t\t\tret = -1;\n+\t\t\tgoto done;\n+\t\t}\n+\n+\t\tdata.old_refs = wt_data->old_refs;\n+\t\tdata.transaction = transaction;\n+\t\tdata.errbuf = errbuf;\n+\t\tdata.is_main_worktree = wt_data->is_main;\n+\n+\t\t/*\n+\t\t * We need to use the internal `do_for_each_ref()` here so that\n+\t\t * we can also include broken refs and symrefs. These would\n+\t\t * otherwise be skipped silently.\n+\t\t *\n+\t\t * Ideally, we would do this call while locking the old ref\n+\t\t * storage such that there cannot be any concurrent modifications.\n+\t\t * We do not have the infra for that though, and the \"files\"\n+\t\t * backend does not allow for a central lock due to its design.\n+\t\t * It's thus on the user to ensure that there are no concurrent\n+\t\t * writes.\n+\t\t */\n+\t\tret = do_for_each_ref(wt_data->old_refs, \"\", NULL, migrate_one_ref, 0,\n+\t\t\t\t      DO_FOR_EACH_INCLUDE_ROOT_REFS | DO_FOR_EACH_INCLUDE_BROKEN,\n+\t\t\t\t      &data);\n+\t\tif (ret < 0) {\n+\t\t\tref_transaction_free(transaction);\n+\t\t\tstrbuf_release(&data.sb);\n+\t\t\tstrbuf_release(&data.name);\n+\t\t\tstrbuf_release(&data.mail);\n+\t\t\tgoto done;\n+\t\t}\n+\n+\t\tif (!(flags & REPO_MIGRATE_REF_STORAGE_FORMAT_SKIP_REFLOG)) {\n+\t\t\tret = refs_for_each_reflog(wt_data->old_refs, migrate_one_reflog, &data);\n+\t\t\tif (ret < 0) {\n+\t\t\t\tref_transaction_free(transaction);\n+\t\t\t\tstrbuf_release(&data.sb);\n+\t\t\t\tstrbuf_release(&data.name);\n+\t\t\t\tstrbuf_release(&data.mail);\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t}\n+\n+\t\tret = ref_transaction_commit(transaction, errbuf);\n+\t\tref_transaction_free(transaction);\n+\t\tstrbuf_release(&data.sb);\n+\t\tstrbuf_release(&data.name);\n+\t\tstrbuf_release(&data.mail);\n \n-\tif (!(flags & REPO_MIGRATE_REF_STORAGE_FORMAT_SKIP_REFLOG)) {\n-\t\tret = refs_for_each_reflog(old_refs, migrate_one_reflog, &data);\n \t\tif (ret < 0)\n \t\t\tgoto done;\n-\t}\n \n-\tret = ref_transaction_commit(transaction, errbuf);\n-\tif (ret < 0)\n-\t\tgoto done;\n-\tdid_migrate_refs = 1;\n+\t\t/*\n+\t\t * Linked worktrees should not have a packed-refs file. If one\n+\t\t * was created during the transaction, remove it before moving\n+\t\t * files into place.\n+\t\t */\n+\t\tif (!wt_data->is_main) {\n+\t\t\tstruct strbuf packed_refs = STRBUF_INIT;\n+\t\t\tstrbuf_addf(&packed_refs, \"%s/packed-refs\", wt_data->new_dir.buf);\n+\t\t\tif (unlink(packed_refs.buf) < 0 && errno != ENOENT)\n+\t\t\t\twarning_errno(_(\"could not remove packed-refs from linked worktree at '%s'\"),\n+\t\t\t\t\t      packed_refs.buf);\n+\t\t\tstrbuf_release(&packed_refs);\n+\t\t}\n+\n+\t\t/*\n+\t\t * Release the new ref store to close any open files. This is\n+\t\t * required for platforms like Cygwin where renaming an open\n+\t\t * file results in EPERM.\n+\t\t */\n+\t\tref_store_release(wt_data->new_refs);\n+\t\tFREE_AND_NULL(wt_data->new_refs);\n+\t}\n \n \tif (flags & REPO_MIGRATE_REF_STORAGE_FORMAT_DRYRUN) {\n-\t\tprintf(_(\"Finished dry-run migration of refs, \"\n-\t\t\t \"the result can be found at '%s'\\n\"), new_gitdir.buf);\n+\t\tprintf(_(\"Finished dry-run migration of refs for %\"PRIuMAX\" worktree(s)\\n\"),\n+\t\t       (uintmax_t)nr_worktrees);\n+\t\tfor (size_t i = 0; i < nr_worktrees; i++) {\n+\t\t\tconst char *path = wt_migrations[i].new_dir.buf;\n+\n+\t\t\t/* Show absolute paths consistently for both main and linked worktrees */\n+\t\t\tif (!is_absolute_path(path))\n+\t\t\t\tpath = absolute_path(path);\n+\n+\t\t\tprintf(_(\"  Worktree '%s': %s\\n\"),\n+\t\t\t       worktrees[i]->path,\n+\t\t\t       path);\n+\t\t}\n \t\tret = 0;\n \t\tgoto done;\n \t}\n \n-\t/*\n-\t * Release the new ref store such that any potentially-open files will\n-\t * be closed. This is required for platforms like Cygwin, where\n-\t * renaming an open file results in EPERM.\n-\t */\n-\tref_store_release(new_refs);\n-\tFREE_AND_NULL(new_refs);\n+\tfor (size_t i = 0; i < nr_worktrees; i++) {\n+\t\tstruct worktree_migration_data *wt_data = &wt_migrations[i];\n \n-\t/*\n-\t * Until now we were in the non-destructive phase, where we only\n-\t * populated the new ref store. From hereon though we are about\n-\t * to get hands by deleting the old ref store and then moving\n-\t * the new one into place.\n-\t *\n-\t * Assuming that there were no concurrent writes, the new ref\n-\t * store should have all information. So if we fail from hereon\n-\t * we may be in an in-between state, but it would still be able\n-\t * to recover by manually moving remaining files from the\n-\t * temporary migration directory into place.\n-\t */\n-\tret = ref_store_remove_on_disk(old_refs, errbuf);\n-\tif (ret < 0)\n-\t\tgoto done;\n+\t\t/* Create backup directory */\n+\t\tstrbuf_addf(&wt_data->backup_dir, \"%s/ref_migration_backup.XXXXXX\",\n+\t\t\t    wt_data->old_refs->gitdir);\n+\t\tif (!mkdtemp(wt_data->backup_dir.buf)) {\n+\t\t\tstrbuf_addf(errbuf, _(\"cannot create backup directory for worktree '%s': %s\"),\n+\t\t\t\t    worktrees[i]->path, strerror(errno));\n+\t\t\tret = -1;\n+\t\t\tgoto done;\n+\t\t}\n \n-\tret = move_files(new_gitdir.buf, old_refs->gitdir, errbuf);\n-\tif (ret < 0)\n-\t\tgoto done;\n+\t\t/* Backup old ref storage by moving it to backup directory */\n+\t\tret = backup_ref_storage(wt_data->old_refs->gitdir,\n+\t\t\t\t\t wt_data->backup_dir.buf,\n+\t\t\t\t\t wt_data->is_main, errbuf);\n+\t\tif (ret < 0) {\n+\t\t\tstrbuf_addf(errbuf, _(\" (worktree: %s)\"), worktrees[i]->path);\n+\t\t\tgoto done;\n+\t\t}\n \n-\tif (rmdir(new_gitdir.buf) < 0)\n-\t\twarning_errno(_(\"could not remove temporary migration directory '%s'\"),\n-\t\t\t      new_gitdir.buf);\n+\t\t/* Move new ref storage into place */\n+\t\tret = move_files(wt_data->new_dir.buf, wt_data->old_refs->gitdir, errbuf);\n+\t\tif (ret < 0) {\n+\t\t\tstrbuf_addf(errbuf, _(\" (worktree: %s)\"), worktrees[i]->path);\n+\t\t\tgoto done;\n+\t\t}\n+\n+\t\t/* Remove temporary migration directory */\n+\t\tif (rmdir(wt_data->new_dir.buf) < 0)\n+\t\t\twarning_errno(_(\"could not remove temporary migration directory '%s'\"),\n+\t\t\t\t      wt_data->new_dir.buf);\n+\t}\n \n \t/*\n-\t * We have migrated the repository, so we now need to adjust the\n-\t * repository format so that clients will use the new ref store.\n-\t * We also need to swap out the repository's main ref store.\n+\t * Update the repository format so that clients will use the new ref\n+\t * store.\n \t */\n \tinitialize_repository_version(hash_algo_by_ptr(repo->hash_algo), format, 1);\n \n \t/*\n-\t * Unset the old ref store and release it. `get_main_ref_store()` will\n-\t * make sure to lazily re-initialize the repository's ref store with\n-\t * the new format.\n+\t * Reinitialize all worktree ref stores with the new format. We release\n+\t * the old ones and clear cached pointers so they get lazily\n+\t * reinitialized with the new format.\n \t */\n-\tref_store_release(old_refs);\n-\tFREE_AND_NULL(old_refs);\n-\trepo->refs_private = NULL;\n+\tfor (size_t i = 0; i < nr_worktrees; i++) {\n+\t\tif (wt_migrations[i].is_main) {\n+\t\t\t/* Main worktree: clear the cached main ref store */\n+\t\t\tif (repo->refs_private) {\n+\t\t\t\tref_store_release(repo->refs_private);\n+\t\t\t\tFREE_AND_NULL(repo->refs_private);\n+\t\t\t}\n+\t\t}\n+\t\t/* Worktree ref stores will be lazily reinitialized on next access */\n+\t}\n+\n+\t/* Delete backup directories since migration succeeded */\n+\tfor (size_t i = 0; i < nr_worktrees; i++) {\n+\t\tif (wt_migrations[i].backup_dir.len)\n+\t\t\tdelete_backup(wt_migrations[i].backup_dir.buf);\n+\t}\n \n \tret = 0;\n \n done:\n-\tif (ret && did_migrate_refs) {\n-\t\tstrbuf_complete(errbuf, '\\n');\n-\t\tstrbuf_addf(errbuf, _(\"migrated refs can be found at '%s'\"),\n-\t\t\t    new_gitdir.buf);\n-\t}\n+\tif (ret) {\n+\t\t/*\n+\t\t * Migration failed. Attempt to restore from backups if we made\n+\t\t * it to Phase 2.\n+\t\t */\n+\t\tfor (size_t i = 0; wt_migrations && i < nr_worktrees; i++) {\n+\t\t\tif (wt_migrations[i].backup_dir.len) {\n+\t\t\t\trestore_ref_storage_from_backup(\n+\t\t\t\t\twt_migrations[i].old_refs->gitdir,\n+\t\t\t\t\twt_migrations[i].backup_dir.buf,\n+\t\t\t\t\twt_migrations[i].is_main);\n+\t\t\t\tdelete_backup(wt_migrations[i].backup_dir.buf);\n+\t\t\t}\n+\t\t}\n \n-\tif (new_refs) {\n-\t\tref_store_release(new_refs);\n-\t\tfree(new_refs);\n+\t\t/*\n+\t\t * Report where migrated refs can be found for manual recovery.\n+\t\t * We keep these directories as insurance - if the restore failed,\n+\t\t * they may be the only way to recover the migrated refs.\n+\t\t */\n+\t\tfor (size_t i = 0; wt_migrations && i < nr_worktrees; i++) {\n+\t\t\tif (wt_migrations[i].new_dir.len) {\n+\t\t\t\tstrbuf_complete(errbuf, '\\n');\n+\t\t\t\tstrbuf_addf(errbuf, _(\"migrated refs for worktree '%s' can be found at '%s'\"),\n+\t\t\t\t\t    worktrees[i]->path, wt_migrations[i].new_dir.buf);\n+\t\t\t}\n+\t\t}\n \t}\n-\tref_transaction_free(transaction);\n-\tstrbuf_release(&new_gitdir);\n-\tstrbuf_release(&data.sb);\n-\tstrbuf_release(&data.name);\n-\tstrbuf_release(&data.mail);\n+\n+\tif (wt_migrations)\n+\t\tworktree_migration_data_array_release(wt_migrations, nr_worktrees);\n+\tif (worktrees)\n+\t\tfree_worktrees(worktrees);\n+\n \treturn ret;\n }\n \ndiff --git a/refs/files-backend.c b/refs/files-backend.c\nindex 8d7007f4aa..6be56274d3 100644\n--- a/refs/files-backend.c\n+++ b/refs/files-backend.c\n@@ -3210,11 +3210,13 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n \t\tstring_list_append(&refnames_to_check, update->refname);\n \n \t\t/*\n-\t\t * packed-refs don't support symbolic refs, root refs and reflogs,\n-\t\t * so we have to queue these references via the loose transaction.\n+\t\t * packed-refs don't support symbolic refs, root refs, per-worktree\n+\t\t * refs, and reflogs, so we have to queue these references via the\n+\t\t * loose transaction.\n \t\t */\n \t\tif (update->new_target ||\n \t\t    is_root_ref(update->refname) ||\n+\t\t    is_per_worktree_ref(update->refname) ||\n \t\t    (update->flags & REF_LOG_ONLY)) {\n \t\t\tif (!loose_transaction) {\n \t\t\t\tloose_transaction = ref_store_transaction_begin(&refs->base, 0, err);\ndiff --git a/t/t1460-refs-migrate.sh b/t/t1460-refs-migrate.sh\nindex 0e1116a319..2ea8d31361 100755\n--- a/t/t1460-refs-migrate.sh\n+++ b/t/t1460-refs-migrate.sh\n@@ -114,16 +114,40 @@ do\n \t\t\ttest_cmp expect err\n \t\t'\n \n-\t\ttest_expect_success \"$from_format -> $to_format: migration with worktree fails\" '\n+\t\ttest_expect_success \"$from_format -> $to_format: migration with worktree\" '\n \t\t\ttest_when_finished \"rm -rf repo\" &&\n \t\t\tgit init --ref-format=$from_format repo &&\n+\t\t\ttest_commit -C repo initial &&\n \t\t\tgit -C repo worktree add wt &&\n-\t\t\ttest_must_fail git -C repo refs migrate \\\n-\t\t\t\t--ref-format=$to_format 2>err &&\n-\t\t\tcat >expect <<-EOF &&\n-\t\t\terror: migrating repositories with worktrees is not supported yet\n-\t\t\tEOF\n-\t\t\ttest_cmp expect err\n+\n+\t\t\t# Create some refs and reflogs in both worktrees\n+\t\t\ttest_commit -C repo second &&\n+\t\t\tgit -C repo update-ref refs/heads/from-main HEAD &&\n+\t\t\tgit -C repo/wt checkout -b wt-branch &&\n+\t\t\ttest_commit -C repo/wt wt-commit &&\n+\t\t\tgit -C repo/wt update-ref refs/bisect/wt-ref HEAD &&\n+\n+\t\t\t# Capture refs from both worktrees before migration\n+\t\t\tgit -C repo for-each-ref --include-root-refs \\\n+\t\t\t\t--format=\"%(refname) %(objectname) %(symref)\" >expect-main &&\n+\t\t\tgit -C repo/wt for-each-ref --include-root-refs \\\n+\t\t\t\t--format=\"%(refname) %(objectname) %(symref)\" >expect-wt &&\n+\n+\t\t\t# Perform migration\n+\t\t\tgit -C repo refs migrate --ref-format=$to_format &&\n+\n+\t\t\t# Verify refs in both worktrees after migration\n+\t\t\tgit -C repo for-each-ref --include-root-refs \\\n+\t\t\t\t--format=\"%(refname) %(objectname) %(symref)\" >actual-main &&\n+\t\t\tgit -C repo/wt for-each-ref --include-root-refs \\\n+\t\t\t\t--format=\"%(refname) %(objectname) %(symref)\" >actual-wt &&\n+\t\t\ttest_cmp expect-main actual-main &&\n+\t\t\ttest_cmp expect-wt actual-wt &&\n+\n+\t\t\t# Verify repository format changed\n+\t\t\tgit -C repo rev-parse --show-ref-format >actual &&\n+\t\t\techo \"$to_format\" >expect &&\n+\t\t\ttest_cmp expect actual\n \t\t'\n \n \t\ttest_expect_success \"$from_format -> $to_format: unborn HEAD\" '\n@@ -325,4 +349,412 @@ test_expect_success 'migrating from reftable format deletes backend files' '\n \ttest_path_is_file repo/.git/packed-refs\n '\n \n+test_expect_success 'files -> reftable: migration with multiple worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init --ref-format=files repo &&\n+\ttest_commit -C repo initial &&\n+\tgit -C repo worktree add wt1 &&\n+\tgit -C repo worktree add wt2 &&\n+\n+\t# Create unique refs in each worktree\n+\ttest_commit -C repo main-commit &&\n+\ttest_commit -C repo/wt1 wt1-commit &&\n+\ttest_commit -C repo/wt2 wt2-commit &&\n+\tgit -C repo update-ref refs/bisect/main-bisect HEAD &&\n+\tgit -C repo/wt1 update-ref refs/bisect/wt1-bisect HEAD &&\n+\tgit -C repo/wt2 update-ref refs/bisect/wt2-bisect HEAD &&\n+\n+\t# Capture state before migration\n+\tgit -C repo for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >expect-main &&\n+\tgit -C repo/wt1 for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >expect-wt1 &&\n+\tgit -C repo/wt2 for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >expect-wt2 &&\n+\n+\t# Migrate\n+\tgit -C repo refs migrate --ref-format=reftable &&\n+\n+\t# Verify all worktrees still work\n+\tgit -C repo for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >actual-main &&\n+\tgit -C repo/wt1 for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >actual-wt1 &&\n+\tgit -C repo/wt2 for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >actual-wt2 &&\n+\ttest_cmp expect-main actual-main &&\n+\ttest_cmp expect-wt1 actual-wt1 &&\n+\ttest_cmp expect-wt2 actual-wt2 &&\n+\n+\t# Verify format changed\n+\tgit -C repo rev-parse --show-ref-format >actual &&\n+\techo \"reftable\" >expect &&\n+\ttest_cmp expect actual &&\n+\n+\t# Verify operations still work in all worktrees\n+\ttest_commit -C repo post-migrate-main &&\n+\ttest_commit -C repo/wt1 post-migrate-wt1 &&\n+\ttest_commit -C repo/wt2 post-migrate-wt2\n+'\n+\n+test_expect_success 'files -> reftable: dry-run with worktrees' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init --ref-format=files repo &&\n+\ttest_commit -C repo initial &&\n+\tgit -C repo worktree add wt &&\n+\n+\tgit -C repo refs migrate --ref-format=reftable --dry-run >output &&\n+\tgrep \"Finished dry-run migration\" output &&\n+\tgrep \"2 worktree\" output &&\n+\n+\t# Format should not have changed\n+\tgit -C repo rev-parse --show-ref-format >actual &&\n+\techo \"files\" >expect &&\n+\ttest_cmp expect actual &&\n+\n+\t# Files backend should still be present\n+\ttest_path_is_file repo/.git/refs/heads/main\n+'\n+\n+test_expect_success 'reftable -> files: migration with worktrees and per-worktree refs' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init --ref-format=reftable repo &&\n+\ttest_commit -C repo initial &&\n+\tgit -C repo worktree add wt &&\n+\n+\t# Create various types of per-worktree refs\n+\ttest_commit -C repo main-work &&\n+\tgit -C repo update-ref refs/bisect/bad HEAD &&\n+\tgit -C repo update-ref refs/rewritten/main HEAD &&\n+\tgit -C repo update-ref refs/worktree/custom HEAD &&\n+\n+\ttest_commit -C repo/wt wt-work &&\n+\tgit -C repo/wt update-ref refs/bisect/good HEAD &&\n+\tgit -C repo/wt update-ref refs/rewritten/wt HEAD &&\n+\tgit -C repo/wt update-ref refs/worktree/wt-custom HEAD &&\n+\n+\t# Capture all refs including per-worktree ones\n+\tgit -C repo for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >expect-main &&\n+\tgit -C repo/wt for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >expect-wt &&\n+\n+\t# Migrate back to files\n+\tgit -C repo refs migrate --ref-format=files &&\n+\n+\t# Verify per-worktree refs are still separate\n+\tgit -C repo for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >actual-main &&\n+\tgit -C repo/wt for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >actual-wt &&\n+\ttest_cmp expect-main actual-main &&\n+\ttest_cmp expect-wt actual-wt &&\n+\n+\t# Verify physical separation of per-worktree refs\n+\ttest_path_is_file repo/.git/refs/bisect/bad &&\n+\ttest_path_is_file repo/.git/worktrees/wt/refs/bisect/good &&\n+\ttest_path_is_missing repo/.git/refs/bisect/good &&\n+\ttest_path_is_missing repo/.git/worktrees/wt/refs/bisect/bad\n+'\n+\n+test_expect_success 'bare repository with worktrees: bidirectional migration' '\n+\ttest_when_finished \"rm -rf bare-repo worktrees\" &&\n+\n+\t# Create a bare repository\n+\tgit init --bare --ref-format=files bare-repo &&\n+\n+\t# Add worktrees to the bare repository\n+\tmkdir worktrees &&\n+\tgit -C bare-repo worktree add ../worktrees/main &&\n+\tgit -C bare-repo worktree add ../worktrees/feature &&\n+\n+\t# Create initial commits and refs in main worktree\n+\ttest_commit -C worktrees/main initial &&\n+\tgit -C worktrees/main update-ref refs/heads/main HEAD &&\n+\tgit -C worktrees/main update-ref refs/bisect/main-bad HEAD &&\n+\tgit -C worktrees/main update-ref refs/worktree/main-custom HEAD &&\n+\n+\t# Create commits and refs in feature worktree\n+\ttest_commit -C worktrees/feature feature-work &&\n+\tgit -C worktrees/feature update-ref refs/bisect/feature-bad HEAD &&\n+\tgit -C worktrees/feature update-ref refs/worktree/feature-custom HEAD &&\n+\n+\t# Capture all refs before migration\n+\tgit -C worktrees/main for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >expect-main &&\n+\tgit -C worktrees/feature for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >expect-feature &&\n+\n+\t# Migrate bare repo to reftable\n+\tgit -C bare-repo refs migrate --ref-format=reftable &&\n+\n+\t# Verify format changed\n+\tgit -C bare-repo rev-parse --show-ref-format >actual &&\n+\techo \"reftable\" >expect-format &&\n+\ttest_cmp expect-format actual &&\n+\n+\t# Verify all refs still exist and are correct\n+\tgit -C worktrees/main for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >actual-main &&\n+\tgit -C worktrees/feature for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >actual-feature &&\n+\ttest_cmp expect-main actual-main &&\n+\ttest_cmp expect-feature actual-feature &&\n+\n+\t# Migrate back to files\n+\tgit -C bare-repo refs migrate --ref-format=files &&\n+\n+\t# Verify format changed back\n+\tgit -C bare-repo rev-parse --show-ref-format >actual &&\n+\techo \"files\" >expect-format &&\n+\ttest_cmp expect-format actual &&\n+\n+\t# Verify all refs still exist and are correct after round-trip\n+\tgit -C worktrees/main for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >actual-main &&\n+\tgit -C worktrees/feature for-each-ref --include-root-refs \\\n+\t\t--format=\"%(refname) %(objectname)\" | sort >actual-feature &&\n+\ttest_cmp expect-main actual-main &&\n+\ttest_cmp expect-feature actual-feature &&\n+\n+\t# Verify physical separation of per-worktree refs\n+\ttest_path_is_file bare-repo/worktrees/main/refs/bisect/main-bad &&\n+\ttest_path_is_file bare-repo/worktrees/feature/refs/bisect/feature-bad &&\n+\ttest_path_is_missing bare-repo/worktrees/main/refs/bisect/feature-bad &&\n+\ttest_path_is_missing bare-repo/worktrees/feature/refs/bisect/main-bad\n+'\n+\n+test_expect_success SANITY 'files -> reftable: migration fails with read-only .git' '\n+\ttest_when_finished \"chmod -R u+w read-only-git\" &&\n+\tgit init --ref-format=files read-only-git &&\n+\ttest_commit -C read-only-git initial &&\n+\tchmod -R a-w read-only-git/.git &&\n+\ttest_must_fail git -C read-only-git refs migrate --ref-format=reftable 2>err &&\n+\tgrep -i \"permission denied\\|read-only\" err\n+'\n+\n+test_expect_success SANITY 'files -> reftable: read-only refs directory prevents backup' '\n+\ttest_when_finished \"chmod -R u+w read-only-refs\" &&\n+\tgit init --ref-format=files read-only-refs &&\n+\ttest_commit -C read-only-refs initial &&\n+\tchmod a-w read-only-refs/.git/refs &&\n+\ttest_must_fail git -C read-only-refs refs migrate --ref-format=reftable 2>err &&\n+\tchmod u+w read-only-refs/.git/refs &&\n+\tgrep -i \"could not\\|permission denied\" err\n+'\n+\n+test_expect_success 'files -> reftable: git status works in all worktrees after migration' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init --ref-format=files repo &&\n+\ttest_commit -C repo initial &&\n+\tgit -C repo worktree add wt1 &&\n+\tgit -C repo worktree add wt2 &&\n+\n+\t# Make some commits in each worktree\n+\ttest_commit -C repo main-work &&\n+\ttest_commit -C repo/wt1 wt1-work &&\n+\ttest_commit -C repo/wt2 wt2-work &&\n+\n+\t# Verify status works before migration using -C\n+\tgit -C repo status &&\n+\tgit -C repo/wt1 status &&\n+\tgit -C repo/wt2 status &&\n+\n+\t# Verify status works before migration by cd-ing into worktree\n+\t(cd repo && git status) &&\n+\t(cd repo/wt1 && git status) &&\n+\t(cd repo/wt2 && git status) &&\n+\n+\t# Migrate to reftable\n+\tgit -C repo refs migrate --ref-format=reftable &&\n+\n+\t# Verify status still works after migration using -C\n+\tgit -C repo status &&\n+\tgit -C repo/wt1 status &&\n+\tgit -C repo/wt2 status &&\n+\n+\t# Verify status works after migration by cd-ing into worktree\n+\t(cd repo && git status) &&\n+\t(cd repo/wt1 && git status) &&\n+\t(cd repo/wt2 && git status) &&\n+\n+\t# Verify other common commands work in all worktrees\n+\tgit -C repo log --oneline &&\n+\tgit -C repo/wt1 log --oneline &&\n+\tgit -C repo/wt2 log --oneline &&\n+\n+\tgit -C repo branch &&\n+\tgit -C repo/wt1 branch &&\n+\tgit -C repo/wt2 branch &&\n+\n+\t# Migrate back to files\n+\tgit -C repo refs migrate --ref-format=files &&\n+\n+\t# Verify status still works after migrating back\n+\tgit -C repo status &&\n+\tgit -C repo/wt1 status &&\n+\tgit -C repo/wt2 status &&\n+\n+\t(cd repo && git status) &&\n+\t(cd repo/wt1 && git status) &&\n+\t(cd repo/wt2 && git status)\n+'\n+\n+test_expect_success 'files -> reftable: migration fails from inside linked worktree' '\n+\ttest_when_finished \"rm -rf from-wt-bare.git from-wt-trees\" &&\n+\n+\t# Create a bare repo with worktrees\n+\tgit init --bare --ref-format=files from-wt-bare.git &&\n+\n+\t# Add two worktrees\n+\tmkdir from-wt-trees &&\n+\tgit -C from-wt-bare.git worktree add ../from-wt-trees/wt1 &&\n+\tgit -C from-wt-bare.git worktree add ../from-wt-trees/wt2 &&\n+\n+\t# Create commits in first worktree\n+\ttest_commit -C from-wt-trees/wt1 initial &&\n+\ttest_commit -C from-wt-trees/wt1 second &&\n+\n+\t# Migration from inside a linked worktree should fail with helpful error\n+\t(\n+\t\tcd from-wt-trees/wt1 &&\n+\t\ttest_must_fail git refs migrate --ref-format=reftable 2>err\n+\t) &&\n+\tgrep \"migration must be run from the main worktree\" from-wt-trees/wt1/err &&\n+\n+\t# Verify repository is not corrupted - refs format should still be files\n+\tgit -C from-wt-bare.git rev-parse --show-ref-format >actual-format &&\n+\techo \"files\" >expect-format &&\n+\ttest_cmp expect-format actual-format &&\n+\n+\t# Verify git status still works in the worktree\n+\tgit -C from-wt-trees/wt1 status &&\n+\t(cd from-wt-trees/wt1 && git status) &&\n+\n+\t# Verify migration succeeds when run from the main repository\n+\tgit -C from-wt-bare.git refs migrate --ref-format=reftable &&\n+\n+\t# Verify migration actually happened\n+\tgit -C from-wt-bare.git rev-parse --show-ref-format >actual-format-after &&\n+\techo \"reftable\" >expect-format-after &&\n+\ttest_cmp expect-format-after actual-format-after &&\n+\n+\t# Verify worktree still works after successful migration\n+\tgit -C from-wt-trees/wt1 status &&\n+\t(cd from-wt-trees/wt1 && git status)\n+'\n+\n+test_expect_success 'files -> reftable: migration with uncommitted changes in worktrees' '\n+\ttest_when_finished \"rm -rf dirty-wt-repo dirty-wt\" &&\n+\n+\t# Create repo with initial commit\n+\tgit init --ref-format=files dirty-wt-repo &&\n+\ttest_commit -C dirty-wt-repo initial &&\n+\n+\t# Create worktree and make a commit there so it has tracked files\n+\tgit -C dirty-wt-repo worktree add ../dirty-wt &&\n+\ttest_commit -C dirty-wt base &&\n+\n+\t# Create uncommitted changes in worktree:\n+\t# 1. Untracked file\n+\techo \"untracked content\" >dirty-wt/untracked.txt &&\n+\n+\t# 2. Modified tracked file (not staged)\n+\techo \"modified\" >>dirty-wt/base.t &&\n+\n+\t# 3. Staged new file\n+\techo \"staged new content\" >dirty-wt/staged-new.txt &&\n+\tgit -C dirty-wt add staged-new.txt &&\n+\n+\t# 4. Staged modification to tracked file\n+\techo \"staged modification\" >>dirty-wt/initial.t &&\n+\tgit -C dirty-wt add initial.t &&\n+\n+\t# 5. File with both staged AND unstaged changes\n+\techo \"staged change\" >dirty-wt/both.txt &&\n+\tgit -C dirty-wt add both.txt &&\n+\techo \"unstaged change\" >>dirty-wt/both.txt &&\n+\n+\t# Record status before migration\n+\tgit -C dirty-wt status --porcelain >status-before &&\n+\n+\t# Verify status works before migration\n+\tgit -C dirty-wt status &&\n+\t(cd dirty-wt && git status) &&\n+\n+\t# Migrate from main worktree\n+\tgit -C dirty-wt-repo refs migrate --ref-format=reftable &&\n+\n+\t# Verify migration succeeded\n+\tgit -C dirty-wt-repo rev-parse --show-ref-format >actual &&\n+\techo \"reftable\" >expect &&\n+\ttest_cmp expect actual &&\n+\n+\t# Verify status still works after migration\n+\tgit -C dirty-wt status &&\n+\t(cd dirty-wt && git status) &&\n+\n+\t# Verify all uncommitted changes are preserved exactly\n+\tgit -C dirty-wt status --porcelain >status-after &&\n+\ttest_cmp status-before status-after &&\n+\n+\t# Verify file contents are preserved\n+\ttest \"$(cat dirty-wt/untracked.txt)\" = \"untracked content\" &&\n+\tgrep \"modified\" dirty-wt/base.t &&\n+\ttest \"$(cat dirty-wt/staged-new.txt)\" = \"staged new content\" &&\n+\tgrep \"staged modification\" dirty-wt/initial.t &&\n+\ttest \"$(cat dirty-wt/both.txt)\" = \"staged change\n+unstaged change\" &&\n+\n+\t# Verify all 5 types of changes are still present in status\n+\ttest_line_count = 5 status-after\n+'\n+\n+test_expect_success 'files -> reftable: migration with prunable worktree' '\n+\ttest_when_finished \"rm -rf prunable-repo\" &&\n+\n+\t# Create repo with worktree, then delete the worktree directory\n+\tgit init --ref-format=files prunable-repo &&\n+\ttest_commit -C prunable-repo initial &&\n+\tgit -C prunable-repo worktree add ../prunable-wt &&\n+\trm -rf ../prunable-wt &&\n+\n+\t# Migration should still succeed\n+\tgit -C prunable-repo refs migrate --ref-format=reftable &&\n+\n+\t# Verify migration succeeded\n+\tgit -C prunable-repo rev-parse --show-ref-format >actual &&\n+\techo \"reftable\" >expect &&\n+\ttest_cmp expect actual &&\n+\n+\t# Verify worktree is marked as prunable but metadata exists\n+\tgit -C prunable-repo worktree list --porcelain >list &&\n+\tgrep \"prunable\" list\n+'\n+\n+test_expect_success 'files -> reftable: migration works from main worktree .git directory' '\n+\ttest_when_finished \"rm -rf from-gitdir-repo\" &&\n+\n+\tgit init --ref-format=files from-gitdir-repo &&\n+\ttest_commit -C from-gitdir-repo initial &&\n+\n+\t# Verify status works before migration\n+\t(cd from-gitdir-repo && git status) &&\n+\n+\t# Run migration from inside .git directory\n+\t(\n+\t\tcd from-gitdir-repo/.git &&\n+\t\tgit refs migrate --ref-format=reftable\n+\t) &&\n+\n+\t# Verify migration succeeded\n+\tgit -C from-gitdir-repo rev-parse --show-ref-format >actual &&\n+\techo \"reftable\" >expect &&\n+\ttest_cmp expect actual &&\n+\n+\t# Verify repo still works\n+\tgit -C from-gitdir-repo status &&\n+\t(cd from-gitdir-repo && git status)\n+'\n+\n test_done\n\nbase-commit: 419c72cb8ada252b260efc38ff91fe201de7c8c3\n-- \ngitgitgadget\n"},{"id":"529763","messageId":"aQBwiE-bhqcaSHG_@pks.im","threadId":"64395","inReplyTo":"pull.2077.git.git.1761589580028.gitgitgadget@gmail.com","subject":"Re: [PATCH] refs: support migration with worktrees","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-10-28T07:28:08Z","receivedAt":"2025-10-28T07:28:22Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 27, 2025 at 06:26:20PM +0000, Sam Bostock via GitGitGadget wrote:\n> From: Sam Bostock <sam.bostock@shopify.com>\n> \n> Remove the worktree limitation from `git refs migrate` by implementing\n> migration support for repositories with linked worktrees.\n> \n> Previously, attempting to migrate a repository with worktrees would fail\n> with \"migrating repositories with worktrees is not supported yet\". This\n> limitation existed because each worktree has its own ref storage that\n> needed to be migrated separately.\n> \n> Migration now uses a multi-phase approach to safely handle multiple\n> worktrees:\n> \n> 1. Phase 1: Iterate through all worktrees and create temporary new ref\n>    storage for each in a staging directory.\n> \n> 2. Phase 2: For each worktree, backup the existing ref storage, then\n>    move the new storage into place.\n> \n> 3. Phase 3: Update the repository format config, clear cached ref stores,\n>    and delete all backups. On failure, restore from backups and report\n>    where the migrated refs can be found for manual recovery.\n\nMakes sense.\n\n> This approach ensures that if migration fails partway through, the\n> repository can be restored to its original state.\n> \n> Key implementation details:\n> \n> - For files backend: Create a commondir file in temp directories for\n>   linked worktrees so the files backend knows where the common git\n>   directory is located.\n\nHm, okay. Not yet sure why we need this, but let's read on.\n\n> - For linked worktrees: Use non-INITIAL transactions to avoid creating\n>   packed-refs files (linked worktrees should never have packed-refs).\n\nHm, this is unfortunate. The reason why we use initial transactions is\ntwofold:\n\n  - First, we want to avoid creating loose refs, only. This is indeed\n    something we must not do with worktrees, as you point out.\n\n  - But second, we also want to skip pointless checks like the conflict\n    checks. This results in quite a saving.\n\nWould've been great to retain the second property, but I guess as long\nas we only do this for worktrees it's okayish and something we can worry\nabout at a later point in time. Better to migrate the refs slowish than\nnot at all.\n\n> - Filter refs during iteration: Linked worktrees only migrate their\n>   per-worktree refs (refs/bisect/*, refs/rewritten/*, refs/worktree/*).\n>   Shared refs are migrated once in the main worktree.\n\nMakes sense.\n\n> - Write per-worktree refs as loose files: The files backend's\n>   transaction_finish_initial() optimization writes most refs to\n>   packed-refs, but per-worktree refs must be stored as loose files\n>   to maintain proper worktree isolation.\n\nIsn't this roughly the same as the second bullet point? Feels like they\nshould be merged together.\n\n> diff --git a/Documentation/git-refs.adoc b/Documentation/git-refs.adoc\n> index fa33680cc7..f6a3bf4f03 100644\n> --- a/Documentation/git-refs.adoc\n> +++ b/Documentation/git-refs.adoc\n> @@ -30,7 +30,8 @@ COMMANDS\n>  --------\n>  \n>  `migrate`::\n> -\tMigrate ref store between different formats.\n> +\tMigrate ref store between different formats. Supports repositories\n> +\twith worktrees; migration must be run from the main worktree.\n\nIt feels a bit weird to single our worktrees specifically. We don't say\nthat the tool supports bare and non-bare repositories, either, so the\nonly reason why we'd have the note about worktrees is historic legacy.\nHow about this instead:\n\n    Migrate ref storage between different formats. Must be run from the\n    main worktree in case the repository uses worktrees.\n\n> @@ -95,7 +96,7 @@ KNOWN LIMITATIONS\n>  \n>  The ref format migration has several known limitations in its current form:\n>  \n> -* It is not possible to migrate repositories that have worktrees.\n> +* Migration must be run from the main worktree.\n>  \n\nI'd drop this bullet point entirely, as I don't really see this as a\nlimitation anymore.\n\n> diff --git a/refs.c b/refs.c\n> index 965381367e..0834329e5a 100644\n> --- a/refs.c\n> +++ b/refs.c\n\nI'm sorry, but this is _extremely_ hard to review as we're changing\nalmost all of the implementation at once with random changes left and\nright. Furthermore, I feel like we're getting way to intimate with the\ndifferent backends here -- that shouldn't be the case though, the logic\nthat is specific to the backends should really live in the backends\nthemselves.\n\nThe way I'd expect a series like this to look like is to have commits\nthat:\n\n  1. Do preparatory changes, e.g. teach the files backend to not create\n     a packed-refs file for worktrees during migration.\n\n  2. Pull out the logic to migrate a single reference backend that we\n     already have into a separate function that can be called in a loop.\n     The end result should be a function that accepts the old refdb as\n     input and that returns the new refdb.\n\n  3. Implement the logic that calls the function we introduced in (2)\n     for each worktree. This can be done by iterating through all the\n     worktrees, calling `get_worktree_ref_store()` on it and then\n     passing the refdb to the new function.\n\n> @@ -2974,8 +2976,296 @@ struct migration_data {\n[snip]\n> +/*\n> + * Create a commondir file in the temporary migration directory for a linked\n> + * worktree. The files backend needs this to locate the common git directory.\n> + * Returns 0 on success, -1 on failure.\n> + */\n> +static int create_commondir_file(const char *new_dir, const char *worktree_path,\n> +\t\t\t\t  struct strbuf *errbuf)\n> +{\n\nI still don't get why we need this. We should have access to both the\nref store of the worktree and the repository, and both of these are\nhandled in the same process. So there shouldn't be a need to propagate\nthe commondir via a file.\n\n[snip]\n> +/*\n> + * Returns the list of ref storage items to backup/restore for a worktree.\n> + * Main worktrees include packed-refs, linked worktrees do not.\n> + */\n> +static const char **get_ref_storage_items(int is_main_worktree)\n> +{\n> +\tstatic const char *main_items[] = {\"refs\", \"logs\", \"reftable\", \"packed-refs\", NULL};\n> +\tstatic const char *linked_items[] = {\"refs\", \"logs\", \"reftable\", NULL};\n> +\n> +\treturn is_main_worktree ? main_items : linked_items;\n> +}\n\nThis here is what I was referring to as \"too intimate with the\nbackends\". This logic should be entirely self-contained in the backends,\nand it already is for migrating the main worktree. We have\n`ref_store_remove_on_disk()` to prune old data, and as the new ref\nstorage is written into a temporary directory we don't need to enumerate\nits contents, but can instead move all of its entries into the gitdir\ndirectly.\n\nI guess the reason why you have this and all of the following functions\nis to create the backups. But that logic must not live in \"refs.c\", but\nit really should live in the backends. I could for example see a new\nfunction that moves a ref store to a different directory.\n\nAn alternative would be to not do the backups at all. We only start\ndoing \"destructive\" operations when all the new backends have already\nbeen created, so the last step would be to rename everything into place.\nIf this operation fails or gets cancelled we are left with a broken\nrepository, true. But the data is not lost, as we can in theory continue\nto rename the remaining data into place.\n\nSo maybe that's good enough? The user would have to manually restore in\neither of the cases, so we don't really gain that much by having a\nbackup in the first place.\n\n> diff --git a/refs/files-backend.c b/refs/files-backend.c\n> index 8d7007f4aa..6be56274d3 100644\n> --- a/refs/files-backend.c\n> +++ b/refs/files-backend.c\n> @@ -3210,11 +3210,13 @@ static int files_transaction_finish_initial(struct files_ref_store *refs,\n>  \t\tstring_list_append(&refnames_to_check, update->refname);\n>  \n>  \t\t/*\n> -\t\t * packed-refs don't support symbolic refs, root refs and reflogs,\n> -\t\t * so we have to queue these references via the loose transaction.\n> +\t\t * packed-refs don't support symbolic refs, root refs, per-worktree\n> +\t\t * refs, and reflogs, so we have to queue these references via the\n> +\t\t * loose transaction.\n>  \t\t */\n>  \t\tif (update->new_target ||\n>  \t\t    is_root_ref(update->refname) ||\n> +\t\t    is_per_worktree_ref(update->refname) ||\n>  \t\t    (update->flags & REF_LOG_ONLY)) {\n>  \t\t\tif (!loose_transaction) {\n>  \t\t\t\tloose_transaction = ref_store_transaction_begin(&refs->base, 0, err);\n\nThis is one of these changes where I think it would make sense to split\nthem out into separate commits so that they can be properly singled out\nand explained.\n\nPatrick\n"},{"id":"529794","messageId":"xmqqfrb3dnis.fsf@gitster.g","threadId":"64395","inReplyTo":"aQBwiE-bhqcaSHG_@pks.im","subject":"Re: [PATCH] refs: support migration with worktrees","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-28T16:00:43Z","receivedAt":"2025-10-28T16:00:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n>>  `migrate`::\n>> -\tMigrate ref store between different formats.\n>> +\tMigrate ref store between different formats. Supports repositories\n>> +\twith worktrees; migration must be run from the main worktree.\n>\n> It feels a bit weird to single our worktrees specifically. We don't say\n> that the tool supports bare and non-bare repositories, either, so the\n> only reason why we'd have the note about worktrees is historic legacy.\n> How about this instead:\n>\n>     Migrate ref storage between different formats. Must be run from the\n>     main worktree in case the repository uses worktrees.\n\nTwo thoughts.\n\n * Would it be unacceptable if the primary repository and refstore\n   uses reftable backend, and a newly attached worktree to the\n   repository uses ref-files only for its per-worktree refs?  If we\n   should allow it, then \"if the ref store you are migrating is in a\n   repository with multiple worktrees, you must migrate from the\n   primary and migrate _all_ ref store for all worktrees at once,\n   into the same backend\", which the design of this patch seems to\n   aim at, would contradict with it, no?\n\n * If \"you must do so from the primary worktree and we convert all\n   the worktrees attached to the same repository\" is the only mode\n   of operation we support (which by the way I have no problem\n   with---the first bullet point above was asking question, not\n   suggesting change of design), then would it be easier for the\n   user to use if the command noticed that it is not in the primary\n   worktree and switched to it for the user, instead of complaining\n   and failing?\n\n>> @@ -95,7 +96,7 @@ KNOWN LIMITATIONS\n>>  \n>>  The ref format migration has several known limitations in its current form:\n>>  \n>> -* It is not possible to migrate repositories that have worktrees.\n>> +* Migration must be run from the main worktree.\n>>  \n>\n> I'd drop this bullet point entirely, as I don't really see this as a\n> limitation anymore.\n\nI agree that such a limitation should be lifted, but if we have to\nsay \"you must do it this way, not that way\", that is still a\nlimitation ;-).\n"},{"id":"529858","messageId":"aQHoKXtrbDx6eNpH@pks.im","threadId":"64395","inReplyTo":"xmqqfrb3dnis.fsf@gitster.g","subject":"Re: [PATCH] refs: support migration with worktrees","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-10-29T10:10:49Z","receivedAt":"2025-10-29T10:11:01Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Oct 28, 2025 at 09:00:43AM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> >>  `migrate`::\n> >> -\tMigrate ref store between different formats.\n> >> +\tMigrate ref store between different formats. Supports repositories\n> >> +\twith worktrees; migration must be run from the main worktree.\n> >\n> > It feels a bit weird to single our worktrees specifically. We don't say\n> > that the tool supports bare and non-bare repositories, either, so the\n> > only reason why we'd have the note about worktrees is historic legacy.\n> > How about this instead:\n> >\n> >     Migrate ref storage between different formats. Must be run from the\n> >     main worktree in case the repository uses worktrees.\n> \n> Two thoughts.\n> \n>  * Would it be unacceptable if the primary repository and refstore\n>    uses reftable backend, and a newly attached worktree to the\n>    repository uses ref-files only for its per-worktree refs?  If we\n>    should allow it, then \"if the ref store you are migrating is in a\n>    repository with multiple worktrees, you must migrate from the\n>    primary and migrate _all_ ref store for all worktrees at once,\n>    into the same backend\", which the design of this patch seems to\n>    aim at, would contradict with it, no?\n\nThe problem we have here is backwards compatibility. Right now we assume\nthat `extensions.refStorage` applies to all worktrees, so if we wanted\nto change it like you propose then we'd have to introduce a backwards\nincompatible change.\n\nI agree though that it would've been great if we would have said from\nthe beginning that the worktree-specific configuration is allowed to\noverride the ref storage format for a worktree. If so, we could easily\nconvert any of the worktrees (including the main one) by without having\nany impact on all the other worktrees.\n\nBut we do not live in such a world right now, and getting there would\nrequire some significant reworking of how we handle per-worktree\nreferences. Unfortunate, but I also don't think there's a strong enough\nreason to change this.\n\n>  * If \"you must do so from the primary worktree and we convert all\n>    the worktrees attached to the same repository\" is the only mode\n>    of operation we support (which by the way I have no problem\n>    with---the first bullet point above was asking question, not\n>    suggesting change of design), then would it be easier for the\n>    user to use if the command noticed that it is not in the primary\n>    worktree and switched to it for the user, instead of complaining\n>    and failing?\n\nI'm not sure. The question is whether the user recognizes that migrating\nreferences in the worktree would also migrate references in the main\nrepository. It might be surprising behaviour if we did that without\nasking.\n\nIt might of course also be surprising if you do that from the main\nworking tree. But I think there's an argument to be made that it's at\nleast _less_ surprising.\n\n> >> @@ -95,7 +96,7 @@ KNOWN LIMITATIONS\n> >>  \n> >>  The ref format migration has several known limitations in its current form:\n> >>  \n> >> -* It is not possible to migrate repositories that have worktrees.\n> >> +* Migration must be run from the main worktree.\n> >>  \n> >\n> > I'd drop this bullet point entirely, as I don't really see this as a\n> > limitation anymore.\n> \n> I agree that such a limitation should be lifted, but if we have to\n> say \"you must do it this way, not that way\", that is still a\n> limitation ;-).\n\nSo with the above reasoning I'm not sure I'd call this a limitation.\nIt's rather a mechanism to protect users from unexpected consequences.\n\nPatrick\n"},{"id":"529860","messageId":"85d6fdcc-cee3-448a-8bda-72791f342be3@app.fastmail.com","threadId":"64395","inReplyTo":"aQHoKXtrbDx6eNpH@pks.im","subject":"Re: [PATCH] refs: support migration with worktrees","fromName":"Kristoffer Haugsbakk","fromEmail":"kristofferhaugsbakk@fastmail.com","sentAt":"2025-10-29T11:33:15Z","receivedAt":"2025-10-29T11:33:38Z","isPatch":true,"sender":{"key":"kristofferhaugsbakk@fastmail.com","avatar":null},"body":"On Wed, Oct 29, 2025, at 11:10, Patrick Steinhardt wrote:\n> On Tue, Oct 28, 2025 at 09:00:43AM -0700, Junio C Hamano wrote:\n>> Patrick Steinhardt <ps@pks.im> writes:\n>>\n>> >>  `migrate`::\n>> >> -\tMigrate ref store between different formats.\n>> >> +\tMigrate ref store between different formats. Supports repositories\n>> >> +\twith worktrees; migration must be run from the main worktree.\n>> >\n>> > It feels a bit weird to single our worktrees specifically. We don't say\n>> > that the tool supports bare and non-bare repositories, either, so the\n>> > only reason why we'd have the note about worktrees is historic legacy.\n>> > How about this instead:\n>> >\n>> >     Migrate ref storage between different formats. Must be run from the\n>> >     main worktree in case the repository uses worktrees.\n>>\n>> Two thoughts.\n>>\n>>  * Would it be unacceptable if the primary repository and refstore\n>>    uses reftable backend, and a newly attached worktree to the\n>>    repository uses ref-files only for its per-worktree refs?  If we\n>>    should allow it, then \"if the ref store you are migrating is in a\n>>    repository with multiple worktrees, you must migrate from the\n>>    primary and migrate _all_ ref store for all worktrees at once,\n>>    into the same backend\", which the design of this patch seems to\n>>    aim at, would contradict with it, no?\n>\n> The problem we have here is backwards compatibility. Right now we assume\n> that `extensions.refStorage` applies to all worktrees, so if we wanted\n> to change it like you propose then we'd have to introduce a backwards\n> incompatible change.\n\nI don’t understand the motivation or use case for supporting different\nbackends for different worktrees. But Junio would have to explain that.\n\nMaybe the motivation is this weird (from a user’s perspective) limi-\ntation that you have to run a command from the main worktree?  Okay,\nthat’s strange but you get the error and switch to wherever the main\nworktree is (that the error message hopefully helpfully provides you\nwith) and run the command there.  Then you forget that weird thing five\nminutes later since this was a one-off command.\n\n>\n> I agree though that it would've been great if we would have said from\n> the beginning that the worktree-specific configuration is allowed to\n> override the ref storage format for a worktree. If so, we could easily\n> convert any of the worktrees (including the main one) by without having\n> any impact on all the other worktrees.\n\nAs a user I don’t understand why that is a great thing to have.\n\n>\n> But we do not live in such a world right now, and getting there would\n> require some significant reworking of how we handle per-worktree\n> references. Unfortunate, but I also don't think there's a strong enough\n> reason to change this.\n>\n>>  * If \"you must do so from the primary worktree and we convert all\n>>    the worktrees attached to the same repository\" is the only mode\n>>    of operation we support (which by the way I have no problem\n>>    with---the first bullet point above was asking question, not\n>>    suggesting change of design), then would it be easier for the\n>>    user to use if the command noticed that it is not in the primary\n>>    worktree and switched to it for the user, instead of complaining\n>>    and failing?\n>\n> I'm not sure. The question is whether the user recognizes that migrating\n> references in the worktree would also migrate references in the main\n> repository. It might be surprising behaviour if we did that without\n> asking.\n\nOn the contrary, as a user I think it mattering what worktree I run this\ncommand from sounds very weird.  (But again I can tolerate it requiring\nme to run it from the main worktree if there are technical difficulties/\nlimitations.  But using different backends for different\nworktrees is very weird, again.)\n\nIf I run `git gc` I don’t want it to do different things based on what\nworktree I am. I want to operate on the repository, and the repository\nis the same no matter what worktree I am in.  The same principle applies\nto this command in my mind.\n\nIs the “main worktree” even something that makes sense from the user’s\nperspective?  It seems like it’s just a side-effect of the fact that the\nrepository itself has to live somewhere.  Imagine I have one main\nworktree and two linked ones.  I delete the main worktree.  Imagine that\nit works because the repository itself is moved to one of the linked\nworktrees (arbitrary).  Which then becomes the main worktree.  But the\nuser does not have to care as long the user does not poke inside the\n`.git` directory.  Which the user should not have to do (there should be\ncommands to answer whatever `.git`-poking motivations).\n\nI am of course not suggesting such a change.  But the point is that the\n“main worktree” is not such a useful end-user concept.\n\nSure, I happen to use a “main worktree” in the informal sense that I\noften have the original path where I cloned or created the repository\nand I have the other ones in satellite locations with more\npointed/topical names (e.g. `git-mine` is the basename of the Git\nworktree that I use to `make install`).  But I never ever consult `git\nworktree list` to remind me what the main worktree is.\n\nOkay.  Let’s say I get tripped up by the gitlink or whatever it is kind\nof file that worktrees use for `.git`.  Because I really want to poke at\nthe `.git` directory.  Then I think “I need to find the main worktree”\nbecause it happens to have the repository and the link to that directory\ncould not be implemented using a symlink, maybe because of Windows\nfilesystems, I don’t know.  Again a technical limitation to my mind.\nNo worktree is special except because of technical limitations.\n\n(The “main worktree” even becomes a technically contradictory concept in\nthe case when the “main worktree” is bare. And that is a popular\npractice for some reason.)\n\nAnd I wonder how many worktree users even actively think about the fact\nthat per-worktree refs exist.  It’s the kind of thing that you have to\nlogically conclude *has* to be the case:\n\n1. `HEAD` is a ref and you need that for a worktree\n2. You can have a bisect session in a worktree and that uses refs under\n   the hood\n\nBut:\n\n1. Conceptually I never really think about `HEAD` as a ref; “what\n   branch/commit am I on” is what I care about. It’s the only builtin\n   symref that I know of (or ref or symref depending on...). Not a usual\n   ref at all.\n2. I use git-bisect(1) to find a commit given a criteria.  Ones I have\n   it I note the commit.  I don’t care that refs are used to store the\n   bisect state while a session is active.\n\nAccording to gitglossary(7) these are currently the only per-worktree\nrefs.  I do not know if you are allowed to use the `refs/worktree/`\nhierarchy to create refs beyond that.\n\n>\n> It might of course also be surprising if you do that from the main\n> working tree. But I think there's an argument to be made that it's at\n> least _less_ surprising.\n>\n>> >> @@ -95,7 +96,7 @@ KNOWN LIMITATIONS\n>> >>\n>> >>  The ref format migration has several known limitations in its current form:\n>> >>\n>> >> -* It is not possible to migrate repositories that have worktrees.\n>> >> +* Migration must be run from the main worktree.\n>> >>\n>> >\n>> > I'd drop this bullet point entirely, as I don't really see this as a\n>> > limitation anymore.\n>>\n>> I agree that such a limitation should be lifted, but if we have to\n>> say \"you must do it this way, not that way\", that is still a\n>> limitation ;-).\n>\n> So with the above reasoning I'm not sure I'd call this a limitation.\n> It's rather a mechanism to protect users from unexpected consequences.\n"},{"id":"529869","messageId":"xmqqzf99ahop.fsf@gitster.g","threadId":"64395","inReplyTo":"aQHoKXtrbDx6eNpH@pks.im","subject":"Re: [PATCH] refs: support migration with worktrees","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-29T14:47:18Z","receivedAt":"2025-10-29T14:47:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> The problem we have here is backwards compatibility. Right now we assume\n> that `extensions.refStorage` applies to all worktrees, so if we wanted\n> to change it like you propose then we'd have to introduce a backwards\n> incompatible change.\n\nThat settles it.  If we have long declared that a set of worktrees\nattached to a repository share the same backend, then we do not have\nto worry about overlaying refs stored in a different backend on top\nof the base set of refs at all.  That simplifies things a lot, I\nwould imagine.\n\n> So with the above reasoning I'm not sure I'd call this a limitation.\n> It's rather a mechanism to protect users from unexpected consequences.\n\nThe need for that mechanism would imply that it may not be clear to\nthe users that worktrees of the same repository must use the same\nref backend.  Some education is needed, and erroring this operation\nout may be one of the ways to give that, perhaps.\n\nThanks.\n"},{"id":"529878","messageId":"598941EF-43F6-4642-B665-C0D65C5CDABB@gmail.com","threadId":"64395","inReplyTo":"85d6fdcc-cee3-448a-8bda-72791f342be3@app.fastmail.com","subject":"Re: [PATCH] refs: support migration with worktrees","fromName":"Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2025-10-29T16:22:18Z","receivedAt":"2025-10-29T16:22:30Z","isPatch":true,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"\n\n> Le 29 oct. 2025 à 07:37, Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail.com> a écrit :\n> \n> On Wed, Oct 29, 2025, at 11:10, Patrick Steinhardt wrote:\n>>> On Tue, Oct 28, 2025 at 09:00:43AM -0700, Junio C Hamano wrote:\n>>> Patrick Steinhardt <ps@pks.im> writes:\n\n[snip]\n\n>>> * If \"you must do so from the primary worktree and we convert all\n>>>   the worktrees attached to the same repository\" is the only mode\n>>>   of operation we support (which by the way I have no problem\n>>>   with---the first bullet point above was asking question, not\n>>>   suggesting change of design), then would it be easier for the\n>>>   user to use if the command noticed that it is not in the primary\n>>>   worktree and switched to it for the user, instead of complaining\n>>>   and failing?\n>> \n>> I'm not sure. The question is whether the user recognizes that migrating\n>> references in the worktree would also migrate references in the main\n>> repository. It might be surprising behaviour if we did that without\n>> asking.\n> \n> On the contrary, as a user I think it mattering what worktree I run this\n> command from sounds very weird.  (But again I can tolerate it requiring\n> me to run it from the main worktree if there are technical difficulties/\n> limitations.  But using different backends for different\n> worktrees is very weird, again.)\n\n[snip]\n\nThe fewer concepts we ask a user to manage at a time, likely the better. In this case, “migrate the refs” should probably just work. While things are experimental, rough edges are more tolerable of course, but as we are lifting limitations towards making things official I think polishing such edges is a good idea.\n\nIn sum, it can be done later, but I think automatically changing the process directory to the main worktree and carrying on is fine. The curious folks would even see that under the TRACE output ;)\n"},{"id":"529921","messageId":"aQMHpfwZs2eqRAwS@pks.im","threadId":"64395","inReplyTo":"85d6fdcc-cee3-448a-8bda-72791f342be3@app.fastmail.com","subject":"Re: [PATCH] refs: support migration with worktrees","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-10-30T06:37:25Z","receivedAt":"2025-10-30T06:37:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Oct 29, 2025 at 12:33:15PM +0100, Kristoffer Haugsbakk wrote:\n> On Wed, Oct 29, 2025, at 11:10, Patrick Steinhardt wrote:\n> > On Tue, Oct 28, 2025 at 09:00:43AM -0700, Junio C Hamano wrote:\n> >> Patrick Steinhardt <ps@pks.im> writes:\n> >>\n> >> >>  `migrate`::\n> >> >> -\tMigrate ref store between different formats.\n> >> >> +\tMigrate ref store between different formats. Supports repositories\n> >> >> +\twith worktrees; migration must be run from the main worktree.\n> >> >\n> >> > It feels a bit weird to single our worktrees specifically. We don't say\n> >> > that the tool supports bare and non-bare repositories, either, so the\n> >> > only reason why we'd have the note about worktrees is historic legacy.\n> >> > How about this instead:\n> >> >\n> >> >     Migrate ref storage between different formats. Must be run from the\n> >> >     main worktree in case the repository uses worktrees.\n> >>\n> >> Two thoughts.\n> >>\n> >>  * Would it be unacceptable if the primary repository and refstore\n> >>    uses reftable backend, and a newly attached worktree to the\n> >>    repository uses ref-files only for its per-worktree refs?  If we\n> >>    should allow it, then \"if the ref store you are migrating is in a\n> >>    repository with multiple worktrees, you must migrate from the\n> >>    primary and migrate _all_ ref store for all worktrees at once,\n> >>    into the same backend\", which the design of this patch seems to\n> >>    aim at, would contradict with it, no?\n> >\n> > The problem we have here is backwards compatibility. Right now we assume\n> > that `extensions.refStorage` applies to all worktrees, so if we wanted\n> > to change it like you propose then we'd have to introduce a backwards\n> > incompatible change.\n> \n> I don’t understand the motivation or use case for supporting different\n> backends for different worktrees. But Junio would have to explain that.\n> \n> Maybe the motivation is this weird (from a user’s perspective) limi-\n> tation that you have to run a command from the main worktree?  Okay,\n> that’s strange but you get the error and switch to wherever the main\n> worktree is (that the error message hopefully helpfully provides you\n> with) and run the command there.  Then you forget that weird thing five\n> minutes later since this was a one-off command.\n> \n> >\n> > I agree though that it would've been great if we would have said from\n> > the beginning that the worktree-specific configuration is allowed to\n> > override the ref storage format for a worktree. If so, we could easily\n> > convert any of the worktrees (including the main one) by without having\n> > any impact on all the other worktrees.\n> \n> As a user I don’t understand why that is a great thing to have.\n\nI am commenting more from the developer side here. In the best case the\nuser wouldn't ever care what ref storage format they use. We simply pick\nthe best format available and the user lives happily ever after.\n\nBut from a developer standpoint it matters. If we had per-worktree ref\nformats we would for example be able to make the ref migration code a\nlot more robust, as we could now migrate worktrees one by one. In the\ncurrent situation we basically have to migrate all worktrees at once,\nand that significantly increases the risk of the migration going wrong\nat any point in time.\n\n> >\n> > But we do not live in such a world right now, and getting there would\n> > require some significant reworking of how we handle per-worktree\n> > references. Unfortunate, but I also don't think there's a strong enough\n> > reason to change this.\n> >\n> >>  * If \"you must do so from the primary worktree and we convert all\n> >>    the worktrees attached to the same repository\" is the only mode\n> >>    of operation we support (which by the way I have no problem\n> >>    with---the first bullet point above was asking question, not\n> >>    suggesting change of design), then would it be easier for the\n> >>    user to use if the command noticed that it is not in the primary\n> >>    worktree and switched to it for the user, instead of complaining\n> >>    and failing?\n> >\n> > I'm not sure. The question is whether the user recognizes that migrating\n> > references in the worktree would also migrate references in the main\n> > repository. It might be surprising behaviour if we did that without\n> > asking.\n> \n> On the contrary, as a user I think it mattering what worktree I run this\n> command from sounds very weird.  (But again I can tolerate it requiring\n> me to run it from the main worktree if there are technical difficulties/\n> limitations.  But using different backends for different\n> worktrees is very weird, again.)\n> \n> If I run `git gc` I don’t want it to do different things based on what\n> worktree I am. I want to operate on the repository, and the repository\n> is the same no matter what worktree I am in.  The same principle applies\n> to this command in my mind.\n\nIt does though :) Only very slightly so, but for example maintenance of\nreferences is dependent on the worktree you are in. We don't maintain\nreferences from other worktrees. So it's not really a new thing that I'm\nproposing here.\n\nIn any case, I'm happy to change my stance if the majority of folks\nthinks that migrating the whole repository from secondary worktrees is\nfine. I mostly wanted to avoid that operations that the user perform\nhave a wider blast radius than they understood, but if everyone agrees\nthat this is a non-issue then I don't mind much. It's only going to make\nthe implementation simpler.\n\nPatrick\n"}]}