{"thread":{"id":"58504","subject":"[PATCH 00/10] Add the Git Change command","startedAt":"2022-09-23T18:55:47Z","lastAt":"2022-10-12T19:19:50Z","messageCount":66,"participants":["Christophe Poucet via GitGitGadget","Chris Poucet via GitGitGadget","Stefan Xenos via GitGitGadget","Jerry Zhang","Phillip Wood","Ævar Arnfjörð Bjarmason","Chris Poucet","Junio C Hamano","Jonathan Tan","Chris P","Glen Choo","Victoria Dye"],"isPatch":true,"patchVersion":1,"patchTotal":10},"messages":[{"id":"463532","messageId":"pull.1356.git.1663959324.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":null,"subject":"[PATCH 00/10] Add the Git Change command","fromName":"Christophe Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:14Z","receivedAt":"2022-09-23T18:55:47Z","isPatch":true,"sender":{"key":"name:Christophe Poucet","avatar":null},"body":"I'm reviving the original git evolve work that was started by\nsxenos@google.com\n(https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n\nThis work is intended to make it easier to deal with stacked changes.\n\nThe following set of patches introduces the design doc on the evolve command\nas well as the basics of the git change command.\n\nChris Poucet (4):\n sha1-array: implement oid_array_readonly_contains\n ref-filter: add the metas namespace to ref-filter\n evolve: add delete command\n evolve: add documentation for `git change`\n\nStefan Xenos (6):\n technical doc: add a design doc for the evolve command\n evolve: add support for parsing metacommits\n evolve: add the change-table structure\n evolve: add support for writing metacommits\n evolve: implement the git change command\n evolve: add the git change list command\n\n .gitignore | 1 +\n Documentation/git-change.txt | 55 ++\n Documentation/technical/evolve.txt | 1051 ++++++++++++++++++++++++++++\n Makefile | 4 +\n builtin.h | 1 +\n builtin/change.c | 342 +++++++++\n change-table.c | 179 +++++\n change-table.h | 132 ++++\n git.c | 1 +\n metacommit-parser.c | 110 +++\n metacommit-parser.h | 19 +\n metacommit.c | 404 +++++++++++\n metacommit.h | 58 ++\n oid-array.c | 12 +\n oid-array.h | 7 +\n ref-filter.c | 10 +-\n ref-filter.h | 8 +-\n t/helper/test-oid-array.c | 6 +\n t/t0064-oid-array.sh | 22 +\n 19 files changed, 2418 insertions(+), 4 deletions(-)\n create mode 100644 Documentation/git-change.txt\n create mode 100644 Documentation/technical/evolve.txt\n create mode 100644 builtin/change.c\n create mode 100644 change-table.c\n create mode 100644 change-table.h\n create mode 100644 metacommit-parser.c\n create mode 100644 metacommit-parser.h\n create mode 100644 metacommit.c\n create mode 100644 metacommit.h\n\n\nbase-commit: 4b79ee4b0cd1130ba8907029cdc5f6a1632aca26\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1356%2Fpoucet%2Fevolve-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1356/poucet/evolve-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1356\n-- \ngitgitgadget\n"},{"id":"463533","messageId":"84588312c1d4a62ff6c6211e85b4e58ab0563daa.1663959324.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 02/10] sha1-array: implement oid_array_readonly_contains","fromName":"Chris Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:16Z","receivedAt":"2022-09-23T18:55:48Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"From: Chris Poucet \n\nImplement a \"readonly_contains\" function for oid_array that won't\nsort the array if it is unsorted. This can be used to test containment in\nthe rare situations where the array order matters.\n\nThe function has intentionally been given a name that is more cumbersome\nthan the \"lookup\" function, which is what most callers will will want\nin most situations.\n\nSigned-off-by: Chris Poucet \n---\n oid-array.c | 12 ++++++++++++\n oid-array.h | 7 +++++++\n t/helper/test-oid-array.c | 6 ++++++\n t/t0064-oid-array.sh | 22 ++++++++++++++++++++++\n 4 files changed, 47 insertions(+)\n\ndiff --git a/oid-array.c b/oid-array.c\nindex 73ba76e9e9a..1e12651d245 100644\n--- a/oid-array.c\n+++ b/oid-array.c\n@@ -28,6 +28,18 @@ static const struct object_id *oid_access(size_t index, const void *table)\n \treturn &array[index];\n }\n \n+int oid_array_readonly_contains(const struct oid_array *array,\n+\t\t\t\tconst struct object_id* oid) {\n+\tint i;\n+\n+\tif (array->sorted)\n+\t\treturn oid_pos(oid, array->oid, array->nr, oid_access) >= 0;\n+\tfor (i = 0; i < array->nr; i++)\n+\t\tif (oideq(&array->oid[i], oid))\n+\t\t\treturn 1;\n+\treturn 0;\n+}\n+\n int oid_array_lookup(struct oid_array *array, const struct object_id *oid)\n {\n \toid_array_sort(array);\ndiff --git a/oid-array.h b/oid-array.h\nindex f60f9af6741..e056eb61fa2 100644\n--- a/oid-array.h\n+++ b/oid-array.h\n@@ -58,6 +58,13 @@ struct oid_array {\n \n #define OID_ARRAY_INIT { 0 }\n \n+/**\n+ * Sees whether an array contains an object ID. Optimized for when the array is\n+ * sorted but does not require the array to be sorted.\n+ */\n+int oid_array_readonly_contains(const struct oid_array *array,\n+\t\t\t\tconst struct object_id* oid);\n+\n /**\n * Add an item to the set. The object ID will be placed at the end of the array\n * (but note that some operations below may lose this ordering).\ndiff --git a/t/helper/test-oid-array.c b/t/helper/test-oid-array.c\nindex d1324d086a2..0dbfc91ca8d 100644\n--- a/t/helper/test-oid-array.c\n+++ b/t/helper/test-oid-array.c\n@@ -28,10 +28,16 @@ int cmd__oid_array(int argc, const char **argv)\n \t\t\tif (get_oid_hex(arg, &oid))\n \t\t\t\tdie(\"not a hexadecimal oid: %s\", arg);\n \t\t\tprintf(\"%d\\n\", oid_array_lookup(&array, &oid));\n+\t\t} else if (skip_prefix(line.buf, \"readonly_contains \", &arg)) {\n+\t\t\tif (get_oid_hex(arg, &oid))\n+\t\t\t\tdie(\"not a hexadecimal oid: %s\", arg);\n+\t\t\tprintf(\"%d\\n\", oid_array_readonly_contains(&array, &oid));\n \t\t} else if (!strcmp(line.buf, \"clear\"))\n \t\t\toid_array_clear(&array);\n \t\telse if (!strcmp(line.buf, \"for_each_unique\"))\n \t\t\toid_array_for_each_unique(&array, print_oid, NULL);\n+\t\telse if (!strcmp(line.buf, \"for_each\"))\n+\t\t\toid_array_for_each(&array, print_oid, NULL);\n \t\telse\n \t\t\tdie(\"unknown command: %s\", line.buf);\n \t}\ndiff --git a/t/t0064-oid-array.sh b/t/t0064-oid-array.sh\nindex 88c89e8f48a..aa677af132d 100755\n--- a/t/t0064-oid-array.sh\n+++ b/t/t0064-oid-array.sh\n@@ -35,6 +35,28 @@ test_expect_success 'ordered enumeration with duplicate suppression' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'readonly_contains finds existing' '\n+\techo 1 >expect &&\n+\techoid \"\" 88 44 aa 55 >>expect &&\n+\t{\n+\t\techoid append 88 44 aa 55 &&\n+\t\techoid readonly_contains 55 &&\n+\t\techo for_each\n+\t} | test-tool oid-array >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'readonly_contains non-existing query' '\n+\techo 0 >expect &&\n+\techoid \"\" 88 44 aa 55 >>expect &&\n+\t{\n+\t\techoid append 88 44 aa 55 &&\n+\t\techoid readonly_contains 33 &&\n+\t\techo for_each\n+\t} | test-tool oid-array >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'lookup' '\n \t{\n \t\techoid append 88 44 aa 55 &&\n-- \ngitgitgadget\n\n"},{"id":"463534","messageId":"54e559967df55ca314e629b65927a88c7f804a98.1663959324.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 03/10] ref-filter: add the metas namespace to ref-filter","fromName":"Chris Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:17Z","receivedAt":"2022-09-23T18:55:51Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"From: Chris Poucet \n\nThe metas namespace will contain refs for changes in progress. Add\nsupport for searching this namespace.\n\nSigned-off-by: Chris Poucet \n---\n ref-filter.c | 8 ++++++--\n ref-filter.h | 4 +++-\n 2 files changed, 9 insertions(+), 3 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex fd1cb14b0f1..6a1789c623f 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -2200,7 +2200,8 @@ static int ref_kind_from_refname(const char *refname)\n \t} ref_kind[] = {\n \t\t{ \"refs/heads/\" , FILTER_REFS_BRANCHES },\n \t\t{ \"refs/remotes/\" , FILTER_REFS_REMOTES },\n-\t\t{ \"refs/tags/\", FILTER_REFS_TAGS}\n+\t\t{ \"refs/tags/\", FILTER_REFS_TAGS},\n+\t\t{ \"refs/metas/\", FILTER_REFS_CHANGES }\n \t};\n \n \tif (!strcmp(refname, \"HEAD\"))\n@@ -2218,7 +2219,8 @@ static int filter_ref_kind(struct ref_filter *filter, const char *refname)\n {\n \tif (filter->kind == FILTER_REFS_BRANCHES ||\n \t filter->kind == FILTER_REFS_REMOTES ||\n-\t filter->kind == FILTER_REFS_TAGS)\n+\t filter->kind == FILTER_REFS_TAGS ||\n+\t filter->kind == FILTER_REFS_CHANGES)\n \t\treturn filter->kind;\n \treturn ref_kind_from_refname(refname);\n }\n@@ -2435,6 +2437,8 @@ int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int\n \t\t\tret = for_each_fullref_in(\"refs/remotes/\", ref_filter_handler, &ref_cbdata);\n \t\telse if (filter->kind == FILTER_REFS_TAGS)\n \t\t\tret = for_each_fullref_in(\"refs/tags/\", ref_filter_handler, &ref_cbdata);\n+\t\telse if (filter->kind == FILTER_REFS_CHANGES)\n+\t\t\tret = for_each_fullref_in(\"refs/metas/\", ref_filter_handler, &ref_cbdata);\n \t\telse if (filter->kind & FILTER_REFS_ALL)\n \t\t\tret = for_each_fullref_in_pattern(filter, ref_filter_handler, &ref_cbdata);\n \t\tif (!ret && (filter->kind & FILTER_REFS_DETACHED_HEAD))\ndiff --git a/ref-filter.h b/ref-filter.h\nindex aa0eea4ecf5..064fbef8e50 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -17,8 +17,10 @@\n #define FILTER_REFS_BRANCHES 0x0004\n #define FILTER_REFS_REMOTES 0x0008\n #define FILTER_REFS_OTHERS 0x0010\n+#define FILTER_REFS_CHANGES 0x0040\n #define FILTER_REFS_ALL (FILTER_REFS_TAGS | FILTER_REFS_BRANCHES | \\\n-\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS)\n+\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS | \\\n+\t\t\t\t FILTER_REFS_CHANGES)\n #define FILTER_REFS_DETACHED_HEAD 0x0020\n #define FILTER_REFS_KIND_MASK (FILTER_REFS_ALL | FILTER_REFS_DETACHED_HEAD)\n \n-- \ngitgitgadget\n\n"},{"id":"463535","messageId":"2e9a4a9bd819785404e8a5343385f4fb2bc06109.1663959325.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 04/10] evolve: add support for parsing metacommits","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:18Z","receivedAt":"2022-09-23T18:55:53Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nThis patch adds the get_metacommit_content method, which can classify\ncommits as either metacommits or normal commits, determine whether they\nare abandoned, and extract the content commit's object id from the\nmetacommit.\n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n Makefile | 1 +\n metacommit-parser.c | 110 ++++++++++++++++++++++++++++++++++++++++++++\n metacommit-parser.h | 19 ++++++++\n 3 files changed, 130 insertions(+)\n create mode 100644 metacommit-parser.c\n create mode 100644 metacommit-parser.h\n\ndiff --git a/Makefile b/Makefile\nindex cac3452edb9..b2bcc00c289 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -999,6 +999,7 @@ LIB_OBJS += merge-ort.o\n LIB_OBJS += merge-ort-wrappers.o\n LIB_OBJS += merge-recursive.o\n LIB_OBJS += merge.o\n+LIB_OBJS += metacommit-parser.o\n LIB_OBJS += midx.o\n LIB_OBJS += name-hash.o\n LIB_OBJS += negotiator/default.o\ndiff --git a/metacommit-parser.c b/metacommit-parser.c\nnew file mode 100644\nindex 00000000000..70c1428bfc6\n--- /dev/null\n+++ b/metacommit-parser.c\n@@ -0,0 +1,110 @@\n+#include \"cache.h\"\n+#include \"metacommit-parser.h\"\n+#include \"commit.h\"\n+\n+/*\n+ * Search the commit buffer for a line starting with the given key. Unlike\n+ * find_commit_header, this also searches the commit message body.\n+ */\n+static const char *find_key(const char *msg, const char *key, size_t *out_len)\n+{\n+\tint key_len = strlen(key);\n+\tconst char *line = msg;\n+\n+\twhile (line) {\n+\t\tconst char *eol = strchrnul(line, '\\n');\n+\n+\t\tif (eol - line > key_len && !memcmp(line, key, key_len) &&\n+\t\t line[key_len] == ' ') {\n+\t\t\t*out_len = eol - line - key_len - 1;\n+\t\t\treturn line + key_len + 1;\n+\t\t}\n+\t\tline = *eol ? eol + 1 : NULL;\n+\t}\n+\treturn NULL;\n+}\n+\n+static struct commit *get_commit_by_index(struct commit_list *to_search, int index)\n+{\n+\twhile (to_search && index) {\n+\t\tto_search = to_search->next;\n+\t\tindex--;\n+\t}\n+\n+\tif (!to_search)\n+\t\treturn NULL;\n+\n+\treturn to_search->item;\n+}\n+\n+/*\n+ * Writes the index of the content parent to \"result\". Returns the metacommit\n+ * type. See the METACOMMIT_TYPE_* constants.\n+ */\n+static int index_of_content_commit(const char *buffer, int *result)\n+{\n+\tint index = 0;\n+\tint ret = METACOMMIT_TYPE_NONE;\n+\tsize_t parent_types_size;\n+\tconst char *parent_types = find_key(buffer, \"parent-type\",\n+\t\t&parent_types_size);\n+\tconst char *end;\n+\tconst char *enum_start = parent_types;\n+\tint enum_length = 0;\n+\n+\tif (!parent_types)\n+\t\treturn METACOMMIT_TYPE_NONE;\n+\n+\tend = &parent_types[parent_types_size];\n+\n+\twhile (1) {\n+\t\tchar next = *parent_types;\n+\t\tif (next == ' ' || parent_types >= end) {\n+\t\t\tif (enum_length == 1) {\n+\t\t\t\tchar first_char_in_enum = *enum_start;\n+\t\t\t\tif (first_char_in_enum == 'c') {\n+\t\t\t\t\tret = METACOMMIT_TYPE_NORMAL;\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t\tif (first_char_in_enum == 'a') {\n+\t\t\t\t\tret = METACOMMIT_TYPE_ABANDONED;\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t\tif (parent_types >= end)\n+\t\t\t\treturn METACOMMIT_TYPE_NONE;\n+\t\t\tenum_start = parent_types + 1;\n+\t\t\tenum_length = 0;\n+\t\t\tindex++;\n+\t\t} else {\n+\t\t\tenum_length++;\n+\t\t}\n+\t\tparent_types++;\n+\t}\n+\n+\t*result = index;\n+\treturn ret;\n+}\n+\n+/*\n+ * Writes the content parent's object id to \"content\".\n+ * Returns the metacommit type. See the METACOMMIT_TYPE_* constants.\n+ */\n+int get_metacommit_content(struct commit *commit, struct object_id *content)\n+{\n+\tconst char *buffer = get_commit_buffer(commit, NULL);\n+\tint index = 0;\n+\tint ret = index_of_content_commit(buffer, &index);\n+\tstruct commit *content_parent;\n+\n+\tif (ret == METACOMMIT_TYPE_NONE)\n+\t\treturn ret;\n+\n+\tcontent_parent = get_commit_by_index(commit->parents, index);\n+\n+\tif (!content_parent)\n+\t\treturn METACOMMIT_TYPE_NONE;\n+\n+\toidcpy(content, &(content_parent->object.oid));\n+\treturn ret;\n+}\ndiff --git a/metacommit-parser.h b/metacommit-parser.h\nnew file mode 100644\nindex 00000000000..1c74bd6d699\n--- /dev/null\n+++ b/metacommit-parser.h\n@@ -0,0 +1,19 @@\n+#ifndef METACOMMIT_PARSER_H\n+#define METACOMMIT_PARSER_H\n+\n+#include \"commit.h\"\n+#include \"hash.h\"\n+\n+/* Indicates a normal commit (non-metacommit) */\n+#define METACOMMIT_TYPE_NONE 0\n+/* Indicates a metacommit with normal content (non-abandoned) */\n+#define METACOMMIT_TYPE_NORMAL 1\n+/* Indicates a metacommit with abandoned content */\n+#define METACOMMIT_TYPE_ABANDONED 2\n+\n+struct commit;\n+\n+extern int get_metacommit_content(\n+\tstruct commit *commit, struct object_id *content);\n+\n+#endif\n-- \ngitgitgadget\n\n"},{"id":"463536","messageId":"a0cf68f8ba2adefae4fceeab0d438d05e355e695.1663959324.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 01/10] technical doc: add a design doc for the evolve command","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:15Z","receivedAt":"2022-09-23T18:55:58Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nThis document describes what a change graph for\ngit would look like, the behavior of the evolve command,\nand the changes planned for other commands.\n\nIt was originally proposed in 2018, see\nhttps://public-inbox.org/git/20181115005546.212538-1-sxenos@google.com/\n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n Documentation/technical/evolve.txt | 1051 ++++++++++++++++++++++++++++\n 1 file changed, 1051 insertions(+)\n create mode 100644 Documentation/technical/evolve.txt\n\ndiff --git a/Documentation/technical/evolve.txt b/Documentation/technical/evolve.txt\nnew file mode 100644\nindex 00000000000..68ee2457e52\n--- /dev/null\n+++ b/Documentation/technical/evolve.txt\n@@ -0,0 +1,1051 @@\n+Evolve\n+======\n+\n+Objective\n+=========\n+Create an \"evolve\" command to help users craft a high quality commit history.\n+Users can improve commits one at a time and in any order, then run git evolve to\n+rewrite their recent history to ensure everything is up-to-date. We track\n+amendments to a commit over time in a change graph. Users can share their\n+progress with others by exchanging their change graphs using the standard push,\n+fetch, and format-patch commands.\n+\n+Status\n+======\n+This proposal has not been implemented yet.\n+\n+Background\n+==========\n+Imagine you have three sequential changes up for review and you receive feedback\n+that requires editing all three changes. We'll define the word \"change\"\n+formally later, but for the moment let's say that a change is a work-in-progress\n+whose final version will be submitted as a commit in the future.\n+\n+While you're editing one change, more feedback arrives on one of the others.\n+What do you do?\n+\n+The evolve command is a convenient way to work with chains of commits that are\n+under review. Whenever you rebase or amend a commit, the repository remembers\n+that the old commit is obsolete and has been replaced by the new one. Then, at\n+some point in the future, you can run \"git evolve\" and the correct sequence of\n+rebases will occur in the correct order such that no commit has an obsolete\n+parent.\n+\n+Part of making the \"evolve\" command work involves tracking the edits to a commit\n+over time, which is why we need an change graph. However, the change\n+graph will also bring other benefits:\n+\n+- Users can view the history of a change directly (the sequence of amends and\n+ rebases it has undergone, orthogonal to the history of the branch it is on).\n+- It will be possible to quickly locate and list all the changes the user\n+ currently has in progress.\n+- It can be used as part of other high-level commands that combine or split\n+ changes.\n+- It can be used to decorate commits (in git log, gitk, etc) that are either\n+ obsolete or are the tip of a work in progress.\n+- By pushing and pulling the change graph, users can collaborate more\n+ easily on changes-in-progress. This is better than pushing and pulling the\n+ commits themselves since the change graph can be used to locate a more\n+ specific merge base, allowing for better merges between different versions of\n+ the same change.\n+- It could be used to correctly rebase local changes and other local branches\n+ after running git-filter-branch.\n+- It can replace the change-id footer used by gerrit.\n+\n+Goals\n+-----\n+Legend: Goals marked with P0 are required. Goals marked with Pn should be\n+attempted unless they interfere with goals marked with Pn-1.\n+\n+P0. All commands that modify commits (such as the normal commit --amend or\n+ rebase command) should mark the old commit as being obsolete and replaced by\n+ the new one. No additional commands should be required to keep the\n+ change graph up-to-date.\n+P0. Any commit that may be involved in a future evolve command should not be\n+ garbage collected. Specifically:\n+ - Commits that obsolete another should not be garbage collected until\n+ user-specified conditions have occurred and the change has expired from\n+ the reflog. User specified conditions for removing changes include:\n+ - The user explicitly deleted the change.\n+ - The change was merged into a specific branch.\n+ - Commits that have been obsoleted by another should not be garbage\n+ collected if any of their replacements are still being retained.\n+P0. A commit can be obsoleted by more than one replacement (called divergence).\n+P0. Users must be able to resolve divergence (convergence).\n+P1. Users should be able to share chains of obsolete changes in order to\n+ collaborate on WIP changes.\n+P2. Such sharing should be at the user’s option. That is, it should be possible\n+ to directly share a change without also sharing the file states or commit\n+ comments from the obsolete changes that led up to it, and the choice not to\n+ share those commits should not require changing any commit hashes.\n+P2. It should be possible to discard part or all of the change graph\n+ without discarding the commits themselves that are already present in\n+ branches and the reflog.\n+P2. Provide sufficient information to replace gerrit's Change-Id footers.\n+\n+Similar technologies\n+--------------------\n+There are some other technologies that address the same end-user problem.\n+\n+Rebase -i can be used to solve the same problem, but users can't easily switch\n+tasks midway through an interactive rebase or have more than one interactive\n+rebase going on at the same time. It can't handle the case where you have\n+multiple changes sharing the same parent when that parent needs to be rebased\n+and won't let you collaborate with others on resolving a complicated interactive\n+rebase. You can think of rebase -i as a top-down approach and the evolve command\n+as the bottom-up approach to the same problem.\n+\n+Several patch queue managers have been built on top of git (such as topgit,\n+stgit, and quilt). They address the same user need. However they also rely on\n+state managed outside git that needs to be kept in sync. Such state can be\n+easily damaged when running a git native command that is unaware of the patch\n+queue. They also typically require an explicit initialization step to be done by\n+the user which creates workflow problems.\n+\n+Mercurial implements a very similar feature in its EvolveExtension. The behavior\n+of the evolve command itself is very similar, but the storage format for the\n+change graph differs. In the case of mercurial, each change set can have one or\n+more obsolescence markers that point to other changesets that they replace. This\n+is similar to the \"Commit Headers\" approach considered in the other options\n+appendix. The approach proposed here stores obsolescence information in a\n+separate metacommit graph, which makes exchanging of obsolescence information\n+optional.\n+\n+Mercurial's default behavior makes it easy to find and switch between\n+non-obsolete changesets that aren't currently on any branch. We introduce the\n+notion of a new ref namespace that enables a similar workflow via a different\n+mechanism. Mercurial has the notion of changeset phases which isn't present\n+in git and creates new ways for a changeset to diverge. Git doesn't need\n+to deal with these issues, but it has to deal with the problems of picking an\n+upstream branch as a target for rebases and protecting obsolescence information\n+from GC. We also introduce some additional transformations (see\n+obsolescence-over-cherry-pick, below) that aren't present in the mercurial\n+implementation.\n+\n+Semi-related work\n+-----------------\n+There are other technologies that address different problems but have some\n+similarities with this proposal.\n+\n+Replacements (refs/replace) are superficially similar to obsolescences in that\n+they describe that one commit should be replaced by another. However, they\n+differ in both how they are created and how they are intended to be used.\n+Obsolescences are created automatically by the commands a user runs, and they\n+describe the user’s intent to perform a future rebase. Obsolete commits still\n+appear in branches, logs, etc like normal commits (possibly with an extra\n+decoration that marks them as obsolete). Replacements are typically created\n+explicitly by the user, they are meant to be kept around for a long time, and\n+they describe a replacement to be applied at read-time rather than as the input\n+to a future operation. When a replaced commit is queried, it is typically hidden\n+and swapped out with its replacement as though the replacement has already\n+occurred.\n+\n+Git-imerge is a project to help make complicated merges easier, particularly\n+when merging or rebasing long chains of patches. It is not an alternative to\n+the change graph, but its algorithm of applying smaller incremental merges\n+could be used as part of the evolve algorithm in the future.\n+\n+Overview\n+========\n+We introduce the notion of “meta-commits” which describe how one commit was\n+created from other commits. A branch of meta-commits is known as a change.\n+Changes are created and updated automatically whenever a user runs a command\n+that creates a commit. They are used for locating obsolete commits, providing a\n+list of a user’s unsubmitted work in progress, and providing a stable name for\n+each unsubmitted change.\n+\n+Users can exchange edit histories by pushing and fetching changes.\n+\n+New commands will be introduced for manipulating changes and resolving\n+divergence between them. Existing commands that create commits will be updated\n+to modify the meta-commit graph and create changes where necessary.\n+\n+Example usage\n+-------------\n+# First create three dependent changes\n+$ echo foo>bar.txt && git add .\n+$ git commit -m \"This is a test\"\n+created change metas/this_is_a_test\n+$ echo foo2>bar2.txt && git add .\n+$ git commit -m \"This is also a test\"\n+created change metas/this_is_also_a_test\n+$ echo foo3>bar3.txt && git add .\n+$ git commit -m \"More testing\"\n+created change metas/more_testing\n+\n+# List all our changes in progress\n+$ git change list\n+metas/this_is_a_test\n+metas/this_is_also_a_test\n+* metas/more_testing\n+metas/some_change_already_merged_upstream\n+\n+# Now modify the earliest change, using its stable name\n+$ git reset --hard metas/this_is_a_test\n+$ echo morefoo>>bar.txt && git add . && git commit --amend --no-edit\n+\n+# Use git-evolve to fix up any dependent changes\n+$ git evolve\n+rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n+rebasing metas/more_testing onto metas/this_is_also_a_test\n+Done\n+\n+# Use git-obslog to view the history of the this_is_a_test change\n+$ git log --obslog\n+93f110 metas/this_is_a_test@{0} commit (amend): This is a test\n+930219 metas/this_is_a_test@{1} commit: This is a test\n+\n+# Now create an unrelated change\n+$ git reset --hard origin/master\n+$ echo newchange>unrelated.txt && git add .\n+$ git commit -m \"Unrelated change\"\n+created change metas/unrelated_change\n+\n+# Fetch the latest code from origin/master and use git-evolve\n+# to rebase all dependent changes.\n+$ git fetch origin master\n+$ git evolve origin/master\n+deleting metas/some_change_already_merged_upstream\n+rebasing metas/this_is_a_test onto origin/master\n+rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n+rebasing metas/more_testing onto metas/this_is_also_a_test\n+rebasing metas/unrelated_change onto origin/master\n+Conflict detected! Resolve it and then use git evolve --continue to resume.\n+\n+# Sort out the conflict\n+$ git mergetool\n+$ git evolve origin/master\n+Done\n+\n+# Share the full history of edits for the this_is_a_test change\n+# with a review server\n+$ git push origin metas/this_is_a_test:refs/for/master\n+# Share the lastest commit for “Unrelated change”, without history\n+$ git push origin HEAD:refs/for/master\n+\n+Detailed design\n+===============\n+Obsolescence information is stored as a graph of meta-commits. A meta-commit is\n+a specially-formatted merge commit that describes how one commit was created\n+from others.\n+\n+Meta-commits look like this:\n+\n+$ git cat-file -p \n+tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\n+parent aa7ce55545bf2c14bef48db91af1a74e2347539a\n+parent d64309ee51d0af12723b6cb027fc9f195b15a5e9\n+parent 7e1bbcd3a0fa854a7a9eac9bf1eea6465de98136\n+author Stefan Xenos 1540841596 -0700\n+committer Stefan Xenos 1540841596 -0700\n+parent-type c r o\n+\n+This says “commit aa7ce555 makes commit d64309ee obsolete. It was created by\n+cherry-picking commit 7e1bbcd3”.\n+\n+The tree for meta-commits is always the empty tree, but future versions of git\n+may attach other trees here. For forward-compatibility fsck should ignore such\n+trees if found on future repository versions. This will allow future versions of\n+git to add metadata to the meta-commit tree without breaking forwards\n+compatibility.\n+\n+The commit comment for a meta-commit is an auto-generated user-readable string\n+describing the command that produced the meta commit. These strings are shown\n+to the user when they view the obslog.\n+\n+Parent-type\n+-----------\n+The “parent-type” field in the commit header identifies a commit as a\n+meta-commit and indicates the meaning for each of its parents. It is never\n+present for normal commits. It contains a space-deliminated list of enum values\n+whose order matches the order of the parents. Possible parent types are:\n+\n+- c: (content) the content parent identifies the commit that this meta-commit is\n+ describing.\n+- r: (replaced) indicates that this parent is made obsolete by the content\n+ parent.\n+- o: (origin) indicates that the content parent was generated by cherry-picking\n+ this parent.\n+- a: (abandoned) used in place of a content parent for abandoned changes. Points\n+ to the final content commit for the change at the time it was abandoned.\n+\n+There must be exactly one content or abandoned parent for each meta-commit and\n+it is always the first parent. The content commit will always be a normal commit\n+and not a meta-commit. However, future versions of git may create meta-commits\n+for other meta-commits and the fsck tool must be aware of this for forwards\n+compatibility.\n+\n+A meta-commit can have zero or more replaced parents. An amend operation creates\n+a single replaced parent. A merge used to resolve divergence (see divergence,\n+below) will create multiple replaced parents. A meta-commit may have no\n+replaced parents if it describes a cherry-pick or squash merge that copies one\n+or more commits but does not replace them.\n+\n+A meta-commit can have zero or more origin parents. A cherry-pick creates a\n+single origin parent. Certain types of squash merge will create multiple origin\n+parents. Origin parents don't directly cause their origin to become obsolete,\n+but are used when computing blame or locating a merge base. The section\n+on obsolescence over cherry-picks describes how the evolve command uses\n+origin parents.\n+\n+A replaced parent or origin parent may be either a normal commit (indicating\n+the oldest-known version of a change) or another meta-commit (for a change that\n+has already been modified one or more times).\n+\n+The parent-type field needs to go after the committer field since git's rules\n+for forwards-compatibility require that new fields to be at the end of the\n+header. Putting a new field in the middle of the header would break fsck.\n+\n+The presence of an abandoned parent indicates that the change should be pruned\n+by the evolve command, and removed from the repository's history. Any follow-up\n+changes should rebased onto the parent of the pruned commit. The abandoned\n+parent points to the version of the change that should be restored if the user\n+attempts to restore the change.\n+\n+Changes\n+-------\n+A branch of meta-commits describes how a commit was produced and what previous\n+commits it is based on. It is also an identifier for a thing the user is\n+currently working on. We refer to such a meta-branch as a change.\n+\n+Local changes are stored in the new refs/metas namespace. Remote changes are\n+stored in the refs/remote//metas namespace.\n+\n+The list of changes in refs/metas is more than just a mechanism for the evolve\n+command to locate obsolete commits. It is also a convenient list of all of a\n+user’s work in progress and their current state - a list of things they’re\n+likely to want to come back to.\n+\n+Strictly speaking, it is the presence of the branch in the refs/metas namespace\n+that marks a branch as being a change, not the fact that it points to a\n+metacommit. Metacommits are only created when a commit is amended or rebased, so\n+in the case where a change points to a commit that has never been modified, the\n+change points to that initial commit rather than a metacommit.\n+\n+Changes are also stored in the refs/hiddenmetas namespace. Hiddenmetas holds\n+metadata for historical changes that are not currently in progress by the user.\n+Commands like filter-branch and other bulk import commands create metadata in\n+this namespace.\n+\n+Note that the changes in hiddenmetas get special treatment in several ways:\n+\n+- They are not cleaned up automatically once merged, since it is expected that\n+ they refer to historical changes.\n+- User commands that modify changes don't append to these changes as they would\n+ to a change in refs/metas.\n+- They are not displayed when the user lists their local changes.\n+\n+Obsolescence\n+------------\n+A commit is considered obsolete if it is reachable from the “replaces” edges\n+anywhere in the history of a change and it isn’t the head of that change.\n+Commits may be the content for 0 or more meta-commits. If the same commit\n+appears in multiple changes, it is not obsolete if it is the head of any of\n+those changes.\n+\n+Note that there is an exception to this rule. The metas namespace takes\n+precedence over the hiddenmetas namespace for the purpose of obsolescence. That\n+is, if a change appears in a replaces edge of a change in the metas namespace,\n+it is obsolete even if it also appears as the head of a change in the\n+hiddenmetas namespace.\n+\n+This special case prevents the hiddenmetas namespace from creating divergence\n+with the user's work in progress, and allows the user to resolve historical\n+divergence by creating new changes in the metas namespace.\n+\n+Divergence\n+----------\n+From the user’s perspective, two changes are divergent if they both ask for\n+different replacements to the same commit. More precisely, a target commit is\n+considered divergent if there is more than one commit at the head of a change in\n+refs/metas that leads to the target commit via an unbroken chain of “replaces”\n+parents.\n+\n+Much like a merge conflict, divergence is a situation that requires user\n+intervention to resolve. The evolve command will stop when it encounters\n+divergence and prompt the user to resolve the problem. Users can solve the\n+problem in several ways:\n+\n+- Discard one of the changes (by deleting its change branch).\n+- Merge the two changes (producing a single change branch).\n+- Copy one of the changes (keep both commits, but one of them gets a new\n+ metacommit appended to its history that is connected to its predecessor via an\n+ origin edge rather than a replaces edge. That new change no longer obsoletes\n+ the original.)\n+\n+Obsolescence across cherry-picks\n+--------------------------------\n+By default the evolve command will treat cherry-picks and squash merges as being\n+completely separate from the original. Further amendments to the original commit\n+will have no effect on the cherry-picked copy. However, this behavior may not be\n+desirable in all circumstances.\n+\n+The evolve command may at some point support an option to look for cases where\n+the source of a cherry-pick or squash merge has itself been amended, and\n+automatically apply that same change to the cherry-picked copy. In such cases,\n+it would traverse origin edges rather than ignoring them, and would treat a\n+commit with origin edges as being obsolete if any of its origins were obsolete.\n+\n+Garbage collection\n+------------------\n+For GC purposes, meta-commits are normal commits. Just as a commit causes its\n+parents and tree to be retained, a meta-commit also causes its parents to be\n+retained.\n+\n+Change creation\n+---------------\n+Changes are created automatically whenever the user runs a command like “commit”\n+that has the semantics of creating a new change. They also move forward\n+automatically even if they’re not checked out. For example, whenever the user\n+runs a command like “commit --amend” that modifies a commit, all branches in\n+refs/metas that pointed to the old commit move forward to point to its\n+replacement instead. This also happens when the user is working from a detached\n+head.\n+\n+This does not mean that every commit has a corresponding change. By default,\n+changes only exist for recent locally-created commits. Users may explicitly pull\n+changes from other users or keep their changes around for a long time, but\n+either behavior requires a user to opt-in. Code review systems like gerrit may\n+also choose to keep changes around forever.\n+\n+Note that the changes in refs/metas serve a dual function as both a way to\n+identify obsolete changes and as a way for the user to keep track of their work\n+in progress. If we were only concerned with identifying obsolete changes, it\n+would be sufficient to create the change branch lazily the first time a commit\n+is obsoleted. Addressing the second use - of refs/metas as a mechanism for\n+keeping track of work in progress - is the reason for eagerly creating the\n+change on first commit.\n+\n+Change naming\n+-------------\n+When a change is first created, the only requirement for its name is that it\n+must be unique. Good names would also serve as useful mnemonics and be easy to\n+type. For example, a short word from the commit message containing no numbers or\n+special characters and that shows up with low frequency in other commit messages\n+would make a good choice.\n+\n+Different users may prefer different heuristics for their change names. For this\n+reason a new hook will be introduced to compute change names. Git will invoke\n+the hook for all newly-created changes and will append a numeric suffix if the\n+name isn’t unique. The default heuristics are not specified by this proposal and\n+may change during implementation.\n+\n+Change deletion\n+---------------\n+Changes are normally only interesting to a user while a commit is still in\n+development and under review. Once the commit has submitted wherever it is\n+going, its change can be discarded.\n+\n+The normal way of deleting changes makes this easy to do - changes are deleted\n+by the evolve command when it detects that the change is present in an upstream\n+branch. It does this in two ways: if the latest commit in a change either shows\n+up in the branch history or the change becomes empty after a rebase, it is\n+considered merged and the change is discarded. In this context, an “upstream\n+branch” is any branch passed in as the upstream argument of the evolve command.\n+\n+In case this sometimes deletes a useful change, such automatic deletions are\n+recorded in the reflog allowing them to be easily recovered.\n+\n+Sharing changes\n+---------------\n+Change histories are shared by pushing or fetching meta-commits and change\n+branches. This provides users with a lot of control of what to share and\n+repository implementations with control over what to retain.\n+\n+Users that only want to share the content of a commit can do so by pushing the\n+commit itself as they currently would. Users that want to share an edit history\n+for the commit can push its change, which would point to a meta-commit rather\n+than the commit itself if there is any history to share. Note that multiple\n+changes can refer to the same commits, so it’s possible to construct and push a\n+different history for the same commit in order to remove sensitive or irrelevant\n+intermediate states.\n+\n+Imagine the user is working on a change “mychange” that is currently the latest\n+commit on master. They have two ways to share it:\n+\n+# User shares just a commit without its history\n+> git push origin master\n+\n+# User shares the full history of the commit to a review system\n+> git push origin metas/mychange:refs/for/master\n+\n+# User fetches a collaborator’s modifications to their change\n+> git fetch remotename metas/mychange\n+# Which updates the ref remote/remotename/metas/mychange\n+\n+This will cause more intermediate states to be shared with the server than would\n+have been shared previously. A review system like gerrit would need to keep\n+track of which states had been explicitly pushed versus other intermediate\n+states in order to de-emphasize (or hide) the extra intermediate states from the\n+user interface.\n+\n+Merge-base\n+----------\n+Merge-base will be changed to search the meta-commit graph for common ancestors\n+as well as the commit graph, and will generally prefer results from the\n+meta-commit graph over the commit graph. Merge-base will consider meta-commits\n+from all changes, and will traverse both origin and obsolete edges.\n+\n+The reason for this is that - when merging two versions of the same commit\n+together - an earlier version of that same commit will usually be much more\n+similar than their common parent. This should make the workflow of collaborating\n+on unsubmitted patches as convenient as the workflow for collaborating in a\n+topic branch by eliminating repeated merges.\n+\n+Configuration\n+-------------\n+The core.enableChanges configuration variable enables the creation and update\n+of change branches. This is enabled by default.\n+\n+User interface\n+--------------\n+All git porcelain commands that create commits are classified as having one of\n+four behaviors: modify, create, copy, or import. These behaviors are discussed\n+in more detail below.\n+\n+Modify commands\n+---------------\n+Modification commands (commit --amend, rebase) will mark the old commit as\n+obsolete by creating a new meta-commit that references the old one as a\n+replaced parent. In the event that multiple changes point to the same commit,\n+this is done independently for every such change.\n+\n+More specifically, modifications work like this:\n+\n+1. Locate all existing changes for which the old commit is the content for the\n+ head of the change branch. If no such branch exists, create one that points\n+ to the old commit. Changes that include this commit in their history but not\n+ at their head are explicitly not included.\n+2. For every such change, create a new meta-commit that references the new\n+ commit as its content and references the old head of the change as a\n+ replaced parent.\n+3. Move the change branch forward to point to the new meta-commit.\n+\n+Copy commands\n+-------------\n+Copy commands (cherry-pick, merge --squash) create a new meta-commit that\n+references the old commits as origin parents. Besides the fact that the new\n+parents are tagged differently, copy commands work the same way as modify\n+commands.\n+\n+Create commands\n+---------------\n+Creation commands (commit, merge) create a new commit and a new change that\n+points to that commit. The do not create any meta-commits.\n+\n+Import commands\n+---------------\n+Import commands (fetch, pull) do not create any new meta-commits or changes\n+unless that is specifically what they are importing. For example, the fetch\n+command would update remote/origin/metas/change35 and fetch all referenced\n+meta-commits if asked to do so directly, but it wouldn’t create any changes or\n+meta-commits for commits discovered on the master branch when running “git fetch\n+origin master”.\n+\n+Other commands\n+--------------\n+Some commands don’t fit cleanly into one of the above categories.\n+\n+Semantically, filter-branch should be treated as a modify command, but doing so\n+is likely to create a lot of irrelevant clutter in the changes namespace and the\n+large number of extra change refs may introduce performance problems. We\n+recommend treating filter-branch as an import command initially, but making it\n+behave more like a modify command in future follow-up work. One possible\n+solution may be to treat commits that are part of existing changes as being\n+modified but to avoid creating changes for other rewritten changes. Another\n+solution may be to record the modifications as changes in the hiddenmetas\n+namespace.\n+\n+Once the evolve command can handle obsolescence across cherry-picks, such\n+cherry-picks will result in a hybrid move-and-copy operation. It will create\n+cherry-picks that replace other cherry-picks, which will have both origin edges\n+(pointing to the new source commit being picked) and replacement edges (pointing\n+to the previous cherry-pick being replaced).\n+\n+Evolve\n+------\n+The evolve command performs the correct sequence of rebases such that no change\n+has an obsolete parent. The syntax looks like this:\n+\n+git evolve [upstream…]\n+\n+It takes an optional list of upstream branches. All changes whose parent shows\n+up in the history of one of the upstream branches will be rebased onto the\n+upstream branch before resolving obsolete parents.\n+\n+Any change whose latest state is found in an upstream branch (or that ends up\n+empty after rebase) will be deleted. This is the normal mechanism for deleting\n+changes. Changes are created automatically on the first commit, and are deleted\n+automatically when evolve determines that they’ve been merged upstream.\n+\n+Orphan commits are commits with obsolete parents. The evolve command then\n+repeatedly rebases orphan commits with non-orphan parents until there are either\n+no orphan commits left, or a merge conflict is discovered. It will also\n+terminate if it detects a divergent parent or a cycle that can't be resolved\n+using any of the enabled transformations.\n+\n+When evolve discovers divergence, it will first check if it can resolve the\n+divergence automatically using one of its enabled transformations. Supported\n+transformations are:\n+\n+- Check if the user has already merged the divergent changes in a follow-up\n+ change. That is, look for an existing merge in a follow-up change where all\n+ the parents are divergent versions of the same change. Squash that merge with\n+ its parents and use the result as the resolution for the divergence.\n+\n+- Attempt to auto-merge all the divergent changes (disabled by default).\n+\n+Each of the transformations can be enabled or disabled by command line options.\n+\n+Cycles can occur when two changes reference one another as parents. This can\n+happen when both changes use an obsolete version of the other change as their\n+parent. Although there are never cycles in the commit graph, users can create\n+cycles in the change graph by rebasing changes onto obsolete commits. The evolve\n+command has a transformation that will detect and break cycles by arbitrarily\n+picking one of the changes to go first. If this generates a merge conflict,\n+it tries each of the other changes in sequence to see if any ordering merges\n+cleanly. If no possible ordering merges cleanly, it picks one and terminates\n+to let the user resolve the merge conflict.\n+\n+If the working tree is dirty, evolve will attempt to stash the user's changes\n+before applying the evolve and then reapply those changes afterward, in much\n+the same way as rebase --autostash does.\n+\n+Checkout\n+--------\n+Running checkout on a change by name has the same effect as checking out a\n+detached head pointing to the latest commit on that change-branch. There is no\n+need to ever have HEAD point to a change since changes always move forward when\n+necessary, no matter what branch the user has checked out\n+\n+Meta-commits themselves cannot be checked out by their hash.\n+\n+Reset\n+-----\n+Resetting a branch to a change by name is the same as resetting to the content\n+(or abandoned) commit at that change’s head.\n+\n+Commit\n+------\n+Commit --amend gets modify semantics and will move existing changes forward. The\n+normal form of commit gets create semantics and will create a new change.\n+\n+$ touch foo && git add . && git commit -m \"foo\" && git tag A\n+$ touch bar && git add . && git commit -m \"bar\" && git tag B\n+$ touch baz && git add . && git commit -m \"baz\" && git tag C\n+\n+This produces the following commits:\n+A(tree=[foo])\n+B(tree=[foo, bar], parent=A)\n+C(tree=[foo, bar, baz], parent=B)\n+\n+...along with three changes:\n+metas/foo = A\n+metas/bar = B\n+metas/baz = C\n+\n+Running commit --amend does the following:\n+$ git checkout B\n+$ touch zoom && git add . && git commit --amend -m \"baz and zoom\"\n+$ git tag D\n+\n+Commits:\n+A(tree=[foo])\n+B(tree=[foo, bar], parent=A)\n+C(tree=[foo, bar, baz], parent=B)\n+D(tree=[foo, bar, zoom], parent=A)\n+Dmeta(content=D, obsolete=B)\n+\n+Changes:\n+metas/foo = A\n+metas/bar = Dmeta\n+metas/baz = C\n+\n+Merge\n+-----\n+Merge gets create, modify, or copy semantics based on what is being merged and\n+the options being used.\n+\n+The --squash version of merge gets copy semantics (it produces a new change that\n+is marked as a copy of all the original changes that were squashed into it).\n+\n+The “modify” version of merge replaces both of the original commits with the\n+resulting merge commit. This is one of the standard mechanisms for resolving\n+divergence. The parents of the merge commit are the parents of the two commits\n+being merged. The resulting commit will not be a merge commit if both of the\n+original commits had the same parent or if one was the parent of the other.\n+\n+The “create” version of merge creates a new change pointing to a merge commit\n+that has both original commits as parents. The result is what merge produces now\n+- a new merge commit. However, this version of merge doesn’t directly resolve\n+divergence.\n+\n+To select between these two behaviors, merge gets new “--amend” and “--noamend”\n+options which select between the “create” and “modify” behaviors respectively,\n+with noamend being the default.\n+\n+For example, imagine we created two divergent changes like this:\n+\n+$ touch foo && git add . && git commit -m \"foo\" && git tag A\n+$ touch bar && git add . && git commit -m \"bar\" && git tag B\n+$ touch baz && git add . && git commit --amend -m \"bar and baz\"\n+$ git tag C\n+$ git checkout B\n+$ touch bam && git add . && git commit --amend -m \"bar and bam\"\n+$ git tag D\n+\n+At this point the commit graph looks like this:\n+\n+A(tree=[foo])\n+B(tree=[bar], parent=A)\n+C(tree=[bar, baz], parent=A)\n+D(tree=[bar, bam], parent=A)\n+Cmeta(content=C, obsoletes=B)\n+Dmeta(content=D, obsoletes=B)\n+\n+There would be three active changes with heads pointing as follows:\n+\n+metas/changeA=A\n+metas/changeB=Cmeta\n+metas/changeB2=Dmeta\n+\n+ChangeB and changeB2 are divergent at this point. Lets consider what happens if\n+perform each type of merge between changeB and changeB2.\n+\n+Merge example: Amend merge\n+One way to resolve divergent changes is to use an amend merge. Recall that HEAD\n+is currently pointing to D at this point.\n+\n+$ git merge --amend metas/changeB\n+\n+Here we’ve asked for an amend merge since we’re trying to resolve divergence\n+between two versions of the same change. There are no conflicts so we end up\n+with this:\n+\n+E(tree=[bar, baz, bam], parent=A)\n+Emeta(content=E, obsoletes=[Cmeta, Dmeta])\n+\n+With the following branches:\n+\n+metas/changeA=A\n+metas/changeB=Emeta\n+metas/changeB2=Emeta\n+\n+Notice that the result of the “amend merge” is a replacement for C and D rather\n+than a new commit with C and D as parents (as a normal merge would have\n+produced). The parents of the amend merge are the parents of C and D which - in\n+this case - is just A, so the result is not a merge commit. Also notice that\n+changeB and changeB2 are now aliases for the same change.\n+\n+Merge example: Noamend merge\n+Consider what would have happened if we’d used a noamend merge instead. Recall\n+that HEAD was at D and our branches looked like this:\n+\n+metas/changeA=A\n+metas/changeB=Cmeta\n+metas/changeB2=Dmeta\n+\n+$ git merge --noamend metas/changeB\n+\n+That would produce the sort of merge we’d normally expect today:\n+\n+F(tree=[bar, baz, bam], parent=[C, D])\n+\n+And our changes would look like this:\n+metas/changeA=A\n+metas/changeB=Cmeta\n+metas/changeB2=Dmeta\n+metas/changeF=F\n+\n+In this case, changeB and changeB2 are still divergent and we’ve created a new\n+change for our merge commit. However, this is just a temporary state. The next\n+time we run the “evolve” command, it will discover the divergence but also\n+discover the merge commit F that resolves it. Evolve will suggest converting F\n+into an amend merge in order to resolve the divergence and will display the\n+command for doing so.\n+\n+Rebase\n+------\n+In general the rebase command is treated as a modify command. When a change is\n+rebased, the new commit replaces the original.\n+\n+Rebase --abort is special. Its intent is to restore git to the state it had\n+prior to running rebase. It should move back any changes to point to the refs\n+they had prior to running rebase and delete any new changes that were created as\n+part of the rebase. To achieve this, rebase will save the state of all changes\n+in refs/metas prior to running rebase and will restore the entire namespace\n+after rebase completes (deleting any newly-created changes). Newly-created\n+metacommits are left in place, but will have no effect until garbage collected\n+since metacommits are only used if they are reachable from refs/metas.\n+\n+Change\n+------\n+The “change” command can be used to list, rename, reset or delete change. It has\n+a number of subcommands.\n+\n+The \"list\" subcommand lists local changes. If given the -r argument, it lists\n+remote changes.\n+\n+The \"rename\" subcommand renames a change, given its old and new name. If the old\n+name is omitted and there is exactly one change pointing to the current HEAD,\n+that change is renamed. If there are no changes pointing to the current HEAD,\n+one is created with the given name.\n+\n+The \"forget\" subcommand deletes a change by deleting its ref from the metas/\n+namespace. This is the normal way to delete extra aliases for a change if the\n+change has more than one name. By default, this will refuse to delete the last\n+alias for a change if there are any other changes that reference this change as\n+a parent.\n+\n+The \"update\" subcommand adds a new state to a change. It uses the default\n+algorithm for assigning change names. If the content commit is omitted, HEAD is\n+used. If given the optional --force argument, it will overwrite any existing\n+change of the same name. This latter form of \"update\" can be used to effectively\n+reset changes.\n+\n+The \"update\" command can accept any number of --origin and --replace arguments.\n+If any are present, the resulting change branch will point to a metacommit\n+containing the given origin and replacement edges.\n+\n+The \"abandon\" command deletes a change using obsolescence markers. It marks the\n+change as being obsolete and having been replaced by its parent. If given no\n+arguments, it applies to the current commit. Running evolve will cause any\n+abandoned changes to be removed from the branch. Any child changes will be\n+reparented on top of the parent of the abandoned change. If the current change\n+is abandoned, HEAD will move to point to its parent.\n+\n+The \"restore\" command restores a previously-abandoned change.\n+\n+The \"prune\" command deletes all obsolete changes and all changes that are\n+present in the given branch. Note that such changes can be recovered from the\n+reflog.\n+\n+Combined with the GC protection that is offered, this is intended to facilitate\n+a workflow that relies on changes instead of branches. Users could choose to\n+work with no local branches and use changes instead - both for mailing list and\n+gerrit workflows.\n+\n+Log\n+---\n+When a commit is shown in git log that is part of a change, it is decorated with\n+extra change information. If it is the head of a change, the name of the change\n+is shown next to the list of branches. If it is obsolete, it is decorated with\n+the text “obsolete, commits behind ”.\n+\n+Log gets a new --obslog argument indicating that the obsolescence graph should\n+be followed instead of the commit graph. This also changes the default\n+formatting options to make them more appropriate for viewing different\n+iterations of the same commit.\n+\n+Pull\n+----\n+\n+Pull gets an --evolve argument that will automatically attempt to run \"evolve\"\n+on any affected branches after pulling.\n+\n+We also introduce an \"evolve\" enum value for the branch..rebase config\n+value. When set, the evolve behavior will happen automatically for that branch\n+after every pull even if the --evolve argument is not used.\n+\n+Next\n+----\n+\n+The \"next\" command will reset HEAD to a non-obsolete commit that refers to this\n+change as its parent. If there is more than one such change, the user will be\n+prompted. If given the --evolve argument, the next commit will be evolved if\n+necessary first.\n+\n+The \"next\" command can be thought of as the opposite of\n+\"git reset --hard HEAD^\" in that it navigates to a child commit rather than a\n+parent.\n+\n+Prev\n+----\n+\n+The \"prev\" command will reset HEAD to the latest version of the parent change.\n+If the parent change isn't obsolete, this is equivalent to\n+\"git reset --hard HEAD^\". If the parent commit is obsolete, it resets to the\n+latest replacement for the parent commit.\n+\n+Other options considered\n+========================\n+We considered several other options for storing the obsolescence graph. This\n+section describes the other options and why they were rejected.\n+\n+Commit header\n+-------------\n+Add an “obsoletes” field to the commit header that points backwards from a\n+commit to the previous commits it obsoletes.\n+\n+Pros:\n+- Very simple\n+- Easy to traverse from a commit to the previous commits it obsoletes.\n+Cons:\n+- Adds a cost to the storage format, even for commits where the change history\n+ is uninteresting.\n+- Unconditionally prevents the change history from being garbage collected.\n+- Always causes the change history to be shared when pushing or pulling changes.\n+\n+Git notes\n+---------\n+Instead of storing obsolescence information in metacommits, the metacommit\n+content could go in a new notes namespace - say refs/notes/metacommit. Each note\n+would contain the list of obsolete and origin parents. An automerger could\n+be supplied to make it easy to merge the metacommit notes from different remotes.\n+\n+Pros:\n+- Easy to locate all commits obsoleted by a given commit (since there would only\n+ be one metacommit for any given commit).\n+Cons:\n+- Wrong GC behavior (obsolete commits wouldn’t automatically be retained by GC)\n+ unless we introduced a special case for these kinds of notes.\n+- No way to selectively share or pull the metacommits for one specific change.\n+ It would be all-or-nothing, which would be expensive. This could be addressed\n+ by changes to the protocol, but this would be invasive.\n+- Requires custom auto-merging behavior on fetch.\n+\n+Tags\n+----\n+Put the content of the metacommit in a message attached to tag on the\n+replacement commit. This is very similar to the git notes approach and has the\n+same pros and cons.\n+\n+Simple forward references\n+-------------------------\n+Record an edge from an obsolete commit to its replacement in this form:\n+\n+refs/obsoletes/\n+\n+pointing to commit as an indication that B is the replacement for the\n+obsolete commit A.\n+\n+Pros:\n+- Protects from being garbage collected.\n+- Fast lookup for the evolve operation, without additional search structures\n+ (“what is the replacement for ?” is very fast).\n+\n+Cons:\n+- Can’t represent divergence (which is a P0 requirement).\n+- Creates lots of refs (which can be inefficient)\n+- Doesn’t provide a way to fetch only refs for a specific change.\n+- The obslog command requires a search of all refs.\n+\n+Complex forward references\n+--------------------------\n+Record an edge from an obsolete commit to its replacement in this form:\n+\n+refs/obsoletes//obs_\n+\n+Pointing to commit as an indication that B is the replacement for obsolete\n+commit A.\n+\n+Pros:\n+- Permits sharing and fetching refs for only a specific change.\n+- Supports divergence\n+- Protects from being garbage collected.\n+\n+Cons:\n+- Creates lots of refs, which is inefficient.\n+- Doesn’t provide a good lookup structure for lookups in either direction.\n+\n+Backward references\n+-------------------\n+Record an edge from a replacement commit to the obsolete one in this form:\n+\n+refs/obsolescences/\n+\n+Cons:\n+- Doesn’t provide a way to resolve divergence (which is a P0 requirement).\n+- Doesn’t protect from being garbage collected (which could be fixed by\n+ combining this with a refs/metas namespace, as in the metacommit variant).\n+\n+Obsolescences file\n+------------------\n+Create a custom file (or files) in .git recording obsolescences.\n+\n+Pros:\n+- Can store exactly the information we want with exactly the performance we want\n+ for all operations. For example, there could be a disk-based hashtable\n+ permitting constant time lookups in either direction.\n+\n+Cons:\n+- Handling GC, pushing, and pulling would all require custom solutions. GC\n+ issues could be addressed with a repository format extension.\n+\n+Squash points\n+-------------\n+We treat changes like topic branches, and use special squash points to mark\n+places in the commit graph that separate changes.\n+\n+We create and update change branches in refs/metas at the same time we\n+would have in the metacommit proposal. However, rather than pointing to a\n+metacommit branch they point to normal commits and are treated as “squash\n+points” - markers for sequences of commits intended to be squashed together on\n+submission.\n+\n+Amends and rebases work differently than they do now. Rather than actually\n+containing the desired state of a commit, they contain a delta from the previous\n+version along with a squash point indicating that the preceding changes are\n+intended to be squashed on submission. Specifically, amends would become new\n+changes and rebases would become merge commits with the old commit and new\n+parent as parents.\n+\n+When the changes are finally submitted, the squashes are executed, producing the\n+final version of the commit.\n+\n+In addition to the squash points, git would maintain a set of “nosquash” tags\n+for commits that were used as ancestors of a change that are not meant to be\n+included in the squash.\n+\n+For example, if we have this commit graph:\n+\n+A(...)\n+B(parent=A)\n+C(parent=B)\n+\n+...and we amend B to produce D, we’d get:\n+\n+A(...)\n+B(parent=A)\n+C(parent=B)\n+D(parent=B)\n+\n+...along with a new change branch indicating D should be squashed with its\n+parents when submitted:\n+\n+metas/changeB = D\n+metas/changeC = C\n+\n+We’d also create a nosquash tag for A indicating that A shouldn’t be included\n+when changeB is squashed.\n+\n+If a user amends the change again, they’d get:\n+\n+A(...)\n+B(parent=A)\n+C(parent=B)\n+D(parent=B)\n+E(parent=D)\n+\n+metas/changeB = E\n+metas/changeC = C\n+\n+Pros:\n+- Good GC behavior.\n+- Provides a natural way to share changes (they’re just normal branches).\n+- Merge-base works automatically without special cases.\n+- Rewriting the obslog would be easy using existing git commands.\n+- No new data types needed.\n+Cons:\n+- No way to connect the squashed version of a change to the original, so no way\n+ to automatically clean up old changes. This also means users lose all benefits\n+ of the evolve command if they prematurely squash their commits. This may occur\n+ if a user thinks a change is ready for submission, squashes it, and then later\n+ discovers an additional change to make.\n+- Histories would look very cluttered (users would see all previous edits to\n+ their commit in the commit log, and all previous rebases would show up as\n+ merges). Could be quite hard for users to tell what is going on. (Possible\n+ fix: also implement a new smart log feature that displays the log as though\n+ the squashes had occurred).\n+- Need to change the current behavior of current commands (like amend and\n+ rebase) in ways that will be unexpected to many users.\n-- \ngitgitgadget\n\n"},{"id":"463537","messageId":"2b3a00a6702eb8fb12e45b833ca74155939588ef.1663959325.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 05/10] evolve: add the change-table structure","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:19Z","receivedAt":"2022-09-23T18:56:00Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nA change table stores a list of changes, and supports efficient lookup\nfrom a commit hash to the list of changes that reference that commit\ndirectly.\n\nIt can be used to look up content commits or metacommits at the head\nof a change, but does not support lookup of commits referenced as part\nof the commit history.\n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n Makefile | 1 +\n change-table.c | 179 +++++++++++++++++++++++++++++++++++++++++++++++++\n change-table.h | 132 ++++++++++++++++++++++++++++++++++++\n 3 files changed, 312 insertions(+)\n create mode 100644 change-table.c\n create mode 100644 change-table.h\n\ndiff --git a/Makefile b/Makefile\nindex b2bcc00c289..2b847e7e7de 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -913,6 +913,7 @@ LIB_OBJS += bulk-checkin.o\n LIB_OBJS += bundle-uri.o\n LIB_OBJS += bundle.o\n LIB_OBJS += cache-tree.o\n+LIB_OBJS += change-table.o\n LIB_OBJS += cbtree.o\n LIB_OBJS += chdir-notify.o\n LIB_OBJS += checkout.o\ndiff --git a/change-table.c b/change-table.c\nnew file mode 100644\nindex 00000000000..c61ba29f1ed\n--- /dev/null\n+++ b/change-table.c\n@@ -0,0 +1,179 @@\n+#include \"cache.h\"\n+#include \"change-table.h\"\n+#include \"commit.h\"\n+#include \"ref-filter.h\"\n+#include \"metacommit-parser.h\"\n+\n+void change_table_init(struct change_table *to_initialize)\n+{\n+\tmemset(to_initialize, 0, sizeof(*to_initialize));\n+\tmem_pool_init(&to_initialize->memory_pool, 0);\n+\tto_initialize->memory_pool.block_alloc = 4*1024 - sizeof(struct mp_block);\n+\toidmap_init(&to_initialize->oid_to_metadata_index, 0);\n+\tstring_list_init_dup(&to_initialize->refname_to_change_head);\n+}\n+\n+static void change_list_clear(struct change_list *to_clear) {\n+\tstring_list_clear(&to_clear->additional_refnames, 0);\n+}\n+\n+static void commit_change_list_entry_clear(\n+\tstruct commit_change_list_entry *to_clear) {\n+\tchange_list_clear(&to_clear->changes);\n+}\n+\n+void change_table_clear(struct change_table *to_clear)\n+{\n+\tstruct oidmap_iter iter;\n+\tstruct commit_change_list_entry *next;\n+\tfor (next = oidmap_iter_first(&to_clear->oid_to_metadata_index, &iter);\n+\t\tnext;\n+\t\tnext = oidmap_iter_next(&iter)) {\n+\n+\t\tcommit_change_list_entry_clear(next);\n+\t}\n+\n+\toidmap_free(&to_clear->oid_to_metadata_index, 0);\n+\tstring_list_clear(&to_clear->refname_to_change_head, 0);\n+\tmem_pool_discard(&to_clear->memory_pool, 0);\n+}\n+\n+static void add_head_to_commit(struct change_table *to_modify,\n+\tconst struct object_id *to_add, const char *refname)\n+{\n+\tstruct commit_change_list_entry *entry;\n+\n+\t/**\n+\t * Note: the indices in the map are 1-based. 0 is used to indicate a missing\n+\t * element.\n+\t */\n+\tentry = oidmap_get(&to_modify->oid_to_metadata_index, to_add);\n+\tif (!entry) {\n+\t\tentry = mem_pool_calloc(&to_modify->memory_pool, 1,\n+\t\t\tsizeof(*entry));\n+\t\toidcpy(&entry->entry.oid, to_add);\n+\t\toidmap_put(&to_modify->oid_to_metadata_index, entry);\n+\t\tstring_list_init_nodup(&entry->changes.additional_refnames);\n+\t}\n+\n+\tif (!entry->changes.first_refname)\n+\t\tentry->changes.first_refname = refname;\n+\telse\n+\t\tstring_list_insert(&entry->changes.additional_refnames, refname);\n+}\n+\n+void change_table_add(struct change_table *to_modify, const char *refname,\n+\tstruct commit *to_add)\n+{\n+\tstruct change_head *new_head;\n+\tstruct string_list_item *new_item;\n+\tint metacommit_type;\n+\n+\tnew_head = mem_pool_calloc(&to_modify->memory_pool, 1,\n+\t\tsizeof(*new_head));\n+\n+\toidcpy(&new_head->head, &to_add->object.oid);\n+\n+\tmetacommit_type = get_metacommit_content(to_add, &new_head->content);\n+\tif (metacommit_type == METACOMMIT_TYPE_NONE)\n+\t\toidcpy(&new_head->content, &to_add->object.oid);\n+\tnew_head->abandoned = (metacommit_type == METACOMMIT_TYPE_ABANDONED);\n+\tnew_head->remote = starts_with(refname, \"refs/remote/\");\n+\tnew_head->hidden = starts_with(refname, \"refs/hiddenmetas/\");\n+\n+\tnew_item = string_list_insert(&to_modify->refname_to_change_head, refname);\n+\tnew_item->util = new_head;\n+\t/* Use pointers to the copy of the string we're retaining locally */\n+\trefname = new_item->string;\n+\n+\tif (!oideq(&new_head->content, &new_head->head))\n+\t\tadd_head_to_commit(to_modify, &new_head->content, refname);\n+\tadd_head_to_commit(to_modify, &new_head->head, refname);\n+}\n+\n+void change_table_add_all_visible(struct change_table *to_modify,\n+\tstruct repository* repo)\n+{\n+\tstruct ref_filter filter;\n+\tconst char *name_patterns[] = {NULL};\n+\tmemset(&filter, 0, sizeof(filter));\n+\tfilter.kind = FILTER_REFS_CHANGES;\n+\tfilter.name_patterns = name_patterns;\n+\n+\tchange_table_add_matching_filter(to_modify, repo, &filter);\n+}\n+\n+void change_table_add_matching_filter(struct change_table *to_modify,\n+\tstruct repository* repo, struct ref_filter *filter)\n+{\n+\tstruct ref_array matching_refs;\n+\tint i;\n+\n+\tmemset(&matching_refs, 0, sizeof(matching_refs));\n+\tfilter_refs(&matching_refs, filter, filter->kind);\n+\n+\t/**\n+\t * Determine the object id for the latest content commit for each change.\n+\t * Fetch the commit at the head of each change ref. If it's a normal commit,\n+\t * that's the commit we want. If it's a metacommit, locate its content parent\n+\t * and use that.\n+\t */\n+\n+\tfor (i = 0; i < matching_refs.nr; i++) {\n+\t\tstruct ref_array_item *item = matching_refs.items[i];\n+\t\tstruct commit *commit = item->commit;\n+\n+\t\tcommit = lookup_commit_reference_gently(repo, &item->objectname, 1);\n+\n+\t\tif (commit)\n+\t\t\tchange_table_add(to_modify, item->refname, commit);\n+\t}\n+\n+\tref_array_clear(&matching_refs);\n+}\n+\n+static int return_true_callback(const char *refname, void *cb_data)\n+{\n+\treturn 1;\n+}\n+\n+int change_table_has_change_referencing(struct change_table *changes,\n+\tconst struct object_id *referenced_commit_id)\n+{\n+\treturn for_each_change_referencing(changes, referenced_commit_id,\n+\t\treturn_true_callback, NULL);\n+}\n+\n+int for_each_change_referencing(struct change_table *table,\n+\tconst struct object_id *referenced_commit_id, each_change_fn fn, void *cb_data)\n+{\n+\tconst struct change_list *changes;\n+\tint i;\n+\tint retvalue;\n+\tstruct commit_change_list_entry *entry;\n+\n+\tentry = oidmap_get(&table->oid_to_metadata_index,\n+\t\treferenced_commit_id);\n+\t/* If this commit isn't referenced by any changes, it won't be in the map */\n+\tif (!entry)\n+\t\treturn 0;\n+\tchanges = &entry->changes;\n+\tif (!changes->first_refname)\n+\t\treturn 0;\n+\tretvalue = fn(changes->first_refname, cb_data);\n+\tfor (i = 0; retvalue == 0 && i < changes->additional_refnames.nr; i++)\n+\t\tretvalue = fn(changes->additional_refnames.items[i].string, cb_data);\n+\treturn retvalue;\n+}\n+\n+struct change_head* get_change_head(struct change_table *heads,\n+\tconst char* refname)\n+{\n+\tstruct string_list_item *item = string_list_lookup(\n+\t\t&heads->refname_to_change_head, refname);\n+\n+\tif (!item)\n+\t\treturn NULL;\n+\n+\treturn (struct change_head *)item->util;\n+}\ndiff --git a/change-table.h b/change-table.h\nnew file mode 100644\nindex 00000000000..166b5ed8073\n--- /dev/null\n+++ b/change-table.h\n@@ -0,0 +1,132 @@\n+#ifndef CHANGE_TABLE_H\n+#define CHANGE_TABLE_H\n+\n+#include \"oidmap.h\"\n+\n+struct commit;\n+struct ref_filter;\n+\n+/**\n+ * This struct holds a list of change refs. The first element is stored inline,\n+ * to optimize for small lists.\n+ */\n+struct change_list {\n+\t/**\n+\t * Ref name for the first change in the list, or null if none.\n+\t *\n+\t * This field is private. Use for_each_change_in to read.\n+\t */\n+\tconst char* first_refname;\n+\t/**\n+\t * List of additional change refs. Note that this is empty if the list\n+\t * contains 0 or 1 elements.\n+\t *\n+\t * This field is private. Use for_each_change_in to read.\n+\t */\n+\tstruct string_list additional_refnames;\n+};\n+\n+/**\n+ * Holds information about the head of a single change.\n+ */\n+struct change_head {\n+\t/**\n+\t * The location pointed to by the head of the change. May be a commit or a\n+\t * metacommit.\n+\t */\n+\tstruct object_id head;\n+\t/**\n+\t * The content commit for the latest commit in the change. Always points to a\n+\t * real commit, never a metacommit.\n+\t */\n+\tstruct object_id content;\n+\t/**\n+\t * Abandoned: indicates that the content commit should be removed from the\n+\t * history.\n+\t *\n+\t * Hidden: indicates that the change is an inactive change from the\n+\t * hiddenmetas namespace. Such changes will be hidden from the user by\n+\t * default.\n+\t *\n+\t * Deleted: indicates that the change has been removed from the repository.\n+\t * That is the ref was deleted since the time this struct was created. Such\n+\t * entries should be ignored.\n+\t */\n+\tunsigned int abandoned:1,\n+\t\thidden:1,\n+\t\tremote:1,\n+\t\tdeleted:1;\n+};\n+\n+/**\n+ * Holds the list of change refs whose content points to a particular content\n+ * commit.\n+ */\n+struct commit_change_list_entry {\n+\tstruct oidmap_entry entry;\n+\tstruct change_list changes;\n+};\n+\n+/**\n+ * Holds information about the heads of each change, and permits effecient\n+ * lookup from a commit to the changes that reference it directly.\n+ *\n+ * All fields should be considered private. Use the change_table functions\n+ * to interact with this struct.\n+ */\n+struct change_table {\n+\t/**\n+\t * Memory pool for the objects allocated by the change table.\n+\t */\n+\tstruct mem_pool memory_pool;\n+\t/* Map object_id to commit_change_list_entry structs. */\n+\tstruct oidmap oid_to_metadata_index;\n+\t/**\n+\t * List of ref names. The util value points to a change_head structure\n+\t * allocated from memory_pool.\n+\t */\n+\tstruct string_list refname_to_change_head;\n+};\n+\n+extern void change_table_init(struct change_table *to_initialize);\n+extern void change_table_clear(struct change_table *to_clear);\n+\n+/* Adds the given change head to the change_table struct */\n+extern void change_table_add(struct change_table *to_modify,\n+\tconst char *refname, struct commit *target);\n+\n+/**\n+ * Adds the non-hidden local changes to the given change_table struct.\n+ */\n+extern void change_table_add_all_visible(struct change_table *to_modify,\n+\tstruct repository *repo);\n+\n+/*\n+ * Adds all changes matching the given ref filter to the given change_table\n+ * struct.\n+ */\n+extern void change_table_add_matching_filter(struct change_table *to_modify,\n+\tstruct repository* repo, struct ref_filter *filter);\n+\n+typedef int each_change_fn(const char *refname, void *cb_data);\n+\n+extern int change_table_has_change_referencing(struct change_table *changes,\n+\tconst struct object_id *referenced_commit_id);\n+\n+/**\n+ * Iterates over all changes that reference the given commit. For metacommits,\n+ * this is the list of changes that point directly to that metacommit.\n+ * For normal commits, this is the list of changes that have this commit as\n+ * their latest content.\n+ */\n+extern int for_each_change_referencing(struct change_table *heads,\n+\tconst struct object_id *referenced_commit_id, each_change_fn fn, void *cb_data);\n+\n+/**\n+ * Returns the change head for the given refname. Returns NULL if no such change\n+ * exists.\n+ */\n+extern struct change_head* get_change_head(struct change_table *heads,\n+\tconst char* refname);\n+\n+#endif\n-- \ngitgitgadget\n\n"},{"id":"463538","messageId":"56c6770997bbdb1b3b87c2c410dd7f158b03f2d6.1663959325.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 06/10] evolve: add support for writing metacommits","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:20Z","receivedAt":"2022-09-23T18:56:02Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nmetacommit.c supports the creation of metacommits and\nadds the API needed to create and update changes.\n\nCreate the \"modify_change\" function that can be called from modification\ncommands like \"rebase\" and \"git amend\" to record obsolescences in the\nchange graph.\n\nCreate the \"record_metacommit\" function for recording more complicated\ncommit relationships in the commit graph.\n\nCreate the \"write_metacommit\" function for low-level creation of\nmetacommits.\n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n Makefile | 1 +\n metacommit.c | 404 +++++++++++++++++++++++++++++++++++++++++++++++++++\n metacommit.h | 58 ++++++++\n 3 files changed, 463 insertions(+)\n create mode 100644 metacommit.c\n create mode 100644 metacommit.h\n\ndiff --git a/Makefile b/Makefile\nindex 2b847e7e7de..68082ef94c7 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1000,6 +1000,7 @@ LIB_OBJS += merge-ort.o\n LIB_OBJS += merge-ort-wrappers.o\n LIB_OBJS += merge-recursive.o\n LIB_OBJS += merge.o\n+LIB_OBJS += metacommit.o\n LIB_OBJS += metacommit-parser.o\n LIB_OBJS += midx.o\n LIB_OBJS += name-hash.o\ndiff --git a/metacommit.c b/metacommit.c\nnew file mode 100644\nindex 00000000000..d2b859a4d3b\n--- /dev/null\n+++ b/metacommit.c\n@@ -0,0 +1,404 @@\n+#include \"cache.h\"\n+#include \"metacommit.h\"\n+#include \"commit.h\"\n+#include \"change-table.h\"\n+#include \"refs.h\"\n+\n+void init_metacommit_data(struct metacommit_data *state)\n+{\n+\tmemset(state, 0, sizeof(*state));\n+}\n+\n+void clear_metacommit_data(struct metacommit_data *state)\n+{\n+\toid_array_clear(&state->replace);\n+\toid_array_clear(&state->origin);\n+}\n+\n+static void compute_default_change_name(struct commit *initial_commit,\n+\tstruct strbuf* result)\n+{\n+\tstruct strbuf default_name;\n+\tconst char *buffer;\n+\tconst char *subject;\n+\tconst char *eol;\n+\tint len;\n+\tstrbuf_init(&default_name, 0);\n+\tbuffer = get_commit_buffer(initial_commit, NULL);\n+\tfind_commit_subject(buffer, &subject);\n+\teol = strchrnul(subject, '\\n');\n+\tfor (len = 0;subject < eol && len < 10; ++subject, ++len) {\n+\t\tchar next = *subject;\n+\t\tif (isspace(next))\n+\t\t\tcontinue;\n+\n+\t\tstrbuf_addch(&default_name, next);\n+\t}\n+\tsanitize_refname_component(default_name.buf, result);\n+}\n+\n+/**\n+ * Computes a change name for a change rooted at the given initial commit. Good\n+ * change names should be memorable, unique, and easy to type. They are not\n+ * required to match the commit comment.\n+ */\n+static void compute_change_name(struct commit *initial_commit, struct strbuf* result)\n+{\n+\tstruct strbuf default_name;\n+\tstruct object_id unused;\n+\n+\tstrbuf_init(&default_name, 0);\n+\tif (initial_commit)\n+\t\tcompute_default_change_name(initial_commit, &default_name);\n+\telse\n+\t\tstrbuf_addstr(&default_name, \"change\");\n+\tstrbuf_addstr(result, \"refs/metas/\");\n+\tstrbuf_addbuf(result, &default_name);\n+\n+\t/* If there is already a change of this name, append a suffix */\n+\tif (!read_ref(result->buf, &unused)) {\n+\t\tint suffix = 2;\n+\t\tint original_length = result->len;\n+\n+\t\twhile (1) {\n+\t\t\tstrbuf_addf(result, \"%d\", suffix);\n+\t\t\tif (read_ref(result->buf, &unused))\n+\t\t\t\tbreak;\n+\t\t\tstrbuf_remove(result, original_length, result->len - original_length);\n+\t\t\t++suffix;\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&default_name);\n+}\n+\n+struct resolve_metacommit_callback_data\n+{\n+\tstruct change_table* active_changes;\n+\tstruct string_list *changes;\n+\tstruct oid_array *heads;\n+};\n+\n+static int resolve_metacommit_callback(const char *refname, void *cb_data)\n+{\n+\tstruct resolve_metacommit_callback_data *data = (struct resolve_metacommit_callback_data *)cb_data;\n+\tstruct change_head *chhead;\n+\n+\tchhead = get_change_head(data->active_changes, refname);\n+\n+\tif (data->changes)\n+\t\tstring_list_append(data->changes, refname)->util = &(chhead->head);\n+\tif (data->heads)\n+\t\toid_array_append(data->heads, &(chhead->head));\n+\n+\treturn 0;\n+}\n+\n+/**\n+ * Produces the final form of a metacommit based on the current change refs.\n+ */\n+static void resolve_metacommit(\n+\tstruct repository* repo,\n+\tstruct change_table* active_changes,\n+\tconst struct metacommit_data *to_resolve,\n+\tstruct metacommit_data *resolved_output,\n+\tstruct string_list *to_advance,\n+\tint allow_append)\n+{\n+\tint i;\n+\tint len = to_resolve->replace.nr;\n+\tstruct resolve_metacommit_callback_data cbdata;\n+\tint old_change_list_length = to_advance->nr;\n+\tstruct commit* content;\n+\n+\toidcpy(&resolved_output->content, &to_resolve->content);\n+\n+\t/* First look for changes that point to any of the replacement edges in the\n+\t * metacommit. These will be the changes that get advanced by this\n+\t * metacommit. */\n+\tresolved_output->abandoned = to_resolve->abandoned;\n+\tcbdata.active_changes = active_changes;\n+\tcbdata.changes = to_advance;\n+\tcbdata.heads = &(resolved_output->replace);\n+\n+\tif (allow_append) {\n+\t\tfor (i = 0; i < len; i++) {\n+\t\t\tint old_number = resolved_output->replace.nr;\n+\t\t\tfor_each_change_referencing(active_changes, &(to_resolve->replace.oid[i]),\n+\t\t\t\tresolve_metacommit_callback, &cbdata);\n+\t\t\t/* If no changes were found, use the unresolved value. */\n+\t\t\tif (old_number == resolved_output->replace.nr)\n+\t\t\t\toid_array_append(&(resolved_output->replace), &(to_resolve->replace.oid[i]));\n+\t\t}\n+\t}\n+\n+\tcbdata.changes = NULL;\n+\tcbdata.heads = &(resolved_output->origin);\n+\n+\tlen = to_resolve->origin.nr;\n+\tfor (i = 0; i < len; i++) {\n+\t\tint old_number = resolved_output->origin.nr;\n+\t\tfor_each_change_referencing(active_changes, &(to_resolve->origin.oid[i]),\n+\t\t\tresolve_metacommit_callback, &cbdata);\n+\t\tif (old_number == resolved_output->origin.nr)\n+\t\t\toid_array_append(&(resolved_output->origin), &(to_resolve->origin.oid[i]));\n+\t}\n+\n+\t/* If no changes were advanced by this metacommit, we'll need to create a new\n+\t * one. */\n+\tif (to_advance->nr == old_change_list_length) {\n+\t\tstruct strbuf change_name;\n+\n+\t\tstrbuf_init(&change_name, 80);\n+\t\tcontent = lookup_commit_reference_gently(repo, &(to_resolve->content), 1);\n+\n+\t\tcompute_change_name(content, &change_name);\n+\t\tstring_list_append(to_advance, change_name.buf);\n+\t\tstrbuf_release(&change_name);\n+\t}\n+}\n+\n+static void lookup_commits(\n+\tstruct repository *repo,\n+\tstruct oid_array *to_lookup,\n+\tstruct commit_list **result)\n+{\n+\tint i = to_lookup->nr;\n+\n+\twhile (--i >= 0) {\n+\t\tstruct object_id *next = &(to_lookup->oid[i]);\n+\t\tstruct commit *commit = lookup_commit_reference_gently(repo, next, 1);\n+\t\tcommit_list_insert(commit, result);\n+\t}\n+}\n+\n+#define PARENT_TYPE_PREFIX \"parent-type \"\n+\n+/**\n+ * Creates a new metacommit object with the given content. Writes the object\n+ * id of the newly-created commit to result.\n+ */\n+int write_metacommit(struct repository *repo, struct metacommit_data *state,\n+\tstruct object_id *result)\n+{\n+\tstruct commit_list *parents = NULL;\n+\tstruct strbuf comment;\n+\tint i;\n+\tstruct commit *content;\n+\n+\tstrbuf_init(&comment, strlen(PARENT_TYPE_PREFIX)\n+\t\t+ 1 + 2 * (state->origin.nr + state->replace.nr));\n+\tlookup_commits(repo, &state->origin, &parents);\n+\tlookup_commits(repo, &state->replace, &parents);\n+\tcontent = lookup_commit_reference_gently(repo, &state->content, 1);\n+\tif (!content) {\n+\t\tstrbuf_release(&comment);\n+\t\tfree_commit_list(parents);\n+\t\treturn -1;\n+\t}\n+\tcommit_list_insert(content, &parents);\n+\n+\tstrbuf_addstr(&comment, PARENT_TYPE_PREFIX);\n+\tstrbuf_addstr(&comment, state->abandoned ? \"a\" : \"c\");\n+\tfor (i = 0; i < state->replace.nr; i++)\n+\t\tstrbuf_addstr(&comment, \" r\");\n+\n+\tfor (i = 0; i < state->origin.nr; i++)\n+\t\tstrbuf_addstr(&comment, \" o\");\n+\n+\t/* The parents list will be freed by this call. */\n+\tcommit_tree(comment.buf, comment.len, repo->hash_algo->empty_tree, parents,\n+\t\tresult, NULL, NULL);\n+\n+\tstrbuf_release(&comment);\n+\treturn 0;\n+}\n+\n+/**\n+ * Returns true iff the given metacommit is abandoned, has one or more origin\n+ * parents, or has one or more replacement parents.\n+ */\n+static int is_nontrivial_metacommit(struct metacommit_data *state)\n+{\n+\treturn state->replace.nr || state->origin.nr || state->abandoned;\n+}\n+\n+/*\n+ * Records the relationships described by the given metacommit in the\n+ * repository.\n+ *\n+ * If override_change is NULL (the default), an attempt will be made\n+ * to append to existing changes wherever possible instead of creating new ones.\n+ * If override_change is non-null, only the given change ref will be updated.\n+ *\n+ * options is a bitwise combination of the UPDATE_OPTION_* flags.\n+ */\n+int record_metacommit(\n+\tstruct repository *repo,\n+\tconst struct metacommit_data *metacommit, const char *override_change,\n+\tint options, struct strbuf *err)\n+{\n+\t\tstruct change_table chtable;\n+\t\tstruct string_list changes;\n+\t\tint result;\n+\n+\t\tchange_table_init(&chtable);\n+\t\tchange_table_add_all_visible(&chtable, repo);\n+\t\tstring_list_init_dup(&changes);\n+\n+\t\tresult = record_metacommit_withresult(repo, &chtable, metacommit,\n+\t\t\toverride_change, options, err, &changes);\n+\n+\t\tstring_list_clear(&changes, 0);\n+\t\tchange_table_clear(&chtable);\n+\t\treturn result;\n+}\n+\n+/*\n+ * Records the relationships described by the given metacommit in the\n+ * repository.\n+ *\n+ * If override_change is NULL (the default), an attempt will be made\n+ * to append to existing changes wherever possible instead of creating new ones.\n+ * If override_change is non-null, only the given change ref will be updated.\n+ *\n+ * The changes list is filled in with the list of change refs that were updated,\n+ * with the util pointers pointing to the old object IDS for those changes.\n+ * The object ID pointers all point to objects owned by the change_table and\n+ * will go out of scope when the change_table is destroyed.\n+ *\n+ * options is a bitwise combination of the UPDATE_OPTION_* flags.\n+ */\n+int record_metacommit_withresult(\n+\tstruct repository *repo,\n+\tstruct change_table *chtable,\n+\tconst struct metacommit_data *metacommit,\n+\tconst char *override_change,\n+\tint options, struct strbuf *err,\n+\tstruct string_list *changes)\n+{\n+\tstatic const char *msg = \"updating change\";\n+\tstruct metacommit_data resolved_metacommit;\n+\tstruct object_id commit_target;\n+\tstruct ref_transaction *transaction = NULL;\n+\tstruct change_head *overridden_head;\n+\tconst struct object_id *old_head;\n+\n+\tint i;\n+\tint ret = 0;\n+\tint force = (options & UPDATE_OPTION_FORCE);\n+\n+\tinit_metacommit_data(&resolved_metacommit);\n+\n+\tresolve_metacommit(repo, chtable, metacommit, &resolved_metacommit, changes,\n+\t\t(options & UPDATE_OPTION_NOAPPEND) == 0);\n+\n+\tif (override_change) {\n+\t\tstring_list_clear(changes, 0);\n+\t\toverridden_head = get_change_head(chtable, override_change);\n+\t\tif (!overridden_head) {\n+\t\t\t/* This is an existing change */\n+\t\t\told_head = &overridden_head->head;\n+\t\t\tif (!force) {\n+\t\t\t\tif (!oid_array_readonly_contains(&(resolved_metacommit.replace),\n+\t\t\t\t\t&overridden_head->head)) {\n+\t\t\t\t\t/* Attempted non-fast-forward change */\n+\t\t\t\t\tstrbuf_addf(err, _(\"non-fast-forward update to '%s'\"),\n+\t\t\t\t\t\toverride_change);\n+\t\t\t\t\tret = -1;\n+\t\t\t\t\tgoto cleanup;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t} else\n+\t\t\t/* ...then this is a newly-created change */\n+\t\t\told_head = null_oid();\n+\n+\t\t/* The expected \"current\" head of the change is stored in the util\n+\t\t * pointer. */\n+\t\tstring_list_append(changes, override_change)->util = (void*)old_head;\n+\t}\n+\n+\tif (is_nontrivial_metacommit(&resolved_metacommit)) {\n+\t\t/* If there are any origin or replacement parents, create a new metacommit\n+\t\t * object. */\n+\t\tif (write_metacommit(repo, &resolved_metacommit, &commit_target) < 0) {\n+\t\t\tret = -1;\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t} else\n+\t\t/**\n+\t\t * If the metacommit would only contain a content commit, point to the\n+\t\t * commit itself rather than creating a trivial metacommit.\n+\t\t */\n+\t\toidcpy(&commit_target, &(resolved_metacommit.content));\n+\n+\t/**\n+\t * If a change already exists with this target and we're not forcing an\n+\t * update to some specific override_change && change, there's nothing to do.\n+\t */\n+\tif (!override_change\n+\t\t&& change_table_has_change_referencing(chtable, &commit_target))\n+\t\t/* Not an error */\n+\t\tgoto cleanup;\n+\n+\ttransaction = ref_transaction_begin(err);\n+\n+\t/* Update the refs for each affected change */\n+\tif (!transaction)\n+\t\tret = -1;\n+\telse {\n+\t\tfor (i = 0; i < changes->nr; i++) {\n+\t\t\tstruct string_list_item *it = &changes->items[i];\n+\n+\t\t\t/**\n+\t\t\t * The expected current head of the change is stored in the util pointer.\n+\t\t\t * It is null if the change should be newly-created.\n+\t\t\t */\n+\t\t\tif (it->util) {\n+\t\t\t\tif (ref_transaction_update(transaction, it->string, &commit_target,\n+\t\t\t\t\tforce ? NULL : it->util, 0, msg, err))\n+\n+\t\t\t\t\tret = -1;\n+\t\t\t} else {\n+\t\t\t\tif (ref_transaction_create(transaction, it->string,\n+\t\t\t\t\t&commit_target, 0, msg, err))\n+\n+\t\t\t\t\tret = -1;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!ret)\n+\t\t\tif (ref_transaction_commit(transaction, err))\n+\t\t\t\tret = -1;\n+\t}\n+\n+cleanup:\n+\tref_transaction_free(transaction);\n+\tclear_metacommit_data(&resolved_metacommit);\n+\n+\treturn ret;\n+}\n+\n+/**\n+ * Should be invoked after a command that has \"modify\" semantics - commands that\n+ * create a new commit based on an old commit and treat the new one as a\n+ * replacement for the old one. This method records the replacement in the\n+ * change graph, such that a future evolve operation will rebase children of\n+ * the old commit onto the new commit.\n+ */\n+void modify_change(\n+\tstruct repository *repo,\n+\tconst struct object_id *old_commit,\n+\tconst struct object_id *new_commit,\n+\tstruct strbuf *err)\n+{\n+\tstruct metacommit_data metacommit;\n+\n+\tinit_metacommit_data(&metacommit);\n+\toidcpy(&(metacommit.content), new_commit);\n+\toid_array_append(&(metacommit.replace), old_commit);\n+\n+\trecord_metacommit(repo, &metacommit, NULL, 0, err);\n+\n+\tclear_metacommit_data(&metacommit);\n+}\ndiff --git a/metacommit.h b/metacommit.h\nnew file mode 100644\nindex 00000000000..fdb253f0f04\n--- /dev/null\n+++ b/metacommit.h\n@@ -0,0 +1,58 @@\n+#ifndef METACOMMIT_H\n+#define METACOMMIT_H\n+\n+#include \"hash.h\"\n+#include \"oid-array.h\"\n+#include \"repository.h\"\n+#include \"string-list.h\"\n+\n+\n+struct change_table;\n+\n+/* If specified, non-fast-forward changes are permitted. */\n+#define UPDATE_OPTION_FORCE 0x0001\n+/**\n+ * If specified, no attempt will be made to append to existing changes.\n+ * Normally, if a metacommit points to a commit in its replace or origin\n+ * list and an existing change points to that same commit as its content, the\n+ * new metacommit will attempt to append to that same change. This may replace\n+ * the commit parent with one or more metacommits from the head of the appended\n+ * changes. This option disables this behavior, and will always create a new\n+ * change rather than reusing existing changes.\n+ */\n+#define UPDATE_OPTION_NOAPPEND 0x0002\n+\n+/* Metacommit Data */\n+\n+struct metacommit_data {\n+\tstruct object_id content;\n+\tstruct oid_array replace;\n+\tstruct oid_array origin;\n+\tint abandoned;\n+};\n+\n+extern void init_metacommit_data(struct metacommit_data *state);\n+\n+extern void clear_metacommit_data(struct metacommit_data *state);\n+\n+extern int record_metacommit(struct repository *repo,\n+\tconst struct metacommit_data *metacommit,\n+\tconst char* override_change, int options, struct strbuf *err);\n+\n+extern int record_metacommit_withresult(\n+\tstruct repository *repo,\n+\tstruct change_table *chtable,\n+\tconst struct metacommit_data *metacommit,\n+\tconst char *override_change,\n+\tint options,\n+\tstruct strbuf *err,\n+\tstruct string_list *changes);\n+\n+extern void modify_change(struct repository *repo,\n+\tconst struct object_id *old_commit, const struct object_id *new_commit,\n+\tstruct strbuf *err);\n+\n+extern int write_metacommit(struct repository *repo, struct metacommit_data *state,\n+\tstruct object_id *result);\n+\n+#endif\n-- \ngitgitgadget\n\n"},{"id":"463539","messageId":"811d516e5d272acc40835aa6bdd4f79a001f72c0.1663959325.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 10/10] evolve: add documentation for `git change`","fromName":"Chris Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:24Z","receivedAt":"2022-09-23T18:56:04Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"From: Chris Poucet \n\nSigned-off-by: Chris Poucet \n---\n Documentation/git-change.txt | 55 ++++++++++++++++++++++++++++++++++++\n 1 file changed, 55 insertions(+)\n create mode 100644 Documentation/git-change.txt\n\ndiff --git a/Documentation/git-change.txt b/Documentation/git-change.txt\nnew file mode 100644\nindex 00000000000..ea9a8e619b9\n--- /dev/null\n+++ b/Documentation/git-change.txt\n@@ -0,0 +1,55 @@\n+git-change(1)\n+=============\n+\n+NAME\n+----\n+git-change - Create, list, update or delete changes\n+\n+SYNOPSIS\n+--------\n+[verse]\n+'git change' list [...]\n+'git change' update [-g | -n] [--force] [--replace ...] [--origin ...] [--content ]\n+'git change' delete ...\n+\n+DESCRIPTION\n+-----------\n+\n+`git change list`: lists all existing s.\n+\n+`git change delete`: deletes the given s.\n+\n+`git change update`: creates or updates a .\n+\n+If no arguments are given to `update` then a change is added to the\n+`refs/metas/` directory, unless a change already exists for the given commit.\n+\n+A starts with `metas/` and represents the current change that is\n+being worked on.\n+\n+OPTIONS\n+-------\n+-c::\n+--content::\n+\tIdentifies the content commit for the change\n+\n+-o::\n+--origin::\n+\tMarks the given commit as being the origin of this commit.\n+\n+-r::\n+--replace::\n+\tMarks the given commit as being obsoleted by the new commit.\n+\n+-g::\n+\t to update\n+\n+-n::\n+\tIndicates that the change is new and an existing change should not be updated.\n+\n+--force::\n+\tOverwite an existing change of the same name.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\n-- \ngitgitgadget\n"},{"id":"463540","messageId":"914028341842a4d57e02ec42a7426d3aa83640f9.1663959325.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 07/10] evolve: implement the git change command","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:21Z","receivedAt":"2022-09-23T18:56:06Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nImplement the git change update command, which\nare sufficient for constructing change graphs.\n\nFor example, to create a new change (a stable name) that refers to HEAD:\n\ngit change update -c HEAD\n\nTo record a rebase or amend in the change graph:\n\ngit change update -c -r \n\nTo record a cherry-pick in the change graph:\n\ngit change update -c -o \n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n .gitignore | 1 +\n Makefile | 1 +\n builtin.h | 1 +\n builtin/change.c | 199 +++++++++++++++++++++++++++++++++++++++++++++++\n git.c | 1 +\n ref-filter.c | 2 +-\n ref-filter.h | 4 +\n 7 files changed, 208 insertions(+), 1 deletion(-)\n create mode 100644 builtin/change.c\n\ndiff --git a/.gitignore b/.gitignore\nindex b3dcafcb331..a57fd8d8897 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -28,6 +28,7 @@\n /git-bugreport\n /git-bundle\n /git-cat-file\n+/git-change\n /git-check-attr\n /git-check-ignore\n /git-check-mailmap\ndiff --git a/Makefile b/Makefile\nindex 68082ef94c7..82f68f13d9f 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1142,6 +1142,7 @@ BUILTIN_OBJS += builtin/branch.o\n BUILTIN_OBJS += builtin/bugreport.o\n BUILTIN_OBJS += builtin/bundle.o\n BUILTIN_OBJS += builtin/cat-file.o\n+BUILTIN_OBJS += builtin/change.o\n BUILTIN_OBJS += builtin/check-attr.o\n BUILTIN_OBJS += builtin/check-ignore.o\n BUILTIN_OBJS += builtin/check-mailmap.o\ndiff --git a/builtin.h b/builtin.h\nindex 8901a34d6bf..c10f20c972c 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -122,6 +122,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix);\n int cmd_bugreport(int argc, const char **argv, const char *prefix);\n int cmd_bundle(int argc, const char **argv, const char *prefix);\n int cmd_cat_file(int argc, const char **argv, const char *prefix);\n+int cmd_change(int argc, const char **argv, const char *prefix);\n int cmd_checkout(int argc, const char **argv, const char *prefix);\n int cmd_checkout__worker(int argc, const char **argv, const char *prefix);\n int cmd_checkout_index(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/change.c b/builtin/change.c\nnew file mode 100644\nindex 00000000000..b0e29e87ec9\n--- /dev/null\n+++ b/builtin/change.c\n@@ -0,0 +1,199 @@\n+#include \"builtin.h\"\n+#include \"ref-filter.h\"\n+#include \"parse-options.h\"\n+#include \"metacommit.h\"\n+#include \"change-table.h\"\n+#include \"config.h\"\n+\n+static const char * const builtin_change_usage[] = {\n+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n+\tNULL\n+};\n+\n+static const char * const builtin_update_usage[] = {\n+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n+\tNULL\n+};\n+\n+struct update_state {\n+\tint options;\n+\tconst char* change;\n+\tconst char* content;\n+\tstruct string_list replace;\n+\tstruct string_list origin;\n+};\n+\n+static void init_update_state(struct update_state *state)\n+{\n+\tmemset(state, 0, sizeof(*state));\n+\tstate->content = \"HEAD\";\n+\tstring_list_init_nodup(&state->replace);\n+\tstring_list_init_nodup(&state->origin);\n+}\n+\n+static void clear_update_state(struct update_state *state)\n+{\n+\tstring_list_clear(&state->replace, 0);\n+\tstring_list_clear(&state->origin, 0);\n+}\n+\n+static int update_option_parse_replace(const struct option *opt,\n+\t\t\t\t const char *arg, int unset)\n+{\n+\tstruct update_state *state = opt->value;\n+\tstring_list_append(&state->replace, arg);\n+\treturn 0;\n+}\n+\n+static int update_option_parse_origin(const struct option *opt,\n+\t\t\t\t const char *arg, int unset)\n+{\n+\tstruct update_state *state = opt->value;\n+\tstring_list_append(&state->origin, arg);\n+\treturn 0;\n+}\n+\n+static int resolve_commit(const char *committish, struct object_id *result)\n+{\n+\tstruct commit *commit;\n+\tif (get_oid_committish(committish, result))\n+\t\tdie(_(\"Failed to resolve '%s' as a valid revision.\"), committish);\n+\tcommit = lookup_commit_reference(the_repository, result);\n+\tif (!commit)\n+\t\tdie(_(\"Could not parse object '%s'.\"), committish);\n+\toidcpy(result, &commit->object.oid);\n+\treturn 0;\n+}\n+\n+static void resolve_commit_list(const struct string_list *commitsish_list,\n+\tstruct oid_array* result)\n+{\n+\tint i;\n+\tfor (i = 0; i < commitsish_list->nr; i++) {\n+\t\tstruct string_list_item *item = &commitsish_list->items[i];\n+\t\tstruct object_id next;\n+\t\tresolve_commit(item->string, &next);\n+\t\toid_array_append(result, &next);\n+\t}\n+}\n+\n+/*\n+ * Given the command-line options for the update command, fills in a\n+ * metacommit_data with the corresponding changes.\n+ */\n+static void get_metacommit_from_command_line(\n+\tconst struct update_state* commands, struct metacommit_data *result)\n+{\n+\tresolve_commit(commands->content, &(result->content));\n+\tresolve_commit_list(&(commands->replace), &(result->replace));\n+\tresolve_commit_list(&(commands->origin), &(result->origin));\n+}\n+\n+static int perform_update(\n+\tstruct repository *repo,\n+\tconst struct update_state *state,\n+\tstruct strbuf *err)\n+{\n+\tstruct metacommit_data metacommit;\n+\tstruct change_table chtable;\n+\tstruct string_list changes;\n+\tint ret;\n+\tint i;\n+\n+\tchange_table_init(&chtable);\n+\tchange_table_add_all_visible(&chtable, repo);\n+\tstring_list_init_dup(&changes);\n+\n+\tinit_metacommit_data(&metacommit);\n+\n+\tget_metacommit_from_command_line(state, &metacommit);\n+\n+\tret = record_metacommit_withresult(repo, &chtable, &metacommit,\n+\t\tstate->change, state->options, err, &changes);\n+\n+\tfor (i = 0; i < changes.nr; i++) {\n+\t\tstruct string_list_item *it = &changes.items[i];\n+\n+\t\tconst char* name = lstrip_ref_components(it->string, 1);\n+\t\tif (!name)\n+\t\t\tdie(_(\"Failed to remove `refs/` from %s\"), it->string);\n+\n+\t\tif (it->util)\n+\t\t\tfprintf(stdout, N_(\"Updated change %s\\n\"), name);\n+\t\telse\n+\t\t\tfprintf(stdout, N_(\"Created change %s\\n\"), name);\n+\t}\n+\n+\tstring_list_clear(&changes, 0);\n+\tchange_table_clear(&chtable);\n+\tclear_metacommit_data(&metacommit);\n+\n+\treturn ret;\n+}\n+\n+static int change_update(int argc, const char **argv, const char* prefix)\n+{\n+\tint result;\n+\tint force = 0;\n+\tint newchange = 0;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tstruct update_state state;\n+\tstruct option options[] = {\n+\t\t{ OPTION_CALLBACK, 'r', \"replace\", &state, N_(\"commit\"),\n+\t\t\tN_(\"marks the given commit as being obsolete\"),\n+\t\t\t0, update_option_parse_replace },\n+\t\t{ OPTION_CALLBACK, 'o', \"origin\", &state, N_(\"commit\"),\n+\t\t\tN_(\"marks the given commit as being the origin of this commit\"),\n+\t\t\t0, update_option_parse_origin },\n+\t\tOPT_BOOL('F', \"force\", &force,\n+\t\t\tN_(\"overwrite an existing change of the same name\")),\n+\t\tOPT_STRING('c', \"content\", &state.content, N_(\"commit\"),\n+\t\t\t\t N_(\"identifies the new content commit for the change\")),\n+\t\tOPT_STRING('g', \"change\", &state.change, N_(\"commit\"),\n+\t\t\t\t N_(\"name of the change to update\")),\n+\t\tOPT_BOOL('n', \"new\", &newchange,\n+\t\t\tN_(\"create a new change - do not append to any existing change\")),\n+\t\tOPT_END()\n+\t};\n+\n+\tinit_update_state(&state);\n+\n+\targc = parse_options(argc, argv, prefix, options, builtin_update_usage, 0);\n+\n+\tif (force) state.options |= UPDATE_OPTION_FORCE;\n+\tif (newchange) state.options |= UPDATE_OPTION_NOAPPEND;\n+\n+\tresult = perform_update(the_repository, &state, &err);\n+\n+\tif (result < 0) {\n+\t\terror(\"%s\", err.buf);\n+\t\tstrbuf_release(&err);\n+\t}\n+\n+\tclear_update_state(&state);\n+\n+\treturn result;\n+}\n+\n+int cmd_change(int argc, const char **argv, const char *prefix)\n+{\n+\t/* No options permitted before subcommand currently */\n+\tstruct option options[] = {\n+\t\tOPT_END()\n+\t};\n+\tint result = 1;\n+\n+\targc = parse_options(argc, argv, prefix, options, builtin_change_usage,\n+\t\tPARSE_OPT_STOP_AT_NON_OPTION);\n+\n+\tif (argc < 1)\n+\t\tusage_with_options(builtin_change_usage, options);\n+\telse if (!strcmp(argv[0], \"update\"))\n+\t\tresult = change_update(argc, argv, prefix);\n+\telse {\n+\t\terror(_(\"Unknown subcommand: %s\"), argv[0]);\n+\t\tusage_with_options(builtin_change_usage, options);\n+\t}\n+\n+\treturn result ? 1 : 0;\n+}\ndiff --git a/git.c b/git.c\nindex da411c53822..837b1abc53b 100644\n--- a/git.c\n+++ b/git.c\n@@ -498,6 +498,7 @@ static struct cmd_struct commands[] = {\n \t{ \"bugreport\", cmd_bugreport, RUN_SETUP_GENTLY },\n \t{ \"bundle\", cmd_bundle, RUN_SETUP_GENTLY },\n \t{ \"cat-file\", cmd_cat_file, RUN_SETUP },\n+\t{ \"change\", cmd_change, RUN_SETUP},\n \t{ \"check-attr\", cmd_check_attr, RUN_SETUP },\n \t{ \"check-ignore\", cmd_check_ignore, RUN_SETUP | NEED_WORK_TREE },\n \t{ \"check-mailmap\", cmd_check_mailmap, RUN_SETUP },\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 6a1789c623f..2d7a919d547 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1557,7 +1557,7 @@ static inline char *copy_advance(char *dst, const char *src)\n \treturn dst;\n }\n \n-static const char *lstrip_ref_components(const char *refname, int len)\n+const char *lstrip_ref_components(const char *refname, int len)\n {\n \tlong remaining = len;\n \tconst char *start = xstrdup(refname);\ndiff --git a/ref-filter.h b/ref-filter.h\nindex 064fbef8e50..7a7737e9552 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -145,4 +145,8 @@ struct ref_array_item *ref_array_push(struct ref_array *array,\n \t\t\t\t const char *refname,\n \t\t\t\t const struct object_id *oid);\n \n+/* Strips `len` prefix components from the refname. */\n+const char *lstrip_ref_components(const char *refname, int len);\n+\n+\n #endif /* REF_FILTER_H */\n-- \ngitgitgadget\n\n"},{"id":"463541","messageId":"d087d467e3fe3000eb19939c2bb5e5c0723fd908.1663959325.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 09/10] evolve: add delete command","fromName":"Chris Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:23Z","receivedAt":"2022-09-23T18:56:09Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"From: Chris Poucet \n\nThe delete command allows a user to delete one or more changes.\nThis effectively deletes the corresponding /refs/metas/foo ref.\n\nSigned-off-by: Chris Poucet \n---\n builtin/change.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++--\n 1 file changed, 80 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/change.c b/builtin/change.c\nindex 67d708dc8de..07d029d82d5 100644\n--- a/builtin/change.c\n+++ b/builtin/change.c\n@@ -4,10 +4,12 @@\n #include \"metacommit.h\"\n #include \"change-table.h\"\n #include \"config.h\"\n+#include \"refs.h\"\n \n static const char * const builtin_change_usage[] = {\n \tN_(\"git change list [...]\"),\n-\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n+\tN_(\"git change delete ...\"),\n \tNULL\n };\n \n@@ -17,7 +19,12 @@ static const char * const builtin_list_usage[] = {\n };\n \n static const char * const builtin_update_usage[] = {\n-\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n+\tNULL\n+};\n+\n+static const char * const builtin_delete_usage[] = {\n+\tN_(\"git change delete ...\"),\n \tNULL\n };\n \n@@ -238,6 +245,75 @@ static int change_update(int argc, const char **argv, const char* prefix)\n \treturn result;\n }\n \n+typedef int (*each_change_name_fn)(const char *name, const char *ref,\n+\t\t\t\t const struct object_id *oid, void *cb_data);\n+\n+static int for_each_change_name(const char **argv, each_change_name_fn fn,\n+\t\t\t\tvoid *cb_data)\n+{\n+\tconst char **p;\n+\tstruct strbuf ref = STRBUF_INIT;\n+\tint had_error = 0;\n+\tstruct object_id oid;\n+\n+\tfor (p = argv; *p; p++) {\n+\t\tstrbuf_reset(&ref);\n+\t\t/* Convenience functionality to avoid having to type `metas/` */\n+\t\tif (strncmp(\"metas/\", *p, 5)) {\n+\t\t\tstrbuf_addf(&ref, \"refs/metas/%s\", *p);\n+\t\t} else {\n+\t\t\tstrbuf_addf(&ref, \"refs/%s\", *p);\n+\t\t}\n+\t\tif (read_ref(ref.buf, &oid)) {\n+\t\t\terror(_(\"change '%s' not found.\"), *p);\n+\t\t\thad_error = 1;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (fn(*p, ref.buf, &oid, cb_data))\n+\t\t\thad_error = 1;\n+\t}\n+\tstrbuf_release(&ref);\n+\treturn had_error;\n+}\n+\n+static int collect_changes(const char *name, const char *ref,\n+\t\t\t const struct object_id *oid, void *cb_data)\n+{\n+\tstruct string_list *ref_list = cb_data;\n+\n+\tstring_list_append(ref_list, ref);\n+\tref_list->items[ref_list->nr - 1].util = oiddup(oid);\n+\treturn 0;\n+}\n+\n+static int change_delete(int argc, const char **argv, const char* prefix) {\n+\tint result = 0;\n+\tstruct string_list refs_to_delete = STRING_LIST_INIT_DUP;\n+\tstruct string_list_item *item;\n+\tstruct option options[] = {\n+\t\tOPT_END()\n+\t};\n+\n+\targc = parse_options(argc, argv, prefix, options, builtin_delete_usage, 0);\n+\n+\tresult = for_each_change_name(argv, collect_changes, (void *)&refs_to_delete);\n+\tif (delete_refs(NULL, &refs_to_delete, REF_NO_DEREF))\n+\t\tresult = 1;\n+\n+\tfor_each_string_list_item(item, &refs_to_delete) {\n+\t\tconst char *name = item->string;\n+\t\tstruct object_id *oid = item->util;\n+\t\tif (!ref_exists(name))\n+\t\t\tprintf(_(\"Deleted change '%s' (was %s)\\n\"),\n+\t\t\t\titem->string + 5,\n+\t\t\t\tfind_unique_abbrev(oid, DEFAULT_ABBREV));\n+\n+\t\tfree(oid);\n+\t}\n+\tstring_list_clear(&refs_to_delete, 0);\n+\treturn result;\n+}\n+\n int cmd_change(int argc, const char **argv, const char *prefix)\n {\n \t/* No options permitted before subcommand currently */\n@@ -255,6 +331,8 @@ int cmd_change(int argc, const char **argv, const char *prefix)\n \t\tresult = change_list(argc, argv, prefix);\n \telse if (!strcmp(argv[0], \"update\"))\n \t\tresult = change_update(argc, argv, prefix);\n+\telse if (!strcmp(argv[0], \"delete\"))\n+\t\tresult = change_delete(argc, argv, prefix);\n \telse {\n \t\terror(_(\"Unknown subcommand: %s\"), argv[0]);\n \t\tusage_with_options(builtin_change_usage, options);\n-- \ngitgitgadget\n\n"},{"id":"463542","messageId":"b83a79beeb456fa4c55aee5cfd204752a7b992c2.1663959325.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH 08/10] evolve: add the git change list command","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-09-23T18:55:22Z","receivedAt":"2022-09-23T18:56:11Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nThis command lists the ongoing changes from the refs/metas\nnamespace.\n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n builtin/change.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 65 insertions(+)\n\ndiff --git a/builtin/change.c b/builtin/change.c\nindex b0e29e87ec9..67d708dc8de 100644\n--- a/builtin/change.c\n+++ b/builtin/change.c\n@@ -6,15 +6,78 @@\n #include \"config.h\"\n \n static const char * const builtin_change_usage[] = {\n+\tN_(\"git change list [...]\"),\n \tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n \tNULL\n };\n \n+static const char * const builtin_list_usage[] = {\n+\tN_(\"git change list [...]\"),\n+\tNULL\n+};\n+\n static const char * const builtin_update_usage[] = {\n \tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n \tNULL\n };\n \n+static int change_list(int argc, const char **argv, const char* prefix)\n+{\n+\tstruct option options[] = {\n+\t\tOPT_END()\n+\t};\n+\tstruct ref_filter filter;\n+\t/* TODO: See below\n+\tstruct ref_sorting *sorting;\n+\tstruct string_list sorting_options = STRING_LIST_INIT_DUP; */\n+\tstruct ref_format format = REF_FORMAT_INIT;\n+\tstruct ref_array array;\n+\tint i;\n+\n+\targc = parse_options(argc, argv, prefix, options, builtin_list_usage, 0);\n+\n+\tsetup_ref_filter_porcelain_msg();\n+\n+\tmemset(&filter, 0, sizeof(filter));\n+\tmemset(&array, 0, sizeof(array));\n+\n+\tfilter.kind = FILTER_REFS_CHANGES;\n+\tfilter.name_patterns = argv;\n+\n+\tfilter_refs(&array, &filter, FILTER_REFS_CHANGES);\n+\n+\t/* TODO: This causes a crash. It sets one of the atom_value handlers to\n+\t * something invalid, which causes a crash later when we call\n+\t * show_ref_array_item. Figure out why this happens and put back the sorting.\n+\t *\n+\t * sorting = ref_sorting_options(&sorting_options);\n+\t * ref_array_sort(sorting, &array); */\n+\n+\tif (!format.format)\n+\t\tformat.format = \"%(refname:lstrip=1)\";\n+\n+\tif (verify_ref_format(&format))\n+\t\tdie(_(\"unable to parse format string\"));\n+\n+\tfor (i = 0; i < array.nr; i++) {\n+\t\tstruct strbuf output = STRBUF_INIT;\n+\t\tstruct strbuf err = STRBUF_INIT;\n+\t\tif (format_ref_array_item(array.items[i], &format, &output, &err))\n+\t\t\tdie(\"%s\", err.buf);\n+\t\tfwrite(output.buf, 1, output.len, stdout);\n+\t\tputchar('\\n');\n+\n+\t\tstrbuf_release(&err);\n+\t\tstrbuf_release(&output);\n+\t}\n+\n+\tref_array_clear(&array);\n+\t/* TODO: see above\n+\tref_sorting_release(sorting); */\n+\n+\treturn 0;\n+}\n+\n struct update_state {\n \tint options;\n \tconst char* change;\n@@ -188,6 +251,8 @@ int cmd_change(int argc, const char **argv, const char *prefix)\n \n \tif (argc < 1)\n \t\tusage_with_options(builtin_change_usage, options);\n+\telse if (!strcmp(argv[0], \"list\"))\n+\t\tresult = change_list(argc, argv, prefix);\n \telse if (!strcmp(argv[0], \"update\"))\n \t\tresult = change_update(argc, argv, prefix);\n \telse {\n-- \ngitgitgadget\n\n"},{"id":"463548","messageId":"CAMKO5CsXAhb0jM=sFOZCwnMf=ZdO1HrQ1EcTj9bW8OJ5aj6ZPw@mail.gmail.com","threadId":"58504","inReplyTo":"a0cf68f8ba2adefae4fceeab0d438d05e355e695.1663959324.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 01/10] technical doc: add a design doc for the evolve command","fromName":"Jerry Zhang","fromEmail":"jerry@skydio.com","sentAt":"2022-09-23T19:59:56Z","receivedAt":"2022-09-23T20:00:19Z","isPatch":true,"sender":{"key":"jerry@skydio.com","avatar":"https://avatars.githubusercontent.com/u/81337184?v=4"},"body":"On Fri, Sep 23, 2022 at 11:56 AM Stefan Xenos via GitGitGadget\n wrote:\n>\n> From: Stefan Xenos \n>\n> This document describes what a change graph for\n> git would look like, the behavior of the evolve command,\n> and the changes planned for other commands.\n>\n> It was originally proposed in 2018, see\n> https://public-inbox.org/git/20181115005546.212538-1-sxenos@google.com/\n>\n> Signed-off-by: Stefan Xenos \n> Signed-off-by: Chris Poucet \n> ---\n> Documentation/technical/evolve.txt | 1051 ++++++++++++++++++++++++++++\n> 1 file changed, 1051 insertions(+)\n> create mode 100644 Documentation/technical/evolve.txt\n>\n> diff --git a/Documentation/technical/evolve.txt b/Documentation/technical/evolve.txt\n> new file mode 100644\n> index 00000000000..68ee2457e52\n> --- /dev/null\n> +++ b/Documentation/technical/evolve.txt\n> @@ -0,0 +1,1051 @@\n> +Evolve\n> +======\n> +\n> +Objective\n> +=========\n> +Create an \"evolve\" command to help users craft a high quality commit history.\n> +Users can improve commits one at a time and in any order, then run git evolve to\n> +rewrite their recent history to ensure everything is up-to-date. We track\n> +amendments to a commit over time in a change graph. Users can share their\n> +progress with others by exchanging their change graphs using the standard push,\n> +fetch, and format-patch commands.\n> +\n> +Status\n> +======\n> +This proposal has not been implemented yet.\n> +\n> +Background\n> +==========\n> +Imagine you have three sequential changes up for review and you receive feedback\n> +that requires editing all three changes. We'll define the word \"change\"\n> +formally later, but for the moment let's say that a change is a work-in-progress\n> +whose final version will be submitted as a commit in the future.\n> +\n> +While you're editing one change, more feedback arrives on one of the others.\n> +What do you do?\n> +\n> +The evolve command is a convenient way to work with chains of commits that are\n> +under review. Whenever you rebase or amend a commit, the repository remembers\n> +that the old commit is obsolete and has been replaced by the new one. Then, at\n> +some point in the future, you can run \"git evolve\" and the correct sequence of\n> +rebases will occur in the correct order such that no commit has an obsolete\n> +parent.\n> +\n> +Part of making the \"evolve\" command work involves tracking the edits to a commit\n> +over time, which is why we need an change graph. However, the change\n> +graph will also bring other benefits:\n> +\n> +- Users can view the history of a change directly (the sequence of amends and\n> + rebases it has undergone, orthogonal to the history of the branch it is on).\n> +- It will be possible to quickly locate and list all the changes the user\n> + currently has in progress.\n> +- It can be used as part of other high-level commands that combine or split\n> + changes.\n> +- It can be used to decorate commits (in git log, gitk, etc) that are either\n> + obsolete or are the tip of a work in progress.\n> +- By pushing and pulling the change graph, users can collaborate more\n> + easily on changes-in-progress. This is better than pushing and pulling the\n> + commits themselves since the change graph can be used to locate a more\n> + specific merge base, allowing for better merges between different versions of\n> + the same change.\n> +- It could be used to correctly rebase local changes and other local branches\n> + after running git-filter-branch.\n> +- It can replace the change-id footer used by gerrit.\n> +\n> +Goals\n> +-----\n> +Legend: Goals marked with P0 are required. Goals marked with Pn should be\n> +attempted unless they interfere with goals marked with Pn-1.\n> +\n> +P0. All commands that modify commits (such as the normal commit --amend or\n> + rebase command) should mark the old commit as being obsolete and replaced by\n> + the new one. No additional commands should be required to keep the\n> + change graph up-to-date.\n> +P0. Any commit that may be involved in a future evolve command should not be\n> + garbage collected. Specifically:\n> + - Commits that obsolete another should not be garbage collected until\n> + user-specified conditions have occurred and the change has expired from\n> + the reflog. User specified conditions for removing changes include:\n> + - The user explicitly deleted the change.\n> + - The change was merged into a specific branch.\n> + - Commits that have been obsoleted by another should not be garbage\n> + collected if any of their replacements are still being retained.\n> +P0. A commit can be obsoleted by more than one replacement (called divergence).\n> +P0. Users must be able to resolve divergence (convergence).\n> +P1. Users should be able to share chains of obsolete changes in order to\n> + collaborate on WIP changes.\n> +P2. Such sharing should be at the user’s option. That is, it should be possible\n> + to directly share a change without also sharing the file states or commit\n> + comments from the obsolete changes that led up to it, and the choice not to\n> + share those commits should not require changing any commit hashes.\n> +P2. It should be possible to discard part or all of the change graph\n> + without discarding the commits themselves that are already present in\n> + branches and the reflog.\n> +P2. Provide sufficient information to replace gerrit's Change-Id footers.\n> +\n> +Similar technologies\n> +--------------------\n> +There are some other technologies that address the same end-user problem.\n> +\n> +Rebase -i can be used to solve the same problem, but users can't easily switch\n> +tasks midway through an interactive rebase or have more than one interactive\n> +rebase going on at the same time. It can't handle the case where you have\n> +multiple changes sharing the same parent when that parent needs to be rebased\n> +and won't let you collaborate with others on resolving a complicated interactive\n> +rebase. You can think of rebase -i as a top-down approach and the evolve command\n> +as the bottom-up approach to the same problem.\nI'll mention some other tools in this space too:\n\nrevup amend (https://github.com/Skydio/revup/blob/main/docs/amend.md)\n(I'm the author) allows insertion of cached changes into any commit in\nthe current history, and then reapplies the rest of history on top of\nthose changes. It uses a \"git apply --cached\" engine under the hood so\ndoesn't touch the working directory (although it will soon use the new\ngit merge-tree). When paired with \"revup upload\" which creates and\npushes multiple branches in the background for you, its possible to\nwork on a \"graph\" of changes on a single branch linearly, then have\nthe true graph structure created at upload time.\n\ngit-revise (https://github.com/mystor/git-revise) does some very\nsimilar things except it uses \"git merge-file\" combined with manually\nmerging the resulting trees. git branchstack\n(https://github.com/krobelus/git-branchstack) can also create branches\nin the background for you with the same mechanism.\n\nThese tools don't store any external state, but as such also don't\nprovide any specific collaboration mechanism for individual changes,\nso I'm interested in the \"evolve\" approach as well.\n> +\n> +Several patch queue managers have been built on top of git (such as topgit,\n> +stgit, and quilt). They address the same user need. However they also rely on\n> +state managed outside git that needs to be kept in sync. Such state can be\n> +easily damaged when running a git native command that is unaware of the patch\n> +queue. They also typically require an explicit initialization step to be done by\n> +the user which creates workflow problems.\n> +\n> +Mercurial implements a very similar feature in its EvolveExtension. The behavior\n> +of the evolve command itself is very similar, but the storage format for the\n> +change graph differs. In the case of mercurial, each change set can have one or\n> +more obsolescence markers that point to other changesets that they replace. This\n> +is similar to the \"Commit Headers\" approach considered in the other options\n> +appendix. The approach proposed here stores obsolescence information in a\n> +separate metacommit graph, which makes exchanging of obsolescence information\n> +optional.\n> +\n> +Mercurial's default behavior makes it easy to find and switch between\n> +non-obsolete changesets that aren't currently on any branch. We introduce the\n> +notion of a new ref namespace that enables a similar workflow via a different\n> +mechanism. Mercurial has the notion of changeset phases which isn't present\n> +in git and creates new ways for a changeset to diverge. Git doesn't need\n> +to deal with these issues, but it has to deal with the problems of picking an\n> +upstream branch as a target for rebases and protecting obsolescence information\n> +from GC. We also introduce some additional transformations (see\n> +obsolescence-over-cherry-pick, below) that aren't present in the mercurial\n> +implementation.\n> +\n> +Semi-related work\n> +-----------------\n> +There are other technologies that address different problems but have some\n> +similarities with this proposal.\n> +\n> +Replacements (refs/replace) are superficially similar to obsolescences in that\n> +they describe that one commit should be replaced by another. However, they\n> +differ in both how they are created and how they are intended to be used.\n> +Obsolescences are created automatically by the commands a user runs, and they\n> +describe the user’s intent to perform a future rebase. Obsolete commits still\n> +appear in branches, logs, etc like normal commits (possibly with an extra\n> +decoration that marks them as obsolete). Replacements are typically created\n> +explicitly by the user, they are meant to be kept around for a long time, and\n> +they describe a replacement to be applied at read-time rather than as the input\n> +to a future operation. When a replaced commit is queried, it is typically hidden\n> +and swapped out with its replacement as though the replacement has already\n> +occurred.\n> +\n> +Git-imerge is a project to help make complicated merges easier, particularly\n> +when merging or rebasing long chains of patches. It is not an alternative to\n> +the change graph, but its algorithm of applying smaller incremental merges\n> +could be used as part of the evolve algorithm in the future.\n> +\n> +Overview\n> +========\n> +We introduce the notion of “meta-commits” which describe how one commit was\n> +created from other commits. A branch of meta-commits is known as a change.\n> +Changes are created and updated automatically whenever a user runs a command\n> +that creates a commit. They are used for locating obsolete commits, providing a\n> +list of a user’s unsubmitted work in progress, and providing a stable name for\n> +each unsubmitted change.\n> +\n> +Users can exchange edit histories by pushing and fetching changes.\n> +\n> +New commands will be introduced for manipulating changes and resolving\n> +divergence between them. Existing commands that create commits will be updated\n> +to modify the meta-commit graph and create changes where necessary.\n> +\n> +Example usage\n> +-------------\n> +# First create three dependent changes\n> +$ echo foo>bar.txt && git add .\n> +$ git commit -m \"This is a test\"\n> +created change metas/this_is_a_test\n> +$ echo foo2>bar2.txt && git add .\n> +$ git commit -m \"This is also a test\"\n> +created change metas/this_is_also_a_test\n> +$ echo foo3>bar3.txt && git add .\n> +$ git commit -m \"More testing\"\n> +created change metas/more_testing\n> +\n> +# List all our changes in progress\n> +$ git change list\n> +metas/this_is_a_test\n> +metas/this_is_also_a_test\n> +* metas/more_testing\n> +metas/some_change_already_merged_upstream\n> +\n> +# Now modify the earliest change, using its stable name\n> +$ git reset --hard metas/this_is_a_test\n> +$ echo morefoo>>bar.txt && git add . && git commit --amend --no-edit\n> +\n> +# Use git-evolve to fix up any dependent changes\n> +$ git evolve\n> +rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n> +rebasing metas/more_testing onto metas/this_is_also_a_test\n> +Done\n> +\n> +# Use git-obslog to view the history of the this_is_a_test change\n> +$ git log --obslog\n> +93f110 metas/this_is_a_test@{0} commit (amend): This is a test\n> +930219 metas/this_is_a_test@{1} commit: This is a test\n> +\n> +# Now create an unrelated change\n> +$ git reset --hard origin/master\n> +$ echo newchange>unrelated.txt && git add .\n> +$ git commit -m \"Unrelated change\"\n> +created change metas/unrelated_change\n> +\n> +# Fetch the latest code from origin/master and use git-evolve\n> +# to rebase all dependent changes.\n> +$ git fetch origin master\n> +$ git evolve origin/master\n> +deleting metas/some_change_already_merged_upstream\n> +rebasing metas/this_is_a_test onto origin/master\n> +rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n> +rebasing metas/more_testing onto metas/this_is_also_a_test\n> +rebasing metas/unrelated_change onto origin/master\n> +Conflict detected! Resolve it and then use git evolve --continue to resume.\n> +\n> +# Sort out the conflict\n> +$ git mergetool\n> +$ git evolve origin/master\n> +Done\n> +\n> +# Share the full history of edits for the this_is_a_test change\n> +# with a review server\n> +$ git push origin metas/this_is_a_test:refs/for/master\n> +# Share the lastest commit for “Unrelated change”, without history\n> +$ git push origin HEAD:refs/for/master\n> +\n> +Detailed design\n> +===============\n> +Obsolescence information is stored as a graph of meta-commits. A meta-commit is\n> +a specially-formatted merge commit that describes how one commit was created\n> +from others.\n> +\n> +Meta-commits look like this:\n> +\n> +$ git cat-file -p \n> +tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\n> +parent aa7ce55545bf2c14bef48db91af1a74e2347539a\n> +parent d64309ee51d0af12723b6cb027fc9f195b15a5e9\n> +parent 7e1bbcd3a0fa854a7a9eac9bf1eea6465de98136\n> +author Stefan Xenos 1540841596 -0700\n> +committer Stefan Xenos 1540841596 -0700\n> +parent-type c r o\n> +\n> +This says “commit aa7ce555 makes commit d64309ee obsolete. It was created by\n> +cherry-picking commit 7e1bbcd3”.\n> +\n> +The tree for meta-commits is always the empty tree, but future versions of git\n> +may attach other trees here. For forward-compatibility fsck should ignore such\n> +trees if found on future repository versions. This will allow future versions of\n> +git to add metadata to the meta-commit tree without breaking forwards\n> +compatibility.\n> +\n> +The commit comment for a meta-commit is an auto-generated user-readable string\n> +describing the command that produced the meta commit. These strings are shown\n> +to the user when they view the obslog.\n> +\n> +Parent-type\n> +-----------\n> +The “parent-type” field in the commit header identifies a commit as a\n> +meta-commit and indicates the meaning for each of its parents. It is never\n> +present for normal commits. It contains a space-deliminated list of enum values\n> +whose order matches the order of the parents. Possible parent types are:\n> +\n> +- c: (content) the content parent identifies the commit that this meta-commit is\n> + describing.\n> +- r: (replaced) indicates that this parent is made obsolete by the content\n> + parent.\n> +- o: (origin) indicates that the content parent was generated by cherry-picking\n> + this parent.\n> +- a: (abandoned) used in place of a content parent for abandoned changes. Points\n> + to the final content commit for the change at the time it was abandoned.\n> +\n> +There must be exactly one content or abandoned parent for each meta-commit and\n> +it is always the first parent. The content commit will always be a normal commit\n> +and not a meta-commit. However, future versions of git may create meta-commits\n> +for other meta-commits and the fsck tool must be aware of this for forwards\n> +compatibility.\n> +\n> +A meta-commit can have zero or more replaced parents. An amend operation creates\n> +a single replaced parent. A merge used to resolve divergence (see divergence,\n> +below) will create multiple replaced parents. A meta-commit may have no\n> +replaced parents if it describes a cherry-pick or squash merge that copies one\n> +or more commits but does not replace them.\n> +\n> +A meta-commit can have zero or more origin parents. A cherry-pick creates a\n> +single origin parent. Certain types of squash merge will create multiple origin\n> +parents. Origin parents don't directly cause their origin to become obsolete,\n> +but are used when computing blame or locating a merge base. The section\n> +on obsolescence over cherry-picks describes how the evolve command uses\n> +origin parents.\n> +\n> +A replaced parent or origin parent may be either a normal commit (indicating\n> +the oldest-known version of a change) or another meta-commit (for a change that\n> +has already been modified one or more times).\n> +\n> +The parent-type field needs to go after the committer field since git's rules\n> +for forwards-compatibility require that new fields to be at the end of the\n> +header. Putting a new field in the middle of the header would break fsck.\n> +\n> +The presence of an abandoned parent indicates that the change should be pruned\n> +by the evolve command, and removed from the repository's history. Any follow-up\n> +changes should rebased onto the parent of the pruned commit. The abandoned\n> +parent points to the version of the change that should be restored if the user\n> +attempts to restore the change.\n> +\n> +Changes\n> +-------\n> +A branch of meta-commits describes how a commit was produced and what previous\n> +commits it is based on. It is also an identifier for a thing the user is\n> +currently working on. We refer to such a meta-branch as a change.\n> +\n> +Local changes are stored in the new refs/metas namespace. Remote changes are\n> +stored in the refs/remote//metas namespace.\n> +\n> +The list of changes in refs/metas is more than just a mechanism for the evolve\n> +command to locate obsolete commits. It is also a convenient list of all of a\n> +user’s work in progress and their current state - a list of things they’re\n> +likely to want to come back to.\n> +\n> +Strictly speaking, it is the presence of the branch in the refs/metas namespace\n> +that marks a branch as being a change, not the fact that it points to a\n> +metacommit. Metacommits are only created when a commit is amended or rebased, so\n> +in the case where a change points to a commit that has never been modified, the\n> +change points to that initial commit rather than a metacommit.\n> +\n> +Changes are also stored in the refs/hiddenmetas namespace. Hiddenmetas holds\n> +metadata for historical changes that are not currently in progress by the user.\n> +Commands like filter-branch and other bulk import commands create metadata in\n> +this namespace.\n> +\n> +Note that the changes in hiddenmetas get special treatment in several ways:\n> +\n> +- They are not cleaned up automatically once merged, since it is expected that\n> + they refer to historical changes.\n> +- User commands that modify changes don't append to these changes as they would\n> + to a change in refs/metas.\n> +- They are not displayed when the user lists their local changes.\n> +\n> +Obsolescence\n> +------------\n> +A commit is considered obsolete if it is reachable from the “replaces” edges\n> +anywhere in the history of a change and it isn’t the head of that change.\n> +Commits may be the content for 0 or more meta-commits. If the same commit\n> +appears in multiple changes, it is not obsolete if it is the head of any of\n> +those changes.\n> +\n> +Note that there is an exception to this rule. The metas namespace takes\n> +precedence over the hiddenmetas namespace for the purpose of obsolescence. That\n> +is, if a change appears in a replaces edge of a change in the metas namespace,\n> +it is obsolete even if it also appears as the head of a change in the\n> +hiddenmetas namespace.\n> +\n> +This special case prevents the hiddenmetas namespace from creating divergence\n> +with the user's work in progress, and allows the user to resolve historical\n> +divergence by creating new changes in the metas namespace.\n> +\n> +Divergence\n> +----------\n> +From the user’s perspective, two changes are divergent if they both ask for\n> +different replacements to the same commit. More precisely, a target commit is\n> +considered divergent if there is more than one commit at the head of a change in\n> +refs/metas that leads to the target commit via an unbroken chain of “replaces”\n> +parents.\n> +\n> +Much like a merge conflict, divergence is a situation that requires user\n> +intervention to resolve. The evolve command will stop when it encounters\n> +divergence and prompt the user to resolve the problem. Users can solve the\n> +problem in several ways:\n> +\n> +- Discard one of the changes (by deleting its change branch).\n> +- Merge the two changes (producing a single change branch).\n> +- Copy one of the changes (keep both commits, but one of them gets a new\n> + metacommit appended to its history that is connected to its predecessor via an\n> + origin edge rather than a replaces edge. That new change no longer obsoletes\n> + the original.)\n> +\n> +Obsolescence across cherry-picks\n> +--------------------------------\n> +By default the evolve command will treat cherry-picks and squash merges as being\n> +completely separate from the original. Further amendments to the original commit\n> +will have no effect on the cherry-picked copy. However, this behavior may not be\n> +desirable in all circumstances.\n> +\n> +The evolve command may at some point support an option to look for cases where\n> +the source of a cherry-pick or squash merge has itself been amended, and\n> +automatically apply that same change to the cherry-picked copy. In such cases,\n> +it would traverse origin edges rather than ignoring them, and would treat a\n> +commit with origin edges as being obsolete if any of its origins were obsolete.\n> +\n> +Garbage collection\n> +------------------\n> +For GC purposes, meta-commits are normal commits. Just as a commit causes its\n> +parents and tree to be retained, a meta-commit also causes its parents to be\n> +retained.\n> +\n> +Change creation\n> +---------------\n> +Changes are created automatically whenever the user runs a command like “commit”\n> +that has the semantics of creating a new change. They also move forward\n> +automatically even if they’re not checked out. For example, whenever the user\n> +runs a command like “commit --amend” that modifies a commit, all branches in\n> +refs/metas that pointed to the old commit move forward to point to its\n> +replacement instead. This also happens when the user is working from a detached\n> +head.\n> +\n> +This does not mean that every commit has a corresponding change. By default,\n> +changes only exist for recent locally-created commits. Users may explicitly pull\n> +changes from other users or keep their changes around for a long time, but\n> +either behavior requires a user to opt-in. Code review systems like gerrit may\n> +also choose to keep changes around forever.\n> +\n> +Note that the changes in refs/metas serve a dual function as both a way to\n> +identify obsolete changes and as a way for the user to keep track of their work\n> +in progress. If we were only concerned with identifying obsolete changes, it\n> +would be sufficient to create the change branch lazily the first time a commit\n> +is obsoleted. Addressing the second use - of refs/metas as a mechanism for\n> +keeping track of work in progress - is the reason for eagerly creating the\n> +change on first commit.\n> +\n> +Change naming\n> +-------------\n> +When a change is first created, the only requirement for its name is that it\n> +must be unique. Good names would also serve as useful mnemonics and be easy to\n> +type. For example, a short word from the commit message containing no numbers or\n> +special characters and that shows up with low frequency in other commit messages\n> +would make a good choice.\n> +\n> +Different users may prefer different heuristics for their change names. For this\n> +reason a new hook will be introduced to compute change names. Git will invoke\n> +the hook for all newly-created changes and will append a numeric suffix if the\n> +name isn’t unique. The default heuristics are not specified by this proposal and\n> +may change during implementation.\n> +\n> +Change deletion\n> +---------------\n> +Changes are normally only interesting to a user while a commit is still in\n> +development and under review. Once the commit has submitted wherever it is\n> +going, its change can be discarded.\n> +\n> +The normal way of deleting changes makes this easy to do - changes are deleted\n> +by the evolve command when it detects that the change is present in an upstream\n> +branch. It does this in two ways: if the latest commit in a change either shows\n> +up in the branch history or the change becomes empty after a rebase, it is\n> +considered merged and the change is discarded. In this context, an “upstream\n> +branch” is any branch passed in as the upstream argument of the evolve command.\n> +\n> +In case this sometimes deletes a useful change, such automatic deletions are\n> +recorded in the reflog allowing them to be easily recovered.\n> +\n> +Sharing changes\n> +---------------\n> +Change histories are shared by pushing or fetching meta-commits and change\n> +branches. This provides users with a lot of control of what to share and\n> +repository implementations with control over what to retain.\n> +\n> +Users that only want to share the content of a commit can do so by pushing the\n> +commit itself as they currently would. Users that want to share an edit history\n> +for the commit can push its change, which would point to a meta-commit rather\n> +than the commit itself if there is any history to share. Note that multiple\n> +changes can refer to the same commits, so it’s possible to construct and push a\n> +different history for the same commit in order to remove sensitive or irrelevant\n> +intermediate states.\n> +\n> +Imagine the user is working on a change “mychange” that is currently the latest\n> +commit on master. They have two ways to share it:\n> +\n> +# User shares just a commit without its history\n> +> git push origin master\n> +\n> +# User shares the full history of the commit to a review system\n> +> git push origin metas/mychange:refs/for/master\n> +\n> +# User fetches a collaborator’s modifications to their change\n> +> git fetch remotename metas/mychange\n> +# Which updates the ref remote/remotename/metas/mychange\n> +\n> +This will cause more intermediate states to be shared with the server than would\n> +have been shared previously. A review system like gerrit would need to keep\n> +track of which states had been explicitly pushed versus other intermediate\n> +states in order to de-emphasize (or hide) the extra intermediate states from the\n> +user interface.\n> +\n> +Merge-base\n> +----------\n> +Merge-base will be changed to search the meta-commit graph for common ancestors\n> +as well as the commit graph, and will generally prefer results from the\n> +meta-commit graph over the commit graph. Merge-base will consider meta-commits\n> +from all changes, and will traverse both origin and obsolete edges.\n> +\n> +The reason for this is that - when merging two versions of the same commit\n> +together - an earlier version of that same commit will usually be much more\n> +similar than their common parent. This should make the workflow of collaborating\n> +on unsubmitted patches as convenient as the workflow for collaborating in a\n> +topic branch by eliminating repeated merges.\n> +\n> +Configuration\n> +-------------\n> +The core.enableChanges configuration variable enables the creation and update\n> +of change branches. This is enabled by default.\n> +\n> +User interface\n> +--------------\n> +All git porcelain commands that create commits are classified as having one of\n> +four behaviors: modify, create, copy, or import. These behaviors are discussed\n> +in more detail below.\n> +\n> +Modify commands\n> +---------------\n> +Modification commands (commit --amend, rebase) will mark the old commit as\n> +obsolete by creating a new meta-commit that references the old one as a\n> +replaced parent. In the event that multiple changes point to the same commit,\n> +this is done independently for every such change.\n> +\n> +More specifically, modifications work like this:\n> +\n> +1. Locate all existing changes for which the old commit is the content for the\n> + head of the change branch. If no such branch exists, create one that points\n> + to the old commit. Changes that include this commit in their history but not\n> + at their head are explicitly not included.\n> +2. For every such change, create a new meta-commit that references the new\n> + commit as its content and references the old head of the change as a\n> + replaced parent.\n> +3. Move the change branch forward to point to the new meta-commit.\n> +\n> +Copy commands\n> +-------------\n> +Copy commands (cherry-pick, merge --squash) create a new meta-commit that\n> +references the old commits as origin parents. Besides the fact that the new\n> +parents are tagged differently, copy commands work the same way as modify\n> +commands.\n> +\n> +Create commands\n> +---------------\n> +Creation commands (commit, merge) create a new commit and a new change that\n> +points to that commit. The do not create any meta-commits.\n> +\n> +Import commands\n> +---------------\n> +Import commands (fetch, pull) do not create any new meta-commits or changes\n> +unless that is specifically what they are importing. For example, the fetch\n> +command would update remote/origin/metas/change35 and fetch all referenced\n> +meta-commits if asked to do so directly, but it wouldn’t create any changes or\n> +meta-commits for commits discovered on the master branch when running “git fetch\n> +origin master”.\n> +\n> +Other commands\n> +--------------\n> +Some commands don’t fit cleanly into one of the above categories.\n> +\n> +Semantically, filter-branch should be treated as a modify command, but doing so\n> +is likely to create a lot of irrelevant clutter in the changes namespace and the\n> +large number of extra change refs may introduce performance problems. We\n> +recommend treating filter-branch as an import command initially, but making it\n> +behave more like a modify command in future follow-up work. One possible\n> +solution may be to treat commits that are part of existing changes as being\n> +modified but to avoid creating changes for other rewritten changes. Another\n> +solution may be to record the modifications as changes in the hiddenmetas\n> +namespace.\n> +\n> +Once the evolve command can handle obsolescence across cherry-picks, such\n> +cherry-picks will result in a hybrid move-and-copy operation. It will create\n> +cherry-picks that replace other cherry-picks, which will have both origin edges\n> +(pointing to the new source commit being picked) and replacement edges (pointing\n> +to the previous cherry-pick being replaced).\n> +\n> +Evolve\n> +------\n> +The evolve command performs the correct sequence of rebases such that no change\n> +has an obsolete parent. The syntax looks like this:\n> +\n> +git evolve [upstream…]\n> +\n> +It takes an optional list of upstream branches. All changes whose parent shows\n> +up in the history of one of the upstream branches will be rebased onto the\n> +upstream branch before resolving obsolete parents.\n> +\n> +Any change whose latest state is found in an upstream branch (or that ends up\n> +empty after rebase) will be deleted. This is the normal mechanism for deleting\n> +changes. Changes are created automatically on the first commit, and are deleted\n> +automatically when evolve determines that they’ve been merged upstream.\n> +\n> +Orphan commits are commits with obsolete parents. The evolve command then\n> +repeatedly rebases orphan commits with non-orphan parents until there are either\n> +no orphan commits left, or a merge conflict is discovered. It will also\n> +terminate if it detects a divergent parent or a cycle that can't be resolved\n> +using any of the enabled transformations.\n> +\n> +When evolve discovers divergence, it will first check if it can resolve the\n> +divergence automatically using one of its enabled transformations. Supported\n> +transformations are:\n> +\n> +- Check if the user has already merged the divergent changes in a follow-up\n> + change. That is, look for an existing merge in a follow-up change where all\n> + the parents are divergent versions of the same change. Squash that merge with\n> + its parents and use the result as the resolution for the divergence.\n> +\n> +- Attempt to auto-merge all the divergent changes (disabled by default).\n> +\n> +Each of the transformations can be enabled or disabled by command line options.\n> +\n> +Cycles can occur when two changes reference one another as parents. This can\n> +happen when both changes use an obsolete version of the other change as their\n> +parent. Although there are never cycles in the commit graph, users can create\n> +cycles in the change graph by rebasing changes onto obsolete commits. The evolve\n> +command has a transformation that will detect and break cycles by arbitrarily\n> +picking one of the changes to go first. If this generates a merge conflict,\n> +it tries each of the other changes in sequence to see if any ordering merges\n> +cleanly. If no possible ordering merges cleanly, it picks one and terminates\n> +to let the user resolve the merge conflict.\n> +\n> +If the working tree is dirty, evolve will attempt to stash the user's changes\n> +before applying the evolve and then reapply those changes afterward, in much\n> +the same way as rebase --autostash does.\n> +\n> +Checkout\n> +--------\n> +Running checkout on a change by name has the same effect as checking out a\n> +detached head pointing to the latest commit on that change-branch. There is no\n> +need to ever have HEAD point to a change since changes always move forward when\n> +necessary, no matter what branch the user has checked out\n> +\n> +Meta-commits themselves cannot be checked out by their hash.\n> +\n> +Reset\n> +-----\n> +Resetting a branch to a change by name is the same as resetting to the content\n> +(or abandoned) commit at that change’s head.\n> +\n> +Commit\n> +------\n> +Commit --amend gets modify semantics and will move existing changes forward. The\n> +normal form of commit gets create semantics and will create a new change.\n> +\n> +$ touch foo && git add . && git commit -m \"foo\" && git tag A\n> +$ touch bar && git add . && git commit -m \"bar\" && git tag B\n> +$ touch baz && git add . && git commit -m \"baz\" && git tag C\n> +\n> +This produces the following commits:\n> +A(tree=[foo])\n> +B(tree=[foo, bar], parent=A)\n> +C(tree=[foo, bar, baz], parent=B)\n> +\n> +...along with three changes:\n> +metas/foo = A\n> +metas/bar = B\n> +metas/baz = C\n> +\n> +Running commit --amend does the following:\n> +$ git checkout B\n> +$ touch zoom && git add . && git commit --amend -m \"baz and zoom\"\n> +$ git tag D\n> +\n> +Commits:\n> +A(tree=[foo])\n> +B(tree=[foo, bar], parent=A)\n> +C(tree=[foo, bar, baz], parent=B)\n> +D(tree=[foo, bar, zoom], parent=A)\n> +Dmeta(content=D, obsolete=B)\n> +\n> +Changes:\n> +metas/foo = A\n> +metas/bar = Dmeta\n> +metas/baz = C\n> +\n> +Merge\n> +-----\n> +Merge gets create, modify, or copy semantics based on what is being merged and\n> +the options being used.\n> +\n> +The --squash version of merge gets copy semantics (it produces a new change that\n> +is marked as a copy of all the original changes that were squashed into it).\n> +\n> +The “modify” version of merge replaces both of the original commits with the\n> +resulting merge commit. This is one of the standard mechanisms for resolving\n> +divergence. The parents of the merge commit are the parents of the two commits\n> +being merged. The resulting commit will not be a merge commit if both of the\n> +original commits had the same parent or if one was the parent of the other.\n> +\n> +The “create” version of merge creates a new change pointing to a merge commit\n> +that has both original commits as parents. The result is what merge produces now\n> +- a new merge commit. However, this version of merge doesn’t directly resolve\n> +divergence.\n> +\n> +To select between these two behaviors, merge gets new “--amend” and “--noamend”\n> +options which select between the “create” and “modify” behaviors respectively,\n> +with noamend being the default.\n> +\n> +For example, imagine we created two divergent changes like this:\n> +\n> +$ touch foo && git add . && git commit -m \"foo\" && git tag A\n> +$ touch bar && git add . && git commit -m \"bar\" && git tag B\n> +$ touch baz && git add . && git commit --amend -m \"bar and baz\"\n> +$ git tag C\n> +$ git checkout B\n> +$ touch bam && git add . && git commit --amend -m \"bar and bam\"\n> +$ git tag D\n> +\n> +At this point the commit graph looks like this:\n> +\n> +A(tree=[foo])\n> +B(tree=[bar], parent=A)\n> +C(tree=[bar, baz], parent=A)\n> +D(tree=[bar, bam], parent=A)\n> +Cmeta(content=C, obsoletes=B)\n> +Dmeta(content=D, obsoletes=B)\n> +\n> +There would be three active changes with heads pointing as follows:\n> +\n> +metas/changeA=A\n> +metas/changeB=Cmeta\n> +metas/changeB2=Dmeta\n> +\n> +ChangeB and changeB2 are divergent at this point. Lets consider what happens if\n> +perform each type of merge between changeB and changeB2.\n> +\n> +Merge example: Amend merge\n> +One way to resolve divergent changes is to use an amend merge. Recall that HEAD\n> +is currently pointing to D at this point.\n> +\n> +$ git merge --amend metas/changeB\n> +\n> +Here we’ve asked for an amend merge since we’re trying to resolve divergence\n> +between two versions of the same change. There are no conflicts so we end up\n> +with this:\n> +\n> +E(tree=[bar, baz, bam], parent=A)\n> +Emeta(content=E, obsoletes=[Cmeta, Dmeta])\n> +\n> +With the following branches:\n> +\n> +metas/changeA=A\n> +metas/changeB=Emeta\n> +metas/changeB2=Emeta\n> +\n> +Notice that the result of the “amend merge” is a replacement for C and D rather\n> +than a new commit with C and D as parents (as a normal merge would have\n> +produced). The parents of the amend merge are the parents of C and D which - in\n> +this case - is just A, so the result is not a merge commit. Also notice that\n> +changeB and changeB2 are now aliases for the same change.\n> +\n> +Merge example: Noamend merge\n> +Consider what would have happened if we’d used a noamend merge instead. Recall\n> +that HEAD was at D and our branches looked like this:\n> +\n> +metas/changeA=A\n> +metas/changeB=Cmeta\n> +metas/changeB2=Dmeta\n> +\n> +$ git merge --noamend metas/changeB\n> +\n> +That would produce the sort of merge we’d normally expect today:\n> +\n> +F(tree=[bar, baz, bam], parent=[C, D])\n> +\n> +And our changes would look like this:\n> +metas/changeA=A\n> +metas/changeB=Cmeta\n> +metas/changeB2=Dmeta\n> +metas/changeF=F\n> +\n> +In this case, changeB and changeB2 are still divergent and we’ve created a new\n> +change for our merge commit. However, this is just a temporary state. The next\n> +time we run the “evolve” command, it will discover the divergence but also\n> +discover the merge commit F that resolves it. Evolve will suggest converting F\n> +into an amend merge in order to resolve the divergence and will display the\n> +command for doing so.\n> +\n> +Rebase\n> +------\n> +In general the rebase command is treated as a modify command. When a change is\n> +rebased, the new commit replaces the original.\n> +\n> +Rebase --abort is special. Its intent is to restore git to the state it had\n> +prior to running rebase. It should move back any changes to point to the refs\n> +they had prior to running rebase and delete any new changes that were created as\n> +part of the rebase. To achieve this, rebase will save the state of all changes\n> +in refs/metas prior to running rebase and will restore the entire namespace\n> +after rebase completes (deleting any newly-created changes). Newly-created\n> +metacommits are left in place, but will have no effect until garbage collected\n> +since metacommits are only used if they are reachable from refs/metas.\n> +\n> +Change\n> +------\n> +The “change” command can be used to list, rename, reset or delete change. It has\n> +a number of subcommands.\n> +\n> +The \"list\" subcommand lists local changes. If given the -r argument, it lists\n> +remote changes.\n> +\n> +The \"rename\" subcommand renames a change, given its old and new name. If the old\n> +name is omitted and there is exactly one change pointing to the current HEAD,\n> +that change is renamed. If there are no changes pointing to the current HEAD,\n> +one is created with the given name.\n> +\n> +The \"forget\" subcommand deletes a change by deleting its ref from the metas/\n> +namespace. This is the normal way to delete extra aliases for a change if the\n> +change has more than one name. By default, this will refuse to delete the last\n> +alias for a change if there are any other changes that reference this change as\n> +a parent.\n> +\n> +The \"update\" subcommand adds a new state to a change. It uses the default\n> +algorithm for assigning change names. If the content commit is omitted, HEAD is\n> +used. If given the optional --force argument, it will overwrite any existing\n> +change of the same name. This latter form of \"update\" can be used to effectively\n> +reset changes.\n> +\n> +The \"update\" command can accept any number of --origin and --replace arguments.\n> +If any are present, the resulting change branch will point to a metacommit\n> +containing the given origin and replacement edges.\n> +\n> +The \"abandon\" command deletes a change using obsolescence markers. It marks the\n> +change as being obsolete and having been replaced by its parent. If given no\n> +arguments, it applies to the current commit. Running evolve will cause any\n> +abandoned changes to be removed from the branch. Any child changes will be\n> +reparented on top of the parent of the abandoned change. If the current change\n> +is abandoned, HEAD will move to point to its parent.\n> +\n> +The \"restore\" command restores a previously-abandoned change.\n> +\n> +The \"prune\" command deletes all obsolete changes and all changes that are\n> +present in the given branch. Note that such changes can be recovered from the\n> +reflog.\n> +\n> +Combined with the GC protection that is offered, this is intended to facilitate\n> +a workflow that relies on changes instead of branches. Users could choose to\n> +work with no local branches and use changes instead - both for mailing list and\n> +gerrit workflows.\n> +\n> +Log\n> +---\n> +When a commit is shown in git log that is part of a change, it is decorated with\n> +extra change information. If it is the head of a change, the name of the change\n> +is shown next to the list of branches. If it is obsolete, it is decorated with\n> +the text “obsolete, commits behind ”.\n> +\n> +Log gets a new --obslog argument indicating that the obsolescence graph should\n> +be followed instead of the commit graph. This also changes the default\n> +formatting options to make them more appropriate for viewing different\n> +iterations of the same commit.\n> +\n> +Pull\n> +----\n> +\n> +Pull gets an --evolve argument that will automatically attempt to run \"evolve\"\n> +on any affected branches after pulling.\n> +\n> +We also introduce an \"evolve\" enum value for the branch..rebase config\n> +value. When set, the evolve behavior will happen automatically for that branch\n> +after every pull even if the --evolve argument is not used.\n> +\n> +Next\n> +----\n> +\n> +The \"next\" command will reset HEAD to a non-obsolete commit that refers to this\n> +change as its parent. If there is more than one such change, the user will be\n> +prompted. If given the --evolve argument, the next commit will be evolved if\n> +necessary first.\n> +\n> +The \"next\" command can be thought of as the opposite of\n> +\"git reset --hard HEAD^\" in that it navigates to a child commit rather than a\n> +parent.\n> +\n> +Prev\n> +----\n> +\n> +The \"prev\" command will reset HEAD to the latest version of the parent change.\n> +If the parent change isn't obsolete, this is equivalent to\n> +\"git reset --hard HEAD^\". If the parent commit is obsolete, it resets to the\n> +latest replacement for the parent commit.\n> +\n> +Other options considered\n> +========================\n> +We considered several other options for storing the obsolescence graph. This\n> +section describes the other options and why they were rejected.\n> +\n> +Commit header\n> +-------------\n> +Add an “obsoletes” field to the commit header that points backwards from a\n> +commit to the previous commits it obsoletes.\n> +\n> +Pros:\n> +- Very simple\n> +- Easy to traverse from a commit to the previous commits it obsoletes.\n> +Cons:\n> +- Adds a cost to the storage format, even for commits where the change history\n> + is uninteresting.\n> +- Unconditionally prevents the change history from being garbage collected.\n> +- Always causes the change history to be shared when pushing or pulling changes.\n> +\n> +Git notes\n> +---------\n> +Instead of storing obsolescence information in metacommits, the metacommit\n> +content could go in a new notes namespace - say refs/notes/metacommit. Each note\n> +would contain the list of obsolete and origin parents. An automerger could\n> +be supplied to make it easy to merge the metacommit notes from different remotes.\n> +\n> +Pros:\n> +- Easy to locate all commits obsoleted by a given commit (since there would only\n> + be one metacommit for any given commit).\n> +Cons:\n> +- Wrong GC behavior (obsolete commits wouldn’t automatically be retained by GC)\n> + unless we introduced a special case for these kinds of notes.\n> +- No way to selectively share or pull the metacommits for one specific change.\n> + It would be all-or-nothing, which would be expensive. This could be addressed\n> + by changes to the protocol, but this would be invasive.\n> +- Requires custom auto-merging behavior on fetch.\n> +\n> +Tags\n> +----\n> +Put the content of the metacommit in a message attached to tag on the\n> +replacement commit. This is very similar to the git notes approach and has the\n> +same pros and cons.\n> +\n> +Simple forward references\n> +-------------------------\n> +Record an edge from an obsolete commit to its replacement in this form:\n> +\n> +refs/obsoletes/\n> +\n> +pointing to commit as an indication that B is the replacement for the\n> +obsolete commit A.\n> +\n> +Pros:\n> +- Protects from being garbage collected.\n> +- Fast lookup for the evolve operation, without additional search structures\n> + (“what is the replacement for ?” is very fast).\n> +\n> +Cons:\n> +- Can’t represent divergence (which is a P0 requirement).\n> +- Creates lots of refs (which can be inefficient)\n> +- Doesn’t provide a way to fetch only refs for a specific change.\n> +- The obslog command requires a search of all refs.\n> +\n> +Complex forward references\n> +--------------------------\n> +Record an edge from an obsolete commit to its replacement in this form:\n> +\n> +refs/obsoletes//obs_\n> +\n> +Pointing to commit as an indication that B is the replacement for obsolete\n> +commit A.\n> +\n> +Pros:\n> +- Permits sharing and fetching refs for only a specific change.\n> +- Supports divergence\n> +- Protects from being garbage collected.\n> +\n> +Cons:\n> +- Creates lots of refs, which is inefficient.\n> +- Doesn’t provide a good lookup structure for lookups in either direction.\n> +\n> +Backward references\n> +-------------------\n> +Record an edge from a replacement commit to the obsolete one in this form:\n> +\n> +refs/obsolescences/\n> +\n> +Cons:\n> +- Doesn’t provide a way to resolve divergence (which is a P0 requirement).\n> +- Doesn’t protect from being garbage collected (which could be fixed by\n> + combining this with a refs/metas namespace, as in the metacommit variant).\n> +\n> +Obsolescences file\n> +------------------\n> +Create a custom file (or files) in .git recording obsolescences.\n> +\n> +Pros:\n> +- Can store exactly the information we want with exactly the performance we want\n> + for all operations. For example, there could be a disk-based hashtable\n> + permitting constant time lookups in either direction.\n> +\n> +Cons:\n> +- Handling GC, pushing, and pulling would all require custom solutions. GC\n> + issues could be addressed with a repository format extension.\n> +\n> +Squash points\n> +-------------\n> +We treat changes like topic branches, and use special squash points to mark\n> +places in the commit graph that separate changes.\n> +\n> +We create and update change branches in refs/metas at the same time we\n> +would have in the metacommit proposal. However, rather than pointing to a\n> +metacommit branch they point to normal commits and are treated as “squash\n> +points” - markers for sequences of commits intended to be squashed together on\n> +submission.\n> +\n> +Amends and rebases work differently than they do now. Rather than actually\n> +containing the desired state of a commit, they contain a delta from the previous\n> +version along with a squash point indicating that the preceding changes are\n> +intended to be squashed on submission. Specifically, amends would become new\n> +changes and rebases would become merge commits with the old commit and new\n> +parent as parents.\n> +\n> +When the changes are finally submitted, the squashes are executed, producing the\n> +final version of the commit.\n> +\n> +In addition to the squash points, git would maintain a set of “nosquash” tags\n> +for commits that were used as ancestors of a change that are not meant to be\n> +included in the squash.\n> +\n> +For example, if we have this commit graph:\n> +\n> +A(...)\n> +B(parent=A)\n> +C(parent=B)\n> +\n> +...and we amend B to produce D, we’d get:\n> +\n> +A(...)\n> +B(parent=A)\n> +C(parent=B)\n> +D(parent=B)\n> +\n> +...along with a new change branch indicating D should be squashed with its\n> +parents when submitted:\n> +\n> +metas/changeB = D\n> +metas/changeC = C\n> +\n> +We’d also create a nosquash tag for A indicating that A shouldn’t be included\n> +when changeB is squashed.\n> +\n> +If a user amends the change again, they’d get:\n> +\n> +A(...)\n> +B(parent=A)\n> +C(parent=B)\n> +D(parent=B)\n> +E(parent=D)\n> +\n> +metas/changeB = E\n> +metas/changeC = C\n> +\n> +Pros:\n> +- Good GC behavior.\n> +- Provides a natural way to share changes (they’re just normal branches).\n> +- Merge-base works automatically without special cases.\n> +- Rewriting the obslog would be easy using existing git commands.\n> +- No new data types needed.\n> +Cons:\n> +- No way to connect the squashed version of a change to the original, so no way\n> + to automatically clean up old changes. This also means users lose all benefits\n> + of the evolve command if they prematurely squash their commits. This may occur\n> + if a user thinks a change is ready for submission, squashes it, and then later\n> + discovers an additional change to make.\n> +- Histories would look very cluttered (users would see all previous edits to\n> + their commit in the commit log, and all previous rebases would show up as\n> + merges). Could be quite hard for users to tell what is going on. (Possible\n> + fix: also implement a new smart log feature that displays the log as though\n> + the squashes had occurred).\n> +- Need to change the current behavior of current commands (like amend and\n> + rebase) in ways that will be unexpected to many users.\n> --\n> gitgitgadget\n>\n"},{"id":"463596","messageId":"e301d4c1-8f80-b9cf-142b-cd7bd183d625@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"Re: [PATCH 00/10] Add the Git Change command","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-25T08:39:45Z","receivedAt":"2022-09-25T08:40:54Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Christophe\n\nOn 23/09/2022 19:55, Christophe Poucet via GitGitGadget wrote:\n> I'm reviving the original git evolve work that was started by\n> sxenos@google.com\n> (https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n> \n> This work is intended to make it easier to deal with stacked changes.\n> \n> The following set of patches introduces the design doc on the evolve command\n> as well as the basics of the git change command.\n\nThanks for picking this up, having an evolve command would be a really \nuseful addition to git. I read the final four patches as I was \ninterested to see how a user would use \"git change\" to track changes to \na set of commits. Unfortunately because there are no tests and scant \ndocumentation there are no examples of how to do this. Looking at the \npatches I felt like it would have been helpful to mark them as RFC to \nindicate that the author is requesting feedback but does not consider \nthem ready for merging.\n\nI'm confused as to why the command is called \"change\" (which I don't \nfind particularly descriptive) when every patch subject is \"evolve\". It \ndefinitely makes sense to request feedback on a large topic like this \nbefore everything is implemented but I'd be nervous of merging the early \nstages before there is a working evolve command. For an example of a \nsuccessful multipart topic see \nhttps://lore.kernel.org/git/pull.1248.git.1654545325.gitgitgadget@gmail.com/ \nKnowing the author of that series the commit messages should also give \nyou a good idea of the level of detail expected.\n\nBest Wishes\n\nPhillip\n\n> Chris Poucet (4):\n> sha1-array: implement oid_array_readonly_contains\n> ref-filter: add the metas namespace to ref-filter\n> evolve: add delete command\n> evolve: add documentation for `git change`\n> \n> Stefan Xenos (6):\n> technical doc: add a design doc for the evolve command\n> evolve: add support for parsing metacommits\n> evolve: add the change-table structure\n> evolve: add support for writing metacommits\n> evolve: implement the git change command\n> evolve: add the git change list command\n> \n> .gitignore | 1 +\n> Documentation/git-change.txt | 55 ++\n> Documentation/technical/evolve.txt | 1051 ++++++++++++++++++++++++++++\n> Makefile | 4 +\n> builtin.h | 1 +\n> builtin/change.c | 342 +++++++++\n> change-table.c | 179 +++++\n> change-table.h | 132 ++++\n> git.c | 1 +\n> metacommit-parser.c | 110 +++\n> metacommit-parser.h | 19 +\n> metacommit.c | 404 +++++++++++\n> metacommit.h | 58 ++\n> oid-array.c | 12 +\n> oid-array.h | 7 +\n> ref-filter.c | 10 +-\n> ref-filter.h | 8 +-\n> t/helper/test-oid-array.c | 6 +\n> t/t0064-oid-array.sh | 22 +\n> 19 files changed, 2418 insertions(+), 4 deletions(-)\n> create mode 100644 Documentation/git-change.txt\n> create mode 100644 Documentation/technical/evolve.txt\n> create mode 100644 builtin/change.c\n> create mode 100644 change-table.c\n> create mode 100644 change-table.h\n> create mode 100644 metacommit-parser.c\n> create mode 100644 metacommit-parser.h\n> create mode 100644 metacommit.c\n> create mode 100644 metacommit.h\n> \n> \n> base-commit: 4b79ee4b0cd1130ba8907029cdc5f6a1632aca26\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1356%2Fpoucet%2Fevolve-v1\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1356/poucet/evolve-v1\n> Pull-Request: https://github.com/gitgitgadget/git/pull/1356\n"},{"id":"463597","messageId":"edad7b00-cf77-4538-9a1d-4e0e7e5d4261@gmail.com","threadId":"58504","inReplyTo":"811d516e5d272acc40835aa6bdd4f79a001f72c0.1663959325.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 10/10] evolve: add documentation for `git change`","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-25T08:41:50Z","receivedAt":"2022-09-25T08:42:28Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 23/09/2022 19:55, Chris Poucet via GitGitGadget wrote:\n> From: Chris Poucet \n> \n> Signed-off-by: Chris Poucet \n> ---\n> Documentation/git-change.txt | 55 ++++++++++++++++++++++++++++++++++++\n> 1 file changed, 55 insertions(+)\n> create mode 100644 Documentation/git-change.txt\n> \n> diff --git a/Documentation/git-change.txt b/Documentation/git-change.txt\n> new file mode 100644\n> index 00000000000..ea9a8e619b9\n> --- /dev/null\n> +++ b/Documentation/git-change.txt\n> @@ -0,0 +1,55 @@\n> +git-change(1)\n> +=============\n> +\n> +NAME\n> +----\n> +git-change - Create, list, update or delete changes\n> +\n> +SYNOPSIS\n> +--------\n> +[verse]\n> +'git change' list [...]\n> +'git change' update [-g | -n] [--force] [--replace ...] [--origin ...] [--content ]\n> +'git change' delete ...\n> +\n> +DESCRIPTION\n> +-----------\n> +\n> +`git change list`: lists all existing s.\n> +\n> +`git change delete`: deletes the given s.\n> +\n> +`git change update`: creates or updates a .\n> +\n> +If no arguments are given to `update` then a change is added to the\n> +`refs/metas/` directory, unless a change already exists for the given commit.\n> +\n> +A starts with `metas/` and represents the current change that is\n> +being worked on.\n\nIt would be really useful for users if this documentation included an \nintroduction to the concepts behind the command and examples of how they \nshould use it to track changes to a patch series.\n\nBest Wishes\n\nPhillip\n\n> +OPTIONS\n> +-------\n> +-c::\n> +--content::\n> +\tIdentifies the content commit for the change\n> +\n> +-o::\n> +--origin::\n> +\tMarks the given commit as being the origin of this commit.\n> +\n> +-r::\n> +--replace::\n> +\tMarks the given commit as being obsoleted by the new commit.\n> +\n> +-g::\n> +\t to update\n> +\n> +-n::\n> +\tIndicates that the change is new and an existing change should not be updated.\n> +\n> +--force::\n> +\tOverwite an existing change of the same name.\n> +\n> +GIT\n> +---\n> +Part of the linkgit:git[1] suite\n"},{"id":"463598","messageId":"61a01fdf-9805-ff8d-6306-ff49f31e93c2@gmail.com","threadId":"58504","inReplyTo":"914028341842a4d57e02ec42a7426d3aa83640f9.1663959325.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 07/10] evolve: implement the git change command","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-25T09:10:55Z","receivedAt":"2022-09-25T09:11:03Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n> From: Stefan Xenos \n> \n> Implement the git change update command, which\n> are sufficient for constructing change graphs.\n> \n> For example, to create a new change (a stable name) that refers to HEAD:\n> \n> git change update -c HEAD\n> \n> To record a rebase or amend in the change graph:\n> \n> git change update -c -r \n> \n> To record a cherry-pick in the change graph:\n> \n> git change update -c -o \n\nWhile it is good to have this example it would be better to have some \ndocumentation about how to use this command. It would be very helpful to \nhave the documentation added before the code so that reviewers have an \noverview of the command when they come to review the code.\n\nThe commit message should also discuss why it is called \"change\" rather \nthan \"evolve\". For more details on commit messages for this project see \n\"Describe your changes well\" in Documentation/SubmittingPatches. Having \nsome tests would make it clear how this command is intended to be used \nas well as demonstrating that the implementation works.\n\n> Signed-off-by: Stefan Xenos \n> Signed-off-by: Chris Poucet \n> ---\n\n> +struct update_state {\n> +\tint options;\n> +\tconst char* change;\n> +\tconst char* content;\n> +\tstruct string_list replace;\n> +\tstruct string_list origin;\n> +};\n> +\n> +static void init_update_state(struct update_state *state)\n> +{\n> +\tmemset(state, 0, sizeof(*state));\n> +\tstate->content = \"HEAD\";\n> +\tstring_list_init_nodup(&state->replace);\n> +\tstring_list_init_nodup(&state->origin);\n> +}\n\nIn general we prefer to use initializer macros over functions. So this \nwould become\n\n#define UPDATE_STATE_INIT {\t\t\t\\\n\t.content = \"HEAD\",\t\t\t\\\n\t.replace = STRING_LIST_INIT_NODUP,\t\\\n\t.origin = STRING_LIST_INIT_NODUP\t\\\n}\n\nand lower down we'd have\n\nstruct update_state state = UPDATE_STATE_INIT;\n\nLikewise we prefer\n\n\tstruct foo = { 0 };\n\nover\n\n\tstruct foo foo;\n\tmemset(&foo, 0, sizeof(foo));\n\n> +int cmd_change(int argc, const char **argv, const char *prefix)\n> +{\n> +\t/* No options permitted before subcommand currently */\n> +\tstruct option options[] = {\n> +\t\tOPT_END()\n> +\t};\n> +\tint result = 1;\n> +\n> +\targc = parse_options(argc, argv, prefix, options, builtin_change_usage,\n> +\t\tPARSE_OPT_STOP_AT_NON_OPTION);\n> +\n> +\tif (argc < 1)\n> +\t\tusage_with_options(builtin_change_usage, options);\n> +\telse if (!strcmp(argv[0], \"update\"))\n> +\t\tresult = change_update(argc, argv, prefix);\n\nSince Stefan wrote this code the parse options api has been improved to \nsupport sub commands so this should be updated to use that support.\n\n\nThanks again for picking up these patches, I'm excited to see an evolve \ncommand for git.\n\nBest Wishes\n\nPhillip\n"},{"id":"463607","messageId":"220926.867d1q4k7t.gmgdl@evledraar.gmail.com","threadId":"58504","inReplyTo":"61a01fdf-9805-ff8d-6306-ff49f31e93c2@gmail.com","subject":"Re: [PATCH 07/10] evolve: implement the git change command","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-26T08:23:24Z","receivedAt":"2022-09-26T08:24:15Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sun, Sep 25 2022, Phillip Wood wrote:\n\n> Hi Chris\n>\n> On 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n>> From: Stefan Xenos \n>> +static void init_update_state(struct update_state *state)\n>> +{\n>> +\tmemset(state, 0, sizeof(*state));\n>> +\tstate->content = \"HEAD\";\n>> +\tstring_list_init_nodup(&state->replace);\n>> +\tstring_list_init_nodup(&state->origin);\n>> +}\n>\n> In general we prefer to use initializer macros over functions. So this\n> would become\n>\n> #define UPDATE_STATE_INIT {\t\t\t\\\n> \t.content = \"HEAD\",\t\t\t\\\n> \t.replace = STRING_LIST_INIT_NODUP,\t\\\n> \t.origin = STRING_LIST_INIT_NODUP\t\\\n> }\n\n*nod*, although our usual style is not to indent the \"\\\"'s like that. But just:\n\n\t#define FOO { \\\n\t\t.bar = \"baz\", \\\n\t\t[...]\n"},{"id":"463608","messageId":"220926.8635ce4jox.gmgdl@evledraar.gmail.com","threadId":"58504","inReplyTo":"914028341842a4d57e02ec42a7426d3aa83640f9.1663959325.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 07/10] evolve: implement the git change command","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-26T08:25:50Z","receivedAt":"2022-09-26T08:35:38Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Sep 23 2022, Stefan Xenos via GitGitGadget wrote:\n\n> From: Stefan Xenos \n\n> +static const char * const builtin_change_usage[] = {\n> +\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> +\tNULL\n> +};\n> +\n> +static const char * const builtin_update_usage[] = {\n> +\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> +\tNULL\n> +};\n\nThis (and the corresponding later *.txt version) should indent the\noverly long -h line, probably after \"[--replace ...]\".\n\n> +struct update_state {\n> +\tint options;\n\nI think this should be an enum in your earlier 06/10. Makes things more\n\n> +\t\tdie(_(\"Failed to resolve '%s' as a valid revision.\"), committish);\n\nThis and other error should start with a lower-case letter, see\nCodingGuidelines on errors.\n\n> [...]\n> +\t\tdie(_(\"Could not parse object '%s'.\"), committish);\n\nDitto etc.\n\n> +\tint i;\n> +\tfor (i = 0; i < commitsish_list->nr; i++) {\n\nA string_list uses a size_t for a nr, not int, so lets make that \"size_t\ni\".\n\nThis both makes things more obvious, and helps some compilers spot\nunsigned v.s. signed issues.\n\n\n> +\tint i;\n\nditto size_t above...\n\n> +\tfor (i = 0; i < changes.nr; i++) {\n\n...for this iteration...\n\n> +\t\tstruct string_list_item *it = &changes.items[i];\n\n...but actually don't you just want for_each_string_list_item() instead?\n\n> +\t\tif (it->util)\n> +\t\t\tfprintf(stdout, N_(\"Updated change %s\\n\"), name);\n> +\t\telse\n> +\t\t\tfprintf(stdout, N_(\"Created change %s\\n\"), name);\n\nThe use of N_() here is wrong, you should use _(), N_() just marks\nthings for translation, but doesn't use it.\n\nWe also tend to try to avoid adding \\n in translations needlessly. And\nsince you're printing to stdout this can be:\n\n\n\tif (...)\n\t\tprintf(_(\"Updated change %s\"), name);\n\t...\n\tputchar('\\n') \n\n\n\n> +\t}\n> +\n> +\tstring_list_clear(&changes, 0);\n> +\tchange_table_clear(&chtable);\n> +\tclear_metacommit_data(&metacommit);\n> +\n> +\treturn ret;\n> +}\n> +\n> +static int change_update(int argc, const char **argv, const char* prefix)\n> +{\n> +\tint result;\n> +\tint force = 0;\n> +\tint newchange = 0;\n> +\tstruct strbuf err = STRBUF_INIT;\n> +\tstruct update_state state;\n> +\tstruct option options[] = {\n> +\t\t{ OPTION_CALLBACK, 'r', \"replace\", &state, N_(\"commit\"),\n> +\t\t\tN_(\"marks the given commit as being obsolete\"),\n> +\t\t\t0, update_option_parse_replace },\n> +\t\t{ OPTION_CALLBACK, 'o', \"origin\", &state, N_(\"commit\"),\n> +\t\t\tN_(\"marks the given commit as being the origin of this commit\"),\n> +\t\t\t0, update_option_parse_origin },\n> +\t\tOPT_BOOL('F', \"force\", &force,\n> +\t\t\tN_(\"overwrite an existing change of the same name\")),\n> +\t\tOPT_STRING('c', \"content\", &state.content, N_(\"commit\"),\n> +\t\t\t\t N_(\"identifies the new content commit for the change\")),\n> +\t\tOPT_STRING('g', \"change\", &state.change, N_(\"commit\"),\n> +\t\t\t\t N_(\"name of the change to update\")),\n> +\t\tOPT_BOOL('n', \"new\", &newchange,\n> +\t\t\tN_(\"create a new change - do not append to any existing change\")),\n> +\t\tOPT_END()\n> +\t};\n> +\n> +\tinit_update_state(&state);\n> +\n> +\targc = parse_options(argc, argv, prefix, options, builtin_update_usage, 0);\n> +\n> +\tif (force) state.options |= UPDATE_OPTION_FORCE;\n> +\tif (newchange) state.options |= UPDATE_OPTION_NOAPPEND;\n\nJust use OPT_SET_INT_F() and skip the indirection thorugh OPT_BOOL(),\nthat macro itself is a thin wrapper for OPT_SET_INT_F().\n\nI.e. you can drop these \"force\" and \"newchange\" variables, andjust set\nyour state.options directly.\n\n> +int cmd_change(int argc, const char **argv, const char *prefix)\n> +{\n> +\t/* No options permitted before subcommand currently */\n> +\tstruct option options[] = {\n> +\t\tOPT_END()\n> +\t};\n> +\tint result = 1;\n> +\n> +\targc = parse_options(argc, argv, prefix, options, builtin_change_usage,\n> +\t\tPARSE_OPT_STOP_AT_NON_OPTION);\n> +\n> +\tif (argc < 1)\n> +\t\tusage_with_options(builtin_change_usage, options);\n> +\telse if (!strcmp(argv[0], \"update\"))\n> +\t\tresult = change_update(argc, argv, prefix);\n> +\telse {\n> +\t\terror(_(\"Unknown subcommand: %s\"), argv[0]);\n> +\t\tusage_with_options(builtin_change_usage, options);\n> +\t}\n\nThis was presumably written before the recent OPT_SUBCOMMAND(), and\nshould instead use that API.\n"},{"id":"463609","messageId":"220926.86y1u634yy.gmgdl@evledraar.gmail.com","threadId":"58504","inReplyTo":"d087d467e3fe3000eb19939c2bb5e5c0723fd908.1663959325.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 09/10] evolve: add delete command","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-26T08:38:14Z","receivedAt":"2022-09-26T08:38:54Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Sep 23 2022, Chris Poucet via GitGitGadget wrote:\n\n> From: Chris Poucet \n> static const char * const builtin_change_usage[] = {\n> \tN_(\"git change list [...]\"),\n> -\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> +\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n\nHere you're just correcting a typo in an earlier commit, squash it into that one instead.\n\n> static const char * const builtin_update_usage[] = {\n> -\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> +\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n\nDitto.\n"},{"id":"463612","messageId":"CAN9+7XcvS92YU45qaBZjQ4i-g8uJOJQraXhViFQ2uJHDCAcSjg@mail.gmail.com","threadId":"58504","inReplyTo":"220926.86y1u634yy.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH 09/10] evolve: add delete command","fromName":"Chris Poucet","fromEmail":"poucet@google.com","sentAt":"2022-09-26T09:10:11Z","receivedAt":"2022-09-26T09:10:28Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"On Mon, Sep 26, 2022 at 10:38 AM Ævar Arnfjörð Bjarmason\n wrote:\n>\n>\n> On Fri, Sep 23 2022, Chris Poucet via GitGitGadget wrote:\n>\n> > From: Chris Poucet \n> > static const char * const builtin_change_usage[] = {\n> > N_(\"git change list [...]\"),\n> > - N_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> > + N_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n>\n> Here you're just correcting a typo in an earlier commit, squash it into that one instead.\n\nDone, thank you.\n>\n> > static const char * const builtin_update_usage[] = {\n> > - N_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> > + N_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n>\n> Ditto.\n\nDone, thank you.\n"},{"id":"463621","messageId":"2d886c19-09a8-c3cb-308b-b30ece02fb32@gmail.com","threadId":"58504","inReplyTo":"84588312c1d4a62ff6c6211e85b4e58ab0563daa.1663959324.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 02/10] sha1-array: implement oid_array_readonly_contains","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-26T13:08:17Z","receivedAt":"2022-09-26T14:45:46Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 23/09/2022 19:55, Chris Poucet via GitGitGadget wrote:\n> From: Chris Poucet \n> \n> Implement a \"readonly_contains\" function for oid_array that won't\n> sort the array if it is unsorted. This can be used to test containment in\n> the rare situations where the array order matters.\n> \n> The function has intentionally been given a name that is more cumbersome\n> than the \"lookup\" function, which is what most callers will will want\n> in most situations.\n\nIt certainly is more cumbersome. I also find it completely impenetrable, \nI wonder if lookup_unsorted or lookup_no_sort strike better balance \nbetween being cumbersome and descriptive.\n\n> Signed-off-by: Chris Poucet \n> ---\n> oid-array.c | 12 ++++++++++++\n> oid-array.h | 7 +++++++\n> t/helper/test-oid-array.c | 6 ++++++\n> t/t0064-oid-array.sh | 22 ++++++++++++++++++++++\n> 4 files changed, 47 insertions(+)\n> \n> diff --git a/oid-array.c b/oid-array.c\n> index 73ba76e9e9a..1e12651d245 100644\n> --- a/oid-array.c\n> +++ b/oid-array.c\n> @@ -28,6 +28,18 @@ static const struct object_id *oid_access(size_t index, const void *table)\n> \treturn &array[index];\n> }\n> \n> +int oid_array_readonly_contains(const struct oid_array *array,\n> +\t\t\t\tconst struct object_id* oid) {\n> +\tint i;\n\narray->nr is size_t so i should be as well.\n\nBest Wishes\n\nPhillip\n\n"},{"id":"463622","messageId":"fc291c07-55e9-64f8-1251-20bd2422024d@gmail.com","threadId":"58504","inReplyTo":"54e559967df55ca314e629b65927a88c7f804a98.1663959324.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 03/10] ref-filter: add the metas namespace to ref-filter","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-26T13:13:50Z","receivedAt":"2022-09-26T14:49:14Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 23/09/2022 19:55, Chris Poucet via GitGitGadget wrote:\n> From: Chris Poucet \n> \n> The metas namespace will contain refs for changes in progress. Add\n> support for searching this namespace.\n\nI assume this is to save having to write \"refs/metas/\" when we want to \nsearch for meta commits?\n\n> Signed-off-by: Chris Poucet \n> --- > diff --git a/ref-filter.h b/ref-filter.h\n> index aa0eea4ecf5..064fbef8e50 100644\n> --- a/ref-filter.h\n> +++ b/ref-filter.h\n> @@ -17,8 +17,10 @@\n> #define FILTER_REFS_BRANCHES 0x0004\n> #define FILTER_REFS_REMOTES 0x0008\n> #define FILTER_REFS_OTHERS 0x0010\n> +#define FILTER_REFS_CHANGES 0x0040\n\nIt would be nice to keep FILTER_REFS_OTHERS at the end I think (we don't \nneed to worry about abi compatibility), also what happened to 0x0020?\n\nBest Wishes\n\nPhillip\n\n> #define FILTER_REFS_ALL (FILTER_REFS_TAGS | FILTER_REFS_BRANCHES | \\\n> -\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS)\n> +\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS | \\\n> +\t\t\t\t FILTER_REFS_CHANGES)\n> #define FILTER_REFS_DETACHED_HEAD 0x0020\n> #define FILTER_REFS_KIND_MASK (FILTER_REFS_ALL | FILTER_REFS_DETACHED_HEAD)\n> \n\n\n"},{"id":"463626","messageId":"e7278794-428d-4aff-e91b-d2e6527f142d@gmail.com","threadId":"58504","inReplyTo":"2e9a4a9bd819785404e8a5343385f4fb2bc06109.1663959325.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 04/10] evolve: add support for parsing metacommits","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-26T13:27:31Z","receivedAt":"2022-09-26T14:57:20Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n> From: Stefan Xenos \n> \n> This patch adds the get_metacommit_content method, which can classify\n> commits as either metacommits or normal commits, determine whether they\n> are abandoned, and extract the content commit's object id from the\n> metacommit.\n> \n> Signed-off-by: Stefan Xenos \n> Signed-off-by: Chris Poucet \n> ---\n> Makefile | 1 +\n> metacommit-parser.c | 110 ++++++++++++++++++++++++++++++++++++++++++++\n> metacommit-parser.h | 19 ++++++++\n> 3 files changed, 130 insertions(+)\n> create mode 100644 metacommit-parser.c\n> create mode 100644 metacommit-parser.h\n> \n> diff --git a/Makefile b/Makefile\n> index cac3452edb9..b2bcc00c289 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -999,6 +999,7 @@ LIB_OBJS += merge-ort.o\n> LIB_OBJS += merge-ort-wrappers.o\n> LIB_OBJS += merge-recursive.o\n> LIB_OBJS += merge.o\n> +LIB_OBJS += metacommit-parser.o\n\nThere seems to be a problem with the indent here\n\n> LIB_OBJS += midx.o\n> LIB_OBJS += name-hash.o\n> LIB_OBJS += negotiator/default.o\n\n > diff --git a/metacommit-parser.h b/metacommit-parser.h\n > new file mode 100644\n > index 00000000000..1c74bd6d699\n > --- /dev/null\n > +++ b/metacommit-parser.h\n > @@ -0,0 +1,19 @@\n > +#ifndef METACOMMIT_PARSER_H\n > +#define METACOMMIT_PARSER_H\n > +\n > +#include \"commit.h\"\n > +#include \"hash.h\"\n > +\n > +/* Indicates a normal commit (non-metacommit) */\n > +#define METACOMMIT_TYPE_NONE 0\n > +/* Indicates a metacommit with normal content (non-abandoned) */\n > +#define METACOMMIT_TYPE_NORMAL 1\n > +/* Indicates a metacommit with abandoned content */\n > +#define METACOMMIT_TYPE_ABANDONED 2\n\nIs it possible to define these as an enum? It would make the signature \nof get_meta_commit_content() nicer.\n\n > +struct commit;\n\nWhat's this for? We're including commit.h above.\n\n > +extern int get_metacommit_content(\n > +\tstruct commit *commit, struct object_id *content);\n\n> diff --git a/metacommit-parser.c b/metacommit-parser.c\n> new file mode 100644\n> index 00000000000..70c1428bfc6\n> --- /dev/null\n> +++ b/metacommit-parser.c\n> @@ -0,0 +1,110 @@\n> +#include \"cache.h\"\n> +#include \"metacommit-parser.h\"\n> +#include \"commit.h\"\n> +\n> +/*\n> + * Search the commit buffer for a line starting with the given key. Unlike\n> + * find_commit_header, this also searches the commit message body.\n> + */\n\nThere is no explanation in the code or commit message as to why this \nfunction is needed. The documentation added in the first commit says \nthat \"parent-type\" header is a commit header. I think the answer is that \nthis series does not implement that header but uses the commit message \ninstead. That's perfectly fine for a proof of concept but it is \nprecisely the sort of detail that should be described it the commit \nmessage and probably flagged up in the cover letter.\n\n> +static const char *find_key(const char *msg, const char *key, size_t *out_len)\n> +{\n> +\tint key_len = strlen(key);\n> +\tconst char *line = msg;\n> +\n> +\twhile (line) {\n> +\t\tconst char *eol = strchrnul(line, '\\n');\n> +\n> +\t\tif (eol - line > key_len && !memcmp(line, key, key_len) &&\n> +\t\t line[key_len] == ' ') {\n> +\t\t\t*out_len = eol - line - key_len - 1;\n> +\t\t\treturn line + key_len + 1;\n> +\t\t}\n> +\t\tline = *eol ? eol + 1 : NULL;\n> +\t}\n> +\treturn NULL;\n> +}\n> +\n> +static struct commit *get_commit_by_index(struct commit_list *to_search, int index)\n> +{\n> +\twhile (to_search && index) {\n> +\t\tto_search = to_search->next;\n> +\t\tindex--;\n> +\t}\n> +\n> +\tif (!to_search)\n> +\t\treturn NULL;\n> +\n> +\treturn to_search->item;\n> +}\n\nThis function is a useful utility for struct commit_list and should live \nin commit.c. It could be used to simplify object-name.c:get_parent() for \nexample.\n\n> +/*\n> + * Writes the index of the content parent to \"result\". Returns the metacommit\n> + * type. See the METACOMMIT_TYPE_* constants.\n> + */\n> +static int index_of_content_commit(const char *buffer, int *result)\n\nI found the signature confusing as it is returning an int but that is \nnot the index. Switching to an enum for the metacommit types would \nclarify that.\n\n> +{\n> +\tint index = 0;\n> +\tint ret = METACOMMIT_TYPE_NONE;\n> +\tsize_t parent_types_size;\n> +\tconst char *parent_types = find_key(buffer, \"parent-type\",\n> +\t\t&parent_types_size);\n> +\tconst char *end;\n> +\tconst char *enum_start = parent_types;\n> +\tint enum_length = 0;\n> +\n> +\tif (!parent_types)\n> +\t\treturn METACOMMIT_TYPE_NONE;\n> +\n> +\tend = &parent_types[parent_types_size];\n> +\n> +\twhile (1) {\n> +\t\tchar next = *parent_types;\n> +\t\tif (next == ' ' || parent_types >= end) {\n> +\t\t\tif (enum_length == 1) {\n\nif enum_length != 1 then there is an error in the parent-type header and \nwe should probably bail out.\n\n> +\t\t\t\tchar first_char_in_enum = *enum_start;\n\nIt's not just the first character, it's the only character, do we really \nneed such a long variable name? (how about just calling it \"type\")\n\nI'll try and take at look at the next couple of patches later in the week.\n\nBest Wishes\n\nPhillip\n\n"},{"id":"463710","messageId":"3c61e0b3-5526-f42e-48a7-c4465d06ccb3@dunelm.org.uk","threadId":"58504","inReplyTo":"2b3a00a6702eb8fb12e45b833ca74155939588ef.1663959325.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 05/10] evolve: add the change-table structure","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-27T13:27:45Z","receivedAt":"2022-09-27T13:32:01Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n> From: Stefan Xenos \n> \n> A change table stores a list of changes, and supports efficient lookup\n> from a commit hash to the list of changes that reference that commit\n> directly.\n> \n> It can be used to look up content commits or metacommits at the head\n> of a change, but does not support lookup of commits referenced as part\n> of the commit history.\n> \n> Signed-off-by: Stefan Xenos \n> Signed-off-by: Chris Poucet \n\n > diff --git a/change-table.h b/change-table.h\n > new file mode 100644\n > index 00000000000..166b5ed8073\n > --- /dev/null\n > +++ b/change-table.h\n > @@ -0,0 +1,132 @@\n > +#ifndef CHANGE_TABLE_H\n > +#define CHANGE_TABLE_H\n > +\n > +#include \"oidmap.h\"\n > +\n > +struct commit;\n > +struct ref_filter;\n > +\n > +/**\n\nWe tend to just use '/*' rather than '/**'\n\n > + * This struct holds a list of change refs. The first element is \nstored inline,\n > + * to optimize for small lists.\n > + */\n > +struct change_list {\n > +\t/**\n > +\t * Ref name for the first change in the list, or null if none.\n > +\t *\n > +\t * This field is private. Use for_each_change_in to read.\n > +\t */\n > +\tconst char* first_refname;\n > +\t/**\n > +\t * List of additional change refs. Note that this is empty if the list\n > +\t * contains 0 or 1 elements.\n > +\t *\n > +\t * This field is private. Use for_each_change_in to read.\n > +\t */\n > +\tstruct string_list additional_refnames;\n\nSplitting this feels like a premature optimization. We don't have any \ntests yet, let alone any real-world experience using this code. Also if \nwe want to save memory for lists with a single entry why are we \nembedding the struct string_list rather than just storing a pointer to it?\n\nI think it would be simpler to use a struct strset to hold the refnames \nas we don't need the util field offered by struct string_list.\n\n > +/**\n > + * Holds information about the head of a single change.\n > + */\n > +struct change_head {\n > +\t/**\n > +\t * The location pointed to by the head of the change. May be a \ncommit or a\n > +\t * metacommit.\n > +\t */\n > +\tstruct object_id head;\n\nI found this duality between commits and metacommits rather confusing - \nwhy isn't the head always a metacommit?\n\n > +/**\n > + * Holds information about the heads of each change, and permits \neffecient\n\ns/effecient/efficient/\n\n > + * lookup from a commit to the changes that reference it directly.\n > + *\n > + * All fields should be considered private. Use the change_table \nfunctions\n > + * to interact with this struct.\n > + */\n > +struct change_table {\n > +\t/**\n > +\t * Memory pool for the objects allocated by the change table.\n > +\t */\n > +\tstruct mem_pool memory_pool;\n > +\t/* Map object_id to commit_change_list_entry structs. */\n > +\tstruct oidmap oid_to_metadata_index;\n > +\t/**\n > +\t * List of ref names. The util value points to a change_head structure\n > +\t * allocated from memory_pool.\n > +\t */\n > +\tstruct string_list refname_to_change_head;\n\nI think these days we'd use a strmap for this for O(1) lookups.\n\n > +};\n > +\n > +extern void change_table_init(struct change_table *to_initialize);\n\nThe struct change_table argument to all these functions changes its name \nmore often than a criminal on the run. I would find it much easier to \nfollow the code if we consistently called this argument \"table\"\n\n > + * Adds all changes matching the given ref filter to the given \nchange_table\n > + * struct.\n > + */\n > +extern void change_table_add_matching_filter(struct change_table \n*to_modify,\n > +\tstruct repository* repo, struct ref_filter *filter);\n\nI can't see any callers outside of change-table.c so do we really need \nto export this function.\n\n> diff --git a/change-table.c b/change-table.c\n> new file mode 100644\n> index 00000000000..c61ba29f1ed\n> --- /dev/null\n> +++ b/change-table.c\n> @@ -0,0 +1,179 @@\n> +#include \"cache.h\"\n> +#include \"change-table.h\"\n> +#include \"commit.h\"\n> +#include \"ref-filter.h\"\n> +#include \"metacommit-parser.h\"\n> +\n> +void change_table_init(struct change_table *to_initialize)\n> +{\n> +\tmemset(to_initialize, 0, sizeof(*to_initialize));\n> +\tmem_pool_init(&to_initialize->memory_pool, 0);\n> +\tto_initialize->memory_pool.block_alloc = 4*1024 - sizeof(struct mp_block);\n\nIf we're using a mempool to minimize the allocation overhead we should \nleave .block_alloc set to the default value of 1MB rather than changing \nit to 4kB\n\n> +\toidmap_init(&to_initialize->oid_to_metadata_index, 0);\n> +\tstring_list_init_dup(&to_initialize->refname_to_change_head);\n> +}\n> +\n> +static void change_list_clear(struct change_list *to_clear) {\n> +\tstring_list_clear(&to_clear->additional_refnames, 0);\n> +}\n> +\n> +static void commit_change_list_entry_clear(\n> +\tstruct commit_change_list_entry *to_clear) {\n> +\tchange_list_clear(&to_clear->changes);\n> +}\n> +\n> +void change_table_clear(struct change_table *to_clear)\n> +{\n> +\tstruct oidmap_iter iter;\n> +\tstruct commit_change_list_entry *next;\n> +\tfor (next = oidmap_iter_first(&to_clear->oid_to_metadata_index, &iter);\n> +\t\tnext;\n> +\t\tnext = oidmap_iter_next(&iter)) {\n> +\n> +\t\tcommit_change_list_entry_clear(next);\n> +\t}\n> +\n> +\toidmap_free(&to_clear->oid_to_metadata_index, 0);\n> +\tstring_list_clear(&to_clear->refname_to_change_head, 0);\n> +\tmem_pool_discard(&to_clear->memory_pool, 0);\n> +}\n> +\n> +static void add_head_to_commit(struct change_table *to_modify,\n> +\tconst struct object_id *to_add, const char *refname)\n\nI found the function and argument names rather confusing. If I've \nunderstood the code correctly then this function is adding an assoation \nbetween the commit \"to_add\" and \"refname\". Despite its name \"to_add\" may \nalready exist in the change table.\n\nThe formatting is a bit off as well (as are most of the function \ndeclarations in this patch and the next), we'd write that as\n\nstatic void add_head_to_commit(struct change_table *table,\n\t\t\t const struct object_id *to_add,\n\t\t\t const char *refname)\n\n> +{\n> +\tstruct commit_change_list_entry *entry;\n> +\n> +\t/**\n> +\t * Note: the indices in the map are 1-based. 0 is used to indicate a missing\n> +\t * element.\n> +\t */\n\nI'm confused by this comment, what indices is it talking about?\n\n> +\tentry = oidmap_get(&to_modify->oid_to_metadata_index, to_add);\n> +\tif (!entry) {\n> +\t\tentry = mem_pool_calloc(&to_modify->memory_pool, 1,\n> +\t\t\tsizeof(*entry));\n> +\t\toidcpy(&entry->entry.oid, to_add);\n> +\t\toidmap_put(&to_modify->oid_to_metadata_index, entry);\n> +\t\tstring_list_init_nodup(&entry->changes.additional_refnames);\n> +\t}\n> +\n> +\tif (!entry->changes.first_refname)\n> +\t\tentry->changes.first_refname = refname;\n> +\telse\n> +\t\tstring_list_insert(&entry->changes.additional_refnames, refname);\n\nThis is an example of the complexity added by the current definition of \nstruct change_list.\n\n> +void change_table_add(struct change_table *to_modify, const char *refname,\n> +\tstruct commit *to_add)\n> +{\n> +\tstruct change_head *new_head;\n> +\tstruct string_list_item *new_item;\n> +\tint metacommit_type;\n> +\n> +\tnew_head = mem_pool_calloc(&to_modify->memory_pool, 1,\n> +\t\tsizeof(*new_head));\n> +\n> +\toidcpy(&new_head->head, &to_add->object.oid);\n> +\n> +\tmetacommit_type = get_metacommit_content(to_add, &new_head->content);\n> +\tif (metacommit_type == METACOMMIT_TYPE_NONE)\n> +\t\toidcpy(&new_head->content, &to_add->object.oid);\n\nIf to_add is not a metacommit then the content is to_add itself, \notherwise it will have been set by the call to get_metacommit_content().\n\n> +\tnew_head->abandoned = (metacommit_type == METACOMMIT_TYPE_ABANDONED);\n\nStyle: I don't think we normally bother with parentheses here\n\n> +\tnew_head->remote = starts_with(refname, \"refs/remote/\");\n> +\tnew_head->hidden = starts_with(refname, \"refs/hiddenmetas/\");\n> +\n> +\tnew_item = string_list_insert(&to_modify->refname_to_change_head, refname);\n> +\tnew_item->util = new_head;\n> +\t/* Use pointers to the copy of the string we're retaining locally */\n\nstring_list_insert() copied the string and we're using that copy. Saying \nwe're retaining it locally when it will outlive this function call is \nconfusing.\n\n> +\trefname = new_item->string;\n> +\n> +\tif (!oideq(&new_head->content, &new_head->head))\n> +\t\tadd_head_to_commit(to_modify, &new_head->content, refname);\n\nIf to_add is a metacommit then we remember the link between refname and \nthe content commit.\n\n> +\tadd_head_to_commit(to_modify, &new_head->head, refname);\n\nWe also remember the link between refname and to_add\n\n> +}\n> +\n> +void change_table_add_all_visible(struct change_table *to_modify,\n> +\tstruct repository* repo)\n> +{\n> +\tstruct ref_filter filter;\n\nrather than using memset we'd write (the same goes for all the other \nmemset() calls in this series, unless they're operation on a heap \nallocation)\n\n\tstruct ref_filter filter = { 0 };\n\n> +\tconst char *name_patterns[] = {NULL};\n> +\tmemset(&filter, 0, sizeof(filter));\n> +\tfilter.kind = FILTER_REFS_CHANGES;\n> +\tfilter.name_patterns = name_patterns;\n> +\n> +\tchange_table_add_matching_filter(to_modify, repo, &filter);\n> +}\n> +\n> +void change_table_add_matching_filter(struct change_table *to_modify,\n> +\tstruct repository* repo, struct ref_filter *filter)\n> +{\n> +\tstruct ref_array matching_refs;\n> +\tint i;\n> +\n> +\tmemset(&matching_refs, 0, sizeof(matching_refs));\n> +\tfilter_refs(&matching_refs, filter, filter->kind);\n> +\n> +\t/**\n> +\t * Determine the object id for the latest content commit for each change.\n> +\t * Fetch the commit at the head of each change ref. If it's a normal commit,\n> +\t * that's the commit we want. If it's a metacommit, locate its content parent\n> +\t * and use that.\n> +\t */\n> +\n> +\tfor (i = 0; i < matching_refs.nr; i++) {\n> +\t\tstruct ref_array_item *item = matching_refs.items[i];\n> +\t\tstruct commit *commit = item->commit;\n> +\n> +\t\tcommit = lookup_commit_reference_gently(repo, &item->objectname, 1);\n\nWe're assigning commit twice - why do we need to look it up if \nfilter_refs returns it?\n\nThere are a number of places where we call \nlookup_commit_reference_gently(..., 1) to silence the warning if the \nobjectname does not dereference to a commit. It is not clear to me that \nwe want to hide those errors. Indeed I think we should be doing\n\n\t\tcommit = lookup_commit_reference(repo, oid)\n\t\tif (!commit)\n\t\t\tBUG(\"commit missing ...\")\n\nunless there is a good reason that the lookup can fail.\n\n> +\t\tif (commit)\n> +\t\t\tchange_table_add(to_modify, item->refname, commit);\n> +\t}\n> +\n> +\tref_array_clear(&matching_refs);\n> +}\n\n> +int for_each_change_referencing(struct change_table *table,\n> +\tconst struct object_id *referenced_commit_id, each_change_fn fn, void *cb_data)\n> +{\n> +\tconst struct change_list *changes;\n> +\tint i;\n> +\tint retvalue;\n\nWe normally use \"ret\" for this\n\n> +\tstruct commit_change_list_entry *entry;\n> +\n> +\tentry = oidmap_get(&table->oid_to_metadata_index,\n> +\t\treferenced_commit_id);\n\nThis should be indented to start below the '(' of the function call.\n\n> +\t/* If this commit isn't referenced by any changes, it won't be in the map */\n> +\tif (!entry)\n> +\t\treturn 0;\n> +\tchanges = &entry->changes;\n> +\tif (!changes->first_refname)\n> +\t\treturn 0;\n> +\tretvalue = fn(changes->first_refname, cb_data);\n> +\tfor (i = 0; retvalue == 0 && i < changes->additional_refnames.nr; i++)\n> +\t\tretvalue = fn(changes->additional_refnames.items[i].string, cb_data);\n\nUsing an strset for struct change_list would simplify this\n\n> +\treturn retvalue;\n> +}\n> +\n> +struct change_head* get_change_head(struct change_table *heads,\n> +\tconst char* refname)\n> +{\n> +\tstruct string_list_item *item = string_list_lookup(\n> +\t\t&heads->refname_to_change_head, refname);\n> +\n> +\tif (!item)\n> +\t\treturn NULL;\n> +\n> +\treturn (struct change_head *)item->util;\n\nWe don't bother with casting void* pointers like this. In any case this \nwhole function could become\n\n\treturn strmap_get(table, refname)\n\nif we used an strmap instead of a string_list.\n\n\nAside from the style issues and using api's that have been added since \nStefan wrote these patches this looks pretty sound. The only thing I \ndon't really get why the public api allows normal commits to be added to \nthe change table (I can see why we might want to add the content commit \nas well when we add a metacommit but that should be done internally)\n\nBest Wishes\n\nPhillip\n"},{"id":"463713","messageId":"220927.865yh8zzux.gmgdl@evledraar.gmail.com","threadId":"58504","inReplyTo":"3c61e0b3-5526-f42e-48a7-c4465d06ccb3@dunelm.org.uk","subject":"Re: [PATCH 05/10] evolve: add the change-table structure","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-27T13:50:29Z","receivedAt":"2022-09-27T13:55:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Sep 27 2022, Phillip Wood wrote:\n\n> On 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n>> From: Stefan Xenos \n>> A change table stores a list of changes, and supports efficient\n>> lookup\n>> from a commit hash to the list of changes that reference that commit\n>> directly.\n>> It can be used to look up content commits or metacommits at the head\n>> of a change, but does not support lookup of commits referenced as part\n>> of the commit history.\n>> Signed-off-by: Stefan Xenos \n>> Signed-off-by: Chris Poucet \n>\n>> diff --git a/change-table.h b/change-table.h\n>> new file mode 100644\n>> index 00000000000..166b5ed8073\n>> --- /dev/null\n>> +++ b/change-table.h\n>> @@ -0,0 +1,132 @@\n>> +#ifndef CHANGE_TABLE_H\n>> +#define CHANGE_TABLE_H\n>> +\n>> +#include \"oidmap.h\"\n>> +\n>> +struct commit;\n>> +struct ref_filter;\n>> +\n>> +/**\n>\n> We tend to just use '/*' rather than '/**'\n\nNo, we use both, and /** is correct here. It's an API-doc syntax, see\ne.g. strbuf.h.\n\nIt's explicitly meant for this sort of thing, i.e. comments on public\nstructs in a header & the functions in a header (and struct members,\netc.).\n\n>> + * This struct holds a list of change refs. The first element is\n> stored inline,\n>> + * to optimize for small lists.\n>> + */\n>> +struct change_list {\n>> +\t/**\n>> +\t * Ref name for the first change in the list, or null if none.\n>> +\t *\n>> +\t * This field is private. Use for_each_change_in to read.\n>> +\t */\n>> +\tconst char* first_refname;\n>> +\t/**\n\nWhile we're on nits we tend to add an extra \\n before the next API\ncomment...\n"},{"id":"463726","messageId":"ae031714-ba51-fe39-6351-ebd638840c32@dunelm.org.uk","threadId":"58504","inReplyTo":"220927.865yh8zzux.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH 05/10] evolve: add the change-table structure","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-27T14:13:56Z","receivedAt":"2022-09-27T14:14:03Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"\n\nOn 27/09/2022 14:50, Ævar Arnfjörð Bjarmason wrote:\n> \n> On Tue, Sep 27 2022, Phillip Wood wrote:\n> \n>> On 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n>>> From: Stefan Xenos \n>>> A change table stores a list of changes, and supports efficient\n>>> lookup\n>>> from a commit hash to the list of changes that reference that commit\n>>> directly.\n>>> It can be used to look up content commits or metacommits at the head\n>>> of a change, but does not support lookup of commits referenced as part\n>>> of the commit history.\n>>> Signed-off-by: Stefan Xenos \n>>> Signed-off-by: Chris Poucet \n>>\n>>> diff --git a/change-table.h b/change-table.h\n>>> new file mode 100644\n>>> index 00000000000..166b5ed8073\n>>> --- /dev/null\n>>> +++ b/change-table.h\n>>> @@ -0,0 +1,132 @@\n>>> +#ifndef CHANGE_TABLE_H\n>>> +#define CHANGE_TABLE_H\n>>> +\n>>> +#include \"oidmap.h\"\n>>> +\n>>> +struct commit;\n>>> +struct ref_filter;\n>>> +\n>>> +/**\n>>\n>> We tend to just use '/*' rather than '/**'\n> \n> No, we use both, and /** is correct here. It's an API-doc syntax, see\n> e.g. strbuf.h.\n> \n> It's explicitly meant for this sort of thing, i.e. comments on public\n> structs in a header & the functions in a header (and struct members,\n> etc.).\n\nWe don't do that consistently, we don't mention them in CodingGuidelines \nand we don't use anything that processes API-doc comments. It would be a \nlot simpler and it would be consistent with our coding guidelines just \nto use the same style everywhere. That would avoid problems where this \nseries uses API-doc comments for in-code comments in .c files and where \nsingle line comments in header files do not use the API-doc syntax.\n\nBest Wishes\n\nPhillip\n\n>>> + * This struct holds a list of change refs. The first element is\n>> stored inline,\n>>> + * to optimize for small lists.\n>>> + */\n>>> +struct change_list {\n>>> +\t/**\n>>> +\t * Ref name for the first change in the list, or null if none.\n>>> +\t *\n>>> +\t * This field is private. Use for_each_change_in to read.\n>>> +\t */\n>>> +\tconst char* first_refname;\n>>> +\t/**\n> \n> While we're on nits we tend to add an extra \\n before the next API\n> comment...\n"},{"id":"463727","messageId":"7c310e1a-1011-c426-fe64-7be21b69052c@dunelm.org.uk","threadId":"58504","inReplyTo":"3c61e0b3-5526-f42e-48a7-c4465d06ccb3@dunelm.org.uk","subject":"Re: [PATCH 05/10] evolve: add the change-table structure","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-27T14:18:57Z","receivedAt":"2022-09-27T14:19:05Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 27/09/2022 14:27, Phillip Wood wrote:\n>> +    /**\n>> +     * Determine the object id for the latest content commit for each \n>> change.\n>> +     * Fetch the commit at the head of each change ref. If it's a \n>> normal commit,\n>> +     * that's the commit we want. If it's a metacommit, locate its \n>> content parent\n>> +     * and use that.\n>> +     */\n>> +\n>> +    for (i = 0; i < matching_refs.nr; i++) {\n>> +        struct ref_array_item *item = matching_refs.items[i];\n>> +        struct commit *commit = item->commit;\n>> +\n>> +        commit = lookup_commit_reference_gently(repo, \n>> &item->objectname, 1);\n> \n> We're assigning commit twice - why do we need to look it up if \n> filter_refs returns it?\n\nI think we want to look it up to check that item->objectname is a \ncommit. item->commit is not filled unless we specify the verbose flag \nand I'm not sure what the implications of setting that are. If we get an \nobjectname that does not name a commit we should call BUG() as suggested \nbelow.\n\n> There are a number of places where we call \n> lookup_commit_reference_gently(..., 1) to silence the warning if the \n> objectname does not dereference to a commit. It is not clear to me that \n> we want to hide those errors. Indeed I think we should be doing\n> \n>         commit = lookup_commit_reference(repo, oid)\n>         if (!commit)\n>             BUG(\"commit missing ...\")\n> \n> unless there is a good reason that the lookup can fail.\n"},{"id":"463730","messageId":"220927.861qrwzvhe.gmgdl@evledraar.gmail.com","threadId":"58504","inReplyTo":"ae031714-ba51-fe39-6351-ebd638840c32@dunelm.org.uk","subject":"Re: [PATCH 05/10] evolve: add the change-table structure","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-27T15:28:09Z","receivedAt":"2022-09-27T15:29:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Sep 27 2022, Phillip Wood wrote:\n\n> On 27/09/2022 14:50, Ævar Arnfjörð Bjarmason wrote:\n>> On Tue, Sep 27 2022, Phillip Wood wrote:\n>> \n>>> On 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n>>>> From: Stefan Xenos \n>>>> A change table stores a list of changes, and supports efficient\n>>>> lookup\n>>>> from a commit hash to the list of changes that reference that commit\n>>>> directly.\n>>>> It can be used to look up content commits or metacommits at the head\n>>>> of a change, but does not support lookup of commits referenced as part\n>>>> of the commit history.\n>>>> Signed-off-by: Stefan Xenos \n>>>> Signed-off-by: Chris Poucet \n>>>\n>>>> diff --git a/change-table.h b/change-table.h\n>>>> new file mode 100644\n>>>> index 00000000000..166b5ed8073\n>>>> --- /dev/null\n>>>> +++ b/change-table.h\n>>>> @@ -0,0 +1,132 @@\n>>>> +#ifndef CHANGE_TABLE_H\n>>>> +#define CHANGE_TABLE_H\n>>>> +\n>>>> +#include \"oidmap.h\"\n>>>> +\n>>>> +struct commit;\n>>>> +struct ref_filter;\n>>>> +\n>>>> +/**\n>>>\n>>> We tend to just use '/*' rather than '/**'\n>> No, we use both, and /** is correct here. It's an API-doc syntax,\n>> see\n>> e.g. strbuf.h.\n>> It's explicitly meant for this sort of thing, i.e. comments on\n>> public\n>> structs in a header & the functions in a header (and struct members,\n>> etc.).\n>\n> We don't do that consistently, we don't mention them in\n> CodingGuidelines and we don't use anything that processes API-doc\n> comments. It would be a lot simpler and it would be consistent with\n> our coding guidelines just to use the same style everywhere. That\n> would avoid problems where this series uses API-doc comments for\n> in-code comments in .c files and where single line comments in header\n> files do not use the API-doc syntax.\n\nYes, this isn't documented in CodingGuidelines (but FWIW is in various\ncommit messages).\n\nI'm pointing out that this isn't a mistake, but the preferred style for\nnew API docs.\n\nAt least Emacs knows how to highlight these differently, which is the\nmain use I personally get out of them, I don't know what other use-cases\nthere are for them...\n\n"},{"id":"463843","messageId":"a7ddab8a-ddd6-a8bf-496d-4ce7757d89cf@dunelm.org.uk","threadId":"58504","inReplyTo":"56c6770997bbdb1b3b87c2c410dd7f158b03f2d6.1663959325.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 06/10] evolve: add support for writing metacommits","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-28T14:27:25Z","receivedAt":"2022-09-28T14:27:35Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n> From: Stefan Xenos \n> \n> metacommit.c supports the creation of metacommits and\n> adds the API needed to create and update changes.\n> \n> Create the \"modify_change\" function that can be called from modification\n> commands like \"rebase\" and \"git amend\" to record obsolescences in the\n> change graph.\n> \n> Create the \"record_metacommit\" function for recording more complicated\n> commit relationships in the commit graph.\n> \n> Create the \"write_metacommit\" function for low-level creation of\n> metacommits.\n\nThe commit message fails to mention that we do not create a \n\"parent-type\" header when we create a metacommit but instead abuse the \ncommit message.\n\nAs with the other patches there are a lot of style comments, but to try \nand limit the noise I've only commented on one instance of each - you \nshould apply the comments to all occurrences in all patches.\n\nI've left a couple of questions where I'm not sure exactly what the code \nis trying to do but apart from an easily fixed NULL pointer de-reference \nand not actually creating the \"parent-type\" header it looks pretty good. \nI was glad to see that there are no obvious memory leaks. I do think the \npatches in this series would be easier to follow if the function \nparameter names were nouns rather than verbs.\n\n> Signed-off-by: Stefan Xenos \n> Signed-off-by: Chris Poucet \n> ---\n> Makefile | 1 +\n> metacommit.c | 404 +++++++++++++++++++++++++++++++++++++++++++++++++++\n> metacommit.h | 58 ++++++++\n> 3 files changed, 463 insertions(+)\n> create mode 100644 metacommit.c\n> create mode 100644 metacommit.h\n> \n> diff --git a/Makefile b/Makefile\n> index 2b847e7e7de..68082ef94c7 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -1000,6 +1000,7 @@ LIB_OBJS += merge-ort.o\n> LIB_OBJS += merge-ort-wrappers.o\n> LIB_OBJS += merge-recursive.o\n> LIB_OBJS += merge.o\n> +LIB_OBJS += metacommit.o\n> LIB_OBJS += metacommit-parser.o\n\nI think the code to parse and create metacommits (as well as the change \ntable code) could quite happily live in the same file.\n\n> diff --git a/metacommit.c b/metacommit.c\n> new file mode 100644\n> index 00000000000..d2b859a4d3b\n> --- /dev/null\n> +++ b/metacommit.c\n> @@ -0,0 +1,404 @@\n> +#include \"cache.h\"\n> +#include \"metacommit.h\"\n> +#include \"commit.h\"\n> +#include \"change-table.h\"\n> +#include \"refs.h\"\n> +\n> +void init_metacommit_data(struct metacommit_data *state)\n> +{\n> +\tmemset(state, 0, sizeof(*state));\n> +}\nWe'd normally use an initializer macro instead\n\n\t#define METACOMMIT_DATA_INIT = { 0 }\n\n> +void clear_metacommit_data(struct metacommit_data *state)\n> +{\n> +\toid_array_clear(&state->replace);\n> +\toid_array_clear(&state->origin);\n> +}\n> +\n> +static void compute_default_change_name(struct commit *initial_commit,\n> +\tstruct strbuf* result)\n> +{\n> +\tstruct strbuf default_name;\n\nThe canonical way to initialize an strbuf that is not on the heap is\n\n\tstruct strbuf buf = STRBUF_INIT;\n\n> +\tconst char *buffer;\n> +\tconst char *subject;\n> +\tconst char *eol;\n> +\tint len;\n> +\tstrbuf_init(&default_name, 0);\n> +\tbuffer = get_commit_buffer(initial_commit, NULL);\n> +\tfind_commit_subject(buffer, &subject);\n> +\teol = strchrnul(subject, '\\n');\n> +\tfor (len = 0;subject < eol && len < 10; ++subject, ++len) {\n\nThere's a space missing after the first ';'. We prefer post-increments \nto pre-increments unless the pre-increment is significant.\n\n> +\t\tchar next = *subject;\n> +\t\tif (isspace(next))\n> +\t\t\tcontinue;\n> +\n> +\t\tstrbuf_addch(&default_name, next);\n> +\t}\n> +\tsanitize_refname_component(default_name.buf, result);\n\nI suspect we need to call unuse_commit_buffer(initial_commit) here.\n\n> +}\n> +\n> +/**\n> + * Computes a change name for a change rooted at the given initial commit. Good\n> + * change names should be memorable, unique, and easy to type. They are not\n> + * required to match the commit comment.\n> + */\n> +static void compute_change_name(struct commit *initial_commit, struct strbuf* result)\n> +{\n> +\tstruct strbuf default_name;\n> +\tstruct object_id unused;\n> +\n> +\tstrbuf_init(&default_name, 0);\n> +\tif (initial_commit)\n> +\t\tcompute_default_change_name(initial_commit, &default_name);\n> +\telse\n> +\t\tstrbuf_addstr(&default_name, \"change\");\n\nWhat does it mean to call this function with initial_commit == NULL?\n\n> +\tstrbuf_addstr(result, \"refs/metas/\");\n> +\tstrbuf_addbuf(result, &default_name);\n> +\t/* If there is already a change of this name, append a suffix */\n> +\tif (!read_ref(result->buf, &unused)) {\n> +\t\tint suffix = 2;\n> +\t\tint original_length = result->len;\n\nThis is one of many places where we have a size_t len or nr member and \nassign it to an int. I think it would be clearer to use a size_t instead \nto avoid adding any more signed<->unsigned conversions.\n\n> +\n> +\t\twhile (1) {\n> +\t\t\tstrbuf_addf(result, \"%d\", suffix);\n> +\t\t\tif (read_ref(result->buf, &unused))\n> +\t\t\t\tbreak;\n> +\t\t\tstrbuf_remove(result, original_length, result->len - original_length);\n> +\t\t\t++suffix;\n> +\t\t}\n> +\t}\n> +\n> +\tstrbuf_release(&default_name);\n> +}\n> +\n> +struct resolve_metacommit_callback_data\n\nWhile there are some structs with a _callback_data suffix in the code \nbase, it is far more common to use _context and name any corresponding \nvariables ctx.\n\n> +{\n> +\tstruct change_table* active_changes;\n> +\tstruct string_list *changes;\n> +\tstruct oid_array *heads;\n> +};\n> +\n> +static int resolve_metacommit_callback(const char *refname, void *cb_data)\n> +{\n> +\tstruct resolve_metacommit_callback_data *data = (struct resolve_metacommit_callback_data *)cb_data;\n\nWe don't use redundant casts such as this.\n\n> +\tstruct change_head *chhead;\n> +\n> +\tchhead = get_change_head(data->active_changes, refname);\n\nThis is really a comment on the previous patch but are there uses of \nfor_each_change_referencing() for which just the refname is sufficient? \nIt might be more convenient to pass the change head into the callback as \nwell.\n\n> +\n> +\tif (data->changes)\n> +\t\tstring_list_append(data->changes, refname)->util = &(chhead->head);\n\nWe don't use redundant parentheses such as this (and this patch does not \nuse them consistently)\n\n> +\tif (data->heads)\n> +\t\toid_array_append(data->heads, &(chhead->head));\n> +\n> +\treturn 0;\n> +}\n> +\n> +/**\n> + * Produces the final form of a metacommit based on the current change refs.\n> + */\n> +static void resolve_metacommit(\n> +\tstruct repository* repo,\n> +\tstruct change_table* active_changes,\n> +\tconst struct metacommit_data *to_resolve,\n\n[testing my understanding] This is the metacommit we want to update\n\n> +\tstruct metacommit_data *resolved_output,\n\nThis is the updated metacommit returned to the user\n\n> +\tstruct string_list *to_advance,\n\nIs also an output? It ends up as a list of refname to change head mappings\n\n> +\tint allow_append)\n> +{\n> +\tint i;\n> +\tint len = to_resolve->replace.nr;\n> +\tstruct resolve_metacommit_callback_data cbdata;\n\nThis would be a good place to a designated initializer.\n\n\tstruct resolve_metacommit_context ctx = {\n\t\t.active_changes = active_changes,\n\t\t.changes = to_advance,\n\t\t.heads = &resolved_output->replace\n\t};\n\n> +\tint old_change_list_length = to_advance->nr;\n> +\tstruct commit* content;\n> +\n> +\toidcpy(&resolved_output->content, &to_resolve->content);\n> +\n> +\t/* First look for changes that point to any of the replacement edges in the\n> +\t * metacommit. These will be the changes that get advanced by this\n> +\t * metacommit. */\n\nStyle: '/*' & '*/' should be on their own lines.\n\n> +\tresolved_output->abandoned = to_resolve->abandoned;\n> +\tcbdata.active_changes = active_changes;\n> +\tcbdata.changes = to_advance;\n> +\tcbdata.heads = &(resolved_output->replace);\n> +\n> +\tif (allow_append) {\n> +\t\tfor (i = 0; i < len; i++) {\n> +\t\t\tint old_number = resolved_output->replace.nr;\n> +\t\t\tfor_each_change_referencing(active_changes, &(to_resolve->replace.oid[i]),\n> +\t\t\t\tresolve_metacommit_callback, &cbdata);\n> +\t\t\t/* If no changes were found, use the unresolved value. */\n> +\t\t\tif (old_number == resolved_output->replace.nr)\n> +\t\t\t\toid_array_append(&(resolved_output->replace), &(to_resolve->replace.oid[i]));\n\nWe see if there are any refs under refs/metas/ which point to \n'to_resolve' or its content and if there are we add those refs and the \ncorresponding change head to 'to_advance'. If we don't find any refs \nthen we copy the replace oid from 'to_resolve' to 'resolved_output'\n\nIf allow_append is false then we ignore all the replace oids in 'to_resolve'\n\n> +\t\t}\n> +\t}\n> +\n> +\tcbdata.changes = NULL;\n> +\tcbdata.heads = &(resolved_output->origin);\n> +\n> +\tlen = to_resolve->origin.nr;\n> +\tfor (i = 0; i < len; i++) {\n> +\t\tint old_number = resolved_output->origin.nr;\n> +\t\tfor_each_change_referencing(active_changes, &(to_resolve->origin.oid[i]),\n> +\t\t\tresolve_metacommit_callback, &cbdata);\n> +\t\tif (old_number == resolved_output->origin.nr)\n> +\t\t\toid_array_append(&(resolved_output->origin), &(to_resolve->origin.oid[i]));\n> +\t}\n\nThis is copying the origin oids in the same way as we copied the replace \noids above.\n\n> +\t/* If no changes were advanced by this metacommit, we'll need to create a new\n> +\t * one. */\n> +\tif (to_advance->nr == old_change_list_length) {\n> +\t\tstruct strbuf change_name;\n> +\n> +\t\tstrbuf_init(&change_name, 80);\n> +\t\tcontent = lookup_commit_reference_gently(repo, &(to_resolve->content), 1);\n> +\n> +\t\tcompute_change_name(content, &change_name);\n> +\t\tstring_list_append(to_advance, change_name.buf);\n> +\t\tstrbuf_release(&change_name);\n> +\t}\n> +}\n> +\n> +static void lookup_commits(\n> +\tstruct repository *repo,\n> +\tstruct oid_array *to_lookup,\n> +\tstruct commit_list **result)\n> +{\n> +\tint i = to_lookup->nr;\n> +\n> +\twhile (--i >= 0) {\n> +\t\tstruct object_id *next = &(to_lookup->oid[i]);\n> +\t\tstruct commit *commit = lookup_commit_reference_gently(repo, next, 1);\n> +\t\tcommit_list_insert(commit, result);\n> +\t}\n\nWe walk backwards because commit_list_insert prepends to the list - good.\n\n> +}\n> +\n> +#define PARENT_TYPE_PREFIX \"parent-type \"\n> +\n> +/**\n> + * Creates a new metacommit object with the given content. Writes the object\n> + * id of the newly-created commit to result.\n> + */\n> +int write_metacommit(struct repository *repo, struct metacommit_data *state,\n> +\tstruct object_id *result)\n> +{\n> +\tstruct commit_list *parents = NULL;\n> +\tstruct strbuf comment;\n> +\tint i;\n> +\tstruct commit *content;\n> +\n> +\tstrbuf_init(&comment, strlen(PARENT_TYPE_PREFIX)\n> +\t\t+ 1 + 2 * (state->origin.nr + state->replace.nr));\n> +\tlookup_commits(repo, &state->origin, &parents);\n> +\tlookup_commits(repo, &state->replace, &parents);\n> +\tcontent = lookup_commit_reference_gently(repo, &state->content, 1);\n> +\tif (!content) {\n> +\t\tstrbuf_release(&comment);\n> +\t\tfree_commit_list(parents);\n> +\t\treturn -1;\n> +\t}\n> +\tcommit_list_insert(content, &parents);\n> +\n> +\tstrbuf_addstr(&comment, PARENT_TYPE_PREFIX);\n> +\tstrbuf_addstr(&comment, state->abandoned ? \"a\" : \"c\");\n> +\tfor (i = 0; i < state->replace.nr; i++)\n> +\t\tstrbuf_addstr(&comment, \" r\");\n> +\n> +\tfor (i = 0; i < state->origin.nr; i++)\n> +\t\tstrbuf_addstr(&comment, \" o\"); > +\t/* The parents list will be freed by this call. */\n> +\tcommit_tree(comment.buf, comment.len, repo->hash_algo->empty_tree, parents,\n> +\t\tresult, NULL, NULL);\n\nIt would be relatively easy to use commit_tree_extended() with \nextra_headers so that we create a commit with a \"parent-type\" header \nrather than abusing the commit message.\n\n\tstruct commit_extra_header extra = { .key = \"parent-type\" };\n\t\n\t/* build header value in strbuf */\n\n\textra.value = buf.buf;\n\textra.len = buf.len;\n\tcommit_tree_extended(\"\", 0, repo->hash_algo->empty_tree,\n\t\t\t parents, result, NULL, NULL, NULL,\n\t\t\t &extra);\n\n> +\n> +\tstrbuf_release(&comment);\n> +\treturn 0;\n> +}\n> +\n> +/**\n> + * Returns true iff the given metacommit is abandoned, has one or more origin\n> + * parents, or has one or more replacement parents.\n> + */\n> +static int is_nontrivial_metacommit(struct metacommit_data *state)\n> +{\n> +\treturn state->replace.nr || state->origin.nr || state->abandoned;\n> +}\n> +\n> +/*\n> + * Records the relationships described by the given metacommit in the\n> + * repository.\n> + *\n> + * If override_change is NULL (the default), an attempt will be made\n> + * to append to existing changes wherever possible instead of creating new ones.\n> + * If override_change is non-null, only the given change ref will be updated.\n\nSo override_head is the refname of an existing change?\n\n> + * options is a bitwise combination of the UPDATE_OPTION_* flags.\n> + */\n> +int record_metacommit(\n> +\tstruct repository *repo,\n> +\tconst struct metacommit_data *metacommit, const char *override_change,\n> +\tint options, struct strbuf *err)\n> +{\n> +\t\tstruct change_table chtable;\n> +\t\tstruct string_list changes;\n> +\t\tint result;\n> +\n> +\t\tchange_table_init(&chtable);\n> +\t\tchange_table_add_all_visible(&chtable, repo);\n> +\t\tstring_list_init_dup(&changes);\n> +\n> +\t\tresult = record_metacommit_withresult(repo, &chtable, metacommit,\n> +\t\t\toverride_change, options, err, &changes);\n> +\n> +\t\tstring_list_clear(&changes, 0);\n> +\t\tchange_table_clear(&chtable);\n> +\t\treturn result;\n> +}\n> +\n> +/*\n> + * Records the relationships described by the given metacommit in the\n> + * repository.\n> + *\n> + * If override_change is NULL (the default), an attempt will be made\n> + * to append to existing changes wherever possible instead of creating new ones.\n> + * If override_change is non-null, only the given change ref will be updated.\n> + *\n> + * The changes list is filled in with the list of change refs that were updated,\n> + * with the util pointers pointing to the old object IDS for those changes.\n> + * The object ID pointers all point to objects owned by the change_table and\n> + * will go out of scope when the change_table is destroyed.\n\nThat potentially sounds like an invitation to create use after free bugs \nunless we're careful. Does this function need to be public?\n\n> + *\n> + * options is a bitwise combination of the UPDATE_OPTION_* flags.\n> + */\n> +int record_metacommit_withresult(\n> +\tstruct repository *repo,\n> +\tstruct change_table *chtable,\n> +\tconst struct metacommit_data *metacommit,\n> +\tconst char *override_change,\n> +\tint options, struct strbuf *err,\n> +\tstruct string_list *changes)\n> +{\n> +\tstatic const char *msg = \"updating change\";\n> +\tstruct metacommit_data resolved_metacommit;\n> +\tstruct object_id commit_target;\n> +\tstruct ref_transaction *transaction = NULL;\n> +\tstruct change_head *overridden_head;\n> +\tconst struct object_id *old_head;\n> +\n> +\tint i;\n> +\tint ret = 0;\n> +\tint force = (options & UPDATE_OPTION_FORCE);\n> +\n> +\tinit_metacommit_data(&resolved_metacommit);\n> +\n> +\tresolve_metacommit(repo, chtable, metacommit, &resolved_metacommit, changes,\n> +\t\t(options & UPDATE_OPTION_NOAPPEND) == 0);\n> +\n> +\tif (override_change) {\n> +\t\tstring_list_clear(changes, 0);\n> +\t\toverridden_head = get_change_head(chtable, override_change);\n> +\t\tif (!overridden_head) {\n\nWe enter this branch if overridden_head is NULL\n\n> +\t\t\t/* This is an existing change */\n> +\t\t\told_head = &overridden_head->head;\n\nHere we de-reference overridden_head which is NULL\n\n> +\t\t\tif (!force) {\n> +\t\t\t\tif (!oid_array_readonly_contains(&(resolved_metacommit.replace),\n> +\t\t\t\t\t&overridden_head->head)) {\n> +\t\t\t\t\t/* Attempted non-fast-forward change */\n> +\t\t\t\t\tstrbuf_addf(err, _(\"non-fast-forward update to '%s'\"),\n> +\t\t\t\t\t\toverride_change);\n> +\t\t\t\t\tret = -1;\n> +\t\t\t\t\tgoto cleanup;\n> +\t\t\t\t}\n> +\t\t\t}\n> +\t\t} else\n\nStyle: if one branch of an if statement requires braces then all \nbranches should have braces.\n\n> +\t\t\t/* ...then this is a newly-created change */\n> +\t\t\told_head = null_oid();\n> +\n> +\t\t/* The expected \"current\" head of the change is stored in the util\n> +\t\t * pointer. */\n> +\t\tstring_list_append(changes, override_change)->util = (void*)old_head;\n\nNo need to cast here\n\n> +\t}\n> +\n> +\tif (is_nontrivial_metacommit(&resolved_metacommit)) {\n> +\t\t/* If there are any origin or replacement parents, create a new metacommit\n> +\t\t * object. */\n> +\t\tif (write_metacommit(repo, &resolved_metacommit, &commit_target) < 0) {\n> +\t\t\tret = -1;\n> +\t\t\tgoto cleanup;\n> +\t\t}\n> +\t} else\n> +\t\t/**\n> +\t\t * If the metacommit would only contain a content commit, point to the\n> +\t\t * commit itself rather than creating a trivial metacommit.\n> +\t\t */\n> +\t\toidcpy(&commit_target, &(resolved_metacommit.content));\n\nOh, is this optimization why we don't insist on metacommits but also \nallow ordinary commits to be added to the change table?\n\n> diff --git a/metacommit.h b/metacommit.h\n> new file mode 100644\n> index 00000000000..fdb253f0f04\n> --- /dev/null\n> +++ b/metacommit.h\n> @@ -0,0 +1,58 @@\n> +#ifndef METACOMMIT_H\n> +#define METACOMMIT_H\n> +\n> +#include \"hash.h\"\n> +#include \"oid-array.h\"\n> +#include \"repository.h\"\n> +#include \"string-list.h\"\n> +\n> +\n> +struct change_table;\n> +\n> +/* If specified, non-fast-forward changes are permitted. */\n> +#define UPDATE_OPTION_FORCE 0x0001\n> +/**\n> + * If specified, no attempt will be made to append to existing changes.\n> + * Normally, if a metacommit points to a commit in its replace or origin\n> + * list and an existing change points to that same commit as its content, the\n> + * new metacommit will attempt to append to that same change. This may replace\n> + * the commit parent with one or more metacommits from the head of the appended\n> + * changes. This option disables this behavior, and will always create a new\n> + * change rather than reusing existing changes.\n> + */\n> +#define UPDATE_OPTION_NOAPPEND 0x0002\n> +\n> +/* Metacommit Data */\n> +\n> +struct metacommit_data {\n> +\tstruct object_id content;\n> +\tstruct oid_array replace;\n> +\tstruct oid_array origin;\n> +\tint abandoned;\n> +};\n> +\n> +extern void init_metacommit_data(struct metacommit_data *state);\n> +\n> +extern void clear_metacommit_data(struct metacommit_data *state);\n> +\n> +extern int record_metacommit(struct repository *repo,\n> +\tconst struct metacommit_data *metacommit,\n> +\tconst char* override_change, int options, struct strbuf *err);\n> +\n> +extern int record_metacommit_withresult(\n> +\tstruct repository *repo,\n> +\tstruct change_table *chtable,\n> +\tconst struct metacommit_data *metacommit,\n> +\tconst char *override_change,\n> +\tint options,\n> +\tstruct strbuf *err,\n> +\tstruct string_list *changes);\n\nDoes this need to be public? i.e. why would one call this rather than \nrecord_metacommit()?\n\n> +extern void modify_change(struct repository *repo,\n> +\tconst struct object_id *old_commit, const struct object_id *new_commit,\n> +\tstruct strbuf *err);\n> +\n> +extern int write_metacommit(struct repository *repo, struct metacommit_data *state,\n> +\tstruct object_id *result);\n\nThe documentation for the flags is very welcome but this header could to \nwith the api being documented as well.\n\nBest Wishes\n\nPhillip\n"},{"id":"463844","messageId":"72c4e8c9-62bb-96aa-7b42-d7103b7873eb@dunelm.org.uk","threadId":"58504","inReplyTo":"220927.861qrwzvhe.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH 05/10] evolve: add the change-table structure","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-28T14:33:02Z","receivedAt":"2022-09-28T14:33:08Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 27/09/2022 16:28, Ævar Arnfjörð Bjarmason wrote:\n> \n> On Tue, Sep 27 2022, Phillip Wood wrote:\n> \n>> On 27/09/2022 14:50, Ævar Arnfjörð Bjarmason wrote:\n>>> On Tue, Sep 27 2022, Phillip Wood wrote:\n>>>\n>>>> On 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n>>>>> +/**\n>>>>\n>>>> We tend to just use '/*' rather than '/**'\n>>> No, we use both, and /** is correct here. It's an API-doc syntax,\n>>> see\n>>> e.g. strbuf.h.\n>>> It's explicitly meant for this sort of thing, i.e. comments on\n>>> public\n>>> structs in a header & the functions in a header (and struct members,\n>>> etc.).\n>>\n>> We don't do that consistently, we don't mention them in\n>> CodingGuidelines and we don't use anything that processes API-doc\n>> comments. It would be a lot simpler and it would be consistent with\n>> our coding guidelines just to use the same style everywhere. That\n>> would avoid problems where this series uses API-doc comments for\n>> in-code comments in .c files and where single line comments in header\n>> files do not use the API-doc syntax.\n> \n> Yes, this isn't documented in CodingGuidelines (but FWIW is in various\n> commit messages).\n> \n> I'm pointing out that this isn't a mistake, but the preferred style for\n> new API docs.\n\nIt seems a bit a stretch to call it the preferred style, however I had \nthought all our uses were historic but that's not the case.\n\nChris if you want to use '/**' style comments for the API docs then \nplease do so consistently and do not use them elsewhere.\n\n> At least Emacs knows how to highlight these differently, which is the\n> main use I personally get out of them, I don't know what other use-cases\n> there are for them...\n\nI've come across them in projects that use gtk-doc or other \ndocumentation generators where it is necessary to distinguish \n'documentation' from 'code comments'. I don't think they add much value \nif one is not generating documentation from the source, it is just one \nmore thing for contributors to remember.\n\nBest Wishes\n\nPhillip\n\n\n"},{"id":"463848","messageId":"220928.86pmffwmft.gmgdl@evledraar.gmail.com","threadId":"58504","inReplyTo":"72c4e8c9-62bb-96aa-7b42-d7103b7873eb@dunelm.org.uk","subject":"Re: [PATCH 05/10] evolve: add the change-table structure","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-09-28T15:14:14Z","receivedAt":"2022-09-28T15:26:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Sep 28 2022, Phillip Wood wrote:\n\n> On 27/09/2022 16:28, Ævar Arnfjörð Bjarmason wrote:\n>> On Tue, Sep 27 2022, Phillip Wood wrote:\n>> \n>>> On 27/09/2022 14:50, Ævar Arnfjörð Bjarmason wrote:\n>>>> On Tue, Sep 27 2022, Phillip Wood wrote:\n>>>>\n>>>>> On 23/09/2022 19:55, Stefan Xenos via GitGitGadget wrote:\n>>>>>> +/**\n>>>>>\n>>>>> We tend to just use '/*' rather than '/**'\n>>>> No, we use both, and /** is correct here. It's an API-doc syntax,\n>>>> see\n>>>> e.g. strbuf.h.\n>>>> It's explicitly meant for this sort of thing, i.e. comments on\n>>>> public\n>>>> structs in a header & the functions in a header (and struct members,\n>>>> etc.).\n>>>\n>>> We don't do that consistently, we don't mention them in\n>>> CodingGuidelines and we don't use anything that processes API-doc\n>>> comments. It would be a lot simpler and it would be consistent with\n>>> our coding guidelines just to use the same style everywhere. That\n>>> would avoid problems where this series uses API-doc comments for\n>>> in-code comments in .c files and where single line comments in header\n>>> files do not use the API-doc syntax.\n>> Yes, this isn't documented in CodingGuidelines (but FWIW is in\n>> various\n>> commit messages).\n>> I'm pointing out that this isn't a mistake, but the preferred style\n>> for\n>> new API docs.\n>\n> It seems a bit a stretch to call it the preferred style, however I had\n> thought all our uses were historic but that's not the case.\n\nFWIW that claim of mine comes from my (admittedly fuzzy recollection of)\nhistory, around the time that this was being worked out (e.g. where API\ndocs should live, they used to mostly be in Documentation/technical/,\nnow they're mostly in *.h) we ended up with this mention in CodingGudielines:\n\n - When you come up with an API, document its functions and structures\n in the header file that exposes the API to its callers. Use what is\n in \"strbuf.h\" as a model for the appropriate tone and level of\n detail.\n\nI.e. at some point we decreed strbuf.h as a best-practice model to\nfollow. I think every one of my own use of \"/**\" has come after reading\nthat file...\n\nBut patches to make it more explicit are most welcome. FWIW I think I\nlooked into that once, but couldn't find a canonical reference for what\nthis syntax is even called.\n\nHrm, looking a bit more I see it's probably JavaDoc (at least Emacs\nhighlights it as such). A grep of:\n\n\tgit grep '^ \\* @' -- '*.[ch]'\n\nShows that we make almost no use of the meta-syntax (i.e. \"tags\":\nhttps://en.wikipedia.org/wiki/Javadoc#Table_of_Javadoc_tags)\n\n> Chris if you want to use '/**' style comments for the API docs then\n> please do so consistently and do not use them elsewhere.\n>\n>> At least Emacs knows how to highlight these differently, which is the\n>> main use I personally get out of them, I don't know what other use-cases\n>> there are for them...\n>\n> I've come across them in projects that use gtk-doc or other\n> documentation generators where it is necessary to distinguish \n> 'documentation' from 'code comments'. I don't think they add much\n> value if one is not generating documentation from the source, it is\n> just one more thing for contributors to remember.\n\nI for one find it very useful these \"API docs\" are shown differently in\nmy editor, even if I'm not using some fancier (and hypothetical) \"make\njavadoc\" target.\n\nI.e. you can ideally skim the *.h file and see at a glance what's an\nimplementation comment v.s. API doc comment.\n\nExcept that even for the supposed role model of strbuf.h we forgot in\nsome cases, and should have the below applied to it, oh well... :)\n\ndiff --git a/strbuf.h b/strbuf.h\nindex 76965a17d44..46549986ae3 100644\n--- a/strbuf.h\n+++ b/strbuf.h\n@@ -492,7 +492,7 @@ int strbuf_getline_lf(struct strbuf *sb, FILE *fp);\n /* Uses NUL as the line terminator */\n int strbuf_getline_nul(struct strbuf *sb, FILE *fp);\n \n-/*\n+/**\n * Similar to strbuf_getline_lf(), but additionally treats a CR that\n * comes immediately before the LF as part of the terminator.\n * This is the most friendly version to be used to read \"text\" files\n@@ -610,7 +610,7 @@ static inline struct strbuf **strbuf_split(const struct strbuf *sb,\n \treturn strbuf_split_max(sb, terminator, 0);\n }\n \n-/*\n+/**\n * Adds all strings of a string list to the strbuf, separated by the given\n * separator. For example, if sep is\n * ', '\n@@ -653,7 +653,7 @@ int launch_editor(const char *path, struct strbuf *buffer,\n int launch_sequence_editor(const char *path, struct strbuf *buffer,\n \t\t\t const char *const *env);\n \n-/*\n+/**\n * In contrast to `launch_editor()`, this function writes out the contents\n * of the specified file first, then clears the `buffer`, then launches\n * the editor and reads back in the file contents into the `buffer`.\n@@ -693,7 +693,7 @@ static inline void strbuf_complete_line(struct strbuf *sb)\n \tstrbuf_complete(sb, '\\n');\n }\n \n-/*\n+/**\n * Copy \"name\" to \"sb\", expanding any special @-marks as handled by\n * interpret_branch_name(). The result is a non-qualified branch name\n * (so \"foo\" or \"origin/master\" instead of \"refs/heads/foo\" or\n@@ -707,7 +707,7 @@ static inline void strbuf_complete_line(struct strbuf *sb)\n void strbuf_branchname(struct strbuf *sb, const char *name,\n \t\t unsigned allowed);\n \n-/*\n+/**\n * Like strbuf_branchname() above, but confirm that the result is\n * syntactically valid to be used as a local branch name in refs/heads/.\n *\n"},{"id":"463853","messageId":"xmqqh70rzdz9.fsf@gitster.g","threadId":"58504","inReplyTo":"72c4e8c9-62bb-96aa-7b42-d7103b7873eb@dunelm.org.uk","subject":"Re: [PATCH 05/10] evolve: add the change-table structure","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-09-28T15:59:54Z","receivedAt":"2022-09-28T16:00:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood writes:\n\n> Chris if you want to use '/**' style comments for the API docs then\n> please do so consistently and do not use them elsewhere.\n\nThanks.\n\n> I've come across them in projects that use gtk-doc or other\n> documentation generators where it is necessary to distinguish\n> 'documentation' from 'code comments'. I don't think they add much\n> value if one is not generating documentation from the source, it is\n> just one more thing for contributors to remember.\n\nYes, I personally find them annoying, but has tolerated them so far,\nhoping that something good (read: automated documentation out of\ncomments) emerge someday, simply because the first ones were added\nby folks who were interested in that direction, which unfortunately\nhas never materialized.\n\nMaybe it would eventually happen, but I think there are a lot of\nclean-up to do before it happens. I somehow suspect that the sooner\nthe mechanism to create the documentation set, however incomplete\nand messy the result is with the current material, the more incentive\nthe contributors have to apply /** vs /* distinction properly, but\nof course on the other hand, until the existing material gets\ncleaned up, the care they need to take to make the distinction does\nfeel like a makework. So, I dunno.\n\nThanks.\n\n"},{"id":"463884","messageId":"xmqqsfkbqjgz.fsf@gitster.g","threadId":"58504","inReplyTo":"a0cf68f8ba2adefae4fceeab0d438d05e355e695.1663959324.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 01/10] technical doc: add a design doc for the evolve command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-09-28T21:26:04Z","receivedAt":"2022-09-28T21:26:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Stefan Xenos via GitGitGadget\" writes:\n\n> From: Stefan Xenos \n\n[the above address should bounce, and has been removed from cc: list]\n\n> +Background\n> +==========\n> +Imagine you have three sequential changes up for review and you receive feedback\n> +that requires editing all three changes. We'll define the word \"change\"\n> +formally later, but for the moment let's say that a change is a work-in-progress\n> +whose final version will be submitted as a commit in the future.\n> +\n> +While you're editing one change, more feedback arrives on one of the others.\n> +What do you do?\n> +\n> +The evolve command is a convenient way to work with chains of commits that are\n> +under review. Whenever you rebase or amend a commit, the repository remembers\n> +that the old commit is obsolete and has been replaced by the new one. Then, at\n> +some point in the future, you can run \"git evolve\" and the correct sequence of\n> +rebases will occur in the correct order such that no commit has an obsolete\n> +parent.\n> +\n> +Part of making the \"evolve\" command work involves tracking the edits to a commit\n> +over time, which is why we need an change graph. However, the change\n> +graph will also bring other benefits:\n\nIt would be assuring to hear that \"change graph\" will also be\ndefined and explained formally later, just like \"change\" will in the\nprevious paragraph. We will later see mention of \"metacommits\" and\n\"meta-commits\" in this document, and I am guessing both of them are\nquasi-synonyms to \"change graph\". If that is true, it is better to\nstick to a single terminology.\n\n> +Goals\n> +-----\n> +Legend: Goals marked with P0 are required. Goals marked with Pn should be\n> +attempted unless they interfere with goals marked with Pn-1.\n> +\n> +P0. All commands that modify commits (such as the normal commit --amend or\n> + rebase command) should mark the old commit as being obsolete and replaced by\n> + the new one. No additional commands should be required to keep the\n> + change graph up-to-date.\n> +P0. Any commit that may be involved in a future evolve command should not be\n> + garbage collected. Specifically:\n> + - Commits that obsolete another should not be garbage collected until\n> + user-specified conditions have occurred and the change has expired from\n> + the reflog. User specified conditions for removing changes include:\n> + - The user explicitly deleted the change.\n> + - The change was merged into a specific branch.\n> + - Commits that have been obsoleted by another should not be garbage\n> + collected if any of their replacements are still being retained.\n> +P0. A commit can be obsoleted by more than one replacement (called divergence).\n> +P0. Users must be able to resolve divergence (convergence).\n\nP0: a single parent commit should keep only one parent. IOW, the\n\"change graph\" implementation should not contaminate the end-result\ncommit in the regular part of the history, which is the product of\nthe final iteration of a \"change\"\n\nIOW ...\n\n> +P2. It should be possible to discard part or all of the change graph\n> + without discarding the commits themselves that are already present in\n> + branches and the reflog.\n\n... this item should be P0.\n\n> +Overview\n> +========\n> +We introduce the notion of “meta-commits” which describe how one commit was\n\nRandom appearance of smart quotes are annoying. We'll be formatting\nthe doc via AsciiDoc, so let's stick to vanilla double or single quotes.\n\n> +created from other commits. A branch of meta-commits is known as a change.\n> +Changes are created and updated automatically whenever a user runs a command\n> +that creates a commit. They are used for locating obsolete commits, providing a\n> +list of a user’s unsubmitted work in progress, and providing a stable name for\n> +each unsubmitted change.\n\nCan \"change graph\" also be defined and explained here, too? Or if\nit is pretty much a synonym to \"a branch of meta-commits\", then\nperhaps the document does not have to introduce the term \"change\ngraph\" and still stay understandable?\n\n> +Detailed design\n> +===============\n> +Obsolescence information is stored as a graph of meta-commits. A meta-commit is\n> +a specially-formatted merge commit that describes how one commit was created\n> +from others.\n> +\n> +Meta-commits look like this:\n> +\n> +$ git cat-file -p \n> +tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\n> +parent aa7ce55545bf2c14bef48db91af1a74e2347539a\n> +parent d64309ee51d0af12723b6cb027fc9f195b15a5e9\n> +parent 7e1bbcd3a0fa854a7a9eac9bf1eea6465de98136\n> +author Stefan Xenos 1540841596 -0700\n> +committer Stefan Xenos 1540841596 -0700\n> +parent-type c r o\n> +\n> +This says “commit aa7ce555 makes commit d64309ee obsolete. It was created by\n> +cherry-picking commit 7e1bbcd3”.\n> +\n> +The tree for meta-commits is always the empty tree, but future versions of git\n> +may attach other trees here. For forward-compatibility fsck should ignore such\n> +trees if found on future repository versions. This will allow future versions of\n> +git to add metadata to the meta-commit tree without breaking forwards\n> +compatibility.\n\nNot clear why \"trees\" need to be ignored only by fsck but not others\nlike fetch/push, and I'd strongly advise against making such a\nspecial case. If they are missing, they are missing and should be\nreported as corruptoin, and if you do not like it, do not add a\nmissing tree.\n\n> +Parent-type\n> +-----------\n> +The “parent-type” field in the commit header identifies a commit as a\n> +meta-commit and indicates the meaning for each of its parents. It is never\n> +present for normal commits. It contains a space-deliminated list of enum values\n> +whose order matches the order of the parents. Possible parent types are:\n\n> +- c: (content) the content parent identifies the commit that this meta-commit is\n> + describing.\n> +- r: (replaced) indicates that this parent is made obsolete by the content\n> + parent.\n> +- o: (origin) indicates that the content parent was generated by cherry-picking\n> + this parent.\n> +- a: (abandoned) used in place of a content parent for abandoned changes. Points\n> + to the final content commit for the change at the time it was abandoned.\n\nDon't be cute with parent-type using single letters. You'll thank me\nlater when you need two types that share the first letter.\n\n> +A meta-commit can have zero or more origin parents. A cherry-pick creates a\n> +single origin parent. Certain types of squash merge will create multiple origin\n> +parents. Origin parents don't directly cause their origin to become obsolete,\n> +but are used when computing blame or locating a merge base. The section\n> +on obsolescence over cherry-picks describes how the evolve command uses\n> +origin parents.\n\nShould it make a difference among doing these operations?\n\n - running \"commit --amend\" after \"cherry-pick --no-commit\" possibly with editing\n\n - running \"commit --amend\" after manually editing the same way, and\n\n - running \"commit --amend\" after \"cherry-pick\", possibly with editing?\n\nIt seems that the first two will not be captured while the last one\nleaves 'origin'. What should happen after running \"commit --amend\"\nafter \"apply --index\" possibly with editing?\n\nWhat's the point of giving 'origin' only for \"cherry-pick\" and\nsquash merge? I am wondering if we want to record contributions\nsourced from an e-mailed patch from elsewhere (currently people use\nexternal services like patchwork to do this)?\n\nFor the purpose of discussing \"evolve\", should \"rebase\" (with or\nwithout \"-i\") be treated pretty much the same as a series of\n\"cherry-pick\" mixed with \"commit --amend\" (possibly preceded with a\nmanual edit), followed by finally replacing the tip of the branch?\nIn the end result, the replaced commits after a \"rebase\" become\naccessible only from reflog, but other than that, these two bulk\ntransplanting operations shouldn't be all that different.\n\n> +The parent-type field needs to go after the committer field since git's rules\n> +for forwards-compatibility require that new fields to be at the end of the\n> +header. Putting a new field in the middle of the header would break fsck.\n\nYou can do without introducing a new header to avoid compatibility\nissue by recording the information in the body of the commit object,\nwhich would be even cleaner.\n\n> +Change deletion\n> +---------------\n> +Changes are normally only interesting to a user while a commit is still in\n> +development and under review. Once the commit has submitted wherever it is\n> +going, its change can be discarded.\n> +\n> +The normal way of deleting changes makes this easy to do - changes are deleted\n> +by the evolve command when it detects that the change is present in an upstream\n> +branch. It does this in two ways: if the latest commit in a change either shows\n> +up in the branch history or the change becomes empty after a rebase, it is\n> +considered merged and the change is discarded. In this context, an “upstream\n> +branch” is any branch passed in as the upstream argument of the evolve command.\n> +\n> +In case this sometimes deletes a useful change, such automatic deletions are\n> +recorded in the reflog allowing them to be easily recovered.\n\nDeleting a useful change is recorded in the reflog? Isn't a change\nrecorded as a ref in metas/ hierarchy? Doesn't the removal of such a\nref remove its reflog as well?\n\nI guess the above silly questions come from the fact that the\ndocument does not make it clear reflog of what ref it is recorded.\n\n> +Modify commands\n> +---------------\n> +Modification commands (commit --amend, rebase) will mark the old commit as\n> +obsolete by creating a new meta-commit that references the old one as a\n> +replaced parent. In the event that multiple changes point to the same commit,\n> +this is done independently for every such change.\n> +\n> +More specifically, modifications work like this:\n> +\n> +1. Locate all existing changes for which the old commit is the content for the\n> + head of the change branch. If no such branch exists, create one that points\n> + to the old commit. Changes that include this commit in their history but not\n> + at their head are explicitly not included.\n> +2. For every such change, create a new meta-commit that references the new\n> + commit as its content and references the old head of the change as a\n> + replaced parent.\n> +3. Move the change branch forward to point to the new meta-commit.\n> +\n> +Copy commands\n> +-------------\n> +Copy commands (cherry-pick, merge --squash) create a new meta-commit that\n> +references the old commits as origin parents. Besides the fact that the new\n> +parents are tagged differently, copy commands work the same way as modify\n> +commands.\n\nIt is unclear what benefit we will get by separating \"Copy\" commands\nfrom \"Modify\" commands. \"checkout A && cherry-pick B\" may make a\nnew copy of the edit the commit at the tip of branch B wanted to\nmake at the tip of branch A, but \"commit --amend\" is the same, in\nthat it makes a new copy of the edit the commit at the tip of the\ncurrent branch wanted to make, and the original copy is available in\nboth cases. It is just that the original of \"cherry-pick B\" is\nslightly easier to access (i.e. it is still at the tip of branch B,\nuntil the branch gains new commits on top of it) than the original\nof \"commit --amend\" (i.e. the user needs to know that @{1} is the\nprevious state). Shouldn't all commands that create a new commit\nobject using some existing material (i.e. not from scratch) be\ntreated equally, without splitting them into two camps?\n\nIOW, the above explains that the new parents are tagged differently,\nbut it does not explain why it is a good idea to do so.\n"},{"id":"463885","messageId":"xmqqedvvqgxh.fsf@gitster.g","threadId":"58504","inReplyTo":"a0cf68f8ba2adefae4fceeab0d438d05e355e695.1663959324.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 01/10] technical doc: add a design doc for the evolve command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-09-28T22:20:58Z","receivedAt":"2022-09-28T22:21:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Stefan Xenos via GitGitGadget\" writes:\n\n> +Rebase\n> +------\n> +In general the rebase command is treated as a modify command. When a change is\n> +rebased, the new commit replaces the original.\n> +\n> +Rebase --abort is special. Its intent is to restore git to the state it had\n> +prior to running rebase. It should move back any changes to point to the refs\n> +they had prior to running rebase and delete any new changes that were created as\n> +part of the rebase. To achieve this, rebase will save the state of all changes\n> +in refs/metas prior to running rebase and will restore the entire namespace\n> +after rebase completes (deleting any newly-created changes). Newly-created\n> +metacommits are left in place, but will have no effect until garbage collected\n> +since metacommits are only used if they are reachable from refs/metas.\n\nOne thing that makes me nervous is how well your analysis capture\n\"unusual\" but still reasonable ways to use these commands, as the\nworkflows of people are quite different.\n\nFor example, I almost never do \"git checkout topic && git rebase\norigin\"; instead I would do \"git checkout topic && git rebase origin\nHEAD^0\" to first make a detached HEAD out of the topic, in order to\nhave two copies explicitly available to be compared after \"rebase\"\nfinishes. After doing so and get satisfied by the result of\ncomparison between topic and HEAD, I may do \"git checkout -B topic\"\nto update. Would that leave exactly the same set of metacommits as\nthe case where I didn't do the \"first rebase the detached HEAD and\nthen update the bracnh for real\" and instead \"rebase the topic\"\ndirectly?\n"},{"id":"463899","messageId":"87e7da54-b9cf-cf35-3742-640462e11ceb@dunelm.org.uk","threadId":"58504","inReplyTo":"xmqqedvvqgxh.fsf@gitster.g","subject":"Re: [PATCH 01/10] technical doc: add a design doc for the evolve command","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-09-29T09:17:01Z","receivedAt":"2022-09-29T09:17:08Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 28/09/2022 23:20, Junio C Hamano wrote:\n> \"Stefan Xenos via GitGitGadget\" writes:\n> \n>> +Rebase\n>> +------\n>> +In general the rebase command is treated as a modify command. When a change is\n>> +rebased, the new commit replaces the original.\n>> +\n>> +Rebase --abort is special. Its intent is to restore git to the state it had\n>> +prior to running rebase. It should move back any changes to point to the refs\n>> +they had prior to running rebase and delete any new changes that were created as\n>> +part of the rebase. To achieve this, rebase will save the state of all changes\n>> +in refs/metas prior to running rebase and will restore the entire namespace\n>> +after rebase completes (deleting any newly-created changes).\n\nThat wont work now that we have multiple worktrees. If a user starts two \nrebases of two different branches in two different worktrees and aborts \none of them we loose the new meta-commits of both. Each rebase will need \nto track which refs under refs/metas/ it has updated (that will also \nsave the overhead of copying the entire refs/metas/ subtree).\n\n> Newly-created\n>> +metacommits are left in place, but will have no effect until garbage collected\n>> +since metacommits are only used if they are reachable from refs/metas.\n> \n> One thing that makes me nervous is how well your analysis capture\n> \"unusual\" but still reasonable ways to use these commands, as the\n> workflows of people are quite different.\n> \n> For example, I almost never do \"git checkout topic && git rebase\n> origin\"; instead I would do \"git checkout topic && git rebase origin\n> HEAD^0\" to first make a detached HEAD out of the topic, in order to\n> have two copies explicitly available to be compared after \"rebase\"\n> finishes. After doing so and get satisfied by the result of\n> comparison between topic and HEAD, I may do \"git checkout -B topic\"\n> to update. Would that leave exactly the same set of metacommits as\n> the case where I didn't do the \"first rebase the detached HEAD and\n> then update the bracnh for real\" and instead \"rebase the topic\"\n> directly?\n\nAs I understand it we have a ref under refs/metas/ for each commit. If \nthat understanding is correct the two cases you describe should be \nrecorded identically I think.\n\nBest Wishes\n\nPhillip\n"},{"id":"463916","messageId":"20220929195725.1420647-1-jonathantanmy@google.com","threadId":"58504","inReplyTo":"a0cf68f8ba2adefae4fceeab0d438d05e355e695.1663959324.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 01/10] technical doc: add a design doc for the evolve command","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2022-09-29T19:57:24Z","receivedAt":"2022-09-29T19:57:34Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"\"Stefan Xenos via GitGitGadget\" writes:\n> +Background\n> +==========\n> +Imagine you have three sequential changes up for review and you receive feedback\n> +that requires editing all three changes. We'll define the word \"change\"\n> +formally later, but for the moment let's say that a change is a work-in-progress\n> +whose final version will be submitted as a commit in the future.\n[snip]\n> +Part of making the \"evolve\" command work involves tracking the edits to a commit\n> +over time, which is why we need an change graph. \n\nReading later, I thought that a \"change\" is a connected subset of\nelements in the set of metacommits, so a \"change\" is already a graph.\nI'll mentally substitute \"the concept of a change\" for \"an [sic] change\ngraph\" for now - hopefully that's correct.\n\n> +- It can be used as part of other high-level commands that combine or split\n> + changes.\n\nIs the current concept of a change suitable for combining and splitting?\nThere is divergence, but that seems like a commit being modified in 2\ndifferent ways, not a commit being split into 2.\n\n> +Goals\n> +-----\n[snip]\n> +P0. A commit can be obsoleted by more than one replacement (called divergence).\n> +P0. Users must be able to resolve divergence (convergence).\n\nIs divergence important? It seems to me that both the internals and the\nUX could be simplified if we don't allow divergence, and systems like\nGerrit don't have it either (as far as I know, in Gerrit, all commits\nbearing the same Change-Id just form a sequence in the order that they\nwere pushed to the server, with no branching).\n\n> +Overview\n> +========\n> +We introduce the notion of “meta-commits” which describe how one commit was\n> +created from other commits. A branch of meta-commits is known as a change.\n\n\"Branch\" here is confusing. Do you mean a connected set of meta-commits?\n\n(\"Branch\" has a specific meaning in Git - a ref of the form\nrefs/heads/??. If you mean that refs of the form refs/metas/?? point to\nchanges in a 1:1 manner, then the term \"ref\" is appropriate.)\n\n> +Example usage\n> +-------------\n> +# First create three dependent changes\n> +$ echo foo>bar.txt && git add .\n> +$ git commit -m \"This is a test\"\n> +created change metas/this_is_a_test\n> +$ echo foo2>bar2.txt && git add .\n> +$ git commit -m \"This is also a test\"\n> +created change metas/this_is_also_a_test\n> +$ echo foo3>bar3.txt && git add .\n> +$ git commit -m \"More testing\"\n> +created change metas/more_testing\n> +\n> +# List all our changes in progress\n> +$ git change list\n> +metas/this_is_a_test\n> +metas/this_is_also_a_test\n> +* metas/more_testing\n> +metas/some_change_already_merged_upstream\n> +\n> +# Now modify the earliest change, using its stable name\n> +$ git reset --hard metas/this_is_a_test\n> +$ echo morefoo>>bar.txt && git add . && git commit --amend --no-edit\n\nSo up to here, I thought that we would have 2 refs metas/this_is_a_test2\nand metas/this_is_a_test, with the latter's commit being one of the\nparents of the former's commit. (This is because we presumably need to\nbe able to represent the situation in which the user checks out the\noriginal \"This is a test\" commit and modifies it, so we still need to\nhang on to the metas/this_is_a_test.)\n\n> +# Use git-evolve to fix up any dependent changes\n> +$ git evolve\n> +rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n> +rebasing metas/more_testing onto metas/this_is_also_a_test\n> +Done\n\nSo I'm surprised that there's no mention of this_is_a_test2 here. In\naddition, linearizing a change like this doesn't seem to be described in\nthis document.\n\nHaving said that, I don't think that linearizing changes is important to\nthe goals of this \"evolve\" concept, so maybe one thing we can do is to\nnot support it at all.\n\nFast-forward...\n\n> +# Fetch the latest code from origin/master and use git-evolve\n> +# to rebase all dependent changes.\n> +$ git fetch origin master\n> +$ git evolve origin/master\n> +deleting metas/some_change_already_merged_upstream\n> +rebasing metas/this_is_a_test onto origin/master\n> +rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n> +rebasing metas/more_testing onto metas/this_is_also_a_test\n> +rebasing metas/unrelated_change onto origin/master\n> +Conflict detected! Resolve it and then use git evolve --continue to resume.\n> +\n> +# Sort out the conflict\n> +$ git mergetool\n> +$ git evolve origin/master\n> +Done\n\nThis is what I expected from the evolve mechanism, so that's great :-)\n\nThe conflict resolution needs to be discussed further, though. It is\nsuperficially similar to rebase, but with rebase, the ref being rebased\nis only rewritten at the end of the process, so it is always possible to\nabort halfway. Here, multiple refs are written during the process, so it\nis not as easy to abort.\n\n> +Parent-type\n> +-----------\n> +The “parent-type” field in the commit header identifies a commit as a\n> +meta-commit and indicates the meaning for each of its parents. It is never\n> +present for normal commits. It contains a space-deliminated list of enum values\n> +whose order matches the order of the parents. Possible parent types are:\n> +\n> +- c: (content) the content parent identifies the commit that this meta-commit is\n> + describing.\n> +- r: (replaced) indicates that this parent is made obsolete by the content\n> + parent.\n> +- o: (origin) indicates that the content parent was generated by cherry-picking\n> + this parent.\n> +- a: (abandoned) used in place of a content parent for abandoned changes. Points\n> + to the final content commit for the change at the time it was abandoned.\n\nHow would the \"o\" parent be useful to the user?\n\n> +Changes\n> +-------\n[snip]\n> +Changes are also stored in the refs/hiddenmetas namespace. Hiddenmetas holds\n> +metadata for historical changes that are not currently in progress by the user.\n> +Commands like filter-branch and other bulk import commands create metadata in\n> +this namespace.\n> +\n> +Note that the changes in hiddenmetas get special treatment in several ways:\n> +\n> +- They are not cleaned up automatically once merged, since it is expected that\n> + they refer to historical changes.\n> +- User commands that modify changes don't append to these changes as they would\n> + to a change in refs/metas.\n> +- They are not displayed when the user lists their local changes.\n\nThe presence of refs/hiddenmetas further muddies the already unclear\nlifecycle of meta-commits and their refs. Non-hidden meta-commits get\ncleaned up when their latest commit appears upstream, so they may get\ndeleted when the user doesn't expect it (especially if the user is\nusing, say, a prefetch mechanism that downloads refs at night). We're\nadding to this a class of refs that don't get cleaned up at all.\n\nBesides the disk space taken by the meta-commits, having more refs\ntypically reduces performance e.g. because all refs generally take part\nin packfile negotiation during fetching. (And they probably should\ncontinue with this behavior because sharing meta-commits is one of the\nfeatures we want.) So I think that having a clear cleanup strategy is a\ngood idea, and permanent archiving probably shouldn't be it.\n\n> +Change creation\n> +---------------\n> +Changes are created automatically whenever the user runs a command like “commit”\n> +that has the semantics of creating a new change. They also move forward\n> +automatically even if they’re not checked out. For example, whenever the user\n> +runs a command like “commit --amend” that modifies a commit, all branches in\n> +refs/metas that pointed to the old commit move forward to point to its\n> +replacement instead.\n\nWhat happens in the following?\n\n $ echo \"hello\" >hello.txt\n $ git add hello.txt\n $ git commit -m \"hello\"\n $ git tag hello\n $ echo \"one\" >hello.txt\n $ git commit -a --amend # this updates refs/metas/hello\n $ git checkout hello\n $ echo \"one\" >hello.txt\n $ git commit -a --amend # does this update refs/metas/hello too?\n\n> +Sharing changes\n> +---------------\n> +Change histories are shared by pushing or fetching meta-commits and change\n> +branches. This provides users with a lot of control of what to share and\n> +repository implementations with control over what to retain.\n> +\n> +Users that only want to share the content of a commit can do so by pushing the\n> +commit itself as they currently would. Users that want to share an edit history\n> +for the commit can push its change, which would point to a meta-commit rather\n> +than the commit itself if there is any history to share. Note that multiple\n> +changes can refer to the same commits, so it’s possible to construct and push a\n> +different history for the same commit in order to remove sensitive or irrelevant\n> +intermediate states.\n\nIt looks difficult to remove such intermediate states, but maybe that\ndoesn't have to be dealt with in the initial design.\n\n> +Checkout\n> +--------\n> +Running checkout on a change by name has the same effect as checking out a\n> +detached head pointing to the latest commit on that change-branch. There is no\n> +need to ever have HEAD point to a change since changes always move forward when\n> +necessary, no matter what branch the user has checked out\n> +\n> +Meta-commits themselves cannot be checked out by their hash.\n\nThis is the same behavior as for annotated tags, but I guess we can't\nuse them because those can only have one referent.\n"},{"id":"464114","messageId":"CAN84kK=XKYDzF3tmUiwb4vCGcnWvXvewz7QtZwzNEsvRZ8Em+g@mail.gmail.com","threadId":"58504","inReplyTo":"e301d4c1-8f80-b9cf-142b-cd7bd183d625@gmail.com","subject":"Re: [PATCH 00/10] Add the Git Change command","fromName":"Chris P","fromEmail":"christophe.poucet@gmail.com","sentAt":"2022-10-04T09:33:38Z","receivedAt":"2022-10-04T09:37:44Z","isPatch":true,"sender":{"key":"christophe.poucet@gmail.com","avatar":null},"body":"> Thanks for picking this up, having an evolve command would be a really\n> useful addition to git. I read the final four patches as I was\n> interested to see how a user would use \"git change\" to track changes to\n> a set of commits. Unfortunately because there are no tests and scant\n> documentation there are no examples of how to do this. Looking at the\n> patches I felt like it would have been helpful to mark them as RFC to\n> indicate that the author is requesting feedback but does not consider\n> them ready for merging.\n\nThanks for the feedback, I'll mark them as RFC.\n\n> I'm confused as to why the command is called \"change\" (which I don't\n> find particularly descriptive) when every patch subject is \"evolve\". It\n> definitely makes sense to request feedback on a large topic like this\n> before everything is implemented but I'd be nervous of merging the early\n> stages before there is a working evolve command. For an example of a\n> successful multipart topic see\n> https://lore.kernel.org/git/pull.1248.git.1654545325.gitgitgadget@gmail.com/\n> Knowing the author of that series the commit messages should also give\n> you a good idea of the level of detail expected.\n\nThe `git change` command is a lower-level command used to directly\nmanipulate changes, as a user you should not be engaging with those.\nWhat is missing is the more complicated `git evolve` command.\nI admit that I don't yet know how to implement that or the changes that\nneed to happen to all create/modify commands.\n\nStill learning git, so apologies for any mistakes and thank you for your\nconsideration\n\n- simply chris\n\nOn Sun, Sep 25, 2022 at 10:40 AM Phillip Wood wrote:\n>\n> Hi Christophe\n>\n> On 23/09/2022 19:55, Christophe Poucet via GitGitGadget wrote:\n> > I'm reviving the original git evolve work that was started by\n> > sxenos@google.com\n> > (https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n> >\n> > This work is intended to make it easier to deal with stacked changes.\n> >\n> > The following set of patches introduces the design doc on the evolve command\n> > as well as the basics of the git change command.\n>\n> Thanks for picking this up, having an evolve command would be a really\n> useful addition to git. I read the final four patches as I was\n> interested to see how a user would use \"git change\" to track changes to\n> a set of commits. Unfortunately because there are no tests and scant\n> documentation there are no examples of how to do this. Looking at the\n> patches I felt like it would have been helpful to mark them as RFC to\n> indicate that the author is requesting feedback but does not consider\n> them ready for merging.\n>\n> I'm confused as to why the command is called \"change\" (which I don't\n> find particularly descriptive) when every patch subject is \"evolve\". It\n> definitely makes sense to request feedback on a large topic like this\n> before everything is implemented but I'd be nervous of merging the early\n> stages before there is a working evolve command. For an example of a\n> successful multipart topic see\n> https://lore.kernel.org/git/pull.1248.git.1654545325.gitgitgadget@gmail.com/\n> Knowing the author of that series the commit messages should also give\n> you a good idea of the level of detail expected.\n>\n> Best Wishes\n>\n> Phillip\n>\n> > Chris Poucet (4):\n> > sha1-array: implement oid_array_readonly_contains\n> > ref-filter: add the metas namespace to ref-filter\n> > evolve: add delete command\n> > evolve: add documentation for `git change`\n> >\n> > Stefan Xenos (6):\n> > technical doc: add a design doc for the evolve command\n> > evolve: add support for parsing metacommits\n> > evolve: add the change-table structure\n> > evolve: add support for writing metacommits\n> > evolve: implement the git change command\n> > evolve: add the git change list command\n> >\n> > .gitignore | 1 +\n> > Documentation/git-change.txt | 55 ++\n> > Documentation/technical/evolve.txt | 1051 ++++++++++++++++++++++++++++\n> > Makefile | 4 +\n> > builtin.h | 1 +\n> > builtin/change.c | 342 +++++++++\n> > change-table.c | 179 +++++\n> > change-table.h | 132 ++++\n> > git.c | 1 +\n> > metacommit-parser.c | 110 +++\n> > metacommit-parser.h | 19 +\n> > metacommit.c | 404 +++++++++++\n> > metacommit.h | 58 ++\n> > oid-array.c | 12 +\n> > oid-array.h | 7 +\n> > ref-filter.c | 10 +-\n> > ref-filter.h | 8 +-\n> > t/helper/test-oid-array.c | 6 +\n> > t/t0064-oid-array.sh | 22 +\n> > 19 files changed, 2418 insertions(+), 4 deletions(-)\n> > create mode 100644 Documentation/git-change.txt\n> > create mode 100644 Documentation/technical/evolve.txt\n> > create mode 100644 builtin/change.c\n> > create mode 100644 change-table.c\n> > create mode 100644 change-table.h\n> > create mode 100644 metacommit-parser.c\n> > create mode 100644 metacommit-parser.h\n> > create mode 100644 metacommit.c\n> > create mode 100644 metacommit.h\n> >\n> >\n> > base-commit: 4b79ee4b0cd1130ba8907029cdc5f6a1632aca26\n> > Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1356%2Fpoucet%2Fevolve-v1\n> > Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1356/poucet/evolve-v1\n> > Pull-Request: https://github.com/gitgitgadget/git/pull/1356\n"},{"id":"464115","messageId":"CAN84kK=vuvmaPJeVupV-ZN2Cnn_E+9STvxRqEbQc4m3dQbD34w@mail.gmail.com","threadId":"58504","inReplyTo":"fc291c07-55e9-64f8-1251-20bd2422024d@gmail.com","subject":"Re: [PATCH 03/10] ref-filter: add the metas namespace to ref-filter","fromName":"Chris P","fromEmail":"christophe.poucet@gmail.com","sentAt":"2022-10-04T09:50:50Z","receivedAt":"2022-10-04T09:52:39Z","isPatch":true,"sender":{"key":"christophe.poucet@gmail.com","avatar":null},"body":"> I assume this is to save having to write \"refs/metas/\" when we want to\n> search for meta commits?\n\nYes, though currently it still requires \"metas/\", I'm trying to figure\nout how to\nremove that.\n\n> Signed-off-by: Chris Poucet \n> --- > diff --git a/ref-filter.h b/ref-filter.h\n> index aa0eea4ecf5..064fbef8e50 100644\n> --- a/ref-filter.h\n> +++ b/ref-filter.h\n> @@ -17,8 +17,10 @@\n> #define FILTER_REFS_BRANCHES 0x0004\n> #define FILTER_REFS_REMOTES 0x0008\n> #define FILTER_REFS_OTHERS 0x0010\n> +#define FILTER_REFS_CHANGES 0x0040\n\n> It would be nice to keep FILTER_REFS_OTHERS at the end I think (we don't\n> need to worry about abi compatibility), also what happened to 0x0020?\n\nThe 0x0020 is listed further below, I don't know why the previous\nauthor decided to\nput them out of order. I've renumbered them as you've requested with\nthe assumption\nthat this data never gets serialized to storage.\n\n> Best Wishes\n\nThank you for all the feedback!\n\n- simply chris\n"},{"id":"464120","messageId":"CAN84kKmsFiGm2W+74aBbe=fXjDeK05ujCxNF+wTHGEjEkQwjDw@mail.gmail.com","threadId":"58504","inReplyTo":"e7278794-428d-4aff-e91b-d2e6527f142d@gmail.com","subject":"Re: [PATCH 04/10] evolve: add support for parsing metacommits","fromName":"Chris P","fromEmail":"christophe.poucet@gmail.com","sentAt":"2022-10-04T11:21:46Z","receivedAt":"2022-10-04T11:22:13Z","isPatch":true,"sender":{"key":"christophe.poucet@gmail.com","avatar":null},"body":"> > This patch adds the get_metacommit_content method, which can classify\n> > commits as either metacommits or normal commits, determine whether they\n> > are abandoned, and extract the content commit's object id from the\n> > metacommit.\n> > diff --git a/Makefile b/Makefile\n> > index cac3452edb9..b2bcc00c289 100644\n> > --- a/Makefile\n> > +++ b/Makefile\n> > @@ -999,6 +999,7 @@ LIB_OBJS += merge-ort.o\n> > LIB_OBJS += merge-ort-wrappers.o\n> > LIB_OBJS += merge-recursive.o\n> > LIB_OBJS += merge.o\n> > +LIB_OBJS += metacommit-parser.o\n>\n> There seems to be a problem with the indent here\n\nI'm not sure I follow, there's not indentation on that line?\n>\n> > LIB_OBJS += midx.o\n> > LIB_OBJS += name-hash.o\n> > LIB_OBJS += negotiator/default.o\n>\n> > diff --git a/metacommit-parser.h b/metacommit-parser.h\n> > new file mode 100644\n> > index 00000000000..1c74bd6d699\n> > --- /dev/null\n> > +++ b/metacommit-parser.h\n> > @@ -0,0 +1,19 @@\n> > +#ifndef METACOMMIT_PARSER_H\n> > +#define METACOMMIT_PARSER_H\n> > +\n> > +#include \"commit.h\"\n> > +#include \"hash.h\"\n> > +\n> > +/* Indicates a normal commit (non-metacommit) */\n> > +#define METACOMMIT_TYPE_NONE 0\n> > +/* Indicates a metacommit with normal content (non-abandoned) */\n> > +#define METACOMMIT_TYPE_NORMAL 1\n> > +/* Indicates a metacommit with abandoned content */\n> > +#define METACOMMIT_TYPE_ABANDONED 2\n>\n> Is it possible to define these as an enum? It would make the signature\n> of get_meta_commit_content() nicer.\n>\n> > +struct commit;\n>\n> What's this for? We're including commit.h above.\n\nForgot to remove this as I added the include commit.h later.\n\n>\n> > +extern int get_metacommit_content(\n> > + struct commit *commit, struct object_id *content);\n>\n> > diff --git a/metacommit-parser.c b/metacommit-parser.c\n> > new file mode 100644\n> > index 00000000000..70c1428bfc6\n> > --- /dev/null\n> > +++ b/metacommit-parser.c\n> > @@ -0,0 +1,110 @@\n> > +#include \"cache.h\"\n> > +#include \"metacommit-parser.h\"\n> > +#include \"commit.h\"\n> > +\n> > +/*\n> > + * Search the commit buffer for a line starting with the given key. Unlike\n> > + * find_commit_header, this also searches the commit message body.\n> > + */\n>\n> There is no explanation in the code or commit message as to why this\n> function is needed. The documentation added in the first commit says\n> that \"parent-type\" header is a commit header. I think the answer is that\n> this series does not implement that header but uses the commit message\n> instead. That's perfectly fine for a proof of concept but it is\n> precisely the sort of detail that should be described it the commit\n> message and probably flagged up in the cover letter.\n\nI admit I thought I thought this was part of the header because it\nshows up before\nthe blank line before the commit title.\n\nHow do I make this a commit header?\n\n>\n> > +static const char *find_key(const char *msg, const char *key, size_t *out_len)\n> > +{\n> > + int key_len = strlen(key);\n> > + const char *line = msg;\n> > +\n> > + while (line) {\n> > + const char *eol = strchrnul(line, '\\n');\n> > +\n> > + if (eol - line > key_len && !memcmp(line, key, key_len) &&\n> > + line[key_len] == ' ') {\n> > + *out_len = eol - line - key_len - 1;\n> > + return line + key_len + 1;\n> > + }\n> > + line = *eol ? eol + 1 : NULL;\n> > + }\n> > + return NULL;\n> > +}\n> > +\n> > +static struct commit *get_commit_by_index(struct commit_list *to_search, int index)\n> > +{\n> > + while (to_search && index) {\n> > + to_search = to_search->next;\n> > + index--;\n> > + }\n> > +\n> > + if (!to_search)\n> > + return NULL;\n> > +\n> > + return to_search->item;\n> > +}\n>\n> This function is a useful utility for struct commit_list and should live\n> in commit.c. It could be used to simplify object-name.c:get_parent() for\n> example.\n\nDone. I'll defer cleaning up get_parent to a potentially later change to avoid\nmuddying up this change too much.\n\n>\n> > +/*\n> > + * Writes the index of the content parent to \"result\". Returns the metacommit\n> > + * type. See the METACOMMIT_TYPE_* constants.\n> > + */\n> > +static int index_of_content_commit(const char *buffer, int *result)\n>\n> I found the signature confusing as it is returning an int but that is\n> not the index. Switching to an enum for the metacommit types would\n> clarify that.\n\nDone.\n\n>\n> > +{\n> > + int index = 0;\n> > + int ret = METACOMMIT_TYPE_NONE;\n> > + size_t parent_types_size;\n> > + const char *parent_types = find_key(buffer, \"parent-type\",\n> > + &parent_types_size);\n> > + const char *end;\n> > + const char *enum_start = parent_types;\n> > + int enum_length = 0;\n> > +\n> > + if (!parent_types)\n> > + return METACOMMIT_TYPE_NONE;\n> > +\n> > + end = &parent_types[parent_types_size];\n> > +\n> > + while (1) {\n> > + char next = *parent_types;\n> > + if (next == ' ' || parent_types >= end) {\n> > + if (enum_length == 1) {\n>\n> if enum_length != 1 then there is an error in the parent-type header and\n> we should probably bail out.\n>\n> > + char first_char_in_enum = *enum_start;\n>\n> It's not just the first character, it's the only character, do we really\n> need such a long variable name? (how about just calling it \"type\")\n\nDone.\n\n> I'll try and take at look at the next couple of patches later in the week.\n\nThank you for all the reviews!\n\n-- simply chris\n"},{"id":"464170","messageId":"003a4462-dd98-9ffb-6bd8-771dd18693e8@dunelm.org.uk","threadId":"58504","inReplyTo":"CAN84kKmsFiGm2W+74aBbe=fXjDeK05ujCxNF+wTHGEjEkQwjDw@mail.gmail.com","subject":"Re: [PATCH 04/10] evolve: add support for parsing metacommits","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-04T14:10:27Z","receivedAt":"2022-10-04T14:10:36Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 04/10/2022 12:21, Chris P wrote:\n>>> This patch adds the get_metacommit_content method, which can classify\n>>> commits as either metacommits or normal commits, determine whether they\n>>> are abandoned, and extract the content commit's object id from the\n>>> metacommit.\n>>> diff --git a/Makefile b/Makefile\n>>> index cac3452edb9..b2bcc00c289 100644\n>>> --- a/Makefile\n>>> +++ b/Makefile\n>>> @@ -999,6 +999,7 @@ LIB_OBJS += merge-ort.o\n>>> LIB_OBJS += merge-ort-wrappers.o\n>>> LIB_OBJS += merge-recursive.o\n>>> LIB_OBJS += merge.o\n>>> +LIB_OBJS += metacommit-parser.o\n>>\n>> There seems to be a problem with the indent here\n> \n> I'm not sure I follow, there's not indentation on that line?\n\nFor some reason LIB_OBJS on that line does not line up with the lines \neither side of it in my mailer, but looking at the patch on \nlore.kernel.org it seems fine so I think the problem was at my end.\n\n>>> diff --git a/metacommit-parser.c b/metacommit-parser.c\n>>> new file mode 100644\n>>> index 00000000000..70c1428bfc6\n>>> --- /dev/null\n>>> +++ b/metacommit-parser.c\n>>> @@ -0,0 +1,110 @@\n>>> +#include \"cache.h\"\n>>> +#include \"metacommit-parser.h\"\n>>> +#include \"commit.h\"\n>>> +\n>>> +/*\n>>> + * Search the commit buffer for a line starting with the given key. Unlike\n>>> + * find_commit_header, this also searches the commit message body.\n>>> + */\n>>\n>> There is no explanation in the code or commit message as to why this\n>> function is needed. The documentation added in the first commit says\n>> that \"parent-type\" header is a commit header. I think the answer is that\n>> this series does not implement that header but uses the commit message\n>> instead. That's perfectly fine for a proof of concept but it is\n>> precisely the sort of detail that should be described it the commit\n>> message and probably flagged up in the cover letter.\n> \n> I admit I thought I thought this was part of the header because it\n> shows up before\n> the blank line before the commit title.\n\nIf I create a meta-commit and then run \"git cat-file commit\" on it I see\n\ntree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\nparent fd7e455287603d5bb2e3623dc442b592411cbfe9\nparent d79ce1670bdcb76e6d1da2ae095e890ccb326ae9\nauthor A U Thor 1112912113 -0700\ncommitter C O Mitter 1112912113 -0700\n\nparent-type c r\n\ni.e. the parent-type comes after the blank line that separates the \nheaders from the message\n\n> How do I make this a commit header?\n\nI've left some comments on the patch that creates the meta-commits. \nSince I wrote the above Junio has commented[1] that he prefers the \ncommit message approach to adding a new header so I'd leave the creation \nas it is for now and change find_key() just to look at the commit \nmessage. (I do prefer the idea of a new header as it provides an \nunambiguous way to distinguish meta-commits from normal commits but lets \nsee how using the commit message pans out)\n\n[1] https://lore.kernel.org/git/xmqqsfkbqjgz.fsf@gitster.g/\n\n>>> +static const char *find_key(const char *msg, const char *key, size_t *out_len)\n>>> +{\n>>> + int key_len = strlen(key);\n>>> + const char *line = msg;\n>>> +\n>>> + while (line) {\n>>> + const char *eol = strchrnul(line, '\\n');\n>>> +\n>>> + if (eol - line > key_len && !memcmp(line, key, key_len) &&\n>>> + line[key_len] == ' ') {\n>>> + *out_len = eol - line - key_len - 1;\n>>> + return line + key_len + 1;\n>>> + }\n>>> + line = *eol ? eol + 1 : NULL;\n>>> + }\n>>> + return NULL;\n>>> +}\n>>> +\n>>> +static struct commit *get_commit_by_index(struct commit_list *to_search, int index)\n>>> +{\n>>> + while (to_search && index) {\n>>> + to_search = to_search->next;\n>>> + index--;\n>>> + }\n>>> +\n>>> + if (!to_search)\n>>> + return NULL;\n>>> +\n>>> + return to_search->item;\n>>> +}\n>>\n>> This function is a useful utility for struct commit_list and should live\n>> in commit.c. It could be used to simplify object-name.c:get_parent() for\n>> example.\n> \n> Done. I'll defer cleaning up get_parent to a potentially later change to avoid\n> muddying up this change too much.\n\nSure, get_parent() was meant as an example of why the function is useful \noutside of this work, while you're very welcome to clean it up please \ndon't feel that you are obliged to.\n\n>> I'll try and take at look at the next couple of patches later in the week.\n> \n> Thank you for all the reviews!\n\nYou're welcome, I'm excited to see evolve getting some attention again.\n\nPhillip\n\n\n> -- simply chris\n"},{"id":"464171","messageId":"2a1c3861-c752-4f40-59d3-803a62cd1bc2@dunelm.org.uk","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"Re: [PATCH 00/10] Add the Git Change command","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-04T14:24:48Z","receivedAt":"2022-10-04T14:25:01Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 23/09/2022 19:55, Christophe Poucet via GitGitGadget wrote:\n> I'm reviving the original git evolve work that was started by\n> sxenos@google.com\n> (https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n> \n> This work is intended to make it easier to deal with stacked changes.\n> \n> The following set of patches introduces the design doc on the evolve command\n> as well as the basics of the git change command.\n\nOur test suite can be a little tricky to get started with and I was impatient to\ncheck the basic functionality of these patches so I've written some simple\nexample tests for the change command and a couple of fixups to make them pass.\n\nBest Wishes\n\nPhillip\n\n---- >8 ----\n\n From a7c38d0f388e4d8a1f3debcc3069a7fb43084eda Mon Sep 17 00:00:00 2001\nFrom: Phillip Wood \nDate: Tue, 4 Oct 2022 15:12:36 +0100\nSubject: [PATCH 1/3] fixup! evolve: add support for writing metacommits\n\n---\n metacommit.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/metacommit.c b/metacommit.c\nindex d2b859a4d3..8f970fa104 100644\n--- a/metacommit.c\n+++ b/metacommit.c\n@@ -296,7 +296,7 @@ int record_metacommit_withresult(\n if (override_change) {\n string_list_clear(changes, 0);\n overridden_head = get_change_head(chtable, override_change);\n- if (!overridden_head) {\n+ if (overridden_head) {\n /* This is an existing change */\n old_head = &overridden_head->head;\n if (!force) {\n-- \n2.37.3.947.g1b8ba4da7f.dirty\n\n\n From cc7e8ba0b1a90268ced85d3f0c91aed49f2246d6 Mon Sep 17 00:00:00 2001\nFrom: Phillip Wood \nDate: Tue, 4 Oct 2022 15:15:32 +0100\nSubject: [PATCH 2/3] fixup! evolve: implement the git change command\n\n---\n t/t9999-changes.sh | 126 +++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 126 insertions(+)\n create mode 100755 t/t9999-changes.sh\n\ndiff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\nnew file mode 100755\nindex 0000000000..9e58925b23\n--- /dev/null\n+++ b/t/t9999-changes.sh\n@@ -0,0 +1,126 @@\n+#!/bin/sh\n+\n+test_description='git change - low level meta-commit management'\n+\n+. ./test-lib.sh\n+\n+. \"$TEST_DIRECTORY\"/lib-rebase.sh\n+\n+test_expect_success 'setup commits and meta-commits' '\n+ for c in one two three\n+ do\n+ test_commit $c &&\n+ git change update --content $c >actual 2>err &&\n+ echo \"Created change metas/$c\" >expect &&\n+ test_cmp expect actual &&\n+ test_must_be_empty err &&\n+ test_cmp_rev refs/metas/$c $c || return 1\n+ done\n+'\n+\n+# Check a meta-commit has the correct parents Call with the object\n+# name of the meta-commit followed by pairs of type and parent\n+check_meta_commit () {\n+ name=$1\n+ shift\n+ while test $# -gt 0\n+ do\n+ printf '%s %s\\n' $1 $(git rev-parse --verify $2)\n+ shift\n+ shift\n+ done | sort >expect\n+ git cat-file commit $name >metacommit &&\n+ # commit body should consist of parent-type\n+ types=\"$(sed -n '/^$/ {\n+ :loop\n+ n\n+ s/^parent-type //\n+ p\n+ b loop\n+ }' metacommit)\" &&\n+ while read key value\n+ do\n+ # TODO: don't sort the first parent\n+ if test \"$key\" = \"parent\"\n+ then\n+ type=\"${types%% *}\"\n+ test -n \"$type\" || return 1\n+ printf '%s %s\\n' $type $value\n+ types=\"${types#?}\"\n+ types=\"${types# }\"\n+ elif test \"$key\" = \"tree\"\n+ then\n+ test_cmp_rev \"$value\" $EMPTY_TREE || return 1\n+ elif test -z \"$key\"\n+ then\n+ # only parse commit headers\n+ break\n+ fi\n+ done actual-unsorted &&\n+ test -z \"$types\" &&\n+ sort >actual out 2>err &&\n+ echo \"Updated change metas/one\" >expect &&\n+ test_cmp expect out &&\n+ test_must_be_empty err &&\n+ git change update --replace tags/two --content HEAD@{2} &&\n+ oid=$(git rev-parse --verify metas/two) &&\n+ git change update --replace HEAD@{2} --replace tags/three \\\n+ --content HEAD &&\n+\n+ # check meta-commits\n+ check_meta_commit metas/one c HEAD~1 r tags/one &&\n+ check_meta_commit $oid c HEAD@{2} r tags/two &&\n+ # NB this checks that \"git change update\" uses the meta-commit ($oid)\n+ # corresponding to the replaces commit (HEAD@2 above) given on the\n+ # commandline.\n+ check_meta_commit metas/two c HEAD r $oid r tags/three &&\n+ check_meta_commit metas/three c HEAD r $oid r tags/three\n+'\n+\n+reset_meta_commits () {\n+ for c in one two three\n+ do\n+ echo \"update refs/metas/$c refs/tags/$c^0\"\n+ done | git update-ref --stdin\n+}\n+\n+test_expect_success 'override change name' '\n+ # TODO: builtin/change.c expects --change to be the full refname,\n+ # ideally it would prepend refs/metas to the string given by the\n+ # user.\n+ git change update --change refs/metas/another-one --content one &&\n+ test_cmp_rev metas/another-one one\n+'\n+\n+test_expect_success 'non-fast forward meta-commit update refused' '\n+ test_must_fail git change update --change refs/metas/one --content two \\\n+ >out 2>err &&\n+ echo \"error: non-fast-forward update to ${SQ}refs/metas/one${SQ}\" \\\n+ >expect &&\n+ test_cmp expect err &&\n+ test_must_be_empty out\n+'\n+\n+test_expect_success 'forced non-fast forward update succeeds' '\n+ git change update --change refs/metas/one --content two --force \\\n+ >out 2>err &&\n+ echo \"Updated change metas/one\" >expect &&\n+ test_cmp expect out &&\n+ test_must_be_empty err\n+'\n+\n+test_done\n-- \n2.37.3.947.g1b8ba4da7f.dirty\n\n\n From 7784f253fa799dd11fcbc81fe815fb387af52d97 Mon Sep 17 00:00:00 2001\nFrom: Phillip Wood \nDate: Tue, 4 Oct 2022 15:16:05 +0100\nSubject: [PATCH 3/3] fixup! evolve: add the git change list command\n\n---\n builtin/change.c | 16 +++++-----------\n t/t9999-changes.sh | 11 +++++++++++\n 2 files changed, 16 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/change.c b/builtin/change.c\nindex 07d029d82d..888ef648fa 100644\n--- a/builtin/change.c\n+++ b/builtin/change.c\n@@ -34,9 +34,8 @@ static int change_list(int argc, const char **argv, const char* prefix)\n OPT_END()\n };\n struct ref_filter filter;\n- /* TODO: See below\n struct ref_sorting *sorting;\n- struct string_list sorting_options = STRING_LIST_INIT_DUP; */\n+ struct string_list sorting_options = STRING_LIST_INIT_DUP;\n struct ref_format format = REF_FORMAT_INIT;\n struct ref_array array;\n int i;\n@@ -53,19 +52,15 @@ static int change_list(int argc, const char **argv, const char* prefix)\n \n filter_refs(&array, &filter, FILTER_REFS_CHANGES);\n \n- /* TODO: This causes a crash. It sets one of the atom_value handlers to\n- * something invalid, which causes a crash later when we call\n- * show_ref_array_item. Figure out why this happens and put back the sorting.\n- *\n- * sorting = ref_sorting_options(&sorting_options);\n- * ref_array_sort(sorting, &array); */\n-\n if (!format.format)\n format.format = \"%(refname:lstrip=1)\";\n \n if (verify_ref_format(&format))\n die(_(\"unable to parse format string\"));\n \n+ sorting = ref_sorting_options(&sorting_options);\n+ ref_array_sort(sorting, &array);\n+\n for (i = 0; i < array.nr; i++) {\n struct strbuf output = STRBUF_INIT;\n struct strbuf err = STRBUF_INIT;\n@@ -79,8 +74,7 @@ static int change_list(int argc, const char **argv, const char* prefix)\n }\n \n ref_array_clear(&array);\n- /* TODO: see above\n- ref_sorting_release(sorting); */\n+ ref_sorting_release(sorting);\n \n return 0;\n }\ndiff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\nindex 9e58925b23..9312eba86d 100755\n--- a/t/t9999-changes.sh\n+++ b/t/t9999-changes.sh\n@@ -123,4 +123,15 @@ test_expect_success 'forced non-fast forward update succeeds' '\n test_must_be_empty err\n '\n \n+test_expect_success 'list changes' '\n+ cat >expect <<-\\EOF &&\n+ metas/another-one\n+ metas/one\n+ metas/three\n+ metas/two\n+ EOF\n+ git change list >actual &&\n+ test_cmp expect actual\n+'\n+\n test_done\n-- \n2.37.3.947.g1b8ba4da7f.dirty\n"},{"id":"464173","messageId":"CAN84kKnsxZ2upEFD9Miv51KfxV-rFL7iZmPDS4nx6zb9agSXRA@mail.gmail.com","threadId":"58504","inReplyTo":"3c61e0b3-5526-f42e-48a7-c4465d06ccb3@dunelm.org.uk","subject":"Re: [PATCH 05/10] evolve: add the change-table structure","fromName":"Chris P","fromEmail":"christophe.poucet@gmail.com","sentAt":"2022-10-04T14:48:42Z","receivedAt":"2022-10-04T14:48:58Z","isPatch":true,"sender":{"key":"christophe.poucet@gmail.com","avatar":null},"body":"> > +/**\n>\n> We tend to just use '/*' rather than '/**'\n\nIt seems there's some disagreement on this. Regardless, I changed the\nones in the implementation to be \"/*\"\n\n>\n> > + * This struct holds a list of change refs. The first element is\n> stored inline,\n> > + * to optimize for small lists.\n> > + */\n> > +struct change_list {\n> > + /**\n> > + * Ref name for the first change in the list, or null if none.\n> > + *\n> > + * This field is private. Use for_each_change_in to read.\n> > + */\n> > + const char* first_refname;\n> > + /**\n> > + * List of additional change refs. Note that this is empty if the list\n> > + * contains 0 or 1 elements.\n> > + *\n> > + * This field is private. Use for_each_change_in to read.\n> > + */\n> > + struct string_list additional_refnames;\n>\n> Splitting this feels like a premature optimization. We don't have any\n> tests yet, let alone any real-world experience using this code. Also if\n> we want to save memory for lists with a single entry why are we\n> embedding the struct string_list rather than just storing a pointer to it?\n\nAgreed, simplified to a strset. Thanks for the suggestion.\n\n>\n> I think it would be simpler to use a struct strset to hold the refnames\n> as we don't need the util field offered by struct string_list.\n\nDone.\n\n>\n> > +/**\n> > + * Holds information about the head of a single change.\n> > + */\n> > +struct change_head {\n> > + /**\n> > + * The location pointed to by the head of the change. May be a\n> commit or a\n> > + * metacommit.\n> > + */\n> > + struct object_id head;\n>\n> I found this duality between commits and metacommits rather confusing -\n> why isn't the head always a metacommit?\n\nThere is no reason to create a metacommit for the first commit you create.\nYou only need one if you're replacing a commit with another commit.\n\n>\n> > +/**\n> > + * Holds information about the heads of each change, and permits\n> effecient\n>\n> s/effecient/efficient/\n\nDone.\n\n>\n> > + * lookup from a commit to the changes that reference it directly.\n> > + *\n> > + * All fields should be considered private. Use the change_table\n> functions\n> > + * to interact with this struct.\n> > + */\n> > +struct change_table {\n> > + /**\n> > + * Memory pool for the objects allocated by the change table.\n> > + */\n> > + struct mem_pool memory_pool;\n> > + /* Map object_id to commit_change_list_entry structs. */\n> > + struct oidmap oid_to_metadata_index;\n> > + /**\n> > + * List of ref names. The util value points to a change_head structure\n> > + * allocated from memory_pool.\n> > + */\n> > + struct string_list refname_to_change_head;\n>\n> I think these days we'd use a strmap for this for O(1) lookups.\n\nWay better!\n\n>\n> > +};\n> > +\n> > +extern void change_table_init(struct change_table *to_initialize);\n>\n> The struct change_table argument to all these functions changes its name\n> more often than a criminal on the run. I would find it much easier to\n> follow the code if we consistently called this argument \"table\"\n\nAgreed, changed them all to \"table\".\n\n>\n> > + * Adds all changes matching the given ref filter to the given\n> change_table\n> > + * struct.\n> > + */\n> > +extern void change_table_add_matching_filter(struct change_table\n> *to_modify,\n> > + struct repository* repo, struct ref_filter *filter);\n>\n> I can't see any callers outside of change-table.c so do we really need\n> to export this function.\n\nThanks for verifying, done.\n\n> > +\n> > +void change_table_init(struct change_table *to_initialize)\n> > +{\n> > + memset(to_initialize, 0, sizeof(*to_initialize));\n> > + mem_pool_init(&to_initialize->memory_pool, 0);\n> > + to_initialize->memory_pool.block_alloc = 4*1024 - sizeof(struct mp_block);\n>\n> If we're using a mempool to minimize the allocation overhead we should\n> leave .block_alloc set to the default value of 1MB rather than changing\n> it to 4kB\n\nGood question, I don't know the typical sizes that we'll get for these,\nso for now just sticking with the default seems sensible.\n\n> > +\n> > +static void add_head_to_commit(struct change_table *to_modify,\n> > + const struct object_id *to_add, const char *refname)\n>\n> I found the function and argument names rather confusing. If I've\n> understood the code correctly then this function is adding an assoation\n> between the commit \"to_add\" and \"refname\". Despite its name \"to_add\" may\n> already exist in the change table.\n>\n> The formatting is a bit off as well (as are most of the function\n> declarations in this patch and the next), we'd write that as\n>\n> static void add_head_to_commit(struct change_table *table,\n> const struct object_id *to_add,\n> const char *refname)\n\nThanks, I wasn't clear on the guidelines. I hope the new format makes\nmore sense.\n\n>\n> > +{\n> > + struct commit_change_list_entry *entry;\n> > +\n> > + /**\n> > + * Note: the indices in the map are 1-based. 0 is used to indicate a missing\n> > + * element.\n> > + */\n>\n> I'm confused by this comment, what indices is it talking about?\n\nNo idea, removed.\n\n> > +\n> > + if (!entry->changes.first_refname)\n> > + entry->changes.first_refname = refname;\n> > + else\n> > + string_list_insert(&entry->changes.additional_refnames, refname);\n>\n> This is an example of the complexity added by the current definition of\n> struct change_list.\n\nYes, simplified.\n\n>\n> > +void change_table_add(struct change_table *to_modify, const char *refname,\n> > + struct commit *to_add)\n> > +{\n> > + struct change_head *new_head;\n> > + struct string_list_item *new_item;\n> > + int metacommit_type;\n> > +\n> > + new_head = mem_pool_calloc(&to_modify->memory_pool, 1,\n> > + sizeof(*new_head));\n> > +\n> > + oidcpy(&new_head->head, &to_add->object.oid);\n> > +\n> > + metacommit_type = get_metacommit_content(to_add, &new_head->content);\n> > + if (metacommit_type == METACOMMIT_TYPE_NONE)\n> > + oidcpy(&new_head->content, &to_add->object.oid);\n>\n> If to_add is not a metacommit then the content is to_add itself,\n> otherwise it will have been set by the call to get_metacommit_content().\n\nYes, added the comment.\n\n>\n> > + new_head->abandoned = (metacommit_type == METACOMMIT_TYPE_ABANDONED);\n>\n> Style: I don't think we normally bother with parentheses here\n\nI admit I prefer it here because operator priority isn't always\nobvious (it could be read as\n(new_head->abandoned = metacommit_type) == METACOMMIT_TYPE_ABANDONED;\n\n>\n> > + new_head->remote = starts_with(refname, \"refs/remote/\");\n> > + new_head->hidden = starts_with(refname, \"refs/hiddenmetas/\");\n> > +\n> > + new_item = string_list_insert(&to_modify->refname_to_change_head, refname);\n> > + new_item->util = new_head;\n> > + /* Use pointers to the copy of the string we're retaining locally */\n>\n> string_list_insert() copied the string and we're using that copy. Saying\n> we're retaining it locally when it will outlive this function call is\n> confusing.\n\nThis is now obsolete with the move to strmap.\n\n>\n> > + refname = new_item->string;\n> > +\n> > + if (!oideq(&new_head->content, &new_head->head))\n> > + add_head_to_commit(to_modify, &new_head->content, refname);\n>\n> If to_add is a metacommit then we remember the link between refname and\n> the content commit.\n>\n> > + add_head_to_commit(to_modify, &new_head->head, refname);\n>\n> We also remember the link between refname and to_add\n\nThanks, added the comment.\n\n>\n> > +}\n> > +\n> > +void change_table_add_all_visible(struct change_table *to_modify,\n> > + struct repository* repo)\n> > +{\n> > + struct ref_filter filter;\n>\n> rather than using memset we'd write (the same goes for all the other\n> memset() calls in this series, unless they're operation on a heap\n> allocation)\n>\n> struct ref_filter filter = { 0 };\n\nThanks, I wasn't aware of that trick.\n\n>\n> > + const char *name_patterns[] = {NULL};\n> > + memset(&filter, 0, sizeof(filter));\n> > + filter.kind = FILTER_REFS_CHANGES;\n> > + filter.name_patterns = name_patterns;\n> > +\n> > + change_table_add_matching_filter(to_modify, repo, &filter);\n> > +}\n> > +\n> > +void change_table_add_matching_filter(struct change_table *to_modify,\n> > + struct repository* repo, struct ref_filter *filter)\n> > +{\n> > + struct ref_array matching_refs;\n> > + int i;\n> > +\n> > + memset(&matching_refs, 0, sizeof(matching_refs));\n> > + filter_refs(&matching_refs, filter, filter->kind);\n> > +\n> > + /**\n> > + * Determine the object id for the latest content commit for each change.\n> > + * Fetch the commit at the head of each change ref. If it's a normal commit,\n> > + * that's the commit we want. If it's a metacommit, locate its content parent\n> > + * and use that.\n> > + */\n> > +\n> > + for (i = 0; i < matching_refs.nr; i++) {\n> > + struct ref_array_item *item = matching_refs.items[i];\n> > + struct commit *commit = item->commit;\n> > +\n> > + commit = lookup_commit_reference_gently(repo, &item->objectname, 1);\n>\n> We're assigning commit twice - why do we need to look it up if\n> filter_refs returns it?\n\nI think this is a case of missing logic if you look at what the\ncomment above it says.\n\n>\n> There are a number of places where we call\n> lookup_commit_reference_gently(..., 1) to silence the warning if the\n> objectname does not dereference to a commit. It is not clear to me that\n> we want to hide those errors. Indeed I think we should be doing\n\nAgreed, move to this.\n\n>\n> commit = lookup_commit_reference(repo, oid)\n> if (!commit)\n> BUG(\"commit missing ...\")\n>\n> unless there is a good reason that the lookup can fail.\n\nI can't think of any but then I'm not the original author.\n\n>\n> > + if (commit)\n> > + change_table_add(to_modify, item->refname, commit);\n> > + }\n> > +\n> > + ref_array_clear(&matching_refs);\n> > +}\n>\n> > +int for_each_change_referencing(struct change_table *table,\n> > + const struct object_id *referenced_commit_id, each_change_fn fn, void *cb_data)\n> > +{\n> > + const struct change_list *changes;\n> > + int i;\n> > + int retvalue;\n>\n> We normally use \"ret\" for this\n\nDone.\n\n>\n> > + struct commit_change_list_entry *entry;\n> > +\n> > + entry = oidmap_get(&table->oid_to_metadata_index,\n> > + referenced_commit_id);\n>\n> This should be indented to start below the '(' of the function call.\n\nDone.\n\n>\n> > + /* If this commit isn't referenced by any changes, it won't be in the map */\n> > + if (!entry)\n> > + return 0;\n> > + changes = &entry->changes;\n> > + if (!changes->first_refname)\n> > + return 0;\n> > + retvalue = fn(changes->first_refname, cb_data);\n> > + for (i = 0; retvalue == 0 && i < changes->additional_refnames.nr; i++)\n> > + retvalue = fn(changes->additional_refnames.items[i].string, cb_data);\n>\n> Using an strset for struct change_list would simplify this\n\nAgreed! Simplified.\n\n>\n> > + return retvalue;\n> > +}\n> > +\n> > +struct change_head* get_change_head(struct change_table *heads,\n> > + const char* refname)\n> > +{\n> > + struct string_list_item *item = string_list_lookup(\n> > + &heads->refname_to_change_head, refname);\n> > +\n> > + if (!item)\n> > + return NULL;\n> > +\n> > + return (struct change_head *)item->util;\n>\n> We don't bother with casting void* pointers like this. In any case this\n> whole function could become\n>\n> return strmap_get(table, refname)\n>\n> if we used an strmap instead of a string_list.\n>\n>\n> Aside from the style issues and using api's that have been added since\n> Stefan wrote these patches this looks pretty sound. The only thing I\n> don't really get why the public api allows normal commits to be added to\n> the change table (I can see why we might want to add the content commit\n> as well when we add a metacommit but that should be done internally)\n>\n> Best Wishes\n>\n> Phillip\n"},{"id":"464176","messageId":"CAN84kKnDvvG5V=eCnTPiNVC+CfWu9NFqwF+L4QuX26TAPov2Zg@mail.gmail.com","threadId":"58504","inReplyTo":"2a1c3861-c752-4f40-59d3-803a62cd1bc2@dunelm.org.uk","subject":"Re: [PATCH 00/10] Add the Git Change command","fromName":"Chris P","fromEmail":"christophe.poucet@gmail.com","sentAt":"2022-10-04T15:19:55Z","receivedAt":"2022-10-04T15:20:34Z","isPatch":true,"sender":{"key":"christophe.poucet@gmail.com","avatar":null},"body":"Thanks a lot.\n\nIs there something special I must do to get these scripts to work? The\nentire script fails for me, despite having build git-change.\n\nOn Tue, Oct 4, 2022 at 4:24 PM Phillip Wood wrote:\n>\n> Hi Chris\n>\n> On 23/09/2022 19:55, Christophe Poucet via GitGitGadget wrote:\n> > I'm reviving the original git evolve work that was started by\n> > sxenos@google.com\n> > (https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n> >\n> > This work is intended to make it easier to deal with stacked changes.\n> >\n> > The following set of patches introduces the design doc on the evolve command\n> > as well as the basics of the git change command.\n>\n> Our test suite can be a little tricky to get started with and I was impatient to\n> check the basic functionality of these patches so I've written some simple\n> example tests for the change command and a couple of fixups to make them pass.\n>\n> Best Wishes\n>\n> Phillip\n>\n> ---- >8 ----\n>\n> From a7c38d0f388e4d8a1f3debcc3069a7fb43084eda Mon Sep 17 00:00:00 2001\n> From: Phillip Wood \n> Date: Tue, 4 Oct 2022 15:12:36 +0100\n> Subject: [PATCH 1/3] fixup! evolve: add support for writing metacommits\n>\n> ---\n> metacommit.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/metacommit.c b/metacommit.c\n> index d2b859a4d3..8f970fa104 100644\n> --- a/metacommit.c\n> +++ b/metacommit.c\n> @@ -296,7 +296,7 @@ int record_metacommit_withresult(\n> if (override_change) {\n> string_list_clear(changes, 0);\n> overridden_head = get_change_head(chtable, override_change);\n> - if (!overridden_head) {\n> + if (overridden_head) {\n> /* This is an existing change */\n> old_head = &overridden_head->head;\n> if (!force) {\n> --\n> 2.37.3.947.g1b8ba4da7f.dirty\n>\n>\n> From cc7e8ba0b1a90268ced85d3f0c91aed49f2246d6 Mon Sep 17 00:00:00 2001\n> From: Phillip Wood \n> Date: Tue, 4 Oct 2022 15:15:32 +0100\n> Subject: [PATCH 2/3] fixup! evolve: implement the git change command\n>\n> ---\n> t/t9999-changes.sh | 126 +++++++++++++++++++++++++++++++++++++++++++++\n> 1 file changed, 126 insertions(+)\n> create mode 100755 t/t9999-changes.sh\n>\n> diff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\n> new file mode 100755\n> index 0000000000..9e58925b23\n> --- /dev/null\n> +++ b/t/t9999-changes.sh\n> @@ -0,0 +1,126 @@\n> +#!/bin/sh\n> +\n> +test_description='git change - low level meta-commit management'\n> +\n> +. ./test-lib.sh\n> +\n> +. \"$TEST_DIRECTORY\"/lib-rebase.sh\n> +\n> +test_expect_success 'setup commits and meta-commits' '\n> + for c in one two three\n> + do\n> + test_commit $c &&\n> + git change update --content $c >actual 2>err &&\n> + echo \"Created change metas/$c\" >expect &&\n> + test_cmp expect actual &&\n> + test_must_be_empty err &&\n> + test_cmp_rev refs/metas/$c $c || return 1\n> + done\n> +'\n> +\n> +# Check a meta-commit has the correct parents Call with the object\n> +# name of the meta-commit followed by pairs of type and parent\n> +check_meta_commit () {\n> + name=$1\n> + shift\n> + while test $# -gt 0\n> + do\n> + printf '%s %s\\n' $1 $(git rev-parse --verify $2)\n> + shift\n> + shift\n> + done | sort >expect\n> + git cat-file commit $name >metacommit &&\n> + # commit body should consist of parent-type\n> + types=\"$(sed -n '/^$/ {\n> + :loop\n> + n\n> + s/^parent-type //\n> + p\n> + b loop\n> + }' metacommit)\" &&\n> + while read key value\n> + do\n> + # TODO: don't sort the first parent\n> + if test \"$key\" = \"parent\"\n> + then\n> + type=\"${types%% *}\"\n> + test -n \"$type\" || return 1\n> + printf '%s %s\\n' $type $value\n> + types=\"${types#?}\"\n> + types=\"${types# }\"\n> + elif test \"$key\" = \"tree\"\n> + then\n> + test_cmp_rev \"$value\" $EMPTY_TREE || return 1\n> + elif test -z \"$key\"\n> + then\n> + # only parse commit headers\n> + break\n> + fi\n> + done actual-unsorted &&\n> + test -z \"$types\" &&\n> + sort >actual + test_cmp expect actual\n> +}\n> +\n> +test_expect_success 'update meta-commits after rebase' '\n> + (\n> + set_fake_editor &&\n> + FAKE_AMEND=edited &&\n> + FAKE_LINES=\"reword 1 pick 2 fixup 3\" &&\n> + export FAKE_AMEND FAKE_LINES &&\n> + git rebase -i --root\n> + ) &&\n> +\n> + # update meta-commits\n> + git change update --replace tags/one --content HEAD~1 >out 2>err &&\n> + echo \"Updated change metas/one\" >expect &&\n> + test_cmp expect out &&\n> + test_must_be_empty err &&\n> + git change update --replace tags/two --content HEAD@{2} &&\n> + oid=$(git rev-parse --verify metas/two) &&\n> + git change update --replace HEAD@{2} --replace tags/three \\\n> + --content HEAD &&\n> +\n> + # check meta-commits\n> + check_meta_commit metas/one c HEAD~1 r tags/one &&\n> + check_meta_commit $oid c HEAD@{2} r tags/two &&\n> + # NB this checks that \"git change update\" uses the meta-commit ($oid)\n> + # corresponding to the replaces commit (HEAD@2 above) given on the\n> + # commandline.\n> + check_meta_commit metas/two c HEAD r $oid r tags/three &&\n> + check_meta_commit metas/three c HEAD r $oid r tags/three\n> +'\n> +\n> +reset_meta_commits () {\n> + for c in one two three\n> + do\n> + echo \"update refs/metas/$c refs/tags/$c^0\"\n> + done | git update-ref --stdin\n> +}\n> +\n> +test_expect_success 'override change name' '\n> + # TODO: builtin/change.c expects --change to be the full refname,\n> + # ideally it would prepend refs/metas to the string given by the\n> + # user.\n> + git change update --change refs/metas/another-one --content one &&\n> + test_cmp_rev metas/another-one one\n> +'\n> +\n> +test_expect_success 'non-fast forward meta-commit update refused' '\n> + test_must_fail git change update --change refs/metas/one --content two \\\n> + >out 2>err &&\n> + echo \"error: non-fast-forward update to ${SQ}refs/metas/one${SQ}\" \\\n> + >expect &&\n> + test_cmp expect err &&\n> + test_must_be_empty out\n> +'\n> +\n> +test_expect_success 'forced non-fast forward update succeeds' '\n> + git change update --change refs/metas/one --content two --force \\\n> + >out 2>err &&\n> + echo \"Updated change metas/one\" >expect &&\n> + test_cmp expect out &&\n> + test_must_be_empty err\n> +'\n> +\n> +test_done\n> --\n> 2.37.3.947.g1b8ba4da7f.dirty\n>\n>\n> From 7784f253fa799dd11fcbc81fe815fb387af52d97 Mon Sep 17 00:00:00 2001\n> From: Phillip Wood \n> Date: Tue, 4 Oct 2022 15:16:05 +0100\n> Subject: [PATCH 3/3] fixup! evolve: add the git change list command\n>\n> ---\n> builtin/change.c | 16 +++++-----------\n> t/t9999-changes.sh | 11 +++++++++++\n> 2 files changed, 16 insertions(+), 11 deletions(-)\n>\n> diff --git a/builtin/change.c b/builtin/change.c\n> index 07d029d82d..888ef648fa 100644\n> --- a/builtin/change.c\n> +++ b/builtin/change.c\n> @@ -34,9 +34,8 @@ static int change_list(int argc, const char **argv, const char* prefix)\n> OPT_END()\n> };\n> struct ref_filter filter;\n> - /* TODO: See below\n> struct ref_sorting *sorting;\n> - struct string_list sorting_options = STRING_LIST_INIT_DUP; */\n> + struct string_list sorting_options = STRING_LIST_INIT_DUP;\n> struct ref_format format = REF_FORMAT_INIT;\n> struct ref_array array;\n> int i;\n> @@ -53,19 +52,15 @@ static int change_list(int argc, const char **argv, const char* prefix)\n>\n> filter_refs(&array, &filter, FILTER_REFS_CHANGES);\n>\n> - /* TODO: This causes a crash. It sets one of the atom_value handlers to\n> - * something invalid, which causes a crash later when we call\n> - * show_ref_array_item. Figure out why this happens and put back the sorting.\n> - *\n> - * sorting = ref_sorting_options(&sorting_options);\n> - * ref_array_sort(sorting, &array); */\n> -\n> if (!format.format)\n> format.format = \"%(refname:lstrip=1)\";\n>\n> if (verify_ref_format(&format))\n> die(_(\"unable to parse format string\"));\n>\n> + sorting = ref_sorting_options(&sorting_options);\n> + ref_array_sort(sorting, &array);\n> +\n> for (i = 0; i < array.nr; i++) {\n> struct strbuf output = STRBUF_INIT;\n> struct strbuf err = STRBUF_INIT;\n> @@ -79,8 +74,7 @@ static int change_list(int argc, const char **argv, const char* prefix)\n> }\n>\n> ref_array_clear(&array);\n> - /* TODO: see above\n> - ref_sorting_release(sorting); */\n> + ref_sorting_release(sorting);\n>\n> return 0;\n> }\n> diff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\n> index 9e58925b23..9312eba86d 100755\n> --- a/t/t9999-changes.sh\n> +++ b/t/t9999-changes.sh\n> @@ -123,4 +123,15 @@ test_expect_success 'forced non-fast forward update succeeds' '\n> test_must_be_empty err\n> '\n>\n> +test_expect_success 'list changes' '\n> + cat >expect <<-\\EOF &&\n> + metas/another-one\n> + metas/one\n> + metas/three\n> + metas/two\n> + EOF\n> + git change list >actual &&\n> + test_cmp expect actual\n> +'\n> +\n> test_done\n> --\n> 2.37.3.947.g1b8ba4da7f.dirty\n"},{"id":"464177","messageId":"CAN84kKmx8fMppFdFG98VyKfK_vpb5nbXLXGUbJWupyVTNqak3A@mail.gmail.com","threadId":"58504","inReplyTo":"CAN84kKnDvvG5V=eCnTPiNVC+CfWu9NFqwF+L4QuX26TAPov2Zg@mail.gmail.com","subject":"Re: [PATCH 00/10] Add the Git Change command","fromName":"Chris P","fromEmail":"christophe.poucet@gmail.com","sentAt":"2022-10-04T15:55:54Z","receivedAt":"2022-10-04T15:56:10Z","isPatch":true,"sender":{"key":"christophe.poucet@gmail.com","avatar":null},"body":"I got the tests to work, except the last one, that one fails.\n\nHow can I see the output of actual and expect?\n\nOn Tue, Oct 4, 2022 at 5:19 PM Chris P wrote:\n>\n> Thanks a lot.\n>\n> Is there something special I must do to get these scripts to work? The\n> entire script fails for me, despite having build git-change.\n>\n> On Tue, Oct 4, 2022 at 4:24 PM Phillip Wood wrote:\n> >\n> > Hi Chris\n> >\n> > On 23/09/2022 19:55, Christophe Poucet via GitGitGadget wrote:\n> > > I'm reviving the original git evolve work that was started by\n> > > sxenos@google.com\n> > > (https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n> > >\n> > > This work is intended to make it easier to deal with stacked changes.\n> > >\n> > > The following set of patches introduces the design doc on the evolve command\n> > > as well as the basics of the git change command.\n> >\n> > Our test suite can be a little tricky to get started with and I was impatient to\n> > check the basic functionality of these patches so I've written some simple\n> > example tests for the change command and a couple of fixups to make them pass.\n> >\n> > Best Wishes\n> >\n> > Phillip\n> >\n> > ---- >8 ----\n> >\n> > From a7c38d0f388e4d8a1f3debcc3069a7fb43084eda Mon Sep 17 00:00:00 2001\n> > From: Phillip Wood \n> > Date: Tue, 4 Oct 2022 15:12:36 +0100\n> > Subject: [PATCH 1/3] fixup! evolve: add support for writing metacommits\n> >\n> > ---\n> > metacommit.c | 2 +-\n> > 1 file changed, 1 insertion(+), 1 deletion(-)\n> >\n> > diff --git a/metacommit.c b/metacommit.c\n> > index d2b859a4d3..8f970fa104 100644\n> > --- a/metacommit.c\n> > +++ b/metacommit.c\n> > @@ -296,7 +296,7 @@ int record_metacommit_withresult(\n> > if (override_change) {\n> > string_list_clear(changes, 0);\n> > overridden_head = get_change_head(chtable, override_change);\n> > - if (!overridden_head) {\n> > + if (overridden_head) {\n> > /* This is an existing change */\n> > old_head = &overridden_head->head;\n> > if (!force) {\n> > --\n> > 2.37.3.947.g1b8ba4da7f.dirty\n> >\n> >\n> > From cc7e8ba0b1a90268ced85d3f0c91aed49f2246d6 Mon Sep 17 00:00:00 2001\n> > From: Phillip Wood \n> > Date: Tue, 4 Oct 2022 15:15:32 +0100\n> > Subject: [PATCH 2/3] fixup! evolve: implement the git change command\n> >\n> > ---\n> > t/t9999-changes.sh | 126 +++++++++++++++++++++++++++++++++++++++++++++\n> > 1 file changed, 126 insertions(+)\n> > create mode 100755 t/t9999-changes.sh\n> >\n> > diff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\n> > new file mode 100755\n> > index 0000000000..9e58925b23\n> > --- /dev/null\n> > +++ b/t/t9999-changes.sh\n> > @@ -0,0 +1,126 @@\n> > +#!/bin/sh\n> > +\n> > +test_description='git change - low level meta-commit management'\n> > +\n> > +. ./test-lib.sh\n> > +\n> > +. \"$TEST_DIRECTORY\"/lib-rebase.sh\n> > +\n> > +test_expect_success 'setup commits and meta-commits' '\n> > + for c in one two three\n> > + do\n> > + test_commit $c &&\n> > + git change update --content $c >actual 2>err &&\n> > + echo \"Created change metas/$c\" >expect &&\n> > + test_cmp expect actual &&\n> > + test_must_be_empty err &&\n> > + test_cmp_rev refs/metas/$c $c || return 1\n> > + done\n> > +'\n> > +\n> > +# Check a meta-commit has the correct parents Call with the object\n> > +# name of the meta-commit followed by pairs of type and parent\n> > +check_meta_commit () {\n> > + name=$1\n> > + shift\n> > + while test $# -gt 0\n> > + do\n> > + printf '%s %s\\n' $1 $(git rev-parse --verify $2)\n> > + shift\n> > + shift\n> > + done | sort >expect\n> > + git cat-file commit $name >metacommit &&\n> > + # commit body should consist of parent-type\n> > + types=\"$(sed -n '/^$/ {\n> > + :loop\n> > + n\n> > + s/^parent-type //\n> > + p\n> > + b loop\n> > + }' metacommit)\" &&\n> > + while read key value\n> > + do\n> > + # TODO: don't sort the first parent\n> > + if test \"$key\" = \"parent\"\n> > + then\n> > + type=\"${types%% *}\"\n> > + test -n \"$type\" || return 1\n> > + printf '%s %s\\n' $type $value\n> > + types=\"${types#?}\"\n> > + types=\"${types# }\"\n> > + elif test \"$key\" = \"tree\"\n> > + then\n> > + test_cmp_rev \"$value\" $EMPTY_TREE || return 1\n> > + elif test -z \"$key\"\n> > + then\n> > + # only parse commit headers\n> > + break\n> > + fi\n> > + done actual-unsorted &&\n> > + test -z \"$types\" &&\n> > + sort >actual > + test_cmp expect actual\n> > +}\n> > +\n> > +test_expect_success 'update meta-commits after rebase' '\n> > + (\n> > + set_fake_editor &&\n> > + FAKE_AMEND=edited &&\n> > + FAKE_LINES=\"reword 1 pick 2 fixup 3\" &&\n> > + export FAKE_AMEND FAKE_LINES &&\n> > + git rebase -i --root\n> > + ) &&\n> > +\n> > + # update meta-commits\n> > + git change update --replace tags/one --content HEAD~1 >out 2>err &&\n> > + echo \"Updated change metas/one\" >expect &&\n> > + test_cmp expect out &&\n> > + test_must_be_empty err &&\n> > + git change update --replace tags/two --content HEAD@{2} &&\n> > + oid=$(git rev-parse --verify metas/two) &&\n> > + git change update --replace HEAD@{2} --replace tags/three \\\n> > + --content HEAD &&\n> > +\n> > + # check meta-commits\n> > + check_meta_commit metas/one c HEAD~1 r tags/one &&\n> > + check_meta_commit $oid c HEAD@{2} r tags/two &&\n> > + # NB this checks that \"git change update\" uses the meta-commit ($oid)\n> > + # corresponding to the replaces commit (HEAD@2 above) given on the\n> > + # commandline.\n> > + check_meta_commit metas/two c HEAD r $oid r tags/three &&\n> > + check_meta_commit metas/three c HEAD r $oid r tags/three\n> > +'\n> > +\n> > +reset_meta_commits () {\n> > + for c in one two three\n> > + do\n> > + echo \"update refs/metas/$c refs/tags/$c^0\"\n> > + done | git update-ref --stdin\n> > +}\n> > +\n> > +test_expect_success 'override change name' '\n> > + # TODO: builtin/change.c expects --change to be the full refname,\n> > + # ideally it would prepend refs/metas to the string given by the\n> > + # user.\n> > + git change update --change refs/metas/another-one --content one &&\n> > + test_cmp_rev metas/another-one one\n> > +'\n> > +\n> > +test_expect_success 'non-fast forward meta-commit update refused' '\n> > + test_must_fail git change update --change refs/metas/one --content two \\\n> > + >out 2>err &&\n> > + echo \"error: non-fast-forward update to ${SQ}refs/metas/one${SQ}\" \\\n> > + >expect &&\n> > + test_cmp expect err &&\n> > + test_must_be_empty out\n> > +'\n> > +\n> > +test_expect_success 'forced non-fast forward update succeeds' '\n> > + git change update --change refs/metas/one --content two --force \\\n> > + >out 2>err &&\n> > + echo \"Updated change metas/one\" >expect &&\n> > + test_cmp expect out &&\n> > + test_must_be_empty err\n> > +'\n> > +\n> > +test_done\n> > --\n> > 2.37.3.947.g1b8ba4da7f.dirty\n> >\n> >\n> > From 7784f253fa799dd11fcbc81fe815fb387af52d97 Mon Sep 17 00:00:00 2001\n> > From: Phillip Wood \n> > Date: Tue, 4 Oct 2022 15:16:05 +0100\n> > Subject: [PATCH 3/3] fixup! evolve: add the git change list command\n> >\n> > ---\n> > builtin/change.c | 16 +++++-----------\n> > t/t9999-changes.sh | 11 +++++++++++\n> > 2 files changed, 16 insertions(+), 11 deletions(-)\n> >\n> > diff --git a/builtin/change.c b/builtin/change.c\n> > index 07d029d82d..888ef648fa 100644\n> > --- a/builtin/change.c\n> > +++ b/builtin/change.c\n> > @@ -34,9 +34,8 @@ static int change_list(int argc, const char **argv, const char* prefix)\n> > OPT_END()\n> > };\n> > struct ref_filter filter;\n> > - /* TODO: See below\n> > struct ref_sorting *sorting;\n> > - struct string_list sorting_options = STRING_LIST_INIT_DUP; */\n> > + struct string_list sorting_options = STRING_LIST_INIT_DUP;\n> > struct ref_format format = REF_FORMAT_INIT;\n> > struct ref_array array;\n> > int i;\n> > @@ -53,19 +52,15 @@ static int change_list(int argc, const char **argv, const char* prefix)\n> >\n> > filter_refs(&array, &filter, FILTER_REFS_CHANGES);\n> >\n> > - /* TODO: This causes a crash. It sets one of the atom_value handlers to\n> > - * something invalid, which causes a crash later when we call\n> > - * show_ref_array_item. Figure out why this happens and put back the sorting.\n> > - *\n> > - * sorting = ref_sorting_options(&sorting_options);\n> > - * ref_array_sort(sorting, &array); */\n> > -\n> > if (!format.format)\n> > format.format = \"%(refname:lstrip=1)\";\n> >\n> > if (verify_ref_format(&format))\n> > die(_(\"unable to parse format string\"));\n> >\n> > + sorting = ref_sorting_options(&sorting_options);\n> > + ref_array_sort(sorting, &array);\n> > +\n> > for (i = 0; i < array.nr; i++) {\n> > struct strbuf output = STRBUF_INIT;\n> > struct strbuf err = STRBUF_INIT;\n> > @@ -79,8 +74,7 @@ static int change_list(int argc, const char **argv, const char* prefix)\n> > }\n> >\n> > ref_array_clear(&array);\n> > - /* TODO: see above\n> > - ref_sorting_release(sorting); */\n> > + ref_sorting_release(sorting);\n> >\n> > return 0;\n> > }\n> > diff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\n> > index 9e58925b23..9312eba86d 100755\n> > --- a/t/t9999-changes.sh\n> > +++ b/t/t9999-changes.sh\n> > @@ -123,4 +123,15 @@ test_expect_success 'forced non-fast forward update succeeds' '\n> > test_must_be_empty err\n> > '\n> >\n> > +test_expect_success 'list changes' '\n> > + cat >expect <<-\\EOF &&\n> > + metas/another-one\n> > + metas/one\n> > + metas/three\n> > + metas/two\n> > + EOF\n> > + git change list >actual &&\n> > + test_cmp expect actual\n> > +'\n> > +\n> > test_done\n> > --\n> > 2.37.3.947.g1b8ba4da7f.dirty\n"},{"id":"464178","messageId":"f65d596b-7479-1551-0763-00c5d211f4fe@dunelm.org.uk","threadId":"58504","inReplyTo":"CAN84kKnDvvG5V=eCnTPiNVC+CfWu9NFqwF+L4QuX26TAPov2Zg@mail.gmail.com","subject":"Re: [PATCH 00/10] Add the Git Change command","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-04T15:57:17Z","receivedAt":"2022-10-04T15:57:25Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 04/10/2022 16:19, Chris P wrote:\n> Thanks a lot.\n> \n> Is there something special I must do to get these scripts to work? The\n> entire script fails for me, despite having build git-change.\n\nIf you do\n\nmake\ncd t\n./t9999-changes.sh -v -i [--root=/dev/shm]\n\nit should run. You need to change into the t directory before running \nour tests and if you've just run \"make git\" before then it wont have \ncreated the scripts in bin-wrappers that the tests use. Are you able to \nrun any of the other tests successfully?\n\nPhillip\n\n> On Tue, Oct 4, 2022 at 4:24 PM Phillip Wood wrote:\n>>\n>> Hi Chris\n>>\n>> On 23/09/2022 19:55, Christophe Poucet via GitGitGadget wrote:\n>>> I'm reviving the original git evolve work that was started by\n>>> sxenos@google.com\n>>> (https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n>>>\n>>> This work is intended to make it easier to deal with stacked changes.\n>>>\n>>> The following set of patches introduces the design doc on the evolve command\n>>> as well as the basics of the git change command.\n>>\n>> Our test suite can be a little tricky to get started with and I was impatient to\n>> check the basic functionality of these patches so I've written some simple\n>> example tests for the change command and a couple of fixups to make them pass.\n>>\n>> Best Wishes\n>>\n>> Phillip\n>>\n>> ---- >8 ----\n>>\n>> From a7c38d0f388e4d8a1f3debcc3069a7fb43084eda Mon Sep 17 00:00:00 2001\n>> From: Phillip Wood \n>> Date: Tue, 4 Oct 2022 15:12:36 +0100\n>> Subject: [PATCH 1/3] fixup! evolve: add support for writing metacommits\n>>\n>> ---\n>> metacommit.c | 2 +-\n>> 1 file changed, 1 insertion(+), 1 deletion(-)\n>>\n>> diff --git a/metacommit.c b/metacommit.c\n>> index d2b859a4d3..8f970fa104 100644\n>> --- a/metacommit.c\n>> +++ b/metacommit.c\n>> @@ -296,7 +296,7 @@ int record_metacommit_withresult(\n>> if (override_change) {\n>> string_list_clear(changes, 0);\n>> overridden_head = get_change_head(chtable, override_change);\n>> - if (!overridden_head) {\n>> + if (overridden_head) {\n>> /* This is an existing change */\n>> old_head = &overridden_head->head;\n>> if (!force) {\n>> --\n>> 2.37.3.947.g1b8ba4da7f.dirty\n>>\n>>\n>> From cc7e8ba0b1a90268ced85d3f0c91aed49f2246d6 Mon Sep 17 00:00:00 2001\n>> From: Phillip Wood \n>> Date: Tue, 4 Oct 2022 15:15:32 +0100\n>> Subject: [PATCH 2/3] fixup! evolve: implement the git change command\n>>\n>> ---\n>> t/t9999-changes.sh | 126 +++++++++++++++++++++++++++++++++++++++++++++\n>> 1 file changed, 126 insertions(+)\n>> create mode 100755 t/t9999-changes.sh\n>>\n>> diff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\n>> new file mode 100755\n>> index 0000000000..9e58925b23\n>> --- /dev/null\n>> +++ b/t/t9999-changes.sh\n>> @@ -0,0 +1,126 @@\n>> +#!/bin/sh\n>> +\n>> +test_description='git change - low level meta-commit management'\n>> +\n>> +. ./test-lib.sh\n>> +\n>> +. \"$TEST_DIRECTORY\"/lib-rebase.sh\n>> +\n>> +test_expect_success 'setup commits and meta-commits' '\n>> + for c in one two three\n>> + do\n>> + test_commit $c &&\n>> + git change update --content $c >actual 2>err &&\n>> + echo \"Created change metas/$c\" >expect &&\n>> + test_cmp expect actual &&\n>> + test_must_be_empty err &&\n>> + test_cmp_rev refs/metas/$c $c || return 1\n>> + done\n>> +'\n>> +\n>> +# Check a meta-commit has the correct parents Call with the object\n>> +# name of the meta-commit followed by pairs of type and parent\n>> +check_meta_commit () {\n>> + name=$1\n>> + shift\n>> + while test $# -gt 0\n>> + do\n>> + printf '%s %s\\n' $1 $(git rev-parse --verify $2)\n>> + shift\n>> + shift\n>> + done | sort >expect\n>> + git cat-file commit $name >metacommit &&\n>> + # commit body should consist of parent-type\n>> + types=\"$(sed -n '/^$/ {\n>> + :loop\n>> + n\n>> + s/^parent-type //\n>> + p\n>> + b loop\n>> + }' metacommit)\" &&\n>> + while read key value\n>> + do\n>> + # TODO: don't sort the first parent\n>> + if test \"$key\" = \"parent\"\n>> + then\n>> + type=\"${types%% *}\"\n>> + test -n \"$type\" || return 1\n>> + printf '%s %s\\n' $type $value\n>> + types=\"${types#?}\"\n>> + types=\"${types# }\"\n>> + elif test \"$key\" = \"tree\"\n>> + then\n>> + test_cmp_rev \"$value\" $EMPTY_TREE || return 1\n>> + elif test -z \"$key\"\n>> + then\n>> + # only parse commit headers\n>> + break\n>> + fi\n>> + done actual-unsorted &&\n>> + test -z \"$types\" &&\n>> + sort >actual > + test_cmp expect actual\n>> +}\n>> +\n>> +test_expect_success 'update meta-commits after rebase' '\n>> + (\n>> + set_fake_editor &&\n>> + FAKE_AMEND=edited &&\n>> + FAKE_LINES=\"reword 1 pick 2 fixup 3\" &&\n>> + export FAKE_AMEND FAKE_LINES &&\n>> + git rebase -i --root\n>> + ) &&\n>> +\n>> + # update meta-commits\n>> + git change update --replace tags/one --content HEAD~1 >out 2>err &&\n>> + echo \"Updated change metas/one\" >expect &&\n>> + test_cmp expect out &&\n>> + test_must_be_empty err &&\n>> + git change update --replace tags/two --content HEAD@{2} &&\n>> + oid=$(git rev-parse --verify metas/two) &&\n>> + git change update --replace HEAD@{2} --replace tags/three \\\n>> + --content HEAD &&\n>> +\n>> + # check meta-commits\n>> + check_meta_commit metas/one c HEAD~1 r tags/one &&\n>> + check_meta_commit $oid c HEAD@{2} r tags/two &&\n>> + # NB this checks that \"git change update\" uses the meta-commit ($oid)\n>> + # corresponding to the replaces commit (HEAD@2 above) given on the\n>> + # commandline.\n>> + check_meta_commit metas/two c HEAD r $oid r tags/three &&\n>> + check_meta_commit metas/three c HEAD r $oid r tags/three\n>> +'\n>> +\n>> +reset_meta_commits () {\n>> + for c in one two three\n>> + do\n>> + echo \"update refs/metas/$c refs/tags/$c^0\"\n>> + done | git update-ref --stdin\n>> +}\n>> +\n>> +test_expect_success 'override change name' '\n>> + # TODO: builtin/change.c expects --change to be the full refname,\n>> + # ideally it would prepend refs/metas to the string given by the\n>> + # user.\n>> + git change update --change refs/metas/another-one --content one &&\n>> + test_cmp_rev metas/another-one one\n>> +'\n>> +\n>> +test_expect_success 'non-fast forward meta-commit update refused' '\n>> + test_must_fail git change update --change refs/metas/one --content two \\\n>> + >out 2>err &&\n>> + echo \"error: non-fast-forward update to ${SQ}refs/metas/one${SQ}\" \\\n>> + >expect &&\n>> + test_cmp expect err &&\n>> + test_must_be_empty out\n>> +'\n>> +\n>> +test_expect_success 'forced non-fast forward update succeeds' '\n>> + git change update --change refs/metas/one --content two --force \\\n>> + >out 2>err &&\n>> + echo \"Updated change metas/one\" >expect &&\n>> + test_cmp expect out &&\n>> + test_must_be_empty err\n>> +'\n>> +\n>> +test_done\n>> --\n>> 2.37.3.947.g1b8ba4da7f.dirty\n>>\n>>\n>> From 7784f253fa799dd11fcbc81fe815fb387af52d97 Mon Sep 17 00:00:00 2001\n>> From: Phillip Wood \n>> Date: Tue, 4 Oct 2022 15:16:05 +0100\n>> Subject: [PATCH 3/3] fixup! evolve: add the git change list command\n>>\n>> ---\n>> builtin/change.c | 16 +++++-----------\n>> t/t9999-changes.sh | 11 +++++++++++\n>> 2 files changed, 16 insertions(+), 11 deletions(-)\n>>\n>> diff --git a/builtin/change.c b/builtin/change.c\n>> index 07d029d82d..888ef648fa 100644\n>> --- a/builtin/change.c\n>> +++ b/builtin/change.c\n>> @@ -34,9 +34,8 @@ static int change_list(int argc, const char **argv, const char* prefix)\n>> OPT_END()\n>> };\n>> struct ref_filter filter;\n>> - /* TODO: See below\n>> struct ref_sorting *sorting;\n>> - struct string_list sorting_options = STRING_LIST_INIT_DUP; */\n>> + struct string_list sorting_options = STRING_LIST_INIT_DUP;\n>> struct ref_format format = REF_FORMAT_INIT;\n>> struct ref_array array;\n>> int i;\n>> @@ -53,19 +52,15 @@ static int change_list(int argc, const char **argv, const char* prefix)\n>>\n>> filter_refs(&array, &filter, FILTER_REFS_CHANGES);\n>>\n>> - /* TODO: This causes a crash. It sets one of the atom_value handlers to\n>> - * something invalid, which causes a crash later when we call\n>> - * show_ref_array_item. Figure out why this happens and put back the sorting.\n>> - *\n>> - * sorting = ref_sorting_options(&sorting_options);\n>> - * ref_array_sort(sorting, &array); */\n>> -\n>> if (!format.format)\n>> format.format = \"%(refname:lstrip=1)\";\n>>\n>> if (verify_ref_format(&format))\n>> die(_(\"unable to parse format string\"));\n>>\n>> + sorting = ref_sorting_options(&sorting_options);\n>> + ref_array_sort(sorting, &array);\n>> +\n>> for (i = 0; i < array.nr; i++) {\n>> struct strbuf output = STRBUF_INIT;\n>> struct strbuf err = STRBUF_INIT;\n>> @@ -79,8 +74,7 @@ static int change_list(int argc, const char **argv, const char* prefix)\n>> }\n>>\n>> ref_array_clear(&array);\n>> - /* TODO: see above\n>> - ref_sorting_release(sorting); */\n>> + ref_sorting_release(sorting);\n>>\n>> return 0;\n>> }\n>> diff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\n>> index 9e58925b23..9312eba86d 100755\n>> --- a/t/t9999-changes.sh\n>> +++ b/t/t9999-changes.sh\n>> @@ -123,4 +123,15 @@ test_expect_success 'forced non-fast forward update succeeds' '\n>> test_must_be_empty err\n>> '\n>>\n>> +test_expect_success 'list changes' '\n>> + cat >expect <<-\\EOF &&\n>> + metas/another-one\n>> + metas/one\n>> + metas/three\n>> + metas/two\n>> + EOF\n>> + git change list >actual &&\n>> + test_cmp expect actual\n>> +'\n>> +\n>> test_done\n>> --\n>> 2.37.3.947.g1b8ba4da7f.dirty\n"},{"id":"464179","messageId":"4b09dadf-7b56-fecd-02c0-08fec8b9bd12@dunelm.org.uk","threadId":"58504","inReplyTo":"CAN84kKmx8fMppFdFG98VyKfK_vpb5nbXLXGUbJWupyVTNqak3A@mail.gmail.com","subject":"Re: [PATCH 00/10] Add the Git Change command","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-04T16:00:25Z","receivedAt":"2022-10-04T16:00:34Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 04/10/2022 16:55, Chris P wrote:\n> I got the tests to work, except the last one, that one fails.\n> \n> How can I see the output of actual and expect?\n\nIf you run the test with -i then it will stop at the first failure and \nyou can inspect the files in the directory 'trash \ndirectory.t9999-changes'. Running with -v (verbose) -x (shell tracing) \nis also useful when debugging.\n\nPhillip\n\n> On Tue, Oct 4, 2022 at 5:19 PM Chris P wrote:\n>>\n>> Thanks a lot.\n>>\n>> Is there something special I must do to get these scripts to work? The\n>> entire script fails for me, despite having build git-change.\n>>\n>> On Tue, Oct 4, 2022 at 4:24 PM Phillip Wood wrote:\n>>>\n>>> Hi Chris\n>>>\n>>> On 23/09/2022 19:55, Christophe Poucet via GitGitGadget wrote:\n>>>> I'm reviving the original git evolve work that was started by\n>>>> sxenos@google.com\n>>>> (https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n>>>>\n>>>> This work is intended to make it easier to deal with stacked changes.\n>>>>\n>>>> The following set of patches introduces the design doc on the evolve command\n>>>> as well as the basics of the git change command.\n>>>\n>>> Our test suite can be a little tricky to get started with and I was impatient to\n>>> check the basic functionality of these patches so I've written some simple\n>>> example tests for the change command and a couple of fixups to make them pass.\n>>>\n>>> Best Wishes\n>>>\n>>> Phillip\n>>>\n>>> ---- >8 ----\n>>>\n>>> From a7c38d0f388e4d8a1f3debcc3069a7fb43084eda Mon Sep 17 00:00:00 2001\n>>> From: Phillip Wood \n>>> Date: Tue, 4 Oct 2022 15:12:36 +0100\n>>> Subject: [PATCH 1/3] fixup! evolve: add support for writing metacommits\n>>>\n>>> ---\n>>> metacommit.c | 2 +-\n>>> 1 file changed, 1 insertion(+), 1 deletion(-)\n>>>\n>>> diff --git a/metacommit.c b/metacommit.c\n>>> index d2b859a4d3..8f970fa104 100644\n>>> --- a/metacommit.c\n>>> +++ b/metacommit.c\n>>> @@ -296,7 +296,7 @@ int record_metacommit_withresult(\n>>> if (override_change) {\n>>> string_list_clear(changes, 0);\n>>> overridden_head = get_change_head(chtable, override_change);\n>>> - if (!overridden_head) {\n>>> + if (overridden_head) {\n>>> /* This is an existing change */\n>>> old_head = &overridden_head->head;\n>>> if (!force) {\n>>> --\n>>> 2.37.3.947.g1b8ba4da7f.dirty\n>>>\n>>>\n>>> From cc7e8ba0b1a90268ced85d3f0c91aed49f2246d6 Mon Sep 17 00:00:00 2001\n>>> From: Phillip Wood \n>>> Date: Tue, 4 Oct 2022 15:15:32 +0100\n>>> Subject: [PATCH 2/3] fixup! evolve: implement the git change command\n>>>\n>>> ---\n>>> t/t9999-changes.sh | 126 +++++++++++++++++++++++++++++++++++++++++++++\n>>> 1 file changed, 126 insertions(+)\n>>> create mode 100755 t/t9999-changes.sh\n>>>\n>>> diff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\n>>> new file mode 100755\n>>> index 0000000000..9e58925b23\n>>> --- /dev/null\n>>> +++ b/t/t9999-changes.sh\n>>> @@ -0,0 +1,126 @@\n>>> +#!/bin/sh\n>>> +\n>>> +test_description='git change - low level meta-commit management'\n>>> +\n>>> +. ./test-lib.sh\n>>> +\n>>> +. \"$TEST_DIRECTORY\"/lib-rebase.sh\n>>> +\n>>> +test_expect_success 'setup commits and meta-commits' '\n>>> + for c in one two three\n>>> + do\n>>> + test_commit $c &&\n>>> + git change update --content $c >actual 2>err &&\n>>> + echo \"Created change metas/$c\" >expect &&\n>>> + test_cmp expect actual &&\n>>> + test_must_be_empty err &&\n>>> + test_cmp_rev refs/metas/$c $c || return 1\n>>> + done\n>>> +'\n>>> +\n>>> +# Check a meta-commit has the correct parents Call with the object\n>>> +# name of the meta-commit followed by pairs of type and parent\n>>> +check_meta_commit () {\n>>> + name=$1\n>>> + shift\n>>> + while test $# -gt 0\n>>> + do\n>>> + printf '%s %s\\n' $1 $(git rev-parse --verify $2)\n>>> + shift\n>>> + shift\n>>> + done | sort >expect\n>>> + git cat-file commit $name >metacommit &&\n>>> + # commit body should consist of parent-type\n>>> + types=\"$(sed -n '/^$/ {\n>>> + :loop\n>>> + n\n>>> + s/^parent-type //\n>>> + p\n>>> + b loop\n>>> + }' metacommit)\" &&\n>>> + while read key value\n>>> + do\n>>> + # TODO: don't sort the first parent\n>>> + if test \"$key\" = \"parent\"\n>>> + then\n>>> + type=\"${types%% *}\"\n>>> + test -n \"$type\" || return 1\n>>> + printf '%s %s\\n' $type $value\n>>> + types=\"${types#?}\"\n>>> + types=\"${types# }\"\n>>> + elif test \"$key\" = \"tree\"\n>>> + then\n>>> + test_cmp_rev \"$value\" $EMPTY_TREE || return 1\n>>> + elif test -z \"$key\"\n>>> + then\n>>> + # only parse commit headers\n>>> + break\n>>> + fi\n>>> + done actual-unsorted &&\n>>> + test -z \"$types\" &&\n>>> + sort >actual >> + test_cmp expect actual\n>>> +}\n>>> +\n>>> +test_expect_success 'update meta-commits after rebase' '\n>>> + (\n>>> + set_fake_editor &&\n>>> + FAKE_AMEND=edited &&\n>>> + FAKE_LINES=\"reword 1 pick 2 fixup 3\" &&\n>>> + export FAKE_AMEND FAKE_LINES &&\n>>> + git rebase -i --root\n>>> + ) &&\n>>> +\n>>> + # update meta-commits\n>>> + git change update --replace tags/one --content HEAD~1 >out 2>err &&\n>>> + echo \"Updated change metas/one\" >expect &&\n>>> + test_cmp expect out &&\n>>> + test_must_be_empty err &&\n>>> + git change update --replace tags/two --content HEAD@{2} &&\n>>> + oid=$(git rev-parse --verify metas/two) &&\n>>> + git change update --replace HEAD@{2} --replace tags/three \\\n>>> + --content HEAD &&\n>>> +\n>>> + # check meta-commits\n>>> + check_meta_commit metas/one c HEAD~1 r tags/one &&\n>>> + check_meta_commit $oid c HEAD@{2} r tags/two &&\n>>> + # NB this checks that \"git change update\" uses the meta-commit ($oid)\n>>> + # corresponding to the replaces commit (HEAD@2 above) given on the\n>>> + # commandline.\n>>> + check_meta_commit metas/two c HEAD r $oid r tags/three &&\n>>> + check_meta_commit metas/three c HEAD r $oid r tags/three\n>>> +'\n>>> +\n>>> +reset_meta_commits () {\n>>> + for c in one two three\n>>> + do\n>>> + echo \"update refs/metas/$c refs/tags/$c^0\"\n>>> + done | git update-ref --stdin\n>>> +}\n>>> +\n>>> +test_expect_success 'override change name' '\n>>> + # TODO: builtin/change.c expects --change to be the full refname,\n>>> + # ideally it would prepend refs/metas to the string given by the\n>>> + # user.\n>>> + git change update --change refs/metas/another-one --content one &&\n>>> + test_cmp_rev metas/another-one one\n>>> +'\n>>> +\n>>> +test_expect_success 'non-fast forward meta-commit update refused' '\n>>> + test_must_fail git change update --change refs/metas/one --content two \\\n>>> + >out 2>err &&\n>>> + echo \"error: non-fast-forward update to ${SQ}refs/metas/one${SQ}\" \\\n>>> + >expect &&\n>>> + test_cmp expect err &&\n>>> + test_must_be_empty out\n>>> +'\n>>> +\n>>> +test_expect_success 'forced non-fast forward update succeeds' '\n>>> + git change update --change refs/metas/one --content two --force \\\n>>> + >out 2>err &&\n>>> + echo \"Updated change metas/one\" >expect &&\n>>> + test_cmp expect out &&\n>>> + test_must_be_empty err\n>>> +'\n>>> +\n>>> +test_done\n>>> --\n>>> 2.37.3.947.g1b8ba4da7f.dirty\n>>>\n>>>\n>>> From 7784f253fa799dd11fcbc81fe815fb387af52d97 Mon Sep 17 00:00:00 2001\n>>> From: Phillip Wood \n>>> Date: Tue, 4 Oct 2022 15:16:05 +0100\n>>> Subject: [PATCH 3/3] fixup! evolve: add the git change list command\n>>>\n>>> ---\n>>> builtin/change.c | 16 +++++-----------\n>>> t/t9999-changes.sh | 11 +++++++++++\n>>> 2 files changed, 16 insertions(+), 11 deletions(-)\n>>>\n>>> diff --git a/builtin/change.c b/builtin/change.c\n>>> index 07d029d82d..888ef648fa 100644\n>>> --- a/builtin/change.c\n>>> +++ b/builtin/change.c\n>>> @@ -34,9 +34,8 @@ static int change_list(int argc, const char **argv, const char* prefix)\n>>> OPT_END()\n>>> };\n>>> struct ref_filter filter;\n>>> - /* TODO: See below\n>>> struct ref_sorting *sorting;\n>>> - struct string_list sorting_options = STRING_LIST_INIT_DUP; */\n>>> + struct string_list sorting_options = STRING_LIST_INIT_DUP;\n>>> struct ref_format format = REF_FORMAT_INIT;\n>>> struct ref_array array;\n>>> int i;\n>>> @@ -53,19 +52,15 @@ static int change_list(int argc, const char **argv, const char* prefix)\n>>>\n>>> filter_refs(&array, &filter, FILTER_REFS_CHANGES);\n>>>\n>>> - /* TODO: This causes a crash. It sets one of the atom_value handlers to\n>>> - * something invalid, which causes a crash later when we call\n>>> - * show_ref_array_item. Figure out why this happens and put back the sorting.\n>>> - *\n>>> - * sorting = ref_sorting_options(&sorting_options);\n>>> - * ref_array_sort(sorting, &array); */\n>>> -\n>>> if (!format.format)\n>>> format.format = \"%(refname:lstrip=1)\";\n>>>\n>>> if (verify_ref_format(&format))\n>>> die(_(\"unable to parse format string\"));\n>>>\n>>> + sorting = ref_sorting_options(&sorting_options);\n>>> + ref_array_sort(sorting, &array);\n>>> +\n>>> for (i = 0; i < array.nr; i++) {\n>>> struct strbuf output = STRBUF_INIT;\n>>> struct strbuf err = STRBUF_INIT;\n>>> @@ -79,8 +74,7 @@ static int change_list(int argc, const char **argv, const char* prefix)\n>>> }\n>>>\n>>> ref_array_clear(&array);\n>>> - /* TODO: see above\n>>> - ref_sorting_release(sorting); */\n>>> + ref_sorting_release(sorting);\n>>>\n>>> return 0;\n>>> }\n>>> diff --git a/t/t9999-changes.sh b/t/t9999-changes.sh\n>>> index 9e58925b23..9312eba86d 100755\n>>> --- a/t/t9999-changes.sh\n>>> +++ b/t/t9999-changes.sh\n>>> @@ -123,4 +123,15 @@ test_expect_success 'forced non-fast forward update succeeds' '\n>>> test_must_be_empty err\n>>> '\n>>>\n>>> +test_expect_success 'list changes' '\n>>> + cat >expect <<-\\EOF &&\n>>> + metas/another-one\n>>> + metas/one\n>>> + metas/three\n>>> + metas/two\n>>> + EOF\n>>> + git change list >actual &&\n>>> + test_cmp expect actual\n>>> +'\n>>> +\n>>> test_done\n>>> --\n>>> 2.37.3.947.g1b8ba4da7f.dirty\n"},{"id":"464220","messageId":"CAN84kKke+vSQ18wNM7h6BvTF9XBtSH96L0_qNOFn+V3fj2yNhg@mail.gmail.com","threadId":"58504","inReplyTo":"a7ddab8a-ddd6-a8bf-496d-4ce7757d89cf@dunelm.org.uk","subject":"Re: [PATCH 06/10] evolve: add support for writing metacommits","fromName":"Chris P","fromEmail":"christophe.poucet@gmail.com","sentAt":"2022-10-05T09:40:57Z","receivedAt":"2022-10-05T09:41:14Z","isPatch":true,"sender":{"key":"christophe.poucet@gmail.com","avatar":null},"body":"+cc my work account due to a bug in gitgitgadget\n\n>\n> I think the code to parse and create metacommits (as well as the change\n> table code) could quite happily live in the same file.\n\nI am considering this, it would hide the change_table as that doesn't\nneed to be exposed (for now at least). My only concern is that it\nwould make the commit-msg rather unwieldy.\n\n>\n> > diff --git a/metacommit.c b/metacommit.c\n> > new file mode 100644\n> > index 00000000000..d2b859a4d3b\n> > --- /dev/null\n> > +++ b/metacommit.c\n> > @@ -0,0 +1,404 @@\n> > +#include \"cache.h\"\n> > +#include \"metacommit.h\"\n> > +#include \"commit.h\"\n> > +#include \"change-table.h\"\n> > +#include \"refs.h\"\n> > +\n> > +void init_metacommit_data(struct metacommit_data *state)\n> > +{\n> > + memset(state, 0, sizeof(*state));\n> > +}\n> We'd normally use an initializer macro instead\n>\n> #define METACOMMIT_DATA_INIT = { 0 }\n\nThanks, done.\n\n>\n> > +void clear_metacommit_data(struct metacommit_data *state)\n> > +{\n> > + oid_array_clear(&state->replace);\n> > + oid_array_clear(&state->origin);\n> > +}\n> > +\n> > +static void compute_default_change_name(struct commit *initial_commit,\n> > + struct strbuf* result)\n> > +{\n> > + struct strbuf default_name;\n>\n> The canonical way to initialize an strbuf that is not on the heap is\n>\n> struct strbuf buf = STRBUF_INIT;\n\nDone.\n\n>\n> > + const char *buffer;\n> > + const char *subject;\n> > + const char *eol;\n> > + int len;\n> > + strbuf_init(&default_name, 0);\n> > + buffer = get_commit_buffer(initial_commit, NULL);\n> > + find_commit_subject(buffer, &subject);\n> > + eol = strchrnul(subject, '\\n');\n> > + for (len = 0;subject < eol && len < 10; ++subject, ++len) {\n>\n> There's a space missing after the first ';'. We prefer post-increments\n> to pre-increments unless the pre-increment is significant.\n\nDone and done :)\n\n>\n> > + char next = *subject;\n> > + if (isspace(next))\n> > + continue;\n> > +\n> > + strbuf_addch(&default_name, next);\n> > + }\n> > + sanitize_refname_component(default_name.buf, result);\n>\n> I suspect we need to call unuse_commit_buffer(initial_commit) here.\n\nOh interesting, yes.\n\n>\n> > +}\n> > +\n> > +/**\n> > + * Computes a change name for a change rooted at the given initial commit. Good\n> > + * change names should be memorable, unique, and easy to type. They are not\n> > + * required to match the commit comment.\n> > + */\n> > +static void compute_change_name(struct commit *initial_commit, struct strbuf* result)\n> > +{\n> > + struct strbuf default_name;\n> > + struct object_id unused;\n> > +\n> > + strbuf_init(&default_name, 0);\n> > + if (initial_commit)\n> > + compute_default_change_name(initial_commit, &default_name);\n> > + else\n> > + strbuf_addstr(&default_name, \"change\");\n>\n> What does it mean to call this function with initial_commit == NULL?\n\nI don't know to be honest, the call site always seems to pass one in.\nChanged it to BUG.\n\n>\n> > + strbuf_addstr(result, \"refs/metas/\");\n> > + strbuf_addbuf(result, &default_name);\n> > + /* If there is already a change of this name, append a suffix */\n> > + if (!read_ref(result->buf, &unused)) {\n> > + int suffix = 2;\n> > + int original_length = result->len;\n>\n> This is one of many places where we have a size_t len or nr member and\n> assign it to an int. I think it would be clearer to use a size_t instead\n> to avoid adding any more signed<->unsigned conversions.\n\nDone, I had to leave one place because it did while(i >= 0);\n\n>\n> > +\n> > + while (1) {\n> > + strbuf_addf(result, \"%d\", suffix);\n> > + if (read_ref(result->buf, &unused))\n> > + break;\n> > + strbuf_remove(result, original_length, result->len - original_length);\n> > + ++suffix;\n> > + }\n> > + }\n> > +\n> > + strbuf_release(&default_name);\n> > +}\n> > +\n> > +struct resolve_metacommit_callback_data\n>\n> While there are some structs with a _callback_data suffix in the code\n> base, it is far more common to use _context and name any corresponding\n> variables ctx.\n\nDone.\n\n>\n> > +{\n> > + struct change_table* active_changes;\n> > + struct string_list *changes;\n> > + struct oid_array *heads;\n> > +};\n> > +\n> > +static int resolve_metacommit_callback(const char *refname, void *cb_data)\n> > +{\n> > + struct resolve_metacommit_callback_data *data = (struct resolve_metacommit_callback_data *)cb_data;\n>\n> We don't use redundant casts such as this.\n\nThanks :)\n\n>\n> > + struct change_head *chhead;\n> > +\n> > + chhead = get_change_head(data->active_changes, refname);\n>\n> This is really a comment on the previous patch but are there uses of\n> for_each_change_referencing() for which just the refname is sufficient?\n> It might be more convenient to pass the change head into the callback as\n> well.\n\nI'm not sure, will investigate post-squash\n\n>\n> > +\n> > + if (data->changes)\n> > + string_list_append(data->changes, refname)->util = &(chhead->head);\n>\n> We don't use redundant parentheses such as this (and this patch does not\n> use them consistently)\n\nDone.\n\n>\n> > + if (data->heads)\n> > + oid_array_append(data->heads, &(chhead->head));\n> > +\n> > + return 0;\n> > +}\n> > +\n> > +/**\n> > + * Produces the final form of a metacommit based on the current change refs.\n> > + */\n> > +static void resolve_metacommit(\n> > + struct repository* repo,\n> > + struct change_table* active_changes,\n> > + const struct metacommit_data *to_resolve,\n>\n> [testing my understanding] This is the metacommit we want to update\n\nMaybe you can help me find a bug. If you run `git-change update`\ntwice without changing commits, it prints that it created a second\none, but then if you `git-change list` it doesn't show that last one\nbecause it doesn't create an extra one if there's already a change\npointing at HEAD.\n\nAlso, thanks for all the comments, it's helping my understanding too.\nIn general do you want all these comments added to the code?\n\n>\n> > + struct metacommit_data *resolved_output,\n>\n> This is the updated metacommit returned to the user\n>\n> > + struct string_list *to_advance,\n>\n> Is also an output? It ends up as a list of refname to change head mappings\n\nYes, this is consumed in the change.c command. I was considering\nmaking this a strintmap that maps to an enum, because we need a\ntristate (updated, created, untouched). But unfortunately the oids\nassigned to `->util` are consumed later in the function.\n\n>\n> > + int allow_append)\n> > +{\n> > + int i;\n> > + int len = to_resolve->replace.nr;\n> > + struct resolve_metacommit_callback_data cbdata;\n>\n> This would be a good place to a designated initializer.\n>\n> struct resolve_metacommit_context ctx = {\n> .active_changes = active_changes,\n> .changes = to_advance,\n> .heads = &resolved_output->replace\n> };\n\nI'm learning new C :D\n\n>\n> > + int old_change_list_length = to_advance->nr;\n> > + struct commit* content;\n> > +\n> > + oidcpy(&resolved_output->content, &to_resolve->content);\n> > +\n> > + /* First look for changes that point to any of the replacement edges in the\n> > + * metacommit. These will be the changes that get advanced by this\n> > + * metacommit. */\n>\n> Style: '/*' & '*/' should be on their own lines.\n\nDone\n\n>\n> > + resolved_output->abandoned = to_resolve->abandoned;\n> > + cbdata.active_changes = active_changes;\n> > + cbdata.changes = to_advance;\n> > + cbdata.heads = &(resolved_output->replace);\n> > +\n> > + if (allow_append) {\n> > + for (i = 0; i < len; i++) {\n> > + int old_number = resolved_output->replace.nr;\n> > + for_each_change_referencing(active_changes, &(to_resolve->replace.oid[i]),\n> > + resolve_metacommit_callback, &cbdata);\n> > + /* If no changes were found, use the unresolved value. */\n> > + if (old_number == resolved_output->replace.nr)\n> > + oid_array_append(&(resolved_output->replace), &(to_resolve->replace.oid[i]));\n>\n> We see if there are any refs under refs/metas/ which point to\n> 'to_resolve' or its content and if there are we add those refs and the\n> corresponding change head to 'to_advance'. If we don't find any refs\n> then we copy the replace oid from 'to_resolve' to 'resolved_output'\n>\n> If allow_append is false then we ignore all the replace oids in 'to_resolve'\n>\n> > + }\n> > + }\n> > +\n> > + cbdata.changes = NULL;\n> > + cbdata.heads = &(resolved_output->origin);\n> > +\n> > + len = to_resolve->origin.nr;\n> > + for (i = 0; i < len; i++) {\n> > + int old_number = resolved_output->origin.nr;\n> > + for_each_change_referencing(active_changes, &(to_resolve->origin.oid[i]),\n> > + resolve_metacommit_callback, &cbdata);\n> > + if (old_number == resolved_output->origin.nr)\n> > + oid_array_append(&(resolved_output->origin), &(to_resolve->origin.oid[i]));\n> > + }\n>\n> This is copying the origin oids in the same way as we copied the replace\n> oids above.\n>\n> > + /* If no changes were advanced by this metacommit, we'll need to create a new\n> > + * one. */\n> > + if (to_advance->nr == old_change_list_length) {\n> > + struct strbuf change_name;\n> > +\n> > + strbuf_init(&change_name, 80);\n> > + content = lookup_commit_reference_gently(repo, &(to_resolve->content), 1);\n> > +\n> > + compute_change_name(content, &change_name);\n> > + string_list_append(to_advance, change_name.buf);\n> > + strbuf_release(&change_name);\n> > + }\n> > +}\n> > +\n> > +static void lookup_commits(\n> > + struct repository *repo,\n> > + struct oid_array *to_lookup,\n> > + struct commit_list **result)\n> > +{\n> > + int i = to_lookup->nr;\n> > +\n> > + while (--i >= 0) {\n> > + struct object_id *next = &(to_lookup->oid[i]);\n> > + struct commit *commit = lookup_commit_reference_gently(repo, next, 1);\n> > + commit_list_insert(commit, result);\n> > + }\n>\n> We walk backwards because commit_list_insert prepends to the list - good.\n>\n> > +}\n> > +\n> > +#define PARENT_TYPE_PREFIX \"parent-type \"\n> > +\n> > +/**\n> > + * Creates a new metacommit object with the given content. Writes the object\n> > + * id of the newly-created commit to result.\n> > + */\n> > +int write_metacommit(struct repository *repo, struct metacommit_data *state,\n> > + struct object_id *result)\n> > +{\n> > + struct commit_list *parents = NULL;\n> > + struct strbuf comment;\n> > + int i;\n> > + struct commit *content;\n> > +\n> > + strbuf_init(&comment, strlen(PARENT_TYPE_PREFIX)\n> > + + 1 + 2 * (state->origin.nr + state->replace.nr));\n> > + lookup_commits(repo, &state->origin, &parents);\n> > + lookup_commits(repo, &state->replace, &parents);\n> > + content = lookup_commit_reference_gently(repo, &state->content, 1);\n> > + if (!content) {\n> > + strbuf_release(&comment);\n> > + free_commit_list(parents);\n> > + return -1;\n> > + }\n> > + commit_list_insert(content, &parents);\n> > +\n> > + strbuf_addstr(&comment, PARENT_TYPE_PREFIX);\n> > + strbuf_addstr(&comment, state->abandoned ? \"a\" : \"c\");\n> > + for (i = 0; i < state->replace.nr; i++)\n> > + strbuf_addstr(&comment, \" r\");\n> > +\n> > + for (i = 0; i < state->origin.nr; i++)\n> > + strbuf_addstr(&comment, \" o\"); > + /* The parents list will be freed by this call. */\n> > + commit_tree(comment.buf, comment.len, repo->hash_algo->empty_tree, parents,\n> > + result, NULL, NULL);\n>\n> It would be relatively easy to use commit_tree_extended() with\n> extra_headers so that we create a commit with a \"parent-type\" header\n> rather than abusing the commit message.\n>\n> struct commit_extra_header extra = { .key = \"parent-type\" };\n>\n> /* build header value in strbuf */\n>\n> extra.value = buf.buf;\n> extra.len = buf.len;\n> commit_tree_extended(\"\", 0, repo->hash_algo->empty_tree,\n> parents, result, NULL, NULL, NULL,\n> &extra);\n\nSounds like a potentially good idea, to avoid that people accidentally\ncreate metacommits that aren't real.\nWhat would that look like on the parsing side as well as the test-setup?\n\n>\n> > +\n> > + strbuf_release(&comment);\n> > + return 0;\n> > +}\n> > +\n> > +/**\n> > + * Returns true iff the given metacommit is abandoned, has one or more origin\n> > + * parents, or has one or more replacement parents.\n> > + */\n> > +static int is_nontrivial_metacommit(struct metacommit_data *state)\n> > +{\n> > + return state->replace.nr || state->origin.nr || state->abandoned;\n> > +}\n> > +\n> > +/*\n> > + * Records the relationships described by the given metacommit in the\n> > + * repository.\n> > + *\n> > + * If override_change is NULL (the default), an attempt will be made\n> > + * to append to existing changes wherever possible instead of creating new ones.\n> > + * If override_change is non-null, only the given change ref will be updated.\n>\n> So override_head is the refname of an existing change?\n\nYes, this comes from the commandline with the option of '-g' (Which\nunfortunately is not documented).\n>\n> > + * options is a bitwise combination of the UPDATE_OPTION_* flags.\n> > + */\n> > +int record_metacommit(\n> > + struct repository *repo,\n> > + const struct metacommit_data *metacommit, const char *override_change,\n> > + int options, struct strbuf *err)\n> > +{\n> > + struct change_table chtable;\n> > + struct string_list changes;\n> > + int result;\n> > +\n> > + change_table_init(&chtable);\n> > + change_table_add_all_visible(&chtable, repo);\n> > + string_list_init_dup(&changes);\n> > +\n> > + result = record_metacommit_withresult(repo, &chtable, metacommit,\n> > + override_change, options, err, &changes);\n> > +\n> > + string_list_clear(&changes, 0);\n> > + change_table_clear(&chtable);\n> > + return result;\n> > +}\n> > +\n> > +/*\n> > + * Records the relationships described by the given metacommit in the\n> > + * repository.\n> > + *\n> > + * If override_change is NULL (the default), an attempt will be made\n> > + * to append to existing changes wherever possible instead of creating new ones.\n> > + * If override_change is non-null, only the given change ref will be updated.\n> > + *\n> > + * The changes list is filled in with the list of change refs that were updated,\n> > + * with the util pointers pointing to the old object IDS for those changes.\n> > + * The object ID pointers all point to objects owned by the change_table and\n> > + * will go out of scope when the change_table is destroyed.\n>\n> That potentially sounds like an invitation to create use after free bugs\n> unless we're careful. Does this function need to be public?\n\nSo the change command uses the changes list to determine what to print\nto the output. It looks whether it's a new change or a created change\nbased on whether the oid is null or not.\nWe could return a strintmap instead that points at status per change.\nI tried that approach but unfortunately, `changes` is used later in\nthis function. I could still consider a strintmap, it would just be\nthat there's some duplicative storage of information until the\nfunction exits. LMKWYT.\n\n>\n> > + *\n> > + * options is a bitwise combination of the UPDATE_OPTION_* flags.\n> > + */\n> > +int record_metacommit_withresult(\n> > + struct repository *repo,\n> > + struct change_table *chtable,\n> > + const struct metacommit_data *metacommit,\n> > + const char *override_change,\n> > + int options, struct strbuf *err,\n> > + struct string_list *changes)\n> > +{\n> > + static const char *msg = \"updating change\";\n> > + struct metacommit_data resolved_metacommit;\n> > + struct object_id commit_target;\n> > + struct ref_transaction *transaction = NULL;\n> > + struct change_head *overridden_head;\n> > + const struct object_id *old_head;\n> > +\n> > + int i;\n> > + int ret = 0;\n> > + int force = (options & UPDATE_OPTION_FORCE);\n> > +\n> > + init_metacommit_data(&resolved_metacommit);\n> > +\n> > + resolve_metacommit(repo, chtable, metacommit, &resolved_metacommit, changes,\n> > + (options & UPDATE_OPTION_NOAPPEND) == 0);\n> > +\n> > + if (override_change) {\n> > + string_list_clear(changes, 0);\n> > + overridden_head = get_change_head(chtable, override_change);\n> > + if (!overridden_head) {\n>\n> We enter this branch if overridden_head is NULL\n\nGood catch!\n\n>\n> > + /* This is an existing change */\n> > + old_head = &overridden_head->head;\n>\n> Here we de-reference overridden_head which is NULL\n\nYep.\n\n>\n> > + if (!force) {\n> > + if (!oid_array_readonly_contains(&(resolved_metacommit.replace),\n> > + &overridden_head->head)) {\n> > + /* Attempted non-fast-forward change */\n> > + strbuf_addf(err, _(\"non-fast-forward update to '%s'\"),\n> > + override_change);\n> > + ret = -1;\n> > + goto cleanup;\n> > + }\n> > + }\n> > + } else\n>\n> Style: if one branch of an if statement requires braces then all\n> branches should have braces.\n\nThanks, fixed.\n\n>\n> > + /* ...then this is a newly-created change */\n> > + old_head = null_oid();\n> > +\n> > + /* The expected \"current\" head of the change is stored in the util\n> > + * pointer. */\n> > + string_list_append(changes, override_change)->util = (void*)old_head;\n>\n> No need to cast here\n\nActually it's required because old_head is a const*\n>\n> > + }\n> > +\n> > + if (is_nontrivial_metacommit(&resolved_metacommit)) {\n> > + /* If there are any origin or replacement parents, create a new metacommit\n> > + * object. */\n> > + if (write_metacommit(repo, &resolved_metacommit, &commit_target) < 0) {\n> > + ret = -1;\n> > + goto cleanup;\n> > + }\n> > + } else\n> > + /**\n> > + * If the metacommit would only contain a content commit, point to the\n> > + * commit itself rather than creating a trivial metacommit.\n> > + */\n> > + oidcpy(&commit_target, &(resolved_metacommit.content));\n>\n> Oh, is this optimization why we don't insist on metacommits but also\n> allow ordinary commits to be added to the change table?\n\nYes.\n\n> > +\n> > +extern int record_metacommit_withresult(\n> > + struct repository *repo,\n> > + struct change_table *chtable,\n> > + const struct metacommit_data *metacommit,\n> > + const char *override_change,\n> > + int options,\n> > + struct strbuf *err,\n> > + struct string_list *changes);\n>\n> Does this need to be public? i.e. why would one call this rather than\n> record_metacommit()?\n\nTurns out the change command needs the `changes` string_list. I've\nmade this one private and extended the public one with the changes\nlist.\nI could potentially still return a strintmap but I'm on the fence.\n\n>\n> > +extern void modify_change(struct repository *repo,\n> > + const struct object_id *old_commit, const struct object_id *new_commit,\n> > + struct strbuf *err);\n> > +\n> > +extern int write_metacommit(struct repository *repo, struct metacommit_data *state,\n> > + struct object_id *result);\n>\n> The documentation for the flags is very welcome but this header could to\n> with the api being documented as well.\n>\n> Best Wishes\n>\n> Phillip\n\n\n\n>\n> > +extern void modify_change(struct repository *repo,\n> > + const struct object_id *old_commit, const struct object_id *new_commit,\n> > + struct strbuf *err);\n> > +\n> > +extern int write_metacommit(struct repository *repo, struct metacommit_data *state,\n> > + struct object_id *result);\n>\n> The documentation for the flags is very welcome but this header could to\n> with the api being documented as well.\n>\n> Best Wishes\n>\n> Phillip\n"},{"id":"464223","messageId":"a2e2bf70-95cc-41ee-c129-ef2f2e38fe79@gmail.com","threadId":"58504","inReplyTo":"CAN84kKke+vSQ18wNM7h6BvTF9XBtSH96L0_qNOFn+V3fj2yNhg@mail.gmail.com","subject":"Re: [PATCH 06/10] evolve: add support for writing metacommits","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-05T11:09:42Z","receivedAt":"2022-10-05T11:09:54Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 05/10/2022 10:40, Chris P wrote:\n>>> +/**\n>>> + * Produces the final form of a metacommit based on the current change refs.\n>>> + */\n>>> +static void resolve_metacommit(\n>>> + struct repository* repo,\n>>> + struct change_table* active_changes,\n>>> + const struct metacommit_data *to_resolve,\n>>\n>> [testing my understanding] This is the metacommit we want to update\n> \n> Maybe you can help me find a bug. If you run `git-change update`\n> twice without changing commits, it prints that it created a second\n> one, but then if you `git-change list` it doesn't show that last one\n> because it doesn't create an extra one if there's already a change\n> pointing at HEAD.\n\nThat's something I thought that we should add a test for but didn't get \nround to. I'll have a look and get back to you\n\n> Also, thanks for all the comments, it's helping my understanding too.\n> In general do you want all these comments added to the code?\n\nNot unless you think the code needs them (from what I remember it is \nalready fairly well commented), they're just me thinking out loud as I \nread the code.\n\nI'm going to be offline for the rest of the day, I'll catch up with the \nrest of your comments tomorrow or Friday.\n\nPhillip\n\n"},{"id":"464225","messageId":"CAN84kK=kTsy5stN7rR=EeLiTOq0si1UcmHfAE-UNvcofMSPKCg@mail.gmail.com","threadId":"58504","inReplyTo":"220926.8635ce4jox.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH 07/10] evolve: implement the git change command","fromName":"Chris P","fromEmail":"christophe.poucet@gmail.com","sentAt":"2022-10-05T12:30:18Z","receivedAt":"2022-10-05T12:30:36Z","isPatch":true,"sender":{"key":"christophe.poucet@gmail.com","avatar":null},"body":"On Mon, Sep 26, 2022 at 10:35 AM Ævar Arnfjörð Bjarmason\n wrote:\n>\n>\n> On Fri, Sep 23 2022, Stefan Xenos via GitGitGadget wrote:\n>\n> > From: Stefan Xenos \n>\n> > +static const char * const builtin_change_usage[] = {\n> > + N_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> > + NULL\n> > +};\n> > +\n> > +static const char * const builtin_update_usage[] = {\n> > + N_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> > + NULL\n> > +};\n>\n> This (and the corresponding later *.txt version) should indent the\n> overly long -h line, probably after \"[--replace ...]\".\n>\n> > +struct update_state {\n> > + int options;\n>\n> I think this should be an enum in your earlier 06/10. Makes things more\n>\n> > + die(_(\"Failed to resolve '%s' as a valid revision.\"), committish);\n>\n> This and other error should start with a lower-case letter, see\n> CodingGuidelines on errors.\n\nDone.\n\n>\n> > [...]\n> > + die(_(\"Could not parse object '%s'.\"), committish);\n>\n> Ditto etc.\n>\n> > + int i;\n> > + for (i = 0; i < commitsish_list->nr; i++) {\n>\n> A string_list uses a size_t for a nr, not int, so lets make that \"size_t\n> i\".\n>\n> This both makes things more obvious, and helps some compilers spot\n> unsigned v.s. signed issues.\n\nDone.\n\n>\n>\n> > + int i;\n>\n> ditto size_t above...\n>\n> > + for (i = 0; i < changes.nr; i++) {\n>\n> ...for this iteration...\n\nObsolete, moved to using for_each_string_list_item\n\n>\n> > + struct string_list_item *it = &changes.items[i];\n>\n> ...but actually don't you just want for_each_string_list_item() instead?\n>\n> > + if (it->util)\n> > + fprintf(stdout, N_(\"Updated change %s\\n\"), name);\n> > + else\n> > + fprintf(stdout, N_(\"Created change %s\\n\"), name);\n>\n> The use of N_() here is wrong, you should use _(), N_() just marks\n> things for translation, but doesn't use it.\n>\n\nDone.\n\n> We also tend to try to avoid adding \\n in translations needlessly. And\n> since you're printing to stdout this can be:\n>\n>\n> if (...)\n> printf(_(\"Updated change %s\"), name);\n> ...\n> putchar('\\n')\n\nDone\n\n>\n>\n> > + }\n> > +\n> > + string_list_clear(&changes, 0);\n> > + change_table_clear(&chtable);\n> > + clear_metacommit_data(&metacommit);\n> > +\n> > + return ret;\n> > +}\n> > +\n> > +static int change_update(int argc, const char **argv, const char* prefix)\n> > +{\n> > + int result;\n> > + int force = 0;\n> > + int newchange = 0;\n> > + struct strbuf err = STRBUF_INIT;\n> > + struct update_state state;\n> > + struct option options[] = {\n> > + { OPTION_CALLBACK, 'r', \"replace\", &state, N_(\"commit\"),\n> > + N_(\"marks the given commit as being obsolete\"),\n> > + 0, update_option_parse_replace },\n> > + { OPTION_CALLBACK, 'o', \"origin\", &state, N_(\"commit\"),\n> > + N_(\"marks the given commit as being the origin of this commit\"),\n> > + 0, update_option_parse_origin },\n> > + OPT_BOOL('F', \"force\", &force,\n> > + N_(\"overwrite an existing change of the same name\")),\n> > + OPT_STRING('c', \"content\", &state.content, N_(\"commit\"),\n> > + N_(\"identifies the new content commit for the change\")),\n> > + OPT_STRING('g', \"change\", &state.change, N_(\"commit\"),\n> > + N_(\"name of the change to update\")),\n> > + OPT_BOOL('n', \"new\", &newchange,\n> > + N_(\"create a new change - do not append to any existing change\")),\n> > + OPT_END()\n> > + };\n> > +\n> > + init_update_state(&state);\n> > +\n> > + argc = parse_options(argc, argv, prefix, options, builtin_update_usage, 0);\n> > +\n> > + if (force) state.options |= UPDATE_OPTION_FORCE;\n> > + if (newchange) state.options |= UPDATE_OPTION_NOAPPEND;\n>\n> Just use OPT_SET_INT_F() and skip the indirection thorugh OPT_BOOL(),\n> that macro itself is a thin wrapper for OPT_SET_INT_F().\n>\n> I.e. you can drop these \"force\" and \"newchange\" variables, andjust set\n> your state.options directly.\n\nDone.\n\n>\n> > +int cmd_change(int argc, const char **argv, const char *prefix)\n> > +{\n> > + /* No options permitted before subcommand currently */\n> > + struct option options[] = {\n> > + OPT_END()\n> > + };\n> > + int result = 1;\n> > +\n> > + argc = parse_options(argc, argv, prefix, options, builtin_change_usage,\n> > + PARSE_OPT_STOP_AT_NON_OPTION);\n> > +\n> > + if (argc < 1)\n> > + usage_with_options(builtin_change_usage, options);\n> > + else if (!strcmp(argv[0], \"update\"))\n> > + result = change_update(argc, argv, prefix);\n> > + else {\n> > + error(_(\"Unknown subcommand: %s\"), argv[0]);\n> > + usage_with_options(builtin_change_usage, options);\n> > + }\n>\n> This was presumably written before the recent OPT_SUBCOMMAND(), and\n> should instead use that API.\n\nDone, thanks!\n"},{"id":"464231","messageId":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.git.1663959324.gitgitgadget@gmail.com","subject":"[PATCH v2 00/10] RFC: Git Evolve / Change","fromName":"Christophe Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:07Z","receivedAt":"2022-10-05T15:00:52Z","isPatch":true,"sender":{"key":"name:Christophe Poucet","avatar":null},"body":"I'm reviving the original git evolve work that was started by\nsxenos@google.com\n(https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n\nThis work is intended to make it easier to deal with stacked changes.\n\nThe following set of patches introduces the design doc on the evolve command\nas well as the basics of the git change command.\n\nChris Poucet (5):\n sha1-array: implement oid_array_readonly_contains\n ref-filter: add the metas namespace to ref-filter\n evolve: add delete command\n evolve: add documentation for `git change`\n evolve: add tests for the git-change command\n\nStefan Xenos (5):\n technical doc: add a design doc for the evolve command\n evolve: add support for parsing metacommits\n evolve: add the change-table structure\n evolve: add support for writing metacommits\n evolve: implement the git change command\n\n .gitignore | 1 +\n Documentation/git-change.txt | 55 ++\n Documentation/technical/evolve.txt | 1070 ++++++++++++++++++++++++++++\n Makefile | 4 +\n builtin.h | 1 +\n builtin/change.c | 330 +++++++++\n change-table.c | 164 +++++\n change-table.h | 122 ++++\n commit.c | 13 +\n commit.h | 5 +\n git.c | 1 +\n metacommit-parser.c | 97 +++\n metacommit-parser.h | 19 +\n metacommit.c | 410 +++++++++++\n metacommit.h | 75 ++\n oid-array.c | 12 +\n oid-array.h | 7 +\n ref-filter.c | 10 +-\n ref-filter.h | 10 +-\n t/helper/test-oid-array.c | 6 +\n t/t0064-oid-array.sh | 22 +\n t/t9990-changes.sh | 148 ++++\n 22 files changed, 2577 insertions(+), 5 deletions(-)\n create mode 100644 Documentation/git-change.txt\n create mode 100644 Documentation/technical/evolve.txt\n create mode 100644 builtin/change.c\n create mode 100644 change-table.c\n create mode 100644 change-table.h\n create mode 100644 metacommit-parser.c\n create mode 100644 metacommit-parser.h\n create mode 100644 metacommit.c\n create mode 100644 metacommit.h\n create mode 100755 t/t9990-changes.sh\n\n\nbase-commit: 3dcec76d9df911ed8321007b1d197c1a206dc164\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1356%2Fpoucet%2Fevolve-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1356/poucet/evolve-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/1356\n\nRange-diff vs v1:\n\n 1: a0cf68f8ba2 ! 1: a5eb9325419 technical doc: add a design doc for the evolve command\n @@ Documentation/technical/evolve.txt (new)\n +rebase. You can think of rebase -i as a top-down approach and the evolve command\n +as the bottom-up approach to the same problem.\n +\n ++Revup amend (https://github.com/Skydio/revup/blob/main/docs/amend.md)\n ++allows insertion of cached changes into any commit in\n ++the current history, and then reapplies the rest of history on top of\n ++those changes. It uses a \"git apply --cached\" engine under the hood so\n ++doesn't touch the working directory (although it will soon use the new\n ++git merge-tree). When paired with \"revup upload\" which creates and\n ++pushes multiple branches in the background for you, its possible to\n ++work on a \"graph\" of changes on a single branch linearly, then have\n ++the true graph structure created at upload time.\n ++\n ++git-revise (https://github.com/mystor/git-revise) does some very\n ++similar things except it uses \"git merge-file\" combined with manually\n ++merging the resulting trees. git branchstack\n ++(https://github.com/krobelus/git-branchstack) can also create branches\n ++in the background with the same mechanism.\n ++\n ++These tools don't store any external state, but as such also don't\n ++provide any specific collaboration mechanism for individual changes.\n ++\n +Several patch queue managers have been built on top of git (such as topgit,\n +stgit, and quilt). They address the same user need. However they also rely on\n +state managed outside git that needs to be kept in sync. Such state can be\n 2: 84588312c1d = 2: ed5106d6080 sha1-array: implement oid_array_readonly_contains\n 3: 54e559967df ! 3: c59066ebc10 ref-filter: add the metas namespace to ref-filter\n @@ ref-filter.c: int filter_refs(struct ref_array *array, struct ref_filter *filter\n \n ## ref-filter.h ##\n @@\n + #define FILTER_REFS_TAGS 0x0002\n #define FILTER_REFS_BRANCHES 0x0004\n #define FILTER_REFS_REMOTES 0x0008\n - #define FILTER_REFS_OTHERS 0x0010\n -+#define FILTER_REFS_CHANGES 0x0040\n +-#define FILTER_REFS_OTHERS 0x0010\n ++#define FILTER_REFS_CHANGES 0x0010\n ++#define FILTER_REFS_OTHERS 0x0040\n #define FILTER_REFS_ALL (FILTER_REFS_TAGS | FILTER_REFS_BRANCHES | \\\n -\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS)\n +\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS | \\\n 4: 2e9a4a9bd81 ! 4: 408941e7400 evolve: add support for parsing metacommits\n @@ Makefile: LIB_OBJS += merge-ort.o\n LIB_OBJS += name-hash.o\n LIB_OBJS += negotiator/default.o\n \n + ## commit.c ##\n +@@ commit.c: struct commit_list *reverse_commit_list(struct commit_list *list)\n + \treturn next;\n + }\n + \n ++struct commit *get_commit_by_index(struct commit_list *to_search, int index)\n ++{\n ++\twhile (to_search && index) {\n ++\t\tto_search = to_search->next;\n ++\t\tindex--;\n ++\t}\n ++\n ++\tif (!to_search)\n ++\t\treturn NULL;\n ++\n ++\treturn to_search->item;\n ++}\n ++\n + void free_commit_list(struct commit_list *list)\n + {\n + \twhile (list)\n +\n + ## commit.h ##\n +@@ commit.h: struct commit_list *copy_commit_list(struct commit_list *list);\n + /* Modify list in-place to reverse it, returning new head; list will be tail */\n + struct commit_list *reverse_commit_list(struct commit_list *list);\n + \n ++/* Returns the commit at `index` or NULL if the index exceeds the `to_search`\n ++ * list */\n ++struct commit *get_commit_by_index(struct commit_list *to_search, int index);\n ++\n + void free_commit_list(struct commit_list *list);\n + \n ++\n + struct rev_info; /* in revision.h, it circularly uses enum cmit_fmt */\n + \n + int has_non_ascii(const char *text);\n +\n ## metacommit-parser.c (new) ##\n @@\n +#include \"cache.h\"\n @@ metacommit-parser.c (new)\n +\treturn NULL;\n +}\n +\n -+static struct commit *get_commit_by_index(struct commit_list *to_search, int index)\n -+{\n -+\twhile (to_search && index) {\n -+\t\tto_search = to_search->next;\n -+\t\tindex--;\n -+\t}\n -+\n -+\tif (!to_search)\n -+\t\treturn NULL;\n -+\n -+\treturn to_search->item;\n -+}\n -+\n +/*\n + * Writes the index of the content parent to \"result\". Returns the metacommit\n + * type. See the METACOMMIT_TYPE_* constants.\n + */\n -+static int index_of_content_commit(const char *buffer, int *result)\n ++static enum metacommit_type index_of_content_commit(const char *buffer, int *result)\n +{\n +\tint index = 0;\n +\tint ret = METACOMMIT_TYPE_NONE;\n @@ metacommit-parser.c (new)\n +\t\tchar next = *parent_types;\n +\t\tif (next == ' ' || parent_types >= end) {\n +\t\t\tif (enum_length == 1) {\n -+\t\t\t\tchar first_char_in_enum = *enum_start;\n -+\t\t\t\tif (first_char_in_enum == 'c') {\n ++\t\t\t\tchar type = *enum_start;\n ++\t\t\t\tif (type == 'c') {\n +\t\t\t\t\tret = METACOMMIT_TYPE_NORMAL;\n +\t\t\t\t\tbreak;\n +\t\t\t\t}\n -+\t\t\t\tif (first_char_in_enum == 'a') {\n ++\t\t\t\tif (type == 'a') {\n +\t\t\t\t\tret = METACOMMIT_TYPE_ABANDONED;\n +\t\t\t\t\tbreak;\n +\t\t\t\t}\n @@ metacommit-parser.c (new)\n + * Writes the content parent's object id to \"content\".\n + * Returns the metacommit type. See the METACOMMIT_TYPE_* constants.\n + */\n -+int get_metacommit_content(struct commit *commit, struct object_id *content)\n ++enum metacommit_type get_metacommit_content(struct commit *commit, struct object_id *content)\n +{\n +\tconst char *buffer = get_commit_buffer(commit, NULL);\n +\tint index = 0;\n -+\tint ret = index_of_content_commit(buffer, &index);\n ++\tenum metacommit_type ret = index_of_content_commit(buffer, &index);\n +\tstruct commit *content_parent;\n +\n +\tif (ret == METACOMMIT_TYPE_NONE)\n @@ metacommit-parser.h (new)\n +#include \"commit.h\"\n +#include \"hash.h\"\n +\n -+/* Indicates a normal commit (non-metacommit) */\n -+#define METACOMMIT_TYPE_NONE 0\n -+/* Indicates a metacommit with normal content (non-abandoned) */\n -+#define METACOMMIT_TYPE_NORMAL 1\n -+/* Indicates a metacommit with abandoned content */\n -+#define METACOMMIT_TYPE_ABANDONED 2\n -+\n -+struct commit;\n ++enum metacommit_type {\n ++\t/* Indicates a normal commit (non-metacommit) */\n ++\tMETACOMMIT_TYPE_NONE = 0,\n ++\t/* Indicates a metacommit with normal content (non-abandoned) */\n ++\tMETACOMMIT_TYPE_NORMAL = 1,\n ++\t/* Indicates a metacommit with abandoned content */\n ++\tMETACOMMIT_TYPE_ABANDONED = 2,\n ++};\n +\n -+extern int get_metacommit_content(\n ++enum metacommit_type get_metacommit_content(\n +\tstruct commit *commit, struct object_id *content);\n +\n +#endif\n 5: 2b3a00a6702 ! 5: 48cd92d35ef evolve: add the change-table structure\n @@ change-table.c (new)\n +#include \"ref-filter.h\"\n +#include \"metacommit-parser.h\"\n +\n -+void change_table_init(struct change_table *to_initialize)\n ++void change_table_init(struct change_table *table)\n +{\n -+\tmemset(to_initialize, 0, sizeof(*to_initialize));\n -+\tmem_pool_init(&to_initialize->memory_pool, 0);\n -+\tto_initialize->memory_pool.block_alloc = 4*1024 - sizeof(struct mp_block);\n -+\toidmap_init(&to_initialize->oid_to_metadata_index, 0);\n -+\tstring_list_init_dup(&to_initialize->refname_to_change_head);\n ++\tmemset(table, 0, sizeof(*table));\n ++\tmem_pool_init(&table->memory_pool, 0);\n ++\toidmap_init(&table->oid_to_metadata_index, 0);\n ++\tstrmap_init(&table->refname_to_change_head);\n +}\n +\n -+static void change_list_clear(struct change_list *to_clear) {\n -+\tstring_list_clear(&to_clear->additional_refnames, 0);\n ++static void change_list_clear(struct change_list *change_list) {\n ++\tstrset_clear(&change_list->refnames);\n +}\n +\n +static void commit_change_list_entry_clear(\n -+\tstruct commit_change_list_entry *to_clear) {\n -+\tchange_list_clear(&to_clear->changes);\n ++\tstruct commit_change_list_entry *entry) {\n ++\tchange_list_clear(&entry->changes);\n +}\n +\n -+void change_table_clear(struct change_table *to_clear)\n ++void change_table_clear(struct change_table *table)\n +{\n +\tstruct oidmap_iter iter;\n +\tstruct commit_change_list_entry *next;\n -+\tfor (next = oidmap_iter_first(&to_clear->oid_to_metadata_index, &iter);\n ++\tfor (next = oidmap_iter_first(&table->oid_to_metadata_index, &iter);\n +\t\tnext;\n +\t\tnext = oidmap_iter_next(&iter)) {\n +\n +\t\tcommit_change_list_entry_clear(next);\n +\t}\n +\n -+\toidmap_free(&to_clear->oid_to_metadata_index, 0);\n -+\tstring_list_clear(&to_clear->refname_to_change_head, 0);\n -+\tmem_pool_discard(&to_clear->memory_pool, 0);\n ++\toidmap_free(&table->oid_to_metadata_index, 0);\n ++\tstrmap_clear(&table->refname_to_change_head, 0);\n ++\tmem_pool_discard(&table->memory_pool, 0);\n +}\n +\n -+static void add_head_to_commit(struct change_table *to_modify,\n -+\tconst struct object_id *to_add, const char *refname)\n ++static void add_head_to_commit(struct change_table *table,\n ++\t\t\t const struct object_id *to_add,\n ++\t\t\t const char *refname)\n +{\n +\tstruct commit_change_list_entry *entry;\n +\n -+\t/**\n -+\t * Note: the indices in the map are 1-based. 0 is used to indicate a missing\n -+\t * element.\n -+\t */\n -+\tentry = oidmap_get(&to_modify->oid_to_metadata_index, to_add);\n ++\tentry = oidmap_get(&table->oid_to_metadata_index, to_add);\n +\tif (!entry) {\n -+\t\tentry = mem_pool_calloc(&to_modify->memory_pool, 1,\n -+\t\t\tsizeof(*entry));\n ++\t\tentry = mem_pool_calloc(&table->memory_pool, 1, sizeof(*entry));\n +\t\toidcpy(&entry->entry.oid, to_add);\n -+\t\toidmap_put(&to_modify->oid_to_metadata_index, entry);\n -+\t\tstring_list_init_nodup(&entry->changes.additional_refnames);\n ++\t\tstrset_init(&entry->changes.refnames);\n ++\t\toidmap_put(&table->oid_to_metadata_index, entry);\n +\t}\n -+\n -+\tif (!entry->changes.first_refname)\n -+\t\tentry->changes.first_refname = refname;\n -+\telse\n -+\t\tstring_list_insert(&entry->changes.additional_refnames, refname);\n ++\tstrset_add(&entry->changes.refnames, refname);\n +}\n +\n -+void change_table_add(struct change_table *to_modify, const char *refname,\n -+\tstruct commit *to_add)\n ++void change_table_add(struct change_table *table,\n ++\t\t const char *refname,\n ++\t\t struct commit *to_add)\n +{\n +\tstruct change_head *new_head;\n -+\tstruct string_list_item *new_item;\n +\tint metacommit_type;\n +\n -+\tnew_head = mem_pool_calloc(&to_modify->memory_pool, 1,\n -+\t\tsizeof(*new_head));\n ++\tnew_head = mem_pool_calloc(&table->memory_pool, 1, sizeof(*new_head));\n +\n +\toidcpy(&new_head->head, &to_add->object.oid);\n +\n +\tmetacommit_type = get_metacommit_content(to_add, &new_head->content);\n ++\t/* If to_add is not a metacommit then the content is to_add itself,\n ++\t * otherwise it will have been set by the call to\n ++\t * get_metacommit_content.\n ++\t */\n +\tif (metacommit_type == METACOMMIT_TYPE_NONE)\n +\t\toidcpy(&new_head->content, &to_add->object.oid);\n +\tnew_head->abandoned = (metacommit_type == METACOMMIT_TYPE_ABANDONED);\n +\tnew_head->remote = starts_with(refname, \"refs/remote/\");\n +\tnew_head->hidden = starts_with(refname, \"refs/hiddenmetas/\");\n +\n -+\tnew_item = string_list_insert(&to_modify->refname_to_change_head, refname);\n -+\tnew_item->util = new_head;\n -+\t/* Use pointers to the copy of the string we're retaining locally */\n -+\trefname = new_item->string;\n -+\n -+\tif (!oideq(&new_head->content, &new_head->head))\n -+\t\tadd_head_to_commit(to_modify, &new_head->content, refname);\n -+\tadd_head_to_commit(to_modify, &new_head->head, refname);\n -+}\n -+\n -+void change_table_add_all_visible(struct change_table *to_modify,\n -+\tstruct repository* repo)\n -+{\n -+\tstruct ref_filter filter;\n -+\tconst char *name_patterns[] = {NULL};\n -+\tmemset(&filter, 0, sizeof(filter));\n -+\tfilter.kind = FILTER_REFS_CHANGES;\n -+\tfilter.name_patterns = name_patterns;\n ++\tstrmap_put(&table->refname_to_change_head, refname, new_head);\n +\n -+\tchange_table_add_matching_filter(to_modify, repo, &filter);\n ++\tif (!oideq(&new_head->content, &new_head->head)) {\n ++\t\t/* We also remember to link between refname and the content oid */\n ++\t\tadd_head_to_commit(table, &new_head->content, refname);\n ++\t}\n ++\tadd_head_to_commit(table, &new_head->head, refname);\n +}\n +\n -+void change_table_add_matching_filter(struct change_table *to_modify,\n -+\tstruct repository* repo, struct ref_filter *filter)\n ++static void change_table_add_matching_filter(struct change_table *table,\n ++\t\t\t\t\t struct repository* repo,\n ++\t\t\t\t\t struct ref_filter *filter)\n +{\n -+\tstruct ref_array matching_refs;\n +\tint i;\n ++\tstruct ref_array matching_refs = { 0 };\n +\n -+\tmemset(&matching_refs, 0, sizeof(matching_refs));\n +\tfilter_refs(&matching_refs, filter, filter->kind);\n +\n -+\t/**\n ++\t/*\n +\t * Determine the object id for the latest content commit for each change.\n +\t * Fetch the commit at the head of each change ref. If it's a normal commit,\n +\t * that's the commit we want. If it's a metacommit, locate its content parent\n @@ change-table.c (new)\n +\n +\tfor (i = 0; i < matching_refs.nr; i++) {\n +\t\tstruct ref_array_item *item = matching_refs.items[i];\n -+\t\tstruct commit *commit = item->commit;\n ++\t\tstruct commit *commit;\n +\n -+\t\tcommit = lookup_commit_reference_gently(repo, &item->objectname, 1);\n -+\n -+\t\tif (commit)\n -+\t\t\tchange_table_add(to_modify, item->refname, commit);\n ++\t\tcommit = lookup_commit_reference(repo, &item->objectname);\n ++\t\tif (!commit) {\n ++\t\t\tBUG(\"Invalid commit for refs/meta: %s\", item->refname);\n ++\t\t}\n ++\t\tchange_table_add(table, item->refname, commit);\n +\t}\n +\n +\tref_array_clear(&matching_refs);\n +}\n +\n ++void change_table_add_all_visible(struct change_table *table,\n ++\tstruct repository* repo)\n ++{\n ++\tstruct ref_filter filter = { 0 };\n ++\tconst char *name_patterns[] = {NULL};\n ++\tfilter.kind = FILTER_REFS_CHANGES;\n ++\tfilter.name_patterns = name_patterns;\n ++\n ++\tchange_table_add_matching_filter(table, repo, &filter);\n ++}\n ++\n +static int return_true_callback(const char *refname, void *cb_data)\n +{\n +\treturn 1;\n +}\n +\n -+int change_table_has_change_referencing(struct change_table *changes,\n ++int change_table_has_change_referencing(struct change_table *table,\n +\tconst struct object_id *referenced_commit_id)\n +{\n -+\treturn for_each_change_referencing(changes, referenced_commit_id,\n ++\treturn for_each_change_referencing(table, referenced_commit_id,\n +\t\treturn_true_callback, NULL);\n +}\n +\n +int for_each_change_referencing(struct change_table *table,\n +\tconst struct object_id *referenced_commit_id, each_change_fn fn, void *cb_data)\n +{\n -+\tconst struct change_list *changes;\n -+\tint i;\n -+\tint retvalue;\n -+\tstruct commit_change_list_entry *entry;\n ++\tint ret;\n ++\tstruct commit_change_list_entry *ccl_entry;\n ++\tstruct hashmap_iter iter;\n ++\tstruct strmap_entry *entry;\n +\n -+\tentry = oidmap_get(&table->oid_to_metadata_index,\n -+\t\treferenced_commit_id);\n ++\tccl_entry = oidmap_get(&table->oid_to_metadata_index,\n ++\t\t\t referenced_commit_id);\n +\t/* If this commit isn't referenced by any changes, it won't be in the map */\n -+\tif (!entry)\n ++\tif (!ccl_entry)\n +\t\treturn 0;\n -+\tchanges = &entry->changes;\n -+\tif (!changes->first_refname)\n -+\t\treturn 0;\n -+\tretvalue = fn(changes->first_refname, cb_data);\n -+\tfor (i = 0; retvalue == 0 && i < changes->additional_refnames.nr; i++)\n -+\t\tretvalue = fn(changes->additional_refnames.items[i].string, cb_data);\n -+\treturn retvalue;\n ++\tstrset_for_each_entry(&ccl_entry->changes.refnames, &iter, entry) {\n ++\t\tret = fn(entry->key, cb_data);\n ++\t\tif (ret != 0) break;\n ++\t}\n ++\treturn ret;\n +}\n +\n -+struct change_head* get_change_head(struct change_table *heads,\n ++struct change_head* get_change_head(struct change_table *table,\n +\tconst char* refname)\n +{\n -+\tstruct string_list_item *item = string_list_lookup(\n -+\t\t&heads->refname_to_change_head, refname);\n -+\n -+\tif (!item)\n -+\t\treturn NULL;\n -+\n -+\treturn (struct change_head *)item->util;\n ++\treturn strmap_get(&table->refname_to_change_head, refname);\n +}\n \n ## change-table.h (new) ##\n @@ change-table.h (new)\n +#define CHANGE_TABLE_H\n +\n +#include \"oidmap.h\"\n ++#include \"strmap.h\"\n +\n +struct commit;\n +struct ref_filter;\n +\n +/**\n -+ * This struct holds a list of change refs. The first element is stored inline,\n -+ * to optimize for small lists.\n ++ * This struct holds a set of change refs.\n + */\n +struct change_list {\n +\t/**\n -+\t * Ref name for the first change in the list, or null if none.\n -+\t *\n ++\t * The refnames in this set.\n +\t * This field is private. Use for_each_change_in to read.\n +\t */\n -+\tconst char* first_refname;\n -+\t/**\n -+\t * List of additional change refs. Note that this is empty if the list\n -+\t * contains 0 or 1 elements.\n -+\t *\n -+\t * This field is private. Use for_each_change_in to read.\n -+\t */\n -+\tstruct string_list additional_refnames;\n ++\tstruct strset refnames;\n +};\n +\n +/**\n @@ change-table.h (new)\n +};\n +\n +/**\n -+ * Holds information about the heads of each change, and permits effecient\n ++ * Holds information about the heads of each change, and permits efficient\n + * lookup from a commit to the changes that reference it directly.\n + *\n + * All fields should be considered private. Use the change_table functions\n @@ change-table.h (new)\n +\t/* Map object_id to commit_change_list_entry structs. */\n +\tstruct oidmap oid_to_metadata_index;\n +\t/**\n -+\t * List of ref names. The util value points to a change_head structure\n -+\t * allocated from memory_pool.\n ++\t * Map of refnames to change_head structure which are allocated from\n ++\t * memory_pool.\n +\t */\n -+\tstruct string_list refname_to_change_head;\n ++\tstruct strmap refname_to_change_head;\n +};\n +\n -+extern void change_table_init(struct change_table *to_initialize);\n -+extern void change_table_clear(struct change_table *to_clear);\n ++extern void change_table_init(struct change_table *table);\n ++extern void change_table_clear(struct change_table *table);\n +\n +/* Adds the given change head to the change_table struct */\n -+extern void change_table_add(struct change_table *to_modify,\n -+\tconst char *refname, struct commit *target);\n ++extern void change_table_add(struct change_table *table,\n ++\t\t\t const char *refname,\n ++\t\t\t struct commit *target);\n +\n +/**\n + * Adds the non-hidden local changes to the given change_table struct.\n + */\n -+extern void change_table_add_all_visible(struct change_table *to_modify,\n -+\tstruct repository *repo);\n -+\n -+/*\n -+ * Adds all changes matching the given ref filter to the given change_table\n -+ * struct.\n -+ */\n -+extern void change_table_add_matching_filter(struct change_table *to_modify,\n -+\tstruct repository* repo, struct ref_filter *filter);\n ++extern void change_table_add_all_visible(struct change_table *table,\n ++\t\t\t\t\t struct repository *repo);\n +\n +typedef int each_change_fn(const char *refname, void *cb_data);\n +\n -+extern int change_table_has_change_referencing(struct change_table *changes,\n ++extern int change_table_has_change_referencing(\n ++\tstruct change_table *table,\n +\tconst struct object_id *referenced_commit_id);\n +\n +/**\n @@ change-table.h (new)\n + * For normal commits, this is the list of changes that have this commit as\n + * their latest content.\n + */\n -+extern int for_each_change_referencing(struct change_table *heads,\n -+\tconst struct object_id *referenced_commit_id, each_change_fn fn, void *cb_data);\n ++extern int for_each_change_referencing(\n ++\tstruct change_table *table,\n ++\tconst struct object_id *referenced_commit_id,\n ++\teach_change_fn fn,\n ++\tvoid *cb_data);\n +\n +/**\n + * Returns the change head for the given refname. Returns NULL if no such change\n + * exists.\n + */\n -+extern struct change_head* get_change_head(struct change_table *heads,\n ++extern struct change_head* get_change_head(struct change_table *table,\n +\tconst char* refname);\n +\n +#endif\n 6: 56c6770997b ! 6: 353d97d0f38 evolve: add support for writing metacommits\n @@ metacommit.c (new)\n +#include \"change-table.h\"\n +#include \"refs.h\"\n +\n -+void init_metacommit_data(struct metacommit_data *state)\n -+{\n -+\tmemset(state, 0, sizeof(*state));\n -+}\n -+\n +void clear_metacommit_data(struct metacommit_data *state)\n +{\n ++\toidcpy(&state->content, null_oid());\n +\toid_array_clear(&state->replace);\n +\toid_array_clear(&state->origin);\n ++\tstate->abandoned = 0;\n +}\n +\n +static void compute_default_change_name(struct commit *initial_commit,\n -+\tstruct strbuf* result)\n ++\t\t\t\t\tstruct strbuf* result)\n +{\n -+\tstruct strbuf default_name;\n ++\tstruct strbuf default_name = STRBUF_INIT;\n +\tconst char *buffer;\n +\tconst char *subject;\n +\tconst char *eol;\n -+\tint len;\n -+\tstrbuf_init(&default_name, 0);\n ++\tsize_t len;\n +\tbuffer = get_commit_buffer(initial_commit, NULL);\n +\tfind_commit_subject(buffer, &subject);\n +\teol = strchrnul(subject, '\\n');\n -+\tfor (len = 0;subject < eol && len < 10; ++subject, ++len) {\n ++\tfor (len = 0; subject < eol && len < 10; subject++, len++) {\n +\t\tchar next = *subject;\n +\t\tif (isspace(next))\n +\t\t\tcontinue;\n @@ metacommit.c (new)\n +\t\tstrbuf_addch(&default_name, next);\n +\t}\n +\tsanitize_refname_component(default_name.buf, result);\n ++\tunuse_commit_buffer(initial_commit, buffer);\n +}\n +\n -+/**\n ++/*\n + * Computes a change name for a change rooted at the given initial commit. Good\n + * change names should be memorable, unique, and easy to type. They are not\n + * required to match the commit comment.\n + */\n +static void compute_change_name(struct commit *initial_commit, struct strbuf* result)\n +{\n -+\tstruct strbuf default_name;\n ++\tstruct strbuf default_name = STRBUF_INIT;\n +\tstruct object_id unused;\n +\n -+\tstrbuf_init(&default_name, 0);\n +\tif (initial_commit)\n +\t\tcompute_default_change_name(initial_commit, &default_name);\n +\telse\n -+\t\tstrbuf_addstr(&default_name, \"change\");\n ++\t\tBUG(\"initial commit is NULL\");\n +\tstrbuf_addstr(result, \"refs/metas/\");\n +\tstrbuf_addbuf(result, &default_name);\n +\n +\t/* If there is already a change of this name, append a suffix */\n +\tif (!read_ref(result->buf, &unused)) {\n +\t\tint suffix = 2;\n -+\t\tint original_length = result->len;\n ++\t\tsize_t original_length = result->len;\n +\n +\t\twhile (1) {\n +\t\t\tstrbuf_addf(result, \"%d\", suffix);\n +\t\t\tif (read_ref(result->buf, &unused))\n +\t\t\t\tbreak;\n -+\t\t\tstrbuf_remove(result, original_length, result->len - original_length);\n ++\t\t\tstrbuf_remove(result, original_length,\n ++\t\t\t\t result->len - original_length);\n +\t\t\t++suffix;\n +\t\t}\n +\t}\n @@ metacommit.c (new)\n +\tstrbuf_release(&default_name);\n +}\n +\n -+struct resolve_metacommit_callback_data\n ++struct resolve_metacommit_context\n +{\n +\tstruct change_table* active_changes;\n +\tstruct string_list *changes;\n @@ metacommit.c (new)\n +\n +static int resolve_metacommit_callback(const char *refname, void *cb_data)\n +{\n -+\tstruct resolve_metacommit_callback_data *data = (struct resolve_metacommit_callback_data *)cb_data;\n ++\tstruct resolve_metacommit_context *data = cb_data;\n +\tstruct change_head *chhead;\n +\n +\tchhead = get_change_head(data->active_changes, refname);\n +\n +\tif (data->changes)\n -+\t\tstring_list_append(data->changes, refname)->util = &(chhead->head);\n ++\t\tstring_list_append(data->changes, refname)->util = &chhead->head;\n +\tif (data->heads)\n +\t\toid_array_append(data->heads, &(chhead->head));\n +\n +\treturn 0;\n +}\n +\n -+/**\n ++/*\n + * Produces the final form of a metacommit based on the current change refs.\n + */\n +static void resolve_metacommit(\n @@ metacommit.c (new)\n +\tstruct string_list *to_advance,\n +\tint allow_append)\n +{\n -+\tint i;\n -+\tint len = to_resolve->replace.nr;\n -+\tstruct resolve_metacommit_callback_data cbdata;\n ++\tsize_t i;\n ++\tsize_t len = to_resolve->replace.nr;\n ++\tstruct resolve_metacommit_context ctx = {\n ++\t\t.active_changes = active_changes,\n ++\t\t.changes = to_advance,\n ++\t\t.heads = &resolved_output->replace\n ++\t};\n +\tint old_change_list_length = to_advance->nr;\n +\tstruct commit* content;\n +\n +\toidcpy(&resolved_output->content, &to_resolve->content);\n +\n -+\t/* First look for changes that point to any of the replacement edges in the\n ++\t/*\n ++\t * First look for changes that point to any of the replacement edges in the\n +\t * metacommit. These will be the changes that get advanced by this\n -+\t * metacommit. */\n ++\t * metacommit.\n ++\t */\n +\tresolved_output->abandoned = to_resolve->abandoned;\n -+\tcbdata.active_changes = active_changes;\n -+\tcbdata.changes = to_advance;\n -+\tcbdata.heads = &(resolved_output->replace);\n +\n +\tif (allow_append) {\n +\t\tfor (i = 0; i < len; i++) {\n +\t\t\tint old_number = resolved_output->replace.nr;\n -+\t\t\tfor_each_change_referencing(active_changes, &(to_resolve->replace.oid[i]),\n -+\t\t\t\tresolve_metacommit_callback, &cbdata);\n ++\t\t\tfor_each_change_referencing(\n ++\t\t\t\tactive_changes,\n ++\t\t\t\t&(to_resolve->replace.oid[i]),\n ++\t\t\t\tresolve_metacommit_callback,\n ++\t\t\t\t&ctx);\n +\t\t\t/* If no changes were found, use the unresolved value. */\n +\t\t\tif (old_number == resolved_output->replace.nr)\n -+\t\t\t\toid_array_append(&(resolved_output->replace), &(to_resolve->replace.oid[i]));\n ++\t\t\t\toid_array_append(&(resolved_output->replace),\n ++\t\t\t\t\t\t &(to_resolve->replace.oid[i]));\n +\t\t}\n +\t}\n +\n -+\tcbdata.changes = NULL;\n -+\tcbdata.heads = &(resolved_output->origin);\n ++\tctx.changes = NULL;\n ++\tctx.heads = &(resolved_output->origin);\n +\n +\tlen = to_resolve->origin.nr;\n +\tfor (i = 0; i < len; i++) {\n +\t\tint old_number = resolved_output->origin.nr;\n -+\t\tfor_each_change_referencing(active_changes, &(to_resolve->origin.oid[i]),\n -+\t\t\tresolve_metacommit_callback, &cbdata);\n ++\t\tfor_each_change_referencing(\n ++\t\t\tactive_changes,\n ++\t\t\t&(to_resolve->origin.oid[i]),\n ++\t\t\tresolve_metacommit_callback,\n ++\t\t\t&ctx);\n +\t\tif (old_number == resolved_output->origin.nr)\n -+\t\t\toid_array_append(&(resolved_output->origin), &(to_resolve->origin.oid[i]));\n ++\t\t\toid_array_append(&(resolved_output->origin),\n ++\t\t\t\t\t &(to_resolve->origin.oid[i]));\n +\t}\n +\n -+\t/* If no changes were advanced by this metacommit, we'll need to create a new\n -+\t * one. */\n ++\t/*\n ++\t * If no changes were advanced by this metacommit, we'll need to create\n ++\t * a new one. */\n +\tif (to_advance->nr == old_change_list_length) {\n +\t\tstruct strbuf change_name;\n +\n +\t\tstrbuf_init(&change_name, 80);\n -+\t\tcontent = lookup_commit_reference_gently(repo, &(to_resolve->content), 1);\n ++\n ++\t\tcontent = lookup_commit_reference_gently(\n ++\t\t\trepo, &(to_resolve->content), 1);\n +\n +\t\tcompute_change_name(content, &change_name);\n +\t\tstring_list_append(to_advance, change_name.buf);\n @@ metacommit.c (new)\n +\n +\twhile (--i >= 0) {\n +\t\tstruct object_id *next = &(to_lookup->oid[i]);\n -+\t\tstruct commit *commit = lookup_commit_reference_gently(repo, next, 1);\n ++\t\tstruct commit *commit =\n ++\t\t\tlookup_commit_reference_gently(repo, next, 1);\n +\t\tcommit_list_insert(commit, result);\n +\t}\n +}\n +\n +#define PARENT_TYPE_PREFIX \"parent-type \"\n +\n -+/**\n -+ * Creates a new metacommit object with the given content. Writes the object\n -+ * id of the newly-created commit to result.\n -+ */\n +int write_metacommit(struct repository *repo, struct metacommit_data *state,\n +\tstruct object_id *result)\n +{\n +\tstruct commit_list *parents = NULL;\n +\tstruct strbuf comment;\n -+\tint i;\n ++\tsize_t i;\n +\tstruct commit *content;\n +\n +\tstrbuf_init(&comment, strlen(PARENT_TYPE_PREFIX)\n @@ metacommit.c (new)\n +\t\tstrbuf_addstr(&comment, \" o\");\n +\n +\t/* The parents list will be freed by this call. */\n -+\tcommit_tree(comment.buf, comment.len, repo->hash_algo->empty_tree, parents,\n -+\t\tresult, NULL, NULL);\n ++\tcommit_tree(\n ++\t\tcomment.buf,\n ++\t\tcomment.len,\n ++\t\trepo->hash_algo->empty_tree,\n ++\t\tparents,\n ++\t\tresult,\n ++\t\tNULL,\n ++\t\tNULL);\n +\n +\tstrbuf_release(&comment);\n +\treturn 0;\n +}\n +\n -+/**\n ++/*\n + * Returns true iff the given metacommit is abandoned, has one or more origin\n + * parents, or has one or more replacement parents.\n + */\n @@ metacommit.c (new)\n + * to append to existing changes wherever possible instead of creating new ones.\n + * If override_change is non-null, only the given change ref will be updated.\n + *\n -+ * options is a bitwise combination of the UPDATE_OPTION_* flags.\n -+ */\n -+int record_metacommit(\n -+\tstruct repository *repo,\n -+\tconst struct metacommit_data *metacommit, const char *override_change,\n -+\tint options, struct strbuf *err)\n -+{\n -+\t\tstruct change_table chtable;\n -+\t\tstruct string_list changes;\n -+\t\tint result;\n -+\n -+\t\tchange_table_init(&chtable);\n -+\t\tchange_table_add_all_visible(&chtable, repo);\n -+\t\tstring_list_init_dup(&changes);\n -+\n -+\t\tresult = record_metacommit_withresult(repo, &chtable, metacommit,\n -+\t\t\toverride_change, options, err, &changes);\n -+\n -+\t\tstring_list_clear(&changes, 0);\n -+\t\tchange_table_clear(&chtable);\n -+\t\treturn result;\n -+}\n -+\n -+/*\n -+ * Records the relationships described by the given metacommit in the\n -+ * repository.\n -+ *\n -+ * If override_change is NULL (the default), an attempt will be made\n -+ * to append to existing changes wherever possible instead of creating new ones.\n -+ * If override_change is non-null, only the given change ref will be updated.\n -+ *\n + * The changes list is filled in with the list of change refs that were updated,\n + * with the util pointers pointing to the old object IDS for those changes.\n + * The object ID pointers all point to objects owned by the change_table and\n @@ metacommit.c (new)\n + *\n + * options is a bitwise combination of the UPDATE_OPTION_* flags.\n + */\n -+int record_metacommit_withresult(\n ++static int record_metacommit_withresult(\n +\tstruct repository *repo,\n +\tstruct change_table *chtable,\n +\tconst struct metacommit_data *metacommit,\n +\tconst char *override_change,\n -+\tint options, struct strbuf *err,\n ++\tint options,\n ++\tstruct strbuf *err,\n +\tstruct string_list *changes)\n +{\n +\tstatic const char *msg = \"updating change\";\n -+\tstruct metacommit_data resolved_metacommit;\n ++\tstruct metacommit_data resolved_metacommit = METACOMMIT_DATA_INIT;\n +\tstruct object_id commit_target;\n +\tstruct ref_transaction *transaction = NULL;\n +\tstruct change_head *overridden_head;\n +\tconst struct object_id *old_head;\n +\n -+\tint i;\n ++\tsize_t i;\n +\tint ret = 0;\n +\tint force = (options & UPDATE_OPTION_FORCE);\n +\n -+\tinit_metacommit_data(&resolved_metacommit);\n -+\n +\tresolve_metacommit(repo, chtable, metacommit, &resolved_metacommit, changes,\n +\t\t(options & UPDATE_OPTION_NOAPPEND) == 0);\n +\n +\tif (override_change) {\n +\t\tstring_list_clear(changes, 0);\n +\t\toverridden_head = get_change_head(chtable, override_change);\n -+\t\tif (!overridden_head) {\n ++\t\tif (overridden_head) {\n +\t\t\t/* This is an existing change */\n +\t\t\told_head = &overridden_head->head;\n +\t\t\tif (!force) {\n @@ metacommit.c (new)\n +\t\t\t/* ...then this is a newly-created change */\n +\t\t\told_head = null_oid();\n +\n -+\t\t/* The expected \"current\" head of the change is stored in the util\n -+\t\t * pointer. */\n -+\t\tstring_list_append(changes, override_change)->util = (void*)old_head;\n ++\t\t/*\n ++\t\t * The expected \"current\" head of the change is stored in the\n ++\t\t * util pointer. Cast required because old_head is const*\n ++\t\t */\n ++\t\tstring_list_append(changes, override_change)->util = (void *)old_head;\n +\t}\n +\n +\tif (is_nontrivial_metacommit(&resolved_metacommit)) {\n @@ metacommit.c (new)\n +\t\t\tret = -1;\n +\t\t\tgoto cleanup;\n +\t\t}\n -+\t} else\n -+\t\t/**\n ++\t} else {\n ++\t\t/*\n +\t\t * If the metacommit would only contain a content commit, point to the\n +\t\t * commit itself rather than creating a trivial metacommit.\n +\t\t */\n +\t\toidcpy(&commit_target, &(resolved_metacommit.content));\n ++\t}\n +\n -+\t/**\n ++\t/*\n +\t * If a change already exists with this target and we're not forcing an\n +\t * update to some specific override_change && change, there's nothing to do.\n +\t */\n @@ metacommit.c (new)\n +\t\tfor (i = 0; i < changes->nr; i++) {\n +\t\t\tstruct string_list_item *it = &changes->items[i];\n +\n -+\t\t\t/**\n ++\t\t\t/*\n +\t\t\t * The expected current head of the change is stored in the util pointer.\n +\t\t\t * It is null if the change should be newly-created.\n +\t\t\t */\n @@ metacommit.c (new)\n +\treturn ret;\n +}\n +\n -+/**\n -+ * Should be invoked after a command that has \"modify\" semantics - commands that\n -+ * create a new commit based on an old commit and treat the new one as a\n -+ * replacement for the old one. This method records the replacement in the\n -+ * change graph, such that a future evolve operation will rebase children of\n -+ * the old commit onto the new commit.\n -+ */\n ++int record_metacommit(\n ++\tstruct repository *repo,\n ++\tconst struct metacommit_data *metacommit,\n ++\tconst char *override_change,\n ++\tint options,\n ++\tstruct strbuf *err,\n ++\tstruct string_list *changes)\n ++{\n ++\t\tstruct change_table chtable;\n ++\t\tint result;\n ++\n ++\t\tchange_table_init(&chtable);\n ++\t\tchange_table_add_all_visible(&chtable, repo);\n ++\n ++\t\tresult = record_metacommit_withresult(\n ++\t\t\trepo,\n ++\t\t\t&chtable,\n ++\t\t\tmetacommit,\n ++\t\t\toverride_change,\n ++\t\t\toptions,\n ++\t\t\terr,\n ++\t\t\tchanges);\n ++\n ++\t\tchange_table_clear(&chtable);\n ++\t\treturn result;\n ++}\n ++\n +void modify_change(\n +\tstruct repository *repo,\n +\tconst struct object_id *old_commit,\n +\tconst struct object_id *new_commit,\n +\tstruct strbuf *err)\n +{\n -+\tstruct metacommit_data metacommit;\n ++\tstruct string_list changes = STRING_LIST_INIT_DUP;\n ++\tstruct metacommit_data metacommit = METACOMMIT_DATA_INIT;\n +\n -+\tinit_metacommit_data(&metacommit);\n +\toidcpy(&(metacommit.content), new_commit);\n +\toid_array_append(&(metacommit.replace), old_commit);\n +\n -+\trecord_metacommit(repo, &metacommit, NULL, 0, err);\n ++\trecord_metacommit(repo, &metacommit, NULL, 0, err, &changes);\n +\n +\tclear_metacommit_data(&metacommit);\n ++\tstring_list_clear(&changes, 0);\n +}\n \n ## metacommit.h (new) ##\n @@ metacommit.h (new)\n +#include \"repository.h\"\n +#include \"string-list.h\"\n +\n -+\n -+struct change_table;\n -+\n +/* If specified, non-fast-forward changes are permitted. */\n +#define UPDATE_OPTION_FORCE 0x0001\n +/**\n @@ metacommit.h (new)\n +\tint abandoned;\n +};\n +\n -+extern void init_metacommit_data(struct metacommit_data *state);\n ++#define METACOMMIT_DATA_INIT { 0 }\n +\n +extern void clear_metacommit_data(struct metacommit_data *state);\n +\n -+extern int record_metacommit(struct repository *repo,\n -+\tconst struct metacommit_data *metacommit,\n -+\tconst char* override_change, int options, struct strbuf *err);\n -+\n -+extern int record_metacommit_withresult(\n ++/**\n ++ * Records the relationships described by the given metacommit in the\n ++ * repository.\n ++ *\n ++ * If override_change is NULL (the default), an attempt will be made\n ++ * to append to existing changes wherever possible instead of creating new ones.\n ++ * If override_change is non-null, only the given change ref will be updated.\n ++ *\n ++ * options is a bitwise combination of the UPDATE_OPTION_* flags.\n ++ */\n ++int record_metacommit(\n +\tstruct repository *repo,\n -+\tstruct change_table *chtable,\n +\tconst struct metacommit_data *metacommit,\n -+\tconst char *override_change,\n ++\tconst char* override_change,\n +\tint options,\n +\tstruct strbuf *err,\n +\tstruct string_list *changes);\n +\n -+extern void modify_change(struct repository *repo,\n -+\tconst struct object_id *old_commit, const struct object_id *new_commit,\n ++/**\n ++ * Should be invoked after a command that has \"modify\" semantics - commands that\n ++ * create a new commit based on an old commit and treat the new one as a\n ++ * replacement for the old one. This method records the replacement in the\n ++ * change graph, such that a future evolve operation will rebase children of\n ++ * the old commit onto the new commit.\n ++ */\n ++void modify_change(\n ++\tstruct repository *repo,\n ++\tconst struct object_id *old_commit,\n ++\tconst struct object_id *new_commit,\n +\tstruct strbuf *err);\n +\n -+extern int write_metacommit(struct repository *repo, struct metacommit_data *state,\n ++/**\n ++ * Creates a new metacommit object with the given content. Writes the object\n ++ * id of the newly-created commit to result.\n ++ */\n ++int write_metacommit(\n ++\tstruct repository *repo,\n ++\tstruct metacommit_data *state,\n +\tstruct object_id *result);\n +\n +#endif\n 7: 91402834184 ! 7: f7a90700e0e evolve: implement the git change command\n @@ builtin/change.c (new)\n +#include \"ref-filter.h\"\n +#include \"parse-options.h\"\n +#include \"metacommit.h\"\n -+#include \"change-table.h\"\n +#include \"config.h\"\n +\n +static const char * const builtin_change_usage[] = {\n -+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n ++\tN_(\"git change list [...]\"),\n ++\tN_(\"git change update [--force] [--replace ...] \"\n ++\t \"[--origin ...] [--content ]\"),\n ++\tNULL\n ++};\n ++\n ++static const char * const builtin_list_usage[] = {\n ++\tN_(\"git change list [...]\"),\n +\tNULL\n +};\n +\n +static const char * const builtin_update_usage[] = {\n -+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n ++\tN_(\"git change update [--force] [--replace ...] \"\n ++\t\"[--origin ...] [--content ]\"),\n +\tNULL\n +};\n +\n ++static int change_list(int argc, const char **argv, const char* prefix)\n ++{\n ++\tstruct option options[] = {\n ++\t\tOPT_END()\n ++\t};\n ++\tstruct ref_filter filter = { 0 };\n ++\tstruct ref_sorting *sorting;\n ++\tstruct string_list sorting_options = STRING_LIST_INIT_DUP;\n ++\tstruct ref_format format = REF_FORMAT_INIT;\n ++\tstruct ref_array array = { 0 };\n ++\tsize_t i;\n ++\n ++\targc = parse_options(argc, argv, prefix, options, builtin_list_usage, 0);\n ++\n ++\tsetup_ref_filter_porcelain_msg();\n ++\n ++\tfilter.kind = FILTER_REFS_CHANGES;\n ++\tfilter.name_patterns = argv;\n ++\n ++\tfilter_refs(&array, &filter, FILTER_REFS_CHANGES);\n ++\n ++\t/* TODO: This causes a crash. It sets one of the atom_value handlers to\n ++\t * something invalid, which causes a crash later when we call\n ++\t * show_ref_array_item. Figure out why this happens and put back the sorting.\n ++\t *\n ++\t * sorting = ref_sorting_options(&sorting_options);\n ++\t * ref_array_sort(sorting, &array); */\n ++\n ++\tif (!format.format)\n ++\t\tformat.format = \"%(refname:lstrip=1)\";\n ++\n ++\tif (verify_ref_format(&format))\n ++\t\tdie(_(\"unable to parse format string\"));\n ++\n ++\tsorting = ref_sorting_options(&sorting_options);\n ++\tref_array_sort(sorting, &array);\n ++\n ++\n ++\tfor (i = 0; i < array.nr; i++) {\n ++\t\tstruct strbuf output = STRBUF_INIT;\n ++\t\tstruct strbuf err = STRBUF_INIT;\n ++\t\tif (format_ref_array_item(array.items[i], &format, &output, &err))\n ++\t\t\tdie(\"%s\", err.buf);\n ++\t\tfwrite(output.buf, 1, output.len, stdout);\n ++\t\tputchar('\\n');\n ++\n ++\t\tstrbuf_release(&err);\n ++\t\tstrbuf_release(&output);\n ++\t}\n ++\n ++\tref_array_clear(&array);\n ++\tref_sorting_release(sorting);\n ++\n ++\treturn 0;\n ++}\n ++\n +struct update_state {\n +\tint options;\n +\tconst char* change;\n @@ builtin/change.c (new)\n +\tstruct string_list origin;\n +};\n +\n -+static void init_update_state(struct update_state *state)\n -+{\n -+\tmemset(state, 0, sizeof(*state));\n -+\tstate->content = \"HEAD\";\n -+\tstring_list_init_nodup(&state->replace);\n -+\tstring_list_init_nodup(&state->origin);\n ++#define UPDATE_STATE_INIT { \\\n ++\t.content = \"HEAD\", \\\n ++\t.replace = STRING_LIST_INIT_NODUP, \\\n ++\t.origin = STRING_LIST_INIT_NODUP \\\n +}\n +\n +static void clear_update_state(struct update_state *state)\n @@ builtin/change.c (new)\n +{\n +\tstruct commit *commit;\n +\tif (get_oid_committish(committish, result))\n -+\t\tdie(_(\"Failed to resolve '%s' as a valid revision.\"), committish);\n ++\t\tdie(_(\"failed to resolve '%s' as a valid revision.\"), committish);\n +\tcommit = lookup_commit_reference(the_repository, result);\n +\tif (!commit)\n -+\t\tdie(_(\"Could not parse object '%s'.\"), committish);\n ++\t\tdie(_(\"could not parse object '%s'.\"), committish);\n +\toidcpy(result, &commit->object.oid);\n +\treturn 0;\n +}\n @@ builtin/change.c (new)\n +static void resolve_commit_list(const struct string_list *commitsish_list,\n +\tstruct oid_array* result)\n +{\n -+\tint i;\n -+\tfor (i = 0; i < commitsish_list->nr; i++) {\n -+\t\tstruct string_list_item *item = &commitsish_list->items[i];\n ++\tstruct string_list_item *item;\n ++\n ++\tfor_each_string_list_item(item, commitsish_list) {\n +\t\tstruct object_id next;\n +\t\tresolve_commit(item->string, &next);\n +\t\toid_array_append(result, &next);\n @@ builtin/change.c (new)\n +\tconst struct update_state *state,\n +\tstruct strbuf *err)\n +{\n -+\tstruct metacommit_data metacommit;\n -+\tstruct change_table chtable;\n -+\tstruct string_list changes;\n ++\tstruct metacommit_data metacommit = METACOMMIT_DATA_INIT;\n ++\tstruct string_list changes = STRING_LIST_INIT_DUP;\n +\tint ret;\n -+\tint i;\n -+\n -+\tchange_table_init(&chtable);\n -+\tchange_table_add_all_visible(&chtable, repo);\n -+\tstring_list_init_dup(&changes);\n -+\n -+\tinit_metacommit_data(&metacommit);\n ++\tstruct string_list_item *item;\n +\n +\tget_metacommit_from_command_line(state, &metacommit);\n +\n -+\tret = record_metacommit_withresult(repo, &chtable, &metacommit,\n -+\t\tstate->change, state->options, err, &changes);\n ++\tret = record_metacommit(\n ++\t\trepo,\n ++\t\t&metacommit,\n ++\t\tstate->change,\n ++\t\tstate->options,\n ++\t\terr,\n ++\t\t&changes);\n +\n -+\tfor (i = 0; i < changes.nr; i++) {\n -+\t\tstruct string_list_item *it = &changes.items[i];\n ++\tfor_each_string_list_item(item, &changes) {\n +\n -+\t\tconst char* name = lstrip_ref_components(it->string, 1);\n ++\t\tconst char* name = lstrip_ref_components(item->string, 1);\n +\t\tif (!name)\n -+\t\t\tdie(_(\"Failed to remove `refs/` from %s\"), it->string);\n ++\t\t\tdie(_(\"failed to remove `refs/` from %s\"), item->string);\n +\n -+\t\tif (it->util)\n -+\t\t\tfprintf(stdout, N_(\"Updated change %s\\n\"), name);\n ++\t\tif (item->util)\n ++\t\t\tfprintf(stdout, _(\"Updated change %s\"), name);\n +\t\telse\n -+\t\t\tfprintf(stdout, N_(\"Created change %s\\n\"), name);\n ++\t\t\tfprintf(stdout, _(\"Created change %s\"), name);\n ++\t\tputchar('\\n');\n +\t}\n +\n +\tstring_list_clear(&changes, 0);\n -+\tchange_table_clear(&chtable);\n +\tclear_metacommit_data(&metacommit);\n +\n +\treturn ret;\n @@ builtin/change.c (new)\n +static int change_update(int argc, const char **argv, const char* prefix)\n +{\n +\tint result;\n -+\tint force = 0;\n -+\tint newchange = 0;\n +\tstruct strbuf err = STRBUF_INIT;\n -+\tstruct update_state state;\n ++\tstruct update_state state = UPDATE_STATE_INIT;\n +\tstruct option options[] = {\n +\t\t{ OPTION_CALLBACK, 'r', \"replace\", &state, N_(\"commit\"),\n +\t\t\tN_(\"marks the given commit as being obsolete\"),\n @@ builtin/change.c (new)\n +\t\t{ OPTION_CALLBACK, 'o', \"origin\", &state, N_(\"commit\"),\n +\t\t\tN_(\"marks the given commit as being the origin of this commit\"),\n +\t\t\t0, update_option_parse_origin },\n -+\t\tOPT_BOOL('F', \"force\", &force,\n -+\t\t\tN_(\"overwrite an existing change of the same name\")),\n ++\n +\t\tOPT_STRING('c', \"content\", &state.content, N_(\"commit\"),\n +\t\t\t\t N_(\"identifies the new content commit for the change\")),\n +\t\tOPT_STRING('g', \"change\", &state.change, N_(\"commit\"),\n +\t\t\t\t N_(\"name of the change to update\")),\n -+\t\tOPT_BOOL('n', \"new\", &newchange,\n -+\t\t\tN_(\"create a new change - do not append to any existing change\")),\n ++\t\tOPT_SET_INT_F('n', \"new\", &state.options,\n ++\t\t\t N_(\"create a new change - do not append to any existing change\"),\n ++\t\t\t UPDATE_OPTION_NOAPPEND, 0),\n ++\t\tOPT_SET_INT_F('F', \"force\", &state.options,\n ++\t\t\t N_(\"overwrite an existing change of the same name\"),\n ++\t\t\t UPDATE_OPTION_FORCE, 0),\n +\t\tOPT_END()\n +\t};\n +\n -+\tinit_update_state(&state);\n -+\n +\targc = parse_options(argc, argv, prefix, options, builtin_update_usage, 0);\n -+\n -+\tif (force) state.options |= UPDATE_OPTION_FORCE;\n -+\tif (newchange) state.options |= UPDATE_OPTION_NOAPPEND;\n -+\n +\tresult = perform_update(the_repository, &state, &err);\n +\n +\tif (result < 0) {\n @@ builtin/change.c (new)\n +\n +int cmd_change(int argc, const char **argv, const char *prefix)\n +{\n ++\tparse_opt_subcommand_fn *fn = NULL;\n +\t/* No options permitted before subcommand currently */\n +\tstruct option options[] = {\n ++\t\tOPT_SUBCOMMAND(\"list\", &fn, change_list),\n ++\t\tOPT_SUBCOMMAND(\"update\", &fn, change_update),\n +\t\tOPT_END()\n +\t};\n -+\tint result = 1;\n +\n +\targc = parse_options(argc, argv, prefix, options, builtin_change_usage,\n -+\t\tPARSE_OPT_STOP_AT_NON_OPTION);\n -+\n -+\tif (argc < 1)\n -+\t\tusage_with_options(builtin_change_usage, options);\n -+\telse if (!strcmp(argv[0], \"update\"))\n -+\t\tresult = change_update(argc, argv, prefix);\n -+\telse {\n -+\t\terror(_(\"Unknown subcommand: %s\"), argv[0]);\n -+\t\tusage_with_options(builtin_change_usage, options);\n ++\t\tPARSE_OPT_SUBCOMMAND_OPTIONAL);\n ++\n ++\tif (!fn) {\n ++\t\tif (argc) {\n ++\t\t\terror(_(\"unknown subcommand: `%s'\"), argv[0]);\n ++\t\t\tusage_with_options(builtin_change_usage, options);\n ++\t\t}\n ++\t\tfn = change_list;\n +\t}\n +\n -+\treturn result ? 1 : 0;\n ++\treturn !!fn(argc, argv, prefix);\n +}\n \n ## git.c ##\n 9: d087d467e3f ! 8: a0669fa63a1 evolve: add delete command\n @@ Commit message\n \n ## builtin/change.c ##\n @@\n + #include \"parse-options.h\"\n #include \"metacommit.h\"\n - #include \"change-table.h\"\n #include \"config.h\"\n +#include \"refs.h\"\n \n static const char * const builtin_change_usage[] = {\n \tN_(\"git change list [...]\"),\n --\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n -+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n + \tN_(\"git change update [--force] [--replace ...] \"\n + \t \"[--origin ...] [--content ]\"),\n +\tN_(\"git change delete ...\"),\n \tNULL\n };\n \n -@@ builtin/change.c: static const char * const builtin_list_usage[] = {\n +@@ builtin/change.c: static const char * const builtin_update_usage[] = {\n + \tNULL\n };\n \n - static const char * const builtin_update_usage[] = {\n --\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n -+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n ++static const char * const builtin_delete_usage[] = {\n ++\tN_(\"git change delete ...\"),\n +\tNULL\n +};\n +\n -+static const char * const builtin_delete_usage[] = {\n -+\tN_(\"git change delete ...\"),\n - \tNULL\n - };\n - \n + static int change_list(int argc, const char **argv, const char* prefix)\n + {\n + \tstruct option options[] = {\n @@ builtin/change.c: static int change_update(int argc, const char **argv, const char* prefix)\n \treturn result;\n }\n @@ builtin/change.c: static int change_update(int argc, const char **argv, const ch\n +\n int cmd_change(int argc, const char **argv, const char *prefix)\n {\n - \t/* No options permitted before subcommand currently */\n + \tparse_opt_subcommand_fn *fn = NULL;\n @@ builtin/change.c: int cmd_change(int argc, const char **argv, const char *prefix)\n - \t\tresult = change_list(argc, argv, prefix);\n - \telse if (!strcmp(argv[0], \"update\"))\n - \t\tresult = change_update(argc, argv, prefix);\n -+\telse if (!strcmp(argv[0], \"delete\"))\n -+\t\tresult = change_delete(argc, argv, prefix);\n - \telse {\n - \t\terror(_(\"Unknown subcommand: %s\"), argv[0]);\n - \t\tusage_with_options(builtin_change_usage, options);\n + \tstruct option options[] = {\n + \t\tOPT_SUBCOMMAND(\"list\", &fn, change_list),\n + \t\tOPT_SUBCOMMAND(\"update\", &fn, change_update),\n ++\t\tOPT_SUBCOMMAND(\"delete\", &fn, change_delete),\n + \t\tOPT_END()\n + \t};\n + \n 10: 811d516e5d2 = 9: e67ff668fff evolve: add documentation for `git change`\n 8: b83a79beeb4 ! 10: 37042b58cda evolve: add the git change list command\n @@\n ## Metadata ##\n -Author: Stefan Xenos \n +Author: Chris Poucet \n \n ## Commit message ##\n - evolve: add the git change list command\n + evolve: add tests for the git-change command\n \n - This command lists the ongoing changes from the refs/metas\n - namespace.\n -\n - Signed-off-by: Stefan Xenos \n + Signed-off-by: Phillip Wood \n Signed-off-by: Chris Poucet \n \n - ## builtin/change.c ##\n + ## t/t9990-changes.sh (new) ##\n @@\n - #include \"config.h\"\n - \n - static const char * const builtin_change_usage[] = {\n -+\tN_(\"git change list [...]\"),\n - \tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n - \tNULL\n - };\n - \n -+static const char * const builtin_list_usage[] = {\n -+\tN_(\"git change list [...]\"),\n -+\tNULL\n -+};\n ++#!/bin/sh\n +\n - static const char * const builtin_update_usage[] = {\n - \tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n - \tNULL\n - };\n - \n -+static int change_list(int argc, const char **argv, const char* prefix)\n -+{\n -+\tstruct option options[] = {\n -+\t\tOPT_END()\n -+\t};\n -+\tstruct ref_filter filter;\n -+\t/* TODO: See below\n -+\tstruct ref_sorting *sorting;\n -+\tstruct string_list sorting_options = STRING_LIST_INIT_DUP; */\n -+\tstruct ref_format format = REF_FORMAT_INIT;\n -+\tstruct ref_array array;\n -+\tint i;\n ++test_description='git change - low level meta-commit management'\n +\n -+\targc = parse_options(argc, argv, prefix, options, builtin_list_usage, 0);\n ++. ./test-lib.sh\n +\n -+\tsetup_ref_filter_porcelain_msg();\n ++. \"$TEST_DIRECTORY\"/lib-rebase.sh\n +\n -+\tmemset(&filter, 0, sizeof(filter));\n -+\tmemset(&array, 0, sizeof(array));\n ++test_expect_success 'setup commits and meta-commits' '\n ++ for c in one two three\n ++ do\n ++ test_commit $c &&\n ++ git change update --content $c >actual 2>err &&\n ++ echo \"Created change metas/$c\" >expect &&\n ++ test_cmp expect actual &&\n ++ test_must_be_empty err &&\n ++ test_cmp_rev refs/metas/$c $c || return 1\n ++ done\n ++'\n +\n -+\tfilter.kind = FILTER_REFS_CHANGES;\n -+\tfilter.name_patterns = argv;\n ++# Check a meta-commit has the correct parents Call with the object\n ++# name of the meta-commit followed by pairs of type and parent\n ++check_meta_commit () {\n ++ name=$1\n ++ shift\n ++ while test $# -gt 0\n ++ do\n ++ printf '%s %s\\n' $1 $(git rev-parse --verify $2)\n ++ shift\n ++ shift\n ++ done | sort >expect\n ++ git cat-file commit $name >metacommit &&\n ++ # commit body should consist of parent-type\n ++ types=\"$(sed -n '/^$/ {\n ++ :loop\n ++ n\n ++ s/^parent-type //\n ++ p\n ++ b loop\n ++ }' metacommit)\" &&\n ++ while read key value\n ++ do\n ++ # TODO: don't sort the first parent\n ++ if test \"$key\" = \"parent\"\n ++ then\n ++ type=\"${types%% *}\"\n ++ test -n \"$type\" || return 1\n ++ printf '%s %s\\n' $type $value\n ++ types=\"${types#?}\"\n ++ types=\"${types# }\"\n ++ elif test \"$key\" = \"tree\"\n ++ then\n ++ test_cmp_rev \"$value\" $EMPTY_TREE || return 1\n ++ elif test -z \"$key\"\n ++ then\n ++ # only parse commit headers\n ++ break\n ++ fi\n ++ done actual-unsorted &&\n ++ test -z \"$types\" &&\n ++ sort >actual out 2>err &&\n ++ echo \"Updated change metas/one\" >expect &&\n ++ test_cmp expect out &&\n ++ test_must_be_empty err &&\n ++ git change update --replace tags/two --content HEAD@{2} &&\n ++ oid=$(git rev-parse --verify metas/two) &&\n ++ git change update --replace HEAD@{2} --replace tags/three \\\n ++ --content HEAD &&\n +\n -+\tif (!format.format)\n -+\t\tformat.format = \"%(refname:lstrip=1)\";\n ++ # check meta-commits\n ++ check_meta_commit metas/one c HEAD~1 r tags/one &&\n ++ check_meta_commit $oid c HEAD@{2} r tags/two &&\n ++ # NB this checks that \"git change update\" uses the meta-commit ($oid)\n ++ # corresponding to the replaces commit (HEAD@2 above) given on the\n ++ # commandline.\n ++ check_meta_commit metas/two c HEAD r $oid r tags/three &&\n ++ check_meta_commit metas/three c HEAD r $oid r tags/three\n ++'\n +\n -+\tif (verify_ref_format(&format))\n -+\t\tdie(_(\"unable to parse format string\"));\n ++reset_meta_commits () {\n ++ for c in one two three\n ++ do\n ++ echo \"update refs/metas/$c refs/tags/$c^0\"\n ++ done | git update-ref --stdin\n ++}\n +\n -+\tfor (i = 0; i < array.nr; i++) {\n -+\t\tstruct strbuf output = STRBUF_INIT;\n -+\t\tstruct strbuf err = STRBUF_INIT;\n -+\t\tif (format_ref_array_item(array.items[i], &format, &output, &err))\n -+\t\t\tdie(\"%s\", err.buf);\n -+\t\tfwrite(output.buf, 1, output.len, stdout);\n -+\t\tputchar('\\n');\n ++test_expect_success 'override change name' '\n ++ # TODO: builtin/change.c expects --change to be the full refname,\n ++ # ideally it would prepend refs/metas to the string given by the\n ++ # user.\n ++ git change update --change refs/metas/another-one --content one &&\n ++ test_cmp_rev metas/another-one one\n ++'\n +\n -+\t\tstrbuf_release(&err);\n -+\t\tstrbuf_release(&output);\n -+\t}\n ++test_expect_success 'non-fast forward meta-commit update refused' '\n ++ test_must_fail git change update --change refs/metas/one --content two \\\n ++ >out 2>err &&\n ++ echo \"error: non-fast-forward update to ${SQ}refs/metas/one${SQ}\" \\\n ++ >expect &&\n ++ test_cmp expect err &&\n ++ test_must_be_empty out\n ++'\n +\n -+\tref_array_clear(&array);\n -+\t/* TODO: see above\n -+\tref_sorting_release(sorting); */\n ++test_expect_success 'forced non-fast forward update succeeds' '\n ++ git change update --change refs/metas/one --content two --force \\\n ++ >out 2>err &&\n ++ echo \"Updated change metas/one\" >expect &&\n ++ test_cmp expect out &&\n ++ test_must_be_empty err\n ++'\n +\n -+\treturn 0;\n -+}\n ++test_expect_success 'list changes' '\n ++ cat >expect <<-\\EOF &&\n ++metas/another-one\n ++metas/one\n ++metas/three\n ++metas/two\n ++EOF\n ++ git change list >actual &&\n ++ test_cmp expect actual\n ++'\n ++\n ++test_expect_success 'delete change' '\n ++ git change delete metas/one &&\n ++ cat >expect <<-\\EOF &&\n ++metas/another-one\n ++metas/three\n ++metas/two\n ++EOF\n ++ git change list >actual &&\n ++ test_cmp expect actual\n ++'\n +\n - struct update_state {\n - \tint options;\n - \tconst char* change;\n -@@ builtin/change.c: int cmd_change(int argc, const char **argv, const char *prefix)\n - \n - \tif (argc < 1)\n - \t\tusage_with_options(builtin_change_usage, options);\n -+\telse if (!strcmp(argv[0], \"list\"))\n -+\t\tresult = change_list(argc, argv, prefix);\n - \telse if (!strcmp(argv[0], \"update\"))\n - \t\tresult = change_update(argc, argv, prefix);\n - \telse {\n ++test_done\n\n-- \ngitgitgadget\n"},{"id":"464230","messageId":"c59066ebc102301817659d5e886a310bd4d84271.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 03/10] ref-filter: add the metas namespace to ref-filter","fromName":"Chris Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:10Z","receivedAt":"2022-10-05T15:00:54Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"From: Chris Poucet \n\nThe metas namespace will contain refs for changes in progress. Add\nsupport for searching this namespace.\n\nSigned-off-by: Chris Poucet \n---\n ref-filter.c | 8 ++++++--\n ref-filter.h | 6 ++++--\n 2 files changed, 10 insertions(+), 4 deletions(-)\n\ndiff --git a/ref-filter.c b/ref-filter.c\nindex fd1cb14b0f1..6a1789c623f 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -2200,7 +2200,8 @@ static int ref_kind_from_refname(const char *refname)\n \t} ref_kind[] = {\n \t\t{ \"refs/heads/\" , FILTER_REFS_BRANCHES },\n \t\t{ \"refs/remotes/\" , FILTER_REFS_REMOTES },\n-\t\t{ \"refs/tags/\", FILTER_REFS_TAGS}\n+\t\t{ \"refs/tags/\", FILTER_REFS_TAGS},\n+\t\t{ \"refs/metas/\", FILTER_REFS_CHANGES }\n \t};\n \n \tif (!strcmp(refname, \"HEAD\"))\n@@ -2218,7 +2219,8 @@ static int filter_ref_kind(struct ref_filter *filter, const char *refname)\n {\n \tif (filter->kind == FILTER_REFS_BRANCHES ||\n \t filter->kind == FILTER_REFS_REMOTES ||\n-\t filter->kind == FILTER_REFS_TAGS)\n+\t filter->kind == FILTER_REFS_TAGS ||\n+\t filter->kind == FILTER_REFS_CHANGES)\n \t\treturn filter->kind;\n \treturn ref_kind_from_refname(refname);\n }\n@@ -2435,6 +2437,8 @@ int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int\n \t\t\tret = for_each_fullref_in(\"refs/remotes/\", ref_filter_handler, &ref_cbdata);\n \t\telse if (filter->kind == FILTER_REFS_TAGS)\n \t\t\tret = for_each_fullref_in(\"refs/tags/\", ref_filter_handler, &ref_cbdata);\n+\t\telse if (filter->kind == FILTER_REFS_CHANGES)\n+\t\t\tret = for_each_fullref_in(\"refs/metas/\", ref_filter_handler, &ref_cbdata);\n \t\telse if (filter->kind & FILTER_REFS_ALL)\n \t\t\tret = for_each_fullref_in_pattern(filter, ref_filter_handler, &ref_cbdata);\n \t\tif (!ret && (filter->kind & FILTER_REFS_DETACHED_HEAD))\ndiff --git a/ref-filter.h b/ref-filter.h\nindex aa0eea4ecf5..db3ee44e4dc 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -16,9 +16,11 @@\n #define FILTER_REFS_TAGS 0x0002\n #define FILTER_REFS_BRANCHES 0x0004\n #define FILTER_REFS_REMOTES 0x0008\n-#define FILTER_REFS_OTHERS 0x0010\n+#define FILTER_REFS_CHANGES 0x0010\n+#define FILTER_REFS_OTHERS 0x0040\n #define FILTER_REFS_ALL (FILTER_REFS_TAGS | FILTER_REFS_BRANCHES | \\\n-\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS)\n+\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS | \\\n+\t\t\t\t FILTER_REFS_CHANGES)\n #define FILTER_REFS_DETACHED_HEAD 0x0020\n #define FILTER_REFS_KIND_MASK (FILTER_REFS_ALL | FILTER_REFS_DETACHED_HEAD)\n \n-- \ngitgitgadget\n\n"},{"id":"464232","messageId":"ed5106d6080363ae47e1126dd9717c8772302a6e.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 02/10] sha1-array: implement oid_array_readonly_contains","fromName":"Chris Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:09Z","receivedAt":"2022-10-05T15:00:56Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"From: Chris Poucet \n\nImplement a \"readonly_contains\" function for oid_array that won't\nsort the array if it is unsorted. This can be used to test containment in\nthe rare situations where the array order matters.\n\nThe function has intentionally been given a name that is more cumbersome\nthan the \"lookup\" function, which is what most callers will will want\nin most situations.\n\nSigned-off-by: Chris Poucet \n---\n oid-array.c | 12 ++++++++++++\n oid-array.h | 7 +++++++\n t/helper/test-oid-array.c | 6 ++++++\n t/t0064-oid-array.sh | 22 ++++++++++++++++++++++\n 4 files changed, 47 insertions(+)\n\ndiff --git a/oid-array.c b/oid-array.c\nindex 73ba76e9e9a..1e12651d245 100644\n--- a/oid-array.c\n+++ b/oid-array.c\n@@ -28,6 +28,18 @@ static const struct object_id *oid_access(size_t index, const void *table)\n \treturn &array[index];\n }\n \n+int oid_array_readonly_contains(const struct oid_array *array,\n+\t\t\t\tconst struct object_id* oid) {\n+\tint i;\n+\n+\tif (array->sorted)\n+\t\treturn oid_pos(oid, array->oid, array->nr, oid_access) >= 0;\n+\tfor (i = 0; i < array->nr; i++)\n+\t\tif (oideq(&array->oid[i], oid))\n+\t\t\treturn 1;\n+\treturn 0;\n+}\n+\n int oid_array_lookup(struct oid_array *array, const struct object_id *oid)\n {\n \toid_array_sort(array);\ndiff --git a/oid-array.h b/oid-array.h\nindex f60f9af6741..e056eb61fa2 100644\n--- a/oid-array.h\n+++ b/oid-array.h\n@@ -58,6 +58,13 @@ struct oid_array {\n \n #define OID_ARRAY_INIT { 0 }\n \n+/**\n+ * Sees whether an array contains an object ID. Optimized for when the array is\n+ * sorted but does not require the array to be sorted.\n+ */\n+int oid_array_readonly_contains(const struct oid_array *array,\n+\t\t\t\tconst struct object_id* oid);\n+\n /**\n * Add an item to the set. The object ID will be placed at the end of the array\n * (but note that some operations below may lose this ordering).\ndiff --git a/t/helper/test-oid-array.c b/t/helper/test-oid-array.c\nindex d1324d086a2..0dbfc91ca8d 100644\n--- a/t/helper/test-oid-array.c\n+++ b/t/helper/test-oid-array.c\n@@ -28,10 +28,16 @@ int cmd__oid_array(int argc, const char **argv)\n \t\t\tif (get_oid_hex(arg, &oid))\n \t\t\t\tdie(\"not a hexadecimal oid: %s\", arg);\n \t\t\tprintf(\"%d\\n\", oid_array_lookup(&array, &oid));\n+\t\t} else if (skip_prefix(line.buf, \"readonly_contains \", &arg)) {\n+\t\t\tif (get_oid_hex(arg, &oid))\n+\t\t\t\tdie(\"not a hexadecimal oid: %s\", arg);\n+\t\t\tprintf(\"%d\\n\", oid_array_readonly_contains(&array, &oid));\n \t\t} else if (!strcmp(line.buf, \"clear\"))\n \t\t\toid_array_clear(&array);\n \t\telse if (!strcmp(line.buf, \"for_each_unique\"))\n \t\t\toid_array_for_each_unique(&array, print_oid, NULL);\n+\t\telse if (!strcmp(line.buf, \"for_each\"))\n+\t\t\toid_array_for_each(&array, print_oid, NULL);\n \t\telse\n \t\t\tdie(\"unknown command: %s\", line.buf);\n \t}\ndiff --git a/t/t0064-oid-array.sh b/t/t0064-oid-array.sh\nindex 88c89e8f48a..aa677af132d 100755\n--- a/t/t0064-oid-array.sh\n+++ b/t/t0064-oid-array.sh\n@@ -35,6 +35,28 @@ test_expect_success 'ordered enumeration with duplicate suppression' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'readonly_contains finds existing' '\n+\techo 1 >expect &&\n+\techoid \"\" 88 44 aa 55 >>expect &&\n+\t{\n+\t\techoid append 88 44 aa 55 &&\n+\t\techoid readonly_contains 55 &&\n+\t\techo for_each\n+\t} | test-tool oid-array >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'readonly_contains non-existing query' '\n+\techo 0 >expect &&\n+\techoid \"\" 88 44 aa 55 >>expect &&\n+\t{\n+\t\techoid append 88 44 aa 55 &&\n+\t\techoid readonly_contains 33 &&\n+\t\techo for_each\n+\t} | test-tool oid-array >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'lookup' '\n \t{\n \t\techoid append 88 44 aa 55 &&\n-- \ngitgitgadget\n\n"},{"id":"464233","messageId":"48cd92d35ef40b523255ea3b0470a3873a251933.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 05/10] evolve: add the change-table structure","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:12Z","receivedAt":"2022-10-05T15:01:00Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nA change table stores a list of changes, and supports efficient lookup\nfrom a commit hash to the list of changes that reference that commit\ndirectly.\n\nIt can be used to look up content commits or metacommits at the head\nof a change, but does not support lookup of commits referenced as part\nof the commit history.\n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n Makefile | 1 +\n change-table.c | 164 +++++++++++++++++++++++++++++++++++++++++++++++++\n change-table.h | 122 ++++++++++++++++++++++++++++++++++++\n 3 files changed, 287 insertions(+)\n create mode 100644 change-table.c\n create mode 100644 change-table.h\n\ndiff --git a/Makefile b/Makefile\nindex b2bcc00c289..2b847e7e7de 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -913,6 +913,7 @@ LIB_OBJS += bulk-checkin.o\n LIB_OBJS += bundle-uri.o\n LIB_OBJS += bundle.o\n LIB_OBJS += cache-tree.o\n+LIB_OBJS += change-table.o\n LIB_OBJS += cbtree.o\n LIB_OBJS += chdir-notify.o\n LIB_OBJS += checkout.o\ndiff --git a/change-table.c b/change-table.c\nnew file mode 100644\nindex 00000000000..1d3d64b36d8\n--- /dev/null\n+++ b/change-table.c\n@@ -0,0 +1,164 @@\n+#include \"cache.h\"\n+#include \"change-table.h\"\n+#include \"commit.h\"\n+#include \"ref-filter.h\"\n+#include \"metacommit-parser.h\"\n+\n+void change_table_init(struct change_table *table)\n+{\n+\tmemset(table, 0, sizeof(*table));\n+\tmem_pool_init(&table->memory_pool, 0);\n+\toidmap_init(&table->oid_to_metadata_index, 0);\n+\tstrmap_init(&table->refname_to_change_head);\n+}\n+\n+static void change_list_clear(struct change_list *change_list) {\n+\tstrset_clear(&change_list->refnames);\n+}\n+\n+static void commit_change_list_entry_clear(\n+\tstruct commit_change_list_entry *entry) {\n+\tchange_list_clear(&entry->changes);\n+}\n+\n+void change_table_clear(struct change_table *table)\n+{\n+\tstruct oidmap_iter iter;\n+\tstruct commit_change_list_entry *next;\n+\tfor (next = oidmap_iter_first(&table->oid_to_metadata_index, &iter);\n+\t\tnext;\n+\t\tnext = oidmap_iter_next(&iter)) {\n+\n+\t\tcommit_change_list_entry_clear(next);\n+\t}\n+\n+\toidmap_free(&table->oid_to_metadata_index, 0);\n+\tstrmap_clear(&table->refname_to_change_head, 0);\n+\tmem_pool_discard(&table->memory_pool, 0);\n+}\n+\n+static void add_head_to_commit(struct change_table *table,\n+\t\t\t const struct object_id *to_add,\n+\t\t\t const char *refname)\n+{\n+\tstruct commit_change_list_entry *entry;\n+\n+\tentry = oidmap_get(&table->oid_to_metadata_index, to_add);\n+\tif (!entry) {\n+\t\tentry = mem_pool_calloc(&table->memory_pool, 1, sizeof(*entry));\n+\t\toidcpy(&entry->entry.oid, to_add);\n+\t\tstrset_init(&entry->changes.refnames);\n+\t\toidmap_put(&table->oid_to_metadata_index, entry);\n+\t}\n+\tstrset_add(&entry->changes.refnames, refname);\n+}\n+\n+void change_table_add(struct change_table *table,\n+\t\t const char *refname,\n+\t\t struct commit *to_add)\n+{\n+\tstruct change_head *new_head;\n+\tint metacommit_type;\n+\n+\tnew_head = mem_pool_calloc(&table->memory_pool, 1, sizeof(*new_head));\n+\n+\toidcpy(&new_head->head, &to_add->object.oid);\n+\n+\tmetacommit_type = get_metacommit_content(to_add, &new_head->content);\n+\t/* If to_add is not a metacommit then the content is to_add itself,\n+\t * otherwise it will have been set by the call to\n+\t * get_metacommit_content.\n+\t */\n+\tif (metacommit_type == METACOMMIT_TYPE_NONE)\n+\t\toidcpy(&new_head->content, &to_add->object.oid);\n+\tnew_head->abandoned = (metacommit_type == METACOMMIT_TYPE_ABANDONED);\n+\tnew_head->remote = starts_with(refname, \"refs/remote/\");\n+\tnew_head->hidden = starts_with(refname, \"refs/hiddenmetas/\");\n+\n+\tstrmap_put(&table->refname_to_change_head, refname, new_head);\n+\n+\tif (!oideq(&new_head->content, &new_head->head)) {\n+\t\t/* We also remember to link between refname and the content oid */\n+\t\tadd_head_to_commit(table, &new_head->content, refname);\n+\t}\n+\tadd_head_to_commit(table, &new_head->head, refname);\n+}\n+\n+static void change_table_add_matching_filter(struct change_table *table,\n+\t\t\t\t\t struct repository* repo,\n+\t\t\t\t\t struct ref_filter *filter)\n+{\n+\tint i;\n+\tstruct ref_array matching_refs = { 0 };\n+\n+\tfilter_refs(&matching_refs, filter, filter->kind);\n+\n+\t/*\n+\t * Determine the object id for the latest content commit for each change.\n+\t * Fetch the commit at the head of each change ref. If it's a normal commit,\n+\t * that's the commit we want. If it's a metacommit, locate its content parent\n+\t * and use that.\n+\t */\n+\n+\tfor (i = 0; i < matching_refs.nr; i++) {\n+\t\tstruct ref_array_item *item = matching_refs.items[i];\n+\t\tstruct commit *commit;\n+\n+\t\tcommit = lookup_commit_reference(repo, &item->objectname);\n+\t\tif (!commit) {\n+\t\t\tBUG(\"Invalid commit for refs/meta: %s\", item->refname);\n+\t\t}\n+\t\tchange_table_add(table, item->refname, commit);\n+\t}\n+\n+\tref_array_clear(&matching_refs);\n+}\n+\n+void change_table_add_all_visible(struct change_table *table,\n+\tstruct repository* repo)\n+{\n+\tstruct ref_filter filter = { 0 };\n+\tconst char *name_patterns[] = {NULL};\n+\tfilter.kind = FILTER_REFS_CHANGES;\n+\tfilter.name_patterns = name_patterns;\n+\n+\tchange_table_add_matching_filter(table, repo, &filter);\n+}\n+\n+static int return_true_callback(const char *refname, void *cb_data)\n+{\n+\treturn 1;\n+}\n+\n+int change_table_has_change_referencing(struct change_table *table,\n+\tconst struct object_id *referenced_commit_id)\n+{\n+\treturn for_each_change_referencing(table, referenced_commit_id,\n+\t\treturn_true_callback, NULL);\n+}\n+\n+int for_each_change_referencing(struct change_table *table,\n+\tconst struct object_id *referenced_commit_id, each_change_fn fn, void *cb_data)\n+{\n+\tint ret;\n+\tstruct commit_change_list_entry *ccl_entry;\n+\tstruct hashmap_iter iter;\n+\tstruct strmap_entry *entry;\n+\n+\tccl_entry = oidmap_get(&table->oid_to_metadata_index,\n+\t\t\t referenced_commit_id);\n+\t/* If this commit isn't referenced by any changes, it won't be in the map */\n+\tif (!ccl_entry)\n+\t\treturn 0;\n+\tstrset_for_each_entry(&ccl_entry->changes.refnames, &iter, entry) {\n+\t\tret = fn(entry->key, cb_data);\n+\t\tif (ret != 0) break;\n+\t}\n+\treturn ret;\n+}\n+\n+struct change_head* get_change_head(struct change_table *table,\n+\tconst char* refname)\n+{\n+\treturn strmap_get(&table->refname_to_change_head, refname);\n+}\ndiff --git a/change-table.h b/change-table.h\nnew file mode 100644\nindex 00000000000..85c2fb80d18\n--- /dev/null\n+++ b/change-table.h\n@@ -0,0 +1,122 @@\n+#ifndef CHANGE_TABLE_H\n+#define CHANGE_TABLE_H\n+\n+#include \"oidmap.h\"\n+#include \"strmap.h\"\n+\n+struct commit;\n+struct ref_filter;\n+\n+/**\n+ * This struct holds a set of change refs.\n+ */\n+struct change_list {\n+\t/**\n+\t * The refnames in this set.\n+\t * This field is private. Use for_each_change_in to read.\n+\t */\n+\tstruct strset refnames;\n+};\n+\n+/**\n+ * Holds information about the head of a single change.\n+ */\n+struct change_head {\n+\t/**\n+\t * The location pointed to by the head of the change. May be a commit or a\n+\t * metacommit.\n+\t */\n+\tstruct object_id head;\n+\t/**\n+\t * The content commit for the latest commit in the change. Always points to a\n+\t * real commit, never a metacommit.\n+\t */\n+\tstruct object_id content;\n+\t/**\n+\t * Abandoned: indicates that the content commit should be removed from the\n+\t * history.\n+\t *\n+\t * Hidden: indicates that the change is an inactive change from the\n+\t * hiddenmetas namespace. Such changes will be hidden from the user by\n+\t * default.\n+\t *\n+\t * Deleted: indicates that the change has been removed from the repository.\n+\t * That is the ref was deleted since the time this struct was created. Such\n+\t * entries should be ignored.\n+\t */\n+\tunsigned int abandoned:1,\n+\t\thidden:1,\n+\t\tremote:1,\n+\t\tdeleted:1;\n+};\n+\n+/**\n+ * Holds the list of change refs whose content points to a particular content\n+ * commit.\n+ */\n+struct commit_change_list_entry {\n+\tstruct oidmap_entry entry;\n+\tstruct change_list changes;\n+};\n+\n+/**\n+ * Holds information about the heads of each change, and permits efficient\n+ * lookup from a commit to the changes that reference it directly.\n+ *\n+ * All fields should be considered private. Use the change_table functions\n+ * to interact with this struct.\n+ */\n+struct change_table {\n+\t/**\n+\t * Memory pool for the objects allocated by the change table.\n+\t */\n+\tstruct mem_pool memory_pool;\n+\t/* Map object_id to commit_change_list_entry structs. */\n+\tstruct oidmap oid_to_metadata_index;\n+\t/**\n+\t * Map of refnames to change_head structure which are allocated from\n+\t * memory_pool.\n+\t */\n+\tstruct strmap refname_to_change_head;\n+};\n+\n+extern void change_table_init(struct change_table *table);\n+extern void change_table_clear(struct change_table *table);\n+\n+/* Adds the given change head to the change_table struct */\n+extern void change_table_add(struct change_table *table,\n+\t\t\t const char *refname,\n+\t\t\t struct commit *target);\n+\n+/**\n+ * Adds the non-hidden local changes to the given change_table struct.\n+ */\n+extern void change_table_add_all_visible(struct change_table *table,\n+\t\t\t\t\t struct repository *repo);\n+\n+typedef int each_change_fn(const char *refname, void *cb_data);\n+\n+extern int change_table_has_change_referencing(\n+\tstruct change_table *table,\n+\tconst struct object_id *referenced_commit_id);\n+\n+/**\n+ * Iterates over all changes that reference the given commit. For metacommits,\n+ * this is the list of changes that point directly to that metacommit.\n+ * For normal commits, this is the list of changes that have this commit as\n+ * their latest content.\n+ */\n+extern int for_each_change_referencing(\n+\tstruct change_table *table,\n+\tconst struct object_id *referenced_commit_id,\n+\teach_change_fn fn,\n+\tvoid *cb_data);\n+\n+/**\n+ * Returns the change head for the given refname. Returns NULL if no such change\n+ * exists.\n+ */\n+extern struct change_head* get_change_head(struct change_table *table,\n+\tconst char* refname);\n+\n+#endif\n-- \ngitgitgadget\n\n"},{"id":"464234","messageId":"408941e74006e711dd592bd8ba8a93901dbf99bf.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 04/10] evolve: add support for parsing metacommits","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:11Z","receivedAt":"2022-10-05T15:01:02Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nThis patch adds the get_metacommit_content method, which can classify\ncommits as either metacommits or normal commits, determine whether they\nare abandoned, and extract the content commit's object id from the\nmetacommit.\n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n Makefile | 1 +\n commit.c | 13 ++++++\n commit.h | 5 +++\n metacommit-parser.c | 97 +++++++++++++++++++++++++++++++++++++++++++++\n metacommit-parser.h | 19 +++++++++\n 5 files changed, 135 insertions(+)\n create mode 100644 metacommit-parser.c\n create mode 100644 metacommit-parser.h\n\ndiff --git a/Makefile b/Makefile\nindex cac3452edb9..b2bcc00c289 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -999,6 +999,7 @@ LIB_OBJS += merge-ort.o\n LIB_OBJS += merge-ort-wrappers.o\n LIB_OBJS += merge-recursive.o\n LIB_OBJS += merge.o\n+LIB_OBJS += metacommit-parser.o\n LIB_OBJS += midx.o\n LIB_OBJS += name-hash.o\n LIB_OBJS += negotiator/default.o\ndiff --git a/commit.c b/commit.c\nindex 89b8efc6116..3eabb66fb6b 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -623,6 +623,19 @@ struct commit_list *reverse_commit_list(struct commit_list *list)\n \treturn next;\n }\n \n+struct commit *get_commit_by_index(struct commit_list *to_search, int index)\n+{\n+\twhile (to_search && index) {\n+\t\tto_search = to_search->next;\n+\t\tindex--;\n+\t}\n+\n+\tif (!to_search)\n+\t\treturn NULL;\n+\n+\treturn to_search->item;\n+}\n+\n void free_commit_list(struct commit_list *list)\n {\n \twhile (list)\ndiff --git a/commit.h b/commit.h\nindex 21e4d25ce78..11861a5a78c 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -188,8 +188,13 @@ struct commit_list *copy_commit_list(struct commit_list *list);\n /* Modify list in-place to reverse it, returning new head; list will be tail */\n struct commit_list *reverse_commit_list(struct commit_list *list);\n \n+/* Returns the commit at `index` or NULL if the index exceeds the `to_search`\n+ * list */\n+struct commit *get_commit_by_index(struct commit_list *to_search, int index);\n+\n void free_commit_list(struct commit_list *list);\n \n+\n struct rev_info; /* in revision.h, it circularly uses enum cmit_fmt */\n \n int has_non_ascii(const char *text);\ndiff --git a/metacommit-parser.c b/metacommit-parser.c\nnew file mode 100644\nindex 00000000000..baccfb4dd5c\n--- /dev/null\n+++ b/metacommit-parser.c\n@@ -0,0 +1,97 @@\n+#include \"cache.h\"\n+#include \"metacommit-parser.h\"\n+#include \"commit.h\"\n+\n+/*\n+ * Search the commit buffer for a line starting with the given key. Unlike\n+ * find_commit_header, this also searches the commit message body.\n+ */\n+static const char *find_key(const char *msg, const char *key, size_t *out_len)\n+{\n+\tint key_len = strlen(key);\n+\tconst char *line = msg;\n+\n+\twhile (line) {\n+\t\tconst char *eol = strchrnul(line, '\\n');\n+\n+\t\tif (eol - line > key_len && !memcmp(line, key, key_len) &&\n+\t\t line[key_len] == ' ') {\n+\t\t\t*out_len = eol - line - key_len - 1;\n+\t\t\treturn line + key_len + 1;\n+\t\t}\n+\t\tline = *eol ? eol + 1 : NULL;\n+\t}\n+\treturn NULL;\n+}\n+\n+/*\n+ * Writes the index of the content parent to \"result\". Returns the metacommit\n+ * type. See the METACOMMIT_TYPE_* constants.\n+ */\n+static enum metacommit_type index_of_content_commit(const char *buffer, int *result)\n+{\n+\tint index = 0;\n+\tint ret = METACOMMIT_TYPE_NONE;\n+\tsize_t parent_types_size;\n+\tconst char *parent_types = find_key(buffer, \"parent-type\",\n+\t\t&parent_types_size);\n+\tconst char *end;\n+\tconst char *enum_start = parent_types;\n+\tint enum_length = 0;\n+\n+\tif (!parent_types)\n+\t\treturn METACOMMIT_TYPE_NONE;\n+\n+\tend = &parent_types[parent_types_size];\n+\n+\twhile (1) {\n+\t\tchar next = *parent_types;\n+\t\tif (next == ' ' || parent_types >= end) {\n+\t\t\tif (enum_length == 1) {\n+\t\t\t\tchar type = *enum_start;\n+\t\t\t\tif (type == 'c') {\n+\t\t\t\t\tret = METACOMMIT_TYPE_NORMAL;\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t\tif (type == 'a') {\n+\t\t\t\t\tret = METACOMMIT_TYPE_ABANDONED;\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t\tif (parent_types >= end)\n+\t\t\t\treturn METACOMMIT_TYPE_NONE;\n+\t\t\tenum_start = parent_types + 1;\n+\t\t\tenum_length = 0;\n+\t\t\tindex++;\n+\t\t} else {\n+\t\t\tenum_length++;\n+\t\t}\n+\t\tparent_types++;\n+\t}\n+\n+\t*result = index;\n+\treturn ret;\n+}\n+\n+/*\n+ * Writes the content parent's object id to \"content\".\n+ * Returns the metacommit type. See the METACOMMIT_TYPE_* constants.\n+ */\n+enum metacommit_type get_metacommit_content(struct commit *commit, struct object_id *content)\n+{\n+\tconst char *buffer = get_commit_buffer(commit, NULL);\n+\tint index = 0;\n+\tenum metacommit_type ret = index_of_content_commit(buffer, &index);\n+\tstruct commit *content_parent;\n+\n+\tif (ret == METACOMMIT_TYPE_NONE)\n+\t\treturn ret;\n+\n+\tcontent_parent = get_commit_by_index(commit->parents, index);\n+\n+\tif (!content_parent)\n+\t\treturn METACOMMIT_TYPE_NONE;\n+\n+\toidcpy(content, &(content_parent->object.oid));\n+\treturn ret;\n+}\ndiff --git a/metacommit-parser.h b/metacommit-parser.h\nnew file mode 100644\nindex 00000000000..ef4a121d433\n--- /dev/null\n+++ b/metacommit-parser.h\n@@ -0,0 +1,19 @@\n+#ifndef METACOMMIT_PARSER_H\n+#define METACOMMIT_PARSER_H\n+\n+#include \"commit.h\"\n+#include \"hash.h\"\n+\n+enum metacommit_type {\n+\t/* Indicates a normal commit (non-metacommit) */\n+\tMETACOMMIT_TYPE_NONE = 0,\n+\t/* Indicates a metacommit with normal content (non-abandoned) */\n+\tMETACOMMIT_TYPE_NORMAL = 1,\n+\t/* Indicates a metacommit with abandoned content */\n+\tMETACOMMIT_TYPE_ABANDONED = 2,\n+};\n+\n+enum metacommit_type get_metacommit_content(\n+\tstruct commit *commit, struct object_id *content);\n+\n+#endif\n-- \ngitgitgadget\n\n"},{"id":"464235","messageId":"353d97d0f38870f287d862c626e0728dd7b84193.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 06/10] evolve: add support for writing metacommits","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:13Z","receivedAt":"2022-10-05T15:01:04Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nmetacommit.c supports the creation of metacommits and\nadds the API needed to create and update changes.\n\nCreate the \"modify_change\" function that can be called from modification\ncommands like \"rebase\" and \"git amend\" to record obsolescences in the\nchange graph.\n\nCreate the \"record_metacommit\" function for recording more complicated\ncommit relationships in the commit graph.\n\nCreate the \"write_metacommit\" function for low-level creation of\nmetacommits.\n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n Makefile | 1 +\n metacommit.c | 410 +++++++++++++++++++++++++++++++++++++++++++++++++++\n metacommit.h | 75 ++++++++++\n 3 files changed, 486 insertions(+)\n create mode 100644 metacommit.c\n create mode 100644 metacommit.h\n\ndiff --git a/Makefile b/Makefile\nindex 2b847e7e7de..68082ef94c7 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1000,6 +1000,7 @@ LIB_OBJS += merge-ort.o\n LIB_OBJS += merge-ort-wrappers.o\n LIB_OBJS += merge-recursive.o\n LIB_OBJS += merge.o\n+LIB_OBJS += metacommit.o\n LIB_OBJS += metacommit-parser.o\n LIB_OBJS += midx.o\n LIB_OBJS += name-hash.o\ndiff --git a/metacommit.c b/metacommit.c\nnew file mode 100644\nindex 00000000000..3c2e3ae1031\n--- /dev/null\n+++ b/metacommit.c\n@@ -0,0 +1,410 @@\n+#include \"cache.h\"\n+#include \"metacommit.h\"\n+#include \"commit.h\"\n+#include \"change-table.h\"\n+#include \"refs.h\"\n+\n+void clear_metacommit_data(struct metacommit_data *state)\n+{\n+\toidcpy(&state->content, null_oid());\n+\toid_array_clear(&state->replace);\n+\toid_array_clear(&state->origin);\n+\tstate->abandoned = 0;\n+}\n+\n+static void compute_default_change_name(struct commit *initial_commit,\n+\t\t\t\t\tstruct strbuf* result)\n+{\n+\tstruct strbuf default_name = STRBUF_INIT;\n+\tconst char *buffer;\n+\tconst char *subject;\n+\tconst char *eol;\n+\tsize_t len;\n+\tbuffer = get_commit_buffer(initial_commit, NULL);\n+\tfind_commit_subject(buffer, &subject);\n+\teol = strchrnul(subject, '\\n');\n+\tfor (len = 0; subject < eol && len < 10; subject++, len++) {\n+\t\tchar next = *subject;\n+\t\tif (isspace(next))\n+\t\t\tcontinue;\n+\n+\t\tstrbuf_addch(&default_name, next);\n+\t}\n+\tsanitize_refname_component(default_name.buf, result);\n+\tunuse_commit_buffer(initial_commit, buffer);\n+}\n+\n+/*\n+ * Computes a change name for a change rooted at the given initial commit. Good\n+ * change names should be memorable, unique, and easy to type. They are not\n+ * required to match the commit comment.\n+ */\n+static void compute_change_name(struct commit *initial_commit, struct strbuf* result)\n+{\n+\tstruct strbuf default_name = STRBUF_INIT;\n+\tstruct object_id unused;\n+\n+\tif (initial_commit)\n+\t\tcompute_default_change_name(initial_commit, &default_name);\n+\telse\n+\t\tBUG(\"initial commit is NULL\");\n+\tstrbuf_addstr(result, \"refs/metas/\");\n+\tstrbuf_addbuf(result, &default_name);\n+\n+\t/* If there is already a change of this name, append a suffix */\n+\tif (!read_ref(result->buf, &unused)) {\n+\t\tint suffix = 2;\n+\t\tsize_t original_length = result->len;\n+\n+\t\twhile (1) {\n+\t\t\tstrbuf_addf(result, \"%d\", suffix);\n+\t\t\tif (read_ref(result->buf, &unused))\n+\t\t\t\tbreak;\n+\t\t\tstrbuf_remove(result, original_length,\n+\t\t\t\t result->len - original_length);\n+\t\t\t++suffix;\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&default_name);\n+}\n+\n+struct resolve_metacommit_context\n+{\n+\tstruct change_table* active_changes;\n+\tstruct string_list *changes;\n+\tstruct oid_array *heads;\n+};\n+\n+static int resolve_metacommit_callback(const char *refname, void *cb_data)\n+{\n+\tstruct resolve_metacommit_context *data = cb_data;\n+\tstruct change_head *chhead;\n+\n+\tchhead = get_change_head(data->active_changes, refname);\n+\n+\tif (data->changes)\n+\t\tstring_list_append(data->changes, refname)->util = &chhead->head;\n+\tif (data->heads)\n+\t\toid_array_append(data->heads, &(chhead->head));\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Produces the final form of a metacommit based on the current change refs.\n+ */\n+static void resolve_metacommit(\n+\tstruct repository* repo,\n+\tstruct change_table* active_changes,\n+\tconst struct metacommit_data *to_resolve,\n+\tstruct metacommit_data *resolved_output,\n+\tstruct string_list *to_advance,\n+\tint allow_append)\n+{\n+\tsize_t i;\n+\tsize_t len = to_resolve->replace.nr;\n+\tstruct resolve_metacommit_context ctx = {\n+\t\t.active_changes = active_changes,\n+\t\t.changes = to_advance,\n+\t\t.heads = &resolved_output->replace\n+\t};\n+\tint old_change_list_length = to_advance->nr;\n+\tstruct commit* content;\n+\n+\toidcpy(&resolved_output->content, &to_resolve->content);\n+\n+\t/*\n+\t * First look for changes that point to any of the replacement edges in the\n+\t * metacommit. These will be the changes that get advanced by this\n+\t * metacommit.\n+\t */\n+\tresolved_output->abandoned = to_resolve->abandoned;\n+\n+\tif (allow_append) {\n+\t\tfor (i = 0; i < len; i++) {\n+\t\t\tint old_number = resolved_output->replace.nr;\n+\t\t\tfor_each_change_referencing(\n+\t\t\t\tactive_changes,\n+\t\t\t\t&(to_resolve->replace.oid[i]),\n+\t\t\t\tresolve_metacommit_callback,\n+\t\t\t\t&ctx);\n+\t\t\t/* If no changes were found, use the unresolved value. */\n+\t\t\tif (old_number == resolved_output->replace.nr)\n+\t\t\t\toid_array_append(&(resolved_output->replace),\n+\t\t\t\t\t\t &(to_resolve->replace.oid[i]));\n+\t\t}\n+\t}\n+\n+\tctx.changes = NULL;\n+\tctx.heads = &(resolved_output->origin);\n+\n+\tlen = to_resolve->origin.nr;\n+\tfor (i = 0; i < len; i++) {\n+\t\tint old_number = resolved_output->origin.nr;\n+\t\tfor_each_change_referencing(\n+\t\t\tactive_changes,\n+\t\t\t&(to_resolve->origin.oid[i]),\n+\t\t\tresolve_metacommit_callback,\n+\t\t\t&ctx);\n+\t\tif (old_number == resolved_output->origin.nr)\n+\t\t\toid_array_append(&(resolved_output->origin),\n+\t\t\t\t\t &(to_resolve->origin.oid[i]));\n+\t}\n+\n+\t/*\n+\t * If no changes were advanced by this metacommit, we'll need to create\n+\t * a new one. */\n+\tif (to_advance->nr == old_change_list_length) {\n+\t\tstruct strbuf change_name;\n+\n+\t\tstrbuf_init(&change_name, 80);\n+\n+\t\tcontent = lookup_commit_reference_gently(\n+\t\t\trepo, &(to_resolve->content), 1);\n+\n+\t\tcompute_change_name(content, &change_name);\n+\t\tstring_list_append(to_advance, change_name.buf);\n+\t\tstrbuf_release(&change_name);\n+\t}\n+}\n+\n+static void lookup_commits(\n+\tstruct repository *repo,\n+\tstruct oid_array *to_lookup,\n+\tstruct commit_list **result)\n+{\n+\tint i = to_lookup->nr;\n+\n+\twhile (--i >= 0) {\n+\t\tstruct object_id *next = &(to_lookup->oid[i]);\n+\t\tstruct commit *commit =\n+\t\t\tlookup_commit_reference_gently(repo, next, 1);\n+\t\tcommit_list_insert(commit, result);\n+\t}\n+}\n+\n+#define PARENT_TYPE_PREFIX \"parent-type \"\n+\n+int write_metacommit(struct repository *repo, struct metacommit_data *state,\n+\tstruct object_id *result)\n+{\n+\tstruct commit_list *parents = NULL;\n+\tstruct strbuf comment;\n+\tsize_t i;\n+\tstruct commit *content;\n+\n+\tstrbuf_init(&comment, strlen(PARENT_TYPE_PREFIX)\n+\t\t+ 1 + 2 * (state->origin.nr + state->replace.nr));\n+\tlookup_commits(repo, &state->origin, &parents);\n+\tlookup_commits(repo, &state->replace, &parents);\n+\tcontent = lookup_commit_reference_gently(repo, &state->content, 1);\n+\tif (!content) {\n+\t\tstrbuf_release(&comment);\n+\t\tfree_commit_list(parents);\n+\t\treturn -1;\n+\t}\n+\tcommit_list_insert(content, &parents);\n+\n+\tstrbuf_addstr(&comment, PARENT_TYPE_PREFIX);\n+\tstrbuf_addstr(&comment, state->abandoned ? \"a\" : \"c\");\n+\tfor (i = 0; i < state->replace.nr; i++)\n+\t\tstrbuf_addstr(&comment, \" r\");\n+\n+\tfor (i = 0; i < state->origin.nr; i++)\n+\t\tstrbuf_addstr(&comment, \" o\");\n+\n+\t/* The parents list will be freed by this call. */\n+\tcommit_tree(\n+\t\tcomment.buf,\n+\t\tcomment.len,\n+\t\trepo->hash_algo->empty_tree,\n+\t\tparents,\n+\t\tresult,\n+\t\tNULL,\n+\t\tNULL);\n+\n+\tstrbuf_release(&comment);\n+\treturn 0;\n+}\n+\n+/*\n+ * Returns true iff the given metacommit is abandoned, has one or more origin\n+ * parents, or has one or more replacement parents.\n+ */\n+static int is_nontrivial_metacommit(struct metacommit_data *state)\n+{\n+\treturn state->replace.nr || state->origin.nr || state->abandoned;\n+}\n+\n+/*\n+ * Records the relationships described by the given metacommit in the\n+ * repository.\n+ *\n+ * If override_change is NULL (the default), an attempt will be made\n+ * to append to existing changes wherever possible instead of creating new ones.\n+ * If override_change is non-null, only the given change ref will be updated.\n+ *\n+ * The changes list is filled in with the list of change refs that were updated,\n+ * with the util pointers pointing to the old object IDS for those changes.\n+ * The object ID pointers all point to objects owned by the change_table and\n+ * will go out of scope when the change_table is destroyed.\n+ *\n+ * options is a bitwise combination of the UPDATE_OPTION_* flags.\n+ */\n+static int record_metacommit_withresult(\n+\tstruct repository *repo,\n+\tstruct change_table *chtable,\n+\tconst struct metacommit_data *metacommit,\n+\tconst char *override_change,\n+\tint options,\n+\tstruct strbuf *err,\n+\tstruct string_list *changes)\n+{\n+\tstatic const char *msg = \"updating change\";\n+\tstruct metacommit_data resolved_metacommit = METACOMMIT_DATA_INIT;\n+\tstruct object_id commit_target;\n+\tstruct ref_transaction *transaction = NULL;\n+\tstruct change_head *overridden_head;\n+\tconst struct object_id *old_head;\n+\n+\tsize_t i;\n+\tint ret = 0;\n+\tint force = (options & UPDATE_OPTION_FORCE);\n+\n+\tresolve_metacommit(repo, chtable, metacommit, &resolved_metacommit, changes,\n+\t\t(options & UPDATE_OPTION_NOAPPEND) == 0);\n+\n+\tif (override_change) {\n+\t\tstring_list_clear(changes, 0);\n+\t\toverridden_head = get_change_head(chtable, override_change);\n+\t\tif (overridden_head) {\n+\t\t\t/* This is an existing change */\n+\t\t\told_head = &overridden_head->head;\n+\t\t\tif (!force) {\n+\t\t\t\tif (!oid_array_readonly_contains(&(resolved_metacommit.replace),\n+\t\t\t\t\t&overridden_head->head)) {\n+\t\t\t\t\t/* Attempted non-fast-forward change */\n+\t\t\t\t\tstrbuf_addf(err, _(\"non-fast-forward update to '%s'\"),\n+\t\t\t\t\t\toverride_change);\n+\t\t\t\t\tret = -1;\n+\t\t\t\t\tgoto cleanup;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t} else\n+\t\t\t/* ...then this is a newly-created change */\n+\t\t\told_head = null_oid();\n+\n+\t\t/*\n+\t\t * The expected \"current\" head of the change is stored in the\n+\t\t * util pointer. Cast required because old_head is const*\n+\t\t */\n+\t\tstring_list_append(changes, override_change)->util = (void *)old_head;\n+\t}\n+\n+\tif (is_nontrivial_metacommit(&resolved_metacommit)) {\n+\t\t/* If there are any origin or replacement parents, create a new metacommit\n+\t\t * object. */\n+\t\tif (write_metacommit(repo, &resolved_metacommit, &commit_target) < 0) {\n+\t\t\tret = -1;\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t} else {\n+\t\t/*\n+\t\t * If the metacommit would only contain a content commit, point to the\n+\t\t * commit itself rather than creating a trivial metacommit.\n+\t\t */\n+\t\toidcpy(&commit_target, &(resolved_metacommit.content));\n+\t}\n+\n+\t/*\n+\t * If a change already exists with this target and we're not forcing an\n+\t * update to some specific override_change && change, there's nothing to do.\n+\t */\n+\tif (!override_change\n+\t\t&& change_table_has_change_referencing(chtable, &commit_target))\n+\t\t/* Not an error */\n+\t\tgoto cleanup;\n+\n+\ttransaction = ref_transaction_begin(err);\n+\n+\t/* Update the refs for each affected change */\n+\tif (!transaction)\n+\t\tret = -1;\n+\telse {\n+\t\tfor (i = 0; i < changes->nr; i++) {\n+\t\t\tstruct string_list_item *it = &changes->items[i];\n+\n+\t\t\t/*\n+\t\t\t * The expected current head of the change is stored in the util pointer.\n+\t\t\t * It is null if the change should be newly-created.\n+\t\t\t */\n+\t\t\tif (it->util) {\n+\t\t\t\tif (ref_transaction_update(transaction, it->string, &commit_target,\n+\t\t\t\t\tforce ? NULL : it->util, 0, msg, err))\n+\n+\t\t\t\t\tret = -1;\n+\t\t\t} else {\n+\t\t\t\tif (ref_transaction_create(transaction, it->string,\n+\t\t\t\t\t&commit_target, 0, msg, err))\n+\n+\t\t\t\t\tret = -1;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!ret)\n+\t\t\tif (ref_transaction_commit(transaction, err))\n+\t\t\t\tret = -1;\n+\t}\n+\n+cleanup:\n+\tref_transaction_free(transaction);\n+\tclear_metacommit_data(&resolved_metacommit);\n+\n+\treturn ret;\n+}\n+\n+int record_metacommit(\n+\tstruct repository *repo,\n+\tconst struct metacommit_data *metacommit,\n+\tconst char *override_change,\n+\tint options,\n+\tstruct strbuf *err,\n+\tstruct string_list *changes)\n+{\n+\t\tstruct change_table chtable;\n+\t\tint result;\n+\n+\t\tchange_table_init(&chtable);\n+\t\tchange_table_add_all_visible(&chtable, repo);\n+\n+\t\tresult = record_metacommit_withresult(\n+\t\t\trepo,\n+\t\t\t&chtable,\n+\t\t\tmetacommit,\n+\t\t\toverride_change,\n+\t\t\toptions,\n+\t\t\terr,\n+\t\t\tchanges);\n+\n+\t\tchange_table_clear(&chtable);\n+\t\treturn result;\n+}\n+\n+void modify_change(\n+\tstruct repository *repo,\n+\tconst struct object_id *old_commit,\n+\tconst struct object_id *new_commit,\n+\tstruct strbuf *err)\n+{\n+\tstruct string_list changes = STRING_LIST_INIT_DUP;\n+\tstruct metacommit_data metacommit = METACOMMIT_DATA_INIT;\n+\n+\toidcpy(&(metacommit.content), new_commit);\n+\toid_array_append(&(metacommit.replace), old_commit);\n+\n+\trecord_metacommit(repo, &metacommit, NULL, 0, err, &changes);\n+\n+\tclear_metacommit_data(&metacommit);\n+\tstring_list_clear(&changes, 0);\n+}\ndiff --git a/metacommit.h b/metacommit.h\nnew file mode 100644\nindex 00000000000..45625cd0d02\n--- /dev/null\n+++ b/metacommit.h\n@@ -0,0 +1,75 @@\n+#ifndef METACOMMIT_H\n+#define METACOMMIT_H\n+\n+#include \"hash.h\"\n+#include \"oid-array.h\"\n+#include \"repository.h\"\n+#include \"string-list.h\"\n+\n+/* If specified, non-fast-forward changes are permitted. */\n+#define UPDATE_OPTION_FORCE 0x0001\n+/**\n+ * If specified, no attempt will be made to append to existing changes.\n+ * Normally, if a metacommit points to a commit in its replace or origin\n+ * list and an existing change points to that same commit as its content, the\n+ * new metacommit will attempt to append to that same change. This may replace\n+ * the commit parent with one or more metacommits from the head of the appended\n+ * changes. This option disables this behavior, and will always create a new\n+ * change rather than reusing existing changes.\n+ */\n+#define UPDATE_OPTION_NOAPPEND 0x0002\n+\n+/* Metacommit Data */\n+\n+struct metacommit_data {\n+\tstruct object_id content;\n+\tstruct oid_array replace;\n+\tstruct oid_array origin;\n+\tint abandoned;\n+};\n+\n+#define METACOMMIT_DATA_INIT { 0 }\n+\n+extern void clear_metacommit_data(struct metacommit_data *state);\n+\n+/**\n+ * Records the relationships described by the given metacommit in the\n+ * repository.\n+ *\n+ * If override_change is NULL (the default), an attempt will be made\n+ * to append to existing changes wherever possible instead of creating new ones.\n+ * If override_change is non-null, only the given change ref will be updated.\n+ *\n+ * options is a bitwise combination of the UPDATE_OPTION_* flags.\n+ */\n+int record_metacommit(\n+\tstruct repository *repo,\n+\tconst struct metacommit_data *metacommit,\n+\tconst char* override_change,\n+\tint options,\n+\tstruct strbuf *err,\n+\tstruct string_list *changes);\n+\n+/**\n+ * Should be invoked after a command that has \"modify\" semantics - commands that\n+ * create a new commit based on an old commit and treat the new one as a\n+ * replacement for the old one. This method records the replacement in the\n+ * change graph, such that a future evolve operation will rebase children of\n+ * the old commit onto the new commit.\n+ */\n+void modify_change(\n+\tstruct repository *repo,\n+\tconst struct object_id *old_commit,\n+\tconst struct object_id *new_commit,\n+\tstruct strbuf *err);\n+\n+/**\n+ * Creates a new metacommit object with the given content. Writes the object\n+ * id of the newly-created commit to result.\n+ */\n+int write_metacommit(\n+\tstruct repository *repo,\n+\tstruct metacommit_data *state,\n+\tstruct object_id *result);\n+\n+#endif\n-- \ngitgitgadget\n\n"},{"id":"464237","messageId":"a5eb93254191b7ae9c17ce52e056955c669ea007.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 01/10] technical doc: add a design doc for the evolve command","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:08Z","receivedAt":"2022-10-05T15:01:10Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nThis document describes what a change graph for\ngit would look like, the behavior of the evolve command,\nand the changes planned for other commands.\n\nIt was originally proposed in 2018, see\nhttps://public-inbox.org/git/20181115005546.212538-1-sxenos@google.com/\n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n Documentation/technical/evolve.txt | 1070 ++++++++++++++++++++++++++++\n 1 file changed, 1070 insertions(+)\n create mode 100644 Documentation/technical/evolve.txt\n\ndiff --git a/Documentation/technical/evolve.txt b/Documentation/technical/evolve.txt\nnew file mode 100644\nindex 00000000000..2051ea77b8a\n--- /dev/null\n+++ b/Documentation/technical/evolve.txt\n@@ -0,0 +1,1070 @@\n+Evolve\n+======\n+\n+Objective\n+=========\n+Create an \"evolve\" command to help users craft a high quality commit history.\n+Users can improve commits one at a time and in any order, then run git evolve to\n+rewrite their recent history to ensure everything is up-to-date. We track\n+amendments to a commit over time in a change graph. Users can share their\n+progress with others by exchanging their change graphs using the standard push,\n+fetch, and format-patch commands.\n+\n+Status\n+======\n+This proposal has not been implemented yet.\n+\n+Background\n+==========\n+Imagine you have three sequential changes up for review and you receive feedback\n+that requires editing all three changes. We'll define the word \"change\"\n+formally later, but for the moment let's say that a change is a work-in-progress\n+whose final version will be submitted as a commit in the future.\n+\n+While you're editing one change, more feedback arrives on one of the others.\n+What do you do?\n+\n+The evolve command is a convenient way to work with chains of commits that are\n+under review. Whenever you rebase or amend a commit, the repository remembers\n+that the old commit is obsolete and has been replaced by the new one. Then, at\n+some point in the future, you can run \"git evolve\" and the correct sequence of\n+rebases will occur in the correct order such that no commit has an obsolete\n+parent.\n+\n+Part of making the \"evolve\" command work involves tracking the edits to a commit\n+over time, which is why we need an change graph. However, the change\n+graph will also bring other benefits:\n+\n+- Users can view the history of a change directly (the sequence of amends and\n+ rebases it has undergone, orthogonal to the history of the branch it is on).\n+- It will be possible to quickly locate and list all the changes the user\n+ currently has in progress.\n+- It can be used as part of other high-level commands that combine or split\n+ changes.\n+- It can be used to decorate commits (in git log, gitk, etc) that are either\n+ obsolete or are the tip of a work in progress.\n+- By pushing and pulling the change graph, users can collaborate more\n+ easily on changes-in-progress. This is better than pushing and pulling the\n+ commits themselves since the change graph can be used to locate a more\n+ specific merge base, allowing for better merges between different versions of\n+ the same change.\n+- It could be used to correctly rebase local changes and other local branches\n+ after running git-filter-branch.\n+- It can replace the change-id footer used by gerrit.\n+\n+Goals\n+-----\n+Legend: Goals marked with P0 are required. Goals marked with Pn should be\n+attempted unless they interfere with goals marked with Pn-1.\n+\n+P0. All commands that modify commits (such as the normal commit --amend or\n+ rebase command) should mark the old commit as being obsolete and replaced by\n+ the new one. No additional commands should be required to keep the\n+ change graph up-to-date.\n+P0. Any commit that may be involved in a future evolve command should not be\n+ garbage collected. Specifically:\n+ - Commits that obsolete another should not be garbage collected until\n+ user-specified conditions have occurred and the change has expired from\n+ the reflog. User specified conditions for removing changes include:\n+ - The user explicitly deleted the change.\n+ - The change was merged into a specific branch.\n+ - Commits that have been obsoleted by another should not be garbage\n+ collected if any of their replacements are still being retained.\n+P0. A commit can be obsoleted by more than one replacement (called divergence).\n+P0. Users must be able to resolve divergence (convergence).\n+P1. Users should be able to share chains of obsolete changes in order to\n+ collaborate on WIP changes.\n+P2. Such sharing should be at the user’s option. That is, it should be possible\n+ to directly share a change without also sharing the file states or commit\n+ comments from the obsolete changes that led up to it, and the choice not to\n+ share those commits should not require changing any commit hashes.\n+P2. It should be possible to discard part or all of the change graph\n+ without discarding the commits themselves that are already present in\n+ branches and the reflog.\n+P2. Provide sufficient information to replace gerrit's Change-Id footers.\n+\n+Similar technologies\n+--------------------\n+There are some other technologies that address the same end-user problem.\n+\n+Rebase -i can be used to solve the same problem, but users can't easily switch\n+tasks midway through an interactive rebase or have more than one interactive\n+rebase going on at the same time. It can't handle the case where you have\n+multiple changes sharing the same parent when that parent needs to be rebased\n+and won't let you collaborate with others on resolving a complicated interactive\n+rebase. You can think of rebase -i as a top-down approach and the evolve command\n+as the bottom-up approach to the same problem.\n+\n+Revup amend (https://github.com/Skydio/revup/blob/main/docs/amend.md)\n+allows insertion of cached changes into any commit in\n+the current history, and then reapplies the rest of history on top of\n+those changes. It uses a \"git apply --cached\" engine under the hood so\n+doesn't touch the working directory (although it will soon use the new\n+git merge-tree). When paired with \"revup upload\" which creates and\n+pushes multiple branches in the background for you, its possible to\n+work on a \"graph\" of changes on a single branch linearly, then have\n+the true graph structure created at upload time.\n+\n+git-revise (https://github.com/mystor/git-revise) does some very\n+similar things except it uses \"git merge-file\" combined with manually\n+merging the resulting trees. git branchstack\n+(https://github.com/krobelus/git-branchstack) can also create branches\n+in the background with the same mechanism.\n+\n+These tools don't store any external state, but as such also don't\n+provide any specific collaboration mechanism for individual changes.\n+\n+Several patch queue managers have been built on top of git (such as topgit,\n+stgit, and quilt). They address the same user need. However they also rely on\n+state managed outside git that needs to be kept in sync. Such state can be\n+easily damaged when running a git native command that is unaware of the patch\n+queue. They also typically require an explicit initialization step to be done by\n+the user which creates workflow problems.\n+\n+Mercurial implements a very similar feature in its EvolveExtension. The behavior\n+of the evolve command itself is very similar, but the storage format for the\n+change graph differs. In the case of mercurial, each change set can have one or\n+more obsolescence markers that point to other changesets that they replace. This\n+is similar to the \"Commit Headers\" approach considered in the other options\n+appendix. The approach proposed here stores obsolescence information in a\n+separate metacommit graph, which makes exchanging of obsolescence information\n+optional.\n+\n+Mercurial's default behavior makes it easy to find and switch between\n+non-obsolete changesets that aren't currently on any branch. We introduce the\n+notion of a new ref namespace that enables a similar workflow via a different\n+mechanism. Mercurial has the notion of changeset phases which isn't present\n+in git and creates new ways for a changeset to diverge. Git doesn't need\n+to deal with these issues, but it has to deal with the problems of picking an\n+upstream branch as a target for rebases and protecting obsolescence information\n+from GC. We also introduce some additional transformations (see\n+obsolescence-over-cherry-pick, below) that aren't present in the mercurial\n+implementation.\n+\n+Semi-related work\n+-----------------\n+There are other technologies that address different problems but have some\n+similarities with this proposal.\n+\n+Replacements (refs/replace) are superficially similar to obsolescences in that\n+they describe that one commit should be replaced by another. However, they\n+differ in both how they are created and how they are intended to be used.\n+Obsolescences are created automatically by the commands a user runs, and they\n+describe the user’s intent to perform a future rebase. Obsolete commits still\n+appear in branches, logs, etc like normal commits (possibly with an extra\n+decoration that marks them as obsolete). Replacements are typically created\n+explicitly by the user, they are meant to be kept around for a long time, and\n+they describe a replacement to be applied at read-time rather than as the input\n+to a future operation. When a replaced commit is queried, it is typically hidden\n+and swapped out with its replacement as though the replacement has already\n+occurred.\n+\n+Git-imerge is a project to help make complicated merges easier, particularly\n+when merging or rebasing long chains of patches. It is not an alternative to\n+the change graph, but its algorithm of applying smaller incremental merges\n+could be used as part of the evolve algorithm in the future.\n+\n+Overview\n+========\n+We introduce the notion of “meta-commits” which describe how one commit was\n+created from other commits. A branch of meta-commits is known as a change.\n+Changes are created and updated automatically whenever a user runs a command\n+that creates a commit. They are used for locating obsolete commits, providing a\n+list of a user’s unsubmitted work in progress, and providing a stable name for\n+each unsubmitted change.\n+\n+Users can exchange edit histories by pushing and fetching changes.\n+\n+New commands will be introduced for manipulating changes and resolving\n+divergence between them. Existing commands that create commits will be updated\n+to modify the meta-commit graph and create changes where necessary.\n+\n+Example usage\n+-------------\n+# First create three dependent changes\n+$ echo foo>bar.txt && git add .\n+$ git commit -m \"This is a test\"\n+created change metas/this_is_a_test\n+$ echo foo2>bar2.txt && git add .\n+$ git commit -m \"This is also a test\"\n+created change metas/this_is_also_a_test\n+$ echo foo3>bar3.txt && git add .\n+$ git commit -m \"More testing\"\n+created change metas/more_testing\n+\n+# List all our changes in progress\n+$ git change list\n+metas/this_is_a_test\n+metas/this_is_also_a_test\n+* metas/more_testing\n+metas/some_change_already_merged_upstream\n+\n+# Now modify the earliest change, using its stable name\n+$ git reset --hard metas/this_is_a_test\n+$ echo morefoo>>bar.txt && git add . && git commit --amend --no-edit\n+\n+# Use git-evolve to fix up any dependent changes\n+$ git evolve\n+rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n+rebasing metas/more_testing onto metas/this_is_also_a_test\n+Done\n+\n+# Use git-obslog to view the history of the this_is_a_test change\n+$ git log --obslog\n+93f110 metas/this_is_a_test@{0} commit (amend): This is a test\n+930219 metas/this_is_a_test@{1} commit: This is a test\n+\n+# Now create an unrelated change\n+$ git reset --hard origin/master\n+$ echo newchange>unrelated.txt && git add .\n+$ git commit -m \"Unrelated change\"\n+created change metas/unrelated_change\n+\n+# Fetch the latest code from origin/master and use git-evolve\n+# to rebase all dependent changes.\n+$ git fetch origin master\n+$ git evolve origin/master\n+deleting metas/some_change_already_merged_upstream\n+rebasing metas/this_is_a_test onto origin/master\n+rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n+rebasing metas/more_testing onto metas/this_is_also_a_test\n+rebasing metas/unrelated_change onto origin/master\n+Conflict detected! Resolve it and then use git evolve --continue to resume.\n+\n+# Sort out the conflict\n+$ git mergetool\n+$ git evolve origin/master\n+Done\n+\n+# Share the full history of edits for the this_is_a_test change\n+# with a review server\n+$ git push origin metas/this_is_a_test:refs/for/master\n+# Share the lastest commit for “Unrelated change”, without history\n+$ git push origin HEAD:refs/for/master\n+\n+Detailed design\n+===============\n+Obsolescence information is stored as a graph of meta-commits. A meta-commit is\n+a specially-formatted merge commit that describes how one commit was created\n+from others.\n+\n+Meta-commits look like this:\n+\n+$ git cat-file -p \n+tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\n+parent aa7ce55545bf2c14bef48db91af1a74e2347539a\n+parent d64309ee51d0af12723b6cb027fc9f195b15a5e9\n+parent 7e1bbcd3a0fa854a7a9eac9bf1eea6465de98136\n+author Stefan Xenos 1540841596 -0700\n+committer Stefan Xenos 1540841596 -0700\n+parent-type c r o\n+\n+This says “commit aa7ce555 makes commit d64309ee obsolete. It was created by\n+cherry-picking commit 7e1bbcd3”.\n+\n+The tree for meta-commits is always the empty tree, but future versions of git\n+may attach other trees here. For forward-compatibility fsck should ignore such\n+trees if found on future repository versions. This will allow future versions of\n+git to add metadata to the meta-commit tree without breaking forwards\n+compatibility.\n+\n+The commit comment for a meta-commit is an auto-generated user-readable string\n+describing the command that produced the meta commit. These strings are shown\n+to the user when they view the obslog.\n+\n+Parent-type\n+-----------\n+The “parent-type” field in the commit header identifies a commit as a\n+meta-commit and indicates the meaning for each of its parents. It is never\n+present for normal commits. It contains a space-deliminated list of enum values\n+whose order matches the order of the parents. Possible parent types are:\n+\n+- c: (content) the content parent identifies the commit that this meta-commit is\n+ describing.\n+- r: (replaced) indicates that this parent is made obsolete by the content\n+ parent.\n+- o: (origin) indicates that the content parent was generated by cherry-picking\n+ this parent.\n+- a: (abandoned) used in place of a content parent for abandoned changes. Points\n+ to the final content commit for the change at the time it was abandoned.\n+\n+There must be exactly one content or abandoned parent for each meta-commit and\n+it is always the first parent. The content commit will always be a normal commit\n+and not a meta-commit. However, future versions of git may create meta-commits\n+for other meta-commits and the fsck tool must be aware of this for forwards\n+compatibility.\n+\n+A meta-commit can have zero or more replaced parents. An amend operation creates\n+a single replaced parent. A merge used to resolve divergence (see divergence,\n+below) will create multiple replaced parents. A meta-commit may have no\n+replaced parents if it describes a cherry-pick or squash merge that copies one\n+or more commits but does not replace them.\n+\n+A meta-commit can have zero or more origin parents. A cherry-pick creates a\n+single origin parent. Certain types of squash merge will create multiple origin\n+parents. Origin parents don't directly cause their origin to become obsolete,\n+but are used when computing blame or locating a merge base. The section\n+on obsolescence over cherry-picks describes how the evolve command uses\n+origin parents.\n+\n+A replaced parent or origin parent may be either a normal commit (indicating\n+the oldest-known version of a change) or another meta-commit (for a change that\n+has already been modified one or more times).\n+\n+The parent-type field needs to go after the committer field since git's rules\n+for forwards-compatibility require that new fields to be at the end of the\n+header. Putting a new field in the middle of the header would break fsck.\n+\n+The presence of an abandoned parent indicates that the change should be pruned\n+by the evolve command, and removed from the repository's history. Any follow-up\n+changes should rebased onto the parent of the pruned commit. The abandoned\n+parent points to the version of the change that should be restored if the user\n+attempts to restore the change.\n+\n+Changes\n+-------\n+A branch of meta-commits describes how a commit was produced and what previous\n+commits it is based on. It is also an identifier for a thing the user is\n+currently working on. We refer to such a meta-branch as a change.\n+\n+Local changes are stored in the new refs/metas namespace. Remote changes are\n+stored in the refs/remote//metas namespace.\n+\n+The list of changes in refs/metas is more than just a mechanism for the evolve\n+command to locate obsolete commits. It is also a convenient list of all of a\n+user’s work in progress and their current state - a list of things they’re\n+likely to want to come back to.\n+\n+Strictly speaking, it is the presence of the branch in the refs/metas namespace\n+that marks a branch as being a change, not the fact that it points to a\n+metacommit. Metacommits are only created when a commit is amended or rebased, so\n+in the case where a change points to a commit that has never been modified, the\n+change points to that initial commit rather than a metacommit.\n+\n+Changes are also stored in the refs/hiddenmetas namespace. Hiddenmetas holds\n+metadata for historical changes that are not currently in progress by the user.\n+Commands like filter-branch and other bulk import commands create metadata in\n+this namespace.\n+\n+Note that the changes in hiddenmetas get special treatment in several ways:\n+\n+- They are not cleaned up automatically once merged, since it is expected that\n+ they refer to historical changes.\n+- User commands that modify changes don't append to these changes as they would\n+ to a change in refs/metas.\n+- They are not displayed when the user lists their local changes.\n+\n+Obsolescence\n+------------\n+A commit is considered obsolete if it is reachable from the “replaces” edges\n+anywhere in the history of a change and it isn’t the head of that change.\n+Commits may be the content for 0 or more meta-commits. If the same commit\n+appears in multiple changes, it is not obsolete if it is the head of any of\n+those changes.\n+\n+Note that there is an exception to this rule. The metas namespace takes\n+precedence over the hiddenmetas namespace for the purpose of obsolescence. That\n+is, if a change appears in a replaces edge of a change in the metas namespace,\n+it is obsolete even if it also appears as the head of a change in the\n+hiddenmetas namespace.\n+\n+This special case prevents the hiddenmetas namespace from creating divergence\n+with the user's work in progress, and allows the user to resolve historical\n+divergence by creating new changes in the metas namespace.\n+\n+Divergence\n+----------\n+From the user’s perspective, two changes are divergent if they both ask for\n+different replacements to the same commit. More precisely, a target commit is\n+considered divergent if there is more than one commit at the head of a change in\n+refs/metas that leads to the target commit via an unbroken chain of “replaces”\n+parents.\n+\n+Much like a merge conflict, divergence is a situation that requires user\n+intervention to resolve. The evolve command will stop when it encounters\n+divergence and prompt the user to resolve the problem. Users can solve the\n+problem in several ways:\n+\n+- Discard one of the changes (by deleting its change branch).\n+- Merge the two changes (producing a single change branch).\n+- Copy one of the changes (keep both commits, but one of them gets a new\n+ metacommit appended to its history that is connected to its predecessor via an\n+ origin edge rather than a replaces edge. That new change no longer obsoletes\n+ the original.)\n+\n+Obsolescence across cherry-picks\n+--------------------------------\n+By default the evolve command will treat cherry-picks and squash merges as being\n+completely separate from the original. Further amendments to the original commit\n+will have no effect on the cherry-picked copy. However, this behavior may not be\n+desirable in all circumstances.\n+\n+The evolve command may at some point support an option to look for cases where\n+the source of a cherry-pick or squash merge has itself been amended, and\n+automatically apply that same change to the cherry-picked copy. In such cases,\n+it would traverse origin edges rather than ignoring them, and would treat a\n+commit with origin edges as being obsolete if any of its origins were obsolete.\n+\n+Garbage collection\n+------------------\n+For GC purposes, meta-commits are normal commits. Just as a commit causes its\n+parents and tree to be retained, a meta-commit also causes its parents to be\n+retained.\n+\n+Change creation\n+---------------\n+Changes are created automatically whenever the user runs a command like “commit”\n+that has the semantics of creating a new change. They also move forward\n+automatically even if they’re not checked out. For example, whenever the user\n+runs a command like “commit --amend” that modifies a commit, all branches in\n+refs/metas that pointed to the old commit move forward to point to its\n+replacement instead. This also happens when the user is working from a detached\n+head.\n+\n+This does not mean that every commit has a corresponding change. By default,\n+changes only exist for recent locally-created commits. Users may explicitly pull\n+changes from other users or keep their changes around for a long time, but\n+either behavior requires a user to opt-in. Code review systems like gerrit may\n+also choose to keep changes around forever.\n+\n+Note that the changes in refs/metas serve a dual function as both a way to\n+identify obsolete changes and as a way for the user to keep track of their work\n+in progress. If we were only concerned with identifying obsolete changes, it\n+would be sufficient to create the change branch lazily the first time a commit\n+is obsoleted. Addressing the second use - of refs/metas as a mechanism for\n+keeping track of work in progress - is the reason for eagerly creating the\n+change on first commit.\n+\n+Change naming\n+-------------\n+When a change is first created, the only requirement for its name is that it\n+must be unique. Good names would also serve as useful mnemonics and be easy to\n+type. For example, a short word from the commit message containing no numbers or\n+special characters and that shows up with low frequency in other commit messages\n+would make a good choice.\n+\n+Different users may prefer different heuristics for their change names. For this\n+reason a new hook will be introduced to compute change names. Git will invoke\n+the hook for all newly-created changes and will append a numeric suffix if the\n+name isn’t unique. The default heuristics are not specified by this proposal and\n+may change during implementation.\n+\n+Change deletion\n+---------------\n+Changes are normally only interesting to a user while a commit is still in\n+development and under review. Once the commit has submitted wherever it is\n+going, its change can be discarded.\n+\n+The normal way of deleting changes makes this easy to do - changes are deleted\n+by the evolve command when it detects that the change is present in an upstream\n+branch. It does this in two ways: if the latest commit in a change either shows\n+up in the branch history or the change becomes empty after a rebase, it is\n+considered merged and the change is discarded. In this context, an “upstream\n+branch” is any branch passed in as the upstream argument of the evolve command.\n+\n+In case this sometimes deletes a useful change, such automatic deletions are\n+recorded in the reflog allowing them to be easily recovered.\n+\n+Sharing changes\n+---------------\n+Change histories are shared by pushing or fetching meta-commits and change\n+branches. This provides users with a lot of control of what to share and\n+repository implementations with control over what to retain.\n+\n+Users that only want to share the content of a commit can do so by pushing the\n+commit itself as they currently would. Users that want to share an edit history\n+for the commit can push its change, which would point to a meta-commit rather\n+than the commit itself if there is any history to share. Note that multiple\n+changes can refer to the same commits, so it’s possible to construct and push a\n+different history for the same commit in order to remove sensitive or irrelevant\n+intermediate states.\n+\n+Imagine the user is working on a change “mychange” that is currently the latest\n+commit on master. They have two ways to share it:\n+\n+# User shares just a commit without its history\n+> git push origin master\n+\n+# User shares the full history of the commit to a review system\n+> git push origin metas/mychange:refs/for/master\n+\n+# User fetches a collaborator’s modifications to their change\n+> git fetch remotename metas/mychange\n+# Which updates the ref remote/remotename/metas/mychange\n+\n+This will cause more intermediate states to be shared with the server than would\n+have been shared previously. A review system like gerrit would need to keep\n+track of which states had been explicitly pushed versus other intermediate\n+states in order to de-emphasize (or hide) the extra intermediate states from the\n+user interface.\n+\n+Merge-base\n+----------\n+Merge-base will be changed to search the meta-commit graph for common ancestors\n+as well as the commit graph, and will generally prefer results from the\n+meta-commit graph over the commit graph. Merge-base will consider meta-commits\n+from all changes, and will traverse both origin and obsolete edges.\n+\n+The reason for this is that - when merging two versions of the same commit\n+together - an earlier version of that same commit will usually be much more\n+similar than their common parent. This should make the workflow of collaborating\n+on unsubmitted patches as convenient as the workflow for collaborating in a\n+topic branch by eliminating repeated merges.\n+\n+Configuration\n+-------------\n+The core.enableChanges configuration variable enables the creation and update\n+of change branches. This is enabled by default.\n+\n+User interface\n+--------------\n+All git porcelain commands that create commits are classified as having one of\n+four behaviors: modify, create, copy, or import. These behaviors are discussed\n+in more detail below.\n+\n+Modify commands\n+---------------\n+Modification commands (commit --amend, rebase) will mark the old commit as\n+obsolete by creating a new meta-commit that references the old one as a\n+replaced parent. In the event that multiple changes point to the same commit,\n+this is done independently for every such change.\n+\n+More specifically, modifications work like this:\n+\n+1. Locate all existing changes for which the old commit is the content for the\n+ head of the change branch. If no such branch exists, create one that points\n+ to the old commit. Changes that include this commit in their history but not\n+ at their head are explicitly not included.\n+2. For every such change, create a new meta-commit that references the new\n+ commit as its content and references the old head of the change as a\n+ replaced parent.\n+3. Move the change branch forward to point to the new meta-commit.\n+\n+Copy commands\n+-------------\n+Copy commands (cherry-pick, merge --squash) create a new meta-commit that\n+references the old commits as origin parents. Besides the fact that the new\n+parents are tagged differently, copy commands work the same way as modify\n+commands.\n+\n+Create commands\n+---------------\n+Creation commands (commit, merge) create a new commit and a new change that\n+points to that commit. The do not create any meta-commits.\n+\n+Import commands\n+---------------\n+Import commands (fetch, pull) do not create any new meta-commits or changes\n+unless that is specifically what they are importing. For example, the fetch\n+command would update remote/origin/metas/change35 and fetch all referenced\n+meta-commits if asked to do so directly, but it wouldn’t create any changes or\n+meta-commits for commits discovered on the master branch when running “git fetch\n+origin master”.\n+\n+Other commands\n+--------------\n+Some commands don’t fit cleanly into one of the above categories.\n+\n+Semantically, filter-branch should be treated as a modify command, but doing so\n+is likely to create a lot of irrelevant clutter in the changes namespace and the\n+large number of extra change refs may introduce performance problems. We\n+recommend treating filter-branch as an import command initially, but making it\n+behave more like a modify command in future follow-up work. One possible\n+solution may be to treat commits that are part of existing changes as being\n+modified but to avoid creating changes for other rewritten changes. Another\n+solution may be to record the modifications as changes in the hiddenmetas\n+namespace.\n+\n+Once the evolve command can handle obsolescence across cherry-picks, such\n+cherry-picks will result in a hybrid move-and-copy operation. It will create\n+cherry-picks that replace other cherry-picks, which will have both origin edges\n+(pointing to the new source commit being picked) and replacement edges (pointing\n+to the previous cherry-pick being replaced).\n+\n+Evolve\n+------\n+The evolve command performs the correct sequence of rebases such that no change\n+has an obsolete parent. The syntax looks like this:\n+\n+git evolve [upstream…]\n+\n+It takes an optional list of upstream branches. All changes whose parent shows\n+up in the history of one of the upstream branches will be rebased onto the\n+upstream branch before resolving obsolete parents.\n+\n+Any change whose latest state is found in an upstream branch (or that ends up\n+empty after rebase) will be deleted. This is the normal mechanism for deleting\n+changes. Changes are created automatically on the first commit, and are deleted\n+automatically when evolve determines that they’ve been merged upstream.\n+\n+Orphan commits are commits with obsolete parents. The evolve command then\n+repeatedly rebases orphan commits with non-orphan parents until there are either\n+no orphan commits left, or a merge conflict is discovered. It will also\n+terminate if it detects a divergent parent or a cycle that can't be resolved\n+using any of the enabled transformations.\n+\n+When evolve discovers divergence, it will first check if it can resolve the\n+divergence automatically using one of its enabled transformations. Supported\n+transformations are:\n+\n+- Check if the user has already merged the divergent changes in a follow-up\n+ change. That is, look for an existing merge in a follow-up change where all\n+ the parents are divergent versions of the same change. Squash that merge with\n+ its parents and use the result as the resolution for the divergence.\n+\n+- Attempt to auto-merge all the divergent changes (disabled by default).\n+\n+Each of the transformations can be enabled or disabled by command line options.\n+\n+Cycles can occur when two changes reference one another as parents. This can\n+happen when both changes use an obsolete version of the other change as their\n+parent. Although there are never cycles in the commit graph, users can create\n+cycles in the change graph by rebasing changes onto obsolete commits. The evolve\n+command has a transformation that will detect and break cycles by arbitrarily\n+picking one of the changes to go first. If this generates a merge conflict,\n+it tries each of the other changes in sequence to see if any ordering merges\n+cleanly. If no possible ordering merges cleanly, it picks one and terminates\n+to let the user resolve the merge conflict.\n+\n+If the working tree is dirty, evolve will attempt to stash the user's changes\n+before applying the evolve and then reapply those changes afterward, in much\n+the same way as rebase --autostash does.\n+\n+Checkout\n+--------\n+Running checkout on a change by name has the same effect as checking out a\n+detached head pointing to the latest commit on that change-branch. There is no\n+need to ever have HEAD point to a change since changes always move forward when\n+necessary, no matter what branch the user has checked out\n+\n+Meta-commits themselves cannot be checked out by their hash.\n+\n+Reset\n+-----\n+Resetting a branch to a change by name is the same as resetting to the content\n+(or abandoned) commit at that change’s head.\n+\n+Commit\n+------\n+Commit --amend gets modify semantics and will move existing changes forward. The\n+normal form of commit gets create semantics and will create a new change.\n+\n+$ touch foo && git add . && git commit -m \"foo\" && git tag A\n+$ touch bar && git add . && git commit -m \"bar\" && git tag B\n+$ touch baz && git add . && git commit -m \"baz\" && git tag C\n+\n+This produces the following commits:\n+A(tree=[foo])\n+B(tree=[foo, bar], parent=A)\n+C(tree=[foo, bar, baz], parent=B)\n+\n+...along with three changes:\n+metas/foo = A\n+metas/bar = B\n+metas/baz = C\n+\n+Running commit --amend does the following:\n+$ git checkout B\n+$ touch zoom && git add . && git commit --amend -m \"baz and zoom\"\n+$ git tag D\n+\n+Commits:\n+A(tree=[foo])\n+B(tree=[foo, bar], parent=A)\n+C(tree=[foo, bar, baz], parent=B)\n+D(tree=[foo, bar, zoom], parent=A)\n+Dmeta(content=D, obsolete=B)\n+\n+Changes:\n+metas/foo = A\n+metas/bar = Dmeta\n+metas/baz = C\n+\n+Merge\n+-----\n+Merge gets create, modify, or copy semantics based on what is being merged and\n+the options being used.\n+\n+The --squash version of merge gets copy semantics (it produces a new change that\n+is marked as a copy of all the original changes that were squashed into it).\n+\n+The “modify” version of merge replaces both of the original commits with the\n+resulting merge commit. This is one of the standard mechanisms for resolving\n+divergence. The parents of the merge commit are the parents of the two commits\n+being merged. The resulting commit will not be a merge commit if both of the\n+original commits had the same parent or if one was the parent of the other.\n+\n+The “create” version of merge creates a new change pointing to a merge commit\n+that has both original commits as parents. The result is what merge produces now\n+- a new merge commit. However, this version of merge doesn’t directly resolve\n+divergence.\n+\n+To select between these two behaviors, merge gets new “--amend” and “--noamend”\n+options which select between the “create” and “modify” behaviors respectively,\n+with noamend being the default.\n+\n+For example, imagine we created two divergent changes like this:\n+\n+$ touch foo && git add . && git commit -m \"foo\" && git tag A\n+$ touch bar && git add . && git commit -m \"bar\" && git tag B\n+$ touch baz && git add . && git commit --amend -m \"bar and baz\"\n+$ git tag C\n+$ git checkout B\n+$ touch bam && git add . && git commit --amend -m \"bar and bam\"\n+$ git tag D\n+\n+At this point the commit graph looks like this:\n+\n+A(tree=[foo])\n+B(tree=[bar], parent=A)\n+C(tree=[bar, baz], parent=A)\n+D(tree=[bar, bam], parent=A)\n+Cmeta(content=C, obsoletes=B)\n+Dmeta(content=D, obsoletes=B)\n+\n+There would be three active changes with heads pointing as follows:\n+\n+metas/changeA=A\n+metas/changeB=Cmeta\n+metas/changeB2=Dmeta\n+\n+ChangeB and changeB2 are divergent at this point. Lets consider what happens if\n+perform each type of merge between changeB and changeB2.\n+\n+Merge example: Amend merge\n+One way to resolve divergent changes is to use an amend merge. Recall that HEAD\n+is currently pointing to D at this point.\n+\n+$ git merge --amend metas/changeB\n+\n+Here we’ve asked for an amend merge since we’re trying to resolve divergence\n+between two versions of the same change. There are no conflicts so we end up\n+with this:\n+\n+E(tree=[bar, baz, bam], parent=A)\n+Emeta(content=E, obsoletes=[Cmeta, Dmeta])\n+\n+With the following branches:\n+\n+metas/changeA=A\n+metas/changeB=Emeta\n+metas/changeB2=Emeta\n+\n+Notice that the result of the “amend merge” is a replacement for C and D rather\n+than a new commit with C and D as parents (as a normal merge would have\n+produced). The parents of the amend merge are the parents of C and D which - in\n+this case - is just A, so the result is not a merge commit. Also notice that\n+changeB and changeB2 are now aliases for the same change.\n+\n+Merge example: Noamend merge\n+Consider what would have happened if we’d used a noamend merge instead. Recall\n+that HEAD was at D and our branches looked like this:\n+\n+metas/changeA=A\n+metas/changeB=Cmeta\n+metas/changeB2=Dmeta\n+\n+$ git merge --noamend metas/changeB\n+\n+That would produce the sort of merge we’d normally expect today:\n+\n+F(tree=[bar, baz, bam], parent=[C, D])\n+\n+And our changes would look like this:\n+metas/changeA=A\n+metas/changeB=Cmeta\n+metas/changeB2=Dmeta\n+metas/changeF=F\n+\n+In this case, changeB and changeB2 are still divergent and we’ve created a new\n+change for our merge commit. However, this is just a temporary state. The next\n+time we run the “evolve” command, it will discover the divergence but also\n+discover the merge commit F that resolves it. Evolve will suggest converting F\n+into an amend merge in order to resolve the divergence and will display the\n+command for doing so.\n+\n+Rebase\n+------\n+In general the rebase command is treated as a modify command. When a change is\n+rebased, the new commit replaces the original.\n+\n+Rebase --abort is special. Its intent is to restore git to the state it had\n+prior to running rebase. It should move back any changes to point to the refs\n+they had prior to running rebase and delete any new changes that were created as\n+part of the rebase. To achieve this, rebase will save the state of all changes\n+in refs/metas prior to running rebase and will restore the entire namespace\n+after rebase completes (deleting any newly-created changes). Newly-created\n+metacommits are left in place, but will have no effect until garbage collected\n+since metacommits are only used if they are reachable from refs/metas.\n+\n+Change\n+------\n+The “change” command can be used to list, rename, reset or delete change. It has\n+a number of subcommands.\n+\n+The \"list\" subcommand lists local changes. If given the -r argument, it lists\n+remote changes.\n+\n+The \"rename\" subcommand renames a change, given its old and new name. If the old\n+name is omitted and there is exactly one change pointing to the current HEAD,\n+that change is renamed. If there are no changes pointing to the current HEAD,\n+one is created with the given name.\n+\n+The \"forget\" subcommand deletes a change by deleting its ref from the metas/\n+namespace. This is the normal way to delete extra aliases for a change if the\n+change has more than one name. By default, this will refuse to delete the last\n+alias for a change if there are any other changes that reference this change as\n+a parent.\n+\n+The \"update\" subcommand adds a new state to a change. It uses the default\n+algorithm for assigning change names. If the content commit is omitted, HEAD is\n+used. If given the optional --force argument, it will overwrite any existing\n+change of the same name. This latter form of \"update\" can be used to effectively\n+reset changes.\n+\n+The \"update\" command can accept any number of --origin and --replace arguments.\n+If any are present, the resulting change branch will point to a metacommit\n+containing the given origin and replacement edges.\n+\n+The \"abandon\" command deletes a change using obsolescence markers. It marks the\n+change as being obsolete and having been replaced by its parent. If given no\n+arguments, it applies to the current commit. Running evolve will cause any\n+abandoned changes to be removed from the branch. Any child changes will be\n+reparented on top of the parent of the abandoned change. If the current change\n+is abandoned, HEAD will move to point to its parent.\n+\n+The \"restore\" command restores a previously-abandoned change.\n+\n+The \"prune\" command deletes all obsolete changes and all changes that are\n+present in the given branch. Note that such changes can be recovered from the\n+reflog.\n+\n+Combined with the GC protection that is offered, this is intended to facilitate\n+a workflow that relies on changes instead of branches. Users could choose to\n+work with no local branches and use changes instead - both for mailing list and\n+gerrit workflows.\n+\n+Log\n+---\n+When a commit is shown in git log that is part of a change, it is decorated with\n+extra change information. If it is the head of a change, the name of the change\n+is shown next to the list of branches. If it is obsolete, it is decorated with\n+the text “obsolete, commits behind ”.\n+\n+Log gets a new --obslog argument indicating that the obsolescence graph should\n+be followed instead of the commit graph. This also changes the default\n+formatting options to make them more appropriate for viewing different\n+iterations of the same commit.\n+\n+Pull\n+----\n+\n+Pull gets an --evolve argument that will automatically attempt to run \"evolve\"\n+on any affected branches after pulling.\n+\n+We also introduce an \"evolve\" enum value for the branch..rebase config\n+value. When set, the evolve behavior will happen automatically for that branch\n+after every pull even if the --evolve argument is not used.\n+\n+Next\n+----\n+\n+The \"next\" command will reset HEAD to a non-obsolete commit that refers to this\n+change as its parent. If there is more than one such change, the user will be\n+prompted. If given the --evolve argument, the next commit will be evolved if\n+necessary first.\n+\n+The \"next\" command can be thought of as the opposite of\n+\"git reset --hard HEAD^\" in that it navigates to a child commit rather than a\n+parent.\n+\n+Prev\n+----\n+\n+The \"prev\" command will reset HEAD to the latest version of the parent change.\n+If the parent change isn't obsolete, this is equivalent to\n+\"git reset --hard HEAD^\". If the parent commit is obsolete, it resets to the\n+latest replacement for the parent commit.\n+\n+Other options considered\n+========================\n+We considered several other options for storing the obsolescence graph. This\n+section describes the other options and why they were rejected.\n+\n+Commit header\n+-------------\n+Add an “obsoletes” field to the commit header that points backwards from a\n+commit to the previous commits it obsoletes.\n+\n+Pros:\n+- Very simple\n+- Easy to traverse from a commit to the previous commits it obsoletes.\n+Cons:\n+- Adds a cost to the storage format, even for commits where the change history\n+ is uninteresting.\n+- Unconditionally prevents the change history from being garbage collected.\n+- Always causes the change history to be shared when pushing or pulling changes.\n+\n+Git notes\n+---------\n+Instead of storing obsolescence information in metacommits, the metacommit\n+content could go in a new notes namespace - say refs/notes/metacommit. Each note\n+would contain the list of obsolete and origin parents. An automerger could\n+be supplied to make it easy to merge the metacommit notes from different remotes.\n+\n+Pros:\n+- Easy to locate all commits obsoleted by a given commit (since there would only\n+ be one metacommit for any given commit).\n+Cons:\n+- Wrong GC behavior (obsolete commits wouldn’t automatically be retained by GC)\n+ unless we introduced a special case for these kinds of notes.\n+- No way to selectively share or pull the metacommits for one specific change.\n+ It would be all-or-nothing, which would be expensive. This could be addressed\n+ by changes to the protocol, but this would be invasive.\n+- Requires custom auto-merging behavior on fetch.\n+\n+Tags\n+----\n+Put the content of the metacommit in a message attached to tag on the\n+replacement commit. This is very similar to the git notes approach and has the\n+same pros and cons.\n+\n+Simple forward references\n+-------------------------\n+Record an edge from an obsolete commit to its replacement in this form:\n+\n+refs/obsoletes/\n+\n+pointing to commit as an indication that B is the replacement for the\n+obsolete commit A.\n+\n+Pros:\n+- Protects from being garbage collected.\n+- Fast lookup for the evolve operation, without additional search structures\n+ (“what is the replacement for ?” is very fast).\n+\n+Cons:\n+- Can’t represent divergence (which is a P0 requirement).\n+- Creates lots of refs (which can be inefficient)\n+- Doesn’t provide a way to fetch only refs for a specific change.\n+- The obslog command requires a search of all refs.\n+\n+Complex forward references\n+--------------------------\n+Record an edge from an obsolete commit to its replacement in this form:\n+\n+refs/obsoletes//obs_\n+\n+Pointing to commit as an indication that B is the replacement for obsolete\n+commit A.\n+\n+Pros:\n+- Permits sharing and fetching refs for only a specific change.\n+- Supports divergence\n+- Protects from being garbage collected.\n+\n+Cons:\n+- Creates lots of refs, which is inefficient.\n+- Doesn’t provide a good lookup structure for lookups in either direction.\n+\n+Backward references\n+-------------------\n+Record an edge from a replacement commit to the obsolete one in this form:\n+\n+refs/obsolescences/\n+\n+Cons:\n+- Doesn’t provide a way to resolve divergence (which is a P0 requirement).\n+- Doesn’t protect from being garbage collected (which could be fixed by\n+ combining this with a refs/metas namespace, as in the metacommit variant).\n+\n+Obsolescences file\n+------------------\n+Create a custom file (or files) in .git recording obsolescences.\n+\n+Pros:\n+- Can store exactly the information we want with exactly the performance we want\n+ for all operations. For example, there could be a disk-based hashtable\n+ permitting constant time lookups in either direction.\n+\n+Cons:\n+- Handling GC, pushing, and pulling would all require custom solutions. GC\n+ issues could be addressed with a repository format extension.\n+\n+Squash points\n+-------------\n+We treat changes like topic branches, and use special squash points to mark\n+places in the commit graph that separate changes.\n+\n+We create and update change branches in refs/metas at the same time we\n+would have in the metacommit proposal. However, rather than pointing to a\n+metacommit branch they point to normal commits and are treated as “squash\n+points” - markers for sequences of commits intended to be squashed together on\n+submission.\n+\n+Amends and rebases work differently than they do now. Rather than actually\n+containing the desired state of a commit, they contain a delta from the previous\n+version along with a squash point indicating that the preceding changes are\n+intended to be squashed on submission. Specifically, amends would become new\n+changes and rebases would become merge commits with the old commit and new\n+parent as parents.\n+\n+When the changes are finally submitted, the squashes are executed, producing the\n+final version of the commit.\n+\n+In addition to the squash points, git would maintain a set of “nosquash” tags\n+for commits that were used as ancestors of a change that are not meant to be\n+included in the squash.\n+\n+For example, if we have this commit graph:\n+\n+A(...)\n+B(parent=A)\n+C(parent=B)\n+\n+...and we amend B to produce D, we’d get:\n+\n+A(...)\n+B(parent=A)\n+C(parent=B)\n+D(parent=B)\n+\n+...along with a new change branch indicating D should be squashed with its\n+parents when submitted:\n+\n+metas/changeB = D\n+metas/changeC = C\n+\n+We’d also create a nosquash tag for A indicating that A shouldn’t be included\n+when changeB is squashed.\n+\n+If a user amends the change again, they’d get:\n+\n+A(...)\n+B(parent=A)\n+C(parent=B)\n+D(parent=B)\n+E(parent=D)\n+\n+metas/changeB = E\n+metas/changeC = C\n+\n+Pros:\n+- Good GC behavior.\n+- Provides a natural way to share changes (they’re just normal branches).\n+- Merge-base works automatically without special cases.\n+- Rewriting the obslog would be easy using existing git commands.\n+- No new data types needed.\n+Cons:\n+- No way to connect the squashed version of a change to the original, so no way\n+ to automatically clean up old changes. This also means users lose all benefits\n+ of the evolve command if they prematurely squash their commits. This may occur\n+ if a user thinks a change is ready for submission, squashes it, and then later\n+ discovers an additional change to make.\n+- Histories would look very cluttered (users would see all previous edits to\n+ their commit in the commit log, and all previous rebases would show up as\n+ merges). Could be quite hard for users to tell what is going on. (Possible\n+ fix: also implement a new smart log feature that displays the log as though\n+ the squashes had occurred).\n+- Need to change the current behavior of current commands (like amend and\n+ rebase) in ways that will be unexpected to many users.\n-- \ngitgitgadget\n\n"},{"id":"464236","messageId":"f7a90700e0e79b3ae6fbc5cf32723da04e907347.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 07/10] evolve: implement the git change command","fromName":"Stefan Xenos via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:14Z","receivedAt":"2022-10-05T15:01:11Z","isPatch":true,"sender":{"key":"sxenos@google.com","avatar":null},"body":"From: Stefan Xenos \n\nImplement the git change update command, which\nare sufficient for constructing change graphs.\n\nFor example, to create a new change (a stable name) that refers to HEAD:\n\ngit change update -c HEAD\n\nTo record a rebase or amend in the change graph:\n\ngit change update -c -r \n\nTo record a cherry-pick in the change graph:\n\ngit change update -c -o \n\nSigned-off-by: Stefan Xenos \nSigned-off-by: Chris Poucet \n---\n .gitignore | 1 +\n Makefile | 1 +\n builtin.h | 1 +\n builtin/change.c | 253 +++++++++++++++++++++++++++++++++++++++++++++++\n git.c | 1 +\n ref-filter.c | 2 +-\n ref-filter.h | 4 +\n 7 files changed, 262 insertions(+), 1 deletion(-)\n create mode 100644 builtin/change.c\n\ndiff --git a/.gitignore b/.gitignore\nindex b3dcafcb331..a57fd8d8897 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -28,6 +28,7 @@\n /git-bugreport\n /git-bundle\n /git-cat-file\n+/git-change\n /git-check-attr\n /git-check-ignore\n /git-check-mailmap\ndiff --git a/Makefile b/Makefile\nindex 68082ef94c7..82f68f13d9f 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1142,6 +1142,7 @@ BUILTIN_OBJS += builtin/branch.o\n BUILTIN_OBJS += builtin/bugreport.o\n BUILTIN_OBJS += builtin/bundle.o\n BUILTIN_OBJS += builtin/cat-file.o\n+BUILTIN_OBJS += builtin/change.o\n BUILTIN_OBJS += builtin/check-attr.o\n BUILTIN_OBJS += builtin/check-ignore.o\n BUILTIN_OBJS += builtin/check-mailmap.o\ndiff --git a/builtin.h b/builtin.h\nindex 8901a34d6bf..c10f20c972c 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -122,6 +122,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix);\n int cmd_bugreport(int argc, const char **argv, const char *prefix);\n int cmd_bundle(int argc, const char **argv, const char *prefix);\n int cmd_cat_file(int argc, const char **argv, const char *prefix);\n+int cmd_change(int argc, const char **argv, const char *prefix);\n int cmd_checkout(int argc, const char **argv, const char *prefix);\n int cmd_checkout__worker(int argc, const char **argv, const char *prefix);\n int cmd_checkout_index(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/change.c b/builtin/change.c\nnew file mode 100644\nindex 00000000000..e4e8e15b768\n--- /dev/null\n+++ b/builtin/change.c\n@@ -0,0 +1,253 @@\n+#include \"builtin.h\"\n+#include \"ref-filter.h\"\n+#include \"parse-options.h\"\n+#include \"metacommit.h\"\n+#include \"config.h\"\n+\n+static const char * const builtin_change_usage[] = {\n+\tN_(\"git change list [...]\"),\n+\tN_(\"git change update [--force] [--replace ...] \"\n+\t \"[--origin ...] [--content ]\"),\n+\tNULL\n+};\n+\n+static const char * const builtin_list_usage[] = {\n+\tN_(\"git change list [...]\"),\n+\tNULL\n+};\n+\n+static const char * const builtin_update_usage[] = {\n+\tN_(\"git change update [--force] [--replace ...] \"\n+\t\"[--origin ...] [--content ]\"),\n+\tNULL\n+};\n+\n+static int change_list(int argc, const char **argv, const char* prefix)\n+{\n+\tstruct option options[] = {\n+\t\tOPT_END()\n+\t};\n+\tstruct ref_filter filter = { 0 };\n+\tstruct ref_sorting *sorting;\n+\tstruct string_list sorting_options = STRING_LIST_INIT_DUP;\n+\tstruct ref_format format = REF_FORMAT_INIT;\n+\tstruct ref_array array = { 0 };\n+\tsize_t i;\n+\n+\targc = parse_options(argc, argv, prefix, options, builtin_list_usage, 0);\n+\n+\tsetup_ref_filter_porcelain_msg();\n+\n+\tfilter.kind = FILTER_REFS_CHANGES;\n+\tfilter.name_patterns = argv;\n+\n+\tfilter_refs(&array, &filter, FILTER_REFS_CHANGES);\n+\n+\t/* TODO: This causes a crash. It sets one of the atom_value handlers to\n+\t * something invalid, which causes a crash later when we call\n+\t * show_ref_array_item. Figure out why this happens and put back the sorting.\n+\t *\n+\t * sorting = ref_sorting_options(&sorting_options);\n+\t * ref_array_sort(sorting, &array); */\n+\n+\tif (!format.format)\n+\t\tformat.format = \"%(refname:lstrip=1)\";\n+\n+\tif (verify_ref_format(&format))\n+\t\tdie(_(\"unable to parse format string\"));\n+\n+\tsorting = ref_sorting_options(&sorting_options);\n+\tref_array_sort(sorting, &array);\n+\n+\n+\tfor (i = 0; i < array.nr; i++) {\n+\t\tstruct strbuf output = STRBUF_INIT;\n+\t\tstruct strbuf err = STRBUF_INIT;\n+\t\tif (format_ref_array_item(array.items[i], &format, &output, &err))\n+\t\t\tdie(\"%s\", err.buf);\n+\t\tfwrite(output.buf, 1, output.len, stdout);\n+\t\tputchar('\\n');\n+\n+\t\tstrbuf_release(&err);\n+\t\tstrbuf_release(&output);\n+\t}\n+\n+\tref_array_clear(&array);\n+\tref_sorting_release(sorting);\n+\n+\treturn 0;\n+}\n+\n+struct update_state {\n+\tint options;\n+\tconst char* change;\n+\tconst char* content;\n+\tstruct string_list replace;\n+\tstruct string_list origin;\n+};\n+\n+#define UPDATE_STATE_INIT { \\\n+\t.content = \"HEAD\", \\\n+\t.replace = STRING_LIST_INIT_NODUP, \\\n+\t.origin = STRING_LIST_INIT_NODUP \\\n+}\n+\n+static void clear_update_state(struct update_state *state)\n+{\n+\tstring_list_clear(&state->replace, 0);\n+\tstring_list_clear(&state->origin, 0);\n+}\n+\n+static int update_option_parse_replace(const struct option *opt,\n+\t\t\t\t const char *arg, int unset)\n+{\n+\tstruct update_state *state = opt->value;\n+\tstring_list_append(&state->replace, arg);\n+\treturn 0;\n+}\n+\n+static int update_option_parse_origin(const struct option *opt,\n+\t\t\t\t const char *arg, int unset)\n+{\n+\tstruct update_state *state = opt->value;\n+\tstring_list_append(&state->origin, arg);\n+\treturn 0;\n+}\n+\n+static int resolve_commit(const char *committish, struct object_id *result)\n+{\n+\tstruct commit *commit;\n+\tif (get_oid_committish(committish, result))\n+\t\tdie(_(\"failed to resolve '%s' as a valid revision.\"), committish);\n+\tcommit = lookup_commit_reference(the_repository, result);\n+\tif (!commit)\n+\t\tdie(_(\"could not parse object '%s'.\"), committish);\n+\toidcpy(result, &commit->object.oid);\n+\treturn 0;\n+}\n+\n+static void resolve_commit_list(const struct string_list *commitsish_list,\n+\tstruct oid_array* result)\n+{\n+\tstruct string_list_item *item;\n+\n+\tfor_each_string_list_item(item, commitsish_list) {\n+\t\tstruct object_id next;\n+\t\tresolve_commit(item->string, &next);\n+\t\toid_array_append(result, &next);\n+\t}\n+}\n+\n+/*\n+ * Given the command-line options for the update command, fills in a\n+ * metacommit_data with the corresponding changes.\n+ */\n+static void get_metacommit_from_command_line(\n+\tconst struct update_state* commands, struct metacommit_data *result)\n+{\n+\tresolve_commit(commands->content, &(result->content));\n+\tresolve_commit_list(&(commands->replace), &(result->replace));\n+\tresolve_commit_list(&(commands->origin), &(result->origin));\n+}\n+\n+static int perform_update(\n+\tstruct repository *repo,\n+\tconst struct update_state *state,\n+\tstruct strbuf *err)\n+{\n+\tstruct metacommit_data metacommit = METACOMMIT_DATA_INIT;\n+\tstruct string_list changes = STRING_LIST_INIT_DUP;\n+\tint ret;\n+\tstruct string_list_item *item;\n+\n+\tget_metacommit_from_command_line(state, &metacommit);\n+\n+\tret = record_metacommit(\n+\t\trepo,\n+\t\t&metacommit,\n+\t\tstate->change,\n+\t\tstate->options,\n+\t\terr,\n+\t\t&changes);\n+\n+\tfor_each_string_list_item(item, &changes) {\n+\n+\t\tconst char* name = lstrip_ref_components(item->string, 1);\n+\t\tif (!name)\n+\t\t\tdie(_(\"failed to remove `refs/` from %s\"), item->string);\n+\n+\t\tif (item->util)\n+\t\t\tfprintf(stdout, _(\"Updated change %s\"), name);\n+\t\telse\n+\t\t\tfprintf(stdout, _(\"Created change %s\"), name);\n+\t\tputchar('\\n');\n+\t}\n+\n+\tstring_list_clear(&changes, 0);\n+\tclear_metacommit_data(&metacommit);\n+\n+\treturn ret;\n+}\n+\n+static int change_update(int argc, const char **argv, const char* prefix)\n+{\n+\tint result;\n+\tstruct strbuf err = STRBUF_INIT;\n+\tstruct update_state state = UPDATE_STATE_INIT;\n+\tstruct option options[] = {\n+\t\t{ OPTION_CALLBACK, 'r', \"replace\", &state, N_(\"commit\"),\n+\t\t\tN_(\"marks the given commit as being obsolete\"),\n+\t\t\t0, update_option_parse_replace },\n+\t\t{ OPTION_CALLBACK, 'o', \"origin\", &state, N_(\"commit\"),\n+\t\t\tN_(\"marks the given commit as being the origin of this commit\"),\n+\t\t\t0, update_option_parse_origin },\n+\n+\t\tOPT_STRING('c', \"content\", &state.content, N_(\"commit\"),\n+\t\t\t\t N_(\"identifies the new content commit for the change\")),\n+\t\tOPT_STRING('g', \"change\", &state.change, N_(\"commit\"),\n+\t\t\t\t N_(\"name of the change to update\")),\n+\t\tOPT_SET_INT_F('n', \"new\", &state.options,\n+\t\t\t N_(\"create a new change - do not append to any existing change\"),\n+\t\t\t UPDATE_OPTION_NOAPPEND, 0),\n+\t\tOPT_SET_INT_F('F', \"force\", &state.options,\n+\t\t\t N_(\"overwrite an existing change of the same name\"),\n+\t\t\t UPDATE_OPTION_FORCE, 0),\n+\t\tOPT_END()\n+\t};\n+\n+\targc = parse_options(argc, argv, prefix, options, builtin_update_usage, 0);\n+\tresult = perform_update(the_repository, &state, &err);\n+\n+\tif (result < 0) {\n+\t\terror(\"%s\", err.buf);\n+\t\tstrbuf_release(&err);\n+\t}\n+\n+\tclear_update_state(&state);\n+\n+\treturn result;\n+}\n+\n+int cmd_change(int argc, const char **argv, const char *prefix)\n+{\n+\tparse_opt_subcommand_fn *fn = NULL;\n+\t/* No options permitted before subcommand currently */\n+\tstruct option options[] = {\n+\t\tOPT_SUBCOMMAND(\"list\", &fn, change_list),\n+\t\tOPT_SUBCOMMAND(\"update\", &fn, change_update),\n+\t\tOPT_END()\n+\t};\n+\n+\targc = parse_options(argc, argv, prefix, options, builtin_change_usage,\n+\t\tPARSE_OPT_SUBCOMMAND_OPTIONAL);\n+\n+\tif (!fn) {\n+\t\tif (argc) {\n+\t\t\terror(_(\"unknown subcommand: `%s'\"), argv[0]);\n+\t\t\tusage_with_options(builtin_change_usage, options);\n+\t\t}\n+\t\tfn = change_list;\n+\t}\n+\n+\treturn !!fn(argc, argv, prefix);\n+}\ndiff --git a/git.c b/git.c\nindex da411c53822..837b1abc53b 100644\n--- a/git.c\n+++ b/git.c\n@@ -498,6 +498,7 @@ static struct cmd_struct commands[] = {\n \t{ \"bugreport\", cmd_bugreport, RUN_SETUP_GENTLY },\n \t{ \"bundle\", cmd_bundle, RUN_SETUP_GENTLY },\n \t{ \"cat-file\", cmd_cat_file, RUN_SETUP },\n+\t{ \"change\", cmd_change, RUN_SETUP},\n \t{ \"check-attr\", cmd_check_attr, RUN_SETUP },\n \t{ \"check-ignore\", cmd_check_ignore, RUN_SETUP | NEED_WORK_TREE },\n \t{ \"check-mailmap\", cmd_check_mailmap, RUN_SETUP },\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 6a1789c623f..2d7a919d547 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1557,7 +1557,7 @@ static inline char *copy_advance(char *dst, const char *src)\n \treturn dst;\n }\n \n-static const char *lstrip_ref_components(const char *refname, int len)\n+const char *lstrip_ref_components(const char *refname, int len)\n {\n \tlong remaining = len;\n \tconst char *start = xstrdup(refname);\ndiff --git a/ref-filter.h b/ref-filter.h\nindex db3ee44e4dc..193700694ad 100644\n--- a/ref-filter.h\n+++ b/ref-filter.h\n@@ -145,4 +145,8 @@ struct ref_array_item *ref_array_push(struct ref_array *array,\n \t\t\t\t const char *refname,\n \t\t\t\t const struct object_id *oid);\n \n+/* Strips `len` prefix components from the refname. */\n+const char *lstrip_ref_components(const char *refname, int len);\n+\n+\n #endif /* REF_FILTER_H */\n-- \ngitgitgadget\n\n"},{"id":"464238","messageId":"e67ff668fffaf6d38ec77d332319d60acc9b2454.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 09/10] evolve: add documentation for `git change`","fromName":"Chris Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:16Z","receivedAt":"2022-10-05T15:01:15Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"From: Chris Poucet \n\nSigned-off-by: Chris Poucet \n---\n Documentation/git-change.txt | 55 ++++++++++++++++++++++++++++++++++++\n 1 file changed, 55 insertions(+)\n create mode 100644 Documentation/git-change.txt\n\ndiff --git a/Documentation/git-change.txt b/Documentation/git-change.txt\nnew file mode 100644\nindex 00000000000..ea9a8e619b9\n--- /dev/null\n+++ b/Documentation/git-change.txt\n@@ -0,0 +1,55 @@\n+git-change(1)\n+=============\n+\n+NAME\n+----\n+git-change - Create, list, update or delete changes\n+\n+SYNOPSIS\n+--------\n+[verse]\n+'git change' list [...]\n+'git change' update [-g | -n] [--force] [--replace ...] [--origin ...] [--content ]\n+'git change' delete ...\n+\n+DESCRIPTION\n+-----------\n+\n+`git change list`: lists all existing s.\n+\n+`git change delete`: deletes the given s.\n+\n+`git change update`: creates or updates a .\n+\n+If no arguments are given to `update` then a change is added to the\n+`refs/metas/` directory, unless a change already exists for the given commit.\n+\n+A starts with `metas/` and represents the current change that is\n+being worked on.\n+\n+OPTIONS\n+-------\n+-c::\n+--content::\n+\tIdentifies the content commit for the change\n+\n+-o::\n+--origin::\n+\tMarks the given commit as being the origin of this commit.\n+\n+-r::\n+--replace::\n+\tMarks the given commit as being obsoleted by the new commit.\n+\n+-g::\n+\t to update\n+\n+-n::\n+\tIndicates that the change is new and an existing change should not be updated.\n+\n+--force::\n+\tOverwite an existing change of the same name.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\n-- \ngitgitgadget\n\n"},{"id":"464239","messageId":"a0669fa63a1c3887798d3f8461dc5bd38704c0a0.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 08/10] evolve: add delete command","fromName":"Chris Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:15Z","receivedAt":"2022-10-05T15:01:18Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"From: Chris Poucet \n\nThe delete command allows a user to delete one or more changes.\nThis effectively deletes the corresponding /refs/metas/foo ref.\n\nSigned-off-by: Chris Poucet \n---\n builtin/change.c | 77 ++++++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 77 insertions(+)\n\ndiff --git a/builtin/change.c b/builtin/change.c\nindex e4e8e15b768..12ea5f68197 100644\n--- a/builtin/change.c\n+++ b/builtin/change.c\n@@ -3,11 +3,13 @@\n #include \"parse-options.h\"\n #include \"metacommit.h\"\n #include \"config.h\"\n+#include \"refs.h\"\n \n static const char * const builtin_change_usage[] = {\n \tN_(\"git change list [...]\"),\n \tN_(\"git change update [--force] [--replace ...] \"\n \t \"[--origin ...] [--content ]\"),\n+\tN_(\"git change delete ...\"),\n \tNULL\n };\n \n@@ -22,6 +24,11 @@ static const char * const builtin_update_usage[] = {\n \tNULL\n };\n \n+static const char * const builtin_delete_usage[] = {\n+\tN_(\"git change delete ...\"),\n+\tNULL\n+};\n+\n static int change_list(int argc, const char **argv, const char* prefix)\n {\n \tstruct option options[] = {\n@@ -228,6 +235,75 @@ static int change_update(int argc, const char **argv, const char* prefix)\n \treturn result;\n }\n \n+typedef int (*each_change_name_fn)(const char *name, const char *ref,\n+\t\t\t\t const struct object_id *oid, void *cb_data);\n+\n+static int for_each_change_name(const char **argv, each_change_name_fn fn,\n+\t\t\t\tvoid *cb_data)\n+{\n+\tconst char **p;\n+\tstruct strbuf ref = STRBUF_INIT;\n+\tint had_error = 0;\n+\tstruct object_id oid;\n+\n+\tfor (p = argv; *p; p++) {\n+\t\tstrbuf_reset(&ref);\n+\t\t/* Convenience functionality to avoid having to type `metas/` */\n+\t\tif (strncmp(\"metas/\", *p, 5)) {\n+\t\t\tstrbuf_addf(&ref, \"refs/metas/%s\", *p);\n+\t\t} else {\n+\t\t\tstrbuf_addf(&ref, \"refs/%s\", *p);\n+\t\t}\n+\t\tif (read_ref(ref.buf, &oid)) {\n+\t\t\terror(_(\"change '%s' not found.\"), *p);\n+\t\t\thad_error = 1;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (fn(*p, ref.buf, &oid, cb_data))\n+\t\t\thad_error = 1;\n+\t}\n+\tstrbuf_release(&ref);\n+\treturn had_error;\n+}\n+\n+static int collect_changes(const char *name, const char *ref,\n+\t\t\t const struct object_id *oid, void *cb_data)\n+{\n+\tstruct string_list *ref_list = cb_data;\n+\n+\tstring_list_append(ref_list, ref);\n+\tref_list->items[ref_list->nr - 1].util = oiddup(oid);\n+\treturn 0;\n+}\n+\n+static int change_delete(int argc, const char **argv, const char* prefix) {\n+\tint result = 0;\n+\tstruct string_list refs_to_delete = STRING_LIST_INIT_DUP;\n+\tstruct string_list_item *item;\n+\tstruct option options[] = {\n+\t\tOPT_END()\n+\t};\n+\n+\targc = parse_options(argc, argv, prefix, options, builtin_delete_usage, 0);\n+\n+\tresult = for_each_change_name(argv, collect_changes, (void *)&refs_to_delete);\n+\tif (delete_refs(NULL, &refs_to_delete, REF_NO_DEREF))\n+\t\tresult = 1;\n+\n+\tfor_each_string_list_item(item, &refs_to_delete) {\n+\t\tconst char *name = item->string;\n+\t\tstruct object_id *oid = item->util;\n+\t\tif (!ref_exists(name))\n+\t\t\tprintf(_(\"Deleted change '%s' (was %s)\\n\"),\n+\t\t\t\titem->string + 5,\n+\t\t\t\tfind_unique_abbrev(oid, DEFAULT_ABBREV));\n+\n+\t\tfree(oid);\n+\t}\n+\tstring_list_clear(&refs_to_delete, 0);\n+\treturn result;\n+}\n+\n int cmd_change(int argc, const char **argv, const char *prefix)\n {\n \tparse_opt_subcommand_fn *fn = NULL;\n@@ -235,6 +311,7 @@ int cmd_change(int argc, const char **argv, const char *prefix)\n \tstruct option options[] = {\n \t\tOPT_SUBCOMMAND(\"list\", &fn, change_list),\n \t\tOPT_SUBCOMMAND(\"update\", &fn, change_update),\n+\t\tOPT_SUBCOMMAND(\"delete\", &fn, change_delete),\n \t\tOPT_END()\n \t};\n \n-- \ngitgitgadget\n\n"},{"id":"464240","messageId":"37042b58cda070bd562d7f64d654bd407e8572e4.1664981958.git.gitgitgadget@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"[PATCH v2 10/10] evolve: add tests for the git-change command","fromName":"Chris Poucet via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-05T14:59:17Z","receivedAt":"2022-10-05T15:01:52Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"From: Chris Poucet \n\nSigned-off-by: Phillip Wood \nSigned-off-by: Chris Poucet \n---\n t/t9990-changes.sh | 148 +++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 148 insertions(+)\n create mode 100755 t/t9990-changes.sh\n\ndiff --git a/t/t9990-changes.sh b/t/t9990-changes.sh\nnew file mode 100755\nindex 00000000000..11fbd8ba49c\n--- /dev/null\n+++ b/t/t9990-changes.sh\n@@ -0,0 +1,148 @@\n+#!/bin/sh\n+\n+test_description='git change - low level meta-commit management'\n+\n+. ./test-lib.sh\n+\n+. \"$TEST_DIRECTORY\"/lib-rebase.sh\n+\n+test_expect_success 'setup commits and meta-commits' '\n+ for c in one two three\n+ do\n+ test_commit $c &&\n+ git change update --content $c >actual 2>err &&\n+ echo \"Created change metas/$c\" >expect &&\n+ test_cmp expect actual &&\n+ test_must_be_empty err &&\n+ test_cmp_rev refs/metas/$c $c || return 1\n+ done\n+'\n+\n+# Check a meta-commit has the correct parents Call with the object\n+# name of the meta-commit followed by pairs of type and parent\n+check_meta_commit () {\n+ name=$1\n+ shift\n+ while test $# -gt 0\n+ do\n+ printf '%s %s\\n' $1 $(git rev-parse --verify $2)\n+ shift\n+ shift\n+ done | sort >expect\n+ git cat-file commit $name >metacommit &&\n+ # commit body should consist of parent-type\n+ types=\"$(sed -n '/^$/ {\n+ :loop\n+ n\n+ s/^parent-type //\n+ p\n+ b loop\n+ }' metacommit)\" &&\n+ while read key value\n+ do\n+ # TODO: don't sort the first parent\n+ if test \"$key\" = \"parent\"\n+ then\n+ type=\"${types%% *}\"\n+ test -n \"$type\" || return 1\n+ printf '%s %s\\n' $type $value\n+ types=\"${types#?}\"\n+ types=\"${types# }\"\n+ elif test \"$key\" = \"tree\"\n+ then\n+ test_cmp_rev \"$value\" $EMPTY_TREE || return 1\n+ elif test -z \"$key\"\n+ then\n+ # only parse commit headers\n+ break\n+ fi\n+ done actual-unsorted &&\n+ test -z \"$types\" &&\n+ sort >actual out 2>err &&\n+ echo \"Updated change metas/one\" >expect &&\n+ test_cmp expect out &&\n+ test_must_be_empty err &&\n+ git change update --replace tags/two --content HEAD@{2} &&\n+ oid=$(git rev-parse --verify metas/two) &&\n+ git change update --replace HEAD@{2} --replace tags/three \\\n+ --content HEAD &&\n+\n+ # check meta-commits\n+ check_meta_commit metas/one c HEAD~1 r tags/one &&\n+ check_meta_commit $oid c HEAD@{2} r tags/two &&\n+ # NB this checks that \"git change update\" uses the meta-commit ($oid)\n+ # corresponding to the replaces commit (HEAD@2 above) given on the\n+ # commandline.\n+ check_meta_commit metas/two c HEAD r $oid r tags/three &&\n+ check_meta_commit metas/three c HEAD r $oid r tags/three\n+'\n+\n+reset_meta_commits () {\n+ for c in one two three\n+ do\n+ echo \"update refs/metas/$c refs/tags/$c^0\"\n+ done | git update-ref --stdin\n+}\n+\n+test_expect_success 'override change name' '\n+ # TODO: builtin/change.c expects --change to be the full refname,\n+ # ideally it would prepend refs/metas to the string given by the\n+ # user.\n+ git change update --change refs/metas/another-one --content one &&\n+ test_cmp_rev metas/another-one one\n+'\n+\n+test_expect_success 'non-fast forward meta-commit update refused' '\n+ test_must_fail git change update --change refs/metas/one --content two \\\n+ >out 2>err &&\n+ echo \"error: non-fast-forward update to ${SQ}refs/metas/one${SQ}\" \\\n+ >expect &&\n+ test_cmp expect err &&\n+ test_must_be_empty out\n+'\n+\n+test_expect_success 'forced non-fast forward update succeeds' '\n+ git change update --change refs/metas/one --content two --force \\\n+ >out 2>err &&\n+ echo \"Updated change metas/one\" >expect &&\n+ test_cmp expect out &&\n+ test_must_be_empty err\n+'\n+\n+test_expect_success 'list changes' '\n+ cat >expect <<-\\EOF &&\n+metas/another-one\n+metas/one\n+metas/three\n+metas/two\n+EOF\n+ git change list >actual &&\n+ test_cmp expect actual\n+'\n+\n+test_expect_success 'delete change' '\n+ git change delete metas/one &&\n+ cat >expect <<-\\EOF &&\n+metas/another-one\n+metas/three\n+metas/two\n+EOF\n+ git change list >actual &&\n+ test_cmp expect actual\n+'\n+\n+test_done\n-- \ngitgitgadget\n"},{"id":"464241","messageId":"CAN9+7XcYFa+Y9jsJSEmQhf29TUZADoz8=SzcNbjCH8ewqYriYg@mail.gmail.com","threadId":"58504","inReplyTo":"a5eb93254191b7ae9c17ce52e056955c669ea007.1664981958.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 01/10] technical doc: add a design doc for the evolve command","fromName":"Chris Poucet","fromEmail":"poucet@google.com","sentAt":"2022-10-05T15:16:10Z","receivedAt":"2022-10-05T15:16:33Z","isPatch":true,"sender":{"key":"poucet@google.com","avatar":null},"body":"One thing that is not clear to me is whether this is the desired\ndirection. I took at look at the git review notes but it was hard to\nget a sense of where people are at.\n\nWould love input on the design.\n\n\nOn Wed, Oct 5, 2022 at 4:59 PM Stefan Xenos via GitGitGadget\n wrote:\n>\n> From: Stefan Xenos \n>\n> This document describes what a change graph for\n> git would look like, the behavior of the evolve command,\n> and the changes planned for other commands.\n>\n> It was originally proposed in 2018, see\n> https://public-inbox.org/git/20181115005546.212538-1-sxenos@google.com/\n>\n> Signed-off-by: Stefan Xenos \n> Signed-off-by: Chris Poucet \n> ---\n> Documentation/technical/evolve.txt | 1070 ++++++++++++++++++++++++++++\n> 1 file changed, 1070 insertions(+)\n> create mode 100644 Documentation/technical/evolve.txt\n>\n> diff --git a/Documentation/technical/evolve.txt b/Documentation/technical/evolve.txt\n> new file mode 100644\n> index 00000000000..2051ea77b8a\n> --- /dev/null\n> +++ b/Documentation/technical/evolve.txt\n> @@ -0,0 +1,1070 @@\n> +Evolve\n> +======\n> +\n> +Objective\n> +=========\n> +Create an \"evolve\" command to help users craft a high quality commit history.\n> +Users can improve commits one at a time and in any order, then run git evolve to\n> +rewrite their recent history to ensure everything is up-to-date. We track\n> +amendments to a commit over time in a change graph. Users can share their\n> +progress with others by exchanging their change graphs using the standard push,\n> +fetch, and format-patch commands.\n> +\n> +Status\n> +======\n> +This proposal has not been implemented yet.\n> +\n> +Background\n> +==========\n> +Imagine you have three sequential changes up for review and you receive feedback\n> +that requires editing all three changes. We'll define the word \"change\"\n> +formally later, but for the moment let's say that a change is a work-in-progress\n> +whose final version will be submitted as a commit in the future.\n> +\n> +While you're editing one change, more feedback arrives on one of the others.\n> +What do you do?\n> +\n> +The evolve command is a convenient way to work with chains of commits that are\n> +under review. Whenever you rebase or amend a commit, the repository remembers\n> +that the old commit is obsolete and has been replaced by the new one. Then, at\n> +some point in the future, you can run \"git evolve\" and the correct sequence of\n> +rebases will occur in the correct order such that no commit has an obsolete\n> +parent.\n> +\n> +Part of making the \"evolve\" command work involves tracking the edits to a commit\n> +over time, which is why we need an change graph. However, the change\n> +graph will also bring other benefits:\n> +\n> +- Users can view the history of a change directly (the sequence of amends and\n> + rebases it has undergone, orthogonal to the history of the branch it is on).\n> +- It will be possible to quickly locate and list all the changes the user\n> + currently has in progress.\n> +- It can be used as part of other high-level commands that combine or split\n> + changes.\n> +- It can be used to decorate commits (in git log, gitk, etc) that are either\n> + obsolete or are the tip of a work in progress.\n> +- By pushing and pulling the change graph, users can collaborate more\n> + easily on changes-in-progress. This is better than pushing and pulling the\n> + commits themselves since the change graph can be used to locate a more\n> + specific merge base, allowing for better merges between different versions of\n> + the same change.\n> +- It could be used to correctly rebase local changes and other local branches\n> + after running git-filter-branch.\n> +- It can replace the change-id footer used by gerrit.\n> +\n> +Goals\n> +-----\n> +Legend: Goals marked with P0 are required. Goals marked with Pn should be\n> +attempted unless they interfere with goals marked with Pn-1.\n> +\n> +P0. All commands that modify commits (such as the normal commit --amend or\n> + rebase command) should mark the old commit as being obsolete and replaced by\n> + the new one. No additional commands should be required to keep the\n> + change graph up-to-date.\n> +P0. Any commit that may be involved in a future evolve command should not be\n> + garbage collected. Specifically:\n> + - Commits that obsolete another should not be garbage collected until\n> + user-specified conditions have occurred and the change has expired from\n> + the reflog. User specified conditions for removing changes include:\n> + - The user explicitly deleted the change.\n> + - The change was merged into a specific branch.\n> + - Commits that have been obsoleted by another should not be garbage\n> + collected if any of their replacements are still being retained.\n> +P0. A commit can be obsoleted by more than one replacement (called divergence).\n> +P0. Users must be able to resolve divergence (convergence).\n> +P1. Users should be able to share chains of obsolete changes in order to\n> + collaborate on WIP changes.\n> +P2. Such sharing should be at the user’s option. That is, it should be possible\n> + to directly share a change without also sharing the file states or commit\n> + comments from the obsolete changes that led up to it, and the choice not to\n> + share those commits should not require changing any commit hashes.\n> +P2. It should be possible to discard part or all of the change graph\n> + without discarding the commits themselves that are already present in\n> + branches and the reflog.\n> +P2. Provide sufficient information to replace gerrit's Change-Id footers.\n> +\n> +Similar technologies\n> +--------------------\n> +There are some other technologies that address the same end-user problem.\n> +\n> +Rebase -i can be used to solve the same problem, but users can't easily switch\n> +tasks midway through an interactive rebase or have more than one interactive\n> +rebase going on at the same time. It can't handle the case where you have\n> +multiple changes sharing the same parent when that parent needs to be rebased\n> +and won't let you collaborate with others on resolving a complicated interactive\n> +rebase. You can think of rebase -i as a top-down approach and the evolve command\n> +as the bottom-up approach to the same problem.\n> +\n> +Revup amend (https://github.com/Skydio/revup/blob/main/docs/amend.md)\n> +allows insertion of cached changes into any commit in\n> +the current history, and then reapplies the rest of history on top of\n> +those changes. It uses a \"git apply --cached\" engine under the hood so\n> +doesn't touch the working directory (although it will soon use the new\n> +git merge-tree). When paired with \"revup upload\" which creates and\n> +pushes multiple branches in the background for you, its possible to\n> +work on a \"graph\" of changes on a single branch linearly, then have\n> +the true graph structure created at upload time.\n> +\n> +git-revise (https://github.com/mystor/git-revise) does some very\n> +similar things except it uses \"git merge-file\" combined with manually\n> +merging the resulting trees. git branchstack\n> +(https://github.com/krobelus/git-branchstack) can also create branches\n> +in the background with the same mechanism.\n> +\n> +These tools don't store any external state, but as such also don't\n> +provide any specific collaboration mechanism for individual changes.\n> +\n> +Several patch queue managers have been built on top of git (such as topgit,\n> +stgit, and quilt). They address the same user need. However they also rely on\n> +state managed outside git that needs to be kept in sync. Such state can be\n> +easily damaged when running a git native command that is unaware of the patch\n> +queue. They also typically require an explicit initialization step to be done by\n> +the user which creates workflow problems.\n> +\n> +Mercurial implements a very similar feature in its EvolveExtension. The behavior\n> +of the evolve command itself is very similar, but the storage format for the\n> +change graph differs. In the case of mercurial, each change set can have one or\n> +more obsolescence markers that point to other changesets that they replace. This\n> +is similar to the \"Commit Headers\" approach considered in the other options\n> +appendix. The approach proposed here stores obsolescence information in a\n> +separate metacommit graph, which makes exchanging of obsolescence information\n> +optional.\n> +\n> +Mercurial's default behavior makes it easy to find and switch between\n> +non-obsolete changesets that aren't currently on any branch. We introduce the\n> +notion of a new ref namespace that enables a similar workflow via a different\n> +mechanism. Mercurial has the notion of changeset phases which isn't present\n> +in git and creates new ways for a changeset to diverge. Git doesn't need\n> +to deal with these issues, but it has to deal with the problems of picking an\n> +upstream branch as a target for rebases and protecting obsolescence information\n> +from GC. We also introduce some additional transformations (see\n> +obsolescence-over-cherry-pick, below) that aren't present in the mercurial\n> +implementation.\n> +\n> +Semi-related work\n> +-----------------\n> +There are other technologies that address different problems but have some\n> +similarities with this proposal.\n> +\n> +Replacements (refs/replace) are superficially similar to obsolescences in that\n> +they describe that one commit should be replaced by another. However, they\n> +differ in both how they are created and how they are intended to be used.\n> +Obsolescences are created automatically by the commands a user runs, and they\n> +describe the user’s intent to perform a future rebase. Obsolete commits still\n> +appear in branches, logs, etc like normal commits (possibly with an extra\n> +decoration that marks them as obsolete). Replacements are typically created\n> +explicitly by the user, they are meant to be kept around for a long time, and\n> +they describe a replacement to be applied at read-time rather than as the input\n> +to a future operation. When a replaced commit is queried, it is typically hidden\n> +and swapped out with its replacement as though the replacement has already\n> +occurred.\n> +\n> +Git-imerge is a project to help make complicated merges easier, particularly\n> +when merging or rebasing long chains of patches. It is not an alternative to\n> +the change graph, but its algorithm of applying smaller incremental merges\n> +could be used as part of the evolve algorithm in the future.\n> +\n> +Overview\n> +========\n> +We introduce the notion of “meta-commits” which describe how one commit was\n> +created from other commits. A branch of meta-commits is known as a change.\n> +Changes are created and updated automatically whenever a user runs a command\n> +that creates a commit. They are used for locating obsolete commits, providing a\n> +list of a user’s unsubmitted work in progress, and providing a stable name for\n> +each unsubmitted change.\n> +\n> +Users can exchange edit histories by pushing and fetching changes.\n> +\n> +New commands will be introduced for manipulating changes and resolving\n> +divergence between them. Existing commands that create commits will be updated\n> +to modify the meta-commit graph and create changes where necessary.\n> +\n> +Example usage\n> +-------------\n> +# First create three dependent changes\n> +$ echo foo>bar.txt && git add .\n> +$ git commit -m \"This is a test\"\n> +created change metas/this_is_a_test\n> +$ echo foo2>bar2.txt && git add .\n> +$ git commit -m \"This is also a test\"\n> +created change metas/this_is_also_a_test\n> +$ echo foo3>bar3.txt && git add .\n> +$ git commit -m \"More testing\"\n> +created change metas/more_testing\n> +\n> +# List all our changes in progress\n> +$ git change list\n> +metas/this_is_a_test\n> +metas/this_is_also_a_test\n> +* metas/more_testing\n> +metas/some_change_already_merged_upstream\n> +\n> +# Now modify the earliest change, using its stable name\n> +$ git reset --hard metas/this_is_a_test\n> +$ echo morefoo>>bar.txt && git add . && git commit --amend --no-edit\n> +\n> +# Use git-evolve to fix up any dependent changes\n> +$ git evolve\n> +rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n> +rebasing metas/more_testing onto metas/this_is_also_a_test\n> +Done\n> +\n> +# Use git-obslog to view the history of the this_is_a_test change\n> +$ git log --obslog\n> +93f110 metas/this_is_a_test@{0} commit (amend): This is a test\n> +930219 metas/this_is_a_test@{1} commit: This is a test\n> +\n> +# Now create an unrelated change\n> +$ git reset --hard origin/master\n> +$ echo newchange>unrelated.txt && git add .\n> +$ git commit -m \"Unrelated change\"\n> +created change metas/unrelated_change\n> +\n> +# Fetch the latest code from origin/master and use git-evolve\n> +# to rebase all dependent changes.\n> +$ git fetch origin master\n> +$ git evolve origin/master\n> +deleting metas/some_change_already_merged_upstream\n> +rebasing metas/this_is_a_test onto origin/master\n> +rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n> +rebasing metas/more_testing onto metas/this_is_also_a_test\n> +rebasing metas/unrelated_change onto origin/master\n> +Conflict detected! Resolve it and then use git evolve --continue to resume.\n> +\n> +# Sort out the conflict\n> +$ git mergetool\n> +$ git evolve origin/master\n> +Done\n> +\n> +# Share the full history of edits for the this_is_a_test change\n> +# with a review server\n> +$ git push origin metas/this_is_a_test:refs/for/master\n> +# Share the lastest commit for “Unrelated change”, without history\n> +$ git push origin HEAD:refs/for/master\n> +\n> +Detailed design\n> +===============\n> +Obsolescence information is stored as a graph of meta-commits. A meta-commit is\n> +a specially-formatted merge commit that describes how one commit was created\n> +from others.\n> +\n> +Meta-commits look like this:\n> +\n> +$ git cat-file -p \n> +tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\n> +parent aa7ce55545bf2c14bef48db91af1a74e2347539a\n> +parent d64309ee51d0af12723b6cb027fc9f195b15a5e9\n> +parent 7e1bbcd3a0fa854a7a9eac9bf1eea6465de98136\n> +author Stefan Xenos 1540841596 -0700\n> +committer Stefan Xenos 1540841596 -0700\n> +parent-type c r o\n> +\n> +This says “commit aa7ce555 makes commit d64309ee obsolete. It was created by\n> +cherry-picking commit 7e1bbcd3”.\n> +\n> +The tree for meta-commits is always the empty tree, but future versions of git\n> +may attach other trees here. For forward-compatibility fsck should ignore such\n> +trees if found on future repository versions. This will allow future versions of\n> +git to add metadata to the meta-commit tree without breaking forwards\n> +compatibility.\n> +\n> +The commit comment for a meta-commit is an auto-generated user-readable string\n> +describing the command that produced the meta commit. These strings are shown\n> +to the user when they view the obslog.\n> +\n> +Parent-type\n> +-----------\n> +The “parent-type” field in the commit header identifies a commit as a\n> +meta-commit and indicates the meaning for each of its parents. It is never\n> +present for normal commits. It contains a space-deliminated list of enum values\n> +whose order matches the order of the parents. Possible parent types are:\n> +\n> +- c: (content) the content parent identifies the commit that this meta-commit is\n> + describing.\n> +- r: (replaced) indicates that this parent is made obsolete by the content\n> + parent.\n> +- o: (origin) indicates that the content parent was generated by cherry-picking\n> + this parent.\n> +- a: (abandoned) used in place of a content parent for abandoned changes. Points\n> + to the final content commit for the change at the time it was abandoned.\n> +\n> +There must be exactly one content or abandoned parent for each meta-commit and\n> +it is always the first parent. The content commit will always be a normal commit\n> +and not a meta-commit. However, future versions of git may create meta-commits\n> +for other meta-commits and the fsck tool must be aware of this for forwards\n> +compatibility.\n> +\n> +A meta-commit can have zero or more replaced parents. An amend operation creates\n> +a single replaced parent. A merge used to resolve divergence (see divergence,\n> +below) will create multiple replaced parents. A meta-commit may have no\n> +replaced parents if it describes a cherry-pick or squash merge that copies one\n> +or more commits but does not replace them.\n> +\n> +A meta-commit can have zero or more origin parents. A cherry-pick creates a\n> +single origin parent. Certain types of squash merge will create multiple origin\n> +parents. Origin parents don't directly cause their origin to become obsolete,\n> +but are used when computing blame or locating a merge base. The section\n> +on obsolescence over cherry-picks describes how the evolve command uses\n> +origin parents.\n> +\n> +A replaced parent or origin parent may be either a normal commit (indicating\n> +the oldest-known version of a change) or another meta-commit (for a change that\n> +has already been modified one or more times).\n> +\n> +The parent-type field needs to go after the committer field since git's rules\n> +for forwards-compatibility require that new fields to be at the end of the\n> +header. Putting a new field in the middle of the header would break fsck.\n> +\n> +The presence of an abandoned parent indicates that the change should be pruned\n> +by the evolve command, and removed from the repository's history. Any follow-up\n> +changes should rebased onto the parent of the pruned commit. The abandoned\n> +parent points to the version of the change that should be restored if the user\n> +attempts to restore the change.\n> +\n> +Changes\n> +-------\n> +A branch of meta-commits describes how a commit was produced and what previous\n> +commits it is based on. It is also an identifier for a thing the user is\n> +currently working on. We refer to such a meta-branch as a change.\n> +\n> +Local changes are stored in the new refs/metas namespace. Remote changes are\n> +stored in the refs/remote//metas namespace.\n> +\n> +The list of changes in refs/metas is more than just a mechanism for the evolve\n> +command to locate obsolete commits. It is also a convenient list of all of a\n> +user’s work in progress and their current state - a list of things they’re\n> +likely to want to come back to.\n> +\n> +Strictly speaking, it is the presence of the branch in the refs/metas namespace\n> +that marks a branch as being a change, not the fact that it points to a\n> +metacommit. Metacommits are only created when a commit is amended or rebased, so\n> +in the case where a change points to a commit that has never been modified, the\n> +change points to that initial commit rather than a metacommit.\n> +\n> +Changes are also stored in the refs/hiddenmetas namespace. Hiddenmetas holds\n> +metadata for historical changes that are not currently in progress by the user.\n> +Commands like filter-branch and other bulk import commands create metadata in\n> +this namespace.\n> +\n> +Note that the changes in hiddenmetas get special treatment in several ways:\n> +\n> +- They are not cleaned up automatically once merged, since it is expected that\n> + they refer to historical changes.\n> +- User commands that modify changes don't append to these changes as they would\n> + to a change in refs/metas.\n> +- They are not displayed when the user lists their local changes.\n> +\n> +Obsolescence\n> +------------\n> +A commit is considered obsolete if it is reachable from the “replaces” edges\n> +anywhere in the history of a change and it isn’t the head of that change.\n> +Commits may be the content for 0 or more meta-commits. If the same commit\n> +appears in multiple changes, it is not obsolete if it is the head of any of\n> +those changes.\n> +\n> +Note that there is an exception to this rule. The metas namespace takes\n> +precedence over the hiddenmetas namespace for the purpose of obsolescence. That\n> +is, if a change appears in a replaces edge of a change in the metas namespace,\n> +it is obsolete even if it also appears as the head of a change in the\n> +hiddenmetas namespace.\n> +\n> +This special case prevents the hiddenmetas namespace from creating divergence\n> +with the user's work in progress, and allows the user to resolve historical\n> +divergence by creating new changes in the metas namespace.\n> +\n> +Divergence\n> +----------\n> +From the user’s perspective, two changes are divergent if they both ask for\n> +different replacements to the same commit. More precisely, a target commit is\n> +considered divergent if there is more than one commit at the head of a change in\n> +refs/metas that leads to the target commit via an unbroken chain of “replaces”\n> +parents.\n> +\n> +Much like a merge conflict, divergence is a situation that requires user\n> +intervention to resolve. The evolve command will stop when it encounters\n> +divergence and prompt the user to resolve the problem. Users can solve the\n> +problem in several ways:\n> +\n> +- Discard one of the changes (by deleting its change branch).\n> +- Merge the two changes (producing a single change branch).\n> +- Copy one of the changes (keep both commits, but one of them gets a new\n> + metacommit appended to its history that is connected to its predecessor via an\n> + origin edge rather than a replaces edge. That new change no longer obsoletes\n> + the original.)\n> +\n> +Obsolescence across cherry-picks\n> +--------------------------------\n> +By default the evolve command will treat cherry-picks and squash merges as being\n> +completely separate from the original. Further amendments to the original commit\n> +will have no effect on the cherry-picked copy. However, this behavior may not be\n> +desirable in all circumstances.\n> +\n> +The evolve command may at some point support an option to look for cases where\n> +the source of a cherry-pick or squash merge has itself been amended, and\n> +automatically apply that same change to the cherry-picked copy. In such cases,\n> +it would traverse origin edges rather than ignoring them, and would treat a\n> +commit with origin edges as being obsolete if any of its origins were obsolete.\n> +\n> +Garbage collection\n> +------------------\n> +For GC purposes, meta-commits are normal commits. Just as a commit causes its\n> +parents and tree to be retained, a meta-commit also causes its parents to be\n> +retained.\n> +\n> +Change creation\n> +---------------\n> +Changes are created automatically whenever the user runs a command like “commit”\n> +that has the semantics of creating a new change. They also move forward\n> +automatically even if they’re not checked out. For example, whenever the user\n> +runs a command like “commit --amend” that modifies a commit, all branches in\n> +refs/metas that pointed to the old commit move forward to point to its\n> +replacement instead. This also happens when the user is working from a detached\n> +head.\n> +\n> +This does not mean that every commit has a corresponding change. By default,\n> +changes only exist for recent locally-created commits. Users may explicitly pull\n> +changes from other users or keep their changes around for a long time, but\n> +either behavior requires a user to opt-in. Code review systems like gerrit may\n> +also choose to keep changes around forever.\n> +\n> +Note that the changes in refs/metas serve a dual function as both a way to\n> +identify obsolete changes and as a way for the user to keep track of their work\n> +in progress. If we were only concerned with identifying obsolete changes, it\n> +would be sufficient to create the change branch lazily the first time a commit\n> +is obsoleted. Addressing the second use - of refs/metas as a mechanism for\n> +keeping track of work in progress - is the reason for eagerly creating the\n> +change on first commit.\n> +\n> +Change naming\n> +-------------\n> +When a change is first created, the only requirement for its name is that it\n> +must be unique. Good names would also serve as useful mnemonics and be easy to\n> +type. For example, a short word from the commit message containing no numbers or\n> +special characters and that shows up with low frequency in other commit messages\n> +would make a good choice.\n> +\n> +Different users may prefer different heuristics for their change names. For this\n> +reason a new hook will be introduced to compute change names. Git will invoke\n> +the hook for all newly-created changes and will append a numeric suffix if the\n> +name isn’t unique. The default heuristics are not specified by this proposal and\n> +may change during implementation.\n> +\n> +Change deletion\n> +---------------\n> +Changes are normally only interesting to a user while a commit is still in\n> +development and under review. Once the commit has submitted wherever it is\n> +going, its change can be discarded.\n> +\n> +The normal way of deleting changes makes this easy to do - changes are deleted\n> +by the evolve command when it detects that the change is present in an upstream\n> +branch. It does this in two ways: if the latest commit in a change either shows\n> +up in the branch history or the change becomes empty after a rebase, it is\n> +considered merged and the change is discarded. In this context, an “upstream\n> +branch” is any branch passed in as the upstream argument of the evolve command.\n> +\n> +In case this sometimes deletes a useful change, such automatic deletions are\n> +recorded in the reflog allowing them to be easily recovered.\n> +\n> +Sharing changes\n> +---------------\n> +Change histories are shared by pushing or fetching meta-commits and change\n> +branches. This provides users with a lot of control of what to share and\n> +repository implementations with control over what to retain.\n> +\n> +Users that only want to share the content of a commit can do so by pushing the\n> +commit itself as they currently would. Users that want to share an edit history\n> +for the commit can push its change, which would point to a meta-commit rather\n> +than the commit itself if there is any history to share. Note that multiple\n> +changes can refer to the same commits, so it’s possible to construct and push a\n> +different history for the same commit in order to remove sensitive or irrelevant\n> +intermediate states.\n> +\n> +Imagine the user is working on a change “mychange” that is currently the latest\n> +commit on master. They have two ways to share it:\n> +\n> +# User shares just a commit without its history\n> +> git push origin master\n> +\n> +# User shares the full history of the commit to a review system\n> +> git push origin metas/mychange:refs/for/master\n> +\n> +# User fetches a collaborator’s modifications to their change\n> +> git fetch remotename metas/mychange\n> +# Which updates the ref remote/remotename/metas/mychange\n> +\n> +This will cause more intermediate states to be shared with the server than would\n> +have been shared previously. A review system like gerrit would need to keep\n> +track of which states had been explicitly pushed versus other intermediate\n> +states in order to de-emphasize (or hide) the extra intermediate states from the\n> +user interface.\n> +\n> +Merge-base\n> +----------\n> +Merge-base will be changed to search the meta-commit graph for common ancestors\n> +as well as the commit graph, and will generally prefer results from the\n> +meta-commit graph over the commit graph. Merge-base will consider meta-commits\n> +from all changes, and will traverse both origin and obsolete edges.\n> +\n> +The reason for this is that - when merging two versions of the same commit\n> +together - an earlier version of that same commit will usually be much more\n> +similar than their common parent. This should make the workflow of collaborating\n> +on unsubmitted patches as convenient as the workflow for collaborating in a\n> +topic branch by eliminating repeated merges.\n> +\n> +Configuration\n> +-------------\n> +The core.enableChanges configuration variable enables the creation and update\n> +of change branches. This is enabled by default.\n> +\n> +User interface\n> +--------------\n> +All git porcelain commands that create commits are classified as having one of\n> +four behaviors: modify, create, copy, or import. These behaviors are discussed\n> +in more detail below.\n> +\n> +Modify commands\n> +---------------\n> +Modification commands (commit --amend, rebase) will mark the old commit as\n> +obsolete by creating a new meta-commit that references the old one as a\n> +replaced parent. In the event that multiple changes point to the same commit,\n> +this is done independently for every such change.\n> +\n> +More specifically, modifications work like this:\n> +\n> +1. Locate all existing changes for which the old commit is the content for the\n> + head of the change branch. If no such branch exists, create one that points\n> + to the old commit. Changes that include this commit in their history but not\n> + at their head are explicitly not included.\n> +2. For every such change, create a new meta-commit that references the new\n> + commit as its content and references the old head of the change as a\n> + replaced parent.\n> +3. Move the change branch forward to point to the new meta-commit.\n> +\n> +Copy commands\n> +-------------\n> +Copy commands (cherry-pick, merge --squash) create a new meta-commit that\n> +references the old commits as origin parents. Besides the fact that the new\n> +parents are tagged differently, copy commands work the same way as modify\n> +commands.\n> +\n> +Create commands\n> +---------------\n> +Creation commands (commit, merge) create a new commit and a new change that\n> +points to that commit. The do not create any meta-commits.\n> +\n> +Import commands\n> +---------------\n> +Import commands (fetch, pull) do not create any new meta-commits or changes\n> +unless that is specifically what they are importing. For example, the fetch\n> +command would update remote/origin/metas/change35 and fetch all referenced\n> +meta-commits if asked to do so directly, but it wouldn’t create any changes or\n> +meta-commits for commits discovered on the master branch when running “git fetch\n> +origin master”.\n> +\n> +Other commands\n> +--------------\n> +Some commands don’t fit cleanly into one of the above categories.\n> +\n> +Semantically, filter-branch should be treated as a modify command, but doing so\n> +is likely to create a lot of irrelevant clutter in the changes namespace and the\n> +large number of extra change refs may introduce performance problems. We\n> +recommend treating filter-branch as an import command initially, but making it\n> +behave more like a modify command in future follow-up work. One possible\n> +solution may be to treat commits that are part of existing changes as being\n> +modified but to avoid creating changes for other rewritten changes. Another\n> +solution may be to record the modifications as changes in the hiddenmetas\n> +namespace.\n> +\n> +Once the evolve command can handle obsolescence across cherry-picks, such\n> +cherry-picks will result in a hybrid move-and-copy operation. It will create\n> +cherry-picks that replace other cherry-picks, which will have both origin edges\n> +(pointing to the new source commit being picked) and replacement edges (pointing\n> +to the previous cherry-pick being replaced).\n> +\n> +Evolve\n> +------\n> +The evolve command performs the correct sequence of rebases such that no change\n> +has an obsolete parent. The syntax looks like this:\n> +\n> +git evolve [upstream…]\n> +\n> +It takes an optional list of upstream branches. All changes whose parent shows\n> +up in the history of one of the upstream branches will be rebased onto the\n> +upstream branch before resolving obsolete parents.\n> +\n> +Any change whose latest state is found in an upstream branch (or that ends up\n> +empty after rebase) will be deleted. This is the normal mechanism for deleting\n> +changes. Changes are created automatically on the first commit, and are deleted\n> +automatically when evolve determines that they’ve been merged upstream.\n> +\n> +Orphan commits are commits with obsolete parents. The evolve command then\n> +repeatedly rebases orphan commits with non-orphan parents until there are either\n> +no orphan commits left, or a merge conflict is discovered. It will also\n> +terminate if it detects a divergent parent or a cycle that can't be resolved\n> +using any of the enabled transformations.\n> +\n> +When evolve discovers divergence, it will first check if it can resolve the\n> +divergence automatically using one of its enabled transformations. Supported\n> +transformations are:\n> +\n> +- Check if the user has already merged the divergent changes in a follow-up\n> + change. That is, look for an existing merge in a follow-up change where all\n> + the parents are divergent versions of the same change. Squash that merge with\n> + its parents and use the result as the resolution for the divergence.\n> +\n> +- Attempt to auto-merge all the divergent changes (disabled by default).\n> +\n> +Each of the transformations can be enabled or disabled by command line options.\n> +\n> +Cycles can occur when two changes reference one another as parents. This can\n> +happen when both changes use an obsolete version of the other change as their\n> +parent. Although there are never cycles in the commit graph, users can create\n> +cycles in the change graph by rebasing changes onto obsolete commits. The evolve\n> +command has a transformation that will detect and break cycles by arbitrarily\n> +picking one of the changes to go first. If this generates a merge conflict,\n> +it tries each of the other changes in sequence to see if any ordering merges\n> +cleanly. If no possible ordering merges cleanly, it picks one and terminates\n> +to let the user resolve the merge conflict.\n> +\n> +If the working tree is dirty, evolve will attempt to stash the user's changes\n> +before applying the evolve and then reapply those changes afterward, in much\n> +the same way as rebase --autostash does.\n> +\n> +Checkout\n> +--------\n> +Running checkout on a change by name has the same effect as checking out a\n> +detached head pointing to the latest commit on that change-branch. There is no\n> +need to ever have HEAD point to a change since changes always move forward when\n> +necessary, no matter what branch the user has checked out\n> +\n> +Meta-commits themselves cannot be checked out by their hash.\n> +\n> +Reset\n> +-----\n> +Resetting a branch to a change by name is the same as resetting to the content\n> +(or abandoned) commit at that change’s head.\n> +\n> +Commit\n> +------\n> +Commit --amend gets modify semantics and will move existing changes forward. The\n> +normal form of commit gets create semantics and will create a new change.\n> +\n> +$ touch foo && git add . && git commit -m \"foo\" && git tag A\n> +$ touch bar && git add . && git commit -m \"bar\" && git tag B\n> +$ touch baz && git add . && git commit -m \"baz\" && git tag C\n> +\n> +This produces the following commits:\n> +A(tree=[foo])\n> +B(tree=[foo, bar], parent=A)\n> +C(tree=[foo, bar, baz], parent=B)\n> +\n> +...along with three changes:\n> +metas/foo = A\n> +metas/bar = B\n> +metas/baz = C\n> +\n> +Running commit --amend does the following:\n> +$ git checkout B\n> +$ touch zoom && git add . && git commit --amend -m \"baz and zoom\"\n> +$ git tag D\n> +\n> +Commits:\n> +A(tree=[foo])\n> +B(tree=[foo, bar], parent=A)\n> +C(tree=[foo, bar, baz], parent=B)\n> +D(tree=[foo, bar, zoom], parent=A)\n> +Dmeta(content=D, obsolete=B)\n> +\n> +Changes:\n> +metas/foo = A\n> +metas/bar = Dmeta\n> +metas/baz = C\n> +\n> +Merge\n> +-----\n> +Merge gets create, modify, or copy semantics based on what is being merged and\n> +the options being used.\n> +\n> +The --squash version of merge gets copy semantics (it produces a new change that\n> +is marked as a copy of all the original changes that were squashed into it).\n> +\n> +The “modify” version of merge replaces both of the original commits with the\n> +resulting merge commit. This is one of the standard mechanisms for resolving\n> +divergence. The parents of the merge commit are the parents of the two commits\n> +being merged. The resulting commit will not be a merge commit if both of the\n> +original commits had the same parent or if one was the parent of the other.\n> +\n> +The “create” version of merge creates a new change pointing to a merge commit\n> +that has both original commits as parents. The result is what merge produces now\n> +- a new merge commit. However, this version of merge doesn’t directly resolve\n> +divergence.\n> +\n> +To select between these two behaviors, merge gets new “--amend” and “--noamend”\n> +options which select between the “create” and “modify” behaviors respectively,\n> +with noamend being the default.\n> +\n> +For example, imagine we created two divergent changes like this:\n> +\n> +$ touch foo && git add . && git commit -m \"foo\" && git tag A\n> +$ touch bar && git add . && git commit -m \"bar\" && git tag B\n> +$ touch baz && git add . && git commit --amend -m \"bar and baz\"\n> +$ git tag C\n> +$ git checkout B\n> +$ touch bam && git add . && git commit --amend -m \"bar and bam\"\n> +$ git tag D\n> +\n> +At this point the commit graph looks like this:\n> +\n> +A(tree=[foo])\n> +B(tree=[bar], parent=A)\n> +C(tree=[bar, baz], parent=A)\n> +D(tree=[bar, bam], parent=A)\n> +Cmeta(content=C, obsoletes=B)\n> +Dmeta(content=D, obsoletes=B)\n> +\n> +There would be three active changes with heads pointing as follows:\n> +\n> +metas/changeA=A\n> +metas/changeB=Cmeta\n> +metas/changeB2=Dmeta\n> +\n> +ChangeB and changeB2 are divergent at this point. Lets consider what happens if\n> +perform each type of merge between changeB and changeB2.\n> +\n> +Merge example: Amend merge\n> +One way to resolve divergent changes is to use an amend merge. Recall that HEAD\n> +is currently pointing to D at this point.\n> +\n> +$ git merge --amend metas/changeB\n> +\n> +Here we’ve asked for an amend merge since we’re trying to resolve divergence\n> +between two versions of the same change. There are no conflicts so we end up\n> +with this:\n> +\n> +E(tree=[bar, baz, bam], parent=A)\n> +Emeta(content=E, obsoletes=[Cmeta, Dmeta])\n> +\n> +With the following branches:\n> +\n> +metas/changeA=A\n> +metas/changeB=Emeta\n> +metas/changeB2=Emeta\n> +\n> +Notice that the result of the “amend merge” is a replacement for C and D rather\n> +than a new commit with C and D as parents (as a normal merge would have\n> +produced). The parents of the amend merge are the parents of C and D which - in\n> +this case - is just A, so the result is not a merge commit. Also notice that\n> +changeB and changeB2 are now aliases for the same change.\n> +\n> +Merge example: Noamend merge\n> +Consider what would have happened if we’d used a noamend merge instead. Recall\n> +that HEAD was at D and our branches looked like this:\n> +\n> +metas/changeA=A\n> +metas/changeB=Cmeta\n> +metas/changeB2=Dmeta\n> +\n> +$ git merge --noamend metas/changeB\n> +\n> +That would produce the sort of merge we’d normally expect today:\n> +\n> +F(tree=[bar, baz, bam], parent=[C, D])\n> +\n> +And our changes would look like this:\n> +metas/changeA=A\n> +metas/changeB=Cmeta\n> +metas/changeB2=Dmeta\n> +metas/changeF=F\n> +\n> +In this case, changeB and changeB2 are still divergent and we’ve created a new\n> +change for our merge commit. However, this is just a temporary state. The next\n> +time we run the “evolve” command, it will discover the divergence but also\n> +discover the merge commit F that resolves it. Evolve will suggest converting F\n> +into an amend merge in order to resolve the divergence and will display the\n> +command for doing so.\n> +\n> +Rebase\n> +------\n> +In general the rebase command is treated as a modify command. When a change is\n> +rebased, the new commit replaces the original.\n> +\n> +Rebase --abort is special. Its intent is to restore git to the state it had\n> +prior to running rebase. It should move back any changes to point to the refs\n> +they had prior to running rebase and delete any new changes that were created as\n> +part of the rebase. To achieve this, rebase will save the state of all changes\n> +in refs/metas prior to running rebase and will restore the entire namespace\n> +after rebase completes (deleting any newly-created changes). Newly-created\n> +metacommits are left in place, but will have no effect until garbage collected\n> +since metacommits are only used if they are reachable from refs/metas.\n> +\n> +Change\n> +------\n> +The “change” command can be used to list, rename, reset or delete change. It has\n> +a number of subcommands.\n> +\n> +The \"list\" subcommand lists local changes. If given the -r argument, it lists\n> +remote changes.\n> +\n> +The \"rename\" subcommand renames a change, given its old and new name. If the old\n> +name is omitted and there is exactly one change pointing to the current HEAD,\n> +that change is renamed. If there are no changes pointing to the current HEAD,\n> +one is created with the given name.\n> +\n> +The \"forget\" subcommand deletes a change by deleting its ref from the metas/\n> +namespace. This is the normal way to delete extra aliases for a change if the\n> +change has more than one name. By default, this will refuse to delete the last\n> +alias for a change if there are any other changes that reference this change as\n> +a parent.\n> +\n> +The \"update\" subcommand adds a new state to a change. It uses the default\n> +algorithm for assigning change names. If the content commit is omitted, HEAD is\n> +used. If given the optional --force argument, it will overwrite any existing\n> +change of the same name. This latter form of \"update\" can be used to effectively\n> +reset changes.\n> +\n> +The \"update\" command can accept any number of --origin and --replace arguments.\n> +If any are present, the resulting change branch will point to a metacommit\n> +containing the given origin and replacement edges.\n> +\n> +The \"abandon\" command deletes a change using obsolescence markers. It marks the\n> +change as being obsolete and having been replaced by its parent. If given no\n> +arguments, it applies to the current commit. Running evolve will cause any\n> +abandoned changes to be removed from the branch. Any child changes will be\n> +reparented on top of the parent of the abandoned change. If the current change\n> +is abandoned, HEAD will move to point to its parent.\n> +\n> +The \"restore\" command restores a previously-abandoned change.\n> +\n> +The \"prune\" command deletes all obsolete changes and all changes that are\n> +present in the given branch. Note that such changes can be recovered from the\n> +reflog.\n> +\n> +Combined with the GC protection that is offered, this is intended to facilitate\n> +a workflow that relies on changes instead of branches. Users could choose to\n> +work with no local branches and use changes instead - both for mailing list and\n> +gerrit workflows.\n> +\n> +Log\n> +---\n> +When a commit is shown in git log that is part of a change, it is decorated with\n> +extra change information. If it is the head of a change, the name of the change\n> +is shown next to the list of branches. If it is obsolete, it is decorated with\n> +the text “obsolete, commits behind ”.\n> +\n> +Log gets a new --obslog argument indicating that the obsolescence graph should\n> +be followed instead of the commit graph. This also changes the default\n> +formatting options to make them more appropriate for viewing different\n> +iterations of the same commit.\n> +\n> +Pull\n> +----\n> +\n> +Pull gets an --evolve argument that will automatically attempt to run \"evolve\"\n> +on any affected branches after pulling.\n> +\n> +We also introduce an \"evolve\" enum value for the branch..rebase config\n> +value. When set, the evolve behavior will happen automatically for that branch\n> +after every pull even if the --evolve argument is not used.\n> +\n> +Next\n> +----\n> +\n> +The \"next\" command will reset HEAD to a non-obsolete commit that refers to this\n> +change as its parent. If there is more than one such change, the user will be\n> +prompted. If given the --evolve argument, the next commit will be evolved if\n> +necessary first.\n> +\n> +The \"next\" command can be thought of as the opposite of\n> +\"git reset --hard HEAD^\" in that it navigates to a child commit rather than a\n> +parent.\n> +\n> +Prev\n> +----\n> +\n> +The \"prev\" command will reset HEAD to the latest version of the parent change.\n> +If the parent change isn't obsolete, this is equivalent to\n> +\"git reset --hard HEAD^\". If the parent commit is obsolete, it resets to the\n> +latest replacement for the parent commit.\n> +\n> +Other options considered\n> +========================\n> +We considered several other options for storing the obsolescence graph. This\n> +section describes the other options and why they were rejected.\n> +\n> +Commit header\n> +-------------\n> +Add an “obsoletes” field to the commit header that points backwards from a\n> +commit to the previous commits it obsoletes.\n> +\n> +Pros:\n> +- Very simple\n> +- Easy to traverse from a commit to the previous commits it obsoletes.\n> +Cons:\n> +- Adds a cost to the storage format, even for commits where the change history\n> + is uninteresting.\n> +- Unconditionally prevents the change history from being garbage collected.\n> +- Always causes the change history to be shared when pushing or pulling changes.\n> +\n> +Git notes\n> +---------\n> +Instead of storing obsolescence information in metacommits, the metacommit\n> +content could go in a new notes namespace - say refs/notes/metacommit. Each note\n> +would contain the list of obsolete and origin parents. An automerger could\n> +be supplied to make it easy to merge the metacommit notes from different remotes.\n> +\n> +Pros:\n> +- Easy to locate all commits obsoleted by a given commit (since there would only\n> + be one metacommit for any given commit).\n> +Cons:\n> +- Wrong GC behavior (obsolete commits wouldn’t automatically be retained by GC)\n> + unless we introduced a special case for these kinds of notes.\n> +- No way to selectively share or pull the metacommits for one specific change.\n> + It would be all-or-nothing, which would be expensive. This could be addressed\n> + by changes to the protocol, but this would be invasive.\n> +- Requires custom auto-merging behavior on fetch.\n> +\n> +Tags\n> +----\n> +Put the content of the metacommit in a message attached to tag on the\n> +replacement commit. This is very similar to the git notes approach and has the\n> +same pros and cons.\n> +\n> +Simple forward references\n> +-------------------------\n> +Record an edge from an obsolete commit to its replacement in this form:\n> +\n> +refs/obsoletes/\n> +\n> +pointing to commit as an indication that B is the replacement for the\n> +obsolete commit A.\n> +\n> +Pros:\n> +- Protects from being garbage collected.\n> +- Fast lookup for the evolve operation, without additional search structures\n> + (“what is the replacement for ?” is very fast).\n> +\n> +Cons:\n> +- Can’t represent divergence (which is a P0 requirement).\n> +- Creates lots of refs (which can be inefficient)\n> +- Doesn’t provide a way to fetch only refs for a specific change.\n> +- The obslog command requires a search of all refs.\n> +\n> +Complex forward references\n> +--------------------------\n> +Record an edge from an obsolete commit to its replacement in this form:\n> +\n> +refs/obsoletes//obs_\n> +\n> +Pointing to commit as an indication that B is the replacement for obsolete\n> +commit A.\n> +\n> +Pros:\n> +- Permits sharing and fetching refs for only a specific change.\n> +- Supports divergence\n> +- Protects from being garbage collected.\n> +\n> +Cons:\n> +- Creates lots of refs, which is inefficient.\n> +- Doesn’t provide a good lookup structure for lookups in either direction.\n> +\n> +Backward references\n> +-------------------\n> +Record an edge from a replacement commit to the obsolete one in this form:\n> +\n> +refs/obsolescences/\n> +\n> +Cons:\n> +- Doesn’t provide a way to resolve divergence (which is a P0 requirement).\n> +- Doesn’t protect from being garbage collected (which could be fixed by\n> + combining this with a refs/metas namespace, as in the metacommit variant).\n> +\n> +Obsolescences file\n> +------------------\n> +Create a custom file (or files) in .git recording obsolescences.\n> +\n> +Pros:\n> +- Can store exactly the information we want with exactly the performance we want\n> + for all operations. For example, there could be a disk-based hashtable\n> + permitting constant time lookups in either direction.\n> +\n> +Cons:\n> +- Handling GC, pushing, and pulling would all require custom solutions. GC\n> + issues could be addressed with a repository format extension.\n> +\n> +Squash points\n> +-------------\n> +We treat changes like topic branches, and use special squash points to mark\n> +places in the commit graph that separate changes.\n> +\n> +We create and update change branches in refs/metas at the same time we\n> +would have in the metacommit proposal. However, rather than pointing to a\n> +metacommit branch they point to normal commits and are treated as “squash\n> +points” - markers for sequences of commits intended to be squashed together on\n> +submission.\n> +\n> +Amends and rebases work differently than they do now. Rather than actually\n> +containing the desired state of a commit, they contain a delta from the previous\n> +version along with a squash point indicating that the preceding changes are\n> +intended to be squashed on submission. Specifically, amends would become new\n> +changes and rebases would become merge commits with the old commit and new\n> +parent as parents.\n> +\n> +When the changes are finally submitted, the squashes are executed, producing the\n> +final version of the commit.\n> +\n> +In addition to the squash points, git would maintain a set of “nosquash” tags\n> +for commits that were used as ancestors of a change that are not meant to be\n> +included in the squash.\n> +\n> +For example, if we have this commit graph:\n> +\n> +A(...)\n> +B(parent=A)\n> +C(parent=B)\n> +\n> +...and we amend B to produce D, we’d get:\n> +\n> +A(...)\n> +B(parent=A)\n> +C(parent=B)\n> +D(parent=B)\n> +\n> +...along with a new change branch indicating D should be squashed with its\n> +parents when submitted:\n> +\n> +metas/changeB = D\n> +metas/changeC = C\n> +\n> +We’d also create a nosquash tag for A indicating that A shouldn’t be included\n> +when changeB is squashed.\n> +\n> +If a user amends the change again, they’d get:\n> +\n> +A(...)\n> +B(parent=A)\n> +C(parent=B)\n> +D(parent=B)\n> +E(parent=D)\n> +\n> +metas/changeB = E\n> +metas/changeC = C\n> +\n> +Pros:\n> +- Good GC behavior.\n> +- Provides a natural way to share changes (they’re just normal branches).\n> +- Merge-base works automatically without special cases.\n> +- Rewriting the obslog would be easy using existing git commands.\n> +- No new data types needed.\n> +Cons:\n> +- No way to connect the squashed version of a change to the original, so no way\n> + to automatically clean up old changes. This also means users lose all benefits\n> + of the evolve command if they prematurely squash their commits. This may occur\n> + if a user thinks a change is ready for submission, squashes it, and then later\n> + discovers an additional change to make.\n> +- Histories would look very cluttered (users would see all previous edits to\n> + their commit in the commit log, and all previous rebases would show up as\n> + merges). Could be quite hard for users to tell what is going on. (Possible\n> + fix: also implement a new smart log feature that displays the log as though\n> + the squashes had occurred).\n> +- Need to change the current behavior of current commands (like amend and\n> + rebase) in ways that will be unexpected to many users.\n> --\n> gitgitgadget\n>\n"},{"id":"464329","messageId":"kl6ltu4gwu6b.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"58504","inReplyTo":"CAN9+7XcYFa+Y9jsJSEmQhf29TUZADoz8=SzcNbjCH8ewqYriYg@mail.gmail.com","subject":"Re: [PATCH v2 01/10] technical doc: add a design doc for the evolve command","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-10-06T20:53:16Z","receivedAt":"2022-10-06T20:53:30Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"\nHi Chris!\n\nChris Poucet writes:\n\n> One thing that is not clear to me is whether this is the desired\n> direction. I took at look at the git review notes but it was hard to\n> get a sense of where people are at.\n\nI'm really sorry, I meant to get back to this sooner with the takeaways\nfrom Review Club. Hopefully this will still be useful.\n\nYou can find the Review Club notes here:\n\n https://docs.google.com/document/d/14L8BAumGTpsXpjDY8VzZ4rRtpAjuGrFSRqn3stCuS_w/edit?pli=1\n\n> Would love input on the design.\n\nOthers have given a lot of input on the design, so instead, I'll focus\nmostly on how to make the doc better on the mailing list.\n\n>\n> On Wed, Oct 5, 2022 at 4:59 PM Stefan Xenos via GitGitGadget\n> wrote:\n>>\n>> From: Stefan Xenos \n>>\n>> This document describes what a change graph for\n>> git would look like, the behavior of the evolve command,\n>> and the changes planned for other commands.\n>>\n>> It was originally proposed in 2018, see\n>> https://public-inbox.org/git/20181115005546.212538-1-sxenos@google.com/\n\nThis doc is quite well-thought-out and surprisingly readable despite its\nlength. That said, it is a lot to review in one sitting, and a reviewer\nmight get easily fatigued. I suspect that reviewers will find it hard to\nkeep up with the discussion if they have to review the entire doc on\nevery iteration.\n\nAs Victoria suggested in Review Club, it might be helpful to split up\nthe design over multiple patches to make feedback more focused. I think\nthis will make it easier for you (and others) to get a sense of how we\nfeel about each part of the design. e.g. here's one way to split up the\ndoc:\n\n- Motivation, Background, High level idea of how a user would use this. \n\n (Roughly corresponding to the sections \"Objective\", \"Status\",\n \"Background\", \"Goals\", \"Similar technologies\", \"Semi-related work\")\n\n- Local change tracking, Changes to existing commands, Meta-commits\n\n (The parts about the data format and their implications for GC,\n negotiation, etc. Maybe include the `change` subcommand if it helps\n reviewers visualize the impact.)\n\n- How evolve works, e.g. convergence, divergence, merge base finding.\n CLI\n\n- Sharing changes\n\nBesides the design, here other sections that I would find useful:\n\n- Glossary. I thought that terms like \"change\", \"change branch\" and\n \"change graph\" were underdefined. This would also be a useful\n reference during the implementation phase.\n\n- Implementation Plan (you can find examples in\n Documentation/technical/bundle-uri.txt and\n Documentation/technical/sparse-index.txt). Making the concrete next\n steps visible has numerous benefits:\n - Reviewers of future patches know what problem is being tackled and\n value is being delivered.\n - The list gains confidence that the author can deliver the work being\n promised.\n - The shared direction makes it easier for others to contribute\n patches.\n\n- Open questions (e.g. \"Implementation questions\" in [1]). It would be\n useful to know what questions can be answered later instead of right\n now. Also, since you are not the original author, perhaps you also\n have questions about the design that you want answered by reviewers.\n I also wouldn't mind this being in the cover letter or \"---\" section.\n\n[1] https://lore.kernel.org/git/pull.1367.git.1664064588846.gitgitgadget@gmail.com\n\nAs mentioned earlier, I'll comment only very lightly on the design.\n\n>> +Similar technologies\n>> +--------------------\n\nI'd personally love to see \"git evolve\". If it helps to consider some\nother tools, I use the following tools that implement similar workflows:\n\n- git-branchless [2] features anonymous heads, obsolescence tracking, \n history manipulations and \"git evolve\". Having used this for a while,\n I'm of the opnion that having any of these features without the\n others is still very useful, and implementing them in phases \n will still deliver value without having to complete all of the work\n (granted, each of these features is incrementally dependent on the\n others).\n\n Case in point: I don't use the \"evolve\" equivalent of git-branchless\n (IIRC \"restack); being able to see obsolescence and manually\n manipulating history is good enough for me.\n\n- Jujutsu [3] also features anonymous heads, obsolescence tracking and\n advanced history manipulations. Instead of \"evolve\", descendents of an\n obsolete commit are automatically rebased on the obsoleting commit.\n\n[2] https://github.com/arxanas/git-branchless\n[3] https://github.com/martinvonz/jj\n\n>> +Changes\n>> +-------\n>> +A branch of meta-commits describes how a commit was produced and what previous\n>> +commits it is based on. It is also an identifier for a thing the user is\n>> +currently working on. We refer to such a meta-branch as a change.\n>> +\n>> +Local changes are stored in the new refs/metas namespace. Remote changes are\n>> +stored in the refs/remote//metas namespace.\n\nI find this terminology of \"changes\" and \"metas\" more confusing than\nnecessary. A glossary would help, but it might be even better to also\nuse an appropriate ref namespace. \"refs/changes/\" is an obvious\ncandidate, though I assume this wasn't mentioned because Gerrit uses\nthat namespace extensively.\n\nMaybe `refs/changelists`, `refs/change-requests`, `refs/proposals`? Idk.\n\n>> +Sharing changes\n>> +---------------\n>> +Change histories are shared by pushing or fetching meta-commits and change\n>> +branches. This provides users with a lot of control of what to share and\n>> +repository implementations with control over what to retain.\n>> +\n>> +Users that only want to share the content of a commit can do so by pushing the\n>> +commit itself as they currently would. Users that want to share an edit history\n>> +for the commit can push its change, which would point to a meta-commit rather\n>> +than the commit itself if there is any history to share. Note that multiple\n>> +changes can refer to the same commits, so it’s possible to construct and push a\n>> +different history for the same commit in order to remove sensitive or irrelevant\n>> +intermediate states.\n\nI would not like to see the ability to share all intermediate states\nwith the server because this increases the risk of unintentional\ndisclosure by a lot.\n\nHow exactly we could tweak this can be an open discussion for later.\nSome examples I can think of:\n - Asking the user to go through the obsolescence log and manually\n prune revisions (sounds too onerous for users IMO).\n - Push a truncated history consisting of only the latest version and\n commits that the server already knows (somewhat similar to Gerrit).\n\n>> +Evolve\n>> +------\n>> +The evolve command performs the correct sequence of rebases such that no change\n>> +has an obsolete parent. The syntax looks like this:\n>> +\n>> +git evolve [upstream…]\n>> +\n>> +It takes an optional list of upstream branches. All changes whose parent shows\n>> +up in the history of one of the upstream branches will be rebased onto the\n>> +upstream branch before resolving obsolete parents.\n>> +\n\nThis CLI is an example of something that can be reviewed largely\nindependently of the implementing data structures.\n\n>> +Merge\n>> +-----\n>> +\n>> +To select between these two behaviors, merge gets new “--amend” and “--noamend”\n>> +options which select between the “create” and “modify” behaviors respectively,\n>> +with noamend being the default.\n\nDitto.\n\n"},{"id":"464474","messageId":"bdc09e1c-3208-9d03-5ab4-28b818081aea@gmail.com","threadId":"58504","inReplyTo":"pull.1356.v2.git.1664981957.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 00/10] RFC: Git Evolve / Change","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-10T09:23:08Z","receivedAt":"2022-10-10T09:23:19Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Chris\n\nOn 05/10/2022 15:59, Christophe Poucet via GitGitGadget wrote:\n> I'm reviving the original git evolve work that was started by\n> sxenos@google.com\n> (https://public-inbox.org/git/20190215043105.163688-1-sxenos@google.com/)\n> \n> This work is intended to make it easier to deal with stacked changes.\n> \n> The following set of patches introduces the design doc on the evolve command\n> as well as the basics of the git change command.\n\nThanks for the new version. When you post a new version of a patch \nseries it is helpful to give a brief outline of what you have changed \nsince the last version. The overview should also explain the reasons for \nreordering or squashing patches. In this case the old patches 7 & 8 have \nbeen squashed together. I think it would have been better to leave them \nas separate patches and add the tests at the same time as the commands \n(that's why I provided separate fixups). The style is now closer to our \nnormal style but there are still some deviations.\n\n* Comments:\n\nSingle line comments should look like\n\t/* single line */\n\nMulti-line comments should look like\n\t/*\n\t * Multi-line\n\t * comment.\n\t */\nNot\n+\t/* If to_add is not a metacommit then the content is to_add itself,\n+\t * otherwise it will have been set by the call to\n+\t * get_metacommit_content.\n+\t */\n\nAPI comments may be formatted as\n\t/**\n\t * API docs\n\t */\n\nbut then you should not mix styles. For example in patch 5 you have\n+\t/**\n+\t * Memory pool for the objects allocated by the change table.\n+\t */\n+\tstruct mem_pool memory_pool;\n+\t/* Map object_id to commit_change_list_entry structs. */\n+\tstruct oidmap oid_to_metadata_index;\n\n* Functions\n\nWrap function arguments at 80 columns and indent the continuation lines \nto align with the opening parenthesis unless the function name is \nexceptionally long. There may be more than one function argument per \nline. For example\n\nstatic void resolve_metacommit(struct repository* repo,\n struct change_table* active_changes,\n const struct metacommit_data *to_resolve,\n struct metacommit_data *resolved_output,\n struct string_list *to_advance, int \nallow_append)\n\nNot\n\nstatic void resolve_metacommit(\n\tstruct repository* repo,\n\tstruct change_table* active_changes,\n\tconst struct metacommit_data *to_resolve,\n\tstruct metacommit_data *resolved_output,\n\tstruct string_list *to_advance,\n\tint allow_append)\n\nYou can use git-clang-format to get a fairly close approximation to the \nrequired style.\n\nFor variables and function arguments we generally prefer names to be \nnouns rather than verbs.\n\nCommit messages are expected to explain the motivation the changes not \njust list the functions that are added.\n\nOverall the changes are moving in the right direction, though it's a \nshame the fixup for sorting the output of \"git change list\" isn't \nincluded here.\n\nAs well as aligning the code style, I think we need to pin down the \ndetails in patch 1 as there seems to be some on-going discussion about \nthe design.\n\nBest Wishes\n\nPhillip\n\n> Chris Poucet (5):\n> sha1-array: implement oid_array_readonly_contains\n> ref-filter: add the metas namespace to ref-filter\n> evolve: add delete command\n> evolve: add documentation for `git change`\n> evolve: add tests for the git-change command\n> \n> Stefan Xenos (5):\n> technical doc: add a design doc for the evolve command\n> evolve: add support for parsing metacommits\n> evolve: add the change-table structure\n> evolve: add support for writing metacommits\n> evolve: implement the git change command\n> \n> .gitignore | 1 +\n> Documentation/git-change.txt | 55 ++\n> Documentation/technical/evolve.txt | 1070 ++++++++++++++++++++++++++++\n> Makefile | 4 +\n> builtin.h | 1 +\n> builtin/change.c | 330 +++++++++\n> change-table.c | 164 +++++\n> change-table.h | 122 ++++\n> commit.c | 13 +\n> commit.h | 5 +\n> git.c | 1 +\n> metacommit-parser.c | 97 +++\n> metacommit-parser.h | 19 +\n> metacommit.c | 410 +++++++++++\n> metacommit.h | 75 ++\n> oid-array.c | 12 +\n> oid-array.h | 7 +\n> ref-filter.c | 10 +-\n> ref-filter.h | 10 +-\n> t/helper/test-oid-array.c | 6 +\n> t/t0064-oid-array.sh | 22 +\n> t/t9990-changes.sh | 148 ++++\n> 22 files changed, 2577 insertions(+), 5 deletions(-)\n> create mode 100644 Documentation/git-change.txt\n> create mode 100644 Documentation/technical/evolve.txt\n> create mode 100644 builtin/change.c\n> create mode 100644 change-table.c\n> create mode 100644 change-table.h\n> create mode 100644 metacommit-parser.c\n> create mode 100644 metacommit-parser.h\n> create mode 100644 metacommit.c\n> create mode 100644 metacommit.h\n> create mode 100755 t/t9990-changes.sh\n> \n> \n> base-commit: 3dcec76d9df911ed8321007b1d197c1a206dc164\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1356%2Fpoucet%2Fevolve-v2\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1356/poucet/evolve-v2\n> Pull-Request: https://github.com/gitgitgadget/git/pull/1356\n> \n> Range-diff vs v1:\n> \n> 1: a0cf68f8ba2 ! 1: a5eb9325419 technical doc: add a design doc for the evolve command\n> @@ Documentation/technical/evolve.txt (new)\n> +rebase. You can think of rebase -i as a top-down approach and the evolve command\n> +as the bottom-up approach to the same problem.\n> +\n> ++Revup amend (https://github.com/Skydio/revup/blob/main/docs/amend.md)\n> ++allows insertion of cached changes into any commit in\n> ++the current history, and then reapplies the rest of history on top of\n> ++those changes. It uses a \"git apply --cached\" engine under the hood so\n> ++doesn't touch the working directory (although it will soon use the new\n> ++git merge-tree). When paired with \"revup upload\" which creates and\n> ++pushes multiple branches in the background for you, its possible to\n> ++work on a \"graph\" of changes on a single branch linearly, then have\n> ++the true graph structure created at upload time.\n> ++\n> ++git-revise (https://github.com/mystor/git-revise) does some very\n> ++similar things except it uses \"git merge-file\" combined with manually\n> ++merging the resulting trees. git branchstack\n> ++(https://github.com/krobelus/git-branchstack) can also create branches\n> ++in the background with the same mechanism.\n> ++\n> ++These tools don't store any external state, but as such also don't\n> ++provide any specific collaboration mechanism for individual changes.\n> ++\n> +Several patch queue managers have been built on top of git (such as topgit,\n> +stgit, and quilt). They address the same user need. However they also rely on\n> +state managed outside git that needs to be kept in sync. Such state can be\n> 2: 84588312c1d = 2: ed5106d6080 sha1-array: implement oid_array_readonly_contains\n> 3: 54e559967df ! 3: c59066ebc10 ref-filter: add the metas namespace to ref-filter\n> @@ ref-filter.c: int filter_refs(struct ref_array *array, struct ref_filter *filter\n> \n> ## ref-filter.h ##\n> @@\n> + #define FILTER_REFS_TAGS 0x0002\n> #define FILTER_REFS_BRANCHES 0x0004\n> #define FILTER_REFS_REMOTES 0x0008\n> - #define FILTER_REFS_OTHERS 0x0010\n> -+#define FILTER_REFS_CHANGES 0x0040\n> +-#define FILTER_REFS_OTHERS 0x0010\n> ++#define FILTER_REFS_CHANGES 0x0010\n> ++#define FILTER_REFS_OTHERS 0x0040\n> #define FILTER_REFS_ALL (FILTER_REFS_TAGS | FILTER_REFS_BRANCHES | \\\n> -\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS)\n> +\t\t\t\t FILTER_REFS_REMOTES | FILTER_REFS_OTHERS | \\\n\nThis looks good\n\n> 4: 2e9a4a9bd81 ! 4: 408941e7400 evolve: add support for parsing metacommits\n> @@ Makefile: LIB_OBJS += merge-ort.o\n> LIB_OBJS += name-hash.o\n> LIB_OBJS += negotiator/default.o\n> \n> + ## commit.c ##\n> +@@ commit.c: struct commit_list *reverse_commit_list(struct commit_list *list)\n> + \treturn next;\n> + }\n> +\n> ++struct commit *get_commit_by_index(struct commit_list *to_search, int index)\n> ++{\n> ++\twhile (to_search && index) {\n> ++\t\tto_search = to_search->next;\n> ++\t\tindex--;\n> ++\t}\n> ++\n> ++\tif (!to_search)\n> ++\t\treturn NULL;\n> ++\n> ++\treturn to_search->item;\n> ++}\n> ++\n> + void free_commit_list(struct commit_list *list)\n> + {\n> + \twhile (list)\n> +\n> + ## commit.h ##\n> +@@ commit.h: struct commit_list *copy_commit_list(struct commit_list *list);\n> + /* Modify list in-place to reverse it, returning new head; list will be tail */\n> + struct commit_list *reverse_commit_list(struct commit_list *list);\n> +\n> ++/* Returns the commit at `index` or NULL if the index exceeds the `to_search`\n> ++ * list */\n> ++struct commit *get_commit_by_index(struct commit_list *to_search, int index);\n> ++\n> + void free_commit_list(struct commit_list *list);\n> +\n> ++\n> + struct rev_info; /* in revision.h, it circularly uses enum cmit_fmt */\n> +\n> + int has_non_ascii(const char *text);\n> +\n> ## metacommit-parser.c (new) ##\n> @@\n> +#include \"cache.h\"\n> @@ metacommit-parser.c (new)\n> +\treturn NULL;\n> +}\n> +\n> -+static struct commit *get_commit_by_index(struct commit_list *to_search, int index)\n> -+{\n> -+\twhile (to_search && index) {\n> -+\t\tto_search = to_search->next;\n> -+\t\tindex--;\n> -+\t}\n> -+\n> -+\tif (!to_search)\n> -+\t\treturn NULL;\n> -+\n> -+\treturn to_search->item;\n> -+}\n> -+\n> +/*\n> + * Writes the index of the content parent to \"result\". Returns the metacommit\n> + * type. See the METACOMMIT_TYPE_* constants.\n> + */\n> -+static int index_of_content_commit(const char *buffer, int *result)\n> ++static enum metacommit_type index_of_content_commit(const char *buffer, int *result)\n> +{\n> +\tint index = 0;\n> +\tint ret = METACOMMIT_TYPE_NONE;\n> @@ metacommit-parser.c (new)\n> +\t\tchar next = *parent_types;\n> +\t\tif (next == ' ' || parent_types >= end) {\n> +\t\t\tif (enum_length == 1) {\n> -+\t\t\t\tchar first_char_in_enum = *enum_start;\n> -+\t\t\t\tif (first_char_in_enum == 'c') {\n> ++\t\t\t\tchar type = *enum_start;\n> ++\t\t\t\tif (type == 'c') {\n> +\t\t\t\t\tret = METACOMMIT_TYPE_NORMAL;\n> +\t\t\t\t\tbreak;\n> +\t\t\t\t}\n> -+\t\t\t\tif (first_char_in_enum == 'a') {\n> ++\t\t\t\tif (type == 'a') {\n> +\t\t\t\t\tret = METACOMMIT_TYPE_ABANDONED;\n> +\t\t\t\t\tbreak;\n> +\t\t\t\t}\n> @@ metacommit-parser.c (new)\n> + * Writes the content parent's object id to \"content\".\n> + * Returns the metacommit type. See the METACOMMIT_TYPE_* constants.\n> + */\n> -+int get_metacommit_content(struct commit *commit, struct object_id *content)\n> ++enum metacommit_type get_metacommit_content(struct commit *commit, struct object_id *content)\n> +{\n> +\tconst char *buffer = get_commit_buffer(commit, NULL);\n> +\tint index = 0;\n> -+\tint ret = index_of_content_commit(buffer, &index);\n> ++\tenum metacommit_type ret = index_of_content_commit(buffer, &index);\n> +\tstruct commit *content_parent;\n> +\n> +\tif (ret == METACOMMIT_TYPE_NONE)\n> @@ metacommit-parser.h (new)\n> +#include \"commit.h\"\n> +#include \"hash.h\"\n> +\n> -+/* Indicates a normal commit (non-metacommit) */\n> -+#define METACOMMIT_TYPE_NONE 0\n> -+/* Indicates a metacommit with normal content (non-abandoned) */\n> -+#define METACOMMIT_TYPE_NORMAL 1\n> -+/* Indicates a metacommit with abandoned content */\n> -+#define METACOMMIT_TYPE_ABANDONED 2\n> -+\n> -+struct commit;\n> ++enum metacommit_type {\n> ++\t/* Indicates a normal commit (non-metacommit) */\n> ++\tMETACOMMIT_TYPE_NONE = 0,\n> ++\t/* Indicates a metacommit with normal content (non-abandoned) */\n> ++\tMETACOMMIT_TYPE_NORMAL = 1,\n> ++\t/* Indicates a metacommit with abandoned content */\n> ++\tMETACOMMIT_TYPE_ABANDONED = 2,\n> ++};\n> +\n> -+extern int get_metacommit_content(\n> ++enum metacommit_type get_metacommit_content(\n> +\tstruct commit *commit, struct object_id *content);\n> +\n> +#endif\n> 5: 2b3a00a6702 ! 5: 48cd92d35ef evolve: add the change-table structure\n> @@ change-table.c (new)\n> +#include \"ref-filter.h\"\n> +#include \"metacommit-parser.h\"\n> +\n> -+void change_table_init(struct change_table *to_initialize)\n> ++void change_table_init(struct change_table *table)\n> +{\n> -+\tmemset(to_initialize, 0, sizeof(*to_initialize));\n> -+\tmem_pool_init(&to_initialize->memory_pool, 0);\n> -+\tto_initialize->memory_pool.block_alloc = 4*1024 - sizeof(struct mp_block);\n> -+\toidmap_init(&to_initialize->oid_to_metadata_index, 0);\n> -+\tstring_list_init_dup(&to_initialize->refname_to_change_head);\n> ++\tmemset(table, 0, sizeof(*table));\n> ++\tmem_pool_init(&table->memory_pool, 0);\n> ++\toidmap_init(&table->oid_to_metadata_index, 0);\n> ++\tstrmap_init(&table->refname_to_change_head);\n> +}\n> +\n> -+static void change_list_clear(struct change_list *to_clear) {\n> -+\tstring_list_clear(&to_clear->additional_refnames, 0);\n> ++static void change_list_clear(struct change_list *change_list) {\n> ++\tstrset_clear(&change_list->refnames);\n> +}\n> +\n> +static void commit_change_list_entry_clear(\n> -+\tstruct commit_change_list_entry *to_clear) {\n> -+\tchange_list_clear(&to_clear->changes);\n> ++\tstruct commit_change_list_entry *entry) {\n> ++\tchange_list_clear(&entry->changes);\n> +}\n> +\n> -+void change_table_clear(struct change_table *to_clear)\n> ++void change_table_clear(struct change_table *table)\n> +{\n> +\tstruct oidmap_iter iter;\n> +\tstruct commit_change_list_entry *next;\n> -+\tfor (next = oidmap_iter_first(&to_clear->oid_to_metadata_index, &iter);\n> ++\tfor (next = oidmap_iter_first(&table->oid_to_metadata_index, &iter);\n> +\t\tnext;\n> +\t\tnext = oidmap_iter_next(&iter)) {\n> +\n> +\t\tcommit_change_list_entry_clear(next);\n> +\t}\n> +\n> -+\toidmap_free(&to_clear->oid_to_metadata_index, 0);\n> -+\tstring_list_clear(&to_clear->refname_to_change_head, 0);\n> -+\tmem_pool_discard(&to_clear->memory_pool, 0);\n> ++\toidmap_free(&table->oid_to_metadata_index, 0);\n> ++\tstrmap_clear(&table->refname_to_change_head, 0);\n> ++\tmem_pool_discard(&table->memory_pool, 0);\n> +}\n> +\n> -+static void add_head_to_commit(struct change_table *to_modify,\n> -+\tconst struct object_id *to_add, const char *refname)\n> ++static void add_head_to_commit(struct change_table *table,\n> ++\t\t\t const struct object_id *to_add,\n> ++\t\t\t const char *refname)\n> +{\n> +\tstruct commit_change_list_entry *entry;\n> +\n> -+\t/**\n> -+\t * Note: the indices in the map are 1-based. 0 is used to indicate a missing\n> -+\t * element.\n> -+\t */\n> -+\tentry = oidmap_get(&to_modify->oid_to_metadata_index, to_add);\n> ++\tentry = oidmap_get(&table->oid_to_metadata_index, to_add);\n> +\tif (!entry) {\n> -+\t\tentry = mem_pool_calloc(&to_modify->memory_pool, 1,\n> -+\t\t\tsizeof(*entry));\n> ++\t\tentry = mem_pool_calloc(&table->memory_pool, 1, sizeof(*entry));\n> +\t\toidcpy(&entry->entry.oid, to_add);\n> -+\t\toidmap_put(&to_modify->oid_to_metadata_index, entry);\n> -+\t\tstring_list_init_nodup(&entry->changes.additional_refnames);\n> ++\t\tstrset_init(&entry->changes.refnames);\n> ++\t\toidmap_put(&table->oid_to_metadata_index, entry);\n> +\t}\n> -+\n> -+\tif (!entry->changes.first_refname)\n> -+\t\tentry->changes.first_refname = refname;\n> -+\telse\n> -+\t\tstring_list_insert(&entry->changes.additional_refnames, refname);\n> ++\tstrset_add(&entry->changes.refnames, refname);\n> +}\n> +\n> -+void change_table_add(struct change_table *to_modify, const char *refname,\n> -+\tstruct commit *to_add)\n> ++void change_table_add(struct change_table *table,\n> ++\t\t const char *refname,\n> ++\t\t struct commit *to_add)\n> +{\n> +\tstruct change_head *new_head;\n> -+\tstruct string_list_item *new_item;\n> +\tint metacommit_type;\n> +\n> -+\tnew_head = mem_pool_calloc(&to_modify->memory_pool, 1,\n> -+\t\tsizeof(*new_head));\n> ++\tnew_head = mem_pool_calloc(&table->memory_pool, 1, sizeof(*new_head));\n> +\n> +\toidcpy(&new_head->head, &to_add->object.oid);\n> +\n> +\tmetacommit_type = get_metacommit_content(to_add, &new_head->content);\n> ++\t/* If to_add is not a metacommit then the content is to_add itself,\n> ++\t * otherwise it will have been set by the call to\n> ++\t * get_metacommit_content.\n> ++\t */\n> +\tif (metacommit_type == METACOMMIT_TYPE_NONE)\n> +\t\toidcpy(&new_head->content, &to_add->object.oid);\n> +\tnew_head->abandoned = (metacommit_type == METACOMMIT_TYPE_ABANDONED);\n> +\tnew_head->remote = starts_with(refname, \"refs/remote/\");\n> +\tnew_head->hidden = starts_with(refname, \"refs/hiddenmetas/\");\n> +\n> -+\tnew_item = string_list_insert(&to_modify->refname_to_change_head, refname);\n> -+\tnew_item->util = new_head;\n> -+\t/* Use pointers to the copy of the string we're retaining locally */\n> -+\trefname = new_item->string;\n> -+\n> -+\tif (!oideq(&new_head->content, &new_head->head))\n> -+\t\tadd_head_to_commit(to_modify, &new_head->content, refname);\n> -+\tadd_head_to_commit(to_modify, &new_head->head, refname);\n> -+}\n> -+\n> -+void change_table_add_all_visible(struct change_table *to_modify,\n> -+\tstruct repository* repo)\n> -+{\n> -+\tstruct ref_filter filter;\n> -+\tconst char *name_patterns[] = {NULL};\n> -+\tmemset(&filter, 0, sizeof(filter));\n> -+\tfilter.kind = FILTER_REFS_CHANGES;\n> -+\tfilter.name_patterns = name_patterns;\n> ++\tstrmap_put(&table->refname_to_change_head, refname, new_head);\n> +\n> -+\tchange_table_add_matching_filter(to_modify, repo, &filter);\n> ++\tif (!oideq(&new_head->content, &new_head->head)) {\n> ++\t\t/* We also remember to link between refname and the content oid */\n> ++\t\tadd_head_to_commit(table, &new_head->content, refname);\n> ++\t}\n> ++\tadd_head_to_commit(table, &new_head->head, refname);\n> +}\n> +\n> -+void change_table_add_matching_filter(struct change_table *to_modify,\n> -+\tstruct repository* repo, struct ref_filter *filter)\n> ++static void change_table_add_matching_filter(struct change_table *table,\n> ++\t\t\t\t\t struct repository* repo,\n> ++\t\t\t\t\t struct ref_filter *filter)\n> +{\n> -+\tstruct ref_array matching_refs;\n> +\tint i;\n> ++\tstruct ref_array matching_refs = { 0 };\n> +\n> -+\tmemset(&matching_refs, 0, sizeof(matching_refs));\n> +\tfilter_refs(&matching_refs, filter, filter->kind);\n> +\n> -+\t/**\n> ++\t/*\n> +\t * Determine the object id for the latest content commit for each change.\n> +\t * Fetch the commit at the head of each change ref. If it's a normal commit,\n> +\t * that's the commit we want. If it's a metacommit, locate its content parent\n> @@ change-table.c (new)\n> +\n> +\tfor (i = 0; i < matching_refs.nr; i++) {\n> +\t\tstruct ref_array_item *item = matching_refs.items[i];\n> -+\t\tstruct commit *commit = item->commit;\n> ++\t\tstruct commit *commit;\n> +\n> -+\t\tcommit = lookup_commit_reference_gently(repo, &item->objectname, 1);\n> -+\n> -+\t\tif (commit)\n> -+\t\t\tchange_table_add(to_modify, item->refname, commit);\n> ++\t\tcommit = lookup_commit_reference(repo, &item->objectname);\n> ++\t\tif (!commit) {\n> ++\t\t\tBUG(\"Invalid commit for refs/meta: %s\", item->refname);\n> ++\t\t}\n> ++\t\tchange_table_add(table, item->refname, commit);\n> +\t}\n> +\n> +\tref_array_clear(&matching_refs);\n> +}\n> +\n> ++void change_table_add_all_visible(struct change_table *table,\n> ++\tstruct repository* repo)\n> ++{\n> ++\tstruct ref_filter filter = { 0 };\n> ++\tconst char *name_patterns[] = {NULL};\n> ++\tfilter.kind = FILTER_REFS_CHANGES;\n> ++\tfilter.name_patterns = name_patterns;\n> ++\n> ++\tchange_table_add_matching_filter(table, repo, &filter);\n> ++}\n> ++\n> +static int return_true_callback(const char *refname, void *cb_data)\n> +{\n> +\treturn 1;\n> +}\n> +\n> -+int change_table_has_change_referencing(struct change_table *changes,\n> ++int change_table_has_change_referencing(struct change_table *table,\n> +\tconst struct object_id *referenced_commit_id)\n> +{\n> -+\treturn for_each_change_referencing(changes, referenced_commit_id,\n> ++\treturn for_each_change_referencing(table, referenced_commit_id,\n> +\t\treturn_true_callback, NULL);\n> +}\n> +\n> +int for_each_change_referencing(struct change_table *table,\n> +\tconst struct object_id *referenced_commit_id, each_change_fn fn, void *cb_data)\n> +{\n> -+\tconst struct change_list *changes;\n> -+\tint i;\n> -+\tint retvalue;\n> -+\tstruct commit_change_list_entry *entry;\n> ++\tint ret;\n> ++\tstruct commit_change_list_entry *ccl_entry;\n> ++\tstruct hashmap_iter iter;\n> ++\tstruct strmap_entry *entry;\n> +\n> -+\tentry = oidmap_get(&table->oid_to_metadata_index,\n> -+\t\treferenced_commit_id);\n> ++\tccl_entry = oidmap_get(&table->oid_to_metadata_index,\n> ++\t\t\t referenced_commit_id);\n> +\t/* If this commit isn't referenced by any changes, it won't be in the map */\n> -+\tif (!entry)\n> ++\tif (!ccl_entry)\n> +\t\treturn 0;\n> -+\tchanges = &entry->changes;\n> -+\tif (!changes->first_refname)\n> -+\t\treturn 0;\n> -+\tretvalue = fn(changes->first_refname, cb_data);\n> -+\tfor (i = 0; retvalue == 0 && i < changes->additional_refnames.nr; i++)\n> -+\t\tretvalue = fn(changes->additional_refnames.items[i].string, cb_data);\n> -+\treturn retvalue;\n> ++\tstrset_for_each_entry(&ccl_entry->changes.refnames, &iter, entry) {\n> ++\t\tret = fn(entry->key, cb_data);\n> ++\t\tif (ret != 0) break;\n> ++\t}\n> ++\treturn ret;\n> +}\n> +\n> -+struct change_head* get_change_head(struct change_table *heads,\n> ++struct change_head* get_change_head(struct change_table *table,\n> +\tconst char* refname)\n> +{\n> -+\tstruct string_list_item *item = string_list_lookup(\n> -+\t\t&heads->refname_to_change_head, refname);\n> -+\n> -+\tif (!item)\n> -+\t\treturn NULL;\n> -+\n> -+\treturn (struct change_head *)item->util;\n> ++\treturn strmap_get(&table->refname_to_change_head, refname);\n> +}\n> \n> ## change-table.h (new) ##\n> @@ change-table.h (new)\n> +#define CHANGE_TABLE_H\n> +\n> +#include \"oidmap.h\"\n> ++#include \"strmap.h\"\n> +\n> +struct commit;\n> +struct ref_filter;\n> +\n> +/**\n> -+ * This struct holds a list of change refs. The first element is stored inline,\n> -+ * to optimize for small lists.\n> ++ * This struct holds a set of change refs.\n> + */\n> +struct change_list {\n> +\t/**\n> -+\t * Ref name for the first change in the list, or null if none.\n> -+\t *\n> ++\t * The refnames in this set.\n> +\t * This field is private. Use for_each_change_in to read.\n> +\t */\n> -+\tconst char* first_refname;\n> -+\t/**\n> -+\t * List of additional change refs. Note that this is empty if the list\n> -+\t * contains 0 or 1 elements.\n> -+\t *\n> -+\t * This field is private. Use for_each_change_in to read.\n> -+\t */\n> -+\tstruct string_list additional_refnames;\n> ++\tstruct strset refnames;\n> +};\n> +\n> +/**\n> @@ change-table.h (new)\n> +};\n> +\n> +/**\n> -+ * Holds information about the heads of each change, and permits effecient\n> ++ * Holds information about the heads of each change, and permits efficient\n> + * lookup from a commit to the changes that reference it directly.\n> + *\n> + * All fields should be considered private. Use the change_table functions\n> @@ change-table.h (new)\n> +\t/* Map object_id to commit_change_list_entry structs. */\n> +\tstruct oidmap oid_to_metadata_index;\n> +\t/**\n> -+\t * List of ref names. The util value points to a change_head structure\n> -+\t * allocated from memory_pool.\n> ++\t * Map of refnames to change_head structure which are allocated from\n> ++\t * memory_pool.\n> +\t */\n> -+\tstruct string_list refname_to_change_head;\n> ++\tstruct strmap refname_to_change_head;\n> +};\n> +\n> -+extern void change_table_init(struct change_table *to_initialize);\n> -+extern void change_table_clear(struct change_table *to_clear);\n> ++extern void change_table_init(struct change_table *table);\n> ++extern void change_table_clear(struct change_table *table);\n> +\n> +/* Adds the given change head to the change_table struct */\n> -+extern void change_table_add(struct change_table *to_modify,\n> -+\tconst char *refname, struct commit *target);\n> ++extern void change_table_add(struct change_table *table,\n> ++\t\t\t const char *refname,\n> ++\t\t\t struct commit *target);\n> +\n> +/**\n> + * Adds the non-hidden local changes to the given change_table struct.\n> + */\n> -+extern void change_table_add_all_visible(struct change_table *to_modify,\n> -+\tstruct repository *repo);\n> -+\n> -+/*\n> -+ * Adds all changes matching the given ref filter to the given change_table\n> -+ * struct.\n> -+ */\n> -+extern void change_table_add_matching_filter(struct change_table *to_modify,\n> -+\tstruct repository* repo, struct ref_filter *filter);\n> ++extern void change_table_add_all_visible(struct change_table *table,\n> ++\t\t\t\t\t struct repository *repo);\n> +\n> +typedef int each_change_fn(const char *refname, void *cb_data);\n> +\n> -+extern int change_table_has_change_referencing(struct change_table *changes,\n> ++extern int change_table_has_change_referencing(\n> ++\tstruct change_table *table,\n> +\tconst struct object_id *referenced_commit_id);\n> +\n> +/**\n> @@ change-table.h (new)\n> + * For normal commits, this is the list of changes that have this commit as\n> + * their latest content.\n> + */\n> -+extern int for_each_change_referencing(struct change_table *heads,\n> -+\tconst struct object_id *referenced_commit_id, each_change_fn fn, void *cb_data);\n> ++extern int for_each_change_referencing(\n> ++\tstruct change_table *table,\n> ++\tconst struct object_id *referenced_commit_id,\n> ++\teach_change_fn fn,\n> ++\tvoid *cb_data);\n> +\n> +/**\n> + * Returns the change head for the given refname. Returns NULL if no such change\n> + * exists.\n> + */\n> -+extern struct change_head* get_change_head(struct change_table *heads,\n> ++extern struct change_head* get_change_head(struct change_table *table,\n> +\tconst char* refname);\n> +\n> +#endif\n> 6: 56c6770997b ! 6: 353d97d0f38 evolve: add support for writing metacommits\n> @@ metacommit.c (new)\n> +#include \"change-table.h\"\n> +#include \"refs.h\"\n> +\n> -+void init_metacommit_data(struct metacommit_data *state)\n> -+{\n> -+\tmemset(state, 0, sizeof(*state));\n> -+}\n> -+\n> +void clear_metacommit_data(struct metacommit_data *state)\n> +{\n> ++\toidcpy(&state->content, null_oid());\n> +\toid_array_clear(&state->replace);\n> +\toid_array_clear(&state->origin);\n> ++\tstate->abandoned = 0;\n> +}\n> +\n> +static void compute_default_change_name(struct commit *initial_commit,\n> -+\tstruct strbuf* result)\n> ++\t\t\t\t\tstruct strbuf* result)\n> +{\n> -+\tstruct strbuf default_name;\n> ++\tstruct strbuf default_name = STRBUF_INIT;\n> +\tconst char *buffer;\n> +\tconst char *subject;\n> +\tconst char *eol;\n> -+\tint len;\n> -+\tstrbuf_init(&default_name, 0);\n> ++\tsize_t len;\n> +\tbuffer = get_commit_buffer(initial_commit, NULL);\n> +\tfind_commit_subject(buffer, &subject);\n> +\teol = strchrnul(subject, '\\n');\n> -+\tfor (len = 0;subject < eol && len < 10; ++subject, ++len) {\n> ++\tfor (len = 0; subject < eol && len < 10; subject++, len++) {\n> +\t\tchar next = *subject;\n> +\t\tif (isspace(next))\n> +\t\t\tcontinue;\n> @@ metacommit.c (new)\n> +\t\tstrbuf_addch(&default_name, next);\n> +\t}\n> +\tsanitize_refname_component(default_name.buf, result);\n> ++\tunuse_commit_buffer(initial_commit, buffer);\n> +}\n> +\n> -+/**\n> ++/*\n> + * Computes a change name for a change rooted at the given initial commit. Good\n> + * change names should be memorable, unique, and easy to type. They are not\n> + * required to match the commit comment.\n> + */\n> +static void compute_change_name(struct commit *initial_commit, struct strbuf* result)\n> +{\n> -+\tstruct strbuf default_name;\n> ++\tstruct strbuf default_name = STRBUF_INIT;\n> +\tstruct object_id unused;\n> +\n> -+\tstrbuf_init(&default_name, 0);\n> +\tif (initial_commit)\n> +\t\tcompute_default_change_name(initial_commit, &default_name);\n> +\telse\n> -+\t\tstrbuf_addstr(&default_name, \"change\");\n> ++\t\tBUG(\"initial commit is NULL\");\n> +\tstrbuf_addstr(result, \"refs/metas/\");\n> +\tstrbuf_addbuf(result, &default_name);\n> +\n> +\t/* If there is already a change of this name, append a suffix */\n> +\tif (!read_ref(result->buf, &unused)) {\n> +\t\tint suffix = 2;\n> -+\t\tint original_length = result->len;\n> ++\t\tsize_t original_length = result->len;\n> +\n> +\t\twhile (1) {\n> +\t\t\tstrbuf_addf(result, \"%d\", suffix);\n> +\t\t\tif (read_ref(result->buf, &unused))\n> +\t\t\t\tbreak;\n> -+\t\t\tstrbuf_remove(result, original_length, result->len - original_length);\n> ++\t\t\tstrbuf_remove(result, original_length,\n> ++\t\t\t\t result->len - original_length);\n> +\t\t\t++suffix;\n> +\t\t}\n> +\t}\n> @@ metacommit.c (new)\n> +\tstrbuf_release(&default_name);\n> +}\n> +\n> -+struct resolve_metacommit_callback_data\n> ++struct resolve_metacommit_context\n> +{\n> +\tstruct change_table* active_changes;\n> +\tstruct string_list *changes;\n> @@ metacommit.c (new)\n> +\n> +static int resolve_metacommit_callback(const char *refname, void *cb_data)\n> +{\n> -+\tstruct resolve_metacommit_callback_data *data = (struct resolve_metacommit_callback_data *)cb_data;\n> ++\tstruct resolve_metacommit_context *data = cb_data;\n> +\tstruct change_head *chhead;\n> +\n> +\tchhead = get_change_head(data->active_changes, refname);\n> +\n> +\tif (data->changes)\n> -+\t\tstring_list_append(data->changes, refname)->util = &(chhead->head);\n> ++\t\tstring_list_append(data->changes, refname)->util = &chhead->head;\n> +\tif (data->heads)\n> +\t\toid_array_append(data->heads, &(chhead->head));\n> +\n> +\treturn 0;\n> +}\n> +\n> -+/**\n> ++/*\n> + * Produces the final form of a metacommit based on the current change refs.\n> + */\n> +static void resolve_metacommit(\n> @@ metacommit.c (new)\n> +\tstruct string_list *to_advance,\n> +\tint allow_append)\n> +{\n> -+\tint i;\n> -+\tint len = to_resolve->replace.nr;\n> -+\tstruct resolve_metacommit_callback_data cbdata;\n> ++\tsize_t i;\n> ++\tsize_t len = to_resolve->replace.nr;\n> ++\tstruct resolve_metacommit_context ctx = {\n> ++\t\t.active_changes = active_changes,\n> ++\t\t.changes = to_advance,\n> ++\t\t.heads = &resolved_output->replace\n> ++\t};\n> +\tint old_change_list_length = to_advance->nr;\n> +\tstruct commit* content;\n> +\n> +\toidcpy(&resolved_output->content, &to_resolve->content);\n> +\n> -+\t/* First look for changes that point to any of the replacement edges in the\n> ++\t/*\n> ++\t * First look for changes that point to any of the replacement edges in the\n> +\t * metacommit. These will be the changes that get advanced by this\n> -+\t * metacommit. */\n> ++\t * metacommit.\n> ++\t */\n> +\tresolved_output->abandoned = to_resolve->abandoned;\n> -+\tcbdata.active_changes = active_changes;\n> -+\tcbdata.changes = to_advance;\n> -+\tcbdata.heads = &(resolved_output->replace);\n> +\n> +\tif (allow_append) {\n> +\t\tfor (i = 0; i < len; i++) {\n> +\t\t\tint old_number = resolved_output->replace.nr;\n> -+\t\t\tfor_each_change_referencing(active_changes, &(to_resolve->replace.oid[i]),\n> -+\t\t\t\tresolve_metacommit_callback, &cbdata);\n> ++\t\t\tfor_each_change_referencing(\n> ++\t\t\t\tactive_changes,\n> ++\t\t\t\t&(to_resolve->replace.oid[i]),\n> ++\t\t\t\tresolve_metacommit_callback,\n> ++\t\t\t\t&ctx);\n> +\t\t\t/* If no changes were found, use the unresolved value. */\n> +\t\t\tif (old_number == resolved_output->replace.nr)\n> -+\t\t\t\toid_array_append(&(resolved_output->replace), &(to_resolve->replace.oid[i]));\n> ++\t\t\t\toid_array_append(&(resolved_output->replace),\n> ++\t\t\t\t\t\t &(to_resolve->replace.oid[i]));\n> +\t\t}\n> +\t}\n> +\n> -+\tcbdata.changes = NULL;\n> -+\tcbdata.heads = &(resolved_output->origin);\n> ++\tctx.changes = NULL;\n> ++\tctx.heads = &(resolved_output->origin);\n> +\n> +\tlen = to_resolve->origin.nr;\n> +\tfor (i = 0; i < len; i++) {\n> +\t\tint old_number = resolved_output->origin.nr;\n> -+\t\tfor_each_change_referencing(active_changes, &(to_resolve->origin.oid[i]),\n> -+\t\t\tresolve_metacommit_callback, &cbdata);\n> ++\t\tfor_each_change_referencing(\n> ++\t\t\tactive_changes,\n> ++\t\t\t&(to_resolve->origin.oid[i]),\n> ++\t\t\tresolve_metacommit_callback,\n> ++\t\t\t&ctx);\n> +\t\tif (old_number == resolved_output->origin.nr)\n> -+\t\t\toid_array_append(&(resolved_output->origin), &(to_resolve->origin.oid[i]));\n> ++\t\t\toid_array_append(&(resolved_output->origin),\n> ++\t\t\t\t\t &(to_resolve->origin.oid[i]));\n> +\t}\n> +\n> -+\t/* If no changes were advanced by this metacommit, we'll need to create a new\n> -+\t * one. */\n> ++\t/*\n> ++\t * If no changes were advanced by this metacommit, we'll need to create\n> ++\t * a new one. */\n> +\tif (to_advance->nr == old_change_list_length) {\n> +\t\tstruct strbuf change_name;\n> +\n> +\t\tstrbuf_init(&change_name, 80);\n> -+\t\tcontent = lookup_commit_reference_gently(repo, &(to_resolve->content), 1);\n> ++\n> ++\t\tcontent = lookup_commit_reference_gently(\n> ++\t\t\trepo, &(to_resolve->content), 1);\n> +\n> +\t\tcompute_change_name(content, &change_name);\n> +\t\tstring_list_append(to_advance, change_name.buf);\n> @@ metacommit.c (new)\n> +\n> +\twhile (--i >= 0) {\n> +\t\tstruct object_id *next = &(to_lookup->oid[i]);\n> -+\t\tstruct commit *commit = lookup_commit_reference_gently(repo, next, 1);\n> ++\t\tstruct commit *commit =\n> ++\t\t\tlookup_commit_reference_gently(repo, next, 1);\n> +\t\tcommit_list_insert(commit, result);\n> +\t}\n> +}\n> +\n> +#define PARENT_TYPE_PREFIX \"parent-type \"\n> +\n> -+/**\n> -+ * Creates a new metacommit object with the given content. Writes the object\n> -+ * id of the newly-created commit to result.\n> -+ */\n> +int write_metacommit(struct repository *repo, struct metacommit_data *state,\n> +\tstruct object_id *result)\n> +{\n> +\tstruct commit_list *parents = NULL;\n> +\tstruct strbuf comment;\n> -+\tint i;\n> ++\tsize_t i;\n> +\tstruct commit *content;\n> +\n> +\tstrbuf_init(&comment, strlen(PARENT_TYPE_PREFIX)\n> @@ metacommit.c (new)\n> +\t\tstrbuf_addstr(&comment, \" o\");\n> +\n> +\t/* The parents list will be freed by this call. */\n> -+\tcommit_tree(comment.buf, comment.len, repo->hash_algo->empty_tree, parents,\n> -+\t\tresult, NULL, NULL);\n> ++\tcommit_tree(\n> ++\t\tcomment.buf,\n> ++\t\tcomment.len,\n> ++\t\trepo->hash_algo->empty_tree,\n> ++\t\tparents,\n> ++\t\tresult,\n> ++\t\tNULL,\n> ++\t\tNULL);\n> +\n> +\tstrbuf_release(&comment);\n> +\treturn 0;\n> +}\n> +\n> -+/**\n> ++/*\n> + * Returns true iff the given metacommit is abandoned, has one or more origin\n> + * parents, or has one or more replacement parents.\n> + */\n> @@ metacommit.c (new)\n> + * to append to existing changes wherever possible instead of creating new ones.\n> + * If override_change is non-null, only the given change ref will be updated.\n> + *\n> -+ * options is a bitwise combination of the UPDATE_OPTION_* flags.\n> -+ */\n> -+int record_metacommit(\n> -+\tstruct repository *repo,\n> -+\tconst struct metacommit_data *metacommit, const char *override_change,\n> -+\tint options, struct strbuf *err)\n> -+{\n> -+\t\tstruct change_table chtable;\n> -+\t\tstruct string_list changes;\n> -+\t\tint result;\n> -+\n> -+\t\tchange_table_init(&chtable);\n> -+\t\tchange_table_add_all_visible(&chtable, repo);\n> -+\t\tstring_list_init_dup(&changes);\n> -+\n> -+\t\tresult = record_metacommit_withresult(repo, &chtable, metacommit,\n> -+\t\t\toverride_change, options, err, &changes);\n> -+\n> -+\t\tstring_list_clear(&changes, 0);\n> -+\t\tchange_table_clear(&chtable);\n> -+\t\treturn result;\n> -+}\n> -+\n> -+/*\n> -+ * Records the relationships described by the given metacommit in the\n> -+ * repository.\n> -+ *\n> -+ * If override_change is NULL (the default), an attempt will be made\n> -+ * to append to existing changes wherever possible instead of creating new ones.\n> -+ * If override_change is non-null, only the given change ref will be updated.\n> -+ *\n> + * The changes list is filled in with the list of change refs that were updated,\n> + * with the util pointers pointing to the old object IDS for those changes.\n> + * The object ID pointers all point to objects owned by the change_table and\n> @@ metacommit.c (new)\n> + *\n> + * options is a bitwise combination of the UPDATE_OPTION_* flags.\n> + */\n> -+int record_metacommit_withresult(\n> ++static int record_metacommit_withresult(\n> +\tstruct repository *repo,\n> +\tstruct change_table *chtable,\n> +\tconst struct metacommit_data *metacommit,\n> +\tconst char *override_change,\n> -+\tint options, struct strbuf *err,\n> ++\tint options,\n> ++\tstruct strbuf *err,\n> +\tstruct string_list *changes)\n> +{\n> +\tstatic const char *msg = \"updating change\";\n> -+\tstruct metacommit_data resolved_metacommit;\n> ++\tstruct metacommit_data resolved_metacommit = METACOMMIT_DATA_INIT;\n> +\tstruct object_id commit_target;\n> +\tstruct ref_transaction *transaction = NULL;\n> +\tstruct change_head *overridden_head;\n> +\tconst struct object_id *old_head;\n> +\n> -+\tint i;\n> ++\tsize_t i;\n> +\tint ret = 0;\n> +\tint force = (options & UPDATE_OPTION_FORCE);\n> +\n> -+\tinit_metacommit_data(&resolved_metacommit);\n> -+\n> +\tresolve_metacommit(repo, chtable, metacommit, &resolved_metacommit, changes,\n> +\t\t(options & UPDATE_OPTION_NOAPPEND) == 0);\n> +\n> +\tif (override_change) {\n> +\t\tstring_list_clear(changes, 0);\n> +\t\toverridden_head = get_change_head(chtable, override_change);\n> -+\t\tif (!overridden_head) {\n> ++\t\tif (overridden_head) {\n> +\t\t\t/* This is an existing change */\n> +\t\t\told_head = &overridden_head->head;\n> +\t\t\tif (!force) {\n> @@ metacommit.c (new)\n> +\t\t\t/* ...then this is a newly-created change */\n> +\t\t\told_head = null_oid();\n> +\n> -+\t\t/* The expected \"current\" head of the change is stored in the util\n> -+\t\t * pointer. */\n> -+\t\tstring_list_append(changes, override_change)->util = (void*)old_head;\n> ++\t\t/*\n> ++\t\t * The expected \"current\" head of the change is stored in the\n> ++\t\t * util pointer. Cast required because old_head is const*\n> ++\t\t */\n> ++\t\tstring_list_append(changes, override_change)->util = (void *)old_head;\n> +\t}\n> +\n> +\tif (is_nontrivial_metacommit(&resolved_metacommit)) {\n> @@ metacommit.c (new)\n> +\t\t\tret = -1;\n> +\t\t\tgoto cleanup;\n> +\t\t}\n> -+\t} else\n> -+\t\t/**\n> ++\t} else {\n> ++\t\t/*\n> +\t\t * If the metacommit would only contain a content commit, point to the\n> +\t\t * commit itself rather than creating a trivial metacommit.\n> +\t\t */\n> +\t\toidcpy(&commit_target, &(resolved_metacommit.content));\n> ++\t}\n> +\n> -+\t/**\n> ++\t/*\n> +\t * If a change already exists with this target and we're not forcing an\n> +\t * update to some specific override_change && change, there's nothing to do.\n> +\t */\n> @@ metacommit.c (new)\n> +\t\tfor (i = 0; i < changes->nr; i++) {\n> +\t\t\tstruct string_list_item *it = &changes->items[i];\n> +\n> -+\t\t\t/**\n> ++\t\t\t/*\n> +\t\t\t * The expected current head of the change is stored in the util pointer.\n> +\t\t\t * It is null if the change should be newly-created.\n> +\t\t\t */\n> @@ metacommit.c (new)\n> +\treturn ret;\n> +}\n> +\n> -+/**\n> -+ * Should be invoked after a command that has \"modify\" semantics - commands that\n> -+ * create a new commit based on an old commit and treat the new one as a\n> -+ * replacement for the old one. This method records the replacement in the\n> -+ * change graph, such that a future evolve operation will rebase children of\n> -+ * the old commit onto the new commit.\n> -+ */\n> ++int record_metacommit(\n> ++\tstruct repository *repo,\n> ++\tconst struct metacommit_data *metacommit,\n> ++\tconst char *override_change,\n> ++\tint options,\n> ++\tstruct strbuf *err,\n> ++\tstruct string_list *changes)\n> ++{\n> ++\t\tstruct change_table chtable;\n> ++\t\tint result;\n> ++\n> ++\t\tchange_table_init(&chtable);\n> ++\t\tchange_table_add_all_visible(&chtable, repo);\n> ++\n> ++\t\tresult = record_metacommit_withresult(\n> ++\t\t\trepo,\n> ++\t\t\t&chtable,\n> ++\t\t\tmetacommit,\n> ++\t\t\toverride_change,\n> ++\t\t\toptions,\n> ++\t\t\terr,\n> ++\t\t\tchanges);\n> ++\n> ++\t\tchange_table_clear(&chtable);\n> ++\t\treturn result;\n> ++}\n> ++\n> +void modify_change(\n> +\tstruct repository *repo,\n> +\tconst struct object_id *old_commit,\n> +\tconst struct object_id *new_commit,\n> +\tstruct strbuf *err)\n> +{\n> -+\tstruct metacommit_data metacommit;\n> ++\tstruct string_list changes = STRING_LIST_INIT_DUP;\n> ++\tstruct metacommit_data metacommit = METACOMMIT_DATA_INIT;\n> +\n> -+\tinit_metacommit_data(&metacommit);\n> +\toidcpy(&(metacommit.content), new_commit);\n> +\toid_array_append(&(metacommit.replace), old_commit);\n> +\n> -+\trecord_metacommit(repo, &metacommit, NULL, 0, err);\n> ++\trecord_metacommit(repo, &metacommit, NULL, 0, err, &changes);\n> +\n> +\tclear_metacommit_data(&metacommit);\n> ++\tstring_list_clear(&changes, 0);\n> +}\n> \n> ## metacommit.h (new) ##\n> @@ metacommit.h (new)\n> +#include \"repository.h\"\n> +#include \"string-list.h\"\n> +\n> -+\n> -+struct change_table;\n> -+\n> +/* If specified, non-fast-forward changes are permitted. */\n> +#define UPDATE_OPTION_FORCE 0x0001\n> +/**\n> @@ metacommit.h (new)\n> +\tint abandoned;\n> +};\n> +\n> -+extern void init_metacommit_data(struct metacommit_data *state);\n> ++#define METACOMMIT_DATA_INIT { 0 }\n> +\n> +extern void clear_metacommit_data(struct metacommit_data *state);\n> +\n> -+extern int record_metacommit(struct repository *repo,\n> -+\tconst struct metacommit_data *metacommit,\n> -+\tconst char* override_change, int options, struct strbuf *err);\n> -+\n> -+extern int record_metacommit_withresult(\n> ++/**\n> ++ * Records the relationships described by the given metacommit in the\n> ++ * repository.\n> ++ *\n> ++ * If override_change is NULL (the default), an attempt will be made\n> ++ * to append to existing changes wherever possible instead of creating new ones.\n> ++ * If override_change is non-null, only the given change ref will be updated.\n> ++ *\n> ++ * options is a bitwise combination of the UPDATE_OPTION_* flags.\n> ++ */\n> ++int record_metacommit(\n> +\tstruct repository *repo,\n> -+\tstruct change_table *chtable,\n> +\tconst struct metacommit_data *metacommit,\n> -+\tconst char *override_change,\n> ++\tconst char* override_change,\n> +\tint options,\n> +\tstruct strbuf *err,\n> +\tstruct string_list *changes);\n> +\n> -+extern void modify_change(struct repository *repo,\n> -+\tconst struct object_id *old_commit, const struct object_id *new_commit,\n> ++/**\n> ++ * Should be invoked after a command that has \"modify\" semantics - commands that\n> ++ * create a new commit based on an old commit and treat the new one as a\n> ++ * replacement for the old one. This method records the replacement in the\n> ++ * change graph, such that a future evolve operation will rebase children of\n> ++ * the old commit onto the new commit.\n> ++ */\n> ++void modify_change(\n> ++\tstruct repository *repo,\n> ++\tconst struct object_id *old_commit,\n> ++\tconst struct object_id *new_commit,\n> +\tstruct strbuf *err);\n> +\n> -+extern int write_metacommit(struct repository *repo, struct metacommit_data *state,\n> ++/**\n> ++ * Creates a new metacommit object with the given content. Writes the object\n> ++ * id of the newly-created commit to result.\n> ++ */\n> ++int write_metacommit(\n> ++\tstruct repository *repo,\n> ++\tstruct metacommit_data *state,\n> +\tstruct object_id *result);\n> +\n> +#endif\n> 7: 91402834184 ! 7: f7a90700e0e evolve: implement the git change command\n> @@ builtin/change.c (new)\n> +#include \"ref-filter.h\"\n> +#include \"parse-options.h\"\n> +#include \"metacommit.h\"\n> -+#include \"change-table.h\"\n> +#include \"config.h\"\n> +\n> +static const char * const builtin_change_usage[] = {\n> -+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> ++\tN_(\"git change list [...]\"),\n> ++\tN_(\"git change update [--force] [--replace ...] \"\n> ++\t \"[--origin ...] [--content ]\"),\n> ++\tNULL\n> ++};\n> ++\n> ++static const char * const builtin_list_usage[] = {\n> ++\tN_(\"git change list [...]\"),\n> +\tNULL\n> +};\n> +\n> +static const char * const builtin_update_usage[] = {\n> -+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> ++\tN_(\"git change update [--force] [--replace ...] \"\n> ++\t\"[--origin ...] [--content ]\"),\n> +\tNULL\n> +};\n> +\n> ++static int change_list(int argc, const char **argv, const char* prefix)\n> ++{\n> ++\tstruct option options[] = {\n> ++\t\tOPT_END()\n> ++\t};\n> ++\tstruct ref_filter filter = { 0 };\n> ++\tstruct ref_sorting *sorting;\n> ++\tstruct string_list sorting_options = STRING_LIST_INIT_DUP;\n> ++\tstruct ref_format format = REF_FORMAT_INIT;\n> ++\tstruct ref_array array = { 0 };\n> ++\tsize_t i;\n> ++\n> ++\targc = parse_options(argc, argv, prefix, options, builtin_list_usage, 0);\n> ++\n> ++\tsetup_ref_filter_porcelain_msg();\n> ++\n> ++\tfilter.kind = FILTER_REFS_CHANGES;\n> ++\tfilter.name_patterns = argv;\n> ++\n> ++\tfilter_refs(&array, &filter, FILTER_REFS_CHANGES);\n> ++\n> ++\t/* TODO: This causes a crash. It sets one of the atom_value handlers to\n> ++\t * something invalid, which causes a crash later when we call\n> ++\t * show_ref_array_item. Figure out why this happens and put back the sorting.\n> ++\t *\n> ++\t * sorting = ref_sorting_options(&sorting_options);\n> ++\t * ref_array_sort(sorting, &array); */\n> ++\n> ++\tif (!format.format)\n> ++\t\tformat.format = \"%(refname:lstrip=1)\";\n> ++\n> ++\tif (verify_ref_format(&format))\n> ++\t\tdie(_(\"unable to parse format string\"));\n> ++\n> ++\tsorting = ref_sorting_options(&sorting_options);\n> ++\tref_array_sort(sorting, &array);\n> ++\n> ++\n> ++\tfor (i = 0; i < array.nr; i++) {\n> ++\t\tstruct strbuf output = STRBUF_INIT;\n> ++\t\tstruct strbuf err = STRBUF_INIT;\n> ++\t\tif (format_ref_array_item(array.items[i], &format, &output, &err))\n> ++\t\t\tdie(\"%s\", err.buf);\n> ++\t\tfwrite(output.buf, 1, output.len, stdout);\n> ++\t\tputchar('\\n');\n> ++\n> ++\t\tstrbuf_release(&err);\n> ++\t\tstrbuf_release(&output);\n> ++\t}\n> ++\n> ++\tref_array_clear(&array);\n> ++\tref_sorting_release(sorting);\n> ++\n> ++\treturn 0;\n> ++}\n> ++\n> +struct update_state {\n> +\tint options;\n> +\tconst char* change;\n> @@ builtin/change.c (new)\n> +\tstruct string_list origin;\n> +};\n> +\n> -+static void init_update_state(struct update_state *state)\n> -+{\n> -+\tmemset(state, 0, sizeof(*state));\n> -+\tstate->content = \"HEAD\";\n> -+\tstring_list_init_nodup(&state->replace);\n> -+\tstring_list_init_nodup(&state->origin);\n> ++#define UPDATE_STATE_INIT { \\\n> ++\t.content = \"HEAD\", \\\n> ++\t.replace = STRING_LIST_INIT_NODUP, \\\n> ++\t.origin = STRING_LIST_INIT_NODUP \\\n> +}\n> +\n> +static void clear_update_state(struct update_state *state)\n> @@ builtin/change.c (new)\n> +{\n> +\tstruct commit *commit;\n> +\tif (get_oid_committish(committish, result))\n> -+\t\tdie(_(\"Failed to resolve '%s' as a valid revision.\"), committish);\n> ++\t\tdie(_(\"failed to resolve '%s' as a valid revision.\"), committish);\n> +\tcommit = lookup_commit_reference(the_repository, result);\n> +\tif (!commit)\n> -+\t\tdie(_(\"Could not parse object '%s'.\"), committish);\n> ++\t\tdie(_(\"could not parse object '%s'.\"), committish);\n> +\toidcpy(result, &commit->object.oid);\n> +\treturn 0;\n> +}\n> @@ builtin/change.c (new)\n> +static void resolve_commit_list(const struct string_list *commitsish_list,\n> +\tstruct oid_array* result)\n> +{\n> -+\tint i;\n> -+\tfor (i = 0; i < commitsish_list->nr; i++) {\n> -+\t\tstruct string_list_item *item = &commitsish_list->items[i];\n> ++\tstruct string_list_item *item;\n> ++\n> ++\tfor_each_string_list_item(item, commitsish_list) {\n> +\t\tstruct object_id next;\n> +\t\tresolve_commit(item->string, &next);\n> +\t\toid_array_append(result, &next);\n> @@ builtin/change.c (new)\n> +\tconst struct update_state *state,\n> +\tstruct strbuf *err)\n> +{\n> -+\tstruct metacommit_data metacommit;\n> -+\tstruct change_table chtable;\n> -+\tstruct string_list changes;\n> ++\tstruct metacommit_data metacommit = METACOMMIT_DATA_INIT;\n> ++\tstruct string_list changes = STRING_LIST_INIT_DUP;\n> +\tint ret;\n> -+\tint i;\n> -+\n> -+\tchange_table_init(&chtable);\n> -+\tchange_table_add_all_visible(&chtable, repo);\n> -+\tstring_list_init_dup(&changes);\n> -+\n> -+\tinit_metacommit_data(&metacommit);\n> ++\tstruct string_list_item *item;\n> +\n> +\tget_metacommit_from_command_line(state, &metacommit);\n> +\n> -+\tret = record_metacommit_withresult(repo, &chtable, &metacommit,\n> -+\t\tstate->change, state->options, err, &changes);\n> ++\tret = record_metacommit(\n> ++\t\trepo,\n> ++\t\t&metacommit,\n> ++\t\tstate->change,\n> ++\t\tstate->options,\n> ++\t\terr,\n> ++\t\t&changes);\n> +\n> -+\tfor (i = 0; i < changes.nr; i++) {\n> -+\t\tstruct string_list_item *it = &changes.items[i];\n> ++\tfor_each_string_list_item(item, &changes) {\n> +\n> -+\t\tconst char* name = lstrip_ref_components(it->string, 1);\n> ++\t\tconst char* name = lstrip_ref_components(item->string, 1);\n> +\t\tif (!name)\n> -+\t\t\tdie(_(\"Failed to remove `refs/` from %s\"), it->string);\n> ++\t\t\tdie(_(\"failed to remove `refs/` from %s\"), item->string);\n> +\n> -+\t\tif (it->util)\n> -+\t\t\tfprintf(stdout, N_(\"Updated change %s\\n\"), name);\n> ++\t\tif (item->util)\n> ++\t\t\tfprintf(stdout, _(\"Updated change %s\"), name);\n> +\t\telse\n> -+\t\t\tfprintf(stdout, N_(\"Created change %s\\n\"), name);\n> ++\t\t\tfprintf(stdout, _(\"Created change %s\"), name);\n> ++\t\tputchar('\\n');\n> +\t}\n> +\n> +\tstring_list_clear(&changes, 0);\n> -+\tchange_table_clear(&chtable);\n> +\tclear_metacommit_data(&metacommit);\n> +\n> +\treturn ret;\n> @@ builtin/change.c (new)\n> +static int change_update(int argc, const char **argv, const char* prefix)\n> +{\n> +\tint result;\n> -+\tint force = 0;\n> -+\tint newchange = 0;\n> +\tstruct strbuf err = STRBUF_INIT;\n> -+\tstruct update_state state;\n> ++\tstruct update_state state = UPDATE_STATE_INIT;\n> +\tstruct option options[] = {\n> +\t\t{ OPTION_CALLBACK, 'r', \"replace\", &state, N_(\"commit\"),\n> +\t\t\tN_(\"marks the given commit as being obsolete\"),\n> @@ builtin/change.c (new)\n> +\t\t{ OPTION_CALLBACK, 'o', \"origin\", &state, N_(\"commit\"),\n> +\t\t\tN_(\"marks the given commit as being the origin of this commit\"),\n> +\t\t\t0, update_option_parse_origin },\n> -+\t\tOPT_BOOL('F', \"force\", &force,\n> -+\t\t\tN_(\"overwrite an existing change of the same name\")),\n> ++\n> +\t\tOPT_STRING('c', \"content\", &state.content, N_(\"commit\"),\n> +\t\t\t\t N_(\"identifies the new content commit for the change\")),\n> +\t\tOPT_STRING('g', \"change\", &state.change, N_(\"commit\"),\n> +\t\t\t\t N_(\"name of the change to update\")),\n> -+\t\tOPT_BOOL('n', \"new\", &newchange,\n> -+\t\t\tN_(\"create a new change - do not append to any existing change\")),\n> ++\t\tOPT_SET_INT_F('n', \"new\", &state.options,\n> ++\t\t\t N_(\"create a new change - do not append to any existing change\"),\n> ++\t\t\t UPDATE_OPTION_NOAPPEND, 0),\n> ++\t\tOPT_SET_INT_F('F', \"force\", &state.options,\n> ++\t\t\t N_(\"overwrite an existing change of the same name\"),\n> ++\t\t\t UPDATE_OPTION_FORCE, 0),\n> +\t\tOPT_END()\n> +\t};\n> +\n> -+\tinit_update_state(&state);\n> -+\n> +\targc = parse_options(argc, argv, prefix, options, builtin_update_usage, 0);\n> -+\n> -+\tif (force) state.options |= UPDATE_OPTION_FORCE;\n> -+\tif (newchange) state.options |= UPDATE_OPTION_NOAPPEND;\n> -+\n> +\tresult = perform_update(the_repository, &state, &err);\n> +\n> +\tif (result < 0) {\n> @@ builtin/change.c (new)\n> +\n> +int cmd_change(int argc, const char **argv, const char *prefix)\n> +{\n> ++\tparse_opt_subcommand_fn *fn = NULL;\n> +\t/* No options permitted before subcommand currently */\n> +\tstruct option options[] = {\n> ++\t\tOPT_SUBCOMMAND(\"list\", &fn, change_list),\n> ++\t\tOPT_SUBCOMMAND(\"update\", &fn, change_update),\n> +\t\tOPT_END()\n> +\t};\n> -+\tint result = 1;\n> +\n> +\targc = parse_options(argc, argv, prefix, options, builtin_change_usage,\n> -+\t\tPARSE_OPT_STOP_AT_NON_OPTION);\n> -+\n> -+\tif (argc < 1)\n> -+\t\tusage_with_options(builtin_change_usage, options);\n> -+\telse if (!strcmp(argv[0], \"update\"))\n> -+\t\tresult = change_update(argc, argv, prefix);\n> -+\telse {\n> -+\t\terror(_(\"Unknown subcommand: %s\"), argv[0]);\n> -+\t\tusage_with_options(builtin_change_usage, options);\n> ++\t\tPARSE_OPT_SUBCOMMAND_OPTIONAL);\n> ++\n> ++\tif (!fn) {\n> ++\t\tif (argc) {\n> ++\t\t\terror(_(\"unknown subcommand: `%s'\"), argv[0]);\n> ++\t\t\tusage_with_options(builtin_change_usage, options);\n> ++\t\t}\n> ++\t\tfn = change_list;\n> +\t}\n> +\n> -+\treturn result ? 1 : 0;\n> ++\treturn !!fn(argc, argv, prefix);\n> +}\n> \n> ## git.c ##\n> 9: d087d467e3f ! 8: a0669fa63a1 evolve: add delete command\n> @@ Commit message\n> \n> ## builtin/change.c ##\n> @@\n> + #include \"parse-options.h\"\n> #include \"metacommit.h\"\n> - #include \"change-table.h\"\n> #include \"config.h\"\n> +#include \"refs.h\"\n> \n> static const char * const builtin_change_usage[] = {\n> \tN_(\"git change list [...]\"),\n> --\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> -+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> + \tN_(\"git change update [--force] [--replace ...] \"\n> + \t \"[--origin ...] [--content ]\"),\n> +\tN_(\"git change delete ...\"),\n> \tNULL\n> };\n> \n> -@@ builtin/change.c: static const char * const builtin_list_usage[] = {\n> +@@ builtin/change.c: static const char * const builtin_update_usage[] = {\n> + \tNULL\n> };\n> \n> - static const char * const builtin_update_usage[] = {\n> --\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> -+\tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> ++static const char * const builtin_delete_usage[] = {\n> ++\tN_(\"git change delete ...\"),\n> +\tNULL\n> +};\n> +\n> -+static const char * const builtin_delete_usage[] = {\n> -+\tN_(\"git change delete ...\"),\n> - \tNULL\n> - };\n> -\n> + static int change_list(int argc, const char **argv, const char* prefix)\n> + {\n> + \tstruct option options[] = {\n> @@ builtin/change.c: static int change_update(int argc, const char **argv, const char* prefix)\n> \treturn result;\n> }\n> @@ builtin/change.c: static int change_update(int argc, const char **argv, const ch\n> +\n> int cmd_change(int argc, const char **argv, const char *prefix)\n> {\n> - \t/* No options permitted before subcommand currently */\n> + \tparse_opt_subcommand_fn *fn = NULL;\n> @@ builtin/change.c: int cmd_change(int argc, const char **argv, const char *prefix)\n> - \t\tresult = change_list(argc, argv, prefix);\n> - \telse if (!strcmp(argv[0], \"update\"))\n> - \t\tresult = change_update(argc, argv, prefix);\n> -+\telse if (!strcmp(argv[0], \"delete\"))\n> -+\t\tresult = change_delete(argc, argv, prefix);\n> - \telse {\n> - \t\terror(_(\"Unknown subcommand: %s\"), argv[0]);\n> - \t\tusage_with_options(builtin_change_usage, options);\n> + \tstruct option options[] = {\n> + \t\tOPT_SUBCOMMAND(\"list\", &fn, change_list),\n> + \t\tOPT_SUBCOMMAND(\"update\", &fn, change_update),\n> ++\t\tOPT_SUBCOMMAND(\"delete\", &fn, change_delete),\n> + \t\tOPT_END()\n> + \t};\n> +\n> 10: 811d516e5d2 = 9: e67ff668fff evolve: add documentation for `git change`\n> 8: b83a79beeb4 ! 10: 37042b58cda evolve: add the git change list command\n> @@\n> ## Metadata ##\n> -Author: Stefan Xenos \n> +Author: Chris Poucet \n> \n> ## Commit message ##\n> - evolve: add the git change list command\n> + evolve: add tests for the git-change command\n> \n> - This command lists the ongoing changes from the refs/metas\n> - namespace.\n> -\n> - Signed-off-by: Stefan Xenos \n> + Signed-off-by: Phillip Wood \n> Signed-off-by: Chris Poucet \n> \n> - ## builtin/change.c ##\n> + ## t/t9990-changes.sh (new) ##\n> @@\n> - #include \"config.h\"\n> -\n> - static const char * const builtin_change_usage[] = {\n> -+\tN_(\"git change list [...]\"),\n> - \tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> - \tNULL\n> - };\n> -\n> -+static const char * const builtin_list_usage[] = {\n> -+\tN_(\"git change list [...]\"),\n> -+\tNULL\n> -+};\n> ++#!/bin/sh\n> +\n> - static const char * const builtin_update_usage[] = {\n> - \tN_(\"git change update [--force] [--replace ...] [--origin ...] [--content ]\"),\n> - \tNULL\n> - };\n> -\n> -+static int change_list(int argc, const char **argv, const char* prefix)\n> -+{\n> -+\tstruct option options[] = {\n> -+\t\tOPT_END()\n> -+\t};\n> -+\tstruct ref_filter filter;\n> -+\t/* TODO: See below\n> -+\tstruct ref_sorting *sorting;\n> -+\tstruct string_list sorting_options = STRING_LIST_INIT_DUP; */\n> -+\tstruct ref_format format = REF_FORMAT_INIT;\n> -+\tstruct ref_array array;\n> -+\tint i;\n> ++test_description='git change - low level meta-commit management'\n> +\n> -+\targc = parse_options(argc, argv, prefix, options, builtin_list_usage, 0);\n> ++. ./test-lib.sh\n> +\n> -+\tsetup_ref_filter_porcelain_msg();\n> ++. \"$TEST_DIRECTORY\"/lib-rebase.sh\n> +\n> -+\tmemset(&filter, 0, sizeof(filter));\n> -+\tmemset(&array, 0, sizeof(array));\n> ++test_expect_success 'setup commits and meta-commits' '\n> ++ for c in one two three\n> ++ do\n> ++ test_commit $c &&\n> ++ git change update --content $c >actual 2>err &&\n> ++ echo \"Created change metas/$c\" >expect &&\n> ++ test_cmp expect actual &&\n> ++ test_must_be_empty err &&\n> ++ test_cmp_rev refs/metas/$c $c || return 1\n> ++ done\n> ++'\n> +\n> -+\tfilter.kind = FILTER_REFS_CHANGES;\n> -+\tfilter.name_patterns = argv;\n> ++# Check a meta-commit has the correct parents Call with the object\n> ++# name of the meta-commit followed by pairs of type and parent\n> ++check_meta_commit () {\n> ++ name=$1\n> ++ shift\n> ++ while test $# -gt 0\n> ++ do\n> ++ printf '%s %s\\n' $1 $(git rev-parse --verify $2)\n> ++ shift\n> ++ shift\n> ++ done | sort >expect\n> ++ git cat-file commit $name >metacommit &&\n> ++ # commit body should consist of parent-type\n> ++ types=\"$(sed -n '/^$/ {\n> ++ :loop\n> ++ n\n> ++ s/^parent-type //\n> ++ p\n> ++ b loop\n> ++ }' metacommit)\" &&\n> ++ while read key value\n> ++ do\n> ++ # TODO: don't sort the first parent\n> ++ if test \"$key\" = \"parent\"\n> ++ then\n> ++ type=\"${types%% *}\"\n> ++ test -n \"$type\" || return 1\n> ++ printf '%s %s\\n' $type $value\n> ++ types=\"${types#?}\"\n> ++ types=\"${types# }\"\n> ++ elif test \"$key\" = \"tree\"\n> ++ then\n> ++ test_cmp_rev \"$value\" $EMPTY_TREE || return 1\n> ++ elif test -z \"$key\"\n> ++ then\n> ++ # only parse commit headers\n> ++ break\n> ++ fi\n> ++ done actual-unsorted &&\n> ++ test -z \"$types\" &&\n> ++ sort >actual ++ test_cmp expect actual\n> ++}\n> +\n> -+\tfilter_refs(&array, &filter, FILTER_REFS_CHANGES);\n> ++test_expect_success 'update meta-commits after rebase' '\n> ++ (\n> ++ set_fake_editor &&\n> ++ FAKE_AMEND=edited &&\n> ++ FAKE_LINES=\"reword 1 pick 2 fixup 3\" &&\n> ++ export FAKE_AMEND FAKE_LINES &&\n> ++ git rebase -i --root\n> ++ ) &&\n> +\n> -+\t/* TODO: This causes a crash. It sets one of the atom_value handlers to\n> -+\t * something invalid, which causes a crash later when we call\n> -+\t * show_ref_array_item. Figure out why this happens and put back the sorting.\n> -+\t *\n> -+\t * sorting = ref_sorting_options(&sorting_options);\n> -+\t * ref_array_sort(sorting, &array); */\n> ++ # update meta-commits\n> ++ git change update --replace tags/one --content HEAD~1 >out 2>err &&\n> ++ echo \"Updated change metas/one\" >expect &&\n> ++ test_cmp expect out &&\n> ++ test_must_be_empty err &&\n> ++ git change update --replace tags/two --content HEAD@{2} &&\n> ++ oid=$(git rev-parse --verify metas/two) &&\n> ++ git change update --replace HEAD@{2} --replace tags/three \\\n> ++ --content HEAD &&\n> +\n> -+\tif (!format.format)\n> -+\t\tformat.format = \"%(refname:lstrip=1)\";\n> ++ # check meta-commits\n> ++ check_meta_commit metas/one c HEAD~1 r tags/one &&\n> ++ check_meta_commit $oid c HEAD@{2} r tags/two &&\n> ++ # NB this checks that \"git change update\" uses the meta-commit ($oid)\n> ++ # corresponding to the replaces commit (HEAD@2 above) given on the\n> ++ # commandline.\n> ++ check_meta_commit metas/two c HEAD r $oid r tags/three &&\n> ++ check_meta_commit metas/three c HEAD r $oid r tags/three\n> ++'\n> +\n> -+\tif (verify_ref_format(&format))\n> -+\t\tdie(_(\"unable to parse format string\"));\n> ++reset_meta_commits () {\n> ++ for c in one two three\n> ++ do\n> ++ echo \"update refs/metas/$c refs/tags/$c^0\"\n> ++ done | git update-ref --stdin\n> ++}\n> +\n> -+\tfor (i = 0; i < array.nr; i++) {\n> -+\t\tstruct strbuf output = STRBUF_INIT;\n> -+\t\tstruct strbuf err = STRBUF_INIT;\n> -+\t\tif (format_ref_array_item(array.items[i], &format, &output, &err))\n> -+\t\t\tdie(\"%s\", err.buf);\n> -+\t\tfwrite(output.buf, 1, output.len, stdout);\n> -+\t\tputchar('\\n');\n> ++test_expect_success 'override change name' '\n> ++ # TODO: builtin/change.c expects --change to be the full refname,\n> ++ # ideally it would prepend refs/metas to the string given by the\n> ++ # user.\n> ++ git change update --change refs/metas/another-one --content one &&\n> ++ test_cmp_rev metas/another-one one\n> ++'\n> +\n> -+\t\tstrbuf_release(&err);\n> -+\t\tstrbuf_release(&output);\n> -+\t}\n> ++test_expect_success 'non-fast forward meta-commit update refused' '\n> ++ test_must_fail git change update --change refs/metas/one --content two \\\n> ++ >out 2>err &&\n> ++ echo \"error: non-fast-forward update to ${SQ}refs/metas/one${SQ}\" \\\n> ++ >expect &&\n> ++ test_cmp expect err &&\n> ++ test_must_be_empty out\n> ++'\n> +\n> -+\tref_array_clear(&array);\n> -+\t/* TODO: see above\n> -+\tref_sorting_release(sorting); */\n> ++test_expect_success 'forced non-fast forward update succeeds' '\n> ++ git change update --change refs/metas/one --content two --force \\\n> ++ >out 2>err &&\n> ++ echo \"Updated change metas/one\" >expect &&\n> ++ test_cmp expect out &&\n> ++ test_must_be_empty err\n> ++'\n> +\n> -+\treturn 0;\n> -+}\n> ++test_expect_success 'list changes' '\n> ++ cat >expect <<-\\EOF &&\n> ++metas/another-one\n> ++metas/one\n> ++metas/three\n> ++metas/two\n> ++EOF\n> ++ git change list >actual &&\n> ++ test_cmp expect actual\n> ++'\n> ++\n> ++test_expect_success 'delete change' '\n> ++ git change delete metas/one &&\n> ++ cat >expect <<-\\EOF &&\n> ++metas/another-one\n> ++metas/three\n> ++metas/two\n> ++EOF\n> ++ git change list >actual &&\n> ++ test_cmp expect actual\n> ++'\n> +\n> - struct update_state {\n> - \tint options;\n> - \tconst char* change;\n> -@@ builtin/change.c: int cmd_change(int argc, const char **argv, const char *prefix)\n> -\n> - \tif (argc < 1)\n> - \t\tusage_with_options(builtin_change_usage, options);\n> -+\telse if (!strcmp(argv[0], \"list\"))\n> -+\t\tresult = change_list(argc, argv, prefix);\n> - \telse if (!strcmp(argv[0], \"update\"))\n> - \t\tresult = change_update(argc, argv, prefix);\n> - \telse {\n> ++test_done\n> \n\n"},{"id":"464528","messageId":"35d65b75-c5c4-132a-bbd5-49d3c012c69f@github.com","threadId":"58504","inReplyTo":"a5eb93254191b7ae9c17ce52e056955c669ea007.1664981958.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 01/10] technical doc: add a design doc for the evolve command","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-10-10T19:35:34Z","receivedAt":"2022-10-10T19:35:51Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Stefan Xenos via GitGitGadget wrote:\n> From: Stefan Xenos \n> \n> This document describes what a change graph for\n> git would look like, the behavior of the evolve command,\n> and the changes planned for other commands.\n> \n> It was originally proposed in 2018, see\n> https://public-inbox.org/git/20181115005546.212538-1-sxenos@google.com/\n> \n> Signed-off-by: Stefan Xenos \n> Signed-off-by: Chris Poucet \n> ---\n> Documentation/technical/evolve.txt | 1070 ++++++++++++++++++++++++++++\n> 1 file changed, 1070 insertions(+)\n> create mode 100644 Documentation/technical/evolve.txt\n> \n> diff --git a/Documentation/technical/evolve.txt b/Documentation/technical/evolve.txt\n> new file mode 100644\n> index 00000000000..2051ea77b8a\n> --- /dev/null\n> +++ b/Documentation/technical/evolve.txt\n> @@ -0,0 +1,1070 @@\n> +Evolve\n> +======\n> +\n> +Objective\n> +=========\n> +Create an \"evolve\" command to help users craft a high quality commit history.\n> +Users can improve commits one at a time and in any order, then run git evolve to\n> +rewrite their recent history to ensure everything is up-to-date. We track\n> +amendments to a commit over time in a change graph. Users can share their\n> +progress with others by exchanging their change graphs using the standard push,\n> +fetch, and format-patch commands.\n> +\n> +Status\n> +======\n> +This proposal has not been implemented yet.\n> +\n> +Background\n> +==========\n> +Imagine you have three sequential changes up for review and you receive feedback\n> +that requires editing all three changes. We'll define the word \"change\"\n> +formally later, but for the moment let's say that a change is a work-in-progress\n> +whose final version will be submitted as a commit in the future.\n> +\n> +While you're editing one change, more feedback arrives on one of the others.\n> +What do you do?\n\nFor the sake of providing additional perspectives, I can say that I'd:\n\n$ git stash\n# Make changes based on new feedback\n$ git add . & git commit --fixup \n$ git stash pop\n# Continue working\n\nor something along those lines.\n\n> +\n> +The evolve command is a convenient way to work with chains of commits that are\n> +under review. Whenever you rebase or amend a commit, the repository remembers\n> +that the old commit is obsolete and has been replaced by the new one. Then, at\n> +some point in the future, you can run \"git evolve\" and the correct sequence of\n> +rebases will occur in the correct order such that no commit has an obsolete\n> +parent.\n> +\n> +Part of making the \"evolve\" command work involves tracking the edits to a commit\n> +over time, which is why we need an change graph. However, the change\n> +graph will also bring other benefits:\n> +\n> +- Users can view the history of a change directly (the sequence of amends and\n> + rebases it has undergone, orthogonal to the history of the branch it is on).\n> +- It will be possible to quickly locate and list all the changes the user\n> + currently has in progress.\n> +- It can be used as part of other high-level commands that combine or split\n> + changes.\n> +- It can be used to decorate commits (in git log, gitk, etc) that are either\n> + obsolete or are the tip of a work in progress.\n> +- By pushing and pulling the change graph, users can collaborate more\n> + easily on changes-in-progress. This is better than pushing and pulling the\n> + commits themselves since the change graph can be used to locate a more\n> + specific merge base, allowing for better merges between different versions of\n> + the same change.\n> +- It could be used to correctly rebase local changes and other local branches\n> + after running git-filter-branch.\n> +- It can replace the change-id footer used by gerrit.\n\nWhile the first part of the \"Background\" section is good (talking about a\nuser scenario), I don't think this description of 'git evolve' belongs here.\nI'd expect the background to talk about what you wish Git could do (but it\nhard/impossible to do now), or what prompted you to (re-)submit this\nproposal. By prescribing the 'git evolve' command as the solution up-front,\nthis doc makes it difficult to interpret the underlying \"why\" of the\nproposal.\n\n> +\n> +Goals\n> +-----\n> +Legend: Goals marked with P0 are required. Goals marked with Pn should be\n> +attempted unless they interfere with goals marked with Pn-1.\n> +\n> +P0. All commands that modify commits (such as the normal commit --amend or\n> + rebase command) should mark the old commit as being obsolete and replaced by\n> + the new one. No additional commands should be required to keep the\n> + change graph up-to-date.\n\nYou elaborate on it more later, but this design is proposing that this\n\"change\" workflow becomes the default for everyone, with a config option\n('core.enableChanges') to opt out. This would be massively disruptive (and\nconfusing) to the huge swath of users that have no desire to use this\nparticular workflow.\n\n> +P0. Any commit that may be involved in a future evolve command should not be\n> + garbage collected. Specifically:\n> + - Commits that obsolete another should not be garbage collected until\n> + user-specified conditions have occurred and the change has expired from\n> + the reflog. User specified conditions for removing changes include:\n> + - The user explicitly deleted the change.\n> + - The change was merged into a specific branch.\n> + - Commits that have been obsoleted by another should not be garbage\n> + collected if any of their replacements are still being retained.\n\nIf the creation of these linkages is passive, but requires active user\nintervention to clean up, this requirement could result in creating an\nenormous amount of cruft in repositories. I might rebase a branch 10+ times\nbetween pushes to make little tweaks to phrasing in commit messages, or fix\ntypos, etc. It sounds like I'd be pushing an order of magnitude more objects\nthan I am now, let alone the fact that they wouldn't be GC'd automatically.\n\n> +P0. A commit can be obsoleted by more than one replacement (called divergence).\n> +P0. Users must be able to resolve divergence (convergence).\n> +P1. Users should be able to share chains of obsolete changes in order to\n> + collaborate on WIP changes.\n> +P2. Such sharing should be at the user’s option. That is, it should be possible\n> + to directly share a change without also sharing the file states or commit\n> + comments from the obsolete changes that led up to it, and the choice not to\n> + share those commits should not require changing any commit hashes.\n> +P2. It should be possible to discard part or all of the change graph\n> + without discarding the commits themselves that are already present in\n> + branches and the reflog.\n> +P2. Provide sufficient information to replace gerrit's Change-Id footers.\n\nA general comment on the \"Goals\" section (similar to the one on \"Background)\n- all of the listed goals are written assuming you've already decided on the\nsolution, rather than elaborating on the problems you're trying to solve\n(e.g., \"I want to create a persistent linkage between iterations of a commit\nand be able to query that linkage\"). Talking about the features you want\n(and why) will not only make it much easier to understand the proposal, it\ncould inspire other contributors to suggest solutions you may not have\nconsidered.\n\n> +\n> +Similar technologies\n> +--------------------\n> +There are some other technologies that address the same end-user problem.\n> +\n> +Rebase -i can be used to solve the same problem, but users can't easily switch\n> +tasks midway through an interactive rebase or have more than one interactive\n> +rebase going on at the same time. It can't handle the case where you have\n> +multiple changes sharing the same parent when that parent needs to be rebased\n> +and won't let you collaborate with others on resolving a complicated interactive\n> +rebase. You can think of rebase -i as a top-down approach and the evolve command\n> +as the bottom-up approach to the same problem.\n\nI think it's worth considering whether 'rebase' can be updated to handle\nthese cases (since it might simplify and/or pare down your proposed design).\n\n1. Can't easily switch tasks midway through an interactive rebase\n - I could imagine us introducing a 'git rebase pause' that does this,\n although it would require changes to how rebases are tracked\n internally.\n2. Can't have more than one interactive rebase going on at the same time\n - Do you mean nested rebases, or just separate ones? I think both of them\n could be possible (with the changes to rebase tracking in #1), but\n nested ones might be tough to mentally keep track of.\n3. Can't handle multiple changes sharing the same parent when the parent\n needs to be rebased\n - Since the introduction of '--update-refs' [1], this is technically\n possible (although it needs a UI for the use case you mentioned).\n4. Won't let you collaborate with others on resolving a complicated\n interactive rebase\n - This is an interesting one, since it requires being able to push a\n mid-merge state. However, if you're planning on solving that for 'git\n evolve', a similar solution could probably be used for 'rebase'.\n Pushing a whole rebase script, though, would be more complicated.\n\nThe \"top-down\"/\"bottom-up\" analogy is a bit lost on me, I'm afraid. Could\nyou clarify what you mean by that?\n\n[1] https://lore.kernel.org/git/pull.1247.v5.git.1658255624.gitgitgadget@gmail.com/\n\n> +Overview\n> +========\n> +We introduce the notion of “meta-commits” which describe how one commit was\n> +created from other commits. A branch of meta-commits is known as a change.\n> +Changes are created and updated automatically whenever a user runs a command\n> +that creates a commit. They are used for locating obsolete commits, providing a\n> +list of a user’s unsubmitted work in progress, and providing a stable name for\n> +each unsubmitted change.\n\nThe term \"change\" is overly generic; I and others already use it as a\ngeneral term for anything from \"part of a patch\" to \"an entire patch\nseries\". Maybe \"evolution\" (in keeping with 'git evolve' being the command\nthat updates all of the meta-commit branches)? Or \"iteration\" (although that\nmight also be overloaded with how we colloquially refer to [PATCH vN]\nsubmissions)?\n\nAlso, I'm not sure what you mean by \"unsubmitted\". Un-pushed? \n\n> +\n> +Users can exchange edit histories by pushing and fetching changes.\n> +\n> +New commands will be introduced for manipulating changes and resolving\n> +divergence between them. Existing commands that create commits will be updated\n> +to modify the meta-commit graph and create changes where necessary.\n> +\n> +Example usage\n> +-------------\n\nnit: please include information about where HEAD starts (I think you start\non 'metas/some_change_already_merged_upstream'?).\n\n> +# First create three dependent changes\n> +$ echo foo>bar.txt && git add .\n> +$ git commit -m \"This is a test\"\n> +created change metas/this_is_a_test\n> +$ echo foo2>bar2.txt && git add .\n> +$ git commit -m \"This is also a test\"\n> +created change metas/this_is_also_a_test\n> +$ echo foo3>bar3.txt && git add .\n> +$ git commit -m \"More testing\"\n> +created change metas/more_testing\n> +\n> +# List all our changes in progress\n> +$ git change list\n> +metas/this_is_a_test\n> +metas/this_is_also_a_test\n> +* metas/more_testing\n> +metas/some_change_already_merged_upstream\n\nIf this is a list of all of the unmerged commits/changes you have, it's\ngoing to be exceptionally long for people with multiple local branches.\nUnless the results of 'git change list' are scoped to those reachable from\nthe meta-commit pointing at HEAD?\n\n> +\n> +# Now modify the earliest change, using its stable name\n> +$ git reset --hard metas/this_is_a_test\n> +$ echo morefoo>>bar.txt && git add . && git commit --amend --no-edit\n> +\n> +# Use git-evolve to fix up any dependent changes\n> +$ git evolve\n> +rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n> +rebasing metas/more_testing onto metas/this_is_also_a_test\n> +Done\n> +\nWhere is HEAD at this point? Still 'metas/this_is_a_test'? \n\n> +# Use git-obslog to view the history of the this_is_a_test change\n> +$ git log --obslog\n> +93f110 metas/this_is_a_test@{0} commit (amend): This is a test\n> +930219 metas/this_is_a_test@{1} commit: This is a test\n> +\n> +# Now create an unrelated change\n> +$ git reset --hard origin/master\n> +$ echo newchange>unrelated.txt && git add .\n> +$ git commit -m \"Unrelated change\"\n> +created change metas/unrelated_change\n> +\n> +# Fetch the latest code from origin/master and use git-evolve\n> +# to rebase all dependent changes.\n> +$ git fetch origin master\n> +$ git evolve origin/master\n> +deleting metas/some_change_already_merged_upstream\n> +rebasing metas/this_is_a_test onto origin/master\n> +rebasing metas/this_is_also_a_test onto metas/this_is_a_test\n> +rebasing metas/more_testing onto metas/this_is_also_a_test\n> +rebasing metas/unrelated_change onto origin/master\n> +Conflict detected! Resolve it and then use git evolve --continue to resume.\n> +\n> +# Sort out the conflict\n> +$ git mergetool\n> +$ git evolve origin/master\n> +Done\n\nYou ran 'git evolve origin/master', but the message said to use 'git evolve\n--continue'. Is that a typo, or do they actually do something different\nafter resolving a conflict?\n\n> +\n> +# Share the full history of edits for the this_is_a_test change\n> +# with a review server\n> +$ git push origin metas/this_is_a_test:refs/for/master\n> +# Share the lastest commit for “Unrelated change”, without history\n> +$ git push origin HEAD:refs/for/master\n\nIt would be nice to also show here how a change is \"finalized\" (unlinked\nfrom previous iterations, allowing them to be garbage collected). I think\nyou add this detail later in the doc, but it'd be nice to have up-front to\nshow the full end-to-end workflow.\n\n> +\n> +Detailed design\n> +===============\n> +Obsolescence information is stored as a graph of meta-commits. A meta-commit is\n> +a specially-formatted merge commit that describes how one commit was created\n> +from others.\n> +\n> +Meta-commits look like this:\n> +\n> +$ git cat-file -p \n> +tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904\n> +parent aa7ce55545bf2c14bef48db91af1a74e2347539a\n> +parent d64309ee51d0af12723b6cb027fc9f195b15a5e9\n> +parent 7e1bbcd3a0fa854a7a9eac9bf1eea6465de98136\n> +author Stefan Xenos 1540841596 -0700\n> +committer Stefan Xenos 1540841596 -0700\n> +parent-type c r o\n> +\n> +This says “commit aa7ce555 makes commit d64309ee obsolete. It was created by\n> +cherry-picking commit 7e1bbcd3”.\n> +\n> +The tree for meta-commits is always the empty tree, but future versions of git\n> +may attach other trees here. For forward-compatibility fsck should ignore such\n> +trees if found on future repository versions. This will allow future versions of\n> +git to add metadata to the meta-commit tree without breaking forwards\n> +compatibility.\n> +\n> +The commit comment for a meta-commit is an auto-generated user-readable string\n> +describing the command that produced the meta commit. These strings are shown\n> +to the user when they view the obslog.\n> +\n\nThe rest of this section provides (very detailed) descriptions of what you\nwant to do with each command. I appreciate the detail, but I wanted to focus\nthe review/discussion on the high-level approach before diving into\nimplementation details.\n\nMy overall take on this proposal is that, while (I think) I understand how\nyour proposed solution works, I'm unsure as to whether it's the best way to\nsolve the problems you're hoping to solve. Reworking the \"Background\" and\n\"Goals\" sections to focus more on pain points/what you want to get out of\nthe new workflow would help substantially with figuring that out, so please\nconsider updating those in your next re-roll.\n\nAs far as the described approach, I have some concerns with both the\nuser-facing side of things and the internal architecture. In terms of\nuser impact:\n\n- Regardless of the decided approach, I don't think this workflow should be\n made the default for all users; it's just too different from typical\n workflows that Git users follow.\n- Even if users do \"opt-in\" to using this workflow, I think it'd be valuable\n to have an \"iterate only when I ask you to\" option for users that don't\n want every single fixup be saved as a persistent \"version\" of a change.\n- The term \"change\" is already a loose/overloaded concept in the context of\n Git, I'd strongly suggest picking something else. Relatedly, a\n \"terminology\" subsection (for things like \"meta-commit\", \"evolve\", etc.)\n under the \"Overview\" section would help a lot with reading through this.\n\nThe backend concerns are mostly related to massively increasing the number\nof pushed objects and the reachability of those objects. An \"iterate only\nwhen I ask you to\" approach would help with this, but I don't know whether\nthat fits your needs or not.\n\nThanks!\n- Victoria\n"},{"id":"464595","messageId":"3384d8ab-ddbb-6e57-1663-d039fc99e0a6@dunelm.org.uk","threadId":"58504","inReplyTo":"35d65b75-c5c4-132a-bbd5-49d3c012c69f@github.com","subject":"Re: [PATCH v2 01/10] technical doc: add a design doc for the evolve command","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-11T08:59:44Z","receivedAt":"2022-10-11T08:59:53Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 10/10/2022 20:35, Victoria Dye wrote:\n> Stefan Xenos via GitGitGadget wrote:\n>> From: Stefan Xenos \n>>\n>> This document describes what a change graph for\n>> git would look like, the behavior of the evolve command,\n>> and the changes planned for other commands.\n>>\n>> It was originally proposed in 2018, see\n>> https://public-inbox.org/git/20181115005546.212538-1-sxenos@google.com/\n>>\n>> Signed-off-by: Stefan Xenos \n>> Signed-off-by: Chris Poucet \n>> ---\n>> Documentation/technical/evolve.txt | 1070 ++++++++++++++++++++++++++++\n>> 1 file changed, 1070 insertions(+)\n>> create mode 100644 Documentation/technical/evolve.txt\n>>\n>> diff --git a/Documentation/technical/evolve.txt b/Documentation/technical/evolve.txt\n>> new file mode 100644\n>> index 00000000000..2051ea77b8a\n>> --- /dev/null\n>> +++ b/Documentation/technical/evolve.txt\n\n...\n\n>> +P0. Any commit that may be involved in a future evolve command should not be\n>> + garbage collected. Specifically:\n>> + - Commits that obsolete another should not be garbage collected until\n>> + user-specified conditions have occurred and the change has expired from\n>> + the reflog. User specified conditions for removing changes include:\n>> + - The user explicitly deleted the change.\n>> + - The change was merged into a specific branch.\n>> + - Commits that have been obsoleted by another should not be garbage\n>> + collected if any of their replacements are still being retained.\n> \n> If the creation of these linkages is passive, but requires active user\n> intervention to clean up, this requirement could result in creating an\n> enormous amount of cruft in repositories. I might rebase a branch 10+ times\n> between pushes to make little tweaks to phrasing in commit messages, or fix\n> typos, etc. It sounds like I'd be pushing an order of magnitude more objects\n> than I am now, let alone the fact that they wouldn't be GC'd automatically.\n\nThat's an interesting point. When we push we only really need to push a \nmap of \"commits we pulled\" to \"commits we're pushing\", we don't need to \nsend all the intermediate changes. That would also help to address \nGlen's review club concerns about accidentally pushing secret information.\n\nOne of the things which I hope comes out of having all the intermediate \nchanges tracked locally is a way to view the history of a particular \ncommit. If I make a mistake when rebasing and don't notice it for a \nwhile it would be really helpful to be able to view the history and \nfigure out which change introduced the mistake (You can do something \nsimilar with \"git rev-list -g $branch | git log -p --stdin \n^${branch}@{upstream}\" but you have to wade through all the commits on \n$branch).\n\n...\n\n>> +Similar technologies\n>> +--------------------\n>> +There are some other technologies that address the same end-user problem.\n>> +\n>> +Rebase -i can be used to solve the same problem, but users can't easily switch\n>> +tasks midway through an interactive rebase or have more than one interactive\n>> +rebase going on at the same time. It can't handle the case where you have\n>> +multiple changes sharing the same parent when that parent needs to be rebased\n>> +and won't let you collaborate with others on resolving a complicated interactive\n>> +rebase. You can think of rebase -i as a top-down approach and the evolve command\n>> +as the bottom-up approach to the same problem.\n> \n> I think it's worth considering whether 'rebase' can be updated to handle\n> these cases (since it might simplify and/or pare down your proposed design).\n> \n> 1. Can't easily switch tasks midway through an interactive rebase\n> - I could imagine us introducing a 'git rebase pause' that does this,\n> although it would require changes to how rebases are tracked\n> internally.\n\nI'm not sure how much of a problem this is in practice as one can use \n\"git worktree add\" to work on a different branch or is the idea to be \nable to start several rebases on the same branch? - That sounds like a \nrecipe for conflicts that cannot be resolved automatically unless the \nuser is very disciplined.\n\n> 2. Can't have more than one interactive rebase going on at the same time\n> - Do you mean nested rebases, or just separate ones? I think both of them\n> could be possible (with the changes to rebase tracking in #1), but\n> nested ones might be tough to mentally keep track of.\n> 3. Can't handle multiple changes sharing the same parent when the parent\n> needs to be rebased\n> - Since the introduction of '--update-refs' [1], this is technically\n> possible (although it needs a UI for the use case you mentioned).\n\n'--update-refs' is more limited though I think. With evolve if I have\n\n D (topic-2)\n /\n\tA - B - C (topic-1)\n \\\n E (topic-3)\n\nthen if I checkout topic-1 and amend one of the commits I can run \"git \nevolve\" to automatically rebase topic-2 & topic-3. One cannot do that \nwith \"rebase --update-refs\". We could extend rebase (or have a new \ncommand) so that users can say \"amend commit X and rebase all the \nbranches that contain it\".\n\n> 4. Won't let you collaborate with others on resolving a complicated\n> interactive rebase\n> - This is an interesting one, since it requires being able to push a\n> mid-merge state. However, if you're planning on solving that for 'git\n> evolve', a similar solution could probably be used for 'rebase'.\n> Pushing a whole rebase script, though, would be more complicated.\n> \n> The \"top-down\"/\"bottom-up\" analogy is a bit lost on me, I'm afraid. Could\n> you clarify what you mean by that?\n\nI was confused by that as well\n\n> [1] https://lore.kernel.org/git/pull.1247.v5.git.1658255624.gitgitgadget@gmail.com/\n\nBest Wishes\n\nPhillip\n"},{"id":"464622","messageId":"0c7a87bc-f2b7-4c9e-cfe5-b4ba6b33fee7@github.com","threadId":"58504","inReplyTo":"3384d8ab-ddbb-6e57-1663-d039fc99e0a6@dunelm.org.uk","subject":"Re: [PATCH v2 01/10] technical doc: add a design doc for the evolve command","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-10-11T16:59:02Z","receivedAt":"2022-10-11T16:59:07Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Phillip Wood wrote:\n> On 10/10/2022 20:35, Victoria Dye wrote:\n>> Stefan Xenos via GitGitGadget wrote:\n>> 3. Can't handle multiple changes sharing the same parent when the parent\n>>     needs to be rebased\n>>     - Since the introduction of '--update-refs' [1], this is technically\n>>       possible (although it needs a UI for the use case you mentioned).\n> \n> '--update-refs' is more limited though I think. With evolve if I have\n> \n>                   D (topic-2)\n>                  /\n>     A - B - C (topic-1)\n>                  \\\n>                   E (topic-3)\n> \n> then if I checkout topic-1 and amend one of the commits I can run \"git\n> evolve\" to automatically rebase topic-2 & topic-3. One cannot do that with\n> \"rebase --update-refs\". We could extend rebase (or have a new command) so\n> that users can say \"amend commit X and rebase all the branches that contain\n> it\".\n\nSorry, let me clarify what I mean. The 'update-ref' command in a\n'rebase-todo' script (not the '--update-refs' option) can be used to create\na rebase script that does what's described in your example:\n\n label onto # A\n\n reset onto\n pick 1342ab B\n fixup 8a7f3e fixup! B\n label branch-point-1\n\n pick 90d7fc C\n label topic-1\n update-ref refs/heads/topic-1\n\n reset branch-point-1\n pick 42f92b D\n label topic-2\n update-ref refs/heads/topic-2\n\n reset branch-point-1\n pick 06d8ec E\n label topic-3\n update-ref refs/heads/topic-3\n\nSo, while it'd need a less manual UI (e.g., a 'rebase --evolve' option) to\ngenerate that script, the 'update-ref' command makes this functionality\npossible in a rebase.\n"},{"id":"464745","messageId":"37f56330-36c5-d333-0863-75e369d43d8d@dunelm.org.uk","threadId":"58504","inReplyTo":"0c7a87bc-f2b7-4c9e-cfe5-b4ba6b33fee7@github.com","subject":"Re: [PATCH v2 01/10] technical doc: add a design doc for the evolve command","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-10-12T19:19:42Z","receivedAt":"2022-10-12T19:19:50Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Victoria\n\nOn 11/10/2022 17:59, Victoria Dye wrote:\n> Phillip Wood wrote:\n>> On 10/10/2022 20:35, Victoria Dye wrote:\n>>> Stefan Xenos via GitGitGadget wrote:\n>>> 3. Can't handle multiple changes sharing the same parent when the parent\n>>>     needs to be rebased\n>>>     - Since the introduction of '--update-refs' [1], this is technically\n>>>       possible (although it needs a UI for the use case you mentioned).\n>>\n>> '--update-refs' is more limited though I think. With evolve if I have\n>>\n>>                   D (topic-2)\n>>                  /\n>>     A - B - C (topic-1)\n>>                  \\\n>>                   E (topic-3)\n>>\n>> then if I checkout topic-1 and amend one of the commits I can run \"git\n>> evolve\" to automatically rebase topic-2 & topic-3. One cannot do that with\n>> \"rebase --update-refs\". We could extend rebase (or have a new command) so\n>> that users can say \"amend commit X and rebase all the branches that contain\n>> it\".\n> \n> Sorry, let me clarify what I mean. The 'update-ref' command in a\n> 'rebase-todo' script (not the '--update-refs' option) can be used to create\n> a rebase script that does what's described in your example:\n> \n> label onto # A\n> \n> reset onto\n> pick 1342ab B\n> fixup 8a7f3e fixup! B\n> label branch-point-1\n> \n> pick 90d7fc C\n> label topic-1\n> update-ref refs/heads/topic-1\n> \n> reset branch-point-1\n> pick 42f92b D\n> label topic-2\n> update-ref refs/heads/topic-2\n> \n> reset branch-point-1\n> pick 06d8ec E\n> label topic-3\n> update-ref refs/heads/topic-3\n> \n> So, while it'd need a less manual UI (e.g., a 'rebase --evolve' option) to\n> generate that script, the 'update-ref' command makes this functionality\n> possible in a rebase.\n\nAh, I'd misunderstood what you meant, that makes sense - thanks for \nclarifying.\n\nBest Wishes\n\nPhillip\n"}]}