{"thread":{"id":"48889","subject":"[PATCH 00/16] Consolidate reachability logic","startedAt":"2018-06-19T20:25:07Z","lastAt":"2020-11-18T17:47:54Z","messageCount":118,"participants":["Derrick Stolee via GitGitGadget","Ramsay Jones","Jeff King","Stefan Beller","Eric Sunshine","Derrick Stolee","Jonathan Tan","Johannes Schindelin","Duy Nguyen","Junio C Hamano","Jonathan Nieder","René Scharfe","Ævar Arnfjörð Bjarmason"],"isPatch":true,"patchVersion":1,"patchTotal":16},"messages":[{"id":"352611","messageId":"a8df7674dff42c2bba2eca2e803b6fd43e7e4606.1531746010.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 04/16] upload-pack: make reachable() more generic","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-06-19T20:25:07Z","receivedAt":"2018-06-19T20:25:07Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nIn anticipation of moving the reachable() method to commit-reach.c,\nmodify the prototype to be more generic to flags known outside of\nupload-pack.c. Also rename 'want' to 'from' to make the statement\nmore clear outside of the context of haves/wants negotiation.\n\nSigned-off-by: Derrick Stolee \n---\n upload-pack.c | 16 ++++++++--------\n 1 file changed, 8 insertions(+), 8 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 4ca052d0b..0ed470713 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -336,17 +336,17 @@ static int got_oid(const char *hex, struct object_id *oid)\n \treturn 0;\n }\n \n-static int reachable(struct commit *want)\n+static int reachable(struct commit *from, int with_flag, int assign_flag)\n {\n \tstruct prio_queue work = { compare_commits_by_commit_date };\n \n-\tprio_queue_put(&work, want);\n+\tprio_queue_put(&work, from);\n \twhile (work.nr) {\n \t\tstruct commit_list *list;\n \t\tstruct commit *commit = prio_queue_get(&work);\n \n-\t\tif (commit->object.flags & THEY_HAVE) {\n-\t\t\twant->object.flags |= COMMON_KNOWN;\n+\t\tif (commit->object.flags & with_flag) {\n+\t\t\tfrom->object.flags |= assign_flag;\n \t\t\tbreak;\n \t\t}\n \t\tif (!commit->object.parsed)\n@@ -362,10 +362,10 @@ static int reachable(struct commit *want)\n \t\t\t\tprio_queue_put(&work, parent);\n \t\t}\n \t}\n-\twant->object.flags |= REACHABLE;\n-\tclear_commit_marks(want, REACHABLE);\n+\tfrom->object.flags |= REACHABLE;\n+\tclear_commit_marks(from, REACHABLE);\n \tclear_prio_queue(&work);\n-\treturn (want->object.flags & COMMON_KNOWN);\n+\treturn (from->object.flags & assign_flag);\n }\n \n static int ok_to_give_up(void)\n@@ -390,7 +390,7 @@ static int ok_to_give_up(void)\n \t\t\twant_obj.objects[i].item->flags |= COMMON_KNOWN;\n \t\t\tcontinue;\n \t\t}\n-\t\tif (!reachable((struct commit *)want))\n+\t\tif (!reachable((struct commit *)want, THEY_HAVE, COMMON_KNOWN))\n \t\t\treturn 0;\n \t}\n \treturn 1;\n-- \ngitgitgadget\n\n"},{"id":"352613","messageId":"01daf73121a97abc128c52b2705eccaff6c7c92b.1531746010.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 05/16] upload-pack: refactor ok_to_give_up()","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-06-19T20:35:36Z","receivedAt":"2018-06-19T20:35:36Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nIn anticipation of consolidating all commit reachability algorithms,\nrefactor ok_to_give_up() in order to allow splitting its logic into\nan external method.\n\nSigned-off-by: Derrick Stolee \n---\n upload-pack.c | 33 ++++++++++++++++++++++-----------\n 1 file changed, 22 insertions(+), 11 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 0ed470713..e7ad7f24b 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -368,34 +368,45 @@ static int reachable(struct commit *from, int with_flag, int assign_flag)\n \treturn (from->object.flags & assign_flag);\n }\n \n-static int ok_to_give_up(void)\n+/*\n+ * Determine if every commit in 'from' can reach at least one commit\n+ * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n+ * as a marker for commits that are already visited.\n+ */\n+static int can_all_from_reach_with_flag(struct object_array *from,\n+\t\t\t\t\tint with_flag, int assign_flag)\n {\n \tint i;\n \n-\tif (!have_obj.nr)\n-\t\treturn 0;\n-\n-\tfor (i = 0; i < want_obj.nr; i++) {\n-\t\tstruct object *want = want_obj.objects[i].item;\n+\tfor (i = 0; i < from->nr; i++) {\n+\t\tstruct object *from_one = from->objects[i].item;\n \n-\t\tif (want->flags & COMMON_KNOWN)\n+\t\tif (from_one->flags & assign_flag)\n \t\t\tcontinue;\n-\t\twant = deref_tag(the_repository, want, \"a want line\", 0);\n-\t\tif (!want || want->type != OBJ_COMMIT) {\n+\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n+\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n \t\t\t/* no way to tell if this is reachable by\n \t\t\t * looking at the ancestry chain alone, so\n \t\t\t * leave a note to ourselves not to worry about\n \t\t\t * this object anymore.\n \t\t\t */\n-\t\t\twant_obj.objects[i].item->flags |= COMMON_KNOWN;\n+\t\t\tfrom->objects[i].item->flags |= assign_flag;\n \t\t\tcontinue;\n \t\t}\n-\t\tif (!reachable((struct commit *)want, THEY_HAVE, COMMON_KNOWN))\n+\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag))\n \t\t\treturn 0;\n \t}\n \treturn 1;\n }\n \n+static int ok_to_give_up(void)\n+{\n+\tif (!have_obj.nr)\n+\t\treturn 0;\n+\n+\treturn can_all_from_reach_with_flag(&want_obj, THEY_HAVE, COMMON_KNOWN);\n+}\n+\n static int get_common_commits(void)\n {\n \tstruct object_id oid;\n-- \ngitgitgadget\n\n"},{"id":"352609","messageId":"01bef3d654845236e9115ebc33d80673b7b4a5a2.1531746009.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 01/16] commit-reach: move walk methods from commit.c","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-06-25T17:16:47Z","receivedAt":"2018-06-25T17:16:47Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nSigned-off-by: Derrick Stolee \n---\n Makefile | 1 +\n commit-reach.c | 359 +++++++++++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 41 ++++++\n commit.c | 358 ------------------------------------------------\n 4 files changed, 401 insertions(+), 358 deletions(-)\n create mode 100644 commit-reach.c\n create mode 100644 commit-reach.h\n\ndiff --git a/Makefile b/Makefile\nindex bb8bd6720..59781f4bc 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -829,6 +829,7 @@ LIB_OBJS += column.o\n LIB_OBJS += combine-diff.o\n LIB_OBJS += commit.o\n LIB_OBJS += commit-graph.o\n+LIB_OBJS += commit-reach.o\n LIB_OBJS += compat/obstack.o\n LIB_OBJS += compat/terminal.o\n LIB_OBJS += config.o\ndiff --git a/commit-reach.c b/commit-reach.c\nnew file mode 100644\nindex 000000000..f2e2f7461\n--- /dev/null\n+++ b/commit-reach.c\n@@ -0,0 +1,359 @@\n+#include \"cache.h\"\n+#include \"prio-queue.h\"\n+#include \"commit-reach.h\"\n+\n+/* Remember to update object flag allocation in object.h */\n+#define PARENT1\t\t(1u<<16)\n+#define PARENT2\t\t(1u<<17)\n+#define STALE\t\t(1u<<18)\n+#define RESULT\t\t(1u<<19)\n+\n+static const unsigned all_flags = (PARENT1 | PARENT2 | STALE | RESULT);\n+\n+static int queue_has_nonstale(struct prio_queue *queue)\n+{\n+\tint i;\n+\tfor (i = 0; i < queue->nr; i++) {\n+\t\tstruct commit *commit = queue->array[i].data;\n+\t\tif (!(commit->object.flags & STALE))\n+\t\t\treturn 1;\n+\t}\n+\treturn 0;\n+}\n+\n+/* all input commits in one and twos[] must have been parsed! */\n+static struct commit_list *paint_down_to_common(struct commit *one, int n,\n+\t\t\t\t\t\tstruct commit **twos,\n+\t\t\t\t\t\tint min_generation)\n+{\n+\tstruct prio_queue queue = { compare_commits_by_gen_then_commit_date };\n+\tstruct commit_list *result = NULL;\n+\tint i;\n+\tuint32_t last_gen = GENERATION_NUMBER_INFINITY;\n+\n+\tone->object.flags |= PARENT1;\n+\tif (!n) {\n+\t\tcommit_list_append(one, &result);\n+\t\treturn result;\n+\t}\n+\tprio_queue_put(&queue, one);\n+\n+\tfor (i = 0; i < n; i++) {\n+\t\ttwos[i]->object.flags |= PARENT2;\n+\t\tprio_queue_put(&queue, twos[i]);\n+\t}\n+\n+\twhile (queue_has_nonstale(&queue)) {\n+\t\tstruct commit *commit = prio_queue_get(&queue);\n+\t\tstruct commit_list *parents;\n+\t\tint flags;\n+\n+\t\tif (commit->generation > last_gen)\n+\t\t\tBUG(\"bad generation skip %8x > %8x at %s\",\n+\t\t\t commit->generation, last_gen,\n+\t\t\t oid_to_hex(&commit->object.oid));\n+\t\tlast_gen = commit->generation;\n+\n+\t\tif (commit->generation < min_generation)\n+\t\t\tbreak;\n+\n+\t\tflags = commit->object.flags & (PARENT1 | PARENT2 | STALE);\n+\t\tif (flags == (PARENT1 | PARENT2)) {\n+\t\t\tif (!(commit->object.flags & RESULT)) {\n+\t\t\t\tcommit->object.flags |= RESULT;\n+\t\t\t\tcommit_list_insert_by_date(commit, &result);\n+\t\t\t}\n+\t\t\t/* Mark parents of a found merge stale */\n+\t\t\tflags |= STALE;\n+\t\t}\n+\t\tparents = commit->parents;\n+\t\twhile (parents) {\n+\t\t\tstruct commit *p = parents->item;\n+\t\t\tparents = parents->next;\n+\t\t\tif ((p->object.flags & flags) == flags)\n+\t\t\t\tcontinue;\n+\t\t\tif (parse_commit(p))\n+\t\t\t\treturn NULL;\n+\t\t\tp->object.flags |= flags;\n+\t\t\tprio_queue_put(&queue, p);\n+\t\t}\n+\t}\n+\n+\tclear_prio_queue(&queue);\n+\treturn result;\n+}\n+\n+static struct commit_list *merge_bases_many(struct commit *one, int n, struct commit **twos)\n+{\n+\tstruct commit_list *list = NULL;\n+\tstruct commit_list *result = NULL;\n+\tint i;\n+\n+\tfor (i = 0; i < n; i++) {\n+\t\tif (one == twos[i])\n+\t\t\t/*\n+\t\t\t * We do not mark this even with RESULT so we do not\n+\t\t\t * have to clean it up.\n+\t\t\t */\n+\t\t\treturn commit_list_insert(one, &result);\n+\t}\n+\n+\tif (parse_commit(one))\n+\t\treturn NULL;\n+\tfor (i = 0; i < n; i++) {\n+\t\tif (parse_commit(twos[i]))\n+\t\t\treturn NULL;\n+\t}\n+\n+\tlist = paint_down_to_common(one, n, twos, 0);\n+\n+\twhile (list) {\n+\t\tstruct commit *commit = pop_commit(&list);\n+\t\tif (!(commit->object.flags & STALE))\n+\t\t\tcommit_list_insert_by_date(commit, &result);\n+\t}\n+\treturn result;\n+}\n+\n+struct commit_list *get_octopus_merge_bases(struct commit_list *in)\n+{\n+\tstruct commit_list *i, *j, *k, *ret = NULL;\n+\n+\tif (!in)\n+\t\treturn ret;\n+\n+\tcommit_list_insert(in->item, &ret);\n+\n+\tfor (i = in->next; i; i = i->next) {\n+\t\tstruct commit_list *new_commits = NULL, *end = NULL;\n+\n+\t\tfor (j = ret; j; j = j->next) {\n+\t\t\tstruct commit_list *bases;\n+\t\t\tbases = get_merge_bases(i->item, j->item);\n+\t\t\tif (!new_commits)\n+\t\t\t\tnew_commits = bases;\n+\t\t\telse\n+\t\t\t\tend->next = bases;\n+\t\t\tfor (k = bases; k; k = k->next)\n+\t\t\t\tend = k;\n+\t\t}\n+\t\tret = new_commits;\n+\t}\n+\treturn ret;\n+}\n+\n+static int remove_redundant(struct commit **array, int cnt)\n+{\n+\t/*\n+\t * Some commit in the array may be an ancestor of\n+\t * another commit. Move such commit to the end of\n+\t * the array, and return the number of commits that\n+\t * are independent from each other.\n+\t */\n+\tstruct commit **work;\n+\tunsigned char *redundant;\n+\tint *filled_index;\n+\tint i, j, filled;\n+\n+\twork = xcalloc(cnt, sizeof(*work));\n+\tredundant = xcalloc(cnt, 1);\n+\tALLOC_ARRAY(filled_index, cnt - 1);\n+\n+\tfor (i = 0; i < cnt; i++)\n+\t\tparse_commit(array[i]);\n+\tfor (i = 0; i < cnt; i++) {\n+\t\tstruct commit_list *common;\n+\t\tuint32_t min_generation = array[i]->generation;\n+\n+\t\tif (redundant[i])\n+\t\t\tcontinue;\n+\t\tfor (j = filled = 0; j < cnt; j++) {\n+\t\t\tif (i == j || redundant[j])\n+\t\t\t\tcontinue;\n+\t\t\tfilled_index[filled] = j;\n+\t\t\twork[filled++] = array[j];\n+\n+\t\t\tif (array[j]->generation < min_generation)\n+\t\t\t\tmin_generation = array[j]->generation;\n+\t\t}\n+\t\tcommon = paint_down_to_common(array[i], filled, work,\n+\t\t\t\t\t min_generation);\n+\t\tif (array[i]->object.flags & PARENT2)\n+\t\t\tredundant[i] = 1;\n+\t\tfor (j = 0; j < filled; j++)\n+\t\t\tif (work[j]->object.flags & PARENT1)\n+\t\t\t\tredundant[filled_index[j]] = 1;\n+\t\tclear_commit_marks(array[i], all_flags);\n+\t\tclear_commit_marks_many(filled, work, all_flags);\n+\t\tfree_commit_list(common);\n+\t}\n+\n+\t/* Now collect the result */\n+\tCOPY_ARRAY(work, array, cnt);\n+\tfor (i = filled = 0; i < cnt; i++)\n+\t\tif (!redundant[i])\n+\t\t\tarray[filled++] = work[i];\n+\tfor (j = filled, i = 0; i < cnt; i++)\n+\t\tif (redundant[i])\n+\t\t\tarray[j++] = work[i];\n+\tfree(work);\n+\tfree(redundant);\n+\tfree(filled_index);\n+\treturn filled;\n+}\n+\n+static struct commit_list *get_merge_bases_many_0(struct commit *one,\n+\t\t\t\t\t\t int n,\n+\t\t\t\t\t\t struct commit **twos,\n+\t\t\t\t\t\t int cleanup)\n+{\n+\tstruct commit_list *list;\n+\tstruct commit **rslt;\n+\tstruct commit_list *result;\n+\tint cnt, i;\n+\n+\tresult = merge_bases_many(one, n, twos);\n+\tfor (i = 0; i < n; i++) {\n+\t\tif (one == twos[i])\n+\t\t\treturn result;\n+\t}\n+\tif (!result || !result->next) {\n+\t\tif (cleanup) {\n+\t\t\tclear_commit_marks(one, all_flags);\n+\t\t\tclear_commit_marks_many(n, twos, all_flags);\n+\t\t}\n+\t\treturn result;\n+\t}\n+\n+\t/* There are more than one */\n+\tcnt = commit_list_count(result);\n+\trslt = xcalloc(cnt, sizeof(*rslt));\n+\tfor (list = result, i = 0; list; list = list->next)\n+\t\trslt[i++] = list->item;\n+\tfree_commit_list(result);\n+\n+\tclear_commit_marks(one, all_flags);\n+\tclear_commit_marks_many(n, twos, all_flags);\n+\n+\tcnt = remove_redundant(rslt, cnt);\n+\tresult = NULL;\n+\tfor (i = 0; i < cnt; i++)\n+\t\tcommit_list_insert_by_date(rslt[i], &result);\n+\tfree(rslt);\n+\treturn result;\n+}\n+\n+struct commit_list *get_merge_bases_many(struct commit *one,\n+\t\t\t\t\t int n,\n+\t\t\t\t\t struct commit **twos)\n+{\n+\treturn get_merge_bases_many_0(one, n, twos, 1);\n+}\n+\n+struct commit_list *get_merge_bases_many_dirty(struct commit *one,\n+\t\t\t\t\t int n,\n+\t\t\t\t\t struct commit **twos)\n+{\n+\treturn get_merge_bases_many_0(one, n, twos, 0);\n+}\n+\n+struct commit_list *get_merge_bases(struct commit *one, struct commit *two)\n+{\n+\treturn get_merge_bases_many_0(one, 1, &two, 1);\n+}\n+\n+/*\n+ * Is \"commit\" a descendant of one of the elements on the \"with_commit\" list?\n+ */\n+int is_descendant_of(struct commit *commit, struct commit_list *with_commit)\n+{\n+\tif (!with_commit)\n+\t\treturn 1;\n+\twhile (with_commit) {\n+\t\tstruct commit *other;\n+\n+\t\tother = with_commit->item;\n+\t\twith_commit = with_commit->next;\n+\t\tif (in_merge_bases(other, commit))\n+\t\t\treturn 1;\n+\t}\n+\treturn 0;\n+}\n+\n+/*\n+ * Is \"commit\" an ancestor of one of the \"references\"?\n+ */\n+int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit **reference)\n+{\n+\tstruct commit_list *bases;\n+\tint ret = 0, i;\n+\tuint32_t min_generation = GENERATION_NUMBER_INFINITY;\n+\n+\tif (parse_commit(commit))\n+\t\treturn ret;\n+\tfor (i = 0; i < nr_reference; i++) {\n+\t\tif (parse_commit(reference[i]))\n+\t\t\treturn ret;\n+\t\tif (reference[i]->generation < min_generation)\n+\t\t\tmin_generation = reference[i]->generation;\n+\t}\n+\n+\tif (commit->generation > min_generation)\n+\t\treturn ret;\n+\n+\tbases = paint_down_to_common(commit, nr_reference, reference, commit->generation);\n+\tif (commit->object.flags & PARENT2)\n+\t\tret = 1;\n+\tclear_commit_marks(commit, all_flags);\n+\tclear_commit_marks_many(nr_reference, reference, all_flags);\n+\tfree_commit_list(bases);\n+\treturn ret;\n+}\n+\n+/*\n+ * Is \"commit\" an ancestor of (i.e. reachable from) the \"reference\"?\n+ */\n+int in_merge_bases(struct commit *commit, struct commit *reference)\n+{\n+\treturn in_merge_bases_many(commit, 1, &reference);\n+}\n+\n+struct commit_list *reduce_heads(struct commit_list *heads)\n+{\n+\tstruct commit_list *p;\n+\tstruct commit_list *result = NULL, **tail = &result;\n+\tstruct commit **array;\n+\tint num_head, i;\n+\n+\tif (!heads)\n+\t\treturn NULL;\n+\n+\t/* Uniquify */\n+\tfor (p = heads; p; p = p->next)\n+\t\tp->item->object.flags &= ~STALE;\n+\tfor (p = heads, num_head = 0; p; p = p->next) {\n+\t\tif (p->item->object.flags & STALE)\n+\t\t\tcontinue;\n+\t\tp->item->object.flags |= STALE;\n+\t\tnum_head++;\n+\t}\n+\tarray = xcalloc(num_head, sizeof(*array));\n+\tfor (p = heads, i = 0; p; p = p->next) {\n+\t\tif (p->item->object.flags & STALE) {\n+\t\t\tarray[i++] = p->item;\n+\t\t\tp->item->object.flags &= ~STALE;\n+\t\t}\n+\t}\n+\tnum_head = remove_redundant(array, num_head);\n+\tfor (i = 0; i < num_head; i++)\n+\t\ttail = &commit_list_insert(array[i], tail)->next;\n+\tfree(array);\n+\treturn result;\n+}\n+\n+void reduce_heads_replace(struct commit_list **heads)\n+{\n+\tstruct commit_list *result = reduce_heads(*heads);\n+\tfree_commit_list(*heads);\n+\t*heads = result;\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nnew file mode 100644\nindex 000000000..244f48c5f\n--- /dev/null\n+++ b/commit-reach.h\n@@ -0,0 +1,41 @@\n+#ifndef __COMMIT_REACH_H__\n+#define __COMMIT_REACH_H__\n+\n+#include \"commit.h\"\n+\n+struct commit_list *get_merge_bases_many(struct commit *one,\n+\t\t\t\t\t int n,\n+\t\t\t\t\t struct commit **twos);\n+struct commit_list *get_merge_bases_many_dirty(struct commit *one,\n+\t\t\t\t\t int n,\n+\t\t\t\t\t struct commit **twos);\n+struct commit_list *get_merge_bases(struct commit *one, struct commit *two);\n+struct commit_list *get_octopus_merge_bases(struct commit_list *in);\n+\n+/* To be used only when object flags after this call no longer matter */\n+struct commit_list *get_merge_bases_many_dirty(struct commit *one, int n, struct commit **twos);\n+\n+int is_descendant_of(struct commit *commit, struct commit_list *with_commit);\n+int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit **reference);\n+int in_merge_bases(struct commit *commit, struct commit *reference);\n+\n+\n+/*\n+ * Takes a list of commits and returns a new list where those\n+ * have been removed that can be reached from other commits in\n+ * the list. It is useful for, e.g., reducing the commits\n+ * randomly thrown at the git-merge command and removing\n+ * redundant commits that the user shouldn't have given to it.\n+ *\n+ * This function destroys the STALE bit of the commit objects'\n+ * flags.\n+ */\n+struct commit_list *reduce_heads(struct commit_list *heads);\n+\n+/*\n+ * Like `reduce_heads()`, except it replaces the list. Use this\n+ * instead of `foo = reduce_heads(foo);` to avoid memory leaks.\n+ */\n+void reduce_heads_replace(struct commit_list **heads);\n+\n+#endif\ndiff --git a/commit.c b/commit.c\nindex 39b80bd21..32d1234bd 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -843,364 +843,6 @@ void sort_in_topological_order(struct commit_list **list, enum rev_sort_order so\n \t\tclear_author_date_slab(&author_date);\n }\n \n-/* merge-base stuff */\n-\n-/* Remember to update object flag allocation in object.h */\n-#define PARENT1\t\t(1u<<16)\n-#define PARENT2\t\t(1u<<17)\n-#define STALE\t\t(1u<<18)\n-#define RESULT\t\t(1u<<19)\n-\n-static const unsigned all_flags = (PARENT1 | PARENT2 | STALE | RESULT);\n-\n-static int queue_has_nonstale(struct prio_queue *queue)\n-{\n-\tint i;\n-\tfor (i = 0; i < queue->nr; i++) {\n-\t\tstruct commit *commit = queue->array[i].data;\n-\t\tif (!(commit->object.flags & STALE))\n-\t\t\treturn 1;\n-\t}\n-\treturn 0;\n-}\n-\n-/* all input commits in one and twos[] must have been parsed! */\n-static struct commit_list *paint_down_to_common(struct commit *one, int n,\n-\t\t\t\t\t\tstruct commit **twos,\n-\t\t\t\t\t\tint min_generation)\n-{\n-\tstruct prio_queue queue = { compare_commits_by_gen_then_commit_date };\n-\tstruct commit_list *result = NULL;\n-\tint i;\n-\tuint32_t last_gen = GENERATION_NUMBER_INFINITY;\n-\n-\tone->object.flags |= PARENT1;\n-\tif (!n) {\n-\t\tcommit_list_append(one, &result);\n-\t\treturn result;\n-\t}\n-\tprio_queue_put(&queue, one);\n-\n-\tfor (i = 0; i < n; i++) {\n-\t\ttwos[i]->object.flags |= PARENT2;\n-\t\tprio_queue_put(&queue, twos[i]);\n-\t}\n-\n-\twhile (queue_has_nonstale(&queue)) {\n-\t\tstruct commit *commit = prio_queue_get(&queue);\n-\t\tstruct commit_list *parents;\n-\t\tint flags;\n-\n-\t\tif (commit->generation > last_gen)\n-\t\t\tBUG(\"bad generation skip %8x > %8x at %s\",\n-\t\t\t commit->generation, last_gen,\n-\t\t\t oid_to_hex(&commit->object.oid));\n-\t\tlast_gen = commit->generation;\n-\n-\t\tif (commit->generation < min_generation)\n-\t\t\tbreak;\n-\n-\t\tflags = commit->object.flags & (PARENT1 | PARENT2 | STALE);\n-\t\tif (flags == (PARENT1 | PARENT2)) {\n-\t\t\tif (!(commit->object.flags & RESULT)) {\n-\t\t\t\tcommit->object.flags |= RESULT;\n-\t\t\t\tcommit_list_insert_by_date(commit, &result);\n-\t\t\t}\n-\t\t\t/* Mark parents of a found merge stale */\n-\t\t\tflags |= STALE;\n-\t\t}\n-\t\tparents = commit->parents;\n-\t\twhile (parents) {\n-\t\t\tstruct commit *p = parents->item;\n-\t\t\tparents = parents->next;\n-\t\t\tif ((p->object.flags & flags) == flags)\n-\t\t\t\tcontinue;\n-\t\t\tif (parse_commit(p))\n-\t\t\t\treturn NULL;\n-\t\t\tp->object.flags |= flags;\n-\t\t\tprio_queue_put(&queue, p);\n-\t\t}\n-\t}\n-\n-\tclear_prio_queue(&queue);\n-\treturn result;\n-}\n-\n-static struct commit_list *merge_bases_many(struct commit *one, int n, struct commit **twos)\n-{\n-\tstruct commit_list *list = NULL;\n-\tstruct commit_list *result = NULL;\n-\tint i;\n-\n-\tfor (i = 0; i < n; i++) {\n-\t\tif (one == twos[i])\n-\t\t\t/*\n-\t\t\t * We do not mark this even with RESULT so we do not\n-\t\t\t * have to clean it up.\n-\t\t\t */\n-\t\t\treturn commit_list_insert(one, &result);\n-\t}\n-\n-\tif (parse_commit(one))\n-\t\treturn NULL;\n-\tfor (i = 0; i < n; i++) {\n-\t\tif (parse_commit(twos[i]))\n-\t\t\treturn NULL;\n-\t}\n-\n-\tlist = paint_down_to_common(one, n, twos, 0);\n-\n-\twhile (list) {\n-\t\tstruct commit *commit = pop_commit(&list);\n-\t\tif (!(commit->object.flags & STALE))\n-\t\t\tcommit_list_insert_by_date(commit, &result);\n-\t}\n-\treturn result;\n-}\n-\n-struct commit_list *get_octopus_merge_bases(struct commit_list *in)\n-{\n-\tstruct commit_list *i, *j, *k, *ret = NULL;\n-\n-\tif (!in)\n-\t\treturn ret;\n-\n-\tcommit_list_insert(in->item, &ret);\n-\n-\tfor (i = in->next; i; i = i->next) {\n-\t\tstruct commit_list *new_commits = NULL, *end = NULL;\n-\n-\t\tfor (j = ret; j; j = j->next) {\n-\t\t\tstruct commit_list *bases;\n-\t\t\tbases = get_merge_bases(i->item, j->item);\n-\t\t\tif (!new_commits)\n-\t\t\t\tnew_commits = bases;\n-\t\t\telse\n-\t\t\t\tend->next = bases;\n-\t\t\tfor (k = bases; k; k = k->next)\n-\t\t\t\tend = k;\n-\t\t}\n-\t\tret = new_commits;\n-\t}\n-\treturn ret;\n-}\n-\n-static int remove_redundant(struct commit **array, int cnt)\n-{\n-\t/*\n-\t * Some commit in the array may be an ancestor of\n-\t * another commit. Move such commit to the end of\n-\t * the array, and return the number of commits that\n-\t * are independent from each other.\n-\t */\n-\tstruct commit **work;\n-\tunsigned char *redundant;\n-\tint *filled_index;\n-\tint i, j, filled;\n-\n-\twork = xcalloc(cnt, sizeof(*work));\n-\tredundant = xcalloc(cnt, 1);\n-\tALLOC_ARRAY(filled_index, cnt - 1);\n-\n-\tfor (i = 0; i < cnt; i++)\n-\t\tparse_commit(array[i]);\n-\tfor (i = 0; i < cnt; i++) {\n-\t\tstruct commit_list *common;\n-\t\tuint32_t min_generation = array[i]->generation;\n-\n-\t\tif (redundant[i])\n-\t\t\tcontinue;\n-\t\tfor (j = filled = 0; j < cnt; j++) {\n-\t\t\tif (i == j || redundant[j])\n-\t\t\t\tcontinue;\n-\t\t\tfilled_index[filled] = j;\n-\t\t\twork[filled++] = array[j];\n-\n-\t\t\tif (array[j]->generation < min_generation)\n-\t\t\t\tmin_generation = array[j]->generation;\n-\t\t}\n-\t\tcommon = paint_down_to_common(array[i], filled, work,\n-\t\t\t\t\t min_generation);\n-\t\tif (array[i]->object.flags & PARENT2)\n-\t\t\tredundant[i] = 1;\n-\t\tfor (j = 0; j < filled; j++)\n-\t\t\tif (work[j]->object.flags & PARENT1)\n-\t\t\t\tredundant[filled_index[j]] = 1;\n-\t\tclear_commit_marks(array[i], all_flags);\n-\t\tclear_commit_marks_many(filled, work, all_flags);\n-\t\tfree_commit_list(common);\n-\t}\n-\n-\t/* Now collect the result */\n-\tCOPY_ARRAY(work, array, cnt);\n-\tfor (i = filled = 0; i < cnt; i++)\n-\t\tif (!redundant[i])\n-\t\t\tarray[filled++] = work[i];\n-\tfor (j = filled, i = 0; i < cnt; i++)\n-\t\tif (redundant[i])\n-\t\t\tarray[j++] = work[i];\n-\tfree(work);\n-\tfree(redundant);\n-\tfree(filled_index);\n-\treturn filled;\n-}\n-\n-static struct commit_list *get_merge_bases_many_0(struct commit *one,\n-\t\t\t\t\t\t int n,\n-\t\t\t\t\t\t struct commit **twos,\n-\t\t\t\t\t\t int cleanup)\n-{\n-\tstruct commit_list *list;\n-\tstruct commit **rslt;\n-\tstruct commit_list *result;\n-\tint cnt, i;\n-\n-\tresult = merge_bases_many(one, n, twos);\n-\tfor (i = 0; i < n; i++) {\n-\t\tif (one == twos[i])\n-\t\t\treturn result;\n-\t}\n-\tif (!result || !result->next) {\n-\t\tif (cleanup) {\n-\t\t\tclear_commit_marks(one, all_flags);\n-\t\t\tclear_commit_marks_many(n, twos, all_flags);\n-\t\t}\n-\t\treturn result;\n-\t}\n-\n-\t/* There are more than one */\n-\tcnt = commit_list_count(result);\n-\trslt = xcalloc(cnt, sizeof(*rslt));\n-\tfor (list = result, i = 0; list; list = list->next)\n-\t\trslt[i++] = list->item;\n-\tfree_commit_list(result);\n-\n-\tclear_commit_marks(one, all_flags);\n-\tclear_commit_marks_many(n, twos, all_flags);\n-\n-\tcnt = remove_redundant(rslt, cnt);\n-\tresult = NULL;\n-\tfor (i = 0; i < cnt; i++)\n-\t\tcommit_list_insert_by_date(rslt[i], &result);\n-\tfree(rslt);\n-\treturn result;\n-}\n-\n-struct commit_list *get_merge_bases_many(struct commit *one,\n-\t\t\t\t\t int n,\n-\t\t\t\t\t struct commit **twos)\n-{\n-\treturn get_merge_bases_many_0(one, n, twos, 1);\n-}\n-\n-struct commit_list *get_merge_bases_many_dirty(struct commit *one,\n-\t\t\t\t\t int n,\n-\t\t\t\t\t struct commit **twos)\n-{\n-\treturn get_merge_bases_many_0(one, n, twos, 0);\n-}\n-\n-struct commit_list *get_merge_bases(struct commit *one, struct commit *two)\n-{\n-\treturn get_merge_bases_many_0(one, 1, &two, 1);\n-}\n-\n-/*\n- * Is \"commit\" a descendant of one of the elements on the \"with_commit\" list?\n- */\n-int is_descendant_of(struct commit *commit, struct commit_list *with_commit)\n-{\n-\tif (!with_commit)\n-\t\treturn 1;\n-\twhile (with_commit) {\n-\t\tstruct commit *other;\n-\n-\t\tother = with_commit->item;\n-\t\twith_commit = with_commit->next;\n-\t\tif (in_merge_bases(other, commit))\n-\t\t\treturn 1;\n-\t}\n-\treturn 0;\n-}\n-\n-/*\n- * Is \"commit\" an ancestor of one of the \"references\"?\n- */\n-int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit **reference)\n-{\n-\tstruct commit_list *bases;\n-\tint ret = 0, i;\n-\tuint32_t min_generation = GENERATION_NUMBER_INFINITY;\n-\n-\tif (parse_commit(commit))\n-\t\treturn ret;\n-\tfor (i = 0; i < nr_reference; i++) {\n-\t\tif (parse_commit(reference[i]))\n-\t\t\treturn ret;\n-\t\tif (reference[i]->generation < min_generation)\n-\t\t\tmin_generation = reference[i]->generation;\n-\t}\n-\n-\tif (commit->generation > min_generation)\n-\t\treturn ret;\n-\n-\tbases = paint_down_to_common(commit, nr_reference, reference, commit->generation);\n-\tif (commit->object.flags & PARENT2)\n-\t\tret = 1;\n-\tclear_commit_marks(commit, all_flags);\n-\tclear_commit_marks_many(nr_reference, reference, all_flags);\n-\tfree_commit_list(bases);\n-\treturn ret;\n-}\n-\n-/*\n- * Is \"commit\" an ancestor of (i.e. reachable from) the \"reference\"?\n- */\n-int in_merge_bases(struct commit *commit, struct commit *reference)\n-{\n-\treturn in_merge_bases_many(commit, 1, &reference);\n-}\n-\n-struct commit_list *reduce_heads(struct commit_list *heads)\n-{\n-\tstruct commit_list *p;\n-\tstruct commit_list *result = NULL, **tail = &result;\n-\tstruct commit **array;\n-\tint num_head, i;\n-\n-\tif (!heads)\n-\t\treturn NULL;\n-\n-\t/* Uniquify */\n-\tfor (p = heads; p; p = p->next)\n-\t\tp->item->object.flags &= ~STALE;\n-\tfor (p = heads, num_head = 0; p; p = p->next) {\n-\t\tif (p->item->object.flags & STALE)\n-\t\t\tcontinue;\n-\t\tp->item->object.flags |= STALE;\n-\t\tnum_head++;\n-\t}\n-\tarray = xcalloc(num_head, sizeof(*array));\n-\tfor (p = heads, i = 0; p; p = p->next) {\n-\t\tif (p->item->object.flags & STALE) {\n-\t\t\tarray[i++] = p->item;\n-\t\t\tp->item->object.flags &= ~STALE;\n-\t\t}\n-\t}\n-\tnum_head = remove_redundant(array, num_head);\n-\tfor (i = 0; i < num_head; i++)\n-\t\ttail = &commit_list_insert(array[i], tail)->next;\n-\tfree(array);\n-\treturn result;\n-}\n-\n-void reduce_heads_replace(struct commit_list **heads)\n-{\n-\tstruct commit_list *result = reduce_heads(*heads);\n-\tfree_commit_list(*heads);\n-\t*heads = result;\n-}\n-\n static const char gpg_sig_header[] = \"gpgsig\";\n static const int gpg_sig_header_len = sizeof(gpg_sig_header) - 1;\n \n-- \ngitgitgadget\n\n"},{"id":"352610","messageId":"7a6d1cbe12efa0a452a977f1f6bb8c618421040b.1531746010.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 02/16] commit-reach: move ref_newer from remote.c","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-06-25T17:35:03Z","receivedAt":"2018-06-25T17:35:03Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nSigned-off-by: Derrick Stolee \n---\n builtin/remote.c | 1 +\n commit-reach.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 2 ++\n http-push.c | 1 +\n remote.c | 50 +-------------------------------------------\n remote.h | 1 -\n 6 files changed, 59 insertions(+), 50 deletions(-)\n\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex c74ee8869..79b032644 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -10,6 +10,7 @@\n #include \"refspec.h\"\n #include \"object-store.h\"\n #include \"argv-array.h\"\n+#include \"commit-reach.h\"\n \n static const char * const builtin_remote_usage[] = {\n \tN_(\"git remote [-v | --verbose]\"),\ndiff --git a/commit-reach.c b/commit-reach.c\nindex f2e2f7461..a6bc4781a 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -1,5 +1,10 @@\n #include \"cache.h\"\n+#include \"commit.h\"\n+#include \"decorate.h\"\n #include \"prio-queue.h\"\n+#include \"tree.h\"\n+#include \"revision.h\"\n+#include \"tag.h\"\n #include \"commit-reach.h\"\n \n /* Remember to update object flag allocation in object.h */\n@@ -357,3 +362,52 @@ void reduce_heads_replace(struct commit_list **heads)\n \tfree_commit_list(*heads);\n \t*heads = result;\n }\n+\n+static void unmark_and_free(struct commit_list *list, unsigned int mark)\n+{\n+\twhile (list) {\n+\t\tstruct commit *commit = pop_commit(&list);\n+\t\tcommit->object.flags &= ~mark;\n+\t}\n+}\n+\n+int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n+{\n+\tstruct object *o;\n+\tstruct commit *old_commit, *new_commit;\n+\tstruct commit_list *list, *used;\n+\tint found = 0;\n+\n+\t/*\n+\t * Both new_commit and old_commit must be commit-ish and new_commit is descendant of\n+\t * old_commit. Otherwise we require --force.\n+\t */\n+\to = deref_tag(the_repository, parse_object(the_repository, old_oid),\n+\t\t NULL, 0);\n+\tif (!o || o->type != OBJ_COMMIT)\n+\t\treturn 0;\n+\told_commit = (struct commit *) o;\n+\n+\to = deref_tag(the_repository, parse_object(the_repository, new_oid),\n+\t\t NULL, 0);\n+\tif (!o || o->type != OBJ_COMMIT)\n+\t\treturn 0;\n+\tnew_commit = (struct commit *) o;\n+\n+\tif (parse_commit(new_commit) < 0)\n+\t\treturn 0;\n+\n+\tused = list = NULL;\n+\tcommit_list_insert(new_commit, &list);\n+\twhile (list) {\n+\t\tnew_commit = pop_most_recent_commit(&list, TMP_MARK);\n+\t\tcommit_list_insert(new_commit, &used);\n+\t\tif (new_commit == old_commit) {\n+\t\t\tfound = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tunmark_and_free(list, TMP_MARK);\n+\tunmark_and_free(used, TMP_MARK);\n+\treturn found;\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nindex 244f48c5f..35ec9f0dd 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -38,4 +38,6 @@ struct commit_list *reduce_heads(struct commit_list *heads);\n */\n void reduce_heads_replace(struct commit_list **heads);\n \n+int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid);\n+\n #endif\ndiff --git a/http-push.c b/http-push.c\nindex 5eaf551b5..e007cb5a6 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -14,6 +14,7 @@\n #include \"argv-array.h\"\n #include \"packfile.h\"\n #include \"object-store.h\"\n+#include \"commit-reach.h\"\n \n \n #ifdef EXPAT_NEEDS_XMLPARSE_H\ndiff --git a/remote.c b/remote.c\nindex 26b1fbd9a..f0c23bae4 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -12,6 +12,7 @@\n #include \"string-list.h\"\n #include \"mergesort.h\"\n #include \"argv-array.h\"\n+#include \"commit-reach.h\"\n \n enum map_direction { FROM_SRC, FROM_DST };\n \n@@ -1783,55 +1784,6 @@ int resolve_remote_symref(struct ref *ref, struct ref *list)\n \treturn 1;\n }\n \n-static void unmark_and_free(struct commit_list *list, unsigned int mark)\n-{\n-\twhile (list) {\n-\t\tstruct commit *commit = pop_commit(&list);\n-\t\tcommit->object.flags &= ~mark;\n-\t}\n-}\n-\n-int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n-{\n-\tstruct object *o;\n-\tstruct commit *old_commit, *new_commit;\n-\tstruct commit_list *list, *used;\n-\tint found = 0;\n-\n-\t/*\n-\t * Both new_commit and old_commit must be commit-ish and new_commit is descendant of\n-\t * old_commit. Otherwise we require --force.\n-\t */\n-\to = deref_tag(the_repository, parse_object(the_repository, old_oid),\n-\t\t NULL, 0);\n-\tif (!o || o->type != OBJ_COMMIT)\n-\t\treturn 0;\n-\told_commit = (struct commit *) o;\n-\n-\to = deref_tag(the_repository, parse_object(the_repository, new_oid),\n-\t\t NULL, 0);\n-\tif (!o || o->type != OBJ_COMMIT)\n-\t\treturn 0;\n-\tnew_commit = (struct commit *) o;\n-\n-\tif (parse_commit(new_commit) < 0)\n-\t\treturn 0;\n-\n-\tused = list = NULL;\n-\tcommit_list_insert(new_commit, &list);\n-\twhile (list) {\n-\t\tnew_commit = pop_most_recent_commit(&list, TMP_MARK);\n-\t\tcommit_list_insert(new_commit, &used);\n-\t\tif (new_commit == old_commit) {\n-\t\t\tfound = 1;\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tunmark_and_free(list, TMP_MARK);\n-\tunmark_and_free(used, TMP_MARK);\n-\treturn found;\n-}\n-\n /*\n * Lookup the upstream branch for the given branch and if present, optionally\n * compute the commit ahead/behind values for the pair.\ndiff --git a/remote.h b/remote.h\nindex 45ecc6cef..56fb9cbb2 100644\n--- a/remote.h\n+++ b/remote.h\n@@ -149,7 +149,6 @@ extern struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,\n \t\t\t\t const struct string_list *server_options);\n \n int resolve_remote_symref(struct ref *ref, struct ref *list);\n-int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid);\n \n /*\n * Remove and free all but the first of any entries in the input list\n-- \ngitgitgadget\n\n"},{"id":"352612","messageId":"b9687d2b21c4560178a509572770e3a058582a8b.1531746010.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 03/16] commit-reach: move commit_contains from ref-filter","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-06-25T18:01:28Z","receivedAt":"2018-06-25T18:01:28Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 120 ++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 18 +++++-\n fast-import.c | 1 +\n ref-filter.c | 147 +++----------------------------------------------\n 4 files changed, 146 insertions(+), 140 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex a6bc4781a..9e56f90ea 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"commit.h\"\n+#include \"commit-graph.h\"\n #include \"decorate.h\"\n #include \"prio-queue.h\"\n #include \"tree.h\"\n@@ -411,3 +412,122 @@ int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n \tunmark_and_free(used, TMP_MARK);\n \treturn found;\n }\n+\n+/*\n+ * Mimicking the real stack, this stack lives on the heap, avoiding stack\n+ * overflows.\n+ *\n+ * At each recursion step, the stack items points to the commits whose\n+ * ancestors are to be inspected.\n+ */\n+struct contains_stack {\n+\tint nr, alloc;\n+\tstruct contains_stack_entry {\n+\t\tstruct commit *commit;\n+\t\tstruct commit_list *parents;\n+\t} *contains_stack;\n+};\n+\n+static int in_commit_list(const struct commit_list *want, struct commit *c)\n+{\n+\tfor (; want; want = want->next)\n+\t\tif (!oidcmp(&want->item->object.oid, &c->object.oid))\n+\t\t\treturn 1;\n+\treturn 0;\n+}\n+\n+/*\n+ * Test whether the candidate is contained in the list.\n+ * Do not recurse to find out, though, but return -1 if inconclusive.\n+ */\n+static enum contains_result contains_test(struct commit *candidate,\n+\t\t\t\t\t const struct commit_list *want,\n+\t\t\t\t\t struct contains_cache *cache,\n+\t\t\t\t\t uint32_t cutoff)\n+{\n+\tenum contains_result *cached = contains_cache_at(cache, candidate);\n+\n+\t/* If we already have the answer cached, return that. */\n+\tif (*cached)\n+\t\treturn *cached;\n+\n+\t/* or are we it? */\n+\tif (in_commit_list(want, candidate)) {\n+\t\t*cached = CONTAINS_YES;\n+\t\treturn CONTAINS_YES;\n+\t}\n+\n+\t/* Otherwise, we don't know; prepare to recurse */\n+\tparse_commit_or_die(candidate);\n+\n+\tif (candidate->generation < cutoff)\n+\t\treturn CONTAINS_NO;\n+\n+\treturn CONTAINS_UNKNOWN;\n+}\n+\n+static void push_to_contains_stack(struct commit *candidate, struct contains_stack *contains_stack)\n+{\n+\tALLOC_GROW(contains_stack->contains_stack, contains_stack->nr + 1, contains_stack->alloc);\n+\tcontains_stack->contains_stack[contains_stack->nr].commit = candidate;\n+\tcontains_stack->contains_stack[contains_stack->nr++].parents = candidate->parents;\n+}\n+\n+static enum contains_result contains_tag_algo(struct commit *candidate,\n+\t\t\t\t\t const struct commit_list *want,\n+\t\t\t\t\t struct contains_cache *cache)\n+{\n+\tstruct contains_stack contains_stack = { 0, 0, NULL };\n+\tenum contains_result result;\n+\tuint32_t cutoff = GENERATION_NUMBER_INFINITY;\n+\tconst struct commit_list *p;\n+\n+\tfor (p = want; p; p = p->next) {\n+\t\tstruct commit *c = p->item;\n+\t\tload_commit_graph_info(the_repository, c);\n+\t\tif (c->generation < cutoff)\n+\t\t\tcutoff = c->generation;\n+\t}\n+\n+\tresult = contains_test(candidate, want, cache, cutoff);\n+\tif (result != CONTAINS_UNKNOWN)\n+\t\treturn result;\n+\n+\tpush_to_contains_stack(candidate, &contains_stack);\n+\twhile (contains_stack.nr) {\n+\t\tstruct contains_stack_entry *entry = &contains_stack.contains_stack[contains_stack.nr - 1];\n+\t\tstruct commit *commit = entry->commit;\n+\t\tstruct commit_list *parents = entry->parents;\n+\n+\t\tif (!parents) {\n+\t\t\t*contains_cache_at(cache, commit) = CONTAINS_NO;\n+\t\t\tcontains_stack.nr--;\n+\t\t}\n+\t\t/*\n+\t\t * If we just popped the stack, parents->item has been marked,\n+\t\t * therefore contains_test will return a meaningful yes/no.\n+\t\t */\n+\t\telse switch (contains_test(parents->item, want, cache, cutoff)) {\n+\t\tcase CONTAINS_YES:\n+\t\t\t*contains_cache_at(cache, commit) = CONTAINS_YES;\n+\t\t\tcontains_stack.nr--;\n+\t\t\tbreak;\n+\t\tcase CONTAINS_NO:\n+\t\t\tentry->parents = parents->next;\n+\t\t\tbreak;\n+\t\tcase CONTAINS_UNKNOWN:\n+\t\t\tpush_to_contains_stack(parents->item, &contains_stack);\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tfree(contains_stack.contains_stack);\n+\treturn contains_test(candidate, want, cache, cutoff);\n+}\n+\n+int commit_contains(struct ref_filter *filter, struct commit *commit,\n+\t\t struct commit_list *list, struct contains_cache *cache)\n+{\n+\tif (filter->with_commit_tag_algo)\n+\t\treturn contains_tag_algo(commit, list, cache) == CONTAINS_YES;\n+\treturn is_descendant_of(commit, list);\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nindex 35ec9f0dd..925cb05d7 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -2,6 +2,8 @@\n #define __COMMIT_REACH_H__\n \n #include \"commit.h\"\n+#include \"commit-slab.h\"\n+#include \"ref-filter.h\"\n \n struct commit_list *get_merge_bases_many(struct commit *one,\n \t\t\t\t\t int n,\n@@ -19,7 +21,6 @@ int is_descendant_of(struct commit *commit, struct commit_list *with_commit);\n int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit **reference);\n int in_merge_bases(struct commit *commit, struct commit *reference);\n \n-\n /*\n * Takes a list of commits and returns a new list where those\n * have been removed that can be reached from other commits in\n@@ -40,4 +41,19 @@ void reduce_heads_replace(struct commit_list **heads);\n \n int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid);\n \n+/*\n+ * Unknown has to be \"0\" here, because that's the default value for\n+ * contains_cache slab entries that have not yet been assigned.\n+ */\n+enum contains_result {\n+\tCONTAINS_UNKNOWN = 0,\n+\tCONTAINS_NO,\n+\tCONTAINS_YES\n+};\n+\n+define_commit_slab(contains_cache, enum contains_result);\n+\n+int commit_contains(struct ref_filter *filter, struct commit *commit,\n+\t\t struct commit_list *list, struct contains_cache *cache);\n+\n #endif\ndiff --git a/fast-import.c b/fast-import.c\nindex 3ea578102..4a93df383 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -171,6 +171,7 @@ Format of STDIN stream:\n #include \"packfile.h\"\n #include \"object-store.h\"\n #include \"mem-pool.h\"\n+#include \"commit-reach.h\"\n \n #define PACK_ID_BITS 16\n #define MAX_PACK_ID ((1<next)\n-\t\tif (!oidcmp(&want->item->object.oid, &c->object.oid))\n-\t\t\treturn 1;\n-\treturn 0;\n-}\n-\n-/*\n- * Test whether the candidate is contained in the list.\n- * Do not recurse to find out, though, but return -1 if inconclusive.\n- */\n-static enum contains_result contains_test(struct commit *candidate,\n-\t\t\t\t\t const struct commit_list *want,\n-\t\t\t\t\t struct contains_cache *cache,\n-\t\t\t\t\t uint32_t cutoff)\n-{\n-\tenum contains_result *cached = contains_cache_at(cache, candidate);\n-\n-\t/* If we already have the answer cached, return that. */\n-\tif (*cached)\n-\t\treturn *cached;\n-\n-\t/* or are we it? */\n-\tif (in_commit_list(want, candidate)) {\n-\t\t*cached = CONTAINS_YES;\n-\t\treturn CONTAINS_YES;\n-\t}\n-\n-\t/* Otherwise, we don't know; prepare to recurse */\n-\tparse_commit_or_die(candidate);\n-\n-\tif (candidate->generation < cutoff)\n-\t\treturn CONTAINS_NO;\n-\n-\treturn CONTAINS_UNKNOWN;\n-}\n-\n-static void push_to_contains_stack(struct commit *candidate, struct contains_stack *contains_stack)\n-{\n-\tALLOC_GROW(contains_stack->contains_stack, contains_stack->nr + 1, contains_stack->alloc);\n-\tcontains_stack->contains_stack[contains_stack->nr].commit = candidate;\n-\tcontains_stack->contains_stack[contains_stack->nr++].parents = candidate->parents;\n-}\n-\n-static enum contains_result contains_tag_algo(struct commit *candidate,\n-\t\t\t\t\t const struct commit_list *want,\n-\t\t\t\t\t struct contains_cache *cache)\n-{\n-\tstruct contains_stack contains_stack = { 0, 0, NULL };\n-\tenum contains_result result;\n-\tuint32_t cutoff = GENERATION_NUMBER_INFINITY;\n-\tconst struct commit_list *p;\n-\n-\tfor (p = want; p; p = p->next) {\n-\t\tstruct commit *c = p->item;\n-\t\tload_commit_graph_info(the_repository, c);\n-\t\tif (c->generation < cutoff)\n-\t\t\tcutoff = c->generation;\n-\t}\n-\n-\tresult = contains_test(candidate, want, cache, cutoff);\n-\tif (result != CONTAINS_UNKNOWN)\n-\t\treturn result;\n-\n-\tpush_to_contains_stack(candidate, &contains_stack);\n-\twhile (contains_stack.nr) {\n-\t\tstruct contains_stack_entry *entry = &contains_stack.contains_stack[contains_stack.nr - 1];\n-\t\tstruct commit *commit = entry->commit;\n-\t\tstruct commit_list *parents = entry->parents;\n-\n-\t\tif (!parents) {\n-\t\t\t*contains_cache_at(cache, commit) = CONTAINS_NO;\n-\t\t\tcontains_stack.nr--;\n-\t\t}\n-\t\t/*\n-\t\t * If we just popped the stack, parents->item has been marked,\n-\t\t * therefore contains_test will return a meaningful yes/no.\n-\t\t */\n-\t\telse switch (contains_test(parents->item, want, cache, cutoff)) {\n-\t\tcase CONTAINS_YES:\n-\t\t\t*contains_cache_at(cache, commit) = CONTAINS_YES;\n-\t\t\tcontains_stack.nr--;\n-\t\t\tbreak;\n-\t\tcase CONTAINS_NO:\n-\t\t\tentry->parents = parents->next;\n-\t\t\tbreak;\n-\t\tcase CONTAINS_UNKNOWN:\n-\t\t\tpush_to_contains_stack(parents->item, &contains_stack);\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tfree(contains_stack.contains_stack);\n-\treturn contains_test(candidate, want, cache, cutoff);\n-}\n-\n-static int commit_contains(struct ref_filter *filter, struct commit *commit,\n-\t\t\t struct commit_list *list, struct contains_cache *cache)\n-{\n-\tif (filter->with_commit_tag_algo)\n-\t\treturn contains_tag_algo(commit, list, cache) == CONTAINS_YES;\n-\treturn is_descendant_of(commit, list);\n-}\n-\n /*\n * Return 1 if the refname matches one of the patterns, otherwise 0.\n * A pattern can be a literal prefix (e.g. a refname \"refs/heads/master\"\n@@ -1987,6 +1849,13 @@ static int filter_ref_kind(struct ref_filter *filter, const char *refname)\n \treturn ref_kind_from_refname(refname);\n }\n \n+struct ref_filter_cbdata {\n+\tstruct ref_array *array;\n+\tstruct ref_filter *filter;\n+\tstruct contains_cache contains_cache;\n+\tstruct contains_cache no_contains_cache;\n+};\n+\n /*\n * A call-back given to for_each_ref(). Filter refs and keep them for\n * later object processing.\n-- \ngitgitgadget\n\n"},{"id":"352623","messageId":"816821eec9ba476ccdfbfdf6e3cdd3619743ea2e.1531746012.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-06-28T12:31:05Z","receivedAt":"2018-06-28T12:31:05Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe can_all_from_reach_with_flags() algorithm is currently quadratic in\nthe worst case, because it calls the reachable() method for every 'from'\nwithout tracking which commits have already been walked or which can\nalready reach a commit in 'to'.\n\nRewrite the algorithm to walk each commit a constant number of times.\n\nWe also add some optimizations that should work for the main consumer of\nthis method: fetch negotitation (haves/wants).\n\nThe first step includes using a depth-first-search (DFS) from each from\ncommit, sorted by ascending generation number. We do not walk beyond the\nminimum generation number or the minimum commit date. This DFS is likely\nto be faster than the existing reachable() method because we expect\nprevious ref values to be along the first-parent history.\n\nIf we find a target commit, then we mark everything in the DFS stack as\na RESULT. This expands the set of targets for the other from commits. We\nalso mark the visited commits using 'assign_flag' to prevent re-walking\nthe same code.\n\nWe still need to clear our flags at the end, which is why we will have a\ntotal of three visits to each commit.\n\nPerformance was measured on the Linux repository using\n'test-tool reach can_all_from_reach'. The input included rows seeded by\ntag values. The \"small\" case included X-rows as v4.[0-9]* and Y-rows as\nv3.[0-9]*. This mimics a (very large) fetch that says \"I have all major\nv3 releases and want all major v4 releases.\" The \"large\" case included\nX-rows as \"v4.*\" and Y-rows as \"v3.*\". This adds all release-candidate\ntags to the set, which does not greatly increase the number of objects\nthat are considered, but does increase the number of 'from' commits,\ndemonstrating the quadratic nature of the previous code.\n\nSmall Case\n----------\n\nBefore: 1.52 s\n After: 0.26 s\n\nLarge Case\n----------\n\nBefore: 3.50 s\n After: 0.27 s\n\nNote how the time increases between the two cases in the two versions.\nThe new code increases relative to the number of commits that need to be\nwalked, but not directly relative to the number of 'from' commits.\n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 124 ++++++++++++++++++++++++++++++-------------------\n commit-reach.h | 6 ++-\n upload-pack.c | 5 +-\n 3 files changed, 85 insertions(+), 50 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex c58e50fbb..ac132c8e4 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -513,65 +513,88 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n \treturn is_descendant_of(commit, list);\n }\n \n-int reachable(struct commit *from, int with_flag, int assign_flag,\n-\t time_t min_commit_date)\n+static int compare_commits_by_gen(const void *_a, const void *_b)\n {\n-\tstruct prio_queue work = { compare_commits_by_commit_date };\n+\tconst struct commit *a = (const struct commit *)_a;\n+\tconst struct commit *b = (const struct commit *)_b;\n \n-\tprio_queue_put(&work, from);\n-\twhile (work.nr) {\n-\t\tstruct commit_list *list;\n-\t\tstruct commit *commit = prio_queue_get(&work);\n-\n-\t\tif (commit->object.flags & with_flag) {\n-\t\t\tfrom->object.flags |= assign_flag;\n-\t\t\tbreak;\n-\t\t}\n-\t\tif (!commit->object.parsed)\n-\t\t\tparse_object(the_repository, &commit->object.oid);\n-\t\tif (commit->object.flags & REACHABLE)\n-\t\t\tcontinue;\n-\t\tcommit->object.flags |= REACHABLE;\n-\t\tif (commit->date < min_commit_date)\n-\t\t\tcontinue;\n-\t\tfor (list = commit->parents; list; list = list->next) {\n-\t\t\tstruct commit *parent = list->item;\n-\t\t\tif (!(parent->object.flags & REACHABLE))\n-\t\t\t\tprio_queue_put(&work, parent);\n-\t\t}\n-\t}\n-\tfrom->object.flags |= REACHABLE;\n-\tclear_commit_marks(from, REACHABLE);\n-\tclear_prio_queue(&work);\n-\treturn (from->object.flags & assign_flag);\n+\tif (a->generation < b->generation)\n+\t\treturn -1;\n+\tif (a->generation > b->generation)\n+\t\treturn 1;\n+\treturn 0;\n }\n \n int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\t int with_flag, int assign_flag,\n-\t\t\t\t time_t min_commit_date)\n+\t\t\t\t time_t min_commit_date,\n+\t\t\t\t uint32_t min_generation)\n {\n+\tstruct commit **list = NULL;\n \tint i;\n+\tint result = 1;\n \n+\tALLOC_ARRAY(list, from->nr);\n \tfor (i = 0; i < from->nr; i++) {\n-\t\tstruct object *from_one = from->objects[i].item;\n+\t\tlist[i] = (struct commit *)from->objects[i].item;\n \n-\t\tif (from_one->flags & assign_flag)\n-\t\t\tcontinue;\n-\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n-\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n-\t\t\t/* no way to tell if this is reachable by\n-\t\t\t * looking at the ancestry chain alone, so\n-\t\t\t * leave a note to ourselves not to worry about\n-\t\t\t * this object anymore.\n-\t\t\t */\n-\t\t\tfrom->objects[i].item->flags |= assign_flag;\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag,\n-\t\t\t min_commit_date))\n+\t\tparse_commit(list[i]);\n+\n+\t\tif (list[i]->generation < min_generation)\n \t\t\treturn 0;\n \t}\n-\treturn 1;\n+\n+\tQSORT(list, from->nr, compare_commits_by_gen);\n+\n+\tfor (i = 0; i < from->nr; i++) {\n+\t\t/* DFS from list[i] */\n+\t\tstruct commit_list *stack = NULL;\n+\n+\t\tlist[i]->object.flags |= assign_flag;\n+\t\tcommit_list_insert(list[i], &stack);\n+\n+\t\twhile (stack) {\n+\t\t\tstruct commit_list *parent;\n+\n+\t\t\tif (stack->item->object.flags & with_flag) {\n+\t\t\t\tpop_commit(&stack);\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\tfor (parent = stack->item->parents; parent; parent = parent->next) {\n+\t\t\t\tif (parent->item->object.flags & (with_flag | RESULT))\n+\t\t\t\t\tstack->item->object.flags |= RESULT;\n+\n+\t\t\t\tif (!(parent->item->object.flags & assign_flag)) {\n+\t\t\t\t\tparent->item->object.flags |= assign_flag;\n+\n+\t\t\t\t\tparse_commit(parent->item);\n+\n+\t\t\t\t\tif (parent->item->date < min_commit_date ||\n+\t\t\t\t\t parent->item->generation < min_generation)\n+\t\t\t\t\t\tcontinue;\n+\n+\t\t\t\t\tcommit_list_insert(parent->item, &stack);\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t}\n+\n+\t\t\tif (!parent)\n+\t\t\t\tpop_commit(&stack);\n+\t\t}\n+\n+\t\tif (!(list[i]->object.flags & (with_flag | RESULT))) {\n+\t\t\tresult = 0;\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n+cleanup:\n+\tfor (i = 0; i < from->nr; i++) {\n+\t\tclear_commit_marks(list[i], RESULT);\n+\t\tclear_commit_marks(list[i], assign_flag);\n+\t}\n+\treturn result;\n }\n \n int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n@@ -581,6 +604,7 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \ttime_t min_commit_date = cutoff_by_min_date ? from->item->date : 0;\n \tstruct commit_list *from_iter = from, *to_iter = to;\n \tint result;\n+\tuint32_t min_generation = GENERATION_NUMBER_INFINITY;\n \n \twhile (from_iter) {\n \t\tadd_object_array(&from_iter->item->object, NULL, &from_objs);\n@@ -588,6 +612,9 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \t\tif (!parse_commit(from_iter->item)) {\n \t\t\tif (from_iter->item->date < min_commit_date)\n \t\t\t\tmin_commit_date = from_iter->item->date;\n+\n+\t\t\tif (from_iter->item->generation < min_generation)\n+\t\t\t\tmin_generation = from_iter->item->generation;\n \t\t}\n \n \t\tfrom_iter = from_iter->next;\n@@ -597,6 +624,9 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \t\tif (!parse_commit(to_iter->item)) {\n \t\t\tif (to_iter->item->date < min_commit_date)\n \t\t\t\tmin_commit_date = to_iter->item->date;\n+\n+\t\t\tif (to_iter->item->generation < min_generation)\n+\t\t\t\tmin_generation = to_iter->item->generation;\n \t\t}\n \n \t\tto_iter->item->object.flags |= PARENT2;\n@@ -605,7 +635,7 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \t}\n \n \tresult = can_all_from_reach_with_flag(&from_objs, PARENT2, PARENT1,\n-\t\t\t\t\t min_commit_date);\n+\t\t\t\t\t min_commit_date, min_generation);\n \n \twhile (from) {\n \t\tclear_commit_marks(from->item, PARENT1);\ndiff --git a/commit-reach.h b/commit-reach.h\nindex 58de0df56..482d9eb5d 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -63,11 +63,13 @@ int reachable(struct commit *from, int with_flag, int assign_flag,\n * Determine if every commit in 'from' can reach at least one commit\n * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n * as a marker for commits that are already visited. Do not walk\n- * commits with date below 'min_commit_date'.\n+ * commits with date below 'min_commit_date' or generation below\n+ * 'min_generation'.\n */\n int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\t int with_flag, int assign_flag,\n-\t\t\t\t time_t min_commit_date);\n+\t\t\t\t time_t min_commit_date,\n+\t\t\t\t uint32_t min_generation);\n int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \t\t int commit_date_cutoff);\n \ndiff --git a/upload-pack.c b/upload-pack.c\nindex 11c426685..1e498f118 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -338,11 +338,14 @@ static int got_oid(const char *hex, struct object_id *oid)\n \n static int ok_to_give_up(void)\n {\n+\tuint32_t min_generation = GENERATION_NUMBER_ZERO;\n+\n \tif (!have_obj.nr)\n \t\treturn 0;\n \n \treturn can_all_from_reach_with_flag(&want_obj, THEY_HAVE,\n-\t\t\t\t\t COMMON_KNOWN, oldest_have);\n+\t\t\t\t\t COMMON_KNOWN, oldest_have,\n+\t\t\t\t\t min_generation);\n }\n \n static int get_common_commits(void)\n-- \ngitgitgadget\n\n"},{"id":"352608","messageId":"pull.10.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":null,"subject":"[PATCH 00/16] Consolidate reachability logic","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-16T13:00:04Z","receivedAt":"2018-07-16T13:00:18Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"There are many places in Git that use a commit walk to determine\nreachability between commits and/or refs. A lot of this logic is\nduplicated.\n\nI wanted to achieve the following:\n\n1. Consolidate several different commit walks into one file\n2. Reduce duplicate reachability logic\n3. Increase testability (correctness and performance)\n4. Improve performance of reachability queries\n\nMy approach is mostly in three parts:\n\n I. Move code to a new commit-reach.c file.\n II. Add a 'test-tool reach' command to test these methods directly.\nIII. Modify the logic by improving performance and calling methods with\n similar logic but different prototypes.\n\nThe 'test-tool reach' command is helpful to make sure I don't break\nanything as I change the logic, but also so I can test methods that are\nnormally only exposed by other more complicated commands. For instance,\nref_newer() is part of 'git push -f' and ok_to_give_up() is buried deep\nwithin fetch negotiation. Both of these methods have some problematic\nperformance issues that are corrected by this series. As I discovered\nthem, it was clear that it would be better to consolidate walk logic\ninstead of discovering a new walk in another file hidden somewhere.\n\nFor the ok_to_give_up() method, I refactored the method so I could pull\nthe logic out of the depths of fetch negotiation. In the commit\n\"commit-reach: make can_all_from_reach... linear\" I discuss how the\nexisting algorithm is quadratic and how we can make it linear. Also, we\ncan use heuristic knowledge about the shape of the commit graph and the\nusual haves/wants to get some extra performance bonus. (The heuristic is\nto do a DFS with first-parents first, and stop on first found result. We\nexpect haves/wants to include ref tips, which typically have their\nprevious values in their first-parent history.)\n\nOne major difference in this series versus the RFC is that I added a new\nmethod 'generation_numbers_enabled()' to detect if we have a commit-graph\nfile with non-zero generation numbers. Using can_all_from_reach in\nis_descendant_of is only faster if we have generation numbers as a cutoff.\n\nThanks,\n-Stolee\n\nThis series is based on jt/commit-graph-per-object-store\n\nCC: sbeller@google.com\n\nDerrick Stolee (16):\n commit-reach: move walk methods from commit.c\n commit-reach: move ref_newer from remote.c\n commit-reach: move commit_contains from ref-filter\n upload-pack: make reachable() more generic\n upload-pack: refactor ok_to_give_up()\n upload-pack: generalize commit date cutoff\n commit-reach: move can_all_from_reach_with_flags\n test-reach: create new test tool for ref_newer\n test-reach: test in_merge_bases\n test-reach: test is_descendant_of\n test-reach: test get_merge_bases_many\n test-reach: test reduce_heads\n test-reach: test can_all_from_reach_with_flags\n commit-reach: replace ref_newer logic\n commit-reach: make can_all_from_reach... linear\n commit-reach: use can_all_from_reach\n\n Makefile | 2 +\n builtin/remote.c | 1 +\n commit-graph.c | 18 ++\n commit-graph.h | 6 +\n commit-reach.c | 662 ++++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 76 +++++\n commit.c | 358 -----------------------\n fast-import.c | 1 +\n http-push.c | 1 +\n ref-filter.c | 147 +---------\n remote.c | 50 +---\n remote.h | 1 -\n t/helper/test-reach.c | 104 +++++++\n t/helper/test-tool.c | 1 +\n t/helper/test-tool.h | 1 +\n t/t6600-test-reach.sh | 208 +++++++++++++\n upload-pack.c | 58 +---\n 17 files changed, 1095 insertions(+), 600 deletions(-)\n create mode 100644 commit-reach.c\n create mode 100644 commit-reach.h\n create mode 100644 t/helper/test-reach.c\n create mode 100755 t/t6600-test-reach.sh\n\n\nbase-commit: 596e28576ef3ca69432dbe5953b7bdcd18a32876\nPublished-As: https://github.com/gitgitgadget/git/releases/tags/pr-10%2Fderrickstolee%2Freach%2Frefactor-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-10/derrickstolee/reach/refactor-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/10\n-- \ngitgitgadget\n"},{"id":"352614","messageId":"74b7352b580ae920d9d5f206b9389e207aada4fe.1531746010.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 06/16] upload-pack: generalize commit date cutoff","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-12T20:47:52Z","receivedAt":"2018-07-16T13:00:26Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe ok_to_give_up() method uses the commit date as a cutoff to avoid\nwalking the entire reachble set of commits. Before moving the\nreachable() method to commit-reach.c, pull out the dependence on the\nglobal constant 'oldest_have' with a 'min_commit_date' parameter.\n\nSigned-off-by: Derrick Stolee \n---\n upload-pack.c | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex e7ad7f24b..96298e50e 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -336,7 +336,8 @@ static int got_oid(const char *hex, struct object_id *oid)\n \treturn 0;\n }\n \n-static int reachable(struct commit *from, int with_flag, int assign_flag)\n+static int reachable(struct commit *from, int with_flag, int assign_flag,\n+\t\t time_t min_commit_date)\n {\n \tstruct prio_queue work = { compare_commits_by_commit_date };\n \n@@ -354,7 +355,7 @@ static int reachable(struct commit *from, int with_flag, int assign_flag)\n \t\tif (commit->object.flags & REACHABLE)\n \t\t\tcontinue;\n \t\tcommit->object.flags |= REACHABLE;\n-\t\tif (commit->date < oldest_have)\n+\t\tif (commit->date < min_commit_date)\n \t\t\tcontinue;\n \t\tfor (list = commit->parents; list; list = list->next) {\n \t\t\tstruct commit *parent = list->item;\n@@ -371,10 +372,12 @@ static int reachable(struct commit *from, int with_flag, int assign_flag)\n /*\n * Determine if every commit in 'from' can reach at least one commit\n * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n- * as a marker for commits that are already visited.\n+ * as a marker for commits that are already visited. Do not walk\n+ * commits with date below 'min_commit_date'.\n */\n static int can_all_from_reach_with_flag(struct object_array *from,\n-\t\t\t\t\tint with_flag, int assign_flag)\n+\t\t\t\t\tint with_flag, int assign_flag,\n+\t\t\t\t\ttime_t min_commit_date)\n {\n \tint i;\n \n@@ -393,7 +396,8 @@ static int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\tfrom->objects[i].item->flags |= assign_flag;\n \t\t\tcontinue;\n \t\t}\n-\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag))\n+\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag,\n+\t\t\t min_commit_date))\n \t\t\treturn 0;\n \t}\n \treturn 1;\n@@ -404,7 +408,8 @@ static int ok_to_give_up(void)\n \tif (!have_obj.nr)\n \t\treturn 0;\n \n-\treturn can_all_from_reach_with_flag(&want_obj, THEY_HAVE, COMMON_KNOWN);\n+\treturn can_all_from_reach_with_flag(&want_obj, THEY_HAVE,\n+\t\t\t\t\t COMMON_KNOWN, oldest_have);\n }\n \n static int get_common_commits(void)\n-- \ngitgitgadget\n\n"},{"id":"352616","messageId":"cf4559e08c47adcaf4c938a325acf829a5759285.1531746011.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 07/16] commit-reach: move can_all_from_reach_with_flags","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-12T20:52:23Z","receivedAt":"2018-07-16T13:00:27Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 62 +++++++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 13 ++++++++++\n upload-pack.c | 69 +-------------------------------------------------\n 3 files changed, 76 insertions(+), 68 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 9e56f90ea..e07ff993f 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -9,6 +9,7 @@\n #include \"commit-reach.h\"\n \n /* Remember to update object flag allocation in object.h */\n+#define REACHABLE (1u<<15)\n #define PARENT1\t\t(1u<<16)\n #define PARENT2\t\t(1u<<17)\n #define STALE\t\t(1u<<18)\n@@ -531,3 +532,64 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n \t\treturn contains_tag_algo(commit, list, cache) == CONTAINS_YES;\n \treturn is_descendant_of(commit, list);\n }\n+\n+int reachable(struct commit *from, int with_flag, int assign_flag,\n+\t time_t min_commit_date)\n+{\n+\tstruct prio_queue work = { compare_commits_by_commit_date };\n+\n+\tprio_queue_put(&work, from);\n+\twhile (work.nr) {\n+\t\tstruct commit_list *list;\n+\t\tstruct commit *commit = prio_queue_get(&work);\n+\n+\t\tif (commit->object.flags & with_flag) {\n+\t\t\tfrom->object.flags |= assign_flag;\n+\t\t\tbreak;\n+\t\t}\n+\t\tif (!commit->object.parsed)\n+\t\t\tparse_object(the_repository, &commit->object.oid);\n+\t\tif (commit->object.flags & REACHABLE)\n+\t\t\tcontinue;\n+\t\tcommit->object.flags |= REACHABLE;\n+\t\tif (commit->date < min_commit_date)\n+\t\t\tcontinue;\n+\t\tfor (list = commit->parents; list; list = list->next) {\n+\t\t\tstruct commit *parent = list->item;\n+\t\t\tif (!(parent->object.flags & REACHABLE))\n+\t\t\t\tprio_queue_put(&work, parent);\n+\t\t}\n+\t}\n+\tfrom->object.flags |= REACHABLE;\n+\tclear_commit_marks(from, REACHABLE);\n+\tclear_prio_queue(&work);\n+\treturn (from->object.flags & assign_flag);\n+}\n+\n+int can_all_from_reach_with_flag(struct object_array *from,\n+\t\t\t\t int with_flag, int assign_flag,\n+\t\t\t\t time_t min_commit_date)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i < from->nr; i++) {\n+\t\tstruct object *from_one = from->objects[i].item;\n+\n+\t\tif (from_one->flags & assign_flag)\n+\t\t\tcontinue;\n+\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n+\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n+\t\t\t/* no way to tell if this is reachable by\n+\t\t\t * looking at the ancestry chain alone, so\n+\t\t\t * leave a note to ourselves not to worry about\n+\t\t\t * this object anymore.\n+\t\t\t */\n+\t\t\tfrom->objects[i].item->flags |= assign_flag;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag,\n+\t\t\t min_commit_date))\n+\t\t\treturn 0;\n+\t}\n+\treturn 1;\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nindex 925cb05d7..ce3b963e6 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -56,4 +56,17 @@ define_commit_slab(contains_cache, enum contains_result);\n int commit_contains(struct ref_filter *filter, struct commit *commit,\n \t\t struct commit_list *list, struct contains_cache *cache);\n \n+int reachable(struct commit *from, int with_flag, int assign_flag,\n+\t time_t min_commit_date);\n+\n+/*\n+ * Determine if every commit in 'from' can reach at least one commit\n+ * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n+ * as a marker for commits that are already visited. Do not walk\n+ * commits with date below 'min_commit_date'.\n+ */\n+int can_all_from_reach_with_flag(struct object_array *from,\n+\t\t\t\t int with_flag, int assign_flag,\n+\t\t\t\t time_t min_commit_date);\n+\n #endif\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 96298e50e..11c426685 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -24,13 +24,13 @@\n #include \"quote.h\"\n #include \"upload-pack.h\"\n #include \"serve.h\"\n+#include \"commit-reach.h\"\n \n /* Remember to update object flag allocation in object.h */\n #define THEY_HAVE\t(1u << 11)\n #define OUR_REF\t\t(1u << 12)\n #define WANTED\t\t(1u << 13)\n #define COMMON_KNOWN\t(1u << 14)\n-#define REACHABLE\t(1u << 15)\n \n #define SHALLOW\t\t(1u << 16)\n #define NOT_SHALLOW\t(1u << 17)\n@@ -336,73 +336,6 @@ static int got_oid(const char *hex, struct object_id *oid)\n \treturn 0;\n }\n \n-static int reachable(struct commit *from, int with_flag, int assign_flag,\n-\t\t time_t min_commit_date)\n-{\n-\tstruct prio_queue work = { compare_commits_by_commit_date };\n-\n-\tprio_queue_put(&work, from);\n-\twhile (work.nr) {\n-\t\tstruct commit_list *list;\n-\t\tstruct commit *commit = prio_queue_get(&work);\n-\n-\t\tif (commit->object.flags & with_flag) {\n-\t\t\tfrom->object.flags |= assign_flag;\n-\t\t\tbreak;\n-\t\t}\n-\t\tif (!commit->object.parsed)\n-\t\t\tparse_object(the_repository, &commit->object.oid);\n-\t\tif (commit->object.flags & REACHABLE)\n-\t\t\tcontinue;\n-\t\tcommit->object.flags |= REACHABLE;\n-\t\tif (commit->date < min_commit_date)\n-\t\t\tcontinue;\n-\t\tfor (list = commit->parents; list; list = list->next) {\n-\t\t\tstruct commit *parent = list->item;\n-\t\t\tif (!(parent->object.flags & REACHABLE))\n-\t\t\t\tprio_queue_put(&work, parent);\n-\t\t}\n-\t}\n-\tfrom->object.flags |= REACHABLE;\n-\tclear_commit_marks(from, REACHABLE);\n-\tclear_prio_queue(&work);\n-\treturn (from->object.flags & assign_flag);\n-}\n-\n-/*\n- * Determine if every commit in 'from' can reach at least one commit\n- * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n- * as a marker for commits that are already visited. Do not walk\n- * commits with date below 'min_commit_date'.\n- */\n-static int can_all_from_reach_with_flag(struct object_array *from,\n-\t\t\t\t\tint with_flag, int assign_flag,\n-\t\t\t\t\ttime_t min_commit_date)\n-{\n-\tint i;\n-\n-\tfor (i = 0; i < from->nr; i++) {\n-\t\tstruct object *from_one = from->objects[i].item;\n-\n-\t\tif (from_one->flags & assign_flag)\n-\t\t\tcontinue;\n-\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n-\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n-\t\t\t/* no way to tell if this is reachable by\n-\t\t\t * looking at the ancestry chain alone, so\n-\t\t\t * leave a note to ourselves not to worry about\n-\t\t\t * this object anymore.\n-\t\t\t */\n-\t\t\tfrom->objects[i].item->flags |= assign_flag;\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag,\n-\t\t\t min_commit_date))\n-\t\t\treturn 0;\n-\t}\n-\treturn 1;\n-}\n-\n static int ok_to_give_up(void)\n {\n \tif (!have_obj.nr)\n-- \ngitgitgadget\n\n"},{"id":"352615","messageId":"6acc5e804ea7beabc7c813e4c1c067915a16087b.1531746011.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 08/16] test-reach: create new test tool for ref_newer","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-13T14:06:29Z","receivedAt":"2018-07-16T13:00:29Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nAs we prepare to change the behavior of the algorithms in\ncommit-reach.c, create a new test-tool subcommand 'reach' to test these\nmethods on interesting commit-graph shapes.\n\nTo use the new test-tool, use 'test-tool reach ' and provide\ninput to stdin that describes the inputs to the method. Currently, we\nonly implement the ref_newer method, which requires two commits. Use\nlines \"A:\" and \"B:\" for the two inputs. We will\nexpand this input later to accommodate methods that take lists of\ncommits.\n\nThe test t6600-test-reach.sh creates a repo whose commits form a\ntwo-dimensional grid. This grid makes it easy for us to determine\nreachability because commit-A-B can reach commit-X-Y if and only if A is\nat least X and B is at least Y. This helps create interesting test cases\nfor each result of the methods in commit-reach.c.\n\nWe test all methods in three different states of the commit-graph file:\nNon-existent (no generation numbers), fully computed, and mixed (some\ncommits have generation numbers and others do not).\n\nSigned-off-by: Derrick Stolee \n---\n Makefile | 1 +\n t/helper/test-reach.c | 62 +++++++++++++++++++++++++++++++\n t/helper/test-tool.c | 1 +\n t/helper/test-tool.h | 1 +\n t/t6600-test-reach.sh | 86 +++++++++++++++++++++++++++++++++++++++++++\n 5 files changed, 151 insertions(+)\n create mode 100644 t/helper/test-reach.c\n create mode 100755 t/t6600-test-reach.sh\n\ndiff --git a/Makefile b/Makefile\nindex 59781f4bc..d69f9d415 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -716,6 +716,7 @@ TEST_BUILTINS_OBJS += test-mktemp.o\n TEST_BUILTINS_OBJS += test-online-cpus.o\n TEST_BUILTINS_OBJS += test-path-utils.o\n TEST_BUILTINS_OBJS += test-prio-queue.o\n+TEST_BUILTINS_OBJS += test-reach.o\n TEST_BUILTINS_OBJS += test-read-cache.o\n TEST_BUILTINS_OBJS += test-ref-store.o\n TEST_BUILTINS_OBJS += test-regex.o\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nnew file mode 100644\nindex 000000000..8cc570f3b\n--- /dev/null\n+++ b/t/helper/test-reach.c\n@@ -0,0 +1,62 @@\n+#include \"test-tool.h\"\n+#include \"cache.h\"\n+#include \"commit-reach.h\"\n+#include \"config.h\"\n+#include \"parse-options.h\"\n+#include \"tag.h\"\n+\n+int cmd__reach(int ac, const char **av)\n+{\n+\tstruct object_id oid_A, oid_B;\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct repository *r = the_repository;\n+\n+\tsetup_git_directory();\n+\n+\tif (ac < 2)\n+\t\texit(1);\n+\n+\n+\twhile (strbuf_getline(&buf, stdin) != EOF) {\n+\t\tstruct object_id oid;\n+\t\tstruct object *o;\n+\t\tstruct commit *c;\n+\t\tif (buf.len < 3)\n+\t\t\tcontinue;\n+\n+\t\tif (get_oid_committish(buf.buf + 2, &oid))\n+\t\t\tdie(\"failed to resolve %s\", buf.buf + 2);\n+\n+\t\to = parse_object(r, &oid);\n+\t\to = deref_tag_noverify(o);\n+\n+\t\tif (!o)\n+\t\t\tdie(\"failed to load commit for input %s resulting in oid %s\\n\",\n+\t\t\t buf.buf, oid_to_hex(&oid));\n+\n+\t\tc = object_as_type(r, o, OBJ_COMMIT, 0);\n+\n+\t\tif (!c)\n+\t\t\tdie(\"failed to load commit for input %s resulting in oid %s\\n\",\n+\t\t\t buf.buf, oid_to_hex(&oid));\n+\n+\t\tswitch (buf.buf[0]) {\n+\t\t\tcase 'A':\n+\t\t\t\toidcpy(&oid_A, &oid);\n+\t\t\t\tbreak;\n+\n+\t\t\tcase 'B':\n+\t\t\t\toidcpy(&oid_B, &oid);\n+\t\t\t\tbreak;\n+\n+\t\t\tdefault:\n+\t\t\t\tdie(\"unexpected start of line: %c\", buf.buf[0]);\n+\t\t}\n+\t}\n+\tstrbuf_release(&buf);\n+\n+\tif (!strcmp(av[1], \"ref_newer\"))\n+\t\tprintf(\"%s:%d\\n\", av[1], ref_newer(&oid_A, &oid_B));\n+\n+\texit(0);\n+}\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex dafc91c24..582d02adf 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -26,6 +26,7 @@ static struct test_cmd cmds[] = {\n \t{ \"online-cpus\", cmd__online_cpus },\n \t{ \"path-utils\", cmd__path_utils },\n \t{ \"prio-queue\", cmd__prio_queue },\n+\t{ \"reach\", cmd__reach },\n \t{ \"read-cache\", cmd__read_cache },\n \t{ \"ref-store\", cmd__ref_store },\n \t{ \"regex\", cmd__regex },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex 80cbcf085..a7e53c420 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -20,6 +20,7 @@ int cmd__mktemp(int argc, const char **argv);\n int cmd__online_cpus(int argc, const char **argv);\n int cmd__path_utils(int argc, const char **argv);\n int cmd__prio_queue(int argc, const char **argv);\n+int cmd__reach(int argc, const char **argv);\n int cmd__read_cache(int argc, const char **argv);\n int cmd__ref_store(int argc, const char **argv);\n int cmd__regex(int argc, const char **argv);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nnew file mode 100755\nindex 000000000..4ffe0174d\n--- /dev/null\n+++ b/t/t6600-test-reach.sh\n@@ -0,0 +1,86 @@\n+#!/bin/sh\n+\n+test_description='basic commit reachability tests'\n+\n+. ./test-lib.sh\n+\n+# Construct a grid-like commit graph with points (x,y)\n+# with 1 <= x <= 10, 1 <= y <= 10, where (x,y) has\n+# parents (x-1, y) and (x, y-1), keeping in mind that\n+# we drop a parent if a coordinate is nonpositive.\n+#\n+# (10,10)\n+# / \\\n+# (10,9) (9,10)\n+# / \\ / \\\n+# (10,8) (9,9) (8,10)\n+# / \\ / \\ / \\\n+# ( continued...)\n+# \\ / \\ / \\ /\n+# (3,1) (2,2) (1,3)\n+# \\ / \\ /\n+# (2,1) (2,1)\n+# \\ /\n+# (1,1)\n+#\n+# We use branch 'comit-x-y' to refer to (x,y).\n+# This grid allows interesting reachability and\n+# non-reachability queries: (x,y) can reach (x',y')\n+# if and only if x' <= x and y' <= y.\n+test_expect_success 'setup' '\n+\tfor i in $(test_seq 1 10)\n+\tdo\n+\t\ttest_commit \"1-$i\" &&\n+\t\tgit branch -f commit-1-$i\n+\tdone &&\n+\tfor j in $(test_seq 1 9)\n+\tdo\n+\t\tgit reset --hard commit-$j-1 &&\n+\t\tx=$(($j + 1)) &&\n+\t\ttest_commit \"$x-1\" &&\n+\t\tgit branch -f commit-$x-1 &&\n+\n+\t\tfor i in $(test_seq 2 10)\n+\t\tdo\n+\t\t\tgit merge commit-$j-$i -m \"$x-$i\" &&\n+\t\t\tgit branch -f commit-$x-$i\n+\t\tdone\n+\tdone &&\n+\tgit commit-graph write --reachable &&\n+\tmv .git/objects/info/commit-graph commit-graph-full &&\n+\tgit show-ref -s commit-7-7 | git commit-graph write --stdin-commits &&\n+\tmv .git/objects/info/commit-graph commit-graph-half &&\n+\tgit config core.commitGraph true\n+'\n+\n+test_three_modes () {\n+\ttest_when_finished rm -rf .git/objects/info/commit-graph &&\n+\ttest-tool reach $1 actual &&\n+\ttest_cmp expect actual &&\n+\tcp commit-graph-full .git/objects/info/commit-graph &&\n+\ttest-tool reach $1 actual &&\n+\ttest_cmp expect actual &&\n+\tcp commit-graph-half .git/objects/info/commit-graph &&\n+\ttest-tool reach $1 actual &&\n+\ttest_cmp expect actual\n+}\n+\n+test_expect_success 'ref_newer:miss' '\n+\tcat >input <<-\\EOF &&\n+\t\tA:commit-5-7\n+\t\tB:commit-4-9\n+\tEOF\n+\tprintf \"ref_newer:0\\n\" >expect &&\n+\ttest_three_modes ref_newer\n+'\n+\n+test_expect_success 'ref_newer:hit' '\n+\tcat >input <<-\\EOF &&\n+\t\tA:commit-5-7\n+\t\tB:commit-2-3\n+\tEOF\n+\tprintf \"ref_newer:1\\n\" >expect &&\n+\ttest_three_modes ref_newer\n+'\n+\n+test_done\n-- \ngitgitgadget\n\n"},{"id":"352617","messageId":"dbe75e7d2192a493825ef34fecfecbc021e3d4fb.1531746011.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 09/16] test-reach: test in_merge_bases","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-13T14:28:05Z","receivedAt":"2018-07-16T13:00:30Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nSigned-off-by: Derrick Stolee \n---\n t/helper/test-reach.c | 6 ++++++\n t/t6600-test-reach.sh | 18 ++++++++++++++++++\n 2 files changed, 24 insertions(+)\n\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex 8cc570f3b..29104d41a 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -8,6 +8,7 @@\n int cmd__reach(int ac, const char **av)\n {\n \tstruct object_id oid_A, oid_B;\n+\tstruct commit *A, *B;\n \tstruct strbuf buf = STRBUF_INIT;\n \tstruct repository *r = the_repository;\n \n@@ -16,6 +17,7 @@ int cmd__reach(int ac, const char **av)\n \tif (ac < 2)\n \t\texit(1);\n \n+\tA = B = NULL;\n \n \twhile (strbuf_getline(&buf, stdin) != EOF) {\n \t\tstruct object_id oid;\n@@ -43,10 +45,12 @@ int cmd__reach(int ac, const char **av)\n \t\tswitch (buf.buf[0]) {\n \t\t\tcase 'A':\n \t\t\t\toidcpy(&oid_A, &oid);\n+\t\t\t\tA = c;\n \t\t\t\tbreak;\n \n \t\t\tcase 'B':\n \t\t\t\toidcpy(&oid_B, &oid);\n+\t\t\t\tB = c;\n \t\t\t\tbreak;\n \n \t\t\tdefault:\n@@ -57,6 +61,8 @@ int cmd__reach(int ac, const char **av)\n \n \tif (!strcmp(av[1], \"ref_newer\"))\n \t\tprintf(\"%s:%d\\n\", av[1], ref_newer(&oid_A, &oid_B));\n+\telse if (!strcmp(av[1], \"in_merge_bases\"))\n+\t\tprintf(\"%s(A,B):%d\\n\", av[1], in_merge_bases(A, B));\n \n \texit(0);\n }\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 4ffe0174d..d065f2602 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -83,4 +83,22 @@ test_expect_success 'ref_newer:hit' '\n \ttest_three_modes ref_newer\n '\n \n+test_expect_success 'in_merge_bases:hit' '\n+\tcat >input <<- EOF &&\n+\t\tA:commit-5-7\n+\t\tB:commit-8-8\n+\tEOF\n+\tprintf \"in_merge_bases(A,B):1\\n\" >expect &&\n+\ttest_three_modes in_merge_bases\n+'\n+\n+test_expect_success 'in_merge_bases:miss' '\n+\tcat >input <<- EOF &&\n+\t\tA:commit-6-8\n+\t\tB:commit-5-9\n+\tEOF\n+\tprintf \"in_merge_bases(A,B):0\\n\" >expect &&\n+\ttest_three_modes in_merge_bases\n+'\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"352618","messageId":"074a9e04c980fabf73d0d042f96fc2c13429f9b5.1531746011.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 10/16] test-reach: test is_descendant_of","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-13T14:38:44Z","receivedAt":"2018-07-16T13:00:32Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe is_descendant_of method takes a single commit as its first parameter\nand a list of commits as its second parameter. Extend the input of the\n'test-tool reach' command to take multiple lines of the form\n\"X:\" to construct a list of commits. Pass these to\nis_descendant_of and create tests that check each result.\n\nSigned-off-by: Derrick Stolee \n---\n t/helper/test-reach.c | 8 ++++++++\n t/t6600-test-reach.sh | 22 ++++++++++++++++++++++\n 2 files changed, 30 insertions(+)\n\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex 29104d41a..149e8f32c 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -9,6 +9,7 @@ int cmd__reach(int ac, const char **av)\n {\n \tstruct object_id oid_A, oid_B;\n \tstruct commit *A, *B;\n+\tstruct commit_list *X;\n \tstruct strbuf buf = STRBUF_INIT;\n \tstruct repository *r = the_repository;\n \n@@ -18,6 +19,7 @@ int cmd__reach(int ac, const char **av)\n \t\texit(1);\n \n \tA = B = NULL;\n+\tX = NULL;\n \n \twhile (strbuf_getline(&buf, stdin) != EOF) {\n \t\tstruct object_id oid;\n@@ -53,6 +55,10 @@ int cmd__reach(int ac, const char **av)\n \t\t\t\tB = c;\n \t\t\t\tbreak;\n \n+\t\t\tcase 'X':\n+\t\t\t\tcommit_list_insert(c, &X);\n+\t\t\t\tbreak;\n+\n \t\t\tdefault:\n \t\t\t\tdie(\"unexpected start of line: %c\", buf.buf[0]);\n \t\t}\n@@ -63,6 +69,8 @@ int cmd__reach(int ac, const char **av)\n \t\tprintf(\"%s:%d\\n\", av[1], ref_newer(&oid_A, &oid_B));\n \telse if (!strcmp(av[1], \"in_merge_bases\"))\n \t\tprintf(\"%s(A,B):%d\\n\", av[1], in_merge_bases(A, B));\n+\telse if (!strcmp(av[1], \"is_descendant_of\"))\n+\t\tprintf(\"%s(A,X):%d\\n\", av[1], is_descendant_of(A, X));\n \n \texit(0);\n }\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex d065f2602..99444e0f7 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -101,4 +101,26 @@ test_expect_success 'in_merge_bases:miss' '\n \ttest_three_modes in_merge_bases\n '\n \n+test_expect_success 'is_descendant_of:hit' '\n+\tcat >input <<-\\EOF &&\n+\t\tA:commit-5-7\n+\t\tX:commit-4-8\n+\t\tX:commit-6-6\n+\t\tX:commit-1-1\n+\tEOF\n+\tprintf \"is_descendant_of(A,X):1\\n\" >expect &&\n+\ttest_three_modes is_descendant_of\n+'\n+\n+test_expect_success 'is_descendant_of:miss' '\n+\tcat >input <<-\\EOF &&\n+\t\tA:commit-6-8\n+\t\tX:commit-5-9\n+\t\tX:commit-4-10\n+\t\tX:commit-7-6\n+\tEOF\n+\tprintf \"is_descendant_of(A,X):0\\n\" >expect &&\n+\ttest_three_modes is_descendant_of\n+'\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"352619","messageId":"7a3e3815c585dd3972e9bea6798a7664232157a3.1531746012.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 11/16] test-reach: test get_merge_bases_many","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-13T14:51:05Z","receivedAt":"2018-07-16T13:00:32Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe get_merge_bases_many method returns a list of merge bases for a\nsingle commit (A) against a list of commits (X). Some care is needed in\nconstructing the expected behavior because the result is not the\nexpected merge-base for an octopus merge with those parents but instead\nthe set of maximal commits that are reachable from A and at least one of\nthe commits in X.\n\nAdd get_merge_bases_many to 'test-tool reach' and create a test that\ndemonstrates that this output returns multiple results. Specifically, we\nselect a list of three commits such that we output two commits that are\nreachable from one of the first two, respectively, and none are\nreachable from the third.\n\nSigned-off-by: Derrick Stolee \n---\n t/helper/test-reach.c | 15 +++++++++++++++\n t/t6600-test-reach.sh | 15 +++++++++++++++\n 2 files changed, 30 insertions(+)\n\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex 149e8f32c..97c726040 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -10,6 +10,8 @@ int cmd__reach(int ac, const char **av)\n \tstruct object_id oid_A, oid_B;\n \tstruct commit *A, *B;\n \tstruct commit_list *X;\n+\tstruct commit **X_array;\n+\tint X_nr, X_alloc;\n \tstruct strbuf buf = STRBUF_INIT;\n \tstruct repository *r = the_repository;\n \n@@ -20,6 +22,9 @@ int cmd__reach(int ac, const char **av)\n \n \tA = B = NULL;\n \tX = NULL;\n+\tX_nr = 0;\n+\tX_alloc = 16;\n+\tALLOC_ARRAY(X_array, X_alloc);\n \n \twhile (strbuf_getline(&buf, stdin) != EOF) {\n \t\tstruct object_id oid;\n@@ -57,6 +62,8 @@ int cmd__reach(int ac, const char **av)\n \n \t\t\tcase 'X':\n \t\t\t\tcommit_list_insert(c, &X);\n+\t\t\t\tALLOC_GROW(X_array, X_nr + 1, X_alloc);\n+\t\t\t\tX_array[X_nr++] = c;\n \t\t\t\tbreak;\n \n \t\t\tdefault:\n@@ -71,6 +78,14 @@ int cmd__reach(int ac, const char **av)\n \t\tprintf(\"%s(A,B):%d\\n\", av[1], in_merge_bases(A, B));\n \telse if (!strcmp(av[1], \"is_descendant_of\"))\n \t\tprintf(\"%s(A,X):%d\\n\", av[1], is_descendant_of(A, X));\n+\telse if (!strcmp(av[1], \"get_merge_bases_many\")) {\n+\t\tstruct commit_list *list = get_merge_bases_many(A, X_nr, X_array);\n+\t\tprintf(\"%s(A,X):\\n\", av[1]);\n+\t\twhile (list) {\n+\t\t\tprintf(\"%s\\n\", oid_to_hex(&list->item->object.oid));\n+\t\t\tlist = list->next;\n+\t\t}\n+\t}\n \n \texit(0);\n }\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 99444e0f7..19e39ad7c 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -123,4 +123,19 @@ test_expect_success 'is_descendant_of:miss' '\n \ttest_three_modes is_descendant_of\n '\n \n+test_expect_success 'get_merge_bases_many' '\n+\tcat >input <<-\\EOF &&\n+\t\tA:commit-5-7\n+\t\tX:commit-4-8\n+\t\tX:commit-6-6\n+\t\tX:commit-8-3\n+\tEOF\n+\t{\n+\t\tprintf \"get_merge_bases_many(A,X):\\n\" &&\n+\t\tgit rev-parse commit-5-6 &&\n+\t\tgit rev-parse commit-4-7\n+\t} >expect &&\n+\ttest_three_modes get_merge_bases_many\n+'\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"352620","messageId":"3b0379cf6fb42ca5ee4e10cb20bf42ff333e1456.1531746012.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 12/16] test-reach: test reduce_heads","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-13T16:51:23Z","receivedAt":"2018-07-16T13:00:34Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nSigned-off-by: Derrick Stolee \n---\n t/helper/test-reach.c | 7 +++++++\n t/t6600-test-reach.sh | 22 ++++++++++++++++++++++\n 2 files changed, 29 insertions(+)\n\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex 97c726040..73cb55208 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -85,6 +85,13 @@ int cmd__reach(int ac, const char **av)\n \t\t\tprintf(\"%s\\n\", oid_to_hex(&list->item->object.oid));\n \t\t\tlist = list->next;\n \t\t}\n+\t} else if (!strcmp(av[1], \"reduce_heads\")) {\n+\t\tstruct commit_list *list = reduce_heads(X);\n+\t\tprintf(\"%s(X):\\n\", av[1]);\n+\t\twhile (list) {\n+\t\t\tprintf(\"%s\\n\", oid_to_hex(&list->item->object.oid));\n+\t\t\tlist = list->next;\n+\t\t}\n \t}\n \n \texit(0);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 19e39ad7c..e365a4ea6 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -138,4 +138,26 @@ test_expect_success 'get_merge_bases_many' '\n \ttest_three_modes get_merge_bases_many\n '\n \n+test_expect_success 'reduce_heads' '\n+\tcat >input <<-\\EOF &&\n+\t\tX:commit-1-10\n+\t\tX:commit-2-8\n+\t\tX:commit-3-6\n+\t\tX:commit-4-4\n+\t\tX:commit-1-7\n+\t\tX:commit-2-5\n+\t\tX:commit-3-3\n+\t\tX:commit-5-1\n+\tEOF\n+\t{\n+\t\tprintf \"reduce_heads(X):\\n\" &&\n+\t\tgit rev-parse commit-5-1 &&\n+\t\tgit rev-parse commit-4-4 &&\n+\t\tgit rev-parse commit-3-6 &&\n+\t\tgit rev-parse commit-2-8 &&\n+\t\tgit rev-parse commit-1-10\n+\t} >expect &&\n+\ttest_three_modes reduce_heads\n+'\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"352621","messageId":"95b11ced4c0355aabf458539a284bb3895d73f6f.1531746012.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 13/16] test-reach: test can_all_from_reach_with_flags","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-13T17:22:57Z","receivedAt":"2018-07-16T13:00:36Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe can_all_from_reach_with_flags method is used by ok_to_give_up in\nupload-pack.c to see if we have done enough negotiation during a fetch.\nThis method is intentionally created to preserve state between calls to\nassist with stateful negotiation, such as over SSH.\n\nTo make this method testable, add a new can_all_from_reach method that\ndoes the initial setup and final tear-down. Call the method from\n'test-tool reach'.\n\nSince this is a many-to-many reachability query, add a new type of input\nto the 'test-tool reach' input format. Lines \"Y:\" create a\nlist of commits to be the reachability targets from the commits in the\n'X' list. In the context of fetch negotiation, the 'X' commits are the\n'want' commits and the 'Y' commits are the 'have' commits.\n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 47 +++++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 2 ++\n t/helper/test-reach.c | 10 +++++++--\n t/t6600-test-reach.sh | 45 +++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 102 insertions(+), 2 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex e07ff993f..c5e89a2de 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -593,3 +593,50 @@ int can_all_from_reach_with_flag(struct object_array *from,\n \t}\n \treturn 1;\n }\n+\n+int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n+\t\t int cutoff_by_min_date)\n+{\n+\tstruct object_array from_objs = OBJECT_ARRAY_INIT;\n+\ttime_t min_commit_date = cutoff_by_min_date ? from->item->date : 0;\n+\tstruct commit_list *from_iter = from, *to_iter = to;\n+\tint result;\n+\n+\twhile (from_iter) {\n+\t\tadd_object_array(&from_iter->item->object, NULL, &from_objs);\n+\n+\t\tif (!parse_commit(from_iter->item)) {\n+\t\t\tif (from_iter->item->date < min_commit_date)\n+\t\t\t\tmin_commit_date = from_iter->item->date;\n+\t\t}\n+\n+\t\tfrom_iter = from_iter->next;\n+\t}\n+\n+\twhile (to_iter) {\n+\t\tif (!parse_commit(to_iter->item)) {\n+\t\t\tif (to_iter->item->date < min_commit_date)\n+\t\t\t\tmin_commit_date = to_iter->item->date;\n+\t\t}\n+\n+\t\tto_iter->item->object.flags |= PARENT2;\n+\n+\t\tto_iter = to_iter->next;\n+\t}\n+\n+\tresult = can_all_from_reach_with_flag(&from_objs, PARENT2, PARENT1,\n+\t\t\t\t\t min_commit_date);\n+\n+\twhile (from) {\n+\t\tclear_commit_marks(from->item, PARENT1);\n+\t\tfrom = from->next;\n+\t}\n+\n+\twhile (to) {\n+\t\tclear_commit_marks(to->item, PARENT2);\n+\t\tto = to->next;\n+\t}\n+\n+\tobject_array_clear(&from_objs);\n+\treturn result;\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nindex ce3b963e6..58de0df56 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -68,5 +68,7 @@ int reachable(struct commit *from, int with_flag, int assign_flag,\n int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\t int with_flag, int assign_flag,\n \t\t\t\t time_t min_commit_date);\n+int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n+\t\t int commit_date_cutoff);\n \n #endif\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex 73cb55208..dc9710099 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -9,7 +9,7 @@ int cmd__reach(int ac, const char **av)\n {\n \tstruct object_id oid_A, oid_B;\n \tstruct commit *A, *B;\n-\tstruct commit_list *X;\n+\tstruct commit_list *X, *Y;\n \tstruct commit **X_array;\n \tint X_nr, X_alloc;\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -21,7 +21,7 @@ int cmd__reach(int ac, const char **av)\n \t\texit(1);\n \n \tA = B = NULL;\n-\tX = NULL;\n+\tX = Y = NULL;\n \tX_nr = 0;\n \tX_alloc = 16;\n \tALLOC_ARRAY(X_array, X_alloc);\n@@ -66,6 +66,10 @@ int cmd__reach(int ac, const char **av)\n \t\t\t\tX_array[X_nr++] = c;\n \t\t\t\tbreak;\n \n+\t\t\tcase 'Y':\n+\t\t\t\tcommit_list_insert(c, &Y);\n+\t\t\t\tbreak;\n+\n \t\t\tdefault:\n \t\t\t\tdie(\"unexpected start of line: %c\", buf.buf[0]);\n \t\t}\n@@ -92,6 +96,8 @@ int cmd__reach(int ac, const char **av)\n \t\t\tprintf(\"%s\\n\", oid_to_hex(&list->item->object.oid));\n \t\t\tlist = list->next;\n \t\t}\n+\t} else if (!strcmp(av[1], \"can_all_from_reach\")) {\n+\t\tprintf(\"%s(X,Y):%d\\n\", av[1], can_all_from_reach(X, Y, 1));\n \t}\n \n \texit(0);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex e365a4ea6..cb07c6431 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -160,4 +160,49 @@ test_expect_success 'reduce_heads' '\n \ttest_three_modes reduce_heads\n '\n \n+test_expect_success 'can_all_from_reach:hit' '\n+\tcat >input <<-\\EOF &&\n+\t\tX:commit-2-10\n+\t\tX:commit-3-9\n+\t\tX:commit-4-8\n+\t\tX:commit-5-7\n+\t\tX:commit-6-6\n+\t\tX:commit-7-5\n+\t\tX:commit-8-4\n+\t\tX:commit-9-3\n+\t\tY:commit-1-9\n+\t\tY:commit-2-8\n+\t\tY:commit-3-7\n+\t\tY:commit-4-6\n+\t\tY:commit-5-5\n+\t\tY:commit-6-4\n+\t\tY:commit-7-3\n+\t\tY:commit-8-1\n+\tEOF\n+\tprintf \"can_all_from_reach(X,Y):1\\n\" >expect &&\n+\ttest_three_modes can_all_from_reach\n+'\n+\n+test_expect_success 'can_all_from_reach:miss' '\n+\tcat >input <<-\\EOF &&\n+\t\tX:commit-2-10\n+\t\tX:commit-3-9\n+\t\tX:commit-4-8\n+\t\tX:commit-5-7\n+\t\tX:commit-6-6\n+\t\tX:commit-7-5\n+\t\tX:commit-8-4\n+\t\tX:commit-9-3\n+\t\tY:commit-1-9\n+\t\tY:commit-2-8\n+\t\tY:commit-3-7\n+\t\tY:commit-4-6\n+\t\tY:commit-5-5\n+\t\tY:commit-6-4\n+\t\tY:commit-8-5\n+\tEOF\n+\tprintf \"can_all_from_reach(X,Y):0\\n\" >expect &&\n+\ttest_three_modes can_all_from_reach\n+'\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"352622","messageId":"1fd45ef2b15524c18322183231608fe13a733506.1531746012.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 14/16] commit-reach: replace ref_newer logic","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-13T18:37:32Z","receivedAt":"2018-07-16T13:00:37Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe ref_newer method is used by 'git push' to check if a force-push is\nrequired. This method does not use any kind of cutoff when walking, so\nin the case of a force-push will walk all reachable commits.\n\nThe is_descendant_of method already uses paint_down_to_common along with\ncutoffs. By translating the ref_newer arguments into the commit and\ncommit_list required by is_descendant_of, we can have one fewer commit\nwalk and also improve our performance!\n\nFor a copy of the Linux repository, 'test-tool reach ref_newer' presents\nthe following improvements with the specified input. In the case that\nref_newer returns 1, there is no improvement. The improvement is in the\nsecond case where ref_newer returns 0.\n\nInput\n-----\nA:v4.9\nB:v3.19\n\nBefore: 0.09 s\n After: 0.09 s\n\nTo test the negative case, add a new commit with parent v3.19,\nregenerate the commit-graph, and then run with B pointing at that\ncommit.\n\nBefore: 0.43 s\n After: 0.09 s\n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 26 +++-----------------------\n 1 file changed, 3 insertions(+), 23 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex c5e89a2de..c58e50fbb 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -365,20 +365,11 @@ void reduce_heads_replace(struct commit_list **heads)\n \t*heads = result;\n }\n \n-static void unmark_and_free(struct commit_list *list, unsigned int mark)\n-{\n-\twhile (list) {\n-\t\tstruct commit *commit = pop_commit(&list);\n-\t\tcommit->object.flags &= ~mark;\n-\t}\n-}\n-\n int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n {\n \tstruct object *o;\n \tstruct commit *old_commit, *new_commit;\n-\tstruct commit_list *list, *used;\n-\tint found = 0;\n+\tstruct commit_list *old_commit_list = NULL;\n \n \t/*\n \t * Both new_commit and old_commit must be commit-ish and new_commit is descendant of\n@@ -399,19 +390,8 @@ int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n \tif (parse_commit(new_commit) < 0)\n \t\treturn 0;\n \n-\tused = list = NULL;\n-\tcommit_list_insert(new_commit, &list);\n-\twhile (list) {\n-\t\tnew_commit = pop_most_recent_commit(&list, TMP_MARK);\n-\t\tcommit_list_insert(new_commit, &used);\n-\t\tif (new_commit == old_commit) {\n-\t\t\tfound = 1;\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tunmark_and_free(list, TMP_MARK);\n-\tunmark_and_free(used, TMP_MARK);\n-\treturn found;\n+\tcommit_list_insert(old_commit, &old_commit_list);\n+\treturn is_descendant_of(new_commit, old_commit_list);\n }\n \n /*\n-- \ngitgitgadget\n\n"},{"id":"352624","messageId":"a60b7011adfd107633d7d3158c9c3fa127dc1a61.1531746012.git.gitgitgadget@gmail.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH 16/16] commit-reach: use can_all_from_reach","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2018-07-13T19:25:08Z","receivedAt":"2018-07-16T13:00:40Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe is_descendant_of method previously used in_merge_bases() to check if\nthe commit can reach any of the commits in the provided list. This had\ntwo performance problems:\n\n1. The performance is quadratic in worst-case.\n\n2. A single in_merge_bases() call requires walking beyond the target\n commit in order to find the full set of boundary commits that may be\n merge-bases.\n\nThe can_all_from_reach method avoids this quadratic behavior and can\nlimit the search beyond the target commits using generation numbers. It\nrequires a small prototype adjustment to stop using commit-date as a\ncutoff, as that optimization is no longer appropriate here.\n\nSince in_merge_bases() uses paint_down_to_common(), is_descendant_of()\nnaturally found cutoffs to avoid walking the entire commit graph. Since\nwe want to always return the correct result, we cannot use the\nmin_commit_date cutoff in can_all_from_reach. We then rely on generation\nnumbers to provide the cutoff.\n\nSince not all repos will have a commit-graph file, nor will we always\nhave generation numbers computed for a commit-graph file, create a new\nmethod, generation_numbers_enabled(), that checks for a commit-graph\nfile and sees if the first commit in the file has a non-zero generation\nnumber. In the case that we do not have generation numbers, use the old\nlogic for is_descendant_of().\n\nPerformance was meausured on a copy of the Linux repository using the\n'test-tool reach is_descendant_of' command using this input:\n\nA:v4.9\nX:v4.10\nX:v4.11\nX:v4.12\nX:v4.13\nX:v4.14\nX:v4.15\nX:v4.16\nX:v4.17\nX.v3.0\n\nNote that this input is tailored to demonstrate the quadratic nature of\nthe previous method, as it will compute merge-bases for v4.9 versus all\nof the later versions before checking against v4.1.\n\nBefore: 0.26 s\n After: 0.21 s\n\nSince we previously used the is_descendant_of method in the ref_newer\nmethod, we also measured performance there using\n'test-tool reach ref_newer' with this input:\n\nA:v4.9\nB:v3.19\n\nBefore: 0.10 s\n After: 0.08 s\n\nBy adding a new commit with parent v3.19, we test the non-reachable case\nof ref_newer:\n\nBefore: 0.09 s\n After: 0.08 s\n\nSigned-off-by: Derrick Stolee \n---\n commit-graph.c | 18 ++++++++++++++++++\n commit-graph.h | 6 ++++++\n commit-reach.c | 24 +++++++++++++++++-------\n 3 files changed, 41 insertions(+), 7 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex b0a55ad12..e9786fa86 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -233,6 +233,24 @@ static int prepare_commit_graph(struct repository *r)\n \treturn !!r->objects->commit_graph;\n }\n \n+int generation_numbers_enabled(struct repository *r)\n+{\n+\tuint32_t first_generation;\n+\tstruct commit_graph *g;\n+\tif (!prepare_commit_graph(r))\n+\t return 0;\n+\n+\tg = r->objects->commit_graph;\n+\n+\tif (!g->num_commits)\n+\t\treturn 0;\n+\n+\tfirst_generation = get_be32(g->chunk_commit_data +\n+\t\t\t\t g->hash_len + 8) >> 2;\n+\n+\treturn !!first_generation;\n+}\n+\n static void close_commit_graph(void)\n {\n \tfree_commit_graph(the_repository->objects->commit_graph);\ndiff --git a/commit-graph.h b/commit-graph.h\nindex 76e098934..0de8f8831 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -51,6 +51,12 @@ struct commit_graph {\n \n struct commit_graph *load_commit_graph_one(const char *graph_file);\n \n+/*\n+ * Return 1 if and only if the repository has a commit-graph\n+ * file and generation numbers are computed in that file.\n+ */\n+int generation_numbers_enabled(struct repository *r);\n+\n void write_commit_graph_reachable(const char *obj_dir, int append);\n void write_commit_graph(const char *obj_dir,\n \t\t\tstruct string_list *pack_indexes,\ndiff --git a/commit-reach.c b/commit-reach.c\nindex ac132c8e4..9eb622540 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -276,15 +276,25 @@ int is_descendant_of(struct commit *commit, struct commit_list *with_commit)\n {\n \tif (!with_commit)\n \t\treturn 1;\n-\twhile (with_commit) {\n-\t\tstruct commit *other;\n \n-\t\tother = with_commit->item;\n-\t\twith_commit = with_commit->next;\n-\t\tif (in_merge_bases(other, commit))\n-\t\t\treturn 1;\n+\tif (generation_numbers_enabled(the_repository)) {\n+\t\tstruct commit_list *from_list = NULL;\n+\t\tint result;\n+\t\tcommit_list_insert(commit, &from_list);\n+\t\tresult = can_all_from_reach(from_list, with_commit, 0);\n+\t\tfree_commit_list(from_list);\n+\t\treturn result;\n+\t} else {\n+\t\twhile (with_commit) {\n+\t\t\tstruct commit *other;\n+\n+\t\t\tother = with_commit->item;\n+\t\t\twith_commit = with_commit->next;\n+\t\t\tif (in_merge_bases(other, commit))\n+\t\t\t\treturn 1;\n+\t\t}\n+\t\treturn 0;\n \t}\n-\treturn 0;\n }\n \n /*\n-- \ngitgitgadget\n"},{"id":"352627","messageId":"6b7a50dc-6b71-0e31-030a-42dd1b26bde4@ramsayjones.plus.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2018-07-16T13:54:38Z","receivedAt":"2018-07-16T13:54:42Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 16/07/18 14:00, Derrick Stolee via GitGitGadget wrote:\n> There are many places in Git that use a commit walk to determine\n> reachability between commits and/or refs. A lot of this logic is\n> duplicated.\n[snip] ...\n\nThis is not your problem, but I find these GitGitGadget\nsubmissions somewhat annoying. This series has been spewed\nall over my in-box in, what I assume, is commit date order.\n\nSo, patches #4,5 dated 19/06, then #1,2,3 dated 25/06,\nthen #15 dated 28/06, then #6,7 dated 12/07, then #8-16\ndated 13/07, then 00/16 dated today.\n\nNo I don't use a threaded display (I hate it), be even with\nthat turned on, the patches still appear in the above order\nunder the cover letter (but at least all together).\n\nAnnoyed.\n\nATB,\nRamsay Jones\n\n"},{"id":"352633","messageId":"20180716161821.GB18150@sigill.intra.peff.net","threadId":"48889","inReplyTo":"6b7a50dc-6b71-0e31-030a-42dd1b26bde4@ramsayjones.plus.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-07-16T16:18:22Z","receivedAt":"2018-07-16T16:18:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 16, 2018 at 02:54:38PM +0100, Ramsay Jones wrote:\n\n> On 16/07/18 14:00, Derrick Stolee via GitGitGadget wrote:\n> > There are many places in Git that use a commit walk to determine\n> > reachability between commits and/or refs. A lot of this logic is\n> > duplicated.\n> [snip] ...\n> \n> This is not your problem, but I find these GitGitGadget\n> submissions somewhat annoying. This series has been spewed\n> all over my in-box in, what I assume, is commit date order.\n> \n> So, patches #4,5 dated 19/06, then #1,2,3 dated 25/06,\n> then #15 dated 28/06, then #6,7 dated 12/07, then #8-16\n> dated 13/07, then 00/16 dated today.\n> \n> No I don't use a threaded display (I hate it), be even with\n> that turned on, the patches still appear in the above order\n> under the cover letter (but at least all together).\n\nYeah, they're out of order in mutt's threaded display. And the\nback-dating means there's a much higher chance of them getting blocked\nas spam (e.g., some of the dates are from weeks ago).\n\ngit-send-email uses the current time minus an offset, and then\nmonotonically increases for each patch:\n\n $time = time - scalar $#files;\n ...\n my $date = format_2822_time($time++);\n\nwhich seems to work pretty well in practice. It does mean the original\ndates are lost. The committer date is not interesting at all (there will\nbe a new committer via \"git am\" anyway). The original author date is\npotentially of interest, but could be included as an in-body header.\nAFAIK send-email doesn't have such an option, though, and people are\nfine with date-of-sending becoming the new author date.\n\n+cc Johannes as the GitGitGadget author\n\n-Peff\n"},{"id":"352634","messageId":"CAGZ79kYem+uMrhv+CUSDNXtE3C8Pv1cx=aZOyBLG=uo-aQWXeA@mail.gmail.com","threadId":"48889","inReplyTo":"6b7a50dc-6b71-0e31-030a-42dd1b26bde4@ramsayjones.plus.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T17:26:42Z","receivedAt":"2018-07-16T17:26:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"> This is not your problem, but I find these GitGitGadget\n> submissions somewhat annoying.\n\nAnother pain point of the Gadget is that CC's in the cover letter\ndo not work as I would imagine. The line\n\nCC: sbeller@google.com\n\ndid not put that email into the cc field.\n\nHow did you get Junios email into the TO, though?\n\nAnyway I'll have a look at this series.\n\nThanks,\nStefan\n"},{"id":"352645","messageId":"CAPig+cTVP2kubTEZBms_x1FvnqjGcX3pjTRXfjBMfMtdpbJt=Q@mail.gmail.com","threadId":"48889","inReplyTo":"20180716161821.GB18150@sigill.intra.peff.net","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-07-16T18:40:21Z","receivedAt":"2018-07-16T18:40:34Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 16, 2018 at 12:18 PM Jeff King wrote:\n> On Mon, Jul 16, 2018 at 02:54:38PM +0100, Ramsay Jones wrote:\n> > This is not your problem, but I find these GitGitGadget\n> > submissions somewhat annoying. This series has been spewed\n> > all over my in-box in, what I assume, is commit date order.\n> >\n> > So, patches #4,5 dated 19/06, then #1,2,3 dated 25/06,\n> > then #15 dated 28/06, then #6,7 dated 12/07, then #8-16\n> > dated 13/07, then 00/16 dated today.\n>\n> Yeah, they're out of order in mutt's threaded display. And the\n> back-dating means there's a much higher chance of them getting blocked\n> as spam (e.g., some of the dates are from weeks ago).\n>\n> git-send-email uses the current time minus an offset, and then\n> monotonically increases for each patch:\n\nJunio pointed this out to gitgitgadget developers in [1], which led to\nan issue being opened[2]. That issue was merged today.\n\n[1]: https://public-inbox.org/git/xmqq7em7gg3j.fsf@gitster-ct.c.googlers.com/\n[2]: https://github.com/gitgitgadget/gitgitgadget/pull/15\n"},{"id":"352647","messageId":"CAPig+cQj5_bEiLyAAS4xnsUQEPzqT_2yerdAQxd8FidVpJ5CxA@mail.gmail.com","threadId":"48889","inReplyTo":"CAGZ79kYem+uMrhv+CUSDNXtE3C8Pv1cx=aZOyBLG=uo-aQWXeA@mail.gmail.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-07-16T18:44:22Z","receivedAt":"2018-07-16T18:44:35Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 16, 2018 at 1:27 PM Stefan Beller wrote:\n> Another pain point of the Gadget is that CC's in the cover letter\n> do not work as I would imagine. The line\n>\n> CC: sbeller@google.com\n>\n> did not put that email into the cc field.\n\ngitgitgadget recognizes case-sensitive \"Cc:\" only[1].\n\n[1]: https://github.com/gitgitgadget/gitgitgadget/blob/c4805370f59532aa438283431b8ea7d4484c530f/lib/patch-series.ts#L188\n"},{"id":"352648","messageId":"65d51c26-e248-f16d-bac2-e6ba0a3be8e5@gmail.com","threadId":"48889","inReplyTo":"CAPig+cQj5_bEiLyAAS4xnsUQEPzqT_2yerdAQxd8FidVpJ5CxA@mail.gmail.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-07-16T18:47:19Z","receivedAt":"2018-07-16T18:47:24Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/16/2018 2:44 PM, Eric Sunshine wrote:\n> On Mon, Jul 16, 2018 at 1:27 PM Stefan Beller wrote:\n>> Another pain point of the Gadget is that CC's in the cover letter\n>> do not work as I would imagine. The line\n>>\n>> CC: sbeller@google.com\n>>\n>> did not put that email into the cc field.\n> gitgitgadget recognizes case-sensitive \"Cc:\" only[1].\n>\n> [1]: https://github.com/gitgitgadget/gitgitgadget/blob/c4805370f59532aa438283431b8ea7d4484c530f/lib/patch-series.ts#L188\n\nThanks for everyone's patience while we improve gitgitgadget (and - in \nthis case - I learn how to use it).\n\nThanks,\n\n-Stolee\n\n"},{"id":"352652","messageId":"20180716185618.GD22298@sigill.intra.peff.net","threadId":"48889","inReplyTo":"CAPig+cTVP2kubTEZBms_x1FvnqjGcX3pjTRXfjBMfMtdpbJt=Q@mail.gmail.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-07-16T18:56:18Z","receivedAt":"2018-07-16T18:56:22Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 16, 2018 at 02:40:21PM -0400, Eric Sunshine wrote:\n\n> On Mon, Jul 16, 2018 at 12:18 PM Jeff King wrote:\n> > On Mon, Jul 16, 2018 at 02:54:38PM +0100, Ramsay Jones wrote:\n> > > This is not your problem, but I find these GitGitGadget\n> > > submissions somewhat annoying. This series has been spewed\n> > > all over my in-box in, what I assume, is commit date order.\n> > >\n> > > So, patches #4,5 dated 19/06, then #1,2,3 dated 25/06,\n> > > then #15 dated 28/06, then #6,7 dated 12/07, then #8-16\n> > > dated 13/07, then 00/16 dated today.\n> >\n> > Yeah, they're out of order in mutt's threaded display. And the\n> > back-dating means there's a much higher chance of them getting blocked\n> > as spam (e.g., some of the dates are from weeks ago).\n> >\n> > git-send-email uses the current time minus an offset, and then\n> > monotonically increases for each patch:\n> \n> Junio pointed this out to gitgitgadget developers in [1], which led to\n> an issue being opened[2]. That issue was merged today.\n> \n> [1]: https://public-inbox.org/git/xmqq7em7gg3j.fsf@gitster-ct.c.googlers.com/\n> [2]: https://github.com/gitgitgadget/gitgitgadget/pull/15\n\nI was going to say \"oh good, fixed\", but it looks like it just merged\nadding that line to the TODO list. :)\n\nStill, it looks like wheels are in motion, which is nice.\n\n-Peff\n"},{"id":"352653","messageId":"CAGZ79kYjshSeZFoxqn9eY8A6X5p+WSrpwV-YSj2kZ9MX2K2Yqg@mail.gmail.com","threadId":"48889","inReplyTo":"01bef3d654845236e9115ebc33d80673b7b4a5a2.1531746009.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 01/16] commit-reach: move walk methods from commit.c","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T18:57:57Z","receivedAt":"2018-07-16T18:58:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> Signed-off-by: Derrick Stolee \n\nThis looks good, apart from nits below.\n\nThanks,\nStefan\n\n> diff --git a/commit-reach.c b/commit-reach.c\n> new file mode 100644\n> index 000000000..f2e2f7461\n> --- /dev/null\n> +++ b/commit-reach.c\n> @@ -0,0 +1,359 @@\n> +#include \"cache.h\"\n> +#include \"prio-queue.h\"\n> +#include \"commit-reach.h\"\n\nand commit.h (see discussion below) ?\n\n> diff --git a/commit-reach.h b/commit-reach.h\n> new file mode 100644\n> index 000000000..244f48c5f\n> --- /dev/null\n> +++ b/commit-reach.h\n> @@ -0,0 +1,41 @@\n> +#ifndef __COMMIT_REACH_H__\n> +#define __COMMIT_REACH_H__\n> +\n> +#include \"commit.h\"\n\nDo we really need to include the header file in another header file?\nI'd think forward declarations would work fine here?\n(The benefit of forward declaring the structs commits, commit_list\nis purely for the poor saps of developers that we are, as then touching\ncommit.h would not trigger a compilation of files that only include this\nheader but not commit.h. That are not many in this particular case,\nbut I consider it good practice that we should follow)\n\n> +\n> +struct commit_list *get_merge_bases_many(struct commit *one,\n> + int n,\n> + struct commit **twos);\n> +struct commit_list *get_merge_bases_many_dirty(struct commit *one,\n> + int n,\n> + struct commit **twos);\n> +struct commit_list *get_merge_bases(struct commit *one, struct commit *two);\n> +struct commit_list *get_octopus_merge_bases(struct commit_list *in);\n> +\n> +/* To be used only when object flags after this call no longer matter */\n> +struct commit_list *get_merge_bases_many_dirty(struct commit *one, int n, struct commit **twos);\n> +\n> +int is_descendant_of(struct commit *commit, struct commit_list *with_commit);\n> +int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit **reference);\n> +int in_merge_bases(struct commit *commit, struct commit *reference);\n> +\n> +\n> +/*\n> + * Takes a list of commits and returns a new list where those\n> + * have been removed that can be reached from other commits in\n> + * the list. It is useful for, e.g., reducing the commits\n> + * randomly thrown at the git-merge command and removing\n> + * redundant commits that the user shouldn't have given to it.\n> + *\n> + * This function destroys the STALE bit of the commit objects'\n> + * flags.\n> + */\n> +struct commit_list *reduce_heads(struct commit_list *heads);\n> +\n> +/*\n> + * Like `reduce_heads()`, except it replaces the list. Use this\n> + * instead of `foo = reduce_heads(foo);` to avoid memory leaks.\n> + */\n> +void reduce_heads_replace(struct commit_list **heads);\n\nThanks for the docs! Bonus points for also documenting the\nother functions (is_descendant_of etc. For example is\nis_descendant_of destroying some bit state?)\n\n> +#endif\n> diff --git a/commit.c b/commit.c\n> index 39b80bd21..32d1234bd 100644\n> --- a/commit.c\n> +++ b/commit.c\n> @@ -843,364 +843,6 @@ void sort_in_topological_order(struct commit_list **list, enum rev_sort_order so\n> clear_author_date_slab(&author_date);\n> }\n>\n> -/* merge-base stuff */\n\nThis is the only line that did not make it to the other file. :-)\nI don't think it is needed in commit-reach.c\n"},{"id":"352654","messageId":"CAPig+cQ0t8nDnE9fVbo_wgth_Y2yuYM1O9AD1y6kvyHTqD57Yg@mail.gmail.com","threadId":"48889","inReplyTo":"20180716185618.GD22298@sigill.intra.peff.net","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-07-16T18:59:42Z","receivedAt":"2018-07-16T18:59:55Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 16, 2018 at 2:56 PM Jeff King wrote:\n> On Mon, Jul 16, 2018 at 02:40:21PM -0400, Eric Sunshine wrote:\n> > On Mon, Jul 16, 2018 at 12:18 PM Jeff King wrote:\n> > > git-send-email uses the current time minus an offset, and then\n> > > monotonically increases for each patch:\n> >\n> > Junio pointed this out to gitgitgadget developers in [1], which led to\n> > an issue being opened[2]. That issue was merged today.\n> >\n> > [1]: https://public-inbox.org/git/xmqq7em7gg3j.fsf@gitster-ct.c.googlers.com/\n> > [2]: https://github.com/gitgitgadget/gitgitgadget/pull/15\n>\n> I was going to say \"oh good, fixed\", but it looks like it just merged\n> adding that line to the TODO list. :)\n\nErm, right. I actually knew a couple days ago that that issue was just\na change to the TODO list but forgot that important tidbit when I\nwrote the above \"was merged today\". Anyhow, at least it's on the\nradar.\n"},{"id":"352656","messageId":"CAGZ79kZh42em1mQ+LQuzN8UTqY2MCutrS8eXAh-FvzEBVGcmGg@mail.gmail.com","threadId":"48889","inReplyTo":"7a6d1cbe12efa0a452a977f1f6bb8c618421040b.1531746010.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 02/16] commit-reach: move ref_newer from remote.c","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T19:10:50Z","receivedAt":"2018-07-16T19:11:03Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> Signed-off-by: Derrick Stolee \n\nAnother verbatim move!\n(I'll just re-iterate that the --color-moved option is very helpful in\nthese reviews)\n\nThanks,\nStefan\n\n> +++ b/commit-reach.h\n> @@ -38,4 +38,6 @@ struct commit_list *reduce_heads(struct commit_list *heads);\n> */\n> void reduce_heads_replace(struct commit_list **heads);\n>\n> +int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid);\n> +\n\nBonus points for docs on ref_newer!\n\n> +++ b/http-push.c\n> @@ -14,6 +14,7 @@\n> #include \"argv-array.h\"\n> #include \"packfile.h\"\n> #include \"object-store.h\"\n> +#include \"commit-reach.h\"\n>\n>\n\nDouble new line here?\nI missed that in p1, it would be nice if you could fix that up.\n"},{"id":"352657","messageId":"CAGZ79kY76rnp8VEBjzfcZ+kB7JWh+5urASWdAGkYejwGY21QJQ@mail.gmail.com","threadId":"48889","inReplyTo":"b9687d2b21c4560178a509572770e3a058582a8b.1531746010.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 03/16] commit-reach: move commit_contains from ref-filter","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T19:14:39Z","receivedAt":"2018-07-16T19:14:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n wrote:\n\n> +\n> +int commit_contains(struct ref_filter *filter, struct commit *commit,\n> + struct commit_list *list, struct contains_cache *cache)\n\n[...]\n\n> -\n> -static int commit_contains(struct ref_filter *filter, struct commit *commit,\n> - struct commit_list *list, struct contains_cache *cache)\n\nAll moved code, but this one, which was exposed to the public.\nMight be worth calling out in the commit message?\nWhile exposing it, it is a good idea to question its name and if\nit is good enough for public use (I think it is -- despite not understanding\nwhat the function does by its arguments; so bonus points for docs!)\n\nThanks,\nStefan\n"},{"id":"352661","messageId":"CAGZ79kZVyd1BgWGW85PxF_4Skv4pEB=a5iH_1i=5B6X07cfyoA@mail.gmail.com","threadId":"48889","inReplyTo":"74b7352b580ae920d9d5f206b9389e207aada4fe.1531746010.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 06/16] upload-pack: generalize commit date cutoff","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T19:38:36Z","receivedAt":"2018-07-16T19:38:49Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> The ok_to_give_up() method uses the commit date as a cutoff to avoid\n> walking the entire reachble set of commits. Before moving the\n> reachable() method to commit-reach.c, pull out the dependence on the\n> global constant 'oldest_have' with a 'min_commit_date' parameter.\n\n\n 'oldest_have' seems to be used in only one method after that\n (function got_oid); but as that function is called many times\n we either have to make it a function-global or pass around as a parameter,\n we'll defer that to later.\n\nCode (of all previous patches and this one) look good!\nThanks,\nStefan\n"},{"id":"352686","messageId":"CAGZ79ka921=OxV9FUQWx2=yxeoSsWPzVMLYQR_2zq=DJ6=puRA@mail.gmail.com","threadId":"48889","inReplyTo":"7a3e3815c585dd3972e9bea6798a7664232157a3.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 11/16] test-reach: test get_merge_bases_many","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T21:24:30Z","receivedAt":"2018-07-16T21:24:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"> +test_expect_success 'get_merge_bases_many' '\n> + cat >input <<-\\EOF &&\n> + A:commit-5-7\n> + X:commit-4-8\n> + X:commit-6-6\n> + X:commit-8-3\n> + EOF\n> + {\n> + printf \"get_merge_bases_many(A,X):\\n\" &&\n> + git rev-parse commit-5-6 &&\n> + git rev-parse commit-4-7\n\nPlease call rev-parse only once, giving both tips as argument, i.e.\n\n printf \"get_merge_bases_many(A,X):\\n\" &&\n git rev-parse commit-5-6 \\\n commit-4-7\n\nought to produce the same output\n\nThanks,\nStefan\n"},{"id":"352687","messageId":"CAGZ79kbV-ahEYdCPyp6-mo+hVYr1c2+Xf-F5xYD3mCwBX9g16g@mail.gmail.com","threadId":"48889","inReplyTo":"3b0379cf6fb42ca5ee4e10cb20bf42ff333e1456.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 12/16] test-reach: test reduce_heads","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T21:30:17Z","receivedAt":"2018-07-16T21:30:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"> +test_expect_success 'reduce_heads' '\n> + cat >input <<-\\EOF &&\n> + X:commit-1-10\n> + X:commit-2-8\n> + X:commit-3-6\n> + X:commit-4-4\n> + X:commit-1-7\n> + X:commit-2-5\n> + X:commit-3-3\n> + X:commit-5-1\n> + EOF\n> + {\n> + printf \"reduce_heads(X):\\n\" &&\n> + git rev-parse commit-5-1 &&\n> + git rev-parse commit-4-4 &&\n> + git rev-parse commit-3-6 &&\n> + git rev-parse commit-2-8 &&\n> + git rev-parse commit-1-10\n\nPlease use rev-parse only once.\n\nI am not sure about the usage of { braces } in the test suite,\n+cc Eric who sent a test suite linting series recently.\nDo we need to em-'brace' the statements that describe the\nexpected behavior? (Or is it supposed to be easier to read\nfor the reviewers? I found these very readable so far... but\nthis question just came up)\n\nThanks,\nStefan\n"},{"id":"352689","messageId":"20180716213153.242065-1-jonathantanmy@google.com","threadId":"48889","inReplyTo":"01bef3d654845236e9115ebc33d80673b7b4a5a2.1531746009.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 01/16] commit-reach: move walk methods from commit.c","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-07-16T21:31:53Z","receivedAt":"2018-07-16T21:32:01Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> +/* Remember to update object flag allocation in object.h */\n> +#define PARENT1\t\t(1u<<16)\n> +#define PARENT2\t\t(1u<<17)\n> +#define STALE\t\t(1u<<18)\n> +#define RESULT\t\t(1u<<19)\n\nUpdate object.h to point to commit-reach.c instead of commit.c also.\n\n> diff --git a/commit-reach.h b/commit-reach.h\n> new file mode 100644\n> index 000000000..244f48c5f\n> --- /dev/null\n> +++ b/commit-reach.h\n> @@ -0,0 +1,41 @@\n> +#ifndef __COMMIT_REACH_H__\n> +#define __COMMIT_REACH_H__\n> +\n> +#include \"commit.h\"\n> +\n> +struct commit_list *get_merge_bases_many(struct commit *one,\n> +\t\t\t\t\t int n,\n> +\t\t\t\t\t struct commit **twos);\n\n\n\nShould the declarations in commit.h be deleted also?\n\nThanks for copying it over verbatim - it makes it much easier to see\nwhat's going on with --color-moved.\n"},{"id":"352696","messageId":"CAGZ79kYnVxg2XBWa9ib8Ga7iOPE+10sK=ZPwQAYJXMB+Q8tBAg@mail.gmail.com","threadId":"48889","inReplyTo":"95b11ced4c0355aabf458539a284bb3895d73f6f.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 13/16] test-reach: test can_all_from_reach_with_flags","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T21:54:17Z","receivedAt":"2018-07-16T21:54:31Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> The can_all_from_reach_with_flags method is used by ok_to_give_up in\n> upload-pack.c to see if we have done enough negotiation during a fetch.\n> This method is intentionally created to preserve state between calls to\n> assist with stateful negotiation, such as over SSH.\n>\n> To make this method testable, add a new can_all_from_reach method that\n> does the initial setup and final tear-down. Call the method from\n> 'test-tool reach'.\n>\n> Since this is a many-to-many reachability query, add a new type of input\n> to the 'test-tool reach' input format. Lines \"Y:\" create a\n> list of commits to be the reachability targets from the commits in the\n> 'X' list. In the context of fetch negotiation, the 'X' commits are the\n> 'want' commits and the 'Y' commits are the 'have' commits.\n\nMakes sense. I shortly wondered if we want to s/Y/Z/ as I find X and Z\nmore distinguishable than X/Y for reading/skimming.\n\nThanks,\nStefan\n\n> +++ b/commit-reach.c\n> @@ -593,3 +593,50 @@ int can_all_from_reach_with_flag(struct object_array *from,\n> }\n> return 1;\n> }\n> +\n> +int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n> + int cutoff_by_min_date)\n\nWe'll put this method (that is only used by tests so far) here to\nnot clutter the test tool code too much, or do we see more benefits\nfrom the code\nhere? If so, docs would be nice.\n\n> +++ b/t/t6600-test-reach.sh\n\n> +test_expect_success 'can_all_from_reach:hit' '\n [...]\n> + Y:commit-7-3\n> + Y:commit-8-1\n\n> +test_expect_success 'can_all_from_reach:miss' '\n[...]\n> + Y:commit-8-5\n\nIt would be nice if the difference in the list could be easier\nto spot as a reader. (There is a lot of repetition).\n\nMaybe we can teach \"test-tool reach\" to ignore input lines\nstarting with '#' such that we can annotate the last line in\nthe miss case?\n\nWhy do we omit 7-3 in the miss case? (might be nice\nfor symmetry to keep around)\n\nThanks,\nStefan\n"},{"id":"352698","messageId":"CAPig+cQ+pDrhYmsUPe69MzGK8YiH_cGOp_cd592ZEBC4i2Lkhw@mail.gmail.com","threadId":"48889","inReplyTo":"CAGZ79kbV-ahEYdCPyp6-mo+hVYr1c2+Xf-F5xYD3mCwBX9g16g@mail.gmail.com","subject":"Re: [PATCH 12/16] test-reach: test reduce_heads","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-07-16T21:59:04Z","receivedAt":"2018-07-16T21:59:18Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 16, 2018 at 5:30 PM Stefan Beller wrote:\n> > +test_expect_success 'reduce_heads' '\n> > + cat >input <<-\\EOF &&\n> > + X:commit-1-10\n> > + X:commit-2-8\n> > + X:commit-3-6\n> > + X:commit-4-4\n> > + X:commit-1-7\n> > + X:commit-2-5\n> > + X:commit-3-3\n> > + X:commit-5-1\n> > + EOF\n> > + {\n> > + printf \"reduce_heads(X):\\n\" &&\n> > + git rev-parse commit-5-1 &&\n> > + git rev-parse commit-4-4 &&\n> > + git rev-parse commit-3-6 &&\n> > + git rev-parse commit-2-8 &&\n> > + git rev-parse commit-1-10\n> > + } >expect &&\n>\n> Please use rev-parse only once.\n>\n> I am not sure about the usage of { braces } in the test suite,\n> +cc Eric who sent a test suite linting series recently.\n> Do we need to em-'brace' the statements that describe the\n> expected behavior? (Or is it supposed to be easier to read\n> for the reviewers? I found these very readable so far... but\n> this question just came up)\n\nGrouping the commands for redirection via a \"{...}>expect\" block is\nless noisy than redirecting each command separately, thus more\nreviewer-friendly. And, {...} blocks are used regularly in the test\nsuite, so no issue there.\n\nI do agree that a single git-rev-parse with all 5 arguments makes more\nsense (and would be appreciated by Windows folk). Also, the 'printf'\ncould be replaced by a simple 'echo' if we want to get nit-picky.\n\nFinally, a style nit: We don't normally indent the content of a\nhere-doc like that. Instead, the content is normally aligned with the\nclosing EOF, not indented beyond it.\n"},{"id":"352701","messageId":"CAGZ79kYJdQAsVW+=G6d2LkXzSe4vcJOreb_+-962i1evgynjUA@mail.gmail.com","threadId":"48889","inReplyTo":"1fd45ef2b15524c18322183231608fe13a733506.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 14/16] commit-reach: replace ref_newer logic","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T22:16:12Z","receivedAt":"2018-07-16T22:16:27Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> The ref_newer method is used by 'git push' to check if a force-push is\n> required. This method does not use any kind of cutoff when walking, so\n> in the case of a force-push will walk all reachable commits.\n>\n> The is_descendant_of method already uses paint_down_to_common along with\n> cutoffs. By translating the ref_newer arguments into the commit and\n> commit_list required by is_descendant_of, we can have one fewer commit\n> walk and also improve our performance!\n>\n> For a copy of the Linux repository, 'test-tool reach ref_newer' presents\n> the following improvements with the specified input. In the case that\n> ref_newer returns 1, there is no improvement. The improvement is in the\n> second case where ref_newer returns 0.\n>\n> Input\n> -----\n\nI fetched the series as advertised in the cover letter; however Junio\napplies the patches manually, for which there is a problem here in the\npatch format. Three dashes indicate the end of a commit message and\nbelow that you usually have some ephemeral information such as the\nstats, followed by the diffs starting with \"diff --git\", at least that was the\ncase.\n\nI just tested and it applies this patch cleanly keeping the information\nbelow the three dashes intact. Cool!\n\n> A:v4.9\n> B:v3.19\n>\n> Before: 0.09 s\n> After: 0.09 s\n>\n> To test the negative case, add a new commit with parent v3.19,\n> regenerate the commit-graph, and then run with B pointing at that\n> commit.\n>\n> Before: 0.43 s\n> After: 0.09 s\n\nNice! The code looks good, too.\nThanks,\nStefan\n"},{"id":"352704","messageId":"20180716223703.252965-1-jonathantanmy@google.com","threadId":"48889","inReplyTo":"cf4559e08c47adcaf4c938a325acf829a5759285.1531746011.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 07/16] commit-reach: move can_all_from_reach_with_flags","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-07-16T22:37:03Z","receivedAt":"2018-07-16T22:37:10Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> /* Remember to update object flag allocation in object.h */\n> +#define REACHABLE (1u<<15)\n> #define PARENT1\t\t(1u<<16)\n> #define PARENT2\t\t(1u<<17)\n> #define STALE\t\t(1u<<18)\n\nUpdate the object flag allocation in object.h.\n\n> +int reachable(struct commit *from, int with_flag, int assign_flag,\n> +\t time_t min_commit_date)\n\nIn this and previous patches: I think it's better to use \"unsigned int\"\nas the data type for a flag, just like in clear_commit_marks().\n\nOther than that, this and all previous patches look good.\n"},{"id":"352705","messageId":"CAGZ79kY5rEUp3Vsi5ux=KaC0t4vrp8UME-CaQtTvnYsizUGfFQ@mail.gmail.com","threadId":"48889","inReplyTo":"816821eec9ba476ccdfbfdf6e3cdd3619743ea2e.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T22:37:08Z","receivedAt":"2018-07-16T22:37:21Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> Note how the time increases between the two cases in the two versions.\n> The new code increases relative to the number of commits that need to be\n> walked, but not directly relative to the number of 'from' commits.\n\nCool!\n\n> int can_all_from_reach_with_flag(struct object_array *from,\n> int with_flag, int assign_flag,\n> - time_t min_commit_date)\n> + time_t min_commit_date,\n> + uint32_t min_generation)\n> {\n\n> for (i = 0; i < from->nr; i++) {\n[...]\n> + parse_commit(list[i]);\n\nparse_commit_or_die or handle the return code?\n(or a comment why we specifically are allowed to ignore\nthe return code here)\n\n[...]\n> + for (parent = stack->item->parents; parent; parent = parent->next) {\n[...]\n> + parse_commit(parent->item);\n\nsame here.\n"},{"id":"352707","messageId":"CAGZ79kZesqs4=OWcgT_8ycpc5wFTArLKCbu+9dnOPJJLUbuuZw@mail.gmail.com","threadId":"48889","inReplyTo":"a60b7011adfd107633d7d3158c9c3fa127dc1a61.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 16/16] commit-reach: use can_all_from_reach","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-16T22:47:38Z","receivedAt":"2018-07-16T22:47:52Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> The is_descendant_of method previously used in_merge_bases() to check if\n> the commit can reach any of the commits in the provided list. This had\n> two performance problems:\n>\n> 1. The performance is quadratic in worst-case.\n>\n> 2. A single in_merge_bases() call requires walking beyond the target\n> commit in order to find the full set of boundary commits that may be\n> merge-bases.\n>\n> The can_all_from_reach method avoids this quadratic behavior and can\n> limit the search beyond the target commits using generation numbers. It\n> requires a small prototype adjustment to stop using commit-date as a\n> cutoff, as that optimization is no longer appropriate here.\n>\n> Since in_merge_bases() uses paint_down_to_common(), is_descendant_of()\n> naturally found cutoffs to avoid walking the entire commit graph. Since\n> we want to always return the correct result, we cannot use the\n> min_commit_date cutoff in can_all_from_reach. We then rely on generation\n> numbers to provide the cutoff.\n>\n> Since not all repos will have a commit-graph file, nor will we always\n> have generation numbers computed for a commit-graph file, create a new\n> method, generation_numbers_enabled(), that checks for a commit-graph\n> file and sees if the first commit in the file has a non-zero generation\n> number. In the case that we do not have generation numbers, use the old\n> logic for is_descendant_of().\n>\n> Performance was meausured on a copy of the Linux repository using the\n> 'test-tool reach is_descendant_of' command using this input:\n>\n> A:v4.9\n> X:v4.10\n> X:v4.11\n> X:v4.12\n> X:v4.13\n> X:v4.14\n> X:v4.15\n> X:v4.16\n> X:v4.17\n> X.v3.0\n>\n> Note that this input is tailored to demonstrate the quadratic nature of\n> the previous method, as it will compute merge-bases for v4.9 versus all\n> of the later versions before checking against v4.1.\n>\n> Before: 0.26 s\n> After: 0.21 s\n>\n> Since we previously used the is_descendant_of method in the ref_newer\n> method, we also measured performance there using\n> 'test-tool reach ref_newer' with this input:\n>\n> A:v4.9\n> B:v3.19\n>\n> Before: 0.10 s\n> After: 0.08 s\n>\n> By adding a new commit with parent v3.19, we test the non-reachable case\n> of ref_newer:\n>\n> Before: 0.09 s\n> After: 0.08 s\n>\n> Signed-off-by: Derrick Stolee \n> ---\n\nThanks for the commit message. The code itself looks good!\n\nI think this series is nearly done, I have only commented on\nstyle issues so far, which are easier to address than fundamental\ndesign issues or naming things.\n\nThanks,\nStefan\n"},{"id":"352710","messageId":"20180716230019.257742-1-jonathantanmy@google.com","threadId":"48889","inReplyTo":"6acc5e804ea7beabc7c813e4c1c067915a16087b.1531746011.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 08/16] test-reach: create new test tool for ref_newer","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-07-16T23:00:19Z","receivedAt":"2018-07-16T23:00:25Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> To use the new test-tool, use 'test-tool reach ' and provide\n> input to stdin that describes the inputs to the method. Currently, we\n> only implement the ref_newer method, which requires two commits. Use\n> lines \"A:\" and \"B:\" for the two inputs. We will\n> expand this input later to accommodate methods that take lists of\n> commits.\n\nIt would be nice if \"A\" and \"B\" were \"ancestor\" and \"descendant\" (or\nsomething like that) instead, so that I don't have to check which\ndirection the reach is calculated in.\n\n> +int cmd__reach(int ac, const char **av)\n> +{\n> +\tstruct object_id oid_A, oid_B;\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\tstruct repository *r = the_repository;\n> +\n> +\tsetup_git_directory();\n> +\n> +\tif (ac < 2)\n> +\t\texit(1);\n> +\n> +\n> +\twhile (strbuf_getline(&buf, stdin) != EOF) {\n> +\t\tstruct object_id oid;\n> +\t\tstruct object *o;\n> +\t\tstruct commit *c;\n> +\t\tif (buf.len < 3)\n> +\t\t\tcontinue;\n> +\n> +\t\tif (get_oid_committish(buf.buf + 2, &oid))\n> +\t\t\tdie(\"failed to resolve %s\", buf.buf + 2);\n\nYou can also use skip_prefix() instead of using arithmetic to determine\nthe start of the OID.\n\n> +# Construct a grid-like commit graph with points (x,y)\n> +# with 1 <= x <= 10, 1 <= y <= 10, where (x,y) has\n> +# parents (x-1, y) and (x, y-1), keeping in mind that\n> +# we drop a parent if a coordinate is nonpositive.\n> +#\n> +# (10,10)\n> +# / \\\n> +# (10,9) (9,10)\n> +# / \\ / \\\n> +# (10,8) (9,9) (8,10)\n> +# / \\ / \\ / \\\n> +# ( continued...)\n> +# \\ / \\ / \\ /\n> +# (3,1) (2,2) (1,3)\n> +# \\ / \\ /\n> +# (2,1) (2,1)\n> +# \\ /\n> +# (1,1)\n\nThis is quite a good design, thanks.\n\n> +# We use branch 'comit-x-y' to refer to (x,y).\n\ns/comit/commit/\n\n> +\tgit show-ref -s commit-7-7 | git commit-graph write --stdin-commits &&\n> +\tmv .git/objects/info/commit-graph commit-graph-half &&\n\nMy understanding is that this writes for 7-7 and all its ancestors,\nbut...\n\n> +test_expect_success 'ref_newer:hit' '\n> +\tcat >input <<-\\EOF &&\n> +\t\tA:commit-5-7\n> +\t\tB:commit-2-3\n> +\tEOF\n> +\tprintf \"ref_newer:1\\n\" >expect &&\n> +\ttest_three_modes ref_newer\n> +'\n> +\n> +test_done\n\n...both 5-7 and 2-3 are ancestors of 7-7, right? Which means that you\ndon't test the \"half\" commit graph here. (It's probably sufficient to\njust adjust the numbers.)\n"},{"id":"352724","messageId":"20180716230819.259097-1-jonathantanmy@google.com","threadId":"48889","inReplyTo":"7a3e3815c585dd3972e9bea6798a7664232157a3.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 11/16] test-reach: test get_merge_bases_many","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-07-16T23:08:19Z","receivedAt":"2018-07-16T23:08:26Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> @@ -71,6 +78,14 @@ int cmd__reach(int ac, const char **av)\n> \t\tprintf(\"%s(A,B):%d\\n\", av[1], in_merge_bases(A, B));\n> \telse if (!strcmp(av[1], \"is_descendant_of\"))\n> \t\tprintf(\"%s(A,X):%d\\n\", av[1], is_descendant_of(A, X));\n> +\telse if (!strcmp(av[1], \"get_merge_bases_many\")) {\n> +\t\tstruct commit_list *list = get_merge_bases_many(A, X_nr, X_array);\n> +\t\tprintf(\"%s(A,X):\\n\", av[1]);\n> +\t\twhile (list) {\n> +\t\t\tprintf(\"%s\\n\", oid_to_hex(&list->item->object.oid));\n> +\t\t\tlist = list->next;\n> +\t\t}\n\nI don't think get_merge_bases_many defines a sort order on its output?\nIt might be better to sort the resulting commit list here, so that the\noutput is more well-defined. (And omit the informational printf so that\nit's slightly easier to generate the \"expect\" file.)\n"},{"id":"352729","messageId":"20180717001006.8283-1-jonathantanmy@google.com","threadId":"48889","inReplyTo":"95b11ced4c0355aabf458539a284bb3895d73f6f.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 13/16] test-reach: test can_all_from_reach_with_flags","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-07-17T00:10:06Z","receivedAt":"2018-07-17T00:10:11Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"The subject should be can_all_from_reach_with_flag (without the \"s\" at\nthe end). Likewise in the commit message.\n\n> To make this method testable, add a new can_all_from_reach method that\n> does the initial setup and final tear-down. Call the method from\n> 'test-tool reach'.\n\nThis description leads me to believe that can_all_from_reach() is (1)\ntrivial, and (2) will not be used in production code. But (1) the\nfunction itself is non-trivial and the function signature contains a\n\"cutoff_by_min_date\" parameter not found in\ncan_all_from_reach_with_flag():\n\n> +int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n> +\t\t int cutoff_by_min_date)\n\nand (2) this function will be used in production code subsequently in\nthe \"commit-reach: use can_all_from_reach\" commit. It would be clearer,\nmaybe, if there were some rearrangement - maybe a commit introducing\nthis function, especially documenting what cutoff_by_min_date does, and\nthen a test (which just tests can_all_from_reach), and then the\n\"commit-reach: use can_all_from_reach\" commit.\n"},{"id":"352738","messageId":"20180717011600.22362-1-jonathantanmy@google.com","threadId":"48889","inReplyTo":"816821eec9ba476ccdfbfdf6e3cdd3619743ea2e.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-07-17T01:16:00Z","receivedAt":"2018-07-17T01:16:06Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> The first step includes using a depth-first-search (DFS) from each from\n> commit, sorted by ascending generation number. We do not walk beyond the\n> minimum generation number or the minimum commit date. This DFS is likely\n> to be faster than the existing reachable() method because we expect\n> previous ref values to be along the first-parent history.\n> \n> If we find a target commit, then we mark everything in the DFS stack as\n> a RESULT. This expands the set of targets for the other from commits. We\n> also mark the visited commits using 'assign_flag' to prevent re-walking\n> the same code.\n\nThanks for this - it was very helpful in understanding the code.\n\nThe function itself uses a DFS stack that contains only the trail\nleading up to the currently processed node, and not the one that I'm\nmore familiar with, which also contains the siblings of processed nodes.\nI'll annotate the function with my thought process in the hope that it\nwill aid future reviewers. (The diff as seen in the e-mail is confusing\nso I'm reproducing the function itself, not any + or -.)\n\n> int can_all_from_reach_with_flag(struct object_array *from,\n> \t\t\t\t int with_flag, int assign_flag,\n> \t\t\t\t time_t min_commit_date,\n> \t\t\t\t uint32_t min_generation)\n> {\n> \tstruct commit **list = NULL;\n> \tint i;\n> \tint result = 1;\n> \n> \tALLOC_ARRAY(list, from->nr);\n> \tfor (i = 0; i < from->nr; i++) {\n> \t\tlist[i] = (struct commit *)from->objects[i].item;\n> \n> \t\tparse_commit(list[i]);\n> \n> \t\tif (list[i]->generation < min_generation)\n> \t\t\treturn 0;\n> \t}\n> \n> \tQSORT(list, from->nr, compare_commits_by_gen);\n> \n> \tfor (i = 0; i < from->nr; i++) {\n> \t\t/* DFS from list[i] */\n> \t\tstruct commit_list *stack = NULL;\n> \n> \t\tlist[i]->object.flags |= assign_flag;\n> \t\tcommit_list_insert(list[i], &stack);\n> \n> \t\twhile (stack) {\n> \t\t\tstruct commit_list *parent;\n> \n> \t\t\tif (stack->item->object.flags & with_flag) {\n> \t\t\t\tpop_commit(&stack);\n> \t\t\t\tcontinue;\n> \t\t\t}\n\nI wish that the code would refrain from pushing such an object instead\nof popping it at the first opportunity, but I guess that doing so would\nrequire the equivalent of a labeled break/continue. I have no qualms\nwith using \"goto\" in this case, but I know that some people don't like\nit :-P\n\n> \t\t\tfor (parent = stack->item->parents; parent; parent = parent->next) {\n> \t\t\t\tif (parent->item->object.flags & (with_flag | RESULT))\n> \t\t\t\t\tstack->item->object.flags |= RESULT;\n\nStraightforward, and also produces the bubbling up that we want. An\nobject is never popped unless it has the \"with_flag\" flag (see above) or\nall its parents have been processed. The object can encounter the \"if\"\nstatement multiple times; the last one is when all its parents have been\nprocessed (and thus have the RESULT flag set if necessary).\n\n> \t\t\t\tif (!(parent->item->object.flags & assign_flag)) {\n> \t\t\t\t\tparent->item->object.flags |= assign_flag;\n> \n> \t\t\t\t\tparse_commit(parent->item);\n> \n> \t\t\t\t\tif (parent->item->date < min_commit_date ||\n> \t\t\t\t\t parent->item->generation < min_generation)\n> \t\t\t\t\t\tcontinue;\n> \n> \t\t\t\t\tcommit_list_insert(parent->item, &stack);\n> \t\t\t\t\tbreak;\n> \t\t\t\t}\n\nIf not yet processed, push it onto the stack and break. The child commit\nis still left on the stack. The next time the child commit is processed\n(in an iteration of the \"while\" loop), the \"for\" loop will iterate until\nthe next unprocessed parent.\n\nIn the DFS that I'm used to, all parents would be pushed here, but\nperhaps the fact that the iteration is postorder confuses things.\nAnyway, if someone comes up with a better algorithm, replacing it\nshouldn't be too difficult - the algorithm is contained within this\nfunction, and there are tests to check the correctness of the algorithm\nupdate.\n\n> \t\t\t}\n> \n> \t\t\tif (!parent)\n> \t\t\t\tpop_commit(&stack);\n\nOnly when we have no parents left are we completely done with the\ncurrent object.\n\n> \t\t}\n> \n> \t\tif (!(list[i]->object.flags & (with_flag | RESULT))) {\n> \t\t\tresult = 0;\n> \t\t\tgoto cleanup;\n> \t\t}\n\nAnd after the DFS, if the original object did not have an appropriate\nflag set, we do not bother with the other \"want\" objects.\n\n> \t}\n> \n> cleanup:\n> \tfor (i = 0; i < from->nr; i++) {\n> \t\tclear_commit_marks(list[i], RESULT);\n> \t\tclear_commit_marks(list[i], assign_flag);\n> \t}\n> \treturn result;\n> }\n"},{"id":"352859","messageId":"nycvar.QRO.7.76.6.1807181420250.71@tvgsbejvaqbjf.bet","threadId":"48889","inReplyTo":"20180716161821.GB18150@sigill.intra.peff.net","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2018-07-18T12:23:11Z","receivedAt":"2018-07-18T12:23:46Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Mon, 16 Jul 2018, Jeff King wrote:\n\n> On Mon, Jul 16, 2018 at 02:54:38PM +0100, Ramsay Jones wrote:\n> \n> > On 16/07/18 14:00, Derrick Stolee via GitGitGadget wrote:\n> > > There are many places in Git that use a commit walk to determine\n> > > reachability between commits and/or refs. A lot of this logic is\n> > > duplicated.\n> > [snip] ...\n> > \n> > This is not your problem, but I find these GitGitGadget\n> > submissions somewhat annoying. This series has been spewed\n> > all over my in-box in, what I assume, is commit date order.\n> > \n> > So, patches #4,5 dated 19/06, then #1,2,3 dated 25/06,\n> > then #15 dated 28/06, then #6,7 dated 12/07, then #8-16\n> > dated 13/07, then 00/16 dated today.\n> > \n> > No I don't use a threaded display (I hate it), be even with\n> > that turned on, the patches still appear in the above order\n> > under the cover letter (but at least all together).\n> \n> Yeah, they're out of order in mutt's threaded display. And the\n> back-dating means there's a much higher chance of them getting blocked\n> as spam (e.g., some of the dates are from weeks ago).\n> \n> git-send-email uses the current time minus an offset, and then\n> monotonically increases for each patch:\n> \n> $time = time - scalar $#files;\n> ...\n> my $date = format_2822_time($time++);\n> \n> which seems to work pretty well in practice. It does mean the original\n> dates are lost. The committer date is not interesting at all (there will\n> be a new committer via \"git am\" anyway). The original author date is\n> potentially of interest, but could be included as an in-body header.\n> AFAIK send-email doesn't have such an option, though, and people are\n> fine with date-of-sending becoming the new author date.\n> \n> +cc Johannes as the GitGitGadget author\n\nThanks for dumping even more work on my shoulders.\n\nI wanted to help with that insane process we have here, but in a more\ncollaborative manner.\n\nThis time I fixed it, but please do keep in mind that the decision to use\nthe email transport for something it *was not designed for* (it was\ndesigned for humans talking to humans) is the culprit here.\n\nNext time, I will ask you to jump in, instead of putting the onus on me.\n\nI mean, seriously, what is this? \"You can use *any* mail program to work\nwith the Git mailing list, *any* mailer. As long as it is mutt. And as\nlong as you spend hours and hours on tooling that oh BTW nobody else can\nuse.\"\n\nHopefully GitGitGadget will make this situation better. And hopefully not\non the expense of my sanity.\n\nCiao,\nDscho\n"},{"id":"352860","messageId":"nycvar.QRO.7.76.6.1807181427120.71@tvgsbejvaqbjf.bet","threadId":"48889","inReplyTo":"65d51c26-e248-f16d-bac2-e6ba0a3be8e5@gmail.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2018-07-18T12:28:52Z","receivedAt":"2018-07-18T12:29:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 16 Jul 2018, Derrick Stolee wrote:\n\n> On 7/16/2018 2:44 PM, Eric Sunshine wrote:\n> > On Mon, Jul 16, 2018 at 1:27 PM Stefan Beller wrote:\n> > > Another pain point of the Gadget is that CC's in the cover letter\n> > > do not work as I would imagine. The line\n> > >\n> > > CC: sbeller@google.com\n> > >\n> > > did not put that email into the cc field.\n> > gitgitgadget recognizes case-sensitive \"Cc:\" only[1].\n> >\n> > [1]:\n> > https://github.com/gitgitgadget/gitgitgadget/blob/c4805370f59532aa438283431b8ea7d4484c530f/lib/patch-series.ts#L188\n> \n> Thanks for everyone's patience while we improve gitgitgadget (and - in this\n> case - I learn how to use it).\n\nAnd let's please stop pretending that this GitGitGadget project is\nsomebody else's problem.\n\nIt is our best shot at addressing the *constant* pain point that is the code\ncontribution process of Git.\n\nIn other words: if you see something that you don't like about\nGitGitGadget, get your butts off the ground and contribute a fix. The code\ncontribution process of GitGitGadget is very easy: open a PR at\nhttps://github.com/gitgitgadget/gitgitgadget\n\nCiao,\nJohannes\n"},{"id":"352861","messageId":"nycvar.QRO.7.76.6.1807181429580.71@tvgsbejvaqbjf.bet","threadId":"48889","inReplyTo":"CAPig+cQ0t8nDnE9fVbo_wgth_Y2yuYM1O9AD1y6kvyHTqD57Yg@mail.gmail.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2018-07-18T12:32:12Z","receivedAt":"2018-07-18T12:32:40Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Eric & Peff,\n\nOn Mon, 16 Jul 2018, Eric Sunshine wrote:\n\n> On Mon, Jul 16, 2018 at 2:56 PM Jeff King wrote:\n> > On Mon, Jul 16, 2018 at 02:40:21PM -0400, Eric Sunshine wrote:\n> > > On Mon, Jul 16, 2018 at 12:18 PM Jeff King wrote:\n> > > > git-send-email uses the current time minus an offset, and then\n> > > > monotonically increases for each patch:\n> > >\n> > > Junio pointed this out to gitgitgadget developers in [1], which led to\n> > > an issue being opened[2]. That issue was merged today.\n> > >\n> > > [1]: https://public-inbox.org/git/xmqq7em7gg3j.fsf@gitster-ct.c.googlers.com/\n> > > [2]: https://github.com/gitgitgadget/gitgitgadget/pull/15\n> >\n> > I was going to say \"oh good, fixed\", but it looks like it just merged\n> > adding that line to the TODO list. :)\n> \n> Erm, right. I actually knew a couple days ago that that issue was just\n> a change to the TODO list but forgot that important tidbit when I\n> wrote the above \"was merged today\". Anyhow, at least it's on the\n> radar.\n\nIt is always nice to get such active contributions.\n\nSeriously again, do feel free to jump in and contribute improvements to\nGitGitGadget.\n\nWe have a very time-consuming (read: time wasting) code contribution\nprocess, and it is an untenable situation, and GitGitGadget was designed\nto be able to address this huge problem.\n\nBut I can't do it alone. And neither should you pretend that this is my\nproblem alone. This problem is as much your problem as it is mine.\n(Whether you realize it or not.)\n\nCiao,\nDscho\n"},{"id":"352871","messageId":"CACsJy8AbT3=wzKecGqvj8ibHmquU=NfP872f1cAM-VFOLSLSGQ@mail.gmail.com","threadId":"48889","inReplyTo":"nycvar.QRO.7.76.6.1807181427120.71@tvgsbejvaqbjf.bet","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-07-18T15:01:52Z","receivedAt":"2018-07-18T15:02:23Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Jul 18, 2018 at 2:30 PM Johannes Schindelin\n wrote:\n>\n> Hi,\n>\n> On Mon, 16 Jul 2018, Derrick Stolee wrote:\n>\n> > On 7/16/2018 2:44 PM, Eric Sunshine wrote:\n> > > On Mon, Jul 16, 2018 at 1:27 PM Stefan Beller wrote:\n> > > > Another pain point of the Gadget is that CC's in the cover letter\n> > > > do not work as I would imagine. The line\n> > > >\n> > > > CC: sbeller@google.com\n> > > >\n> > > > did not put that email into the cc field.\n> > > gitgitgadget recognizes case-sensitive \"Cc:\" only[1].\n> > >\n> > > [1]:\n> > > https://github.com/gitgitgadget/gitgitgadget/blob/c4805370f59532aa438283431b8ea7d4484c530f/lib/patch-series.ts#L188\n> >\n> > Thanks for everyone's patience while we improve gitgitgadget (and - in this\n> > case - I learn how to use it).\n>\n> And let's please stop pretending that this GitGitGadget project is\n> somebody else's problem.\n>\n> It is our best shot at addressing the *constant* pain point that is the code\n> contribution process of Git.\n>\n> In other words: if you see something that you don't like about\n> GitGitGadget, get your butts off the ground and contribute a fix.\n\nThank you for the frank words. I will choose to not review any mails\ncoming from GitGitGadget.\n-- \nDuy\n"},{"id":"352883","messageId":"7ccf3170-dd7e-6322-a128-b31a29f6ac7b@gmail.com","threadId":"48889","inReplyTo":"CAGZ79kZVyd1BgWGW85PxF_4Skv4pEB=a5iH_1i=5B6X07cfyoA@mail.gmail.com","subject":"Re: [PATCH 06/16] upload-pack: generalize commit date cutoff","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-07-18T16:04:05Z","receivedAt":"2018-07-18T16:04:10Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/16/2018 3:38 PM, Stefan Beller wrote:\n> On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n> wrote:\n>> From: Derrick Stolee \n>>\n>> The ok_to_give_up() method uses the commit date as a cutoff to avoid\n>> walking the entire reachble set of commits. Before moving the\n>> reachable() method to commit-reach.c, pull out the dependence on the\n>> global constant 'oldest_have' with a 'min_commit_date' parameter.\n>\n> 'oldest_have' seems to be used in only one method after that\n> (function got_oid); but as that function is called many times\n> we either have to make it a function-global or pass around as a parameter,\n> we'll defer that to later.\n\nThere is a lot of global state involved in this negotiation code, and it \nlives between negotiation rounds when the transfer is stateful. Tread \ncarefully!\n\nI did not attempt to reduce the global state at all.\n\nThanks,\n\n-Stolee\n\n"},{"id":"352908","messageId":"043a5a75-6181-6a4c-e81f-9d82645d8f1b@gmail.com","threadId":"48889","inReplyTo":"20180716230019.257742-1-jonathantanmy@google.com","subject":"Re: [PATCH 08/16] test-reach: create new test tool for ref_newer","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-07-18T16:14:44Z","receivedAt":"2018-07-18T16:14:49Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/16/2018 7:00 PM, Jonathan Tan wrote:\n>> To use the new test-tool, use 'test-tool reach ' and provide\n>> input to stdin that describes the inputs to the method. Currently, we\n>> only implement the ref_newer method, which requires two commits. Use\n>> lines \"A:\" and \"B:\" for the two inputs. We will\n>> expand this input later to accommodate methods that take lists of\n>> commits.\n> It would be nice if \"A\" and \"B\" were \"ancestor\" and \"descendant\" (or\n> something like that) instead, so that I don't have to check which\n> direction the reach is calculated in.\n\nDifferent methods will use different combinations. I do notice that I \nforgot to list the method parameters as part of the test-tool output. It \nshould print \"ref_newer(A,B)\" so you know which input goes in which place.\n\n>\n>> +int cmd__reach(int ac, const char **av)\n>> +{\n>> +\tstruct object_id oid_A, oid_B;\n>> +\tstruct strbuf buf = STRBUF_INIT;\n>> +\tstruct repository *r = the_repository;\n>> +\n>> +\tsetup_git_directory();\n>> +\n>> +\tif (ac < 2)\n>> +\t\texit(1);\n>> +\n>> +\n>> +\twhile (strbuf_getline(&buf, stdin) != EOF) {\n>> +\t\tstruct object_id oid;\n>> +\t\tstruct object *o;\n>> +\t\tstruct commit *c;\n>> +\t\tif (buf.len < 3)\n>> +\t\t\tcontinue;\n>> +\n>> +\t\tif (get_oid_committish(buf.buf + 2, &oid))\n>> +\t\t\tdie(\"failed to resolve %s\", buf.buf + 2);\n> You can also use skip_prefix() instead of using arithmetic to determine\n> the start of the OID.\n>\n>> +# Construct a grid-like commit graph with points (x,y)\n>> +# with 1 <= x <= 10, 1 <= y <= 10, where (x,y) has\n>> +# parents (x-1, y) and (x, y-1), keeping in mind that\n>> +# we drop a parent if a coordinate is nonpositive.\n>> +#\n>> +# (10,10)\n>> +# / \\\n>> +# (10,9) (9,10)\n>> +# / \\ / \\\n>> +# (10,8) (9,9) (8,10)\n>> +# / \\ / \\ / \\\n>> +# ( continued...)\n>> +# \\ / \\ / \\ /\n>> +# (3,1) (2,2) (1,3)\n>> +# \\ / \\ /\n>> +# (2,1) (2,1)\n>> +# \\ /\n>> +# (1,1)\n> This is quite a good design, thanks.\n>\n>> +# We use branch 'comit-x-y' to refer to (x,y).\n> s/comit/commit/\n>\n>> +\tgit show-ref -s commit-7-7 | git commit-graph write --stdin-commits &&\n>> +\tmv .git/objects/info/commit-graph commit-graph-half &&\n> My understanding is that this writes for 7-7 and all its ancestors,\n> but...\n>\n>> +test_expect_success 'ref_newer:hit' '\n>> +\tcat >input <<-\\EOF &&\n>> +\t\tA:commit-5-7\n>> +\t\tB:commit-2-3\n>> +\tEOF\n>> +\tprintf \"ref_newer:1\\n\" >expect &&\n>> +\ttest_three_modes ref_newer\n>> +'\n>> +\n>> +test_done\n> ...both 5-7 and 2-3 are ancestors of 7-7, right? Which means that you\n> don't test the \"half\" commit graph here. (It's probably sufficient to\n> just adjust the numbers.)\n\nGood point! Thanks. I'll just write the commit-graph starting at \ncommit-5-5 and that should satisfy the point of the \"mixed-mode\" tests.\n\n-Stolee\n\n"},{"id":"352910","messageId":"6c43ef96-a5ba-990b-518a-1992e0907c60@gmail.com","threadId":"48889","inReplyTo":"CAGZ79kYnVxg2XBWa9ib8Ga7iOPE+10sK=ZPwQAYJXMB+Q8tBAg@mail.gmail.com","subject":"Re: [PATCH 13/16] test-reach: test can_all_from_reach_with_flags","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-07-18T16:54:56Z","receivedAt":"2018-07-18T16:55:01Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/16/2018 5:54 PM, Stefan Beller wrote:\n> On Mon, Jul 16, 2018 at 6:00 AM Derrick Stolee via GitGitGadget\n> wrote:\n>> From: Derrick Stolee \n>>\n>> The can_all_from_reach_with_flags method is used by ok_to_give_up in\n>> upload-pack.c to see if we have done enough negotiation during a fetch.\n>> This method is intentionally created to preserve state between calls to\n>> assist with stateful negotiation, such as over SSH.\n>>\n>> To make this method testable, add a new can_all_from_reach method that\n>> does the initial setup and final tear-down. Call the method from\n>> 'test-tool reach'.\n>>\n>> Since this is a many-to-many reachability query, add a new type of input\n>> to the 'test-tool reach' input format. Lines \"Y:\" create a\n>> list of commits to be the reachability targets from the commits in the\n>> 'X' list. In the context of fetch negotiation, the 'X' commits are the\n>> 'want' commits and the 'Y' commits are the 'have' commits.\n> Makes sense. I shortly wondered if we want to s/Y/Z/ as I find X and Z\n> more distinguishable than X/Y for reading/skimming.\n>\n> Thanks,\n> Stefan\n>\n>> +++ b/commit-reach.c\n>> @@ -593,3 +593,50 @@ int can_all_from_reach_with_flag(struct object_array *from,\n>> }\n>> return 1;\n>> }\n>> +\n>> +int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n>> + int cutoff_by_min_date)\n> We'll put this method (that is only used by tests so far) here to\n> not clutter the test tool code too much, or do we see more benefits\n> from the code\n> here? If so, docs would be nice.\n\nWe will use it later as we reduce duplicate walk implementations, but I \ncan hint at that in the message.\n\n\n>\n>> +++ b/t/t6600-test-reach.sh\n>> +test_expect_success 'can_all_from_reach:hit' '\n> [...]\n>> + Y:commit-7-3\n>> + Y:commit-8-1\n>> +test_expect_success 'can_all_from_reach:miss' '\n> [...]\n>> + Y:commit-8-5\n> It would be nice if the difference in the list could be easier\n> to spot as a reader. (There is a lot of repetition).\n>\n> Maybe we can teach \"test-tool reach\" to ignore input lines\n> starting with '#' such that we can annotate the last line in\n> the miss case?\n>\n> Why do we omit 7-3 in the miss case? (might be nice\n> for symmetry to keep around)\n\nThe X-commit that fails to reach a Y-commit in this second case is \ncommit-8-3. That commit can reach both commit-7-3 and commit-8-1, so \nboth need to be removed.\n\nThanks,\n\n-Stolee\n\n"},{"id":"352912","messageId":"xmqqfu0glbqm.fsf@gitster-ct.c.googlers.com","threadId":"48889","inReplyTo":"CACsJy8AbT3=wzKecGqvj8ibHmquU=NfP872f1cAM-VFOLSLSGQ@mail.gmail.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-07-18T17:01:05Z","receivedAt":"2018-07-18T17:01:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Duy Nguyen writes:\n\n>> In other words: if you see something that you don't like about\n>> GitGitGadget, get your butts off the ground and contribute a fix.\n>\n> Thank you for the frank words. I will choose to not review any mails\n> coming from GitGitGadget.\n\nI wouldn't say I will choose not to, but certainly I noticed that\nI'd backburner reading a series that are way out of order in my\nmailbox, no matter who authored them or how they were sent out, as\nthey consume way more concentration-point out of my mind than they\nare often worth X-<. While there are easier-to-read and more nicely\norganized patch series, I'd deal with them first, consciously or\nnot.\n\nNo, fixing a tool that throws such a harder-to-read patch series in\nreader's mailbox is *not* something I'd spend my primary focus on,\nespecially when many contributors are perfectly capable of sending\nreasonably formatted series without using such a tool under\ndevelopment.\n\nThat won't stop those who want to improve the tool. But I'd wish\nthose who want to make Git better spend their time on making Git,\nover making GitGitGadget, better.\n\n"},{"id":"352917","messageId":"afc090ea-840d-6c27-d6d0-2202914f1ccf@gmail.com","threadId":"48889","inReplyTo":"xmqqfu0glbqm.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-07-18T17:11:43Z","receivedAt":"2018-07-18T17:11:48Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/18/2018 1:01 PM, Junio C Hamano wrote:\n> No, fixing a tool that throws such a harder-to-read patch series in\n> reader's mailbox is *not* something I'd spend my primary focus on,\n> especially when many contributors are perfectly capable of sending\n> reasonably formatted series without using such a tool under\n> development.\n>\n> That won't stop those who want to improve the tool. But I'd wish\n> those who want to make Git better spend their time on making Git,\n> over making GitGitGadget, better.\n\nI appreciate the feedback in how this series caused reviewer pain. \nHopefully this date issue is now resolved. Any further feedback is welcome.\n\nI'm choosing to use and contribute to GitGitGadget not because I'm \nincapable of sending series myself, but because I _have_ had difficulty. \nUsing the email submissions creates a friction that I'm willing to \novercome, but we are probably missing out on contributors who are not \nwilling to push through that friction. Perhaps having another way for \nnew contributors to feel welcome is an indirect way to make Git better.\n\nThanks,\n\n-Stolee\n\n"},{"id":"352948","messageId":"20180718192128.GB7778@sigill.intra.peff.net","threadId":"48889","inReplyTo":"nycvar.QRO.7.76.6.1807181420250.71@tvgsbejvaqbjf.bet","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-07-18T19:21:28Z","receivedAt":"2018-07-18T19:21:32Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jul 18, 2018 at 02:23:11PM +0200, Johannes Schindelin wrote:\n\n> > Yeah, they're out of order in mutt's threaded display. And the\n> > back-dating means there's a much higher chance of them getting blocked\n> > as spam (e.g., some of the dates are from weeks ago).\n> > \n> > git-send-email uses the current time minus an offset, and then\n> > monotonically increases for each patch:\n> > \n> > $time = time - scalar $#files;\n> > ...\n> > my $date = format_2822_time($time++);\n> > \n> > which seems to work pretty well in practice. It does mean the original\n> > dates are lost. The committer date is not interesting at all (there will\n> > be a new committer via \"git am\" anyway). The original author date is\n> > potentially of interest, but could be included as an in-body header.\n> > AFAIK send-email doesn't have such an option, though, and people are\n> > fine with date-of-sending becoming the new author date.\n> > \n> > +cc Johannes as the GitGitGadget author\n> \n> Thanks for dumping even more work on my shoulders.\n\nWow. Here's my perspective on what I wrote.\n\nSomebody pointed out an issue in the tool. I tried to add an additional\ndata point (how other clients react, and that I've seen spam-related\nproblems). And I tried to point to an existing solution in another tool,\nin case that was helpful. I thought cc-ing you would be a favor, since\nyou obviously have an interest in the tool, and it is easy to miss\ndiscussions buried deep in a thread.\n\nSo no, I didn't write the patch for you. But I tried to contribute\npositively to the process. And I got yelled at for it. That makes me a\nlot less inclined to try to help in the future.\n\n> Next time, I will ask you to jump in, instead of putting the onus on me.\n>\n> I mean, seriously, what is this? \"You can use *any* mail program to work\n> with the Git mailing list, *any* mailer. As long as it is mutt. And as\n> long as you spend hours and hours on tooling that oh BTW nobody else can\n> use.\"\n\nThe irony here is that I actually _did_ look at the GitGitGadget\nrepository, and thought about making a patch to be helpful. But as it is\nwritten in a language I'm not all that familiar with, using tools that I\ndon't normally use, I didn't want to spend hours and hours in order to\nmake what was probably going to be a one-line patch in software that I\ndon't use myself.\n\n-Peff\n"},{"id":"353033","messageId":"nycvar.QRO.7.76.6.1807191831410.71@tvgsbejvaqbjf.bet","threadId":"48889","inReplyTo":"xmqqfu0glbqm.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2018-07-19T16:32:15Z","receivedAt":"2018-07-19T16:32:42Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Wed, 18 Jul 2018, Junio C Hamano wrote:\n\n> That won't stop those who want to improve the tool. But I'd wish\n> those who want to make Git better spend their time on making Git,\n> over making GitGitGadget, better.\n\nAnd I'd wish that you would not make this task harder by refusing to fix\nyour process to update refs/notes/amlog.\n\nThanks,\nDscho\n"},{"id":"353035","messageId":"nycvar.QRO.7.76.6.1807191833411.71@tvgsbejvaqbjf.bet","threadId":"48889","inReplyTo":"20180718192128.GB7778@sigill.intra.peff.net","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2018-07-19T16:34:09Z","receivedAt":"2018-07-19T16:34:38Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Wed, 18 Jul 2018, Jeff King wrote:\n\n> On Wed, Jul 18, 2018 at 02:23:11PM +0200, Johannes Schindelin wrote:\n> \n> > > Yeah, they're out of order in mutt's threaded display. And the\n> > > back-dating means there's a much higher chance of them getting blocked\n> > > as spam (e.g., some of the dates are from weeks ago).\n> > > \n> > > git-send-email uses the current time minus an offset, and then\n> > > monotonically increases for each patch:\n> > > \n> > > $time = time - scalar $#files;\n> > > ...\n> > > my $date = format_2822_time($time++);\n> > > \n> > > which seems to work pretty well in practice. It does mean the original\n> > > dates are lost. The committer date is not interesting at all (there will\n> > > be a new committer via \"git am\" anyway). The original author date is\n> > > potentially of interest, but could be included as an in-body header.\n> > > AFAIK send-email doesn't have such an option, though, and people are\n> > > fine with date-of-sending becoming the new author date.\n> > > \n> > > +cc Johannes as the GitGitGadget author\n> > \n> > Thanks for dumping even more work on my shoulders.\n> \n> Wow. Here's my perspective on what I wrote.\n> \n> Somebody pointed out an issue in the tool. I tried to add an additional\n> data point (how other clients react, and that I've seen spam-related\n> problems). And I tried to point to an existing solution in another tool,\n> in case that was helpful. I thought cc-ing you would be a favor, since\n> you obviously have an interest in the tool, and it is easy to miss\n> discussions buried deep in a thread.\n> \n> So no, I didn't write the patch for you. But I tried to contribute\n> positively to the process. And I got yelled at for it. That makes me a\n> lot less inclined to try to help in the future.\n> \n> > Next time, I will ask you to jump in, instead of putting the onus on me.\n> >\n> > I mean, seriously, what is this? \"You can use *any* mail program to work\n> > with the Git mailing list, *any* mailer. As long as it is mutt. And as\n> > long as you spend hours and hours on tooling that oh BTW nobody else can\n> > use.\"\n> \n> The irony here is that I actually _did_ look at the GitGitGadget\n> repository, and thought about making a patch to be helpful. But as it is\n> written in a language I'm not all that familiar with, using tools that I\n> don't normally use, I didn't want to spend hours and hours in order to\n> make what was probably going to be a one-line patch in software that I\n> don't use myself.\n\nI understand that. The web is not based on shell scripting, so there is no\ngood way to implement a bot on GitHub using Bash scripts.\n\nCiao,\nDscho\n"},{"id":"353037","messageId":"nycvar.QRO.7.76.6.1807191834390.71@tvgsbejvaqbjf.bet","threadId":"48889","inReplyTo":"afc090ea-840d-6c27-d6d0-2202914f1ccf@gmail.com","subject":"Re: [PATCH 00/16] Consolidate reachability logic","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2018-07-19T16:37:42Z","receivedAt":"2018-07-19T16:38:09Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Stolee,\n\nOn Wed, 18 Jul 2018, Derrick Stolee wrote:\n\n> On 7/18/2018 1:01 PM, Junio C Hamano wrote:\n> > No, fixing a tool that throws such a harder-to-read patch series in\n> > reader's mailbox is *not* something I'd spend my primary focus on,\n> > especially when many contributors are perfectly capable of sending\n> > reasonably formatted series without using such a tool under\n> > development.\n> >\n> > That won't stop those who want to improve the tool. But I'd wish\n> > those who want to make Git better spend their time on making Git,\n> > over making GitGitGadget, better.\n> \n> I appreciate the feedback in how this series caused reviewer pain. Hopefully\n> this date issue is now resolved. Any further feedback is welcome.\n> \n> I'm choosing to use and contribute to GitGitGadget not because I'm incapable\n> of sending series myself, but because I _have_ had difficulty. Using the email\n> submissions creates a friction that I'm willing to overcome, but we are\n> probably missing out on contributors who are not willing to push through that\n> friction. Perhaps having another way for new contributors to feel welcome is\n> an indirect way to make Git better.\n\nWhile I am a seasoned Git contributor, it is *still* too painful to\ncontribute patches *even for me*.\n\nSo hopefully you and I will get this easier contribution process to the\npoint where other oldtimers do not want to take it.\n\nAt least we now have something that does not share the downsides with\nSubmitGit, and is extensible enough that we can teach it new tricks.\n\nWith a little luck, Junio will fix amlog so that it is not utter garbage\nfor anybody but himself, and then GitGitGadget can give contributors\nuseful feedback about the state of their patch series, including automated\nnotifications when their patches have been mentioned in the What's Cooking\nmail (which no irregular contributor reads, as far as I know).\n\nCiao,\nDscho\n"},{"id":"353130","messageId":"20180720163227.105950-3-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 02/18] commit.h: remove method declarations","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:04Z","receivedAt":"2018-07-20T16:33:15Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"These methods are now declared in commit-reach.h. Remove them from\ncommit.h and add new include statements in all files that require these\ndeclarations.\n\nSigned-off-by: Derrick Stolee \n---\n bisect.c | 1 +\n builtin/branch.c | 1 +\n builtin/commit.c | 1 +\n builtin/fetch.c | 1 +\n builtin/fmt-merge-msg.c | 1 +\n builtin/log.c | 1 +\n builtin/merge-base.c | 1 +\n builtin/merge.c | 1 +\n builtin/pull.c | 1 +\n builtin/receive-pack.c | 1 +\n builtin/rev-parse.c | 1 +\n commit.h | 29 -----------------------------\n fast-import.c | 1 +\n http-push.c | 2 +-\n merge-recursive.c | 1 +\n notes-merge.c | 1 +\n pack-bitmap-write.c | 1 +\n ref-filter.c | 1 +\n remote.c | 1 +\n revision.c | 1 +\n sequencer.c | 1 +\n sha1-name.c | 1 +\n shallow.c | 1 +\n submodule.c | 1 +\n 24 files changed, 23 insertions(+), 30 deletions(-)\n\ndiff --git a/bisect.c b/bisect.c\nindex e1275ba79e..d023543c91 100644\n--- a/bisect.c\n+++ b/bisect.c\n@@ -13,6 +13,7 @@\n #include \"sha1-array.h\"\n #include \"argv-array.h\"\n #include \"commit-slab.h\"\n+#include \"commit-reach.h\"\n \n static struct oid_array good_revs;\n static struct oid_array skipped_revs;\ndiff --git a/builtin/branch.c b/builtin/branch.c\nindex a50632fb23..9a787447f4 100644\n--- a/builtin/branch.c\n+++ b/builtin/branch.c\n@@ -23,6 +23,7 @@\n #include \"ref-filter.h\"\n #include \"worktree.h\"\n #include \"help.h\"\n+#include \"commit-reach.h\"\n \n static const char * const builtin_branch_usage[] = {\n \tN_(\"git branch [] [-r | -a] [--merged | --no-merged]\"),\ndiff --git a/builtin/commit.c b/builtin/commit.c\nindex 158e3f843a..b5c608458e 100644\n--- a/builtin/commit.c\n+++ b/builtin/commit.c\n@@ -33,6 +33,7 @@\n #include \"sequencer.h\"\n #include \"mailmap.h\"\n #include \"help.h\"\n+#include \"commit-reach.h\"\n \n static const char * const builtin_commit_usage[] = {\n \tN_(\"git commit [] [--] ...\"),\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex f5d960baec..7de234774b 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -22,6 +22,7 @@\n #include \"utf8.h\"\n #include \"packfile.h\"\n #include \"list-objects-filter-options.h\"\n+#include \"commit-reach.h\"\n \n static const char * const builtin_fetch_usage[] = {\n \tN_(\"git fetch [] [ [...]]\"),\ndiff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c\nindex ff165c0fcd..7277d557b2 100644\n--- a/builtin/fmt-merge-msg.c\n+++ b/builtin/fmt-merge-msg.c\n@@ -12,6 +12,7 @@\n #include \"fmt-merge-msg.h\"\n #include \"gpg-interface.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n static const char * const fmt_merge_msg_usage[] = {\n \tN_(\"git fmt-merge-msg [-m ] [--log[=] | --no-log] [--file ]\"),\ndiff --git a/builtin/log.c b/builtin/log.c\nindex 55a6286d7f..333d97c692 100644\n--- a/builtin/log.c\n+++ b/builtin/log.c\n@@ -31,6 +31,7 @@\n #include \"progress.h\"\n #include \"commit-slab.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n #define MAIL_DEFAULT_WRAP 72\n \ndiff --git a/builtin/merge-base.c b/builtin/merge-base.c\nindex 08d91b1f0c..1c92099070 100644\n--- a/builtin/merge-base.c\n+++ b/builtin/merge-base.c\n@@ -7,6 +7,7 @@\n #include \"revision.h\"\n #include \"parse-options.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n static int show_merge_base(struct commit **rev, int rev_nr, int show_all)\n {\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex d1b547d973..4c601c40a2 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -36,6 +36,7 @@\n #include \"packfile.h\"\n #include \"tag.h\"\n #include \"alias.h\"\n+#include \"commit-reach.h\"\n \n #define DEFAULT_TWOHEAD (1<<0)\n #define DEFAULT_OCTOPUS (1<<1)\ndiff --git a/builtin/pull.c b/builtin/pull.c\nindex 4e78935392..15ad010968 100644\n--- a/builtin/pull.c\n+++ b/builtin/pull.c\n@@ -22,6 +22,7 @@\n #include \"tempfile.h\"\n #include \"lockfile.h\"\n #include \"wt-status.h\"\n+#include \"commit-reach.h\"\n \n enum rebase_type {\n \tREBASE_INVALID = -1,\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex 400d31c18c..d8467f9734 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -27,6 +27,7 @@\n #include \"packfile.h\"\n #include \"object-store.h\"\n #include \"protocol.h\"\n+#include \"commit-reach.h\"\n \n static const char * const receive_pack_usage[] = {\n \tN_(\"git receive-pack \"),\ndiff --git a/builtin/rev-parse.c b/builtin/rev-parse.c\nindex 0f09bbbf65..455f62246d 100644\n--- a/builtin/rev-parse.c\n+++ b/builtin/rev-parse.c\n@@ -14,6 +14,7 @@\n #include \"revision.h\"\n #include \"split-index.h\"\n #include \"submodule.h\"\n+#include \"commit-reach.h\"\n \n #define DO_REVS\t\t1\n #define DO_NOREV\t2\ndiff --git a/commit.h b/commit.h\nindex da0db36eba..e2c99d9b04 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -204,13 +204,6 @@ struct commit_graft *read_graft_line(struct strbuf *line);\n int register_commit_graft(struct repository *r, struct commit_graft *, int);\n struct commit_graft *lookup_commit_graft(struct repository *r, const struct object_id *oid);\n \n-extern struct commit_list *get_merge_bases(struct commit *rev1, struct commit *rev2);\n-extern struct commit_list *get_merge_bases_many(struct commit *one, int n, struct commit **twos);\n-extern struct commit_list *get_octopus_merge_bases(struct commit_list *in);\n-\n-/* To be used only when object flags after this call no longer matter */\n-extern struct commit_list *get_merge_bases_many_dirty(struct commit *one, int n, struct commit **twos);\n-\n /* largest positive number a signed 32-bit integer can contain */\n #define INFINITE_DEPTH 0x7fffffff\n \n@@ -258,32 +251,10 @@ extern int delayed_reachability_test(struct shallow_info *si, int c);\n extern void prune_shallow(int show_only);\n extern struct trace_key trace_shallow;\n \n-int is_descendant_of(struct commit *, struct commit_list *);\n-int in_merge_bases(struct commit *, struct commit *);\n-int in_merge_bases_many(struct commit *, int, struct commit **);\n-\n extern int interactive_add(int argc, const char **argv, const char *prefix, int patch);\n extern int run_add_interactive(const char *revision, const char *patch_mode,\n \t\t\t const struct pathspec *pathspec);\n \n-/*\n- * Takes a list of commits and returns a new list where those\n- * have been removed that can be reached from other commits in\n- * the list. It is useful for, e.g., reducing the commits\n- * randomly thrown at the git-merge command and removing\n- * redundant commits that the user shouldn't have given to it.\n- *\n- * This function destroys the STALE bit of the commit objects'\n- * flags.\n- */\n-extern struct commit_list *reduce_heads(struct commit_list *heads);\n-\n-/*\n- * Like `reduce_heads()`, except it replaces the list. Use this\n- * instead of `foo = reduce_heads(foo);` to avoid memory leaks.\n- */\n-extern void reduce_heads_replace(struct commit_list **heads);\n-\n struct commit_extra_header {\n \tstruct commit_extra_header *next;\n \tchar *key;\ndiff --git a/fast-import.c b/fast-import.c\nindex 3ea5781029..4a93df3839 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -171,6 +171,7 @@ Format of STDIN stream:\n #include \"packfile.h\"\n #include \"object-store.h\"\n #include \"mem-pool.h\"\n+#include \"commit-reach.h\"\n \n #define PACK_ID_BITS 16\n #define MAX_PACK_ID ((1<\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 1dd6ec384d..8155dee9a9 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -27,6 +27,7 @@\n #include \"dir.h\"\n #include \"submodule.h\"\n #include \"revision.h\"\n+#include \"commit-reach.h\"\n \n struct path_hashmap_entry {\n \tstruct hashmap_entry e;\ndiff --git a/notes-merge.c b/notes-merge.c\nindex 76ab19e702..12dfdf6c17 100644\n--- a/notes-merge.c\n+++ b/notes-merge.c\n@@ -12,6 +12,7 @@\n #include \"notes-merge.h\"\n #include \"strbuf.h\"\n #include \"notes-utils.h\"\n+#include \"commit-reach.h\"\n \n struct notes_merge_pair {\n \tstruct object_id obj, base, local, remote;\ndiff --git a/pack-bitmap-write.c b/pack-bitmap-write.c\nindex 953c5dd84d..55bcab907c 100644\n--- a/pack-bitmap-write.c\n+++ b/pack-bitmap-write.c\n@@ -11,6 +11,7 @@\n #include \"pack-bitmap.h\"\n #include \"sha1-lookup.h\"\n #include \"pack-objects.h\"\n+#include \"commit-reach.h\"\n \n struct bitmapped_commit {\n \tstruct commit *commit;\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 9b2da88392..fca3ad040a 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -19,6 +19,7 @@\n #include \"wt-status.h\"\n #include \"commit-slab.h\"\n #include \"commit-graph.h\"\n+#include \"commit-reach.h\"\n \n static struct ref_msg {\n \tconst char *gone;\ndiff --git a/remote.c b/remote.c\nindex 26b1fbd9a8..8e99b9888a 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -12,6 +12,7 @@\n #include \"string-list.h\"\n #include \"mergesort.h\"\n #include \"argv-array.h\"\n+#include \"commit-reach.h\"\n \n enum map_direction { FROM_SRC, FROM_DST };\n \ndiff --git a/revision.c b/revision.c\nindex 4dbe406bed..3205a3947a 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -24,6 +24,7 @@\n #include \"packfile.h\"\n #include \"worktree.h\"\n #include \"argv-array.h\"\n+#include \"commit-reach.h\"\n \n volatile show_early_output_fn_t show_early_output;\n \ndiff --git a/sequencer.c b/sequencer.c\nindex d1d07bed5b..97bdfd48b4 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -30,6 +30,7 @@\n #include \"oidset.h\"\n #include \"commit-slab.h\"\n #include \"alias.h\"\n+#include \"commit-reach.h\"\n \n #define GIT_REFLOG_ACTION \"GIT_REFLOG_ACTION\"\n \ndiff --git a/sha1-name.c b/sha1-name.c\nindex 009faab4ae..7215b30b88 100644\n--- a/sha1-name.c\n+++ b/sha1-name.c\n@@ -12,6 +12,7 @@\n #include \"packfile.h\"\n #include \"object-store.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n \ndiff --git a/shallow.c b/shallow.c\nindex dbe8a2a290..99fd2d1ba0 100644\n--- a/shallow.c\n+++ b/shallow.c\n@@ -16,6 +16,7 @@\n #include \"list-objects.h\"\n #include \"commit-slab.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n void set_alternate_shallow_file(struct repository *r, const char *path, int override)\n {\ndiff --git a/submodule.c b/submodule.c\nindex 6688dd5d45..6650ed7aa0 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -22,6 +22,7 @@\n #include \"worktree.h\"\n #include \"parse-options.h\"\n #include \"object-store.h\"\n+#include \"commit-reach.h\"\n \n static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;\n static struct string_list changed_submodule_names = STRING_LIST_INIT_DUP;\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353131","messageId":"20180720163227.105950-4-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 03/18] commit-reach: move ref_newer from remote.c","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:06Z","receivedAt":"2018-07-20T16:33:15Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"There are several commit walks in the codebase. Group them together into\na new commit-reach.c file and corresponding header. After we group these\nwalks into one place, we can reduce duplicate logic by calling\nequivalent methods.\n\nThe ref_newer() method is used by 'git push -f' to check if a force-push\nis necessary. By making the method public, we make it possible to test\nthe method directly without setting up an envieronment where a 'git\npush' call makes sense.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/remote.c | 1 +\n commit-reach.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++-\n commit-reach.h | 2 ++\n remote.c | 49 ------------------------------------------\n remote.h | 1 -\n 5 files changed, 57 insertions(+), 51 deletions(-)\n\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex c74ee88690..79b0326446 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -10,6 +10,7 @@\n #include \"refspec.h\"\n #include \"object-store.h\"\n #include \"argv-array.h\"\n+#include \"commit-reach.h\"\n \n static const char * const builtin_remote_usage[] = {\n \tN_(\"git remote [-v | --verbose]\"),\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 8ab6044414..a6bc4781a6 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -1,6 +1,10 @@\n #include \"cache.h\"\n-#include \"prio-queue.h\"\n #include \"commit.h\"\n+#include \"decorate.h\"\n+#include \"prio-queue.h\"\n+#include \"tree.h\"\n+#include \"revision.h\"\n+#include \"tag.h\"\n #include \"commit-reach.h\"\n \n /* Remember to update object flag allocation in object.h */\n@@ -358,3 +362,52 @@ void reduce_heads_replace(struct commit_list **heads)\n \tfree_commit_list(*heads);\n \t*heads = result;\n }\n+\n+static void unmark_and_free(struct commit_list *list, unsigned int mark)\n+{\n+\twhile (list) {\n+\t\tstruct commit *commit = pop_commit(&list);\n+\t\tcommit->object.flags &= ~mark;\n+\t}\n+}\n+\n+int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n+{\n+\tstruct object *o;\n+\tstruct commit *old_commit, *new_commit;\n+\tstruct commit_list *list, *used;\n+\tint found = 0;\n+\n+\t/*\n+\t * Both new_commit and old_commit must be commit-ish and new_commit is descendant of\n+\t * old_commit. Otherwise we require --force.\n+\t */\n+\to = deref_tag(the_repository, parse_object(the_repository, old_oid),\n+\t\t NULL, 0);\n+\tif (!o || o->type != OBJ_COMMIT)\n+\t\treturn 0;\n+\told_commit = (struct commit *) o;\n+\n+\to = deref_tag(the_repository, parse_object(the_repository, new_oid),\n+\t\t NULL, 0);\n+\tif (!o || o->type != OBJ_COMMIT)\n+\t\treturn 0;\n+\tnew_commit = (struct commit *) o;\n+\n+\tif (parse_commit(new_commit) < 0)\n+\t\treturn 0;\n+\n+\tused = list = NULL;\n+\tcommit_list_insert(new_commit, &list);\n+\twhile (list) {\n+\t\tnew_commit = pop_most_recent_commit(&list, TMP_MARK);\n+\t\tcommit_list_insert(new_commit, &used);\n+\t\tif (new_commit == old_commit) {\n+\t\t\tfound = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tunmark_and_free(list, TMP_MARK);\n+\tunmark_and_free(used, TMP_MARK);\n+\treturn found;\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nindex 1ea2696e40..f1cf9bfcd8 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -39,4 +39,6 @@ struct commit_list *reduce_heads(struct commit_list *heads);\n */\n void reduce_heads_replace(struct commit_list **heads);\n \n+int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid);\n+\n #endif\ndiff --git a/remote.c b/remote.c\nindex 8e99b9888a..f0c23bae48 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -1784,55 +1784,6 @@ int resolve_remote_symref(struct ref *ref, struct ref *list)\n \treturn 1;\n }\n \n-static void unmark_and_free(struct commit_list *list, unsigned int mark)\n-{\n-\twhile (list) {\n-\t\tstruct commit *commit = pop_commit(&list);\n-\t\tcommit->object.flags &= ~mark;\n-\t}\n-}\n-\n-int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n-{\n-\tstruct object *o;\n-\tstruct commit *old_commit, *new_commit;\n-\tstruct commit_list *list, *used;\n-\tint found = 0;\n-\n-\t/*\n-\t * Both new_commit and old_commit must be commit-ish and new_commit is descendant of\n-\t * old_commit. Otherwise we require --force.\n-\t */\n-\to = deref_tag(the_repository, parse_object(the_repository, old_oid),\n-\t\t NULL, 0);\n-\tif (!o || o->type != OBJ_COMMIT)\n-\t\treturn 0;\n-\told_commit = (struct commit *) o;\n-\n-\to = deref_tag(the_repository, parse_object(the_repository, new_oid),\n-\t\t NULL, 0);\n-\tif (!o || o->type != OBJ_COMMIT)\n-\t\treturn 0;\n-\tnew_commit = (struct commit *) o;\n-\n-\tif (parse_commit(new_commit) < 0)\n-\t\treturn 0;\n-\n-\tused = list = NULL;\n-\tcommit_list_insert(new_commit, &list);\n-\twhile (list) {\n-\t\tnew_commit = pop_most_recent_commit(&list, TMP_MARK);\n-\t\tcommit_list_insert(new_commit, &used);\n-\t\tif (new_commit == old_commit) {\n-\t\t\tfound = 1;\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tunmark_and_free(list, TMP_MARK);\n-\tunmark_and_free(used, TMP_MARK);\n-\treturn found;\n-}\n-\n /*\n * Lookup the upstream branch for the given branch and if present, optionally\n * compute the commit ahead/behind values for the pair.\ndiff --git a/remote.h b/remote.h\nindex 45ecc6cefa..56fb9cbb27 100644\n--- a/remote.h\n+++ b/remote.h\n@@ -149,7 +149,6 @@ extern struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,\n \t\t\t\t const struct string_list *server_options);\n \n int resolve_remote_symref(struct ref *ref, struct ref *list);\n-int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid);\n \n /*\n * Remove and free all but the first of any entries in the input list\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353132","messageId":"20180720163227.105950-5-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 04/18] commit-reach: move commit_contains from ref-filter","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:08Z","receivedAt":"2018-07-20T16:33:17Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"There are several commit walks in the codebase. Group them together into\na new commit-reach.c file and corresponding header. After we group these\nwalks into one place, we can reduce duplicate logic by calling\nequivalent methods.\n\nAll methods are direct moves, except we also make the commit_contains()\nmethod public so its consumers in ref-filter.c can still call it. We can\nalso test this method in a test-tool in a later commit.\n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 121 +++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 20 ++++++-\n ref-filter.c | 145 +++----------------------------------------------\n 3 files changed, 147 insertions(+), 139 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex a6bc4781a6..01d796f011 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -1,8 +1,10 @@\n #include \"cache.h\"\n #include \"commit.h\"\n+#include \"commit-graph.h\"\n #include \"decorate.h\"\n #include \"prio-queue.h\"\n #include \"tree.h\"\n+#include \"ref-filter.c\"\n #include \"revision.h\"\n #include \"tag.h\"\n #include \"commit-reach.h\"\n@@ -411,3 +413,122 @@ int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n \tunmark_and_free(used, TMP_MARK);\n \treturn found;\n }\n+\n+/*\n+ * Mimicking the real stack, this stack lives on the heap, avoiding stack\n+ * overflows.\n+ *\n+ * At each recursion step, the stack items points to the commits whose\n+ * ancestors are to be inspected.\n+ */\n+struct contains_stack {\n+\tint nr, alloc;\n+\tstruct contains_stack_entry {\n+\t\tstruct commit *commit;\n+\t\tstruct commit_list *parents;\n+\t} *contains_stack;\n+};\n+\n+static int in_commit_list(const struct commit_list *want, struct commit *c)\n+{\n+\tfor (; want; want = want->next)\n+\t\tif (!oidcmp(&want->item->object.oid, &c->object.oid))\n+\t\t\treturn 1;\n+\treturn 0;\n+}\n+\n+/*\n+ * Test whether the candidate is contained in the list.\n+ * Do not recurse to find out, though, but return -1 if inconclusive.\n+ */\n+static enum contains_result contains_test(struct commit *candidate,\n+\t\t\t\t\t const struct commit_list *want,\n+\t\t\t\t\t struct contains_cache *cache,\n+\t\t\t\t\t uint32_t cutoff)\n+{\n+\tenum contains_result *cached = contains_cache_at(cache, candidate);\n+\n+\t/* If we already have the answer cached, return that. */\n+\tif (*cached)\n+\t\treturn *cached;\n+\n+\t/* or are we it? */\n+\tif (in_commit_list(want, candidate)) {\n+\t\t*cached = CONTAINS_YES;\n+\t\treturn CONTAINS_YES;\n+\t}\n+\n+\t/* Otherwise, we don't know; prepare to recurse */\n+\tparse_commit_or_die(candidate);\n+\n+\tif (candidate->generation < cutoff)\n+\t\treturn CONTAINS_NO;\n+\n+\treturn CONTAINS_UNKNOWN;\n+}\n+\n+static void push_to_contains_stack(struct commit *candidate, struct contains_stack *contains_stack)\n+{\n+\tALLOC_GROW(contains_stack->contains_stack, contains_stack->nr + 1, contains_stack->alloc);\n+\tcontains_stack->contains_stack[contains_stack->nr].commit = candidate;\n+\tcontains_stack->contains_stack[contains_stack->nr++].parents = candidate->parents;\n+}\n+\n+static enum contains_result contains_tag_algo(struct commit *candidate,\n+\t\t\t\t\t const struct commit_list *want,\n+\t\t\t\t\t struct contains_cache *cache)\n+{\n+\tstruct contains_stack contains_stack = { 0, 0, NULL };\n+\tenum contains_result result;\n+\tuint32_t cutoff = GENERATION_NUMBER_INFINITY;\n+\tconst struct commit_list *p;\n+\n+\tfor (p = want; p; p = p->next) {\n+\t\tstruct commit *c = p->item;\n+\t\tload_commit_graph_info(the_repository, c);\n+\t\tif (c->generation < cutoff)\n+\t\t\tcutoff = c->generation;\n+\t}\n+\n+\tresult = contains_test(candidate, want, cache, cutoff);\n+\tif (result != CONTAINS_UNKNOWN)\n+\t\treturn result;\n+\n+\tpush_to_contains_stack(candidate, &contains_stack);\n+\twhile (contains_stack.nr) {\n+\t\tstruct contains_stack_entry *entry = &contains_stack.contains_stack[contains_stack.nr - 1];\n+\t\tstruct commit *commit = entry->commit;\n+\t\tstruct commit_list *parents = entry->parents;\n+\n+\t\tif (!parents) {\n+\t\t\t*contains_cache_at(cache, commit) = CONTAINS_NO;\n+\t\t\tcontains_stack.nr--;\n+\t\t}\n+\t\t/*\n+\t\t * If we just popped the stack, parents->item has been marked,\n+\t\t * therefore contains_test will return a meaningful yes/no.\n+\t\t */\n+\t\telse switch (contains_test(parents->item, want, cache, cutoff)) {\n+\t\tcase CONTAINS_YES:\n+\t\t\t*contains_cache_at(cache, commit) = CONTAINS_YES;\n+\t\t\tcontains_stack.nr--;\n+\t\t\tbreak;\n+\t\tcase CONTAINS_NO:\n+\t\t\tentry->parents = parents->next;\n+\t\t\tbreak;\n+\t\tcase CONTAINS_UNKNOWN:\n+\t\t\tpush_to_contains_stack(parents->item, &contains_stack);\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tfree(contains_stack.contains_stack);\n+\treturn contains_test(candidate, want, cache, cutoff);\n+}\n+\n+int commit_contains(struct ref_filter *filter, struct commit *commit,\n+\t\t struct commit_list *list, struct contains_cache *cache)\n+{\n+\tif (filter->with_commit_tag_algo)\n+\t\treturn contains_tag_algo(commit, list, cache) == CONTAINS_YES;\n+\treturn is_descendant_of(commit, list);\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nindex f1cf9bfcd8..13dec25cee 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -1,8 +1,12 @@\n #ifndef __COMMIT_REACH_H__\n #define __COMMIT_REACH_H__\n \n+#include \"commit-slab.h\"\n+\n struct commit;\n struct commit_list;\n+struct contains_cache;\n+struct ref_filter;\n \n struct commit_list *get_merge_bases_many(struct commit *one,\n \t\t\t\t\t int n,\n@@ -20,7 +24,6 @@ int is_descendant_of(struct commit *commit, struct commit_list *with_commit);\n int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit **reference);\n int in_merge_bases(struct commit *commit, struct commit *reference);\n \n-\n /*\n * Takes a list of commits and returns a new list where those\n * have been removed that can be reached from other commits in\n@@ -41,4 +44,19 @@ void reduce_heads_replace(struct commit_list **heads);\n \n int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid);\n \n+/*\n+ * Unknown has to be \"0\" here, because that's the default value for\n+ * contains_cache slab entries that have not yet been assigned.\n+ */\n+enum contains_result {\n+\tCONTAINS_UNKNOWN = 0,\n+\tCONTAINS_NO,\n+\tCONTAINS_YES\n+};\n+\n+define_commit_slab(contains_cache, enum contains_result);\n+\n+int commit_contains(struct ref_filter *filter, struct commit *commit,\n+\t\t struct commit_list *list, struct contains_cache *cache);\n+\n #endif\ndiff --git a/ref-filter.c b/ref-filter.c\nindex fca3ad040a..495e830fa5 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -1624,144 +1624,6 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom,\n \treturn 0;\n }\n \n-/*\n- * Unknown has to be \"0\" here, because that's the default value for\n- * contains_cache slab entries that have not yet been assigned.\n- */\n-enum contains_result {\n-\tCONTAINS_UNKNOWN = 0,\n-\tCONTAINS_NO,\n-\tCONTAINS_YES\n-};\n-\n-define_commit_slab(contains_cache, enum contains_result);\n-\n-struct ref_filter_cbdata {\n-\tstruct ref_array *array;\n-\tstruct ref_filter *filter;\n-\tstruct contains_cache contains_cache;\n-\tstruct contains_cache no_contains_cache;\n-};\n-\n-/*\n- * Mimicking the real stack, this stack lives on the heap, avoiding stack\n- * overflows.\n- *\n- * At each recursion step, the stack items points to the commits whose\n- * ancestors are to be inspected.\n- */\n-struct contains_stack {\n-\tint nr, alloc;\n-\tstruct contains_stack_entry {\n-\t\tstruct commit *commit;\n-\t\tstruct commit_list *parents;\n-\t} *contains_stack;\n-};\n-\n-static int in_commit_list(const struct commit_list *want, struct commit *c)\n-{\n-\tfor (; want; want = want->next)\n-\t\tif (!oidcmp(&want->item->object.oid, &c->object.oid))\n-\t\t\treturn 1;\n-\treturn 0;\n-}\n-\n-/*\n- * Test whether the candidate is contained in the list.\n- * Do not recurse to find out, though, but return -1 if inconclusive.\n- */\n-static enum contains_result contains_test(struct commit *candidate,\n-\t\t\t\t\t const struct commit_list *want,\n-\t\t\t\t\t struct contains_cache *cache,\n-\t\t\t\t\t uint32_t cutoff)\n-{\n-\tenum contains_result *cached = contains_cache_at(cache, candidate);\n-\n-\t/* If we already have the answer cached, return that. */\n-\tif (*cached)\n-\t\treturn *cached;\n-\n-\t/* or are we it? */\n-\tif (in_commit_list(want, candidate)) {\n-\t\t*cached = CONTAINS_YES;\n-\t\treturn CONTAINS_YES;\n-\t}\n-\n-\t/* Otherwise, we don't know; prepare to recurse */\n-\tparse_commit_or_die(candidate);\n-\n-\tif (candidate->generation < cutoff)\n-\t\treturn CONTAINS_NO;\n-\n-\treturn CONTAINS_UNKNOWN;\n-}\n-\n-static void push_to_contains_stack(struct commit *candidate, struct contains_stack *contains_stack)\n-{\n-\tALLOC_GROW(contains_stack->contains_stack, contains_stack->nr + 1, contains_stack->alloc);\n-\tcontains_stack->contains_stack[contains_stack->nr].commit = candidate;\n-\tcontains_stack->contains_stack[contains_stack->nr++].parents = candidate->parents;\n-}\n-\n-static enum contains_result contains_tag_algo(struct commit *candidate,\n-\t\t\t\t\t const struct commit_list *want,\n-\t\t\t\t\t struct contains_cache *cache)\n-{\n-\tstruct contains_stack contains_stack = { 0, 0, NULL };\n-\tenum contains_result result;\n-\tuint32_t cutoff = GENERATION_NUMBER_INFINITY;\n-\tconst struct commit_list *p;\n-\n-\tfor (p = want; p; p = p->next) {\n-\t\tstruct commit *c = p->item;\n-\t\tload_commit_graph_info(the_repository, c);\n-\t\tif (c->generation < cutoff)\n-\t\t\tcutoff = c->generation;\n-\t}\n-\n-\tresult = contains_test(candidate, want, cache, cutoff);\n-\tif (result != CONTAINS_UNKNOWN)\n-\t\treturn result;\n-\n-\tpush_to_contains_stack(candidate, &contains_stack);\n-\twhile (contains_stack.nr) {\n-\t\tstruct contains_stack_entry *entry = &contains_stack.contains_stack[contains_stack.nr - 1];\n-\t\tstruct commit *commit = entry->commit;\n-\t\tstruct commit_list *parents = entry->parents;\n-\n-\t\tif (!parents) {\n-\t\t\t*contains_cache_at(cache, commit) = CONTAINS_NO;\n-\t\t\tcontains_stack.nr--;\n-\t\t}\n-\t\t/*\n-\t\t * If we just popped the stack, parents->item has been marked,\n-\t\t * therefore contains_test will return a meaningful yes/no.\n-\t\t */\n-\t\telse switch (contains_test(parents->item, want, cache, cutoff)) {\n-\t\tcase CONTAINS_YES:\n-\t\t\t*contains_cache_at(cache, commit) = CONTAINS_YES;\n-\t\t\tcontains_stack.nr--;\n-\t\t\tbreak;\n-\t\tcase CONTAINS_NO:\n-\t\t\tentry->parents = parents->next;\n-\t\t\tbreak;\n-\t\tcase CONTAINS_UNKNOWN:\n-\t\t\tpush_to_contains_stack(parents->item, &contains_stack);\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tfree(contains_stack.contains_stack);\n-\treturn contains_test(candidate, want, cache, cutoff);\n-}\n-\n-static int commit_contains(struct ref_filter *filter, struct commit *commit,\n-\t\t\t struct commit_list *list, struct contains_cache *cache)\n-{\n-\tif (filter->with_commit_tag_algo)\n-\t\treturn contains_tag_algo(commit, list, cache) == CONTAINS_YES;\n-\treturn is_descendant_of(commit, list);\n-}\n-\n /*\n * Return 1 if the refname matches one of the patterns, otherwise 0.\n * A pattern can be a literal prefix (e.g. a refname \"refs/heads/master\"\n@@ -1988,6 +1850,13 @@ static int filter_ref_kind(struct ref_filter *filter, const char *refname)\n \treturn ref_kind_from_refname(refname);\n }\n \n+struct ref_filter_cbdata {\n+\tstruct ref_array *array;\n+\tstruct ref_filter *filter;\n+\tstruct contains_cache contains_cache;\n+\tstruct contains_cache no_contains_cache;\n+};\n+\n /*\n * A call-back given to for_each_ref(). Filter refs and keep them for\n * later object processing.\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353133","messageId":"20180720163227.105950-6-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 05/18] upload-pack: make reachable() more generic","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:09Z","receivedAt":"2018-07-20T16:33:17Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"In anticipation of moving the reachable() method to commit-reach.c,\nmodify the prototype to be more generic to flags known outside of\nupload-pack.c. Also rename 'want' to 'from' to make the statement\nmore clear outside of the context of haves/wants negotiation.\n\nSigned-off-by: Derrick Stolee \n---\n upload-pack.c | 17 +++++++++--------\n 1 file changed, 9 insertions(+), 8 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 4ca052d0b6..5a639cb47b 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -336,17 +336,18 @@ static int got_oid(const char *hex, struct object_id *oid)\n \treturn 0;\n }\n \n-static int reachable(struct commit *want)\n+static int reachable(struct commit *from, unsigned int with_flag,\n+\t\t unsigned int assign_flag)\n {\n \tstruct prio_queue work = { compare_commits_by_commit_date };\n \n-\tprio_queue_put(&work, want);\n+\tprio_queue_put(&work, from);\n \twhile (work.nr) {\n \t\tstruct commit_list *list;\n \t\tstruct commit *commit = prio_queue_get(&work);\n \n-\t\tif (commit->object.flags & THEY_HAVE) {\n-\t\t\twant->object.flags |= COMMON_KNOWN;\n+\t\tif (commit->object.flags & with_flag) {\n+\t\t\tfrom->object.flags |= assign_flag;\n \t\t\tbreak;\n \t\t}\n \t\tif (!commit->object.parsed)\n@@ -362,10 +363,10 @@ static int reachable(struct commit *want)\n \t\t\t\tprio_queue_put(&work, parent);\n \t\t}\n \t}\n-\twant->object.flags |= REACHABLE;\n-\tclear_commit_marks(want, REACHABLE);\n+\tfrom->object.flags |= REACHABLE;\n+\tclear_commit_marks(from, REACHABLE);\n \tclear_prio_queue(&work);\n-\treturn (want->object.flags & COMMON_KNOWN);\n+\treturn (from->object.flags & assign_flag);\n }\n \n static int ok_to_give_up(void)\n@@ -390,7 +391,7 @@ static int ok_to_give_up(void)\n \t\t\twant_obj.objects[i].item->flags |= COMMON_KNOWN;\n \t\t\tcontinue;\n \t\t}\n-\t\tif (!reachable((struct commit *)want))\n+\t\tif (!reachable((struct commit *)want, THEY_HAVE, COMMON_KNOWN))\n \t\t\treturn 0;\n \t}\n \treturn 1;\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353134","messageId":"20180720163227.105950-8-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 07/18] upload-pack: generalize commit date cutoff","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:12Z","receivedAt":"2018-07-20T16:33:27Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"The ok_to_give_up() method uses the commit date as a cutoff to avoid\nwalking the entire reachble set of commits. Before moving the\nreachable() method to commit-reach.c, pull out the dependence on the\nglobal constant 'oldest_have' with a 'min_commit_date' parameter.\n\nSigned-off-by: Derrick Stolee \n---\n upload-pack.c | 16 ++++++++++------\n 1 file changed, 10 insertions(+), 6 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 9fe19003c6..427de461d8 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -337,7 +337,7 @@ static int got_oid(const char *hex, struct object_id *oid)\n }\n \n static int reachable(struct commit *from, unsigned int with_flag,\n-\t\t unsigned int assign_flag)\n+\t\t unsigned int assign_flag, time_t min_commit_date)\n {\n \tstruct prio_queue work = { compare_commits_by_commit_date };\n \n@@ -355,7 +355,7 @@ static int reachable(struct commit *from, unsigned int with_flag,\n \t\tif (commit->object.flags & REACHABLE)\n \t\t\tcontinue;\n \t\tcommit->object.flags |= REACHABLE;\n-\t\tif (commit->date < oldest_have)\n+\t\tif (commit->date < min_commit_date)\n \t\t\tcontinue;\n \t\tfor (list = commit->parents; list; list = list->next) {\n \t\t\tstruct commit *parent = list->item;\n@@ -372,11 +372,13 @@ static int reachable(struct commit *from, unsigned int with_flag,\n /*\n * Determine if every commit in 'from' can reach at least one commit\n * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n- * as a marker for commits that are already visited.\n+ * as a marker for commits that are already visited. Do not walk\n+ * commits with date below 'min_commit_date'.\n */\n static int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\t\tunsigned int with_flag,\n-\t\t\t\t\tunsigned int assign_flag)\n+\t\t\t\t\tunsigned int assign_flag,\n+\t\t\t\t\ttime_t min_commit_date)\n {\n \tint i;\n \n@@ -395,7 +397,8 @@ static int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\tfrom->objects[i].item->flags |= assign_flag;\n \t\t\tcontinue;\n \t\t}\n-\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag))\n+\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag,\n+\t\t\t min_commit_date))\n \t\t\treturn 0;\n \t}\n \treturn 1;\n@@ -406,7 +409,8 @@ static int ok_to_give_up(void)\n \tif (!have_obj.nr)\n \t\treturn 0;\n \n-\treturn can_all_from_reach_with_flag(&want_obj, THEY_HAVE, COMMON_KNOWN);\n+\treturn can_all_from_reach_with_flag(&want_obj, THEY_HAVE,\n+\t\t\t\t\t COMMON_KNOWN, oldest_have);\n }\n \n static int get_common_commits(void)\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353135","messageId":"20180720163227.105950-7-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 06/18] upload-pack: refactor ok_to_give_up()","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:11Z","receivedAt":"2018-07-20T16:33:31Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"In anticipation of consolidating all commit reachability algorithms,\nrefactor ok_to_give_up() in order to allow splitting its logic into\nan external method.\n\nSigned-off-by: Derrick Stolee \n---\n upload-pack.c | 34 +++++++++++++++++++++++-----------\n 1 file changed, 23 insertions(+), 11 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 5a639cb47b..9fe19003c6 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -369,34 +369,46 @@ static int reachable(struct commit *from, unsigned int with_flag,\n \treturn (from->object.flags & assign_flag);\n }\n \n-static int ok_to_give_up(void)\n+/*\n+ * Determine if every commit in 'from' can reach at least one commit\n+ * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n+ * as a marker for commits that are already visited.\n+ */\n+static int can_all_from_reach_with_flag(struct object_array *from,\n+\t\t\t\t\tunsigned int with_flag,\n+\t\t\t\t\tunsigned int assign_flag)\n {\n \tint i;\n \n-\tif (!have_obj.nr)\n-\t\treturn 0;\n-\n-\tfor (i = 0; i < want_obj.nr; i++) {\n-\t\tstruct object *want = want_obj.objects[i].item;\n+\tfor (i = 0; i < from->nr; i++) {\n+\t\tstruct object *from_one = from->objects[i].item;\n \n-\t\tif (want->flags & COMMON_KNOWN)\n+\t\tif (from_one->flags & assign_flag)\n \t\t\tcontinue;\n-\t\twant = deref_tag(the_repository, want, \"a want line\", 0);\n-\t\tif (!want || want->type != OBJ_COMMIT) {\n+\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n+\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n \t\t\t/* no way to tell if this is reachable by\n \t\t\t * looking at the ancestry chain alone, so\n \t\t\t * leave a note to ourselves not to worry about\n \t\t\t * this object anymore.\n \t\t\t */\n-\t\t\twant_obj.objects[i].item->flags |= COMMON_KNOWN;\n+\t\t\tfrom->objects[i].item->flags |= assign_flag;\n \t\t\tcontinue;\n \t\t}\n-\t\tif (!reachable((struct commit *)want, THEY_HAVE, COMMON_KNOWN))\n+\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag))\n \t\t\treturn 0;\n \t}\n \treturn 1;\n }\n \n+static int ok_to_give_up(void)\n+{\n+\tif (!have_obj.nr)\n+\t\treturn 0;\n+\n+\treturn can_all_from_reach_with_flag(&want_obj, THEY_HAVE, COMMON_KNOWN);\n+}\n+\n static int get_common_commits(void)\n {\n \tstruct object_id oid;\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353136","messageId":"20180720163227.105950-10-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 09/18] test-reach: create new test tool for ref_newer","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:15Z","receivedAt":"2018-07-20T16:33:33Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"As we prepare to change the behavior of the algorithms in\ncommit-reach.c, create a new test-tool subcommand 'reach' to test these\nmethods on interesting commit-graph shapes.\n\nTo use the new test-tool, use 'test-tool reach ' and provide\ninput to stdin that describes the inputs to the method. Currently, we\nonly implement the ref_newer method, which requires two commits. Use\nlines \"A:\" and \"B:\" for the two inputs. We will\nexpand this input later to accommodate methods that take lists of\ncommits.\n\nThe test t6600-test-reach.sh creates a repo whose commits form a\ntwo-dimensional grid. This grid makes it easy for us to determine\nreachability because commit-A-B can reach commit-X-Y if and only if A is\nat least X and B is at least Y. This helps create interesting test cases\nfor each result of the methods in commit-reach.c.\n\nWe test all methods in three different states of the commit-graph file:\nNon-existent (no generation numbers), fully computed, and mixed (some\ncommits have generation numbers and others do not).\n\nSigned-off-by: Derrick Stolee \n---\n Makefile | 1 +\n t/helper/test-reach.c | 63 +++++++++++++++++++++++++++++++\n t/helper/test-tool.c | 1 +\n t/helper/test-tool.h | 1 +\n t/t6600-test-reach.sh | 86 +++++++++++++++++++++++++++++++++++++++++++\n 5 files changed, 152 insertions(+)\n create mode 100644 t/helper/test-reach.c\n create mode 100755 t/t6600-test-reach.sh\n\ndiff --git a/Makefile b/Makefile\nindex 59781f4bc3..d69f9d415d 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -716,6 +716,7 @@ TEST_BUILTINS_OBJS += test-mktemp.o\n TEST_BUILTINS_OBJS += test-online-cpus.o\n TEST_BUILTINS_OBJS += test-path-utils.o\n TEST_BUILTINS_OBJS += test-prio-queue.o\n+TEST_BUILTINS_OBJS += test-reach.o\n TEST_BUILTINS_OBJS += test-read-cache.o\n TEST_BUILTINS_OBJS += test-ref-store.o\n TEST_BUILTINS_OBJS += test-regex.o\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nnew file mode 100644\nindex 0000000000..620bb46041\n--- /dev/null\n+++ b/t/helper/test-reach.c\n@@ -0,0 +1,63 @@\n+#include \"test-tool.h\"\n+#include \"cache.h\"\n+#include \"commit.h\"\n+#include \"commit-reach.h\"\n+#include \"config.h\"\n+#include \"parse-options.h\"\n+#include \"tag.h\"\n+\n+int cmd__reach(int ac, const char **av)\n+{\n+\tstruct object_id oid_A, oid_B;\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct repository *r = the_repository;\n+\n+\tsetup_git_directory();\n+\n+\tif (ac < 2)\n+\t\texit(1);\n+\n+\n+\twhile (strbuf_getline(&buf, stdin) != EOF) {\n+\t\tstruct object_id oid;\n+\t\tstruct object *o;\n+\t\tstruct commit *c;\n+\t\tif (buf.len < 3)\n+\t\t\tcontinue;\n+\n+\t\tif (get_oid_committish(buf.buf + 2, &oid))\n+\t\t\tdie(\"failed to resolve %s\", buf.buf + 2);\n+\n+\t\to = parse_object(r, &oid);\n+\t\to = deref_tag_noverify(o);\n+\n+\t\tif (!o)\n+\t\t\tdie(\"failed to load commit for input %s resulting in oid %s\\n\",\n+\t\t\t buf.buf, oid_to_hex(&oid));\n+\n+\t\tc = object_as_type(r, o, OBJ_COMMIT, 0);\n+\n+\t\tif (!c)\n+\t\t\tdie(\"failed to load commit for input %s resulting in oid %s\\n\",\n+\t\t\t buf.buf, oid_to_hex(&oid));\n+\n+\t\tswitch (buf.buf[0]) {\n+\t\t\tcase 'A':\n+\t\t\t\toidcpy(&oid_A, &oid);\n+\t\t\t\tbreak;\n+\n+\t\t\tcase 'B':\n+\t\t\t\toidcpy(&oid_B, &oid);\n+\t\t\t\tbreak;\n+\n+\t\t\tdefault:\n+\t\t\t\tdie(\"unexpected start of line: %c\", buf.buf[0]);\n+\t\t}\n+\t}\n+\tstrbuf_release(&buf);\n+\n+\tif (!strcmp(av[1], \"ref_newer\"))\n+\t\tprintf(\"%s(A,B):%d\\n\", av[1], ref_newer(&oid_A, &oid_B));\n+\n+\texit(0);\n+}\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex dafc91c240..582d02adfd 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -26,6 +26,7 @@ static struct test_cmd cmds[] = {\n \t{ \"online-cpus\", cmd__online_cpus },\n \t{ \"path-utils\", cmd__path_utils },\n \t{ \"prio-queue\", cmd__prio_queue },\n+\t{ \"reach\", cmd__reach },\n \t{ \"read-cache\", cmd__read_cache },\n \t{ \"ref-store\", cmd__ref_store },\n \t{ \"regex\", cmd__regex },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex 80cbcf0857..a7e53c420e 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -20,6 +20,7 @@ int cmd__mktemp(int argc, const char **argv);\n int cmd__online_cpus(int argc, const char **argv);\n int cmd__path_utils(int argc, const char **argv);\n int cmd__prio_queue(int argc, const char **argv);\n+int cmd__reach(int argc, const char **argv);\n int cmd__read_cache(int argc, const char **argv);\n int cmd__ref_store(int argc, const char **argv);\n int cmd__regex(int argc, const char **argv);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nnew file mode 100755\nindex 0000000000..966309c6cf\n--- /dev/null\n+++ b/t/t6600-test-reach.sh\n@@ -0,0 +1,86 @@\n+#!/bin/sh\n+\n+test_description='basic commit reachability tests'\n+\n+. ./test-lib.sh\n+\n+# Construct a grid-like commit graph with points (x,y)\n+# with 1 <= x <= 10, 1 <= y <= 10, where (x,y) has\n+# parents (x-1, y) and (x, y-1), keeping in mind that\n+# we drop a parent if a coordinate is nonpositive.\n+#\n+# (10,10)\n+# / \\\n+# (10,9) (9,10)\n+# / \\ / \\\n+# (10,8) (9,9) (8,10)\n+# / \\ / \\ / \\\n+# ( continued...)\n+# \\ / \\ / \\ /\n+# (3,1) (2,2) (1,3)\n+# \\ / \\ /\n+# (2,1) (2,1)\n+# \\ /\n+# (1,1)\n+#\n+# We use branch 'commit-x-y' to refer to (x,y).\n+# This grid allows interesting reachability and\n+# non-reachability queries: (x,y) can reach (x',y')\n+# if and only if x' <= x and y' <= y.\n+test_expect_success 'setup' '\n+\tfor i in $(test_seq 1 10)\n+\tdo\n+\t\ttest_commit \"1-$i\" &&\n+\t\tgit branch -f commit-1-$i\n+\tdone &&\n+\tfor j in $(test_seq 1 9)\n+\tdo\n+\t\tgit reset --hard commit-$j-1 &&\n+\t\tx=$(($j + 1)) &&\n+\t\ttest_commit \"$x-1\" &&\n+\t\tgit branch -f commit-$x-1 &&\n+\n+\t\tfor i in $(test_seq 2 10)\n+\t\tdo\n+\t\t\tgit merge commit-$j-$i -m \"$x-$i\" &&\n+\t\t\tgit branch -f commit-$x-$i\n+\t\tdone\n+\tdone &&\n+\tgit commit-graph write --reachable &&\n+\tmv .git/objects/info/commit-graph commit-graph-full &&\n+\tgit show-ref -s commit-5-5 | git commit-graph write --stdin-commits &&\n+\tmv .git/objects/info/commit-graph commit-graph-half &&\n+\tgit config core.commitGraph true\n+'\n+\n+test_three_modes () {\n+\ttest_when_finished rm -rf .git/objects/info/commit-graph &&\n+\ttest-tool reach $1 actual &&\n+\ttest_cmp expect actual &&\n+\tcp commit-graph-full .git/objects/info/commit-graph &&\n+\ttest-tool reach $1 actual &&\n+\ttest_cmp expect actual &&\n+\tcp commit-graph-half .git/objects/info/commit-graph &&\n+\ttest-tool reach $1 actual &&\n+\ttest_cmp expect actual\n+}\n+\n+test_expect_success 'ref_newer:miss' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tB:commit-4-9\n+\tEOF\n+\techo \"ref_newer(A,B):0\" >expect &&\n+\ttest_three_modes ref_newer\n+'\n+\n+test_expect_success 'ref_newer:hit' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tB:commit-2-3\n+\tEOF\n+\techo \"ref_newer(A,B):1\" >expect &&\n+\ttest_three_modes ref_newer\n+'\n+\n+test_done\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353137","messageId":"20180720163227.105950-9-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 08/18] commit-reach: move can_all_from_reach_with_flags","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:13Z","receivedAt":"2018-07-20T16:33:34Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"There are several commit walks in the codebase. Group them together into\na new commit-reach.c file and corresponding header. After we group these\nwalks into one place, we can reduce duplicate logic by calling\nequivalent methods.\n\nThe can_all_from_reach_with_flags method is used in a stateful way by\nupload-pack.c. The parameters are very flexible, so we will be able to\nuse its commit walking logic for many other callers.\n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 63 +++++++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 14 ++++++++++\n object.h | 4 +--\n upload-pack.c | 70 +-------------------------------------------------\n 4 files changed, 80 insertions(+), 71 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 01d796f011..d806291d5d 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -10,6 +10,7 @@\n #include \"commit-reach.h\"\n \n /* Remember to update object flag allocation in object.h */\n+#define REACHABLE (1u<<15)\n #define PARENT1\t\t(1u<<16)\n #define PARENT2\t\t(1u<<17)\n #define STALE\t\t(1u<<18)\n@@ -532,3 +533,65 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n \t\treturn contains_tag_algo(commit, list, cache) == CONTAINS_YES;\n \treturn is_descendant_of(commit, list);\n }\n+\n+int reachable(struct commit *from, unsigned int with_flag,\n+\t unsigned int assign_flag, time_t min_commit_date)\n+{\n+\tstruct prio_queue work = { compare_commits_by_commit_date };\n+\n+\tprio_queue_put(&work, from);\n+\twhile (work.nr) {\n+\t\tstruct commit_list *list;\n+\t\tstruct commit *commit = prio_queue_get(&work);\n+\n+\t\tif (commit->object.flags & with_flag) {\n+\t\t\tfrom->object.flags |= assign_flag;\n+\t\t\tbreak;\n+\t\t}\n+\t\tif (!commit->object.parsed)\n+\t\t\tparse_object(the_repository, &commit->object.oid);\n+\t\tif (commit->object.flags & REACHABLE)\n+\t\t\tcontinue;\n+\t\tcommit->object.flags |= REACHABLE;\n+\t\tif (commit->date < min_commit_date)\n+\t\t\tcontinue;\n+\t\tfor (list = commit->parents; list; list = list->next) {\n+\t\t\tstruct commit *parent = list->item;\n+\t\t\tif (!(parent->object.flags & REACHABLE))\n+\t\t\t\tprio_queue_put(&work, parent);\n+\t\t}\n+\t}\n+\tfrom->object.flags |= REACHABLE;\n+\tclear_commit_marks(from, REACHABLE);\n+\tclear_prio_queue(&work);\n+\treturn (from->object.flags & assign_flag);\n+}\n+\n+int can_all_from_reach_with_flag(struct object_array *from,\n+\t\t\t\t unsigned int with_flag,\n+\t\t\t\t unsigned int assign_flag,\n+\t\t\t\t time_t min_commit_date)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i < from->nr; i++) {\n+\t\tstruct object *from_one = from->objects[i].item;\n+\n+\t\tif (from_one->flags & assign_flag)\n+\t\t\tcontinue;\n+\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n+\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n+\t\t\t/* no way to tell if this is reachable by\n+\t\t\t * looking at the ancestry chain alone, so\n+\t\t\t * leave a note to ourselves not to worry about\n+\t\t\t * this object anymore.\n+\t\t\t */\n+\t\t\tfrom->objects[i].item->flags |= assign_flag;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag,\n+\t\t\t min_commit_date))\n+\t\t\treturn 0;\n+\t}\n+\treturn 1;\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nindex 13dec25cee..b28bc22fcd 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -59,4 +59,18 @@ define_commit_slab(contains_cache, enum contains_result);\n int commit_contains(struct ref_filter *filter, struct commit *commit,\n \t\t struct commit_list *list, struct contains_cache *cache);\n \n+int reachable(struct commit *from, unsigned int with_flag,\n+\t unsigned int assign_flag, time_t min_commit_date);\n+\n+/*\n+ * Determine if every commit in 'from' can reach at least one commit\n+ * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n+ * as a marker for commits that are already visited. Do not walk\n+ * commits with date below 'min_commit_date'.\n+ */\n+int can_all_from_reach_with_flag(struct object_array *from,\n+\t\t\t\t unsigned int with_flag,\n+\t\t\t\t unsigned int assign_flag,\n+\t\t\t\t time_t min_commit_date);\n+\n #endif\ndiff --git a/object.h b/object.h\nindex 18c2b073e3..b132944c51 100644\n--- a/object.h\n+++ b/object.h\n@@ -60,12 +60,12 @@ struct object_array {\n * revision.h: 0---------10 26\n * fetch-pack.c: 0----5\n * walker.c: 0-2\n- * upload-pack.c: 4 11----------------19\n+ * upload-pack.c: 4 11-----14 16-----19\n * builtin/blame.c: 12-13\n * bisect.c: 16\n * bundle.c: 16\n * http-push.c: 16-----19\n- * commit-reach.c: 16-----19\n+ * commit-reach.c: 15-------19\n * sha1-name.c: 20\n * list-objects-filter.c: 21\n * builtin/fsck.c: 0--3\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 427de461d8..11c426685d 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -24,13 +24,13 @@\n #include \"quote.h\"\n #include \"upload-pack.h\"\n #include \"serve.h\"\n+#include \"commit-reach.h\"\n \n /* Remember to update object flag allocation in object.h */\n #define THEY_HAVE\t(1u << 11)\n #define OUR_REF\t\t(1u << 12)\n #define WANTED\t\t(1u << 13)\n #define COMMON_KNOWN\t(1u << 14)\n-#define REACHABLE\t(1u << 15)\n \n #define SHALLOW\t\t(1u << 16)\n #define NOT_SHALLOW\t(1u << 17)\n@@ -336,74 +336,6 @@ static int got_oid(const char *hex, struct object_id *oid)\n \treturn 0;\n }\n \n-static int reachable(struct commit *from, unsigned int with_flag,\n-\t\t unsigned int assign_flag, time_t min_commit_date)\n-{\n-\tstruct prio_queue work = { compare_commits_by_commit_date };\n-\n-\tprio_queue_put(&work, from);\n-\twhile (work.nr) {\n-\t\tstruct commit_list *list;\n-\t\tstruct commit *commit = prio_queue_get(&work);\n-\n-\t\tif (commit->object.flags & with_flag) {\n-\t\t\tfrom->object.flags |= assign_flag;\n-\t\t\tbreak;\n-\t\t}\n-\t\tif (!commit->object.parsed)\n-\t\t\tparse_object(the_repository, &commit->object.oid);\n-\t\tif (commit->object.flags & REACHABLE)\n-\t\t\tcontinue;\n-\t\tcommit->object.flags |= REACHABLE;\n-\t\tif (commit->date < min_commit_date)\n-\t\t\tcontinue;\n-\t\tfor (list = commit->parents; list; list = list->next) {\n-\t\t\tstruct commit *parent = list->item;\n-\t\t\tif (!(parent->object.flags & REACHABLE))\n-\t\t\t\tprio_queue_put(&work, parent);\n-\t\t}\n-\t}\n-\tfrom->object.flags |= REACHABLE;\n-\tclear_commit_marks(from, REACHABLE);\n-\tclear_prio_queue(&work);\n-\treturn (from->object.flags & assign_flag);\n-}\n-\n-/*\n- * Determine if every commit in 'from' can reach at least one commit\n- * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n- * as a marker for commits that are already visited. Do not walk\n- * commits with date below 'min_commit_date'.\n- */\n-static int can_all_from_reach_with_flag(struct object_array *from,\n-\t\t\t\t\tunsigned int with_flag,\n-\t\t\t\t\tunsigned int assign_flag,\n-\t\t\t\t\ttime_t min_commit_date)\n-{\n-\tint i;\n-\n-\tfor (i = 0; i < from->nr; i++) {\n-\t\tstruct object *from_one = from->objects[i].item;\n-\n-\t\tif (from_one->flags & assign_flag)\n-\t\t\tcontinue;\n-\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n-\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n-\t\t\t/* no way to tell if this is reachable by\n-\t\t\t * looking at the ancestry chain alone, so\n-\t\t\t * leave a note to ourselves not to worry about\n-\t\t\t * this object anymore.\n-\t\t\t */\n-\t\t\tfrom->objects[i].item->flags |= assign_flag;\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag,\n-\t\t\t min_commit_date))\n-\t\t\treturn 0;\n-\t}\n-\treturn 1;\n-}\n-\n static int ok_to_give_up(void)\n {\n \tif (!have_obj.nr)\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353139","messageId":"20180720163227.105950-12-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 11/18] test-reach: test is_descendant_of","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:18Z","receivedAt":"2018-07-20T16:33:35Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"The is_descendant_of method takes a single commit as its first parameter\nand a list of commits as its second parameter. Extend the input of the\n'test-tool reach' command to take multiple lines of the form\n\"X:\" to construct a list of commits. Pass these to\nis_descendant_of and create tests that check each result.\n\nSigned-off-by: Derrick Stolee \n---\n t/helper/test-reach.c | 8 ++++++++\n t/t6600-test-reach.sh | 22 ++++++++++++++++++++++\n 2 files changed, 30 insertions(+)\n\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex f93ad5084d..dccbd48178 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -10,6 +10,7 @@ int cmd__reach(int ac, const char **av)\n {\n \tstruct object_id oid_A, oid_B;\n \tstruct commit *A, *B;\n+\tstruct commit_list *X;\n \tstruct strbuf buf = STRBUF_INIT;\n \tstruct repository *r = the_repository;\n \n@@ -19,6 +20,7 @@ int cmd__reach(int ac, const char **av)\n \t\texit(1);\n \n \tA = B = NULL;\n+\tX = NULL;\n \n \twhile (strbuf_getline(&buf, stdin) != EOF) {\n \t\tstruct object_id oid;\n@@ -54,6 +56,10 @@ int cmd__reach(int ac, const char **av)\n \t\t\t\tB = c;\n \t\t\t\tbreak;\n \n+\t\t\tcase 'X':\n+\t\t\t\tcommit_list_insert(c, &X);\n+\t\t\t\tbreak;\n+\n \t\t\tdefault:\n \t\t\t\tdie(\"unexpected start of line: %c\", buf.buf[0]);\n \t\t}\n@@ -64,6 +70,8 @@ int cmd__reach(int ac, const char **av)\n \t\tprintf(\"%s(A,B):%d\\n\", av[1], ref_newer(&oid_A, &oid_B));\n \telse if (!strcmp(av[1], \"in_merge_bases\"))\n \t\tprintf(\"%s(A,B):%d\\n\", av[1], in_merge_bases(A, B));\n+\telse if (!strcmp(av[1], \"is_descendant_of\"))\n+\t\tprintf(\"%s(A,X):%d\\n\", av[1], is_descendant_of(A, X));\n \n \texit(0);\n }\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 5cd6b14c69..98bcb17960 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -101,4 +101,26 @@ test_expect_success 'in_merge_bases:miss' '\n \ttest_three_modes in_merge_bases\n '\n \n+test_expect_success 'is_descendant_of:hit' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tX:commit-4-8\n+\tX:commit-6-6\n+\tX:commit-1-1\n+\tEOF\n+\techo \"is_descendant_of(A,X):1\" >expect &&\n+\ttest_three_modes is_descendant_of\n+'\n+\n+test_expect_success 'is_descendant_of:miss' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-6-8\n+\tX:commit-5-9\n+\tX:commit-4-10\n+\tX:commit-7-6\n+\tEOF\n+\techo \"is_descendant_of(A,X):0\" >expect &&\n+\ttest_three_modes is_descendant_of\n+'\n+\n test_done\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353138","messageId":"20180720163227.105950-13-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 12/18] test-reach: test get_merge_bases_many","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:20Z","receivedAt":"2018-07-20T16:33:36Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"The get_merge_bases_many method returns a list of merge bases for a\nsingle commit (A) against a list of commits (X). Some care is needed in\nconstructing the expected behavior because the result is not the\nexpected merge-base for an octopus merge with those parents but instead\nthe set of maximal commits that are reachable from A and at least one of\nthe commits in X.\n\nAdd get_merge_bases_many to 'test-tool reach' and create a test that\ndemonstrates that this output returns multiple results. Specifically, we\nselect a list of three commits such that we output two commits that are\nreachable from one of the first two, respectively, and none are\nreachable from the third.\n\nSigned-off-by: Derrick Stolee \n---\n t/helper/test-reach.c | 31 +++++++++++++++++++++++++++++++\n t/t6600-test-reach.sh | 15 +++++++++++++++\n 2 files changed, 46 insertions(+)\n\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex dccbd48178..4df01187c9 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -4,13 +4,34 @@\n #include \"commit-reach.h\"\n #include \"config.h\"\n #include \"parse-options.h\"\n+#include \"string-list.h\"\n #include \"tag.h\"\n \n+static void print_sorted_commit_ids(struct commit_list *list)\n+{\n+\tint i;\n+\tstruct string_list s = STRING_LIST_INIT_DUP;\n+\n+\twhile (list) {\n+\t\tstring_list_append(&s, oid_to_hex(&list->item->object.oid));\n+\t\tlist = list->next;\n+\t}\n+\n+\tstring_list_sort(&s);\n+\n+\tfor (i = 0; i < s.nr; i++)\n+\t\tprintf(\"%s\\n\", s.items[i].string);\n+\n+\tstring_list_clear(&s, 0);\n+}\n+\n int cmd__reach(int ac, const char **av)\n {\n \tstruct object_id oid_A, oid_B;\n \tstruct commit *A, *B;\n \tstruct commit_list *X;\n+\tstruct commit **X_array;\n+\tint X_nr, X_alloc;\n \tstruct strbuf buf = STRBUF_INIT;\n \tstruct repository *r = the_repository;\n \n@@ -21,6 +42,9 @@ int cmd__reach(int ac, const char **av)\n \n \tA = B = NULL;\n \tX = NULL;\n+\tX_nr = 0;\n+\tX_alloc = 16;\n+\tALLOC_ARRAY(X_array, X_alloc);\n \n \twhile (strbuf_getline(&buf, stdin) != EOF) {\n \t\tstruct object_id oid;\n@@ -58,6 +82,8 @@ int cmd__reach(int ac, const char **av)\n \n \t\t\tcase 'X':\n \t\t\t\tcommit_list_insert(c, &X);\n+\t\t\t\tALLOC_GROW(X_array, X_nr + 1, X_alloc);\n+\t\t\t\tX_array[X_nr++] = c;\n \t\t\t\tbreak;\n \n \t\t\tdefault:\n@@ -72,6 +98,11 @@ int cmd__reach(int ac, const char **av)\n \t\tprintf(\"%s(A,B):%d\\n\", av[1], in_merge_bases(A, B));\n \telse if (!strcmp(av[1], \"is_descendant_of\"))\n \t\tprintf(\"%s(A,X):%d\\n\", av[1], is_descendant_of(A, X));\n+\telse if (!strcmp(av[1], \"get_merge_bases_many\")) {\n+\t\tstruct commit_list *list = get_merge_bases_many(A, X_nr, X_array);\n+\t\tprintf(\"%s(A,X):\\n\", av[1]);\n+\t\tprint_sorted_commit_ids(list);\n+\t}\n \n \texit(0);\n }\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 98bcb17960..d43e1a61d5 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -123,4 +123,19 @@ test_expect_success 'is_descendant_of:miss' '\n \ttest_three_modes is_descendant_of\n '\n \n+test_expect_success 'get_merge_bases_many' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tX:commit-4-8\n+\tX:commit-6-6\n+\tX:commit-8-3\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse commit-5-6 \\\n+\t\t\t commit-4-7 | sort\n+\t} >expect &&\n+\ttest_three_modes get_merge_bases_many\n+'\n+\n test_done\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353140","messageId":"20180720163227.105950-11-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 10/18] test-reach: test in_merge_bases","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:17Z","receivedAt":"2018-07-20T16:33:38Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"Signed-off-by: Derrick Stolee \n---\n t/helper/test-reach.c | 6 ++++++\n t/t6600-test-reach.sh | 18 ++++++++++++++++++\n 2 files changed, 24 insertions(+)\n\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex 620bb46041..f93ad5084d 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -9,6 +9,7 @@\n int cmd__reach(int ac, const char **av)\n {\n \tstruct object_id oid_A, oid_B;\n+\tstruct commit *A, *B;\n \tstruct strbuf buf = STRBUF_INIT;\n \tstruct repository *r = the_repository;\n \n@@ -17,6 +18,7 @@ int cmd__reach(int ac, const char **av)\n \tif (ac < 2)\n \t\texit(1);\n \n+\tA = B = NULL;\n \n \twhile (strbuf_getline(&buf, stdin) != EOF) {\n \t\tstruct object_id oid;\n@@ -44,10 +46,12 @@ int cmd__reach(int ac, const char **av)\n \t\tswitch (buf.buf[0]) {\n \t\t\tcase 'A':\n \t\t\t\toidcpy(&oid_A, &oid);\n+\t\t\t\tA = c;\n \t\t\t\tbreak;\n \n \t\t\tcase 'B':\n \t\t\t\toidcpy(&oid_B, &oid);\n+\t\t\t\tB = c;\n \t\t\t\tbreak;\n \n \t\t\tdefault:\n@@ -58,6 +62,8 @@ int cmd__reach(int ac, const char **av)\n \n \tif (!strcmp(av[1], \"ref_newer\"))\n \t\tprintf(\"%s(A,B):%d\\n\", av[1], ref_newer(&oid_A, &oid_B));\n+\telse if (!strcmp(av[1], \"in_merge_bases\"))\n+\t\tprintf(\"%s(A,B):%d\\n\", av[1], in_merge_bases(A, B));\n \n \texit(0);\n }\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 966309c6cf..5cd6b14c69 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -83,4 +83,22 @@ test_expect_success 'ref_newer:hit' '\n \ttest_three_modes ref_newer\n '\n \n+test_expect_success 'in_merge_bases:hit' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tB:commit-8-8\n+\tEOF\n+\techo \"in_merge_bases(A,B):1\" >expect &&\n+\ttest_three_modes in_merge_bases\n+'\n+\n+test_expect_success 'in_merge_bases:miss' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-6-8\n+\tB:commit-5-9\n+\tEOF\n+\techo \"in_merge_bases(A,B):0\" >expect &&\n+\ttest_three_modes in_merge_bases\n+'\n+\n test_done\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353142","messageId":"20180720163227.105950-15-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 14/18] test-reach: test can_all_from_reach_with_flags","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:23Z","receivedAt":"2018-07-20T16:33:38Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"The can_all_from_reach_with_flags method is used by ok_to_give_up in\nupload-pack.c to see if we have done enough negotiation during a fetch.\nThis method is intentionally created to preserve state between calls to\nassist with stateful negotiation, such as over SSH.\n\nTo make this method testable, add a new can_all_from_reach method that\ndoes the initial setup and final tear-down. We will later use this\nmethod in production code. Call the method from 'test-tool reach' for\nnow.\n\nSince this is a many-to-many reachability query, add a new type of input\nto the 'test-tool reach' input format. Lines \"Y:\" create a\nlist of commits to be the reachability targets from the commits in the\n'X' list. In the context of fetch negotiation, the 'X' commits are the\n'want' commits and the 'Y' commits are the 'have' commits.\n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 47 +++++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 2 ++\n t/helper/test-reach.c | 10 +++++++--\n t/t6600-test-reach.sh | 45 +++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 102 insertions(+), 2 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex d806291d5d..940fbf2e17 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -595,3 +595,50 @@ int can_all_from_reach_with_flag(struct object_array *from,\n \t}\n \treturn 1;\n }\n+\n+int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n+\t\t int cutoff_by_min_date)\n+{\n+\tstruct object_array from_objs = OBJECT_ARRAY_INIT;\n+\ttime_t min_commit_date = cutoff_by_min_date ? from->item->date : 0;\n+\tstruct commit_list *from_iter = from, *to_iter = to;\n+\tint result;\n+\n+\twhile (from_iter) {\n+\t\tadd_object_array(&from_iter->item->object, NULL, &from_objs);\n+\n+\t\tif (!parse_commit(from_iter->item)) {\n+\t\t\tif (from_iter->item->date < min_commit_date)\n+\t\t\t\tmin_commit_date = from_iter->item->date;\n+\t\t}\n+\n+\t\tfrom_iter = from_iter->next;\n+\t}\n+\n+\twhile (to_iter) {\n+\t\tif (!parse_commit(to_iter->item)) {\n+\t\t\tif (to_iter->item->date < min_commit_date)\n+\t\t\t\tmin_commit_date = to_iter->item->date;\n+\t\t}\n+\n+\t\tto_iter->item->object.flags |= PARENT2;\n+\n+\t\tto_iter = to_iter->next;\n+\t}\n+\n+\tresult = can_all_from_reach_with_flag(&from_objs, PARENT2, PARENT1,\n+\t\t\t\t\t min_commit_date);\n+\n+\twhile (from) {\n+\t\tclear_commit_marks(from->item, PARENT1);\n+\t\tfrom = from->next;\n+\t}\n+\n+\twhile (to) {\n+\t\tclear_commit_marks(to->item, PARENT2);\n+\t\tto = to->next;\n+\t}\n+\n+\tobject_array_clear(&from_objs);\n+\treturn result;\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nindex b28bc22fcd..aa202c9703 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -72,5 +72,7 @@ int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\t unsigned int with_flag,\n \t\t\t\t unsigned int assign_flag,\n \t\t\t\t time_t min_commit_date);\n+int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n+\t\t int commit_date_cutoff);\n \n #endif\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex e32e193b70..c79729cac0 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -29,7 +29,7 @@ int cmd__reach(int ac, const char **av)\n {\n \tstruct object_id oid_A, oid_B;\n \tstruct commit *A, *B;\n-\tstruct commit_list *X;\n+\tstruct commit_list *X, *Y;\n \tstruct commit **X_array;\n \tint X_nr, X_alloc;\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -41,7 +41,7 @@ int cmd__reach(int ac, const char **av)\n \t\texit(1);\n \n \tA = B = NULL;\n-\tX = NULL;\n+\tX = Y = NULL;\n \tX_nr = 0;\n \tX_alloc = 16;\n \tALLOC_ARRAY(X_array, X_alloc);\n@@ -86,6 +86,10 @@ int cmd__reach(int ac, const char **av)\n \t\t\t\tX_array[X_nr++] = c;\n \t\t\t\tbreak;\n \n+\t\t\tcase 'Y':\n+\t\t\t\tcommit_list_insert(c, &Y);\n+\t\t\t\tbreak;\n+\n \t\t\tdefault:\n \t\t\t\tdie(\"unexpected start of line: %c\", buf.buf[0]);\n \t\t}\n@@ -106,6 +110,8 @@ int cmd__reach(int ac, const char **av)\n \t\tstruct commit_list *list = reduce_heads(X);\n \t\tprintf(\"%s(X):\\n\", av[1]);\n \t\tprint_sorted_commit_ids(list);\n+\t} else if (!strcmp(av[1], \"can_all_from_reach\")) {\n+\t\tprintf(\"%s(X,Y):%d\\n\", av[1], can_all_from_reach(X, Y, 1));\n \t}\n \n \texit(0);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 17c6467988..e41eb397a7 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -160,4 +160,49 @@ test_expect_success 'reduce_heads' '\n \ttest_three_modes reduce_heads\n '\n \n+test_expect_success 'can_all_from_reach:hit' '\n+\tcat >input <<-\\EOF &&\n+\tX:commit-2-10\n+\tX:commit-3-9\n+\tX:commit-4-8\n+\tX:commit-5-7\n+\tX:commit-6-6\n+\tX:commit-7-5\n+\tX:commit-8-4\n+\tX:commit-9-3\n+\tY:commit-1-9\n+\tY:commit-2-8\n+\tY:commit-3-7\n+\tY:commit-4-6\n+\tY:commit-5-5\n+\tY:commit-6-4\n+\tY:commit-7-3\n+\tY:commit-8-1\n+\tEOF\n+\techo \"can_all_from_reach(X,Y):1\" >expect &&\n+\ttest_three_modes can_all_from_reach\n+'\n+\n+test_expect_success 'can_all_from_reach:miss' '\n+\tcat >input <<-\\EOF &&\n+\tX:commit-2-10\n+\tX:commit-3-9\n+\tX:commit-4-8\n+\tX:commit-5-7\n+\tX:commit-6-6\n+\tX:commit-7-5\n+\tX:commit-8-4\n+\tX:commit-9-3\n+\tY:commit-1-9\n+\tY:commit-2-8\n+\tY:commit-3-7\n+\tY:commit-4-6\n+\tY:commit-5-5\n+\tY:commit-6-4\n+\tY:commit-8-5\n+\tEOF\n+\techo \"can_all_from_reach(X,Y):0\" >expect &&\n+\ttest_three_modes can_all_from_reach\n+'\n+\n test_done\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353141","messageId":"20180720163227.105950-16-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 15/18] test-reach: test commit_contains","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:25Z","receivedAt":"2018-07-20T16:33:40Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"The commit_contains method has two modes which depend on the given\nref_filter struct. We have the \"normal\" algorithm (which is also the\ntypically-slow operation) and the \"tag\" algorithm. This difference is\nessentially what changes performance for 'git branch --contains' versus\n'git tag --contains'. There are thoughts that the data shapes used by\nthese two applications justify the different implementations.\n\nCreate tests using 'test-tool reach commit_contains [--tag]' to cover\nboth methods.\n\nSigned-off-by: Derrick Stolee \n---\n t/helper/test-reach.c | 12 ++++++++++++\n t/t6600-test-reach.sh | 34 ++++++++++++++++++++++++++++++++++\n 2 files changed, 46 insertions(+)\n\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex c79729cac0..eb21103998 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -4,6 +4,7 @@\n #include \"commit-reach.h\"\n #include \"config.h\"\n #include \"parse-options.h\"\n+#include \"ref-filter.h\"\n #include \"string-list.h\"\n #include \"tag.h\"\n \n@@ -112,6 +113,17 @@ int cmd__reach(int ac, const char **av)\n \t\tprint_sorted_commit_ids(list);\n \t} else if (!strcmp(av[1], \"can_all_from_reach\")) {\n \t\tprintf(\"%s(X,Y):%d\\n\", av[1], can_all_from_reach(X, Y, 1));\n+\t} else if (!strcmp(av[1], \"commit_contains\")) {\n+\t\tstruct ref_filter filter;\n+\t\tstruct contains_cache cache;\n+\t\tinit_contains_cache(&cache);\n+\n+\t\tif (ac > 2 && !strcmp(av[2], \"--tag\"))\n+\t\t\tfilter.with_commit_tag_algo = 1;\n+\t\telse\n+\t\t\tfilter.with_commit_tag_algo = 0;\n+\n+\t\tprintf(\"%s(_,A,X,_):%d\\n\", av[1], commit_contains(&filter, A, X, &cache));\n \t}\n \n \texit(0);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex e41eb397a7..d139a00d1d 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -205,4 +205,38 @@ test_expect_success 'can_all_from_reach:miss' '\n \ttest_three_modes can_all_from_reach\n '\n \n+test_expect_success 'commit_contains:hit' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-7-7\n+\tX:commit-2-10\n+\tX:commit-3-9\n+\tX:commit-4-8\n+\tX:commit-5-7\n+\tX:commit-6-6\n+\tX:commit-7-5\n+\tX:commit-8-4\n+\tX:commit-9-3\n+\tEOF\n+\techo \"commit_contains(_,A,X,_):1\" >expect &&\n+\ttest_three_modes commit_contains &&\n+\ttest_three_modes commit_contains --tag\n+'\n+\n+test_expect_success 'commit_contains:miss' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-6-5\n+\tX:commit-2-10\n+\tX:commit-3-9\n+\tX:commit-4-8\n+\tX:commit-5-7\n+\tX:commit-6-6\n+\tX:commit-7-5\n+\tX:commit-8-4\n+\tX:commit-9-3\n+\tEOF\n+\techo \"commit_contains(_,A,X,_):0\" >expect &&\n+\ttest_three_modes commit_contains &&\n+\ttest_three_modes commit_contains --tag\n+'\n+\n test_done\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353143","messageId":"20180720163227.105950-17-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 16/18] commit-reach: replace ref_newer logic","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:27Z","receivedAt":"2018-07-20T16:33:40Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"The ref_newer method is used by 'git push' to check if a force-push is\nrequired. This method does not use any kind of cutoff when walking, so\nin the case of a force-push will walk all reachable commits.\n\nThe is_descendant_of method already uses paint_down_to_common along with\ncutoffs. By translating the ref_newer arguments into the commit and\ncommit_list required by is_descendant_of, we can have one fewer commit\nwalk and also improve our performance!\n\nFor a copy of the Linux repository, 'test-tool reach ref_newer' presents\nthe following improvements with the specified input. In the case that\nref_newer returns 1, there is no improvement. The improvement is in the\nsecond case where ref_newer returns 0.\n\nInput:\nA:v4.9\nB:v3.19\n\nBefore: 0.09 s\n After: 0.09 s\n\nTo test the negative case, add a new commit with parent v3.19,\nregenerate the commit-graph, and then run with B pointing at that\ncommit.\n\nBefore: 0.43 s\n After: 0.09 s\n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 26 +++-----------------------\n 1 file changed, 3 insertions(+), 23 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 940fbf2e17..f5858944fd 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -366,20 +366,11 @@ void reduce_heads_replace(struct commit_list **heads)\n \t*heads = result;\n }\n \n-static void unmark_and_free(struct commit_list *list, unsigned int mark)\n-{\n-\twhile (list) {\n-\t\tstruct commit *commit = pop_commit(&list);\n-\t\tcommit->object.flags &= ~mark;\n-\t}\n-}\n-\n int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n {\n \tstruct object *o;\n \tstruct commit *old_commit, *new_commit;\n-\tstruct commit_list *list, *used;\n-\tint found = 0;\n+\tstruct commit_list *old_commit_list = NULL;\n \n \t/*\n \t * Both new_commit and old_commit must be commit-ish and new_commit is descendant of\n@@ -400,19 +391,8 @@ int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)\n \tif (parse_commit(new_commit) < 0)\n \t\treturn 0;\n \n-\tused = list = NULL;\n-\tcommit_list_insert(new_commit, &list);\n-\twhile (list) {\n-\t\tnew_commit = pop_most_recent_commit(&list, TMP_MARK);\n-\t\tcommit_list_insert(new_commit, &used);\n-\t\tif (new_commit == old_commit) {\n-\t\t\tfound = 1;\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\tunmark_and_free(list, TMP_MARK);\n-\tunmark_and_free(used, TMP_MARK);\n-\treturn found;\n+\tcommit_list_insert(old_commit, &old_commit_list);\n+\treturn is_descendant_of(new_commit, old_commit_list);\n }\n \n /*\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353145","messageId":"20180720163227.105950-2-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 01/18] commit-reach: move walk methods from commit.c","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:02Z","receivedAt":"2018-07-20T16:33:44Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"There are several commit walks in the codebase. Group them together into\na new commit-reach.c file and corresponding header. After we group these\nwalks into one place, we can reduce duplicate logic by calling\nequivalent methods.\n\nThe method declarations in commit.h are not touched by this commit and\nwill be moved in a following commit. Many consumers need to point to\ncommit-reach.h and that would bloat this commit.\n\nSigned-off-by: Derrick Stolee \n---\n Makefile | 1 +\n commit-reach.c | 360 +++++++++++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 42 ++++++\n commit.c | 358 ------------------------------------------------\n object.h | 2 +-\n 5 files changed, 404 insertions(+), 359 deletions(-)\n create mode 100644 commit-reach.c\n create mode 100644 commit-reach.h\n\ndiff --git a/Makefile b/Makefile\nindex bb8bd67201..59781f4bc3 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -829,6 +829,7 @@ LIB_OBJS += column.o\n LIB_OBJS += combine-diff.o\n LIB_OBJS += commit.o\n LIB_OBJS += commit-graph.o\n+LIB_OBJS += commit-reach.o\n LIB_OBJS += compat/obstack.o\n LIB_OBJS += compat/terminal.o\n LIB_OBJS += config.o\ndiff --git a/commit-reach.c b/commit-reach.c\nnew file mode 100644\nindex 0000000000..8ab6044414\n--- /dev/null\n+++ b/commit-reach.c\n@@ -0,0 +1,360 @@\n+#include \"cache.h\"\n+#include \"prio-queue.h\"\n+#include \"commit.h\"\n+#include \"commit-reach.h\"\n+\n+/* Remember to update object flag allocation in object.h */\n+#define PARENT1\t\t(1u<<16)\n+#define PARENT2\t\t(1u<<17)\n+#define STALE\t\t(1u<<18)\n+#define RESULT\t\t(1u<<19)\n+\n+static const unsigned all_flags = (PARENT1 | PARENT2 | STALE | RESULT);\n+\n+static int queue_has_nonstale(struct prio_queue *queue)\n+{\n+\tint i;\n+\tfor (i = 0; i < queue->nr; i++) {\n+\t\tstruct commit *commit = queue->array[i].data;\n+\t\tif (!(commit->object.flags & STALE))\n+\t\t\treturn 1;\n+\t}\n+\treturn 0;\n+}\n+\n+/* all input commits in one and twos[] must have been parsed! */\n+static struct commit_list *paint_down_to_common(struct commit *one, int n,\n+\t\t\t\t\t\tstruct commit **twos,\n+\t\t\t\t\t\tint min_generation)\n+{\n+\tstruct prio_queue queue = { compare_commits_by_gen_then_commit_date };\n+\tstruct commit_list *result = NULL;\n+\tint i;\n+\tuint32_t last_gen = GENERATION_NUMBER_INFINITY;\n+\n+\tone->object.flags |= PARENT1;\n+\tif (!n) {\n+\t\tcommit_list_append(one, &result);\n+\t\treturn result;\n+\t}\n+\tprio_queue_put(&queue, one);\n+\n+\tfor (i = 0; i < n; i++) {\n+\t\ttwos[i]->object.flags |= PARENT2;\n+\t\tprio_queue_put(&queue, twos[i]);\n+\t}\n+\n+\twhile (queue_has_nonstale(&queue)) {\n+\t\tstruct commit *commit = prio_queue_get(&queue);\n+\t\tstruct commit_list *parents;\n+\t\tint flags;\n+\n+\t\tif (commit->generation > last_gen)\n+\t\t\tBUG(\"bad generation skip %8x > %8x at %s\",\n+\t\t\t commit->generation, last_gen,\n+\t\t\t oid_to_hex(&commit->object.oid));\n+\t\tlast_gen = commit->generation;\n+\n+\t\tif (commit->generation < min_generation)\n+\t\t\tbreak;\n+\n+\t\tflags = commit->object.flags & (PARENT1 | PARENT2 | STALE);\n+\t\tif (flags == (PARENT1 | PARENT2)) {\n+\t\t\tif (!(commit->object.flags & RESULT)) {\n+\t\t\t\tcommit->object.flags |= RESULT;\n+\t\t\t\tcommit_list_insert_by_date(commit, &result);\n+\t\t\t}\n+\t\t\t/* Mark parents of a found merge stale */\n+\t\t\tflags |= STALE;\n+\t\t}\n+\t\tparents = commit->parents;\n+\t\twhile (parents) {\n+\t\t\tstruct commit *p = parents->item;\n+\t\t\tparents = parents->next;\n+\t\t\tif ((p->object.flags & flags) == flags)\n+\t\t\t\tcontinue;\n+\t\t\tif (parse_commit(p))\n+\t\t\t\treturn NULL;\n+\t\t\tp->object.flags |= flags;\n+\t\t\tprio_queue_put(&queue, p);\n+\t\t}\n+\t}\n+\n+\tclear_prio_queue(&queue);\n+\treturn result;\n+}\n+\n+static struct commit_list *merge_bases_many(struct commit *one, int n, struct commit **twos)\n+{\n+\tstruct commit_list *list = NULL;\n+\tstruct commit_list *result = NULL;\n+\tint i;\n+\n+\tfor (i = 0; i < n; i++) {\n+\t\tif (one == twos[i])\n+\t\t\t/*\n+\t\t\t * We do not mark this even with RESULT so we do not\n+\t\t\t * have to clean it up.\n+\t\t\t */\n+\t\t\treturn commit_list_insert(one, &result);\n+\t}\n+\n+\tif (parse_commit(one))\n+\t\treturn NULL;\n+\tfor (i = 0; i < n; i++) {\n+\t\tif (parse_commit(twos[i]))\n+\t\t\treturn NULL;\n+\t}\n+\n+\tlist = paint_down_to_common(one, n, twos, 0);\n+\n+\twhile (list) {\n+\t\tstruct commit *commit = pop_commit(&list);\n+\t\tif (!(commit->object.flags & STALE))\n+\t\t\tcommit_list_insert_by_date(commit, &result);\n+\t}\n+\treturn result;\n+}\n+\n+struct commit_list *get_octopus_merge_bases(struct commit_list *in)\n+{\n+\tstruct commit_list *i, *j, *k, *ret = NULL;\n+\n+\tif (!in)\n+\t\treturn ret;\n+\n+\tcommit_list_insert(in->item, &ret);\n+\n+\tfor (i = in->next; i; i = i->next) {\n+\t\tstruct commit_list *new_commits = NULL, *end = NULL;\n+\n+\t\tfor (j = ret; j; j = j->next) {\n+\t\t\tstruct commit_list *bases;\n+\t\t\tbases = get_merge_bases(i->item, j->item);\n+\t\t\tif (!new_commits)\n+\t\t\t\tnew_commits = bases;\n+\t\t\telse\n+\t\t\t\tend->next = bases;\n+\t\t\tfor (k = bases; k; k = k->next)\n+\t\t\t\tend = k;\n+\t\t}\n+\t\tret = new_commits;\n+\t}\n+\treturn ret;\n+}\n+\n+static int remove_redundant(struct commit **array, int cnt)\n+{\n+\t/*\n+\t * Some commit in the array may be an ancestor of\n+\t * another commit. Move such commit to the end of\n+\t * the array, and return the number of commits that\n+\t * are independent from each other.\n+\t */\n+\tstruct commit **work;\n+\tunsigned char *redundant;\n+\tint *filled_index;\n+\tint i, j, filled;\n+\n+\twork = xcalloc(cnt, sizeof(*work));\n+\tredundant = xcalloc(cnt, 1);\n+\tALLOC_ARRAY(filled_index, cnt - 1);\n+\n+\tfor (i = 0; i < cnt; i++)\n+\t\tparse_commit(array[i]);\n+\tfor (i = 0; i < cnt; i++) {\n+\t\tstruct commit_list *common;\n+\t\tuint32_t min_generation = array[i]->generation;\n+\n+\t\tif (redundant[i])\n+\t\t\tcontinue;\n+\t\tfor (j = filled = 0; j < cnt; j++) {\n+\t\t\tif (i == j || redundant[j])\n+\t\t\t\tcontinue;\n+\t\t\tfilled_index[filled] = j;\n+\t\t\twork[filled++] = array[j];\n+\n+\t\t\tif (array[j]->generation < min_generation)\n+\t\t\t\tmin_generation = array[j]->generation;\n+\t\t}\n+\t\tcommon = paint_down_to_common(array[i], filled, work,\n+\t\t\t\t\t min_generation);\n+\t\tif (array[i]->object.flags & PARENT2)\n+\t\t\tredundant[i] = 1;\n+\t\tfor (j = 0; j < filled; j++)\n+\t\t\tif (work[j]->object.flags & PARENT1)\n+\t\t\t\tredundant[filled_index[j]] = 1;\n+\t\tclear_commit_marks(array[i], all_flags);\n+\t\tclear_commit_marks_many(filled, work, all_flags);\n+\t\tfree_commit_list(common);\n+\t}\n+\n+\t/* Now collect the result */\n+\tCOPY_ARRAY(work, array, cnt);\n+\tfor (i = filled = 0; i < cnt; i++)\n+\t\tif (!redundant[i])\n+\t\t\tarray[filled++] = work[i];\n+\tfor (j = filled, i = 0; i < cnt; i++)\n+\t\tif (redundant[i])\n+\t\t\tarray[j++] = work[i];\n+\tfree(work);\n+\tfree(redundant);\n+\tfree(filled_index);\n+\treturn filled;\n+}\n+\n+static struct commit_list *get_merge_bases_many_0(struct commit *one,\n+\t\t\t\t\t\t int n,\n+\t\t\t\t\t\t struct commit **twos,\n+\t\t\t\t\t\t int cleanup)\n+{\n+\tstruct commit_list *list;\n+\tstruct commit **rslt;\n+\tstruct commit_list *result;\n+\tint cnt, i;\n+\n+\tresult = merge_bases_many(one, n, twos);\n+\tfor (i = 0; i < n; i++) {\n+\t\tif (one == twos[i])\n+\t\t\treturn result;\n+\t}\n+\tif (!result || !result->next) {\n+\t\tif (cleanup) {\n+\t\t\tclear_commit_marks(one, all_flags);\n+\t\t\tclear_commit_marks_many(n, twos, all_flags);\n+\t\t}\n+\t\treturn result;\n+\t}\n+\n+\t/* There are more than one */\n+\tcnt = commit_list_count(result);\n+\trslt = xcalloc(cnt, sizeof(*rslt));\n+\tfor (list = result, i = 0; list; list = list->next)\n+\t\trslt[i++] = list->item;\n+\tfree_commit_list(result);\n+\n+\tclear_commit_marks(one, all_flags);\n+\tclear_commit_marks_many(n, twos, all_flags);\n+\n+\tcnt = remove_redundant(rslt, cnt);\n+\tresult = NULL;\n+\tfor (i = 0; i < cnt; i++)\n+\t\tcommit_list_insert_by_date(rslt[i], &result);\n+\tfree(rslt);\n+\treturn result;\n+}\n+\n+struct commit_list *get_merge_bases_many(struct commit *one,\n+\t\t\t\t\t int n,\n+\t\t\t\t\t struct commit **twos)\n+{\n+\treturn get_merge_bases_many_0(one, n, twos, 1);\n+}\n+\n+struct commit_list *get_merge_bases_many_dirty(struct commit *one,\n+\t\t\t\t\t int n,\n+\t\t\t\t\t struct commit **twos)\n+{\n+\treturn get_merge_bases_many_0(one, n, twos, 0);\n+}\n+\n+struct commit_list *get_merge_bases(struct commit *one, struct commit *two)\n+{\n+\treturn get_merge_bases_many_0(one, 1, &two, 1);\n+}\n+\n+/*\n+ * Is \"commit\" a descendant of one of the elements on the \"with_commit\" list?\n+ */\n+int is_descendant_of(struct commit *commit, struct commit_list *with_commit)\n+{\n+\tif (!with_commit)\n+\t\treturn 1;\n+\twhile (with_commit) {\n+\t\tstruct commit *other;\n+\n+\t\tother = with_commit->item;\n+\t\twith_commit = with_commit->next;\n+\t\tif (in_merge_bases(other, commit))\n+\t\t\treturn 1;\n+\t}\n+\treturn 0;\n+}\n+\n+/*\n+ * Is \"commit\" an ancestor of one of the \"references\"?\n+ */\n+int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit **reference)\n+{\n+\tstruct commit_list *bases;\n+\tint ret = 0, i;\n+\tuint32_t min_generation = GENERATION_NUMBER_INFINITY;\n+\n+\tif (parse_commit(commit))\n+\t\treturn ret;\n+\tfor (i = 0; i < nr_reference; i++) {\n+\t\tif (parse_commit(reference[i]))\n+\t\t\treturn ret;\n+\t\tif (reference[i]->generation < min_generation)\n+\t\t\tmin_generation = reference[i]->generation;\n+\t}\n+\n+\tif (commit->generation > min_generation)\n+\t\treturn ret;\n+\n+\tbases = paint_down_to_common(commit, nr_reference, reference, commit->generation);\n+\tif (commit->object.flags & PARENT2)\n+\t\tret = 1;\n+\tclear_commit_marks(commit, all_flags);\n+\tclear_commit_marks_many(nr_reference, reference, all_flags);\n+\tfree_commit_list(bases);\n+\treturn ret;\n+}\n+\n+/*\n+ * Is \"commit\" an ancestor of (i.e. reachable from) the \"reference\"?\n+ */\n+int in_merge_bases(struct commit *commit, struct commit *reference)\n+{\n+\treturn in_merge_bases_many(commit, 1, &reference);\n+}\n+\n+struct commit_list *reduce_heads(struct commit_list *heads)\n+{\n+\tstruct commit_list *p;\n+\tstruct commit_list *result = NULL, **tail = &result;\n+\tstruct commit **array;\n+\tint num_head, i;\n+\n+\tif (!heads)\n+\t\treturn NULL;\n+\n+\t/* Uniquify */\n+\tfor (p = heads; p; p = p->next)\n+\t\tp->item->object.flags &= ~STALE;\n+\tfor (p = heads, num_head = 0; p; p = p->next) {\n+\t\tif (p->item->object.flags & STALE)\n+\t\t\tcontinue;\n+\t\tp->item->object.flags |= STALE;\n+\t\tnum_head++;\n+\t}\n+\tarray = xcalloc(num_head, sizeof(*array));\n+\tfor (p = heads, i = 0; p; p = p->next) {\n+\t\tif (p->item->object.flags & STALE) {\n+\t\t\tarray[i++] = p->item;\n+\t\t\tp->item->object.flags &= ~STALE;\n+\t\t}\n+\t}\n+\tnum_head = remove_redundant(array, num_head);\n+\tfor (i = 0; i < num_head; i++)\n+\t\ttail = &commit_list_insert(array[i], tail)->next;\n+\tfree(array);\n+\treturn result;\n+}\n+\n+void reduce_heads_replace(struct commit_list **heads)\n+{\n+\tstruct commit_list *result = reduce_heads(*heads);\n+\tfree_commit_list(*heads);\n+\t*heads = result;\n+}\ndiff --git a/commit-reach.h b/commit-reach.h\nnew file mode 100644\nindex 0000000000..1ea2696e40\n--- /dev/null\n+++ b/commit-reach.h\n@@ -0,0 +1,42 @@\n+#ifndef __COMMIT_REACH_H__\n+#define __COMMIT_REACH_H__\n+\n+struct commit;\n+struct commit_list;\n+\n+struct commit_list *get_merge_bases_many(struct commit *one,\n+\t\t\t\t\t int n,\n+\t\t\t\t\t struct commit **twos);\n+struct commit_list *get_merge_bases_many_dirty(struct commit *one,\n+\t\t\t\t\t int n,\n+\t\t\t\t\t struct commit **twos);\n+struct commit_list *get_merge_bases(struct commit *one, struct commit *two);\n+struct commit_list *get_octopus_merge_bases(struct commit_list *in);\n+\n+/* To be used only when object flags after this call no longer matter */\n+struct commit_list *get_merge_bases_many_dirty(struct commit *one, int n, struct commit **twos);\n+\n+int is_descendant_of(struct commit *commit, struct commit_list *with_commit);\n+int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit **reference);\n+int in_merge_bases(struct commit *commit, struct commit *reference);\n+\n+\n+/*\n+ * Takes a list of commits and returns a new list where those\n+ * have been removed that can be reached from other commits in\n+ * the list. It is useful for, e.g., reducing the commits\n+ * randomly thrown at the git-merge command and removing\n+ * redundant commits that the user shouldn't have given to it.\n+ *\n+ * This function destroys the STALE bit of the commit objects'\n+ * flags.\n+ */\n+struct commit_list *reduce_heads(struct commit_list *heads);\n+\n+/*\n+ * Like `reduce_heads()`, except it replaces the list. Use this\n+ * instead of `foo = reduce_heads(foo);` to avoid memory leaks.\n+ */\n+void reduce_heads_replace(struct commit_list **heads);\n+\n+#endif\ndiff --git a/commit.c b/commit.c\nindex 39b80bd21d..32d1234bd7 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -843,364 +843,6 @@ void sort_in_topological_order(struct commit_list **list, enum rev_sort_order so\n \t\tclear_author_date_slab(&author_date);\n }\n \n-/* merge-base stuff */\n-\n-/* Remember to update object flag allocation in object.h */\n-#define PARENT1\t\t(1u<<16)\n-#define PARENT2\t\t(1u<<17)\n-#define STALE\t\t(1u<<18)\n-#define RESULT\t\t(1u<<19)\n-\n-static const unsigned all_flags = (PARENT1 | PARENT2 | STALE | RESULT);\n-\n-static int queue_has_nonstale(struct prio_queue *queue)\n-{\n-\tint i;\n-\tfor (i = 0; i < queue->nr; i++) {\n-\t\tstruct commit *commit = queue->array[i].data;\n-\t\tif (!(commit->object.flags & STALE))\n-\t\t\treturn 1;\n-\t}\n-\treturn 0;\n-}\n-\n-/* all input commits in one and twos[] must have been parsed! */\n-static struct commit_list *paint_down_to_common(struct commit *one, int n,\n-\t\t\t\t\t\tstruct commit **twos,\n-\t\t\t\t\t\tint min_generation)\n-{\n-\tstruct prio_queue queue = { compare_commits_by_gen_then_commit_date };\n-\tstruct commit_list *result = NULL;\n-\tint i;\n-\tuint32_t last_gen = GENERATION_NUMBER_INFINITY;\n-\n-\tone->object.flags |= PARENT1;\n-\tif (!n) {\n-\t\tcommit_list_append(one, &result);\n-\t\treturn result;\n-\t}\n-\tprio_queue_put(&queue, one);\n-\n-\tfor (i = 0; i < n; i++) {\n-\t\ttwos[i]->object.flags |= PARENT2;\n-\t\tprio_queue_put(&queue, twos[i]);\n-\t}\n-\n-\twhile (queue_has_nonstale(&queue)) {\n-\t\tstruct commit *commit = prio_queue_get(&queue);\n-\t\tstruct commit_list *parents;\n-\t\tint flags;\n-\n-\t\tif (commit->generation > last_gen)\n-\t\t\tBUG(\"bad generation skip %8x > %8x at %s\",\n-\t\t\t commit->generation, last_gen,\n-\t\t\t oid_to_hex(&commit->object.oid));\n-\t\tlast_gen = commit->generation;\n-\n-\t\tif (commit->generation < min_generation)\n-\t\t\tbreak;\n-\n-\t\tflags = commit->object.flags & (PARENT1 | PARENT2 | STALE);\n-\t\tif (flags == (PARENT1 | PARENT2)) {\n-\t\t\tif (!(commit->object.flags & RESULT)) {\n-\t\t\t\tcommit->object.flags |= RESULT;\n-\t\t\t\tcommit_list_insert_by_date(commit, &result);\n-\t\t\t}\n-\t\t\t/* Mark parents of a found merge stale */\n-\t\t\tflags |= STALE;\n-\t\t}\n-\t\tparents = commit->parents;\n-\t\twhile (parents) {\n-\t\t\tstruct commit *p = parents->item;\n-\t\t\tparents = parents->next;\n-\t\t\tif ((p->object.flags & flags) == flags)\n-\t\t\t\tcontinue;\n-\t\t\tif (parse_commit(p))\n-\t\t\t\treturn NULL;\n-\t\t\tp->object.flags |= flags;\n-\t\t\tprio_queue_put(&queue, p);\n-\t\t}\n-\t}\n-\n-\tclear_prio_queue(&queue);\n-\treturn result;\n-}\n-\n-static struct commit_list *merge_bases_many(struct commit *one, int n, struct commit **twos)\n-{\n-\tstruct commit_list *list = NULL;\n-\tstruct commit_list *result = NULL;\n-\tint i;\n-\n-\tfor (i = 0; i < n; i++) {\n-\t\tif (one == twos[i])\n-\t\t\t/*\n-\t\t\t * We do not mark this even with RESULT so we do not\n-\t\t\t * have to clean it up.\n-\t\t\t */\n-\t\t\treturn commit_list_insert(one, &result);\n-\t}\n-\n-\tif (parse_commit(one))\n-\t\treturn NULL;\n-\tfor (i = 0; i < n; i++) {\n-\t\tif (parse_commit(twos[i]))\n-\t\t\treturn NULL;\n-\t}\n-\n-\tlist = paint_down_to_common(one, n, twos, 0);\n-\n-\twhile (list) {\n-\t\tstruct commit *commit = pop_commit(&list);\n-\t\tif (!(commit->object.flags & STALE))\n-\t\t\tcommit_list_insert_by_date(commit, &result);\n-\t}\n-\treturn result;\n-}\n-\n-struct commit_list *get_octopus_merge_bases(struct commit_list *in)\n-{\n-\tstruct commit_list *i, *j, *k, *ret = NULL;\n-\n-\tif (!in)\n-\t\treturn ret;\n-\n-\tcommit_list_insert(in->item, &ret);\n-\n-\tfor (i = in->next; i; i = i->next) {\n-\t\tstruct commit_list *new_commits = NULL, *end = NULL;\n-\n-\t\tfor (j = ret; j; j = j->next) {\n-\t\t\tstruct commit_list *bases;\n-\t\t\tbases = get_merge_bases(i->item, j->item);\n-\t\t\tif (!new_commits)\n-\t\t\t\tnew_commits = bases;\n-\t\t\telse\n-\t\t\t\tend->next = bases;\n-\t\t\tfor (k = bases; k; k = k->next)\n-\t\t\t\tend = k;\n-\t\t}\n-\t\tret = new_commits;\n-\t}\n-\treturn ret;\n-}\n-\n-static int remove_redundant(struct commit **array, int cnt)\n-{\n-\t/*\n-\t * Some commit in the array may be an ancestor of\n-\t * another commit. Move such commit to the end of\n-\t * the array, and return the number of commits that\n-\t * are independent from each other.\n-\t */\n-\tstruct commit **work;\n-\tunsigned char *redundant;\n-\tint *filled_index;\n-\tint i, j, filled;\n-\n-\twork = xcalloc(cnt, sizeof(*work));\n-\tredundant = xcalloc(cnt, 1);\n-\tALLOC_ARRAY(filled_index, cnt - 1);\n-\n-\tfor (i = 0; i < cnt; i++)\n-\t\tparse_commit(array[i]);\n-\tfor (i = 0; i < cnt; i++) {\n-\t\tstruct commit_list *common;\n-\t\tuint32_t min_generation = array[i]->generation;\n-\n-\t\tif (redundant[i])\n-\t\t\tcontinue;\n-\t\tfor (j = filled = 0; j < cnt; j++) {\n-\t\t\tif (i == j || redundant[j])\n-\t\t\t\tcontinue;\n-\t\t\tfilled_index[filled] = j;\n-\t\t\twork[filled++] = array[j];\n-\n-\t\t\tif (array[j]->generation < min_generation)\n-\t\t\t\tmin_generation = array[j]->generation;\n-\t\t}\n-\t\tcommon = paint_down_to_common(array[i], filled, work,\n-\t\t\t\t\t min_generation);\n-\t\tif (array[i]->object.flags & PARENT2)\n-\t\t\tredundant[i] = 1;\n-\t\tfor (j = 0; j < filled; j++)\n-\t\t\tif (work[j]->object.flags & PARENT1)\n-\t\t\t\tredundant[filled_index[j]] = 1;\n-\t\tclear_commit_marks(array[i], all_flags);\n-\t\tclear_commit_marks_many(filled, work, all_flags);\n-\t\tfree_commit_list(common);\n-\t}\n-\n-\t/* Now collect the result */\n-\tCOPY_ARRAY(work, array, cnt);\n-\tfor (i = filled = 0; i < cnt; i++)\n-\t\tif (!redundant[i])\n-\t\t\tarray[filled++] = work[i];\n-\tfor (j = filled, i = 0; i < cnt; i++)\n-\t\tif (redundant[i])\n-\t\t\tarray[j++] = work[i];\n-\tfree(work);\n-\tfree(redundant);\n-\tfree(filled_index);\n-\treturn filled;\n-}\n-\n-static struct commit_list *get_merge_bases_many_0(struct commit *one,\n-\t\t\t\t\t\t int n,\n-\t\t\t\t\t\t struct commit **twos,\n-\t\t\t\t\t\t int cleanup)\n-{\n-\tstruct commit_list *list;\n-\tstruct commit **rslt;\n-\tstruct commit_list *result;\n-\tint cnt, i;\n-\n-\tresult = merge_bases_many(one, n, twos);\n-\tfor (i = 0; i < n; i++) {\n-\t\tif (one == twos[i])\n-\t\t\treturn result;\n-\t}\n-\tif (!result || !result->next) {\n-\t\tif (cleanup) {\n-\t\t\tclear_commit_marks(one, all_flags);\n-\t\t\tclear_commit_marks_many(n, twos, all_flags);\n-\t\t}\n-\t\treturn result;\n-\t}\n-\n-\t/* There are more than one */\n-\tcnt = commit_list_count(result);\n-\trslt = xcalloc(cnt, sizeof(*rslt));\n-\tfor (list = result, i = 0; list; list = list->next)\n-\t\trslt[i++] = list->item;\n-\tfree_commit_list(result);\n-\n-\tclear_commit_marks(one, all_flags);\n-\tclear_commit_marks_many(n, twos, all_flags);\n-\n-\tcnt = remove_redundant(rslt, cnt);\n-\tresult = NULL;\n-\tfor (i = 0; i < cnt; i++)\n-\t\tcommit_list_insert_by_date(rslt[i], &result);\n-\tfree(rslt);\n-\treturn result;\n-}\n-\n-struct commit_list *get_merge_bases_many(struct commit *one,\n-\t\t\t\t\t int n,\n-\t\t\t\t\t struct commit **twos)\n-{\n-\treturn get_merge_bases_many_0(one, n, twos, 1);\n-}\n-\n-struct commit_list *get_merge_bases_many_dirty(struct commit *one,\n-\t\t\t\t\t int n,\n-\t\t\t\t\t struct commit **twos)\n-{\n-\treturn get_merge_bases_many_0(one, n, twos, 0);\n-}\n-\n-struct commit_list *get_merge_bases(struct commit *one, struct commit *two)\n-{\n-\treturn get_merge_bases_many_0(one, 1, &two, 1);\n-}\n-\n-/*\n- * Is \"commit\" a descendant of one of the elements on the \"with_commit\" list?\n- */\n-int is_descendant_of(struct commit *commit, struct commit_list *with_commit)\n-{\n-\tif (!with_commit)\n-\t\treturn 1;\n-\twhile (with_commit) {\n-\t\tstruct commit *other;\n-\n-\t\tother = with_commit->item;\n-\t\twith_commit = with_commit->next;\n-\t\tif (in_merge_bases(other, commit))\n-\t\t\treturn 1;\n-\t}\n-\treturn 0;\n-}\n-\n-/*\n- * Is \"commit\" an ancestor of one of the \"references\"?\n- */\n-int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit **reference)\n-{\n-\tstruct commit_list *bases;\n-\tint ret = 0, i;\n-\tuint32_t min_generation = GENERATION_NUMBER_INFINITY;\n-\n-\tif (parse_commit(commit))\n-\t\treturn ret;\n-\tfor (i = 0; i < nr_reference; i++) {\n-\t\tif (parse_commit(reference[i]))\n-\t\t\treturn ret;\n-\t\tif (reference[i]->generation < min_generation)\n-\t\t\tmin_generation = reference[i]->generation;\n-\t}\n-\n-\tif (commit->generation > min_generation)\n-\t\treturn ret;\n-\n-\tbases = paint_down_to_common(commit, nr_reference, reference, commit->generation);\n-\tif (commit->object.flags & PARENT2)\n-\t\tret = 1;\n-\tclear_commit_marks(commit, all_flags);\n-\tclear_commit_marks_many(nr_reference, reference, all_flags);\n-\tfree_commit_list(bases);\n-\treturn ret;\n-}\n-\n-/*\n- * Is \"commit\" an ancestor of (i.e. reachable from) the \"reference\"?\n- */\n-int in_merge_bases(struct commit *commit, struct commit *reference)\n-{\n-\treturn in_merge_bases_many(commit, 1, &reference);\n-}\n-\n-struct commit_list *reduce_heads(struct commit_list *heads)\n-{\n-\tstruct commit_list *p;\n-\tstruct commit_list *result = NULL, **tail = &result;\n-\tstruct commit **array;\n-\tint num_head, i;\n-\n-\tif (!heads)\n-\t\treturn NULL;\n-\n-\t/* Uniquify */\n-\tfor (p = heads; p; p = p->next)\n-\t\tp->item->object.flags &= ~STALE;\n-\tfor (p = heads, num_head = 0; p; p = p->next) {\n-\t\tif (p->item->object.flags & STALE)\n-\t\t\tcontinue;\n-\t\tp->item->object.flags |= STALE;\n-\t\tnum_head++;\n-\t}\n-\tarray = xcalloc(num_head, sizeof(*array));\n-\tfor (p = heads, i = 0; p; p = p->next) {\n-\t\tif (p->item->object.flags & STALE) {\n-\t\t\tarray[i++] = p->item;\n-\t\t\tp->item->object.flags &= ~STALE;\n-\t\t}\n-\t}\n-\tnum_head = remove_redundant(array, num_head);\n-\tfor (i = 0; i < num_head; i++)\n-\t\ttail = &commit_list_insert(array[i], tail)->next;\n-\tfree(array);\n-\treturn result;\n-}\n-\n-void reduce_heads_replace(struct commit_list **heads)\n-{\n-\tstruct commit_list *result = reduce_heads(*heads);\n-\tfree_commit_list(*heads);\n-\t*heads = result;\n-}\n-\n static const char gpg_sig_header[] = \"gpgsig\";\n static const int gpg_sig_header_len = sizeof(gpg_sig_header) - 1;\n \ndiff --git a/object.h b/object.h\nindex fa5ca97567..18c2b073e3 100644\n--- a/object.h\n+++ b/object.h\n@@ -65,7 +65,7 @@ struct object_array {\n * bisect.c: 16\n * bundle.c: 16\n * http-push.c: 16-----19\n- * commit.c: 16-----19\n+ * commit-reach.c: 16-----19\n * sha1-name.c: 20\n * list-objects-filter.c: 21\n * builtin/fsck.c: 0--3\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353144","messageId":"20180720163227.105950-19-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 18/18] commit-reach: use can_all_from_reach","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:30Z","receivedAt":"2018-07-20T16:33:45Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"The is_descendant_of method previously used in_merge_bases() to check if\nthe commit can reach any of the commits in the provided list. This had\ntwo performance problems:\n\n1. The performance is quadratic in worst-case.\n\n2. A single in_merge_bases() call requires walking beyond the target\n commit in order to find the full set of boundary commits that may be\n merge-bases.\n\nThe can_all_from_reach method avoids this quadratic behavior and can\nlimit the search beyond the target commits using generation numbers. It\nrequires a small prototype adjustment to stop using commit-date as a\ncutoff, as that optimization is no longer appropriate here.\n\nSince in_merge_bases() uses paint_down_to_common(), is_descendant_of()\nnaturally found cutoffs to avoid walking the entire commit graph. Since\nwe want to always return the correct result, we cannot use the\nmin_commit_date cutoff in can_all_from_reach. We then rely on generation\nnumbers to provide the cutoff.\n\nSince not all repos will have a commit-graph file, nor will we always\nhave generation numbers computed for a commit-graph file, create a new\nmethod, generation_numbers_enabled(), that checks for a commit-graph\nfile and sees if the first commit in the file has a non-zero generation\nnumber. In the case that we do not have generation numbers, use the old\nlogic for is_descendant_of().\n\nPerformance was meausured on a copy of the Linux repository using the\n'test-tool reach is_descendant_of' command using this input:\n\nA:v4.9\nX:v4.10\nX:v4.11\nX:v4.12\nX:v4.13\nX:v4.14\nX:v4.15\nX:v4.16\nX:v4.17\nX.v3.0\n\nNote that this input is tailored to demonstrate the quadratic nature of\nthe previous method, as it will compute merge-bases for v4.9 versus all\nof the later versions before checking against v4.1.\n\nBefore: 0.26 s\n After: 0.21 s\n\nSince we previously used the is_descendant_of method in the ref_newer\nmethod, we also measured performance there using\n'test-tool reach ref_newer' with this input:\n\nA:v4.9\nB:v3.19\n\nBefore: 0.10 s\n After: 0.08 s\n\nBy adding a new commit with parent v3.19, we test the non-reachable case\nof ref_newer:\n\nBefore: 0.09 s\n After: 0.08 s\n\nSigned-off-by: Derrick Stolee \n---\n commit-graph.c | 18 ++++++++++++++++++\n commit-graph.h | 6 ++++++\n commit-reach.c | 24 +++++++++++++++++-------\n 3 files changed, 41 insertions(+), 7 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex b0a55ad128..e9786fa864 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -233,6 +233,24 @@ static int prepare_commit_graph(struct repository *r)\n \treturn !!r->objects->commit_graph;\n }\n \n+int generation_numbers_enabled(struct repository *r)\n+{\n+\tuint32_t first_generation;\n+\tstruct commit_graph *g;\n+\tif (!prepare_commit_graph(r))\n+\t return 0;\n+\n+\tg = r->objects->commit_graph;\n+\n+\tif (!g->num_commits)\n+\t\treturn 0;\n+\n+\tfirst_generation = get_be32(g->chunk_commit_data +\n+\t\t\t\t g->hash_len + 8) >> 2;\n+\n+\treturn !!first_generation;\n+}\n+\n static void close_commit_graph(void)\n {\n \tfree_commit_graph(the_repository->objects->commit_graph);\ndiff --git a/commit-graph.h b/commit-graph.h\nindex 76e098934a..0de8f88316 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -51,6 +51,12 @@ struct commit_graph {\n \n struct commit_graph *load_commit_graph_one(const char *graph_file);\n \n+/*\n+ * Return 1 if and only if the repository has a commit-graph\n+ * file and generation numbers are computed in that file.\n+ */\n+int generation_numbers_enabled(struct repository *r);\n+\n void write_commit_graph_reachable(const char *obj_dir, int append);\n void write_commit_graph(const char *obj_dir,\n \t\t\tstruct string_list *pack_indexes,\ndiff --git a/commit-reach.c b/commit-reach.c\nindex bc522d6840..c996524032 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -277,15 +277,25 @@ int is_descendant_of(struct commit *commit, struct commit_list *with_commit)\n {\n \tif (!with_commit)\n \t\treturn 1;\n-\twhile (with_commit) {\n-\t\tstruct commit *other;\n \n-\t\tother = with_commit->item;\n-\t\twith_commit = with_commit->next;\n-\t\tif (in_merge_bases(other, commit))\n-\t\t\treturn 1;\n+\tif (generation_numbers_enabled(the_repository)) {\n+\t\tstruct commit_list *from_list = NULL;\n+\t\tint result;\n+\t\tcommit_list_insert(commit, &from_list);\n+\t\tresult = can_all_from_reach(from_list, with_commit, 0);\n+\t\tfree_commit_list(from_list);\n+\t\treturn result;\n+\t} else {\n+\t\twhile (with_commit) {\n+\t\t\tstruct commit *other;\n+\n+\t\t\tother = with_commit->item;\n+\t\t\twith_commit = with_commit->next;\n+\t\t\tif (in_merge_bases(other, commit))\n+\t\t\t\treturn 1;\n+\t\t}\n+\t\treturn 0;\n \t}\n-\treturn 0;\n }\n \n /*\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353146","messageId":"20180720163227.105950-18-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 17/18] commit-reach: make can_all_from_reach... linear","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:28Z","receivedAt":"2018-07-20T16:33:46Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"The can_all_from_reach_with_flags() algorithm is currently quadratic in\nthe worst case, because it calls the reachable() method for every 'from'\nwithout tracking which commits have already been walked or which can\nalready reach a commit in 'to'.\n\nRewrite the algorithm to walk each commit a constant number of times.\n\nWe also add some optimizations that should work for the main consumer of\nthis method: fetch negotitation (haves/wants).\n\nThe first step includes using a depth-first-search (DFS) from each\n'from' commit, sorted by ascending generation number. We do not walk\nbeyond the minimum generation number or the minimum commit date. This\nDFS is likely to be faster than the existing reachable() method because\nwe expect previous ref values to be along the first-parent history.\n\nIf we find a target commit, then we mark everything in the DFS stack as\na RESULT. This expands the set of targets for the other 'from' commits.\nWe also mark the visited commits using 'assign_flag' to prevent re-\nwalking the same commits.\n\nWe still need to clear our flags at the end, which is why we will have a\ntotal of three visits to each commit.\n\nPerformance was measured on the Linux repository using\n'test-tool reach can_all_from_reach'. The input included rows seeded by\ntag values. The \"small\" case included X-rows as v4.[0-9]* and Y-rows as\nv3.[0-9]*. This mimics a (very large) fetch that says \"I have all major\nv3 releases and want all major v4 releases.\" The \"large\" case included\nX-rows as \"v4.*\" and Y-rows as \"v3.*\". This adds all release-candidate\ntags to the set, which does not greatly increase the number of objects\nthat are considered, but does increase the number of 'from' commits,\ndemonstrating the quadratic nature of the previous code.\n\nSmall Case:\n\nBefore: 1.52 s\n After: 0.26 s\n\nLarge Case:\n\nBefore: 3.50 s\n After: 0.27 s\n\nNote how the time increases between the two cases in the two versions.\nThe new code increases relative to the number of commits that need to be\nwalked, but not directly relative to the number of 'from' commits.\n\nSigned-off-by: Derrick Stolee \n---\n commit-reach.c | 122 ++++++++++++++++++++++++++++++-------------------\n commit-reach.h | 9 ++--\n upload-pack.c | 5 +-\n 3 files changed, 83 insertions(+), 53 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex f5858944fd..bc522d6840 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -514,66 +514,87 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n \treturn is_descendant_of(commit, list);\n }\n \n-int reachable(struct commit *from, unsigned int with_flag,\n-\t unsigned int assign_flag, time_t min_commit_date)\n+static int compare_commits_by_gen(const void *_a, const void *_b)\n {\n-\tstruct prio_queue work = { compare_commits_by_commit_date };\n+\tconst struct commit *a = (const struct commit *)_a;\n+\tconst struct commit *b = (const struct commit *)_b;\n \n-\tprio_queue_put(&work, from);\n-\twhile (work.nr) {\n-\t\tstruct commit_list *list;\n-\t\tstruct commit *commit = prio_queue_get(&work);\n-\n-\t\tif (commit->object.flags & with_flag) {\n-\t\t\tfrom->object.flags |= assign_flag;\n-\t\t\tbreak;\n-\t\t}\n-\t\tif (!commit->object.parsed)\n-\t\t\tparse_object(the_repository, &commit->object.oid);\n-\t\tif (commit->object.flags & REACHABLE)\n-\t\t\tcontinue;\n-\t\tcommit->object.flags |= REACHABLE;\n-\t\tif (commit->date < min_commit_date)\n-\t\t\tcontinue;\n-\t\tfor (list = commit->parents; list; list = list->next) {\n-\t\t\tstruct commit *parent = list->item;\n-\t\t\tif (!(parent->object.flags & REACHABLE))\n-\t\t\t\tprio_queue_put(&work, parent);\n-\t\t}\n-\t}\n-\tfrom->object.flags |= REACHABLE;\n-\tclear_commit_marks(from, REACHABLE);\n-\tclear_prio_queue(&work);\n-\treturn (from->object.flags & assign_flag);\n+\tif (a->generation < b->generation)\n+\t\treturn -1;\n+\tif (a->generation > b->generation)\n+\t\treturn 1;\n+\treturn 0;\n }\n \n int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\t unsigned int with_flag,\n \t\t\t\t unsigned int assign_flag,\n-\t\t\t\t time_t min_commit_date)\n+\t\t\t\t time_t min_commit_date,\n+\t\t\t\t uint32_t min_generation)\n {\n+\tstruct commit **list = NULL;\n \tint i;\n+\tint result = 1;\n \n+\tALLOC_ARRAY(list, from->nr);\n \tfor (i = 0; i < from->nr; i++) {\n-\t\tstruct object *from_one = from->objects[i].item;\n+\t\tlist[i] = (struct commit *)from->objects[i].item;\n \n-\t\tif (from_one->flags & assign_flag)\n-\t\t\tcontinue;\n-\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n-\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n-\t\t\t/* no way to tell if this is reachable by\n-\t\t\t * looking at the ancestry chain alone, so\n-\t\t\t * leave a note to ourselves not to worry about\n-\t\t\t * this object anymore.\n-\t\t\t */\n-\t\t\tfrom->objects[i].item->flags |= assign_flag;\n-\t\t\tcontinue;\n-\t\t}\n-\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag,\n-\t\t\t min_commit_date))\n+\t\tif (parse_commit(list[i]) ||\n+\t\t list[i]->generation < min_generation)\n \t\t\treturn 0;\n \t}\n-\treturn 1;\n+\n+\tQSORT(list, from->nr, compare_commits_by_gen);\n+\n+\tfor (i = 0; i < from->nr; i++) {\n+\t\t/* DFS from list[i] */\n+\t\tstruct commit_list *stack = NULL;\n+\n+\t\tlist[i]->object.flags |= assign_flag;\n+\t\tcommit_list_insert(list[i], &stack);\n+\n+\t\twhile (stack) {\n+\t\t\tstruct commit_list *parent;\n+\n+\t\t\tif (stack->item->object.flags & with_flag) {\n+\t\t\t\tpop_commit(&stack);\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\tfor (parent = stack->item->parents; parent; parent = parent->next) {\n+\t\t\t\tif (parent->item->object.flags & (with_flag | RESULT))\n+\t\t\t\t\tstack->item->object.flags |= RESULT;\n+\n+\t\t\t\tif (!(parent->item->object.flags & assign_flag)) {\n+\t\t\t\t\tparent->item->object.flags |= assign_flag;\n+\n+\t\t\t\t\tif (parse_commit(parent->item) ||\n+\t\t\t\t\t parent->item->date < min_commit_date ||\n+\t\t\t\t\t parent->item->generation < min_generation)\n+\t\t\t\t\t\tcontinue;\n+\n+\t\t\t\t\tcommit_list_insert(parent->item, &stack);\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t}\n+\n+\t\t\tif (!parent)\n+\t\t\t\tpop_commit(&stack);\n+\t\t}\n+\n+\t\tif (!(list[i]->object.flags & (with_flag | RESULT))) {\n+\t\t\tresult = 0;\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n+cleanup:\n+\tfor (i = 0; i < from->nr; i++) {\n+\t\tclear_commit_marks(list[i], RESULT);\n+\t\tclear_commit_marks(list[i], assign_flag);\n+\t}\n+\treturn result;\n }\n \n int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n@@ -583,6 +604,7 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \ttime_t min_commit_date = cutoff_by_min_date ? from->item->date : 0;\n \tstruct commit_list *from_iter = from, *to_iter = to;\n \tint result;\n+\tuint32_t min_generation = GENERATION_NUMBER_INFINITY;\n \n \twhile (from_iter) {\n \t\tadd_object_array(&from_iter->item->object, NULL, &from_objs);\n@@ -590,6 +612,9 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \t\tif (!parse_commit(from_iter->item)) {\n \t\t\tif (from_iter->item->date < min_commit_date)\n \t\t\t\tmin_commit_date = from_iter->item->date;\n+\n+\t\t\tif (from_iter->item->generation < min_generation)\n+\t\t\t\tmin_generation = from_iter->item->generation;\n \t\t}\n \n \t\tfrom_iter = from_iter->next;\n@@ -599,6 +624,9 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \t\tif (!parse_commit(to_iter->item)) {\n \t\t\tif (to_iter->item->date < min_commit_date)\n \t\t\t\tmin_commit_date = to_iter->item->date;\n+\n+\t\t\tif (to_iter->item->generation < min_generation)\n+\t\t\t\tmin_generation = to_iter->item->generation;\n \t\t}\n \n \t\tto_iter->item->object.flags |= PARENT2;\n@@ -607,7 +635,7 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \t}\n \n \tresult = can_all_from_reach_with_flag(&from_objs, PARENT2, PARENT1,\n-\t\t\t\t\t min_commit_date);\n+\t\t\t\t\t min_commit_date, min_generation);\n \n \twhile (from) {\n \t\tclear_commit_marks(from->item, PARENT1);\ndiff --git a/commit-reach.h b/commit-reach.h\nindex aa202c9703..7d313e2975 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -59,19 +59,18 @@ define_commit_slab(contains_cache, enum contains_result);\n int commit_contains(struct ref_filter *filter, struct commit *commit,\n \t\t struct commit_list *list, struct contains_cache *cache);\n \n-int reachable(struct commit *from, unsigned int with_flag,\n-\t unsigned int assign_flag, time_t min_commit_date);\n-\n /*\n * Determine if every commit in 'from' can reach at least one commit\n * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n * as a marker for commits that are already visited. Do not walk\n- * commits with date below 'min_commit_date'.\n+ * commits with date below 'min_commit_date' or generation below\n+ * 'min_generation'.\n */\n int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\t unsigned int with_flag,\n \t\t\t\t unsigned int assign_flag,\n-\t\t\t\t time_t min_commit_date);\n+\t\t\t\t time_t min_commit_date,\n+\t\t\t\t uint32_t min_generation);\n int can_all_from_reach(struct commit_list *from, struct commit_list *to,\n \t\t int commit_date_cutoff);\n \ndiff --git a/upload-pack.c b/upload-pack.c\nindex 11c426685d..1e498f1188 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -338,11 +338,14 @@ static int got_oid(const char *hex, struct object_id *oid)\n \n static int ok_to_give_up(void)\n {\n+\tuint32_t min_generation = GENERATION_NUMBER_ZERO;\n+\n \tif (!have_obj.nr)\n \t\treturn 0;\n \n \treturn can_all_from_reach_with_flag(&want_obj, THEY_HAVE,\n-\t\t\t\t\t COMMON_KNOWN, oldest_have);\n+\t\t\t\t\t COMMON_KNOWN, oldest_have,\n+\t\t\t\t\t min_generation);\n }\n \n static int get_common_commits(void)\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353147","messageId":"20180720163227.105950-1-dstolee@microsoft.com","threadId":"48889","inReplyTo":"pull.10.git.gitgitgadget@gmail.com","subject":"[PATCH v2 00/18] Consolidate reachability logic","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:00Z","receivedAt":"2018-07-20T16:33:56Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"There are many places in Git that use a commit walk to determine\nreachability between commits and/or refs. A lot of this logic is\nduplicated.\n\nI wanted to achieve the following:\n\nConsolidate several different commit walks into one file\nReduce duplicate reachability logic\nIncrease testability (correctness and performance)\nImprove performance of reachability queries\nMy approach is mostly in three parts:\n\nI. Move code to a new commit-reach.c file.\nII. Add a 'test-tool reach' command to test these methods directly.\nIII. Modify the logic by improving performance and calling methods with\nsimilar logic but different prototypes.\n\nThe 'test-tool reach' command is helpful to make sure I don't break\nanything as I change the logic, but also so I can test methods that are\nnormally only exposed by other more complicated commands. For instance,\nref_newer() is part of 'git push -f' and ok_to_give_up() is buried deep\nwithin fetch negotiation. Both of these methods have some problematic\nperformance issues that are corrected by this series. As I discovered\nthem, it was clear that it would be better to consolidate walk logic\ninstead of discovering a new walk in another file hidden somewhere.\n\nFor the ok_to_give_up() method, I refactored the method so I could pull\nthe logic out of the depths of fetch negotiation. In the commit\n\"commit-reach: make can_all_from_reach... linear\" I discuss how the\nexisting algorithm is quadratic and how we can make it linear. Also, we\ncan use heuristic knowledge about the shape of the commit graph and the\nusual haves/wants to get some extra performance bonus. (The heuristic is\nto do a DFS with first-parents first, and stop on first found result. We\nexpect haves/wants to include ref tips, which typically have their\nprevious values in their first-parent history.)\n\nOne major difference in this series versus the RFC is that I added a new\nmethod 'generation_numbers_enabled()' to detect if we have a commit-graph\nfile with non-zero generation numbers. Using can_all_from_reach in\nis_descendant_of is only faster if we have generation numbers as a cutoff.\n\nV2 Update: The biggest material change in this version is that we drop the\nmethod declarations from commit.h, which requires adding a lot of references\nto commit-reach.h across the codebase. This change is in a commit on its own.\nIn addition, we have the following smaller changes:\n\n* Use 'unsigned int' for the flag variables.\n\n* Properly align the here-doc test input data.\n\n* Use single rev-parse commands in test output, and pipe the OIDs through 'sort'\n\n* Check output of parse_commit()\n\n* Update flag documentation in object.h\n\n* Add tests for commit_contains() including both algorithms.\n\n* Reduce size of \"mixed-mode\" commit-graph to ensure we start commit walks\n 'above' the graph and then walk into the commits with generation numbers.\n\nThanks,\n-Stolee\n\nThis series is based on jt/commit-graph-per-object-store\n\nDerrick Stolee (18):\n commit-reach: move walk methods from commit.c\n commit.h: remove method declarations\n commit-reach: move ref_newer from remote.c\n commit-reach: move commit_contains from ref-filter\n upload-pack: make reachable() more generic\n upload-pack: refactor ok_to_give_up()\n upload-pack: generalize commit date cutoff\n commit-reach: move can_all_from_reach_with_flags\n test-reach: create new test tool for ref_newer\n test-reach: test in_merge_bases\n test-reach: test is_descendant_of\n test-reach: test get_merge_bases_many\n test-reach: test reduce_heads\n test-reach: test can_all_from_reach_with_flags\n test-reach: test commit_contains\n commit-reach: replace ref_newer logic\n commit-reach: make can_all_from_reach... linear\n commit-reach: use can_all_from_reach\n\n Makefile | 2 +\n bisect.c | 1 +\n builtin/branch.c | 1 +\n builtin/commit.c | 1 +\n builtin/fetch.c | 1 +\n builtin/fmt-merge-msg.c | 1 +\n builtin/log.c | 1 +\n builtin/merge-base.c | 1 +\n builtin/merge.c | 1 +\n builtin/pull.c | 1 +\n builtin/receive-pack.c | 1 +\n builtin/remote.c | 1 +\n builtin/rev-parse.c | 1 +\n commit-graph.c | 18 ++\n commit-graph.h | 6 +\n commit-reach.c | 662 ++++++++++++++++++++++++++++++++++++++++\n commit-reach.h | 77 +++++\n commit.c | 358 ----------------------\n commit.h | 29 --\n fast-import.c | 1 +\n http-push.c | 2 +-\n merge-recursive.c | 1 +\n notes-merge.c | 1 +\n object.h | 4 +-\n pack-bitmap-write.c | 1 +\n ref-filter.c | 146 +--------\n remote.c | 50 +--\n remote.h | 1 -\n revision.c | 1 +\n sequencer.c | 1 +\n sha1-name.c | 1 +\n shallow.c | 1 +\n submodule.c | 1 +\n t/helper/test-reach.c | 130 ++++++++\n t/helper/test-tool.c | 1 +\n t/helper/test-tool.h | 1 +\n t/t6600-test-reach.sh | 242 +++++++++++++++\n upload-pack.c | 58 +---\n 38 files changed, 1177 insertions(+), 631 deletions(-)\n create mode 100644 commit-reach.c\n create mode 100644 commit-reach.h\n create mode 100644 t/helper/test-reach.c\n create mode 100755 t/t6600-test-reach.sh\n\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353148","messageId":"20180720163227.105950-14-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"[PATCH v2 13/18] test-reach: test reduce_heads","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T16:33:22Z","receivedAt":"2018-07-20T16:34:17Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"Signed-off-by: Derrick Stolee \n---\n t/helper/test-reach.c | 4 ++++\n t/t6600-test-reach.sh | 22 ++++++++++++++++++++++\n 2 files changed, 26 insertions(+)\n\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex 4df01187c9..e32e193b70 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -102,6 +102,10 @@ int cmd__reach(int ac, const char **av)\n \t\tstruct commit_list *list = get_merge_bases_many(A, X_nr, X_array);\n \t\tprintf(\"%s(A,X):\\n\", av[1]);\n \t\tprint_sorted_commit_ids(list);\n+\t} else if (!strcmp(av[1], \"reduce_heads\")) {\n+\t\tstruct commit_list *list = reduce_heads(X);\n+\t\tprintf(\"%s(X):\\n\", av[1]);\n+\t\tprint_sorted_commit_ids(list);\n \t}\n \n \texit(0);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex d43e1a61d5..17c6467988 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -138,4 +138,26 @@ test_expect_success 'get_merge_bases_many' '\n \ttest_three_modes get_merge_bases_many\n '\n \n+test_expect_success 'reduce_heads' '\n+\tcat >input <<-\\EOF &&\n+\tX:commit-1-10\n+\tX:commit-2-8\n+\tX:commit-3-6\n+\tX:commit-4-4\n+\tX:commit-1-7\n+\tX:commit-2-5\n+\tX:commit-3-3\n+\tX:commit-5-1\n+\tEOF\n+\t{\n+\t\techo \"reduce_heads(X):\" &&\n+\t\tgit rev-parse commit-5-1 \\\n+\t\t\t commit-4-4 \\\n+\t\t\t commit-3-6 \\\n+\t\t\t commit-2-8 \\\n+\t\t\t commit-1-10 | sort\n+\t} >expect &&\n+\ttest_three_modes reduce_heads\n+'\n+\n test_done\n-- \n2.18.0.118.gd4f65b8d14\n\n"},{"id":"353152","messageId":"CAGZ79kY28vcs5TN9GtKd439vaTt38Qtwv8Lw0zanDaR1OmsTLg@mail.gmail.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-20T17:10:58Z","receivedAt":"2018-07-20T17:11:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Hi Derrick,\n\n> V2 Update: The biggest material change in this version is that we drop the\n> method declarations from commit.h, which requires adding a lot of references\n> to commit-reach.h across the codebase. This change is in a commit on its own.\n> In addition, we have the following smaller changes:\n\nIs there a remote available to get this series from?\n\n> * Use 'unsigned int' for the flag variables.\n>\n> * Properly align the here-doc test input data.\n>\n> * Use single rev-parse commands in test output, and pipe the OIDs through 'sort'\n>\n> * Check output of parse_commit()\n>\n> * Update flag documentation in object.h\n>\n> * Add tests for commit_contains() including both algorithms.\n>\n> * Reduce size of \"mixed-mode\" commit-graph to ensure we start commit walks\n> 'above' the graph and then walk into the commits with generation numbers.\n\nA range diff would be nice (though I can just look at all patches again\neven if it takes longer).\n\nI notice this is not sent via the GGG, but via git send-email?\n\nThanks,\nStefan\n"},{"id":"353153","messageId":"833dea7f-b6dd-b3dd-6282-c0a5f939ecca@gmail.com","threadId":"48889","inReplyTo":"CAGZ79kY28vcs5TN9GtKd439vaTt38Qtwv8Lw0zanDaR1OmsTLg@mail.gmail.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-07-20T17:15:38Z","receivedAt":"2018-07-20T17:15:45Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/20/2018 1:10 PM, Stefan Beller wrote:\n> Hi Derrick,\n>\n>> V2 Update: The biggest material change in this version is that we drop the\n>> method declarations from commit.h, which requires adding a lot of references\n>> to commit-reach.h across the codebase. This change is in a commit on its own.\n>> In addition, we have the following smaller changes:\n> Is there a remote available to get this series from?\n\nSure! It's on my fork [1]\n\n[1] https://github.com/derrickstolee/git/tree/reach/refactor\n\n\n>> * Use 'unsigned int' for the flag variables.\n>>\n>> * Properly align the here-doc test input data.\n>>\n>> * Use single rev-parse commands in test output, and pipe the OIDs through 'sort'\n>>\n>> * Check output of parse_commit()\n>>\n>> * Update flag documentation in object.h\n>>\n>> * Add tests for commit_contains() including both algorithms.\n>>\n>> * Reduce size of \"mixed-mode\" commit-graph to ensure we start commit walks\n>> 'above' the graph and then walk into the commits with generation numbers.\n> A range diff would be nice (though I can just look at all patches again\n> even if it takes longer).\n\nI can send a diff. It's a bit big because of the indenting changes.\n\n> I notice this is not sent via the GGG, but via git send-email?\n\nGGG can do version updates, but is currently having trouble when the \nbranch has conflicts with the target [2]. We will address this issue \nnext week, but I wanted to get this version out. Thank you for your \npatience as we work out the kinks.\n\n[2] https://github.com/gitgitgadget/gitgitgadget/issues/25\n\nThanks,\n\n-Stolee\n\n"},{"id":"353154","messageId":"20180720171836.106091-1-dstolee@microsoft.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Derrick Stolee","fromEmail":"dstolee@microsoft.com","sentAt":"2018-07-20T17:18:43Z","receivedAt":"2018-07-20T17:19:54Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"Here is the diff between v1 and v2.\n\nThanks,\n-Stolee\n\n---\n\ndiff --git a/bisect.c b/bisect.c\nindex e1275ba79e..d023543c91 100644\n--- a/bisect.c\n+++ b/bisect.c\n@@ -13,6 +13,7 @@\n #include \"sha1-array.h\"\n #include \"argv-array.h\"\n #include \"commit-slab.h\"\n+#include \"commit-reach.h\"\n \n static struct oid_array good_revs;\n static struct oid_array skipped_revs;\ndiff --git a/builtin/branch.c b/builtin/branch.c\nindex a50632fb23..9a787447f4 100644\n--- a/builtin/branch.c\n+++ b/builtin/branch.c\n@@ -23,6 +23,7 @@\n #include \"ref-filter.h\"\n #include \"worktree.h\"\n #include \"help.h\"\n+#include \"commit-reach.h\"\n \n static const char * const builtin_branch_usage[] = {\n \tN_(\"git branch [] [-r | -a] [--merged | --no-merged]\"),\ndiff --git a/builtin/commit.c b/builtin/commit.c\nindex 158e3f843a..b5c608458e 100644\n--- a/builtin/commit.c\n+++ b/builtin/commit.c\n@@ -33,6 +33,7 @@\n #include \"sequencer.h\"\n #include \"mailmap.h\"\n #include \"help.h\"\n+#include \"commit-reach.h\"\n \n static const char * const builtin_commit_usage[] = {\n \tN_(\"git commit [] [--] ...\"),\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex f5d960baec..7de234774b 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -22,6 +22,7 @@\n #include \"utf8.h\"\n #include \"packfile.h\"\n #include \"list-objects-filter-options.h\"\n+#include \"commit-reach.h\"\n \n static const char * const builtin_fetch_usage[] = {\n \tN_(\"git fetch [] [ [...]]\"),\ndiff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c\nindex ff165c0fcd..7277d557b2 100644\n--- a/builtin/fmt-merge-msg.c\n+++ b/builtin/fmt-merge-msg.c\n@@ -12,6 +12,7 @@\n #include \"fmt-merge-msg.h\"\n #include \"gpg-interface.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n static const char * const fmt_merge_msg_usage[] = {\n \tN_(\"git fmt-merge-msg [-m ] [--log[=] | --no-log] [--file ]\"),\ndiff --git a/builtin/log.c b/builtin/log.c\nindex 55a6286d7f..333d97c692 100644\n--- a/builtin/log.c\n+++ b/builtin/log.c\n@@ -31,6 +31,7 @@\n #include \"progress.h\"\n #include \"commit-slab.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n #define MAIL_DEFAULT_WRAP 72\n \ndiff --git a/builtin/merge-base.c b/builtin/merge-base.c\nindex 08d91b1f0c..1c92099070 100644\n--- a/builtin/merge-base.c\n+++ b/builtin/merge-base.c\n@@ -7,6 +7,7 @@\n #include \"revision.h\"\n #include \"parse-options.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n static int show_merge_base(struct commit **rev, int rev_nr, int show_all)\n {\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex d1b547d973..4c601c40a2 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -36,6 +36,7 @@\n #include \"packfile.h\"\n #include \"tag.h\"\n #include \"alias.h\"\n+#include \"commit-reach.h\"\n \n #define DEFAULT_TWOHEAD (1<<0)\n #define DEFAULT_OCTOPUS (1<<1)\ndiff --git a/builtin/pull.c b/builtin/pull.c\nindex 4e78935392..15ad010968 100644\n--- a/builtin/pull.c\n+++ b/builtin/pull.c\n@@ -22,6 +22,7 @@\n #include \"tempfile.h\"\n #include \"lockfile.h\"\n #include \"wt-status.h\"\n+#include \"commit-reach.h\"\n \n enum rebase_type {\n \tREBASE_INVALID = -1,\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex 400d31c18c..d8467f9734 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -27,6 +27,7 @@\n #include \"packfile.h\"\n #include \"object-store.h\"\n #include \"protocol.h\"\n+#include \"commit-reach.h\"\n \n static const char * const receive_pack_usage[] = {\n \tN_(\"git receive-pack \"),\ndiff --git a/builtin/rev-parse.c b/builtin/rev-parse.c\nindex 0f09bbbf65..455f62246d 100644\n--- a/builtin/rev-parse.c\n+++ b/builtin/rev-parse.c\n@@ -14,6 +14,7 @@\n #include \"revision.h\"\n #include \"split-index.h\"\n #include \"submodule.h\"\n+#include \"commit-reach.h\"\n \n #define DO_REVS\t\t1\n #define DO_NOREV\t2\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 9eb6225403..c996524032 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -4,6 +4,7 @@\n #include \"decorate.h\"\n #include \"prio-queue.h\"\n #include \"tree.h\"\n+#include \"ref-filter.c\"\n #include \"revision.h\"\n #include \"tag.h\"\n #include \"commit-reach.h\"\n@@ -536,7 +537,8 @@ static int compare_commits_by_gen(const void *_a, const void *_b)\n }\n \n int can_all_from_reach_with_flag(struct object_array *from,\n-\t\t\t\t int with_flag, int assign_flag,\n+\t\t\t\t unsigned int with_flag,\n+\t\t\t\t unsigned int assign_flag,\n \t\t\t\t time_t min_commit_date,\n \t\t\t\t uint32_t min_generation)\n {\n@@ -548,9 +550,8 @@ int can_all_from_reach_with_flag(struct object_array *from,\n \tfor (i = 0; i < from->nr; i++) {\n \t\tlist[i] = (struct commit *)from->objects[i].item;\n \n-\t\tparse_commit(list[i]);\n-\n-\t\tif (list[i]->generation < min_generation)\n+\t\tif (parse_commit(list[i]) ||\n+\t\t list[i]->generation < min_generation)\n \t\t\treturn 0;\n \t}\n \n@@ -578,9 +579,8 @@ int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\tif (!(parent->item->object.flags & assign_flag)) {\n \t\t\t\t\tparent->item->object.flags |= assign_flag;\n \n-\t\t\t\t\tparse_commit(parent->item);\n-\n-\t\t\t\t\tif (parent->item->date < min_commit_date ||\n+\t\t\t\t\tif (parse_commit(parent->item) ||\n+\t\t\t\t\t parent->item->date < min_commit_date ||\n \t\t\t\t\t parent->item->generation < min_generation)\n \t\t\t\t\t\tcontinue;\n \ndiff --git a/commit-reach.h b/commit-reach.h\nindex 482d9eb5dd..7d313e2975 100644\n--- a/commit-reach.h\n+++ b/commit-reach.h\n@@ -1,9 +1,12 @@\n #ifndef __COMMIT_REACH_H__\n #define __COMMIT_REACH_H__\n \n-#include \"commit.h\"\n #include \"commit-slab.h\"\n-#include \"ref-filter.h\"\n+\n+struct commit;\n+struct commit_list;\n+struct contains_cache;\n+struct ref_filter;\n \n struct commit_list *get_merge_bases_many(struct commit *one,\n \t\t\t\t\t int n,\n@@ -56,9 +59,6 @@ define_commit_slab(contains_cache, enum contains_result);\n int commit_contains(struct ref_filter *filter, struct commit *commit,\n \t\t struct commit_list *list, struct contains_cache *cache);\n \n-int reachable(struct commit *from, int with_flag, int assign_flag,\n-\t time_t min_commit_date);\n-\n /*\n * Determine if every commit in 'from' can reach at least one commit\n * that is marked with 'with_flag'. As we traverse, use 'assign_flag'\n@@ -67,7 +67,8 @@ int reachable(struct commit *from, int with_flag, int assign_flag,\n * 'min_generation'.\n */\n int can_all_from_reach_with_flag(struct object_array *from,\n-\t\t\t\t int with_flag, int assign_flag,\n+\t\t\t\t unsigned int with_flag,\n+\t\t\t\t unsigned int assign_flag,\n \t\t\t\t time_t min_commit_date,\n \t\t\t\t uint32_t min_generation);\n int can_all_from_reach(struct commit_list *from, struct commit_list *to,\ndiff --git a/commit.h b/commit.h\nindex da0db36eba..e2c99d9b04 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -204,13 +204,6 @@ struct commit_graft *read_graft_line(struct strbuf *line);\n int register_commit_graft(struct repository *r, struct commit_graft *, int);\n struct commit_graft *lookup_commit_graft(struct repository *r, const struct object_id *oid);\n \n-extern struct commit_list *get_merge_bases(struct commit *rev1, struct commit *rev2);\n-extern struct commit_list *get_merge_bases_many(struct commit *one, int n, struct commit **twos);\n-extern struct commit_list *get_octopus_merge_bases(struct commit_list *in);\n-\n-/* To be used only when object flags after this call no longer matter */\n-extern struct commit_list *get_merge_bases_many_dirty(struct commit *one, int n, struct commit **twos);\n-\n /* largest positive number a signed 32-bit integer can contain */\n #define INFINITE_DEPTH 0x7fffffff\n \n@@ -258,32 +251,10 @@ extern int delayed_reachability_test(struct shallow_info *si, int c);\n extern void prune_shallow(int show_only);\n extern struct trace_key trace_shallow;\n \n-int is_descendant_of(struct commit *, struct commit_list *);\n-int in_merge_bases(struct commit *, struct commit *);\n-int in_merge_bases_many(struct commit *, int, struct commit **);\n-\n extern int interactive_add(int argc, const char **argv, const char *prefix, int patch);\n extern int run_add_interactive(const char *revision, const char *patch_mode,\n \t\t\t const struct pathspec *pathspec);\n \n-/*\n- * Takes a list of commits and returns a new list where those\n- * have been removed that can be reached from other commits in\n- * the list. It is useful for, e.g., reducing the commits\n- * randomly thrown at the git-merge command and removing\n- * redundant commits that the user shouldn't have given to it.\n- *\n- * This function destroys the STALE bit of the commit objects'\n- * flags.\n- */\n-extern struct commit_list *reduce_heads(struct commit_list *heads);\n-\n-/*\n- * Like `reduce_heads()`, except it replaces the list. Use this\n- * instead of `foo = reduce_heads(foo);` to avoid memory leaks.\n- */\n-extern void reduce_heads_replace(struct commit_list **heads);\n-\n struct commit_extra_header {\n \tstruct commit_extra_header *next;\n \tchar *key;\ndiff --git a/contrib/coccinelle/commit.cocci b/contrib/coccinelle/commit.cocci\nindex a7e9215ffc..aec3345adb 100644\n--- a/contrib/coccinelle/commit.cocci\n+++ b/contrib/coccinelle/commit.cocci\n@@ -12,7 +12,7 @@ expression c;\n \n // These excluded functions must access c->maybe_tree direcly.\n @@\n-identifier f !~ \"^(get_commit_tree|get_commit_tree_in_graph|load_tree_for_commit)$\";\n+identifier f !~ \"^(get_commit_tree|get_commit_tree_in_graph_one|load_tree_for_commit)$\";\n expression c;\n @@\n f(...) {...\ndiff --git a/http-push.c b/http-push.c\nindex e007cb5a6b..91fdc7e1d5 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -16,7 +16,6 @@\n #include \"object-store.h\"\n #include \"commit-reach.h\"\n \n-\n #ifdef EXPAT_NEEDS_XMLPARSE_H\n #include \n #else\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 1dd6ec384d..8155dee9a9 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -27,6 +27,7 @@\n #include \"dir.h\"\n #include \"submodule.h\"\n #include \"revision.h\"\n+#include \"commit-reach.h\"\n \n struct path_hashmap_entry {\n \tstruct hashmap_entry e;\ndiff --git a/notes-merge.c b/notes-merge.c\nindex 76ab19e702..12dfdf6c17 100644\n--- a/notes-merge.c\n+++ b/notes-merge.c\n@@ -12,6 +12,7 @@\n #include \"notes-merge.h\"\n #include \"strbuf.h\"\n #include \"notes-utils.h\"\n+#include \"commit-reach.h\"\n \n struct notes_merge_pair {\n \tstruct object_id obj, base, local, remote;\ndiff --git a/object.h b/object.h\nindex fa5ca97567..b132944c51 100644\n--- a/object.h\n+++ b/object.h\n@@ -60,12 +60,12 @@ struct object_array {\n * revision.h: 0---------10 26\n * fetch-pack.c: 0----5\n * walker.c: 0-2\n- * upload-pack.c: 4 11----------------19\n+ * upload-pack.c: 4 11-----14 16-----19\n * builtin/blame.c: 12-13\n * bisect.c: 16\n * bundle.c: 16\n * http-push.c: 16-----19\n- * commit.c: 16-----19\n+ * commit-reach.c: 15-------19\n * sha1-name.c: 20\n * list-objects-filter.c: 21\n * builtin/fsck.c: 0--3\ndiff --git a/pack-bitmap-write.c b/pack-bitmap-write.c\nindex 953c5dd84d..55bcab907c 100644\n--- a/pack-bitmap-write.c\n+++ b/pack-bitmap-write.c\n@@ -11,6 +11,7 @@\n #include \"pack-bitmap.h\"\n #include \"sha1-lookup.h\"\n #include \"pack-objects.h\"\n+#include \"commit-reach.h\"\n \n struct bitmapped_commit {\n \tstruct commit *commit;\ndiff --git a/ref-filter.c b/ref-filter.c\nindex 35b2d25ce5..495e830fa5 100644\n--- a/ref-filter.c\n+++ b/ref-filter.c\n@@ -18,6 +18,7 @@\n #include \"trailer.h\"\n #include \"wt-status.h\"\n #include \"commit-slab.h\"\n+#include \"commit-graph.h\"\n #include \"commit-reach.h\"\n \n static struct ref_msg {\ndiff --git a/revision.c b/revision.c\nindex 4dbe406bed..3205a3947a 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -24,6 +24,7 @@\n #include \"packfile.h\"\n #include \"worktree.h\"\n #include \"argv-array.h\"\n+#include \"commit-reach.h\"\n \n volatile show_early_output_fn_t show_early_output;\n \ndiff --git a/sequencer.c b/sequencer.c\nindex d1d07bed5b..97bdfd48b4 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -30,6 +30,7 @@\n #include \"oidset.h\"\n #include \"commit-slab.h\"\n #include \"alias.h\"\n+#include \"commit-reach.h\"\n \n #define GIT_REFLOG_ACTION \"GIT_REFLOG_ACTION\"\n \ndiff --git a/sha1-name.c b/sha1-name.c\nindex 009faab4ae..7215b30b88 100644\n--- a/sha1-name.c\n+++ b/sha1-name.c\n@@ -12,6 +12,7 @@\n #include \"packfile.h\"\n #include \"object-store.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n \ndiff --git a/shallow.c b/shallow.c\nindex dbe8a2a290..99fd2d1ba0 100644\n--- a/shallow.c\n+++ b/shallow.c\n@@ -16,6 +16,7 @@\n #include \"list-objects.h\"\n #include \"commit-slab.h\"\n #include \"repository.h\"\n+#include \"commit-reach.h\"\n \n void set_alternate_shallow_file(struct repository *r, const char *path, int override)\n {\ndiff --git a/submodule.c b/submodule.c\nindex 6688dd5d45..6650ed7aa0 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -22,6 +22,7 @@\n #include \"worktree.h\"\n #include \"parse-options.h\"\n #include \"object-store.h\"\n+#include \"commit-reach.h\"\n \n static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;\n static struct string_list changed_submodule_names = STRING_LIST_INIT_DUP;\ndiff --git a/t/helper/test-reach.c b/t/helper/test-reach.c\nindex dc97100992..eb21103998 100644\n--- a/t/helper/test-reach.c\n+++ b/t/helper/test-reach.c\n@@ -1,10 +1,31 @@\n #include \"test-tool.h\"\n #include \"cache.h\"\n+#include \"commit.h\"\n #include \"commit-reach.h\"\n #include \"config.h\"\n #include \"parse-options.h\"\n+#include \"ref-filter.h\"\n+#include \"string-list.h\"\n #include \"tag.h\"\n \n+static void print_sorted_commit_ids(struct commit_list *list)\n+{\n+\tint i;\n+\tstruct string_list s = STRING_LIST_INIT_DUP;\n+\n+\twhile (list) {\n+\t\tstring_list_append(&s, oid_to_hex(&list->item->object.oid));\n+\t\tlist = list->next;\n+\t}\n+\n+\tstring_list_sort(&s);\n+\n+\tfor (i = 0; i < s.nr; i++)\n+\t\tprintf(\"%s\\n\", s.items[i].string);\n+\n+\tstring_list_clear(&s, 0);\n+}\n+\n int cmd__reach(int ac, const char **av)\n {\n \tstruct object_id oid_A, oid_B;\n@@ -77,7 +98,7 @@ int cmd__reach(int ac, const char **av)\n \tstrbuf_release(&buf);\n \n \tif (!strcmp(av[1], \"ref_newer\"))\n-\t\tprintf(\"%s:%d\\n\", av[1], ref_newer(&oid_A, &oid_B));\n+\t\tprintf(\"%s(A,B):%d\\n\", av[1], ref_newer(&oid_A, &oid_B));\n \telse if (!strcmp(av[1], \"in_merge_bases\"))\n \t\tprintf(\"%s(A,B):%d\\n\", av[1], in_merge_bases(A, B));\n \telse if (!strcmp(av[1], \"is_descendant_of\"))\n@@ -85,19 +106,24 @@ int cmd__reach(int ac, const char **av)\n \telse if (!strcmp(av[1], \"get_merge_bases_many\")) {\n \t\tstruct commit_list *list = get_merge_bases_many(A, X_nr, X_array);\n \t\tprintf(\"%s(A,X):\\n\", av[1]);\n-\t\twhile (list) {\n-\t\t\tprintf(\"%s\\n\", oid_to_hex(&list->item->object.oid));\n-\t\t\tlist = list->next;\n-\t\t}\n+\t\tprint_sorted_commit_ids(list);\n \t} else if (!strcmp(av[1], \"reduce_heads\")) {\n \t\tstruct commit_list *list = reduce_heads(X);\n \t\tprintf(\"%s(X):\\n\", av[1]);\n-\t\twhile (list) {\n-\t\t\tprintf(\"%s\\n\", oid_to_hex(&list->item->object.oid));\n-\t\t\tlist = list->next;\n-\t\t}\n+\t\tprint_sorted_commit_ids(list);\n \t} else if (!strcmp(av[1], \"can_all_from_reach\")) {\n \t\tprintf(\"%s(X,Y):%d\\n\", av[1], can_all_from_reach(X, Y, 1));\n+\t} else if (!strcmp(av[1], \"commit_contains\")) {\n+\t\tstruct ref_filter filter;\n+\t\tstruct contains_cache cache;\n+\t\tinit_contains_cache(&cache);\n+\n+\t\tif (ac > 2 && !strcmp(av[2], \"--tag\"))\n+\t\t\tfilter.with_commit_tag_algo = 1;\n+\t\telse\n+\t\t\tfilter.with_commit_tag_algo = 0;\n+\n+\t\tprintf(\"%s(_,A,X,_):%d\\n\", av[1], commit_contains(&filter, A, X, &cache));\n \t}\n \n \texit(0);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex cb07c64310..d139a00d1d 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -23,7 +23,7 @@ test_description='basic commit reachability tests'\n # \\ /\n # (1,1)\n #\n-# We use branch 'comit-x-y' to refer to (x,y).\n+# We use branch 'commit-x-y' to refer to (x,y).\n # This grid allows interesting reachability and\n # non-reachability queries: (x,y) can reach (x',y')\n # if and only if x' <= x and y' <= y.\n@@ -48,7 +48,7 @@ test_expect_success 'setup' '\n \tdone &&\n \tgit commit-graph write --reachable &&\n \tmv .git/objects/info/commit-graph commit-graph-full &&\n-\tgit show-ref -s commit-7-7 | git commit-graph write --stdin-commits &&\n+\tgit show-ref -s commit-5-5 | git commit-graph write --stdin-commits &&\n \tmv .git/objects/info/commit-graph commit-graph-half &&\n \tgit config core.commitGraph true\n '\n@@ -67,142 +67,176 @@ test_three_modes () {\n \n test_expect_success 'ref_newer:miss' '\n \tcat >input <<-\\EOF &&\n-\t\tA:commit-5-7\n-\t\tB:commit-4-9\n+\tA:commit-5-7\n+\tB:commit-4-9\n \tEOF\n-\tprintf \"ref_newer:0\\n\" >expect &&\n+\techo \"ref_newer(A,B):0\" >expect &&\n \ttest_three_modes ref_newer\n '\n \n test_expect_success 'ref_newer:hit' '\n \tcat >input <<-\\EOF &&\n-\t\tA:commit-5-7\n-\t\tB:commit-2-3\n+\tA:commit-5-7\n+\tB:commit-2-3\n \tEOF\n-\tprintf \"ref_newer:1\\n\" >expect &&\n+\techo \"ref_newer(A,B):1\" >expect &&\n \ttest_three_modes ref_newer\n '\n \n test_expect_success 'in_merge_bases:hit' '\n-\tcat >input <<- EOF &&\n-\t\tA:commit-5-7\n-\t\tB:commit-8-8\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tB:commit-8-8\n \tEOF\n-\tprintf \"in_merge_bases(A,B):1\\n\" >expect &&\n+\techo \"in_merge_bases(A,B):1\" >expect &&\n \ttest_three_modes in_merge_bases\n '\n \n test_expect_success 'in_merge_bases:miss' '\n-\tcat >input <<- EOF &&\n-\t\tA:commit-6-8\n-\t\tB:commit-5-9\n+\tcat >input <<-\\EOF &&\n+\tA:commit-6-8\n+\tB:commit-5-9\n \tEOF\n-\tprintf \"in_merge_bases(A,B):0\\n\" >expect &&\n+\techo \"in_merge_bases(A,B):0\" >expect &&\n \ttest_three_modes in_merge_bases\n '\n \n test_expect_success 'is_descendant_of:hit' '\n \tcat >input <<-\\EOF &&\n-\t\tA:commit-5-7\n-\t\tX:commit-4-8\n-\t\tX:commit-6-6\n-\t\tX:commit-1-1\n+\tA:commit-5-7\n+\tX:commit-4-8\n+\tX:commit-6-6\n+\tX:commit-1-1\n \tEOF\n-\tprintf \"is_descendant_of(A,X):1\\n\" >expect &&\n+\techo \"is_descendant_of(A,X):1\" >expect &&\n \ttest_three_modes is_descendant_of\n '\n \n test_expect_success 'is_descendant_of:miss' '\n \tcat >input <<-\\EOF &&\n-\t\tA:commit-6-8\n-\t\tX:commit-5-9\n-\t\tX:commit-4-10\n-\t\tX:commit-7-6\n+\tA:commit-6-8\n+\tX:commit-5-9\n+\tX:commit-4-10\n+\tX:commit-7-6\n \tEOF\n-\tprintf \"is_descendant_of(A,X):0\\n\" >expect &&\n+\techo \"is_descendant_of(A,X):0\" >expect &&\n \ttest_three_modes is_descendant_of\n '\n \n test_expect_success 'get_merge_bases_many' '\n \tcat >input <<-\\EOF &&\n-\t\tA:commit-5-7\n-\t\tX:commit-4-8\n-\t\tX:commit-6-6\n-\t\tX:commit-8-3\n+\tA:commit-5-7\n+\tX:commit-4-8\n+\tX:commit-6-6\n+\tX:commit-8-3\n \tEOF\n \t{\n-\t\tprintf \"get_merge_bases_many(A,X):\\n\" &&\n-\t\tgit rev-parse commit-5-6 &&\n-\t\tgit rev-parse commit-4-7\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse commit-5-6 \\\n+\t\t\t commit-4-7 | sort\n \t} >expect &&\n \ttest_three_modes get_merge_bases_many\n '\n \n test_expect_success 'reduce_heads' '\n \tcat >input <<-\\EOF &&\n-\t\tX:commit-1-10\n-\t\tX:commit-2-8\n-\t\tX:commit-3-6\n-\t\tX:commit-4-4\n-\t\tX:commit-1-7\n-\t\tX:commit-2-5\n-\t\tX:commit-3-3\n-\t\tX:commit-5-1\n+\tX:commit-1-10\n+\tX:commit-2-8\n+\tX:commit-3-6\n+\tX:commit-4-4\n+\tX:commit-1-7\n+\tX:commit-2-5\n+\tX:commit-3-3\n+\tX:commit-5-1\n \tEOF\n \t{\n-\t\tprintf \"reduce_heads(X):\\n\" &&\n-\t\tgit rev-parse commit-5-1 &&\n-\t\tgit rev-parse commit-4-4 &&\n-\t\tgit rev-parse commit-3-6 &&\n-\t\tgit rev-parse commit-2-8 &&\n-\t\tgit rev-parse commit-1-10\n+\t\techo \"reduce_heads(X):\" &&\n+\t\tgit rev-parse commit-5-1 \\\n+\t\t\t commit-4-4 \\\n+\t\t\t commit-3-6 \\\n+\t\t\t commit-2-8 \\\n+\t\t\t commit-1-10 | sort\n \t} >expect &&\n \ttest_three_modes reduce_heads\n '\n \n test_expect_success 'can_all_from_reach:hit' '\n \tcat >input <<-\\EOF &&\n-\t\tX:commit-2-10\n-\t\tX:commit-3-9\n-\t\tX:commit-4-8\n-\t\tX:commit-5-7\n-\t\tX:commit-6-6\n-\t\tX:commit-7-5\n-\t\tX:commit-8-4\n-\t\tX:commit-9-3\n-\t\tY:commit-1-9\n-\t\tY:commit-2-8\n-\t\tY:commit-3-7\n-\t\tY:commit-4-6\n-\t\tY:commit-5-5\n-\t\tY:commit-6-4\n-\t\tY:commit-7-3\n-\t\tY:commit-8-1\n+\tX:commit-2-10\n+\tX:commit-3-9\n+\tX:commit-4-8\n+\tX:commit-5-7\n+\tX:commit-6-6\n+\tX:commit-7-5\n+\tX:commit-8-4\n+\tX:commit-9-3\n+\tY:commit-1-9\n+\tY:commit-2-8\n+\tY:commit-3-7\n+\tY:commit-4-6\n+\tY:commit-5-5\n+\tY:commit-6-4\n+\tY:commit-7-3\n+\tY:commit-8-1\n \tEOF\n-\tprintf \"can_all_from_reach(X,Y):1\\n\" >expect &&\n+\techo \"can_all_from_reach(X,Y):1\" >expect &&\n \ttest_three_modes can_all_from_reach\n '\n \n test_expect_success 'can_all_from_reach:miss' '\n \tcat >input <<-\\EOF &&\n-\t\tX:commit-2-10\n-\t\tX:commit-3-9\n-\t\tX:commit-4-8\n-\t\tX:commit-5-7\n-\t\tX:commit-6-6\n-\t\tX:commit-7-5\n-\t\tX:commit-8-4\n-\t\tX:commit-9-3\n-\t\tY:commit-1-9\n-\t\tY:commit-2-8\n-\t\tY:commit-3-7\n-\t\tY:commit-4-6\n-\t\tY:commit-5-5\n-\t\tY:commit-6-4\n-\t\tY:commit-8-5\n+\tX:commit-2-10\n+\tX:commit-3-9\n+\tX:commit-4-8\n+\tX:commit-5-7\n+\tX:commit-6-6\n+\tX:commit-7-5\n+\tX:commit-8-4\n+\tX:commit-9-3\n+\tY:commit-1-9\n+\tY:commit-2-8\n+\tY:commit-3-7\n+\tY:commit-4-6\n+\tY:commit-5-5\n+\tY:commit-6-4\n+\tY:commit-8-5\n \tEOF\n-\tprintf \"can_all_from_reach(X,Y):0\\n\" >expect &&\n+\techo \"can_all_from_reach(X,Y):0\" >expect &&\n \ttest_three_modes can_all_from_reach\n '\n \n+test_expect_success 'commit_contains:hit' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-7-7\n+\tX:commit-2-10\n+\tX:commit-3-9\n+\tX:commit-4-8\n+\tX:commit-5-7\n+\tX:commit-6-6\n+\tX:commit-7-5\n+\tX:commit-8-4\n+\tX:commit-9-3\n+\tEOF\n+\techo \"commit_contains(_,A,X,_):1\" >expect &&\n+\ttest_three_modes commit_contains &&\n+\ttest_three_modes commit_contains --tag\n+'\n+\n+test_expect_success 'commit_contains:miss' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-6-5\n+\tX:commit-2-10\n+\tX:commit-3-9\n+\tX:commit-4-8\n+\tX:commit-5-7\n+\tX:commit-6-6\n+\tX:commit-7-5\n+\tX:commit-8-4\n+\tX:commit-9-3\n+\tEOF\n+\techo \"commit_contains(_,A,X,_):0\" >expect &&\n+\ttest_three_modes commit_contains &&\n+\ttest_three_modes commit_contains --tag\n+'\n+\n test_done\n"},{"id":"353157","messageId":"CACsJy8CTPtxLsSoNO8xynHGoPqQ3k6N4mSykWTQ9Ceap3yV1hw@mail.gmail.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-07-20T17:41:03Z","receivedAt":"2018-07-20T17:41:32Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Jul 20, 2018 at 6:35 PM Derrick Stolee wrote:\n>\n> There are many places in Git that use a commit walk to determine\n> reachability between commits and/or refs. A lot of this logic is\n> duplicated.\n>\n> I wanted to achieve the following:\n>\n> Consolidate several different commit walks into one file\n\nI'm surprised get_shallow_commits() in shallow.c didn't make the cut.\nIt's no problem though if you already considered it and decided it was\nbetter left alone.\n\n> Reduce duplicate reachability logic\n> Increase testability (correctness and performance)\n> Improve performance of reachability queries\n\nWhat's your recommendation on adding new commit reachability code? I\nmight have to add one to fix prune_shallow() if I don't find anything\nfit. I guess the code should go to commit-reach.c too?\n-- \nDuy\n"},{"id":"353165","messageId":"CAPig+cRRa9gcvfu9jKy-+DK+Jjj6maCabuOyprVQMeGx4uJrRg@mail.gmail.com","threadId":"48889","inReplyTo":"20180720171836.106091-1-dstolee@microsoft.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-07-20T18:09:27Z","receivedAt":"2018-07-20T18:09:41Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Jul 20, 2018 at 1:20 PM Derrick Stolee wrote:\n> Here is the diff between v1 and v2.\n>\n> diff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\n> @@ -67,142 +67,176 @@ test_three_modes () {\n> test_expect_success 'get_merge_bases_many' '\n> cat >input <<-\\EOF &&\n> + A:commit-5-7\n> + X:commit-4-8\n> + X:commit-6-6\n> + X:commit-8-3\n> EOF\n> {\n> - printf \"get_merge_bases_many(A,X):\\n\" &&\n> - git rev-parse commit-5-6 &&\n> - git rev-parse commit-4-7\n> + echo \"get_merge_bases_many(A,X):\" &&\n> + git rev-parse commit-5-6 \\\n> + commit-4-7 | sort\n\nPipes lose the exit code of the all upstream commands. When a Git\ncommand is upstream, we'd usually recommend to dump its output to a\nfile, then use the file as input to the rest of the pipe so as not to\nlose the Git command's exit code:\n\n {\n ...\n git rev-parse ... >oids &&\n sort expect &&\n\nOne could argue, in this case, that if git-rev-parse crashes, then it\nwon't have the expected output and the test will fail anyhow despite\nnot seeing its failed exit code. However, git-rev-parse might crash\n_after_ emitting all the normal, expected output, and that crash would\nbe missed altogether, so avoiding git-rev-parse as a pipe upstream is\na good idea.\n\nHowever, one could argue that argument by saying that it isn't the job\nof this particular test script to check git-rev-parse's behavior, so\ncrashy git-rev-parse ought to be caught elsewhere by some other test\nscript. Nevertheless, you'll likely encounter reviewers who don't want\nto see git-rev-parse upstream, even with that argument.\n\nAnyhow, why is that 'sort' even there? It wasn't needed in the\noriginal. Is git-rev-parse outputting the OID's in random order?\n\n> } >expect &&\n> test_three_modes get_merge_bases_many\n> '\n>\n> test_expect_success 'reduce_heads' '\n> [...]\n> + git rev-parse commit-5-1 \\\n> + commit-4-4 \\\n> + commit-3-6 \\\n> + commit-2-8 \\\n> + commit-1-10 | sort\n\nDitto.\n\n> } >expect &&\n> test_three_modes reduce_heads\n> '\n"},{"id":"353172","messageId":"215c6ad7-a403-9173-5965-70f68e796d7a@gmail.com","threadId":"48889","inReplyTo":"CACsJy8CTPtxLsSoNO8xynHGoPqQ3k6N4mSykWTQ9Ceap3yV1hw@mail.gmail.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-07-20T19:09:15Z","receivedAt":"2018-07-20T19:09:19Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/20/2018 1:41 PM, Duy Nguyen wrote:\n> On Fri, Jul 20, 2018 at 6:35 PM Derrick Stolee wrote:\n>> There are many places in Git that use a commit walk to determine\n>> reachability between commits and/or refs. A lot of this logic is\n>> duplicated.\n>>\n>> I wanted to achieve the following:\n>>\n>> Consolidate several different commit walks into one file\n> I'm surprised get_shallow_commits() in shallow.c didn't make the cut.\n> It's no problem though if you already considered it and decided it was\n> better left alone.\n\nThanks for pointing this out. I didn't know about it. It would make an \nexcellent follow-up series.\n\n>> Reduce duplicate reachability logic\n>> Increase testability (correctness and performance)\n>> Improve performance of reachability queries\n> What's your recommendation on adding new commit reachability code? I\n> might have to add one to fix prune_shallow() if I don't find anything\n> fit. I guess the code should go to commit-reach.c too?\n\nIn my opinion, new commit walks should go into commit-reach.c. Then, you \ncan justify why you are using a \"new\" walk instead of using an existing \nwalk. Further, you can probably think of the walk in more generic terms \nthan the specific application you need. Finally, you can use the \n'test-tool reach ' pattern to test the specific walk you create \noutside of the logic for which you needed it.\n\nI understand that while this patch is under review, you will probably \nwant to continue adding your walk where it is, then we can consolidate \nthe code after both have settled.\n\nThanks,\n\n-Stolee\n\n"},{"id":"353173","messageId":"0676494e-8f2d-77ed-acbf-3e082e9fddcb@gmail.com","threadId":"48889","inReplyTo":"CAPig+cRRa9gcvfu9jKy-+DK+Jjj6maCabuOyprVQMeGx4uJrRg@mail.gmail.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-07-20T19:14:03Z","receivedAt":"2018-07-20T19:14:07Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/20/2018 2:09 PM, Eric Sunshine wrote:\n> On Fri, Jul 20, 2018 at 1:20 PM Derrick Stolee wrote:\n>> Here is the diff between v1 and v2.\n>>\n>> diff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\n>> @@ -67,142 +67,176 @@ test_three_modes () {\n>> test_expect_success 'get_merge_bases_many' '\n>> cat >input <<-\\EOF &&\n>> + A:commit-5-7\n>> + X:commit-4-8\n>> + X:commit-6-6\n>> + X:commit-8-3\n>> EOF\n>> {\n>> - printf \"get_merge_bases_many(A,X):\\n\" &&\n>> - git rev-parse commit-5-6 &&\n>> - git rev-parse commit-4-7\n>> + echo \"get_merge_bases_many(A,X):\" &&\n>> + git rev-parse commit-5-6 \\\n>> + commit-4-7 | sort\n> Pipes lose the exit code of the all upstream commands. When a Git\n> command is upstream, we'd usually recommend to dump its output to a\n> file, then use the file as input to the rest of the pipe so as not to\n> lose the Git command's exit code:\n>\n> {\n> ...\n> git rev-parse ... >oids &&\n> sort } >expect &&\n\nThis approach seems fine to me. I'd hate to be in the case where \nrev-parse reports an error, terminating early, resulting in an incorrect \nexpected file, and then having the test pass because the code is \nsimilarly incorrect. No matter how slim the chances are, I want to avoid \na false positive there.\n\n> One could argue, in this case, that if git-rev-parse crashes, then it\n> won't have the expected output and the test will fail anyhow despite\n> not seeing its failed exit code. However, git-rev-parse might crash\n> _after_ emitting all the normal, expected output, and that crash would\n> be missed altogether, so avoiding git-rev-parse as a pipe upstream is\n> a good idea.\n>\n> However, one could argue that argument by saying that it isn't the job\n> of this particular test script to check git-rev-parse's behavior, so\n> crashy git-rev-parse ought to be caught elsewhere by some other test\n> script. Nevertheless, you'll likely encounter reviewers who don't want\n> to see git-rev-parse upstream, even with that argument.\n>\n> Anyhow, why is that 'sort' even there? It wasn't needed in the\n> original. Is git-rev-parse outputting the OID's in random order?\n\nSince the merge-base algorithms provide the commits in an order that \ndepends on the implementation (not the functional contract), we decided \nto sort the output commit ids in the output of 'test-tool reach \n'. Thus, we sort the rev-parse output to match.\n\nThanks,\n\n-Stolee\n\n"},{"id":"353201","messageId":"CAGZ79kb7tWV=cmboA+nsChAUaiC+fVVM-GBCuWfsypC+-wyaVg@mail.gmail.com","threadId":"48889","inReplyTo":"833dea7f-b6dd-b3dd-6282-c0a5f939ecca@gmail.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-07-20T22:16:10Z","receivedAt":"2018-07-20T22:16:24Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Hi Derrick,\n\n> Sure! It's on my fork [1]\n>\n> [1] https://github.com/derrickstolee/git/tree/reach/refactor\n>\n\nThanks!\n\n> >> * Use single rev-parse commands in test output, and pipe the OIDs through 'sort'\n\nWhy do we need to sort them? The order of the answers given by rev-parse\nis the same as the input given and we did not need to sort it before, i.e.\nthe unit under test would not give sorted output but some deterministic(?)\norder, which we can replicate as input to rev-parse.\nAm I missing the obvious?\n\n> >> * Check output of parse_commit()\n> >>\n> >> * Update flag documentation in object.h\n> >>\n> >> * Add tests for commit_contains() including both algorithms.\n> >>\n> >> * Reduce size of \"mixed-mode\" commit-graph to ensure we start commit walks\n> >> 'above' the graph and then walk into the commits with generation numbers.\n\nOverall I like the series as-is, and have found\nno further issues in a quick read.\n\nThanks,\nStefan\n"},{"id":"353205","messageId":"xmqqsh4d8r21.fsf@gitster-ct.c.googlers.com","threadId":"48889","inReplyTo":"20180720163227.105950-1-dstolee@microsoft.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-07-20T22:45:26Z","receivedAt":"2018-07-20T22:45:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> There are many places in Git that use a commit walk to determine\n> reachability between commits and/or refs. A lot of this logic is\n> duplicated.\n>\n> I wanted to achieve the following:\n>\n> Consolidate several different commit walks into one file\n> Reduce duplicate reachability logic\n> Increase testability (correctness and performance)\n> Improve performance of reachability queries\n\nAll of these are good goals to shoot at.\n\n> This series is based on jt/commit-graph-per-object-store\n\nAs such, it has some interactions with another topic [*1*] that is\nbased on the same, but my trial merge seems to suggest that the\ninteractions are minimum and do not pose a serious problem.\n\nWill push out as part of the next integration run, as I've already\npushed out today's.\n\nThanks.\n\n\n[Footnote]\n\n*1* ds/commit-graph-with-grafts topic that is in 'pu', slated to go\n 'next' soonish.\n"},{"id":"353413","messageId":"20180723203500.231932-1-jonathantanmy@google.com","threadId":"48889","inReplyTo":"20180720163227.105950-16-dstolee@microsoft.com","subject":"Re: [PATCH v2 15/18] test-reach: test commit_contains","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-07-23T20:35:00Z","receivedAt":"2018-07-23T20:35:07Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> +\t} else if (!strcmp(av[1], \"commit_contains\")) {\n> +\t\tstruct ref_filter filter;\n> +\t\tstruct contains_cache cache;\n> +\t\tinit_contains_cache(&cache);\n> +\n> +\t\tif (ac > 2 && !strcmp(av[2], \"--tag\"))\n> +\t\t\tfilter.with_commit_tag_algo = 1;\n> +\t\telse\n> +\t\t\tfilter.with_commit_tag_algo = 0;\n> +\n> +\t\tprintf(\"%s(_,A,X,_):%d\\n\", av[1], commit_contains(&filter, A, X, &cache));\n\nShould we initialize filter (with {NULL} or some equivalent)?\n"},{"id":"353414","messageId":"20180723204112.233274-1-jonathantanmy@google.com","threadId":"48889","inReplyTo":"20180720163227.105950-18-dstolee@microsoft.com","subject":"Re: [PATCH v2 17/18] commit-reach: make can_all_from_reach... linear","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-07-23T20:41:12Z","receivedAt":"2018-07-23T20:41:17Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"> +\t\tif (parse_commit(list[i]) ||\n> +\t\t list[i]->generation < min_generation)\n\nHere...\n\n> +\t\t\t\t\tif (parse_commit(parent->item) ||\n> +\t\t\t\t\t parent->item->date < min_commit_date ||\n> +\t\t\t\t\t parent->item->generation < min_generation)\n\n...and here, would parse_commit_or_die() be better? I think that a\nfunction that returns a definitive answer (either the commits are\nreachable or not) should die when the commits cannot be parsed.\n\nOther than that, I've compared the commits in this version to v1, and\nall my review comments have been addressed, thanks. (With the exception\nof the skip_prefix() one, but that is a minor matter - I suggested that\nto make it easier to implement my \"Ancestor:\" and \"Descendant:\"\nsuggestion which Stolee disagreed on with reason.)\n\n[1] https://public-inbox.org/git/20180716230019.257742-1-jonathantanmy@google.com/\n"},{"id":"353579","messageId":"xmqqr2jrur1m.fsf@gitster-ct.c.googlers.com","threadId":"48889","inReplyTo":"20180723203500.231932-1-jonathantanmy@google.com","subject":"Re: [PATCH v2 15/18] test-reach: test commit_contains","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-07-25T18:08:21Z","receivedAt":"2018-07-25T18:08:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n>> +\t} else if (!strcmp(av[1], \"commit_contains\")) {\n>> +\t\tstruct ref_filter filter;\n>> +\t\tstruct contains_cache cache;\n>> +\t\tinit_contains_cache(&cache);\n>> +\n>> +\t\tif (ac > 2 && !strcmp(av[2], \"--tag\"))\n>> +\t\t\tfilter.with_commit_tag_algo = 1;\n>> +\t\telse\n>> +\t\t\tfilter.with_commit_tag_algo = 0;\n>> +\n>> +\t\tprintf(\"%s(_,A,X,_):%d\\n\", av[1], commit_contains(&filter, A, X, &cache));\n>\n> Should we initialize filter (with {NULL} or some equivalent)?\n\nSounds like a sensible suggestion. Wouldn't we segfault otherwise\ndepending on what garbage bytes are on the stack?\n"},{"id":"353581","messageId":"0c8b2860-3de7-e7fd-da07-9f2ae27e2e8f@gmail.com","threadId":"48889","inReplyTo":"xmqqr2jrur1m.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 15/18] test-reach: test commit_contains","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-07-25T18:30:12Z","receivedAt":"2018-07-25T18:30:16Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/25/2018 2:08 PM, Junio C Hamano wrote:\n> Jonathan Tan writes:\n>\n>>> +\t} else if (!strcmp(av[1], \"commit_contains\")) {\n>>> +\t\tstruct ref_filter filter;\n>>> +\t\tstruct contains_cache cache;\n>>> +\t\tinit_contains_cache(&cache);\n>>> +\n>>> +\t\tif (ac > 2 && !strcmp(av[2], \"--tag\"))\n>>> +\t\t\tfilter.with_commit_tag_algo = 1;\n>>> +\t\telse\n>>> +\t\t\tfilter.with_commit_tag_algo = 0;\n>>> +\n>>> +\t\tprintf(\"%s(_,A,X,_):%d\\n\", av[1], commit_contains(&filter, A, X, &cache));\n>> Should we initialize filter (with {NULL} or some equivalent)?\n> Sounds like a sensible suggestion. Wouldn't we segfault otherwise\n> depending on what garbage bytes are on the stack?\n\nIt's a good idea to initialize the struct properly, but the only part of \nthe 'filter' struct that is accessed by that method is the \n'with_commit_tag_algo' member. Everything else is read from A, X, and cache.\n\nThanks,\n-Stolee\n"},{"id":"354189","messageId":"2c5febdb-f6f6-cb92-d959-84505261920d@gmail.com","threadId":"48889","inReplyTo":"CAGZ79kb7tWV=cmboA+nsChAUaiC+fVVM-GBCuWfsypC+-wyaVg@mail.gmail.com","subject":"Re: [PATCH v2 00/18] Consolidate reachability logic","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-08-01T20:33:24Z","receivedAt":"2018-08-01T20:33:31Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/20/2018 6:16 PM, Stefan Beller wrote:\n>>>> * Use single rev-parse commands in test output, and pipe the OIDs through 'sort'\n> Why do we need to sort them? The order of the answers given by rev-parse\n> is the same as the input given and we did not need to sort it before, i.e.\n> the unit under test would not give sorted output but some deterministic(?)\n> order, which we can replicate as input to rev-parse.\n> Am I missing the obvious?\nThe output of the test program is not always deterministic (or at least, \nthe order is determined by the implementation, but not as part of the \nmethod contract). For example: get_all_merge_bases can return the list \nof merge bases in any order.\n\nBy sorting, we can ensure the output values (and their multiplicity) \nmatch expected.\n\nThanks,\n-Stolee\n"},{"id":"354190","messageId":"699cb8a9-dafc-8450-b992-0fb5c8afe1c2@gmail.com","threadId":"48889","inReplyTo":"20180723204112.233274-1-jonathantanmy@google.com","subject":"Re: [PATCH v2 17/18] commit-reach: make can_all_from_reach... linear","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-08-01T20:41:10Z","receivedAt":"2018-08-01T20:41:16Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"\n\nOn 7/23/2018 4:41 PM, Jonathan Tan wrote:\n>> +\t\tif (parse_commit(list[i]) ||\n>> +\t\t list[i]->generation < min_generation)\n> Here...\n>\n>> +\t\t\t\t\tif (parse_commit(parent->item) ||\n>> +\t\t\t\t\t parent->item->date < min_commit_date ||\n>> +\t\t\t\t\t parent->item->generation < min_generation)\n> ...and here, would parse_commit_or_die() be better? I think that a\n> function that returns a definitive answer (either the commits are\n> reachable or not) should die when the commits cannot be parsed.\nI'm hesitant to add _or_die() here, when the previous implementation \nonly used parse_object() or parse_commit(), so would not die when \nparsing fails. The same holds true for the other methods that call \ncan_all_from_reach().\n\nThanks,\n-Stolee\n"},{"id":"356762","messageId":"20180828212457.GA74687@aiede.svl.corp.google.com","threadId":"48889","inReplyTo":"20180720163227.105950-5-dstolee@microsoft.com","subject":"Re: [PATCH v2 04/18] commit-reach: move commit_contains from ref-filter","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-08-28T21:24:57Z","receivedAt":"2018-08-28T21:25:01Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nDerrick Stolee wrote:\n\n> There are several commit walks in the codebase. Group them together into\n> a new commit-reach.c file and corresponding header. After we group these\n> walks into one place, we can reduce duplicate logic by calling\n> equivalent methods.\n>\n> All methods are direct moves, except we also make the commit_contains()\n> method public so its consumers in ref-filter.c can still call it. We can\n> also test this method in a test-tool in a later commit.\n>\n> Signed-off-by: Derrick Stolee \n> ---\n> commit-reach.c | 121 +++++++++++++++++++++++++++++++++++++++++\n> commit-reach.h | 20 ++++++-\n> ref-filter.c | 145 +++----------------------------------------------\n> 3 files changed, 147 insertions(+), 139 deletions(-)\n> \n> diff --git a/commit-reach.c b/commit-reach.c\n> index a6bc4781a6..01d796f011 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -1,8 +1,10 @@\n> #include \"cache.h\"\n> #include \"commit.h\"\n> +#include \"commit-graph.h\"\n> #include \"decorate.h\"\n> #include \"prio-queue.h\"\n> #include \"tree.h\"\n> +#include \"ref-filter.c\"\n\nDid you mean \"ref-filter.h\"?\n\nThis broke the build here. Is there some check that we can use to\nprevent it happening again? I don't think we ever intentionally\n#include a .c file.\n\nThanks,\nJonathan\n"},{"id":"356768","messageId":"e71b2540-430d-d3e8-0c29-f290a12de4b0@gmail.com","threadId":"48889","inReplyTo":"20180828212457.GA74687@aiede.svl.corp.google.com","subject":"Re: [PATCH v2 04/18] commit-reach: move commit_contains from ref-filter","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-08-28T21:33:37Z","receivedAt":"2018-08-28T21:33:44Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/28/2018 5:24 PM, Jonathan Nieder wrote:\n> Hi,\n>\n> Derrick Stolee wrote:\n>\n>> There are several commit walks in the codebase. Group them together into\n>> a new commit-reach.c file and corresponding header. After we group these\n>> walks into one place, we can reduce duplicate logic by calling\n>> equivalent methods.\n>>\n>> All methods are direct moves, except we also make the commit_contains()\n>> method public so its consumers in ref-filter.c can still call it. We can\n>> also test this method in a test-tool in a later commit.\n>>\n>> Signed-off-by: Derrick Stolee \n>> ---\n>> commit-reach.c | 121 +++++++++++++++++++++++++++++++++++++++++\n>> commit-reach.h | 20 ++++++-\n>> ref-filter.c | 145 +++----------------------------------------------\n>> 3 files changed, 147 insertions(+), 139 deletions(-)\n>>\n>> diff --git a/commit-reach.c b/commit-reach.c\n>> index a6bc4781a6..01d796f011 100644\n>> --- a/commit-reach.c\n>> +++ b/commit-reach.c\n>> @@ -1,8 +1,10 @@\n>> #include \"cache.h\"\n>> #include \"commit.h\"\n>> +#include \"commit-graph.h\"\n>> #include \"decorate.h\"\n>> #include \"prio-queue.h\"\n>> #include \"tree.h\"\n>> +#include \"ref-filter.c\"\n> Did you mean \"ref-filter.h\"?\n>\n> This broke the build here. Is there some check that we can use to\n> prevent it happening again? I don't think we ever intentionally\n> #include a .c file.\nWoah! How did that ever work? I definitely built this locally.\n"},{"id":"356772","messageId":"20180828213657.GA74296@aiede.svl.corp.google.com","threadId":"48889","inReplyTo":"20180828212457.GA74687@aiede.svl.corp.google.com","subject":"[PATCH] commit-reach: correct accidental #include of C file","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-08-28T21:36:57Z","receivedAt":"2018-08-28T21:37:02Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Without this change, the build breaks with clang:\n\n libgit/ref-filter.pic.o: multiple definition of 'filter_refs'\n libgit/commit-reach.pic.o: previous definition here\n\nSigned-off-by: Jonathan Nieder \n---\nJonathan Nieder wrote:\n> Derrick Stolee wrote:\n\n>> --- a/commit-reach.c\n>> +++ b/commit-reach.c\n>> @@ -1,8 +1,10 @@\n>> #include \"cache.h\"\n>> #include \"commit.h\"\n>> +#include \"commit-graph.h\"\n>> #include \"decorate.h\"\n>> #include \"prio-queue.h\"\n>> #include \"tree.h\"\n>> +#include \"ref-filter.c\"\n>\n> Did you mean \"ref-filter.h\"?\n>\n> This broke the build here. Is there some check that we can use to\n> prevent it happening again? I don't think we ever intentionally\n> #include a .c file.\n\nHere's what I'm applying locally.\n\nThanks,\nJonathan\n\n commit-reach.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex c996524032..86715c103c 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -4,7 +4,7 @@\n #include \"decorate.h\"\n #include \"prio-queue.h\"\n #include \"tree.h\"\n-#include \"ref-filter.c\"\n+#include \"ref-filter.h\"\n #include \"revision.h\"\n #include \"tag.h\"\n #include \"commit-reach.h\"\n-- \n2.19.0.rc0.228.g281dcd1b4d0\n\n"},{"id":"356773","messageId":"6ae7f8fd-9be5-992e-e5a4-35254c263280@gmail.com","threadId":"48889","inReplyTo":"20180828213657.GA74296@aiede.svl.corp.google.com","subject":"Re: [PATCH] commit-reach: correct accidental #include of C file","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-08-28T21:39:32Z","receivedAt":"2018-08-28T21:39:38Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/28/2018 5:36 PM, Jonathan Nieder wrote:\n> Without this change, the build breaks with clang:\nFor some reason, it didn't fail with GCC for me, but this is an \nobviously correct change to make. Thanks!\n> libgit/ref-filter.pic.o: multiple definition of 'filter_refs'\n> libgit/commit-reach.pic.o: previous definition here\n>\n> Signed-off-by: Jonathan Nieder \n> ---\n> Jonathan Nieder wrote:\n>> Derrick Stolee wrote:\n>>> --- a/commit-reach.c\n>>> +++ b/commit-reach.c\n>>> @@ -1,8 +1,10 @@\n>>> #include \"cache.h\"\n>>> #include \"commit.h\"\n>>> +#include \"commit-graph.h\"\n>>> #include \"decorate.h\"\n>>> #include \"prio-queue.h\"\n>>> #include \"tree.h\"\n>>> +#include \"ref-filter.c\"\n>> Did you mean \"ref-filter.h\"?\n>>\n>> This broke the build here. Is there some check that we can use to\n>> prevent it happening again? I don't think we ever intentionally\n>> #include a .c file.\n> Here's what I'm applying locally.\n>\n> Thanks,\n> Jonathan\n>\n> commit-reach.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/commit-reach.c b/commit-reach.c\n> index c996524032..86715c103c 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -4,7 +4,7 @@\n> #include \"decorate.h\"\n> #include \"prio-queue.h\"\n> #include \"tree.h\"\n> -#include \"ref-filter.c\"\n> +#include \"ref-filter.h\"\n> #include \"revision.h\"\n> #include \"tag.h\"\n> #include \"commit-reach.h\"\n"},{"id":"357944","messageId":"20180912041425.GA10472@sigill.intra.peff.net","threadId":"48889","inReplyTo":"20180720163227.105950-18-dstolee@microsoft.com","subject":"Re: [PATCH v2 17/18] commit-reach: make can_all_from_reach... linear","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-12T04:14:25Z","receivedAt":"2018-09-12T04:14:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Jul 20, 2018 at 04:33:28PM +0000, Derrick Stolee wrote:\n\n> The can_all_from_reach_with_flags() algorithm is currently quadratic in\n> the worst case, because it calls the reachable() method for every 'from'\n> without tracking which commits have already been walked or which can\n> already reach a commit in 'to'.\n> \n> Rewrite the algorithm to walk each commit a constant number of times.\n\nI got a segfault in upload-pack from 'next' today which bisected to this\npatch (which became 4fbcca4eff). I think the problem is the line at the\nbottom of this hunk:\n\n> int can_all_from_reach_with_flag(struct object_array *from,\n> \t\t\t\t unsigned int with_flag,\n> \t\t\t\t unsigned int assign_flag,\n> -\t\t\t\t time_t min_commit_date)\n> +\t\t\t\t time_t min_commit_date,\n> +\t\t\t\t uint32_t min_generation)\n> {\n> +\tstruct commit **list = NULL;\n> \tint i;\n> +\tint result = 1;\n> \n> +\tALLOC_ARRAY(list, from->nr);\n> \tfor (i = 0; i < from->nr; i++) {\n> -\t\tstruct object *from_one = from->objects[i].item;\n> +\t\tlist[i] = (struct commit *)from->objects[i].item;\n\nSome of the objects in my array are not commits, but rather tags, so\nthis is a bogus cast.\n\nYou can see that the original code peeled them and threw away\nnon-commits:\n\n> \n> -\t\tif (from_one->flags & assign_flag)\n> -\t\t\tcontinue;\n> -\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n> -\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n> -\t\t\t/* no way to tell if this is reachable by\n> -\t\t\t * looking at the ancestry chain alone, so\n> -\t\t\t * leave a note to ourselves not to worry about\n> -\t\t\t * this object anymore.\n> -\t\t\t */\n> -\t\t\tfrom->objects[i].item->flags |= assign_flag;\n> -\t\t\tcontinue;\n> -\t\t}\n\nSo presumably we'd need to do something similar.\n\nI think when we're called from can_all_from_reach(), we feed only\ncommits. But in this case the stack trace is:\n\n #0 can_all_from_reach_with_flag (from=0x55f95ff42f80 , with_flag=2048, assign_flag=16384, \n min_commit_date=1513037626, min_generation=0) at commit-reach.c:567\n #1 0x000055f95fe20e92 in ok_to_give_up () at upload-pack.c:346\n #2 0x000055f95fe20efa in get_common_commits () at upload-pack.c:369\n #3 0x000055f95fe22ce9 in upload_pack (options=0x7fff7c1b81f0) at upload-pack.c:1065\n #4 0x000055f95fcdc11b in cmd_upload_pack (argc=1, argv=0x7fff7c1b8498, prefix=0x0)\n at builtin/upload-pack.c:67\n #5 0x000055f95fc39574 in run_builtin (p=0x55f95ff02248 , argc=2, argv=0x7fff7c1b8498)\n at git.c:417\n #6 0x000055f95fc3987c in handle_builtin (argc=2, argv=0x7fff7c1b8498) at git.c:633\n #7 0x000055f95fc39b02 in cmd_main (argc=2, argv=0x7fff7c1b8498) at git.c:732\n #8 0x000055f95fce044b in main (argc=2, argv=0x7fff7c1b8498) at common-main.c:45\n\nand my client was fetching some tags (though it would similarly break\nwith other object types). So I'd think the easy reproduction would be\njust fetching a tag, but a trivial case didn't seem to trigger (it\nprobably needs a more substantial negotiation).\n\n-Peff\n"},{"id":"357945","messageId":"20180912042938.GA15876@sigill.intra.peff.net","threadId":"48889","inReplyTo":"20180912041425.GA10472@sigill.intra.peff.net","subject":"Re: [PATCH v2 17/18] commit-reach: make can_all_from_reach... linear","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-12T04:29:39Z","receivedAt":"2018-09-12T04:29:42Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 12, 2018 at 12:14:25AM -0400, Jeff King wrote:\n\n> > +\tALLOC_ARRAY(list, from->nr);\n> > \tfor (i = 0; i < from->nr; i++) {\n> > -\t\tstruct object *from_one = from->objects[i].item;\n> > +\t\tlist[i] = (struct commit *)from->objects[i].item;\n> \n> Some of the objects in my array are not commits, but rather tags, so\n> this is a bogus cast.\n> \n> You can see that the original code peeled them and threw away\n> non-commits:\n> \n> > \n> > -\t\tif (from_one->flags & assign_flag)\n> > -\t\t\tcontinue;\n> > -\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n> > -\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n> > -\t\t\t/* no way to tell if this is reachable by\n> > -\t\t\t * looking at the ancestry chain alone, so\n> > -\t\t\t * leave a note to ourselves not to worry about\n> > -\t\t\t * this object anymore.\n> > -\t\t\t */\n> > -\t\t\tfrom->objects[i].item->flags |= assign_flag;\n> > -\t\t\tcontinue;\n> > -\t\t}\n> \n> So presumably we'd need to do something similar.\n\nThis patch seems to fix it for me. It's more or less a reversion of the\nhunk above, though I didn't dig into whether I'm violating some other\nassumption in your new code.\n\nI think this function leaks \"list\" both from the location I noted here,\nas well as from normal exit\n\n---\n commit-reach.c | 25 ++++++++++++++++++-------\n 1 file changed, 18 insertions(+), 7 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 622eeb313d..abe90a2f55 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -547,20 +547,31 @@ int can_all_from_reach_with_flag(struct object_array *from,\n {\n \tstruct commit **list = NULL;\n \tint i;\n+\tint nr_commits;\n \tint result = 1;\n \n \tALLOC_ARRAY(list, from->nr);\n+\tnr_commits = 0;\n \tfor (i = 0; i < from->nr; i++) {\n-\t\tlist[i] = (struct commit *)from->objects[i].item;\n+\t\tstruct object *from_one = from->objects[i].item;\n \n-\t\tif (parse_commit(list[i]) ||\n-\t\t list[i]->generation < min_generation)\n-\t\t\treturn 0;\n+\t\tfrom_one = deref_tag(the_repository, from_one,\n+\t\t\t\t \"a from object\", 0);\n+\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n+\t\t\tfrom->objects[i].item->flags |= assign_flag;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tlist[nr_commits] = (struct commit *)from_one;\n+\t\tif (parse_commit(list[nr_commits]) ||\n+\t\t list[nr_commits]->generation < min_generation)\n+\t\t\treturn 0; /* is this a leak? */\n+\t\tnr_commits++;\n \t}\n \n-\tQSORT(list, from->nr, compare_commits_by_gen);\n+\tQSORT(list, nr_commits, compare_commits_by_gen);\n \n-\tfor (i = 0; i < from->nr; i++) {\n+\tfor (i = 0; i < nr_commits; i++) {\n \t\t/* DFS from list[i] */\n \t\tstruct commit_list *stack = NULL;\n \n@@ -603,7 +614,7 @@ int can_all_from_reach_with_flag(struct object_array *from,\n \t}\n \n cleanup:\n-\tfor (i = 0; i < from->nr; i++) {\n+\tfor (i = 0; i < nr_commits; i++) {\n \t\tclear_commit_marks(list[i], RESULT);\n \t\tclear_commit_marks(list[i], assign_flag);\n \t}\n-- \n2.19.0.600.ga229f7d059\n\n"},{"id":"357962","messageId":"0bf9103c-9377-506b-7ad7-e5273d8e94fc@gmail.com","threadId":"48889","inReplyTo":"20180912042938.GA15876@sigill.intra.peff.net","subject":"Re: [PATCH v2 17/18] commit-reach: make can_all_from_reach... linear","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-09-12T13:08:28Z","receivedAt":"2018-09-12T13:08:37Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 9/12/2018 12:29 AM, Jeff King wrote:\n> On Wed, Sep 12, 2018 at 12:14:25AM -0400, Jeff King wrote:\n>\n>>> +\tALLOC_ARRAY(list, from->nr);\n>>> \tfor (i = 0; i < from->nr; i++) {\n>>> -\t\tstruct object *from_one = from->objects[i].item;\n>>> +\t\tlist[i] = (struct commit *)from->objects[i].item;\n>> Some of the objects in my array are not commits, but rather tags, so\n>> this is a bogus cast.\n>>\n>> You can see that the original code peeled them and threw away\n>> non-commits:\n>>\n>>> \n>>> -\t\tif (from_one->flags & assign_flag)\n>>> -\t\t\tcontinue;\n>>> -\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n>>> -\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n>>> -\t\t\t/* no way to tell if this is reachable by\n>>> -\t\t\t * looking at the ancestry chain alone, so\n>>> -\t\t\t * leave a note to ourselves not to worry about\n>>> -\t\t\t * this object anymore.\n>>> -\t\t\t */\n>>> -\t\t\tfrom->objects[i].item->flags |= assign_flag;\n>>> -\t\t\tcontinue;\n>>> -\t\t}\n>> So presumably we'd need to do something similar.\n> This patch seems to fix it for me. It's more or less a reversion of the\n> hunk above, though I didn't dig into whether I'm violating some other\n> assumption in your new code.\n>\n> I think this function leaks \"list\" both from the location I noted here,\n> as well as from normal exit\nThanks for the report and the fix. I'll try to create  test that \ndemonstrates this and then push up a full patch.\n> ---\n> commit-reach.c | 25 ++++++++++++++++++-------\n> 1 file changed, 18 insertions(+), 7 deletions(-)\n>\n> diff --git a/commit-reach.c b/commit-reach.c\n> index 622eeb313d..abe90a2f55 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -547,20 +547,31 @@ int can_all_from_reach_with_flag(struct object_array *from,\n> {\n> \tstruct commit **list = NULL;\n> \tint i;\n> +\tint nr_commits;\n> \tint result = 1;\n> \n> \tALLOC_ARRAY(list, from->nr);\n> +\tnr_commits = 0;\n> \tfor (i = 0; i < from->nr; i++) {\n> -\t\tlist[i] = (struct commit *)from->objects[i].item;\n> +\t\tstruct object *from_one = from->objects[i].item;\n> \n> -\t\tif (parse_commit(list[i]) ||\n> -\t\t list[i]->generation < min_generation)\n> -\t\t\treturn 0;\n> +\t\tfrom_one = deref_tag(the_repository, from_one,\n> +\t\t\t\t \"a from object\", 0);\n> +\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n> +\t\t\tfrom->objects[i].item->flags |= assign_flag;\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n> +\t\tlist[nr_commits] = (struct commit *)from_one;\n> +\t\tif (parse_commit(list[nr_commits]) ||\n> +\t\t list[nr_commits]->generation < min_generation)\n> +\t\t\treturn 0; /* is this a leak? */\n> +\t\tnr_commits++;\n> \t}\n> \n> -\tQSORT(list, from->nr, compare_commits_by_gen);\n> +\tQSORT(list, nr_commits, compare_commits_by_gen);\n> \n> -\tfor (i = 0; i < from->nr; i++) {\n> +\tfor (i = 0; i < nr_commits; i++) {\n> \t\t/* DFS from list[i] */\n> \t\tstruct commit_list *stack = NULL;\n> \n> @@ -603,7 +614,7 @@ int can_all_from_reach_with_flag(struct object_array *from,\n> \t}\n> \n> cleanup:\n> -\tfor (i = 0; i < from->nr; i++) {\n> +\tfor (i = 0; i < nr_commits; i++) {\n> \t\tclear_commit_marks(list[i], RESULT);\n> \t\tclear_commit_marks(list[i], assign_flag);\n> \t}\n\n"},{"id":"359380","messageId":"d1b58614-989f-5998-6c53-c19eee409a2f@web.de","threadId":"48889","inReplyTo":"816821eec9ba476ccdfbfdf6e3cdd3619743ea2e.1531746012.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2018-10-01T19:16:01Z","receivedAt":"2018-10-01T19:16:13Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 28.06.2018 um 14:31 schrieb Derrick Stolee via GitGitGadget:\n> diff --git a/commit-reach.c b/commit-reach.c\n> index c58e50fbb..ac132c8e4 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -513,65 +513,88 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n> \treturn is_descendant_of(commit, list);\n> }\n> \n> -int reachable(struct commit *from, int with_flag, int assign_flag,\n> -\t time_t min_commit_date)\n> +static int compare_commits_by_gen(const void *_a, const void *_b)\n> {\n> -\tstruct prio_queue work = { compare_commits_by_commit_date };\n> +\tconst struct commit *a = (const struct commit *)_a;\n> +\tconst struct commit *b = (const struct commit *)_b;\n\nThis cast is bogus. QSORT gets handed a struct commit **, i.e. an array\nof pointers, and qsort(1) passes references to those pointers to the\ncompare function, and not the pointer values.\n\nAs a result it's unlikely that the array is sorted in the intended\norder. Given that, a silly question: Is sorting even necessary here?\n\nAnyway, the patch below should fix it.\n\n> \n> -\tprio_queue_put(&work, from);\n> -\twhile (work.nr) {\n> -\t\tstruct commit_list *list;\n> -\t\tstruct commit *commit = prio_queue_get(&work);\n> -\n> -\t\tif (commit->object.flags & with_flag) {\n> -\t\t\tfrom->object.flags |= assign_flag;\n> -\t\t\tbreak;\n> -\t\t}\n> -\t\tif (!commit->object.parsed)\n> -\t\t\tparse_object(the_repository, &commit->object.oid);\n> -\t\tif (commit->object.flags & REACHABLE)\n> -\t\t\tcontinue;\n> -\t\tcommit->object.flags |= REACHABLE;\n> -\t\tif (commit->date < min_commit_date)\n> -\t\t\tcontinue;\n> -\t\tfor (list = commit->parents; list; list = list->next) {\n> -\t\t\tstruct commit *parent = list->item;\n> -\t\t\tif (!(parent->object.flags & REACHABLE))\n> -\t\t\t\tprio_queue_put(&work, parent);\n> -\t\t}\n> -\t}\n> -\tfrom->object.flags |= REACHABLE;\n> -\tclear_commit_marks(from, REACHABLE);\n> -\tclear_prio_queue(&work);\n> -\treturn (from->object.flags & assign_flag);\n> +\tif (a->generation < b->generation)\n> +\t\treturn -1;\n> +\tif (a->generation > b->generation)\n> +\t\treturn 1;\n> +\treturn 0;\n> }\n> \n> int can_all_from_reach_with_flag(struct object_array *from,\n> \t\t\t\t int with_flag, int assign_flag,\n> -\t\t\t\t time_t min_commit_date)\n> +\t\t\t\t time_t min_commit_date,\n> +\t\t\t\t uint32_t min_generation)\n> {\n> +\tstruct commit **list = NULL;\n> \tint i;\n> +\tint result = 1;\n> \n> +\tALLOC_ARRAY(list, from->nr);\n> \tfor (i = 0; i < from->nr; i++) {\n> -\t\tstruct object *from_one = from->objects[i].item;\n> +\t\tlist[i] = (struct commit *)from->objects[i].item;\n> \n> -\t\tif (from_one->flags & assign_flag)\n> -\t\t\tcontinue;\n> -\t\tfrom_one = deref_tag(the_repository, from_one, \"a from object\", 0);\n> -\t\tif (!from_one || from_one->type != OBJ_COMMIT) {\n> -\t\t\t/* no way to tell if this is reachable by\n> -\t\t\t * looking at the ancestry chain alone, so\n> -\t\t\t * leave a note to ourselves not to worry about\n> -\t\t\t * this object anymore.\n> -\t\t\t */\n> -\t\t\tfrom->objects[i].item->flags |= assign_flag;\n> -\t\t\tcontinue;\n> -\t\t}\n> -\t\tif (!reachable((struct commit *)from_one, with_flag, assign_flag,\n> -\t\t\t min_commit_date))\n> +\t\tparse_commit(list[i]);\n> +\n> +\t\tif (list[i]->generation < min_generation)\n> \t\t\treturn 0;\n> \t}\n> -\treturn 1;\n> +\n> +\tQSORT(list, from->nr, compare_commits_by_gen);\n\n-- >8 --\nSubject: [PATCH] commit-reach: fix cast in compare_commits_by_gen()\n\nThe elements of the array to be sorted are commit pointers, so the\ncomparison function gets handed references to these pointers, not\npointers to commit objects. Cast to the right type and dereference\nonce to correctly get the commit reference.\n\nFound using Clang's ASan and t5500.\n\nSigned-off-by: Rene Scharfe \n---\nHas this patch a performance impact?\n\n commit-reach.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 00e5ceee6f..2f5e592d16 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -529,8 +529,8 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n \n static int compare_commits_by_gen(const void *_a, const void *_b)\n {\n-\tconst struct commit *a = (const struct commit *)_a;\n-\tconst struct commit *b = (const struct commit *)_b;\n+\tconst struct commit *a = *(const struct commit * const *)_a;\n+\tconst struct commit *b = *(const struct commit * const *)_b;\n \n \tif (a->generation < b->generation)\n \t\treturn -1;\n-- \n2.19.0\n"},{"id":"359386","messageId":"dd798e76-922f-a113-4408-e3892bee3b44@gmail.com","threadId":"48889","inReplyTo":"d1b58614-989f-5998-6c53-c19eee409a2f@web.de","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-10-01T19:26:00Z","receivedAt":"2018-10-01T19:26:04Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 10/1/2018 3:16 PM, René Scharfe wrote:\n> Am 28.06.2018 um 14:31 schrieb Derrick Stolee via GitGitGadget:\n>> diff --git a/commit-reach.c b/commit-reach.c\n>> index c58e50fbb..ac132c8e4 100644\n>> --- a/commit-reach.c\n>> +++ b/commit-reach.c\n>> @@ -513,65 +513,88 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n>> \treturn is_descendant_of(commit, list);\n>> }\n>> \n>> -int reachable(struct commit *from, int with_flag, int assign_flag,\n>> -\t time_t min_commit_date)\n>> +static int compare_commits_by_gen(const void *_a, const void *_b)\n>> {\n>> -\tstruct prio_queue work = { compare_commits_by_commit_date };\n>> +\tconst struct commit *a = (const struct commit *)_a;\n>> +\tconst struct commit *b = (const struct commit *)_b;\n> This cast is bogus. QSORT gets handed a struct commit **, i.e. an array\n> of pointers, and qsort(1) passes references to those pointers to the\n> compare function, and not the pointer values.\n\nGood catch! I'm disappointed that we couldn't use type-checking here, as \nit is quite difficult to discover that the types are wrong here.\n\n\n> As a result it's unlikely that the array is sorted in the intended\n> order. Given that, a silly question: Is sorting even necessary here?\n\nThe reason to sort is to hopefully minimize the amount we walk by \nexploring the \"lower\" commits first. This is a performance-only thing, \nnot a correctness issue (which is why the bug exists). Even then, it is \njust a heuristic.\n> Anyway, the patch below should fix it.\n>\n> -- >8 --\n> Subject: [PATCH] commit-reach: fix cast in compare_commits_by_gen()\n>\n> The elements of the array to be sorted are commit pointers, so the\n> comparison function gets handed references to these pointers, not\n> pointers to commit objects. Cast to the right type and dereference\n> once to correctly get the commit reference.\n>\n> Found using Clang's ASan and t5500.\n>\n> Signed-off-by: Rene Scharfe \n> ---\n> Has this patch a performance impact?\n>\n> commit-reach.c | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/commit-reach.c b/commit-reach.c\n> index 00e5ceee6f..2f5e592d16 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -529,8 +529,8 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n> \n> static int compare_commits_by_gen(const void *_a, const void *_b)\n> {\n> -\tconst struct commit *a = (const struct commit *)_a;\n> -\tconst struct commit *b = (const struct commit *)_b;\n> +\tconst struct commit *a = *(const struct commit * const *)_a;\n> +\tconst struct commit *b = *(const struct commit * const *)_b;\n\nI would expect s/* const */**/ here, but I'm guessing your formulation \nis a bit extra careful about types.\n\nThanks!\n\nReviewed-by: Derrick Stolee \n"},{"id":"359390","messageId":"223b14f7-213f-4d22-4776-22dcfd1806c2@web.de","threadId":"48889","inReplyTo":"dd798e76-922f-a113-4408-e3892bee3b44@gmail.com","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2018-10-01T20:37:29Z","receivedAt":"2018-10-01T20:37:42Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 01.10.2018 um 21:26 schrieb Derrick Stolee:\n> On 10/1/2018 3:16 PM, René Scharfe wrote:\n>> Am 28.06.2018 um 14:31 schrieb Derrick Stolee via GitGitGadget:\n>>> diff --git a/commit-reach.c b/commit-reach.c\n>>> index c58e50fbb..ac132c8e4 100644\n>>> --- a/commit-reach.c\n>>> +++ b/commit-reach.c\n>>> @@ -513,65 +513,88 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n>>> \treturn is_descendant_of(commit, list);\n>>> }\n>>> \n>>> -int reachable(struct commit *from, int with_flag, int assign_flag,\n>>> -\t time_t min_commit_date)\n>>> +static int compare_commits_by_gen(const void *_a, const void *_b)\n>>> {\n>>> -\tstruct prio_queue work = { compare_commits_by_commit_date };\n>>> +\tconst struct commit *a = (const struct commit *)_a;\n>>> +\tconst struct commit *b = (const struct commit *)_b;\n>> This cast is bogus. QSORT gets handed a struct commit **, i.e. an array\n>> of pointers, and qsort(1) passes references to those pointers to the\n>> compare function, and not the pointer values.\n> \n> Good catch! I'm disappointed that we couldn't use type-checking here, as \n> it is quite difficult to discover that the types are wrong here.\n\nGenerics in C are hard, and type checking traditionally falls by the\nwayside. You could use macros for that, like klib [*] does, but that\nhas its own downsides (more object text, debugging the sort macros\nthemselves is harder).\n\n[*] https://github.com/attractivechaos/klib/blob/master/ksort.h\n\n>> diff --git a/commit-reach.c b/commit-reach.c\n>> index 00e5ceee6f..2f5e592d16 100644\n>> --- a/commit-reach.c\n>> +++ b/commit-reach.c\n>> @@ -529,8 +529,8 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n>> \n>> static int compare_commits_by_gen(const void *_a, const void *_b)\n>> {\n>> -\tconst struct commit *a = (const struct commit *)_a;\n>> -\tconst struct commit *b = (const struct commit *)_b;\n>> +\tconst struct commit *a = *(const struct commit * const *)_a;\n>> +\tconst struct commit *b = *(const struct commit * const *)_b;\n> \n> I would expect s/* const */**/ here, but I'm guessing your formulation \n> is a bit extra careful about types.\n\nYeah, that second const is not necessary, as the dereference in the same\nline makes it inconsequential, but I added it to make clear that this\nfunction is really not supposed to write at all..\n\nRené\n"},{"id":"359644","messageId":"7b95417a-c8fb-4f1e-cb09-c36804a3a4d0@web.de","threadId":"48889","inReplyTo":"223b14f7-213f-4d22-4776-22dcfd1806c2@web.de","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2018-10-04T22:59:02Z","receivedAt":"2018-10-04T22:59:16Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 01.10.2018 um 22:37 schrieb René Scharfe:\n> Am 01.10.2018 um 21:26 schrieb Derrick Stolee:\n>> On 10/1/2018 3:16 PM, René Scharfe wrote:\n>>> Am 28.06.2018 um 14:31 schrieb Derrick Stolee via GitGitGadget:\n>>>> diff --git a/commit-reach.c b/commit-reach.c\n>>>> index c58e50fbb..ac132c8e4 100644\n>>>> --- a/commit-reach.c\n>>>> +++ b/commit-reach.c\n>>>> @@ -513,65 +513,88 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n>>>> \treturn is_descendant_of(commit, list);\n>>>> }\n>>>> \n>>>> -int reachable(struct commit *from, int with_flag, int assign_flag,\n>>>> -\t time_t min_commit_date)\n>>>> +static int compare_commits_by_gen(const void *_a, const void *_b)\n>>>> {\n>>>> -\tstruct prio_queue work = { compare_commits_by_commit_date };\n>>>> +\tconst struct commit *a = (const struct commit *)_a;\n>>>> +\tconst struct commit *b = (const struct commit *)_b;\n>>> This cast is bogus. QSORT gets handed a struct commit **, i.e. an array\n>>> of pointers, and qsort(1) passes references to those pointers to the\n>>> compare function, and not the pointer values.\n>>\n>> Good catch! I'm disappointed that we couldn't use type-checking here, as \n>> it is quite difficult to discover that the types are wrong here.\n> \n> Generics in C are hard, and type checking traditionally falls by the\n> wayside. You could use macros for that, like klib [*] does, but that\n> has its own downsides (more object text, debugging the sort macros\n> themselves is harder).\n> \n> [*] https://github.com/attractivechaos/klib/blob/master/ksort.h\n\nWe could also do something like this to reduce the amount of manual\ncasting, but do we want to? (Macro at the bottom, three semi-random\nexamples at the top.)\n---\n bisect.c | 11 +++--------\n commit-graph.c | 9 ++-------\n commit-reach.c | 12 +++++-------\n git-compat-util.h | 12 ++++++++++++\n 4 files changed, 22 insertions(+), 22 deletions(-)\n\ndiff --git a/bisect.c b/bisect.c\nindex e8b17cf7e1..06be3a3c15 100644\n--- a/bisect.c\n+++ b/bisect.c\n@@ -192,16 +192,11 @@ struct commit_dist {\n \tint distance;\n };\n \n-static int compare_commit_dist(const void *a_, const void *b_)\n-{\n-\tstruct commit_dist *a, *b;\n-\n-\ta = (struct commit_dist *)a_;\n-\tb = (struct commit_dist *)b_;\n+DEFINE_SORT(sort_by_commit_dist, struct commit_dist, a, b, {\n \tif (a->distance != b->distance)\n \t\treturn b->distance - a->distance; /* desc sort */\n \treturn oidcmp(&a->commit->object.oid, &b->commit->object.oid);\n-}\n+})\n \n static struct commit_list *best_bisection_sorted(struct commit_list *list, int nr)\n {\n@@ -223,7 +218,7 @@ static struct commit_list *best_bisection_sorted(struct commit_list *list, int n\n \t\tarray[cnt].distance = distance;\n \t\tcnt++;\n \t}\n-\tQSORT(array, cnt, compare_commit_dist);\n+\tsort_by_commit_dist(array, cnt);\n \tfor (p = list, i = 0; i < cnt; i++) {\n \t\tstruct object *obj = &(array[i].commit->object);\n \ndiff --git a/commit-graph.c b/commit-graph.c\nindex 7f4519ec3b..a2202414e0 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -550,12 +550,7 @@ static void write_graph_chunk_large_edges(struct hashfile *f,\n \t}\n }\n \n-static int commit_compare(const void *_a, const void *_b)\n-{\n-\tconst struct object_id *a = (const struct object_id *)_a;\n-\tconst struct object_id *b = (const struct object_id *)_b;\n-\treturn oidcmp(a, b);\n-}\n+DEFINE_SORT(sort_oids, struct object_id, a, b, return oidcmp(a, b))\n \n struct packed_commit_list {\n \tstruct commit **list;\n@@ -780,7 +775,7 @@ void write_commit_graph(const char *obj_dir,\n \n \tclose_reachable(&oids);\n \n-\tQSORT(oids.list, oids.nr, commit_compare);\n+\tsort_oids(oids.list, oids.nr);\n \n \tcount_distinct = 1;\n \tfor (i = 1; i < oids.nr; i++) {\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 2f5e592d16..3aef47c3dd 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -527,17 +527,15 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n \treturn is_descendant_of(commit, list);\n }\n \n-static int compare_commits_by_gen(const void *_a, const void *_b)\n-{\n-\tconst struct commit *a = *(const struct commit * const *)_a;\n-\tconst struct commit *b = *(const struct commit * const *)_b;\n-\n+DEFINE_SORT(sort_commits_by_gen, struct commit *, ap, bp, {\n+\tconst struct commit *a = *ap;\n+\tconst struct commit *b = *bp;\n \tif (a->generation < b->generation)\n \t\treturn -1;\n \tif (a->generation > b->generation)\n \t\treturn 1;\n \treturn 0;\n-}\n+})\n \n int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\t unsigned int with_flag,\n@@ -580,7 +578,7 @@ int can_all_from_reach_with_flag(struct object_array *from,\n \t\tnr_commits++;\n \t}\n \n-\tQSORT(list, nr_commits, compare_commits_by_gen);\n+\tsort_commits_by_gen(list, nr_commits);\n \n \tfor (i = 0; i < nr_commits; i++) {\n \t\t/* DFS from list[i] */\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 5f2e90932f..f9e78d69a2 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -1066,6 +1066,18 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n \t\tqsort(base, nmemb, size, compar);\n }\n \n+#define DEFINE_SORT(name, elemtype, one, two, code)\t\t\t\\\n+static int name##_compare(const void *one##_v_, const void *two##_v_)\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\telemtype const *one = one##_v_;\t\t\t\t\t\\\n+\telemtype const *two = two##_v_;\t\t\t\t\t\\\n+\tcode;\t\t\t\t\t\t\t\t\\\n+}\t\t\t\t\t\t\t\t\t\\\n+static void name(elemtype *array, size_t n)\t\t\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\tQSORT(array, n, name##_compare);\t\t\t\t\\\n+}\n+\n #ifndef HAVE_ISO_QSORT_S\n int git_qsort_s(void *base, size_t nmemb, size_t size,\n \t\tint (*compar)(const void *, const void *, void *), void *ctx);\n-- \n2.19.0\n\n\n"},{"id":"359668","messageId":"f27fd384-6cfa-5acb-3e2b-72668f04d689@gmail.com","threadId":"48889","inReplyTo":"7b95417a-c8fb-4f1e-cb09-c36804a3a4d0@web.de","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-10-05T12:15:09Z","receivedAt":"2018-10-05T12:15:15Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 10/4/2018 6:59 PM, René Scharfe wrote:\n> Am 01.10.2018 um 22:37 schrieb René Scharfe:\n>> Am 01.10.2018 um 21:26 schrieb Derrick Stolee:\n>>> Good catch! I'm disappointed that we couldn't use type-checking here, as\n>>> it is quite difficult to discover that the types are wrong here.\n>> Generics in C are hard, and type checking traditionally falls by the\n>> wayside. You could use macros for that, like klib [*] does, but that\n>> has its own downsides (more object text, debugging the sort macros\n>> themselves is harder).\n>>\n>> [*] https://github.com/attractivechaos/klib/blob/master/ksort.h\n> We could also do something like this to reduce the amount of manual\n> casting, but do we want to? (Macro at the bottom, three semi-random\n> examples at the top.)\n\nI like the idea! It certainly can assist in some of the repeat work when \npreparing to QSORT, and make it less error-prone.\n\n> ---\n> bisect.c | 11 +++--------\n> commit-graph.c | 9 ++-------\n> commit-reach.c | 12 +++++-------\n> git-compat-util.h | 12 ++++++++++++\n> 4 files changed, 22 insertions(+), 22 deletions(-)\n>\n> diff --git a/bisect.c b/bisect.c\n> index e8b17cf7e1..06be3a3c15 100644\n> --- a/bisect.c\n> +++ b/bisect.c\n> @@ -192,16 +192,11 @@ struct commit_dist {\n> \tint distance;\n> };\n> \n> -static int compare_commit_dist(const void *a_, const void *b_)\n> -{\n> -\tstruct commit_dist *a, *b;\n> -\n> -\ta = (struct commit_dist *)a_;\n> -\tb = (struct commit_dist *)b_;\n> +DEFINE_SORT(sort_by_commit_dist, struct commit_dist, a, b, {\n> \tif (a->distance != b->distance)\n> \t\treturn b->distance - a->distance; /* desc sort */\n> \treturn oidcmp(&a->commit->object.oid, &b->commit->object.oid);\n> -}\n> +})\n> \n> static struct commit_list *best_bisection_sorted(struct commit_list *list, int nr)\n> {\n> @@ -223,7 +218,7 @@ static struct commit_list *best_bisection_sorted(struct commit_list *list, int n\n> \t\tarray[cnt].distance = distance;\n> \t\tcnt++;\n> \t}\n> -\tQSORT(array, cnt, compare_commit_dist);\n> +\tsort_by_commit_dist(array, cnt);\n> \tfor (p = list, i = 0; i < cnt; i++) {\n> \t\tstruct object *obj = &(array[i].commit->object);\n> \n> diff --git a/commit-graph.c b/commit-graph.c\n> index 7f4519ec3b..a2202414e0 100644\n> --- a/commit-graph.c\n> +++ b/commit-graph.c\n> @@ -550,12 +550,7 @@ static void write_graph_chunk_large_edges(struct hashfile *f,\n> \t}\n> }\n> \n> -static int commit_compare(const void *_a, const void *_b)\n> -{\n> -\tconst struct object_id *a = (const struct object_id *)_a;\n> -\tconst struct object_id *b = (const struct object_id *)_b;\n> -\treturn oidcmp(a, b);\n> -}\n> +DEFINE_SORT(sort_oids, struct object_id, a, b, return oidcmp(a, b))\n> \n> struct packed_commit_list {\n> \tstruct commit **list;\n> @@ -780,7 +775,7 @@ void write_commit_graph(const char *obj_dir,\n> \n> \tclose_reachable(&oids);\n> \n> -\tQSORT(oids.list, oids.nr, commit_compare);\n> +\tsort_oids(oids.list, oids.nr);\n> \n> \tcount_distinct = 1;\n> \tfor (i = 1; i < oids.nr; i++) {\n> diff --git a/commit-reach.c b/commit-reach.c\n> index 2f5e592d16..3aef47c3dd 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -527,17 +527,15 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n> \treturn is_descendant_of(commit, list);\n> }\n> \n> -static int compare_commits_by_gen(const void *_a, const void *_b)\n> -{\n> -\tconst struct commit *a = *(const struct commit * const *)_a;\n> -\tconst struct commit *b = *(const struct commit * const *)_b;\n> -\n> +DEFINE_SORT(sort_commits_by_gen, struct commit *, ap, bp, {\n> +\tconst struct commit *a = *ap;\n> +\tconst struct commit *b = *bp;\n> \tif (a->generation < b->generation)\n> \t\treturn -1;\n> \tif (a->generation > b->generation)\n> \t\treturn 1;\n> \treturn 0;\n> -}\n> +})\n\nHere, to make the macro version compile you need to cast ap and bp, \nwhich gives us a level of type-safety that wasn't there before. That can \nhelp us find errors at compile-time!\n\n> \n> int can_all_from_reach_with_flag(struct object_array *from,\n> \t\t\t\t unsigned int with_flag,\n> @@ -580,7 +578,7 @@ int can_all_from_reach_with_flag(struct object_array *from,\n> \t\tnr_commits++;\n> \t}\n> \n> -\tQSORT(list, nr_commits, compare_commits_by_gen);\n> +\tsort_commits_by_gen(list, nr_commits);\n> \n> \tfor (i = 0; i < nr_commits; i++) {\n> \t\t/* DFS from list[i] */\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 5f2e90932f..f9e78d69a2 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -1066,6 +1066,18 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n> \t\tqsort(base, nmemb, size, compar);\n> }\n> \n> +#define DEFINE_SORT(name, elemtype, one, two, code)\t\t\t\\\n> +static int name##_compare(const void *one##_v_, const void *two##_v_)\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\telemtype const *one = one##_v_;\t\t\t\t\t\\\n> +\telemtype const *two = two##_v_;\t\t\t\t\t\\\n> +\tcode;\t\t\t\t\t\t\t\t\\\n> +}\t\t\t\t\t\t\t\t\t\\\n> +static void name(elemtype *array, size_t n)\t\t\t\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\tQSORT(array, n, name##_compare);\t\t\t\t\\\n> +}\n> +\n\nI would use this macro.\n\nThanks,\n\n-Stolee\n\n"},{"id":"359703","messageId":"20181005165157.GC11254@sigill.intra.peff.net","threadId":"48889","inReplyTo":"7b95417a-c8fb-4f1e-cb09-c36804a3a4d0@web.de","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-10-05T16:51:58Z","receivedAt":"2018-10-05T16:52:01Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Oct 05, 2018 at 12:59:02AM +0200, René Scharfe wrote:\n\n> We could also do something like this to reduce the amount of manual\n> casting, but do we want to? (Macro at the bottom, three semi-random\n> examples at the top.)\n> [...]\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 5f2e90932f..f9e78d69a2 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -1066,6 +1066,18 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n> \t\tqsort(base, nmemb, size, compar);\n> }\n> \n> +#define DEFINE_SORT(name, elemtype, one, two, code)\t\t\t\\\n> +static int name##_compare(const void *one##_v_, const void *two##_v_)\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\telemtype const *one = one##_v_;\t\t\t\t\t\\\n> +\telemtype const *two = two##_v_;\t\t\t\t\t\\\n> +\tcode;\t\t\t\t\t\t\t\t\\\n> +}\t\t\t\t\t\t\t\t\t\\\n> +static void name(elemtype *array, size_t n)\t\t\t\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\tQSORT(array, n, name##_compare);\t\t\t\t\\\n> +}\n\nInteresting. When I saw the callers of this macro, I first thought you\nwere just removing the casts from the comparison function, but the real\nvalue here is the matching QSORT() wrapper which provides the type\nsafety.\n\nI'm not wild about declaring functions inside macros, just because it\nmakes tools like ctags like useful (but I have certainly been guilty of\nit myself). I'd also worry that taking \"code\" as a macro parameter might\nnot scale (what happens if the code has a comma in it?)\n\nI think we can address that last part by switching the definition order.\nLike:\n\n #define DEFINE_SORT(name, elemtype, one, two) \\\n static int name##_compare(const void *, const void *); \\\n static void name(elemtype *array, size_t n) \\\n { \\\n\tQSORT(array, n, name##_compare); \\\n } \\\n static int name##_compare(const void *one##_v_, const void *two##_v_) \\\n { \\\n\telemtype const *one = one##_v_;\t\t\t\t\t\\\n\telemtype const *two = two##_v_;\t\t\t\t\t\\\n\nAnd then expecting the caller to do:\n\n DEFINE_SORT(foo, struct foo, a, b)\n /* code goes here */\n }\n\nThe unbalanced braces are nasty, though (and likely to screw up editor\nformatting, highlighting, etc).\n\nI wonder if it would be possible to just declare the comparison function\nwith its real types, and then teach QSORT() to do a type check. That\nwould require typeof() at least, but it would be OK for the type-check\nto be available only to gcc/clang users, I think.\n\nI'm not quite sure what that type-check would look like, but I was\nthinking something along the lines of (inside the QSORT macro):\n\n do {\n /* this will yield a type mismatch if fed the wrong function */\n int (*check)(const typeof(array), const typeof(array)) = compar;\n sane_qsort(array, n, sizeof(*array), n);\n } while (0)\n\nI have no idea if that even comes close to compiling, though.\n\n-Peff\n"},{"id":"359714","messageId":"dca35e44-a763-bcf0-f457-b8dab53815cf@web.de","threadId":"48889","inReplyTo":"20181005165157.GC11254@sigill.intra.peff.net","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2018-10-05T18:48:27Z","receivedAt":"2018-10-05T18:48:42Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 05.10.2018 um 18:51 schrieb Jeff King:\n> On Fri, Oct 05, 2018 at 12:59:02AM +0200, René Scharfe wrote:\n> \n>> We could also do something like this to reduce the amount of manual\n>> casting, but do we want to? (Macro at the bottom, three semi-random\n>> examples at the top.)\n>> [...]\n>> diff --git a/git-compat-util.h b/git-compat-util.h\n>> index 5f2e90932f..f9e78d69a2 100644\n>> --- a/git-compat-util.h\n>> +++ b/git-compat-util.h\n>> @@ -1066,6 +1066,18 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n>> \t\tqsort(base, nmemb, size, compar);\n>> }\n>> \n>> +#define DEFINE_SORT(name, elemtype, one, two, code)\t\t\t\\\n>> +static int name##_compare(const void *one##_v_, const void *two##_v_)\t\\\n>> +{\t\t\t\t\t\t\t\t\t\\\n>> +\telemtype const *one = one##_v_;\t\t\t\t\t\\\n>> +\telemtype const *two = two##_v_;\t\t\t\t\t\\\n>> +\tcode;\t\t\t\t\t\t\t\t\\\n>> +}\t\t\t\t\t\t\t\t\t\\\n>> +static void name(elemtype *array, size_t n)\t\t\t\t\\\n>> +{\t\t\t\t\t\t\t\t\t\\\n>> +\tQSORT(array, n, name##_compare);\t\t\t\t\\\n>> +}\n> \n> Interesting. When I saw the callers of this macro, I first thought you\n> were just removing the casts from the comparison function, but the real\n> value here is the matching QSORT() wrapper which provides the type\n> safety.\n\nIndeed.\n\n> I'm not wild about declaring functions inside macros, just because it\n> makes tools like ctags like useful (but I have certainly been guilty of\n> it myself). I'd also worry that taking \"code\" as a macro parameter might\n> not scale (what happens if the code has a comma in it?)\n\nIt works fine, as long as the comma is surrounded by parentheses, so\nfunction calls with more than one parameter are fine without any change.\n\n> I think we can address that last part by switching the definition order.\n> Like:\n> \n> #define DEFINE_SORT(name, elemtype, one, two) \\\n> static int name##_compare(const void *, const void *); \\\n> static void name(elemtype *array, size_t n) \\\n> { \\\n> \tQSORT(array, n, name##_compare); \\\n> } \\\n> static int name##_compare(const void *one##_v_, const void *two##_v_) \\\n> { \\\n> \telemtype const *one = one##_v_;\t\t\t\t\t\\\n> \telemtype const *two = two##_v_;\t\t\t\t\t\\\n> \n> And then expecting the caller to do:\n> \n> DEFINE_SORT(foo, struct foo, a, b)\n> /* code goes here */\n> }\n> \n> The unbalanced braces are nasty, though (and likely to screw up editor\n> formatting, highlighting, etc).\n\nAdding an extra pair of parentheses if needed is also not ideal, but has\nless downsides, I think.\n\n> I wonder if it would be possible to just declare the comparison function\n> with its real types, and then teach QSORT() to do a type check. That\n> would require typeof() at least, but it would be OK for the type-check\n> to be available only to gcc/clang users, I think.\n> \n> I'm not quite sure what that type-check would look like, but I was\n> thinking something along the lines of (inside the QSORT macro):\n> \n> do {\n> /* this will yield a type mismatch if fed the wrong function */\n> int (*check)(const typeof(array), const typeof(array)) = compar;\n> sane_qsort(array, n, sizeof(*array), n);\n> } while (0)\n> \n> I have no idea if that even comes close to compiling, though.\n\nIf the comparison function has proper types then we need to declare a\nversion with void pointer parameters as well to give to qsort(3). I\nthink using cast function pointers is undefined. Perhaps like this?\n\n---\n bisect.c | 11 +++++------\n commit-graph.c | 8 ++++----\n commit-reach.c | 12 +++++++-----\n git-compat-util.h | 14 ++++++++++++++\n 4 files changed, 30 insertions(+), 15 deletions(-)\n\ndiff --git a/bisect.c b/bisect.c\nindex e8b17cf7e1..1fc6278c6b 100644\n--- a/bisect.c\n+++ b/bisect.c\n@@ -192,17 +192,16 @@ struct commit_dist {\n \tint distance;\n };\n \n-static int compare_commit_dist(const void *a_, const void *b_)\n+static int compare_commit_dist(const struct commit_dist *a,\n+\t\t\t const struct commit_dist *b)\n {\n-\tstruct commit_dist *a, *b;\n-\n-\ta = (struct commit_dist *)a_;\n-\tb = (struct commit_dist *)b_;\n \tif (a->distance != b->distance)\n \t\treturn b->distance - a->distance; /* desc sort */\n \treturn oidcmp(&a->commit->object.oid, &b->commit->object.oid);\n }\n \n+DEFINE_SORT(sort_by_commit_dist, struct commit_dist *, compare_commit_dist)\n+\n static struct commit_list *best_bisection_sorted(struct commit_list *list, int nr)\n {\n \tstruct commit_list *p;\n@@ -223,7 +222,7 @@ static struct commit_list *best_bisection_sorted(struct commit_list *list, int n\n \t\tarray[cnt].distance = distance;\n \t\tcnt++;\n \t}\n-\tQSORT(array, cnt, compare_commit_dist);\n+\tsort_by_commit_dist(array, cnt);\n \tfor (p = list, i = 0; i < cnt; i++) {\n \t\tstruct object *obj = &(array[i].commit->object);\n \ndiff --git a/commit-graph.c b/commit-graph.c\nindex 7f4519ec3b..07d302fefd 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -550,13 +550,13 @@ static void write_graph_chunk_large_edges(struct hashfile *f,\n \t}\n }\n \n-static int commit_compare(const void *_a, const void *_b)\n+static int commit_compare(const struct object_id *a, const struct object_id *b)\n {\n-\tconst struct object_id *a = (const struct object_id *)_a;\n-\tconst struct object_id *b = (const struct object_id *)_b;\n \treturn oidcmp(a, b);\n }\n \n+DEFINE_SORT(sort_oids, struct object_id *, commit_compare)\n+\n struct packed_commit_list {\n \tstruct commit **list;\n \tint nr;\n@@ -780,7 +780,7 @@ void write_commit_graph(const char *obj_dir,\n \n \tclose_reachable(&oids);\n \n-\tQSORT(oids.list, oids.nr, commit_compare);\n+\tsort_oids(oids.list, oids.nr);\n \n \tcount_distinct = 1;\n \tfor (i = 1; i < oids.nr; i++) {\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 2f5e592d16..496c4201af 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -527,11 +527,11 @@ int commit_contains(struct ref_filter *filter, struct commit *commit,\n \treturn is_descendant_of(commit, list);\n }\n \n-static int compare_commits_by_gen(const void *_a, const void *_b)\n+static int compare_commits_by_gen(const struct commit * const *ap,\n+\t\t\t\t const struct commit * const *bp)\n {\n-\tconst struct commit *a = *(const struct commit * const *)_a;\n-\tconst struct commit *b = *(const struct commit * const *)_b;\n-\n+\tconst struct commit *a = *ap;\n+\tconst struct commit *b = *bp;\n \tif (a->generation < b->generation)\n \t\treturn -1;\n \tif (a->generation > b->generation)\n@@ -539,6 +539,8 @@ static int compare_commits_by_gen(const void *_a, const void *_b)\n \treturn 0;\n }\n \n+DEFINE_SORT(sort_commits_by_gen, struct commit **, compare_commits_by_gen)\n+\n int can_all_from_reach_with_flag(struct object_array *from,\n \t\t\t\t unsigned int with_flag,\n \t\t\t\t unsigned int assign_flag,\n@@ -580,7 +582,7 @@ int can_all_from_reach_with_flag(struct object_array *from,\n \t\tnr_commits++;\n \t}\n \n-\tQSORT(list, nr_commits, compare_commits_by_gen);\n+\tsort_commits_by_gen(list, nr_commits);\n \n \tfor (i = 0; i < nr_commits; i++) {\n \t\t/* DFS from list[i] */\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 5f2e90932f..2462173790 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -1066,6 +1066,20 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n \t\tqsort(base, nmemb, size, compar);\n }\n \n+#define DEFINE_SORT(name, type, compare)\t\t\t\t\\\n+static int compare##_void(const void *one, const void *two)\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\treturn compare(one, two);\t\t\t\t\t\\\n+}\t\t\t\t\t\t\t\t\t\\\n+static void name(type base, size_t nmemb)\t\t\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\tconst type dummy = NULL;\t\t\t\t\t\\\n+\tif (nmemb > 1)\t\t\t\t\t\t\t\\\n+\t\tqsort(base, nmemb, sizeof(base[0]), compare##_void);\t\\\n+\telse if (0)\t\t\t\t\t\t\t\\\n+\t\tcompare(dummy, dummy);\t\t\t\t\t\\\n+}\n+\n #ifndef HAVE_ISO_QSORT_S\n int git_qsort_s(void *base, size_t nmemb, size_t size,\n \t\tint (*compar)(const void *, const void *, void *), void *ctx);\n-- \n2.19.0\n"},{"id":"359718","messageId":"20181005190847.GC17482@sigill.intra.peff.net","threadId":"48889","inReplyTo":"dca35e44-a763-bcf0-f457-b8dab53815cf@web.de","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-10-05T19:08:48Z","receivedAt":"2018-10-05T19:08:51Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Oct 05, 2018 at 08:48:27PM +0200, René Scharfe wrote:\n\n> If the comparison function has proper types then we need to declare a\n> version with void pointer parameters as well to give to qsort(3). I\n> think using cast function pointers is undefined. Perhaps like this?\n\nI think it's undefined, too, though we have many instances already.\n\n> +#define DEFINE_SORT(name, type, compare)\t\t\t\t\\\n> +static int compare##_void(const void *one, const void *two)\t\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\treturn compare(one, two);\t\t\t\t\t\\\n> +}\t\t\t\t\t\t\t\t\t\\\n> +static void name(type base, size_t nmemb)\t\t\t\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\tconst type dummy = NULL;\t\t\t\t\t\\\n> +\tif (nmemb > 1)\t\t\t\t\t\t\t\\\n> +\t\tqsort(base, nmemb, sizeof(base[0]), compare##_void);\t\\\n> +\telse if (0)\t\t\t\t\t\t\t\\\n> +\t\tcompare(dummy, dummy);\t\t\t\t\t\\\n> +}\n\nI do like that this removes the need to have the code block aspart of\nthe macro.\n\nDid you measure to see if there is any runtime impact?\n\nAs an aside, we may need to take a \"scope\" argument in case somebody\nwants to do this in a non-static way. It would be nice if we could make\nthis \"static inline\", but I don't think even a clever compiler would be\nable to omit the wrapper call.\n\n-Peff\n"},{"id":"359719","messageId":"878t3cgqd2.fsf@evledraar.gmail.com","threadId":"48889","inReplyTo":"20181005165157.GC11254@sigill.intra.peff.net","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-10-05T19:12:09Z","receivedAt":"2018-10-05T19:12:16Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Oct 05 2018, Jeff King wrote:\n\n> On Fri, Oct 05, 2018 at 12:59:02AM +0200, René Scharfe wrote:\n>\n>> We could also do something like this to reduce the amount of manual\n>> casting, but do we want to? (Macro at the bottom, three semi-random\n>> examples at the top.)\n>> [...]\n>> diff --git a/git-compat-util.h b/git-compat-util.h\n>> index 5f2e90932f..f9e78d69a2 100644\n>> --- a/git-compat-util.h\n>> +++ b/git-compat-util.h\n>> @@ -1066,6 +1066,18 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n>> \t\tqsort(base, nmemb, size, compar);\n>> }\n>>\n>> +#define DEFINE_SORT(name, elemtype, one, two, code)\t\t\t\\\n>> +static int name##_compare(const void *one##_v_, const void *two##_v_)\t\\\n>> +{\t\t\t\t\t\t\t\t\t\\\n>> +\telemtype const *one = one##_v_;\t\t\t\t\t\\\n>> +\telemtype const *two = two##_v_;\t\t\t\t\t\\\n>> +\tcode;\t\t\t\t\t\t\t\t\\\n>> +}\t\t\t\t\t\t\t\t\t\\\n>> +static void name(elemtype *array, size_t n)\t\t\t\t\\\n>> +{\t\t\t\t\t\t\t\t\t\\\n>> +\tQSORT(array, n, name##_compare);\t\t\t\t\\\n>> +}\n>\n> Interesting. When I saw the callers of this macro, I first thought you\n> were just removing the casts from the comparison function, but the real\n> value here is the matching QSORT() wrapper which provides the type\n> safety.\n>\n> I'm not wild about declaring functions inside macros, just because it\n> makes tools like ctags like useful (but I have certainly been guilty of\n> it myself). I'd also worry that taking \"code\" as a macro parameter might\n> not scale (what happens if the code has a comma in it?)\n\nThere's always the option of generating the C code as part of some build\nstep and carrying around a big C file with various type-safe functions\nthat only differ in the types they operate on. It can even be committed\nto source control.\n\nThat sucks in some ways for sure, but is a lot friendlier for grepping,\nctags etc.\n\nI've just barely resisted the urge to include that thread where we were\ndiscussing making the code C++-compiler compatible in the References\nheader :)\n"},{"id":"359723","messageId":"20181005192816.GE17482@sigill.intra.peff.net","threadId":"48889","inReplyTo":"878t3cgqd2.fsf@evledraar.gmail.com","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-10-05T19:28:16Z","receivedAt":"2018-10-05T19:28:19Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Oct 05, 2018 at 09:12:09PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> > I'm not wild about declaring functions inside macros, just because it\n> > makes tools like ctags like useful (but I have certainly been guilty of\n> > it myself). I'd also worry that taking \"code\" as a macro parameter might\n> > not scale (what happens if the code has a comma in it?)\n> \n> There's always the option of generating the C code as part of some build\n> step and carrying around a big C file with various type-safe functions\n> that only differ in the types they operate on. It can even be committed\n> to source control.\n> \n> That sucks in some ways for sure, but is a lot friendlier for grepping,\n> ctags etc.\n\nYeah, in a lot of ways the C preprocessor is not great for larger-scale\ncode generation. I was hoping we could get away without having the\nbodies of these functions as part of the generated bit, though.\n\nI think what René showed later in the thread is not too bad in that\nrespect.\n\n> I've just barely resisted the urge to include that thread where we were\n> discussing making the code C++-compiler compatible in the References\n> header :)\n\nYes. The main thing I would want out of using C++ is type-safe,\nefficient data structures. IIRC, early versions of C++ were implemented\nvia code generation, and we're basically walking down that same road.\n\nI'm not sure where the right cutoff is, though. It's nice to pick up\nthe solution somebody else produced, but requiring a C++ compiler to\nbuild Git is a pretty big step that I imagine will create a lot of new\nproblems. (We're just now allowing C99 -- I don't even want to think\nabout what kind of compiler issues we'll run into on antique systems\ntrying to use C++).\n\n-Peff\n"},{"id":"359724","messageId":"c05f192b-4e89-48b0-1c23-b43ec6fdb74b@web.de","threadId":"48889","inReplyTo":"20181005190847.GC17482@sigill.intra.peff.net","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2018-10-05T19:36:28Z","receivedAt":"2018-10-05T19:36:41Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 05.10.2018 um 21:08 schrieb Jeff King:\n> On Fri, Oct 05, 2018 at 08:48:27PM +0200, René Scharfe wrote:\n>> +#define DEFINE_SORT(name, type, compare)\t\t\t\t\\\n>> +static int compare##_void(const void *one, const void *two)\t\t\\\n>> +{\t\t\t\t\t\t\t\t\t\\\n>> +\treturn compare(one, two);\t\t\t\t\t\\\n>> +}\t\t\t\t\t\t\t\t\t\\\n>> +static void name(type base, size_t nmemb)\t\t\t\t\\\n>> +{\t\t\t\t\t\t\t\t\t\\\n>> +\tconst type dummy = NULL;\t\t\t\t\t\\\n>> +\tif (nmemb > 1)\t\t\t\t\t\t\t\\\n>> +\t\tqsort(base, nmemb, sizeof(base[0]), compare##_void);\t\\\n>> +\telse if (0)\t\t\t\t\t\t\t\\\n>> +\t\tcompare(dummy, dummy);\t\t\t\t\t\\\n>> +}\n> \n> I do like that this removes the need to have the code block aspart of\n> the macro.\n> \n> Did you measure to see if there is any runtime impact?\n\nNo, but I wouldn't expect any -- the generated code should be the same\nin most cases.\n\nHere's an example: https://godbolt.org/z/gwXENy.\n\n> As an aside, we may need to take a \"scope\" argument in case somebody\n> wants to do this in a non-static way.\n\nSure. (They could easily wrap the static function, but a macro\nparameter is simpler still.)\n\n> It would be nice if we could make\n> this \"static inline\", but I don't think even a clever compiler would be\n> able to omit the wrapper call.\n\nIt could, if it was to inline qsort(3). Current compilers don't do\nthat AFAIK, but I wouldn't be too surprised if they started to.\n\nThe typed comparison function can be inlined into the one with the void\npointers, though.\n\nRené\n"},{"id":"359726","messageId":"20181005194223.GA19428@sigill.intra.peff.net","threadId":"48889","inReplyTo":"c05f192b-4e89-48b0-1c23-b43ec6fdb74b@web.de","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-10-05T19:42:23Z","receivedAt":"2018-10-05T19:42:26Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Oct 05, 2018 at 09:36:28PM +0200, René Scharfe wrote:\n\n> Am 05.10.2018 um 21:08 schrieb Jeff King:\n> > On Fri, Oct 05, 2018 at 08:48:27PM +0200, René Scharfe wrote:\n> >> +#define DEFINE_SORT(name, type, compare)\t\t\t\t\\\n> >> +static int compare##_void(const void *one, const void *two)\t\t\\\n> >> +{\t\t\t\t\t\t\t\t\t\\\n> >> +\treturn compare(one, two);\t\t\t\t\t\\\n> >> +}\t\t\t\t\t\t\t\t\t\\\n> >> +static void name(type base, size_t nmemb)\t\t\t\t\\\n> >> +{\t\t\t\t\t\t\t\t\t\\\n> >> +\tconst type dummy = NULL;\t\t\t\t\t\\\n> >> +\tif (nmemb > 1)\t\t\t\t\t\t\t\\\n> >> +\t\tqsort(base, nmemb, sizeof(base[0]), compare##_void);\t\\\n> >> +\telse if (0)\t\t\t\t\t\t\t\\\n> >> +\t\tcompare(dummy, dummy);\t\t\t\t\t\\\n> >> +}\n> > \n> > I do like that this removes the need to have the code block aspart of\n> > the macro.\n> > \n> > Did you measure to see if there is any runtime impact?\n> \n> No, but I wouldn't expect any -- the generated code should be the same\n> in most cases.\n> \n> Here's an example: https://godbolt.org/z/gwXENy.\n\nOK, that's good enough for me.\n\n> The typed comparison function can be inlined into the one with the void\n> pointers, though.\n\nRight, that makes sense. I suspect it depends on the comparison function\nbeing static, but in a DEFINE_SORT() world, they generally could be.\n\nSo I like this approach.\n\n-Peff\n"},{"id":"359727","messageId":"877eiwgoxy.fsf@evledraar.gmail.com","threadId":"48889","inReplyTo":"20181005192816.GE17482@sigill.intra.peff.net","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-10-05T19:42:49Z","receivedAt":"2018-10-05T19:42:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Oct 05 2018, Jeff King wrote:\n\n> On Fri, Oct 05, 2018 at 09:12:09PM +0200, Ævar Arnfjörð Bjarmason wrote:\n>\n>> > I'm not wild about declaring functions inside macros, just because it\n>> > makes tools like ctags like useful (but I have certainly been guilty of\n>> > it myself). I'd also worry that taking \"code\" as a macro parameter might\n>> > not scale (what happens if the code has a comma in it?)\n>>\n>> There's always the option of generating the C code as part of some build\n>> step and carrying around a big C file with various type-safe functions\n>> that only differ in the types they operate on. It can even be committed\n>> to source control.\n>>\n>> That sucks in some ways for sure, but is a lot friendlier for grepping,\n>> ctags etc.\n>\n> Yeah, in a lot of ways the C preprocessor is not great for larger-scale\n> code generation. I was hoping we could get away without having the\n> bodies of these functions as part of the generated bit, though.\n>\n> I think what René showed later in the thread is not too bad in that\n> respect.\n>\n>> I've just barely resisted the urge to include that thread where we were\n>> discussing making the code C++-compiler compatible in the References\n>> header :)\n>\n> Yes. The main thing I would want out of using C++ is type-safe,\n> efficient data structures. IIRC, early versions of C++ were implemented\n> via code generation, and we're basically walking down that same road.\n>\n> I'm not sure where the right cutoff is, though. It's nice to pick up\n> the solution somebody else produced, but requiring a C++ compiler to\n> build Git is a pretty big step[...]\n\nNo comment on whether git should use C++...\n\n> that I imagine will create a lot of new problems. (We're just now\n> allowing C99 -- I don't even want to think about what kind of compiler\n> issues we'll run into on antique systems trying to use C++).\n\n...But just on this point: I was under the impression that this problem\nwas way easier with C++. I.e. reason we're just now using C99 for\nportable C projects is because Microsoft for years refused to put any\neffort into updating their compiler to support newer C versions, while\nkeeping up-to-date with C++, and that this has only recently started\nchanging: https://en.wikipedia.org/wiki/C99#Implementations\n\nMaybe there was some other popular vendor of C/C++ compilers that had\nthe inverse of that story, but I'm not aware of any.\n"},{"id":"359728","messageId":"20181005194412.GA19689@sigill.intra.peff.net","threadId":"48889","inReplyTo":"877eiwgoxy.fsf@evledraar.gmail.com","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-10-05T19:44:13Z","receivedAt":"2018-10-05T19:44:16Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Oct 05, 2018 at 09:42:49PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> > that I imagine will create a lot of new problems. (We're just now\n> > allowing C99 -- I don't even want to think about what kind of compiler\n> > issues we'll run into on antique systems trying to use C++).\n> \n> ...But just on this point: I was under the impression that this problem\n> was way easier with C++. I.e. reason we're just now using C99 for\n> portable C projects is because Microsoft for years refused to put any\n> effort into updating their compiler to support newer C versions, while\n> keeping up-to-date with C++, and that this has only recently started\n> changing: https://en.wikipedia.org/wiki/C99#Implementations\n> \n> Maybe there was some other popular vendor of C/C++ compilers that had\n> the inverse of that story, but I'm not aware of any.\n\nI'd worry about what the C++ story is on AIX, etc.\n\n-Peff\n"},{"id":"360408","messageId":"c141fb44-904f-e8b6-119f-7d2d6bcfd81a@web.de","threadId":"48889","inReplyTo":"20181005194223.GA19428@sigill.intra.peff.net","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2018-10-14T14:29:06Z","receivedAt":"2018-10-14T14:29:28Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 05.10.2018 um 21:42 schrieb Jeff King:\n> On Fri, Oct 05, 2018 at 09:36:28PM +0200, René Scharfe wrote:\n> \n>> Am 05.10.2018 um 21:08 schrieb Jeff King:\n>>> On Fri, Oct 05, 2018 at 08:48:27PM +0200, René Scharfe wrote:\n>>>> +#define DEFINE_SORT(name, type, compare)\t\t\t\t\\\n>>>> +static int compare##_void(const void *one, const void *two)\t\t\\\n>>>> +{\t\t\t\t\t\t\t\t\t\\\n>>>> +\treturn compare(one, two);\t\t\t\t\t\\\n>>>> +}\t\t\t\t\t\t\t\t\t\\\n>>>> +static void name(type base, size_t nmemb)\t\t\t\t\\\n>>>> +{\t\t\t\t\t\t\t\t\t\\\n>>>> +\tconst type dummy = NULL;\t\t\t\t\t\\\n>>>> +\tif (nmemb > 1)\t\t\t\t\t\t\t\\\n>>>> +\t\tqsort(base, nmemb, sizeof(base[0]), compare##_void);\t\\\n>>>> +\telse if (0)\t\t\t\t\t\t\t\\\n>>>> +\t\tcompare(dummy, dummy);\t\t\t\t\t\\\n>>>> +}\n>>>\n>>> I do like that this removes the need to have the code block aspart of\n>>> the macro.\n>>>\n>>> Did you measure to see if there is any runtime impact?\n>>\n>> No, but I wouldn't expect any -- the generated code should be the same\n>> in most cases.\n>>\n>> Here's an example: https://godbolt.org/z/gwXENy.\n> \n> OK, that's good enough for me.\n> \n>> The typed comparison function can be inlined into the one with the void\n>> pointers, though.\n> \n> Right, that makes sense. I suspect it depends on the comparison function\n> being static, but in a DEFINE_SORT() world, they generally could be.\n> \n> So I like this approach.\n\nIt still has some repetition, converted code is a bit longer than the\ncurrent one, and I don't know how to build a Coccinelle rule that would\ndo that conversion.\n\nLooked for a possibility to at least leave QSORT call-sites alone by\nenhancing that macro, but didn't find any. Found a few websites\nshowing off mindblowing macros, thouhg, this one in particular:\n\nhttps://github.com/pfultz2/Cloak/wiki/C-Preprocessor-tricks,-tips,-and-idioms\n\nAnyway, drove the generative approach a bit further, and came up with\nthe new DEFINE_SORT below. I'm unsure about the name; perhaps it should\nbe called DEFINE_SORT_BY_COMPARE_FUNCTION_BODY, but that's a bit long.\nIt handles casts and const attributes behind the scenes and avoids\nrepetition, but looks a bit weird, as it is placed where a function\nsignature would go.\n\nApart from that the macro is simple and doesn't use any tricks or\nadded checks. It just sets up boilerplate functions to offer type-safe\nsorting.\n\ndiffcore-rename.c and refs/packed-backend.c receive special treatment in\nthe patch because their compare functions are used outside of sorting as\nwell. I made them take typed pointers nevertheless and used them from\nDEFINE_SORT; the wrapper generated by that macro is supposed to be\nprivate. Given that such reuse is rare and I think we don't need a way\nto make it public.\n\nWhat do y'all think about this direction?\n\n---\n bisect.c | 8 ++------\n builtin/describe.c | 6 ++----\n builtin/fmt-merge-msg.c | 10 ++++------\n builtin/index-pack.c | 14 ++++----------\n builtin/name-rev.c | 5 ++---\n builtin/pack-objects.c | 7 ++-----\n builtin/remote.c | 6 ++----\n builtin/shortlog.c | 15 ++++++++-------\n commit-graph.c | 6 ++----\n delta-islands.c | 8 +++-----\n diff.c | 8 +++-----\n diffcore-delta.c | 7 ++-----\n diffcore-order.c | 7 ++-----\n diffcore-rename.c | 11 +++++++----\n git-compat-util.h | 15 +++++++++++++++\n help.c | 7 ++-----\n line-log.c | 7 ++-----\n midx.c | 12 ++++--------\n pack-check.c | 6 ++----\n pathspec.c | 8 ++------\n refs/packed-backend.c | 11 ++++++++---\n sha1-array.c | 4 ++--\n sha1-name.c | 4 ++--\n 23 files changed, 84 insertions(+), 108 deletions(-)\n\ndiff --git a/bisect.c b/bisect.c\nindex e8b17cf7e1..25257c2e69 100644\n--- a/bisect.c\n+++ b/bisect.c\n@@ -192,12 +192,8 @@ struct commit_dist {\n \tint distance;\n };\n \n-static int compare_commit_dist(const void *a_, const void *b_)\n+DEFINE_SORT(static, sort_by_commit_dist, struct commit_dist *, a, b)\n {\n-\tstruct commit_dist *a, *b;\n-\n-\ta = (struct commit_dist *)a_;\n-\tb = (struct commit_dist *)b_;\n \tif (a->distance != b->distance)\n \t\treturn b->distance - a->distance; /* desc sort */\n \treturn oidcmp(&a->commit->object.oid, &b->commit->object.oid);\n@@ -223,7 +219,7 @@ static struct commit_list *best_bisection_sorted(struct commit_list *list, int n\n \t\tarray[cnt].distance = distance;\n \t\tcnt++;\n \t}\n-\tQSORT(array, cnt, compare_commit_dist);\n+\tsort_by_commit_dist(array, cnt);\n \tfor (p = list, i = 0; i < cnt; i++) {\n \t\tstruct object *obj = &(array[i].commit->object);\n \ndiff --git a/builtin/describe.c b/builtin/describe.c\nindex 22c0541da5..44eaadf0a0 100644\n--- a/builtin/describe.c\n+++ b/builtin/describe.c\n@@ -220,10 +220,8 @@ struct possible_tag {\n \tunsigned flag_within;\n };\n \n-static int compare_pt(const void *a_, const void *b_)\n+DEFINE_SORT(static, sort_pt, struct possible_tag *, a, b)\n {\n-\tstruct possible_tag *a = (struct possible_tag *)a_;\n-\tstruct possible_tag *b = (struct possible_tag *)b_;\n \tif (a->depth != b->depth)\n \t\treturn a->depth - b->depth;\n \tif (a->found_order != b->found_order)\n@@ -410,7 +408,7 @@ static void describe_commit(struct object_id *oid, struct strbuf *dst)\n \t\t\t oid_to_hex(cmit_oid));\n \t}\n \n-\tQSORT(all_matches, match_cnt, compare_pt);\n+\tsort_pt(all_matches, match_cnt);\n \n \tif (gave_up_on) {\n \t\tcommit_list_insert_by_date(gave_up_on, &list);\ndiff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c\nindex 59a40342b6..2c84349a1b 100644\n--- a/builtin/fmt-merge-msg.c\n+++ b/builtin/fmt-merge-msg.c\n@@ -268,9 +268,9 @@ static void record_person(int which, struct string_list *people,\n \tunuse_commit_buffer(commit, buffer);\n }\n \n-static int cmp_string_list_util_as_integral(const void *a_, const void *b_)\n+DEFINE_SORT(static, sort_string_list_by_util_as_integral,\n+\t struct string_list_item *, a, b)\n {\n-\tconst struct string_list_item *a = a_, *b = b_;\n \treturn util_as_integral(b) - util_as_integral(a);\n }\n \n@@ -319,10 +319,8 @@ static void add_people_info(struct strbuf *out,\n \t\t\t struct string_list *authors,\n \t\t\t struct string_list *committers)\n {\n-\tQSORT(authors->items, authors->nr,\n-\t cmp_string_list_util_as_integral);\n-\tQSORT(committers->items, committers->nr,\n-\t cmp_string_list_util_as_integral);\n+\tsort_string_list_by_util_as_integral(authors->items, authors->nr);\n+\tsort_string_list_by_util_as_integral(committers->items, committers->nr);\n \n \tcredit_people(out, authors, 'a');\n \tcredit_people(out, committers, 'c');\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 2004e25da2..d3e91afb50 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1066,21 +1066,15 @@ static void find_unresolved_deltas(struct base_data *base)\n \t}\n }\n \n-static int compare_ofs_delta_entry(const void *a, const void *b)\n+DEFINE_SORT(static, sort_by_offset, struct ofs_delta_entry *, delta_a, delta_b)\n {\n-\tconst struct ofs_delta_entry *delta_a = a;\n-\tconst struct ofs_delta_entry *delta_b = b;\n-\n \treturn delta_a->offset < delta_b->offset ? -1 :\n \t delta_a->offset > delta_b->offset ? 1 :\n \t 0;\n }\n \n-static int compare_ref_delta_entry(const void *a, const void *b)\n+DEFINE_SORT(static, sort_by_oid, struct ref_delta_entry *, delta_a, delta_b)\n {\n-\tconst struct ref_delta_entry *delta_a = a;\n-\tconst struct ref_delta_entry *delta_b = b;\n-\n \treturn oidcmp(&delta_a->oid, &delta_b->oid);\n }\n \n@@ -1206,8 +1200,8 @@ static void resolve_deltas(void)\n \t\treturn;\n \n \t/* Sort deltas by base SHA1/offset for fast searching */\n-\tQSORT(ofs_deltas, nr_ofs_deltas, compare_ofs_delta_entry);\n-\tQSORT(ref_deltas, nr_ref_deltas, compare_ref_delta_entry);\n+\tsort_by_offset(ofs_deltas, nr_ofs_deltas);\n+\tsort_by_oid(ref_deltas, nr_ref_deltas);\n \n \tif (verbose || show_resolving_progress)\n \t\tprogress = start_progress(_(\"Resolving deltas\"),\ndiff --git a/builtin/name-rev.c b/builtin/name-rev.c\nindex f1cb45c227..e8d2da1101 100644\n--- a/builtin/name-rev.c\n+++ b/builtin/name-rev.c\n@@ -196,9 +196,8 @@ static void add_to_tip_table(const struct object_id *oid, const char *refname,\n \ttip_table.sorted = 0;\n }\n \n-static int tipcmp(const void *a_, const void *b_)\n+DEFINE_SORT(static, sort_by_oid, struct tip_table_entry *, a, b)\n {\n-\tconst struct tip_table_entry *a = a_, *b = b_;\n \treturn oidcmp(&a->oid, &b->oid);\n }\n \n@@ -293,7 +292,7 @@ static const char *get_exact_ref_match(const struct object *o)\n \t\treturn NULL;\n \n \tif (!tip_table.sorted) {\n-\t\tQSORT(tip_table.table, tip_table.nr, tipcmp);\n+\t\tsort_by_oid(tip_table.table, tip_table.nr);\n \t\ttip_table.sorted = 1;\n \t}\n \ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex e6316d294d..fc37ae186f 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -2868,11 +2868,8 @@ static void mark_in_pack_object(struct object *object, struct packed_git *p, str\n * Compare the objects in the offset order, in order to emulate the\n * \"git rev-list --objects\" output that produced the pack originally.\n */\n-static int ofscmp(const void *a_, const void *b_)\n+DEFINE_SORT(static, sort_in_pack_objects, struct in_pack_object *, a, b)\n {\n-\tstruct in_pack_object *a = (struct in_pack_object *)a_;\n-\tstruct in_pack_object *b = (struct in_pack_object *)b_;\n-\n \tif (a->offset < b->offset)\n \t\treturn -1;\n \telse if (a->offset > b->offset)\n@@ -2912,7 +2909,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)\n \t}\n \n \tif (in_pack.nr) {\n-\t\tQSORT(in_pack.array, in_pack.nr, ofscmp);\n+\t\tsort_in_pack_objects(in_pack.array, in_pack.nr);\n \t\tfor (i = 0; i < in_pack.nr; i++) {\n \t\t\tstruct object *o = in_pack.array[i].object;\n \t\t\tadd_object_entry(&o->oid, o->type, \"\", 0);\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex f7edf7f2cb..0f9e70d445 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -1011,10 +1011,8 @@ static int add_push_to_show_info(struct string_list_item *push_item, void *cb_da\n * Sorting comparison for a string list that has push_info\n * structs in its util field\n */\n-static int cmp_string_with_push(const void *va, const void *vb)\n+DEFINE_SORT(static, sort_by_push_info, struct string_list_item *, a, b)\n {\n-\tconst struct string_list_item *a = va;\n-\tconst struct string_list_item *b = vb;\n \tconst struct push_info *a_push = a->util;\n \tconst struct push_info *b_push = b->util;\n \tint cmp = strcmp(a->string, b->string);\n@@ -1216,7 +1214,7 @@ static int show(int argc, const char **argv)\n \n \t\tinfo.width = info.width2 = 0;\n \t\tfor_each_string_list(&states.push, add_push_to_show_info, &info);\n-\t\tQSORT(info.list->items, info.list->nr, cmp_string_with_push);\n+\t\tsort_by_push_info(info.list->items, info.list->nr);\n \t\tif (info.list->nr)\n \t\t\tprintf_ln(Q_(\" Local ref configured for 'git push'%s:\",\n \t\t\t\t \" Local refs configured for 'git push'%s:\",\ndiff --git a/builtin/shortlog.c b/builtin/shortlog.c\nindex 3898a2c9c4..ec1ace6ed5 100644\n--- a/builtin/shortlog.c\n+++ b/builtin/shortlog.c\n@@ -29,15 +29,13 @@ static char const * const shortlog_usage[] = {\n */\n #define UTIL_TO_INT(x) ((intptr_t)(x)->util)\n \n-static int compare_by_counter(const void *a1, const void *a2)\n+DEFINE_SORT(static, sort_by_counter, struct string_list_item *, i1, i2)\n {\n-\tconst struct string_list_item *i1 = a1, *i2 = a2;\n \treturn UTIL_TO_INT(i2) - UTIL_TO_INT(i1);\n }\n \n-static int compare_by_list(const void *a1, const void *a2)\n+DEFINE_SORT(static, sort_by_list, struct string_list_item *, i1, i2)\n {\n-\tconst struct string_list_item *i1 = a1, *i2 = a2;\n \tconst struct string_list *l1 = i1->util, *l2 = i2->util;\n \n \tif (l1->nr < l2->nr)\n@@ -338,9 +336,12 @@ void shortlog_output(struct shortlog *log)\n \tint i, j;\n \tstruct strbuf sb = STRBUF_INIT;\n \n-\tif (log->sort_by_number)\n-\t\tQSORT(log->list.items, log->list.nr,\n-\t\t log->summary ? compare_by_counter : compare_by_list);\n+\tif (log->sort_by_number) {\n+\t\tif (log->summary)\n+\t\t\tsort_by_counter(log->list.items, log->list.nr);\n+\t\telse\n+\t\t\tsort_by_list(log->list.items, log->list.nr);\n+\t}\n \tfor (i = 0; i < log->list.nr; i++) {\n \t\tconst struct string_list_item *item = &log->list.items[i];\n \t\tif (log->summary) {\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 7f4519ec3b..fc2fbf22b4 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -550,10 +550,8 @@ static void write_graph_chunk_large_edges(struct hashfile *f,\n \t}\n }\n \n-static int commit_compare(const void *_a, const void *_b)\n+DEFINE_SORT(static, sort_oids, struct object_id *, a, b)\n {\n-\tconst struct object_id *a = (const struct object_id *)_a;\n-\tconst struct object_id *b = (const struct object_id *)_b;\n \treturn oidcmp(a, b);\n }\n \n@@ -780,7 +778,7 @@ void write_commit_graph(const char *obj_dir,\n \n \tclose_reachable(&oids);\n \n-\tQSORT(oids.list, oids.nr, commit_compare);\n+\tsort_oids(oids.list, oids.nr);\n \n \tcount_distinct = 1;\n \tfor (i = 1; i < oids.nr; i++) {\ndiff --git a/delta-islands.c b/delta-islands.c\nindex 8e5018e406..29232831d4 100644\n--- a/delta-islands.c\n+++ b/delta-islands.c\n@@ -229,11 +229,9 @@ struct tree_islands_todo {\n \tunsigned int depth;\n };\n \n-static int tree_depth_compare(const void *a, const void *b)\n+DEFINE_SORT(static, sort_by_tree_depth, struct tree_islands_todo *,\n+\t todo_a, todo_b)\n {\n-\tconst struct tree_islands_todo *todo_a = a;\n-\tconst struct tree_islands_todo *todo_b = b;\n-\n \treturn todo_a->depth - todo_b->depth;\n }\n \n@@ -262,7 +260,7 @@ void resolve_tree_islands(int progress, struct packing_data *to_pack)\n \t\t\tnr++;\n \t\t}\n \t}\n-\tQSORT(todo, nr, tree_depth_compare);\n+\tsort_by_tree_depth(todo, nr);\n \n \tif (progress)\n \t\tprogress_state = start_progress(_(\"Propagating island marks\"), nr);\ndiff --git a/diff.c b/diff.c\nindex f0c7557b40..ecb1961173 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2924,10 +2924,8 @@ static long gather_dirstat(struct diff_options *opt, struct dirstat_dir *dir,\n \treturn sum_changes;\n }\n \n-static int dirstat_compare(const void *_a, const void *_b)\n+DEFINE_SORT(static, sort_by_name, struct dirstat_file *, a, b)\n {\n-\tconst struct dirstat_file *a = _a;\n-\tconst struct dirstat_file *b = _b;\n \treturn strcmp(a->name, b->name);\n }\n \n@@ -3021,7 +3019,7 @@ static void show_dirstat(struct diff_options *options)\n \t\treturn;\n \n \t/* Show all directories with more than x% of the changes */\n-\tQSORT(dir.files, dir.nr, dirstat_compare);\n+\tsort_by_name(dir.files, dir.nr);\n \tgather_dirstat(options, &dir, changed, \"\", 0);\n }\n \n@@ -3065,7 +3063,7 @@ static void show_dirstat_by_line(struct diffstat_t *data, struct diff_options *o\n \t\treturn;\n \n \t/* Show all directories with more than x% of the changes */\n-\tQSORT(dir.files, dir.nr, dirstat_compare);\n+\tsort_by_name(dir.files, dir.nr);\n \tgather_dirstat(options, &dir, changed, \"\", 0);\n }\n \ndiff --git a/diffcore-delta.c b/diffcore-delta.c\nindex c83d45a047..53ed2d5766 100644\n--- a/diffcore-delta.c\n+++ b/diffcore-delta.c\n@@ -107,11 +107,8 @@ static struct spanhash_top *add_spanhash(struct spanhash_top *top,\n \t}\n }\n \n-static int spanhash_cmp(const void *a_, const void *b_)\n+DEFINE_SORT(static, sort_by_spanhash, struct spanhash *, a, b)\n {\n-\tconst struct spanhash *a = a_;\n-\tconst struct spanhash *b = b_;\n-\n \t/* A count of zero compares at the end.. */\n \tif (!a->cnt)\n \t\treturn !b->cnt ? 0 : 1;\n@@ -158,7 +155,7 @@ static struct spanhash_top *hash_chars(struct diff_filespec *one)\n \t\tn = 0;\n \t\taccum1 = accum2 = 0;\n \t}\n-\tQSORT(hash->data, 1ul << hash->alloc_log2, spanhash_cmp);\n+\tsort_by_spanhash(hash->data, 1ul << hash->alloc_log2);\n \treturn hash;\n }\n \ndiff --git a/diffcore-order.c b/diffcore-order.c\nindex 19e73311f9..e8abd84242 100644\n--- a/diffcore-order.c\n+++ b/diffcore-order.c\n@@ -78,11 +78,8 @@ static int match_order(const char *path)\n \treturn order_cnt;\n }\n \n-static int compare_objs_order(const void *a_, const void *b_)\n+DEFINE_SORT(static, sort_by_order, struct obj_order *, a, b)\n {\n-\tstruct obj_order const *a, *b;\n-\ta = (struct obj_order const *)a_;\n-\tb = (struct obj_order const *)b_;\n \tif (a->order != b->order)\n \t\treturn a->order - b->order;\n \treturn a->orig_order - b->orig_order;\n@@ -101,7 +98,7 @@ void order_objects(const char *orderfile, obj_path_fn_t obj_path,\n \t\tobjs[i].orig_order = i;\n \t\tobjs[i].order = match_order(obj_path(objs[i].obj));\n \t}\n-\tQSORT(objs, nr, compare_objs_order);\n+\tsort_by_order(objs, nr);\n }\n \n static const char *pair_pathtwo(void *obj)\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex daddd9b28a..0e256377db 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -234,10 +234,8 @@ static void record_rename_pair(int dst_index, int src_index, int score)\n * We sort the rename similarity matrix with the score, in descending\n * order (the most similar first).\n */\n-static int score_compare(const void *a_, const void *b_)\n+static int score_compare(const struct diff_score *a, const struct diff_score *b)\n {\n-\tconst struct diff_score *a = a_, *b = b_;\n-\n \t/* sink the unused ones to the bottom */\n \tif (a->dst < 0)\n \t\treturn (0 <= b->dst);\n@@ -250,6 +248,11 @@ static int score_compare(const void *a_, const void *b_)\n \treturn b->score - a->score;\n }\n \n+DEFINE_SORT(static, sort_by_score, struct diff_score *, a, b)\n+{\n+\treturn score_compare(a, b);\n+}\n+\n struct file_similarity {\n \tstruct hashmap_entry entry;\n \tint index;\n@@ -576,7 +579,7 @@ void diffcore_rename(struct diff_options *options)\n \tstop_progress(&progress);\n \n \t/* cost matrix sorted by most to least similar pair */\n-\tQSORT(mx, dst_cnt * NUM_CANDIDATE_PER_DST, score_compare);\n+\tsort_by_score(mx, dst_cnt * NUM_CANDIDATE_PER_DST);\n \n \trename_count += find_renames(mx, dst_cnt, minimum_score, 0);\n \tif (detect_rename == DIFF_DETECT_COPY)\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 5f2e90932f..491230fc57 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -1066,6 +1066,21 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n \t\tqsort(base, nmemb, size, compar);\n }\n \n+#define DECLARE_SORT(scope, name, elemtype) \\\n+scope void name(elemtype, size_t)\n+\n+#define DEFINE_SORT(scope, name, elemtype, one, two)\t\t\t\\\n+static int name##_compare(const elemtype, const elemtype);\t\t\\\n+static int name##_compare_void(const void *a, const void *b)\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\treturn name##_compare(a, b);\t\t\t\t\t\\\n+}\t\t\t\t\t\t\t\t\t\\\n+scope void name(elemtype base, size_t nmemb)\t\t\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\tQSORT(base, nmemb, name##_compare_void);\t\t\t\\\n+}\t\t\t\t\t\t\t\t\t\\\n+static int name##_compare(const elemtype one, const elemtype two)\n+\n #ifndef HAVE_ISO_QSORT_S\n int git_qsort_s(void *base, size_t nmemb, size_t size,\n \t\tint (*compar)(const void *, const void *, void *), void *ctx);\ndiff --git a/help.c b/help.c\nindex 96f6d221ed..c50dd58943 100644\n--- a/help.c\n+++ b/help.c\n@@ -90,11 +90,8 @@ static void print_command_list(const struct cmdname_help *cmds,\n \t}\n }\n \n-static int cmd_name_cmp(const void *elem1, const void *elem2)\n+DEFINE_SORT(static, sort_help_by_name, struct cmdname_help *, e1, e2)\n {\n-\tconst struct cmdname_help *e1 = elem1;\n-\tconst struct cmdname_help *e2 = elem2;\n-\n \treturn strcmp(e1->name, e2->name);\n }\n \n@@ -114,7 +111,7 @@ static void print_cmd_by_category(const struct category_description *catdesc)\n \t\tif (longest < strlen(cmds[i].name))\n \t\t\tlongest = strlen(cmds[i].name);\n \t}\n-\tQSORT(cmds, nr, cmd_name_cmp);\n+\tsort_help_by_name(cmds, nr);\n \n \tfor (i = 0; catdesc[i].desc; i++) {\n \t\tuint32_t mask = catdesc[i].category;\ndiff --git a/line-log.c b/line-log.c\nindex 72a5fed661..e240959166 100644\n--- a/line-log.c\n+++ b/line-log.c\n@@ -72,11 +72,8 @@ void range_set_append(struct range_set *rs, long a, long b)\n \trange_set_append_unsafe(rs, a, b);\n }\n \n-static int range_cmp(const void *_r, const void *_s)\n+DEFINE_SORT(static, sort_by_start, struct range *, r, s)\n {\n-\tconst struct range *r = _r;\n-\tconst struct range *s = _s;\n-\n \t/* this could be simply 'return r.start-s.start', but for the types */\n \tif (r->start == s->start)\n \t\treturn 0;\n@@ -113,7 +110,7 @@ void sort_and_merge_range_set(struct range_set *rs)\n \tunsigned int i;\n \tunsigned int o = 0; /* output cursor */\n \n-\tQSORT(rs->ranges, rs->nr, range_cmp);\n+\tsort_by_start(rs->ranges, rs->nr);\n \n \tfor (i = 0; i < rs->nr; i++) {\n \t\tif (rs->ranges[i].start == rs->ranges[i].end)\ndiff --git a/midx.c b/midx.c\nindex 713d6f9dde..4407db7949 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -419,10 +419,8 @@ struct pack_pair {\n \tchar *pack_name;\n };\n \n-static int pack_pair_compare(const void *_a, const void *_b)\n+DEFINE_SORT(static, sort_by_pack_name, struct pack_pair *, a, b)\n {\n-\tstruct pack_pair *a = (struct pack_pair *)_a;\n-\tstruct pack_pair *b = (struct pack_pair *)_b;\n \treturn strcmp(a->pack_name, b->pack_name);\n }\n \n@@ -438,7 +436,7 @@ static void sort_packs_by_name(char **pack_names, uint32_t nr_packs, uint32_t *p\n \t\tpairs[i].pack_name = pack_names[i];\n \t}\n \n-\tQSORT(pairs, nr_packs, pack_pair_compare);\n+\tsort_by_pack_name(pairs, nr_packs);\n \n \tfor (i = 0; i < nr_packs; i++) {\n \t\tpack_names[i] = pairs[i].pack_name;\n@@ -455,10 +453,8 @@ struct pack_midx_entry {\n \tuint64_t offset;\n };\n \n-static int midx_oid_compare(const void *_a, const void *_b)\n+DEFINE_SORT(static, sort_midx, struct pack_midx_entry *, a, b)\n {\n-\tconst struct pack_midx_entry *a = (const struct pack_midx_entry *)_a;\n-\tconst struct pack_midx_entry *b = (const struct pack_midx_entry *)_b;\n \tint cmp = oidcmp(&a->oid, &b->oid);\n \n \tif (cmp)\n@@ -573,7 +569,7 @@ static struct pack_midx_entry *get_sorted_entries(struct multi_pack_index *m,\n \t\t\t}\n \t\t}\n \n-\t\tQSORT(entries_by_fanout, nr_fanout, midx_oid_compare);\n+\t\tsort_midx(entries_by_fanout, nr_fanout);\n \n \t\t/*\n \t\t * The batch is now sorted by OID and then mtime (descending).\ndiff --git a/pack-check.c b/pack-check.c\nindex fa5f0ff8fa..d4825f80e0 100644\n--- a/pack-check.c\n+++ b/pack-check.c\n@@ -15,10 +15,8 @@ struct idx_entry {\n \tunsigned int nr;\n };\n \n-static int compare_entries(const void *e1, const void *e2)\n+DEFINE_SORT(static, sort_by_offset, struct idx_entry *, entry1, entry2)\n {\n-\tconst struct idx_entry *entry1 = e1;\n-\tconst struct idx_entry *entry2 = e2;\n \tif (entry1->offset < entry2->offset)\n \t\treturn -1;\n \tif (entry1->offset > entry2->offset)\n@@ -102,7 +100,7 @@ static int verify_packfile(struct packed_git *p,\n \t\tentries[i].offset = nth_packed_object_offset(p, i);\n \t\tentries[i].nr = i;\n \t}\n-\tQSORT(entries, nr_objects, compare_entries);\n+\tsort_by_offset(entries, nr_objects);\n \n \tfor (i = 0; i < nr_objects; i++) {\n \t\tvoid *data;\ndiff --git a/pathspec.c b/pathspec.c\nindex 6f005996fd..b559c3d783 100644\n--- a/pathspec.c\n+++ b/pathspec.c\n@@ -490,12 +490,8 @@ static void init_pathspec_item(struct pathspec_item *item, unsigned flags,\n \t}\n }\n \n-static int pathspec_item_cmp(const void *a_, const void *b_)\n+DEFINE_SORT(static, sort_by_match, struct pathspec_item *, a, b)\n {\n-\tstruct pathspec_item *a, *b;\n-\n-\ta = (struct pathspec_item *)a_;\n-\tb = (struct pathspec_item *)b_;\n \treturn strcmp(a->match, b->match);\n }\n \n@@ -610,7 +606,7 @@ void parse_pathspec(struct pathspec *pathspec,\n \tif (pathspec->magic & PATHSPEC_MAXDEPTH) {\n \t\tif (flags & PATHSPEC_KEEP_ORDER)\n \t\t\tBUG(\"PATHSPEC_MAXDEPTH_VALID and PATHSPEC_KEEP_ORDER are incompatible\");\n-\t\tQSORT(pathspec->items, pathspec->nr, pathspec_item_cmp);\n+\t\tsort_by_match(pathspec->items, pathspec->nr);\n \t}\n }\n \ndiff --git a/refs/packed-backend.c b/refs/packed-backend.c\nindex 74e2996e93..1339ffd8c3 100644\n--- a/refs/packed-backend.c\n+++ b/refs/packed-backend.c\n@@ -271,9 +271,9 @@ struct snapshot_record {\n \tsize_t len;\n };\n \n-static int cmp_packed_ref_records(const void *v1, const void *v2)\n+static int cmp_packed_ref_records(const struct snapshot_record *e1,\n+\t\t\t\t const struct snapshot_record *e2)\n {\n-\tconst struct snapshot_record *e1 = v1, *e2 = v2;\n \tconst char *r1 = e1->start + GIT_SHA1_HEXSZ + 1;\n \tconst char *r2 = e2->start + GIT_SHA1_HEXSZ + 1;\n \n@@ -291,6 +291,11 @@ static int cmp_packed_ref_records(const void *v1, const void *v2)\n \t}\n }\n \n+DEFINE_SORT(static, sort_packed_ref_records, struct snapshot_record *, a, b)\n+{\n+\treturn cmp_packed_ref_records(a, b);\n+}\n+\n /*\n * Compare a snapshot record at `rec` to the specified NUL-terminated\n * refname.\n@@ -380,7 +385,7 @@ static void sort_snapshot(struct snapshot *snapshot)\n \t\tgoto cleanup;\n \n \t/* We need to sort the memory. First we sort the records array: */\n-\tQSORT(records, nr, cmp_packed_ref_records);\n+\tsort_packed_ref_records(records, nr);\n \n \t/*\n \t * Allocate a new chunk of memory, and copy the old memory to\ndiff --git a/sha1-array.c b/sha1-array.c\nindex b94e0ec0f5..2dcaafe9dc 100644\n--- a/sha1-array.c\n+++ b/sha1-array.c\n@@ -9,14 +9,14 @@ void oid_array_append(struct oid_array *array, const struct object_id *oid)\n \tarray->sorted = 0;\n }\n \n-static int void_hashcmp(const void *a, const void *b)\n+DEFINE_SORT(static, sort_by_oid, struct object_id *, a, b)\n {\n \treturn oidcmp(a, b);\n }\n \n static void oid_array_sort(struct oid_array *array)\n {\n-\tQSORT(array->oid, array->nr, void_hashcmp);\n+\tsort_by_oid(array->oid, array->nr);\n \tarray->sorted = 1;\n }\n \ndiff --git a/sha1-name.c b/sha1-name.c\nindex faa60f69e3..39fc68d3c6 100644\n--- a/sha1-name.c\n+++ b/sha1-name.c\n@@ -412,7 +412,7 @@ static int collect_ambiguous(const struct object_id *oid, void *data)\n \treturn 0;\n }\n \n-static int sort_ambiguous(const void *a, const void *b)\n+DEFINE_SORT(static, sort_ambiguous, struct object_id *, a, b)\n {\n \tint a_type = oid_object_info(the_repository, a, NULL);\n \tint b_type = oid_object_info(the_repository, b, NULL);\n@@ -486,7 +486,7 @@ static int get_short_oid(const char *name, int len, struct object_id *oid,\n \n \t\tadvise(_(\"The candidates are:\"));\n \t\tfor_each_abbrev(ds.hex_pfx, collect_ambiguous, &collect);\n-\t\tQSORT(collect.oid, collect.nr, sort_ambiguous);\n+\t\tsort_ambiguous(collect.oid, collect.nr);\n \n \t\tif (oid_array_for_each(&collect, show_ambiguous_object, &ds))\n \t\t\tBUG(\"show_ambiguous_object shouldn't return non-zero\");\n-- \n2.19.1\n"},{"id":"360537","messageId":"0cd89b69-4b65-f961-a025-c74851f8c999@gmail.com","threadId":"48889","inReplyTo":"c141fb44-904f-e8b6-119f-7d2d6bcfd81a@web.de","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2018-10-15T15:31:12Z","receivedAt":"2018-10-15T15:31:18Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 10/14/2018 10:29 AM, René Scharfe wrote:\n> It still has some repetition, converted code is a bit longer than the\n> current one, and I don't know how to build a Coccinelle rule that would\n> do that conversion.\n>\n> Looked for a possibility to at least leave QSORT call-sites alone by\n> enhancing that macro, but didn't find any. Found a few websites\n> showing off mindblowing macros, thouhg, this one in particular:\n>\n> https://github.com/pfultz2/Cloak/wiki/C-Preprocessor-tricks,-tips,-and-idioms\n>\n> Anyway, drove the generative approach a bit further, and came up with\n> the new DEFINE_SORT below. I'm unsure about the name; perhaps it should\n> be called DEFINE_SORT_BY_COMPARE_FUNCTION_BODY, but that's a bit long.\n> It handles casts and const attributes behind the scenes and avoids\n> repetition, but looks a bit weird, as it is placed where a function\n> signature would go.\n>\n> Apart from that the macro is simple and doesn't use any tricks or\n> added checks. It just sets up boilerplate functions to offer type-safe\n> sorting.\n>\n> diffcore-rename.c and refs/packed-backend.c receive special treatment in\n> the patch because their compare functions are used outside of sorting as\n> well. I made them take typed pointers nevertheless and used them from\n> DEFINE_SORT; the wrapper generated by that macro is supposed to be\n> private. Given that such reuse is rare and I think we don't need a way\n> to make it public.\n>\n> What do y'all think about this direction?\n>\n> ---\n[snip]\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 5f2e90932f..491230fc57 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -1066,6 +1066,21 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n> \t\tqsort(base, nmemb, size, compar);\n> }\n> \n> +#define DECLARE_SORT(scope, name, elemtype) \\\n> +scope void name(elemtype, size_t)\n> +\n> +#define DEFINE_SORT(scope, name, elemtype, one, two)\t\t\t\\\n> +static int name##_compare(const elemtype, const elemtype);\t\t\\\n> +static int name##_compare_void(const void *a, const void *b)\t\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\treturn name##_compare(a, b);\t\t\t\t\t\\\n> +}\t\t\t\t\t\t\t\t\t\\\n> +scope void name(elemtype base, size_t nmemb)\t\t\t\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\tQSORT(base, nmemb, name##_compare_void);\t\t\t\\\n> +}\t\t\t\t\t\t\t\t\t\\\n> +static int name##_compare(const elemtype one, const elemtype two)\n> +\n\nSince you were worried about the \"private\" name of the compare function, \nmaybe split this macro into two: DEFINE_COMPARE and DEFINE_SORT. Then, \nif someone wants direct access to the compare function, they could use \nthe DEFINE_COMPARE to ensure the typing is correct, and use QSORT as \nnormal with name##_compare_void.\n\nAs I think about this, I think this is less of a problem than is worth \nthis split. The commit-slab definitions generate a lot of methods using \nthe \"name##\" convention, so perhaps we should just trust developers \nusing the macros to look up the macro definition or similar examples. In \nthat sense, including a conversion that consumes the compare function \ndirectly can be a signpost for future callers.\n\nI would say that maybe the times where you need to do something special \nshould be pulled out into their own patches, so we can call attention to \nthem directly.\n\n[snip]\n> diff --git a/midx.c b/midx.c\n> index 713d6f9dde..4407db7949 100644\n> --- a/midx.c\n> +++ b/midx.c\n> @@ -419,10 +419,8 @@ struct pack_pair {\n> \tchar *pack_name;\n> };\n> \n> -static int pack_pair_compare(const void *_a, const void *_b)\n> +DEFINE_SORT(static, sort_by_pack_name, struct pack_pair *, a, b)\n> {\n> -\tstruct pack_pair *a = (struct pack_pair *)_a;\n> -\tstruct pack_pair *b = (struct pack_pair *)_b;\n> \treturn strcmp(a->pack_name, b->pack_name);\n> }\n> \n> @@ -438,7 +436,7 @@ static void sort_packs_by_name(char **pack_names, uint32_t nr_packs, uint32_t *p\n> \t\tpairs[i].pack_name = pack_names[i];\n> \t}\n> \n> -\tQSORT(pairs, nr_packs, pack_pair_compare);\n> +\tsort_by_pack_name(pairs, nr_packs);\n\nI like this \"sort_by_\" convention..\n\n> \n> \tfor (i = 0; i < nr_packs; i++) {\n> \t\tpack_names[i] = pairs[i].pack_name;\n> @@ -455,10 +453,8 @@ struct pack_midx_entry {\n> \tuint64_t offset;\n> };\n> \n> -static int midx_oid_compare(const void *_a, const void *_b)\n> +DEFINE_SORT(static, sort_midx, struct pack_midx_entry *, a, b)\n> {\n> -\tconst struct pack_midx_entry *a = (const struct pack_midx_entry *)_a;\n> -\tconst struct pack_midx_entry *b = (const struct pack_midx_entry *)_b;\n> \tint cmp = oidcmp(&a->oid, &b->oid);\n> \n> \tif (cmp)\n> @@ -573,7 +569,7 @@ static struct pack_midx_entry *get_sorted_entries(struct multi_pack_index *m,\n> \t\t\t}\n> \t\t}\n> \n> -\t\tQSORT(entries_by_fanout, nr_fanout, midx_oid_compare);\n> +\t\tsort_midx(entries_by_fanout, nr_fanout);\n\n...but it isn't followed here. Perhaps \"sort_by_oid\"?\n\nThanks,\n-Stolee\n"},{"id":"360542","messageId":"cd1b0e5f-94b0-4905-aee8-edec00dc0a49@web.de","threadId":"48889","inReplyTo":"0cd89b69-4b65-f961-a025-c74851f8c999@gmail.com","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2018-10-15T16:26:40Z","receivedAt":"2018-10-15T16:27:03Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 15.10.2018 um 17:31 schrieb Derrick Stolee:\n> On 10/14/2018 10:29 AM, René Scharfe wrote:\n>> diff --git a/git-compat-util.h b/git-compat-util.h\n>> index 5f2e90932f..491230fc57 100644\n>> --- a/git-compat-util.h\n>> +++ b/git-compat-util.h\n>> @@ -1066,6 +1066,21 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n>> \t\tqsort(base, nmemb, size, compar);\n>> }\n>> \n>> +#define DECLARE_SORT(scope, name, elemtype) \\\n>> +scope void name(elemtype, size_t)\n>> +\n>> +#define DEFINE_SORT(scope, name, elemtype, one, two)\t\t\t\\\n>> +static int name##_compare(const elemtype, const elemtype);\t\t\\\n>> +static int name##_compare_void(const void *a, const void *b)\t\t\\\n>> +{\t\t\t\t\t\t\t\t\t\\\n>> +\treturn name##_compare(a, b);\t\t\t\t\t\\\n>> +}\t\t\t\t\t\t\t\t\t\\\n>> +scope void name(elemtype base, size_t nmemb)\t\t\t\t\\\n>> +{\t\t\t\t\t\t\t\t\t\\\n>> +\tQSORT(base, nmemb, name##_compare_void);\t\t\t\\\n>> +}\t\t\t\t\t\t\t\t\t\\\n>> +static int name##_compare(const elemtype one, const elemtype two)\n>> +\n> \n> Since you were worried about the \"private\" name of the compare function, \n> maybe split this macro into two: DEFINE_COMPARE and DEFINE_SORT. Then, \n> if someone wants direct access to the compare function, they could use \n> the DEFINE_COMPARE to ensure the typing is correct, and use QSORT as \n> normal with name##_compare_void.\n\nThe pointers are converted to const void * somewhere along the way from\nqsort() to compare function. Splitting the macro would require type\ncheck tricks to make sure the types of the compare function matches the\narray to be sorted. Letting a single macro bake it all into a layer\ncake of generated functions is a lot simpler.\n\n> As I think about this, I think this is less of a problem than is worth \n> this split. The commit-slab definitions generate a lot of methods using \n> the \"name##\" convention, so perhaps we should just trust developers \n> using the macros to look up the macro definition or similar examples. In \n> that sense, including a conversion that consumes the compare function \n> directly can be a signpost for future callers.\n\nUsing the generated compare function name directly is a bit awkward; e.g.\nin the two example cases it would be sort_by_score_compare() and\nsort_packed_ref_records_compare(). Defining the real compare function\nthe usual way (with a proper name) and having the DEFINE_SORT block call\nit is a bit more repetitive, but clean and understandable IMHO.\n\nWe also could just leave complicated cases alone..\n\n> I would say that maybe the times where you need to do something special \n> should be pulled out into their own patches, so we can call attention to \n> them directly.\n\nRight; this patch was just a sketch.\n\n> I like this \"sort_by_\" convention..\n> \n>> \n>> \tfor (i = 0; i < nr_packs; i++) {\n>> \t\tpack_names[i] = pairs[i].pack_name;\n>> @@ -455,10 +453,8 @@ struct pack_midx_entry {\n>> \tuint64_t offset;\n>> };\n>> \n>> -static int midx_oid_compare(const void *_a, const void *_b)\n>> +DEFINE_SORT(static, sort_midx, struct pack_midx_entry *, a, b)\n>> {\n>> -\tconst struct pack_midx_entry *a = (const struct pack_midx_entry *)_a;\n>> -\tconst struct pack_midx_entry *b = (const struct pack_midx_entry *)_b;\n>> \tint cmp = oidcmp(&a->oid, &b->oid);\n>> \n>> \tif (cmp)\n>> @@ -573,7 +569,7 @@ static struct pack_midx_entry *get_sorted_entries(struct multi_pack_index *m,\n>> \t\t\t}\n>> \t\t}\n>> \n>> -\t\tQSORT(entries_by_fanout, nr_fanout, midx_oid_compare);\n>> +\t\tsort_midx(entries_by_fanout, nr_fanout);\n> \n> ...but it isn't followed here. Perhaps \"sort_by_oid\"?\n\nThat function sorts by oid, pack_mtime, and pack_int_id, but including\nall these fields in the name is a bit unwieldy. Being unspecific by\ncalling it sort_midx() was the lazy way out. Mentioning only oid is a\nbit misleading. Perhaps sort_by_oid_etc()?\n\nRené\n\n"},{"id":"360691","messageId":"xmqqr2gpxzdd.fsf@gitster-ct.c.googlers.com","threadId":"48889","inReplyTo":"c141fb44-904f-e8b6-119f-7d2d6bcfd81a@web.de","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-10-16T23:09:34Z","receivedAt":"2018-10-16T23:09:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"René Scharfe writes:\n\n> Apart from that the macro is simple and doesn't use any tricks or\n> added checks. It just sets up boilerplate functions to offer type-safe\n> sorting.\n> ...\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 5f2e90932f..491230fc57 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -1066,6 +1066,21 @@ static inline void sane_qsort(void *base, size_t nmemb, size_t size,\n> \t\tqsort(base, nmemb, size, compar);\n> }\n> \n> +#define DECLARE_SORT(scope, name, elemtype) \\\n> +scope void name(elemtype, size_t)\n> +\n> +#define DEFINE_SORT(scope, name, elemtype, one, two)\t\t\t\\\n> +static int name##_compare(const elemtype, const elemtype);\t\t\\\n> +static int name##_compare_void(const void *a, const void *b)\t\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\treturn name##_compare(a, b);\t\t\t\t\t\\\n> +}\t\t\t\t\t\t\t\t\t\\\n> +scope void name(elemtype base, size_t nmemb)\t\t\t\t\\\n> +{\t\t\t\t\t\t\t\t\t\\\n> +\tQSORT(base, nmemb, name##_compare_void);\t\t\t\\\n> +}\t\t\t\t\t\t\t\t\t\\\n> +static int name##_compare(const elemtype one, const elemtype two)\n\n... and here comes the body of the comparison function that takes\ntwo \"things\" we are about, i.e. elements of the array being sorted.\n\nQuite cleanly done and the result looks pleasant, at least to me.\n"},{"id":"360731","messageId":"20181017083340.GB31932@sigill.intra.peff.net","threadId":"48889","inReplyTo":"c141fb44-904f-e8b6-119f-7d2d6bcfd81a@web.de","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-10-17T08:33:41Z","receivedAt":"2018-10-17T08:33:45Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Oct 14, 2018 at 04:29:06PM +0200, René Scharfe wrote:\n\n> Anyway, drove the generative approach a bit further, and came up with\n> the new DEFINE_SORT below. I'm unsure about the name; perhaps it should\n> be called DEFINE_SORT_BY_COMPARE_FUNCTION_BODY, but that's a bit long.\n> It handles casts and const attributes behind the scenes and avoids\n> repetition, but looks a bit weird, as it is placed where a function\n> signature would go.\n> \n> Apart from that the macro is simple and doesn't use any tricks or\n> added checks. It just sets up boilerplate functions to offer type-safe\n> sorting.\n> \n> diffcore-rename.c and refs/packed-backend.c receive special treatment in\n> the patch because their compare functions are used outside of sorting as\n> well. I made them take typed pointers nevertheless and used them from\n> DEFINE_SORT; the wrapper generated by that macro is supposed to be\n> private. Given that such reuse is rare and I think we don't need a way\n> to make it public.\n> \n> What do y'all think about this direction?\n\nI think it's the best we're likely to do, and is an improvement on the\nstatus quo.\n\nThe patch looks overall sane to me. I think DEFINE_SORT() is a fine\nname.\n\nI think given a macro parameter \"foo\" you could generate sort_by_foo()\nand compare_foo(), which would eliminate the extra layer in those\ntwo cases you mentioned. But I'm also fine with the approach you've\nshown here.\n\n-Peff\n"},{"id":"410194","messageId":"20201118021605.GB360414@google.com","threadId":"48889","inReplyTo":"20181017083340.GB31932@sigill.intra.peff.net","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2020-11-18T02:16:05Z","receivedAt":"2020-11-18T02:16:11Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nJeff King wrote:\n> On Sun, Oct 14, 2018 at 04:29:06PM +0200, René Scharfe wrote:\n\n>> Anyway, drove the generative approach a bit further, and came up with\n>> the new DEFINE_SORT below. I'm unsure about the name; perhaps it should\n>> be called DEFINE_SORT_BY_COMPARE_FUNCTION_BODY, but that's a bit long.\n>> It handles casts and const attributes behind the scenes and avoids\n>> repetition, but looks a bit weird, as it is placed where a function\n>> signature would go.\n>>\n>> Apart from that the macro is simple and doesn't use any tricks or\n>> added checks. It just sets up boilerplate functions to offer type-safe\n>> sorting.\n>>\n>> diffcore-rename.c and refs/packed-backend.c receive special treatment in\n>> the patch because their compare functions are used outside of sorting as\n>> well. I made them take typed pointers nevertheless and used them from\n>> DEFINE_SORT; the wrapper generated by that macro is supposed to be\n>> private. Given that such reuse is rare and I think we don't need a way\n>> to make it public.\n>>\n>> What do y'all think about this direction?\n>\n> I think it's the best we're likely to do, and is an improvement on the\n> status quo.\n>\n> The patch looks overall sane to me. I think DEFINE_SORT() is a fine\n> name.\n\nSince this came up in [1], I took a glance at this.\n\nI also think it looks reasonable, though it's possible to do better if\nwe're willing to (1) cast between pointers to function with different\nsignatures, which is portable in practice but I don't believe the C\nstandard speaks to and (2) conditionally make use of gcc extensions,\nfor typechecking.\n\nFor example, CCAN's asort[2] does typechecking on the arrays passed in\nand the callback cookie parameter to qsort_r, with no extra\nboilerplate or run-time overhead involved[3].\n\n(The core of that macro is ccan's typesafe_cb_cast[4]:\n\n /* CC0 (Public domain) - see LICENSE file for details */\n\n #if HAVE_TYPEOF && HAVE_BUILTIN_CHOOSE_EXPR && HAVE_BUILTIN_TYPES_COMPATIBLE_P\n /**\n * typesafe_cb_cast - only cast an expression if it matches a given type\n * @desttype: the type to cast to\n * @oktype: the type we allow\n * @expr: the expression to cast\n *\n * This macro is used to create functions which allow multiple types.\n * The result of this macro is used somewhere that a @desttype type is\n * expected: if @expr is exactly of type @oktype, then it will be\n * cast to @desttype type, otherwise left alone.\n *\n * This macro can be used in static initializers.\n *\n * This is merely useful for warnings: if the compiler does not\n * support the primitives required for typesafe_cb_cast(), it becomes an\n * unconditional cast, and the @oktype argument is not used. In\n * particular, this means that @oktype can be a type which uses the\n * \"typeof\": it will not be evaluated if typeof is not supported.\n *\n * Example:\n * // We can take either an unsigned long or a void *.\n * void _set_some_value(void *val);\n * #define set_some_value(e) \\\n * _set_some_value(typesafe_cb_cast(void *, unsigned long, (e)))\n */\n #define typesafe_cb_cast(desttype, oktype, expr) \\\n __builtin_choose_expr( \\\n __builtin_types_compatible_p(__typeof__(0?(expr):(expr)), \\\n oktype), \\\n (desttype)(expr), (expr))\n #else\n #define typesafe_cb_cast(desttype, oktype, expr) ((desttype)(expr))\n #endif\n)\n\nThanks,\nJonathan\n\n[1] https://lore.kernel.org/git/20201117223011.GA642234@coredump.intra.peff.net/\n[2] https://git.ozlabs.org/?p=ccan;a=blob;f=ccan/asort/asort.h;hb=HEAD\n[3] https://ccodearchive.net/info/asort.html\n[4] https://git.ozlabs.org/?p=ccan;a=blob;f=ccan/typesafe_cb/typesafe_cb.h;hb=HEAD\n"},{"id":"410206","messageId":"20201118065437.GA16996@coredump.intra.peff.net","threadId":"48889","inReplyTo":"20201118021605.GB360414@google.com","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-11-18T06:54:37Z","receivedAt":"2020-11-18T06:54:59Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Nov 17, 2020 at 06:16:05PM -0800, Jonathan Nieder wrote:\n\n> Since this came up in [1], I took a glance at this.\n> \n> I also think it looks reasonable, though it's possible to do better if\n> we're willing to (1) cast between pointers to function with different\n> signatures, which is portable in practice but I don't believe the C\n> standard speaks to and (2) conditionally make use of gcc extensions,\n> for typechecking.\n\nThe C standard definitely is not OK with calling a function through a\nwrong declaration or cast. I won't find chapter and verse, but here's a\npractical example:\n\n-- >8 --\n#include \n#include \n\nvoid foo(uint32_t a, uint32_t b)\n{\n\tprintf(\"got a = %u\\n\", a);\n\tprintf(\"got b = %u\\n\", b);\n}\n\ntypedef void (*almost_foo)(uint64_t, uint64_t);\n\nint main(void)\n{\n\talmost_foo bar = (almost_foo)foo;\n\n\tprintf(\"real call:\\n\");\n\tfoo(1, 2);\n\tprintf(\"via cast:\\n\");\n\tbar(3, 4);\n\treturn 0;\n}\n-- >8 --\n\nThe caller thinks it's passing uint64_t integers, but the function\nthinks it's getting uint32_t integers. The output will depend on your\ncalling conventions. If I compile it on my 64-bit Linux machine, it\nproduces what you'd expect:\n\n $ gcc foo.c\n $ ./a.out\n real call:\n got a = 1\n got b = 2\n via cast:\n got a = 3\n got b = 4\n\nThat's because we're using the System V AMD64 ABI convention, which\npasses the first six parameters via registers. And even after that, each\nparameter on the stack uses 8 bytes (even if it's smaller), so the two\nrepresentations are equivalent.\n\nBut if I compile it in 32-bit mode, it doesn't work:\n\n $ gcc -m32 foo.c\n $ ./a.out\n real call:\n got a = 1\n got b = 2\n via cast:\n got a = 3\n got b = 0\n\nThat's because it's using the cdecl convention, which puts everything on\nthe stack, and which uses a minimum of 4 bytes per parameter. So each\n64-bit value results in two 32-bit pushes onto the stack (of 0, and 3).\n\nNow in practice you're probably fine as long as the number and sizes of\nthe parameters are the same between the function definition and what the\ncaller casts to. And so if we're talking about casting individual\nparameters between a void parameter and another pointer, that would\nusually be fine (in practice; the standard only says that void can store\nthe type of anything, so it _could_ be larger than some other pointers.\nI don't know of any modern systems where this is true, though).\n\nWhich is all a roundabout way of saying that yes, I think this kind of\ncast is probably OK in practice.\n\nI _think_ the ccan type-checking macro you pointed to would catch this\nsufficiently on systems with typeof() that it would also protect systems\nwith different calling conventions. But I admit it's pretty dense.\n\nSo I dunno. The nice thing is that this puts the ugliness all inside of\nQSORT(), which becomes magically type-safe. But it involves importing a\nlot of tricky bits under the hood.\n\nThe downside of René's patch is that it hides the declaration of the\ncomparison function (and the typesafe wrapper) inside a macro. But the\nresulting code is (IMHO) pretty easy to comprehend.\n\n-Peff\n"},{"id":"410247","messageId":"c236ca17-0f9e-a900-8a50-f467eaf6b485@web.de","threadId":"48889","inReplyTo":"20201118065437.GA16996@coredump.intra.peff.net","subject":"Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2020-11-18T17:47:14Z","receivedAt":"2020-11-18T17:47:54Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 18.11.20 um 07:54 schrieb Jeff King:\n> On Tue, Nov 17, 2020 at 06:16:05PM -0800, Jonathan Nieder wrote:\n>\n>> Since this came up in [1], I took a glance at this.\n>>\n>> I also think it looks reasonable, though it's possible to do better if\n>> we're willing to (1) cast between pointers to function with different\n>> signatures, which is portable in practice but I don't believe the C\n>> standard speaks to and (2) conditionally make use of gcc extensions,\n>> for typechecking.\n>\n> The C standard definitely is not OK with calling a function through a\n> wrong declaration or cast. I won't find chapter and verse\n\nhttp://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf is a draft for\nC99, and says under 6.3 Conversions, 6.3.2.3 Pointers, paragraph 8:\n\n If a converted pointer is used to call a function whose type is not\n compatible with the pointed-to type, the behavior is undefined.\n\n> Now in practice you're probably fine as long as the number and sizes of\n> the parameters are the same between the function definition and what the\n> caller casts to. And so if we're talking about casting individual\n> parameters between a void parameter and another pointer, that would\n> usually be fine (in practice; the standard only says that void can store\n> the type of anything, so it _could_ be larger than some other pointers.\n> I don't know of any modern systems where this is true, though).\n>\n> Which is all a roundabout way of saying that yes, I think this kind of\n> cast is probably OK in practice.\n>\n> I _think_ the ccan type-checking macro you pointed to would catch this\n> sufficiently on systems with typeof() that it would also protect systems\n> with different calling conventions. But I admit it's pretty dense.\n>\n> So I dunno. The nice thing is that this puts the ugliness all inside of\n> QSORT(), which becomes magically type-safe. But it involves importing a\n> lot of tricky bits under the hood.\n\nA generic and type-safe QSORT would be nice, but if it calls a function\nvia a converted pointer then it's technically relying on undefined\nbehavior unless I misunderstand the standard. I prefer occasional\nmistakes (that are caught by ASan or USan or when the sort order actually\nmatters) to guaranteed undefined behavior that happens to work, until it\ndoesn't.\n\n> The downside of René's patch is that it hides the declaration of the\n> comparison function (and the typesafe wrapper) inside a macro. But the\n> resulting code is (IMHO) pretty easy to comprehend.\n\nI tried some more variants back then, before I dropped the ball\neventually when RL distracted me. I think my favorite one was a\nDEFINE_SORT macro that took the name of a typed comparison function --\nthe code looks more like normal C.\n\nHandling arrays of pointers was a bit tricky, and I had to introduce\nDEFINE_PTR_SORT and DEFINE_CONST_PTR_SORT for them, but they allowed to\nuse the same comparison functions -- they consistently took element\npointers. Just one rule to follow, and the compiler would yell when\na mismatched macro was used.\n\nAnd at that point it occurred to me that comparison functions would\nideally take two elements, not pointers. All the pointer mangling\ncould be done in generated code. GCC and Clang inline it\nappropriately, so this convenience would be free -- but other\ncompilers don't, and that would make sorting more expensive on those\nplatforms. Dead end.\n\nThe last one I looked at was a dumbed-down version, but I think this\nrequires some weird comparison function signatures in some cases\n(patch below, basically untested).\n\nI can understand now why monomorphization approaches like\nhttps://github.com/attractivechaos/klib/blob/master/ksort.h seem\nattractive (no pun intended)..\n\nRené\n\n---\n git-compat-util.h | 17 +++++++++++++++++\n 1 file changed, 17 insertions(+)\n\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex adfea06897..8d871a8e33 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -1124,6 +1124,23 @@ int git_qsort_s(void *base, size_t nmemb, size_t size,\n \t\tBUG(\"qsort_s() failed\");\t\t\t\\\n } while (0)\n\n+#define DECLARE_COMPARE(scope, compar)\t\t\t\t\t\\\n+scope int compar##__void(const void *, const void *)\n+\n+#define DEFINE_COMPARE(scope, compar)\t\t\t\t\t\\\n+scope int compar##__void(const void *a_void_ptr,\t\t\t\\\n+\t\t\t const void *b_void_ptr)\t\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\treturn compar(a_void_ptr, b_void_ptr);\t\t\t\t\\\n+}\t\t\t\t\t\t\t\t\t\\\n+DECLARE_COMPARE(scope, compar)\n+\n+#define GET_COMPARE(base, compar)\t\t\t\t\t\\\n+(0 && compar((base), (base)) ? NULL : compar##__void)\n+\n+#define SORT_ARRAY(base, n, compar)\t\t\t\t\t\\\n+QSORT((base), (n), GET_COMPARE((base), compar))\n+\n #ifndef REG_STARTEND\n #error \"Git requires REG_STARTEND support. Compile with NO_REGEX=NeedsStartEnd\"\n #endif\n--\n2.29.2\n"}]}