{"thread":{"id":"2164","subject":"[PATCH] Do not use git-rev-list in git-fetch-pack","startedAt":"2005-10-20T15:12:01Z","lastAt":"2005-10-20T15:12:01Z","messageCount":1,"participants":["Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"10356","messageId":"Pine.LNX.4.63.0510201707560.29282@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"2164","inReplyTo":null,"subject":"[PATCH] Do not use git-rev-list in git-fetch-pack","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-10-20T15:12:01Z","receivedAt":"2005-10-20T15:12:01Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The code used to call git-rev-list to enumerate the local revisions.\nA disadvantage of that method was that git-fetch-pack would happily\nenumerate ancestors of acknowledged common commits, which was just\ntaking unnecessary bandwidth.\n\nFurther, git-upload-pack makes use only of the first MAX_HAS == 16\nrevisions! So if git-fetch-pack has something to say, it better does\nso concisely.\n\nTherefore, do not use git-rev-list on the fetching side. Send the\nrevisions starting with the local heads, ignoring the acknowledged\nrevisions and their ancestors.\n\nWhen git-fetch-pack is verbose, it now says which sha1 got acknowledged.\n\nSigned-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>\n\n---\n\n\tI am quite certain that MAX_HAS is too low. Since\n\tMAX_NEEDS == 256, I think this value would be better for\n\tMAX_HAS, too, especially since git-fetch-pack no longer\n\tsends so much crud with this patch.\n\n\tAlso: If I read the code correctly, find_common() only sends\n\t\"have\" lines as long as it does not receive an \"ACK\" line,\n\ti.e. in effect git-fetch-pack sends only up to the first\n\tcommon revision (plus at most 31 others).\n\n\tThat is all well if we are expecting only one upstream\n\trepository, but not in a truly distributed environment.\n\n\tMaybe it is time to revise that behaviour? For example,\n\tfetch-pack could stop when it is out of non-common revs,\n\tor when it got MAX_HAS \"ACK\" lines.\n\n\tAlso, git-upload-pack could accept only up to a certain\n\tnumber of \"have\" lines.\n\n fetch-pack.c |  143 ++++++++++++++++++++++++++++++++++++++++++++--------------\n 1 files changed, 109 insertions(+), 34 deletions(-)\n\napplies-to: 50f9bed9a78e1f51178ebf6be3cd9c20b2ffcb0b\n2b5886e2198ea4b8403839b7188e8ad9e6f1713b\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 8566ab1..cdde008 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -13,18 +13,110 @@ static const char fetch_pack_usage[] =\n static const char *exec = \"git-upload-pack\";\n \n #define COMPLETE\t(1U << 0)\n+#define COMMON\t\t(1U << 1)\n+#define COMMON_REF\t(1U << 2)\n+#define TO_SCAN\t\t(1U << 3)\n+\n+struct commit_list *rev_list = NULL;\n+\n+/*\n+   Unfortunately, we want not only commits, but sometimes also tags.\n+*/\n+\n+static struct commit *fake_commit(struct object *object) {\n+\tstruct commit *ret;\n+\n+\tif (object->type == commit_type)\n+\t\treturn (struct commit *)object;\n+\n+\tret = xmalloc(sizeof(struct commit));\n+\n+\tmemset(ret, 0, sizeof(struct commit));\n+\tret->object = *object;\n+\n+\t/* insert_by_date() shall put it at the end */\n+\tret->date = 0;\n+\n+\treturn ret;\n+}\n+\n+static int rev_list_insert(const char *path, const unsigned char *sha1)\n+{\n+\tstruct commit *commit  = fake_commit(parse_object(sha1));\n+\n+\t/*\n+\t   Local heads are much more likely to be common, so send them first.\n+\t*/\n+\tif (commit->object.type == commit_type)\n+\t\tcommit_list_insert(commit, &rev_list);\n+\telse {\n+\t\tinsert_by_date(commit, &rev_list);\n+\n+\t\tif (commit->object.type == tag_type) {\n+\t\t\t/* This commit has been faked. Reget. */\n+\t\t\tstruct object *o = lookup_object(commit->object.sha1);\n+\t\t\to = deref_tag(o);\n+\t\t\tif (o->type == commit_type)\n+\t\t\t\tinsert_by_date((struct commit *)o, &rev_list);\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/*\n+  Get the next rev to send, ignoring the common.\n+*/\n+\n+static const unsigned char* get_rev()\n+{\n+\tstruct commit *commit = NULL;\n+\n+\twhile (commit == NULL) {\n+\t\tunsigned int mark;\n+\n+\t\tif (rev_list == NULL)\n+\t\t\treturn NULL;\n+\n+\t\tcommit = rev_list->item;\n+\n+\t\tif (commit->object.flags&COMMON) {\n+\t\t\t/* do not send \"have\", and ignore ancestors */\n+\t\t\tcommit = NULL;\n+\t\t\tmark = COMMON | TO_SCAN;\n+\t\t} else if (commit->object.flags&COMMON_REF)\n+\t\t\t/* send \"have\", and ignore ancestors */\n+\t\t\tmark = COMMON | TO_SCAN;\n+\t\telse\n+\t\t\t/* send \"have\", also for ancestors, until one is ack'ed */\n+\t\t\tmark = TO_SCAN;\n+\n+\t\tpop_most_recent_commit(&rev_list, mark);\n+\t}\n+\n+\treturn commit->object.sha1;\n+}\n+\n+static void mark_common(const unsigned char* sha1)\n+{\n+\tstruct object *o = lookup_object(sha1);\n+\n+\tif (o != NULL && !(o->flags&COMMON)) {\n+\t\to->flags |= COMMON;\n+\t\tif (o->type == commit_type)\n+\t\t\tinsert_by_date((struct commit *)o, &rev_list);\n+\t}\n+}\n \n static int find_common(int fd[2], unsigned char *result_sha1,\n \t\t       struct ref *refs)\n {\n \tint fetching;\n-\tstatic char line[1000];\n-\tstatic char rev_command[1024];\n-\tint count = 0, flushes = 0, retval, rev_command_len;\n-\tFILE *revs;\n+\tint count = 0, flushes = 0, retval;\n+\tconst unsigned char *sha1;\n+\n+\tfor_each_ref(rev_list_insert);\n \n-\tstrcpy(rev_command, \"git-rev-list $(git-rev-parse --all)\");\n-\trev_command_len = strlen(rev_command);\n \tfetching = 0;\n \tfor ( ; refs ; refs = refs->next) {\n \t\tunsigned char *remote = refs->old_sha1;\n@@ -42,25 +134,11 @@ static int find_common(int fd[2], unsign\n \t\t */\n \t\tif (((o = lookup_object(remote)) != NULL) &&\n \t\t    (o->flags & COMPLETE)) {\n-\t\t\tstruct commit_list *p;\n-\t\t\tstruct commit *commit =\n-\t\t\t\t(struct commit *) (o = deref_tag(o));\n-\t\t\tif (!o)\n-\t\t\t\tgoto repair;\n-\t\t\tif (o->type != commit_type)\n-\t\t\t\tcontinue;\n-\t\t\tp = commit->parents;\n-\t\t\twhile (p &&\n-\t\t\t       rev_command_len + 44 < sizeof(rev_command)) {\n-\t\t\t\tsnprintf(rev_command + rev_command_len, 44,\n-\t\t\t\t\t \" ^%s\",\n-\t\t\t\t\t sha1_to_hex(p->item->object.sha1));\n-\t\t\t\trev_command_len += 43;\n-\t\t\t\tp = p->next;\n-\t\t\t}\n+\t\t\to->flags |= COMMON_REF | TO_SCAN;\n+\t\t\tcommit_list_insert(fake_commit(o), &rev_list);\n \t\t\tcontinue;\n \t\t}\n-\trepair:\n+\n \t\tpacket_write(fd[1], \"want %s\\n\", sha1_to_hex(remote));\n \t\tfetching++;\n \t}\n@@ -68,21 +146,16 @@ static int find_common(int fd[2], unsign\n \tif (!fetching)\n \t\treturn 1;\n \n-\trevs = popen(rev_command, \"r\");\n-\tif (!revs)\n-\t\tdie(\"unable to run 'git-rev-list'\");\n-\n \tflushes = 1;\n \tretval = -1;\n-\twhile (fgets(line, sizeof(line), revs) != NULL) {\n-\t\tunsigned char sha1[20];\n-\t\tif (get_sha1_hex(line, sha1))\n-\t\t\tdie(\"git-fetch-pack: expected object name, got crud\");\n+\twhile ((sha1 = get_rev())) {\n \t\tpacket_write(fd[1], \"have %s\\n\", sha1_to_hex(sha1));\n \t\tif (verbose)\n \t\t\tfprintf(stderr, \"have %s\\n\", sha1_to_hex(sha1));\n \t\tif (!(31 & ++count)) {\n \t\t\tpacket_flush(fd[1]);\n+\t\t\tif (verbose)\n+\t\t\t\tfprintf(stderr, \"flush\\n\");\n \t\t\tflushes++;\n \n \t\t\t/*\n@@ -92,16 +165,17 @@ static int find_common(int fd[2], unsign\n \t\t\tif (count == 32)\n \t\t\t\tcontinue;\n \t\t\tif (get_ack(fd[0], result_sha1)) {\n+\t\t\t\tmark_common(result_sha1);\n \t\t\t\tflushes = 0;\n \t\t\t\tretval = 0;\n \t\t\t\tif (verbose)\n-\t\t\t\t\tfprintf(stderr, \"got ack\\n\");\n+\t\t\t\t\tfprintf(stderr, \"got ack %s\\n\",\n+\t\t\t\t\t\tsha1_to_hex(result_sha1));\n \t\t\t\tbreak;\n \t\t\t}\n \t\t\tflushes--;\n \t\t}\n \t}\n-\tpclose(revs);\n \tpacket_write(fd[1], \"done\\n\");\n \tif (verbose)\n \t\tfprintf(stderr, \"done\\n\");\n@@ -109,7 +183,8 @@ static int find_common(int fd[2], unsign\n \t\tflushes--;\n \t\tif (get_ack(fd[0], result_sha1)) {\n \t\t\tif (verbose)\n-\t\t\t\tfprintf(stderr, \"got ack\\n\");\n+\t\t\t\tfprintf(stderr, \"got ack %s\\n\",\n+\t\t\t\t\tsha1_to_hex(result_sha1));\n \t\t\treturn 0;\n \t\t}\n \t}\n---\n0.99.8.GIT\n"}]}