{"thread":{"id":"2134","subject":"git-fetch-pack: avoid unnecessary zero packing","startedAt":"2005-10-18T17:45:16Z","lastAt":"2005-10-18T17:48:04Z","messageCount":2,"participants":["Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"10210","messageId":"Pine.LNX.4.64.0510181032400.3369@g5.osdl.org","threadId":"2134","inReplyTo":null,"subject":"git-fetch-pack: avoid unnecessary zero packing","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-10-18T17:45:16Z","receivedAt":"2005-10-18T17:45:16Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\nIf everything is up-to-date locally, we don't need to even ask for a \npack-file from the remote, or try to unpack it.\n\nThis is especially important for tags - since the pack-file common commit \nlogic is based purely on the commit history, it will never be able to find \na common tag, and will thus always end up re-fetching them.\n\nEspecially notably, if the tag points to a non-commit (eg a tagged tree), \nthe pack-file would be unnecessarily big, just because it cannot any most \nrecent common point between commits for pruning.\n\nShort-circuiting the case where we already have that reference means that \nwe avoid a lot of these in the common case.\n\nNOTE! This only matches remote ref names against the same local name, \nwhich works well for tags, but is not as generic as it could be. If we \never need to, we could match against _any_ local ref (if we have it, we \nhave it), but this \"match against same name\" is simpler and more \nefficient, and covers the common case.\n\nRenaming of refs is common for branch heads, but since those are always \ncommits, the pack-file generation can optimize that case.\n\nIn some cases we might still end up fetching pack-files unnecessarily, but \nthis at least avoids the re-fetching of tags over and over if you use a \nregular\n\n\tgit fetch --tags ...\n\nwhich was the main reason behind the change.\n\nSigned-off-by: Linus Torvalds <torvalds@osdl.org>\n---\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 953c0cf..cf21c34 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -12,6 +12,7 @@ static const char *exec = \"git-upload-pa\n static int find_common(int fd[2], unsigned char *result_sha1,\n \t\t       struct ref *refs)\n {\n+\tint fetching;\n \tstatic char line[1000];\n \tint count = 0, flushes = 0, retval;\n \tFILE *revs;\n@@ -20,16 +21,19 @@ static int find_common(int fd[2], unsign\n \tif (!revs)\n \t\tdie(\"unable to run 'git-rev-list'\");\n \n-\twhile (refs) {\n+\tfetching = 0;\n+\tfor ( ; refs ; refs = refs->next) {\n \t\tunsigned char *remote = refs->old_sha1;\n-\t\tif (verbose)\n-\t\t\tfprintf(stderr,\n-\t\t\t\t\"want %s (%s)\\n\", sha1_to_hex(remote),\n-\t\t\t\trefs->name);\n+\t\tunsigned char *local = refs->new_sha1;\n+\n+\t\tif (!memcmp(remote, local, 20))\n+\t\t\tcontinue;\n \t\tpacket_write(fd[1], \"want %s\\n\", sha1_to_hex(remote));\n-\t\trefs = refs->next;\n+\t\tfetching++;\n \t}\n \tpacket_flush(fd[1]);\n+\tif (!fetching)\n+\t\treturn 1;\n \tflushes = 1;\n \tretval = -1;\n \twhile (fgets(line, sizeof(line), revs) != NULL) {\n@@ -74,6 +78,37 @@ static int find_common(int fd[2], unsign\n \treturn retval;\n }\n \n+static int everything_local(struct ref *refs)\n+{\n+\tint retval;\n+\n+\tfor (retval = 1; refs ; refs = refs->next) {\n+\t\tconst unsigned char *remote = refs->old_sha1;\n+\t\tunsigned char local[20];\n+\n+\t\tif (read_ref(git_path(\"%s\", refs->name), local) < 0)\n+\t\t\tcontinue;\n+\n+\t\tif (memcmp(remote, local, 20)) {\n+\t\t\tretval = 0;\n+\t\t\tif (!verbose)\n+\t\t\t\tcontinue;\n+\t\t\tfprintf(stderr,\n+\t\t\t\t\"want %s (%s)\\n\", sha1_to_hex(remote),\n+\t\t\t\trefs->name);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tmemcpy(refs->new_sha1, local, 20);\n+\t\tif (!verbose)\n+\t\t\tcontinue;\n+\t\tfprintf(stderr,\n+\t\t\t\"already have %s (%s)\\n\", sha1_to_hex(remote),\n+\t\t\trefs->name);\n+\t}\n+\treturn retval;\n+}\n+\n static int fetch_pack(int fd[2], int nr_match, char **match)\n {\n \tstruct ref *ref;\n@@ -86,6 +121,10 @@ static int fetch_pack(int fd[2], int nr_\n \t\tpacket_flush(fd[1]);\n \t\tdie(\"no matching remote head\");\n \t}\n+\tif (everything_local(ref)) {\n+\t\tpacket_flush(fd[1]);\n+\t\tgoto all_done;\n+\t}\n \tif (find_common(fd, sha1, ref) < 0)\n \t\tfprintf(stderr, \"warning: no common commits\\n\");\n \tpid = fork();\n@@ -109,6 +148,7 @@ static int fetch_pack(int fd[2], int nr_\n \t\tint code = WEXITSTATUS(status);\n \t\tif (code)\n \t\t\tdie(\"git-unpack-objects died with error code %d\", code);\n+all_done:\n \t\twhile (ref) {\n \t\t\tprintf(\"%s %s\\n\",\n \t\t\t       sha1_to_hex(ref->old_sha1), ref->name);\n"},{"id":"10211","messageId":"Pine.LNX.4.64.0510181046401.3369@g5.osdl.org","threadId":"2134","inReplyTo":"Pine.LNX.4.64.0510181032400.3369@g5.osdl.org","subject":"Re: git-fetch-pack: avoid unnecessary zero packing","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-10-18T17:48:04Z","receivedAt":"2005-10-18T17:48:04Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\nGaah. Ignore this version - I had re-organized the code, and that \nintroduced a stupid bug for the case where the local side didn't have any \nsuch ref at all.\n\nI'll send a corrected patch momentarily.\n\n\t\tLinus\n\nOn Tue, 18 Oct 2005, Linus Torvalds wrote:\n> \n> If everything is up-to-date locally, we don't need to even ask for a \n> pack-file from the remote, or try to unpack it.\n"}]}