{"thread":{"id":"10760","subject":"[PATCH 10/11] git-fetch: avoid local fetching from alternate (again)","startedAt":"2007-11-09T11:06:45Z","lastAt":"2007-11-09T11:06:45Z","messageCount":1,"participants":["Shawn O. Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":11},"messages":[{"id":"59029","messageId":"20071109110645.GJ19368@spearce.org","threadId":"10760","inReplyTo":null,"subject":"[PATCH 10/11] git-fetch: avoid local fetching from alternate (again)","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-11-09T11:06:45Z","receivedAt":"2007-11-09T11:06:45Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Back in e3c6f240fd9c5bdeb33f2d47adc859f37935e2df Junio taught\ngit-fetch to avoid copying objects when we are fetching from\na repository that is already registered as an alternate object\ndatabase.  In such a case there is no reason to copy any objects\nas we can already obtain them through the alternate.\n\nHowever we need to ensure the objects are all reachable, so we\nrun `git rev-list --objects $theirs --not --all` to verify this.\nIf any object is missing or unreadable then we need to fetch/copy\nthe objects from the remote.  When a missing object is detected\nthe git-rev-list process will exit with a non-zero exit status,\nmaking this condition quite easy to detect.\n\nAlthough git-fetch is currently a builtin (and so is rev-list)\nwe cannot invoke the traverse_objects() API at this point in the\ntransport code.  The object walker within traverse_objects() calls\ndie() as soon as it finds an object it cannot read.  If that happens\nwe want to resume the fetch process by calling do_fetch_pack().\nTo get aroaund this we spawn git-rev-list into a background process\nto prevent a die() from killing the foreground fetch process,\nthus allowing the fetch process to resume into do_fetch_pack()\nif copying is necessary.\n\nWe aren't interested in the output of rev-list (a list of SHA-1\nobject names that are reachable) or its errors (a \"spurious\" error\nabout an object not being found as we need to copy it) so we redirect\nboth stdout and stderr to /dev/null.\n\nWe run this git-rev-list based check before any fetch as we may\nalready have the necessary objects local from a prior fetch.  If we\ndon't then its very likely the first $theirs object listed on the\ncommand line won't exist locally and git-rev-list will die very\nquickly, allowing us to start the network transfer.  This test even\non remote URLs may save bandwidth if someone runs `git pull origin`,\nsees a merge conflict, resets out, then redoes the same pull just\na short time later.  If the remote hasn't changed between the two\npulls and the local repository hasn't had git-gc run in it then\nthere is probably no need to perform network transfer as all of\nthe objects are local.\n\nDocumentation for the new fetch_local_nocopy function was suggested\nand written by Junio, based on his original comment in git-fetch.sh.\n\nSigned-off-by: Shawn O. Pearce <spearce@spearce.org>\n---\n transport.c |   68 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++--\n 1 files changed, 65 insertions(+), 3 deletions(-)\n\ndiff --git a/transport.c b/transport.c\nindex f4577b7..883348b 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -615,17 +615,76 @@ static struct ref *get_refs_via_connect(struct transport *transport)\n \treturn refs;\n }\n \n+/*\n+ * We would want to bypass the object transfer altogether if\n+ * everything we are going to fetch already exists and connected\n+ * locally.\n+ *\n+ * The refs we are going to fetch are in to_fetch (nr_heads in\n+ * total).  If running\n+ *\n+ *  $ git-rev-list --objects to_fetch[0] to_fetch[1] ... --not --all\n+ *\n+ * does not error out, that means everything reachable from the\n+ * refs we are going to fetch exists and is connected to some of\n+ * our existing refs.\n+ */\n+static int fetch_local_nocopy(struct transport *transport,\n+\t\t\t       int nr_heads, struct ref **to_fetch)\n+{\n+\tstruct git_transport_data *data = transport->data;\n+\tstruct child_process revlist;\n+\tchar **argv;\n+\tint i, j, err;\n+\n+\t/*\n+\t * If we are deepening a shallow clone we already have these\n+\t * objects reachable.  Running rev-list here will return with\n+\t * a good (0) exit status and we'll bypass the fetch that we\n+\t * really need to perform.  Claiming failure now will ensure\n+\t * we perform the network exchange to deepen our history.\n+\t */\n+\tif (data->depth)\n+\t\treturn -1;\n+\n+\ti = 0;\n+\targv = xmalloc(sizeof(*argv) * (nr_heads + 6));\n+\targv[i++] = xstrdup(\"rev-list\");\n+\targv[i++] = xstrdup(\"--quiet\");\n+\targv[i++] = xstrdup(\"--objects\");\n+\tfor (j = 0; j < nr_heads; j++)\n+\t\targv[i++] = xstrdup(sha1_to_hex(to_fetch[j]->old_sha1));\n+\targv[i++] = xstrdup(\"--not\");\n+\targv[i++] = xstrdup(\"--all\");\n+\targv[i++] = NULL;\n+\n+\tmemset(&revlist, 0, sizeof(revlist));\n+\trevlist.argv = (const char**)argv;\n+\trevlist.git_cmd = 1;\n+\trevlist.no_stdin = 1;\n+\trevlist.no_stdout = 1;\n+\trevlist.no_stderr = 1;\n+\terr = run_command(&revlist);\n+\n+\tfor (i = 0; argv[i]; i++)\n+\t\tfree(argv[i]);\n+\tfree(argv);\n+\treturn err;\n+}\n+\n static int fetch_refs_via_pack(struct transport *transport,\n \t\t\t       int nr_heads, struct ref **to_fetch)\n {\n \tstruct git_transport_data *data = transport->data;\n-\tchar **heads = xmalloc(nr_heads * sizeof(*heads));\n-\tchar **origh = xmalloc(nr_heads * sizeof(*origh));\n+\tchar **heads, **origh;\n \tstruct ref *refs;\n-\tchar *dest = xstrdup(transport->url);\n+\tchar *dest;\n \tstruct fetch_pack_args args;\n \tint i;\n \n+\tif (!fetch_local_nocopy(transport, nr_heads, to_fetch))\n+\t\treturn 0;\n+\n \tmemset(&args, 0, sizeof(args));\n \targs.uploadpack = data->uploadpack;\n \targs.keep_pack = data->keep;\n@@ -634,6 +693,9 @@ static int fetch_refs_via_pack(struct transport *transport,\n \targs.verbose = transport->verbose > 0;\n \targs.depth = data->depth;\n \n+\theads = xmalloc(nr_heads * sizeof(*heads));\n+\torigh = xmalloc(nr_heads * sizeof(*origh));\n+\tdest = xstrdup(transport->url);\n \tfor (i = 0; i < nr_heads; i++)\n \t\torigh[i] = heads[i] = xstrdup(to_fetch[i]->name);\n \trefs = fetch_pack(&args, dest, nr_heads, heads, &transport->pack_lockfile);\n-- \n1.5.3.5.1622.g41d10\n"}]}