{"thread":{"id":"20381","subject":"[PATCH 01/13] Add support for external programs for handling native fetches","startedAt":"2009-08-05T05:01:53Z","lastAt":"2009-08-05T05:01:53Z","messageCount":1,"participants":["Daniel Barkalow"],"isPatch":true,"patchVersion":1,"patchTotal":13},"messages":[{"id":"119561","messageId":"alpine.LNX.2.00.0908050053420.2147@iabervon.org","threadId":"20381","inReplyTo":null,"subject":"[PATCH 01/13] Add support for external programs for handling native fetches","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2009-08-05T05:01:53Z","receivedAt":"2009-08-05T05:01:53Z","isPatch":true,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"transport_get() can call transport_native_helper_init() to have list and\nfetch-ref operations handled by running a separate program as:\n\n git remote-<something> <remote> [<url>]\n\nThis program then accepts, on its stdin, \"list\" and \"fetch <hex>\n<name>\" commands; the former prints out a list of available refs and\neither their hashes or what they are symrefs to, while the latter\nfetches them into the local object database and prints a newline when done.\n\nSigned-off-by: Daniel Barkalow <barkalow@iabervon.org>\n---\n Documentation/git-remote-helpers.txt |   71 ++++++++++++++\n Makefile                             |    1 +\n transport-helper.c                   |  168 ++++++++++++++++++++++++++++++++++\n transport.h                          |    3 +\n 4 files changed, 243 insertions(+), 0 deletions(-)\n create mode 100644 Documentation/git-remote-helpers.txt\n create mode 100644 transport-helper.c\n\ndiff --git a/Documentation/git-remote-helpers.txt b/Documentation/git-remote-helpers.txt\nnew file mode 100644\nindex 0000000..173ee23\n--- /dev/null\n+++ b/Documentation/git-remote-helpers.txt\n@@ -0,0 +1,71 @@\n+git-remote-helpers(1)\n+=====================\n+\n+NAME\n+----\n+git-remote-helpers - Helper programs for interoperation with remote git\n+\n+SYNOPSIS\n+--------\n+'git remote-<transport>' <remote>\n+\n+DESCRIPTION\n+-----------\n+\n+These programs are normally not used directly by end users, but are\n+invoked by various git programs that interact with remote repositories\n+when the repository they would operate on will be accessed using\n+transport code not linked into the main git binary. Various particular\n+helper programs will behave as documented here.\n+\n+COMMANDS\n+--------\n+\n+Commands are given by the caller on the helper's standard input, one per line.\n+\n+'capabilities'::\n+\tLists the capabilities of the helper, one per line, ending\n+\twith a blank line.\n+\n+'list'::\n+\tLists the refs, one per line, in the format \"<value> <name>\n+\t[<attr> ...]\". The value may be a hex sha1 hash, \"@<dest>\" for\n+\ta symref, or \"?\" to indicate that the helper could not get the\n+\tvalue of the ref. A space-separated list of attributes follows\n+\tthe name; unrecognized attributes are ignored. After the\n+\tcomplete list, outputs a blank line.\n+\n+'fetch' <sha1> <name>::\n+\tFetches the given object, writing the necessary objects to the\n+\tdatabase. Outputs a blank line when the fetch is\n+\tcomplete. Only objects which were reported in the ref list\n+\twith a sha1 may be fetched this way.\n++\n+Supported if the helper has the \"fetch\" capability.\n+\n+If a fatal error occurs, the program writes the error message to\n+stderr and exits. The caller should expect that a suitable error\n+message has been printed if the child closes the connection without\n+completing a valid response for the current command.\n+\n+Additional commands may be supported, as may be determined from\n+capabilities reported by the helper.\n+\n+CAPABILITIES\n+------------\n+\n+'fetch'::\n+\tThis helper supports the 'fetch' command.\n+\n+REF LIST ATTRIBUTES\n+-------------------\n+\n+None are defined yet, but the caller must accept any which are supplied.\n+\n+Documentation\n+-------------\n+Documentation by Daniel Barkalow.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\ndiff --git a/Makefile b/Makefile\nindex daf4296..504646a 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -549,6 +549,7 @@ LIB_OBJS += symlinks.o\n LIB_OBJS += tag.o\n LIB_OBJS += trace.o\n LIB_OBJS += transport.o\n+LIB_OBJS += transport-helper.o\n LIB_OBJS += tree-diff.o\n LIB_OBJS += tree.o\n LIB_OBJS += tree-walk.o\ndiff --git a/transport-helper.c b/transport-helper.c\nnew file mode 100644\nindex 0000000..0d19c6c\n--- /dev/null\n+++ b/transport-helper.c\n@@ -0,0 +1,168 @@\n+#include \"cache.h\"\n+#include \"transport.h\"\n+\n+#include \"run-command.h\"\n+#include \"commit.h\"\n+#include \"diff.h\"\n+#include \"revision.h\"\n+\n+struct helper_data\n+{\n+\tconst char *name;\n+\tstruct child_process *helper;\n+\tunsigned fetch : 1;\n+};\n+\n+static struct child_process *get_helper(struct transport *transport)\n+{\n+\tstruct helper_data *data = transport->data;\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct child_process *helper;\n+\tFILE *file;\n+\n+\tif (data->helper)\n+\t\treturn data->helper;\n+\n+\thelper = xcalloc(1, sizeof(*helper));\n+\thelper->in = -1;\n+\thelper->out = -1;\n+\thelper->err = 0;\n+\thelper->argv = xcalloc(4, sizeof(*helper->argv));\n+\tstrbuf_addf(&buf, \"remote-%s\", data->name);\n+\thelper->argv[0] = strbuf_detach(&buf, NULL);\n+\thelper->argv[1] = transport->remote->name;\n+\thelper->argv[2] = transport->url;\n+\thelper->git_cmd = 1;\n+\tif (start_command(helper))\n+\t\tdie(\"Unable to run helper: git %s\", helper->argv[0]);\n+\tdata->helper = helper;\n+\n+\twrite_in_full(data->helper->in, \"capabilities\\n\", 13);\n+\tfile = fdopen(helper->out, \"r\");\n+\twhile (1) {\n+\t\tif (strbuf_getline(&buf, file, '\\n') == EOF)\n+\t\t\texit(128); /* child died, message supplied already */\n+\n+\t\tif (!*buf.buf)\n+\t\t\tbreak;\n+\t\tif (!strcmp(buf.buf, \"fetch\"))\n+\t\t\tdata->fetch = 1;\n+\t}\n+\treturn data->helper;\n+}\n+\n+static int disconnect_helper(struct transport *transport)\n+{\n+\tstruct helper_data *data = transport->data;\n+\tif (data->helper) {\n+\t\twrite_in_full(data->helper->in, \"\\n\", 1);\n+\t\tclose(data->helper->in);\n+\t\tfinish_command(data->helper);\n+\t\tfree((char *)data->helper->argv[0]);\n+\t\tfree(data->helper->argv);\n+\t\tfree(data->helper);\n+\t\tdata->helper = NULL;\n+\t}\n+\treturn 0;\n+}\n+\n+static int fetch_with_fetch(struct transport *transport,\n+\t\t\t    int nr_heads, const struct ref **to_fetch)\n+{\n+\tstruct child_process *helper = get_helper(transport);\n+\tFILE *file = fdopen(helper->out, \"r\");\n+\tint i;\n+\tstruct strbuf buf = STRBUF_INIT;\n+\n+\tfor (i = 0; i < nr_heads; i++) {\n+\t\tstruct ref *posn = to_fetch[i];\n+\t\tif (posn->status & REF_STATUS_UPTODATE)\n+\t\t\tcontinue;\n+\t\twrite_in_full(helper->in, \"fetch \", 6);\n+\t\twrite_in_full(helper->in, sha1_to_hex(posn->old_sha1), 40);\n+\t\twrite_in_full(helper->in, \" \", 1);\n+\t\twrite_in_full(helper->in, posn->name, strlen(posn->name));\n+\t\twrite_in_full(helper->in, \"\\n\", 1);\n+\t\tif (strbuf_getline(&buf, file, '\\n') == EOF)\n+\t\t\texit(128); /* child died, message supplied already */\n+\t}\n+\treturn 0;\n+}\n+\n+static int fetch(struct transport *transport,\n+\t\t int nr_heads, const struct ref **to_fetch)\n+{\n+\tstruct helper_data *data = transport->data;\n+\tint i, count;\n+\n+\tcount = 0;\n+\tfor (i = 0; i < nr_heads; i++)\n+\t\tif (!(to_fetch[i]->status & REF_STATUS_UPTODATE))\n+\t\t\tcount++;\n+\n+\tif (!count)\n+\t\treturn 0;\n+\n+\tif (data->fetch)\n+\t\treturn fetch_with_fetch(transport, nr_heads, to_fetch);\n+\n+\treturn -1;\n+}\n+\n+static struct ref *get_refs_list(struct transport *transport, int for_push)\n+{\n+\tstruct child_process *helper;\n+\tstruct ref *ret = NULL;\n+\tstruct ref **tail = &ret;\n+\tstruct ref *posn;\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tFILE *file;\n+\n+\thelper = get_helper(transport);\n+\twrite_in_full(helper->in, \"list\\n\", 5);\n+\n+\tfile = fdopen(helper->out, \"r\");\n+\twhile (1) {\n+\t\tchar *eov, *eon;\n+\t\tif (strbuf_getline(&buf, file, '\\n') == EOF)\n+\t\t\texit(128); /* child died, message supplied already */\n+\n+\t\tif (!*buf.buf)\n+\t\t\tbreak;\n+\n+\t\teov = strchr(buf.buf, ' ');\n+\t\tif (!eov)\n+\t\t\tdie(\"Malformed response in ref list: %s\", buf.buf);\n+                eon = strchr(eov + 1, ' ');\n+\t\t*eov = '\\0';\n+                if (eon)\n+                        *eon = '\\0';\n+\t\t*tail = alloc_ref(eov + 1);\n+\t\tif (buf.buf[0] == '@')\n+\t\t\t(*tail)->symref = xstrdup(buf.buf + 1);\n+\t\telse if (buf.buf[0] != '?')\n+\t\t\tget_sha1_hex(buf.buf, (*tail)->old_sha1);\n+\t\ttail = &((*tail)->next);\n+\t}\n+\tstrbuf_release(&buf);\n+\n+\tfor (posn = ret; posn; posn = posn->next)\n+\t\tresolve_remote_symref(posn, ret);\n+\n+\treturn ret;\n+}\n+\n+int transport_helper_init(struct transport *transport)\n+{\n+\tstruct helper_data *data = xcalloc(sizeof(*data), 1);\n+\tchar *eom = strchr(transport->url, ':');\n+\tif (!eom)\n+\t\treturn -1;\n+\tdata->name = xstrndup(transport->url, eom - transport->url);\n+\n+\ttransport->data = data;\n+\ttransport->get_refs_list = get_refs_list;\n+\ttransport->fetch = fetch;\n+\ttransport->disconnect = disconnect_helper;\n+\treturn 0;\n+}\ndiff --git a/transport.h b/transport.h\nindex 51b5397..df87264 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -77,4 +77,7 @@ void transport_unlock_pack(struct transport *transport);\n int transport_disconnect(struct transport *transport);\n char *transport_anonymize_url(const char *url);\n \n+/* Transport methods defined outside transport.c */\n+int transport_helper_init(struct transport *transport);\n+\n #endif\n-- \n1.6.4.rc3.27.g95032.dirty\n"}]}