{"thread":{"id":"12089","subject":"[PATCH 5/6] Add side-band-64k extension to send-pack/receive-pack for hook output","startedAt":"2008-02-14T06:23:01Z","lastAt":"2008-02-14T06:23:01Z","messageCount":1,"participants":["Shawn O. Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":6},"messages":[{"id":"68711","messageId":"20080214062301.GE30516@spearce.org","threadId":"12089","inReplyTo":null,"subject":"[PATCH 5/6] Add side-band-64k extension to send-pack/receive-pack for hook output","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-02-14T06:23:01Z","receivedAt":"2008-02-14T06:23:01Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"The fetch-pack/upload-pack protocol currently uses the side-band-64k\nprotocol extension to pass the stderr output from the upload-pack\nside of the connection down to the fetch-pack side.  This lets the\nremote send progress messages from pack-objects down to the user,\nso they know the server is working on their behalf and don't just\nbreak a connection off.\n\nIn the past we haven't needed this secondary channel during push,\nas most users only push over SSH and there is already a multiplexed\nstdout and stderr available as part of the native SSH protocol and\nclient/server implementations.  All output from hooks executed\nby receive-pack on the remote side is sent to stderr, which is\nautomatically routed back to the user's terminal by SSH, completely\nbypassing fetch-pack.\n\nIn Git 1.5.0 Linus allowed users to enable receive-pack as a service\nin git-daemon, which is useful in friendly office LAN situations\nwhere all of the users can be implicitly trusted.  However as\ngit-daemon runs on a straight TCP/IP channel there is no automated\nmultiplexing to bring the stderr output from any hooks back to the\npush client.\n\nThis change adds the sideband multiplexer/demultiplexer and uses\nit to wrap the status response from receive-pack into band #1,\nwhich is the standard payload band.  To keep this change smaller\nwe do not wrap the hook output yet, which should be on band #2,\nthe standard \"stderr/verbose/progress\" band.\n\nOne of the complexities of arranging the status response data into\nband #1 is the use of pkt-line for both the status reponse and for\nthe sideband multiplexing.  We have to wrap the pkt-lines inside of\nanother set of pkt-lines (but with the additional band field) to get\nthe data correctly sent over the wire.  Further when we do actually\nflush we now have two flushes; one to end the status and another to\nend the multiplexing.  The first flush is actually still part of band\n\nThe side-band-64k extension is only enabled if both sides have the\ncapability, and only if send-pack sent commands to receive-pack\nfor processing.  This allows both sides to transparently work with\nolder clients or servers.\n\nSigned-off-by: Shawn O. Pearce <spearce@spearce.org>\n---\n builtin-send-pack.c |   37 +++++++++++++++++++++++++++++++------\n pkt-line.c          |   28 +++++++++++++++++++++++++---\n pkt-line.h          |    2 ++\n receive-pack.c      |   17 ++++++++++++-----\n 4 files changed, 70 insertions(+), 14 deletions(-)\n\ndiff --git a/builtin-send-pack.c b/builtin-send-pack.c\nindex 63fbcd2..5c4844c 100644\n--- a/builtin-send-pack.c\n+++ b/builtin-send-pack.c\n@@ -3,6 +3,7 @@\n #include \"tag.h\"\n #include \"refs.h\"\n #include \"pkt-line.h\"\n+#include \"sideband.h\"\n #include \"run-command.h\"\n #include \"remote.h\"\n #include \"send-pack.h\"\n@@ -374,12 +375,18 @@ static int refs_pushed(struct ref *ref)\n \treturn 0;\n }\n \n+static int sideband_demux(int fd, void *data)\n+{\n+\treturn recv_sideband(\"send-pack\", *((int*)data), fd, 2);\n+}\n+\n static int do_send_pack(int in, int out, struct remote *remote, const char *dest, int nr_refspec, const char **refspec)\n {\n \tstruct ref *ref;\n \tint new_refs;\n \tint allow_deleting_refs = 0;\n \tint expect_status_report = 0;\n+\tint use_sideband = 0;\n \tint flags = MATCH_REFS_NONE;\n \tint pushing = 0;\n \tint ret;\n@@ -398,6 +405,8 @@ static int do_send_pack(int in, int out, struct remote *remote, const char *dest\n \t\texpect_status_report = 1;\n \tif (server_supports(\"delete-refs\"))\n \t\tallow_deleting_refs = 1;\n+\tif (server_supports(\"side-band-64k\"))\n+\t\tuse_sideband = 1;\n \n \t/* match them up */\n \tif (!remote_tail)\n@@ -478,9 +487,10 @@ static int do_send_pack(int in, int out, struct remote *remote, const char *dest\n \t\t\tchar *new_hex = sha1_to_hex(ref->new_sha1);\n \n \t\t\tif (!pushing)\n-\t\t\t\tpacket_write(out, \"%s %s %s%c%s\",\n+\t\t\t\tpacket_write(out, \"%s %s %s%c%s%s\",\n \t\t\t\t\told_hex, new_hex, ref->name, 0,\n-\t\t\t\t\t(expect_status_report ? \" report-status\" : \"\")\n+\t\t\t\t\t(expect_status_report ? \" report-status\" : \"\"),\n+\t\t\t\t\t(use_sideband ? \" side-band-64k\" : \"\")\n \t\t\t\t);\n \t\t\telse\n \t\t\t\tpacket_write(out, \"%s %s %s\",\n@@ -501,11 +511,26 @@ static int do_send_pack(int in, int out, struct remote *remote, const char *dest\n \t}\n \tclose(out);\n \n-\tif (pushing && expect_status_report)\n-\t\tret = receive_status(in, remote_refs);\n-\telse\n-\t\tret = 0;\n+\tret = 0;\n+\tif (pushing) {\n+\t\tstruct async demux;\n+\t\tint demux_in = in;\n+\n+\t\tif (use_sideband) {\n+\t\t\tmemset(&demux, 0, sizeof(demux));\n+\t\t\tdemux.proc = sideband_demux;\n+\t\t\tdemux.data = &demux_in;\n+\t\t\tif (start_async(&demux))\n+\t\t\t\tdie(\"send-pack: unable to start sideband demultiplexer\");\n+\t\t\tin = demux.out;\n+\t\t}\n+\n+\t\tif (expect_status_report)\n+\t\t\tret = receive_status(in, remote_refs);\n \n+\t\tif (use_sideband && (finish_async(&demux) || close(demux.out)))\n+\t\t\tdie(\"error in sideband demultiplexer\");\n+\t}\n \tprint_push_status(dest, remote_refs);\n \n \tif (!args.dry_run && remote) {\ndiff --git a/pkt-line.c b/pkt-line.c\nindex 5917e1d..0b5f422 100644\n--- a/pkt-line.c\n+++ b/pkt-line.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"pkt-line.h\"\n+#include \"sideband.h\"\n \n /*\n  * Write a packetized stream, where each line is preceded by\n@@ -37,14 +38,23 @@ ssize_t safe_write(int fd, const void *buf, ssize_t n)\n  * If we buffered things up above (we don't, but we should),\n  * we'd flush it here\n  */\n+void packet_sideband_flush(int fd, int band)\n+{\n+\tif (band >= 0)\n+\t\tsend_sideband(fd, band, \"0000\", 4, LARGE_PACKET_MAX);\n+\telse\n+\t\tsafe_write(fd, \"0000\", 4);\n+}\n+\n void packet_flush(int fd)\n {\n-\tsafe_write(fd, \"0000\", 4);\n+\tpacket_sideband_flush(fd, -1);\n }\n \n #define hex(a) (hexchar[(a) & 15])\n static void packet_vwrite(\n \tint fd,\n+\tint band,\n \tconst char *fmt,\n \tva_list args)\n {\n@@ -60,7 +70,10 @@ static void packet_vwrite(\n \tbuffer[1] = hex(n >> 8);\n \tbuffer[2] = hex(n >> 4);\n \tbuffer[3] = hex(n);\n-\tsafe_write(fd, buffer, n);\n+\tif (band >= 0)\n+\t\tsend_sideband(fd, band, buffer, n, LARGE_PACKET_MAX);\n+\telse\n+\t\tsafe_write(fd, buffer, n);\n }\n \n void packet_write(int fd, const char *fmt, ...)\n@@ -68,7 +81,16 @@ void packet_write(int fd, const char *fmt, ...)\n \tva_list args;\n \n \tva_start(args, fmt);\n-\tpacket_vwrite(fd, fmt, args);\n+\tpacket_vwrite(fd, -1, fmt, args);\n+\tva_end(args);\n+}\n+\n+void packet_sideband_write(int fd, int band, const char *fmt, ...)\n+{\n+\tva_list args;\n+\n+\tva_start(args, fmt);\n+\tpacket_vwrite(fd, band, fmt, args);\n \tva_end(args);\n }\n \ndiff --git a/pkt-line.h b/pkt-line.h\nindex 9df653f..0b7ded1 100644\n--- a/pkt-line.h\n+++ b/pkt-line.h\n@@ -7,7 +7,9 @@\n  * Silly packetized line writing interface\n  */\n void packet_flush(int fd);\n+void packet_sideband_flush(int fd, int band);\n void packet_write(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));\n+void packet_sideband_write(int fd, int band, const char *fmt, ...) __attribute__((format (printf, 3, 4)));\n \n int packet_read_line(int fd, char *buffer, unsigned size);\n ssize_t safe_write(int, const void *, ssize_t);\ndiff --git a/receive-pack.c b/receive-pack.c\nindex 7380395..8962c4c 100644\n--- a/receive-pack.c\n+++ b/receive-pack.c\n@@ -2,6 +2,7 @@\n #include \"pack.h\"\n #include \"refs.h\"\n #include \"pkt-line.h\"\n+#include \"sideband.h\"\n #include \"run-command.h\"\n #include \"exec_cmd.h\"\n #include \"commit.h\"\n@@ -14,8 +15,9 @@ static int receive_unpack_limit = -1;\n static int transfer_unpack_limit = -1;\n static int unpack_limit = 100;\n static int report_status;\n+static int use_sideband;\n \n-static char capabilities[] = \" report-status delete-refs \";\n+static char capabilities[] = \" report-status delete-refs side-band-64k \";\n static int capabilities_sent;\n \n static int receive_pack_config(const char *var, const char *value)\n@@ -321,6 +323,8 @@ static void read_head_info(void)\n \t\t\tconst char *reqcap = refname + reflen + 1;\n \t\t\tif (strstr(reqcap, \"report-status\"))\n \t\t\t\treport_status = 1;\n+\t\t\tif (strstr(reqcap, \"side-band-64k\"))\n+\t\t\t\tuse_sideband = LARGE_PACKET_MAX;\n \t\t}\n \t\tcmd = xmalloc(sizeof(struct command) + len - 80);\n \t\thashcpy(cmd->old_sha1, old_sha1);\n@@ -426,18 +430,19 @@ static const char *unpack(void)\n \n static void report(const char *unpack_status)\n {\n+\tint band = use_sideband ? 1 : -1;\n \tstruct command *cmd;\n-\tpacket_write(1, \"unpack %s\\n\",\n+\tpacket_sideband_write(1, band, \"unpack %s\\n\",\n \t\t     unpack_status ? unpack_status : \"ok\");\n \tfor (cmd = commands; cmd; cmd = cmd->next) {\n \t\tif (!cmd->error_string)\n-\t\t\tpacket_write(1, \"ok %s\\n\",\n+\t\t\tpacket_sideband_write(1, band, \"ok %s\\n\",\n \t\t\t\t     cmd->ref_name);\n \t\telse\n-\t\t\tpacket_write(1, \"ng %s %s\\n\",\n+\t\t\tpacket_sideband_write(1, band, \"ng %s %s\\n\",\n \t\t\t\t     cmd->ref_name, cmd->error_string);\n \t}\n-\tpacket_flush(1);\n+\tpacket_sideband_flush(1, band);\n }\n \n static int delete_only(struct command *cmd)\n@@ -501,6 +506,8 @@ int main(int argc, char **argv)\n \t\t\treport(unpack_status);\n \t\trun_hook(post_receive_hook);\n \t\trun_update_post_hook(commands);\n+\t\tif (use_sideband)\n+\t\t\tpacket_flush(1);\n \t}\n \treturn 0;\n }\n-- \n1.5.4.1.1309.g833c2\n"}]}