{"thread":{"id":"48348","subject":"[PATCH 0/3] optionally send server-options when using v2","startedAt":"2018-04-23T22:46:35Z","lastAt":"2018-04-23T22:47:59Z","messageCount":5,"participants":["Brandon Williams"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"345515","messageId":"20180423224624.213341-1-bmwill@google.com","threadId":"48348","inReplyTo":null,"subject":"[PATCH 0/3] optionally send server-options when using v2","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-04-23T22:46:21Z","receivedAt":"2018-04-23T22:46:35Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Building on top of protocol version 2 this series adds the ability to\noptionally send server specific options when using protocol v2. This\nresembles the \"push-options\" feature except server options are sent as\ncapability lines during a command request allowing for all current and\nfuture commands to benefit from sending arbitrary server options (and\nnot requiring that sending server specific options be re-implemented for\neach and every command that may want to make use of them in the future).\n\nThese options can be provided by the user via the command line by giving\n\"-o <option>\" or \"--server-option=<option>\" to either ls-remote or\nfetch.\n\nCommand request example:\n\n\tcommand=fetch\n\tserver-option=hello\n\tserver-option=world\n\t0001\n\twant A\n\twant B\n\thave X\n\thave Y\n\t0000\n\nThese options are only transmitted to the remote end when communicating\nusing protocol version 2.\n\nBrandon Williams (3):\n  serve: introduce the server-option capability\n  ls-remote: send server options when using protocol v2\n  fetch: send server options when using protocol v2\n\n Documentation/fetch-options.txt         |  8 +++++++\n Documentation/git-ls-remote.txt         |  8 +++++++\n Documentation/technical/protocol-v2.txt | 10 ++++++++\n builtin/fetch.c                         |  5 ++++\n builtin/ls-remote.c                     |  4 ++++\n connect.c                               |  9 ++++++-\n fetch-pack.c                            |  7 ++++++\n fetch-pack.h                            |  1 +\n remote.h                                |  4 +++-\n serve.c                                 |  1 +\n t/t5701-git-serve.sh                    | 21 ++++++++++++++++\n t/t5702-protocol-v2.sh                  | 32 +++++++++++++++++++++++++\n transport.c                             |  3 ++-\n transport.h                             |  6 +++++\n 14 files changed, 116 insertions(+), 3 deletions(-)\n\n-- \n2.17.0.484.g0c8726318c-goog\n\n"},{"id":"345516","messageId":"20180423224624.213341-2-bmwill@google.com","threadId":"48348","inReplyTo":"20180423224624.213341-1-bmwill@google.com","subject":"[PATCH 1/3] serve: introduce the server-option capability","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-04-23T22:46:22Z","receivedAt":"2018-04-23T22:46:38Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Introduce the \"server-option\" capability to protocol version 2.  This\nenables future clients the ability to send server specific options in\ncommand requests when using protocol version 2.\n\nSigned-off-by: Brandon Williams <bmwill@google.com>\n---\n Documentation/technical/protocol-v2.txt | 10 ++++++++++\n serve.c                                 |  1 +\n t/t5701-git-serve.sh                    | 21 +++++++++++++++++++++\n 3 files changed, 32 insertions(+)\n\ndiff --git a/Documentation/technical/protocol-v2.txt b/Documentation/technical/protocol-v2.txt\nindex 136179d7d..d7b6f38e0 100644\n--- a/Documentation/technical/protocol-v2.txt\n+++ b/Documentation/technical/protocol-v2.txt\n@@ -393,3 +393,13 @@ header.\n \t\t1 - pack data\n \t\t2 - progress messages\n \t\t3 - fatal error message just before stream aborts\n+\n+ server-option\n+~~~~~~~~~~~~~~~\n+\n+If advertised, indicates that any number of server specific options can be\n+included in a request.  This is done by sending each option as a\n+\"server-option=<option>\" capability line in the capability-list section of\n+a request.\n+\n+The provided options must not contain a NUL or LF character.\ndiff --git a/serve.c b/serve.c\nindex a5a7b2f7d..bda085f09 100644\n--- a/serve.c\n+++ b/serve.c\n@@ -56,6 +56,7 @@ static struct protocol_capability capabilities[] = {\n \t{ \"agent\", agent_advertise, NULL },\n \t{ \"ls-refs\", always_advertise, ls_refs },\n \t{ \"fetch\", upload_pack_advertise, upload_pack_v2 },\n+\t{ \"server-option\", always_advertise, NULL },\n };\n \n static void advertise_capabilities(void)\ndiff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh\nindex 72d7bc562..011a5796d 100755\n--- a/t/t5701-git-serve.sh\n+++ b/t/t5701-git-serve.sh\n@@ -10,6 +10,7 @@ test_expect_success 'test capability advertisement' '\n \tagent=git/$(git version | cut -d\" \" -f3)\n \tls-refs\n \tfetch=shallow\n+\tserver-option\n \t0000\n \tEOF\n \n@@ -173,4 +174,24 @@ test_expect_success 'symrefs parameter' '\n \ttest_cmp actual expect\n '\n \n+test_expect_success 'sending server-options' '\n+\ttest-pkt-line pack >in <<-EOF &&\n+\tcommand=ls-refs\n+\tserver-option=hello\n+\tserver-option=world\n+\t0001\n+\tref-prefix HEAD\n+\t0000\n+\tEOF\n+\n+\tcat >expect <<-EOF &&\n+\t$(git rev-parse HEAD) HEAD\n+\t0000\n+\tEOF\n+\n+\tgit serve --stateless-rpc <in >out &&\n+\ttest-pkt-line unpack <out >actual &&\n+\ttest_cmp actual expect\n+'\n+\n test_done\n-- \n2.17.0.484.g0c8726318c-goog\n\n"},{"id":"345517","messageId":"20180423224624.213341-3-bmwill@google.com","threadId":"48348","inReplyTo":"20180423224624.213341-1-bmwill@google.com","subject":"[PATCH 2/3] ls-remote: send server options when using protocol v2","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-04-23T22:46:23Z","receivedAt":"2018-04-23T22:46:41Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Teach ls-remote to optionally accept server options by specifying them\non the cmdline via '-o' or '--server-option'.  These server options are\nsent to the remote end when querying for the remote end's refs using\nprotocol version 2.\n\nIf communicating using a protocol other than v2 the provided options are\nignored and not sent to the remote end.\n\nSigned-off-by: Brandon Williams <bmwill@google.com>\n---\n Documentation/git-ls-remote.txt |  8 ++++++++\n builtin/ls-remote.c             |  4 ++++\n connect.c                       |  9 ++++++++-\n remote.h                        |  4 +++-\n t/t5702-protocol-v2.sh          | 16 ++++++++++++++++\n transport.c                     |  2 +-\n transport.h                     |  6 ++++++\n 7 files changed, 46 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-ls-remote.txt b/Documentation/git-ls-remote.txt\nindex 5f2628c8f..e5defb1b2 100644\n--- a/Documentation/git-ls-remote.txt\n+++ b/Documentation/git-ls-remote.txt\n@@ -60,6 +60,14 @@ OPTIONS\n \tupload-pack only shows the symref HEAD, so it will be the only\n \tone shown by ls-remote.\n \n+-o <option>::\n+--server-option=<option>::\n+\tTransmit the given string to the server when communicating using\n+\tprotocol version 2.  The given string must not contain a NUL or LF\n+\tcharacter.\n+\tWhen multiple `--server-option=<option>` are given, they are all\n+\tsent to the other side in the order listed on the command line.\n+\n <repository>::\n \tThe \"remote\" repository to query.  This parameter can be\n \teither a URL or the name of a remote (see the GIT URLS and\ndiff --git a/builtin/ls-remote.c b/builtin/ls-remote.c\nindex 380c18027..3150bfb92 100644\n--- a/builtin/ls-remote.c\n+++ b/builtin/ls-remote.c\n@@ -45,6 +45,7 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \tconst char *uploadpack = NULL;\n \tconst char **pattern = NULL;\n \tstruct argv_array ref_prefixes = ARGV_ARRAY_INIT;\n+\tstruct string_list server_options = STRING_LIST_INIT_DUP;\n \n \tstruct remote *remote;\n \tstruct transport *transport;\n@@ -67,6 +68,7 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \t\t\t      2, PARSE_OPT_NOCOMPLETE),\n \t\tOPT_BOOL(0, \"symref\", &show_symref_target,\n \t\t\t N_(\"show underlying ref in addition to the object pointed by it\")),\n+\t\tOPT_STRING_LIST('o', \"server-option\", &server_options, N_(\"server-specific\"), N_(\"option to transmit\")),\n \t\tOPT_END()\n \t};\n \n@@ -107,6 +109,8 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \ttransport = transport_get(remote, NULL);\n \tif (uploadpack != NULL)\n \t\ttransport_set_option(transport, TRANS_OPT_UPLOADPACK, uploadpack);\n+\tif (server_options.nr)\n+\t\ttransport->server_options = &server_options;\n \n \tref = transport_get_remote_refs(transport, &ref_prefixes);\n \tif (transport_disconnect(transport))\ndiff --git a/connect.c b/connect.c\nindex 54971166a..3000768c7 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -408,7 +408,8 @@ static int process_ref_v2(const char *line, struct ref ***list)\n \n struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,\n \t\t\t     struct ref **list, int for_push,\n-\t\t\t     const struct argv_array *ref_prefixes)\n+\t\t\t     const struct argv_array *ref_prefixes,\n+\t\t\t     const struct string_list *server_options)\n {\n \tint i;\n \t*list = NULL;\n@@ -419,6 +420,12 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,\n \tif (server_supports_v2(\"agent\", 0))\n \t\tpacket_write_fmt(fd_out, \"agent=%s\", git_user_agent_sanitized());\n \n+\tif (server_options && server_options->nr &&\n+\t    server_supports_v2(\"server-option\", 1))\n+\t\tfor (i = 0; i < server_options->nr; i++)\n+\t\t\tpacket_write_fmt(fd_out, \"server-option=%s\",\n+\t\t\t\t\t server_options->items[i].string);\n+\n \tpacket_delim(fd_out);\n \t/* When pushing we don't want to request the peeled tags */\n \tif (!for_push)\ndiff --git a/remote.h b/remote.h\nindex 2b3180f94..93dd97e25 100644\n--- a/remote.h\n+++ b/remote.h\n@@ -153,6 +153,7 @@ void free_refs(struct ref *ref);\n struct oid_array;\n struct packet_reader;\n struct argv_array;\n+struct string_list;\n extern struct ref **get_remote_heads(struct packet_reader *reader,\n \t\t\t\t     struct ref **list, unsigned int flags,\n \t\t\t\t     struct oid_array *extra_have,\n@@ -161,7 +162,8 @@ extern struct ref **get_remote_heads(struct packet_reader *reader,\n /* Used for protocol v2 in order to retrieve refs from a remote */\n extern struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,\n \t\t\t\t    struct ref **list, int for_push,\n-\t\t\t\t    const struct argv_array *ref_prefixes);\n+\t\t\t\t    const struct argv_array *ref_prefixes,\n+\t\t\t\t    const struct string_list *server_options);\n \n int resolve_remote_symref(struct ref *ref, struct ref *list);\n int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid);\ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 56f7c3c32..71ef1aee1 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -154,6 +154,22 @@ test_expect_success 'ref advertisment is filtered with ls-remote using protocol\n \ttest_cmp actual expect\n '\n \n+test_expect_success 'server-options are sent when using ls-remote' '\n+\ttest_when_finished \"rm -f log\" &&\n+\n+\tGIT_TRACE_PACKET=\"$(pwd)/log\" git -c protocol.version=2 \\\n+\t\tls-remote -o hello -o world \"file://$(pwd)/file_parent\" master >actual &&\n+\n+\tcat >expect <<-EOF &&\n+\t$(git -C file_parent rev-parse refs/heads/master)$(printf \"\\t\")refs/heads/master\n+\tEOF\n+\n+\ttest_cmp actual expect &&\n+\tgrep \"server-option=hello\" log &&\n+\tgrep \"server-option=world\" log\n+'\n+\n+\n test_expect_success 'clone with file:// using protocol v2' '\n \ttest_when_finished \"rm -f log\" &&\n \ndiff --git a/transport.c b/transport.c\nindex 4d8beaaab..42fd468f3 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -218,7 +218,7 @@ static struct ref *get_refs_via_connect(struct transport *transport, int for_pus\n \tswitch (data->version) {\n \tcase protocol_v2:\n \t\tget_remote_refs(data->fd[1], &reader, &refs, for_push,\n-\t\t\t\tref_prefixes);\n+\t\t\t\tref_prefixes, transport->server_options);\n \t\tbreak;\n \tcase protocol_v1:\n \tcase protocol_v0:\ndiff --git a/transport.h b/transport.h\nindex e783cfa07..73a7be3c8 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -71,6 +71,12 @@ struct transport {\n \t */\n \tconst struct string_list *push_options;\n \n+\t/*\n+\t * These strings will be passed to the remote side on each command\n+\t * request, if both sides support the server-option capability.\n+\t */\n+\tconst struct string_list *server_options;\n+\n \tchar *pack_lockfile;\n \tsigned verbose : 3;\n \t/**\n-- \n2.17.0.484.g0c8726318c-goog\n\n"},{"id":"345518","messageId":"20180423224624.213341-4-bmwill@google.com","threadId":"48348","inReplyTo":"20180423224624.213341-1-bmwill@google.com","subject":"[PATCH 3/3] fetch: send server options when using protocol v2","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-04-23T22:46:24Z","receivedAt":"2018-04-23T22:46:44Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Teach fetch to optionally accept server options by specifying them on\nthe cmdline via '-o' or '--server-option'.  These server options are\nsent to the remote end when performing a fetch communicating using\nprotocol version 2.\n\nIf communicating using a protocol other than v2 the provided options are\nignored and not sent to the remote end.\n\nSigned-off-by: Brandon Williams <bmwill@google.com>\n---\n Documentation/fetch-options.txt |  8 ++++++++\n builtin/fetch.c                 |  5 +++++\n fetch-pack.c                    |  7 +++++++\n fetch-pack.h                    |  1 +\n t/t5702-protocol-v2.sh          | 16 ++++++++++++++++\n transport.c                     |  1 +\n 6 files changed, 38 insertions(+)\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex 8631e365f..97d3217df 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -188,6 +188,14 @@ endif::git-pull[]\n \tis specified. This flag forces progress status even if the\n \tstandard error stream is not directed to a terminal.\n \n+-o <option>::\n+--server-option=<option>::\n+\tTransmit the given string to the server when communicating using\n+\tprotocol version 2.  The given string must not contain a NUL or LF\n+\tcharacter.\n+\tWhen multiple `--server-option=<option>` are given, they are all\n+\tsent to the other side in the order listed on the command line.\n+\n -4::\n --ipv4::\n \tUse IPv4 addresses only, ignoring IPv6 addresses.\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 7ee83ac0f..5a6f6b2dc 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -62,6 +62,7 @@ static int shown_url = 0;\n static int refmap_alloc, refmap_nr;\n static const char **refmap_array;\n static struct list_objects_filter_options filter_options;\n+static struct string_list server_options = STRING_LIST_INIT_DUP;\n \n static int git_fetch_config(const char *k, const char *v, void *cb)\n {\n@@ -170,6 +171,7 @@ static struct option builtin_fetch_options[] = {\n \t\t N_(\"accept refs that update .git/shallow\")),\n \t{ OPTION_CALLBACK, 0, \"refmap\", NULL, N_(\"refmap\"),\n \t  N_(\"specify fetch refmap\"), PARSE_OPT_NONEG, parse_refmap_arg },\n+\tOPT_STRING_LIST('o', \"server-option\", &server_options, N_(\"server-specific\"), N_(\"option to transmit\")),\n \tOPT_SET_INT('4', \"ipv4\", &family, N_(\"use IPv4 addresses only\"),\n \t\t\tTRANSPORT_FAMILY_IPV4),\n \tOPT_SET_INT('6', \"ipv6\", &family, N_(\"use IPv6 addresses only\"),\n@@ -1417,6 +1419,9 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru\n \t\t}\n \t}\n \n+\tif (server_options.nr)\n+\t\tgtransport->server_options = &server_options;\n+\n \tsigchain_push_common(unlock_pack_on_signal);\n \tatexit(unlock_pack);\n \trefspec = parse_fetch_refspec(ref_nr, refs);\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 216d1368b..199eb8a1d 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -1174,6 +1174,13 @@ static int send_fetch_request(int fd_out, const struct fetch_pack_args *args,\n \t\tpacket_buf_write(&req_buf, \"command=fetch\");\n \tif (server_supports_v2(\"agent\", 0))\n \t\tpacket_buf_write(&req_buf, \"agent=%s\", git_user_agent_sanitized());\n+\tif (args->server_options && args->server_options->nr &&\n+\t    server_supports_v2(\"server-option\", 1)) {\n+\t\tint i;\n+\t\tfor (i = 0; i < args->server_options->nr; i++)\n+\t\t\tpacket_write_fmt(fd_out, \"server-option=%s\",\n+\t\t\t\t\t args->server_options->items[i].string);\n+\t}\n \n \tpacket_buf_delim(&req_buf);\n \tif (args->use_thin_pack)\ndiff --git a/fetch-pack.h b/fetch-pack.h\nindex 667024a76..f4ba851c6 100644\n--- a/fetch-pack.h\n+++ b/fetch-pack.h\n@@ -15,6 +15,7 @@ struct fetch_pack_args {\n \tconst char *deepen_since;\n \tconst struct string_list *deepen_not;\n \tstruct list_objects_filter_options filter_options;\n+\tconst struct string_list *server_options;\n \tunsigned deepen_relative:1;\n \tunsigned quiet:1;\n \tunsigned keep_pack:1;\ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 71ef1aee1..dbfd0691c 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -217,6 +217,22 @@ test_expect_success 'ref advertisment is filtered during fetch using protocol v2\n \t! grep \"refs/tags/three\" log\n '\n \n+test_expect_success 'server-options are sent when fetching' '\n+\ttest_when_finished \"rm -f log\" &&\n+\n+\ttest_commit -C file_parent four &&\n+\n+\tGIT_TRACE_PACKET=\"$(pwd)/log\" git -C file_child -c protocol.version=2 \\\n+\t\tfetch -o hello -o world origin master &&\n+\n+\tgit -C file_child log -1 --format=%s origin/master >actual &&\n+\tgit -C file_parent log -1 --format=%s >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tgrep \"server-option=hello\" log &&\n+\tgrep \"server-option=world\" log\n+'\n+\n # Test protocol v2 with 'http://' transport\n #\n . \"$TEST_DIRECTORY\"/lib-httpd.sh\ndiff --git a/transport.c b/transport.c\nindex 42fd468f3..b088bcd49 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -266,6 +266,7 @@ static int fetch_refs_via_pack(struct transport *transport,\n \targs.no_dependents = data->options.no_dependents;\n \targs.filter_options = data->options.filter_options;\n \targs.stateless_rpc = transport->stateless_rpc;\n+\targs.server_options = transport->server_options;\n \n \tif (!data->got_remote_heads)\n \t\trefs_tmp = get_refs_via_connect(transport, 0, NULL);\n-- \n2.17.0.484.g0c8726318c-goog\n\n"},{"id":"345519","messageId":"20180423224753.GA29052@google.com","threadId":"48348","inReplyTo":"20180423224624.213341-1-bmwill@google.com","subject":"Re: [PATCH 0/3] optionally send server-options when using v2","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-04-23T22:47:53Z","receivedAt":"2018-04-23T22:47:59Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 04/23, Brandon Williams wrote:\n> Building on top of protocol version 2 this series adds the ability to\n> optionally send server specific options when using protocol v2. This\n> resembles the \"push-options\" feature except server options are sent as\n> capability lines during a command request allowing for all current and\n> future commands to benefit from sending arbitrary server options (and\n> not requiring that sending server specific options be re-implemented for\n> each and every command that may want to make use of them in the future).\n> \n> These options can be provided by the user via the command line by giving\n> \"-o <option>\" or \"--server-option=<option>\" to either ls-remote or\n> fetch.\n> \n> Command request example:\n> \n> \tcommand=fetch\n> \tserver-option=hello\n> \tserver-option=world\n> \t0001\n> \twant A\n> \twant B\n> \thave X\n> \thave Y\n> \t0000\n> \n> These options are only transmitted to the remote end when communicating\n> using protocol version 2.\n\nForgot to mention that this series is based on current upstream master\n(fe0a9eaf3) and a merge of origin/bw/protocol-v2.\n\n> \n> Brandon Williams (3):\n>   serve: introduce the server-option capability\n>   ls-remote: send server options when using protocol v2\n>   fetch: send server options when using protocol v2\n> \n>  Documentation/fetch-options.txt         |  8 +++++++\n>  Documentation/git-ls-remote.txt         |  8 +++++++\n>  Documentation/technical/protocol-v2.txt | 10 ++++++++\n>  builtin/fetch.c                         |  5 ++++\n>  builtin/ls-remote.c                     |  4 ++++\n>  connect.c                               |  9 ++++++-\n>  fetch-pack.c                            |  7 ++++++\n>  fetch-pack.h                            |  1 +\n>  remote.h                                |  4 +++-\n>  serve.c                                 |  1 +\n>  t/t5701-git-serve.sh                    | 21 ++++++++++++++++\n>  t/t5702-protocol-v2.sh                  | 32 +++++++++++++++++++++++++\n>  transport.c                             |  3 ++-\n>  transport.h                             |  6 +++++\n>  14 files changed, 116 insertions(+), 3 deletions(-)\n> \n> -- \n> 2.17.0.484.g0c8726318c-goog\n> \n\n-- \nBrandon Williams\n"}]}