threads / patch / 36360

v10, 12 partstrailer: read and process config information

Subject: [PATCH v10 03/12] trailer: read and process config information

## tl;dr

33 messages between Apr 6, 2014 and May 27, 2014. Diffs are folded; open one to read it.

replies: 32people: 5as markdown or json

Christian Couder· Apr 6, 2014, 17:01 UTC · lore

[PATCH v10 00/12] Add interpret-trailers builtin

This patch series implements a new command:
        git interpret-trailers

and an infrastructure to process trailers that can be reused, for example in "commit.c".

1) Rationale:

This command should help with RFC 822 style headers, called "trailers", that are found at the end of commit messages.

(Note that these headers do not follow and are not intended to follow many rules that are in RFC 822. For example they do not follow the line breaking rules, the encoding rules and probably many other rules.)

For a long time, these trailers have become a de facto standard way to add helpful information into commit messages.

Until now git commit has only supported the well known "Signed-off-by: " trailer, that is used by many projects like the Linux kernel and Git.

It is better to keep builtin/commit.c uncontaminated by any more hard-wired logic, like what we have for the signed-off-by line. Any new things can and should be doable in hooks, and this filter would help writing these hooks.

And that is why the design goal of the filter is to make it at least as powerful as the built-in logic we have for signed-off-by lines; that would allow us to later eject the hard-wired logic for signed-off-by line from the main codepath, if/when we wanted to.

Alternatively, we could build a library-ish API around this filter code and replace the hard-wired logic for signed-off-by line with a call into that API, if/when we wanted to, but that requires (in addition to the "at least as powerful as the built-in logic") that the implementation of this stand-alone filter can be cleanly made into a reusable library, so that is a bit higher bar to cross than "everything can be doable with hooks" alternative.

2) Current state:
Currently the usage string of this command is:
git interpret-trailers [--trim-empty] [(<token>[(=|:)<value>])...]
The following features are implemented:
        - the result is printed on stdout
        - the [<token>[=<value>]>] arguments are interpreted
        - a commit message read from stdin is interpreted
        - the "trailer.<token>.key" options in the config are interpreted
        - the "trailer.<token>.where" options are interpreted
        - the "trailer.<token>.ifExist" options are interpreted
        - the "trailer.<token>.ifMissing" options are interpreted
        - the "trailer.<token>.command" config works
        - $ARG can be used in commands
        - there are 31 tests (4 more than in version 9)
        - there is some documentation
The following features are planned but not yet implemented:
        - add examples in documentation
Possible improvements:
        - integration with "git commit"
        - support GIT_COMMIT_PROTO env variable in commands
3) Changes since version 9, thanks to Jonathan and Junio:
* added 1 test with empty trailers in patch 10/12
* fixed bugs when there was no 'key' in the config in patch
  4/12 and added 2 related tests in patch 10/12
* fixed bug when command failed in patch 9/12 and added 1
  related test in patch 10/12
* added patch 12/12 which add one blank line before the
  trailers if there is not one already

This means code changes only in patches 4/12, 9/12, 10/12 and 12/12.

Christian Couder (12):
  trailer: add data structures and basic functions
  trailer: process trailers from stdin and arguments
  trailer: read and process config information
  trailer: process command line trailer arguments
  trailer: parse trailers from stdin
  trailer: put all the processing together and print
  trailer: add interpret-trailers command
  trailer: add tests for "git interpret-trailers"
  trailer: execute command from 'trailer.<name>.command'
  trailer: add tests for commands in config file
  Documentation: add documentation for 'git interpret-trailers'
  trailer: add blank line before the trailers if needed
 .gitignore                               |   1 +
 Documentation/git-interpret-trailers.txt | 123 ++++++
 Makefile                                 |   2 +
 builtin.h                                |   1 +
 builtin/interpret-trailers.c             |  33 ++
 git.c                                    |   1 +
 t/t7513-interpret-trailers.sh            | 477 +++++++++++++++++++++
 trailer.c                                | 709 +++++++++++++++++++++++++++++++
 trailer.h                                |   6 +
 9 files changed, 1353 insertions(+)
 create mode 100644 Documentation/git-interpret-trailers.txt
 create mode 100644 builtin/interpret-trailers.c
 create mode 100755 t/t7513-interpret-trailers.sh
 create mode 100644 trailer.c
 create mode 100644 trailer.h
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:01 UTC · re: Christian Couder · lore

[PATCH v10 01/12] trailer: add data structures and basic functions

We will use a doubly linked list to store all information about trailers and their configuration.

This way we can easily remove or add trailers to or from trailer lists while traversing the lists in either direction.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Makefile  |  1 +
 trailer.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)
 create mode 100644 trailer.c
Show changes to 2 files +50 −0

Makefile, trailer.c

diff --git a/Makefile b/Makefile
index c5316a3..179be0a 100644
--- a/Makefile
+++ b/Makefile
@@ -879,6 +879,7 @@ LIB_OBJS += submodule.o
 LIB_OBJS += symlinks.o
 LIB_OBJS += tag.o
 LIB_OBJS += trace.o
+LIB_OBJS += trailer.o
 LIB_OBJS += transport.o
 LIB_OBJS += transport-helper.o
 LIB_OBJS += tree-diff.o
diff --git a/trailer.c b/trailer.c
new file mode 100644
index 0000000..db93a63
--- /dev/null
+++ b/trailer.c
@@ -0,0 +1,49 @@
+#include "cache.h"
+/*
+ * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
+ */
+
+enum action_where { WHERE_AFTER, WHERE_BEFORE };
+enum action_if_exists { EXISTS_ADD_IF_DIFFERENT, EXISTS_ADD_IF_DIFFERENT_NEIGHBOR,
+			EXISTS_ADD, EXISTS_OVERWRITE, EXISTS_DO_NOTHING };
+enum action_if_missing { MISSING_ADD, MISSING_DO_NOTHING };
+
+struct conf_info {
+	char *name;
+	char *key;
+	char *command;
+	enum action_where where;
+	enum action_if_exists if_exists;
+	enum action_if_missing if_missing;
+};
+
+struct trailer_item {
+	struct trailer_item *previous;
+	struct trailer_item *next;
+	const char *token;
+	const char *value;
+	struct conf_info conf;
+};
+
+static int same_token(struct trailer_item *a, struct trailer_item *b, int alnum_len)
+{
+	return !strncasecmp(a->token, b->token, alnum_len);
+}
+
+static int same_value(struct trailer_item *a, struct trailer_item *b)
+{
+	return !strcasecmp(a->value, b->value);
+}
+
+static int same_trailer(struct trailer_item *a, struct trailer_item *b, int alnum_len)
+{
+	return same_token(a, b, alnum_len) && same_value(a, b);
+}
+
+/* Get the length of buf from its beginning until its last alphanumeric character */
+static size_t alnum_len(const char *buf, size_t len)
+{
+	while (len > 0 && !isalnum(buf[len - 1]))
+		len--;
+	return len;
+}
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:01 UTC · re: Christian Couder · lore

[PATCH v10 02/12] trailer: process trailers from stdin and arguments

Implement the logic to process trailers from stdin and arguments.

At the beginning trailers from stdin are in their own in_tok doubly linked list, and trailers from arguments are in their own arg_tok doubly linked list.

The lists are traversed and when an arg_tok should be "applied", it is removed from its list and inserted into the in_tok list.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 198 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 198 insertions(+)
Show changes to trailer.c +198 −0
diff --git a/trailer.c b/trailer.c
index db93a63..52108c2 100644
--- a/trailer.c
+++ b/trailer.c
@@ -47,3 +47,201 @@ static size_t alnum_len(const char *buf, size_t len)
 		len--;
 	return len;
 }
+
+static void free_trailer_item(struct trailer_item *item)
+{
+	free(item->conf.name);
+	free(item->conf.key);
+	free(item->conf.command);
+	free((char *)item->token);
+	free((char *)item->value);
+	free(item);
+}
+
+static void add_arg_to_input_list(struct trailer_item *in_tok,
+				  struct trailer_item *arg_tok)
+{
+	if (arg_tok->conf.where == WHERE_AFTER) {
+		arg_tok->next = in_tok->next;
+		in_tok->next = arg_tok;
+		arg_tok->previous = in_tok;
+		if (arg_tok->next)
+			arg_tok->next->previous = arg_tok;
+	} else {
+		arg_tok->previous = in_tok->previous;
+		in_tok->previous = arg_tok;
+		arg_tok->next = in_tok;
+		if (arg_tok->previous)
+			arg_tok->previous->next = arg_tok;
+	}
+}
+
+static int check_if_different(struct trailer_item *in_tok,
+			      struct trailer_item *arg_tok,
+			      int alnum_len, int check_all)
+{
+	enum action_where where = arg_tok->conf.where;
+	do {
+		if (!in_tok)
+			return 1;
+		if (same_trailer(in_tok, arg_tok, alnum_len))
+			return 0;
+		/*
+		 * if we want to add a trailer after another one,
+		 * we have to check those before this one
+		 */
+		in_tok = (where == WHERE_AFTER) ? in_tok->previous : in_tok->next;
+	} while (check_all);
+	return 1;
+}
+
+static void apply_arg_if_exists(struct trailer_item *in_tok,
+				struct trailer_item *arg_tok,
+				int alnum_len)
+{
+	switch (arg_tok->conf.if_exists) {
+	case EXISTS_DO_NOTHING:
+		free_trailer_item(arg_tok);
+		break;
+	case EXISTS_OVERWRITE:
+		free((char *)in_tok->value);
+		in_tok->value = xstrdup(arg_tok->value);
+		free_trailer_item(arg_tok);
+		break;
+	case EXISTS_ADD:
+		add_arg_to_input_list(in_tok, arg_tok);
+		break;
+	case EXISTS_ADD_IF_DIFFERENT:
+		if (check_if_different(in_tok, arg_tok, alnum_len, 1))
+			add_arg_to_input_list(in_tok, arg_tok);
+		else
+			free_trailer_item(arg_tok);
+		break;
+	case EXISTS_ADD_IF_DIFFERENT_NEIGHBOR:
+		if (check_if_different(in_tok, arg_tok, alnum_len, 0))
+			add_arg_to_input_list(in_tok, arg_tok);
+		else
+			free_trailer_item(arg_tok);
+		break;
+	}
+}
+
+static void remove_from_list(struct trailer_item *item,
+			     struct trailer_item **first)
+{
+	if (item->next)
+		item->next->previous = item->previous;
+	if (item->previous)
+		item->previous->next = item->next;
+	else
+		*first = item->next;
+}
+
+static struct trailer_item *remove_first(struct trailer_item **first)
+{
+	struct trailer_item *item = *first;
+	*first = item->next;
+	if (item->next) {
+		item->next->previous = NULL;
+		item->next = NULL;
+	}
+	return item;
+}
+
+static void process_input_token(struct trailer_item *in_tok,
+				struct trailer_item **arg_tok_first,
+				enum action_where where)
+{
+	struct trailer_item *arg_tok;
+	struct trailer_item *next_arg;
+
+	int after = where == WHERE_AFTER;
+	int tok_alnum_len = alnum_len(in_tok->token, strlen(in_tok->token));
+
+	for (arg_tok = *arg_tok_first; arg_tok; arg_tok = next_arg) {
+		next_arg = arg_tok->next;
+		if (!same_token(in_tok, arg_tok, tok_alnum_len))
+			continue;
+		if (arg_tok->conf.where != where)
+			continue;
+		remove_from_list(arg_tok, arg_tok_first);
+		apply_arg_if_exists(in_tok, arg_tok, tok_alnum_len);
+		/*
+		 * If arg has been added to input,
+		 * then we need to process it too now.
+		 */
+		if ((after ? in_tok->next : in_tok->previous) == arg_tok)
+			in_tok = arg_tok;
+	}
+}
+
+static void update_last(struct trailer_item **last)
+{
+	if (*last)
+		while ((*last)->next != NULL)
+			*last = (*last)->next;
+}
+
+static void update_first(struct trailer_item **first)
+{
+	if (*first)
+		while ((*first)->previous != NULL)
+			*first = (*first)->previous;
+}
+
+static void apply_arg_if_missing(struct trailer_item **in_tok_first,
+				 struct trailer_item **in_tok_last,
+				 struct trailer_item *arg_tok)
+{
+	struct trailer_item **in_tok;
+	enum action_where where;
+
+	switch (arg_tok->conf.if_missing) {
+	case MISSING_DO_NOTHING:
+		free_trailer_item(arg_tok);
+		break;
+	case MISSING_ADD:
+		where = arg_tok->conf.where;
+		in_tok = (where == WHERE_AFTER) ? in_tok_last : in_tok_first;
+		if (*in_tok) {
+			add_arg_to_input_list(*in_tok, arg_tok);
+			*in_tok = arg_tok;
+		} else {
+			*in_tok_first = arg_tok;
+			*in_tok_last = arg_tok;
+		}
+		break;
+	}
+}
+
+static void process_trailers_lists(struct trailer_item **in_tok_first,
+				   struct trailer_item **in_tok_last,
+				   struct trailer_item **arg_tok_first)
+{
+	struct trailer_item *in_tok;
+	struct trailer_item *arg_tok;
+
+	if (!*arg_tok_first)
+		return;
+
+	/* Process input from end to start */
+	for (in_tok = *in_tok_last; in_tok; in_tok = in_tok->previous)
+		process_input_token(in_tok, arg_tok_first, WHERE_AFTER);
+
+	update_last(in_tok_last);
+
+	if (!*arg_tok_first)
+		return;
+
+	/* Process input from start to end */
+	for (in_tok = *in_tok_first; in_tok; in_tok = in_tok->next)
+		process_input_token(in_tok, arg_tok_first, WHERE_BEFORE);
+
+	update_first(in_tok_first);
+
+	/* Process args left */
+	while (*arg_tok_first) {
+		arg_tok = remove_first(arg_tok_first);
+		apply_arg_if_missing(in_tok_first, in_tok_last, arg_tok);
+	}
+}
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:01 UTC · re: Christian Couder · lore

Read the configuration to get trailer information, and then process it and storing it in a doubly linked list.

The config information is stored in the list whose first item is pointed to by:

static struct trailer_item *first_conf_item;
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 146 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 146 insertions(+)
Show changes to trailer.c +146 −0
diff --git a/trailer.c b/trailer.c
index 52108c2..c7c0f54 100644
--- a/trailer.c
+++ b/trailer.c
@@ -25,6 +25,8 @@ struct trailer_item {
 	struct conf_info conf;
 };
 
+static struct trailer_item *first_conf_item;
+
 static int same_token(struct trailer_item *a, struct trailer_item *b, int alnum_len)
 {
 	return !strncasecmp(a->token, b->token, alnum_len);
@@ -245,3 +247,147 @@ static void process_trailers_lists(struct trailer_item **in_tok_first,
 		apply_arg_if_missing(in_tok_first, in_tok_last, arg_tok);
 	}
 }
+
+static int set_where(struct conf_info *item, const char *value)
+{
+	if (!strcmp("after", value))
+		item->where = WHERE_AFTER;
+	else if (!strcmp("before", value))
+		item->where = WHERE_BEFORE;
+	else
+		return -1;
+	return 0;
+}
+
+static int set_if_exists(struct conf_info *item, const char *value)
+{
+	if (!strcmp("addIfDifferent", value))
+		item->if_exists = EXISTS_ADD_IF_DIFFERENT;
+	else if (!strcmp("addIfDifferentNeighbor", value))
+		item->if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
+	else if (!strcmp("add", value))
+		item->if_exists = EXISTS_ADD;
+	else if (!strcmp("overwrite", value))
+		item->if_exists = EXISTS_OVERWRITE;
+	else if (!strcmp("doNothing", value))
+		item->if_exists = EXISTS_DO_NOTHING;
+	else
+		return -1;
+	return 0;
+}
+
+static int set_if_missing(struct conf_info *item, const char *value)
+{
+	if (!strcmp("doNothing", value))
+		item->if_missing = MISSING_DO_NOTHING;
+	else if (!strcmp("add", value))
+		item->if_missing = MISSING_ADD;
+	else
+		return -1;
+	return 0;
+}
+
+static struct trailer_item *get_conf_item(const char *name)
+{
+	struct trailer_item *item;
+	struct trailer_item *previous;
+
+	/* Look up item with same name */
+	for (previous = NULL, item = first_conf_item;
+	     item;
+	     previous = item, item = item->next) {
+		if (!strcasecmp(item->conf.name, name))
+			return item;
+	}
+
+	/* Item does not already exists, create it */
+	item = xcalloc(sizeof(struct trailer_item), 1);
+	item->conf.name = xstrdup(name);
+
+	if (!previous)
+		first_conf_item = item;
+	else {
+		previous->next = item;
+		item->previous = previous;
+	}
+
+	return item;
+}
+
+enum trailer_info_type { TRAILER_KEY, TRAILER_COMMAND, TRAILER_WHERE,
+			 TRAILER_IF_EXISTS, TRAILER_IF_MISSING };
+
+static struct {
+	const char *name;
+	enum trailer_info_type type;
+} trailer_config_items[] = {
+	{ "key", TRAILER_KEY },
+	{ "command", TRAILER_COMMAND },
+	{ "where", TRAILER_WHERE },
+	{ "ifexists", TRAILER_IF_EXISTS },
+	{ "ifmissing", TRAILER_IF_MISSING }
+};
+
+static int git_trailer_config(const char *conf_key, const char *value, void *cb)
+{
+	const char *trailer_item, *variable_name;
+	struct trailer_item *item;
+	struct conf_info *conf;
+	char *name = NULL;
+	enum trailer_info_type type;
+	int i;
+
+	trailer_item = skip_prefix(conf_key, "trailer.");
+	if (!trailer_item)
+		return 0;
+
+	variable_name = strrchr(trailer_item, '.');
+	if (!variable_name) {
+		warning(_("two level trailer config variable %s"), conf_key);
+		return 0;
+	}
+
+	variable_name++;
+	for (i = 0; i < ARRAY_SIZE(trailer_config_items); i++) {
+		if (strcmp(trailer_config_items[i].name, variable_name))
+			continue;
+		name = xstrndup(trailer_item,  variable_name - trailer_item - 1);
+		type = trailer_config_items[i].type;
+		break;
+	}
+
+	if (!name)
+		return 0;
+
+	item = get_conf_item(name);
+	conf = &item->conf;
+	free(name);
+
+	switch (type) {
+	case TRAILER_KEY:
+		if (conf->key)
+			warning(_("more than one %s"), conf_key);
+		conf->key = xstrdup(value);
+		break;
+	case TRAILER_COMMAND:
+		if (conf->command)
+			warning(_("more than one %s"), conf_key);
+		conf->command = xstrdup(value);
+		break;
+	case TRAILER_WHERE:
+		if (set_where(conf, value))
+			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
+		break;
+	case TRAILER_IF_EXISTS:
+		if (set_if_exists(conf, value))
+			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
+		break;
+	case TRAILER_IF_MISSING:
+		if (set_if_missing(conf, value))
+			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
+		break;
+	default:
+		die("internal bug in trailer.c");
+	}
+	return 0;
+}
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:01 UTC · re: Christian Couder · lore

[PATCH v10 04/12] trailer: process command line trailer arguments

Parse the trailer command line arguments and put the result into an arg_tok doubly linked list.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 117 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 117 insertions(+)
Show changes to trailer.c +117 −0
diff --git a/trailer.c b/trailer.c
index c7c0f54..89ebff1 100644
--- a/trailer.c
+++ b/trailer.c
@@ -391,3 +391,120 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 	}
 	return 0;
 }
+
+static int parse_trailer(struct strbuf *tok, struct strbuf *val, const char *trailer)
+{
+	size_t len = strcspn(trailer, "=:");
+	if (len == 0)
+		return error(_("empty trailer token in trailer '%s'"), trailer);
+	if (len < strlen(trailer)) {
+		strbuf_add(tok, trailer, len);
+		strbuf_trim(tok);
+		strbuf_addstr(val, trailer + len + 1);
+		strbuf_trim(val);
+	} else {
+		strbuf_addstr(tok, trailer);
+		strbuf_trim(tok);
+	}
+	return 0;
+}
+
+
+static void duplicate_conf(struct conf_info *dst, struct conf_info *src)
+{
+	*dst = *src;
+	if (src->name)
+		dst->name = xstrdup(src->name);
+	if (src->key)
+		dst->key = xstrdup(src->key);
+	if (src->command)
+		dst->command = xstrdup(src->command);
+}
+
+static const char *token_from_item(struct trailer_item *item)
+{
+	if (item->conf.key)
+		return item->conf.key;
+
+	return item->conf.name;
+}
+
+static struct trailer_item *new_trailer_item(struct trailer_item *conf_item,
+					     char *tok, char *val)
+{
+	struct trailer_item *new = xcalloc(sizeof(*new), 1);
+	new->value = val;
+
+	if (conf_item) {
+		duplicate_conf(&new->conf, &conf_item->conf);
+		new->token = xstrdup(token_from_item(conf_item));
+		free(tok);
+	} else
+		new->token = tok;
+
+	return new;
+}
+
+static int token_matches_item(const char *tok, struct trailer_item *item, int alnum_len)
+{
+	if (!strncasecmp(tok, item->conf.name, alnum_len))
+		return 1;
+	return item->conf.key ? !strncasecmp(tok, item->conf.key, alnum_len) : 0;
+}
+
+static struct trailer_item *create_trailer_item(const char *string)
+{
+	struct strbuf tok = STRBUF_INIT;
+	struct strbuf val = STRBUF_INIT;
+	struct trailer_item *item;
+	int tok_alnum_len;
+
+	if (parse_trailer(&tok, &val, string))
+		return NULL;
+
+	tok_alnum_len = alnum_len(tok.buf, tok.len);
+
+	/* Lookup if the token matches something in the config */
+	for (item = first_conf_item; item; item = item->next) {
+		if (token_matches_item(tok.buf, item, tok_alnum_len)) {
+			strbuf_release(&tok);
+			return new_trailer_item(item,
+						NULL,
+						strbuf_detach(&val, NULL));
+		}
+	}
+
+	return new_trailer_item(NULL,
+				strbuf_detach(&tok, NULL),
+				strbuf_detach(&val, NULL));
+}
+
+static void add_trailer_item(struct trailer_item **first,
+			     struct trailer_item **last,
+			     struct trailer_item *new)
+{
+	if (!new)
+		return;
+	if (!*last) {
+		*first = new;
+		*last = new;
+	} else {
+		(*last)->next = new;
+		new->previous = *last;
+		*last = new;
+	}
+}
+
+static struct trailer_item *process_command_line_args(int argc, const char **argv)
+{
+	int i;
+	struct trailer_item *arg_tok_first = NULL;
+	struct trailer_item *arg_tok_last = NULL;
+
+	for (i = 0; i < argc; i++) {
+		struct trailer_item *new = create_trailer_item(argv[i]);
+		add_trailer_item(&arg_tok_first, &arg_tok_last, new);
+	}
+
+	return arg_tok_first;
+}
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:01 UTC · re: Christian Couder · lore

[PATCH v10 05/12] trailer: parse trailers from stdin

Read trailers from stdin, parse them and put the result into a doubly linked list.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)
Show changes to trailer.c +76 −0
diff --git a/trailer.c b/trailer.c
index 89ebff1..6d2da32 100644
--- a/trailer.c
+++ b/trailer.c
@@ -50,6 +50,14 @@ static size_t alnum_len(const char *buf, size_t len)
 	return len;
 }
 
+static inline int contains_only_spaces(const char *str)
+{
+	const char *s = str;
+	while (*s && isspace(*s))
+		s++;
+	return !*s;
+}
+
 static void free_trailer_item(struct trailer_item *item)
 {
 	free(item->conf.name);
@@ -508,3 +516,71 @@ static struct trailer_item *process_command_line_args(int argc, const char **arg
 
 	return arg_tok_first;
 }
+
+static struct strbuf **read_stdin(void)
+{
+	struct strbuf **lines;
+	struct strbuf sb = STRBUF_INIT;
+
+	if (strbuf_read(&sb, fileno(stdin), 0) < 0)
+		die_errno(_("could not read from stdin"));
+
+	lines = strbuf_split(&sb, '\n');
+
+	strbuf_release(&sb);
+
+	return lines;
+}
+
+/*
+ * Return the the (0 based) index of the first trailer line
+ * or the line count if there are no trailers.
+ */
+static int find_trailer_start(struct strbuf **lines)
+{
+	int start, empty = 1, count = 0;
+
+	/* Get the line count */
+	while (lines[count])
+		count++;
+
+	/*
+	 * Get the start of the trailers by looking starting from the end
+	 * for a line with only spaces before lines with one ':'.
+	 */
+	for (start = count - 1; start >= 0; start--) {
+		if (contains_only_spaces(lines[start]->buf)) {
+			if (empty)
+				continue;
+			return start + 1;
+		}
+		if (strchr(lines[start]->buf, ':')) {
+			if (empty)
+				empty = 0;
+			continue;
+		}
+		return count;
+	}
+
+	return empty ? count : start + 1;
+}
+
+static void process_stdin(struct trailer_item **in_tok_first,
+			  struct trailer_item **in_tok_last)
+{
+	struct strbuf **lines = read_stdin();
+	int start = find_trailer_start(lines);
+	int i;
+
+	/* Print non trailer lines as is */
+	for (i = 0; lines[i] && i < start; i++)
+		printf("%s", lines[i]->buf);
+
+	/* Parse trailer lines */
+	for (i = start; lines[i]; i++) {
+		struct trailer_item *new = create_trailer_item(lines[i]->buf);
+		add_trailer_item(in_tok_first, in_tok_last, new);
+	}
+
+	strbuf_list_free(lines);
+}
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:01 UTC · re: Christian Couder · lore

[PATCH v10 06/12] trailer: put all the processing together and print

This patch adds the process_trailers() function that calls all the previously added processing functions and then prints the results on the standard output.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 trailer.h |  6 ++++++
 2 files changed, 55 insertions(+)
 create mode 100644 trailer.h
Show changes to 2 files +55 −0

trailer.c, trailer.h

diff --git a/trailer.c b/trailer.c
index 6d2da32..16465e5 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "trailer.h"
 /*
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
@@ -68,6 +69,26 @@ static void free_trailer_item(struct trailer_item *item)
 	free(item);
 }
 
+static void print_tok_val(const char *tok, const char *val)
+{
+	char c = tok[strlen(tok) - 1];
+	if (isalnum(c))
+		printf("%s: %s\n", tok, val);
+	else if (isspace(c) || c == '#')
+		printf("%s%s\n", tok, val);
+	else
+		printf("%s %s\n", tok, val);
+}
+
+static void print_all(struct trailer_item *first, int trim_empty)
+{
+	struct trailer_item *item;
+	for (item = first; item; item = item->next) {
+		if (!trim_empty || strlen(item->value) > 0)
+			print_tok_val(item->token, item->value);
+	}
+}
+
 static void add_arg_to_input_list(struct trailer_item *in_tok,
 				  struct trailer_item *arg_tok)
 {
@@ -584,3 +605,31 @@ static void process_stdin(struct trailer_item **in_tok_first,
 
 	strbuf_list_free(lines);
 }
+
+static void free_all(struct trailer_item **first)
+{
+	while (*first) {
+		struct trailer_item *item = remove_first(first);
+		free_trailer_item(item);
+	}
+}
+
+void process_trailers(int trim_empty, int argc, const char **argv)
+{
+	struct trailer_item *in_tok_first = NULL;
+	struct trailer_item *in_tok_last = NULL;
+	struct trailer_item *arg_tok_first;
+
+	git_config(git_trailer_config, NULL);
+
+	/* Print the non trailer part of stdin */
+	process_stdin(&in_tok_first, &in_tok_last);
+
+	arg_tok_first = process_command_line_args(argc, argv);
+
+	process_trailers_lists(&in_tok_first, &in_tok_last, &arg_tok_first);
+
+	print_all(in_tok_first, trim_empty);
+
+	free_all(&in_tok_first);
+}
diff --git a/trailer.h b/trailer.h
new file mode 100644
index 0000000..9323b1e
--- /dev/null
+++ b/trailer.h
@@ -0,0 +1,6 @@
+#ifndef TRAILER_H
+#define TRAILER_H
+
+void process_trailers(int trim_empty, int argc, const char **argv);
+
+#endif /* TRAILER_H */
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:01 UTC · re: Christian Couder · lore

[PATCH v10 07/12] trailer: add interpret-trailers command

This patch adds the "git interpret-trailers" command. This command uses the previously added process_trailers() function in trailer.c.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 .gitignore                   |  1 +
 Makefile                     |  1 +
 builtin.h                    |  1 +
 builtin/interpret-trailers.c | 33 +++++++++++++++++++++++++++++++++
 git.c                        |  1 +
 5 files changed, 37 insertions(+)
 create mode 100644 builtin/interpret-trailers.c
Show changes to 5 files +37 −0

.gitignore, Makefile, builtin.h, builtin/interpret-trailers.c, git.c

diff --git a/.gitignore b/.gitignore
index dc600f9..c2a0b19 100644
--- a/.gitignore
+++ b/.gitignore
@@ -74,6 +74,7 @@
 /git-index-pack
 /git-init
 /git-init-db
+/git-interpret-trailers
 /git-instaweb
 /git-log
 /git-ls-files
diff --git a/Makefile b/Makefile
index 179be0a..499ca30 100644
--- a/Makefile
+++ b/Makefile
@@ -944,6 +944,7 @@ BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
+BUILTIN_OBJS += builtin/interpret-trailers.o
 BUILTIN_OBJS += builtin/log.o
 BUILTIN_OBJS += builtin/ls-files.o
 BUILTIN_OBJS += builtin/ls-remote.o
diff --git a/builtin.h b/builtin.h
index c47c110..8ca0065 100644
--- a/builtin.h
+++ b/builtin.h
@@ -73,6 +73,7 @@ extern int cmd_hash_object(int argc, const char **argv, const char *prefix);
 extern int cmd_help(int argc, const char **argv, const char *prefix);
 extern int cmd_index_pack(int argc, const char **argv, const char *prefix);
 extern int cmd_init_db(int argc, const char **argv, const char *prefix);
+extern int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
 extern int cmd_log(int argc, const char **argv, const char *prefix);
 extern int cmd_log_reflog(int argc, const char **argv, const char *prefix);
 extern int cmd_ls_files(int argc, const char **argv, const char *prefix);
diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
new file mode 100644
index 0000000..0c8ca72
--- /dev/null
+++ b/builtin/interpret-trailers.c
@@ -0,0 +1,33 @@
+/*
+ * Builtin "git interpret-trailers"
+ *
+ * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
+ *
+ */
+
+#include "cache.h"
+#include "builtin.h"
+#include "parse-options.h"
+#include "trailer.h"
+
+static const char * const git_interpret_trailers_usage[] = {
+	N_("git interpret-trailers [--trim-empty] [(<token>[(=|:)<value>])...]"),
+	NULL
+};
+
+int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
+{
+	int trim_empty = 0;
+
+	struct option options[] = {
+		OPT_BOOL(0, "trim-empty", &trim_empty, N_("trim empty trailers")),
+		OPT_END()
+	};
+
+	argc = parse_options(argc, argv, prefix, options,
+			     git_interpret_trailers_usage, 0);
+
+	process_trailers(trim_empty, argc, argv);
+
+	return 0;
+}
diff --git a/git.c b/git.c
index 9efd1a3..d432f11 100644
--- a/git.c
+++ b/git.c
@@ -380,6 +380,7 @@ static struct cmd_struct commands[] = {
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
+	{ "interpret-trailers", cmd_interpret_trailers, RUN_SETUP },
 	{ "log", cmd_log, RUN_SETUP },
 	{ "ls-files", cmd_ls_files, RUN_SETUP },
 	{ "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY },
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:01 UTC · re: Christian Couder · lore

[PATCH v10 08/12] trailer: add tests for "git interpret-trailers"

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t7513-interpret-trailers.sh | 351 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 351 insertions(+)
 create mode 100755 t/t7513-interpret-trailers.sh
Show changes to t/t7513-interpret-trailers.sh +351 −0
diff --git a/t/t7513-interpret-trailers.sh b/t/t7513-interpret-trailers.sh
new file mode 100755
index 0000000..0e5d57f
--- /dev/null
+++ b/t/t7513-interpret-trailers.sh
@@ -0,0 +1,351 @@
+#!/bin/sh
+#
+# Copyright (c) 2013 Christian Couder
+#
+
+test_description='git interpret-trailers'
+
+. ./test-lib.sh
+
+# When we want one trailing space at the end of each line, let's use sed
+# to make sure that these spaces are not removed by any automatic tool.
+
+test_expect_success 'setup' '
+	cat >basic_message <<-\EOF &&
+		subject
+
+		body
+	EOF
+	cat >complex_message_body <<-\EOF &&
+		my subject
+
+		my body which is long
+		and contains some special
+		chars like : = ? !
+
+	EOF
+	sed -e "s/ Z\$/ /" >complex_message_trailers <<-\EOF
+		Fixes: Z
+		Acked-by: Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+'
+
+test_expect_success 'without config' '
+	sed -e "s/ Z\$/ /" >expected <<-\EOF &&
+		ack: Peff
+		Reviewed-by: Z
+		Acked-by: Johan
+	EOF
+	git interpret-trailers "ack = Peff" "Reviewed-by" "Acked-by: Johan" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success '--trim-empty without config' '
+	cat >expected <<-\EOF &&
+		ack: Peff
+		Acked-by: Johan
+	EOF
+	git interpret-trailers --trim-empty "ack = Peff" \
+		"Reviewed-by" "Acked-by: Johan" "sob:" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with config setup' '
+	git config trailer.ack.key "Acked-by: " &&
+	cat >expected <<-\EOF &&
+		Acked-by: Peff
+	EOF
+	git interpret-trailers --trim-empty "ack = Peff" >actual &&
+	test_cmp expected actual &&
+	git interpret-trailers --trim-empty "Acked-by = Peff" >actual &&
+	test_cmp expected actual &&
+	git interpret-trailers --trim-empty "Acked-by :Peff" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with config setup and = sign' '
+	git config trailer.ack.key "Acked-by= " &&
+	cat >expected <<-\EOF &&
+		Acked-by= Peff
+	EOF
+	git interpret-trailers --trim-empty "ack = Peff" >actual &&
+	test_cmp expected actual &&
+	git interpret-trailers --trim-empty "Acked-by= Peff" >actual &&
+	test_cmp expected actual &&
+	git interpret-trailers --trim-empty "Acked-by : Peff" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with config setup and # sign' '
+	git config trailer.bug.key "Bug #" &&
+	cat >expected <<-\EOF &&
+		Bug #42
+	EOF
+	git interpret-trailers --trim-empty "bug = 42" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with commit basic message' '
+	git interpret-trailers <basic_message >actual &&
+	test_cmp basic_message actual
+'
+
+test_expect_success 'with commit complex message' '
+	cat complex_message_body complex_message_trailers >complex_message &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with commit complex message and args' '
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Bug #42
+	EOF
+	git interpret-trailers "ack: Peff" "bug: 42" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with commit complex message, args and --trim-empty' '
+	cat complex_message_body >expected &&
+	cat >>expected <<-\EOF &&
+		Acked-by= Peff
+		Bug #42
+	EOF
+	git interpret-trailers --trim-empty "ack: Peff" "bug: 42" \
+		<complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "where = before"' '
+	git config trailer.bug.where "before" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "bug: 42" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "where = before" for a token in the middle of the message' '
+	git config trailer.review.key "Reviewed-by:" &&
+	git config trailer.review.where "before" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Johan
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "bug: 42" "review: Johan" \
+		<complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "where = before" and --trim-empty' '
+	cat complex_message_body >expected &&
+	cat >>expected <<-\EOF &&
+		Bug #46
+		Bug #42
+		Acked-by= Peff
+		Reviewed-by: Johan
+	EOF
+	git interpret-trailers --trim-empty "ack: Peff" "bug: 42" \
+		"review: Johan" "Bug: 46" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'the default is "ifExists = addIfDifferent"' '
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "review:" "bug: 42" \
+		"ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = addIfDifferent"' '
+	git config trailer.review.ifExists "addIfDifferent" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "review:" "bug: 42" \
+		"ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = addIfDifferentNeighbor"' '
+	git config trailer.ack.ifExists "addIfDifferentNeighbor" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "review:" "ack: Junio" "bug: 42" \
+		"ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = addIfDifferentNeighbor" and --trim-empty' '
+	git config trailer.ack.ifExists "addIfDifferentNeighbor" &&
+	cat complex_message_body >expected &&
+	cat >>expected <<-\EOF &&
+		Bug #42
+		Acked-by= Peff
+		Acked-by= Junio
+		Acked-by= Peff
+	EOF
+	git interpret-trailers --trim-empty "ack: Peff" "Acked-by= Peff" \
+		"review:" "ack: Junio" "bug: 42" "ack: Peff" \
+		<complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = add"' '
+	git config trailer.ack.ifExists "add" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Acked-by= Peff
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "Acked-by= Peff" "review:" \
+		"ack: Junio" "bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = overwrite"' '
+	git config trailer.fix.key "Fixes:" &&
+	git config trailer.fix.ifExists "overwrite" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: 22
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "ack: Junio" "fix=22" \
+		"bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = doNothing"' '
+	git config trailer.fix.ifExists "doNothing" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "ack: Junio" "fix=22" \
+		"bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'the default is "ifMissing = add"' '
+	git config trailer.cc.key "Cc: " &&
+	git config trailer.cc.where "before" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Cc: Linus
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "cc=Linus" "ack: Junio" \
+		"fix=22" "bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifMissing = add"' '
+	git config trailer.cc.ifMissing "add" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Cc: Linus
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "ack: Junio" "fix=22" \
+		"bug: 42" "cc=Linus" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifMissing = doNothing"' '
+	git config trailer.cc.ifMissing "doNothing" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "cc=Linus" "ack: Junio" \
+		"fix=22" "bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_done
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:02 UTC · re: Christian Couder · lore

[PATCH v10 09/12] trailer: execute command from 'trailer.<name>.command'

Let the user specify a command that will give on its standard output the value to use for the specified trailer.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
Show changes to trailer.c +64 −0
diff --git a/trailer.c b/trailer.c
index 16465e5..09db2c2 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "run-command.h"
 #include "trailer.h"
 /*
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
@@ -13,11 +14,14 @@ struct conf_info {
 	char *name;
 	char *key;
 	char *command;
+	unsigned command_uses_arg : 1;
 	enum action_where where;
 	enum action_if_exists if_exists;
 	enum action_if_missing if_missing;
 };
 
+#define TRAILER_ARG_STRING "$ARG"
+
 struct trailer_item {
 	struct trailer_item *previous;
 	struct trailer_item *next;
@@ -59,6 +63,13 @@ static inline int contains_only_spaces(const char *str)
 	return !*s;
 }
 
+static inline void strbuf_replace(struct strbuf *sb, const char *a, const char *b)
+{
+	const char *ptr = strstr(sb->buf, a);
+	if (ptr)
+		strbuf_splice(sb, ptr - sb->buf, strlen(a), b, strlen(b));
+}
+
 static void free_trailer_item(struct trailer_item *item)
 {
 	free(item->conf.name);
@@ -402,6 +413,7 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 		if (conf->command)
 			warning(_("more than one %s"), conf_key);
 		conf->command = xstrdup(value);
+		conf->command_uses_arg = !!strstr(conf->command, TRAILER_ARG_STRING);
 		break;
 	case TRAILER_WHERE:
 		if (set_where(conf, value))
@@ -438,6 +450,45 @@ static int parse_trailer(struct strbuf *tok, struct strbuf *val, const char *tra
 	return 0;
 }
 
+static int read_from_command(struct child_process *cp, struct strbuf *buf)
+{
+	if (run_command(cp))
+		return error("running trailer command '%s' failed", cp->argv[0]);
+	if (strbuf_read(buf, cp->out, 1024) < 1)
+		return error("reading from trailer command '%s' failed", cp->argv[0]);
+	strbuf_trim(buf);
+	return 0;
+}
+
+static const char *apply_command(const char *command, const char *arg)
+{
+	struct strbuf cmd = STRBUF_INIT;
+	struct strbuf buf = STRBUF_INIT;
+	struct child_process cp;
+	const char *argv[] = {NULL, NULL};
+	const char *result;
+
+	strbuf_addstr(&cmd, command);
+	if (arg)
+		strbuf_replace(&cmd, TRAILER_ARG_STRING, arg);
+
+	argv[0] = cmd.buf;
+	memset(&cp, 0, sizeof(cp));
+	cp.argv = argv;
+	cp.env = local_repo_env;
+	cp.no_stdin = 1;
+	cp.out = -1;
+	cp.use_shell = 1;
+
+	if (read_from_command(&cp, &buf)) {
+		strbuf_release(&buf);
+		result = xstrdup("");
+	} else
+		result = strbuf_detach(&buf, NULL);
+
+	strbuf_release(&cmd);
+	return result;
+}
 
 static void duplicate_conf(struct conf_info *dst, struct conf_info *src)
 {
@@ -468,6 +519,10 @@ static struct trailer_item *new_trailer_item(struct trailer_item *conf_item,
 		duplicate_conf(&new->conf, &conf_item->conf);
 		new->token = xstrdup(token_from_item(conf_item));
 		free(tok);
+		if (conf_item->conf.command_uses_arg || !val) {
+			new->value = apply_command(conf_item->conf.command, val);
+			free(val);
+		}
 	} else
 		new->token = tok;
 
@@ -529,12 +584,21 @@ static struct trailer_item *process_command_line_args(int argc, const char **arg
 	int i;
 	struct trailer_item *arg_tok_first = NULL;
 	struct trailer_item *arg_tok_last = NULL;
+	struct trailer_item *item;
 
 	for (i = 0; i < argc; i++) {
 		struct trailer_item *new = create_trailer_item(argv[i]);
 		add_trailer_item(&arg_tok_first, &arg_tok_last, new);
 	}
 
+	/* Add conf commands that don't use $ARG */
+	for (item = first_conf_item; item; item = item->next) {
+		if (item->conf.command && !item->conf.command_uses_arg) {
+			struct trailer_item *new = new_trailer_item(item, NULL, NULL);
+			add_trailer_item(&arg_tok_first, &arg_tok_last, new);
+		}
+	}
+
 	return arg_tok_first;
 }
 
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:02 UTC · re: Christian Couder · lore

[PATCH v10 10/12] trailer: add tests for commands in config file

And add a few other tests for some special cases.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t7513-interpret-trailers.sh | 116 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 116 insertions(+)
Show changes to t/t7513-interpret-trailers.sh +116 −0
diff --git a/t/t7513-interpret-trailers.sh b/t/t7513-interpret-trailers.sh
index 0e5d57f..262f7bf 100755
--- a/t/t7513-interpret-trailers.sh
+++ b/t/t7513-interpret-trailers.sh
@@ -348,4 +348,120 @@ test_expect_success 'using "ifMissing = doNothing"' '
 	test_cmp expected actual
 '
 
+test_expect_success 'with simple command' '
+	git config trailer.sign.key "Signed-off-by: " &&
+	git config trailer.sign.where "after" &&
+	git config trailer.sign.ifExists "addIfDifferentNeighbor" &&
+	git config trailer.sign.command "echo \"A U Thor <author@example.com>\"" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=22" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with command using commiter information' '
+	git config trailer.sign.ifExists "addIfDifferent" &&
+	git config trailer.sign.command "echo \"\$GIT_COMMITTER_NAME <\$GIT_COMMITTER_EMAIL>\"" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: C O Mitter <committer@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=22" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with command using author information' '
+	git config trailer.sign.key "Signed-off-by: " &&
+	git config trailer.sign.where "after" &&
+	git config trailer.sign.ifExists "addIfDifferentNeighbor" &&
+	git config trailer.sign.command "echo \"\$GIT_AUTHOR_NAME <\$GIT_AUTHOR_EMAIL>\"" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=22" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'setup a commit' '
+	echo "Content of the first commit." > a.txt &&
+	git add a.txt &&
+	git commit -m "Add file a.txt"
+'
+
+test_expect_success 'with command using $ARG' '
+	git config trailer.fix.ifExists "overwrite" &&
+	git config trailer.fix.command "git log -1 --oneline --format=\"%h (%s)\" --abbrev-commit --abbrev=14 \$ARG" &&
+	FIXED=$(git log -1 --oneline --format="%h (%s)" --abbrev-commit --abbrev=14 HEAD) &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-EOF &&
+		Fixes: $FIXED
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=HEAD" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with failing command using $ARG' '
+	git config trailer.fix.ifExists "overwrite" &&
+	git config trailer.fix.command "false \$ARG" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=HEAD" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with empty tokens' '
+	cat >expected <<-EOF &&
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers ":" ":test" >actual <<-EOF &&
+	EOF
+	test_cmp expected actual
+'
+
+test_expect_success 'with command but no key' '
+	git config --unset trailer.sign.key &&
+	cat >expected <<-EOF &&
+		sign: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers >actual <<-EOF &&
+	EOF
+	test_cmp expected actual
+'
+
+test_expect_success 'with no command and no key' '
+	git config --unset trailer.review.key &&
+	cat >expected <<-EOF &&
+		review: Junio
+		sign: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:Junio" >actual <<-EOF &&
+	EOF
+	test_cmp expected actual
+'
+
 test_done
-- 
1.9.0.163.g8ca203c
Christian Couder· Apr 6, 2014, 17:02 UTC · re: Christian Couder · lore

[PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-interpret-trailers.txt | 123 +++++++++++++++++++++++++++++++
 1 file changed, 123 insertions(+)
 create mode 100644 Documentation/git-interpret-trailers.txt
Show changes to Documentation/git-interpret-trailers.txt +123 −0
diff --git a/Documentation/git-interpret-trailers.txt b/Documentation/git-interpret-trailers.txt
new file mode 100644
index 0000000..75ae386
--- /dev/null
+++ b/Documentation/git-interpret-trailers.txt
@@ -0,0 +1,123 @@
+git-interpret-trailers(1)
+=========================
+
+NAME
+----
+git-interpret-trailers - help add stuctured information into commit messages
+
+SYNOPSIS
+--------
+[verse]
+'git interpret-trailers' [--trim-empty] [(<token>[(=|:)<value>])...]
+
+DESCRIPTION
+-----------
+Help add RFC 822-like headers, called 'trailers', at the end of the
+otherwise free-form part of a commit message.
+
+This command is a filter. It reads the standard input for a commit
+message and applies the `token` arguments, if any, to this
+message. The resulting message is emited on the standard output.
+
+Some configuration variables control the way the `token` arguments are
+applied to the message and the way any existing trailer in the message
+is changed. They also make it possible to automatically add some
+trailers.
+
+By default, a 'token=value' or 'token:value' argument will be added
+only if no trailer with the same (token, value) pair is already in the
+message. The 'token' and 'value' parts will be trimmed to remove
+starting and trailing whitespace, and the resulting trimmed 'token'
+and 'value' will appear in the message like this:
+
+------------------------------------------------
+token: value
+------------------------------------------------
+
+By default, if there are already trailers with the same 'token', the
+new trailer will appear just after the last trailer with the same
+'token'. Otherwise it will appear at the end of the message.
+
+Note that 'trailers' do not follow and are not intended to follow many
+rules that are in RFC 822. For example they do not follow the line
+breaking rules, the encoding rules and probably many other rules.
+
+OPTIONS
+-------
+--trim-empty::
+	If the 'value' part of any trailer contains only whitespace,
+	the whole trailer will be removed from the resulting message.
+
+CONFIGURATION VARIABLES
+-----------------------
+
+trailer.<token>.key::
+	This 'key' will be used instead of 'token' in the
+	trailer. After some alphanumeric characters, it can contain
+	some non alphanumeric characters like ':', '=' or '#' that will
+	be used instead of ':' to separate the token from the value in
+	the trailer, though the default ':' is more standard.
+
+trailer.<token>.where::
+	This can be either `after`, which is the default, or
+	`before`. If it is `before`, then a trailer with the specified
+	token, will appear before, instead of after, other trailers
+	with the same token, or otherwise at the beginning, instead of
+	at the end, of all the trailers.
+
+trailer.<token>.ifexist::
+	This option makes it possible to choose what action will be
+	performed when there is already at least one trailer with the
+	same token in the message.
++
+The valid values for this option are: `addIfDifferent` (this is the
+default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.
++
+With `addIfDifferent`, a new trailer will be added only if no trailer
+with the same (token, value) pair is already in the message.
++
+With `addIfDifferentNeighbor`, a new trailer will be added only if no
+trailer with the same (token, value) pair is above or below the line
+where the new trailer will be added.
++
+With `add`, a new trailer will be added, even if some trailers with
+the same (token, value) pair are already in the message.
++
+With `overwrite`, the new trailer will overwrite an existing trailer
+with the same token.
++
+With `doNothing`, nothing will be done, that is no new trailer will be
+added if there is already one with the same token in the message.
+
+trailer.<token>.ifmissing::
+	This option makes it possible to choose what action will be
+	performed when there is not yet any trailer with the same
+	token in the message.
++
+The valid values for this option are: `add` (this is the default) and
+`doNothing`.
++
+With `add`, a new trailer will be added.
++
+With `doNothing`, nothing will be done.
+
+trailer.<token>.command::
+	This option can be used to specify a shell command that will
+	be used to automatically add or modify a trailer with the
+	specified 'token'.
++
+When this option is specified, it is like if a special 'token=value'
+argument is added at the end of the command line, where 'value' will
+be given by the standard output of the specified command.
++
+If the command contains the `$ARG` string, this string will be
+replaced with the 'value' part of an existing trailer with the same
+token, if any, before the command is launched.
+
+SEE ALSO
+--------
+linkgit:git-commit[1]
+
+GIT
+---
+Part of the linkgit:git[1] suite
-- 
1.9.0.163.g8ca203c
Michael Haggerty· Apr 8, 2014, 07:30 UTC · re: Christian Couder · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

Sorry for reappearing in this thread after such a long absence. I wanted to see what is coming up (I think this interpret-trailers command will be handy!) so I read this documentation patch carefully, and added some questions and suggestions below.

On 04/06/2014 07:02 PM, Christian Couder wrote:
Show 33 quoted lines
> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  Documentation/git-interpret-trailers.txt | 123 +++++++++++++++++++++++++++++++
>  1 file changed, 123 insertions(+)
>  create mode 100644 Documentation/git-interpret-trailers.txt
> 
> diff --git a/Documentation/git-interpret-trailers.txt b/Documentation/git-interpret-trailers.txt
> new file mode 100644
> index 0000000..75ae386
> --- /dev/null
> +++ b/Documentation/git-interpret-trailers.txt
> @@ -0,0 +1,123 @@
> +git-interpret-trailers(1)
> +=========================
> +
> +NAME
> +----
> +git-interpret-trailers - help add stuctured information into commit messages
> +
> +SYNOPSIS
> +--------
> +[verse]
> +'git interpret-trailers' [--trim-empty] [(<token>[(=|:)<value>])...]
> +
> +DESCRIPTION
> +-----------
> +Help add RFC 822-like headers, called 'trailers', at the end of the
> +otherwise free-form part of a commit message.
> +
> +This command is a filter. It reads the standard input for a commit
> +message and applies the `token` arguments, if any, to this
> +message. The resulting message is emited on the standard output.
s/emited/emitted/
Show 19 quoted lines
> +
> +Some configuration variables control the way the `token` arguments are
> +applied to the message and the way any existing trailer in the message
> +is changed. They also make it possible to automatically add some
> +trailers.
> +
> +By default, a 'token=value' or 'token:value' argument will be added
> +only if no trailer with the same (token, value) pair is already in the
> +message. The 'token' and 'value' parts will be trimmed to remove
> +starting and trailing whitespace, and the resulting trimmed 'token'
> +and 'value' will appear in the message like this:
> +
> +------------------------------------------------
> +token: value
> +------------------------------------------------
> +
> +By default, if there are already trailers with the same 'token', the
> +new trailer will appear just after the last trailer with the same
> +'token'. Otherwise it will appear at the end of the message.

How are existing trailers recognized in the input commit message? Do trailers have to be configured to be recognized? Or are all lines matching a specific pattern considered trailers? If so, it might be helpful to include a regexp here that describes the trailer "syntax".

What about blank lines? I see that you try to add a blank line before new trailers. But what about on input? Do the trailer lines have to be separated from the free-form comment by a blank line to be recognized? What if there are blank lines between trailer lines, or after them? Is it allowed to have non-trailer lines between or after trailer lines?

Show 10 quoted lines
> +
> +Note that 'trailers' do not follow and are not intended to follow many
> +rules that are in RFC 822. For example they do not follow the line
> +breaking rules, the encoding rules and probably many other rules.
> +
> +OPTIONS
> +-------
> +--trim-empty::
> +	If the 'value' part of any trailer contains only whitespace,
> +	the whole trailer will be removed from the resulting message.

Does this apply to existing trailers, new trailers, or both? If it applies to existing trailers, then it seems a bit dangerous, in the sense that the command might end up changing trailers that are unrelated to the one that the command is trying to add.

Show 7 quoted lines
> +
> +CONFIGURATION VARIABLES
> +-----------------------
> +
> +trailer.<token>.key::
> +	This 'key' will be used instead of 'token' in the
> +	trailer. After some alphanumeric characters, it can contain
Trailer keys can also contain '-', right?
> +	some non alphanumeric characters like ':', '=' or '#' that will
> +	be used instead of ':' to separate the token from the value in
> +	the trailer, though the default ':' is more standard.

Above it looks like the default separator is not ':' but rather ': ' (with a space). Is the space always added regardless of the value of this configuration variable, or should the configuration value include the trailing space if it is desired? Is there any way to get a trailer that doesn't include a space, like

    foo=bar
?  (Changing this to "foo= bar" would look pretty ugly.)

If a commit message containing trailer lines with separators other than ':' is input to the program, will it recognize them as trailer lines? Do such trailer lines have to have the same separator as the one listed in this configuration setting to be recognized?

I suppose that there is some compelling reason to allow non-colon separators here. If not, it seems like it adds a lot of complexity and should maybe be omitted, or limited to only a few specific separators.

Show 7 quoted lines
> +
> +trailer.<token>.where::
> +	This can be either `after`, which is the default, or
> +	`before`. If it is `before`, then a trailer with the specified
> +	token, will appear before, instead of after, other trailers
> +	with the same token, or otherwise at the beginning, instead of
> +	at the end, of all the trailers.
Brainstorming: some other options that might make sense here someday:

`end`: add new trailer after all existing trailers (even those with different keys). This would allow trailers to be kept in chronological order.

`beginning`: add new trailer before the first existing trailer (allows reverse chronological order).

`sorted`: add new trailer among the existing trailers with the same key so as to keep their values in lexicographic order.

Show 8 quoted lines
> +
> +trailer.<token>.ifexist::
> +	This option makes it possible to choose what action will be
> +	performed when there is already at least one trailer with the
> +	same token in the message.
> ++
> +The valid values for this option are: `addIfDifferent` (this is the
> +default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.

Are these option values case sensitive? If so, it might be a little bit confusing because the same camel-case is often used in documentation for configuration *keys*, which are not case sensitive [1], and users might have gotten used to thinking of strings that look like this to be non-case-sensitive.

Show 13 quoted lines
> ++
> +With `addIfDifferent`, a new trailer will be added only if no trailer
> +with the same (token, value) pair is already in the message.
> ++
> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
> +trailer with the same (token, value) pair is above or below the line
> +where the new trailer will be added.
> ++
> +With `add`, a new trailer will be added, even if some trailers with
> +the same (token, value) pair are already in the message.
> ++
> +With `overwrite`, the new trailer will overwrite an existing trailer
> +with the same token.

What if there are multiple existing trailers with the same token? Are they all overwritten?

Show 24 quoted lines
> ++
> +With `doNothing`, nothing will be done, that is no new trailer will be
> +added if there is already one with the same token in the message.
> +
> +trailer.<token>.ifmissing::
> +	This option makes it possible to choose what action will be
> +	performed when there is not yet any trailer with the same
> +	token in the message.
> ++
> +The valid values for this option are: `add` (this is the default) and
> +`doNothing`.
> ++
> +With `add`, a new trailer will be added.
> ++
> +With `doNothing`, nothing will be done.
> +
> +trailer.<token>.command::
> +	This option can be used to specify a shell command that will
> +	be used to automatically add or modify a trailer with the
> +	specified 'token'.
> ++
> +When this option is specified, it is like if a special 'token=value'
> +argument is added at the end of the command line, where 'value' will
> +be given by the standard output of the specified command.
Maybe reword to
    When this option is specified, the behavior is as if a special
    'token=value' argument were added at the end of the command line,
    where 'value' is taken to be the standard output of the specified
    command.

And if it is the case, maybe add "with leading and trailing whitespace trimmed off" at the end of the sentence.

> ++
> +If the command contains the `$ARG` string, this string will be
> +replaced with the 'value' part of an existing trailer with the same
> +token, if any, before the command is launched.
What if the key appears multiple times in existing trailers?
Show 9 quoted lines
> +
> +SEE ALSO
> +--------
> +linkgit:git-commit[1]
> +
> +GIT
> +---
> +Part of the linkgit:git[1] suite
> 
Doesn't this command have to be added to command-list.txt?
Michael

[1] Anti-nitpick declaration: yes, I know that the middle part of configuration keys is case-sensitive.

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Christian Couder· Apr 8, 2014, 11:35 UTC · re: Michael Haggerty · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

On Tue, Apr 8, 2014 at 9:30 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
Show 6 quoted lines
>
>> +This command is a filter. It reads the standard input for a commit
>> +message and applies the `token` arguments, if any, to this
>> +message. The resulting message is emited on the standard output.
>
> s/emited/emitted/
Ok.
Show 23 quoted lines
>> +Some configuration variables control the way the `token` arguments are
>> +applied to the message and the way any existing trailer in the message
>> +is changed. They also make it possible to automatically add some
>> +trailers.
>> +
>> +By default, a 'token=value' or 'token:value' argument will be added
>> +only if no trailer with the same (token, value) pair is already in the
>> +message. The 'token' and 'value' parts will be trimmed to remove
>> +starting and trailing whitespace, and the resulting trimmed 'token'
>> +and 'value' will appear in the message like this:
>> +
>> +------------------------------------------------
>> +token: value
>> +------------------------------------------------
>> +
>> +By default, if there are already trailers with the same 'token', the
>> +new trailer will appear just after the last trailer with the same
>> +'token'. Otherwise it will appear at the end of the message.
>
> How are existing trailers recognized in the input commit message?  Do
> trailers have to be configured to be recognized?  Or are all lines
> matching a specific pattern considered trailers?  If so, it might be
> helpful to include a regexp here that describes the trailer "syntax".
The trailers are recognized in the input commit message using the
following rules:
 - only lines that contains a ':' are considered trailers,
 - the trailer lines must all be next to each other,
 - after them it's only possible to have some lines that contain only spaces,
 - before them there must be at least one line with only spaces
> What about blank lines?  I see that you try to add a blank line before
> new trailers.  But what about on input?

One line with only spaces has to be before the trailers. Some can be after the trailers.

> Do the trailer lines have to be
> separated from the free-form comment by a blank line to be recognized?
Yes.
> What if there are blank lines between trailer lines, or after them?

After them is ok. Between is not ok (only the trailers after the blank lines will be recognized).

> Is it allowed to have non-trailer lines between or after trailer lines?
No except lines with spaces after the trailers lines.
Show 11 quoted lines
>> +Note that 'trailers' do not follow and are not intended to follow many
>> +rules that are in RFC 822. For example they do not follow the line
>> +breaking rules, the encoding rules and probably many other rules.
>> +
>> +OPTIONS
>> +-------
>> +--trim-empty::
>> +     If the 'value' part of any trailer contains only whitespace,
>> +     the whole trailer will be removed from the resulting message.
>
> Does this apply to existing trailers, new trailers, or both?
Both.
> If it applies to existing trailers, then it seems a bit dangerous, in the
> sense that the command might end up changing trailers that are unrelated
> to the one that the command is trying to add.

The command is not just for adding trailers. But there could be an option to just trim trailers that are added.

Show 8 quoted lines
>> +CONFIGURATION VARIABLES
>> +-----------------------
>> +
>> +trailer.<token>.key::
>> +     This 'key' will be used instead of 'token' in the
>> +     trailer. After some alphanumeric characters, it can contain
>
> Trailer keys can also contain '-', right?

Yes. I should have written "after the last alphanumeric character". I will fix that.

Show 13 quoted lines
>> +     some non alphanumeric characters like ':', '=' or '#' that will
>> +     be used instead of ':' to separate the token from the value in
>> +     the trailer, though the default ':' is more standard.
>
> Above it looks like the default separator is not ':' but rather ': '
> (with a space).  Is the space always added regardless of the value of
> this configuration variable, or should the configuration value include
> the trailing space if it is desired?  Is there any way to get a trailer
> that doesn't include a space, like
>
>     foo=bar
>
> ?  (Changing this to "foo= bar" would look pretty ugly.)
I will have a look, but I think that:
- a space is always added after ':' or '=',
- a space is never added after '#',
- it doesn't matter if there is a space or not in the configured key.
> If a commit message containing trailer lines with separators other than
> ':' is input to the program, will it recognize them as trailer lines?
No, '=' and '#' are not supported in the input message, only in the output.
> Do such trailer lines have to have the same separator as the one listed
> in this configuration setting to be recognized?
No they need to have ':' as a separator.

The reason why only ':' is supported is because it is the cannonical trailer separator and it could create problems with many input messages if other separators where supported.

Maybe we could detect a special line like the following:
# TRAILERS START

in the input message and consider everyhting after that line as trailers. In this case it would be ok to accept other separators.

> I suppose that there is some compelling reason to allow non-colon
> separators here.  If not, it seems like it adds a lot of complexity and
> should maybe be omitted, or limited to only a few specific separators.

Yeah, but in the early threads concerning this subject, someone said that GitHub for example uses "bug #XXX". I will have a look again.

Show 18 quoted lines
>> +trailer.<token>.where::
>> +     This can be either `after`, which is the default, or
>> +     `before`. If it is `before`, then a trailer with the specified
>> +     token, will appear before, instead of after, other trailers
>> +     with the same token, or otherwise at the beginning, instead of
>> +     at the end, of all the trailers.
>
> Brainstorming: some other options that might make sense here someday:
>
> `end`: add new trailer after all existing trailers (even those with
> different keys).  This would allow trailers to be kept in chronological
> order.
>
> `beginning`: add new trailer before the first existing trailer (allows
> reverse chronological order).
>
> `sorted`: add new trailer among the existing trailers with the same key
> so as to keep their values in lexicographic order.

Yeah, I thought about these, but I don't think there is a need for them right now.

Show 13 quoted lines
>> +trailer.<token>.ifexist::
>> +     This option makes it possible to choose what action will be
>> +     performed when there is already at least one trailer with the
>> +     same token in the message.
>> ++
>> +The valid values for this option are: `addIfDifferent` (this is the
>> +default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.
>
> Are these option values case sensitive?  If so, it might be a little bit
> confusing because the same camel-case is often used in documentation for
> configuration *keys*, which are not case sensitive [1], and users might
> have gotten used to thinking of strings that look like this to be
> non-case-sensitive.

There were some discussions a few versions of this series ago with Peff, Junio and perhaps others about this. I thought that being case insensitive was better and Peff kind of agreed with that, but as Junio disagreed it is now case sensitive.

Show 15 quoted lines
>> +With `addIfDifferent`, a new trailer will be added only if no trailer
>> +with the same (token, value) pair is already in the message.
>> ++
>> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
>> +trailer with the same (token, value) pair is above or below the line
>> +where the new trailer will be added.
>> ++
>> +With `add`, a new trailer will be added, even if some trailers with
>> +the same (token, value) pair are already in the message.
>> ++
>> +With `overwrite`, the new trailer will overwrite an existing trailer
>> +with the same token.
>
> What if there are multiple existing trailers with the same token?  Are
> they all overwritten?

No, if where == after, only the last one is overwritten, and if where == before, only the first one is overwritten.

I could add an "overwriteAll" option. It could be interesting to use when a command using "$ARG" is configured, as this way the command would apply to all the trailers with the given token instead of just the last or first one.

Show 33 quoted lines
>> +With `doNothing`, nothing will be done, that is no new trailer will be
>> +added if there is already one with the same token in the message.
>> +
>> +trailer.<token>.ifmissing::
>> +     This option makes it possible to choose what action will be
>> +     performed when there is not yet any trailer with the same
>> +     token in the message.
>> ++
>> +The valid values for this option are: `add` (this is the default) and
>> +`doNothing`.
>> ++
>> +With `add`, a new trailer will be added.
>> ++
>> +With `doNothing`, nothing will be done.
>> +
>> +trailer.<token>.command::
>> +     This option can be used to specify a shell command that will
>> +     be used to automatically add or modify a trailer with the
>> +     specified 'token'.
>> ++
>> +When this option is specified, it is like if a special 'token=value'
>> +argument is added at the end of the command line, where 'value' will
>> +be given by the standard output of the specified command.
>
> Maybe reword to
>
>     When this option is specified, the behavior is as if a special
>     'token=value' argument were added at the end of the command line,
>     where 'value' is taken to be the standard output of the specified
>     command.
>
> And if it is the case, maybe add "with leading and trailing whitespace
> trimmed off" at the end of the sentence.
Ok.
Show 5 quoted lines
>> +If the command contains the `$ARG` string, this string will be
>> +replaced with the 'value' part of an existing trailer with the same
>> +token, if any, before the command is launched.
>
> What if the key appears multiple times in existing trailers?

It will be done only once for the last or first trailer with the key depending on "where".

Show 11 quoted lines
>> +
>> +SEE ALSO
>> +--------
>> +linkgit:git-commit[1]
>> +
>> +GIT
>> +---
>> +Part of the linkgit:git[1] suite
>>
>
> Doesn't this command have to be added to command-list.txt?
Maybe, I will have a look.

Thanks, Christian.

Michael Haggerty· Apr 8, 2014, 15:25 UTC · re: Christian Couder · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

On 04/08/2014 01:35 PM, Christian Couder wrote:
Show 12 quoted lines
> On Tue, Apr 8, 2014 at 9:30 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
>> How are existing trailers recognized in the input commit message?  Do
>> trailers have to be configured to be recognized?  Or are all lines
>> matching a specific pattern considered trailers?  If so, it might be
>> helpful to include a regexp here that describes the trailer "syntax".
> 
> The trailers are recognized in the input commit message using the
> following rules:
>  - only lines that contains a ':' are considered trailers,
>  - the trailer lines must all be next to each other,
>  - after them it's only possible to have some lines that contain only spaces,
>  - before them there must be at least one line with only spaces

Thanks for all the explanation. I think that most/all of this information should be included in the documentation.

Show 16 quoted lines
>>> +OPTIONS
>>> +-------
>>> +--trim-empty::
>>> +     If the 'value' part of any trailer contains only whitespace,
>>> +     the whole trailer will be removed from the resulting message.
>>
>> Does this apply to existing trailers, new trailers, or both?
> 
> Both.
> 
>> If it applies to existing trailers, then it seems a bit dangerous, in the
>> sense that the command might end up changing trailers that are unrelated
>> to the one that the command is trying to add.
> 
> The command is not just for adding trailers.
> But there could be an option to just trim trailers that are added.
Maybe that should be the *only* behavior of this option.
Maybe there should be a trailer.<token>.trimEmpty config option.
Show 53 quoted lines
>>> +CONFIGURATION VARIABLES
>>> +-----------------------
>>> +
>>> +trailer.<token>.key::
>>> +     This 'key' will be used instead of 'token' in the
>>> +     trailer. After some alphanumeric characters, it can contain
>>
>> Trailer keys can also contain '-', right?
> 
> Yes.
> I should have written "after the last alphanumeric character".
> I will fix that.
> 
>>> +     some non alphanumeric characters like ':', '=' or '#' that will
>>> +     be used instead of ':' to separate the token from the value in
>>> +     the trailer, though the default ':' is more standard.
>>
>> Above it looks like the default separator is not ':' but rather ': '
>> (with a space).  Is the space always added regardless of the value of
>> this configuration variable, or should the configuration value include
>> the trailing space if it is desired?  Is there any way to get a trailer
>> that doesn't include a space, like
>>
>>     foo=bar
>>
>> ?  (Changing this to "foo= bar" would look pretty ugly.)
> 
> I will have a look, but I think that:
> 
> - a space is always added after ':' or '=',
> - a space is never added after '#',
> - it doesn't matter if there is a space or not in the configured key.
> 
>> If a commit message containing trailer lines with separators other than
>> ':' is input to the program, will it recognize them as trailer lines?
> 
> No, '=' and '#' are not supported in the input message, only in the output.
> 
>> Do such trailer lines have to have the same separator as the one listed
>> in this configuration setting to be recognized?
> 
> No they need to have ':' as a separator.
> 
> The reason why only ':' is supported is because it is the cannonical
> trailer separator and it could create problems with many input
> messages if other separators where supported.
> 
> Maybe we could detect a special line like the following:
> 
> # TRAILERS START
> 
> in the input message and consider everyhting after that line as trailers.
> In this case it would be ok to accept other separators.

It would be ugly to have to use such a line. I think it would be preferable to be more restrictive about trailer separators than to require something like this.

>From what you've said above, it sounds like your code might get confused
with the following input commit message:
    This is the human-readable comment
    Foo: bar
    Fixes #123
    Plugh: xyzzy

It seems to me that none of these lines would be accepted as trailers, because they include a non-trailer "Fixes" line (non-trailer in the sense that it doesn't use a colon separator).

Show 7 quoted lines
>> I suppose that there is some compelling reason to allow non-colon
>> separators here.  If not, it seems like it adds a lot of complexity and
>> should maybe be omitted, or limited to only a few specific separators.
> 
> Yeah, but in the early threads concerning this subject, someone said
> that GitHub for example uses "bug #XXX".
> I will have a look again.

Yes, that's true: GitHub recognizes strings like "fixes #33" but not if there is an intervening colon like in "fixes: #33". OTOH GitHub recognizes such strings wherever they appear in the commit message (they don't have to be in "trailer" lines). So I'm not sure that the added complication is worth it if GitHub is the only use case. (And maybe we could convince GitHub to recognize "Fixes: #33" if such syntax becomes the de-facto Git standard for trailers.)

Show 21 quoted lines
>>> +trailer.<token>.where::
>>> +     This can be either `after`, which is the default, or
>>> +     `before`. If it is `before`, then a trailer with the specified
>>> +     token, will appear before, instead of after, other trailers
>>> +     with the same token, or otherwise at the beginning, instead of
>>> +     at the end, of all the trailers.
>>
>> Brainstorming: some other options that might make sense here someday:
>>
>> `end`: add new trailer after all existing trailers (even those with
>> different keys).  This would allow trailers to be kept in chronological
>> order.
>>
>> `beginning`: add new trailer before the first existing trailer (allows
>> reverse chronological order).
>>
>> `sorted`: add new trailer among the existing trailers with the same key
>> so as to keep their values in lexicographic order.
> 
> Yeah, I thought about these, but I don't think there is a need for
> them right now.

Yes, I didn't mean to imply that any of these options have to be in the first version.

Show 18 quoted lines
>>> +trailer.<token>.ifexist::
>>> +     This option makes it possible to choose what action will be
>>> +     performed when there is already at least one trailer with the
>>> +     same token in the message.
>>> ++
>>> +The valid values for this option are: `addIfDifferent` (this is the
>>> +default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.
>>
>> Are these option values case sensitive?  If so, it might be a little bit
>> confusing because the same camel-case is often used in documentation for
>> configuration *keys*, which are not case sensitive [1], and users might
>> have gotten used to thinking of strings that look like this to be
>> non-case-sensitive.
> 
> There were some discussions a few versions of this series ago with
> Peff, Junio and perhaps others about this.
> I thought that being case insensitive was better and Peff kind of
> agreed with that, but as Junio disagreed it is now case sensitive.

OK, it's my fault for not having followed along with the history of this patch series.

Show 23 quoted lines
>>> +With `addIfDifferent`, a new trailer will be added only if no trailer
>>> +with the same (token, value) pair is already in the message.
>>> ++
>>> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
>>> +trailer with the same (token, value) pair is above or below the line
>>> +where the new trailer will be added.
>>> ++
>>> +With `add`, a new trailer will be added, even if some trailers with
>>> +the same (token, value) pair are already in the message.
>>> ++
>>> +With `overwrite`, the new trailer will overwrite an existing trailer
>>> +with the same token.
>>
>> What if there are multiple existing trailers with the same token?  Are
>> they all overwritten?
> 
> No, if where == after, only the last one is overwritten, and if where
> == before, only the first one is overwritten.
> 
> I could add an "overwriteAll" option. It could be interesting to use
> when a command using "$ARG" is configured, as this way the command
> would apply to all the trailers with the given token instead of just
> the last or first one.

It seems to me that the current behavior (rewriting exactly one existing line) is not that useful. Why not make "overwrite" overwrite *all* existing matching lines?

>>> +With `doNothing`, nothing will be done, that is no new trailer will be
>>> +added if there is already one with the same token in the message.

I just noticed a punctuation problem (comma -> semicolon and add comma) in the sentence above:

    With `doNothing`, nothing will be done; that is, no new trailer
    will be added if there is already one with the same token in the
    message.
Show 42 quoted lines
>>> +
>>> +trailer.<token>.ifmissing::
>>> +     This option makes it possible to choose what action will be
>>> +     performed when there is not yet any trailer with the same
>>> +     token in the message.
>>> ++
>>> +The valid values for this option are: `add` (this is the default) and
>>> +`doNothing`.
>>> ++
>>> +With `add`, a new trailer will be added.
>>> ++
>>> +With `doNothing`, nothing will be done.
>>> +
>>> +trailer.<token>.command::
>>> +     This option can be used to specify a shell command that will
>>> +     be used to automatically add or modify a trailer with the
>>> +     specified 'token'.
>>> ++
>>> +When this option is specified, it is like if a special 'token=value'
>>> +argument is added at the end of the command line, where 'value' will
>>> +be given by the standard output of the specified command.
>>
>> Maybe reword to
>>
>>     When this option is specified, the behavior is as if a special
>>     'token=value' argument were added at the end of the command line,
>>     where 'value' is taken to be the standard output of the specified
>>     command.
>>
>> And if it is the case, maybe add "with leading and trailing whitespace
>> trimmed off" at the end of the sentence.
> 
> Ok.
> 
>>> +If the command contains the `$ARG` string, this string will be
>>> +replaced with the 'value' part of an existing trailer with the same
>>> +token, if any, before the command is launched.
>>
>> What if the key appears multiple times in existing trailers?
> 
> It will be done only once for the last or first trailer with the key
> depending on "where".

It seems like the UI for "git interpret-trailers" is optimized for trailers that appear only once. That's a bit limiting. Maybe it would be better to pipe the existing values to the command's standard input, one per line? For example, suppose we run

    git interpret-trailers \
        Signed-off-by='Christian Couder <christian.couder@gmail.com>'
with the following input:
    Human-readable subject
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    Signed-off-by: Christian Couder <christian.couder@gmail.com>

Then the following three lines could be piped to the command (i.e., one value per line, without the key):

    Junio C Hamano <gitster@pobox.com>
    Christian Couder <christian.couder@gmail.com>
    Christian Couder <christian.couder@gmail.com>

Then, supposing the command were "sort --unique", the command's output would be

    Christian Couder <christian.couder@gmail.com>
    Junio C Hamano <gitster@pobox.com>

which would be converted back into trailer lines by prepending "Signed-off-by: ", resulting in the modified commit message

    Human-readable subject
    Signed-off-by: Christian Couder <christian.couder@gmail.com>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>

(Not that we would want to do with "Signed-off-by" trailers, but you get the idea.)

I'm really sorry for coming so late to the show. Feel free to ignore any of my comments with the justification "too late".

Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Christian Couder· Apr 25, 2014, 21:07 UTC · re: Michael Haggerty · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

From: Michael Haggerty <mhagger@alum.mit.edu>
Show 12 quoted lines
>
> On 04/08/2014 01:35 PM, Christian Couder wrote:
>> 
>> The trailers are recognized in the input commit message using the
>> following rules:
>>  - only lines that contains a ':' are considered trailers,
>>  - the trailer lines must all be next to each other,
>>  - after them it's only possible to have some lines that contain only spaces,
>>  - before them there must be at least one line with only spaces
> 
> Thanks for all the explanation.  I think that most/all of this
> information should be included in the documentation.

Ok, I included the above rules in v11, but maybe not other pieces of information that you might have wanted.

Show 20 quoted lines
>>>> +OPTIONS
>>>> +-------
>>>> +--trim-empty::
>>>> +     If the 'value' part of any trailer contains only whitespace,
>>>> +     the whole trailer will be removed from the resulting message.
>>>
>>> Does this apply to existing trailers, new trailers, or both?
>> 
>> Both.
>> 
>>> If it applies to existing trailers, then it seems a bit dangerous, in the
>>> sense that the command might end up changing trailers that are unrelated
>>> to the one that the command is trying to add.
>> 
>> The command is not just for adding trailers.
>> But there could be an option to just trim trailers that are added.
> 
> Maybe that should be the *only* behavior of this option.
> 
> Maybe there should be a trailer.<token>.trimEmpty config option.

One possible usage of the "git interpret-trailers" command that was discussed in the early threads was the following:

1) You have a commit message template like the following:

----------------- ***SUBJECT***

***MESSAGE***
Fixes: 
Cc: 
Cc: 
Reviewed-by: 
Reviewed-by: 
Signed-off-by: 
-----------------
2) The user add some information when committing:
$ git commit --trailer "Fixes:534" --trailer "Signed-off-by: Michael <mhagger@alum.mit.edu>"
3) "git interpret-trailers" is used automatically by "git commit"
without --trim-empty, and it is passed the --trailer arguments and the
commit message template, so the user is shown the result which is for
example the following:

----------------- ***SUBJECT***

***MESSAGE***
Fixes: 534
Cc: 
Cc: 
Reviewed-by: 
Reviewed-by: 
Signed-off-by: Michael <mhagger@alum.mit.edu>
-----------------
4) The user adds some information and the resulting message is for
example:

----------------- Doing foo and bar

And also baz.
Fixes: 534
Cc: 
Cc: Peff <peff@peff.net>
Reviewed-by: Junio <gitster@pobox.com>
Reviewed-by: 
Signed-off-by: Michael <mhagger@alum.mit.edu>
-----------------
5) Then a post commit hook automatically uses "git interpret-trailers
--trim-empty" on the result, so the commit message is eventually the
following:

----------------- Doing foo and bar

And also baz.
Fixes: 534
Cc: Peff <peff@peff.net>
Reviewed-by: Junio <gitster@pobox.com>
Signed-off-by: Michael <mhagger@alum.mit.edu>
-----------------

So I think it could be very useful to have --trim-empty work on all the trailers, not just those passed as arguments.

Show 24 quoted lines
>>> If a commit message containing trailer lines with separators other than
>>> ':' is input to the program, will it recognize them as trailer lines?
>> 
>> No, '=' and '#' are not supported in the input message, only in the output.
>> 
>>> Do such trailer lines have to have the same separator as the one listed
>>> in this configuration setting to be recognized?
>> 
>> No they need to have ':' as a separator.
>> 
>> The reason why only ':' is supported is because it is the cannonical
>> trailer separator and it could create problems with many input
>> messages if other separators where supported.
>> 
>> Maybe we could detect a special line like the following:
>> 
>> # TRAILERS START
>> 
>> in the input message and consider everyhting after that line as trailers.
>> In this case it would be ok to accept other separators.
> 
> It would be ugly to have to use such a line.  I think it would be
> preferable to be more restrictive about trailer separators than to
> require something like this.
The code is already very restrictive about trailer separators.
Show 12 quoted lines
> From what you've said above, it sounds like your code might get confused
> with the following input commit message:
> 
>     This is the human-readable comment
> 
>     Foo: bar
>     Fixes #123
>     Plugh: xyzzy
> 
> It seems to me that none of these lines would be accepted as trailers,
> because they include a non-trailer "Fixes" line (non-trailer in the
> sense that it doesn't use a colon separator).
Yeah, they would not be accepted because the code is very restrictive.
The following would be accepted:
     Foo: bar
     Fixes: 123
     Plugh: xyzzy
Show 15 quoted lines
>>> I suppose that there is some compelling reason to allow non-colon
>>> separators here.  If not, it seems like it adds a lot of complexity and
>>> should maybe be omitted, or limited to only a few specific separators.
>> 
>> Yeah, but in the early threads concerning this subject, someone said
>> that GitHub for example uses "bug #XXX".
>> I will have a look again.
> 
> Yes, that's true: GitHub recognizes strings like "fixes #33" but not if
> there is an intervening colon like in "fixes: #33".  OTOH GitHub
> recognizes such strings wherever they appear in the commit message (they
> don't have to be in "trailer" lines).  So I'm not sure that the added
> complication is worth it if GitHub is the only use case.  (And maybe we
> could convince GitHub to recognize "Fixes: #33" if such syntax becomes
> the de-facto Git standard for trailers.)

I don't think there is a lot of complexity. But maybe I need to explain how it works better. Feel free to suggest me sentences I could add.

Show 27 quoted lines
>>>> +With `addIfDifferent`, a new trailer will be added only if no trailer
>>>> +with the same (token, value) pair is already in the message.
>>>> ++
>>>> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
>>>> +trailer with the same (token, value) pair is above or below the line
>>>> +where the new trailer will be added.
>>>> ++
>>>> +With `add`, a new trailer will be added, even if some trailers with
>>>> +the same (token, value) pair are already in the message.
>>>> ++
>>>> +With `overwrite`, the new trailer will overwrite an existing trailer
>>>> +with the same token.
>>>
>>> What if there are multiple existing trailers with the same token?  Are
>>> they all overwritten?
>> 
>> No, if where == after, only the last one is overwritten, and if where
>> == before, only the first one is overwritten.
>> 
>> I could add an "overwriteAll" option. It could be interesting to use
>> when a command using "$ARG" is configured, as this way the command
>> would apply to all the trailers with the given token instead of just
>> the last or first one.
> 
> It seems to me that the current behavior (rewriting exactly one existing
> line) is not that useful.  Why not make "overwrite" overwrite *all*
> existing matching lines?
I was thinking that people could use the following template message:
---------------
Signed-off-by: 
Signed-off-by: YOU-WILL-BE-AUTOMATICALLY-ADDED-HERE
---------------
and the following config:
---------------
[trailer "s-o-b"]
	 key = "Signed-off-by: "
	 ifexist = overwrite
	 command = echo \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\"
---------------

This way the user can add other people's s-o-b before the last one which will always contain his own s-o-b.

Show 11 quoted lines
>>>> +If the command contains the `$ARG` string, this string will be
>>>> +replaced with the 'value' part of an existing trailer with the same
>>>> +token, if any, before the command is launched.
>>>
>>> What if the key appears multiple times in existing trailers?
>> 
>> It will be done only once for the last or first trailer with the key
>> depending on "where".
> 
> It seems like the UI for "git interpret-trailers" is optimized for
> trailers that appear only once.  That's a bit limiting.

As I said, it is possible to add an overwriteAll option. I think that would fix the current limitations.

Show 37 quoted lines
> Maybe it would
> be better to pipe the existing values to the command's standard input,
> one per line?  For example, suppose we run
> 
>     git interpret-trailers \
>         Signed-off-by='Christian Couder <christian.couder@gmail.com>'
> 
> with the following input:
> 
>     Human-readable subject
> 
>     Signed-off-by: Junio C Hamano <gitster@pobox.com>
>     Signed-off-by: Christian Couder <christian.couder@gmail.com>
> 
> Then the following three lines could be piped to the command (i.e., one
> value per line, without the key):
> 
>     Junio C Hamano <gitster@pobox.com>
>     Christian Couder <christian.couder@gmail.com>
>     Christian Couder <christian.couder@gmail.com>
> 
> Then, supposing the command were "sort --unique", the command's output
> would be
> 
>     Christian Couder <christian.couder@gmail.com>
>     Junio C Hamano <gitster@pobox.com>
> 
> which would be converted back into trailer lines by prepending
> "Signed-off-by: ", resulting in the modified commit message
> 
>     Human-readable subject
> 
>     Signed-off-by: Christian Couder <christian.couder@gmail.com>
>     Signed-off-by: Junio C Hamano <gitster@pobox.com>
> 
> (Not that we would want to do with "Signed-off-by" trailers, but you get
> the idea.)

Yeah, but I think the existing options like addIfDifferent and addIfDifferentNeighbor are simpler to do these kind of things.

And it is also possible to add a "where = sorted" option.

And if later we realize that people are still not happy, we can still add a special "trailer.<token>.filter" that could do what you suggest.

Thanks, Christian.

Michael Haggerty· Apr 28, 2014, 10:16 UTC · re: Christian Couder · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

On 04/25/2014 11:07 PM, Christian Couder wrote:
Show 40 quoted lines
> From: Michael Haggerty <mhagger@alum.mit.edu>
>>>>> +OPTIONS
>>>>> +-------
>>>>> +--trim-empty::
>>>>> +     If the 'value' part of any trailer contains only whitespace,
>>>>> +     the whole trailer will be removed from the resulting message.
>>>>
>>>> Does this apply to existing trailers, new trailers, or both?
>>>
>>> Both.
>>>
>>>> If it applies to existing trailers, then it seems a bit dangerous, in the
>>>> sense that the command might end up changing trailers that are unrelated
>>>> to the one that the command is trying to add.
>>>
>>> The command is not just for adding trailers.
>>> But there could be an option to just trim trailers that are added.
>>
>> Maybe that should be the *only* behavior of this option.
>>
>> Maybe there should be a trailer.<token>.trimEmpty config option.
> 
> One possible usage of the "git interpret-trailers" command that was
> discussed in the early threads was the following:
> 
> 1) You have a commit message template like the following:
> 
> -----------------
> ***SUBJECT***
> 
> ***MESSAGE***
> 
> Fixes: 
> Cc: 
> Cc: 
> Reviewed-by: 
> Reviewed-by: 
> Signed-off-by: 
> -----------------
> [...etc...]

Thanks for the explanation. Now the --trim-empty option makes a lot more sense.

Show 67 quoted lines
>>>> If a commit message containing trailer lines with separators other than
>>>> ':' is input to the program, will it recognize them as trailer lines?
>>>
>>> No, '=' and '#' are not supported in the input message, only in the output.
>>>
>>>> Do such trailer lines have to have the same separator as the one listed
>>>> in this configuration setting to be recognized?
>>>
>>> No they need to have ':' as a separator.
>>>
>>> The reason why only ':' is supported is because it is the cannonical
>>> trailer separator and it could create problems with many input
>>> messages if other separators where supported.
>>>
>>> Maybe we could detect a special line like the following:
>>>
>>> # TRAILERS START
>>>
>>> in the input message and consider everyhting after that line as trailers.
>>> In this case it would be ok to accept other separators.
>>
>> It would be ugly to have to use such a line.  I think it would be
>> preferable to be more restrictive about trailer separators than to
>> require something like this.
> 
> The code is already very restrictive about trailer separators.
> 
>> From what you've said above, it sounds like your code might get confused
>> with the following input commit message:
>>
>>     This is the human-readable comment
>>
>>     Foo: bar
>>     Fixes #123
>>     Plugh: xyzzy
>>
>> It seems to me that none of these lines would be accepted as trailers,
>> because they include a non-trailer "Fixes" line (non-trailer in the
>> sense that it doesn't use a colon separator).
> 
> Yeah, they would not be accepted because the code is very restrictive.
> 
> The following would be accepted:
> 
>      Foo: bar
>      Fixes: 123
>      Plugh: xyzzy
> 
>>>> I suppose that there is some compelling reason to allow non-colon
>>>> separators here.  If not, it seems like it adds a lot of complexity and
>>>> should maybe be omitted, or limited to only a few specific separators.
>>>
>>> Yeah, but in the early threads concerning this subject, someone said
>>> that GitHub for example uses "bug #XXX".
>>> I will have a look again.
>>
>> Yes, that's true: GitHub recognizes strings like "fixes #33" but not if
>> there is an intervening colon like in "fixes: #33".  OTOH GitHub
>> recognizes such strings wherever they appear in the commit message (they
>> don't have to be in "trailer" lines).  So I'm not sure that the added
>> complication is worth it if GitHub is the only use case.  (And maybe we
>> could convince GitHub to recognize "Fixes: #33" if such syntax becomes
>> the de-facto Git standard for trailers.)
> 
> I don't think there is a lot of complexity.
> But maybe I need to explain how it works better.
> Feel free to suggest me sentences I could add.

I am really excited about having better support for trailers in Git, and I want to thank you for your work. For me the promise of trailers is

* A way for users to add information to commits for whatever purpose
  they want, without having to convince upstream to built support in.
* A standard format for that information, so that all tools can agree
  how to read/write trailers without being confused by or breaking
  trailers that they didn't know about in advance.
* A format that is straightforward enough that it can be machine-
  readable with minimum ambiguity.
* Some command-line tools to make it easy for scripts to work with
  trailers, and that serve as a reference implementation that other
  Git implementations can imitate.  For example, I totally expect that
  we will soon want a command-line tool for inquiring about the
  presence and contents of trailers, for use in scripting.  Eventually
  we will want to be able to do stuff like
      git trailers --get-all s-o-b origin/master..origin/next
      git rev-list --trailer=s-o-b:gitster@pobox.com master
      git trailers --pipe --draft \
          --add-first fixes \
          --append '# You can delete the following line:' \
          --append s-o-b:"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>" \
          --unset private
      git trailers --pipe --verify --tidy-up

I think it is really important to nail down the format of trailers tightly enough that everybody who reads/writes a commit message agrees about exactly what trailers are there. For example the specification might look something like:

    A commit message can optionally end with a block of trailers.
    The trailers, if present, must be separated from the rest of the
    commit message by one or more blank lines (lines that contain only
    whitespace).  There must be no blank lines within the list of
    trailers.  It is allowed to have blank lines after the trailers.
    Each trailer line must match the following Perl regular
    expression:
        ^([A-Za-z0-9_-]+)\s*:\s*(.*[^\s])\s*$
    The string matching the first group is called the key and the string
    matching the second is called the value.  Keys are considered to be
    case-insensitive [or should they be case-sensitive?].  The
    interpretation of values is left entirely up to the application.
    Values must not be empty.
    However, in --draft and --cleanup modes, empty values *are*
    allowed, as are comments (lines starting with `core.commentchar`)
    within the trailer block.  In --draft mode such lines are passed
    through unchanged, and in --cleanup mode such lines are removed.

I'm not saying this is the exact definition that we want; I'm just providing an example of the level of precision that I think is needed.

With regard to the separator character, my concern is not about how to document the rules for this one tool. It's more about having really well-defined rules that are consistent between reading and writing. For me it seems silly to let "git interpret-trailers" output trailers that it doesn't know how to read back in, and pretty much be a show-stopper if the presence of such trailers makes the tool unable to read other trailers in the same commit message.

So my preference would be to make the format of trailers really strict; for example, only allowing colon separators as in the regexp above. People who want to work with trailers using Git tools will just have to conform to this format.

But if we must support flexibility in the separator characters, then I think it is important that we be able to read whatever we can write. For me this means:

* Enumerating a list of allowed separators (e.g., [:=#])
* Specifying how it is decided what separator to use when generating
  new trailers
* Specifying what appends when a trailer is read and then written again:
  is its separator preserved, or is the trailer converted to use the
  separator configured for that particular key in the config.  And if
  the latter, what happens if a key's separator is not configured?
* Specifying whether whitespace around a separator is adjusted when
  reading then writing a trailer.  For example, is
      Foo SP SP : HT bar SP
  canonicalized to
      Foo: SP bar
  (SP=space, HT=tab)?  What about
      Fixes SP #33
  ?  What if the separator for the "fixes" key is not configured?

The reason that I prefer supporting only colons is that more flexibility inevitably raises lots of questions like this, makes the documentation and implementation more complicated, and makes it harder for other implementations to be sure they agree 100% with the reference implementation.

Show 46 quoted lines
>>>>> +With `addIfDifferent`, a new trailer will be added only if no trailer
>>>>> +with the same (token, value) pair is already in the message.
>>>>> ++
>>>>> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
>>>>> +trailer with the same (token, value) pair is above or below the line
>>>>> +where the new trailer will be added.
>>>>> ++
>>>>> +With `add`, a new trailer will be added, even if some trailers with
>>>>> +the same (token, value) pair are already in the message.
>>>>> ++
>>>>> +With `overwrite`, the new trailer will overwrite an existing trailer
>>>>> +with the same token.
>>>>
>>>> What if there are multiple existing trailers with the same token?  Are
>>>> they all overwritten?
>>>
>>> No, if where == after, only the last one is overwritten, and if where
>>> == before, only the first one is overwritten.
>>>
>>> I could add an "overwriteAll" option. It could be interesting to use
>>> when a command using "$ARG" is configured, as this way the command
>>> would apply to all the trailers with the given token instead of just
>>> the last or first one.
>>
>> It seems to me that the current behavior (rewriting exactly one existing
>> line) is not that useful.  Why not make "overwrite" overwrite *all*
>> existing matching lines?
> 
> I was thinking that people could use the following template message:
> 
> ---------------
> Signed-off-by: 
> Signed-off-by: YOU-WILL-BE-AUTOMATICALLY-ADDED-HERE
> ---------------
> 
> and the following config:
> 
> ---------------
> [trailer "s-o-b"]
> 	 key = "Signed-off-by: "
> 	 ifexist = overwrite
> 	 command = echo \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\"
> ---------------
> 
> This way the user can add other people's s-o-b before the last one
> which will always contain his own s-o-b.
That seems fragile.  For example, if the user changes the lines to
    Signed-off-by: Somebody Else <...>

(deleting the "YOU-WILL-BE" line, maybe because they don't want to sign off the commit) then not only will their wish be contradicted, but also Somebody Else would be deleted.

What about allowing a --draft option, which allows blank trailer values plus comments interspersed in the trailer lines? (I.e., the equivalent of --trim-empty could be the default and --draft would turn it off plus allow interspersed comments.) Then the template could be

    # You can add one or more Signed-off-by lines for other people here.
    # A Signed-off-by line for you will be appended automatically when
    # you commit.
    Signed-off-by:
Or, even better:
    # You can add one or more Signed-off-by lines here:
    Signed-off-by:
    Signed-off-by:
    # You can delete the following line if you don't want it:
    Signed-off-by: Me <me@example.com>

; i.e., the Signed-off-by line for the author could be filled in *before* the user is asked to edit the commit message. There could also be a --cleanup mode that allows blank values and comments on input but removes them from the output.

> [...]

Given Git's requirements for backwards compatibility, a specification that we release now will have to be supported forever (because it will be baked into commits and can *never* be changed), and any trailer-handling tools that we release now will have to be supported for years (until at least Git 3.0).

All in all, I think that there has been a lot of discussion about the
interface of this one command, "git interpret-trailers", including its
quite complicated configuration and a command-line behavior.  And yet it
seems to me that not many Git developers have been very engaged in the
conversation, and Junio (who has) still doesn't seem satisfied with it.
 I (though among the too-little engaged) have the feeling that it is
still a ways from maturity.

On the contrary, the data format and semantics of the finished trailers seem to have gotten too little attention, even though they are simpler to define and even more important than the interface of the command used to manipulate trailers.

I think it would be really helpful to have a careful specification of the data format, and make sure that everybody agrees on what we want. For example, I think it is crucial that the trailers can be read and written unambiguously.

Once that's clear, it will be a lot easier to be sure that the tool(s) for working with trailers conform to the specification.

Even then, I think it might be prudent to mark "git interpret-trailers" as "experimental" and/or put it under "contrib" rather than among the main Git commands for a couple of releases. Luckily, it is very loosely coupled to the rest of Git, so I don't see any urgency to having it in core [1]. After people have had time to experiment with it, then it could be moved to core.

Michael

[1] Having the script in contrib would also make it possible to implement it use a scripting language to make it easier to iterate on the design. When the details are agreed it could have been reimplemented in C. But I guess that ship has already sailed.

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
Christian Couder· May 25, 2014, 08:37 UTC · re: Michael Haggerty · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

From: Michael Haggerty <mhagger@alum.mit.edu>
Show 11 quoted lines
> On 04/25/2014 11:07 PM, Christian Couder wrote:
>> 
>> I don't think there is a lot of complexity.
>> But maybe I need to explain how it works better.
>> Feel free to suggest me sentences I could add.
> 
> I am really excited about having better support for trailers in Git, and
> I want to thank you for your work.  For me the promise of trailers is
> 
> * A way for users to add information to commits for whatever purpose
>   they want, without having to convince upstream to built support in.
Yeah, I agree this is the main purpose of trailers.
> * A standard format for that information, so that all tools can agree
>   how to read/write trailers without being confused by or breaking
>   trailers that they didn't know about in advance.

Yeah, but don't you think this goal can sometimes go against the previous goal?

I mean, if some users for their project think that it's better, for example, if they use trailers like "Fix #42" instead of "Fix: 42", because their bug tracking system supports "Fix #42" better, we should let them do what suits them better, even if Git supports them not as well as if they used "Fix: 42".

> * A format that is straightforward enough that it can be machine-
>   readable with minimum ambiguity.

Yeah, but again this could go against the main purpose of trailers above.

> * Some command-line tools to make it easy for scripts to work with
>   trailers, and that serve as a reference implementation that other
>   Git implementations can imitate.
Yeah, ok, as long as we keep in mind the main purpose.
Show 13 quoted lines
> For example, I totally expect that
>   we will soon want a command-line tool for inquiring about the
>   presence and contents of trailers, for use in scripting.  Eventually
>   we will want to be able to do stuff like
> 
>       git trailers --get-all s-o-b origin/master..origin/next
>       git rev-list --trailer=s-o-b:gitster@pobox.com master
>       git trailers --pipe --draft \
>           --add-first fixes \
>           --append '# You can delete the following line:' \
>           --append s-o-b:"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>" \
>           --unset private
>       git trailers --pipe --verify --tidy-up
Yeah, feel free to help make this kind of things possible :-)
> I think it is really important to nail down the format of trailers
> tightly enough that everybody who reads/writes a commit message agrees
> about exactly what trailers are there.

I think we should have a default format for trailers that is clear, but we should not force users to use this format. Because forcing it would go against the main goal of trailers that you listed first above.

Show 19 quoted lines
> For example the specification
> might look something like:
> 
>     A commit message can optionally end with a block of trailers.
>     The trailers, if present, must be separated from the rest of the
>     commit message by one or more blank lines (lines that contain only
>     whitespace).  There must be no blank lines within the list of
>     trailers.  It is allowed to have blank lines after the trailers.
> 
>     Each trailer line must match the following Perl regular
>     expression:
> 
>         ^([A-Za-z0-9_-]+)\s*:\s*(.*[^\s])\s*$
> 
>     The string matching the first group is called the key and the string
>     matching the second is called the value.  Keys are considered to be
>     case-insensitive [or should they be case-sensitive?].  The
>     interpretation of values is left entirely up to the application.
>     Values must not be empty.

I tried to be clearer in the v12 I just posted, and I think it should be enough to be very clear. We might want to tweak a little the specifications later, so being too strict might be counter productive.

And as other tools might already use trailers in a case-sensitive way and yet other tools in a case-insensitive way, I am not sure we would gain anything by specifying if keys or values should be interpreted in a case-sensitive or case-insensitive way. On the contrary we might upset people already using some of these tools for no good reason.

>     However, in --draft and --cleanup modes, empty values *are*
>     allowed, as are comments (lines starting with `core.commentchar`)
>     within the trailer block.  In --draft mode such lines are passed
>     through unchanged, and in --cleanup mode such lines are removed.

I am not sure we should use modes. I think options like "--trim-empty", "--allow-comments", "--allow-empty" might be clearer.

> I'm not saying this is the exact definition that we want; I'm just
> providing an example of the level of precision that I think is needed.
Yeah, but I think too much precision can be counter productive.
Show 7 quoted lines
> With regard to the separator character, my concern is not about how to
> document the rules for this one tool.  It's more about having really
> well-defined rules that are consistent between reading and writing.  For
> me it seems silly to let "git interpret-trailers" output trailers that
> it doesn't know how to read back in, and pretty much be a show-stopper
> if the presence of such trailers makes the tool unable to read other
> trailers in the same commit message.

We might allow an option to specify witch separator(s) should be allowed in the input messages for example. Right now I think it is enough if we support well the default separator, ':' in the input message.

> So my preference would be to make the format of trailers really strict;
> for example, only allowing colon separators as in the regexp above.
> People who want to work with trailers using Git tools will just have to
> conform to this format.

I don't think we should cast in stone the format for trailers, because of the main purpose of trailers.

The format of the commit header for example is cast in stone, but that's ok because it is mostly for Git internal use. Trailers are mostly for external use by users who already have tools expecting different formats.

There are already users who are not happy that they cannot easily have other commit headers, and we point them to trailers. If we specify trailers too strictly, where will we point them to?

> But if we must support flexibility in the separator characters, then I
> think it is important that we be able to read whatever we can write.
An option like --input-separator might be enough to support this.
> For me this means:
> 
> * Enumerating a list of allowed separators (e.g., [:=#])
Junio suggested in a message that users might use different separators
like '%'.
 
> * Specifying how it is decided what separator to use when generating
>   new trailers

This is already possible with the 'trailer.<token>.key' config variable.

> * Specifying what appends when a trailer is read and then written again:
>   is its separator preserved, or is the trailer converted to use the
>   separator configured for that particular key in the config.  And if
>   the latter, what happens if a key's separator is not configured?

Right now we only accept ':' as input separator for the messages and ':' and '=' for the --trailer option, and the default output separator is ':'. If the user specify a different separator in a key, this separator will be used only in the output for this key.

If this is not clear in the documentation, please susggest specific improvements.

Show 14 quoted lines
> * Specifying whether whitespace around a separator is adjusted when
>   reading then writing a trailer.  For example, is
> 
>       Foo SP SP : HT bar SP
> 
>   canonicalized to
> 
>       Foo: SP bar
> 
>   (SP=space, HT=tab)?  What about
> 
>       Fixes SP #33
> 
>   ?  What if the separator for the "fixes" key is not configured?
I tried to be very clear in the doc in v12.
Show 5 quoted lines
> The reason that I prefer supporting only colons is that more flexibility
> inevitably raises lots of questions like this, makes the documentation
> and implementation more complicated, and makes it harder for other
> implementations to be sure they agree 100% with the reference
> implementation.
Yeah, but we should not forget the main purpose of trailers.
Show 30 quoted lines
>>> It seems to me that the current behavior (rewriting exactly one existing
>>> line) is not that useful.  Why not make "overwrite" overwrite *all*
>>> existing matching lines?
>> 
>> I was thinking that people could use the following template message:
>> 
>> ---------------
>> Signed-off-by: 
>> Signed-off-by: YOU-WILL-BE-AUTOMATICALLY-ADDED-HERE
>> ---------------
>> 
>> and the following config:
>> 
>> ---------------
>> [trailer "s-o-b"]
>> 	 key = "Signed-off-by: "
>> 	 ifexist = overwrite
>> 	 command = echo \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\"
>> ---------------
>> 
>> This way the user can add other people's s-o-b before the last one
>> which will always contain his own s-o-b.
> 
> That seems fragile.  For example, if the user changes the lines to
> 
>     Signed-off-by: Somebody Else <...>
> 
> (deleting the "YOU-WILL-BE" line, maybe because they don't want to sign
> off the commit) then not only will their wish be contradicted, but also
> Somebody Else would be deleted.

I agree that it is fragile, but we can add an overwriteAll option if that suits people needs better. "overwrite" is needed anyway for trailers where there should be only one trailer with a given key.

> What about allowing a --draft option, which allows blank trailer values
> plus comments interspersed in the trailer lines?  (I.e., the equivalent
> of --trim-empty could be the default and --draft would turn it off plus
> allow interspersed comments.)

Yeah, or --allow-comments. As I said above prefer many orthogonal options, rather than some general options like --draft.

Show 19 quoted lines
> Then the template could be
> 
>     # You can add one or more Signed-off-by lines for other people here.
>     # A Signed-off-by line for you will be appended automatically when
>     # you commit.
>     Signed-off-by:
> 
> Or, even better:
> 
>     # You can add one or more Signed-off-by lines here:
>     Signed-off-by:
>     Signed-off-by:
>     # You can delete the following line if you don't want it:
>     Signed-off-by: Me <me@example.com>
> 
> ; i.e., the Signed-off-by line for the author could be filled in
> *before* the user is asked to edit the commit message.  There could also
> be a --cleanup mode that allows blank values and comments on input but
> removes them from the output.
I would prefer to add --trim-comments rather than a --cleanup mode. 
Show 7 quoted lines
>> [...]
> 
> Given Git's requirements for backwards compatibility, a specification
> that we release now will have to be supported forever (because it will
> be baked into commits and can *never* be changed), and any
> trailer-handling tools that we release now will have to be supported for
> years (until at least Git 3.0).

Yeah, I know that. So if we are too strict in the specification will be stuck for a long time.

Show 7 quoted lines
> All in all, I think that there has been a lot of discussion about the
> interface of this one command, "git interpret-trailers", including its
> quite complicated configuration and a command-line behavior.  And yet it
> seems to me that not many Git developers have been very engaged in the
> conversation, and Junio (who has) still doesn't seem satisfied with it.
>  I (though among the too-little engaged) have the feeling that it is
> still a ways from maturity.

My opinion is that many Git developers have been engaged and you can see that in the Cc.

I cannot tell if they are all very happy or not but I suppose that if they were very unhappy they would tell it.

Show 12 quoted lines
> On the contrary, the data format and semantics of the finished trailers
> seem to have gotten too little attention, even though they are simpler
> to define and even more important than the interface of the command used
> to manipulate trailers.
> 
> I think it would be really helpful to have a careful specification of
> the data format, and make sure that everybody agrees on what we want.
> For example, I think it is crucial that the trailers can be read and
> written unambiguously.
> 
> Once that's clear, it will be a lot easier to be sure that the tool(s)
> for working with trailers conform to the specification.

Please realize that too much specification is not always good and that it cuts both ways...

Show 6 quoted lines
> Even then, I think it might be prudent to mark "git interpret-trailers"
> as "experimental" and/or put it under "contrib" rather than among the
> main Git commands for a couple of releases.  Luckily, it is very loosely
> coupled to the rest of Git, so I don't see any urgency to having it in
> core [1].  After people have had time to experiment with it, then it
> could be moved to core.

Yeah, it is very loosely coupled to the rest of Git by design. I posted the first version of this series around last November. So people have had a very long time to review it, comment on it, experiment with it, bikeshed many details... And in the same time many users have asked for some features that "git interpret-trailers" provides...

Regards, Christian.

Michael Haggerty· May 27, 2014, 08:21 UTC · re: Christian Couder · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

tl;dr: This patch series wants to introduce a permanent new Git data format. The current version can write trailers in formats that it is incapable of reading, which I consider broken. I advocate a stricter specification of the format of trailers, at least until we get feedback from users that they need more flexibility.

On 05/25/2014 10:37 AM, Christian Couder wrote:
Show 19 quoted lines
> From: Michael Haggerty <mhagger@alum.mit.edu>
> [...]
>> * A way for users to add information to commits for whatever purpose
>>   they want, without having to convince upstream to built support in.
> 
> Yeah, I agree this is the main purpose of trailers.
> 
>> * A standard format for that information, so that all tools can agree
>>   how to read/write trailers without being confused by or breaking
>>   trailers that they didn't know about in advance.
> 
> Yeah, but don't you think this goal can sometimes go against the
> previous goal?
> 
> I mean, if some users for their project think that it's better, for
> example, if they use trailers like "Fix #42" instead of "Fix: 42",
> because their bug tracking system supports "Fix #42" better, we should
> let them do what suits them better, even if Git supports them not as
> well as if they used "Fix: 42".

The flexibility that comes from offering our users a more-or-less general key/value store already accomplishes the first goal. With that the users *could* store their data as "Fix: 42" or "Fixes: #42" and satisfy their functional requirements.

Giving them the option to use "Fix #42" doesn't make *any* new functionality possible. It is pure eye-candy. And it would come at the IMO high cost of making it harder for *everybody* to work with the metadata. It makes the specification more complicated. It makes the code more complicated. It makes the configuration more complicated. It makes it more likely that there will be "false positives" (text in a commit message that our code recognizes as key/value data even though it was not meant to be). And in my opinion it makes the k/v data itself harder for a human to read because it is not in a uniform format.

The only justification I can think of for allowing more flexible formats would be to "retroactively" support metadata that people already have in their history. Are there "famous" or "important" existing metadata formats that are incompatible with "Key: Value"?

More to the point: do you have a concrete reason for wanting to support alternative formats like "Fix #42", or is it based more on the feeling that users will want it?

Remember, it would be really easy to release v1 of this feature with a strict format, then wait and see if users clamor for more flexibility. We can always add more flexibility later. Whereas if v1 already supports more flexible formats, we pretty much have to support them forever.

Show 5 quoted lines
>> * A format that is straightforward enough that it can be machine-
>>   readable with minimum ambiguity.
> 
> Yeah, but again this could go against the main purpose of trailers
> above.

No, the users have all the flexibility they need if they can choose their own key/value schema. Allowing alternative formats adds very little.

I feel strongly that it would be a bad mistake to leave the specification of trailers so loose that they cannot be machine-readable with a good degree of confidence. Tools that add trailers should be composable. The current scheme is not. For example, suppose one tool wants to add a "Fix #42" line and another one wants to add "Signed-off-by":

    git config trailer.fix.key "Fix #"
    git config trailer.sign.key "Signed-off-by: "
    git config trailer.sign.ifexists doNothing
    echo "subject
    Signed-off-by: Alice <alice@example.com>" |
        git interpret-trailers --trailer fix=42 |
        git interpret-trailers --trailer sign="Bob <bob@example.com>"
    --------- output ------------------------------------------
    subject
    Signed-off-by: Alice <alice@example.com>
    Fix #42
    Signed-off-by: Bob <bob@example.com>
    -----------------------------------------------------------

The result is that the trailers end up not in one block but in two (meaning that the first block is no longer recognized as a trailer block), and the second "Signed-off-by" line, which should have been omitted because of ifexists=doNothing, was incorrectly added.

Or let's do something like the "commit template" example from the documentation, but using "Fix #" instead of "Fixes: ":

    echo "***subject***
    ***message***
    Fix #
    Cc:
    Reviewed-by:
    Signed-off-by:
    " |
        sed -Ee 's/(Reviewed-by.*)/\1me/' |
        git interpret-trailers --trim-empty --trailer "git-version: foo"
    ---------- output ------------------------------------------
    ***subject***
    ***message***
    Fix #
    Cc:
    Reviewed-by: me
    Signed-off-by:
    git-version: foo
    ------------------------------------------------------------

Not only haven't the empty lines been stripped off, but a new trailer block has been created for "git-version".

I consider this broken.
Show 30 quoted lines
> [...]
>> For example the specification
>> might look something like:
>>
>>     A commit message can optionally end with a block of trailers.
>>     The trailers, if present, must be separated from the rest of the
>>     commit message by one or more blank lines (lines that contain only
>>     whitespace).  There must be no blank lines within the list of
>>     trailers.  It is allowed to have blank lines after the trailers.
>>
>>     Each trailer line must match the following Perl regular
>>     expression:
>>
>>         ^([A-Za-z0-9_-]+)\s*:\s*(.*[^\s])\s*$
>>
>>     The string matching the first group is called the key and the string
>>     matching the second is called the value.  Keys are considered to be
>>     case-insensitive [or should they be case-sensitive?].  The
>>     interpretation of values is left entirely up to the application.
>>     Values must not be empty.
> 
> I tried to be clearer in the v12 I just posted, and I think it should
> be enough to be very clear. We might want to tweak a little the
> specifications later, so being too strict might be counter productive.
> 
> And as other tools might already use trailers in a case-sensitive way
> and yet other tools in a case-insensitive way, I am not sure we would
> gain anything by specifying if keys or values should be interpreted in
> a case-sensitive or case-insensitive way. On the contrary we might
> upset people already using some of these tools for no good reason.

No sane tool would interpret a trailer differently depending on how the key is capitalized. So I think that if we ourselves treat the keys case-insensitively but we preserve case when processing metadata, there won't be any problems.

Show 7 quoted lines
>>     However, in --draft and --cleanup modes, empty values *are*
>>     allowed, as are comments (lines starting with `core.commentchar`)
>>     within the trailer block.  In --draft mode such lines are passed
>>     through unchanged, and in --cleanup mode such lines are removed.
> 
> I am not sure we should use modes. I think options like
> "--trim-empty", "--allow-comments", "--allow-empty" might be clearer.

I think we want to train users to think of trailers-with-no-values as temporary helpers that shouldn't end up in commits, just like the "#"-commented lines in commit message templates. Why? Because you can't rely on their being preserved. As soon as your trailer goes through a "--cleanup" (which might be there because of an unrelated tool) it will disappear. For the user it is a simpler mental model to think of modes: as long as I am in draft mode, there might be comment lines and trailer templates in my commit message, but when I commit they will go away. I would go so far as to say that deleting trailer lines with no values should be a standard part of cleaning commit messages and should maybe be an option offered by "git stripspace".

Show 17 quoted lines
> [...]
>> So my preference would be to make the format of trailers really strict;
>> for example, only allowing colon separators as in the regexp above.
>> People who want to work with trailers using Git tools will just have to
>> conform to this format.
> 
> I don't think we should cast in stone the format for trailers, because
> of the main purpose of trailers.
> 
> The format of the commit header for example is cast in stone, but
> that's ok because it is mostly for Git internal use. Trailers are
> mostly for external use by users who already have tools expecting
> different formats.
> 
> There are already users who are not happy that they cannot easily have
> other commit headers, and we point them to trailers. If we specify
> trailers too strictly, where will we point them to?

Nobody is disagreeing that users should be allowed to choose their own keys and values and assign their own interpretations to them. If we let users add their own commit headers, we certainly wouldn't let them define a header formatted like "Fix #42", would we?

Again, the *only* justification for more flexible formats would be if there are a lot of tools that *already* exist out there that don't conform to "Key: value". Are there? New tools will certainly be written to use whatever format we define, and in exchange they will be able to use your awesome new tool and hopefully other upcoming tools for dealing with trailers.

>> But if we must support flexibility in the separator characters, then I
>> think it is important that we be able to read whatever we can write.
> 
> An option like --input-separator might be enough to support this.

This would not be adequate because a single commit message might have multiple trailers with different formats:

    Signed-off-by: me
    Fix #42

Either the input separator would have to be specified for every single trailer (which is impractical because you can't dictate centrally how git clients are configured) or the parsing code would have to be taught to read any allowed format.

Show 6 quoted lines
>> For me this means:
>>
>> * Enumerating a list of allowed separators (e.g., [:=#])
> 
> Junio suggested in a message that users might use different separators
> like '%'.

The fewer and stricter the better, to avoid false positive matches to non-trailer text.

Show 5 quoted lines
>> * Specifying how it is decided what separator to use when generating
>>   new trailers
> 
> This is already possible with the 'trailer.<token>.key' config
> variable.

This means that if one developer has forgotten to configure the "Fix" trailer in one clone, then he will generate trailers that are considered malformed by a colleague who has configured "Fix #".

Show 9 quoted lines
>> * Specifying what appends when a trailer is read and then written again:
>>   is its separator preserved, or is the trailer converted to use the
>>   separator configured for that particular key in the config.  And if
>>   the latter, what happens if a key's separator is not configured?
> 
> Right now we only accept ':' as input separator for the messages and
> ':' and '=' for the --trailer option, and the default output separator
> is ':'. If the user specify a different separator in a key, this
> separator will be used only in the output for this key.
I consider trailers that can only be written but not read to be broken.
 My questions are to probe one possible alternate reality, namely:
"Suppose we want to allow alternative trailer formats.  What would it
take to make them readable *and* writable?"
For example:
* What if I try to add a Signed-off-by trailer to a message that already
contains "Fix #42"?  In what form should the "Fix #42" line be written
to the output
  * if I don't have the separator for "Fix" configured?
  * if I have the separator for "Fix" configured to be "="?
* What if I *do* have the separator for "Fix" set to " #", but the input
contains "Fix: 42"?  How should that line be formatted on output?  Does
the answer change if I have "token.fix.ifexist" set to "overwrite" and
run "git interpret-trailers fix=43"?

By asking these questions, I hope to hint that supporting alternative formats increases the complexity of the specification and the implementation to an unjustifiable extent and/or it unrealistically relies all developers on a project having their trailer configuration set up correctly.

> If this is not clear in the documentation, please susggest specific
> improvements.
> [...]
> I tried to be very clear in the doc in v12.

The doc only covers writing new trailers in "alternative" formats, not reading them.

Show 9 quoted lines
> [...]
>> Given Git's requirements for backwards compatibility, a specification
>> that we release now will have to be supported forever (because it will
>> be baked into commits and can *never* be changed), and any
>> trailer-handling tools that we release now will have to be supported for
>> years (until at least Git 3.0).
> 
> Yeah, I know that. So if we are too strict in the specification will
> be stuck for a long time.

No, that's exactly backwards! We can easily loosen the format in the future. Projects don't need to output trailers in the looser format until everybody involved is using the new Git version.

But the format can never be made *more* strict, because then the metadata that users have added to their history will stop being readable.

Show 14 quoted lines
>> All in all, I think that there has been a lot of discussion about the
>> interface of this one command, "git interpret-trailers", including its
>> quite complicated configuration and a command-line behavior.  And yet it
>> seems to me that not many Git developers have been very engaged in the
>> conversation, and Junio (who has) still doesn't seem satisfied with it.
>>  I (though among the too-little engaged) have the feeling that it is
>> still a ways from maturity.
> 
> My opinion is that many Git developers have been engaged and you can
> see that in the Cc.
> 
> I cannot tell if they are all very happy or not but I suppose that if
> they were very unhappy they would tell it.
> [...]

It was unfair of me to try to characterize the opinions of other developers. On the other hand, even though many people have commented on this proposal over its long lifetime, I didn't get the feeling that it has won a consensus of +1s in its current form.

I'd love to hear the opinion of others because maybe I'm totally out in left field.

And I want to reiterate that the reason I'm so emphatic about this proposal is because I think it will be such a great new feature. I just think that some tweaks would make it a more solid foundation for building even more functionality onto.

Michael
-- 
Michael Haggerty
mhagger@alum.mit.edu
Johan Herland· May 27, 2014, 09:17 UTC · re: Michael Haggerty · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

On Tue, May 27, 2014 at 10:21 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
Show 7 quoted lines
> tl;dr: This patch series wants to introduce a permanent new Git data
> format.  The current version can write trailers in formats that it is
> incapable of reading, which I consider broken.  I advocate a stricter
> specification of the format of trailers, at least until we get feedback
> from users that they need more flexibility.
>
> On 05/25/2014 10:37 AM, Christian Couder wrote:
[...]
Show 14 quoted lines
>> My opinion is that many Git developers have been engaged and you can
>> see that in the Cc.
>>
>> I cannot tell if they are all very happy or not but I suppose that if
>> they were very unhappy they would tell it.
>> [...]
>
> It was unfair of me to try to characterize the opinions of other
> developers.  On the other hand, even though many people have commented
> on this proposal over its long lifetime, I didn't get the feeling that
> it has won a consensus of +1s in its current form.
>
> I'd love to hear the opinion of others because maybe I'm totally out in
> left field.

FWIW, after a quick read, I find myself agreeing very much with Michael's arguments for a stricter format (at least in its initial version).

We are formalizing and applying tools/automation to a part of the commit message that has so far been ad hoc and very informal. There is no expectation that _every_ _single_ existing use of (informal) trailers (except the somewhat-formalized support for --signoff) must be supported by git-interpret-trailers.

However, there _is_ an expectation that git-interpret-trailers is self-consistent and does not stumble over its own trailers. Therefore, it makes perfect sense to make v1 very strict in what formats it produce (i.e. a strict "key: value" format is enough for now).

> And I want to reiterate that the reason I'm so emphatic about this
> proposal is because I think it will be such a great new feature.  I just
> think that some tweaks would make it a more solid foundation for
> building even more functionality onto.

Fully agreed. git-interpret-trailers have come up in several other discussion, both on the git mailing list and elsewhere, and I have no doubt that this will be a very useful feature that will be put to good use in many projects.

...Johan
-- 
Johan Herland, <johan@herland.net>
www.herland.net
Junio C Hamano· May 27, 2014, 19:18 UTC · re: Christian Couder · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

Christian Couder <chriscool@tuxfamily.org> writes:
Show 10 quoted lines
> From: Michael Haggerty <mhagger@alum.mit.edu>
> ...
> An option like --input-separator might be enough to support this.
>
>> For me this means:
>> 
>> * Enumerating a list of allowed separators (e.g., [:=#])
>
> Junio suggested in a message that users might use different separators
> like '%'.

I actually think we shouldn't go any fancier than ":" and nothing else, not even "#".

I was hoping that you would eventually realize that there are only two viable extremes when I suggested "the users may want to use other random characters like '%'" and also "the users can specify the 'key' with colon and trailing SP" (in $gmane/245960).

 - If you want to give the projects greater control of the format,
   then you cannot rely on "separators" anyway.  Your users can list
   all possible footer "keys" the particular project would use, so
   that they are recognized by Git, be that "Fixes: 4a28f16", "Bug
   #12354", without hard-coding what "separator" Git must pay
   attention to.  You can easily find a run of lines that begin with
   any of the "key" (e.g. "Fixes: ", "Signed-off-by: ", "Bug #",
   ...) starting from the tail-end of the log message and that is
   your footer block.  No need for "separators" at all.
 - If you want to give the projects freedom to come up with random
   new kinds of footers without pre-arrangement, then you need to
   have a reliable way to say if any line you have never seen could
   be a footer material.  A colon has been used everywhere, and used
   even in the "Fixes: 4a28f16" example you took from the kernel
   circle.  I think you presented it with '#' but I do not think
   they even want that, looking at:
   http://lists.linuxfoundation.org/pipermail/ksummit-discuss/2014-May/000618.html

I also think that bug tracking system using "Bug #12345" is an unrelated issue, as log viewers would want to highlight and make links out of them anywhere in the log message text, not limited to the log footer part.

As to which one of these two we should take, I tend to think that we should start small and limited; loosening the syntax later is much easier than going the other way, i.e. ":" and nothing else.

Junio C Hamano· Apr 8, 2014, 16:52 UTC · re: Michael Haggerty · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

Michael Haggerty <mhagger@alum.mit.edu> writes:
> Sorry for reappearing in this thread after such a long absence.  I
> wanted to see what is coming up (I think this interpret-trailers command
> will be handy!) so I read this documentation patch carefully, and added
> some questions and suggestions below.

Thanks for reading the patch carefully. It helps to have fresh set of eyes that are not contaminated by the preconception formed by previous discussions, especially when reviewing the documentation whose primary target audiences are those who do not care about these previous back-and-forth.

Show 18 quoted lines
>> +trailer.<token>.where::
>> +	This can be either `after`, which is the default, or
>> +	`before`. If it is `before`, then a trailer with the specified
>> +	token, will appear before, instead of after, other trailers
>> +	with the same token, or otherwise at the beginning, instead of
>> +	at the end, of all the trailers.
>
> Brainstorming: some other options that might make sense here someday:
> ...
>> +trailer.<token>.ifexist::
>> +	This option makes it possible to choose what action will be
>> +	performed when there is already at least one trailer with the
>> +	same token in the message.
>> ++
>> +The valid values for this option are: `addIfDifferent` (this is the
>> +default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.
>
> Are these option values case sensitive?

It is interesting and somewhat sad that it all has to come back together inter-twined. From the very beginning, I was opposed to having logical complexity that requires multi-words in both variable names (e.g. "if-exist") and values (e.g. "add-if-different"), and after $gmane/241929 where I let the devil's advocate "how about making the variable simpler without logical operation and put all the conditional on the value side?" suggestion shot down, I somehow was hoping that the value part got a lot simpler not to require multi-words, which would have meant that we would not have to worry about "Is it addIfDifferent? add-if-different? or Add_If_Different?" at all. Sadly that is not what we have ended up with.

So, with that realization...
Show 5 quoted lines
> If so, it might be a little bit
> confusing because the same camel-case is often used in documentation for
> configuration *keys*, which are not case sensitive [1], and users might
> have gotten used to thinking of strings that look like this to be
> non-case-sensitive.

... very true. Having to have these enum values as so complex to require multi-words is probably the root cause of the confusion, and we might probably be better off if we did not have to, but it would be helpful to allow various different spellings (i.e. make them case insensitive to allow random camel spellings, and also accept things like "add-if-different" as well) if we absolutely have to have these complex values.

But you had a lot of good questions and suggestions for possible future enhancements that we would need to take into account while designing the overall scheme to later allow them to fit into. Maybe a value that is a single-token that consists of just a few words (e.g. "addIfDifferent") may not be the best way to go after all.

I dunno.
> What if there are multiple existing trailers with the same token?  Are
> they all overwritten?
> ...
> What if the key appears multiple times in existing trailers?
All good questions, I would think.
Junio C Hamano· Apr 8, 2014, 21:26 UTC · re: Christian Couder · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

Christian Couder <chriscool@tuxfamily.org> writes:
> +Help add RFC 822-like headers, called 'trailers', at the end of the
> +otherwise free-form part of a commit message.

I think it is somewhat misleading to use the word "headers" like that. 'trailers' look similar to RFC-822-headers but they come at the end. The sentence however reads as if they are "headers" that look like RFC 822. Perhaps shuffling words like so:

	Help adding 'trailers' lines, that look similar to RFC 822
	e-mail headers, at the end of the ...
would make it less confusing.
Show 14 quoted lines
> +Some configuration variables control the way the `token` arguments are
> +applied to the message and the way any existing trailer in the message
> +is changed. They also make it possible to automatically add some
> +trailers.
> +
> +By default, a 'token=value' or 'token:value' argument will be added
> +only if no trailer with the same (token, value) pair is already in the
> +message. The 'token' and 'value' parts will be trimmed to remove
> +starting and trailing whitespace, and the resulting trimmed 'token'
> +and 'value' will appear in the message like this:
> +
> +------------------------------------------------
> +token: value
> +------------------------------------------------

Mental note: this does assume that the final output for the 'token' is to have a line <label> that is followed by a colon ":", SP and the value.

And the natural way to express that on the command line would be to say "token: value", I would think, but let's just read on.

> +Note that 'trailers' do not follow and are not intended to follow many
> +rules that are in RFC 822. For example they do not follow the line
> +breaking rules, the encoding rules and probably many other rules.

s/that are in RFC 822/for RFC 822 headers/. s/line breaking/line folding/. (see RFC 822, 3.1.1)

Show 11 quoted lines
> +OPTIONS
> +-------
> +--trim-empty::
> +	If the 'value' part of any trailer contains only whitespace,
> +	the whole trailer will be removed from the resulting message.
> +
> +CONFIGURATION VARIABLES
> +-----------------------
> +
> +trailer.<token>.key::
> +	This 'key' will be used instead of 'token' in the

As `key` is something that is typed literally, it should be typeset as `key` in the descriptive text. I think other manpages spell the placeholder as `<token>` (or '<token>', I am not sure which...).

> +	trailer. After some alphanumeric characters, it can contain
> +	some non alphanumeric characters like ':', '=' or '#' that will
> +	be used instead of ':' to separate the token from the value in
> +	the trailer, though the default ':' is more standard.
I assume that this is for things like
	bug #538
and the configuration would say something like:
	[trailer "bug"]
        	key = "bug #"

For completeness (of this example), the bog-standard s-o-b would look like

	Signed-off-by: Christian Couder <chriscool@tuxfamily.org>

and the configuration for it that spell the redundant "key" would be:

	[trailer "Signed-off-by"]
        	key = "Signed-off-by: "
Am I reading the intention correctly?

That is, when trailer.<token>.key is not defined, the value defaults to "<token>: " (with one SP after the label and colon), and when it is defined, the value can come directly after it.

Christian Couder· Apr 25, 2014, 19:56 UTC · re: Junio C Hamano · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

From: Junio C Hamano <gitster@pobox.com>
Show 15 quoted lines
>
> Christian Couder <chriscool@tuxfamily.org> writes:
> 
>> +Help add RFC 822-like headers, called 'trailers', at the end of the
>> +otherwise free-form part of a commit message.
> 
> I think it is somewhat misleading to use the word "headers" like
> that.  'trailers' look similar to RFC-822-headers but they come at
> the end.  The sentence however reads as if they are "headers" that
> look like RFC 822.  Perhaps shuffling words like so:
> 
> 	Help adding 'trailers' lines, that look similar to RFC 822
> 	e-mail headers, at the end of the ...
> 
> would make it less confusing.
Ok, I made this change in v11.
Show 28 quoted lines
>> +Some configuration variables control the way the `token` arguments are
>> +applied to the message and the way any existing trailer in the message
>> +is changed. They also make it possible to automatically add some
>> +trailers.
>> +
>> +By default, a 'token=value' or 'token:value' argument will be added
>> +only if no trailer with the same (token, value) pair is already in the
>> +message. The 'token' and 'value' parts will be trimmed to remove
>> +starting and trailing whitespace, and the resulting trimmed 'token'
>> +and 'value' will appear in the message like this:
>> +
>> +------------------------------------------------
>> +token: value
>> +------------------------------------------------
> 
> Mental note: this does assume that the final output for the 'token'
> is to have a line <label> that is followed by a colon ":", SP and
> the value.
> 
> And the natural way to express that on the command line would be to
> say "token: value", I would think, but let's just read on.
> 
>> +Note that 'trailers' do not follow and are not intended to follow many
>> +rules that are in RFC 822. For example they do not follow the line
>> +breaking rules, the encoding rules and probably many other rules.
> 
> s/that are in RFC 822/for RFC 822 headers/.
> s/line breaking/line folding/. (see RFC 822, 3.1.1)
Ok, it's in v11 too.
Show 14 quoted lines
>> +OPTIONS
>> +-------
>> +--trim-empty::
>> +	If the 'value' part of any trailer contains only whitespace,
>> +	the whole trailer will be removed from the resulting message.
>> +
>> +CONFIGURATION VARIABLES
>> +-----------------------
>> +
>> +trailer.<token>.key::
>> +	This 'key' will be used instead of 'token' in the
> 
> As `key` is something that is typed literally, it should be typeset
> as `key` in the descriptive text.
Ok, I used `key` in v11.
> I think other manpages spell the
> placeholder as `<token>` (or '<token>', I am not sure which...).
I found mostly <token>, so I used that in v11.
Show 24 quoted lines
>> +	trailer. After some alphanumeric characters, it can contain
>> +	some non alphanumeric characters like ':', '=' or '#' that will
>> +	be used instead of ':' to separate the token from the value in
>> +	the trailer, though the default ':' is more standard.
> 
> I assume that this is for things like
> 
> 	bug #538
> 
> and the configuration would say something like:
> 
> 	[trailer "bug"]
>         	key = "bug #"
> 
> For completeness (of this example), the bog-standard s-o-b would
> look like
> 
> 	Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> 
> and the configuration for it that spell the redundant "key" would
> be:
> 
> 	[trailer "Signed-off-by"]
>         	key = "Signed-off-by: "
Yeah, but you can use the following instead:
 	[trailer "s-o-b"]
         	key = "Signed-off-by: "
The <token> and the key can be different.
> Am I reading the intention correctly?
Yeah, I think so.
> That is, when trailer.<token>.key is not defined, the value defaults
> to "<token>: " (with one SP after the label and colon),
Yes.
> and when it
> is defined, the value can come directly after it.
The value can come directly after the key, only if the key ends with '#'.

If it ends with something else, except spaces, one SP will be added between the key and the value.

Yeah, I made '#' special in the hope that it would be more compatible with GitHub and other services that might also use '#'.

Thanks, Christian.

Junio C Hamano· Apr 28, 2014, 16:37 UTC · re: Christian Couder · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

Christian Couder <chriscool@tuxfamily.org> writes:
Show 34 quoted lines
> From: Junio C Hamano <gitster@pobox.com>
>>
>> Christian Couder <chriscool@tuxfamily.org> writes:
>> ...
>
>>> +	trailer. After some alphanumeric characters, it can contain
>>> +	some non alphanumeric characters like ':', '=' or '#' that will
>>> +	be used instead of ':' to separate the token from the value in
>>> +	the trailer, though the default ':' is more standard.
>> 
>> I assume that this is for things like
>> 
>> 	bug #538
>> 
>> and the configuration would say something like:
>> 
>> 	[trailer "bug"]
>>         	key = "bug #"
>> 
>> For completeness (of this example), the bog-standard s-o-b would
>> look like
>> 
>> 	Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
>> 
>> and the configuration for it that spell the redundant "key" would
>> be:
>> 
>> 	[trailer "Signed-off-by"]
>>         	key = "Signed-off-by: "
>
> Yeah, but you can use the following instead:
>
>  	[trailer "s-o-b"]
>          	key = "Signed-off-by: "

Sure, but note that both of these have a SP at the end in the value part (which I think is a sensible thing to do).

Show 18 quoted lines
> The <token> and the key can be different.
>
>> Am I reading the intention correctly?
>
> Yeah, I think so.
>
>> That is, when trailer.<token>.key is not defined, the value defaults
>> to "<token>: " (with one SP after the label and colon),
>
> Yes.
>
>> and when it
>> is defined, the value can come directly after it.
>
> The value can come directly after the key, only if the key ends with '#'.
>
> If it ends with something else, except spaces, one SP will be added
> between the key and the value.

And I do not think we want (or even need) this "only when it ends with #" special casing in the code at all. When the project's convention is to say "frotz# value-of-frotz", the users will specify that with 'key = "frotz# "' (with a trailing SP in the value part), and in a project that wants 'nitfol %value-of-nitfol', your parser will find 'key = "nitfol %"'. The users will obtain the result they want for either case, and a hard-coded special casing in the code that only has incomplete knowledge on the project convention will actively harm them. I'd suggest dropping that special case.

Jeremy Morton· Apr 29, 2014, 11:05 UTC · re: Junio C Hamano · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

On 28/04/2014 17:37, Junio C Hamano wrote:
Show 36 quoted lines
> Christian Couder<chriscool@tuxfamily.org>  writes:
>
>> From: Junio C Hamano<gitster@pobox.com>
>>>
>>> Christian Couder<chriscool@tuxfamily.org>  writes:
>>> ...
>>
>>>> +	trailer. After some alphanumeric characters, it can contain
>>>> +	some non alphanumeric characters like ':', '=' or '#' that will
>>>> +	be used instead of ':' to separate the token from the value in
>>>> +	the trailer, though the default ':' is more standard.
>>>
>>> I assume that this is for things like
>>>
>>> 	bug #538
>>>
>>> and the configuration would say something like:
>>>
>>> 	[trailer "bug"]
>>>          	key = "bug #"
>>>
>>> For completeness (of this example), the bog-standard s-o-b would
>>> look like
>>>
>>> 	Signed-off-by: Christian Couder<chriscool@tuxfamily.org>
>>>
>>> and the configuration for it that spell the redundant "key" would
>>> be:
>>>
>>> 	[trailer "Signed-off-by"]
>>>          	key = "Signed-off-by: "
>>
>> Yeah, but you can use the following instead:
>>
>>   	[trailer "s-o-b"]
>>           	key = "Signed-off-by: "

One thing I'm not quite understanding is where the "Christian Couder<chriscool@tuxfamily.org>" bit comes from. So you've defined the trailer token and key, but interpret-trailers then needs to get the value it will give for the key from somewhere. Does it have to just be hardcoded in? We probably want some way to get various variables like current branch name, current git version, etc. So in the case of always adding a trailer for the branch that the commit was checked in to at the time (Developed-on, Made-on-branch, Author-branch, etc. [I think my favourite is Made-on-branch]), you'd want something like:

	[trailer "m-o-b"]
		key = "Made-on-branch: "
		value = "$currentBranch"
... resulting in the trailer (for example):
	Made-on-branch: pacman-minigame
Also, if there were no current branch name because you're committing in 
a detached head state, it would be nice if you could have some logic to 
determine that, and instead write the trailer as:
	Made-on-branch: (detached HEAD: AB12CD34)

... or whatever. And also how about some logic to be able to say that if you're committing to the "master" branch, the trailer doesn't get inserted at all?

-- 
Best regards,
Jeremy Morton (Jez)
Christian Couder· Apr 29, 2014, 11:47 UTC · re: Jeremy Morton · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

On Tue, Apr 29, 2014 at 1:05 PM, Jeremy Morton <admin@game-point.net> wrote:
Show 60 quoted lines
> On 28/04/2014 17:37, Junio C Hamano wrote:
>>
>> Christian Couder<chriscool@tuxfamily.org>  writes:
>>
>>> From: Junio C Hamano<gitster@pobox.com>
>>>>
>>>>
>>>> Christian Couder<chriscool@tuxfamily.org>  writes:
>>>> ...
>>>
>>>
>>>>> +       trailer. After some alphanumeric characters, it can contain
>>>>> +       some non alphanumeric characters like ':', '=' or '#' that will
>>>>> +       be used instead of ':' to separate the token from the value in
>>>>> +       the trailer, though the default ':' is more standard.
>>>>
>>>>
>>>> I assume that this is for things like
>>>>
>>>>         bug #538
>>>>
>>>> and the configuration would say something like:
>>>>
>>>>         [trailer "bug"]
>>>>                 key = "bug #"
>>>>
>>>> For completeness (of this example), the bog-standard s-o-b would
>>>> look like
>>>>
>>>>         Signed-off-by: Christian Couder<chriscool@tuxfamily.org>
>>>>
>>>> and the configuration for it that spell the redundant "key" would
>>>> be:
>>>>
>>>>         [trailer "Signed-off-by"]
>>>>                 key = "Signed-off-by: "
>>>
>>>
>>> Yeah, but you can use the following instead:
>>>
>>>         [trailer "s-o-b"]
>>>                 key = "Signed-off-by: "
>
>
> One thing I'm not quite understanding is where the "Christian
> Couder<chriscool@tuxfamily.org>" bit comes from.  So you've defined the
> trailer token and key, but interpret-trailers then needs to get the value it
> will give for the key from somewhere.  Does it have to just be hardcoded in?
> We probably want some way to get various variables like current branch name,
> current git version, etc.  So in the case of always adding a trailer for the
> branch that the commit was checked in to at the time (Developed-on,
> Made-on-branch, Author-branch, etc. [I think my favourite is
> Made-on-branch]), you'd want something like:
>
>         [trailer "m-o-b"]
>                 key = "Made-on-branch: "
>                 value = "$currentBranch"
>
> ... resulting in the trailer (for example):
>         Made-on-branch: pacman-minigame
In the documentation patch, there is:
trailer.<token>.command::
       This option can be used to specify a shell command that will
       be used to automatically add or modify a trailer with the
       specified 'token'.
       When this option is specified, it is like if a special 'token=value'
       argument is added at the end of the command line, where 'value' will
       be given by the standard output of the specified command.
       If the command contains the `$ARG` string, this string will be
       replaced with the 'value' part of an existing trailer with the same
       token, if any, before the command is launched.

That's why Something like the following should work if "git commit" automitically runs "git interpret-trailers":

         [trailer "m-o-b"]
                 key = "Made-on-branch: "
                 command = "git name-rev --name-only HEAD"
> Also, if there were no current branch name because you're committing in a
> detached head state, it would be nice if you could have some logic to
> determine that, and instead write the trailer as:
>         Made-on-branch: (detached HEAD: AB12CD34)

You may need to write a small script for that. Then you just need the "trailer.m-o-b.command" config value to point to your script.

> ... or whatever.  And also how about some logic to be able to say that if
> you're committing to the "master" branch, the trailer doesn't get inserted
> at all?
You can script that too.

Best, Christian.

Jeremy Morton· Apr 29, 2014, 13:25 UTC · re: Christian Couder · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

On 29/04/2014 12:47, Christian Couder wrote:
Show 14 quoted lines
>> Also, if there were no current branch name because you're committing in a
>> detached head state, it would be nice if you could have some logic to
>> determine that, and instead write the trailer as:
>>          Made-on-branch: (detached HEAD: AB12CD34)
>
> You may need to write a small script for that.
> Then you just need the "trailer.m-o-b.command" config value to point
> to your script.
>
>> ... or whatever.  And also how about some logic to be able to say that if
>> you're committing to the "master" branch, the trailer doesn't get inserted
>> at all?
>
> You can script that too.

But it would be nicer if the logic were built-in, then you wouldn't have to share some script with your work colleagues. :-)

-- 
Best regards,
Jeremy Morton (Jez)
Christian Couder· May 1, 2014, 18:54 UTC · re: Jeremy Morton · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

From: Jeremy Morton <admin@game-point.net>
Show 21 quoted lines
> On 29/04/2014 12:47, Christian Couder wrote:
>>> Also, if there were no current branch name because you're committing
>>> in a
>>> detached head state, it would be nice if you could have some logic to
>>> determine that, and instead write the trailer as:
>>>          Made-on-branch: (detached HEAD: AB12CD34)
>>
>> You may need to write a small script for that.
>> Then you just need the "trailer.m-o-b.command" config value to point
>> to your script.
>>
>>> ... or whatever.  And also how about some logic to be able to say that
>>> if
>>> you're committing to the "master" branch, the trailer doesn't get
>>> inserted
>>> at all?
>>
>> You can script that too.
> 
> But it would be nicer if the logic were built-in, then you wouldn't
> have to share some script with your work colleagues. :-)

The above logic is very specific to your workflow. For example some people might a "Made-on-branch: " trailer only when they are on real branches except "dev" and "master".

Best, Christian.

Jeremy Morton· Apr 29, 2014, 13:25 UTC · re: Christian Couder · lore

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

On 29/04/2014 12:47, Christian Couder wrote:
Show 14 quoted lines
>> Also, if there were no current branch name because you're committing in a
>> detached head state, it would be nice if you could have some logic to
>> determine that, and instead write the trailer as:
>>          Made-on-branch: (detached HEAD: AB12CD34)
>
> You may need to write a small script for that.
> Then you just need the "trailer.m-o-b.command" config value to point
> to your script.
>
>> ... or whatever.  And also how about some logic to be able to say that if
>> you're committing to the "master" branch, the trailer doesn't get inserted
>> at all?
>
> You can script that too.

But it would be nicer if the logic were built-in, then you wouldn't have to share some script with your work colleagues. :-)

-- 
Best regards,
Jeremy Morton (Jez)
Christian Couder· Apr 6, 2014, 17:02 UTC · re: Christian Couder · lore

[PATCH v10 12/12] trailer: add blank line before the trailers if needed

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/t7513-interpret-trailers.sh | 12 +++++++++++-
 trailer.c                     | 26 ++++++++++++++++++--------
 2 files changed, 29 insertions(+), 9 deletions(-)
Show changes to 2 files +29 −9

t/t7513-interpret-trailers.sh, trailer.c

diff --git a/t/t7513-interpret-trailers.sh b/t/t7513-interpret-trailers.sh
index 262f7bf..44a7131 100755
--- a/t/t7513-interpret-trailers.sh
+++ b/t/t7513-interpret-trailers.sh
@@ -34,6 +34,7 @@ test_expect_success 'setup' '
 
 test_expect_success 'without config' '
 	sed -e "s/ Z\$/ /" >expected <<-\EOF &&
+
 		ack: Peff
 		Reviewed-by: Z
 		Acked-by: Johan
@@ -44,6 +45,7 @@ test_expect_success 'without config' '
 
 test_expect_success '--trim-empty without config' '
 	cat >expected <<-\EOF &&
+
 		ack: Peff
 		Acked-by: Johan
 	EOF
@@ -55,6 +57,7 @@ test_expect_success '--trim-empty without config' '
 test_expect_success 'with config setup' '
 	git config trailer.ack.key "Acked-by: " &&
 	cat >expected <<-\EOF &&
+
 		Acked-by: Peff
 	EOF
 	git interpret-trailers --trim-empty "ack = Peff" >actual &&
@@ -68,6 +71,7 @@ test_expect_success 'with config setup' '
 test_expect_success 'with config setup and = sign' '
 	git config trailer.ack.key "Acked-by= " &&
 	cat >expected <<-\EOF &&
+
 		Acked-by= Peff
 	EOF
 	git interpret-trailers --trim-empty "ack = Peff" >actual &&
@@ -81,6 +85,7 @@ test_expect_success 'with config setup and = sign' '
 test_expect_success 'with config setup and # sign' '
 	git config trailer.bug.key "Bug #" &&
 	cat >expected <<-\EOF &&
+
 		Bug #42
 	EOF
 	git interpret-trailers --trim-empty "bug = 42" >actual &&
@@ -88,8 +93,10 @@ test_expect_success 'with config setup and # sign' '
 '
 
 test_expect_success 'with commit basic message' '
+	cat basic_message >expected &&
+	echo >>expected &&
 	git interpret-trailers <basic_message >actual &&
-	test_cmp basic_message actual
+	test_cmp expected actual
 '
 
 test_expect_success 'with commit complex message' '
@@ -436,6 +443,7 @@ test_expect_success 'with failing command using $ARG' '
 
 test_expect_success 'with empty tokens' '
 	cat >expected <<-EOF &&
+
 		Signed-off-by: A U Thor <author@example.com>
 	EOF
 	git interpret-trailers ":" ":test" >actual <<-EOF &&
@@ -446,6 +454,7 @@ test_expect_success 'with empty tokens' '
 test_expect_success 'with command but no key' '
 	git config --unset trailer.sign.key &&
 	cat >expected <<-EOF &&
+
 		sign: A U Thor <author@example.com>
 	EOF
 	git interpret-trailers >actual <<-EOF &&
@@ -456,6 +465,7 @@ test_expect_success 'with command but no key' '
 test_expect_success 'with no command and no key' '
 	git config --unset trailer.review.key &&
 	cat >expected <<-EOF &&
+
 		review: Junio
 		sign: A U Thor <author@example.com>
 	EOF
diff --git a/trailer.c b/trailer.c
index 09db2c2..639f657 100644
--- a/trailer.c
+++ b/trailer.c
@@ -618,12 +618,14 @@ static struct strbuf **read_stdin(void)
 }
 
 /*
- * Return the the (0 based) index of the first trailer line
+ * Return the (0 based) index of the first trailer line
  * or the line count if there are no trailers.
+ * The has_blank_line parameter tells if there is a blank
+ * line before the trailers.
  */
-static int find_trailer_start(struct strbuf **lines)
+static int find_trailer_start(struct strbuf **lines, int *has_blank_line)
 {
-	int start, empty = 1, count = 0;
+	int start, only_spaces = 1, count = 0;
 
 	/* Get the line count */
 	while (lines[count])
@@ -635,32 +637,40 @@ static int find_trailer_start(struct strbuf **lines)
 	 */
 	for (start = count - 1; start >= 0; start--) {
 		if (contains_only_spaces(lines[start]->buf)) {
-			if (empty)
+			if (only_spaces)
 				continue;
+			*has_blank_line = 1;
 			return start + 1;
 		}
 		if (strchr(lines[start]->buf, ':')) {
-			if (empty)
-				empty = 0;
+			if (only_spaces)
+				only_spaces = 0;
 			continue;
 		}
+		*has_blank_line = start == count - 1 ?
+		  0 : contains_only_spaces(lines[start + 1]->buf);
 		return count;
 	}
 
-	return empty ? count : start + 1;
+	*has_blank_line = only_spaces ? count > 0 : 0;
+	return only_spaces ? count : start + 1;
 }
 
 static void process_stdin(struct trailer_item **in_tok_first,
 			  struct trailer_item **in_tok_last)
 {
 	struct strbuf **lines = read_stdin();
-	int start = find_trailer_start(lines);
+	int has_blank_line;
+	int start = find_trailer_start(lines, &has_blank_line);
 	int i;
 
 	/* Print non trailer lines as is */
 	for (i = 0; lines[i] && i < start; i++)
 		printf("%s", lines[i]->buf);
 
+	if (!has_blank_line)
+		printf("\n");
+
 	/* Parse trailer lines */
 	for (i = start; lines[i]; i++) {
 		struct trailer_item *new = create_trailer_item(lines[i]->buf);
-- 
1.9.0.163.g8ca203c
Junio C Hamano· Apr 7, 2014, 21:38 UTC · re: Christian Couder · lore

Re: [PATCH v10 12/12] trailer: add blank line before the trailers if needed

Christian Couder <chriscool@tuxfamily.org> writes:
> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> ---

Hmph, this is more fixing a mistake made earlier in the series at the end than adding a new feature or something. Can you start from a version that does not have the mistake from the beginning?

Christian Couder· Apr 8, 2014, 12:48 UTC · re: Junio C Hamano · lore

Re: [PATCH v10 12/12] trailer: add blank line before the trailers if needed

On Mon, Apr 7, 2014 at 11:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> Hmph, this is more fixing a mistake made earlier in the series at
> the end than adding a new feature or something.  Can you start from
> a version that does not have the mistake from the beginning?
Ok, I will squash this patch in other previous patches.

Thanks, Christian.

← back to recent threads