git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 5/8] shortlog: de-duplicate trailer values

From
Jeff King <peff@peff.net>
Date
Sep 27, 2020, 08:40 UTC
Message-ID
<20200927084007.GE2465761@coredump.intra.peff.net>
In-Reply-To
<20200927083933.GA2222823@coredump.intra.peff.net>

The current documentation is vague about what happens with --group=trailer:signed-off-by when we see a commit with:

  Signed-off-by: One
  Signed-off-by: Two
  Signed-off-by: One

We clearly should credit both "One" and "Two", but should "One" get credited twice? The current code does so, but mostly because that was the easiest thing to do. It's probably more useful to count each commit at most once. This will become especially important when we allow values from multiple sources in a future patch.

Signed-off-by: Jeff King <peff@peff.net>
---
 Documentation/git-shortlog.txt |  3 +-
 builtin/shortlog.c             | 58 ++++++++++++++++++++++++++++++++++
 t/t4201-shortlog.sh            | 28 ++++++++++++++++
 3 files changed, 88 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-shortlog.txt b/Documentation/git-shortlog.txt
index edd6cda58a..9e94613e13 100644
--- a/Documentation/git-shortlog.txt
+++ b/Documentation/git-shortlog.txt
@@ -61,7 +61,8 @@ OPTIONS
 +
 Note that commits that do not include the trailer will not be counted.
 Likewise, commits with multiple trailers (e.g., multiple signoffs) may
-be counted more than once.
+be counted more than once (but only once per unique trailer value in
+that commit).
 +
 The contents of each trailer value are taken literally and completely.
 No mailmap is applied, and the `-e` option has no effect (if the trailer
diff --git a/builtin/shortlog.c b/builtin/shortlog.c
index e1d9ee909f..d2d8103dd3 100644
--- a/builtin/shortlog.c
+++ b/builtin/shortlog.c
@@ -166,13 +166,68 @@ static void read_from_stdin(struct shortlog *log)
 	strbuf_release(&oneline);
 }
 
+struct strset_item {
+	struct hashmap_entry ent;
+	char value[FLEX_ARRAY];
+};
+
+struct strset {
+	struct hashmap map;
+};
+
+#define STRSET_INIT { { NULL } }
+
+static int strset_item_hashcmp(const void *hash_data,
+			       const struct hashmap_entry *entry,
+			       const struct hashmap_entry *entry_or_key,
+			       const void *keydata)
+{
+	const struct strset_item *a, *b;
+
+	a = container_of(entry, const struct strset_item, ent);
+	if (keydata)
+		return strcmp(a->value, keydata);
+
+	b = container_of(entry_or_key, const struct strset_item, ent);
+	return strcmp(a->value, b->value);
+}
+
+/*
+ * Adds "str" to the set if it was not already present; returns true if it was
+ * already there.
+ */
+static int strset_check_and_add(struct strset *ss, const char *str)
+{
+	unsigned int hash = strhash(str);
+	struct strset_item *item;
+
+	if (!ss->map.table)
+		hashmap_init(&ss->map, strset_item_hashcmp, NULL, 0);
+
+	if (hashmap_get_from_hash(&ss->map, hash, str))
+		return 1;
+
+	FLEX_ALLOC_STR(item, value, str);
+	hashmap_entry_init(&item->ent, hash);
+	hashmap_add(&ss->map, &item->ent);
+	return 0;
+}
+
+static void strset_clear(struct strset *ss)
+{
+	if (!ss->map.table)
+		return;
+	hashmap_free_entries(&ss->map, struct strset_item, ent);
+}
+
 static void insert_records_from_trailers(struct shortlog *log,
 					 struct commit *commit,
 					 struct pretty_print_context *ctx,
 					 const char *oneline)
 {
 	struct trailer_iterator iter;
 	const char *commit_buffer, *body;
+	struct strset dups = STRSET_INIT;
 
 	/*
 	 * Using format_commit_message("%B") would be simpler here, but
@@ -190,10 +245,13 @@ static void insert_records_from_trailers(struct shortlog *log,
 		if (strcasecmp(iter.key.buf, log->trailer))
 			continue;
 
+		if (strset_check_and_add(&dups, value))
+			continue;
 		insert_one_record(log, value, oneline);
 	}
 	trailer_iterator_release(&iter);
 
+	strset_clear(&dups);
 	unuse_commit_buffer(commit, commit_buffer);
 }
 
diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
index e97d891a71..83dbbc44e8 100755
--- a/t/t4201-shortlog.sh
+++ b/t/t4201-shortlog.sh
@@ -234,4 +234,32 @@ test_expect_success 'shortlog --group=trailer:signed-off-by' '
 	test_cmp expect actual
 '
 
+test_expect_success 'shortlog de-duplicates trailers in a single commit' '
+	git commit --allow-empty -F - <<-\EOF &&
+	subject one
+
+	this message has two distinct values, plus a repeat
+
+	Repeated-trailer: Foo
+	Repeated-trailer: Bar
+	Repeated-trailer: Foo
+	EOF
+
+	git commit --allow-empty -F - <<-\EOF &&
+	subject two
+
+	similar to the previous, but without the second distinct value
+
+	Repeated-trailer: Foo
+	Repeated-trailer: Foo
+	EOF
+
+	cat >expect <<-\EOF &&
+	     2	Foo
+	     1	Bar
+	EOF
+	git shortlog -ns --group=trailer:repeated-trailer -2 HEAD >actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.28.0.1127.ga65787d918
Previous: Junio C HamanoNext: Junio C Hamano
Message 33 of 44 in “parsing trailers with shortlog”
  1. 0/8 parsing trailers with shortlogJeff King, Sep 25, 2020
  2. 1/8 shortlog: change "author" variables to "ident"Jeff King, Sep 25, 2020
  3. 2/8 shortlog: refactor committer/author groupingJeff King, Sep 25, 2020
  4. Eric SunshineSep 25, 2020
  5. Jeff KingSep 27, 2020
  6. Jeff KingSep 27, 2020
  7. Eric SunshineSep 27, 2020
  8. Martin ÅgrenSep 26, 2020
  9. Jeff KingSep 27, 2020
  10. 3/8 trailer: add interface for iterating over commit trailersJeff King, Sep 25, 2020
  11. Martin ÅgrenSep 26, 2020
  12. Jeff KingSep 27, 2020
  13. 4/8 shortlog: match commit trailers with --groupJeff King, Sep 25, 2020
  14. 5/8 shortlog: de-duplicate trailer valuesJeff King, Sep 25, 2020
  15. 6/8 shortlog: rename parse_stdin_ident()Jeff King, Sep 25, 2020
  16. 7/8 shortlog: parse trailer identsJeff King, Sep 25, 2020
  17. 8/8 shortlog: allow multiple groups to be specifiedJeff King, Sep 25, 2020
  18. Eric SunshineSep 25, 2020
  19. Jeff KingSep 27, 2020
  20. Martin ÅgrenSep 26, 2020
  21. Jeff KingSep 27, 2020
  22. Derrick StoleeSep 25, 2020
  23. Junio C HamanoSep 25, 2020
  24. 0/8 parsing trailers with shortlogJeff King, Sep 27, 2020
  25. 1/8 shortlog: change "author" variables to "ident"Jeff King, Sep 27, 2020
  26. Junio C HamanoSep 27, 2020
  27. 2/8 shortlog: add grouping optionJeff King, Sep 27, 2020
  28. 3/8 trailer: add interface for iterating over commit trailersJeff King, Sep 27, 2020
  29. 4/8 shortlog: match commit trailers with --groupJeff King, Sep 27, 2020
  30. Junio C HamanoSep 27, 2020
  31. Jeff KingSep 28, 2020
  32. Junio C HamanoSep 28, 2020
  33. 5/8 shortlog: de-duplicate trailer valuesJeff King, Sep 27, 2020
  34. Junio C HamanoSep 27, 2020
  35. Jeff KingSep 28, 2020
  36. 6/8 shortlog: rename parse_stdin_ident()Jeff King, Sep 27, 2020
  37. 7/8 shortlog: parse trailer identsJeff King, Sep 27, 2020
  38. Junio C HamanoSep 27, 2020
  39. 8/8 shortlog: allow multiple groups to be specifiedJeff King, Sep 27, 2020
  40. Junio C HamanoSep 27, 2020
  41. Jeff KingSep 28, 2020
  42. Junio C HamanoDec 28, 2020
  43. Junio C HamanoFeb 4, 2021
  44. Martin ÅgrenSep 27, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.