{"thread":{"id":"54816","subject":"[PATCH 0/1] Hashed mailmap support","startedAt":"2020-12-13T01:10:48Z","lastAt":"2020-12-18T05:57:29Z","messageCount":14,"participants":["brian m. carlson","Johannes Sixt","Junio C Hamano","Ævar Arnfjörð Bjarmason","Jeff King","Phillip Wood"],"isPatch":true,"patchVersion":1,"patchTotal":1},"messages":[{"id":"412127","messageId":"20201213010539.544101-1-sandals@crustytoothpaste.net","threadId":"54816","inReplyTo":null,"subject":"[PATCH 0/1] Hashed mailmap support","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-12-13T01:05:38Z","receivedAt":"2020-12-13T01:10:48Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Many people, through the course of their lives, will change either a\nname or an email address. For this reason, we have the mailmap, to map\nfrom a user's former name or email address to their current, canonical\nforms. Normally, this works well as it is.\n\nHowever, sometimes people change a name (or an email) and want to\ncompletely cease use of the former name or email. This could be because\na transgender person has transitioned, because a person has left an\nabusive partner or broken ties with an abusive family member, or for any\nother number of good and valuable reasons. In these cases, placing the\nformer name in the .mailmap may be undesirable.\n\nFor those situations, let's introduce a hashed mailmap, where the user's\nformer name or email address can be in the form @sha256:. This\nobscures the former name or email.\n\nNote that this is not perfect, because a user can simply look up all the\nhashed values and find out the old values. However, for projects which\nwish to adopt the feature, it can be somewhat effective to hash all\nexisting mailmap entries and include some no-op entries from other\ncontributors as well, so as to make this process less convenient.\n\nI've spoken to a variety of folks about this, and while we all agree\nthis design isn't perfect, it is an improvement over the status quo. It\nis obfuscation, not security, and in this case, I think that's fine.\nI'm open to hearing ideas about how to improve this design if there are\nany.\n\nI welcome feedback on this patch, while encouraging people to be mindful\nof our code of conduct.\n\nbrian m. carlson (1):\n mailmap: support hashed entries in mailmaps\n\n mailmap.c | 39 +++++++++++++++++++++++++++++++++++++--\n t/t4203-mailmap.sh | 35 +++++++++++++++++++++++++++++++++++\n 2 files changed, 72 insertions(+), 2 deletions(-)\n\n"},{"id":"412128","messageId":"20201213010539.544101-2-sandals@crustytoothpaste.net","threadId":"54816","inReplyTo":"20201213010539.544101-1-sandals@crustytoothpaste.net","subject":"[PATCH 1/1] mailmap: support hashed entries in mailmaps","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-12-13T01:05:39Z","receivedAt":"2020-12-13T01:10:49Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Many people, through the course of their lives, will change either a\nname or an email address. For this reason, we have the mailmap, to map\nfrom a user's former name or email address to their current, canonical\nforms. Normally, this works well as it is.\n\nHowever, sometimes people change a name or an email address and wish to\nwholly disassociate themselves from that former name or email address.\nFor example, a person may have left a company which engaged in a deeply\nunethical act with which the person does not want to be associated, or\nthey may have changed their name to disassociate themselves from an\nabusive family or partner. In such a case, using the former name or\naddress in any way may be undesirable and the person may wish to replace\nit as completely as possible.\n\nFor projects which wish to support this, introduce hashed forms into the\nmailmap. These forms, which start with \"@sha256:\" followed by a SHA-256\nhash of the entry, can be used in place of the form used in the commit\nfield. This form is intentionally designed to be unlikely to conflict\nwith legitimate use cases. For example, this is not a valid email\naddress according to RFC 5322. In the unlikely event that a user has\nput such a form into the actual commit as their name, we will accept it.\n\nWhile the form of the data is designed to accept multiple hash\nalgorithms, we intentionally do not support SHA-1. There is little\nreason to support such a weak algorithm in new use cases and no\nbackwards compatibility to consider. Moreover, SHA-256 is faster than\nthe SHA1DC implementation we use, so this not only improves performance,\nbut simplifies the current implementation somewhat as well.\n\nNote that it is, of course, possible to perform a lookup on all commit\nobjects to determine the actual entry which matches the hashed form of\nthe data. However, a project for which this feature is valuable may\nsimply insert entries for many contributors in order to make discovery\nof \"interesting\" entries significantly less convenient.\n\nSigned-off-by: brian m. carlson \n---\n mailmap.c | 39 +++++++++++++++++++++++++++++++++++++--\n t/t4203-mailmap.sh | 35 +++++++++++++++++++++++++++++++++++\n 2 files changed, 72 insertions(+), 2 deletions(-)\n\ndiff --git a/mailmap.c b/mailmap.c\nindex 962fd86d6d..09d0ad7ca4 100644\n--- a/mailmap.c\n+++ b/mailmap.c\n@@ -313,6 +313,41 @@ static struct string_list_item *lookup_prefix(struct string_list *map,\n \treturn NULL;\n }\n \n+/*\n+ * Convert an email or name into a hashed form for comparison. The hashed form\n+ * will be created in the form\n+ * @sha256:c68b7a430ac8dee9676ec77a387194e23f234d024e03d844050cf6c01775c8f6,\n+ * which would be the hashed form for \"doe@example.com\".\n+ */\n+static char *hashed_form(struct strbuf *buf, const struct git_hash_algo *algop, const char *key, size_t keylen)\n+{\n+\tgit_hash_ctx ctx;\n+\tunsigned char hashbuf[GIT_MAX_RAWSZ];\n+\tchar hexbuf[GIT_MAX_HEXSZ + 1];\n+\n+\talgop->init_fn(&ctx);\n+\talgop->update_fn(&ctx, key, keylen);\n+\talgop->final_fn(hashbuf, &ctx);\n+\thash_to_hex_algop_r(hexbuf, hashbuf, algop);\n+\n+\tstrbuf_addf(buf, \"@%s:%s\", algop->name, hexbuf);\n+\treturn buf->buf;\n+}\n+\n+static struct string_list_item *lookup_one(struct string_list *map,\n+\t\t\t\t\t const char *string, size_t len)\n+{\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct string_list_item *item = lookup_prefix(map, string, len);\n+\tif (item)\n+\t\treturn item;\n+\n+\thashed_form(&buf, &hash_algos[GIT_HASH_SHA256], string, len);\n+\titem = lookup_prefix(map, buf.buf, buf.len);\n+\tstrbuf_release(&buf);\n+\treturn item;\n+}\n+\n int map_user(struct string_list *map,\n \t const char **email, size_t *emaillen,\n \t const char **name, size_t *namelen)\n@@ -324,7 +359,7 @@ int map_user(struct string_list *map,\n \t\t (int)*namelen, debug_str(*name),\n \t\t (int)*emaillen, debug_str(*email));\n \n-\titem = lookup_prefix(map, *email, *emaillen);\n+\titem = lookup_one(map, *email, *emaillen);\n \tif (item != NULL) {\n \t\tme = (struct mailmap_entry *)item->util;\n \t\tif (me->namemap.nr) {\n@@ -334,7 +369,7 @@ int map_user(struct string_list *map,\n \t\t\t * simple entry.\n \t\t\t */\n \t\t\tstruct string_list_item *subitem;\n-\t\t\tsubitem = lookup_prefix(&me->namemap, *name, *namelen);\n+\t\t\tsubitem = lookup_one(&me->namemap, *name, *namelen);\n \t\t\tif (subitem)\n \t\t\t\titem = subitem;\n \t\t}\ndiff --git a/t/t4203-mailmap.sh b/t/t4203-mailmap.sh\nindex 586c3a86b1..794133ba5d 100755\n--- a/t/t4203-mailmap.sh\n+++ b/t/t4203-mailmap.sh\n@@ -62,6 +62,41 @@ test_expect_success 'check-mailmap --stdin arguments' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'hashed mailmap' '\n+\ttest_config mailmap.file ./hashed &&\n+\thashed_author_name=\"@sha256:$(printf \"$GIT_AUTHOR_NAME\" | test-tool sha256)\" &&\n+\thashed_author_email=\"@sha256:$(printf \"$GIT_AUTHOR_EMAIL\" | test-tool sha256)\" &&\n+\tcat >expect <<-EOF &&\n+\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>\n+\tEOF\n+\n+\tcat >hashed <<-EOF &&\n+\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $hashed_author_name <$GIT_AUTHOR_EMAIL>\n+\tEOF\n+\tgit check-mailmap \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\" >actual &&\n+\ttest_cmp expect actual &&\n+\n+\tcat >hashed <<-EOF &&\n+\tWrong $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\n+\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $hashed_author_name <$GIT_AUTHOR_EMAIL>\n+\tEOF\n+\t# Check that we prefer literal matches over hashed names.\n+\tgit check-mailmap \"$hashed_author_name <$GIT_AUTHOR_EMAIL>\" >actual &&\n+\ttest_cmp expect actual &&\n+\n+\tcat >hashed <<-EOF &&\n+\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $hashed_author_name <$hashed_author_email>\n+\tEOF\n+\tgit check-mailmap \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\" >actual &&\n+\ttest_cmp expect actual &&\n+\n+\tcat >hashed <<-EOF &&\n+\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> <$hashed_author_email>\n+\tEOF\n+\tgit check-mailmap \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'check-mailmap bogus contact' '\n \ttest_must_fail git check-mailmap bogus\n '\n"},{"id":"412153","messageId":"2cc4925f-3661-1dfb-2668-5f56edcb8455@kdbg.org","threadId":"54816","inReplyTo":"20201213010539.544101-2-sandals@crustytoothpaste.net","subject":"Re: [PATCH 1/1] mailmap: support hashed entries in mailmaps","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2020-12-13T09:34:06Z","receivedAt":"2020-12-13T09:35:14Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 13.12.20 um 02:05 schrieb brian m. carlson:\n> Many people, through the course of their lives, will change either a\n> name or an email address. For this reason, we have the mailmap, to map\n> from a user's former name or email address to their current, canonical\n> forms. Normally, this works well as it is.\n> \n> However, sometimes people change a name or an email address and wish to\n> wholly disassociate themselves from that former name or email address.\n> For example, a person may have left a company which engaged in a deeply\n> unethical act with which the person does not want to be associated, or\n> they may have changed their name to disassociate themselves from an\n> abusive family or partner. In such a case, using the former name or\n> address in any way may be undesirable and the person may wish to replace\n> it as completely as possible.\n> \n> For projects which wish to support this, introduce hashed forms into the\n> mailmap. These forms, which start with \"@sha256:\" followed by a SHA-256\n> hash of the entry, can be used in place of the form used in the commit\n> field. This form is intentionally designed to be unlikely to conflict\n> with legitimate use cases. For example, this is not a valid email\n> address according to RFC 5322. In the unlikely event that a user has\n> put such a form into the actual commit as their name, we will accept it.\n> \n> While the form of the data is designed to accept multiple hash\n> algorithms, we intentionally do not support SHA-1. There is little\n> reason to support such a weak algorithm in new use cases and no\n> backwards compatibility to consider. Moreover, SHA-256 is faster than\n> the SHA1DC implementation we use, so this not only improves performance,\n> but simplifies the current implementation somewhat as well.\n> \n> Note that it is, of course, possible to perform a lookup on all commit\n> objects to determine the actual entry which matches the hashed form of\n> the data. However, a project for which this feature is valuable may\n> simply insert entries for many contributors in order to make discovery\n> of \"interesting\" entries significantly less convenient.\n> \n> Signed-off-by: brian m. carlson \n> ---\n\n...\n\n> diff --git a/t/t4203-mailmap.sh b/t/t4203-mailmap.sh\n> index 586c3a86b1..794133ba5d 100755\n> --- a/t/t4203-mailmap.sh\n> +++ b/t/t4203-mailmap.sh\n> @@ -62,6 +62,41 @@ test_expect_success 'check-mailmap --stdin arguments' '\n> \ttest_cmp expect actual\n> '\n> \n> +test_expect_success 'hashed mailmap' '\n> +\ttest_config mailmap.file ./hashed &&\n> +\thashed_author_name=\"@sha256:$(printf \"$GIT_AUTHOR_NAME\" | test-tool sha256)\" &&\n> +\thashed_author_email=\"@sha256:$(printf \"$GIT_AUTHOR_EMAIL\" | test-tool sha256)\" &&\n> +\tcat >expect <<-EOF &&\n> +\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>\n> +\tEOF\n...\n> +\tcat >hashed <<-EOF &&\n> +\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> <$hashed_author_email>\n> +\tEOF\n> +\tgit check-mailmap \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\" >actual &&\n> +\ttest_cmp expect actual\n\nI don't understand the concept. A mailmap entry of the form\n\n A \n\ntells that the former address , which is recorded in old project\nhistory, should be replaced by A when a commit is displayed. I am\nassuming that the idea is that old should be the \"banned\" address.\nHow does a hashed entry help when the hashed value appears at the right\nside of a mailmap entry and that literal string never appears anywhere\nin the history?\n\n-- Hannes\n"},{"id":"412154","messageId":"b297b843-99e1-152d-aed1-20371af42d42@kdbg.org","threadId":"54816","inReplyTo":"2cc4925f-3661-1dfb-2668-5f56edcb8455@kdbg.org","subject":"Re: [PATCH 1/1] mailmap: support hashed entries in mailmaps","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2020-12-13T09:45:58Z","receivedAt":"2020-12-13T09:49:25Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 13.12.20 um 10:34 schrieb Johannes Sixt:\n> I don't understand the concept. A mailmap entry of the form\n> \n> A \n> \n> tells that the former address , which is recorded in old project\n> history, should be replaced by A when a commit is displayed. I am\n> assuming that the idea is that old should be the \"banned\" address.\n> How does a hashed entry help when the hashed value appears at the right\n> side of a mailmap entry and that literal string never appears anywhere\n> in the history?\n\nNever mind, I got it: A wants to be disassociated from , but not\nfrom their contributions whose authorship was recorded as .\nTherefore, Git must always compute the hash of all of , , etc,\njust in case that the hashed form appears anywhere in the mailmap file.\n\n-- Hannes\n"},{"id":"412157","messageId":"X9Z70aNc0cWcohNm@camp.crustytoothpaste.net","threadId":"54816","inReplyTo":"b297b843-99e1-152d-aed1-20371af42d42@kdbg.org","subject":"Re: [PATCH 1/1] mailmap: support hashed entries in mailmaps","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-12-13T20:38:41Z","receivedAt":"2020-12-13T20:39:59Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-12-13 at 09:45:58, Johannes Sixt wrote:\n> Am 13.12.20 um 10:34 schrieb Johannes Sixt:\n> > I don't understand the concept. A mailmap entry of the form\n> > \n> > A \n> > \n> > tells that the former address , which is recorded in old project\n> > history, should be replaced by A when a commit is displayed. I am\n> > assuming that the idea is that old should be the \"banned\" address.\n> > How does a hashed entry help when the hashed value appears at the right\n> > side of a mailmap entry and that literal string never appears anywhere\n> > in the history?\n> \n> Never mind, I got it: A wants to be disassociated from , but not\n> from their contributions whose authorship was recorded as .\n> Therefore, Git must always compute the hash of all of , , etc,\n> just in case that the hashed form appears anywhere in the mailmap file.\n\nYup, exactly. You can't specify the hashed one on the new side\nbecause it has to map to it, but you can on the old side. Sorry if\nthat wasn't clear.\n\nCome to think of it, this probably needs documentation as well, so I'll\nwait for any other feedback and then reroll with that in there.\nHopefully that will clear up any potential confusion.\n-- \nbrian m. carlson (he/him or they/them)\nHouston, Texas, US\n"},{"id":"412159","messageId":"xmqqv9d4ergd.fsf@gitster.c.googlers.com","threadId":"54816","inReplyTo":"20201213010539.544101-2-sandals@crustytoothpaste.net","subject":"Re: [PATCH 1/1] mailmap: support hashed entries in mailmaps","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-12-14T00:09:19Z","receivedAt":"2020-12-14T08:14:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> Come to think of it, this probably needs documentation as well, so I'll\n> wait for any other feedback and then reroll with that in there.\n> Hopefully that will clear up any potential confusion.\n\nNot just \"where does the hashed entry can appear in the file\", but\n\"how exactly does it gets computed\" needs to be described. If it is\nsufficient to do something like\n\n\tset x $(echo doe@example.com | sha256sum) &&\n\techo \"@sha256sum:$2\"\n\nthat exact procedure must be described to the users in the\ndocumentation (note: I know the above is not correct as I looked at\nthe tests---it is a demonstration of the need for a procedure using\ncommonly available tools).\n\nI wonder if somebody may want to do a dedicated tool that lets you\n\n (1) given an e-mail and/or a name, look-up existing entries and\n show what pair it maps to;\n\n (2) take a new pair and add mapping from it to some\n other pair.\n\n (3) take an existing mailmap file, and obfuscate all the existing\n entries.\n\nThe first one is covered by \"check-mailmap\", so the other two could\nbe new features added to the command to be triggered with a command\nline option.\n\n> +\tcat >hashed <<-EOF &&\n> +\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $hashed_author_name <$GIT_AUTHOR_EMAIL>\n> +\tEOF\n> +\tgit check-mailmap \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\" >actual &&\n\nThe two spaces after \"check-mailmap\" is not significant but drew my\nattention. Let's not do so.\n"},{"id":"412162","messageId":"87eejswql6.fsf@evledraar.gmail.com","threadId":"54816","inReplyTo":"20201213010539.544101-2-sandals@crustytoothpaste.net","subject":"Re: [PATCH 1/1] mailmap: support hashed entries in mailmaps","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2020-12-14T11:54:13Z","receivedAt":"2020-12-14T12:41:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sun, Dec 13 2020, brian m. carlson wrote:\n\n> Many people, through the course of their lives, will change either a\n> name or an email address. For this reason, we have the mailmap, to map\n> from a user's former name or email address to their current, canonical\n> forms. Normally, this works well as it is.\n>\n> However, sometimes people change a name or an email address and wish to\n> wholly disassociate themselves from that former name or email address.\n> For example, a person may have left a company which engaged in a deeply\n> unethical act with which the person does not want to be associated, or\n> they may have changed their name to disassociate themselves from an\n> abusive family or partner. In such a case, using the former name or\n> address in any way may be undesirable and the person may wish to replace\n> it as completely as possible.\n>\n> [...]\n>\n> Note that it is, of course, possible to perform a lookup on all commit\n> objects to determine the actual entry which matches the hashed form of\n> the data.\n\nThe commit message & cover letter are subtly different in a way that I\ndidn't even notice at first glance. E.g. I assume based on the cover\nletter that one part of this this is a proposed solution do the whole\n\"deadname\" problem. It would be nice if v2 were more explicit and\nattempted to explicitly summarize the use-cases in the commit message.\n\nBut for now I'll attempt to read between the lines from having read\nboth.\n\nI don't understand why either the problem of \"I don't want to see my old\nname again\" or \"I want to hide from other abusive people\" (as an aside:\nbut not so much that you'd still take the risk of submitting a patch to\n.mailmap?) require a hashing solution, as opposed to just some encoding\nin the .mailmap file such as base64.\n\nYou can still trivially get the same information in the end, on git.git\nrunning --pretty=format:\"%aN %aE %an %ae\" takes under a second. A part\nof your commit message seems to address this:\n\n> However, a project for which this feature is valuable may\n> simply insert entries for many contributors in order to make discovery\n> of \"interesting\" entries significantly less convenient.\n\nBut I don't get how that's helped at all by a sha256 hash. Since you can\ntrivially re-expand these again using log/check-mailmap the hashing\noffers no extra protection beyond a trivial layer of obscurity in those\ncases. You'd get the same safety in numbers by having everything a large\nun-hashed .mailmap file, would you not?\n\nI think the underlying use-case is legitimate, but I read it as\nprimarily a social signaling feature by a trivial addition of\nobscurity. Someone called X would like not to be called Y anymore, or\nnot be found in a search engine or \"git grep\" when searching for \"Y\".\n\nSo I'd think purely from the perspective of the feature's appearance to\nusers matching its underlying security we'd be better served with\nsupport for encoding of some sort. E.g. URL encoding, Base64, or even\njust string_reverse() (ROT13 is out as not working for non-ASCII names).\n\nThe encoding versions of this have the added bonus of expanding the\nuse-case beyond what you're suggesting. If you're trying to map e.g. a\nnon-UTF-8 E-Mail address (in your project due to some encoding error)\nyou'd be able to put it into .mailmap without making the project\nmaintainers deal with invalid non-UTF-8 encoding in the file (the\nexisting support is sufficient to map names in most such cases).\n\nAnother reason I'd prefer some encoding solution is because .mailmap\nisn't just used by git itself. Since the format got added it's become\nhow a lot of downstream systems do this mapping. E.g. I worked once on a\nchange management system that mapped lots of user actions across\ndifferent systems, and piggy-backed on .mailmap files in git to resolve\nE-Mail addresses even in cases where the originating data wasn't within\ngit.\n\nNow because of the trivialness of the format it's easy to e.g. import it\ninto a DB table and do a JOIN against it (or the same after converting\nit from some trivial encoding). Use-cases like that would become a full\nhistory walk for each project to extract the real E-Mails (or a re\nimplementation of the SHA256 trick in some sub-SELECT in the database).\n\nThose are all solvable problems that are rather trivial in the end. I\njust wonder if we're not making things needlessly hard to achieve the\nstated aims. And to be fair, most of those aims I inferred (and might\nhave incorrectly inferred), since as noted above the patch itself\ndoesn't discuss the tradeoffs of potential alternate solutions).\n\n> Signed-off-by: brian m. carlson \n> ---\n> mailmap.c | 39 +++++++++++++++++++++++++++++++++++++--\n> t/t4203-mailmap.sh | 35 +++++++++++++++++++++++++++++++++++\n> 2 files changed, 72 insertions(+), 2 deletions(-)\n>\n> [...]\n>\n> int map_user(struct string_list *map,\n> \t const char **email, size_t *emaillen,\n> \t const char **name, size_t *namelen)\n> @@ -324,7 +359,7 @@ int map_user(struct string_list *map,\n> \t\t (int)*namelen, debug_str(*name),\n> \t\t (int)*emaillen, debug_str(*email));\n> \n> -\titem = lookup_prefix(map, *email, *emaillen);\n> +\titem = lookup_one(map, *email, *emaillen);\n> \tif (item != NULL) {\n> \t\tme = (struct mailmap_entry *)item->util;\n> \t\tif (me->namemap.nr) {\n> @@ -334,7 +369,7 @@ int map_user(struct string_list *map,\n> \t\t\t * simple entry.\n> \t\t\t */\n> \t\t\tstruct string_list_item *subitem;\n> -\t\t\tsubitem = lookup_prefix(&me->namemap, *name, *namelen);\n> +\t\t\tsubitem = lookup_one(&me->namemap, *name, *namelen);\n> \t\t\tif (subitem)\n> \t\t\t\titem = subitem;\n> \t\t}\n\nIf you turn on DEBUG_MAILMAP=1 at the top of the file and run e.g. an\nunbounded --pretty=format=:%aE you can see we'll call map_user() in a\nloop for each commit shown. What I'm suggesting above can be read as\n\"can't we have some solution that achieves the same aims, but which we\ncan handle purely in add_mapping()?\". Both for our case, and for\nexternal parsers/re-implementations.\n\nIn any case it would be interesting if v2 amended\nt/perf/p4205-log-pretty-formats.sh to test e.g. the impact of linux.git\nwith all-sha256 entries to see what the cost in the tight loop could be.\n"},{"id":"412227","messageId":"X9gV3mKwGrHL7PzV@coredump.intra.peff.net","threadId":"54816","inReplyTo":"20201213010539.544101-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH 0/1] Hashed mailmap support","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-12-15T01:48:14Z","receivedAt":"2020-12-15T01:49:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Dec 13, 2020 at 01:05:38AM +0000, brian m. carlson wrote:\n\n> Note that this is not perfect, because a user can simply look up all the\n> hashed values and find out the old values. However, for projects which\n> wish to adopt the feature, it can be somewhat effective to hash all\n> existing mailmap entries and include some no-op entries from other\n> contributors as well, so as to make this process less convenient.\n\nI remain unconvinced of the value of any noop entries. Ultimately it's\neasy to invert a one-way hash that comes from a small known set of\ninputs. And that's true whether there are extra noops or not.\n\nThe interesting argument IMHO is that somebody has to _bother_ to invert\nthe hash. So it means that the old and new identities do not show up\nnext to each other in a file indexed by search engines, etc. That drops\nthe low-hanging fruit.\n\nAnd from that argument, I think the obvious question becomes: is it\nworth using a real one-way function, as opposed to just obscuring the\nraw bytes (which Ævar went into in more detail). I don't have a strong\nopinion either way (the obvious one in favor is that it's less expensive\nto do so; and something like \"git log\" will have to either compute a lot\nof these hashes, or cache the hash computations internally).\n\nI think somebody also mentioned that there's value in the social\nsignaling here, and I agree with that. But that is true even for a\nreversible encoding, I think.\n\n-Peff\n"},{"id":"412232","messageId":"X9giE7ZVtuE7dvTr@coredump.intra.peff.net","threadId":"54816","inReplyTo":"X9gV3mKwGrHL7PzV@coredump.intra.peff.net","subject":"Re: [PATCH 0/1] Hashed mailmap support","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-12-15T02:40:19Z","receivedAt":"2020-12-15T02:41:46Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Dec 14, 2020 at 08:48:14PM -0500, Jeff King wrote:\n\n> On Sun, Dec 13, 2020 at 01:05:38AM +0000, brian m. carlson wrote:\n> \n> > Note that this is not perfect, because a user can simply look up all the\n> > hashed values and find out the old values. However, for projects which\n> > wish to adopt the feature, it can be somewhat effective to hash all\n> > existing mailmap entries and include some no-op entries from other\n> > contributors as well, so as to make this process less convenient.\n> \n> I remain unconvinced of the value of any noop entries. Ultimately it's\n> easy to invert a one-way hash that comes from a small known set of\n> inputs. And that's true whether there are extra noops or not.\n> \n> The interesting argument IMHO is that somebody has to _bother_ to invert\n> the hash. So it means that the old and new identities do not show up\n> next to each other in a file indexed by search engines, etc. That drops\n> the low-hanging fruit.\n> \n> And from that argument, I think the obvious question becomes: is it\n> worth using a real one-way function, as opposed to just obscuring the\n> raw bytes (which Ævar went into in more detail). I don't have a strong\n> opinion either way (the obvious one in favor is that it's less expensive\n> to do so; and something like \"git log\" will have to either compute a lot\n> of these hashes, or cache the hash computations internally).\n> \n> I think somebody also mentioned that there's value in the social\n> signaling here, and I agree with that. But that is true even for a\n> reversible encoding, I think.\n\nAfter re-reading what I wrote, I just wanted to make clear: overall the\nfeature makes sense to me. I am questioning only the argument for it,\nand whether a one-way hash is the right tradeoff there.\n\n-Peff\n"},{"id":"412252","messageId":"d250435c-6f57-3c98-25f1-916269f13b7b@gmail.com","threadId":"54816","inReplyTo":"20201213010539.544101-2-sandals@crustytoothpaste.net","subject":"Re: [PATCH 1/1] mailmap: support hashed entries in mailmaps","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2020-12-15T11:13:51Z","receivedAt":"2020-12-15T11:15:29Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Brian\n\nOn 13/12/2020 01:05, brian m. carlson wrote:\n> Many people, through the course of their lives, will change either a\n> name or an email address. For this reason, we have the mailmap, to map\n> from a user's former name or email address to their current, canonical\n> forms. Normally, this works well as it is.\n> \n> However, sometimes people change a name or an email address and wish to\n> wholly disassociate themselves from that former name or email address.\n> For example, a person may have left a company which engaged in a deeply\n> unethical act with which the person does not want to be associated, or\n> they may have changed their name to disassociate themselves from an\n> abusive family or partner.\n\nI think we should be clear in the documentation that by adding a hashed \n.mailmap entry people are still publicly associating their old identity \nwith their new identity it's just that the association is obscured. They \nshould not rely on it for their safety. An abusive partner knows the old \nidentity so all they have to do to find the new identity is hash the old \nidentity and see if it is in the .mailmap file.\n\nHaving said that I think this is a useful step forward in may cases.\n\nBest Wishes\n\nPhillip\n\n> In such a case, using the former name or\n> address in any way may be undesirable and the person may wish to replace\n> it as completely as possible.\n> \n> For projects which wish to support this, introduce hashed forms into the\n> mailmap. These forms, which start with \"@sha256:\" followed by a SHA-256\n> hash of the entry, can be used in place of the form used in the commit\n> field. This form is intentionally designed to be unlikely to conflict\n> with legitimate use cases. For example, this is not a valid email\n> address according to RFC 5322. In the unlikely event that a user has\n> put such a form into the actual commit as their name, we will accept it.\n> \n> While the form of the data is designed to accept multiple hash\n> algorithms, we intentionally do not support SHA-1. There is little\n> reason to support such a weak algorithm in new use cases and no\n> backwards compatibility to consider. Moreover, SHA-256 is faster than\n> the SHA1DC implementation we use, so this not only improves performance,\n> but simplifies the current implementation somewhat as well.\n> \n> Note that it is, of course, possible to perform a lookup on all commit\n> objects to determine the actual entry which matches the hashed form of\n> the data. However, a project for which this feature is valuable may\n> simply insert entries for many contributors in order to make discovery\n> of \"interesting\" entries significantly less convenient.\n> \n> Signed-off-by: brian m. carlson \n> ---\n> mailmap.c | 39 +++++++++++++++++++++++++++++++++++++--\n> t/t4203-mailmap.sh | 35 +++++++++++++++++++++++++++++++++++\n> 2 files changed, 72 insertions(+), 2 deletions(-)\n> \n> diff --git a/mailmap.c b/mailmap.c\n> index 962fd86d6d..09d0ad7ca4 100644\n> --- a/mailmap.c\n> +++ b/mailmap.c\n> @@ -313,6 +313,41 @@ static struct string_list_item *lookup_prefix(struct string_list *map,\n> \treturn NULL;\n> }\n> \n> +/*\n> + * Convert an email or name into a hashed form for comparison. The hashed form\n> + * will be created in the form\n> + * @sha256:c68b7a430ac8dee9676ec77a387194e23f234d024e03d844050cf6c01775c8f6,\n> + * which would be the hashed form for \"doe@example.com\".\n> + */\n> +static char *hashed_form(struct strbuf *buf, const struct git_hash_algo *algop, const char *key, size_t keylen)\n> +{\n> +\tgit_hash_ctx ctx;\n> +\tunsigned char hashbuf[GIT_MAX_RAWSZ];\n> +\tchar hexbuf[GIT_MAX_HEXSZ + 1];\n> +\n> +\talgop->init_fn(&ctx);\n> +\talgop->update_fn(&ctx, key, keylen);\n> +\talgop->final_fn(hashbuf, &ctx);\n> +\thash_to_hex_algop_r(hexbuf, hashbuf, algop);\n> +\n> +\tstrbuf_addf(buf, \"@%s:%s\", algop->name, hexbuf);\n> +\treturn buf->buf;\n> +}\n> +\n> +static struct string_list_item *lookup_one(struct string_list *map,\n> +\t\t\t\t\t const char *string, size_t len)\n> +{\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\tstruct string_list_item *item = lookup_prefix(map, string, len);\n> +\tif (item)\n> +\t\treturn item;\n> +\n> +\thashed_form(&buf, &hash_algos[GIT_HASH_SHA256], string, len);\n> +\titem = lookup_prefix(map, buf.buf, buf.len);\n> +\tstrbuf_release(&buf);\n> +\treturn item;\n> +}\n> +\n> int map_user(struct string_list *map,\n> \t const char **email, size_t *emaillen,\n> \t const char **name, size_t *namelen)\n> @@ -324,7 +359,7 @@ int map_user(struct string_list *map,\n> \t\t (int)*namelen, debug_str(*name),\n> \t\t (int)*emaillen, debug_str(*email));\n> \n> -\titem = lookup_prefix(map, *email, *emaillen);\n> +\titem = lookup_one(map, *email, *emaillen);\n> \tif (item != NULL) {\n> \t\tme = (struct mailmap_entry *)item->util;\n> \t\tif (me->namemap.nr) {\n> @@ -334,7 +369,7 @@ int map_user(struct string_list *map,\n> \t\t\t * simple entry.\n> \t\t\t */\n> \t\t\tstruct string_list_item *subitem;\n> -\t\t\tsubitem = lookup_prefix(&me->namemap, *name, *namelen);\n> +\t\t\tsubitem = lookup_one(&me->namemap, *name, *namelen);\n> \t\t\tif (subitem)\n> \t\t\t\titem = subitem;\n> \t\t}\n> diff --git a/t/t4203-mailmap.sh b/t/t4203-mailmap.sh\n> index 586c3a86b1..794133ba5d 100755\n> --- a/t/t4203-mailmap.sh\n> +++ b/t/t4203-mailmap.sh\n> @@ -62,6 +62,41 @@ test_expect_success 'check-mailmap --stdin arguments' '\n> \ttest_cmp expect actual\n> '\n> \n> +test_expect_success 'hashed mailmap' '\n> +\ttest_config mailmap.file ./hashed &&\n> +\thashed_author_name=\"@sha256:$(printf \"$GIT_AUTHOR_NAME\" | test-tool sha256)\" &&\n> +\thashed_author_email=\"@sha256:$(printf \"$GIT_AUTHOR_EMAIL\" | test-tool sha256)\" &&\n> +\tcat >expect <<-EOF &&\n> +\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>\n> +\tEOF\n> +\n> +\tcat >hashed <<-EOF &&\n> +\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $hashed_author_name <$GIT_AUTHOR_EMAIL>\n> +\tEOF\n> +\tgit check-mailmap \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\" >actual &&\n> +\ttest_cmp expect actual &&\n> +\n> +\tcat >hashed <<-EOF &&\n> +\tWrong $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\n> +\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $hashed_author_name <$GIT_AUTHOR_EMAIL>\n> +\tEOF\n> +\t# Check that we prefer literal matches over hashed names.\n> +\tgit check-mailmap \"$hashed_author_name <$GIT_AUTHOR_EMAIL>\" >actual &&\n> +\ttest_cmp expect actual &&\n> +\n> +\tcat >hashed <<-EOF &&\n> +\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $hashed_author_name <$hashed_author_email>\n> +\tEOF\n> +\tgit check-mailmap \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\" >actual &&\n> +\ttest_cmp expect actual &&\n> +\n> +\tcat >hashed <<-EOF &&\n> +\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> <$hashed_author_email>\n> +\tEOF\n> +\tgit check-mailmap \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\" >actual &&\n> +\ttest_cmp expect actual\n> +'\n> +\n> test_expect_success 'check-mailmap bogus contact' '\n> \ttest_must_fail git check-mailmap bogus\n> '\n> \n"},{"id":"412253","messageId":"558dc3da-e0c4-2e7e-83b3-e8f8c431a696@gmail.com","threadId":"54816","inReplyTo":"X9gV3mKwGrHL7PzV@coredump.intra.peff.net","subject":"Re: [PATCH 0/1] Hashed mailmap support","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2020-12-15T11:15:06Z","receivedAt":"2020-12-15T11:16:15Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Peff, Ævar, Brian\n\nOn 15/12/2020 01:48, Jeff King wrote:\n> On Sun, Dec 13, 2020 at 01:05:38AM +0000, brian m. carlson wrote:\n> \n>> Note that this is not perfect, because a user can simply look up all the\n>> hashed values and find out the old values. However, for projects which\n>> wish to adopt the feature, it can be somewhat effective to hash all\n>> existing mailmap entries and include some no-op entries from other\n>> contributors as well, so as to make this process less convenient.\n> \n> I remain unconvinced of the value of any noop entries. Ultimately it's\n> easy to invert a one-way hash that comes from a small known set of\n> inputs. And that's true whether there are extra noops or not.\n> \n> The interesting argument IMHO is that somebody has to _bother_ to invert\n> the hash. So it means that the old and new identities do not show up\n> next to each other in a file indexed by search engines, etc. That drops\n> the low-hanging fruit.\n> \n> And from that argument, I think the obvious question becomes: is it\n> worth using a real one-way function, as opposed to just obscuring the\n> raw bytes (which Ævar went into in more detail). I don't have a strong\n> opinion either way (the obvious one in favor is that it's less expensive\n> to do so; and something like \"git log\" will have to either compute a lot\n> of these hashes, or cache the hash computations internally).\n> \n> I think somebody also mentioned that there's value in the social\n> signaling here, and I agree with that. But that is true even for a\n> reversible encoding, I think.\n\n From an obscurity point of view one possible advantage of using a \none-way function as opposed to just obscuring the raw bytes with a \nreversible encoding is that looking up an old identity requires someone \nto have both the .mailmap and the repository, they cannot get the old \nidentities by just downloading the .mailmap file. (I think this the same \nargument as Ævar makes in favor of a reversible encoding as the .mailmap \nfile has other uses)\n\nBest Wishes\n\nPhillip\n\n> -Peff\n> \n"},{"id":"412320","messageId":"X9lZ0ABtWvjS+Zsv@camp.crustytoothpaste.net","threadId":"54816","inReplyTo":"xmqqv9d4ergd.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 1/1] mailmap: support hashed entries in mailmaps","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-12-16T00:50:24Z","receivedAt":"2020-12-16T00:51:30Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-12-14 at 00:09:19, Junio C Hamano wrote:\n> \"brian m. carlson\" writes:\n> \n> > Come to think of it, this probably needs documentation as well, so I'll\n> > wait for any other feedback and then reroll with that in there.\n> > Hopefully that will clear up any potential confusion.\n> \n> Not just \"where does the hashed entry can appear in the file\", but\n> \"how exactly does it gets computed\" needs to be described. If it is\n> sufficient to do something like\n> \n> \tset x $(echo doe@example.com | sha256sum) &&\n> \techo \"@sha256sum:$2\"\n> \n> that exact procedure must be described to the users in the\n> documentation (note: I know the above is not correct as I looked at\n> the tests---it is a demonstration of the need for a procedure using\n> commonly available tools).\n\nI believe the difference is that \"echo\" adds a newline and you probably\nwanted \"printf\" here. But I get your point: we need documentation to\nexplain how to do this that's simple and straightforward, and as we've\nboth pointed out, there isn't any at all. I'll add some.\n\n> I wonder if somebody may want to do a dedicated tool that lets you\n> \n> (1) given an e-mail and/or a name, look-up existing entries and\n> show what pair it maps to;\n> \n> (2) take a new pair and add mapping from it to some\n> other pair.\n> \n> (3) take an existing mailmap file, and obfuscate all the existing\n> entries.\n> \n> The first one is covered by \"check-mailmap\", so the other two could\n> be new features added to the command to be triggered with a command\n> line option.\n\nThat could be a useful tool.\n\n> > +\tcat >hashed <<-EOF &&\n> > +\t$GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $hashed_author_name <$GIT_AUTHOR_EMAIL>\n> > +\tEOF\n> > +\tgit check-mailmap \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\" >actual &&\n> \n> The two spaces after \"check-mailmap\" is not significant but drew my\n> attention. Let's not do so.\n\nThat wasn't intentional. Will fix.\n-- \nbrian m. carlson (he/him or they/them)\nHouston, Texas, US\n"},{"id":"412513","messageId":"X9wUGaR3IXcpV0nT@camp.crustytoothpaste.net","threadId":"54816","inReplyTo":"X9gV3mKwGrHL7PzV@coredump.intra.peff.net","subject":"Re: [PATCH 0/1] Hashed mailmap support","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-12-18T02:29:45Z","receivedAt":"2020-12-18T02:31:31Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-12-15 at 01:48:14, Jeff King wrote:\n> On Sun, Dec 13, 2020 at 01:05:38AM +0000, brian m. carlson wrote:\n> \n> > Note that this is not perfect, because a user can simply look up all the\n> > hashed values and find out the old values. However, for projects which\n> > wish to adopt the feature, it can be somewhat effective to hash all\n> > existing mailmap entries and include some no-op entries from other\n> > contributors as well, so as to make this process less convenient.\n> \n> I remain unconvinced of the value of any noop entries. Ultimately it's\n> easy to invert a one-way hash that comes from a small known set of\n> inputs. And that's true whether there are extra noops or not.\n> \n> The interesting argument IMHO is that somebody has to _bother_ to invert\n> the hash. So it means that the old and new identities do not show up\n> next to each other in a file indexed by search engines, etc. That drops\n> the low-hanging fruit.\n> \n> And from that argument, I think the obvious question becomes: is it\n> worth using a real one-way function, as opposed to just obscuring the\n> raw bytes (which Ævar went into in more detail). I don't have a strong\n> opinion either way (the obvious one in favor is that it's less expensive\n> to do so; and something like \"git log\" will have to either compute a lot\n> of these hashes, or cache the hash computations internally).\n\nI don't disagree that it's easy to invert. The question is, is somebody\ngoing to look at a large set of (e.g., a couple hundred) hashed entries\nand be able to easily find ones of people they'd like to make life\ndifficult for or into whose business they'd like to pry or is it going\nto be too inconvenient? I think base64 makes the job too easy and if it\nwere me in that situation, I'd prefer a little more effort.\n\nI think there's also the benefit, at least for email addresses, in that\npeople can map a \"private\" email address that they used accidentally\ninto one with more robust filtering without letting bad actors invert it\ntrivially. That doesn't mean spammers can't run through the log, but it\ndoes mean that they can't write a simple tool to invert base64 email\naddresses they've harvested out of Git repositories. And we know that\nspammers and recruiters (which, in this case, are also spammers) do\nindeed scrape repositories via the repository web interfaces.\n\nAnd as someone who had to download all 21 GB of the Chromium repository\nfor testing purposes recently, I can tell you that absent a very\ncompelling use case, nobody's going to want to download that entire\nrepository just to extract some personal information, especially since\nthe git index-pack operation is essentially guaranteed to take at least\n7 minutes at maximum speed. So by hashing, we've guaranteed significant\ninconvenience unless you have the repository, whereas that's not the\ncase for base64. And making abuse even slightly harder can often deter\na surprising amount of it[0].\n\nSo I think I'm firmly in favor of hashing. If that means my patch needs\nto implement caching, then I'll reroll with that change. I think by\nswitching to a hash table I may be able to actually improve total\nperformance overall, at least in some cases.\n\n> I think somebody also mentioned that there's value in the social\n> signaling here, and I agree with that. But that is true even for a\n> reversible encoding, I think.\n\nThat's true, I agree. And for many projects, that will be sufficient.\nIf I saw a hashed mailmap entry, I would assume that it was intended to\nbe private and would respect that.\n\n[0] See, for example, greylisting.\n-- \nbrian m. carlson (he/him or they/them)\nHouston, Texas, US\n"},{"id":"412535","messageId":"X9xEnpLeZ4mCjwWF@coredump.intra.peff.net","threadId":"54816","inReplyTo":"X9wUGaR3IXcpV0nT@camp.crustytoothpaste.net","subject":"Re: [PATCH 0/1] Hashed mailmap support","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-12-18T05:56:46Z","receivedAt":"2020-12-18T05:57:29Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Dec 18, 2020 at 02:29:45AM +0000, brian m. carlson wrote:\n\n> > And from that argument, I think the obvious question becomes: is it\n> > worth using a real one-way function, as opposed to just obscuring the\n> > raw bytes (which Ævar went into in more detail). I don't have a strong\n> > opinion either way (the obvious one in favor is that it's less expensive\n> > to do so; and something like \"git log\" will have to either compute a lot\n> > of these hashes, or cache the hash computations internally).\n> [...]\n> So I think I'm firmly in favor of hashing. If that means my patch needs\n> to implement caching, then I'll reroll with that change. I think by\n> switching to a hash table I may be able to actually improve total\n> performance overall, at least in some cases.\n\nOK. I agree it raises the bar a little bit. Whether that matters or not\ndepends on your threat model (e.g., casual spammers versus dedicated\ninformation seekers). I don't have a particularly strong opinion on\nwhat's realistic, but I don't mind erring on the side of caution here.\n\nIt might be worth making a short argument along those lines in the\ncommit message.\n\nAs far as caching goes, my main concern is mostly that people who are\nnot using the feature do not pay a performance penalty. So:\n\n - if the feature is not used in the repository's mailmap, it should\n have zero cost (i.e., we do not bother hashing lookup entries if\n there are no hashed entries in the map)\n\n - as soon as there is one hashed entry, we need to hash the key for\n every lookup in the map. I'm not sure what the overhead is like. It\n might be negligible. But I think we should confirm that before\n proceeding.\n\n> And as someone who had to download all 21 GB of the Chromium repository\n> for testing purposes recently, I can tell you that absent a very\n> compelling use case, nobody's going to want to download that entire\n> repository just to extract some personal information, especially since\n> the git index-pack operation is essentially guaranteed to take at least\n> 7 minutes at maximum speed. So by hashing, we've guaranteed significant\n> inconvenience unless you have the repository, whereas that's not the\n> case for base64. And making abuse even slightly harder can often deter\n> a surprising amount of it[0].\n\nThey just need the objects that have ident lines in them, so:\n\n $ time git clone --bare --filter=tree:0 https://github.com/chromium/chromium\n Cloning into bare repository 'chromium.git'...\n remote: Enumerating objects: 202, done.\n remote: Counting objects: 100% (202/202), done.\n remote: Compressing objects: 100% (161/161), done.\n remote: Total 1105453 (delta 49), reused 194 (delta 41), pack-reused 1105251\n Receiving objects: 100% (1105453/1105453), 462.14 MiB | 11.13 MiB/s, done.\n Resolving deltas: 100% (99790/99790), done.\n \n real\t0m49.304s\n user\t0m21.330s\n sys\t0m4.727s\n\ngets you there much quicker. I don't think that negates your point about\nraising the bar, but my guess is that the threat model of \"casual\nspammer\" would probably be deterred, but \"troll who wants to annoy\nspecific person\" would probably not be.\n\n-Peff\n"}]}