{"thread":{"id":"63330","subject":"[PATCH v3 0/3] send-email: add oauth2 support and fix outlook breaking threads","startedAt":"2025-04-22T15:26:35Z","lastAt":"2025-04-23T16:24:11Z","messageCount":10,"participants":["Aditya Garg","Junio C Hamano"],"isPatch":true,"patchVersion":3,"patchTotal":3},"messages":[{"id":"516505","messageId":"PN3PR01MB95975C7925B6395DEC46F9EAB8BB2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","threadId":"63330","inReplyTo":null,"subject":"[PATCH v3 0/3] send-email: add oauth2 support and fix outlook breaking threads","fromName":"Aditya Garg","fromEmail":"gargaditya08@live.com","sentAt":"2025-04-22T15:23:18Z","receivedAt":"2025-04-22T15:26:35Z","isPatch":true,"sender":{"key":"gargaditya08@live.com","avatar":"https://avatars.githubusercontent.com/u/85610623?v=4"},"body":"Hi all!\n\nThis patch series includes three changes:\n\n1. It adds support for Oauth2 authentication, which is now compulsory by Microsoft.\n This patch has been rebased to the latest version from the original version\n at https://lore.kernel.org/git/20250125190131.48717-1-julian@swagemakers.org/\n\n2. The second patch makes the script reply to the message id set by the outlook,\n since outlook has its own proprietary way to handle message ids,\n and does not allow user to set their own. As a result, threads were breaking.\n\n3. The final patch adds a new option to generate passwords like OAuth2 tokens.\n This is useful for users who want to use a script which generates tokens for\n OAuth2 authentication.\n\nDetailed description of each patch has been done in the respective patches\n\nBTW, I am sending this series using the patched send-email by these patches from\nOutlook!\n\nv2: Fix errors flagged by the CI\nv3: Add third patch to generate passwords like OAuth2 tokens\n\nAditya Garg (2):\n send-email: retrieve Message-ID from outlook SMTP server\n send-email: add option to generate passswords like OAuth2 tokens\n\nJulian Swagemakers (1):\n send-email: implement SMTP bearer authentication\n\n Documentation/git-send-email.adoc | 13 ++++-\n git-send-email.perl | 90 ++++++++++++++++++++++++++++++-\n 2 files changed, 101 insertions(+), 2 deletions(-)\n\n-- \n2.49.0\n\n"},{"id":"516506","messageId":"PN3PR01MB959738CC031F383746224F78B8BB2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","threadId":"63330","inReplyTo":"PN3PR01MB95975C7925B6395DEC46F9EAB8BB2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","subject":"[PATCH v3 1/3] send-email: implement SMTP bearer authentication","fromName":"Aditya Garg","fromEmail":"gargaditya08@live.com","sentAt":"2025-04-22T15:23:19Z","receivedAt":"2025-04-22T15:26:38Z","isPatch":true,"sender":{"key":"gargaditya08@live.com","avatar":"https://avatars.githubusercontent.com/u/85610623?v=4"},"body":"From: Julian Swagemakers \n\nManually send SMTP AUTH command for auth type OAUTHBEARER and XOAUTH2.\nThis is necessary since they are currently not supported by the Perls\nAuthen::SASL module.\n\nThe bearer token needs to be passed in as the password. This can be done\nwith git-credential-oauth[0] after minor modifications[1]. Which will\nallow using git send-email with Gmail and oauth2 authentication:\n\n [credential]\n helper = cache --timeout 7200 # two hours\n helper = oauth\n [sendemail]\n smtpEncryption = tls\n smtpServer = smtp.gmail.com\n smtpUser = example@gmail.com\n smtpServerPort = 587\n smtpauth = OAUTHBEARER\n\nAs well as Office 365 accounts:\n\n [credential]\n helper = cache --timeout 7200 # two hours\n helper = oauth\n [sendemail]\n smtpEncryption = tls\n smtpServer = smtp.office365.com\n smtpUser = example@example.com\n smtpServerPort = 587\n smtpauth = XOAUTH2\n\n[0] https://github.com/hickford/git-credential-oauth\n[1] https://github.com/hickford/git-credential-oauth/issues/48\n\nTested-by: M Hickford \nSigned-off-by: Julian Swagemakers \nSigned-off-by: Aditya Garg \n---\n Documentation/git-send-email.adoc | 5 ++-\n git-send-email.perl | 64 ++++++++++++++++++++++++++++++-\n 2 files changed, 67 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-send-email.adoc b/Documentation/git-send-email.adoc\nindex 7f223db42d..1bf75c060d 100644\n--- a/Documentation/git-send-email.adoc\n+++ b/Documentation/git-send-email.adoc\n@@ -213,7 +213,10 @@ SMTP server and if it is supported by the utilized SASL library, the mechanism\n is used for authentication. If neither 'sendemail.smtpAuth' nor `--smtp-auth`\n is specified, all mechanisms supported by the SASL library can be used. The\n special value 'none' maybe specified to completely disable authentication\n-independently of `--smtp-user`\n+independently of `--smtp-user`. Specifying `OAUTHBEARER` or `XOAUTH2` will\n+bypass SASL negotiation and force bearer authentication. In this case the\n+bearer token must be provided with `--smtp-pass` or using a credential helper\n+and `--smtp-encryption=tls` must be set.\n \n --smtp-pass[=]::\n \tPassword for SMTP-AUTH. The argument is optional: If no\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex 1f613fa979..a6cafda29c 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -1398,6 +1398,63 @@ sub smtp_host_string {\n \t}\n }\n \n+sub generate_oauthbearer_string {\n+\t# This will generate the oauthbearer string used for authentication.\n+\t#\n+\t# \"n,a=\" {User} \",^Ahost=\" {Host} \"^Aport=\" {Port} \"^Aauth=Bearer \" {Access Token} \"^A^A\n+\t#\n+\t# The first part `n,a=\" {User} \",` is the gs2 header described in RFC5801.\n+\t# * gs2-cb-flag `n` -> client does not support CB\n+\t# * gs2-authzid `a=\" {User} \"`\n+\t#\n+\t# The second part are key value pairs containing host, port and auth as\n+\t# described in RFC7628.\n+\t#\n+\t# https://datatracker.ietf.org/doc/html/rfc5801\n+\t# https://datatracker.ietf.org/doc/html/rfc7628\n+\tmy $username = shift;\n+\tmy $token = shift;\n+\treturn \"n,a=$username,\\001port=$smtp_server_port\\001auth=Bearer $token\\001\\001\";\n+}\n+\n+sub generate_xoauth2_string {\n+\t# \"user=\" {User} \"^Aauth=Bearer \" {Access Token} \"^A^A\"\n+\t# https://developers.google.com/gmail/imap/xoauth2-protocol#initial_client_response\n+\tmy $username = shift;\n+\tmy $token = shift;\n+\treturn \"user=$username\\001auth=Bearer $token\\001\\001\";\n+}\n+\n+sub smtp_bearer_auth {\n+\tmy $username = shift;\n+\tmy $token = shift;\n+\tmy $auth_string;\n+\tif ($smtp_encryption ne \"tls\") {\n+\t\t# As described in RFC7628 TLS is required and will be enforced\n+\t\t# at this point.\n+\t\t#\n+\t\t# https://datatracker.ietf.org/doc/html/rfc7628#section-3\n+\t\tdie sprintf(__(\"For %s TLS is required.\"), $smtp_auth);\n+\t}\n+\tif ($smtp_auth eq \"OAUTHBEARER\") {\n+\t\t$auth_string = generate_oauthbearer_string($username, $token);\n+\t} elsif ($smtp_auth eq \"XOAUTH2\") {\n+\t\t$auth_string = generate_xoauth2_string($username, $token);\n+\t}\n+\tmy $encoded_auth_string = MIME::Base64::encode($auth_string, \"\");\n+\t$smtp->command(\"AUTH $smtp_auth $encoded_auth_string\\r\\n\");\n+\tuse Net::Cmd qw(CMD_OK);\n+\tif ($smtp->response() == CMD_OK){\n+\t\treturn 1;\n+\t} else {\n+\t\t# Send dummy request on authentication failure according to rfc7628.\n+\t\t# https://datatracker.ietf.org/doc/html/rfc7628#section-3.2.3\n+\t\t$smtp->command(MIME::Base64::encode(\"\\001\"));\n+\t\t$smtp->response();\n+\t\treturn 0;\n+\t}\n+}\n+\n # Returns 1 if authentication succeeded or was not necessary\n # (smtp_user was not specified), and 0 otherwise.\n \n@@ -1436,7 +1493,12 @@ sub smtp_auth_maybe {\n \n \t\t# catch all SMTP auth error in a unified eval block\n \t\teval {\n-\t\t\tif ($smtp_auth) {\n+\t\t\tif (defined $smtp_auth && ($smtp_auth eq \"OAUTHBEARER\" || $smtp_auth eq \"XOAUTH2\")) {\n+\t\t\t\t# Since Authen:SASL does not support XOAUTH2 nor OAUTHBEARER we will\n+\t\t\t\t# manually authenticate for these types. The password field should\n+\t\t\t\t# contain the auth token at this point.\n+\t\t\t\t$result = smtp_bearer_auth($cred->{'username'}, $cred->{'password'});\n+\t\t\t} elsif ($smtp_auth) {\n \t\t\t\tmy $sasl = Authen::SASL->new(\n \t\t\t\t\tmechanism => $smtp_auth,\n \t\t\t\t\tcallback => {\n-- \n2.49.0\n\n"},{"id":"516507","messageId":"PN3PR01MB9597D68A4E7A570949672832B8BB2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","threadId":"63330","inReplyTo":"PN3PR01MB95975C7925B6395DEC46F9EAB8BB2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","subject":"[PATCH v3 2/3] send-email: retrieve Message-ID from outlook SMTP server","fromName":"Aditya Garg","fromEmail":"gargaditya08@live.com","sentAt":"2025-04-22T15:23:20Z","receivedAt":"2025-04-22T15:26:40Z","isPatch":true,"sender":{"key":"gargaditya08@live.com","avatar":"https://avatars.githubusercontent.com/u/85610623?v=4"},"body":"Outlook does not accept the Message-ID header in the email body. Instead\nit saves it in its own proprietary X-Microsoft-Original-Message-ID\nheader and a random Message-ID is set my the server. As a result,\nreplying to threads does not work.\n\nThe $smtp->message variable in this script for outlook is something like\nthis:\n\n2.0.0 OK [Hostname=Some-hostname]\n\nThis contains the Message-ID set by Microsoft in the first <>.\n\nThis patch retrieves the Message-ID from this server response\nand sets it in the email headers instead of using the self generated one.\n\nSigned-off-by: Aditya Garg \n---\n git-send-email.perl | 11 +++++++++++\n 1 file changed, 11 insertions(+)\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex a6cafda29c..216b23caa5 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -1799,6 +1799,17 @@ sub send_message {\n \t\t\t$smtp->datasend(\"$line\") or die $smtp->message;\n \t\t}\n \t\t$smtp->dataend() or die $smtp->message;\n+\n+\t\t# Retrieve the Message-ID from the server response in case of Outlook\n+\t\tif ($smtp_server eq 'smtp.office365.com' || $smtp_server eq 'smtp-mail.outlook.com') {\n+\t\t\tif ($smtp->message =~ /<([^>]+)>/) {\n+\t\t\t\t$message_id = \"<$1>\";\n+\t\t\t\tprintf __(\"Outlook: Retrieved Message-ID: %s\\n\"), $message_id;\n+\t\t\t} else {\n+\t\t\t\twarn __(\"Warning: Could not retrieve Message-ID from server response.\\n\");\n+\t\t\t}\n+\t\t}\n+\n \t\t$smtp->code =~ /250|200/ or die sprintf(__(\"Failed to send %s\\n\"), $subject).$smtp->message;\n \t}\n \tif ($quiet) {\n-- \n2.49.0\n\n"},{"id":"516508","messageId":"PN3PR01MB9597813722A475923ED1C036B8BB2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","threadId":"63330","inReplyTo":"PN3PR01MB95975C7925B6395DEC46F9EAB8BB2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","subject":"[PATCH v3 3/3] send-email: add option to generate passswords like OAuth2 tokens","fromName":"Aditya Garg","fromEmail":"gargaditya08@live.com","sentAt":"2025-04-22T15:23:21Z","receivedAt":"2025-04-22T15:26:42Z","isPatch":true,"sender":{"key":"gargaditya08@live.com","avatar":"https://avatars.githubusercontent.com/u/85610623?v=4"},"body":"Some email providers like outlook allow only OAuth2 tokens to be used\nfor authentication. This commit adds an option to generate OAuth2 tokens\nusing scripts like M365-IMAP[1]. This option is similar to passwordeval\nin msmtp.\n\nExample usage:\n\n[sendemail]\n smtpEncryption = tls\n smtpServer = smtp.office365.com\n smtpUser = someone@outlook.com\n smtpServerPort = 587\n smtpauth = XOAUTH2\n smtpPassEval = cd /workspaces/codespaces-blank/M365-IMAP && python3 ./refresh_token.py\n\nSigned-off-by: Aditya Garg \n---\n Documentation/git-send-email.adoc | 8 ++++++++\n git-send-email.perl | 15 +++++++++++++++\n 2 files changed, 23 insertions(+)\n\ndiff --git a/Documentation/git-send-email.adoc b/Documentation/git-send-email.adoc\nindex 1bf75c060d..b41c807960 100644\n--- a/Documentation/git-send-email.adoc\n+++ b/Documentation/git-send-email.adoc\n@@ -230,6 +230,14 @@ or on the command line. If a username has been specified (with\n specified (with `--smtp-pass` or `sendemail.smtpPass`), then\n a password is obtained using 'git-credential'.\n \n+--smtp-passeval[=]::\n+\tGenerate password or OAuth2 token for SMTP AUTH. The argument is\n+\toptional. If specified, it will use the output of any password\n+\tor OAuth2 token generated using the command specified.\n++\n+Note that it will override any existing password specified using\n+`--smtp-user` or a `sendemail.smtpUser`.\n+\n --no-smtp-auth::\n \tDisable SMTP authentication. Short hand for `--smtp-auth=none`\n \ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex 216b23caa5..49de6a048e 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -59,6 +59,8 @@ sub usage {\n --smtp-server-port * Outgoing SMTP server port.\n --smtp-user * Username for SMTP-AUTH.\n --smtp-pass * Password for SMTP-AUTH; not necessary.\n+ --smtp-passeval * Path to script or a command to generate\n+ password like OAuth2 token for SMTP-AUTH.\n --smtp-encryption * tls or ssl; anything else disables.\n --smtp-ssl * Deprecated. Use '--smtp-encryption ssl'.\n --smtp-ssl-cert-path * Path to ca-certificates (either directory or file).\n@@ -280,6 +282,7 @@ sub do_edit {\n my ($auto_8bit_encoding);\n my ($compose_encoding);\n my ($sendmail_cmd);\n+my ($smtp_authpasseval);\n my ($mailmap_file, $mailmap_blob);\n # Variables with corresponding config settings & hardcoded defaults\n my ($debug_net_smtp) = 0;\t\t# Net::SMTP, see send_message()\n@@ -316,6 +319,7 @@ sub do_edit {\n \"smtppass\" => \\$smtp_authpass,\n \"smtpdomain\" => \\$smtp_domain,\n \"smtpauth\" => \\$smtp_auth,\n+ \"smtppasseval\" => \\$smtp_authpasseval,\n \"smtpbatchsize\" => \\$batch_size,\n \"smtprelogindelay\" => \\$relogin_delay,\n \"to\" => \\@config_to,\n@@ -516,6 +520,7 @@ sub config_regexp {\n \t\t \"smtp-server-port=s\" => \\$smtp_server_port,\n \t\t \"smtp-user=s\" => \\$smtp_authuser,\n \t\t \"smtp-pass:s\" => \\$smtp_authpass,\n+\t\t \"smtp-passeval=s\" => \\$smtp_authpasseval,\n \t\t \"smtp-ssl\" => sub { $smtp_encryption = 'ssl' },\n \t\t \"smtp-encryption=s\" => \\$smtp_encryption,\n \t\t \"smtp-ssl-cert-path=s\" => \\$smtp_ssl_cert_path,\n@@ -1463,6 +1468,16 @@ sub smtp_auth_maybe {\n \t\treturn 1;\n \t}\n \n+\t# If smtpPassEval is set, run the user specified command to get the password\n+\tif (defined $smtp_authpasseval) {\n+\t\tprintf __(\"Executing smtpPassEval: %s\\n\"), $smtp_authpasseval;\n+\t\tchomp(my $generated_password = `$smtp_authpasseval 2>&1`);\n+\t\tif ($? != 0) {\n+\t\t\tdie sprintf(__(\"Failed to execute smtpPassEval: %s\\n\"), $smtp_authpasseval);\n+\t\t}\n+\t\t$smtp_authpass = $generated_password;\n+\t}\n+\n \t# Workaround AUTH PLAIN/LOGIN interaction defect\n \t# with Authen::SASL::Cyrus\n \teval {\n-- \n2.49.0\n\n"},{"id":"516536","messageId":"xmqqmsc73l8g.fsf@gitster.g","threadId":"63330","inReplyTo":"PN3PR01MB9597D68A4E7A570949672832B8BB2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","subject":"Re: [PATCH v3 2/3] send-email: retrieve Message-ID from outlook SMTP server","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-22T22:05:51Z","receivedAt":"2025-04-22T22:05:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Aditya Garg writes:\n\n> Outlook does not accept the Message-ID header in the email body. Instead\n> it saves it in its own proprietary X-Microsoft-Original-Message-ID\n> header and a random Message-ID is set my the server. As a result,\n> replying to threads does not work.\n>\n> The $smtp->message variable in this script for outlook is something like\n> this:\n>\n> 2.0.0 OK [Hostname=Some-hostname]\n>\n> This contains the Message-ID set by Microsoft in the first <>.\n>\n> This patch retrieves the Message-ID from this server response\n> and sets it in the email headers instead of using the self generated one.\n\nHmph.\n\nsend_message calls gen_header as the first thing. This prepares the\nusual From:/To:/Subject:/Date:/Message-ID: lines and returns the\nheader text as well as recipient addresses broken out into different\nclasses, among other things.\n\n> Signed-off-by: Aditya Garg \n> ---\n> git-send-email.perl | 11 +++++++++++\n> 1 file changed, 11 insertions(+)\n>\n> diff --git a/git-send-email.perl b/git-send-email.perl\n> index a6cafda29c..216b23caa5 100755\n> --- a/git-send-email.perl\n> +++ b/git-send-email.perl\n> @@ -1799,6 +1799,17 @@ sub send_message {\n\nAnd before these pre-context lines, the composed $header that\ncontains the message_id has already been sent by calling datasend()\nmethod on the smtp object. After that, we are ...\n\n> \t\t\t$smtp->datasend(\"$line\") or die $smtp->message;\n> \t\t}\n\n... sending the body of the e-mail here.\n\nSo it is not clear to me how ovewriting the $message_id variable\nafter the message with $header with \"Message-ID:\" line that\ncontained the ID we generated has already given to the SMTP server.\nWhat the code is doing certainly contradicts with what the proposed\nlog message explains it does, i.e.\n\n\t... sets it in the email headers instead of using ...\n\nIt would affect the message-ID that is used by subsequent messages\nwhen they are sent as replies to this message. I do not think we\ncomputed the header (the In-Reply-To: field) for the next message\nat this point of the code, and I can well believe that mucking with\nthe $message variable at this point would make the next message\ncorrectly a response to this one.\n\nPerhaps you meant that Outlook DISCARDS the Message-ID: field in the\nmessage it was instructed to send out, and INSERTS its own? Then I\ncan see how this patch would improve the situation, but the last\nparagraph in the proposed log message needs to be rewritten.\n\n\n\tAfter sending a message, retrieve the message-ID the Outlook\n\tserver assigned to the message and store it in $message_id\n\tvariable; this value will be used when next and subsequent\n\tmessage are sent as replies to the message, preserving the\n\tthreading of the messages.\n\nor something.\n\n> +\t\t# Retrieve the Message-ID from the server response in case of Outlook\n> +\t\tif ($smtp_server eq 'smtp.office365.com' || $smtp_server eq 'smtp-mail.outlook.com') {\n\nPerhaps a small helper sub\n\n\tsub is_outlook {\n\t\tmy ($host) = @_;\n\t\treturn ($host eq '...' ||\n\t\t\t$host eq '...');\n\t}\n\nwould make it easier to maintain (and read).\n\n> +\t\t\tif ($smtp->message =~ /<([^>]+)>/) {\n> +\t\t\t\t$message_id = \"<$1>\";\n> +\t\t\t\tprintf __(\"Outlook: Retrieved Message-ID: %s\\n\"), $message_id;\n\nIs this worth reporting, or more or less a leftover debugging aid?\n\nThe fact that we retrieved (as opposed to \"could not retrieve\" case)\nmay have been significant during the development of this feature,\nbut to end-users who see this message, retrieved is not at all\ninteresting. What is interesting to them is that the server gave\nyour message some random Message-ID you've never heard of, so while\nit may make sense to report what that ID is, perhaps \n\n\tOutlook reassigned Messsage-ID: %s\n\nmight be more meaningful to them. rewritten, reassigned, used,\nthere may be even better verbs, but the point is that the subject of\nthat sentence is the outlook smtp serveron the other end of the\nconnection (as opposed to the actor of \"retrived\" is the program\nrunning on our end).\n\n> +\t\t\t} else {\n> +\t\t\t\twarn __(\"Warning: Could not retrieve Message-ID from server response.\\n\");\n> +\t\t\t}\n> +\t\t}\n> +\n> \t\t$smtp->code =~ /250|200/ or die sprintf(__(\"Failed to send %s\\n\"), $subject).$smtp->message;\n> \t}\n> \tif ($quiet) {\n"},{"id":"516537","messageId":"xmqqh62f3kji.fsf@gitster.g","threadId":"63330","inReplyTo":"PN3PR01MB9597813722A475923ED1C036B8BB2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","subject":"Re: [PATCH v3 3/3] send-email: add option to generate passswords like OAuth2 tokens","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-22T22:20:49Z","receivedAt":"2025-04-22T22:20:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Aditya Garg writes:\n\n> +--smtp-passeval[=]::\n> +\tGenerate password or OAuth2 token for SMTP AUTH. The argument is\n> +\toptional. If specified, it will use the output of any password\n> +\tor OAuth2 token generated using the command specified.\n> ++\n> +Note that it will override any existing password specified using\n> +`--smtp-user` or a `sendemail.smtpUser`.\n\nIf the argument is optional, we should explain what the behaviour is\nwhen the optional argument is omitted, as well as how the given\nargument is used. You are doing only the latter, but not the former.\n\nShouldn't the \"command\" be mandatory, if the option is used? I do\nnot quite see how these invocations\n\n\tgit-send-email ... --smtp-passeval ...\n\tgit-send-email ... --smtp-passeval= ...\n\nthat do not specify the command to be used is useful.\n"},{"id":"516550","messageId":"PN3PR01MB95972EA90559852F9F8328C3B8BA2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","threadId":"63330","inReplyTo":"xmqqh62f3kji.fsf@gitster.g","subject":"Re: [PATCH v3 3/3] send-email: add option to generate passswords like OAuth2 tokens","fromName":"Aditya Garg","fromEmail":"gargaditya08@live.com","sentAt":"2025-04-23T03:10:58Z","receivedAt":"2025-04-23T03:11:03Z","isPatch":true,"sender":{"key":"gargaditya08@live.com","avatar":"https://avatars.githubusercontent.com/u/85610623?v=4"},"body":"\n\n> On 23 Apr 2025, at 3:50 AM, Junio C Hamano wrote:\n> \n> Aditya Garg writes:\n> \n>> +--smtp-passeval[=]::\n>> + Generate password or OAuth2 token for SMTP AUTH. The argument is\n>> + optional. If specified, it will use the output of any password\n>> + or OAuth2 token generated using the command specified.\n>> ++\n>> +Note that it will override any existing password specified using\n>> +`--smtp-user` or a `sendemail.smtpUser`.\n> \n> If the argument is optional, we should explain what the behaviour is\n> when the optional argument is omitted, as well as how the given\n> argument is used. You are doing only the latter, but not the former.\n> \n\nArgument is not optional. It's a left over line I had copied from above for the sake of formatting. My bad here.\n\n> Shouldn't the \"command\" be mandatory, if the option is used? I do\n> not quite see how these invocations\n> \n> git-send-email ... --smtp-passeval ...\n> git-send-email ... --smtp-passeval= ...\n> \n> that do not specify the command to be used is useful.\n"},{"id":"516551","messageId":"PN3PR01MB9597D275C6E76AD7938230F9B8BA2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","threadId":"63330","inReplyTo":"xmqqmsc73l8g.fsf@gitster.g","subject":"Re: [PATCH v3 2/3] send-email: retrieve Message-ID from outlook SMTP server","fromName":"Aditya Garg","fromEmail":"gargaditya08@live.com","sentAt":"2025-04-23T03:22:07Z","receivedAt":"2025-04-23T03:22:12Z","isPatch":true,"sender":{"key":"gargaditya08@live.com","avatar":"https://avatars.githubusercontent.com/u/85610623?v=4"},"body":"\n\n> On 23 Apr 2025, at 3:36 AM, Junio C Hamano wrote:\n> \n> Aditya Garg writes:\n> \n>> Outlook does not accept the Message-ID header in the email body. Instead\n>> it saves it in its own proprietary X-Microsoft-Original-Message-ID\n>> header and a random Message-ID is set my the server. As a result,\n>> replying to threads does not work.\n>> \n>> The $smtp->message variable in this script for outlook is something like\n>> this:\n>> \n>> 2.0.0 OK [Hostname=Some-hostname]\n>> \n>> This contains the Message-ID set by Microsoft in the first <>.\n>> \n>> This patch retrieves the Message-ID from this server response\n>> and sets it in the email headers instead of using the self generated one.\n> \n> Hmph.\n> \n> send_message calls gen_header as the first thing. This prepares the\n> usual From:/To:/Subject:/Date:/Message-ID: lines and returns the\n> header text as well as recipient addresses broken out into different\n> classes, among other things.\n> \n>> Signed-off-by: Aditya Garg \n>> ---\n>> git-send-email.perl | 11 +++++++++++\n>> 1 file changed, 11 insertions(+)\n>> \n>> diff --git a/git-send-email.perl b/git-send-email.perl\n>> index a6cafda29c..216b23caa5 100755\n>> --- a/git-send-email.perl\n>> +++ b/git-send-email.perl\n>> @@ -1799,6 +1799,17 @@ sub send_message {\n> \n> And before these pre-context lines, the composed $header that\n> contains the message_id has already been sent by calling datasend()\n> method on the smtp object. After that, we are ...\n> \n>> $smtp->datasend(\"$line\") or die $smtp->message;\n>> }\n> \n> ... sending the body of the e-mail here.\n> \n> So it is not clear to me how ovewriting the $message_id variable\n> after the message with $header with \"Message-ID:\" line that\n> contained the ID we generated has already given to the SMTP server.\n> What the code is doing certainly contradicts with what the proposed\n> log message explains it does, i.e.\n> \n> ... sets it in the email headers instead of using ...\n> \n> It would affect the message-ID that is used by subsequent messages\n> when they are sent as replies to this message. I do not think we\n> computed the header (the In-Reply-To: field) for the next message\n> at this point of the code, and I can well believe that mucking with\n> the $message variable at this point would make the next message\n> correctly a response to this one.\n> \n> Perhaps you meant that Outlook DISCARDS the Message-ID: field in the\n> message it was instructed to send out, and INSERTS its own? Then I\n> can see how this patch would improve the situation, but the last\n> paragraph in the proposed log message needs to be rewritten.\n\nExactly. We don't care what message id the script is sending here. We just\nsend the first mail, and then retrieve the message ID that was set by outlook.\n\nThen we change the variable so that In-reply-to and References work properly.\n\n> After sending a message, retrieve the message-ID the Outlook\n> server assigned to the message and store it in $message_id\n> variable; this value will be used when next and subsequent\n> message are sent as replies to the message, preserving the\n> threading of the messages.\n> \n> or something.\n> \n\nI'll change the log message, although it seemed clear to me.\n\n>> + # Retrieve the Message-ID from the server response in case of Outlook\n>> + if ($smtp_server eq 'smtp.office365.com' || $smtp_server eq 'smtp-mail.outlook.com') {\n> \n> Perhaps a small helper sub\n> \n> sub is_outlook {\n> my ($host) = @_;\n> return ($host eq '...' ||\n> $host eq '...');\n> }\n> \n> would make it easier to maintain (and read).\n\nOk\n\n> \n>> + if ($smtp->message =~ /<([^>]+)>/) {\n>> + $message_id = \"<$1>\";\n>> + printf __(\"Outlook: Retrieved Message-ID: %s\\n\"), $message_id;\n> \n> Is this worth reporting, or more or less a leftover debugging aid?\n> \n> The fact that we retrieved (as opposed to \"could not retrieve\" case)\n> may have been significant during the development of this feature,\n> but to end-users who see this message, retrieved is not at all\n> interesting. What is interesting to them is that the server gave\n> your message some random Message-ID you've never heard of, so while\n> it may make sense to report what that ID is, perhaps\n> \n> Outlook reassigned Messsage-ID: %s\n> \n> might be more meaningful to them. rewritten, reassigned, used,\n> there may be even better verbs, but the point is that the subject of\n> that sentence is the outlook smtp serveron the other end of the\n> connection (as opposed to the actor of \"retrived\" is the program\n> running on our end).\n\nI'll change the message, and having some message about this IMO\nshould be there.\n> \n>> + } else {\n>> + warn __(\"Warning: Could not retrieve Message-ID from server response.\\n\");\n>> + }\n>> + }\n>> +\n>> $smtp->code =~ /250|200/ or die sprintf(__(\"Failed to send %s\\n\"), $subject).$smtp->message;\n>> }\n>> if ($quiet) {\n"},{"id":"516608","messageId":"xmqqo6wm28ok.fsf@gitster.g","threadId":"63330","inReplyTo":"PN3PR01MB9597D275C6E76AD7938230F9B8BA2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","subject":"Re: [PATCH v3 2/3] send-email: retrieve Message-ID from outlook SMTP server","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-23T15:34:35Z","receivedAt":"2025-04-23T15:34:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Aditya Garg writes:\n\n>> On 23 Apr 2025, at 3:36 AM, Junio C Hamano wrote:\n>> \n>> Aditya Garg writes:\n>> \n>>> Outlook does not accept the Message-ID header in the email body. Instead\n>>> it saves it in its own proprietary X-Microsoft-Original-Message-ID\n>>> header and a random Message-ID is set my the server. As a result,\n>>> replying to threads does not work.\n>>> \n>>> The $smtp->message variable in this script for outlook is something like\n>>> this:\n>>> \n>>> 2.0.0 OK [Hostname=Some-hostname]\n>>> \n>>> This contains the Message-ID set by Microsoft in the first <>.\n>>> \n>>> This patch retrieves the Message-ID from this server response\n>>> and sets it in the email headers instead of using the self generated one.\n>> \n>> Hmph.\n>> \n>> send_message calls gen_header as the first thing. This prepares the\n>> usual From:/To:/Subject:/Date:/Message-ID: lines and returns the\n>> header text as well as recipient addresses broken out into different\n>> classes, among other things.\n>> \n>>> Signed-off-by: Aditya Garg \n>>> ---\n>>> git-send-email.perl | 11 +++++++++++\n>>> 1 file changed, 11 insertions(+)\n>>> \n>>> diff --git a/git-send-email.perl b/git-send-email.perl\n>>> index a6cafda29c..216b23caa5 100755\n>>> --- a/git-send-email.perl\n>>> +++ b/git-send-email.perl\n>>> @@ -1799,6 +1799,17 @@ sub send_message {\n>> \n>> And before these pre-context lines, the composed $header that\n>> contains the message_id has already been sent by calling datasend()\n>> method on the smtp object. After that, we are ...\n>> \n>>> $smtp->datasend(\"$line\") or die $smtp->message;\n>>> }\n>> \n>> ... sending the body of the e-mail here.\n>> \n>> So it is not clear to me how ovewriting the $message_id variable\n>> after the message with $header with \"Message-ID:\" line that\n>> contained the ID we generated has already given to the SMTP server.\n>> What the code is doing certainly contradicts with what the proposed\n>> log message explains it does, i.e.\n>> \n>> ... sets it in the email headers instead of using ...\n>> \n>> It would affect the message-ID that is used by subsequent messages\n>> when they are sent as replies to this message. I do not think we\n>> computed the header (the In-Reply-To: field) for the next message\n>> at this point of the code, and I can well believe that mucking with\n>> the $message variable at this point would make the next message\n>> correctly a response to this one.\n>> \n>> Perhaps you meant that Outlook DISCARDS the Message-ID: field in the\n>> message it was instructed to send out, and INSERTS its own? Then I\n>> can see how this patch would improve the situation, but the last\n>> paragraph in the proposed log message needs to be rewritten.\n>\n> Exactly. We don't care what message id the script is sending here. We just\n> send the first mail, and then retrieve the message ID that was set by outlook.\n>\n>\n> Then we change the variable so that In-reply-to and References work properly.\n>\n>> After sending a message, retrieve the message-ID the Outlook\n>> server assigned to the message and store it in $message_id\n>> variable; this value will be used when next and subsequent\n>> message are sent as replies to the message, preserving the\n>> threading of the messages.\n>> \n>> or something.\n>> \n>\n> I'll change the log message, although it seemed clear to me.\n\nSounds good. What confused me was \"and sets it in the email\nheaders\" in the proposed log message looked as if it was referring\nto the Message-ID: field of the message we retrieved the ID Outlook\nwould assign to it, i.e. as if the order of events were (0) we grab\nan ID from Outlook, (1) we compose the header, embedding the ID in\nits Message-ID: field, (2) we send the result out. What happens in\nreality, if I understand you correctly, is that (0) we compose the\nheader, with our message-ID in its Message-ID: field, (1) Outlook\ndiscard our Message-ID: field, replaces it with its own, (2) we can\nlearn the ID Outlook used, and (3) we'll pretend as if that ID is\nwhat we gave the message we just sent in the first place, which\nwould be used in In-Reply-To: field of the messages sent as replies\nto it, but it was unclear from the description which one you meant.\n\nThanks.\n"},{"id":"516612","messageId":"PN3PR01MB959794B1F342A75D8E4FC559B8BA2@PN3PR01MB9597.INDPRD01.PROD.OUTLOOK.COM","threadId":"63330","inReplyTo":"xmqqo6wm28ok.fsf@gitster.g","subject":"Re: [PATCH v3 2/3] send-email: retrieve Message-ID from outlook SMTP server","fromName":"Aditya Garg","fromEmail":"gargaditya08@live.com","sentAt":"2025-04-23T16:24:06Z","receivedAt":"2025-04-23T16:24:11Z","isPatch":true,"sender":{"key":"gargaditya08@live.com","avatar":"https://avatars.githubusercontent.com/u/85610623?v=4"},"body":"\n\n> On 23 Apr 2025, at 9:04 PM, Junio C Hamano wrote:\n> \n> Aditya Garg writes:\n> \n>>>> On 23 Apr 2025, at 3:36 AM, Junio C Hamano wrote:\n>>> \n>>> Aditya Garg writes:\n>>> \n>>>> Outlook does not accept the Message-ID header in the email body. Instead\n>>>> it saves it in its own proprietary X-Microsoft-Original-Message-ID\n>>>> header and a random Message-ID is set my the server. As a result,\n>>>> replying to threads does not work.\n>>>> \n>>>> The $smtp->message variable in this script for outlook is something like\n>>>> this:\n>>>> \n>>>> 2.0.0 OK [Hostname=Some-hostname]\n>>>> \n>>>> This contains the Message-ID set by Microsoft in the first <>.\n>>>> \n>>>> This patch retrieves the Message-ID from this server response\n>>>> and sets it in the email headers instead of using the self generated one.\n>>> \n>>> Hmph.\n>>> \n>>> send_message calls gen_header as the first thing. This prepares the\n>>> usual From:/To:/Subject:/Date:/Message-ID: lines and returns the\n>>> header text as well as recipient addresses broken out into different\n>>> classes, among other things.\n>>> \n>>>> Signed-off-by: Aditya Garg \n>>>> ---\n>>>> git-send-email.perl | 11 +++++++++++\n>>>> 1 file changed, 11 insertions(+)\n>>>> \n>>>> diff --git a/git-send-email.perl b/git-send-email.perl\n>>>> index a6cafda29c..216b23caa5 100755\n>>>> --- a/git-send-email.perl\n>>>> +++ b/git-send-email.perl\n>>>> @@ -1799,6 +1799,17 @@ sub send_message {\n>>> \n>>> And before these pre-context lines, the composed $header that\n>>> contains the message_id has already been sent by calling datasend()\n>>> method on the smtp object. After that, we are ...\n>>> \n>>>> $smtp->datasend(\"$line\") or die $smtp->message;\n>>>> }\n>>> \n>>> ... sending the body of the e-mail here.\n>>> \n>>> So it is not clear to me how ovewriting the $message_id variable\n>>> after the message with $header with \"Message-ID:\" line that\n>>> contained the ID we generated has already given to the SMTP server.\n>>> What the code is doing certainly contradicts with what the proposed\n>>> log message explains it does, i.e.\n>>> \n>>> ... sets it in the email headers instead of using ...\n>>> \n>>> It would affect the message-ID that is used by subsequent messages\n>>> when they are sent as replies to this message. I do not think we\n>>> computed the header (the In-Reply-To: field) for the next message\n>>> at this point of the code, and I can well believe that mucking with\n>>> the $message variable at this point would make the next message\n>>> correctly a response to this one.\n>>> \n>>> Perhaps you meant that Outlook DISCARDS the Message-ID: field in the\n>>> message it was instructed to send out, and INSERTS its own? Then I\n>>> can see how this patch would improve the situation, but the last\n>>> paragraph in the proposed log message needs to be rewritten.\n>> \n>> Exactly. We don't care what message id the script is sending here. We just\n>> send the first mail, and then retrieve the message ID that was set by outlook.\n>> \n>> \n>> Then we change the variable so that In-reply-to and References work properly.\n>> \n>>> After sending a message, retrieve the message-ID the Outlook\n>>> server assigned to the message and store it in $message_id\n>>> variable; this value will be used when next and subsequent\n>>> message are sent as replies to the message, preserving the\n>>> threading of the messages.\n>>> \n>>> or something.\n>>> \n>> \n>> I'll change the log message, although it seemed clear to me.\n> \n> Sounds good. What confused me was \"and sets it in the email\n> headers\" in the proposed log message looked as if it was referring\n> to the Message-ID: field of the message we retrieved the ID Outlook\n> would assign to it, i.e. as if the order of events were (0) we grab\n> an ID from Outlook, (1) we compose the header, embedding the ID in\n> its Message-ID: field, (2) we send the result out. What happens in\n> reality, if I understand you correctly, is that (0) we compose the\n> header, with our message-ID in its Message-ID: field, (1) Outlook\n> discard our Message-ID: field, replaces it with its own, (2) we can\n> learn the ID Outlook used, and (3) we'll pretend as if that ID is\n> what we gave the message we just sent in the first place, which\n> would be used in In-Reply-To: field of the messages sent as replies\n> to it, but it was unclear from the description which one you meant.\n\nYou understood correctly. I've already sent a v4 with detailed description as well\n> \n> Thanks.\n"}]}