{"thread":{"id":"61935","subject":"Heads up: GMail regularly marking list messages as spam","startedAt":"2024-08-12T15:10:36Z","lastAt":"2024-08-14T07:32:52Z","messageCount":8,"participants":["Mike Castle","rsbecker@nexbridge.com","Konstantin Ryabitsev","brian m. carlson","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"500640","messageId":"CA+t9iMyT8fAR_fvQXOer=ivLnNKDnH8g_M8iQiq7gdrnfG7aCg@mail.gmail.com","threadId":"61935","inReplyTo":null,"subject":"Heads up: GMail regularly marking list messages as spam","fromName":"Mike Castle","fromEmail":"dalgoda@gmail.com","sentAt":"2024-08-12T15:10:23Z","receivedAt":"2024-08-12T15:10:36Z","isPatch":false,"sender":{"key":"dalgoda@gmail.com","avatar":null},"body":"If you read the list via GMail, you may want to regularly check your\nSpam label for messages from the git list.\n\nI'm not sure what's up with that.  A handful of real spam had made it\nthrough vger, but GMail seems to be hitting a fair number of false\npositives lately.\n\nSometimes it is just one message in a patch set.  Others, it is whole\nsets of patches.  I didn't check to see if there was anything in\ncommon across the messages.  Maybe if anyone has a large batch, they\ncan check?\n\nIt would be interesting to see if this affects domain specific users\nas well (e.g., Google itself).\n\nCheers,\nmrc\n"},{"id":"500642","messageId":"030501daeccc$0747f210$15d7d630$@nexbridge.com","threadId":"61935","inReplyTo":"CA+t9iMyT8fAR_fvQXOer=ivLnNKDnH8g_M8iQiq7gdrnfG7aCg@mail.gmail.com","subject":"RE: Heads up: GMail regularly marking list messages as spam","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-08-12T15:26:34Z","receivedAt":"2024-08-12T15:26:41Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Monday, August 12, 2024 11:10 AM, Mike Castle wrote:\n>If you read the list via GMail, you may want to regularly check your Spam label for\n>messages from the git list.\n>\n>I'm not sure what's up with that.  A handful of real spam had made it through vger,\n>but GMail seems to be hitting a fair number of false positives lately.\n>\n>Sometimes it is just one message in a patch set.  Others, it is whole sets of patches.\n>I didn't check to see if there was anything in common across the messages.  Maybe\n>if anyone has a large batch, they can check?\n>\n>It would be interesting to see if this affects domain specific users as well (e.g.,\n>Google itself).\n\nWhat I think you will find is that some spammers use this mailing list to send spam.\nMost of it gets blocked because it is in HTML or RTF. However, various honey-pots\nand organizations like SpamCop detect the mailing list servers as the source of spam\nso the servers get black-listed or marked as spam. Unless the servers themselves\nfilter out spam before redistributing it, this is going to keep happening.\n\n"},{"id":"500646","messageId":"20240812-devout-mindful-dogfish-81cefa@lemur","threadId":"61935","inReplyTo":"CA+t9iMyT8fAR_fvQXOer=ivLnNKDnH8g_M8iQiq7gdrnfG7aCg@mail.gmail.com","subject":"Re: Heads up: GMail regularly marking list messages as spam","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2024-08-12T16:04:24Z","receivedAt":"2024-08-12T16:04:25Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Mon, Aug 12, 2024 at 08:10:23AM GMT, Mike Castle wrote:\n> If you read the list via GMail, you may want to regularly check your\n> Spam label for messages from the git list.\n> \n> I'm not sure what's up with that.  A handful of real spam had made it\n> through vger, but GMail seems to be hitting a fair number of false\n> positives lately.\n\nGmail is using internal spam training on all mail that it receives.\nUnfortunately, mailing list traffic like that on the git list falls\nway outside what is \"normal,\" and this tends to trigger false-positives.\n\n> Sometimes it is just one message in a patch set.  Others, it is whole\n> sets of patches.  I didn't check to see if there was anything in\n> common across the messages.  Maybe if anyone has a large batch, they\n> can check?\n\nMost of the time it's super unhelpful:\n\n    Why is this message in spam?\n\n    It is similar to messages that were identified as spam in the past.\n\nSometimes it's something specific:\n\n    Why is this message in spam?\n\n    Lots of messages from xxxxx.msg.chrl.nc.charter.net were identified as\n    spam in the past.\n\nThis isn't really anything to do with vger.kernel.org -- we are complying with\nall Gmail \"bulk mail sender\" requirements and we try our best to deny spammers\nearly. Unfortunately, my general recommendation is not to use Gmail for this\nkind of work.\n\n-K\n"},{"id":"500663","messageId":"Zrp7yNKMxhPk6Tyt@tapette.crustytoothpaste.net","threadId":"61935","inReplyTo":"CA+t9iMyT8fAR_fvQXOer=ivLnNKDnH8g_M8iQiq7gdrnfG7aCg@mail.gmail.com","subject":"Re: Heads up: GMail regularly marking list messages as spam","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-08-12T21:16:56Z","receivedAt":"2024-08-12T21:16:58Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-08-12 at 15:10:23, Mike Castle wrote:\n> If you read the list via GMail, you may want to regularly check your\n> Spam label for messages from the git list.\n> \n> I'm not sure what's up with that.  A handful of real spam had made it\n> through vger, but GMail seems to be hitting a fair number of false\n> positives lately.\n> \n> Sometimes it is just one message in a patch set.  Others, it is whole\n> sets of patches.  I didn't check to see if there was anything in\n> common across the messages.  Maybe if anyone has a large batch, they\n> can check?\n> \n> It would be interesting to see if this affects domain specific users\n> as well (e.g., Google itself).\n\nGmail and Yahoo have both set up new requirements for senders[0].  This\nrequires that all senders have SPF and DKIM, and senders sending more\nthan 5000 messages per day have DMARC set up.  One-click unsubscribe in\nRFC 8058 is also obligatory, as is TLS for sending messages.  These\nchanges came into effect in February 2024.\n\nI don't believe vger uses DKIM (at least, it doesn't appear to add DKIM\nheaders) and doesn't support RFC 8058 one-click unsubscribe.  Perhaps\nadding support for those, as well as any other relevant requirements,\nwould improve things.\n\n[0] https://support.google.com/a/answer/81126?sjid=9729987135322685176-NC&visit_id=638590935924706038-3386441869&rd=1\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"500791","messageId":"20240813-sociable-fresh-whippet-3ee335@meerkat","threadId":"61935","inReplyTo":"Zrp7yNKMxhPk6Tyt@tapette.crustytoothpaste.net","subject":"Re: Heads up: GMail regularly marking list messages as spam","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2024-08-13T15:06:13Z","receivedAt":"2024-08-13T15:06:14Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Mon, Aug 12, 2024 at 09:16:56PM GMT, brian m. carlson wrote:\n> Gmail and Yahoo have both set up new requirements for senders[0].  This\n> requires that all senders have SPF and DKIM, and senders sending more\n> than 5000 messages per day have DMARC set up.  One-click unsubscribe in\n> RFC 8058 is also obligatory, as is TLS for sending messages.  These\n> changes came into effect in February 2024.\n> \n> I don't believe vger uses DKIM (at least, it doesn't appear to add DKIM\n> headers) and doesn't support RFC 8058 one-click unsubscribe.  Perhaps\n> adding support for those, as well as any other relevant requirements,\n> would improve things.\n\nThese guidelines are only applicable if vger was the origin of the messages,\nnot a forwarder. For forwarders, it is only required to add ARC signatures:\n\nhttps://support.google.com/mail/answer/175365?hl=en\n\nThe only thing we aren't doing is we're not adding X-Forwarded-To headers,\nbecause mlmmj doesn't yet support it (should be in 1.5).\n\n-K\n"},{"id":"500823","messageId":"Zrvbpt3Fkl8yjesP@tapette.crustytoothpaste.net","threadId":"61935","inReplyTo":"20240813-sociable-fresh-whippet-3ee335@meerkat","subject":"Re: Heads up: GMail regularly marking list messages as spam","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-08-13T22:18:14Z","receivedAt":"2024-08-13T22:18:23Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-08-13 at 15:06:13, Konstantin Ryabitsev wrote:\n> On Mon, Aug 12, 2024 at 09:16:56PM GMT, brian m. carlson wrote:\n> > Gmail and Yahoo have both set up new requirements for senders[0].  This\n> > requires that all senders have SPF and DKIM, and senders sending more\n> > than 5000 messages per day have DMARC set up.  One-click unsubscribe in\n> > RFC 8058 is also obligatory, as is TLS for sending messages.  These\n> > changes came into effect in February 2024.\n> > \n> > I don't believe vger uses DKIM (at least, it doesn't appear to add DKIM\n> > headers) and doesn't support RFC 8058 one-click unsubscribe.  Perhaps\n> > adding support for those, as well as any other relevant requirements,\n> > would improve things.\n> \n> These guidelines are only applicable if vger was the origin of the messages,\n> not a forwarder. For forwarders, it is only required to add ARC signatures:\n> \n> https://support.google.com/mail/answer/175365?hl=en\n\nThe article you linked still recommends DKIM.  In addition, the article\nI linked doesn't appear to limit itself to non-mailing lists in any way.\nIt says, \"email senders\", and a mailing list sends emails.  In fact, it\nexplicitly mentions \"[i]f you manage a forwarding service, including\nmailing lists or inbound gateways,\" so it still appears to cover both\ncases and apply to all senders, with only some additional requirements\napplying to forwarding services.\n\nIn general, adding DKIM (whether or not ARC is supported) _will_\ndramatically improve the deliverability of email in general, so it's a\nwin overall.  Adding one-click unsubscribe would also probably keep some\nof those unsubscribe messages from hitting the list since people could\nautomatically use the functionality of their mail client to unsubscribe.\n\nAnd from a personal point of view, it is definitely useful for me, as\nsomeone who manages their own mail server, to heavily negatively score\nall email without DKIM, but I can't do that because vger doesn't add\nany.  That's the only thing preventing me from doing that.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"500831","messageId":"20240813-rampant-quokka-from-betelgeuse-4bc3c9@lemur","threadId":"61935","inReplyTo":"Zrvbpt3Fkl8yjesP@tapette.crustytoothpaste.net","subject":"Re: Heads up: GMail regularly marking list messages as spam","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2024-08-14T00:10:17Z","receivedAt":"2024-08-14T00:10:21Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Tue, Aug 13, 2024 at 10:18:14PM GMT, brian m. carlson wrote:\n> The article you linked still recommends DKIM.  In addition, the article\n> I linked doesn't appear to limit itself to non-mailing lists in any way.\n> It says, \"email senders\", and a mailing list sends emails.  In fact, it\n> explicitly mentions \"[i]f you manage a forwarding service, including\n> mailing lists or inbound gateways,\" so it still appears to cover both\n> cases and apply to all senders, with only some additional requirements\n> applying to forwarding services.\n> \n> In general, adding DKIM (whether or not ARC is supported) _will_\n> dramatically improve the deliverability of email in general, so it's a\n> win overall. \n\nSorry, but this is not factually correct. Adding a DKIM signature to a message\nwhere the From: lists a different domain has no impact on anything at all when\nit comes to the chances of it being marked as Spam at google.\n\nE.g. your message has:\n\n    From: \"brian m. carlson\" <sandals@crustytoothpaste.net>\n\nAnd it already has a DKIM signature for your domain. Adding another DKIM\nsignature for vger.kernel.org will serve no purpose, because there is no\nalignment between \"vger.kernel.org\" and the domain in your From: address.\n\nIn fact, we do add a cryptographic signature, but in the ARC record:\nhttps://lore.kernel.org/git/Zrvbpt3Fkl8yjesP@tapette.crustytoothpaste.net/raw\n\nIt's from the overlord subspace.kernel.org domain, but it doesn't need to\nalign with \"vger.kernel.org\".\n\n> Adding one-click unsubscribe would also probably keep some\n> of those unsubscribe messages from hitting the list since people could\n> automatically use the functionality of their mail client to unsubscribe.\n\nThis is not possible without rewriting the From: header, which will break all\npatches. We cannot modify the message content or any of the main headers that\nare DKIM-signed (this bit is mentioned in the forwarder guidelines I linked\nto).\n\n> And from a personal point of view, it is definitely useful for me, as\n> someone who manages their own mail server, to heavily negatively score\n> all email without DKIM, but I can't do that because vger doesn't add\n> any.  That's the only thing preventing me from doing that.\n\nThere is a small subset of active git mailing list posters who send mail that\nis not DKIM-signed, for example Jeff King:\n\nhttps://lore.kernel.org/git/20240813124550.GC968816@coredump.intra.peff.net/raw\n\nHowever, signing their messages with a DKIM signature from vger.kernel.org\nwill serve no purpose, unless we also rewrite the From: to match the\nvger.kernel.org domain. The From: field would need to be something like:\n\n    From: Jeff King via Git Mailing List <foo+somemagic+peff.peff.net@vger.kernel.org>\n\nThis is called \"From-munging\" and is not acceptable on any mailing list that\nsends and receives patches, because this changes the Author of the patch.\n\nThing is, this has nothing to do with vger, because if Jeff emails you\ndirectly and you set a high negative score for messages without a DKIM\nsignature, you won't get his mail either. The fact that it traversed or didn't\ntraverse vger.kernel.org has no impact on that.\n\n-K\n"},{"id":"500868","messageId":"20240814073251.GB2077794@coredump.intra.peff.net","threadId":"61935","inReplyTo":"20240813-rampant-quokka-from-betelgeuse-4bc3c9@lemur","subject":"Re: Heads up: GMail regularly marking list messages as spam","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-08-14T07:32:51Z","receivedAt":"2024-08-14T07:32:52Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Aug 13, 2024 at 08:10:17PM -0400, Konstantin Ryabitsev wrote:\n\n> > And from a personal point of view, it is definitely useful for me, as\n> > someone who manages their own mail server, to heavily negatively score\n> > all email without DKIM, but I can't do that because vger doesn't add\n> > any.  That's the only thing preventing me from doing that.\n> \n> There is a small subset of active git mailing list posters who send mail that\n> is not DKIM-signed, for example Jeff King:\n> \n> https://lore.kernel.org/git/20240813124550.GC968816@coredump.intra.peff.net/raw\n\nUh oh, now I feel singled out. ;)\n\nIt's true that I use only SPF but not DKIM. FWIW, I haven't had a huge\nproblem with deliverability. But if the world is ready to move on and I\nam one of the last dinosaurs holding things up, I'm OK to be told that I\nreally ought to implement DKIM to participate in the list.\n\n> However, signing their messages with a DKIM signature from vger.kernel.org\n> will serve no purpose, unless we also rewrite the From: to match the\n> vger.kernel.org domain. The From: field would need to be something like:\n> \n>     From: Jeff King via Git Mailing List <foo+somemagic+peff.peff.net@vger.kernel.org>\n> \n> This is called \"From-munging\" and is not acceptable on any mailing list that\n> sends and receives patches, because this changes the Author of the patch.\n> \n> Thing is, this has nothing to do with vger, because if Jeff emails you\n> directly and you set a high negative score for messages without a DKIM\n> signature, you won't get his mail either. The fact that it traversed or didn't\n> traverse vger.kernel.org has no impact on that.\n\nYep, agreed with all of that.\n\n-Peff\n"}]}