{"thread":{"id":"63471","subject":"Small patch to add support for MPTCP on Linux","startedAt":"2025-05-16T17:56:17Z","lastAt":"2025-05-22T11:12:46Z","messageCount":14,"participants":["Muhammad Nuzaihan","brian m. carlson","Phillip Wood","Junio C Hamano","Matthieu Baerts"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"518274","messageId":"JH8DWS.72DKHPTI873H3@unrealasia.net","threadId":"63471","inReplyTo":null,"subject":"Small patch to add support for MPTCP on Linux","fromName":"Muhammad Nuzaihan","fromEmail":"zaihan@unrealasia.net","sentAt":"2025-05-16T17:56:07Z","receivedAt":"2025-05-16T17:56:17Z","isPatch":false,"sender":{"key":"zaihan@unrealasia.net","avatar":"https://gravatar.com/avatar/ec76e57992cf391eef4176d86e3cc48967bfe9842ee754a2ce0ac59c58308ed9?d=mp&s=160"},"body":"\nPatch to enable the use of MPTCP on Linux (when available)\n\nIPPROTO_MPTCP v1 (not the old v0) has been improved to go about the\nlimitations of middleboxes.\n\nMPTCP protocol is an extension of vanilla TCP which enables multiple\nIP to aggregate bandwidth at layer 4 of the OSI stack across\nas said IP(s).\n\nSimilar to link aggregation which works at layer 2. MPTCP works on top\nof IP layer.\n\nOther than aggregating bandwidth, MPTCP also allows seamless failover\nwhen one network path (not just link) is down (or having high latency)\nby reinjecting the packets to a path that is available.\n\nThis patch enables IPPROTO_MPTCP if IPPROTO_MPTCP is available and\nuses plain TCP if the Linux system does not support it.\n\nSigned-off-by: Muhammad Nuzaihan Bin Kamal Luddin \n<zaihan@unrealasia.net>\n\n\n\ndiff --git a/connect.c b/connect.c\nindex 3280435331..8473f0b02e 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -827,8 +827,11 @@ static int git_tcp_connect_sock(char *host, int flags)\n \telse if (flags & CONNECT_IPV6)\n \t\thints.ai_family = AF_INET6;\n \thints.ai_socktype = SOCK_STREAM;\n-\thints.ai_protocol = IPPROTO_TCP;\n-\n+#ifdef IPPROTO_MPTCP\n+        hints.ai_protocol = IPPROTO_MPTCP;\n+#else\n+        hints.ai_protocol = IPPROTO_TCP;\n+#endif\n \tif (flags & CONNECT_VERBOSE)\n \t\tfprintf(stderr, _(\"Looking up %s ... \"), host);\n \n"},{"id":"518310","messageId":"aCeg_wjLCf0Sz_7X@tapette.crustytoothpaste.net","threadId":"63471","inReplyTo":"JH8DWS.72DKHPTI873H3@unrealasia.net","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-05-16T20:33:03Z","receivedAt":"2025-05-16T20:33:10Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-05-16 at 17:56:07, Muhammad Nuzaihan wrote:\n> \n> Patch to enable the use of MPTCP on Linux (when available)\n> \n> IPPROTO_MPTCP v1 (not the old v0) has been improved to go about the\n> limitations of middleboxes.\n> \n> MPTCP protocol is an extension of vanilla TCP which enables multiple\n> IP to aggregate bandwidth at layer 4 of the OSI stack across\n> as said IP(s).\n> \n> Similar to link aggregation which works at layer 2. MPTCP works on top\n> of IP layer.\n> \n> Other than aggregating bandwidth, MPTCP also allows seamless failover\n> when one network path (not just link) is down (or having high latency)\n> by reinjecting the packets to a path that is available.\n> \n> This patch enables IPPROTO_MPTCP if IPPROTO_MPTCP is available and\n> uses plain TCP if the Linux system does not support it.\n\nWhat happens here if I compile this on a system that has a kernel that\nsupports MPTCP but then switch to one that does not?  The reason I ask\nis that I have worked at places where we shipped binaries, including\nGit, based on a standard CentOS or RHEL system, but then some people\nused our software on a system with a very stripped down kernel (in some\ncases, where IPv6 was not even compiled in) because doing so meant that\nthey could make about $5 more per server per month.\n\nDo the operating systems which support MPTCP make it a compulsory part\nof the TCP stack, or could we end up with cases where we're unable to\nconnect here?\n\nIn addition, Wikipedia mentions that FreeBSD has only IPv4 support, but\nI don't know if that's up to date.  What happens if we run on a system\nwhere MPTCP is used, but it doesn't work with IPv6 and the only remote\nIP is IPv6?  Do we fall back properly, or do things fail?\n\nI ask these questions not because I'm opposed to this feature but\nbecause I want to be sure we don't accidentally break things for users.\nI know that for instance Go 1.24 enabled MPTCP and that ended up causing\nproblems in some environments, so I would recommend that we make this a\nconfigurable option instead.  We can definitely default to MPTCP, but we\nprobably need an option to fall back.\n\nOf course, this code path is only used by the unauthenticated Git\nprotocol usually run on port 9418, which practically nobody uses anymore\n(because it lacks the privacy, integrity, and authentication which are\nnecessary and prudent on the modern Internet), so maybe nobody cares\nabout edge cases there.  My guess, though, is that the people most\nlikely to be using something that isn't HTTPS or SSH are also the people\nmost likely to be using odd or unusual configurations, so we may very\nwell want to add an option for them.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"518324","messageId":"BP9EWS.WTYEEEQZEN2U1@unrealasia.net","threadId":"63471","inReplyTo":"aCeg_wjLCf0Sz_7X@tapette.crustytoothpaste.net","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Muhammad Nuzaihan","fromEmail":"zaihan@unrealasia.net","sentAt":"2025-05-17T07:19:59Z","receivedAt":"2025-05-17T07:20:17Z","isPatch":false,"sender":{"key":"zaihan@unrealasia.net","avatar":"https://gravatar.com/avatar/ec76e57992cf391eef4176d86e3cc48967bfe9842ee754a2ce0ac59c58308ed9?d=mp&s=160"},"body":"Hi Brian.\n\nOn Fri, May 16 2025 at 08:33:03 PM +0000, brian m. carlson \n<sandals@crustytoothpaste.net> wrote:\n> On 2025-05-16 at 17:56:07, Muhammad Nuzaihan wrote:\n>> \n>>  Patch to enable the use of MPTCP on Linux (when available)\n>> \n>>  IPPROTO_MPTCP v1 (not the old v0) has been improved to go about the\n>>  limitations of middleboxes.\n>> \n>>  MPTCP protocol is an extension of vanilla TCP which enables multiple\n>>  IP to aggregate bandwidth at layer 4 of the OSI stack across\n>>  as said IP(s).\n>> \n>>  Similar to link aggregation which works at layer 2. MPTCP works on \n>> top\n>>  of IP layer.\n>> \n>>  Other than aggregating bandwidth, MPTCP also allows seamless \n>> failover\n>>  when one network path (not just link) is down (or having high \n>> latency)\n>>  by reinjecting the packets to a path that is available.\n>> \n>>  This patch enables IPPROTO_MPTCP if IPPROTO_MPTCP is available and\n>>  uses plain TCP if the Linux system does not support it.\n> \n> What happens here if I compile this on a system that has a kernel that\n> supports MPTCP but then switch to one that does not?  The reason I ask\n> is that I have worked at places where we shipped binaries, including\n> Git, based on a standard CentOS or RHEL system, but then some people\n> used our software on a system with a very stripped down kernel (in \n> some\n> cases, where IPv6 was not even compiled in) because doing so meant \n> that\n> they could make about $5 more per server per month.\n> \nMPTCP supports *both* IPv4 and IPv6. Don't tell me people would also \nremove\neven IPv4 as well? I had written an #ifdef statement to check if \nIPPROTO_MPTCP\nexists and enables that.\n\n\n> Do the operating systems which support MPTCP make it a compulsory part\n> of the TCP stack, or could we end up with cases where we're unable to\n> connect here?\n> \n> In addition, Wikipedia mentions that FreeBSD has only IPv4 support, \n> but\n> I don't know if that's up to date.  What happens if we run on a system\n> where MPTCP is used, but it doesn't work with IPv6 and the only remote\n> IP is IPv6?  Do we fall back properly, or do things fail?\n\nThis patch *specifically* targets Linux to check if IPPROTO_MPTCP exists\nin the Linux system. I think you have not read my initial patch \ndescription\nproperly nor even read about the new changes for MPTCP.\n\nMPTCP support is now officially in the mainline kernel and not \nout-of-tree.\n\nThis *current* implementation of MPTCP is v1 and not v0 (v0 had \nproblems and\nv1 already solved the issue with middleboxes. again, please read my \npatch\ndescription properly)\n\nPlease read up on how MPTCP falls back to regular TCP if it could not \nconnect\nusing MPTCP.\n> \n> I ask these questions not because I'm opposed to this feature but\n> because I want to be sure we don't accidentally break things for \n> users.\n> \nI'm not sure but you have not even bothered to read the documentation \nabout MPTCP.\n> I know that for instance Go 1.24 enabled MPTCP and that ended up \n> causing\n> problems in some environments, so I would recommend that we make this \n> a\n> configurable option instead.  We can definitely default to MPTCP, but \n> we\n> probably need an option to fall back.\nMPTCP v1 (again i am repeating myself) and not the old MPTCP v0 does \nthe fallback\nmore effectively.\n\nDo you know of any references that mentions that Go 1.24 with MPTCP \nenabled\n(normally this is the current MPTCP v1) is causing the issues?\n\nIf you could give me evidences of such issues, maybe i can reconsider \nit again.\n> \n> Of course, this code path is only used by the unauthenticated Git\n> protocol usually run on port 9418, which practically nobody uses \n> anymore\n> (because it lacks the privacy, integrity, and authentication which are\n> necessary and prudent on the modern Internet), so maybe nobody cares\n> about edge cases there.  My guess, though, is that the people most\n> likely to be using something that isn't HTTPS or SSH are also the \n> people\n> most likely to be using odd or unusual configurations, so we may very\n> well want to add an option for them.\n\nAgain, the unauthenticated Git protocol is the *most basic* setup that \nanyone\ncan use to test MPTCP out. I understand from your point of view but it \ndoes\nnot make sense to support ssh and http when the most basic git protocol \nis\nnot supported.\n\ngit protocol is the *most basic* protocol. For ssh and https that would \nfall\nunder other project's implementing (like openssh or apache)\n\nI would consider adding an option to read from .gitconfig to enable \nMPTCP\nwhere i can leave MPTCP disabled by default.\n\nBut what you explained about the downsides of MPTCP (without evidences)\nand not even implementing MPTCP for git protocol does not make sense.\n\nRegards,\nZaihan\n> --\n> brian m. carlson (they/them)\n> Toronto, Ontario, CA\n\n\n"},{"id":"518325","messageId":"MPDEWS.NYG9NTW5G6LQ@unrealasia.net","threadId":"63471","inReplyTo":"CAOYsWhkMb8hxjnYRTgAb269N=e-Vyw10Go5M=RA-8PyCjXPttA@mail.gmail.com","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Muhammad Nuzaihan","fromEmail":"zaihan@unrealasia.net","sentAt":"2025-05-17T08:46:34Z","receivedAt":"2025-05-17T08:46:50Z","isPatch":false,"sender":{"key":"zaihan@unrealasia.net","avatar":"https://gravatar.com/avatar/ec76e57992cf391eef4176d86e3cc48967bfe9842ee754a2ce0ac59c58308ed9?d=mp&s=160"},"body":"Hi all,\n\nI've made the git daemon to work (for the git protocol), as attached\nin the wireshark screenshot.\nAs i had mentioned that the most basic implementation of the protocol \nshould be\nthe basic git protocol to test the implementation and not git over ssh \nor git over http.\n\n\nUnless if everyone wants to deprecate the git protocol entirely then it\nmight be a different issue.\n\nBut i think there need to be a discussion as i have mentioned this \nthread\nabout the concerns being raised by others.\n\nThanks,\nZaihan\n\nOn Sat, May 17 2025 at 10:21:00 AM +0300, Hridoy Ahmed \n<ariyanhridoy130@gmail.com> wrote:\n> \n> \n> On Sat, 17 May 2025 at 10:20 AM Muhammad Nuzaihan \n> <zaihan@unrealasia.net> wrote:\n>> Hi Brian.\n>> \n>>  On Fri, May 16 2025 at 08:33:03 PM +0000, brian m. carlson\n>>  <sandals@crustytoothpaste.net> wrote:\n>>  > On 2025-05-16 at 17:56:07, Muhammad Nuzaihan wrote:\n>>  >>\n>>  >>  Patch to enable the use of MPTCP on Linux (when available)\n>>  >>\n>>  >>  IPPROTO_MPTCP v1 (not the old v0) has been improved to go about \n>> the\n>>  >>  limitations of middleboxes.\n>>  >>\n>>  >>  MPTCP protocol is an extension of vanilla TCP which enables \n>> multiple\n>>  >>  IP to aggregate bandwidth at layer 4 of the OSI stack across\n>>  >>  as said IP(s).\n>>  >>\n>>  >>  Similar to link aggregation which works at layer 2. MPTCP works \n>> on\n>>  >> top\n>>  >>  of IP layer.\n>>  >>\n>>  >>  Other than aggregating bandwidth, MPTCP also allows seamless\n>>  >> failover\n>>  >>  when one network path (not just link) is down (or having high\n>>  >> latency)\n>>  >>  by reinjecting the packets to a path that is available.\n>>  >>\n>>  >>  This patch enables IPPROTO_MPTCP if IPPROTO_MPTCP is available \n>> and\n>>  >>  uses plain TCP if the Linux system does not support it.\n>>  >\n>>  > What happens here if I compile this on a system that has a kernel \n>> that\n>>  > supports MPTCP but then switch to one that does not?  The reason \n>> I ask\n>>  > is that I have worked at places where we shipped binaries, \n>> including\n>>  > Git, based on a standard CentOS or RHEL system, but then some \n>> people\n>>  > used our software on a system with a very stripped down kernel (in\n>>  > some\n>>  > cases, where IPv6 was not even compiled in) because doing so meant\n>>  > that\n>>  > they could make about $5 more per server per month.\n>>  >\n>>  MPTCP supports *both* IPv4 and IPv6. Don't tell me people would also\n>>  remove\n>>  even IPv4 as well? I had written an #ifdef statement to check if\n>>  IPPROTO_MPTCP\n>>  exists and enables that.\n>> \n>> \n>>  > Do the operating systems which support MPTCP make it a compulsory \n>> part\n>>  > of the TCP stack, or could we end up with cases where we're \n>> unable to\n>>  > connect here?\n>>  >\n>>  > In addition, Wikipedia mentions that FreeBSD has only IPv4 \n>> support,\n>>  > but\n>>  > I don't know if that's up to date.  What happens if we run on a \n>> system\n>>  > where MPTCP is used, but it doesn't work with IPv6 and the only \n>> remote\n>>  > IP is IPv6?  Do we fall back properly, or do things fail?\n>> \n>>  This patch *specifically* targets Linux to check if IPPROTO_MPTCP \n>> exists\n>>  in the Linux system. I think you have not read my initial patch\n>>  description\n>>  properly nor even read about the new changes for MPTCP.\n>> \n>>  MPTCP support is now officially in the mainline kernel and not\n>>  out-of-tree.\n>> \n>>  This *current* implementation of MPTCP is v1 and not v0 (v0 had\n>>  problems and\n>>  v1 already solved the issue with middleboxes. again, please read my\n>>  patch\n>>  description properly)\n>> \n>>  Please read up on how MPTCP falls back to regular TCP if it could \n>> not\n>>  connect\n>>  using MPTCP.\n>>  >\n>>  > I ask these questions not because I'm opposed to this feature but\n>>  > because I want to be sure we don't accidentally break things for\n>>  > users.\n>>  >\n>>  I'm not sure but you have not even bothered to read the \n>> documentation\n>>  about MPTCP.\n>>  > I know that for instance Go 1.24 enabled MPTCP and that ended up\n>>  > causing\n>>  > problems in some environments, so I would recommend that we make \n>> this\n>>  > a\n>>  > configurable option instead.  We can definitely default to MPTCP, \n>> but\n>>  > we\n>>  > probably need an option to fall back.\n>>  MPTCP v1 (again i am repeating myself) and not the old MPTCP v0 does\n>>  the fallback\n>>  more effectively.\n>> \n>>  Do you know of any references that mentions that Go 1.24 with MPTCP\n>>  enabled\n>>  (normally this is the current MPTCP v1) is causing the issues?\n>> \n>>  If you could give me evidences of such issues, maybe i can \n>> reconsider\n>>  it again.\n>>  >\n>>  > Of course, this code path is only used by the unauthenticated Git\n>>  > protocol usually run on port 9418, which practically nobody uses\n>>  > anymore\n>>  > (because it lacks the privacy, integrity, and authentication \n>> which are\n>>  > necessary and prudent on the modern Internet), so maybe nobody \n>> cares\n>>  > about edge cases there.  My guess, though, is that the people most\n>>  > likely to be using something that isn't HTTPS or SSH are also the\n>>  > people\n>>  > most likely to be using odd or unusual configurations, so we may \n>> very\n>>  > well want to add an option for them.\n>> \n>>  Again, the unauthenticated Git protocol is the *most basic* setup \n>> that\n>>  anyone\n>>  can use to test MPTCP out. I understand from your point of view but \n>> it\n>>  does\n>>  not make sense to support ssh and http when the most basic git \n>> protocol\n>>  is\n>>  not supported.\n>> \n>>  git protocol is the *most basic* protocol. For ssh and https that \n>> would\n>>  fall\n>>  under other project's implementing (like openssh or apache)\n>> \n>>  I would consider adding an option to read from .gitconfig to enable\n>>  MPTCP\n>>  where i can leave MPTCP disabled by default.\n>> \n>>  But what you explained about the downsides of MPTCP (without \n>> evidences)\n>>  and not even implementing MPTCP for git protocol does not make \n>> sense.\n>> \n>>  Regards,\n>>  Zaihan\n>>  > --\n>>  > brian m. carlson (they/them)\n>>  > Toronto, Ontario, CA\n>> \n>> \n>> \n\n"},{"id":"518326","messageId":"aChhxRx7sMD47N_s@tapette.crustytoothpaste.net","threadId":"63471","inReplyTo":"BP9EWS.WTYEEEQZEN2U1@unrealasia.net","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-05-17T10:15:33Z","receivedAt":"2025-05-17T10:15:36Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-05-17 at 07:19:59, Muhammad Nuzaihan wrote:\n> Hi Brian.\n> \n> On Fri, May 16 2025 at 08:33:03 PM +0000, brian m. carlson\n> <sandals@crustytoothpaste.net> wrote:\n> > What happens here if I compile this on a system that has a kernel that\n> > supports MPTCP but then switch to one that does not?  The reason I ask\n> > is that I have worked at places where we shipped binaries, including\n> > Git, based on a standard CentOS or RHEL system, but then some people\n> > used our software on a system with a very stripped down kernel (in some\n> > cases, where IPv6 was not even compiled in) because doing so meant that\n> > they could make about $5 more per server per month.\n> > \n> MPTCP supports *both* IPv4 and IPv6. Don't tell me people would also remove\n> even IPv4 as well? I had written an #ifdef statement to check if\n> IPPROTO_MPTCP\n> exists and enables that.\n\nI provide this as an example of people compiling even \"essential\"\nfeatures out of their kernel.  The question remains: if I compile on,\nsay, Debian, which has this, and then I switch to the same version of\nDebian, but with a custom kernel that removes MPTCP from the kernel\ncompletely, does this change continue to work, or do we end up with an\nEINVAL from the `socket` call?\n\nI want to point out that the kernel and libc headers used to compile a\nbinary need not reflect the actual code in the running kernel.  With the\nadvent of containers, people frequently run a different operating system\ninside a container than they do outside a container and thus we need to\nconsider all of the possible combinations.\n\n> > Do the operating systems which support MPTCP make it a compulsory part\n> > of the TCP stack, or could we end up with cases where we're unable to\n> > connect here?\n> > \n> > In addition, Wikipedia mentions that FreeBSD has only IPv4 support, but\n> > I don't know if that's up to date.  What happens if we run on a system\n> > where MPTCP is used, but it doesn't work with IPv6 and the only remote\n> > IP is IPv6?  Do we fall back properly, or do things fail?\n> \n> This patch *specifically* targets Linux to check if IPPROTO_MPTCP exists\n> in the Linux system. I think you have not read my initial patch description\n> properly nor even read about the new changes for MPTCP.\n\nGit runs on lots of operating systems, not just Linux.  If the case is\nthat the `IPPROTO_MPTCP` #define is only ever available on Linux and no\nother operating system ever ships that option or ever will, then that's\nfine, but the commit message needs to say that.  I know that many\noperating systems ship MPTCP, so I'm going to ask about how this works\non some non-Linux systems because your commit message didn't explain\nthat to me.\n\n> Please read up on how MPTCP falls back to regular TCP if it could not\n> connect using MPTCP.\n\nAgain, your patch tells me how things work on Linux.  I am interested in\npatches that work across a variety of other operating systems as well.\n\n> > I ask these questions not because I'm opposed to this feature but\n> > because I want to be sure we don't accidentally break things for users.\n> > \n> I'm not sure but you have not even bothered to read the documentation about\n> MPTCP.\n\nOn the Git list, we try not to assume that everyone has read all of the\ntechnical documentation about a subject and instead we explain, at a\nhigh level, how the change is and how it's supposed to work.  Your\ncommit message should convince me (and everyone else, especially Junio,\nthe maintainer) that your change is valuable and should be applied.\n\n> > I know that for instance Go 1.24 enabled MPTCP and that ended up causing\n> > problems in some environments, so I would recommend that we make this a\n> > configurable option instead.  We can definitely default to MPTCP, but we\n> > probably need an option to fall back.\n> MPTCP v1 (again i am repeating myself) and not the old MPTCP v0 does the\n> fallback\n> more effectively.\n> \n> Do you know of any references that mentions that Go 1.24 with MPTCP enabled\n> (normally this is the current MPTCP v1) is causing the issues?\n\nI know that there were circumstances in which there could be kernel\npanics or similar problems with it enabled[0].  I haven't heard of\nactual network problems, though.  Since most people were previously not\nusing MPTCP and Go 1.24 enabled it by default, upgrading to that version\ncaused some people's systems to panic under load.\n\nI do think that enabling features that cause Git to induce a kernel\npanic or the like, even though that's a bug in the kernel, should be\nconfigurable.\n\n> But what you explained about the downsides of MPTCP (without evidences)\n> and not even implementing MPTCP for git protocol does not make sense.\n\nI'm not arguing any downsides of MPTCP.  I'm stating that we have a\nlarge variety of platforms that have to be supported and you haven't\nexplained how this works or will work anywhere other than Linux; that\nthere are people who compile out important features from their kernel\nand, though that is improvident, we should probably not break Git for\nthem; and that we should be careful about enabling features which have\nbeen known to cause system problems.\n\n[0] https://www.wiz.io/vulnerability-database/cve/cve-2022-49198\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"518328","messageId":"4YPEWS.J5JRNETKLXF1@unrealasia.net","threadId":"63471","inReplyTo":"aChhxRx7sMD47N_s@tapette.crustytoothpaste.net","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Muhammad Nuzaihan","fromEmail":"zaihan@unrealasia.net","sentAt":"2025-05-17T13:10:52Z","receivedAt":"2025-05-17T13:11:07Z","isPatch":false,"sender":{"key":"zaihan@unrealasia.net","avatar":"https://gravatar.com/avatar/ec76e57992cf391eef4176d86e3cc48967bfe9842ee754a2ce0ac59c58308ed9?d=mp&s=160"},"body":"\n\nOn Sat, May 17 2025 at 10:15:33 AM +0000, brian m. carlson \n<sandals@crustytoothpaste.net> wrote:\n> On 2025-05-17 at 07:19:59, Muhammad Nuzaihan wrote:\n>>  Hi Brian.\n>> \n>>  On Fri, May 16 2025 at 08:33:03 PM +0000, brian m. carlson\n>>  <sandals@crustytoothpaste.net> wrote:\n>>  > What happens here if I compile this on a system that has a kernel \n>> that\n>>  > supports MPTCP but then switch to one that does not?  The reason \n>> I ask\n>>  > is that I have worked at places where we shipped binaries, \n>> including\n>>  > Git, based on a standard CentOS or RHEL system, but then some \n>> people\n>>  > used our software on a system with a very stripped down kernel \n>> (in some\n>>  > cases, where IPv6 was not even compiled in) because doing so \n>> meant that\n>>  > they could make about $5 more per server per month.\n>>  >\n>>  MPTCP supports *both* IPv4 and IPv6. Don't tell me people would \n>> also remove\n>>  even IPv4 as well? I had written an #ifdef statement to check if\n>>  IPPROTO_MPTCP\n>>  exists and enables that.\n> \n> I provide this as an example of people compiling even \"essential\"\n> features out of their kernel.  The question remains: if I compile on,\n> say, Debian, which has this, and then I switch to the same version of\n> Debian, but with a custom kernel that removes MPTCP from the kernel\n> completely, does this change continue to work, or do we end up with an\n> EINVAL from the `socket` call?\n> \n> I want to point out that the kernel and libc headers used to compile a\n> binary need not reflect the actual code in the running kernel.  With \n> the\n> advent of containers, people frequently run a different operating \n> system\n> inside a container than they do outside a container and thus we need \n> to\n> consider all of the possible combinations.\n\nIn that case, i'll add a check for the OS that git is built on with \n\"defined(__linux__)\"\nif that helps.\n\nAlso another check if a socket is supported by looking for a return \nvalue of\n\"EAI_SOCKTYPE\" (not EINVAL) and fallback to regular TCP if that is \nreturned.\n\nEAI_SOCKTYPE should work across different UNIX systems as this is a \nposix error code.\n\nMPTCP has been in development for the last 15 years and the major \nchange/overhaul (MPTCP v1)\noccured in 2020 and now is accepted in Linux mainline kernel.\n\nI am working on this git code change as i have large git repositories \nwith about 50 gigabytes\nof code and i have multiple WAN links which i can aggregate bandwidth \nacross and even\nwhen one path (even in between my CPE router to internet) is down, i \nwill not\nget interrupted.\n\nAlso i am using a Linux laptop that has WiFi and 5G module. So this kind\nof adds my drive of adding support for git (on Linux)\n\nMPTCP helps in situations when one of my WAN links have a high latency \nand\nautomatically choose a link with a path with less latency.\n\nMPTCP aggregates the MPTCP connection by using subflows where two or \nmore\nlinks can be utilised with subflows. A single flow of data can have \nmultiple\nsubflows across different IP interfaces and thus increases network\nthroughput.\n\nApple for example had been using MPTCP for their cloud services since \nMPTCP v0 which had\nissues (not MPTCP v1) since 2013.\n\nCompared to MultiPath QUIC which is still years away from being \nimplemented.\n\nThe main issue back then with MPTCP v0 was middleboxes such as \nfirewalls and NAT gateways\nthat discards TCP options header which is crucial when using MPTCP.\n\n> \n>>  > Do the operating systems which support MPTCP make it a compulsory \n>> part\n>>  > of the TCP stack, or could we end up with cases where we're \n>> unable to\n>>  > connect here?\n>>  >\n>>  > In addition, Wikipedia mentions that FreeBSD has only IPv4 \n>> support, but\n>>  > I don't know if that's up to date.  What happens if we run on a \n>> system\n>>  > where MPTCP is used, but it doesn't work with IPv6 and the only \n>> remote\n>>  > IP is IPv6?  Do we fall back properly, or do things fail?\n>> \n>>  This patch *specifically* targets Linux to check if IPPROTO_MPTCP \n>> exists\n>>  in the Linux system. I think you have not read my initial patch \n>> description\n>>  properly nor even read about the new changes for MPTCP.\n> \n> Git runs on lots of operating systems, not just Linux.  If the case is\n> that the `IPPROTO_MPTCP` #define is only ever available on Linux and \n> no\n> other operating system ever ships that option or ever will, then \n> that's\n> fine, but the commit message needs to say that.  I know that many\n> operating systems ship MPTCP, so I'm going to ask about how this works\n> on some non-Linux systems because your commit message didn't explain\n> that to me.\n> \n>>  Please read up on how MPTCP falls back to regular TCP if it could \n>> not\n>>  connect using MPTCP.\n> \n> Again, your patch tells me how things work on Linux.  I am interested \n> in\n> patches that work across a variety of other operating systems as well.\n\nMy main focus is Linux so i will add a check if it's built on a Linux \nmachine.\n\nmacOS would be a later focus but it's not a priority for now. I would \navoid\nadding MPTCP on other systems such as FreeBSD as their implementation\nfor example is still considered experimental.\n> \n>>  > I ask these questions not because I'm opposed to this feature but\n>>  > because I want to be sure we don't accidentally break things for \n>> users.\n>>  >\n>>  I'm not sure but you have not even bothered to read the \n>> documentation about\n>>  MPTCP.\n> \n> On the Git list, we try not to assume that everyone has read all of \n> the\n> technical documentation about a subject and instead we explain, at a\n> high level, how the change is and how it's supposed to work.  Your\n> commit message should convince me (and everyone else, especially \n> Junio,\n> the maintainer) that your change is valuable and should be applied.\n\nIt's just a small trival amount of code but anyway.\n\nI will email my latest patch in a separate email.\n\nIn my latest code i added checks for the OS it's built on \ndefined(__linux__) and if IPPROTO_MPTCP is\ndefined. Additional checks for error if EAI_SOCKTYPE is returned, it \nwill revert to\nregular IPPROTO_TCP (regular TCP)\n> \n>>  > I know that for instance Go 1.24 enabled MPTCP and that ended up \n>> causing\n>>  > problems in some environments, so I would recommend that we make \n>> this a\n>>  > configurable option instead.  We can definitely default to MPTCP, \n>> but we\n>>  > probably need an option to fall back.\n>>  MPTCP v1 (again i am repeating myself) and not the old MPTCP v0 \n>> does the\n>>  fallback\n>>  more effectively.\n>> \n>>  Do you know of any references that mentions that Go 1.24 with MPTCP \n>> enabled\n>>  (normally this is the current MPTCP v1) is causing the issues?\n> \n> I know that there were circumstances in which there could be kernel\n> panics or similar problems with it enabled[0].  I haven't heard of\n> actual network problems, though.  Since most people were previously \n> not\n> using MPTCP and Go 1.24 enabled it by default, upgrading to that \n> version\n> caused some people's systems to panic under load.\n> \nMy initial patch deals with *client* side of git. Not the *server* end \nof git (like daemon.c).\n\nThe crash that was reported was about the network pressure of the \nsoftware that\nruns as on a *server*.\n\nBut nevertheless it might still impact the client although the CVE does \nnot state\nthat.\n\nLook, i'm really under an impression you didn't look at the patch that \nsays the code\nchange is in \"connect.c\" and not \"daemon.c\". If you look closer it does \nnot have to do\nwith server side of things.\n> \n> I do think that enabling features that cause Git to induce a kernel\n> panic or the like, even though that's a bug in the kernel, should be\n> configurable.\nI've also added a flag for the git-daemon (git daemon.c code is a new \ncode).\n\nthe flag would be `--mptcp` which can be enabled on the git-daemon \nserver\nside.\n\nExample: ./git-daemon --reuseaddr --base-path=/all/repos/here \n--export-all --mptcp\n> \n>>  But what you explained about the downsides of MPTCP (without \n>> evidences)\n>>  and not even implementing MPTCP for git protocol does not make \n>> sense.\n> \n> I'm not arguing any downsides of MPTCP.  I'm stating that we have a\n> large variety of platforms that have to be supported and you haven't\n> explained how this works or will work anywhere other than Linux; that\n> there are people who compile out important features from their kernel\n> and, though that is improvident, we should probably not break Git for\n> them; and that we should be careful about enabling features which have\n> been known to cause system problems.\n\nGot it. I'll be more informed the next time. Anyway, i'll pass some \nlinks\nthat you might be interested.\n\nhttps://www.mptcp.dev/faq.html#mptcpv0-vs-mptcpv1\nhttps://www.mptcp.dev/faq.html#what-about-middleboxes\n\n> \n> [0] https://www.wiz.io/vulnerability-database/cve/cve-2022-49198\n> --\n> brian m. carlson (they/them)\n> Toronto, Ontario, CA\n\n\n"},{"id":"518330","messageId":"a76dda61-f60c-4221-83db-5e165a2478b1@gmail.com","threadId":"63471","inReplyTo":"4YPEWS.J5JRNETKLXF1@unrealasia.net","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-05-17T13:39:38Z","receivedAt":"2025-05-17T13:39:43Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 17/05/2025 14:10, Muhammad Nuzaihan wrote:\n>> I want to point out that the kernel and libc headers used to compile a\n>> binary need not reflect the actual code in the running kernel.  With the\n>> advent of containers, people frequently run a different operating system\n>> inside a container than they do outside a container and thus we need to\n>> consider all of the possible combinations.\n> \n> In that case, i'll add a check for the OS that git is built on with \n> \"defined(__linux__)\"\n> if that helps.\n\nAs brian has already said I think it would be better to have a Makefile \nknob to control this which defaults to being on for linux. Take a look \nat the various USE_xxx definitions in the Makefile and config.mak.uname \nfor setting default compile flags for different operating systems.\n\n> Also another check if a socket is supported by looking for a return \n> value of\n> \"EAI_SOCKTYPE\" (not EINVAL) and fallback to regular TCP if that is \n> returned.\n> \n> EAI_SOCKTYPE should work across different UNIX systems as this is a \n> posix error code.\n\nThat error is not mentioned in the documentation for MCTCP on Linux [1]. \nPlease make sure your code checks for the errno values described in the \ndocumentation.\n\n>> On the Git list, we try not to assume that everyone has read all of the\n>> technical documentation about a subject and instead we explain, at a\n>> high level, how the change is and how it's supposed to work.  Your\n>> commit message should convince me (and everyone else, especially Junio,\n>> the maintainer) that your change is valuable and should be applied.\n> \n> It's just a small trival amount of code but anyway.\nThat maybe so but please make sure that the commit message explains the \nreason for this change - what the advantages and disadvantages of using \nMPTCP are and what steps you have taken to make sure git continues to \nwork on systems that do not support MPTCP.\n\nThanks\n\nPhillip\n\n[1] \nhttps://www.kernel.org/doc/html/next/networking/mptcp.html#creating-mptcp-sockets\n"},{"id":"518456","messageId":"xmqqo6vokvpv.fsf@gitster.g","threadId":"63471","inReplyTo":"a76dda61-f60c-4221-83db-5e165a2478b1@gmail.com","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-05-19T23:49:00Z","receivedAt":"2025-05-19T23:49:03Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood <phillip.wood123@gmail.com> writes:\n\n> As brian has already said I think it would be better to have a\n> Makefile knob to control this which defaults to being on for\n> linux. Take a look at the various USE_xxx definitions in the Makefile\n> and config.mak.uname for setting default compile flags for different\n> operating systems.\n>\n>> Also another check if a socket is supported by looking for a return\n>> value of\n>> \"EAI_SOCKTYPE\" (not EINVAL) and fallback to regular TCP if that is\n>> returned.\n>> EAI_SOCKTYPE should work across different UNIX systems as this is a\n>> posix error code.\n>\n> That error is not mentioned in the documentation for MCTCP on Linux\n> [1]. Please make sure your code checks for the errno values described\n> in the documentation.\n\nAlso according to RFC 6897, \"MPTCP is designed to be totally\nbackward compatible to applications\".  I understand that this is\nquite unlike introducing IPv6 into IPv4-only world.  You can tell\nthe system that supports MPTCP to use it in specific ways by\nupdating your application, but your system's local policy may\nallow MPTCP to automatically set up multiple subflows even your\napplication is not quite aware of MPTCP.\n\nSo, ... I somehow would be mildly surprised if Git were a kind of\napplication that needs to take advantage of \"several additional\ndegrees of freedom that applications may wish to exploit\" by using\nAPI that is \"a simple extension of TCP's interface for MPTCP-aware\napplications\".  Requiring a simple application like ours to tweak\nand rebuild in today's world does not sound like a winning strategy\nto promote a technology that \"is designed to be totally backward\ncompatible to applications\", at least to me.\n\n\n"},{"id":"518466","messageId":"ZXLJWS.WPQLCXFNN8TH@unrealasia.net","threadId":"63471","inReplyTo":"xmqqo6vokvpv.fsf@gitster.g","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Muhammad Nuzaihan","fromEmail":"zaihan@unrealasia.net","sentAt":"2025-05-20T04:32:23Z","receivedAt":"2025-05-20T05:07:12Z","isPatch":false,"sender":{"key":"zaihan@unrealasia.net","avatar":"https://gravatar.com/avatar/ec76e57992cf391eef4176d86e3cc48967bfe9842ee754a2ce0ac59c58308ed9?d=mp&s=160"},"body":"\n\nOn Mon, May 19 2025 at 04:49:00 PM -0700, Junio C Hamano \n<gitster@pobox.com> wrote:\n> Phillip Wood <phillip.wood123@gmail.com> writes:\n> \n>>  As brian has already said I think it would be better to have a\n>>  Makefile knob to control this which defaults to being on for\n>>  linux. Take a look at the various USE_xxx definitions in the \n>> Makefile\n>>  and config.mak.uname for setting default compile flags for different\n>>  operating systems.\n>> \n>>>  Also another check if a socket is supported by looking for a return\n>>>  value of\n>>>  \"EAI_SOCKTYPE\" (not EINVAL) and fallback to regular TCP if that is\n>>>  returned.\n>>>  EAI_SOCKTYPE should work across different UNIX systems as this is a\n>>>  posix error code.\n>> \n>>  That error is not mentioned in the documentation for MCTCP on Linux\n>>  [1]. Please make sure your code checks for the errno values \n>> described\n>>  in the documentation.\n> \n> Also according to RFC 6897, \"MPTCP is designed to be totally\n> backward compatible to applications\".  I understand that this is\n> quite unlike introducing IPv6 into IPv4-only world.  You can tell\n> the system that supports MPTCP to use it in specific ways by\n> updating your application, but your system's local policy may\n> allow MPTCP to automatically set up multiple subflows even your\n> application is not quite aware of MPTCP.\n> \n> So, ... I somehow would be mildly surprised if Git were a kind of\n> application that needs to take advantage of \"several additional\n> degrees of freedom that applications may wish to exploit\" by using\n> API that is \"a simple extension of TCP's interface for MPTCP-aware\n> applications\".  Requiring a simple application like ours to tweak\n> and rebuild in today's world does not sound like a winning strategy\n> to promote a technology that \"is designed to be totally backward\n> compatible to applications\", at least to me.\n> \nTaking into scenario that i have WiFi access and regular Ethernet access\non my Laptop and i'm cloning or pulling a large set of code from git \n(which uses\nregular ethernet as primary interface and WiFi as secondary.)\n\nOn the regular TCP, my connection will reset when disconnecting \n(plugging off) from Ethernet\nand switching to WiFi but MPTCP solves that issue for me and allows \nuninterrupted\nwork on git cloning/pulling, especially when i have to work with huge \ncodebases.\n\nI think that's the relevant use-case for a laptop user working with git.\n\nApple had been using MPTCP for 12 years and their cloud services run on\nLinux servers with iOS clients and for that i can say that it's pretty \nmuch production ready.\n\nAnd it's not just Apple. Intel and RedHat had been involved and \nRedHat[1] pretty much\nare into MPTCP.\n\nHonestly, i love reading the git codebase as it is very simple and \nstraightforward and\ni think my previous patches were a bit too big and needs to be \nsimplified so,\n\nI have reached out to Matthieu Baerts <matttbe@kernel.org> who works on \nLinux MPTCP\non this issue and i've greatly simplified the code from his comments.\n(I've also added him to the loop in this email)\n\nThe patch i am attaching is a preview (still WIP) as i need some \nfeedback from the linux\nmptcp developers if my implementation is correct. But it's greatly \nsimplified to follow\ngit codebase's structure.\n\nChanged the part for the git server daemon to enable mptcp by \ndefault[0] and modified\nthe git client to use .gitconfig (global or per repository) with:\n\ngit config --global core.mptcp true (or a single repo without --global)\n\nwhich defaults to false (mptcp disabled for client) as per[0] and \nremoved client-side the env var configuration.\n\n\n[0] \nhttps://www.mptcp.dev/faq.html#why--when-should-mptcp-be-enabled-by-default \n(Thanks @matt!)\n[1] \nhttps://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/getting-started-with-multipath-tcp_configuring-and-managing-networking#proc_monitoring-mptcp-sub-flows_getting-started-with-multipath-tcp\n\n\n> \n\n\n\ndiff --git a/cache.h b/cache.h\nindex eba12487b9..6839b3acbc 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -944,6 +944,7 @@ extern int verify_ce_order;\n \n /* Environment bits from configuration mechanism */\n extern int trust_executable_bit;\n+extern int enable_mptcp;\n extern int trust_ctime;\n extern int check_stat;\n extern int quote_path_fully;\ndiff --git a/config.c b/config.c\nindex 2317a76696..833396aa4b 100644\n--- a/config.c\n+++ b/config.c\n@@ -1464,6 +1464,11 @@ static int git_default_core_config(const char *var, const char *value, void *cb)\n \tif (!strcmp(var, \"core.editor\"))\n \t\treturn git_config_string(&editor_program, var, value);\n \n+\tif (!strcmp(var, \"core.mptcp\")) {\n+\t\tenable_mptcp = git_config_bool(var, value);\n+\t\treturn 0;\n+\t}\n+\n \tif (!strcmp(var, \"core.commentchar\")) {\n \t\tif (!value)\n \t\t\treturn config_error_nonbool(var);\ndiff --git a/connect.c b/connect.c\nindex eaf7d6d261..ebeac99bd6 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -721,6 +721,16 @@ static void enable_keepalive(int sockfd)\n \t\terror_errno(_(\"unable to set SO_KEEPALIVE on socket\"));\n }\n \n+static const int needs_mptcp(void)\n+{\n+\tint mptcp = 0;\n+\n+\tif (git_config_get_bool(\"core.mptcp\", &mptcp))\n+\t\treturn mptcp;\n+\n+\treturn mptcp;\n+}\n+\n #ifndef NO_IPV6\n \n static const char *ai_name(const struct addrinfo *ai)\n@@ -770,7 +780,11 @@ static int git_tcp_connect_sock(char *host, int flags)\n \n \tfor (ai0 = ai; ai; ai = ai->ai_next, cnt++) {\n \t\tsockfd = socket(ai->ai_family,\n-\t\t\t\tai->ai_socktype, ai->ai_protocol);\n+\t\t\t\tai->ai_socktype,\n+#ifdef IPPROTO_MPTCP\n+\t\t\t\tneeds_mptcp() ? IPPROTO_MPTCP :\n+#endif\n+\t\t\t\tai->ai_protocol);\n \t\tif ((sockfd < 0) ||\n \t\t    (connect(sockfd, ai->ai_addr, ai->ai_addrlen) < 0)) {\n \t\t\tstrbuf_addf(&error_message, \"%s[%d: %s]: errno=%s\\n\",\n@@ -817,6 +831,7 @@ static int git_tcp_connect_sock(char *host, int flags)\n \tchar **ap;\n \tunsigned int nport;\n \tint cnt;\n+\tconst int needs_mptcp;\n \n \tget_host_and_port(&host, &port);\n \ndiff --git a/daemon.c b/daemon.c\nindex b1fcbe0d6f..08a16ccf03 100644\n--- a/daemon.c\n+++ b/daemon.c\n@@ -17,6 +17,7 @@ static enum log_destination {\n } log_destination = LOG_DESTINATION_UNSET;\n static int verbose;\n static int reuseaddr;\n+static int mptcp;\n static int informative_errors;\n \n static const char daemon_usage[] =\n@@ -1007,6 +1008,10 @@ static int setup_named_sock(char *listen_addr, int listen_port, struct socketlis\n \tfor (ai = ai0; ai; ai = ai->ai_next) {\n \t\tint sockfd;\n \n+#if defined(__linux__) && defined(IPPROTO_MPTCP)\n+\t\tsockfd = socket(ai->ai_family, ai->ai_socktype, IPPROTO_MPTCP);\n+\t\tif (sockfd < 0)\n+#endif\n \t\tsockfd = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol);\n \t\tif (sockfd < 0)\n \t\t\tcontinue;\n@@ -1360,6 +1365,10 @@ int cmd_main(int argc, const char **argv)\n \t\t\treuseaddr = 1;\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!strcmp(arg, \"--mptcp\")) {\n+\t\t\tmptcp = 1;\n+\t\t\tcontinue;\n+\t\t}\n \t\tif (!strcmp(arg, \"--user-path\")) {\n \t\t\tuser_path = \"\";\n \t\t\tcontinue;\ndiff --git a/environment.c b/environment.c\nindex 9da7f3c1a1..72f1adef6c 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -33,6 +33,7 @@ int warn_ambiguous_refs = 1;\n int warn_on_object_refname_ambiguity = 1;\n int repository_format_precious_objects;\n int repository_format_worktree_config;\n+int enable_mptcp;\n const char *git_commit_encoding;\n const char *git_log_output_encoding;\n char *apply_default_whitespace;\n"},{"id":"518485","messageId":"7b3b8efa-4cc1-4547-b66a-c469626eac46@kernel.org","threadId":"63471","inReplyTo":"xmqqo6vokvpv.fsf@gitster.g","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Matthieu Baerts","fromEmail":"matttbe@kernel.org","sentAt":"2025-05-20T10:54:41Z","receivedAt":"2025-05-20T10:54:46Z","isPatch":false,"sender":{"key":"matttbe@kernel.org","avatar":"https://gravatar.com/avatar/eeddb10cd8e676b4c7eda928eeaae8097b1df50c72fbbc3d535d2eee55a633d8?d=mp&s=160"},"body":"Hi Junio, Phillip, Muhammad, Brian,\n\nI'm part of the team maintaining MPTCP in the Linux kernel. Do not\nhesitate to reach me if you have any questions about MPTCP (I don't know\nif there were still opened questions in this email thread).\n\n@Muhammad: thank you for having initiated this email thread.\n\nOn 20/05/2025 01:49, Junio C Hamano wrote:\n> Phillip Wood <phillip.wood123@gmail.com> writes:\n> \n>> As brian has already said I think it would be better to have a\n>> Makefile knob to control this which defaults to being on for\n>> linux. Take a look at the various USE_xxx definitions in the Makefile\n>> and config.mak.uname for setting default compile flags for different\n>> operating systems.\n>>\n>>> Also another check if a socket is supported by looking for a return\n>>> value of\n>>> \"EAI_SOCKTYPE\" (not EINVAL) and fallback to regular TCP if that is\n>>> returned.\n>>> EAI_SOCKTYPE should work across different UNIX systems as this is a\n>>> posix error code.\n>>\n>> That error is not mentioned in the documentation for MCTCP on Linux\n>> [1]. Please make sure your code checks for the errno values described\n>> in the documentation.\n> \n> Also according to RFC 6897, \"MPTCP is designed to be totally\n> backward compatible to applications\".  I understand that this is\n> quite unlike introducing IPv6 into IPv4-only world.  You can tell\n> the system that supports MPTCP to use it in specific ways by\n> updating your application, but your system's local policy may\n> allow MPTCP to automatically set up multiple subflows even your\n> application is not quite aware of MPTCP.\n> \n> So, ... I somehow would be mildly surprised if Git were a kind of\n> application that needs to take advantage of \"several additional\n> degrees of freedom that applications may wish to exploit\" by using\n> API that is \"a simple extension of TCP's interface for MPTCP-aware\n> applications\".  Requiring a simple application like ours to tweak\n> and rebuild in today's world does not sound like a winning strategy\n> to promote a technology that \"is designed to be totally backward\n> compatible to applications\", at least to me.\n\n@Junio: Good point! This RFC 6897 was a bit optimistic I think. To get\nMPTCP in the upstream Linux kernel, we had to make it opt-in, and the\nmodifications we suggested couldn't impact \"plain\" TCP performances (or\nany other sockets). The previous implementation we were maintaining in a\nfork was following RFC 6897 guidelines, and there was no need to modify\nthe apps at all, but that was not realistic either.\n\nI then agree, this situation is different from the IPv6 vs IPv4 one, and\nMPTCP in the Linux kernel is using the same socket API as with TCP. It\nthen means that to support MPTCP, all you need to do is to create a\nsocket with a specific argument: IPPROTO_MPTCP instead of IPPROTO_TCP\nfor the protocol, that's it [1], the rest doesn't need to be modified.\n\n  socket(AF_INET(6), SOCK_STREAM, IPPROTO_MPTCP);\n\nKnowing that, it is then possible to change the behaviour of some apps\nby forcing them to create an MPTCP socket instead of a TCP one, e.g.\nusing LD_PRELOAD, and that's what \"mptcpize\" does, e.g.\n\n  mptcpize run git clone git://git.kernel.org/(...)\n\nThere are other techniques (eBPF, SystemTap, etc.) [2], but it sounds\nbetter to have a \"native\" support by modifying apps to change how\nsocket() is called, this modification should be minimal -- see\nMuhammad's last WIP patch [4] -- and MPTCP could be used only when\nneeded. That's what many apps are already doing [3]. (Also some\nsysadmins don't want to use other workarounds.)\n\n\n@Brian, Phillip, Muhammad, I think it is better not to set IPPROTO_MPTCP\nin arguments passed to getaddrinfo(), but modify what is given to the\nsocket() syscall. Something closed to what Muhammad suggested in his\nlast WIP patch [4]. I guess Muhammad will do a proper submission with\ngood commit messages when the new version will be ready and tested.\n\n\n[1] https://www.mptcp.dev/implementation.html\n[2] https://www.mptcp.dev/setup.html#force-applications-to-use-mptcp\n[3] https://www.mptcp.dev/apps.html\n[4] https://lore.kernel.org/git/ZXLJWS.WPQLCXFNN8TH@unrealasia.net/\n\nCheers,\nMatt\n-- \nSponsored by the NGI0 Core fund.\n\n"},{"id":"518502","messageId":"xmqqfrgzjnhg.fsf@gitster.g","threadId":"63471","inReplyTo":"7b3b8efa-4cc1-4547-b66a-c469626eac46@kernel.org","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-05-20T15:44:27Z","receivedAt":"2025-05-20T15:44:31Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Baerts <matttbe@kernel.org> writes:\n\n> @Junio: Good point! This RFC 6897 was a bit optimistic I think. To get\n> MPTCP in the upstream Linux kernel, we had to make it opt-in, and the\n> modifications we suggested couldn't impact \"plain\" TCP performances (or\n> any other sockets).\n\n\"Couldn't impact\" meaning that unconditionally passing IPPROTO_MPTCP,\neven when MPTCP is not available, would not hurt at all and falls\nback on using regular TCP?\n\nI am assuming that that is not what you meant.  Otherwise, you would\nnot be calling RFC 6897 optimistic, and either the kernel or libc\nlayer would be tweaking the socket() call \"to make the right thing\nhappen transparently\" for everybody, and there wouldn't be any need\nfor this conversation to happen here.\n\nSo I am assuming that at least for now, the choice to use or not use\nMPTCP needs to be made somehow.  Leaving it at the application\nlevel, by the way, does not sound like a winning strategy, but\nanyway, I think the reason why the platform folks do not take\nresponsibility and make it up to the application is because MPTCP\nmay not always be better than TCP; it may boost throughput by\nutilizing multiple links but may hurt latency, for example?\n\nWhat are the criteria the end-user may want to use to decide its\nuse, then?  If interacting with a specific remote repository over\nMCTCP proves better, would the user safely be able to say \"I'll\nalways use MCTCP when talking to that repository\"?  Would it be per\nhost (i.e. if one repository on a host is better with MCTCP, would\nall other repositories on the same host better off using MCTCP)?\n\nWhat I am getting at is that the choice between IPPROTO_TCP and\n_MPTCP may not be \"If Git is compiled with MPTCP support, always use\nMPTCP\", so we need to see where the configuration knob for end-users\nshould be.\n\nThanks.\n"},{"id":"518532","messageId":"202e1a66-72af-48f6-9b3b-7d7473db699e@kernel.org","threadId":"63471","inReplyTo":"xmqqfrgzjnhg.fsf@gitster.g","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Matthieu Baerts","fromEmail":"matttbe@kernel.org","sentAt":"2025-05-20T20:34:56Z","receivedAt":"2025-05-20T20:35:01Z","isPatch":false,"sender":{"key":"matttbe@kernel.org","avatar":"https://gravatar.com/avatar/eeddb10cd8e676b4c7eda928eeaae8097b1df50c72fbbc3d535d2eee55a633d8?d=mp&s=160"},"body":"Hi Junio,\n\nOn 20/05/2025 17:44, Junio C Hamano wrote:\n> Matthieu Baerts <matttbe@kernel.org> writes:\n> \n>> @Junio: Good point! This RFC 6897 was a bit optimistic I think. To get\n>> MPTCP in the upstream Linux kernel, we had to make it opt-in, and the\n>> modifications we suggested couldn't impact \"plain\" TCP performances (or\n>> any other sockets).\n> \n> \"Couldn't impact\" meaning that unconditionally passing IPPROTO_MPTCP,\n> even when MPTCP is not available, would not hurt at all and falls\n> back on using regular TCP?\n\nSorry, I was not clear. I meant \"introducing MPTCP in the Linux kernel\ncouldn't impact other protocols in terms of memory allocated per socket\nbuffer or performances by adding extra checks a bit everywhere for example\".\n\nThe socket API can be used the same way as with TCP: read, write,\nset/get socket options, etc. Plus the MPTCP protocol is made to be\nresilient: if one host doesn't support MPTCP, the connection continues\nin \"plain\" TCP. In the worst cases, when dealing with middleboxes\naltering packets in a way that it messes up MPTCP options, there will be\na fallback to \"plain\" TCP and the connections can continue using only\none path. It looks like the protocol is quite strong, because Apple has\nbeen using MPTCP around the world since 2013, apparently.\n\nIn the Linux kernel, when the client didn't request to use MPTCP, a\nlistening socket supporting MPTCP on the server side will return a\n\"plain\" TCP socket to the userspace during the accept() call. That's why\nwe recommend enabling MPTCP on the server side by default if supported:\nthe impact is minimal, and MPTCP is only used when requested by the\nclients -- which are usually the ones benefiting more from MPTCP\nfeatures. That's in fact the current behaviour for apps written in Go:\nMPTCP is now enabled by default on the server side, and it is easy to\nenable it on the client side when needed.\n\n> I am assuming that that is not what you meant.  Otherwise, you would\n> not be calling RFC 6897 optimistic, and either the kernel or libc\n> layer would be tweaking the socket() call \"to make the right thing\n> happen transparently\" for everybody, and there wouldn't be any need\n> for this conversation to happen here.\n\nSorry, yes, that was my understanding of this RFC 6897. Indeed, they\nseem to suggest the kernel or the libc would decide when to use MPTCP,\nand the apps would not need to care about that at all. That might work\nfor generic cases, but I guess the users and apps prefer to keep the\ncontrol of that. (This RFC apparently also suggest apps to take control\nwhen needed.) Anyway, there are ways to force apps using MPTCP, but a\ndedicated option handled by the apps seem cleaner and clearer.\n\n> So I am assuming that at least for now, the choice to use or not use\n> MPTCP needs to be made somehow.  Leaving it at the application\n> level, by the way, does not sound like a winning strategy, but\n> anyway, I think the reason why the platform folks do not take\n> responsibility and make it up to the application is because MPTCP\n> may not always be better than TCP; it may boost throughput by\n> utilizing multiple links but may hurt latency, for example?\n\nYes indeed, you are right. To be able to use multiple paths, it is\nrequired to add a few bytes in the TCP headers, in the options. If there\nis only one path between two machines located next to each others, or\nfor very short connections, MPTCP and its few extra bytes are not worth\nit. Or to be more precise, there is no need for a client to initiate the\nconnection with MPTCP in these cases. The servers can continue to create\nMPTCP listening sockets, and let the clients decide.\n\n> What are the criteria the end-user may want to use to decide its\n> use, then?  If interacting with a specific remote repository over\n> MCTCP proves better, would the user safely be able to say \"I'll\n> always use MCTCP when talking to that repository\"?  Would it be per\n> host (i.e. if one repository on a host is better with MCTCP, would\n> all other repositories on the same host better off using MCTCP)?\n\nOn the client side, I see this option similar to using Git v2 or\npush.gpgsign: if supported on the server side, I want to use it when my\nclient supports it. Enabling it would be beneficial when switching from\none network to another, or if I have access to multiple networks. Yet,\nto save a few bytes (12B per connection request), I might not want to\ntry using MPTCP with servers that don't support it. Maybe some people\nwill only want to use it with big repository, or all the ones handled by\nthe same server.\n\nSo yes, I think it would be good to start with a global option, and one\nper repository. Because it would be a new feature, people might want to\nstart using MPTCP only with servers supporting it.\n\n> What I am getting at is that the choice between IPPROTO_TCP and\n> _MPTCP may not be \"If Git is compiled with MPTCP support, always use\n> MPTCP\", so we need to see where the configuration knob for end-users\n> should be.\n\nEven if MPTCP is used by default, I guess it would always be safer to\nhave an option to disable it, just in case. In the team, we are all\nhuman, I don't exclude bugs :) (This sentence doesn't make sense any\nmore: if we were robots/AI, that would make even more sense to have an\noption to disable it :-D )\n\nBTW, again thank you all for maintaining and developing Git, this\ncrucial piece of software :)\n\nCheers,\nMatt\n-- \nSponsored by the NGI0 Core fund.\n\n"},{"id":"518535","messageId":"xmqqfrgzgctv.fsf@gitster.g","threadId":"63471","inReplyTo":"202e1a66-72af-48f6-9b3b-7d7473db699e@kernel.org","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-05-20T22:02:52Z","receivedAt":"2025-05-20T22:02:55Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Baerts <matttbe@kernel.org> writes:\n\n> Sorry, I was not clear. I meant \"introducing MPTCP in the Linux kernel\n> couldn't impact other protocols in terms of memory allocated per socket\n> buffer or performances by adding extra checks a bit everywhere for example\".\n\nAh, OK.  What you meant is that the networking maintainers did not\nallow you to affect the \"normal\" codepath when adding MPTCP support\nto their subsystem.\n\nWhich is conservative and probably a good thing, I guess.\n\nBut that choice means each and every application need to opt-in,\nwhich is cumbersome, inconvenient, and hampers adoption X-<.\n\n> listening socket supporting MPTCP on the server side will return a\n> \"plain\" TCP socket to the userspace during the accept() call. That's why\n> we recommend enabling MPTCP on the server side by default if supported:\n> the impact is minimal, and MPTCP is only used when requested by the\n> clients -- which are usually the ones benefiting more from MPTCP\n> features. That's in fact the current behaviour for apps written in Go:\n> MPTCP is now enabled by default on the server side, and it is easy to\n> enable it on the client side when needed.\n\nThat reminds me about one thing I forgot to ask.\n\nThe git:// protocol is the only one we have control over what to ask\nto the socket() system call and the posted patch was about the\nclient side [*].\n\nOn the other end of the connection, even though you could use the\ndedicatd \"git daemon\" process sitting and listening on a socket, my\nunderstanding is it is more common to spawn it via inetd(8).  Does\nit mean that the host needs to run inetd with MPTCP enabled?  I do\nnot know how common that is.\n\nThanks.\n\n[Footnote]\n\n* On the public Internet, hopefully nobody is using that protocol\n  anymore, and instead using either https:// or ssh:// that gives\n  better integrity assurances.\n"},{"id":"518655","messageId":"79398137-999a-4d88-96d5-86d7184a9101@kernel.org","threadId":"63471","inReplyTo":"xmqqfrgzgctv.fsf@gitster.g","subject":"Re: Small patch to add support for MPTCP on Linux","fromName":"Matthieu Baerts","fromEmail":"matttbe@kernel.org","sentAt":"2025-05-22T11:12:43Z","receivedAt":"2025-05-22T11:12:46Z","isPatch":false,"sender":{"key":"matttbe@kernel.org","avatar":"https://gravatar.com/avatar/eeddb10cd8e676b4c7eda928eeaae8097b1df50c72fbbc3d535d2eee55a633d8?d=mp&s=160"},"body":"Hi Junio,\n\nOn 21/05/2025 00:02, Junio C Hamano wrote:\n> Matthieu Baerts <matttbe@kernel.org> writes:\n> \n>> Sorry, I was not clear. I meant \"introducing MPTCP in the Linux kernel\n>> couldn't impact other protocols in terms of memory allocated per socket\n>> buffer or performances by adding extra checks a bit everywhere for example\".\n> \n> Ah, OK.  What you meant is that the networking maintainers did not\n> allow you to affect the \"normal\" codepath when adding MPTCP support\n> to their subsystem.\n\nYes, that's what I meant to say, but you better said it :)\n\n> Which is conservative and probably a good thing, I guess.\n> \n> But that choice means each and every application need to opt-in,\n> which is cumbersome, inconvenient, and hampers adoption X-<.\n\nIndeed... But it looks like it is often the case with new protocols and\nextensions...\n\n>> listening socket supporting MPTCP on the server side will return a\n>> \"plain\" TCP socket to the userspace during the accept() call. That's why\n>> we recommend enabling MPTCP on the server side by default if supported:\n>> the impact is minimal, and MPTCP is only used when requested by the\n>> clients -- which are usually the ones benefiting more from MPTCP\n>> features. That's in fact the current behaviour for apps written in Go:\n>> MPTCP is now enabled by default on the server side, and it is easy to\n>> enable it on the client side when needed.\n> \n> That reminds me about one thing I forgot to ask.\n> \n> The git:// protocol is the only one we have control over what to ask\n> to the socket() system call and the posted patch was about the\n> client side [*].\n> \n> On the other end of the connection, even though you could use the\n> dedicatd \"git daemon\" process sitting and listening on a socket, my\n> understanding is it is more common to spawn it via inetd(8).  Does\n> it mean that the host needs to run inetd with MPTCP enabled?  I do\n> not know how common that is.\n\nGood point. Indeed, for the server side, someone should then also look\nat inetd. I don't know how Muhammad's servers are deployed on his side.\nFrom what I see, inetd relies on the /etc/protocols file, which should\nalready contain an entry for \"mptcp\", at least on Debian-like and\nFedora-like distributions. So 'inetd' should already support MPTCP.\n\n@Muhammad: do you mind checking this case please?\n\n> \n> Thanks.\n> \n> [Footnote]\n> \n> * On the public Internet, hopefully nobody is using that protocol\n>   anymore, and instead using either https:// or ssh:// that gives\n>   better integrity assurances.\n\nIndeed. I already used MPTCP with ssh:// thanks to 'mptcpize', but that\nlooked more like a workaround. For the client side, if an option can be\nset to ask to use MPTCP, this info should be passed to what is being\nused for the HTTPS and SSH connections.\n\n@Muhammad: do you plan to look at that too?\n\n\nFor HTTP(S), it looks like the libcurl is used. If yes, then\n`CURLOPT_OPENSOCKETFUNCTION` can be used, see:\n\n  https://github.com/curl/curl/pull/13278/files\n\n\nFor SSH, I'm a bit annoyed: we already asked OpenSSH maintainers to add\nMPTCP support by sending small patches, but they didn't want it because\nit is not officially supported by BSD... It is supported on Linux,\nmacOS, Windows with WSL, etc. but that's not enough apparently :-/ (or\nmaybe anyone here is able to convince them to support MPTCP by merging\none of the two patches we already sent them? :-D ). For more details and\nworkarounds:\n\n https://www.mptcp.dev/faq.html#how-to-enable-mptcp-support-with-openssh\n\n\nHopefully we will find a way to support MPTCP here in git (and SSH) :)\n\nCheers,\nMatt\n-- \nSponsored by the NGI0 Core fund.\n\n"}]}