{"thread":{"id":"60721","subject":"[DISCUSS] Introducing Rust into the Git project","startedAt":"2024-01-10T20:16:56Z","lastAt":"2024-01-24T07:55:33Z","messageCount":38,"participants":["Taylor Blau","Dragan Simic","Junio C Hamano","rsbecker@nexbridge.com","brian m. carlson","Elijah Newren","Patrick Steinhardt","Sam James","Trevor Gross","Antoni Boucher","Emily Shaffer"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"486563","messageId":"ZZ77NQkSuiRxRDwt@nand.local","threadId":"60721","inReplyTo":null,"subject":"[DISCUSS] Introducing Rust into the Git project","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2024-01-10T20:16:53Z","receivedAt":"2024-01-10T20:16:56Z","isPatch":false,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Over the holiday break at the end of last year I spent some time\nthinking on what it would take to introduce Rust into the Git project.\n\nThere is significant work underway to introduce Rust into the Linux\nkernel (see [1], [2]). Among their stated goals, I think there are a few\nwhich could be potentially relevant to the Git project:\n\n  - Lower risk of memory safety bugs, data races, memory leaks, etc.\n    thanks to the language's safety guarantees.\n\n  - Easier to gain confidence when refactoring or introducing new code\n    in Rust (assuming little to no use of the language's `unsafe`\n    feature).\n\n  - Contributing to Git becomes easier and accessible to a broader group\n    of programmers by relying on a more modern language.\n\nGiven the allure of these benefits, I think it's at least worth\nconsidering and discussing how Rust might make its way into Junio's\ntree.\n\nI imagine that the transition state would involve some parts of the\nproject being built in C and calling into Rust code via FFI (and perhaps\nvice-versa, with Rust code calling back into the existing C codebase).\nLuckily for us, Rust's FFI provides a zero-cost abstraction [3], meaning\nthere is no performance impact when calling code from one language in\nthe other.\n\nSome open questions from me, at least to get the discussion going are:\n\n  1. Platform support. The Rust compiler (rustc) does not enjoy the same\n     widespread availability that C compilers do. For instance, I\n     suspect that NonStop, AIX, Solaris, among others may not be\n     supported.\n\n     One possible alternative is to have those platforms use a Rust\n     front-end for a compiler that they do support. The gccrs [4]\n     project would allow us to compile Rust anywhere where GCC is\n     available. The rustc_codegen_gcc [5] project uses GCC's libgccjit\n     API to target GCC from rustc itself.\n\n  2. Migration. What parts of Git are easiest to convert to Rust? My\n     hunch is that the answer is any stand-alone libraries, like\n     strbuf.h. I'm not sure how we should identify these, though, and in\n     what order we would want to move them over.\n\n  3. Interaction with the lib-ification effort. There is lots of work\n     going on in an effort to lib-ify much of the Git codebase done by\n     Google. I'm not sure how this would interact with that effort, but\n     we should make sure that one isn't a blocker for the other.\n\nI'm curious to hear what others think about this. I think that this\nwould be an exciting and worthwhile direction for the project. Let's\nsee!\n\nThanks,\nTaylor\n\n[1]: https://rust-for-linux.com/\n[2]: https://lore.kernel.org/rust-for-linux/20210414184604.23473-1-ojeda@kernel.org/\n[3]: https://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywhere.html#c-talking-to-rust\n[4]: https://github.com/Rust-GCC/gccrs\n[5]: https://github.com/rust-lang/rustc_codegen_gcc\n"},{"id":"486567","messageId":"b2651b38a4f7edaf1c5ffee72af00e46@manjaro.org","threadId":"60721","inReplyTo":"ZZ77NQkSuiRxRDwt@nand.local","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Dragan Simic","fromEmail":"dsimic@manjaro.org","sentAt":"2024-01-10T21:57:21Z","receivedAt":"2024-01-10T21:57:23Z","isPatch":false,"sender":{"key":"dsimic@manjaro.org","avatar":null},"body":"On 2024-01-10 21:16, Taylor Blau wrote:\n> Over the holiday break at the end of last year I spent some time\n> thinking on what it would take to introduce Rust into the Git project.\n> \n> There is significant work underway to introduce Rust into the Linux\n> kernel (see [1], [2]). Among their stated goals, I think there are a \n> few\n> which could be potentially relevant to the Git project:\n> \n>   - Lower risk of memory safety bugs, data races, memory leaks, etc.\n>     thanks to the language's safety guarantees.\n> \n>   - Easier to gain confidence when refactoring or introducing new code\n>     in Rust (assuming little to no use of the language's `unsafe`\n>     feature).\n> \n>   - Contributing to Git becomes easier and accessible to a broader \n> group\n>     of programmers by relying on a more modern language.\n> \n> Given the allure of these benefits, I think it's at least worth\n> considering and discussing how Rust might make its way into Junio's\n> tree.\n\nQuite frankly, that would only complicate things and cause \nfragmentation.  The goal of introducing Rust into the Linux kernel is \nto, possibly, have some new \"leafs\" written in Rust, such as some new \ndevice drivers.  No existing kernel code, AFAIK, has been planned to be \nrewritten in Rust.\n\nThus, Git should probably follow the same approach of not converting the \nalready existing code, but frankly, I don't see what would actually be \nthe \"new leafs\" written in Rust.\n\n> I imagine that the transition state would involve some parts of the\n> project being built in C and calling into Rust code via FFI (and \n> perhaps\n> vice-versa, with Rust code calling back into the existing C codebase).\n> Luckily for us, Rust's FFI provides a zero-cost abstraction [3], \n> meaning\n> there is no performance impact when calling code from one language in\n> the other.\n> \n> Some open questions from me, at least to get the discussion going are:\n> \n>   1. Platform support. The Rust compiler (rustc) does not enjoy the \n> same\n>      widespread availability that C compilers do. For instance, I\n>      suspect that NonStop, AIX, Solaris, among others may not be\n>      supported.\n> \n>      One possible alternative is to have those platforms use a Rust\n>      front-end for a compiler that they do support. The gccrs [4]\n>      project would allow us to compile Rust anywhere where GCC is\n>      available. The rustc_codegen_gcc [5] project uses GCC's libgccjit\n>      API to target GCC from rustc itself.\n> \n>   2. Migration. What parts of Git are easiest to convert to Rust? My\n>      hunch is that the answer is any stand-alone libraries, like\n>      strbuf.h. I'm not sure how we should identify these, though, and \n> in\n>      what order we would want to move them over.\n> \n>   3. Interaction with the lib-ification effort. There is lots of work\n>      going on in an effort to lib-ify much of the Git codebase done by\n>      Google. I'm not sure how this would interact with that effort, but\n>      we should make sure that one isn't a blocker for the other.\n> \n> I'm curious to hear what others think about this. I think that this\n> would be an exciting and worthwhile direction for the project. Let's\n> see!\n> \n> Thanks,\n> Taylor\n> \n> [1]: https://rust-for-linux.com/\n> [2]:\n> https://lore.kernel.org/rust-for-linux/20210414184604.23473-1-ojeda@kernel.org/\n> [3]:\n> https://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywhere.html#c-talking-to-rust\n> [4]: https://github.com/Rust-GCC/gccrs\n> [5]: https://github.com/rust-lang/rustc_codegen_gcc\n"},{"id":"486569","messageId":"xmqqjzog96uh.fsf@gitster.g","threadId":"60721","inReplyTo":"b2651b38a4f7edaf1c5ffee72af00e46@manjaro.org","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-01-10T22:11:34Z","receivedAt":"2024-01-10T22:11:37Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Dragan Simic <dsimic@manjaro.org> writes:\n\n> Thus, Git should probably follow the same approach of not converting\n> the already existing code, but frankly, I don't see what would\n> actually be the \"new leafs\" written in Rust.\n\nA few obvious ones that come to my mind are that you should be able\nto write a new merge strategy and link the resulting binary into Git\nwithout much hassle.  You might even want to make that a dynamically\nloaded object.  The interface into a merge strategy is fairly narrow\nIIRC.  Or possibly a new remote helper.\n\nAdding a new refs backend may need to wait for the work Patrick is\ndoing to add reftable support, but once the abstraction gets to the\npoint to sufficiently hide the differences between files and reftables\nbackends, I do not see a reason why you cannot add the third one.\n\nAnd more into the future, we might want to have an object DB\nabstraction, similar to how we abstracted refs API over time, at\nwhich time you might be writing code that stores objects to and\nretrieves objects from persistent redis and whatnot in your favorite\nlanguage.\n"},{"id":"486570","messageId":"006b01da4412$96c6c500$c4544f00$@nexbridge.com","threadId":"60721","inReplyTo":"xmqqjzog96uh.fsf@gitster.g","subject":"RE: [DISCUSS] Introducing Rust into the Git project","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-01-10T22:15:53Z","receivedAt":"2024-01-10T22:16:13Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Wednesday, January 10, 2024 5:12 PM, Junio C Hamano wrote:\n>Dragan Simic <dsimic@manjaro.org> writes:\n>\n>> Thus, Git should probably follow the same approach of not converting\n>> the already existing code, but frankly, I don't see what would\n>> actually be the \"new leafs\" written in Rust.\n>\n>A few obvious ones that come to my mind are that you should be able to\nwrite a\n>new merge strategy and link the resulting binary into Git without much\nhassle.  You\n>might even want to make that a dynamically loaded object.  The interface\ninto a\n>merge strategy is fairly narrow IIRC.  Or possibly a new remote helper.\n>\n>Adding a new refs backend may need to wait for the work Patrick is doing to\nadd\n>reftable support, but once the abstraction gets to the point to\nsufficiently hide the\n>differences between files and reftables backends, I do not see a reason why\nyou\n>cannot add the third one.\n>\n>And more into the future, we might want to have an object DB abstraction,\nsimilar\n>to how we abstracted refs API over time, at which time you might be writing\ncode\n>that stores objects to and retrieves objects from persistent redis and\nwhatnot in\n>your favorite language.\n\nJust a brief concern: Rust is not broadly portable. Adding another\ndependency to git will remove many existing platforms from future releases.\nPlease consider this carefully before going down this path.\n--Randall\n\n"},{"id":"486573","messageId":"ZZ8ZlX6bf+hjmhN+@nand.local","threadId":"60721","inReplyTo":"006b01da4412$96c6c500$c4544f00$@nexbridge.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2024-01-10T22:26:29Z","receivedAt":"2024-01-10T22:26:31Z","isPatch":false,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi Randall,\n\nOn Wed, Jan 10, 2024 at 05:15:53PM -0500, rsbecker@nexbridge.com wrote:\n> Just a brief concern: Rust is not broadly portable. Adding another\n> dependency to git will remove many existing platforms from future releases.\n> Please consider this carefully before going down this path.\n\nI was hoping to hear from you as one of the few (only?) folks who\nparticipate on the list and represent HPE NonStop users.\n\nI'm curious which if any of the compiler frontends that I listed in my\nearlier email would work for you.\n\nThanks,\nTaylor\n"},{"id":"486577","messageId":"ZZ8q40OdPy0v-ZxQ@tapette.crustytoothpaste.net","threadId":"60721","inReplyTo":"xmqqjzog96uh.fsf@gitster.g","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-01-10T23:40:19Z","receivedAt":"2024-01-10T23:40:21Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-01-10 at 22:11:34, Junio C Hamano wrote:\n> A few obvious ones that come to my mind are that you should be able\n> to write a new merge strategy and link the resulting binary into Git\n> without much hassle.  You might even want to make that a dynamically\n> loaded object.  The interface into a merge strategy is fairly narrow\n> IIRC.  Or possibly a new remote helper.\n> \n> Adding a new refs backend may need to wait for the work Patrick is\n> doing to add reftable support, but once the abstraction gets to the\n> point to sufficiently hide the differences between files and reftables\n> backends, I do not see a reason why you cannot add the third one.\n> \n> And more into the future, we might want to have an object DB\n> abstraction, similar to how we abstracted refs API over time, at\n> which time you might be writing code that stores objects to and\n> retrieves objects from persistent redis and whatnot in your favorite\n> language.\n\nThis is definitely a thing people will want to do.  I think Microsoft\nhad some code for Azure DevOps that stored their code in the cloud and\nthe refs database in a real database.  I can imagine that being a\nvaluable set of features people would want to implement in a variety of\nenvironments, with all of the benefits of basing on upstream Git.\n\nI also feel that I would absolutely not want to write those things in C.\nRust is much more ergonomic when writing these things because freeing\nresources (freeing memory, rolling back transactions, closing files,\netc.) becomes as easy as implementing the Drop trait and you write less\nboilerplate.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"486578","messageId":"007c01da4420$10a7b700$31f72500$@nexbridge.com","threadId":"60721","inReplyTo":"ZZ8ZlX6bf+hjmhN+@nand.local","subject":"RE: [DISCUSS] Introducing Rust into the Git project","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-01-10T23:52:21Z","receivedAt":"2024-01-10T23:52:31Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Wednesday, January 10, 2024 5:26 PM, Taylor Blau wrote:\n>On Wed, Jan 10, 2024 at 05:15:53PM -0500, rsbecker@nexbridge.com wrote:\n>> Just a brief concern: Rust is not broadly portable. Adding another\n>> dependency to git will remove many existing platforms from future releases.\n>> Please consider this carefully before going down this path.\n>\n>I was hoping to hear from you as one of the few (only?) folks who participate on\n>the list and represent HPE NonStop users.\n>\n>I'm curious which if any of the compiler frontends that I listed in my earlier email\n>would work for you.\n\nUnfortunately, none of the compiler frontends listed previously can be built for NonStop. These appear to all require gcc either directly or transitively, which cannot be ported to NonStop. I do not expect this to change any time soon - and is outside of my control anyway. An attempt was made to port Rust but it did not succeed primarily because of that dependency. Similarly, Golang is also not portable to NonStop because of architecture assumptions made by the Go team that cannot be satisfied on NonStop at this time. If some of the memory/pointer issues are the primary concern, c11 might be something acceptable with smart pointers. C17 will eventually be deployable, but is not available on most currently supported OS versions on the platform.\n\n"},{"id":"486580","messageId":"CABPp-BFOmwV-xBtjvtenb6RFz9wx2VWVpTeho0k=D8wsCCVwqQ@mail.gmail.com","threadId":"60721","inReplyTo":"ZZ77NQkSuiRxRDwt@nand.local","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2024-01-11T00:12:21Z","receivedAt":"2024-01-11T00:12:35Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jan 10, 2024 at 12:18 PM Taylor Blau <me@ttaylorr.com> wrote:\n>\n> Over the holiday break at the end of last year I spent some time\n> thinking on what it would take to introduce Rust into the Git project.\n\nI'm very happy to see this email.\n\n> There is significant work underway to introduce Rust into the Linux\n> kernel (see [1], [2]). Among their stated goals, I think there are a few\n> which could be potentially relevant to the Git project:\n>\n>   - Lower risk of memory safety bugs, data races, memory leaks, etc.\n>     thanks to the language's safety guarantees.\n>\n>   - Easier to gain confidence when refactoring or introducing new code\n>     in Rust (assuming little to no use of the language's `unsafe`\n>     feature).\n>\n>   - Contributing to Git becomes easier and accessible to a broader group\n>     of programmers by relying on a more modern language.\n>\n> Given the allure of these benefits, I think it's at least worth\n> considering and discussing how Rust might make its way into Junio's\n> tree.\n\nI think there are other benefits as well; I'll list them at the end of\nthe email to avoid side-tracking too much[6].\n\n> I imagine that the transition state would involve some parts of the\n> project being built in C and calling into Rust code via FFI (and perhaps\n> vice-versa, with Rust code calling back into the existing C codebase).\n> Luckily for us, Rust's FFI provides a zero-cost abstraction [3], meaning\n> there is no performance impact when calling code from one language in\n> the other.\n\nI agree with the zero-cost abstraction, but there is a funny caveat\nwith measuring it if anyone is curious[7].\n\n> Some open questions from me, at least to get the discussion going are:\n>\n>   1. Platform support. The Rust compiler (rustc) does not enjoy the same\n>      widespread availability that C compilers do. For instance, I\n>      suspect that NonStop, AIX, Solaris, among others may not be\n>      supported.\n>\n>      One possible alternative is to have those platforms use a Rust\n>      front-end for a compiler that they do support. The gccrs [4]\n>      project would allow us to compile Rust anywhere where GCC is\n>      available. The rustc_codegen_gcc [5] project uses GCC's libgccjit\n>      API to target GCC from rustc itself.\n\nAnother alternative (as discussed at Git Merge when we were last\ntalking about Rust[8]), is requiring all Rust code to be optional for\nnow.  If we choose to go that route, I think that means that (a) for\nexisting components, we have both a Rust and a C implementation\navailable, and (b) for new components (e.g. new top-level commands\nlike git-replay), they can be Rust-only and those compiling without\nRust just don't get them.\n\n>   2. Migration. What parts of Git are easiest to convert to Rust? My\n>      hunch is that the answer is any stand-alone libraries, like\n>      strbuf.h. I'm not sure how we should identify these, though, and in\n>      what order we would want to move them over.\n\nIf we're happy to allow Rust, I'd like to rewrite git-replay in Rust\nas a testcase.  It's almost certainly not \"easiest\", but I think it's\nan interesting testcase because it's a new top-level command that\nhasn't appeared in any release yet.  Further, it is currently only\ndesigned for server-side usecases, so would likely not be affected by\nmore limited platform support.  (I haven't started on this; my\nprevious experiments were with diffcore-delta.)\n\n> I'm curious to hear what others think about this. I think that this\n> would be an exciting and worthwhile direction for the project. Let's\n> see!\n\n:-)\n\n>\n> Thanks,\n> Taylor\n>\n> [1]: https://rust-for-linux.com/\n> [2]: https://lore.kernel.org/rust-for-linux/20210414184604.23473-1-ojeda@kernel.org/\n> [3]: https://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywhere.html#c-talking-to-rust\n> [4]: https://github.com/Rust-GCC/gccrs\n> [5]: https://github.com/rust-lang/rustc_codegen_gcc\n\n[6] Here are some additional benefits I see:\n\n - Parallel performance.  We avoid making things parallel in Git because\n   debugging/maintaining/reviewing parallel code in C often isn't worth\n   the squeeze.  Rust was designed to greatly reduce this effort (the\n   whole \"fearless concurrency\" thing).\n\n - Single-threaded Performance.  Multiple factors:\n\n   - We had (and might still have) O(N^2) stuff in a lot of places in\n     our codebase, because we tend to over-use arrays.  (e.g. with\n     string_list, or with insertions and deletions into the index\n     during a merge, etc.)\n\n   - Relatedly, using hashes in C is quite onerous, to the point that\n     we often simply avoid it.  I know I have, and I also know that\n     even after I introduced strmap and tried to use it outside of\n     merge-ort, that I got pushback because \"string hash-maps are not\n     really typical for a C program. I'm sure they are the best choice\n     for an advanced merge algorithm but they are not really necessary\n     [here; let's use sorted arrays instead]...\"  I then had to go\n     through multiple rounds of responses and ended up reimplementing\n     everything as suggested (before finally convincing others to just\n     use the strmap implementation after all).\n\n   - We use QSORT() which basically calls libc's qsort().  Due to the\n     design of this function (where the comparator is a separate\n     function call), it is slow.  When languages avoid making the\n     comparator a separate function call, they can speed sorts up by a\n     factor of 2 (or even by 3 when an unstable sort is good enough\n     and the platform's qsort() is stable).\n\n   - Difficulty of incorporating other libraries.  For example, our\n     hashmap.[ch] make use of FNV, but picking something else is a big\n     amount of effort.  Now, while FNV is faster than Rust's default\n     of SipHash, cargo makes it easy to pull in alternatives like\n     FnvHashMap or FxHashMap, which we can then use where it matters.\n\nI'm also tempted to include bullet points for having a unit testing\nframework built in, and potentially fewer platform-dependent issues\n(e.g. forgetting to use STABLE_QSORT when required since qsort is\nstable in some libc implementations, since rust defines those more\ncarefully to be consistent across platforms), but I'm not sure these\nadditional advantages are big enough to merit a full bullet point.\n\n[7] If you ignore Rust for a moment, and simply divide your files into\ndifferent libraries (e.g. introducing a new.c file, moving some\nfunctions to it, and then compiling new.c into a new library,\nlibnew.a, and linking both libgit.a and libnew.a into git), you can\nsometimes measure some small performance differences.  At least, I\ndid.  What this scenario has to do with Rust is that if we start\nmoving some code to Rust, that will naturally likely result in a\ndifferent division of files into libraries.  Thus, for me to verify\nthat Rust did provide zero-cost abstractions with my experiments, in\norder to compare the performance of my Rust changes, I had to compare\nto a version of git where I split some functions out into a separate\nlibrary.  When I did that, the performance overhead was actually 0.\nOtherwise, there was a tiny performance degradation in the particular\nsplitting I employed.  However, while splitting did give me a small\nperformance drop, it was completely outweighed by the performance\nadvantages I got elsewhere in the things I converted to Rust.\n\n[8] https://lore.kernel.org/git/ZRrfN2lbg14IOLiK@nand.local/\n"},{"id":"486582","messageId":"CABPp-BH3sva=CNtx8YFGP4Egyau-hR+7njZPFEd-DRTw91BK2w@mail.gmail.com","threadId":"60721","inReplyTo":"b2651b38a4f7edaf1c5ffee72af00e46@manjaro.org","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2024-01-11T00:33:56Z","receivedAt":"2024-01-11T00:34:10Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jan 10, 2024 at 1:57 PM Dragan Simic <dsimic@manjaro.org> wrote:\n>\n> Thus, Git should probably follow the same approach of not converting the\n> already existing code\n\nI disagree with this.  I saw significant performance improvements\nthrough converting some existing Git code to Rust.  Granted, it was\nonly a small amount of code, but the performance benefits I saw\nsuggested we'd see more by also doing similar conversions elsewhere.\n(Note that I kept the old C code and then conditionally compiled\neither Rust or C versions of what I was converting.)\n\nFurther, I found a really old bug from this effort as well[1], and I\nfind it extremely unlikely that I would have found that bug otherwise.\nSo, converting to Rust can even improve our existing C code.\n\n>, but frankly, I don't see what would actually be\n> the \"new leafs\" written in Rust.\n\nIn addition to some of the examples Junio mentioned elsewhere, I think\nnew toplevel commands, like git-replay, would qualify.\n\n\n[1] Yeah, I really need to dig the patch out and send it in.  I'll do\nso shortly.\n"},{"id":"486584","messageId":"CABPp-BEw_HFL-9u6WdSEe-qr_JfJyQtfU6PP7izEdPChKooc6g@mail.gmail.com","threadId":"60721","inReplyTo":"007c01da4420$10a7b700$31f72500$@nexbridge.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2024-01-11T00:59:25Z","receivedAt":"2024-01-11T00:59:40Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jan 10, 2024 at 3:52 PM <rsbecker@nexbridge.com> wrote:\n>\n> On Wednesday, January 10, 2024 5:26 PM, Taylor Blau wrote:\n> >On Wed, Jan 10, 2024 at 05:15:53PM -0500, rsbecker@nexbridge.com wrote:\n> >> Just a brief concern: Rust is not broadly portable. Adding another\n> >> dependency to git will remove many existing platforms from future releases.\n> >> Please consider this carefully before going down this path.\n> >\n> >I was hoping to hear from you as one of the few (only?) folks who participate on\n> >the list and represent HPE NonStop users.\n> >\n> >I'm curious which if any of the compiler frontends that I listed in my earlier email\n> >would work for you.\n>\n> Unfortunately, none of the compiler frontends listed previously can be built for NonStop. These appear to all require gcc either directly or transitively, which cannot be ported to NonStop. I do not expect this to change any time soon - and is outside of my control anyway. An attempt was made to port Rust but it did not succeed primarily because of that dependency. Similarly, Golang is also not portable to NonStop because of architecture assumptions made by the Go team that cannot be satisfied on NonStop at this time. If some of the memory/pointer issues are the primary concern, c11 might be something acceptable with smart pointers. C17 will eventually be deployable, but is not available on most currently supported OS versions on the platform.\n\nWould you be okay with the following alternative: requiring that all\nRust code be optional for now?\n\n(In other words, allow you to build with USE_RUST=0, or something like\nthat.  And then we have both a Rust and a C implementation of anything\nthat is required for backward compatibility, while any new Rust-only\nstuff would not be included in your build.)\n"},{"id":"486585","messageId":"008701da442f$b2dfe420$189fac60$@nexbridge.com","threadId":"60721","inReplyTo":"CABPp-BEw_HFL-9u6WdSEe-qr_JfJyQtfU6PP7izEdPChKooc6g@mail.gmail.com","subject":"RE: [DISCUSS] Introducing Rust into the Git project","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-01-11T01:44:15Z","receivedAt":"2024-01-11T01:44:29Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Wednesday, January 10, 2024 7:59 PM, Elijah Newren wrote:\n>On Wed, Jan 10, 2024 at 3:52 PM <rsbecker@nexbridge.com> wrote:\n>>\n>> On Wednesday, January 10, 2024 5:26 PM, Taylor Blau wrote:\n>> >On Wed, Jan 10, 2024 at 05:15:53PM -0500, rsbecker@nexbridge.com wrote:\n>> >> Just a brief concern: Rust is not broadly portable. Adding another\n>> >> dependency to git will remove many existing platforms from future releases.\n>> >> Please consider this carefully before going down this path.\n>> >\n>> >I was hoping to hear from you as one of the few (only?) folks who\n>> >participate on the list and represent HPE NonStop users.\n>> >\n>> >I'm curious which if any of the compiler frontends that I listed in\n>> >my earlier email would work for you.\n>>\n>> Unfortunately, none of the compiler frontends listed previously can be built for\n>NonStop. These appear to all require gcc either directly or transitively, which cannot\n>be ported to NonStop. I do not expect this to change any time soon - and is outside\n>of my control anyway. An attempt was made to port Rust but it did not succeed\n>primarily because of that dependency. Similarly, Golang is also not portable to\n>NonStop because of architecture assumptions made by the Go team that cannot be\n>satisfied on NonStop at this time. If some of the memory/pointer issuese the\n>primary concern, c11 might be something acceptable with smart pointers. C17 will\n>eventually be deployable, but is not available on most currently supported OS\n>versions on the platform.\n>\n>Would you be okay with the following alternative: requiring that all Rust code be\n>optional for now?\n>\n>(In other words, allow you to build with USE_RUST=0, or something like that.  And\n>then we have both a Rust and a C implementation of anything that is required for\n>backward compatibility, while any new Rust-only stuff would not be included in\n>your build.)\n\nTo address the immediate above, I assume this means that platform maintainers will be responsible for developing non-portable implementations that duplicate Rust functionality, which arguably may not be possible. We do have $DAYJOBS and the expectation that duplicate implementation are cost effective or even viable is a huge assumption that may not be attainable.\n\nOne of the key benefits of git is the ability to deploy it virtually anywhere on virtually any platform - and mirror repositories anywhere for resiliency purposes. It currently runs on (almost) every current platform because it does not have dependencies on Linux-only compilers and tools. Except for LFS, which is Golang, and I do not have access to that functionality, anyone with a C compiler can deploy git processes in their environment. By adding Rust (or any other gcc-only dependency), it eliminates the primary benefit of git. I am honestly very disappointed with this direction and think this detracts significantly from the primary value proposition that git offers: specifically, that we can take any developer from any platform and move them anywhere else without having to design new processes or teach them new processes for their workflows (this comes up at every major customer with whom I interact). I think this direction is a fundamental mistake and will rapidly limit (or eliminate) git's long-term viability. To be honest, if I saw this direction when deciding which VCS to deploy, I would reconsider git and start looking around for another more portable option. It hurts to even contemplate this direction. Please do not do this.\n\n"},{"id":"486586","messageId":"ZZ9K1CVBKdij4tG0@tapette.crustytoothpaste.net","threadId":"60721","inReplyTo":"ZZ77NQkSuiRxRDwt@nand.local","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-01-11T01:56:36Z","receivedAt":"2024-01-11T01:56:39Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-01-10 at 20:16:53, Taylor Blau wrote:\n> Over the holiday break at the end of last year I spent some time\n> thinking on what it would take to introduce Rust into the Git project.\n> \n> There is significant work underway to introduce Rust into the Linux\n> kernel (see [1], [2]). Among their stated goals, I think there are a few\n> which could be potentially relevant to the Git project:\n> \n>   - Lower risk of memory safety bugs, data races, memory leaks, etc.\n>     thanks to the language's safety guarantees.\n> \n>   - Easier to gain confidence when refactoring or introducing new code\n>     in Rust (assuming little to no use of the language's `unsafe`\n>     feature).\n\nI agree with both of these points.  We've found that making our code\nthread safe in Git is hard and it's much easier in Rust, because, for\nthe most part, the code doesn't compile if it would have a data race.\nUnit tests are also easy and built-in, and I think that's a major\nadvantage.\n\nWe also get nice things for free, like sets, maps, lists, and a variety\nof other collections that are all type-safe.  Error handling is also a\nhuge benefit: we'll get typed errors with the ability to pass data back.\n\n>   - Contributing to Git becomes easier and accessible to a broader group\n>     of programmers by relying on a more modern language.\n\nI think this can't be understated.  One of the biggest hurdles for\npeople contributing is that our code requires expert knowledge of C.  We\ndo all sorts of weird things with pointer arithmetic that even I have\ntrouble understanding, and I'd really appreciate not having to worry\nabout memory leaks or freeing resources.[0]  Rust has nice things like the\nDrop trait that make resource management easy.\n\nRust is also a language that people _want_ to use.  I really like it and\nwould probably contribute more if Git were in it.  I don't really want\nto write more C, and outside of Git, won't use it on more than a de\nminimis basis unless paid.\n\nI can confirm that, having partially ported our service that serves Git\ntraffic to Rust from C (without the public having noticed), it's a much\nnicer environment to work in.  I'm also much more efficient at making\nchanges as well.\n\n> Given the allure of these benefits, I think it's at least worth\n> considering and discussing how Rust might make its way into Junio's\n> tree.\n\nA couple of things which I think are worth discussing are as follows:\n\nThe Rust project emits a new release every six weeks and doesn't provide\nLTS versions.  What versions of Rust are supported by crates vary\nwidely, and we'll absolutely need to choose our dependencies wisely.  We\nmay also want to ask crate authors if they'll be willing to commit to\nour version policy before using them; oftentimes, that can work.\n\nThe approach that I aim for is supporting the version of Rust in the\nlatest Debian stable, plus the version in Debian's previous stable\nrelease until the latest stable has been out for a year.  (Thus, if\nDebian 12 was released on 2023-06-10, then I'd support Rust 1.48, Debian\n11's version, until 2024-06-10, and then support would move to 1.63,\nDebian 12's version.)  This provides about three years of support for a\ncompiler version, which I think is fair.\n\nNote that none of this means that we're dropping support for older\nsystems; newer versions of Rust will be available for most targets,\neven often after OSes go end of life.\n\nWe'll also probably need to continue to rely on some C libraries.  For\nexample, reqwest, the main Rust HTTP client, doesn't support any\nauthentication other than Basic, and I assure you from my experience as\nthe Git LFS maintainer, we don't want to implement things like NTLM and\nKerberos on our own.  libcurl is almost certainly going to continue to\nbe a dependency, as will PCRE.  The Rust regex crate doesn't support\nbackreferences, and we've basically tied lots of our regexes to POSIX,\nso we'll need to either rely on PCRE or some call out to a\nPOSIX-compatible interface.  gettext is likely to be another issue,\nalthough its thread-safety is potentially a problem; we could try using\nthe `tr` crate instead, which also provides a Rust-specific string\nripper.\n\n> I imagine that the transition state would involve some parts of the\n> project being built in C and calling into Rust code via FFI (and perhaps\n> vice-versa, with Rust code calling back into the existing C codebase).\n> Luckily for us, Rust's FFI provides a zero-cost abstraction [3], meaning\n> there is no performance impact when calling code from one language in\n> the other.\n\nMoreover, there are even ways to generate Rust bindings for C code and C\nheaders for Rust code automatically.  (These are cbindgen and bindgen,\nrespectively.)  I've used both, and while it's clearly an FFI case, it's\nstill very ergonomic.\n\n> Some open questions from me, at least to get the discussion going are:\n> \n>   1. Platform support. The Rust compiler (rustc) does not enjoy the same\n>      widespread availability that C compilers do. For instance, I\n>      suspect that NonStop, AIX, Solaris, among others may not be\n>      supported.\n> \n>      One possible alternative is to have those platforms use a Rust\n>      front-end for a compiler that they do support. The gccrs [4]\n>      project would allow us to compile Rust anywhere where GCC is\n>      available. The rustc_codegen_gcc [5] project uses GCC's libgccjit\n>      API to target GCC from rustc itself.\n\nI think this is probably the biggest stumbling point.  I know GCC is\nhighly portable and works on AIX, as well as virtually every\narchitecture.  gccrs is still incomplete, but I believe\nrustc_codegen_gcc is mature, and should be a viable option for most\nplatforms.  (Solaris is already supported on Rust[1].)\n\nMy main concerns are with NonStop, since the Rust standard library\nrequires threading and a CSPRNG (although that can definitely be RDRAND,\nand is for some targets).  I seem to recall that neither GCC nor LLVM\nare present there, although I see no reason why GCC could not be ported\n(LLVM lacks support for ia64, I believe, which would make it a bigger\nlift)\n\nI suspect that if we go forward, though, a lot of the work for\narchitecture support in Rust upstream will already have been done, since\nI'm pretty sure the Debian porters for architectures like alpha, hppa,\nand ia64 are going to want to continue to use Git.  NetBSD porters may\nalso have useful patches in pkgsrc.\n\nI am also very sympathetic to the difficulties of running on less common\nsystems, having had a PowerPC Mac running Linux as my first laptop and\nseveral UltraSPARC machines.  I have sent in numerous patches to a wide\nvariety of code so that it works gracefully on lots of architectures,\nand I've also dealt with lots of broken software.  I do, however, think\nit's up to the porters of an OS to keep it running and healthy, and that\nmeans making sure it has suitable compiler toolchains for building,\nincluding for modern, extremely popular languages like Rust and Go.  I'm\nokay with dropping support for systems where nobody upstream wants to or\nis capable of maintaining that tooling.\n\nI actually feel that once Rust is running on a system, it's actually\neasier to write portable code, since you don't have alignment issues and\nendianness must be handled explicitly, and most safe Rust code just\nworks out of the box.\n\n>   2. Migration. What parts of Git are easiest to convert to Rust? My\n>      hunch is that the answer is any stand-alone libraries, like\n>      strbuf.h. I'm not sure how we should identify these, though, and in\n>      what order we would want to move them over.\n\nstrbuf.h is tricky because it uses variadic arguments, which are not\nstable in Rust.  My approach would be to start by getting the main\nfunction up and running, and then we can incrementally port things over.\n\nWe could, for example, use the `sha256` crate for our SHA-256 code\n(which would also dynamically use accelerated hardware implementations\nwhere available).  There are other things which are libraries which\ncould well work, though.  Porting over our hashmap implementation might\nbe a thing to do, for example.  The repository structure might also be\na good idea, since that will allow us to write safe wrappers for its\ncontents.\n\n>   3. Interaction with the lib-ification effort. There is lots of work\n>      going on in an effort to lib-ify much of the Git codebase done by\n>      Google. I'm not sure how this would interact with that effort, but\n>      we should make sure that one isn't a blocker for the other.\n\nI think it's going to work together nicely.  We can and should consider\nbuilding a C library from Rust to expose a lot of what we write.\n\nAlso, in my view, the biggest enemy to libification in our codebase is\nour copious and improvident use of globals.  Mutating static variables\nin Rust is unsafe, so as part of the port, we'll need to get rid of\nthem, which seems like a nice common goal.\n\n> I'm curious to hear what others think about this. I think that this\n> would be an exciting and worthwhile direction for the project. Let's\n> see!\n\nI'm very much in favour of this.  I think I brought it up at the\ncontributor's summit and it caught some attention, but I don't think it\nshould be too controversial and it will offer us a lot of advantages.\n\n[0] And before people say, \"Well, you just need to spend more time with\nC,\" I've been writing it since I was 10 and I think we can all agree\nthat with the SHA-256 work I've spent plenty of time with it.\n[1] rustc --print target-list is a great way to see what's supported.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"486589","messageId":"CABPp-BHx=4HPSN4enkHTL7PPnNBsJ1vGWe4Em5imH7HcOcH2PA@mail.gmail.com","threadId":"60721","inReplyTo":"008701da442f$b2dfe420$189fac60$@nexbridge.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2024-01-11T02:21:28Z","receivedAt":"2024-01-11T02:21:42Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jan 10, 2024 at 5:44 PM <rsbecker@nexbridge.com> wrote:\n>\n> On Wednesday, January 10, 2024 7:59 PM, Elijah Newren wrote:\n[...]\n> >Would you be okay with the following alternative: requiring that all Rust code be\n> >optional for now?\n> >\n> >(In other words, allow you to build with USE_RUST=0, or something like that.  And\n> >then we have both a Rust and a C implementation of anything that is required for\n> >backward compatibility, while any new Rust-only stuff would not be included in\n> >your build.)\n>\n> To address the immediate above, I assume this means that platform maintainers will be responsible for developing non-portable implementations that duplicate Rust functionality\n\nThis doesn't at all sound like what I thought I said.  The whole\nproposal was so that folks like NonStop could continue using Git with\nno more work than setting USE_RUST=0 at build time.\n\nWhy do you feel you'd need to duplicate any functionality?\n"},{"id":"486590","messageId":"ZZ9YrYvW_L9A02aI@tapette.crustytoothpaste.net","threadId":"60721","inReplyTo":"007c01da4420$10a7b700$31f72500$@nexbridge.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-01-11T02:55:41Z","receivedAt":"2024-01-11T02:55:44Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-01-10 at 23:52:21, rsbecker@nexbridge.com wrote:\n> Unfortunately, none of the compiler frontends listed previously can be\n> built for NonStop. These appear to all require gcc either directly or\n> transitively, which cannot be ported to NonStop. I do not expect this\n> to change any time soon - and is outside of my control anyway. An\n> attempt was made to port Rust but it did not succeed primarily because\n> of that dependency.\n\nCan you tell us what the technical limitations are that prevent GCC from\nbeing ported so we can understand better?  I know LLVM doesn't support\nia64, which you do support, but GCC is very likely the most portable\ncompiler on the planet and supports architectures and OSes I've never\notherwise heard of.\n\nI strongly suspect that if GCC did end up on NonStop, Rust would be able\nto be ported, too, and you'd also get access to gccgo, which would make\nGit LFS possible on NonStop as well[0].\n\nI'm not capable of porting GCC, but I have done some portability work in\nthe Rust ecosystem, and I'd be willing to provide context and some\nassistance (within my time and capabilities) to help get Rust working on\nNonStop if you want.\n\n[0] For the record, as a maintainer of Git LFS, I'm happy to accept\nportability patches for virtually any OS.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"486591","messageId":"009c01da4439$f70beef0$e523ccd0$@nexbridge.com","threadId":"60721","inReplyTo":"CABPp-BHx=4HPSN4enkHTL7PPnNBsJ1vGWe4Em5imH7HcOcH2PA@mail.gmail.com","subject":"RE: [DISCUSS] Introducing Rust into the Git project","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-01-11T02:57:44Z","receivedAt":"2024-01-11T02:57:57Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Wednesday, January 10, 2024 9:21 PM, Elijah Newren wrote:\n>On Wed, Jan 10, 2024 at 5:44 PM <rsbecker@nexbridge.com> wrote:\n>>\n>> On Wednesday, January 10, 2024 7:59 PM, Elijah Newren wrote:\n>[...]\n>> >Would you be okay with the following alternative: requiring that all\n>> >Rust code be optional for now?\n>> >\n>> >(In other words, allow you to build with USE_RUST=0, or something\n>> >like that.  And then we have both a Rust and a C implementation of\n>> >anything that is required for backward compatibility, while any new\n>> >Rust-only stuff would not be included in your build.)\n>>\n>> To address the immediate above, I assume this means that platform\n>> maintainers will be responsible for developing non-portable\n>> implementations that duplicate Rust functionality\n>\n>This doesn't at all sound like what I thought I said.  The whole proposal was so that\n>folks like NonStop could continue using Git with no more work than setting\n>USE_RUST=0 at build time.\n>\n>Why do you feel you'd need to duplicate any functionality?\n\nI think I misunderstood. What I took from this is that all new functionality would be in Rust, which would require a custom implementation in C for platforms that did not have Rust available - if that is even practical. Did I get that wrong?\n\n"},{"id":"486592","messageId":"00a801da443d$b1539670$13fac350$@nexbridge.com","threadId":"60721","inReplyTo":"ZZ9YrYvW_L9A02aI@tapette.crustytoothpaste.net","subject":"RE: [DISCUSS] Introducing Rust into the Git project","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-01-11T03:24:25Z","receivedAt":"2024-01-11T03:24:42Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Wednesday, January 10, 2024 9:56 PM, brian m. carlson wrote:\n>On 2024-01-10 at 23:52:21, rsbecker@nexbridge.com wrote:\n>> Unfortunately, none of the compiler frontends listed previously can be\n>> built for NonStop. These appear to all require gcc either directly or\n>> transitively, which cannot be ported to NonStop. I do not expect this\n>> to change any time soon - and is outside of my control anyway. An\n>> attempt was made to port Rust but it did not succeed primarily because\n>> of that dependency.\n>\n>Can you tell us what the technical limitations are that prevent GCC from being\n>ported so we can understand better?  I know LLVM doesn't support ia64, which you\n>do support, but GCC is very likely the most portable compiler on the planet and\n>supports architectures and OSes I've never otherwise heard of.\n>\n>I strongly suspect that if GCC did end up on NonStop, Rust would be able to be\n>ported, too, and you'd also get access to gccgo, which would make Git LFS possible\n>on NonStop as well[0].\n>\n>I'm not capable of porting GCC, but I have done some portability work in the Rust\n>ecosystem, and I'd be willing to provide context and some assistance (within my\n>time and capabilities) to help get Rust working on NonStop if you want.\n>\n>[0] For the record, as a maintainer of Git LFS, I'm happy to accept portability\n>patches for virtually any OS.\n\nThere are a number of issues for porting gcc (and Go). The list is fairly long, but the summary of what I encountered directly (on the last funded effort of 3) is:\n1. There are C syntax constructs required to do anything useful (required for access to the OS API) on NonStop that are not in gcc. I can hand code the parser for that, but it would take time.\n2. The Big Endian x86 architecture is weird to gcc and making that work is not easy.\n3. There is no assembler on NonStop.\n4. The ELF header is very different from standard.\n5. The symbol table structure is radically different, so debugging would be (nearly) impossible or impractical. gdb was ported to account for the platform differences.\n6. The linkage structure is similar but different from standard.\n7. The external fixup structure is radically different.\n8. The loader does not work the same way, so there are required sections of the ELF files on NonStop that are not generated by gcc.\n\nThere are more, but I just did not get to the point if hitting them. Part of my own issue is that I have expertise in parsing and semantic passes of compilers, but my code generation skills are not where I want them to be for taking on this effort. Our last funded attempt had a code generation expert and he gave up in frustration.\n\nIf I was hired on to do this, it might have a chance, but at an estimate (not mine) of 4-5 person years for a gcc port, best case, my $DAYJOB will not permit it.\n\nIf gcc could be ported to NonStop, it would solve so many problems. I have heard of numerous failed efforts beyond what was officially funded by various companies, so this is considered a high-risk project.\n\n"},{"id":"486594","messageId":"CABPp-BGmXw0NQ8yBaMiVXHiKr0-Y_jkZWmJB1CG_oc4UGxt_gA@mail.gmail.com","threadId":"60721","inReplyTo":"009c01da4439$f70beef0$e523ccd0$@nexbridge.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2024-01-11T05:06:23Z","receivedAt":"2024-01-11T05:06:37Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jan 10, 2024 at 6:57 PM <rsbecker@nexbridge.com> wrote:\n>\n> On Wednesday, January 10, 2024 9:21 PM, Elijah Newren wrote:\n> >On Wed, Jan 10, 2024 at 5:44 PM <rsbecker@nexbridge.com> wrote:\n> >>\n> >> On Wednesday, January 10, 2024 7:59 PM, Elijah Newren wrote:\n> >[...]\n> >> >Would you be okay with the following alternative: requiring that all\n> >> >Rust code be optional for now?\n> >> >\n> >> >(In other words, allow you to build with USE_RUST=0, or something\n> >> >like that.  And then we have both a Rust and a C implementation of\n> >> >anything that is required for backward compatibility, while any new\n> >> >Rust-only stuff would not be included in your build.)\n> >>\n> >> To address the immediate above, I assume this means that platform\n> >> maintainers will be responsible for developing non-portable\n> >> implementations that duplicate Rust functionality\n> >\n> >This doesn't at all sound like what I thought I said.  The whole proposal was so that\n> >folks like NonStop could continue using Git with no more work than setting\n> >USE_RUST=0 at build time.\n> >\n> >Why do you feel you'd need to duplicate any functionality?\n>\n> I think I misunderstood. What I took from this is that all new functionality would be in Rust, which would require a custom implementation in C for platforms that did not have Rust available - if that is even practical. Did I get that wrong?\n\nI think you somehow missed the word optional?\n\nI did say that new functionality should be allowed to be Rust only\n(unlike existing functionality), but I'm not sure how you leaped to\nassuming that all new functionality would be in Rust.  Further, I also\ndon't understand why you jump to assuming that all new functionality\nneeds to be supported on all platforms.  The point of the word\n\"optional\" in my proposal is that it is not required.  So, say, if\ngit-replay is in Rust, well you've never had git-replay before in any\nrelease, so you haven't lost any functionality by it being implemented\nin Rust.  And existing things (merge, cherry-pick, rebase, etc.)\ncontinue working with C-only code.  But you may have one less optional\naddition.\n\nAt least that was _my_ proposal -- that Rust be optional for now.  It\ndoes differ from what I think Taylor was originally proposing, but\nthat's why I brought it up as an alternative proposal.\n"},{"id":"486595","messageId":"14951ce7e4ec61712e5455b836dd625c@manjaro.org","threadId":"60721","inReplyTo":"CABPp-BFOmwV-xBtjvtenb6RFz9wx2VWVpTeho0k=D8wsCCVwqQ@mail.gmail.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Dragan Simic","fromEmail":"dsimic@manjaro.org","sentAt":"2024-01-11T05:33:50Z","receivedAt":"2024-01-11T05:33:54Z","isPatch":false,"sender":{"key":"dsimic@manjaro.org","avatar":null},"body":"On 2024-01-11 01:12, Elijah Newren wrote:\n> Another alternative (as discussed at Git Merge when we were last\n> talking about Rust[8]), is requiring all Rust code to be optional for\n> now.  If we choose to go that route, I think that means that (a) for\n> existing components, we have both a Rust and a C implementation\n> available, and (b) for new components (e.g. new top-level commands\n> like git-replay), they can be Rust-only and those compiling without\n> Rust just don't get them.\n\nTo me, this sounds like a horrible option, which is exactly what\nI earlier referred to as introducing fragmentation.\n"},{"id":"486596","messageId":"f5b9a57b6e2b513f1d79a93c6f0ccf45@manjaro.org","threadId":"60721","inReplyTo":"CABPp-BH3sva=CNtx8YFGP4Egyau-hR+7njZPFEd-DRTw91BK2w@mail.gmail.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Dragan Simic","fromEmail":"dsimic@manjaro.org","sentAt":"2024-01-11T05:39:13Z","receivedAt":"2024-01-11T05:39:15Z","isPatch":false,"sender":{"key":"dsimic@manjaro.org","avatar":null},"body":"On 2024-01-11 01:33, Elijah Newren wrote:\n> On Wed, Jan 10, 2024 at 1:57 PM Dragan Simic <dsimic@manjaro.org> \n> wrote:\n>> \n>> Thus, Git should probably follow the same approach of not converting \n>> the\n>> already existing code\n> \n> I disagree with this.  I saw significant performance improvements\n> through converting some existing Git code to Rust.  Granted, it was\n> only a small amount of code, but the performance benefits I saw\n> suggested we'd see more by also doing similar conversions elsewhere.\n> (Note that I kept the old C code and then conditionally compiled\n> either Rust or C versions of what I was converting.)\n\nWell, it's also possible that improving the old C code could also result \nin some performance improvements.  Thus, quite frankly, I don't see that \nas a valid argument to rewrite some existing C code in Rust.\n\n> Further, I found a really old bug from this effort as well[1], and I\n> find it extremely unlikely that I would have found that bug otherwise.\n> So, converting to Rust can even improve our existing C code.\n> \n>> , but frankly, I don't see what would actually be\n>> the \"new leafs\" written in Rust.\n> \n> In addition to some of the examples Junio mentioned elsewhere, I think\n> new toplevel commands, like git-replay, would qualify.\n> \n> \n> [1] Yeah, I really need to dig the patch out and send it in.  I'll do\n> so shortly.\n"},{"id":"486598","messageId":"ZZ-RH8L7H9vLcqoR@tanuki","threadId":"60721","inReplyTo":"CABPp-BGmXw0NQ8yBaMiVXHiKr0-Y_jkZWmJB1CG_oc4UGxt_gA@mail.gmail.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-11T06:56:31Z","receivedAt":"2024-01-11T06:56:37Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Jan 10, 2024 at 09:06:23PM -0800, Elijah Newren wrote:\n> On Wed, Jan 10, 2024 at 6:57 PM <rsbecker@nexbridge.com> wrote:\n> >\n> > On Wednesday, January 10, 2024 9:21 PM, Elijah Newren wrote:\n> > >On Wed, Jan 10, 2024 at 5:44 PM <rsbecker@nexbridge.com> wrote:\n> > >>\n> > >> On Wednesday, January 10, 2024 7:59 PM, Elijah Newren wrote:\n> > >[...]\n> > >> >Would you be okay with the following alternative: requiring that all\n> > >> >Rust code be optional for now?\n> > >> >\n> > >> >(In other words, allow you to build with USE_RUST=0, or something\n> > >> >like that.  And then we have both a Rust and a C implementation of\n> > >> >anything that is required for backward compatibility, while any new\n> > >> >Rust-only stuff would not be included in your build.)\n> > >>\n> > >> To address the immediate above, I assume this means that platform\n> > >> maintainers will be responsible for developing non-portable\n> > >> implementations that duplicate Rust functionality\n> > >\n> > >This doesn't at all sound like what I thought I said.  The whole proposal was so that\n> > >folks like NonStop could continue using Git with no more work than setting\n> > >USE_RUST=0 at build time.\n> > >\n> > >Why do you feel you'd need to duplicate any functionality?\n> >\n> > I think I misunderstood. What I took from this is that all new functionality would be in Rust, which would require a custom implementation in C for platforms that did not have Rust available - if that is even practical. Did I get that wrong?\n> \n> I think you somehow missed the word optional?\n> \n> I did say that new functionality should be allowed to be Rust only\n> (unlike existing functionality), but I'm not sure how you leaped to\n> assuming that all new functionality would be in Rust.  Further, I also\n> don't understand why you jump to assuming that all new functionality\n> needs to be supported on all platforms.  The point of the word\n> \"optional\" in my proposal is that it is not required.  So, say, if\n> git-replay is in Rust, well you've never had git-replay before in any\n> release, so you haven't lost any functionality by it being implemented\n> in Rust.  And existing things (merge, cherry-pick, rebase, etc.)\n> continue working with C-only code.  But you may have one less optional\n> addition.\n> \n> At least that was _my_ proposal -- that Rust be optional for now.  It\n> does differ from what I think Taylor was originally proposing, but\n> that's why I brought it up as an alternative proposal.\n\nThere are two ways to do this that I can see:\n\n  - New features may not be available on some platforms. I think this is\n    what Elijah had in mind.\n\n  - New features may require two implementations, one in C and one in\n    Rust. I think this is what Randall understood.\n\nUltimately, I think both alternatives would end up demoting platforms\nthat do not support Rust to become second-class citizens eventually.\nThis demotion is rather obvious in the case where new features may not\nbe available. But I also think that the second approach, where we\nprovide two implementations, would lead to a demotion of the Rust-less\nplatform because the alternate implementation in C would likely end up\nreceiving less attention than the Rust-based one. It's thus likely that\nthe implementation receiving less attention will deteriorate in code\nquality.\n\nI also think that once we start to accept Rust code, it will only be a\nmatter of time before we want to start using it in central code paths.\nRust does provide interfaces which are a lot nicer to use than the C\nbased ones, but it's hard to really reap the benefits unless we start to\nembrace Rust fully. Also, the most complex interfaces tend to be those\nwhich are deep inside our code base, like for example the object\ndatabase. They are thus also the most profitable targets for a Rust\nconversion, even though likely also the hardest to realize.\n\nTo me this feels like a slippery slope, and the deeper we go the more\nincentive we will have to drop platforms which do not support Rust\naltogether. So I can certainly see where Randall is coming from and why\nthis proposal is not something that he is thrilled about.\n\nPatrick\n"},{"id":"486621","messageId":"87v880m6r3.fsf@gentoo.org","threadId":"60721","inReplyTo":"ZZ77NQkSuiRxRDwt@nand.local","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2024-01-11T11:45:07Z","receivedAt":"2024-01-11T11:47:31Z","isPatch":false,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"Something I'm a bit concerned about is that right now, neither\nrustc_codegen_gcc nor gccrs are ready for use here.\n\nWe've had trouble getting things wired up for rustc_codegen_gcc\n- which is not to speak against their wonderful efforts - because\nthe Rust community hasn't yet figured out how to handle things which\npure rustc supports yet. See\ne.g. https://github.com/rust-lang/libc/pull/3032.\n\nI think care should be taken in citing rustc_codegen_gcc and gccrs\nas options for alternative platforms for now. They will hopefully\nbe great options in the future, but they aren't today, and they probably\nwon't be in the next 6 months at the least.\n\nWe also do use git heavily on platforms which rustc isn't supported\nyet.\n\nthanks,\nsam\n"},{"id":"486626","messageId":"012801da448f$29434ee0$7bc9eca0$@nexbridge.com","threadId":"60721","inReplyTo":"CABPp-BGmXw0NQ8yBaMiVXHiKr0-Y_jkZWmJB1CG_oc4UGxt_gA@mail.gmail.com","subject":"RE: [DISCUSS] Introducing Rust into the Git project","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-01-11T13:07:35Z","receivedAt":"2024-01-11T13:07:51Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Thursday, January 11, 2024 12:06 AM, Elijah Newren wrote:\n>On Wed, Jan 10, 2024 at 6:57 PM <rsbecker@nexbridge.com> wrote:\n>>\n>> On Wednesday, January 10, 2024 9:21 PM, Elijah Newren wrote:\n>> >On Wed, Jan 10, 2024 at 5:44 PM <rsbecker@nexbridge.com> wrote:\n>> >>\n>> >> On Wednesday, January 10, 2024 7:59 PM, Elijah Newren wrote:\n>> >[...]\n>> >> >Would you be okay with the following alternative: requiring that\n>> >> >all Rust code be optional for now?\n>> >> >\n>> >> >(In other words, allow you to build with USE_RUST=0, or something\n>> >> >like that.  And then we have both a Rust and a C implementation of\n>> >> >anything that is required for backward compatibility, while any\n>> >> >new Rust-only stuff would not be included in your build.)\n>> >>\n>> >> To address the immediate above, I assume this means that platform\n>> >> maintainers will be responsible for developing non-portable\n>> >> implementations that duplicate Rust functionality\n>> >\n>> >This doesn't at all sound like what I thought I said.  The whole\n>> >proposal was so that folks like NonStop could continue using Git with\n>> >no more work than setting\n>> >USE_RUST=0 at build time.\n>> >\n>> >Why do you feel you'd need to duplicate any functionality?\n>>\n>> I think I misunderstood. What I took from this is that all new functionality would\n>be in Rust, which would require a custom implementation in C for platforms that did\n>not have Rust available - if that is even practical. Did I get that wrong?\n>\n>I think you somehow missed the word optional?\n>\n>I did say that new functionality should be allowed to be Rust only (unlike existing\n>functionality), but I'm not sure how you leaped to assuming that all new\n>functionality would be in Rust.  Further, I also don't understand why you jump to\n>assuming that all new functionality needs to be supported on all platforms.  The\n>point of the word \"optional\" in my proposal is that it is not required.  So, say, if git-\n>replay is in Rust, well you've never had git-replay before in any release, so you\n>haven't lost any functionality by it being implemented in Rust.  And existing things\n>(merge, cherry-pick, rebase, etc.) continue working with C-only code.  But you may\n>have one less optional addition.\n>\n>At least that was _my_ proposal -- that Rust be optional for now.  It does differ from\n>what I think Taylor was originally proposing, but that's why I brought it up as an\n>alternative proposal.\n\nThank you for the clarification.\n\n"},{"id":"486637","messageId":"CABPp-BFWsWCGogqQ=haMsS4OhOdSwc3frcAxa6soQR5ORTceOA@mail.gmail.com","threadId":"60721","inReplyTo":"f5b9a57b6e2b513f1d79a93c6f0ccf45@manjaro.org","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2024-01-11T16:57:28Z","receivedAt":"2024-01-11T16:57:43Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi Dragan,\n\nOn Wed, Jan 10, 2024 at 9:39 PM Dragan Simic <dsimic@manjaro.org> wrote:\n>\n> On 2024-01-11 01:33, Elijah Newren wrote:\n> > On Wed, Jan 10, 2024 at 1:57 PM Dragan Simic <dsimic@manjaro.org>\n> > wrote:\n> >>\n> >> Thus, Git should probably follow the same approach of not converting\n> >> the\n> >> already existing code\n> >\n> > I disagree with this.  I saw significant performance improvements\n> > through converting some existing Git code to Rust.  Granted, it was\n> > only a small amount of code, but the performance benefits I saw\n> > suggested we'd see more by also doing similar conversions elsewhere.\n> > (Note that I kept the old C code and then conditionally compiled\n> > either Rust or C versions of what I was converting.)\n>\n> Well, it's also possible that improving the old C code could also result\n> in some performance improvements.  Thus, quite frankly, I don't see that\n> as a valid argument to rewrite some existing C code in Rust.\n\nYes, and I've made many performance improvements in the C code in git.\nSometimes I make some of the code 5% or 20% faster.  Sometimes 1-3\norders of magnitude faster.  Once over 60 orders of magnitude\nfaster.[1]  Look around in git's history; I've done a fair amount of\nperformance stuff.\n\nAnd I'm specifically arguing that I feel limited in some of the\nperformance work that can be done by remaining in C.  Part of my\nreason for interest in Rust is exactly because I think it can help us\nimprove performance in ways that are far more difficult to achieve in\nC.  And this isn't just guesswork, I've done some trials with it.\nFurther, I even took the time to document some of these reasons\nelsewhere in this thread[2].  Arguing that some performance\nimprovements can be done in C is thus entirely missing the point.\n\nIf you want to dismiss the performance angle of argument for Rust, you\nshould take the time to address the actual reasons raised for why it\ncould make it easier to improve performance relative to continuing in\nC.\n\nAlso, as a heads up since you seem to be relatively new to the list:\nyour position will probably carry more weight with others if you take\nthe time to understand, acknowledge, and/or address counterpoints of\nthe other party.  It is certainly fine to simply express some concerns\nwithout doing so (Randall and Patrick did a good job of this in this\nthread), but when you simply assert that the benefits others point out\nsimply don't exist (e.g. your \"Quite frankly, that would _only_\ncomplicate things and cause fragmentation.\" (emphasis added) from your\nfirst email in this thread[3], and which this latest email of yours\nsomewhat looks like as well), others may well start applying a\ndiscount to any positions you state.  Granted, it's totally up to you,\nbut I'm just giving a hint about how I think you might be able to be\nmore persuasive.\n\n\nHope that helps,\nElijah\n\n[1] A couple examples: 6a5fb966720 (\"Change default merge backend from\nrecursive to ort\", 2021-08-04) and 8d92fb29270 (\"dir: replace\nexponential algorithm with a linear one\", 2020-04-01)\n[2] Footnote 6 of\nhttps://lore.kernel.org/git/CABPp-BFOmwV-xBtjvtenb6RFz9wx2VWVpTeho0k=D8wsCCVwqQ@mail.gmail.com/\n[3] https://lore.kernel.org/git/b2651b38a4f7edaf1c5ffee72af00e46@manjaro.org/\n"},{"id":"486648","messageId":"CALNs47vfBH9u9B5B3tWRoEkJJiqne5067A4CFnZ3OaMVvz_gSg@mail.gmail.com","threadId":"60721","inReplyTo":"00a801da443d$b1539670$13fac350$@nexbridge.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Trevor Gross","fromEmail":"tmgross@umich.edu","sentAt":"2024-01-11T20:07:47Z","receivedAt":"2024-01-11T20:07:59Z","isPatch":false,"sender":{"key":"tmgross@umich.edu","avatar":null},"body":"On Wed, Jan 10, 2024 at 10:24 PM <rsbecker@nexbridge.com> wrote:\n>\n> There are a number of issues for porting gcc (and Go). The list is fairly long, but the summary of what I encountered directly (on the last funded effort of 3) is:\n> 1. There are C syntax constructs required to do anything useful (required for access to the OS API) on NonStop that are not in gcc. I can hand code the parser for that, but it would take time.\n> 2. The Big Endian x86 architecture is weird to gcc and making that work is not easy.\n> 3. There is no assembler on NonStop.\n> 4. The ELF header is very different from standard.\n> 5. The symbol table structure is radically different, so debugging would be (nearly) impossible or impractical. gdb was ported to account for the platform differences.\n> 6. The linkage structure is similar but different from standard.\n> 7. The external fixup structure is radically different.\n> 8. The loader does not work the same way, so there are required sections of the ELF files on NonStop that are not generated by gcc.\n>\n> There are more, but I just did not get to the point if hitting them. Part of my own issue is that I have expertise in parsing and semantic passes of compilers, but my code generation skills are not where I want them to be for taking on this effort. Our last funded attempt had a code generation expert and he gave up in frustration.\n>\n> If I was hired on to do this, it might have a chance, but at an estimate (not mine) of 4-5 person years for a gcc port, best case, my $DAYJOB will not permit it.\n>\n> If gcc could be ported to NonStop, it would solve so many problems. I have heard of numerous failed efforts beyond what was officially funded by various companies, so this is considered a high-risk project.\n\nOut of curiosity - does the Tandem compiler (assuming that is the\ncorrect name) have a backend that is usable as a library or via an IR?\n\nIf so, maybe it would be possible to write a rustc_codegen_tandem\nbackend like the three that exist (rustc_codegen_{llvm,gcc,cranelift}\nat [1]. GCC and cranelift are still under development). This way you\nsidestep a lot of the codegen-specific problems listed above.\n\nI am, of course, not suggesting this as a solution for git and am sure\nyou would rather have GCC support. But I wonder how feasible this\nwould be if Rust on NonStop is desired at some point.\n\n[1]: https://github.com/rust-lang/rust/tree/062e7c6a951c1e4f33c0a6f6761755949cde15ec/compiler\n"},{"id":"486660","messageId":"01c101da44d5$175f1100$461d3300$@nexbridge.com","threadId":"60721","inReplyTo":"CALNs47vfBH9u9B5B3tWRoEkJJiqne5067A4CFnZ3OaMVvz_gSg@mail.gmail.com","subject":"RE: [DISCUSS] Introducing Rust into the Git project","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-01-11T21:28:10Z","receivedAt":"2024-01-11T21:28:43Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Thursday, January 11, 2024 3:08 PM, Trevor Gross wrote:\n>On Wed, Jan 10, 2024 at 10:24 PM <rsbecker@nexbridge.com> wrote:\n>>\n>> There are a number of issues for porting gcc (and Go). The list is fairly long, but the\n>summary of what I encountered directly (on the last funded effort of 3) is:\n>> 1. There are C syntax constructs required to do anything useful (required for\n>access to the OS API) on NonStop that are not in gcc. I can hand code the parser for\n>that, but it would take time.\n>> 2. The Big Endian x86 architecture is weird to gcc and making that work is not\n>easy.\n>> 3. There is no assembler on NonStop.\n>> 4. The ELF header is very different from standard.\n>> 5. The symbol table structure is radically different, so debugging would be (nearly)\n>impossible or impractical. gdb was ported to account for the platform differences.\n>> 6. The linkage structure is similar but different from standard.\n>> 7. The external fixup structure is radically different.\n>> 8. The loader does not work the same way, so there are required sections of the\n>ELF files on NonStop that are not generated by gcc.\n>>\n>> There are more, but I just did not get to the point if hitting them. Part of my own\n>issue is that I have expertise in parsing and semantic passes of compilers, but my\n>code generation skills are not where I want them to be for taking on this effort. Our\n>last funded attempt had a code generation expert and he gave up in frustration.\n>>\n>> If I was hired on to do this, it might have a chance, but at an estimate (not mine)\n>of 4-5 person years for a gcc port, best case, my $DAYJOB will not permit it.\n>>\n>> If gcc could be ported to NonStop, it would solve so many problems. I have heard\n>of numerous failed efforts beyond what was officially funded by various companies,\n>so this is considered a high-risk project.\n>\n>Out of curiosity - does the Tandem compiler (assuming that is the correct name)\n>have a backend that is usable as a library or via an IR?\n>\n>If so, maybe it would be possible to write a rustc_codegen_tandem backend like the\n>three that exist (rustc_codegen_{llvm,gcc,cranelift}\n>at [1]. GCC and cranelift are still under development). This way you sidestep a lot of\n>the codegen-specific problems listed above.\n>\n>I am, of course, not suggesting this as a solution for git and am sure you would\n>rather have GCC support. But I wonder how feasible this would be if Rust on\n>NonStop is desired at some point.\n\nThe usable compilers and interpreters on NonStop are c89, c99 (what we use for git), c11, perl, and python3 (for the x86 only). The perl and python do not have sufficient modules to do what would be needed by git. The compilers are invoked using a CLI and are not callable using a library. gcc is, for all intents and purposes, not possible - so anything requiring gcc (for example, Rust), cannot be built.  There is no back-end pluggable component for any of the compilers.\n\n"},{"id":"486672","messageId":"CALNs47vSMV4XcFqNPBorAGftU0h87nhP7HJbJx2oWfvKxqtO3g@mail.gmail.com","threadId":"60721","inReplyTo":"01c101da44d5$175f1100$461d3300$@nexbridge.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Trevor Gross","fromEmail":"tmgross@umich.edu","sentAt":"2024-01-11T23:23:11Z","receivedAt":"2024-01-11T23:23:25Z","isPatch":false,"sender":{"key":"tmgross@umich.edu","avatar":null},"body":"On Thu, Jan 11, 2024 at 4:28 PM <rsbecker@nexbridge.com> wrote:\n>\n> The usable compilers and interpreters on NonStop are c89, c99 (what we use for git), c11, perl, and python3 (for the x86 only). The perl and python do not have sufficient modules to do what would be needed by git. The compilers are invoked using a CLI and are not callable using a library. gcc is, for all intents and purposes, not possible - so anything requiring gcc (for example, Rust), cannot be built.  There is no back-end pluggable component for any of the compilers.\n>\n\nAh, no pluggable backend is unfortunate. Rust only uses GCC to build\nthe LLVM backend, it isn't actually needed for the language. It does\nlink libgcc_s for unwinding and I believe some math symbols, but\nunwinding can be disabled and other symbols can come from anywhere.\n\nIf you can build mrustc (C++ program) [1] then you can use it to\ntranspile Rust to C. This is how rustc is bootstrapped, and would be\nhow you bring it up with a different backend on a new platform.\n\nStill, this probably wouldn't be a solution for git.\n\n[1]: https://github.com/thepowersgang/mrustc\n"},{"id":"486674","messageId":"ZaB-ayQuGqrS-mL0@tapette.crustytoothpaste.net","threadId":"60721","inReplyTo":"87v880m6r3.fsf@gentoo.org","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-01-11T23:48:59Z","receivedAt":"2024-01-11T23:49:07Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-01-11 at 11:45:07, Sam James wrote:\n> Something I'm a bit concerned about is that right now, neither\n> rustc_codegen_gcc nor gccrs are ready for use here.\n> \n> We've had trouble getting things wired up for rustc_codegen_gcc\n> - which is not to speak against their wonderful efforts - because\n> the Rust community hasn't yet figured out how to handle things which\n> pure rustc supports yet. See\n> e.g. https://github.com/rust-lang/libc/pull/3032.\n\nIs this simply library support in the libc crate?  That's very easy to add.\n\n> I think care should be taken in citing rustc_codegen_gcc and gccrs\n> as options for alternative platforms for now. They will hopefully\n> be great options in the future, but they aren't today, and they probably\n> won't be in the next 6 months at the least.\n\nWhat specifically is missing for rust_codegen_gcc?  I know gccrs is not\nready at the moment, but I was under the impression that\nrust_codegen_gcc was at least usable.  I'm aware it requires some\npatches to GCC, but distros should be able to carry those.\n\nIf rust_codegen_gcc isn't viable, then I agree we should avoid making\nRust mandatory, but I'd like to learn more.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"486675","messageId":"CALNs47s3tUQoOD4ejdoTn6y12ywjL0j5hWU-fUnBLe_o3vV5SQ@mail.gmail.com","threadId":"60721","inReplyTo":"ZZ77NQkSuiRxRDwt@nand.local","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Trevor Gross","fromEmail":"tmgross@umich.edu","sentAt":"2024-01-11T23:53:01Z","receivedAt":"2024-01-11T23:53:15Z","isPatch":false,"sender":{"key":"tmgross@umich.edu","avatar":null},"body":"On Wed, Jan 10, 2024 at 3:19 PM Taylor Blau <me@ttaylorr.com> wrote:\n>\n> Over the holiday break at the end of last year I spent some time\n> thinking on what it would take to introduce Rust into the Git project.\n>\n> There is significant work underway to introduce Rust into the Linux\n> kernel (see [1], [2]). Among their stated goals, I think there are a few\n> which could be potentially relevant to the Git project:\n>\n>   - Lower risk of memory safety bugs, data races, memory leaks, etc.\n>     thanks to the language's safety guarantees.\n>\n>   - Easier to gain confidence when refactoring or introducing new code\n>     in Rust (assuming little to no use of the language's `unsafe`\n>     feature).\n>\n>   - Contributing to Git becomes easier and accessible to a broader group\n>     of programmers by relying on a more modern language.\n>\n> Given the allure of these benefits, I think it's at least worth\n> considering and discussing how Rust might make its way into Junio's\n> tree.\n>\n> I imagine that the transition state would involve some parts of the\n> project being built in C and calling into Rust code via FFI (and perhaps\n> vice-versa, with Rust code calling back into the existing C codebase).\n> Luckily for us, Rust's FFI provides a zero-cost abstraction [3], meaning\n> there is no performance impact when calling code from one language in\n> the other.\n>\n> Some open questions from me, at least to get the discussion going are:\n>\n>   1. Platform support. The Rust compiler (rustc) does not enjoy the same\n>      widespread availability that C compilers do. For instance, I\n>      suspect that NonStop, AIX, Solaris, among others may not be\n>      supported.\n>\n>      One possible alternative is to have those platforms use a Rust\n>      front-end for a compiler that they do support. The gccrs [4]\n>      project would allow us to compile Rust anywhere where GCC is\n>      available. The rustc_codegen_gcc [5] project uses GCC's libgccjit\n>      API to target GCC from rustc itself.\n>\n>   2. Migration. What parts of Git are easiest to convert to Rust? My\n>      hunch is that the answer is any stand-alone libraries, like\n>      strbuf.h. I'm not sure how we should identify these, though, and in\n>      what order we would want to move them over.\n>\n>   3. Interaction with the lib-ification effort. There is lots of work\n>      going on in an effort to lib-ify much of the Git codebase done by\n>      Google. I'm not sure how this would interact with that effort, but\n>      we should make sure that one isn't a blocker for the other.\n>\n> I'm curious to hear what others think about this. I think that this\n> would be an exciting and worthwhile direction for the project. Let's\n> see!\n>\n> Thanks,\n> Taylor\n>\n> [1]: https://rust-for-linux.com/\n> [2]: https://lore.kernel.org/rust-for-linux/20210414184604.23473-1-ojeda@kernel.org/\n> [3]: https://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywhere.html#c-talking-to-rust\n> [4]: https://github.com/Rust-GCC/gccrs\n> [5]: https://github.com/rust-lang/rustc_codegen_gcc\n>\n\nTwo good reference codebases out there:\n\nAbstractions over libgit2\n    Repo: https://github.com/rust-lang/git2-rs\n    Docs: https://docs.rs/git2/latest/git2/\n\ngix, a WIP reimplementation of git. This is far from complete but does\na lot of threading / async to apparently get quite fast.\n    Repo: https://github.com/Byron/gitoxide\n    Docs: https://docs.rs/gix/latest/gix/\n\nIf the git project does decide to go forward with this, there is\nprobably a lot of completed work that can be pulled from either of\nthose sources.\n"},{"id":"486700","messageId":"87jzofrlm4.fsf@gentoo.org","threadId":"60721","inReplyTo":"ZaB-ayQuGqrS-mL0@tapette.crustytoothpaste.net","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2024-01-12T08:24:46Z","receivedAt":"2024-01-12T08:39:52Z","isPatch":false,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"\n\"brian m. carlson\" <sandals@crustytoothpaste.net> writes:\n\n> [[PGP Signed Part:Undecided]]\n> On 2024-01-11 at 11:45:07, Sam James wrote:\n>> Something I'm a bit concerned about is that right now, neither\n>> rustc_codegen_gcc nor gccrs are ready for use here.\n>> \n>> We've had trouble getting things wired up for rustc_codegen_gcc\n>> - which is not to speak against their wonderful efforts - because\n>> the Rust community hasn't yet figured out how to handle things which\n>> pure rustc supports yet. See\n>> e.g. https://github.com/rust-lang/libc/pull/3032.\n>\n> Is this simply library support in the libc crate?  That's very easy to add.\n\n[CC'd the rustc_codegen_gcc maintainer as well as some folks who have\ntried using rustc_codegen_gcc for their distributions.]\n\nEvidently not on the last point? ;)\n\nEven just patching it in downstream isn't easy because you then have to\ndo it for many many packages. But after that PR stalling because of the\npolicy issue, there wasn't really anywhere to go, because of the\nchicken-and-egg situation.\n\nLet alone then, once the libc crate has it, going around and wiring up\nin other crates.\n\nThe discussion on the PR seems clear that the intention is to not add\nit until some policy is revised/formulated? I also don't want to have\nto have that debate with every crate just because rustc doesn't support\nit.\n\n>\n>> I think care should be taken in citing rustc_codegen_gcc and gccrs\n>> as options for alternative platforms for now. They will hopefully\n>> be great options in the future, but they aren't today, and they probably\n>> won't be in the next 6 months at the least.\n>\n> What specifically is missing for rust_codegen_gcc?  I know gccrs is not\n> ready at the moment, but I was under the impression that\n> rust_codegen_gcc was at least usable.  I'm aware it requires some\n> patches to GCC, but distros should be able to carry those.\n>\n> If rust_codegen_gcc isn't viable, then I agree we should avoid making\n> Rust mandatory, but I'd like to learn more.\n\nIt's in a general state of instability. There's still *very* active work\nongoing in libgccjit (by the rust_codegen_gcc maintainer).\n\nI'd say \"you need to patch your GCC\" is probably not a good state of\naffairs for using something critical like git anyway, but even then,\nI'm not aware of anyone having used it to build real-world common\napplications using Rust for a non-rustc-supported platform, at least\nnot then using those builds day-to-day.\n\nSo, even if we were willing to chase the active flurry of libgccjit\npatches (which is wonderful to see!), it's a significant moving\ntarget. In Gentoo, we're probably better-placed than most people\nto be able to do that, but it's still a lot of work and it doesn't\nsound very robust for us to be doing for core infrastructure.\n\nWe have a lot of packages in Gentoo - partly actually stuff in the\nPython ecosystem - where we're very excited to be able to use\nrust_codegen_gcc (or gccrs, whichever comes first inreadiness, surely\nrust_codegen_gcc) for alt platforms, but it's just not there yet.\n"},{"id":"486710","messageId":"b6fc8dd7be709b350bd50fa507ae03f4c3c89d8d.camel@zoho.com","threadId":"60721","inReplyTo":"87jzofrlm4.fsf@gentoo.org","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Antoni Boucher","fromEmail":"bouanto@zoho.com","sentAt":"2024-01-12T14:46:54Z","receivedAt":"2024-01-12T14:47:15Z","isPatch":false,"sender":{"key":"bouanto@zoho.com","avatar":null},"body":"While usable, there are a few things missing in rustc_codegen_gcc:\n\n * Unwinding doesn't work correctly when compiling Rust code in release\nmode.\n * Rustup distribution: might not be mandatory, but I guess it would be\nvery helpful to have an easy way to install rustc_codegen_gcc and being\nable to pin to a specific version.\n * Debug info: again might not be mandatory, but would be helpful.\n * Have not been tested on many platforms: these platforms had a few\ntests, so while it's possible to use Rust on them, that doesn't mean\neverything works (in particular, I know that changes will be needed to\nboth the Rust spec file and the standard library — or its tests — for\nm68k): SuperH, ARC, m68k [1] and there's currently someone\nexperimenting on AVR. Related to the platform support, could you please\nsend me a list of platforms where git is officially supported?\n * Not sure if it would be needed, but the new inline asm syntax is not\nsupported on architectures not supported by rustc.\n * I also expect bad compilation in some cases.\n\n> Is this simply library support in the libc crate?  That's very easy\nto add.\n\nWe might also need to update the object crate.\n\nAs for the progress, we plan to have most of the patches merged for\nlibgccjit 14, but one important one will be missing because it's not\nready (the one for try/catch that is necessary to support Rust panics).\nI expect there will be much less patches for libgccjit 15: probably\ntry/catch and bug fixing for the most part.\nWe also plan to have rustup distribution in the coming months, so\nthat's something that will help for adoption.\nAlong with rustup distribution, we plan on making architectures\ncurrently not supported by rustc usable more easily in the coming\nmonths.\n\nRecently, I built and ran the tests of a dozen of the most popular\ncrates and all of their tests passed [2]. And rustc_codegen_gcc was\nalready able to build the Rust compiler in March 2022 and while not\ncompletely working, the resulting compiler could compile a \"Hello,\nworld!\" [3].\n\n[1] https://github.com/rust-lang/rustc_codegen_gcc/wiki\n[2]\nhttps://blog.antoyo.xyz/rustc_codegen_gcc-progress-report-26#state_of_compiling_popular_crates\n[3] https://blog.antoyo.xyz/rustc_codegen_gcc-progress-report-10\n\nOn Fri, 2024-01-12 at 08:24 +0000, Sam James wrote:\n> \n> \"brian m. carlson\" <sandals@crustytoothpaste.net> writes:\n> \n> > [[PGP Signed Part:Undecided]]\n> > On 2024-01-11 at 11:45:07, Sam James wrote:\n> > > Something I'm a bit concerned about is that right now, neither\n> > > rustc_codegen_gcc nor gccrs are ready for use here.\n> > > \n> > > We've had trouble getting things wired up for rustc_codegen_gcc\n> > > - which is not to speak against their wonderful efforts - because\n> > > the Rust community hasn't yet figured out how to handle things\n> > > which\n> > > pure rustc supports yet. See\n> > > e.g. https://github.com/rust-lang/libc/pull/3032.\n> > \n> > Is this simply library support in the libc crate?  That's very easy\n> > to add.\n> \n> [CC'd the rustc_codegen_gcc maintainer as well as some folks who have\n> tried using rustc_codegen_gcc for their distributions.]\n> \n> Evidently not on the last point? ;)\n> \n> Even just patching it in downstream isn't easy because you then have\n> to\n> do it for many many packages. But after that PR stalling because of\n> the\n> policy issue, there wasn't really anywhere to go, because of the\n> chicken-and-egg situation.\n> \n> Let alone then, once the libc crate has it, going around and wiring\n> up\n> in other crates.\n> \n> The discussion on the PR seems clear that the intention is to not add\n> it until some policy is revised/formulated? I also don't want to have\n> to have that debate with every crate just because rustc doesn't\n> support\n> it.\n> \n> > \n> > > I think care should be taken in citing rustc_codegen_gcc and\n> > > gccrs\n> > > as options for alternative platforms for now. They will hopefully\n> > > be great options in the future, but they aren't today, and they\n> > > probably\n> > > won't be in the next 6 months at the least.\n> > \n> > What specifically is missing for rust_codegen_gcc?  I know gccrs is\n> > not\n> > ready at the moment, but I was under the impression that\n> > rust_codegen_gcc was at least usable.  I'm aware it requires some\n> > patches to GCC, but distros should be able to carry those.\n> > \n> > If rust_codegen_gcc isn't viable, then I agree we should avoid\n> > making\n> > Rust mandatory, but I'd like to learn more.\n> \n> It's in a general state of instability. There's still *very* active\n> work\n> ongoing in libgccjit (by the rust_codegen_gcc maintainer).\n> \n> I'd say \"you need to patch your GCC\" is probably not a good state of\n> affairs for using something critical like git anyway, but even then,\n> I'm not aware of anyone having used it to build real-world common\n> applications using Rust for a non-rustc-supported platform, at least\n> not then using those builds day-to-day.\n> \n> So, even if we were willing to chase the active flurry of libgccjit\n> patches (which is wonderful to see!), it's a significant moving\n> target. In Gentoo, we're probably better-placed than most people\n> to be able to do that, but it's still a lot of work and it doesn't\n> sound very robust for us to be doing for core infrastructure.\n> \n> We have a lot of packages in Gentoo - partly actually stuff in the\n> Python ecosystem - where we're very excited to be able to use\n> rust_codegen_gcc (or gccrs, whichever comes first inreadiness, surely\n> rust_codegen_gcc) for alt platforms, but it's just not there yet.\n\n"},{"id":"486936","messageId":"45bfda3a350b4040a28a25993a2b22e0@manjaro.org","threadId":"60721","inReplyTo":"CABPp-BFWsWCGogqQ=haMsS4OhOdSwc3frcAxa6soQR5ORTceOA@mail.gmail.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Dragan Simic","fromEmail":"dsimic@manjaro.org","sentAt":"2024-01-17T21:30:45Z","receivedAt":"2024-01-17T21:30:54Z","isPatch":false,"sender":{"key":"dsimic@manjaro.org","avatar":null},"body":"On 2024-01-11 17:57, Elijah Newren wrote:\n> Hi Dragan,\n\nI apologize for my delayed response.\n\n> On Wed, Jan 10, 2024 at 9:39 PM Dragan Simic <dsimic@manjaro.org> \n> wrote:\n>> \n>> On 2024-01-11 01:33, Elijah Newren wrote:\n>> > On Wed, Jan 10, 2024 at 1:57 PM Dragan Simic <dsimic@manjaro.org>\n>> > wrote:\n>> >>\n>> >> Thus, Git should probably follow the same approach of not converting\n>> >> the\n>> >> already existing code\n>> >\n>> > I disagree with this.  I saw significant performance improvements\n>> > through converting some existing Git code to Rust.  Granted, it was\n>> > only a small amount of code, but the performance benefits I saw\n>> > suggested we'd see more by also doing similar conversions elsewhere.\n>> > (Note that I kept the old C code and then conditionally compiled\n>> > either Rust or C versions of what I was converting.)\n>> \n>> Well, it's also possible that improving the old C code could also \n>> result\n>> in some performance improvements.  Thus, quite frankly, I don't see \n>> that\n>> as a valid argument to rewrite some existing C code in Rust.\n> \n> Yes, and I've made many performance improvements in the C code in git.\n> Sometimes I make some of the code 5% or 20% faster.  Sometimes 1-3\n> orders of magnitude faster.  Once over 60 orders of magnitude\n> faster.[1]  Look around in git's history; I've done a fair amount of\n> performance stuff.\n\nThank you very much for your work!\n\n> And I'm specifically arguing that I feel limited in some of the\n> performance work that can be done by remaining in C.  Part of my\n> reason for interest in Rust is exactly because I think it can help us\n> improve performance in ways that are far more difficult to achieve in\n> C.  And this isn't just guesswork, I've done some trials with it.\n> Further, I even took the time to document some of these reasons\n> elsewhere in this thread[2].  Arguing that some performance\n> improvements can be done in C is thus entirely missing the point.\n> \n> If you want to dismiss the performance angle of argument for Rust, you\n> should take the time to address the actual reasons raised for why it\n> could make it easier to improve performance relative to continuing in\n> C.\n> \n> Also, as a heads up since you seem to be relatively new to the list:\n> your position will probably carry more weight with others if you take\n> the time to understand, acknowledge, and/or address counterpoints of\n> the other party.  It is certainly fine to simply express some concerns\n> without doing so (Randall and Patrick did a good job of this in this\n> thread), but when you simply assert that the benefits others point out\n> simply don't exist (e.g. your \"Quite frankly, that would _only_\n> complicate things and cause fragmentation.\" (emphasis added) from your\n> first email in this thread[3], and which this latest email of yours\n> somewhat looks like as well), others may well start applying a\n> discount to any positions you state.  Granted, it's totally up to you,\n> but I'm just giving a hint about how I think you might be able to be\n> more persuasive.\n\nI totally agree with your suggestions, and I'm thankful for the time it \ntook you to write it all down.  I'll take your advice and refrain myself \nfrom expressing my opinions in this thread.\n\n> [1] A couple examples: 6a5fb966720 (\"Change default merge backend from\n> recursive to ort\", 2021-08-04) and 8d92fb29270 (\"dir: replace\n> exponential algorithm with a linear one\", 2020-04-01)\n> [2] Footnote 6 of\n> https://lore.kernel.org/git/CABPp-BFOmwV-xBtjvtenb6RFz9wx2VWVpTeho0k=D8wsCCVwqQ@mail.gmail.com/\n> [3] \n> https://lore.kernel.org/git/b2651b38a4f7edaf1c5ffee72af00e46@manjaro.org/\n"},{"id":"487235","messageId":"CAJoAoZnHGTFhfR6e6r=GMSfVbSNgLoHF-opaWYLbHppiuzi+Rg@mail.gmail.com","threadId":"60721","inReplyTo":"007c01da4420$10a7b700$31f72500$@nexbridge.com","subject":"Defining a platform support policy (Was: [DISCUSS] Introducing Rust into the Git project)","fromName":"Emily Shaffer","fromEmail":"nasamuffin@google.com","sentAt":"2024-01-22T23:17:50Z","receivedAt":"2024-01-22T23:18:08Z","isPatch":false,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Wed, Jan 10, 2024 at 3:52 PM <rsbecker@nexbridge.com> wrote:\n>\n> On Wednesday, January 10, 2024 5:26 PM, Taylor Blau wrote:\n> >On Wed, Jan 10, 2024 at 05:15:53PM -0500, rsbecker@nexbridge.com wrote:\n> >> Just a brief concern: Rust is not broadly portable. Adding another\n> >> dependency to git will remove many existing platforms from future releases.\n> >> Please consider this carefully before going down this path.\n> >\n> >I was hoping to hear from you as one of the few (only?) folks who participate on\n> >the list and represent HPE NonStop users.\n> >\n> >I'm curious which if any of the compiler frontends that I listed in my earlier email\n> >would work for you.\n>\n> Unfortunately, none of the compiler frontends listed previously can be built for NonStop. These appear to all require gcc either directly or transitively, which cannot be ported to NonStop. I do not expect this to change any time soon - and is outside of my control anyway. An attempt was made to port Rust but it did not succeed primarily because of that dependency. Similarly, Golang is also not portable to NonStop because of architecture assumptions made by the Go team that cannot be satisfied on NonStop at this time. If some of the memory/pointer issues are the primary concern, c11 might be something acceptable with smart pointers. C17 will eventually be deployable, but is not available on most currently supported OS versions on the platform.\n\nI hope y'all don't mind me hijacking this part of the thread ;)\n\nBut, Randall's remarks bring up something pretty compelling: I don't\nthink Git has a clearly defined platform support policy. As far as I\ncan tell, the support policy now is \"if you run `make test` on it and\nbreaks, and you let us know, we'll try to fix it\" - without much in\nthe way of additional caveats. If I look in CodingGuidelines I see a\nfew \"this doesn't work on platform X so don't do it\" (like around %z\nin printf), but nowhere do I see \"how to know if your platform is\nsupported\" or even \"here are platforms we have heard Git works OK on\".\n\nThat causes a lot of confusion for the project - threads like this one\n(and presumably a similar one about C99 adoption) become a blend of\n\"is this change good for the project or not?\" and \"will this change\nleave behind platform X?\" that is difficult to pick apart.\n\nDoes it make sense for us to formalize a support policy? For example,\nif we wanted to formalize the status quo, I could envision:\n\n\"\"\"\nPlatform support: We make a best-effort attempt to solve any bugs\nreported to the list, regardless of platform. To prevent breakages in\nthe first place, consider running Git's `make test` regularly on your\nplatform and reporting the results to git@vger.kernel.org; or, better\nyet, consider adding your platform to the GitHub Actions CI\n(configured in `.github/`).\n\"\"\"\n\nOr, if we wanted to be able to move very nimbly, we could imagine\nsomething much more restrictive (note that I'm not endorsing it, just\nillustrating):\n\n\"\"\"\nPlatform support: Git is guaranteed to work well on Linux platforms\nusing a kernel version that is less than 1 year old. Support for all\nother platforms is best-effort; when reporting a bug on another\nplatform, you may need to patch the issue and verify your fix\nyourself.\n\"\"\"\n\nI suspect there's a happy medium in here somewhere - trying to fix (or\navoid) an issue on a platform which the average developer cannot run\ntests on is not a recipe for a happy developer, and a general policy\nof \"patches welcome\" for anything but latest Linux is not a recipe for\nhappy users.\n\nI see a few axes we can play with:\n * which architectures/kernels/OS (do we care about more than the\nusual suspects of Linux/Mac/Windows // x86/amd/arm //\nPOSIX-compliant?)\n * age of architectures/kernels (do we care to offer full support for\na 10 or 15 year old OS?)\n * new feature compatibility guarantees vs. core\nfunctionality/security fix guarantees (which do we really define\n\"support\" as?)\n * test provisioning (do we require a VM we can run CI on, or is a\nreport generated from a nightly build and mailed to the list OK?)\n * test/breakage timing (should the above tests run on every commit to\n'next'? every merge to 'master'? every RC?)\n * who provides the support (is it the patch author's responsibility\nto fix late-breaking platform support bugs? is it the reporter's\nresponsibility? and especially, how does this interplay with test\nprovisioning and frequency above?)\n\nIf we had clearer answers to these questions, it'd be much simpler to\ndetermine whether experimentation with Rust is possible or useful.\nPlus it would make developer lives easier, in general, to understand\nhow much compatibility support work they're potentially signing up for\nwhen sending a change of any size.\n\n - Emily\n"},{"id":"487238","messageId":"038701da4d90$b1bfb010$153f1030$@nexbridge.com","threadId":"60721","inReplyTo":"CAJoAoZnHGTFhfR6e6r=GMSfVbSNgLoHF-opaWYLbHppiuzi+Rg@mail.gmail.com","subject":"RE: Defining a platform support policy (Was: [DISCUSS] Introducing Rust into the Git project)","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-01-23T00:11:15Z","receivedAt":"2024-01-23T00:11:44Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On Monday, January 22, 2024 6:18 PM, Emily Shaffer wrote:\n>To: Randall S. Becker <rsbecker@nexbridge.com>\n>Cc: Taylor Blau <me@ttaylorr.com>; Junio C Hamano <gitster@pobox.com>; Dragan\n>Simic <dsimic@manjaro.org>; Git List <git@vger.kernel.org>; Johannes Schindelin\n><Johannes.Schindelin@gmx.de>\n>Subject: Defining a platform support policy (Was: [DISCUSS] Introducing Rust into\n>the Git project)\n>\n>On Wed, Jan 10, 2024 at 3:52 PM <rsbecker@nexbridge.com> wrote:\n>>\n>> On Wednesday, January 10, 2024 5:26 PM, Taylor Blau wrote:\n>> >On Wed, Jan 10, 2024 at 05:15:53PM -0500, rsbecker@nexbridge.com wrote:\n>> >> Just a brief concern: Rust is not broadly portable. Adding another\n>> >> dependency to git will remove many existing platforms from future releases.\n>> >> Please consider this carefully before going down this path.\n>> >\n>> >I was hoping to hear from you as one of the few (only?) folks who\n>> >participate on the list and represent HPE NonStop users.\n>> >\n>> >I'm curious which if any of the compiler frontends that I listed in\n>> >my earlier email would work for you.\n>>\n>> Unfortunately, none of the compiler frontends listed previously can be built for\n>NonStop. These appear to all require gcc either directly or transitively, which cannot\n>be ported to NonStop. I do not expect this to change any time soon - and is outside\n>of my control anyway. An attempt was made to port Rust but it did not succeed\n>primarily because of that dependency. Similarly, Golang is also not portable to\n>NonStop because of architecture assumptions made by the Go team that cannot be\n>satisfied on NonStop at this time. If some of the memory/pointer issues are the\n>primary concern, c11 might be something acceptable with smart pointers. C17 will\n>eventually be deployable, but is not available on most currently supported OS\n>versions on the platform.\n>\n>I hope y'all don't mind me hijacking this part of the thread ;)\n\nI'm happy you did this. The topic is crucial - if nowhere else but to my ability to sleep at night. Preserving Emily's comments without snipping as these are important questions and comments.\n\n>But, Randall's remarks bring up something pretty compelling: I don't think Git has a\n>clearly defined platform support policy. As far as I can tell, the support policy now is\n>\"if you run `make test` on it and breaks, and you let us know, we'll try to fix it\" -\n>without much in the way of additional caveats. If I look in CodingGuidelines I see a\n>few \"this doesn't work on platform X so don't do it\" (like around %z in printf), but\n>nowhere do I see \"how to know if your platform is supported\" or even \"here are\n>platforms we have heard Git works OK on\".\n>\n>That causes a lot of confusion for the project - threads like this one (and presumably\n>a similar one about C99 adoption) become a blend of \"is this change good for the\n>project or not?\" and \"will this change leave behind platform X?\" that is difficult to\n>pick apart.\n>\n>Does it make sense for us to formalize a support policy? For example, if we wanted\n>to formalize the status quo, I could envision:\n>\n>\"\"\"\n>Platform support: We make a best-effort attempt to solve any bugs reported to the\n>list, regardless of platform. To prevent breakages in the first place, consider running\n>Git's `make test` regularly on your platform and reporting the results to\n>git@vger.kernel.org; or, better yet, consider adding your platform to the GitHub\n>Actions CI (configured in `.github/`).\n>\"\"\"\n>\n>Or, if we wanted to be able to move very nimbly, we could imagine something much\n>more restrictive (note that I'm not endorsing it, just\n>illustrating):\n>\n>\"\"\"\n>Platform support: Git is guaranteed to work well on Linux platforms using a kernel\n>version that is less than 1 year old. Support for all other platforms is best-effort;\n>when reporting a bug on another platform, you may need to patch the issue and\n>verify your fix yourself.\n>\"\"\"\n>\n>I suspect there's a happy medium in here somewhere - trying to fix (or\n>avoid) an issue on a platform which the average developer cannot run tests on is\n>not a recipe for a happy developer, and a general policy of \"patches welcome\" for\n>anything but latest Linux is not a recipe for happy users.\n>\n>I see a few axes we can play with:\n> * which architectures/kernels/OS (do we care about more than the usual suspects\n>of Linux/Mac/Windows // x86/amd/arm //\n>POSIX-compliant?)\n> * age of architectures/kernels (do we care to offer full support for a 10 or 15 year\n>old OS?)\n> * new feature compatibility guarantees vs. core functionality/security fix\n>guarantees (which do we really define \"support\" as?)\n> * test provisioning (do we require a VM we can run CI on, or is a report generated\n>from a nightly build and mailed to the list OK?)\n> * test/breakage timing (should the above tests run on every commit to 'next'?\n>every merge to 'master'? every RC?)\n> * who provides the support (is it the patch author's responsibility to fix late-\n>breaking platform support bugs? is it the reporter's responsibility? and especially,\n>how does this interplay with test provisioning and frequency above?)\n>\n>If we had clearer answers to these questions, it'd be much simpler to determine\n>whether experimentation with Rust is possible or useful.\n>Plus it would make developer lives easier, in general, to understand how much\n>compatibility support work they're potentially signing up for when sending a change\n>of any size.\n\nI think we might want to add some considerations to the above list that go beyond what other projects use, OpenSSL as an example:\n\n* Can support for exotic platforms be delegated to some \"community\" support concept. In NonStop's case, I currently do 99% of the verification that each release runs properly. If I am able to provide a fix, I will. We have been fortunate that most problems/solutions have been of general interest and impact, with my platforms being more of a \"Canary in the Coalmine\" situation where we just encounter it first because of edge conditions, but other platforms may be impacted. The problem here is time of how long a designated community support person(s) can keep supporting git and what happens when they (me) retire or get hit by a bus. Like all good NonStop people, I have a backup, so git does not need to worry about me specifically.\n\n* What is the broad impact of dropping support for a platform that has a significant investment in git, where loss of support could have societal impact. There are platforms with 100 million to 1000 million lines of code managed by git today. This type of investment-related impact is specific to git compared to other Open-Source products. Leaving debit or credit card authorizers without a supported git would be, let's say, \"bad\".\n\n* Could stakeholders be consulted before changing support levels? Yes, I get that commercial fee-based products hit this more than Open-Source. Looking at other products in the Open-Source space, there are fee-based support models that could be developed for long-term support (beyond the obvious LTS-type considerations - see OpenSSL's model for reference). A related question is: \"If there is a bug detected in git, what version is the oldest supported git version to which a fix can be made?\" 2.0.0? 1.8.0 (looking at some Linux distro's RPM or APT repositories as being seriously guilty here). My MacOS server (just a couple of years ago) came with 1.8.0. I can't answer that question if asked by a customer. An alternative model, which seems to be informally embraced by git is \"please upgrade to the latest - or a fix on the latest few releases\". But this position puts pressure on the team to maintain platform compatibility for indefinite periods.\n\n* What level of compatibility will be more appropriate to ensure git's reputation as the gold-standard version control platform? Without a board of directors, or at least an advisory board, this might not be answerable (or even decidable). That role has been taken up, by intent and/or because it has to be done, by our quite awesome committers.\n\nThis last two point put a serious amount of pressure for compatibility on both the customer and the git dev team to keep compatibility in the latest release with all platforms, especially the exotic platforms - mine included - although dropping those is not a good approach either. As an example (not intended as guidance) If we said something like git 2.40.0 is LTS until September 2026, and security fixes (and critical functional ones) will be done going back to that version, and anything older requires an extended fee-based support contract, I think it would make some organizations more comfortable with the support model. This is not a panacea and there are some obviously difficult concerns here - causing git's support model to vary from some Linux distro models from the earliest inception of both products.\n\nI don't have a good answer to any of this that would satisfy everyone. I'm not sure there is one.\n--Randall\n\n"},{"id":"487240","messageId":"xmqqil3kriul.fsf@gitster.g","threadId":"60721","inReplyTo":"CAJoAoZnHGTFhfR6e6r=GMSfVbSNgLoHF-opaWYLbHppiuzi+Rg@mail.gmail.com","subject":"Re: Defining a platform support policy","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-01-23T00:31:14Z","receivedAt":"2024-01-23T00:31:20Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer <nasamuffin@google.com> writes:\n\n> But, Randall's remarks bring up something pretty compelling: I don't\n> think Git has a clearly defined platform support policy. As far as I\n> can tell, the support policy now is \"if you run `make test` on it and\n> breaks, and you let us know, we'll try to fix it\"\n\nI doubt this part.  If there is somebody motivated enough among us\nwho has access to such a platform, then that person may try to fix\nit and if the fix is not too ugly, I may accept such a patch as the\nupstream maintainer.  So your \"you let us know we'll try\" does not\nreflect reality at all.  The major platforms luckily have such\nmotivated somebody almost always available for them.  Niche ones,\nperhaps not.\n\n> ..., but nowhere do I see \"how to know if your platform is\n> supported\" or even \"here are platforms we have heard Git works OK on\".\n\nYup.  Patches, with commitments to keep such lists up-to-date, are\nvery much welcome.\n\nThanks.\n"},{"id":"487243","messageId":"xmqq34uorhmc.fsf@gitster.g","threadId":"60721","inReplyTo":"038701da4d90$b1bfb010$153f1030$@nexbridge.com","subject":"Re: Defining a platform support policy","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-01-23T00:57:47Z","receivedAt":"2024-01-23T00:57:53Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"<rsbecker@nexbridge.com> writes:\n\n> I think we might want to add some considerations to the above list\n> that go beyond what other projects use, OpenSSL as an example:\n\n[jc: if you want to have a meaningful discussion on this list,\nplease stick to a reasonable line width.  I'll rewrap your lines\nbelow].\n\n> * Can support for exotic platforms be delegated to some\n> \"community\" support concept. In NonStop's case, I currently do 99%\n> of the verification that each release runs properly. If I am able\n> to provide a fix, I will. We have been fortunate that most\n> problems/solutions have been of general interest and impact, with\n> my platforms being more of a \"Canary in the Coalmine\" situation\n> where we just encounter it first because of edge conditions, but\n> other platforms may be impacted. The problem here is time of how\n> long a designated community support person(s) can keep supporting\n> git and what happens when they (me) retire or get hit by a\n> bus. Like all good NonStop people, I have a backup, so git does\n> not need to worry about me specifically.\n\nThere are platform packagers that deliver binary releases, and we do\nnot have to worry about them.  We _could_ have a tier of minority\nplatform that we can treat pretty much the same as these packagers,\ni.e. the \"community supported version of Git for platform X\" might\nconsist of many patches on top of what I release, and some patches\nthat are acceptable quality may be given upstream, but there may\nneed hacks that are too ugly to live in my tree, which the\n\"community edition\" may have to keep outside the upstream.  Even in\nsuch a case, if they try to engage with this list, they will often\nfind somebody willing to help them polish such \"ugly hacks\" into\nacceptable patches.\n\n> * What is the broad impact of dropping support for a platform that\n> has a significant investment in git, where loss of support could\n> have societal impact. There are platforms with 100 million to 1000\n> million lines of code managed by git today. This type of\n> investment-related impact is specific to git compared to other\n> Open-Source products. Leaving debit or credit card authorizers\n> without a supported git would be, let's say, \"bad\".\n\nLet's say we may want to start requiring new enough version of\nlibrary that is not yet ported to a minority platform.  Do we deeply\ncare?  It depends but \"investment-related impact\" is unlikely cause\nfor us to personally care.  But those $CORPS who will feel the\n\"investment-related impact\" are welcome to hire quality developers\nand to these employed developers, the \"impact\" might become an issue\nthey care more deeply about.\n\n> * Could stakeholders be consulted before changing support levels?\n> Yes, I get that commercial fee-based products hit this more than\n> Open-Source. Looking at other products in the Open-Source space,\n> there are fee-based support models that could be developed for\n> long-term support (beyond the obvious LTS-type considerations -\n> see OpenSSL's model for reference).\n\nThe stakeholders are already consulted, aren't they?  Every time we\nmake noises like \"let's raise the minimum version of Perl we\nrequire\", we discuss it here.  They have to monitor this list, of\ncourse, and if they lack people to do so, then they may have to\ninvest in it.\n\n> A related question is: \"If there is a bug detected in git, what\n> version is the oldest supported git version to which a fix can be\n> made?\"\n\nThis is a good question.  The latest security-induced maintenance\nrelease was Git 2.40.1 done in March 2023 and the fixes go back to\nthe v2.30 track, and Git 2.30.0 was done at the end of 2021.  This\nwindow was unusually generous from our usual standard, IIRC, so I\nwould say roughly speaking 2 years is the maximum.\n\n> ... But this position puts pressure on the team to maintain\n> platform compatibility for indefinite periods.\n\nSure, but I think we should just say something like \"18 months to 24\nmonths\", if you want backport of a fix to older track, you can do so\nyourself.\n\nThe story is probably the same if a minority platform that lacks\nrecent enough dependencies (e.g. libraries) and stop linking\ncorrectly.  If you care deeply enough, you should be ready to invest\nyourself in porting such dependencies.  We can help, but the primary\ndriving force for porting issues ought to be folks with stake in the\nplatform.  We as the project won't bend over backwards and keep\neverybody else to an ancient version of the dependency if some\nplatforms cannot catch up with the time.\n\n\n"},{"id":"487294","messageId":"CABPp-BGfPXKtdHaz0u5273AwUfBnYRKfMa2VHPFohv5fOtwJtg@mail.gmail.com","threadId":"60721","inReplyTo":"45bfda3a350b4040a28a25993a2b22e0@manjaro.org","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2024-01-24T04:15:37Z","receivedAt":"2024-01-24T04:15:52Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi Dragan,\n\nOn Wed, Jan 17, 2024 at 1:30 PM Dragan Simic <dsimic@manjaro.org> wrote:\n>\n> On 2024-01-11 17:57, Elijah Newren wrote:\n> > Hi Dragan,\n>\n> I apologize for my delayed response.\n\nNo worries; I'm often hit or miss on my responses these days as well.\n\n> > On Wed, Jan 10, 2024 at 9:39 PM Dragan Simic <dsimic@manjaro.org>\n> > wrote:\n> >>\n> >> On 2024-01-11 01:33, Elijah Newren wrote:\n> >> > On Wed, Jan 10, 2024 at 1:57 PM Dragan Simic <dsimic@manjaro.org>\n> >> > wrote:\n> >> >>\n> >> >> Thus, Git should probably follow the same approach of not converting\n> >> >> the\n> >> >> already existing code\n> >> >\n> >> > I disagree with this.  I saw significant performance improvements\n> >> > through converting some existing Git code to Rust.  Granted, it was\n> >> > only a small amount of code, but the performance benefits I saw\n> >> > suggested we'd see more by also doing similar conversions elsewhere.\n> >> > (Note that I kept the old C code and then conditionally compiled\n> >> > either Rust or C versions of what I was converting.)\n> >>\n> >> Well, it's also possible that improving the old C code could also\n> >> result\n> >> in some performance improvements.  Thus, quite frankly, I don't see\n> >> that\n> >> as a valid argument to rewrite some existing C code in Rust.\n> >\n> > Yes, and I've made many performance improvements in the C code in git.\n> > Sometimes I make some of the code 5% or 20% faster.  Sometimes 1-3\n> > orders of magnitude faster.  Once over 60 orders of magnitude\n> > faster.[1]  Look around in git's history; I've done a fair amount of\n> > performance stuff.\n>\n> Thank you very much for your work!\n>\n> > And I'm specifically arguing that I feel limited in some of the\n> > performance work that can be done by remaining in C.  Part of my\n> > reason for interest in Rust is exactly because I think it can help us\n> > improve performance in ways that are far more difficult to achieve in\n> > C.  And this isn't just guesswork, I've done some trials with it.\n> > Further, I even took the time to document some of these reasons\n> > elsewhere in this thread[2].  Arguing that some performance\n> > improvements can be done in C is thus entirely missing the point.\n> >\n> > If you want to dismiss the performance angle of argument for Rust, you\n> > should take the time to address the actual reasons raised for why it\n> > could make it easier to improve performance relative to continuing in\n> > C.\n> >\n> > Also, as a heads up since you seem to be relatively new to the list:\n> > your position will probably carry more weight with others if you take\n> > the time to understand, acknowledge, and/or address counterpoints of\n> > the other party.  It is certainly fine to simply express some concerns\n> > without doing so (Randall and Patrick did a good job of this in this\n> > thread), but when you simply assert that the benefits others point out\n> > simply don't exist (e.g. your \"Quite frankly, that would _only_\n> > complicate things and cause fragmentation.\" (emphasis added) from your\n> > first email in this thread[3], and which this latest email of yours\n> > somewhat looks like as well), others may well start applying a\n> > discount to any positions you state.  Granted, it's totally up to you,\n> > but I'm just giving a hint about how I think you might be able to be\n> > more persuasive.\n>\n> I totally agree with your suggestions, and I'm thankful for the time it\n> took you to write it all down.  I'll take your advice\n\nGreat!\n\n> and refrain myself\n> from expressing my opinions in this thread.\n\n...but that's not what my advice was.  My advice was that you'd be\nmore persuasive if you expressed your opinions differently.  Some\npossible examples:\n\n  * Stating that you are worried about the codebase becoming more\ncomplicated or more fragmented (without dismissing the points Taylor\nraised)\n  * Arguing that you believe various points others raised aren't as\nmuch of an advantage as they perceive, or even potentially aren't even\nan advantage at all, not by mere assertion but by providing additional\ndetails on the topic (statistics, anecdotes, war stories,\ncounter-examples, old commit messages, etc.) that back up your point\n  * Stating that you don't understand why others think that advantages\nthey state are as significant as they pose and ask for clarification.\n\nI think there's potentially some good points behind your positions,\nand I don't want to discourage them.  I want to encourage lively,\nfriendly debate so that we can have the best information possible when\ndecisions are made.\n"},{"id":"487296","messageId":"0db21f3b8068e5df3fe2e84b1c6aea9b@manjaro.org","threadId":"60721","inReplyTo":"CABPp-BGfPXKtdHaz0u5273AwUfBnYRKfMa2VHPFohv5fOtwJtg@mail.gmail.com","subject":"Re: [DISCUSS] Introducing Rust into the Git project","fromName":"Dragan Simic","fromEmail":"dsimic@manjaro.org","sentAt":"2024-01-24T05:14:09Z","receivedAt":"2024-01-24T05:14:17Z","isPatch":false,"sender":{"key":"dsimic@manjaro.org","avatar":null},"body":"Hello Elijah,\n\nOn 2024-01-24 05:15, Elijah Newren wrote:\n> On Wed, Jan 17, 2024 at 1:30 PM Dragan Simic <dsimic@manjaro.org> \n> wrote:\n>> On 2024-01-11 17:57, Elijah Newren wrote:\n>>> On Wed, Jan 10, 2024 at 9:39 PM Dragan Simic <dsimic@manjaro.org> \n>>> wrote:\n>> and refrain myself\n>> from expressing my opinions in this thread.\n> \n> ...but that's not what my advice was.  My advice was that you'd be\n> more persuasive if you expressed your opinions differently.  Some\n> possible examples:\n> \n>   * Stating that you are worried about the codebase becoming more\n> complicated or more fragmented (without dismissing the points Taylor\n> raised)\n>   * Arguing that you believe various points others raised aren't as\n> much of an advantage as they perceive, or even potentially aren't even\n> an advantage at all, not by mere assertion but by providing additional\n> details on the topic (statistics, anecdotes, war stories,\n> counter-examples, old commit messages, etc.) that back up your point\n>   * Stating that you don't understand why others think that advantages\n> they state are as significant as they pose and ask for clarification.\n> \n> I think there's potentially some good points behind your positions,\n> and I don't want to discourage them.  I want to encourage lively,\n> friendly debate so that we can have the best information possible when\n> decisions are made.\n\nOh, I once again totally agree!  I really love the way you expressed it,\nwhich I'm once again thankful for.\n\nI always support making improvements and major changes, introducing\nnew technologies, augmenting or even replacing old technologies, etc.\nIn the end, that's how progress is made, but such major changes also\nneed to be performed very carefully, in a controlled way that provides\na backup plan, and based on solid facts and past experiences.\n\nIn this specific case, please be aware that my health wasn't in great\nshape, because I was (and still am) recovering from some nasty flu,\nwhich has also effectively diminished my mental capacities.  That's\nthe primary reason why I provided only terse comments, without backing\nthem up with more specific details.  That's not the way I usually\noperate, and I apologize for that.\n"},{"id":"487308","messageId":"CABPp-BHk3KXRBfNdfU8gUpZrRMwBu_YwMSGfPN-NOaNDqEUXoA@mail.gmail.com","threadId":"60721","inReplyTo":"CAJoAoZnHGTFhfR6e6r=GMSfVbSNgLoHF-opaWYLbHppiuzi+Rg@mail.gmail.com","subject":"Re: Defining a platform support policy (Was: [DISCUSS] Introducing Rust into the Git project)","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2024-01-24T07:54:00Z","receivedAt":"2024-01-24T07:55:33Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Jan 22, 2024 at 3:18 PM Emily Shaffer <nasamuffin@google.com> wrote:\n>\n> On Wed, Jan 10, 2024 at 3:52 PM <rsbecker@nexbridge.com> wrote:\n> >\n[...]\n> > Unfortunately, none of the compiler frontends listed previously can be built for NonStop. These appear to all require gcc either directly or transitively, which cannot be ported to NonStop. I do not expect this to change any time soon - and is outside of my control anyway. An attempt was made to port Rust but it did not succeed primarily because of that dependency. Similarly, Golang is also not portable to NonStop because of architecture assumptions made by the Go team that cannot be satisfied on NonStop at this time. If some of the memory/pointer issues are the primary concern, c11 might be something acceptable with smart pointers. C17 will eventually be deployable, but is not available on most currently supported OS versions on the platform.\n>\n> I hope y'all don't mind me hijacking this part of the thread ;)\n\nOf course not.  :-)\n\n[...]\n> Does it make sense for us to formalize a support policy?\n\nSome hurdles that may need to be overcome if we want to do so:\n\n* For a significant number of the discussions I remember, a\nsignificant challenge was that we don't even know which platforms Git\nis used on.  That's why we sometimes agree to weather balloon patches\nthat attempt to use some new option, in a way that is really easy to\nremove...and if no one complains for a long enough time, then we\npresume all platforms support it and start adding hard dependencies on\nit.\n* We are often happy to try to fix issues on even obscure platforms if\nwe get a detailed enough description showing exactly what the problem\nis\n* However, when reports don't come with a complete diagnosis, we often\nwill tell people who are reporting issues that we don't have access to\nsuch a platform and someone else will have to dig further.  This\nhappens more often for exotic platforms (AIX, NonStop, etc.) but also\nhappens with mainstream platforms (Mac, Windows, and I think I've even\nseen it happen with Linux).\n* Even when folks report that they can't help the reporter, the work\ndoesn't always go back to the reporter, because someone else on the\nlist may respond and dig in; that happens more for mainstream\nplatforms but can happen with the exotic platforms as well.\n* How exactly can we even enforce continued platform support?  What's\nthe actual mechanism?  I think the only route available to us is\npeople who care and try to provide reports, testing, patches, new\ntools (e.g. our CI runs and gitgitgadget providing reports across\nseveral of the more common platforms, with lots of work to investigate\nthe occasional weird build issues and flakes so it continues to be\nfairly reliable), but what happens if some of those developers start\ncaring less...and yet we still have an encoded policy that their\nplatforms are supported?\n\nI generally think we value portability fairly highly, but it clearly\nhas bounds...fuzzy and even unknown-by-us bounds.  I don't know how to\ntranslate that into a policy, and I'm curious if trying to apply nice\nsharp boundaries risks unreasonable expectations on either or both\nsides.\n\nAlso...\n\n[...]\n\n> I see a few axes we can play with:\n>  * which architectures/kernels/OS (do we care about more than the\n> usual suspects of Linux/Mac/Windows // x86/amd/arm //\n> POSIX-compliant?)\n>  * age of architectures/kernels (do we care to offer full support for\n> a 10 or 15 year old OS?)\n>  * new feature compatibility guarantees vs. core\n> functionality/security fix guarantees (which do we really define\n> \"support\" as?)\n>  * test provisioning (do we require a VM we can run CI on, or is a\n> report generated from a nightly build and mailed to the list OK?)\n>  * test/breakage timing (should the above tests run on every commit to\n> 'next'? every merge to 'master'? every RC?)\n>  * who provides the support (is it the patch author's responsibility\n> to fix late-breaking platform support bugs? is it the reporter's\n> responsibility? and especially, how does this interplay with test\n> provisioning and frequency above?)\n\nThat's a great list of questions, but to me it does seem to lean\ntowards \"whatever is supported is supported equally\".  I don't know if\nthat was intended, or just the way I read it.  But if it was intended,\nI'd say that while equal support may be an ideal, I suspect it is\npragmatically just too expensive as evidenced by the many optional\nfeatures we already have, many (all?) of which have roots in platform\nsupport or the lack thereof:\n\n  * gitk (NO_TCLTK)\n  * dumb http(s) transport (NO_EXPAT)\n  * smart http(s) transport (NO_CURL)\n  * perl regexes (USE_LIBPRCRE)\n  * translations (NO_GETTEXT)\n  * charset conversions (NO_ICONV)\n  * p4 support (NO_PYTHON, affected other scripts in the past too)\n  * svn, send-email, gitweb support (NO_PERL, affected other stuff in\nthe past too)\n  * fsmonitor (FSMONITOR_DAEMON_BACKEND)\n\nAlso, this list isn't just an \"exotic\" vs. \"mainstream\" platform\nthing, since even Linux is \"second class\" in the final category[1].\n\nSo, I think if we create a \"supported platforms\" policy, it should\naddress optional features as well (though perhaps as simply as \"the\nsupport policy only applies to non-optional parts of Git\").\n\n[1] https://lore.kernel.org/git/pull.1352.v5.git.git.1670882286.gitgitgadget@gmail.com/\n\n> If we had clearer answers to these questions, it'd be much simpler to\n> determine whether experimentation with Rust is possible or useful.\n\nHow so?\n"}]}