{"thread":{"id":"55509","subject":"[PATCH] CodingGuidelines: remove suggestion to write commands in Perl/SH","startedAt":"2021-04-17T08:44:18Z","lastAt":"2021-04-17T22:11:08Z","messageCount":9,"participants":["Ævar Arnfjörð Bjarmason","Torsten Bögershausen","Bagas Sanjaya","brian m. carlson","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"422154","messageId":"patch-1.1-83266f30b67-20210417T084346Z-avarab@gmail.com","threadId":"55509","inReplyTo":null,"subject":"[PATCH] CodingGuidelines: remove suggestion to write commands in Perl/SH","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-04-17T08:43:54Z","receivedAt":"2021-04-17T08:44:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove a suggestion to write new commands in Perl or Shell to\nexperiment. This advice was added in 6d0618a820a (Add\nDocumentation/CodingGuidelines, 2007-11-08).\n\nSince then the consensus changed to having no new such commands unless\nnecessary, and existing ones have been actively migrated to C.\n\nSigned-off-by: ��var Arnfj��r�� Bjarmason \n---\n Documentation/CodingGuidelines | 5 -----\n 1 file changed, 5 deletions(-)\n\ndiff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines\nindex 45465bc0c98..b9cd55db6a8 100644\n--- a/Documentation/CodingGuidelines\n+++ b/Documentation/CodingGuidelines\n@@ -416,11 +416,6 @@ For C programs:\n that are made available to it by including one of the header files\n it must include by the previous rule.\n \n- - If you are planning a new command, consider writing it in shell\n- or perl first, so that changes in semantics can be easily\n- changed and discussed. Many Git commands started out like\n- that, and a few are still scripts.\n-\n - Avoid introducing a new dependency into Git. This means you\n usually should stay away from scripting languages not already\n used in the Git core command set (unless your command is clearly\n-- \n2.31.1.687.g1d87aeed692\n\n"},{"id":"422157","messageId":"20210417085335.ib6m4jagiekeulec@tb-raspi4","threadId":"55509","inReplyTo":"patch-1.1-83266f30b67-20210417T084346Z-avarab@gmail.com","subject":"Re: [PATCH] CodingGuidelines: remove suggestion to write commands in Perl/SH","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2021-04-17T08:53:35Z","receivedAt":"2021-04-17T08:54:42Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Sat, Apr 17, 2021 at 10:43:54AM +0200, Ævar Arnfjörð Bjarmason wrote:\n> Remove a suggestion to write new commands in Perl or Shell to\n> experiment. This advice was added in 6d0618a820a (Add\n> Documentation/CodingGuidelines, 2007-11-08).\n>\n> Since then the consensus changed to having no new such commands unless\n> necessary, and existing ones have been actively migrated to C.\n>\n> Signed-off-by: ??var Arnfj??r?? Bjarmason \n\nThe patch looks good - but the Umlauts are garbled:\nISO-2022-JP is used ?\n\n> ---\n> Documentation/CodingGuidelines | 5 -----\n> 1 file changed, 5 deletions(-)\n>\n> diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines\n> index 45465bc0c98..b9cd55db6a8 100644\n> --- a/Documentation/CodingGuidelines\n> +++ b/Documentation/CodingGuidelines\n> @@ -416,11 +416,6 @@ For C programs:\n> that are made available to it by including one of the header files\n> it must include by the previous rule.\n>\n> - - If you are planning a new command, consider writing it in shell\n> - or perl first, so that changes in semantics can be easily\n> - changed and discussed. Many Git commands started out like\n> - that, and a few are still scripts.\n> -\n> - Avoid introducing a new dependency into Git. This means you\n> usually should stay away from scripting languages not already\n> used in the Git core command set (unless your command is clearly\n> --\n> 2.31.1.687.g1d87aeed692\n>\n"},{"id":"422161","messageId":"bcc64c2d-3469-38ab-3234-fa8984a3d0f9@gmail.com","threadId":"55509","inReplyTo":"patch-1.1-83266f30b67-20210417T084346Z-avarab@gmail.com","subject":"Re: [PATCH] CodingGuidelines: remove suggestion to write commands in Perl/SH","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-04-17T12:17:32Z","receivedAt":"2021-04-17T12:17:38Z","isPatch":true,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On 17/04/21 15.43, Ævar Arnfjörð Bjarmason wrote:\n> Since then the consensus changed to having no new such commands unless\n> necessary, and existing ones have been actively migrated to C.\n\nWhat I implied that when we need to implement new commands, it must\nbe directly written in C (steeper learning curve and more tedious\nthan implemented in shell script), so I'm against this proposal.\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"422162","messageId":"patch-1.1-98b1f938f7d-20210417T122959Z-avarab@gmail.com","threadId":"55509","inReplyTo":"patch-1.1-83266f30b67-20210417T084346Z-avarab@gmail.com","subject":"[PATCH v2] CodingGuidelines: remove suggestion to write commands in Perl/SH","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-04-17T12:31:27Z","receivedAt":"2021-04-17T12:31:40Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove a suggestion to write new commands in Perl or Shell to\nexperiment. This advice was added in 6d0618a820a (Add\nDocumentation/CodingGuidelines, 2007-11-08).\n\nSince then the consensus changed to having no new such commands unless\nnecessary, and existing ones have been actively migrated to C.\n\nSo this isn't a new proposal or a suggestion to change the coding\nstyle, but bringing this stale part of the CodingGuidelines in line\nwith reality.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n\nV1 of this was misencoded. I sent it as UTF-8 with\ncharset=ISO-2022-JP.\n\nI'd been hacking the test-lib.sh, and at one point it escaped its\nquarantine and started setting config in my main git.git, and I'd\nmanaged to miss that i18n.logOutputEncoding encoding variable when cleaning it up.\n\nRange-diff against v1:\n1: 83266f30b67 ! 1: 98b1f938f7d CodingGuidelines: remove suggestion to write commands in Perl/SH\n @@ Commit message\n Since then the consensus changed to having no new such commands unless\n necessary, and existing ones have been actively migrated to C.\n \n + So this isn't a new proposal or a suggestion to change the coding\n + style, but bringing this stale part of the CodingGuidelines in line\n + with reality.\n +\n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n ## Documentation/CodingGuidelines ##\n\n Documentation/CodingGuidelines | 5 -----\n 1 file changed, 5 deletions(-)\n\ndiff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines\nindex 45465bc0c98..b9cd55db6a8 100644\n--- a/Documentation/CodingGuidelines\n+++ b/Documentation/CodingGuidelines\n@@ -416,11 +416,6 @@ For C programs:\n that are made available to it by including one of the header files\n it must include by the previous rule.\n \n- - If you are planning a new command, consider writing it in shell\n- or perl first, so that changes in semantics can be easily\n- changed and discussed. Many Git commands started out like\n- that, and a few are still scripts.\n-\n - Avoid introducing a new dependency into Git. This means you\n usually should stay away from scripting languages not already\n used in the Git core command set (unless your command is clearly\n-- \n2.31.1.722.g788886f50a2\n\n"},{"id":"422163","messageId":"87r1j91427.fsf@evledraar.gmail.com","threadId":"55509","inReplyTo":"bcc64c2d-3469-38ab-3234-fa8984a3d0f9@gmail.com","subject":"Re: [PATCH] CodingGuidelines: remove suggestion to write commands in Perl/SH","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-04-17T12:36:00Z","receivedAt":"2021-04-17T12:36:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sat, Apr 17 2021, Bagas Sanjaya wrote:\n\n> On 17/04/21 15.43, Ævar Arnfjörð Bjarmason wrote:\n>> Since then the consensus changed to having no new such commands unless\n>> necessary, and existing ones have been actively migrated to C.\n>\n> What I implied that when we need to implement new commands, it must\n> be directly written in C (steeper learning curve and more tedious\n> than implemented in shell script), so I'm against this proposal.\n\nI updated the v2 of this to note that I'm not really proposing anything\nnew, but just bringing the document in line with reality. For a long\ntime now we've rejected any new non-C things being imported into the\ntree, unless those that fall under the \"such as an importer to convert\nrandom-scm-X\" language that's still retained in the CodingGuidelines.\n\nI think that even if you or someone else wanted to write a new thing in\nPerl or SH we'd want a new way of doing that now anyway,\ne.g. git-send-email.perl should really be a helper for a C program\nrather than a stand-alone thing.\n\nI.e. even if the main logic is still in the *.perl that code would\nreally benefit from first doing the command-line parsing etc. in C, so\nthat it would behave the same as everything else in that regard, ditto\nfor its config parsing.\n\n"},{"id":"422187","messageId":"YHtFBeWxE2cFlShY@camp.crustytoothpaste.net","threadId":"55509","inReplyTo":"87r1j91427.fsf@evledraar.gmail.com","subject":"Re: [PATCH] CodingGuidelines: remove suggestion to write commands in Perl/SH","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2021-04-17T20:28:53Z","receivedAt":"2021-04-17T20:29:03Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2021-04-17 at 12:36:00, Ævar Arnfjörð Bjarmason wrote:\n> \n> On Sat, Apr 17 2021, Bagas Sanjaya wrote:\n> \n> > On 17/04/21 15.43, Ævar Arnfjörð Bjarmason wrote:\n> >> Since then the consensus changed to having no new such commands unless\n> >> necessary, and existing ones have been actively migrated to C.\n> >\n> > What I implied that when we need to implement new commands, it must\n> > be directly written in C (steeper learning curve and more tedious\n> > than implemented in shell script), so I'm against this proposal.\n> \n> I updated the v2 of this to note that I'm not really proposing anything\n> new, but just bringing the document in line with reality. For a long\n> time now we've rejected any new non-C things being imported into the\n> tree, unless those that fall under the \"such as an importer to convert\n> random-scm-X\" language that's still retained in the CodingGuidelines.\n> \n> I think that even if you or someone else wanted to write a new thing in\n> Perl or SH we'd want a new way of doing that now anyway,\n> e.g. git-send-email.perl should really be a helper for a C program\n> rather than a stand-alone thing.\n\nI'm also kind of opposed to this change. For example, I plan on adding\na utility to fill in SHA-1 compatibility things for SHA-256 repos, and\nthat will be written in shell. The performance benefit of C here is\ngoing to be minimal, especially considering the fact that people will be\nrunning it literally at most once per repo, so I don't see a reason to\nspend a lot of time writing C code.\n\nI'm not of the opinion that we should never have shell or Perl code in\nour project, nor does it intrinsically make sense to migrate everything\nto C. Typically we've done that because it performs better, especially\non Windows, but there are many situations in which those are not major\nconsiderations and shell or Perl can be a desirable approach.\n-- \nbrian m. carlson (he/him or they/them)\nHouston, Texas, US\n"},{"id":"422190","messageId":"87k0p01tje.fsf@evledraar.gmail.com","threadId":"55509","inReplyTo":"YHtFBeWxE2cFlShY@camp.crustytoothpaste.net","subject":"Re: [PATCH] CodingGuidelines: remove suggestion to write commands in Perl/SH","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-04-17T21:37:57Z","receivedAt":"2021-04-17T21:38:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sat, Apr 17 2021, brian m. carlson wrote:\n\n> On 2021-04-17 at 12:36:00, Ævar Arnfjörð Bjarmason wrote:\n>> \n>> On Sat, Apr 17 2021, Bagas Sanjaya wrote:\n>> \n>> > On 17/04/21 15.43, Ævar Arnfjörð Bjarmason wrote:\n>> >> Since then the consensus changed to having no new such commands unless\n>> >> necessary, and existing ones have been actively migrated to C.\n>> >\n>> > What I implied that when we need to implement new commands, it must\n>> > be directly written in C (steeper learning curve and more tedious\n>> > than implemented in shell script), so I'm against this proposal.\n>> \n>> I updated the v2 of this to note that I'm not really proposing anything\n>> new, but just bringing the document in line with reality. For a long\n>> time now we've rejected any new non-C things being imported into the\n>> tree, unless those that fall under the \"such as an importer to convert\n>> random-scm-X\" language that's still retained in the CodingGuidelines.\n>> \n>> I think that even if you or someone else wanted to write a new thing in\n>> Perl or SH we'd want a new way of doing that now anyway,\n>> e.g. git-send-email.perl should really be a helper for a C program\n>> rather than a stand-alone thing.\n>\n> I'm also kind of opposed to this change. For example, I plan on adding\n> a utility to fill in SHA-1 compatibility things for SHA-256 repos, and\n> that will be written in shell. The performance benefit of C here is\n> going to be minimal, especially considering the fact that people will be\n> running it literally at most once per repo, so I don't see a reason to\n> spend a lot of time writing C code.\n\n\"This change\" as in the patch or my informal summary of what I think the\ncurrent status quo is?\n\nThe change being proposed here isn't to say that you can never write a\nnew thing in shell, but advice that actively encourages that for\nprototyping.\n\nIt was written at a time when the top-level glob of *.sh in the project was:\n\n\tgit-am.sh git-bisect.sh git-checkout.sh git-clean.sh\n\tgit-clone.sh git-commit.sh git-fetch.sh git-filter-branch.sh\n\tgit-instaweb.sh git-lost-found.sh git-ls-remote.sh\n\tgit-merge-octopus.sh git-merge-one-file.sh git-merge-ours.sh\n\tgit-merge-resolve.sh git-merge.sh git-merge-stupid.sh\n\tgit-mergetool.sh git-parse-remote.sh git-pull.sh\n\tgit-quiltimport.sh git-rebase--interactive.sh git-rebase.sh\n\tgit-repack.sh git-request-pull.sh git-reset.sh git-sh-setup.sh\n\tgit-stash.sh git-submodule.sh\n\nI.e. the structure of the codebase was entirely different than it is\ntoday, and it really made sense to encourage prototyping in *.sh first.\n\nI thought it would be uncontroversial to remove advice actively\nencouraging writing new built-ins in non-C first, which is not the same\nas adding a policy to say that such a thing should never be done.\n\nTo have a policy like that is pretty close to my personal opinion on the\nsubject, but that != the proposed patch.\n\n> I'm not of the opinion that we should never have shell or Perl code in\n> our project, nor does it intrinsically make sense to migrate everything\n> to C. Typically we've done that because it performs better, especially\n> on Windows, but there are many situations in which those are not major\n> considerations and shell or Perl can be a desirable approach.\n\n... but since we're sharing our own opinions :)\n\nAs someone with >100 commits in perl.git, I don't think I can be thought\nto be uncomfortable with the language.\n\nBut FWIW having patched git-send-email.perl the other day, I find myself\nwishing that it and similar code was in C.\n\nBecause it's got nothing to do with performance anymore, but that >=95%\nof our non-trivial code is in C, so doing almost anything in either *.sh\nor *.perl runs up against a wall of not having the same libraries you\ntake for granted, plumbing being subtly lacking or different than the\ncore C APIs etc.\n\nIn an ideal world we'd fix some of our plumbing, but in practice\ne.g. getting something as basic as git config values is some combination\nof incorrect and/or atrociously slow via Git.pm[1].\n\nAnd for shellscript in particular last I looked we were *this* close to\ne.g. being able to git rm sh-i18n--envsubst.c, git-sh-i18n.sh etc. I\nwrote that code, but I very much wish it were entirely gone. So as one\nexample I've put off some general improvement in our i18n infra\n(e.g. writing some minimal lib so we don't need to depend on libintl),\nbut have always given up on the point of dreading having to support such\na thing in shell and Perl too.\n\nSo it sucks for the individual author, but at this point the trade-off\nof whipping something new up in e.g. *.sh isn't just that the thing\ndoesn't need to be performant, but e.g. in the case of the gettext\nintegration means we'll be stuck with the fixed costs of extending\ncertain core APIs to shell-land forever, whereas currently it's looking\nlike we might be able to \"git rm\" much/most of that stuff sooner than\nlater.\n\n1. I eventually got past that and other things and have local patches to\n reduce the time to invoke git-send-email.perl's by 80%, which sped up\n t9001 from ~30s to ~10s, but it wasn't pretty..\n\n"},{"id":"422193","messageId":"xmqqmttwbmfm.fsf@gitster.g","threadId":"55509","inReplyTo":"patch-1.1-83266f30b67-20210417T084346Z-avarab@gmail.com","subject":"Re: [PATCH] CodingGuidelines: remove suggestion to write commands in Perl/SH","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-04-17T22:01:17Z","receivedAt":"2021-04-17T22:01:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Remove a suggestion to write new commands in Perl or Shell to\n> experiment. This advice was added in 6d0618a820a (Add\n> Documentation/CodingGuidelines, 2007-11-08).\n\nI think this needs to be rephrased to clarify that it is mainly to\nallow easier prototyping.\n\n> Since then the consensus changed to having no new such commands unless\n> necessary, and existing ones have been actively migrated to C.\n\nWell, \"unless necessary\" is quite an interesting word here. There\nare trade-offs, and some things are not worth spending cycles to\nwrite in C than leaving quick hackability for developers.\n\nAlso existing ones getting migrated to C does nothing to support the\nimplicit claim the above sentence makes, which is \"writing the first\nversion in C is recommended\".\n\n If you are planning a new command, it may be easier to prototype\n by implementing it as a script, until you and others on the list\n can figure out what the semantics and end-user experience should\n be, instead of starting the initial version in C code. Many Git\n commands started out as scripts and then later rewritten in C.\n\nMore importantly, we probably should stress that we've been very bad\nat maintaining plumbing to allow such quick experimentation. We\nshould encourage (or even require) developers to make the feature\nthey directly hacked into the Porcelain command available also to\nthe plumbing layer, so that those who script with plumbing commands\ncan also use it.\n\n> - - If you are planning a new command, consider writing it in shell\n> - or perl first, so that changes in semantics can be easily\n> - changed and discussed. Many Git commands started out like\n> - that, and a few are still scripts.\n> -\n> - Avoid introducing a new dependency into Git. This means you\n> usually should stay away from scripting languages not already\n> used in the Git core command set (unless your command is clearly\n"},{"id":"422194","messageId":"YHtc1LZCv0PWbFpD@camp.crustytoothpaste.net","threadId":"55509","inReplyTo":"87k0p01tje.fsf@evledraar.gmail.com","subject":"Re: [PATCH] CodingGuidelines: remove suggestion to write commands in Perl/SH","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2021-04-17T22:10:28Z","receivedAt":"2021-04-17T22:11:08Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2021-04-17 at 21:37:57, Ævar Arnfjörð Bjarmason wrote:\n> \n> On Sat, Apr 17 2021, brian m. carlson wrote:\n> > I'm also kind of opposed to this change. For example, I plan on adding\n> > a utility to fill in SHA-1 compatibility things for SHA-256 repos, and\n> > that will be written in shell. The performance benefit of C here is\n> > going to be minimal, especially considering the fact that people will be\n> > running it literally at most once per repo, so I don't see a reason to\n> > spend a lot of time writing C code.\n> \n> \"This change\" as in the patch or my informal summary of what I think the\n> current status quo is?\n\nThe patch.\n\n> The change being proposed here isn't to say that you can never write a\n> new thing in shell, but advice that actively encourages that for\n> prototyping.\n\nI think in many cases it is valuable to use it for prototyping still.\n\n> > I'm not of the opinion that we should never have shell or Perl code in\n> > our project, nor does it intrinsically make sense to migrate everything\n> > to C. Typically we've done that because it performs better, especially\n> > on Windows, but there are many situations in which those are not major\n> > considerations and shell or Perl can be a desirable approach.\n> \n> ... but since we're sharing our own opinions :)\n> \n> As someone with >100 commits in perl.git, I don't think I can be thought\n> to be uncomfortable with the language.\n\nI am duly aware of this fact, having worked on a mainly Perl codebase\nfull-time for 6 years.\n\n> So it sucks for the individual author, but at this point the trade-off\n> of whipping something new up in e.g. *.sh isn't just that the thing\n> doesn't need to be performant, but e.g. in the case of the gettext\n> integration means we'll be stuck with the fixed costs of extending\n> certain core APIs to shell-land forever, whereas currently it's looking\n> like we might be able to \"git rm\" much/most of that stuff sooner than\n> later.\n\nI think it's worth keeping the shell script stuff around because I think\nit adds a lot less friction to building new tooling. I'll be frank: I'm\nnot massively in love with C, despite having known it since I was 10,\nnor is our C particularly elegant (memory leaks, tons of global\nvariables, etc.). I think there can be an argument made that in a lot\nof cases, shell or Perl is the better option when performance isn't\nessential, and so I think those should continue to be first-class\nlanguages in our codebase, even if that means we need to put a little\nmore effort into maintaining them.\n\nMoreover, we still have a decent number of scripts using the shell\ntooling, so I'm not sure they're going away quite as quickly as you'd\nhoped. I agree they're not as numerous as they used to be, but they're\nstill not entirely gone, nor do I think they're going to all disappear\nanytime soon.\n-- \nbrian m. carlson (he/him or they/them)\nHouston, Texas, US\n"}]}