{"thread":{"id":"58322","subject":"Programmatic patches (transform commits)","startedAt":"2022-08-18T15:21:46Z","lastAt":"2022-08-19T10:15:02Z","messageCount":3,"participants":["Mark Fulton","Ævar Arnfjörð Bjarmason","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"461432","messageId":"CAED_RJPh28o8bFsBEEzAYvp1DXhff2JWiTk3zHh3VbaHZephgg@mail.gmail.com","threadId":"58322","inReplyTo":null,"subject":"Programmatic patches (transform commits)","fromName":"Mark Fulton","fromEmail":"mfulton26@gmail.com","sentAt":"2022-08-18T15:21:23Z","receivedAt":"2022-08-18T15:21:46Z","isPatch":false,"sender":{"key":"mfulton26@gmail.com","avatar":null},"body":"Is there a way to commit a transform script that programmatically\napplies file changes rather than committing the file changes directly?\n\ne.g. Imagine in a large repository that a contributor wants to replace\ncertain instances of \"abc\" with \"xyz\". A transform script might be\nlike the following:\n\n```sh\n#!/bin/sh\n\nsed -i 's/abc/xyz/g' $(find .)\n```\n\nApplying such a \"programmatic patch\" will potentially edit many files.\nDoing a code review on such a change is error prone due to authors\nresolving merge conflicts manually, etc. while reviewing the patch in\nsome circumstances is much easier (especially tools for specifically\nthis type of file transformations are used to make it easy to parse\ncode, traverse abstract syntax trees, make edits, etc.).\n\nDoes anything like this exist today? Depending on the implementation I\ncould see there being cross-platform support challenges but maybe\nthere is something that already exists to assist with this which I can\nlearn about.\n\nAs an alternative to making this part of Git I can see tools like\nGitHub Actions being used to look for commits of \"programmatic patch\"\nfiles, pick those up, run them, and commit and push the change but\nhaving a solution for this as part of Git itself would make it\nindependent of GitHub and more reusable, etc.\n"},{"id":"461435","messageId":"220818.86mtc18sb5.gmgdl@evledraar.gmail.com","threadId":"58322","inReplyTo":"CAED_RJPh28o8bFsBEEzAYvp1DXhff2JWiTk3zHh3VbaHZephgg@mail.gmail.com","subject":"Re: Programmatic patches (transform commits)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-08-18T15:48:19Z","receivedAt":"2022-08-18T15:50:51Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Aug 18 2022, Mark Fulton wrote:\n\n> Is there a way to commit a transform script that programmatically\n> applies file changes rather than committing the file changes directly?\n>\n> e.g. Imagine in a large repository that a contributor wants to replace\n> certain instances of \"abc\" with \"xyz\". A transform script might be\n> like the following:\n>\n> ```sh\n> #!/bin/sh\n>\n> sed -i 's/abc/xyz/g' $(find .)\n> ```\n>\n> Applying such a \"programmatic patch\" will potentially edit many files.\n> Doing a code review on such a change is error prone due to authors\n> resolving merge conflicts manually, etc. while reviewing the patch in\n> some circumstances is much easier (especially tools for specifically\n> this type of file transformations are used to make it easy to parse\n> code, traverse abstract syntax trees, make edits, etc.).\n>\n> Does anything like this exist today? Depending on the implementation I\n> could see there being cross-platform support challenges but maybe\n> there is something that already exists to assist with this which I can\n> learn about.\n>\n> As an alternative to making this part of Git I can see tools like\n> GitHub Actions being used to look for commits of \"programmatic patch\"\n> files, pick those up, run them, and commit and push the change but\n> having a solution for this as part of Git itself would make it\n> independent of GitHub and more reusable, etc.\n\nYes, e.g. for C (and used by linux.git and this project):\nhttps://www.kernel.org/doc/html/v4.14/dev-tools/coccinelle.html\n\nGenerally speaking I've made such ad-hoc commits in the past, where I've\ne.g. included the needed script (or one-liner) in the commit message,\nalong with manually adding the diff to the commit message that *isn't*\nthe part changed by the script or one-liner.\n\nBut I don't think there's any sort of widespread standard for this sort\nof thing, other than doing that, and the same (sometimes leading to\nnon-atomic commits) of having the manual part of the change applied\nbefore or after in a separate commit.\n"},{"id":"461538","messageId":"n7prrp62-41o1-r280-qo55-9q7rnqs723p1@tzk.qr","threadId":"58322","inReplyTo":"CAED_RJPh28o8bFsBEEzAYvp1DXhff2JWiTk3zHh3VbaHZephgg@mail.gmail.com","subject":"Re: Programmatic patches (transform commits)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2022-08-19T10:15:02Z","receivedAt":"2022-08-19T10:15:02Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Mark,\n\nOn Thu, 18 Aug 2022, Mark Fulton wrote:\n\n> Is there a way to commit a transform script that programmatically\n> applies file changes rather than committing the file changes directly?\n>\n> e.g. Imagine in a large repository that a contributor wants to replace\n> certain instances of \"abc\" with \"xyz\". A transform script might be\n> like the following:\n>\n> ```sh\n> #!/bin/sh\n>\n> sed -i 's/abc/xyz/g' $(find .)\n> ```\n\nI do have a script to do such a thing, and I am sure other users do, too.\nIt would be a simpler script if it did not have to accommodate for `sed`'s\n`-i` option behaving differently between GNU and BSD variants. For the GNU\nvariant, I use\n\n\tgit grep -zl \"$regex\" | xargs -0r sed -i \"s/$regex/$replacement/g\"\n\nand for the BSD variant I use `sed -i ''` (i.e. an extra, empty argument)\nand skip the `-r` option because BSD `xargs` does not understand it.\n\n> Applying such a \"programmatic patch\" will potentially edit many files.\n> Doing a code review on such a change is error prone due to authors\n> resolving merge conflicts manually, etc. while reviewing the patch in\n> some circumstances is much easier (especially tools for specifically\n> this type of file transformations are used to make it easy to parse\n> code, traverse abstract syntax trees, make edits, etc.).\n\nIn the Git project, we do have a track record of mentioning the exact\ncommands for automated transformation in the commit message, see e.g.\nhttps://github.com/git/git/commit/8dcf73c5c940.\n\nThe idea is that any reviewer could run the same command, verify that the\noutcome is the same, and give their ACK. Or, in case of merge conflicts,\nthey would re-run the command and commit the result using the original\ncommit message.\n\nCiao,\nDscho\n"}]}