{"thread":{"id":"39646","subject":"Using clean/smudge filters with difftool","startedAt":"2015-06-16T14:11:03Z","lastAt":"2015-06-21T19:29:51Z","messageCount":18,"participants":["Florian Aspart","Michael J Gruber","John Keeping","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"263929","messageId":"CAGA3++LiX9NyK94RPiuiG83X8FSRN4EkaxMchir51hGSQY90Tw@mail.gmail.com","threadId":"39646","inReplyTo":null,"subject":"Using clean/smudge filters with difftool","fromName":"Florian Aspart","fromEmail":"florian.aspart@gmail.com","sentAt":"2015-06-16T14:11:03Z","receivedAt":"2015-06-16T14:11:03Z","isPatch":false,"sender":{"key":"florian.aspart@gmail.com","avatar":null},"body":"Hi everyone,\n\nI created a clean filter to apply on some files before commiting them.\nThe filter works correctly when I commit the file and is also applied\nwhen I usethe iff command line tool.\nHowever, when using difftool with meld, the filter is not applied and\nthe different versions of the files are compared without any\nfiltering.\n\nIs there a way to apply the clean/smudge filters when comparing the\nworking copy of a file to the HEAD version in a gui diff tool?\n\nI'm using git version 2.4.3 under Ubuntu.\n\nBest,\nFlorian\n"},{"id":"264177","messageId":"5582BA1F.1030409@drmicha.warpmail.net","threadId":"39646","inReplyTo":"CAGA3++LiX9NyK94RPiuiG83X8FSRN4EkaxMchir51hGSQY90Tw@mail.gmail.com","subject":"Re: Using clean/smudge filters with difftool","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2015-06-18T12:31:27Z","receivedAt":"2015-06-18T12:31:27Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Florian Aspart venit, vidit, dixit 16.06.2015 16:11:\n> Hi everyone,\n> \n> I created a clean filter to apply on some files before commiting them.\n> The filter works correctly when I commit the file and is also applied\n> when I usethe iff command line tool.\n> However, when using difftool with meld, the filter is not applied and\n> the different versions of the files are compared without any\n> filtering.\n> \n> Is there a way to apply the clean/smudge filters when comparing the\n> working copy of a file to the HEAD version in a gui diff tool?\n> \n> I'm using git version 2.4.3 under Ubuntu.\n> \n> Best,\n> Florian\n\nAre you saying that \"difftool\" compares an uncleaned working tree file\nwith a cleaned blob? That would be a bug in either difftool or the way\nwe feed difftool.\n\nMichael\n"},{"id":"264179","messageId":"CAGA3++LrVSs3rMkg=S2Og48pz1yEBxwpcRsPt7sNLENRh1ooAg@mail.gmail.com","threadId":"39646","inReplyTo":"5582BA1F.1030409@drmicha.warpmail.net","subject":"Re: Using clean/smudge filters with difftool","fromName":"Florian Aspart","fromEmail":"florian.aspart@gmail.com","sentAt":"2015-06-18T13:15:38Z","receivedAt":"2015-06-18T13:15:38Z","isPatch":false,"sender":{"key":"florian.aspart@gmail.com","avatar":null},"body":"Hi Michael,\n\nyes in this case \"difftool\" compares an uncleaned working tree file\nwith a cleaned blob. I did not try the smudge filter to see if it\napplied in difftool.\n\nI think the problem comes from the way difftool is feeded, since I\nalso had this problem when setting an external tool for the diff in\nthe gitconfig file.\n\nHowever, I'm not sure if this is a bug or it is designed to be so.\nIf the external tool changes a cleaned working tree file during the\ndiff, then by saving this file the result of the cleaning filter would\nalso be saved in the working tree.\n\n2015-06-18 14:31 GMT+02:00 Michael J Gruber :\n> Florian Aspart venit, vidit, dixit 16.06.2015 16:11:\n>> Hi everyone,\n>>\n>> I created a clean filter to apply on some files before commiting them.\n>> The filter works correctly when I commit the file and is also applied\n>> when I usethe iff command line tool.\n>> However, when using difftool with meld, the filter is not applied and\n>> the different versions of the files are compared without any\n>> filtering.\n>>\n>> Is there a way to apply the clean/smudge filters when comparing the\n>> working copy of a file to the HEAD version in a gui diff tool?\n>>\n>> I'm using git version 2.4.3 under Ubuntu.\n>>\n>> Best,\n>> Florian\n>\n> Are you saying that \"difftool\" compares an uncleaned working tree file\n> with a cleaned blob? That would be a bug in either difftool or the way\n> we feed difftool.\n>\n> Michael\n>\n"},{"id":"264180","messageId":"20150618132622.GJ18226@serenity.lan","threadId":"39646","inReplyTo":"CAGA3++LrVSs3rMkg=S2Og48pz1yEBxwpcRsPt7sNLENRh1ooAg@mail.gmail.com","subject":"Re: Using clean/smudge filters with difftool","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2015-06-18T13:26:22Z","receivedAt":"2015-06-18T13:26:22Z","isPatch":false,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"[Please don't top-post on this list.]\n\nOn Thu, Jun 18, 2015 at 03:15:38PM +0200, Florian Aspart wrote:\n> 2015-06-18 14:31 GMT+02:00 Michael J Gruber :\n> > Florian Aspart venit, vidit, dixit 16.06.2015 16:11:\n> >> Hi everyone,\n> >>\n> >> I created a clean filter to apply on some files before commiting them.\n> >> The filter works correctly when I commit the file and is also applied\n> >> when I usethe iff command line tool.\n> >> However, when using difftool with meld, the filter is not applied and\n> >> the different versions of the files are compared without any\n> >> filtering.\n> >>\n> >> Is there a way to apply the clean/smudge filters when comparing the\n> >> working copy of a file to the HEAD version in a gui diff tool?\n> >>\n> >> I'm using git version 2.4.3 under Ubuntu.\n> >>\n> >> Best,\n> >> Florian\n> >\n> > Are you saying that \"difftool\" compares an uncleaned working tree file\n> > with a cleaned blob? That would be a bug in either difftool or the way\n> > we feed difftool.\n> >\n> yes in this case \"difftool\" compares an uncleaned working tree file\n> with a cleaned blob. I did not try the smudge filter to see if it\n> applied in difftool.\n> \n> I think the problem comes from the way difftool is feeded, since I\n> also had this problem when setting an external tool for the diff in\n> the gitconfig file.\n> \n> However, I'm not sure if this is a bug or it is designed to be so.\n> If the external tool changes a cleaned working tree file during the\n> diff, then by saving this file the result of the cleaning filter would\n> also be saved in the working tree.\n\nHow is your filter configured? Is it using a simple pattern (e.g.\n\"*.c\") or is it using a file path?\n\ngit-difftool uses `git checkout-index --all --prefix=$dir/` and I wonder\nif the prefix means that the attribute specification does not match the\ntemporary file that difftool produces, so no filter is applied.\n"},{"id":"264181","messageId":"CAGA3+++_mx=O=Un0pip8Q41X5PZBLmES=Hd=U=aSowryx5r=8w@mail.gmail.com","threadId":"39646","inReplyTo":"20150618132622.GJ18226@serenity.lan","subject":"Re: Using clean/smudge filters with difftool","fromName":"Florian Aspart","fromEmail":"florian.aspart@gmail.com","sentAt":"2015-06-18T13:51:25Z","receivedAt":"2015-06-18T13:51:25Z","isPatch":false,"sender":{"key":"florian.aspart@gmail.com","avatar":null},"body":"2015-06-18 15:26 GMT+02:00 John Keeping :\n> [Please don't top-post on this list.]\n>\n> On Thu, Jun 18, 2015 at 03:15:38PM +0200, Florian Aspart wrote:\n>> 2015-06-18 14:31 GMT+02:00 Michael J Gruber :\n>> > Florian Aspart venit, vidit, dixit 16.06.2015 16:11:\n>> >> Hi everyone,\n>> >>\n>> >> I created a clean filter to apply on some files before commiting them.\n>> >> The filter works correctly when I commit the file and is also applied\n>> >> when I usethe iff command line tool.\n>> >> However, when using difftool with meld, the filter is not applied and\n>> >> the different versions of the files are compared without any\n>> >> filtering.\n>> >>\n>> >> Is there a way to apply the clean/smudge filters when comparing the\n>> >> working copy of a file to the HEAD version in a gui diff tool?\n>> >>\n>> >> I'm using git version 2.4.3 under Ubuntu.\n>> >>\n>> >> Best,\n>> >> Florian\n>> >\n>> > Are you saying that \"difftool\" compares an uncleaned working tree file\n>> > with a cleaned blob? That would be a bug in either difftool or the way\n>> > we feed difftool.\n>> >\n>> yes in this case \"difftool\" compares an uncleaned working tree file\n>> with a cleaned blob. I did not try the smudge filter to see if it\n>> applied in difftool.\n>>\n>> I think the problem comes from the way difftool is feeded, since I\n>> also had this problem when setting an external tool for the diff in\n>> the gitconfig file.\n>>\n>> However, I'm not sure if this is a bug or it is designed to be so.\n>> If the external tool changes a cleaned working tree file during the\n>> diff, then by saving this file the result of the cleaning filter would\n>> also be saved in the working tree.\n>\n> How is your filter configured? Is it using a simple pattern (e.g.\n> \"*.c\") or is it using a file path?\n>\n> git-difftool uses `git checkout-index --all --prefix=$dir/` and I wonder\n> if the prefix means that the attribute specification does not match the\n> temporary file that difftool produces, so no filter is applied.\n\n\nIt is using a simple pattern:\n*.ipynb filter=clean_ipynb\n"},{"id":"264182","messageId":"20150618141116.GK18226@serenity.lan","threadId":"39646","inReplyTo":"CAGA3+++_mx=O=Un0pip8Q41X5PZBLmES=Hd=U=aSowryx5r=8w@mail.gmail.com","subject":"Re: Using clean/smudge filters with difftool","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2015-06-18T14:11:16Z","receivedAt":"2015-06-18T14:11:16Z","isPatch":false,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Thu, Jun 18, 2015 at 03:51:25PM +0200, Florian Aspart wrote:\n> 2015-06-18 15:26 GMT+02:00 John Keeping :\n> > [Please don't top-post on this list.]\n> >\n> > On Thu, Jun 18, 2015 at 03:15:38PM +0200, Florian Aspart wrote:\n> >> 2015-06-18 14:31 GMT+02:00 Michael J Gruber :\n> >> > Florian Aspart venit, vidit, dixit 16.06.2015 16:11:\n> >> >> Hi everyone,\n> >> >>\n> >> >> I created a clean filter to apply on some files before commiting them.\n> >> >> The filter works correctly when I commit the file and is also applied\n> >> >> when I usethe iff command line tool.\n> >> >> However, when using difftool with meld, the filter is not applied and\n> >> >> the different versions of the files are compared without any\n> >> >> filtering.\n> >> >>\n> >> >> Is there a way to apply the clean/smudge filters when comparing the\n> >> >> working copy of a file to the HEAD version in a gui diff tool?\n> >> >>\n> >> >> I'm using git version 2.4.3 under Ubuntu.\n> >> >>\n> >> >> Best,\n> >> >> Florian\n> >> >\n> >> > Are you saying that \"difftool\" compares an uncleaned working tree file\n> >> > with a cleaned blob? That would be a bug in either difftool or the way\n> >> > we feed difftool.\n> >> >\n> >> yes in this case \"difftool\" compares an uncleaned working tree file\n> >> with a cleaned blob. I did not try the smudge filter to see if it\n> >> applied in difftool.\n> >>\n> >> I think the problem comes from the way difftool is feeded, since I\n> >> also had this problem when setting an external tool for the diff in\n> >> the gitconfig file.\n> >>\n> >> However, I'm not sure if this is a bug or it is designed to be so.\n> >> If the external tool changes a cleaned working tree file during the\n> >> diff, then by saving this file the result of the cleaning filter would\n> >> also be saved in the working tree.\n> >\n> > How is your filter configured? Is it using a simple pattern (e.g.\n> > \"*.c\") or is it using a file path?\n> >\n> > git-difftool uses `git checkout-index --all --prefix=$dir/` and I wonder\n> > if the prefix means that the attribute specification does not match the\n> > temporary file that difftool produces, so no filter is applied.\n> \n> It is using a simple pattern:\n> *.ipynb filter=clean_ipynb\n\nI also realised that the code for file diff is very different from\ndirectory diff do you see any difference between git-difftool acting on\nfiles and with the `--dir-diff` option?\n"},{"id":"264183","messageId":"CAGA3+++ibw=8Q1LtM6yJrZ7Q4eVs_MEHmPAzctSVSREXMmBiMQ@mail.gmail.com","threadId":"39646","inReplyTo":"20150618141116.GK18226@serenity.lan","subject":"Re: Using clean/smudge filters with difftool","fromName":"Florian Aspart","fromEmail":"florian.aspart@gmail.com","sentAt":"2015-06-18T14:17:52Z","receivedAt":"2015-06-18T14:17:52Z","isPatch":false,"sender":{"key":"florian.aspart@gmail.com","avatar":null},"body":"2015-06-18 16:11 GMT+02:00 John Keeping :\n> On Thu, Jun 18, 2015 at 03:51:25PM +0200, Florian Aspart wrote:\n>> 2015-06-18 15:26 GMT+02:00 John Keeping :\n>> > [Please don't top-post on this list.]\n>> >\n>> > On Thu, Jun 18, 2015 at 03:15:38PM +0200, Florian Aspart wrote:\n>> >> 2015-06-18 14:31 GMT+02:00 Michael J Gruber :\n>> >> > Florian Aspart venit, vidit, dixit 16.06.2015 16:11:\n>> >> >> Hi everyone,\n>> >> >>\n>> >> >> I created a clean filter to apply on some files before commiting them.\n>> >> >> The filter works correctly when I commit the file and is also applied\n>> >> >> when I usethe iff command line tool.\n>> >> >> However, when using difftool with meld, the filter is not applied and\n>> >> >> the different versions of the files are compared without any\n>> >> >> filtering.\n>> >> >>\n>> >> >> Is there a way to apply the clean/smudge filters when comparing the\n>> >> >> working copy of a file to the HEAD version in a gui diff tool?\n>> >> >>\n>> >> >> I'm using git version 2.4.3 under Ubuntu.\n>> >> >>\n>> >> >> Best,\n>> >> >> Florian\n>> >> >\n>> >> > Are you saying that \"difftool\" compares an uncleaned working tree file\n>> >> > with a cleaned blob? That would be a bug in either difftool or the way\n>> >> > we feed difftool.\n>> >> >\n>> >> yes in this case \"difftool\" compares an uncleaned working tree file\n>> >> with a cleaned blob. I did not try the smudge filter to see if it\n>> >> applied in difftool.\n>> >>\n>> >> I think the problem comes from the way difftool is feeded, since I\n>> >> also had this problem when setting an external tool for the diff in\n>> >> the gitconfig file.\n>> >>\n>> >> However, I'm not sure if this is a bug or it is designed to be so.\n>> >> If the external tool changes a cleaned working tree file during the\n>> >> diff, then by saving this file the result of the cleaning filter would\n>> >> also be saved in the working tree.\n>> >\n>> > How is your filter configured? Is it using a simple pattern (e.g.\n>> > \"*.c\") or is it using a file path?\n>> >\n>> > git-difftool uses `git checkout-index --all --prefix=$dir/` and I wonder\n>> > if the prefix means that the attribute specification does not match the\n>> > temporary file that difftool produces, so no filter is applied.\n>>\n>> It is using a simple pattern:\n>> *.ipynb filter=clean_ipynb\n>\n> I also realised that the code for file diff is very different from\n> directory diff do you see any difference between git-difftool acting on\n> files and with the `--dir-diff` option?\n\nNo, even with the --dir-diff option, the filter is still not applied.\n"},{"id":"264184","messageId":"20150618142852.GL18226@serenity.lan","threadId":"39646","inReplyTo":"CAGA3+++ibw=8Q1LtM6yJrZ7Q4eVs_MEHmPAzctSVSREXMmBiMQ@mail.gmail.com","subject":"Re: Using clean/smudge filters with difftool","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2015-06-18T14:28:52Z","receivedAt":"2015-06-18T14:28:52Z","isPatch":false,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Thu, Jun 18, 2015 at 04:17:52PM +0200, Florian Aspart wrote:\n> 2015-06-18 16:11 GMT+02:00 John Keeping :\n> > On Thu, Jun 18, 2015 at 03:51:25PM +0200, Florian Aspart wrote:\n> >> 2015-06-18 15:26 GMT+02:00 John Keeping :\n> >> > [Please don't top-post on this list.]\n> >> >\n> >> > On Thu, Jun 18, 2015 at 03:15:38PM +0200, Florian Aspart wrote:\n> >> >> 2015-06-18 14:31 GMT+02:00 Michael J Gruber :\n> >> >> > Florian Aspart venit, vidit, dixit 16.06.2015 16:11:\n> >> >> >> I created a clean filter to apply on some files before commiting them.\n> >> >> >> The filter works correctly when I commit the file and is also applied\n> >> >> >> when I usethe iff command line tool.\n> >> >> >> However, when using difftool with meld, the filter is not applied and\n> >> >> >> the different versions of the files are compared without any\n> >> >> >> filtering.\n> >> >> >>\n> >> >> >> Is there a way to apply the clean/smudge filters when comparing the\n> >> >> >> working copy of a file to the HEAD version in a gui diff tool?\n> >> >> >>\n> >> >> >> I'm using git version 2.4.3 under Ubuntu.\n> > \n> > I also realised that the code for file diff is very different from\n> > directory diff do you see any difference between git-difftool acting on\n> > files and with the `--dir-diff` option?\n> \n> No, even with the --dir-diff option, the filter is still not applied.\n\nI have tried to reproduce this and it works as expected for me (i.e. the\nfilter is applied) both for file diff and directory diff mode:\n\n$ git config filter.quote.clean \"sed -e 's/^> //'\"\n$ git config filter.quote.smudge \"sed -e '/^> /n; s/^/> /'\"\n$ git config filter.quote.required true\n\n$ echo '*.quote filter=quote' >>.gitattributes\n$ cat >1.quote <>1.quote\n\nNow `git-difftool` shows the differences with the filter applied. This can be\nseen running with GIT_TRACE:\n\n$ GIT_TRACE=2 git difftool\n15:26:59.211541 git.c:557 trace: exec: 'git-difftool'\n15:26:59.211674 run-command.c:347 trace: run_command: 'git-difftool'\n15:26:59.338617 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'difftool.trustExitCode'\n15:26:59.342664 git.c:348 trace: built-in: git 'diff'\n15:26:59.344857 run-command.c:347 trace: run_command: 'sed -e '\\''s/^> //'\\'''\n15:26:59.345383 run-command.c:195 trace: exec: '/bin/sh' '-c' 'sed -e '\\''s/^> //'\\''' 'sed -e '\\''s/^> //'\\'''\n15:26:59.351077 run-command.c:347 trace: run_command: 'sed -e '\\''/^> /n; s/^/> /'\\'''\n15:26:59.351605 run-command.c:195 trace: exec: '/bin/sh' '-c' 'sed -e '\\''/^> /n; s/^/> /'\\''' 'sed -e '\\''/^> /n; s/^/> /'\\'''\n15:26:59.355716 run-command.c:347 trace: run_command: 'git-difftool--helper' '1.quote' '/tmp/SUEySx_1.quote' '4cb29ea38f70d7c61b2a3a25b02e3bdf44905402' '100644' '1.quote' '0000000000000000000000000000000000000000' '100644'\n15:26:59.356191 run-command.c:195 trace: exec: 'git-difftool--helper' '1.quote' '/tmp/SUEySx_1.quote' '4cb29ea38f70d7c61b2a3a25b02e3bdf44905402' '100644' '1.quote' '0000000000000000000000000000000000000000' '100644'\n15:26:59.370468 git.c:348 trace: built-in: git 'config' 'diff.tool'\n15:26:59.373485 git.c:348 trace: built-in: git 'config' 'merge.tool'\n15:26:59.378402 git.c:348 trace: built-in: git 'config' 'difftool.vimdiff.cmd'\n15:26:59.381424 git.c:348 trace: built-in: git 'config' 'mergetool.vimdiff.cmd'\n15:26:59.386623 git.c:348 trace: built-in: git 'config' '--bool' 'mergetool.prompt'\n15:26:59.390198 git.c:348 trace: built-in: git 'config' '--bool' 'difftool.prompt'\n\nI think the first run_command of `sed` is cleaning the working tree file\nto figure out *if* it differs, then the second `sed` is smudging the\nversion in the index so that difftool can use it.\n"},{"id":"264189","messageId":"CAGA3+++LqZ8Qv6tpuoqQwi37kO5LLODwcbFQtvneorjiV4KARw@mail.gmail.com","threadId":"39646","inReplyTo":"20150618142852.GL18226@serenity.lan","subject":"Re: Using clean/smudge filters with difftool","fromName":"Florian Aspart","fromEmail":"florian.aspart@gmail.com","sentAt":"2015-06-18T15:39:18Z","receivedAt":"2015-06-18T15:39:18Z","isPatch":false,"sender":{"key":"florian.aspart@gmail.com","avatar":null},"body":"2015-06-18 16:28 GMT+02:00 John Keeping :\n> On Thu, Jun 18, 2015 at 04:17:52PM +0200, Florian Aspart wrote:\n>> 2015-06-18 16:11 GMT+02:00 John Keeping :\n>> > On Thu, Jun 18, 2015 at 03:51:25PM +0200, Florian Aspart wrote:\n>> >> 2015-06-18 15:26 GMT+02:00 John Keeping :\n>> >> > [Please don't top-post on this list.]\n>> >> >\n>> >> > On Thu, Jun 18, 2015 at 03:15:38PM +0200, Florian Aspart wrote:\n>> >> >> 2015-06-18 14:31 GMT+02:00 Michael J Gruber :\n>> >> >> > Florian Aspart venit, vidit, dixit 16.06.2015 16:11:\n>> >> >> >> I created a clean filter to apply on some files before commiting them.\n>> >> >> >> The filter works correctly when I commit the file and is also applied\n>> >> >> >> when I usethe iff command line tool.\n>> >> >> >> However, when using difftool with meld, the filter is not applied and\n>> >> >> >> the different versions of the files are compared without any\n>> >> >> >> filtering.\n>> >> >> >>\n>> >> >> >> Is there a way to apply the clean/smudge filters when comparing the\n>> >> >> >> working copy of a file to the HEAD version in a gui diff tool?\n>> >> >> >>\n>> >> >> >> I'm using git version 2.4.3 under Ubuntu.\n>> >\n>> > I also realised that the code for file diff is very different from\n>> > directory diff do you see any difference between git-difftool acting on\n>> > files and with the `--dir-diff` option?\n>>\n>> No, even with the --dir-diff option, the filter is still not applied.\n>\n> I have tried to reproduce this and it works as expected for me (i.e. the\n> filter is applied) both for file diff and directory diff mode:\n>\n> $ git config filter.quote.clean \"sed -e 's/^> //'\"\n> $ git config filter.quote.smudge \"sed -e '/^> /n; s/^/> /'\"\n> $ git config filter.quote.required true\n>\n> $ echo '*.quote filter=quote' >>.gitattributes\n> $ cat >1.quote < one\n> two\n> three\n> EOF\n> $ git add .gitattributes 1.quote\n> $ git commit -m 'Initial commit'\n> $ echo four >>1.quote\n>\n> Now `git-difftool` shows the differences with the filter applied. This can be\n> seen running with GIT_TRACE:\n>\n> $ GIT_TRACE=2 git difftool\n> 15:26:59.211541 git.c:557 trace: exec: 'git-difftool'\n> 15:26:59.211674 run-command.c:347 trace: run_command: 'git-difftool'\n> 15:26:59.338617 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'difftool.trustExitCode'\n> 15:26:59.342664 git.c:348 trace: built-in: git 'diff'\n> 15:26:59.344857 run-command.c:347 trace: run_command: 'sed -e '\\''s/^> //'\\'''\n> 15:26:59.345383 run-command.c:195 trace: exec: '/bin/sh' '-c' 'sed -e '\\''s/^> //'\\''' 'sed -e '\\''s/^> //'\\'''\n> 15:26:59.351077 run-command.c:347 trace: run_command: 'sed -e '\\''/^> /n; s/^/> /'\\'''\n> 15:26:59.351605 run-command.c:195 trace: exec: '/bin/sh' '-c' 'sed -e '\\''/^> /n; s/^/> /'\\''' 'sed -e '\\''/^> /n; s/^/> /'\\'''\n> 15:26:59.355716 run-command.c:347 trace: run_command: 'git-difftool--helper' '1.quote' '/tmp/SUEySx_1.quote' '4cb29ea38f70d7c61b2a3a25b02e3bdf44905402' '100644' '1.quote' '0000000000000000000000000000000000000000' '100644'\n> 15:26:59.356191 run-command.c:195 trace: exec: 'git-difftool--helper' '1.quote' '/tmp/SUEySx_1.quote' '4cb29ea38f70d7c61b2a3a25b02e3bdf44905402' '100644' '1.quote' '0000000000000000000000000000000000000000' '100644'\n> 15:26:59.370468 git.c:348 trace: built-in: git 'config' 'diff.tool'\n> 15:26:59.373485 git.c:348 trace: built-in: git 'config' 'merge.tool'\n> 15:26:59.378402 git.c:348 trace: built-in: git 'config' 'difftool.vimdiff.cmd'\n> 15:26:59.381424 git.c:348 trace: built-in: git 'config' 'mergetool.vimdiff.cmd'\n> 15:26:59.386623 git.c:348 trace: built-in: git 'config' '--bool' 'mergetool.prompt'\n> 15:26:59.390198 git.c:348 trace: built-in: git 'config' '--bool' 'difftool.prompt'\n>\n> I think the first run_command of `sed` is cleaning the working tree file\n> to figure out *if* it differs, then the second `sed` is smudging the\n> version in the index so that difftool can use it.\n\nI'm not really understanding what your filter is doing, but I tried\nyour code on my machine and I get a different result when using diff\nand difftool on my machine.\n\nThe diff results give me:\n\ndiff --git a/1.quote b/1.quote\nindex 4cb29ea..f384549 100644\n--- a/1.quote\n+++ b/1.quote\n@@ -1,3 +1,4 @@\n one\n two\n three\n+four\n\nWhile the diff tool tells me that the repository file is:\n> one\n> two\n> three\n\nand my working copy:\none\ntwo\nthree\nfour\n\nIn both case the the filters are called twice (cf GIT_TRACE) as in the\nexample your wrote.\n"},{"id":"264194","messageId":"20150618160133.GO18226@serenity.lan","threadId":"39646","inReplyTo":"CAGA3+++LqZ8Qv6tpuoqQwi37kO5LLODwcbFQtvneorjiV4KARw@mail.gmail.com","subject":"Re: Using clean/smudge filters with difftool","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2015-06-18T16:01:33Z","receivedAt":"2015-06-18T16:01:33Z","isPatch":false,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Thu, Jun 18, 2015 at 05:39:18PM +0200, Florian Aspart wrote:\n> 2015-06-18 16:28 GMT+02:00 John Keeping :\n> > On Thu, Jun 18, 2015 at 04:17:52PM +0200, Florian Aspart wrote:\n> >> 2015-06-18 16:11 GMT+02:00 John Keeping :\n> >> > On Thu, Jun 18, 2015 at 03:51:25PM +0200, Florian Aspart wrote:\n> >> >> 2015-06-18 15:26 GMT+02:00 John Keeping :\n> >> >> > [Please don't top-post on this list.]\n> >> >> >\n> >> >> > On Thu, Jun 18, 2015 at 03:15:38PM +0200, Florian Aspart wrote:\n> >> >> >> 2015-06-18 14:31 GMT+02:00 Michael J Gruber :\n> >> >> >> > Florian Aspart venit, vidit, dixit 16.06.2015 16:11:\n> >> >> >> >> I created a clean filter to apply on some files before commiting them.\n> >> >> >> >> The filter works correctly when I commit the file and is also applied\n> >> >> >> >> when I usethe iff command line tool.\n> >> >> >> >> However, when using difftool with meld, the filter is not applied and\n> >> >> >> >> the different versions of the files are compared without any\n> >> >> >> >> filtering.\n> >> >> >> >>\n> >> >> >> >> Is there a way to apply the clean/smudge filters when comparing the\n> >> >> >> >> working copy of a file to the HEAD version in a gui diff tool?\n> >> >> >> >>\n> >> >> >> >> I'm using git version 2.4.3 under Ubuntu.\n> >> >\n> >> > I also realised that the code for file diff is very different from\n> >> > directory diff do you see any difference between git-difftool acting on\n> >> > files and with the `--dir-diff` option?\n> >>\n> >> No, even with the --dir-diff option, the filter is still not applied.\n> >\n> > I have tried to reproduce this and it works as expected for me (i.e. the\n> > filter is applied) both for file diff and directory diff mode:\n> >\n> > $ git config filter.quote.clean \"sed -e 's/^> //'\"\n> > $ git config filter.quote.smudge \"sed -e '/^> /n; s/^/> /'\"\n> > $ git config filter.quote.required true\n> >\n> > $ echo '*.quote filter=quote' >>.gitattributes\n> > $ cat >1.quote < > one\n> > two\n> > three\n> > EOF\n> > $ git add .gitattributes 1.quote\n> > $ git commit -m 'Initial commit'\n> > $ echo four >>1.quote\n> >\n> > Now `git-difftool` shows the differences with the filter applied. This can be\n> > seen running with GIT_TRACE:\n> >\n> > $ GIT_TRACE=2 git difftool\n> > 15:26:59.211541 git.c:557 trace: exec: 'git-difftool'\n> > 15:26:59.211674 run-command.c:347 trace: run_command: 'git-difftool'\n> > 15:26:59.338617 git.c:348 trace: built-in: git 'config' '--bool' '--get' 'difftool.trustExitCode'\n> > 15:26:59.342664 git.c:348 trace: built-in: git 'diff'\n> > 15:26:59.344857 run-command.c:347 trace: run_command: 'sed -e '\\''s/^> //'\\'''\n> > 15:26:59.345383 run-command.c:195 trace: exec: '/bin/sh' '-c' 'sed -e '\\''s/^> //'\\''' 'sed -e '\\''s/^> //'\\'''\n> > 15:26:59.351077 run-command.c:347 trace: run_command: 'sed -e '\\''/^> /n; s/^/> /'\\'''\n> > 15:26:59.351605 run-command.c:195 trace: exec: '/bin/sh' '-c' 'sed -e '\\''/^> /n; s/^/> /'\\''' 'sed -e '\\''/^> /n; s/^/> /'\\'''\n> > 15:26:59.355716 run-command.c:347 trace: run_command: 'git-difftool--helper' '1.quote' '/tmp/SUEySx_1.quote' '4cb29ea38f70d7c61b2a3a25b02e3bdf44905402' '100644' '1.quote' '0000000000000000000000000000000000000000' '100644'\n> > 15:26:59.356191 run-command.c:195 trace: exec: 'git-difftool--helper' '1.quote' '/tmp/SUEySx_1.quote' '4cb29ea38f70d7c61b2a3a25b02e3bdf44905402' '100644' '1.quote' '0000000000000000000000000000000000000000' '100644'\n> > 15:26:59.370468 git.c:348 trace: built-in: git 'config' 'diff.tool'\n> > 15:26:59.373485 git.c:348 trace: built-in: git 'config' 'merge.tool'\n> > 15:26:59.378402 git.c:348 trace: built-in: git 'config' 'difftool.vimdiff.cmd'\n> > 15:26:59.381424 git.c:348 trace: built-in: git 'config' 'mergetool.vimdiff.cmd'\n> > 15:26:59.386623 git.c:348 trace: built-in: git 'config' '--bool' 'mergetool.prompt'\n> > 15:26:59.390198 git.c:348 trace: built-in: git 'config' '--bool' 'difftool.prompt'\n> >\n> > I think the first run_command of `sed` is cleaning the working tree file\n> > to figure out *if* it differs, then the second `sed` is smudging the\n> > version in the index so that difftool can use it.\n> \n> I'm not really understanding what your filter is doing, but I tried\n> your code on my machine and I get a different result when using diff\n> and difftool on my machine.\n\nIt's supposed to be adding \"> \" to the beginning of lines in the working\ntree and stripping them in the repo, but I've just realised that the\ninstructions above are wrong since \"1.quote\" is supposed to have leading\n\"> \"'s (although the test I ran before tidying it up for the email was\ncorrect).\n\n> The diff results give me:\n> \n> diff --git a/1.quote b/1.quote\n> index 4cb29ea..f384549 100644\n> --- a/1.quote\n> +++ b/1.quote\n> @@ -1,3 +1,4 @@\n> one\n> two\n> three\n> +four\n\nIt seems that `git diff` shows the difference between the clean versions\nof the files, which is also shown by the GIT_TRACE output when running\nit.\n\n> While the diff tool tells me that the repository file is:\n> > one\n> > two\n> > three\n\nThis indicates that the smudge filter is being applied, so the\ninfrastructure is working on your machine.\n\nI think this is a difference between git-diff's internal and external\ndiff modes which is working correctly, although possibly not desirably\nin this case. The internal diff always uses clean files (so it runs the\nworking tree file through the \"clean\" filter before applying the diff\nalgorithm) but the external diff uses the working tree file so it\napplies the \"smudge\" filter to any blobs that it needs to checkout.\n\nCommit 4e218f5 (Smudge the files fed to external diff and textconv,\n2009-03-21) was the source of this behaviour.\n"},{"id":"264212","messageId":"xmqqsi9oeqhn.fsf@gitster.dls.corp.google.com","threadId":"39646","inReplyTo":"20150618160133.GO18226@serenity.lan","subject":"Re: Using clean/smudge filters with difftool","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-06-18T20:00:36Z","receivedAt":"2015-06-18T20:00:36Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"John Keeping writes:\n\n> I think this is a difference between git-diff's internal and external\n> diff modes which is working correctly, although possibly not desirably\n> in this case. The internal diff always uses clean files (so it runs the\n> working tree file through the \"clean\" filter before applying the diff\n> algorithm) but the external diff uses the working tree file so it\n> applies the \"smudge\" filter to any blobs that it needs to checkout.\n>\n> Commit 4e218f5 (Smudge the files fed to external diff and textconv,\n> 2009-03-21) was the source of this behaviour.\n\nThe fundamental design to use smudged version when interacting with\nexternal programs actually predates that particular commit, I think.\n\nThe caller of the function that was updated by that commit, i.e.\nprepare_temp_file(), reuses what is checked out to the working tree\nwhen we can (i.e. it hasn't been modified from what we think is\nchecked out) and when it is beneficial to do so (i.e. on a system\nwith FAST_WORKING_DIRECTORY defined), which means the temporary file\ngiven by the prepare_temp_file() that is used by the external tools\n(both --ext-diff program and textconv filter) are designed to be fed\nand work on the smudged version of the file. 4e218f5 did not change\nthat fundamental design; it just made things more consistent between\nthe case where we do create a new temporary file out of blob and we\nallow an unmodified checked out file to be reused.\n"},{"id":"264263","messageId":"20150618223927.GP18226@serenity.lan","threadId":"39646","inReplyTo":"xmqqsi9oeqhn.fsf@gitster.dls.corp.google.com","subject":"Re: Using clean/smudge filters with difftool","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2015-06-18T22:39:27Z","receivedAt":"2015-06-18T22:39:27Z","isPatch":false,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Thu, Jun 18, 2015 at 01:00:36PM -0700, Junio C Hamano wrote:\n> John Keeping writes:\n> \n> > I think this is a difference between git-diff's internal and external\n> > diff modes which is working correctly, although possibly not desirably\n> > in this case. The internal diff always uses clean files (so it runs the\n> > working tree file through the \"clean\" filter before applying the diff\n> > algorithm) but the external diff uses the working tree file so it\n> > applies the \"smudge\" filter to any blobs that it needs to checkout.\n> >\n> > Commit 4e218f5 (Smudge the files fed to external diff and textconv,\n> > 2009-03-21) was the source of this behaviour.\n> \n> The fundamental design to use smudged version when interacting with\n> external programs actually predates that particular commit, I think.\n> \n> The caller of the function that was updated by that commit, i.e.\n> prepare_temp_file(), reuses what is checked out to the working tree\n> when we can (i.e. it hasn't been modified from what we think is\n> checked out) and when it is beneficial to do so (i.e. on a system\n> with FAST_WORKING_DIRECTORY defined), which means the temporary file\n> given by the prepare_temp_file() that is used by the external tools\n> (both --ext-diff program and textconv filter) are designed to be fed\n> and work on the smudged version of the file. 4e218f5 did not change\n> that fundamental design; it just made things more consistent between\n> the case where we do create a new temporary file out of blob and we\n> allow an unmodified checked out file to be reused.\n\nWhen I started looking at this, I assumed the problem would be that\ngit-difftool wasn't smudging the non-working-tree files. But actually\neverything is working \"correctly\", I'm just not sure it's always what\nthe user wants (at least it isn't what was wanted in this case).\n\nCurrently, the behaviour is:\n\n\tinternal diff: compare clean files\n\texternal diff: compare smudged files\n\nThis makes sense for LF/CRLF conversion, where platform-specific tools\nclearly want the platform's line ending but the internal diff machinery\ndoesn't care.\n\nHowever, from the filter description in an earlier email, I think\nFlorian is using a clean filter to remove output from IPython notebook\nfiles (it seems that IPython saves both the input and the output in the\nsame file [1] and the output is the equivalent of, for example, C object\nfiles). In this case, the filter is one-way and discards information\nfrom the working tree file, producing a smaller and more readable diff\nin the process.\n\nI think the summary is that there are some scenarios where the external\ndiff tool should see the smudged version and others where the clean\nversion is more appropriate and Git should support both options. It\nseems this is a property of the filter, so I wonder if the best solution\nis a new \"filter..extdiff = [clean|smudge]\" configuration\nvariable (there's probably a better name for the variable than\n\"extdiff\").\n\n\n[1] http://pascalbugnion.net/blog/ipython-notebooks-and-git.html\n"},{"id":"264266","messageId":"xmqqr3p8bp8l.fsf@gitster.dls.corp.google.com","threadId":"39646","inReplyTo":"20150618223927.GP18226@serenity.lan","subject":"Re: Using clean/smudge filters with difftool","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-06-18T22:55:54Z","receivedAt":"2015-06-18T22:55:54Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"John Keeping writes:\n\n> I think the summary is that there are some scenarios where the external\n> diff tool should see the smudged version and others where the clean\n> version is more appropriate and Git should support both options. It\n> seems this is a property of the filter, so I wonder if the best solution\n> is a new \"filter..extdiff = [clean|smudge]\" configuration\n> variable (there's probably a better name for the variable than\n> \"extdiff\").\n\nNot just the external diff, but the textconv filter obeys the same\nrule. The setting should be done the same way for both, if we are\ngoing to go in that direction.\n"},{"id":"264283","messageId":"5583D993.4090305@drmicha.warpmail.net","threadId":"39646","inReplyTo":"xmqqr3p8bp8l.fsf@gitster.dls.corp.google.com","subject":"Re: Using clean/smudge filters with difftool","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2015-06-19T08:57:55Z","receivedAt":"2015-06-19T08:57:55Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Junio C Hamano venit, vidit, dixit 19.06.2015 00:55:\n> John Keeping writes:\n> \n>> I think the summary is that there are some scenarios where the external\n>> diff tool should see the smudged version and others where the clean\n>> version is more appropriate and Git should support both options. It\n>> seems this is a property of the filter, so I wonder if the best solution\n>> is a new \"filter..extdiff = [clean|smudge]\" configuration\n>> variable (there's probably a better name for the variable than\n>> \"extdiff\").\n> \n> Not just the external diff, but the textconv filter obeys the same\n> rule. The setting should be done the same way for both, if we are\n> going to go in that direction.\n> \n\ntextconv is a \"one-way\" filter from \"blob\" to \"readable blob\". External\ndiffs may prefer to work on \"blob\" rather than \"readable blob\", but the\ncurrect setup does not seem to produce surprises.\n\nclean and smudge are two-way filters: clean from \"worktree blob\" (aka\nfile) to \"repo blob\", smudge the other way round.\n\nTypically, the user perceives these as inverse to each other. But we\nonly require clean to be a left-inverse of smudge, i.e. \"(cat-file then)\nsmudge then clean\" should give the same \"repo blob\" (as \"cat-file\").\n\nWe don't require that the other way round, i.e. we don't require smudge\nto be a left-inverse of clean, and in most setups (like the current one)\nit is not: smudge does not recreate what clean has cleaned out. It is a\nno-op (the \"identity\", while clean is a \"projection\").\n\nNow, since external diff runs on smudged blobs, it appears as if we\nmixed cleaned and smudged blobs when feeding external diffs; whereas\nreally, we mix \"worktree blobs\" and \"smudged repo blobs\", which is okay\nas per our definition of clean/smudge: the difference is irrelevant by\ndefinition.\n\nI still think that feeding cleaned blobs to external diff would be less\nsurprising (and should be the default, but maybe can't be changed any\nmore) and feeding smudged blobs should be the special case requiring a\nspecial config. Because otherwise, the external diff would have to know\nwhich parts of the diff are irrelevant - if it display the complete\n(\"uncleaned\") diff, it shows differences (\"what will be committed\") that\nwill not end up in the commit (because they will get cleaned out before).\n\nAs a guiding principle, a worktree-HEAD diff and an index-HEAD diff\nshould be previews of the result of \"commit -a\" resp. \"commit\", and\ntherefore should diff cleaned versions. textconv, on the other hand, is\na setting by which you tell git: \"Don't show me the 'proper'\ndiff/commit-preview but a readable version.\"\n\nMichael\n"},{"id":"264286","messageId":"20150619093251.GQ18226@serenity.lan","threadId":"39646","inReplyTo":"5583D993.4090305@drmicha.warpmail.net","subject":"Re: Using clean/smudge filters with difftool","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2015-06-19T09:32:51Z","receivedAt":"2015-06-19T09:32:51Z","isPatch":false,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Fri, Jun 19, 2015 at 10:57:55AM +0200, Michael J Gruber wrote:\n> Junio C Hamano venit, vidit, dixit 19.06.2015 00:55:\n> > John Keeping writes:\n> > \n> >> I think the summary is that there are some scenarios where the external\n> >> diff tool should see the smudged version and others where the clean\n> >> version is more appropriate and Git should support both options. It\n> >> seems this is a property of the filter, so I wonder if the best solution\n> >> is a new \"filter..extdiff = [clean|smudge]\" configuration\n> >> variable (there's probably a better name for the variable than\n> >> \"extdiff\").\n> > \n> > Not just the external diff, but the textconv filter obeys the same\n> > rule. The setting should be done the same way for both, if we are\n> > going to go in that direction.\n> > \n> \n> textconv is a \"one-way\" filter from \"blob\" to \"readable blob\". External\n> diffs may prefer to work on \"blob\" rather than \"readable blob\", but the\n> currect setup does not seem to produce surprises.\n> \n> clean and smudge are two-way filters: clean from \"worktree blob\" (aka\n> file) to \"repo blob\", smudge the other way round.\n> \n> Typically, the user perceives these as inverse to each other. But we\n> only require clean to be a left-inverse of smudge, i.e. \"(cat-file then)\n> smudge then clean\" should give the same \"repo blob\" (as \"cat-file\").\n> \n> We don't require that the other way round, i.e. we don't require smudge\n> to be a left-inverse of clean, and in most setups (like the current one)\n> it is not: smudge does not recreate what clean has cleaned out. It is a\n> no-op (the \"identity\", while clean is a \"projection\").\n> \n> Now, since external diff runs on smudged blobs, it appears as if we\n> mixed cleaned and smudged blobs when feeding external diffs; whereas\n> really, we mix \"worktree blobs\" and \"smudged repo blobs\", which is okay\n> as per our definition of clean/smudge: the difference is irrelevant by\n> definition.\n\nI agree with this.\n\nBut I was wrong that \"should diff clean\"/\"should diff smudged\" is a\nproperty of the filter. I can also imagine a situation where a more\nintelligent external diff tool wants to see the smudged version where a\nnaïve tool would want the clean version.\n\nFor example, some of the big file stores (e.g. git-lfs [1]) use\nclean/smudge filters and I can imagine a diff utility that avoids\nneeding to fetch the data for large files and instead shows the diff on\nthe server when both blobs are available there. In that case we\ngenerally want to use the smudged copy for external diff, so the filter\nwould use that setting, but the diff utility knows better and would want\nto override that.\n\n[1] https://github.com/github/git-lfs\n"},{"id":"264333","messageId":"CAGA3+++uZcVQdYme5H5HqayK2R6N41j1DVOndok3LVRDjcMZkQ@mail.gmail.com","threadId":"39646","inReplyTo":"20150619093251.GQ18226@serenity.lan","subject":"Re: Using clean/smudge filters with difftool","fromName":"Florian Aspart","fromEmail":"florian.aspart@gmail.com","sentAt":"2015-06-19T15:04:03Z","receivedAt":"2015-06-19T15:04:03Z","isPatch":false,"sender":{"key":"florian.aspart@gmail.com","avatar":null},"body":"2015-06-19 11:32 GMT+02:00 John Keeping :\n> On Fri, Jun 19, 2015 at 10:57:55AM +0200, Michael J Gruber wrote:\n>> Junio C Hamano venit, vidit, dixit 19.06.2015 00:55:\n>> > John Keeping writes:\n>> >\n>> >> I think the summary is that there are some scenarios where the external\n>> >> diff tool should see the smudged version and others where the clean\n>> >> version is more appropriate and Git should support both options. It\n>> >> seems this is a property of the filter, so I wonder if the best solution\n>> >> is a new \"filter..extdiff = [clean|smudge]\" configuration\n>> >> variable (there's probably a better name for the variable than\n>> >> \"extdiff\").\n>> >\n>> > Not just the external diff, but the textconv filter obeys the same\n>> > rule. The setting should be done the same way for both, if we are\n>> > going to go in that direction.\n>> >\n>>\n>> textconv is a \"one-way\" filter from \"blob\" to \"readable blob\". External\n>> diffs may prefer to work on \"blob\" rather than \"readable blob\", but the\n>> currect setup does not seem to produce surprises.\n>>\n>> clean and smudge are two-way filters: clean from \"worktree blob\" (aka\n>> file) to \"repo blob\", smudge the other way round.\n>>\n>> Typically, the user perceives these as inverse to each other. But we\n>> only require clean to be a left-inverse of smudge, i.e. \"(cat-file then)\n>> smudge then clean\" should give the same \"repo blob\" (as \"cat-file\").\n>>\n>> We don't require that the other way round, i.e. we don't require smudge\n>> to be a left-inverse of clean, and in most setups (like the current one)\n>> it is not: smudge does not recreate what clean has cleaned out. It is a\n>> no-op (the \"identity\", while clean is a \"projection\").\n>>\n>> Now, since external diff runs on smudged blobs, it appears as if we\n>> mixed cleaned and smudged blobs when feeding external diffs; whereas\n>> really, we mix \"worktree blobs\" and \"smudged repo blobs\", which is okay\n>> as per our definition of clean/smudge: the difference is irrelevant by\n>> definition.\n>\n> I agree with this.\n>\n> But I was wrong that \"should diff clean\"/\"should diff smudged\" is a\n> property of the filter. I can also imagine a situation where a more\n> intelligent external diff tool wants to see the smudged version where a\n> naïve tool would want the clean version.\n>\n> For example, some of the big file stores (e.g. git-lfs [1]) use\n> clean/smudge filters and I can imagine a diff utility that avoids\n> needing to fetch the data for large files and instead shows the diff on\n> the server when both blobs are available there. In that case we\n> generally want to use the smudged copy for external diff, so the filter\n> would use that setting, but the diff utility knows better and would want\n> to override that.\n>\n> [1] https://github.com/github/git-lfs\n\nI can understand why they are not fed with the clean copy by default.\nSince some external diff tool enable modifying the working copy file,\nthis would correspond to apply the cleaning filter to the working copy\nversion.\n\nNevertheless, in my case it would be really helpful if there were an\noption to feed the external diff tool with the cleaned version.\nOtherwise, I'll probably write a custom script which does this.\n"},{"id":"264340","messageId":"xmqqsi9naavw.fsf@gitster.dls.corp.google.com","threadId":"39646","inReplyTo":"5583D993.4090305@drmicha.warpmail.net","subject":"Re: Using clean/smudge filters with difftool","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-06-19T17:03:31Z","receivedAt":"2015-06-19T17:03:31Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael J Gruber writes:\n\n> Now, since external diff runs on smudged blobs, it appears as if we\n> mixed cleaned and smudged blobs when feeding external diffs; whereas\n> really, we mix \"worktree blobs\" and \"smudged repo blobs\", which is okay\n> as per our definition of clean/smudge: the difference is irrelevant by\n> definition.\n\nIt does not appear to \"mix cleaned and smudged\" to me (even though\nbefore Dscho's commit that John pointed out, we did mix by mistake)\nto me, but you arrived at the correct conclusion in the rest of your\nsentence.\n\nWe treat \"worktree files\" and \"smudged repo blobs\" as \"comparable\"\nbecause by definition the latter is what you get if you did a\n\"checkout\" of the blob. Indeed, when we know a worktree file is an\nunmodified checkout from a blob and we want to have a read-only\ntemporary file for a \"smudged repo blob\", we allow that worktree\nfile to be used as such.\n\nSo in that sense, the commit by Dscho that John pointed out earlier\nwas not something that changed the semantics; it merely made things\nconsistent (before that commit, we used to use clean version if we\ndo not have a usable worktree file).\n\nIt is a separate question which of clean or smudged an external diff\ntool should be given to work on.\n\n> I still think that feeding cleaned blobs to external diff would be less\n> surprising (and should be the default, but maybe can't be changed any\n> more) and feeding smudged blobs should be the special case requiring a\n> special config.\n\nGo back six years and make a review comment before 4e218f54 (Smudge\nthe files fed to external diff and textconv, 2009-03-21) was taken\n;-). The argument against that commit may have gone like this:\n\n * The current (that is, current as of 4e218f54^) code is\n inconsistent, and your patch has a side effect of making it\n consistent by always feeding smudged version.\n\n * We however could make it consistent by always feeding clean\n version (i.e. disable borrow-from-working-tree codepath when\n driving external diff). And that gives us cleaner semantics; the\n internal diff and external diff will both work on clean, not\n smudged data.\n\n * Of course, going the \"clean\" way would not help your cause of\n allowing external diff to work on smudged version, so you would\n need a separate patch on top of that \"consistently feed 'clean'\n version\" fix to optionally allow \"consistently feed 'smudge'\n version\" mode to help msysGit issue 177.\n\nAnd I would have bought such an argument with 97% chance [*1*].\n\nI do not think 6 years have changed things very much with respect to\nthe above three-bullet point argument, except that it would be too\nlate to set the default to 'clean' all of a sudden. So a plausible\nway forward would be to\n\n * introduce an option to feed 'clean' versions to external diff\n drivers, perhaps with --ext-diff-clean= command line\n option and GIT_EXTERNAL_DIFF_CLEAN environment variable, both of\n which take precedence over existing --ext-diff/GIT_EXTERNAL_DIFF\n\n * optionally add a configuration variable diff.feedCleanToExternal\n that makes --ext-diff/GIT_EXTERNAL_DIFF behave as if their\n 'clean' siblings were given. Default it to false.\n\nMy gut feeling is that textconv should need a similar treatment for\nconsistency (after all, it goes through the same prepare_temp_file()\ninfrastructure).\n\n\n[Footnote]\n\n*1* The 3% reservation is that I am not entirely convinced that\n\"both internal and external get to work on the same 'clean'\nrepresentation gives us cleaner semantics\" is always true.\n"},{"id":"264456","messageId":"558710AF.9040009@drmicha.warpmail.net","threadId":"39646","inReplyTo":"xmqqsi9naavw.fsf@gitster.dls.corp.google.com","subject":"Re: Using clean/smudge filters with difftool","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2015-06-21T19:29:51Z","receivedAt":"2015-06-21T19:29:51Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Junio C Hamano venit, vidit, dixit 19.06.2015 19:03:\n> Michael J Gruber writes:\n> \n>> Now, since external diff runs on smudged blobs, it appears as if we\n>> mixed cleaned and smudged blobs when feeding external diffs; whereas\n>> really, we mix \"worktree blobs\" and \"smudged repo blobs\", which is okay\n>> as per our definition of clean/smudge: the difference is irrelevant by\n>> definition.\n> \n> It does not appear to \"mix cleaned and smudged\" to me (even though\n> before Dscho's commit that John pointed out, we did mix by mistake)\n> to me,\n\n... neither to me. I appears as if you missed the past subjunctive ;)\n\n> but you arrived at the correct conclusion in the rest of your\n> sentence.\n\n> We treat \"worktree files\" and \"smudged repo blobs\" as \"comparable\"\n> because by definition the latter is what you get if you did a\n> \"checkout\" of the blob. Indeed, when we know a worktree file is an\n> unmodified checkout from a blob and we want to have a read-only\n> temporary file for a \"smudged repo blob\", we allow that worktree\n> file to be used as such.\n> \n> So in that sense, the commit by Dscho that John pointed out earlier\n> was not something that changed the semantics; it merely made things\n> consistent (before that commit, we used to use clean version if we\n> do not have a usable worktree file).\n> \n> It is a separate question which of clean or smudged an external diff\n> tool should be given to work on.\n> \n>> I still think that feeding cleaned blobs to external diff would be less\n>> surprising (and should be the default, but maybe can't be changed any\n>> more) and feeding smudged blobs should be the special case requiring a\n>> special config.\n> \n> Go back six years and make a review comment before 4e218f54 (Smudge\n> the files fed to external diff and textconv, 2009-03-21) was taken\n> ;-). The argument against that commit may have gone like this:\n> \n> * The current (that is, current as of 4e218f54^) code is\n> inconsistent, and your patch has a side effect of making it\n> consistent by always feeding smudged version.\n> \n> * We however could make it consistent by always feeding clean\n> version (i.e. disable borrow-from-working-tree codepath when\n> driving external diff). And that gives us cleaner semantics; the\n> internal diff and external diff will both work on clean, not\n> smudged data.\n> \n> * Of course, going the \"clean\" way would not help your cause of\n> allowing external diff to work on smudged version, so you would\n> need a separate patch on top of that \"consistently feed 'clean'\n> version\" fix to optionally allow \"consistently feed 'smudge'\n> version\" mode to help msysGit issue 177.\n> \n> And I would have bought such an argument with 97% chance [*1*].\n> \n> I do not think 6 years have changed things very much with respect to\n> the above three-bullet point argument, except that it would be too\n> late to set the default to 'clean' all of a sudden. So a plausible\n> way forward would be to\n> \n> * introduce an option to feed 'clean' versions to external diff\n> drivers, perhaps with --ext-diff-clean= command line\n> option and GIT_EXTERNAL_DIFF_CLEAN environment variable, both of\n> which take precedence over existing --ext-diff/GIT_EXTERNAL_DIFF\n> \n> * optionally add a configuration variable diff.feedCleanToExternal\n> that makes --ext-diff/GIT_EXTERNAL_DIFF behave as if their\n> 'clean' siblings were given. Default it to false.\n> \n> My gut feeling is that textconv should need a similar treatment for\n> consistency (after all, it goes through the same prepare_temp_file()\n> infrastructure).\n> \n> \n> [Footnote]\n> \n> *1* The 3% reservation is that I am not entirely convinced that\n> \"both internal and external get to work on the same 'clean'\n> representation gives us cleaner semantics\" is always true.\n\nWith consistency stepping back behind compatibility, I don't expect any\ndefaults to change.\n\nBut a knob to change defaults would be nice, yes, and in that case for\nexternal diff as well as textconv. A config variable should suffice\ngiven that we have \"git -c\" these days.\n\nMichael\n"}]}