{"thread":{"id":"49231","subject":"GIT_TRACE doesn't show content filter files it's operating on","startedAt":"2018-08-27T23:23:40Z","lastAt":"2018-08-28T19:44:58Z","messageCount":6,"participants":["Stas Bekman","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"356639","messageId":"3fdc0c24-bbee-e7b3-ec43-7e926cee71e1@stason.org","threadId":"49231","inReplyTo":null,"subject":"GIT_TRACE doesn't show content filter files it's operating on","fromName":"Stas Bekman","fromEmail":"stas@stason.org","sentAt":"2018-08-27T23:23:34Z","receivedAt":"2018-08-27T23:23:40Z","isPatch":false,"sender":{"key":"stas@stason.org","avatar":null},"body":"Hello,\n\nI'm debugging the workings of a configured git content filter\n(nbstripout) and I'm trying to get GIT_TRACE to show me the files it's\noperating on, but it doesn't. Consider:\n\n$ GIT_TRACE=1 git pull origin master\n[...] removed irrelevant sections of the output\n16:49:28.846707 run-command.c:640       trace: run_command: git merge\nFETCH_HEAD\n16:49:28.849309 git.c:344               trace: built-in: git merge\nFETCH_HEAD\nUpdating 1ea49ad..ae0ba93\n16:49:28.863291 run-command.c:640       trace: run_command: nbstripout\n16:49:28.864700 run-command.c:640       trace: run_command: nbstripout\n16:49:28.866060 run-command.c:640       trace: run_command: nbstripout\n[...] many more of the same\n\nHow can I get GIT_TRACE's run_command to show the arguments passed to\nthe filter? I looked at various other debug environment variables in\ngit's manual, but I don't see anything that would enable that level of\ndebug.\n\nI'm aware of git check-attr, but I'd like to see the run time trace on\nwhich files it's running the filter on and with which arguments.\n\ngit version 2.17.1\n\nI originally posted this as a question here:\n<https://stackoverflow.com/questions/51995773/getting-git-to-show-specific-filenames-it-is-running-content-filters-on>\nand it appears that sq_quote_argv_pretty() doesn't get argv when a\ncontent filter is invoked, so it doesn't show this information.\n\nWould it be possible to make the files and any other passed arguments\nshow in the trace?\n\nThank you.\n\n-- \n________________________________________________\nStas Bekman       <'))))><       <'))))><\nhttps://stasosphere.com  https://chestofbooks.com\nhttps://experientialsexlab.com https://stason.org\nhttps://stasosphere.com/experience-life/my-books\n"},{"id":"356641","messageId":"20180827235321.GB11663@sigill.intra.peff.net","threadId":"49231","inReplyTo":"3fdc0c24-bbee-e7b3-ec43-7e926cee71e1@stason.org","subject":"Re: GIT_TRACE doesn't show content filter files it's operating on","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-08-27T23:53:21Z","receivedAt":"2018-08-27T23:53:25Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Aug 27, 2018 at 04:23:34PM -0700, Stas Bekman wrote:\n\n> I'm debugging the workings of a configured git content filter\n> (nbstripout) and I'm trying to get GIT_TRACE to show me the files it's\n> operating on, but it doesn't. Consider:\n> \n> $ GIT_TRACE=1 git pull origin master\n> [...] removed irrelevant sections of the output\n> 16:49:28.846707 run-command.c:640       trace: run_command: git merge\n> FETCH_HEAD\n> 16:49:28.849309 git.c:344               trace: built-in: git merge\n> FETCH_HEAD\n> Updating 1ea49ad..ae0ba93\n> 16:49:28.863291 run-command.c:640       trace: run_command: nbstripout\n> 16:49:28.864700 run-command.c:640       trace: run_command: nbstripout\n> 16:49:28.866060 run-command.c:640       trace: run_command: nbstripout\n> [...] many more of the same\n> \n> How can I get GIT_TRACE's run_command to show the arguments passed to\n> the filter? I looked at various other debug environment variables in\n> git's manual, but I don't see anything that would enable that level of\n> debug.\n\nGIT_TRACE should always show the arguments. But unless you specify\narguments in the clean/smudge filter config, then Git won't pass any.\nThe stdin/stdout stream is all that matters.\n\nSo e.g.:\n\n  $ echo '* filter=foo' >.gitattributes\n  $ git config filter.foo.clean 'myfilter'\n  $ GIT_TRACE=1 git add .\n  19:42:16.516401 [pid=14112] git.c:415             trace: built-in: git add .\n  19:42:16.517454 [pid=14112] run-command.c:637     trace: run_command: myfilter\n\n  $ git config filter.foo.clean 'myfilter --foo'\n  $ touch .gitattributes ;# make sure we actually read it again ;)\n  $ GIT_TRACE=1 git add .\n  19:42:58.122942 [pid=14156] git.c:415             trace: built-in: git add .\n  19:42:58.124023 [pid=14156] run-command.c:637     trace: run_command: 'myfilter --foo'\n\nYou can use \"%f\" to pass the name of the file, like:\n\n  $ git config filter.foo.clean 'myfilter %f'\n  $ touch .gitattributes\n  $ GIT_TRACE=1 git add .\n  19:44:51.187177 [pid=14318] git.c:415             trace: built-in: git add .\n  19:44:51.188256 [pid=14318] run-command.c:637     trace: run_command: 'myfilter '\\''.gitattributes'\\'''\n\nOf course that won't be helpful if your filter actually respects the\nargument. For a \"clean\" filter that might be OK (e.g., if it just tells\nyour filter to read from the filesystem instead of stdin), but it's\nalmost certainly not what you want for a \"smudge\" filter.\n\nYou can work around it with some shell hackery:\n\n  git config filter.foo.clean 'f() { echo >&2 \"cleaning $1\"; myfilter ...; }; f %f'\n\nand then even without GIT_TRACE, you get:\n\n  $ git add .\n  cleaning .gitattributes\n\nOr if you really just want to trigger for GIT_TRACE, try just this:\n\n  $ git config filter.foo.clean 'f() { myfilter; }; f %f'\n  19:52:52.874064 [pid=14719] git.c:415             trace: built-in: git add .\n  19:52:52.875115 [pid=14719] run-command.c:637     trace: run_command: 'f() { myfilter; }; f '\\''.gitattributes'\\'''\n\nThere you get the name in the trace output, but the invoked command\ndoesn't actually do anything with it.\n\n-Peff\n"},{"id":"356643","messageId":"28045b26-4822-b00c-30f3-1076d2e49d1f@stason.org","threadId":"49231","inReplyTo":"20180827235321.GB11663@sigill.intra.peff.net","subject":"Re: GIT_TRACE doesn't show content filter files it's operating on","fromName":"Stas Bekman","fromEmail":"stas@stason.org","sentAt":"2018-08-28T00:22:13Z","receivedAt":"2018-08-28T00:22:18Z","isPatch":false,"sender":{"key":"stas@stason.org","avatar":null},"body":"On 2018-08-27 04:53 PM, Jeff King wrote:\n> On Mon, Aug 27, 2018 at 04:23:34PM -0700, Stas Bekman wrote:\n[...]\n>> How can I get GIT_TRACE's run_command to show the arguments passed to\n>> the filter? I looked at various other debug environment variables in\n>> git's manual, but I don't see anything that would enable that level of\n>> debug.\n[...]\n> You can work around it with some shell hackery:\n> \n>   git config filter.foo.clean 'f() { echo >&2 \"cleaning $1\"; myfilter ...; }; f %f'\n> \n> and then even without GIT_TRACE, you get:\n> \n>   $ git add .\n>   cleaning .gitattributes\n> \n> Or if you really just want to trigger for GIT_TRACE, try just this:\n> \n>   $ git config filter.foo.clean 'f() { myfilter; }; f %f'\n>   19:52:52.874064 [pid=14719] git.c:415             trace: built-in: git add .\n>   19:52:52.875115 [pid=14719] run-command.c:637     trace: run_command: 'f() { myfilter; }; f '\\''.gitattributes'\\'''\n\nYour suggestions do the trick, Jeff. Thank you.\n\n1. To benefit others who might be looking for something similar may I\npost your suggestions as an answer to:\n<https://stackoverflow.com/questions/51995773/getting-git-to-show-specific-filenames-it-is-running-content-filters-on>?\n\n2. Is there no way to get git to do the filename reporting as a normal\nGIT_TRACE behavior? I don't know anything about its internal workings,\nbut it surely must knows which file it operates on when it opens it and\nsends its data as stdin to the content filter. It makes the debugging so\nmuch easier when one can see what files are being worked on. So perhaps\nthis utility can be made available to all not just as a hack/workaround.\n\n\n-- \n________________________________________________\nStas Bekman       <'))))><       <'))))><\nhttps://stasosphere.com  https://chestofbooks.com\nhttps://experientialsexlab.com https://stason.org\nhttps://stasosphere.com/experience-life/my-books\n"},{"id":"356645","messageId":"20180828005810.GA18659@sigill.intra.peff.net","threadId":"49231","inReplyTo":"28045b26-4822-b00c-30f3-1076d2e49d1f@stason.org","subject":"Re: GIT_TRACE doesn't show content filter files it's operating on","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-08-28T00:58:10Z","receivedAt":"2018-08-28T00:58:14Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Aug 27, 2018 at 05:22:13PM -0700, Stas Bekman wrote:\n\n> Your suggestions do the trick, Jeff. Thank you.\n> \n> 1. To benefit others who might be looking for something similar may I\n> post your suggestions as an answer to:\n> <https://stackoverflow.com/questions/51995773/getting-git-to-show-specific-filenames-it-is-running-content-filters-on>?\n\nGreat, thanks.\n\n> 2. Is there no way to get git to do the filename reporting as a normal\n> GIT_TRACE behavior? I don't know anything about its internal workings,\n> but it surely must knows which file it operates on when it opens it and\n> sends its data as stdin to the content filter. It makes the debugging so\n> much easier when one can see what files are being worked on. So perhaps\n> this utility can be made available to all not just as a hack/workaround.\n\nNo, because GIT_TRACE itself only reports on the execution of commands\nand sub-processes. There are other GIT_TRACE_* variables for various\nsubsystems, but AFAIK nobody has instrumented the smudge/clean filter\ncode. IMHO it would be reasonable to have a GIT_TRACE_CONVERT\nthat covered convert.c (so these filters, but also newline conversion,\netc).\n\n-Peff\n"},{"id":"356646","messageId":"9fc6cdcf-2e72-8eb1-9bcd-f513babd9c6f@stason.org","threadId":"49231","inReplyTo":"20180828005810.GA18659@sigill.intra.peff.net","subject":"Re: GIT_TRACE doesn't show content filter files it's operating on","fromName":"Stas Bekman","fromEmail":"stas@stason.org","sentAt":"2018-08-28T01:13:03Z","receivedAt":"2018-08-28T01:13:07Z","isPatch":false,"sender":{"key":"stas@stason.org","avatar":null},"body":"On 2018-08-27 05:58 PM, Jeff King wrote:\n[...]\n>> 2. Is there no way to get git to do the filename reporting as a normal\n>> GIT_TRACE behavior? I don't know anything about its internal workings,\n>> but it surely must knows which file it operates on when it opens it and\n>> sends its data as stdin to the content filter. It makes the debugging so\n>> much easier when one can see what files are being worked on. So perhaps\n>> this utility can be made available to all not just as a hack/workaround.\n> \n> No, because GIT_TRACE itself only reports on the execution of commands\n> and sub-processes. There are other GIT_TRACE_* variables for various\n> subsystems, but AFAIK nobody has instrumented the smudge/clean filter\n> code. IMHO it would be reasonable to have a GIT_TRACE_CONVERT\n> that covered convert.c (so these filters, but also newline conversion,\n> etc).\n\nThank you, Jeff.\n\nWould you please add this to the TODO list (if that's how new feature\nrequests are made)?\n\nI now know how get the filenames for \"clean/smudge\" filters. Can you\nplease help with the same for \"textconv\". %f doesn't work - it gets\nstuck there waiting for stdin, the following seems to pass, but I'm not\nsure it's correct:\n\n[diff \"ipynb-code\"]\n\ttextconv = \"f() { echo >&2 \\\"textconv: nbstripout -t $*\\\"; nbstripout\n-t; }; cat $1 > f\"\n\nIn a trace (git diff w/ one modified file) it gets invoked twice:\n\n18:05:27.640955 run-command.c:640       trace: run_command: 'f() { echo\n>&2 \"textconv: nbstripout $*\"; nbstripout -t; }; cat $1 > f'\n/tmp/4nCYaT_004_callbacks.ipynb\n18:05:27.642205 run-command.c:640       trace: run_command: 'f() { echo\n>&2 \"textconv: nbstripout $*\"; nbstripout -t; }; cat $1 > f'\ndev_nb/004_callbacks.ipynb\n\nThe first trace gives a tmp filename, whereas I need the actual filename.\n\nThank you.\n\n-- \n________________________________________________\nStas Bekman       <'))))><       <'))))><\nhttps://stasosphere.com  https://chestofbooks.com\nhttps://experientialsexlab.com https://stason.org\nhttps://stasosphere.com/experience-life/my-books\n"},{"id":"356727","messageId":"a3a1e935-8eb7-0894-4474-74f79dade11e@stason.org","threadId":"49231","inReplyTo":"9fc6cdcf-2e72-8eb1-9bcd-f513babd9c6f@stason.org","subject":"Re: GIT_TRACE doesn't show content filter files it's operating on","fromName":"Stas Bekman","fromEmail":"stas@stason.org","sentAt":"2018-08-28T19:44:52Z","receivedAt":"2018-08-28T19:44:58Z","isPatch":false,"sender":{"key":"stas@stason.org","avatar":null},"body":"On 2018-08-27 06:13 PM, Stas Bekman wrote:\n[...]\n> I now know how get the filenames for \"clean/smudge\" filters. Can you\n> please help with the same for \"textconv\". %f doesn't work - it gets\n> stuck there waiting for stdin, the following seems to pass, but I'm not\n> sure it's correct:\n\nOn a closer look this is not needed for \"textconv\" as it already logs\nthe filename in the GIT_TRACE=1 trace.\n\nI have no more remaining questions, Thank you for your help, Jeff.\n\n-- \n________________________________________________\nStas Bekman       <'))))><       <'))))><\nhttps://stasosphere.com  https://chestofbooks.com\nhttps://experientialsexlab.com https://stason.org\nhttps://stasosphere.com/experience-life/my-books\n"}]}