{"thread":{"id":"13233","subject":"[PATCH 1/3] Add --blob-filter option to filter-branch.","startedAt":"2008-04-23T19:42:35Z","lastAt":"2008-04-24T01:33:00Z","messageCount":11,"participants":["Avery Pennarun","Johannes Schindelin","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"75095","messageId":"1208979757-30860-1-git-send-email-apenwarr@gmail.com","threadId":"13233","inReplyTo":null,"subject":"[PATCH 1/3] Add --blob-filter option to filter-branch.","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-23T19:42:35Z","receivedAt":"2008-04-23T19:42:35Z","isPatch":true,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"From: Jeff King \n\nOn Tue, Apr 22, 2008 at 12:51:14PM -0400, Avery Pennarun wrote:\n\n> Do you think git would benefit from having a generalized version of\n> this script? Basically, the user provides a \"munge\" script on the\n> command line, and there's a git-filter-branch mode for auto-munging\n> (with a cache) every file in every checkin. Even if it's *only* ever\n> used for CRLF, I can imagine this being useful to a lot of people.\n\nIt was easy enough to work up the patch below, which allows\n\n git filter-branch --blob-filter 'tr a-z A-Z'\n\nHowever, it's _still_ horribly slow. Shell script is nice and flexible,\nbut running a tight loop like this is just painful. I suspect\nfilter-branch in something like perl would be a lot faster and just as\nflexible (you could even do it in C, but you'd probably have to invent a\nlittle domain-specific scripting language).\n\nIt is still much better performance than a tree filter, though:\n\n $ cd git && time git filter-branch --tree-filter '\n find . -type f | while read f; do\n tr a-z A-Z <\"$f\" >tmp\n mv tmp \"$f\"\n done\n ' HEAD~10..HEAD\n\n real 4m38.626s\n user 1m32.726s\n sys 2m51.163s\n\n $ cd git && git filter-branch --blob-filter 'tr a-z A-Z' HEAD~10..HEAD\n real 1m40.809s\n user 0m36.822s\n sys 1m14.273s\n\nLots of system time in both. I'm sure we spend a fair bit of time\nhitting our very large map and blob-cache directories, which would be\nmuch more nicely implemented as associative arrays in memory (if we were\nusing a more featureful language).\n\nAnyway, here is the patch. I don't know if it is even worth applying,\nsince it is still painfully slow.\n\nAcked-by: Johannes Schindelin \n---\n git-filter-branch.sh | 30 ++++++++++++++++++++++++++++++\n 1 files changed, 30 insertions(+), 0 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex ea59015..980c431 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -54,6 +54,23 @@ EOF\n \n eval \"$functions\"\n \n+munge_blobs() {\n+\twhile read mode sha1 stage path\n+\tdo\n+\t\tif ! test -r \"$workdir/../blob-cache/$sha1\"\n+\t\tthen\n+\t\t\tnew=`git cat-file blob $sha1 |\n+\t\t\t eval \"$filter_blob\" |\n+\t\t\t git hash-object -w --stdin`\n+\t\t\tprintf $new >$workdir/../blob-cache/$sha1\n+\t\tfi\n+\t\tprintf \"%s %s\\t%s\\n\" \\\n+\t\t\t\"$mode\" \\\n+\t\t\t$(cat \"$workdir/../blob-cache/$sha1\") \\\n+\t\t\t\"$path\"\n+\tdone\n+}\n+\n # When piped a commit, output a script to set the ident of either\n # \"author\" or \"committer\n \n@@ -105,6 +122,7 @@ tempdir=.git-rewrite\n filter_env=\n filter_tree=\n filter_index=\n+filter_blob=\n filter_parent=\n filter_msg=cat\n filter_commit='git commit-tree \"$@\"'\n@@ -150,6 +168,9 @@ do\n \t--index-filter)\n \t\tfilter_index=\"$OPTARG\"\n \t\t;;\n+\t--blob-filter)\n+\t\tfilter_blob=\"$OPTARG\"\n+\t\t;;\n \t--parent-filter)\n \t\tfilter_parent=\"$OPTARG\"\n \t\t;;\n@@ -227,6 +248,9 @@ ret=0\n # map old->new commit ids for rewriting parents\n mkdir ../map || die \"Could not create map/ directory\"\n \n+# cache rewritten blobs for blob filter\n+mkdir ../blob-cache || die \"Could not create blob-cache/ directory\"\n+\n case \"$filter_subdir\" in\n \"\")\n \tgit rev-list --reverse --topo-order --default HEAD \\\n@@ -295,6 +319,12 @@ while read commit parents; do\n \teval \"$filter_index\" < /dev/null ||\n \t\tdie \"index filter failed: $filter_index\"\n \n+\tif test -n \"$filter_blob\"; then\n+\t\tgit ls-files --stage |\n+\t\tmunge_blobs |\n+\t\tgit update-index --index-info\n+\tfi\n+\n \tparentstr=\n \tfor parent in $parents; do\n \t\tfor reparent in $(map \"$parent\"); do\n-- \n1.5.4.3\n"},{"id":"75097","messageId":"1208979757-30860-2-git-send-email-apenwarr@gmail.com","threadId":"13233","inReplyTo":"1208979757-30860-1-git-send-email-apenwarr@gmail.com","subject":"[PATCH 2/3] Make filter-branch --glob-filter much faster by not calling 'cat'","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-23T19:42:36Z","receivedAt":"2008-04-23T19:42:36Z","isPatch":true,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"The main loop of munge_blobs() had to fork-exec \"cat\" every time through the\nloop, even when a blob was already cached. Let's use the sh builtin 'read'\ninstead for a huge speedup.\n\ncd git\ntime git filter-branch --blob-filter 'tr a-z A-Z' HEAD~10..HEAD\n\n(original --blob-filter)\nreal 3m58.569s\nuser 0m22.900s\nsys 3m32.030s\n\n(with 'cat' calls removed)\nreal\t1m11.931s\nuser\t0m8.520s\nsys\t1m2.900s\n\n(with 'cat' calls removed and blob cache already filled)\nreal\t0m19.660s\nuser\t0m3.930s\nsys\t0m15.720s\n\nSigned-off-by: Avery Pennarun \n---\n git-filter-branch.sh | 18 +++++++++++-------\n 1 files changed, 11 insertions(+), 7 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex 980c431..37ac99d 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -57,16 +57,18 @@ eval \"$functions\"\n munge_blobs() {\n \twhile read mode sha1 stage path\n \tdo\n-\t\tif ! test -r \"$workdir/../blob-cache/$sha1\"\n+\t\tif ! test -r \"$cachedir/$sha1\"\n \t\tthen\n-\t\t\tnew=`git cat-file blob $sha1 |\n-\t\t\t eval \"$filter_blob\" |\n-\t\t\t git hash-object -w --stdin`\n-\t\t\tprintf $new >$workdir/../blob-cache/$sha1\n+\t\t\tnew=$(git cat-file blob $sha1 |\n+\t\t\t eval \"$filter_blob\" |\n+\t\t\t git hash-object -w --stdin)\n+\t\t\tprintf $new >$cachedir/$sha1\n+\t\telse\n+\t\t\tread new <\"$cachedir/$sha1\"\n \t\tfi\n \t\tprintf \"%s %s\\t%s\\n\" \\\n \t\t\t\"$mode\" \\\n-\t\t\t$(cat \"$workdir/../blob-cache/$sha1\") \\\n+\t\t\t\"$new\" \\\n \t\t\t\"$path\"\n \tdone\n }\n@@ -108,6 +110,7 @@ USAGE=\"[--env-filter ] [--tree-filter ] \\\n [--index-filter ] [--parent-filter ] \\\n [--msg-filter ] [--commit-filter ] \\\n [--tag-name-filter ] [--subdirectory-filter ] \\\n+[--blob-filter ] \\\n [--original ] [-d ] [-f | --force] \\\n [...]\"\n \n@@ -249,7 +252,8 @@ ret=0\n mkdir ../map || die \"Could not create map/ directory\"\n \n # cache rewritten blobs for blob filter\n-mkdir ../blob-cache || die \"Could not create blob-cache/ directory\"\n+cachedir=\"$workdir/../blob-cache\"\n+mkdir \"$cachedir\" || die \"Could not create blob-cache/ directory\"\n \n case \"$filter_subdir\" in\n \"\")\n-- \n1.5.4.3\n"},{"id":"75096","messageId":"1208979757-30860-3-git-send-email-apenwarr@gmail.com","threadId":"13233","inReplyTo":"1208979757-30860-2-git-send-email-apenwarr@gmail.com","subject":"[PATCH 3/3] Update documentation to describe git filter-branch --blob-filter.","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-23T19:42:37Z","receivedAt":"2008-04-23T19:42:37Z","isPatch":true,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"Signed-off-by: Avery Pennarun \n---\n Documentation/git-filter-branch.txt | 17 +++++++++++++++++\n 1 files changed, 17 insertions(+), 0 deletions(-)\n\ndiff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt\nindex 2a78549..367f119 100644\n--- a/Documentation/git-filter-branch.txt\n+++ b/Documentation/git-filter-branch.txt\n@@ -12,6 +12,7 @@ SYNOPSIS\n \t[--index-filter ] [--parent-filter ]\n \t[--msg-filter ] [--commit-filter ]\n \t[--tag-name-filter ] [--subdirectory-filter ]\n+\t[--blob-filter ] [-d ] [-f | --force]\n \t[...]\n \n@@ -143,6 +144,15 @@ definition impossible to preserve signatures at any rate.)\n \tThe result will contain that directory (and only that) as its\n \tproject root.\n \n+--blob-filter ::\n+\tThis is the filter for modifying the contents of each file (blob)\n+\tin the tree. The contents of a file are provided on stdin, and\n+\tthe new file contents should be provided on stdout. For efficiency,\n+\tthe before/after results of a given blob are only calculated once\n+\tand then cached, so your filter must always return the same output\n+\tblob for any given input blob. You might use this filter for\n+\tconverting CRLF to LF in all your files, for example.\n+\n --original ::\n \tUse this option to set the namespace where the original commits\n \twill be stored. The default value is 'refs/original'.\n@@ -185,6 +195,13 @@ git filter-branch --index-filter 'git update-index --remove filename' HEAD\n \n Now, you will get the rewritten history saved in HEAD.\n \n+To convert CRLF to LF in all your files using the \"fromdos\" program (be\n+careful: this will attempt to modify binary files too!):\n+\n+----------------------------------------------\n+git filter-branch --blob-filter 'fromdos' HEAD\n+----------------------------------------------\n+\n To set a commit (which typically is at the tip of another\n history) to be the parent of the current initial commit, in\n order to paste the other history behind the current history:\n-- \n1.5.4.3\n"},{"id":"75099","messageId":"alpine.DEB.1.00.0804232104160.20267@eeepc-johanness","threadId":"13233","inReplyTo":"1208979757-30860-1-git-send-email-apenwarr@gmail.com","subject":"Re: [PATCH 1/3] Add --blob-filter option to filter-branch.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-04-23T20:05:19Z","receivedAt":"2008-04-23T20:05:19Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 23 Apr 2008, Avery Pennarun wrote:\n\n> From: Jeff King \n> \n> On Tue, Apr 22, 2008 at 12:51:14PM -0400, Avery Pennarun wrote:\n> \n> > Do you think git would benefit from having a generalized version of\n> > this script? Basically, the user provides a \"munge\" script on the\n> > command line, and there's a git-filter-branch mode for auto-munging\n> > (with a cache) every file in every checkin. Even if it's *only* ever\n> > used for CRLF, I can imagine this being useful to a lot of people.\n> \n> It was easy enough to work up the patch below, which allows\n> \n> git filter-branch --blob-filter 'tr a-z A-Z'\n> \n> However, it's _still_ horribly slow. Shell script is nice and flexible,\n> but running a tight loop like this is just painful. I suspect\n> filter-branch in something like perl would be a lot faster and just as\n> flexible (you could even do it in C, but you'd probably have to invent a\n> little domain-specific scripting language).\n> \n> It is still much better performance than a tree filter, though:\n> \n> $ cd git && time git filter-branch --tree-filter '\n> find . -type f | while read f; do\n> tr a-z A-Z <\"$f\" >tmp\n> mv tmp \"$f\"\n> done\n> ' HEAD~10..HEAD\n> \n> real 4m38.626s\n> user 1m32.726s\n> sys 2m51.163s\n> \n> $ cd git && git filter-branch --blob-filter 'tr a-z A-Z' HEAD~10..HEAD\n> real 1m40.809s\n> user 0m36.822s\n> sys 1m14.273s\n> \n> Lots of system time in both. I'm sure we spend a fair bit of time\n> hitting our very large map and blob-cache directories, which would be\n> much more nicely implemented as associative arrays in memory (if we were\n> using a more featureful language).\n> \n> Anyway, here is the patch. I don't know if it is even worth applying,\n> since it is still painfully slow.\n\nNot all of this belongs in the commit messaage.\n\n> Acked-by: Johannes Schindelin \n\nThis does.\n\nA good general rule is: if you think it would be funny/strange to read \nthis message in the output of \"git log\", it should be changed.\n\nCiao,\nDscho\n"},{"id":"75101","messageId":"32541b130804231312n50e86ce6g2b2331ca514ba982@mail.gmail.com","threadId":"13233","inReplyTo":"alpine.DEB.1.00.0804232104160.20267@eeepc-johanness","subject":"Re: [PATCH 1/3] Add --blob-filter option to filter-branch.","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-23T20:12:25Z","receivedAt":"2008-04-23T20:12:25Z","isPatch":true,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/23/08, Johannes Schindelin wrote:\n> Not all of this belongs in the commit messaage.\n>\n> > Acked-by: Johannes Schindelin \n>\n> This does.\n>\n> A good general rule is: if you think it would be funny/strange to read\n> this message in the output of \"git log\", it should be changed.\n\nI felt uncomfortable modifying the message attached to Jeff's patch\n(which actually does pretty clearly explain what's going on), since\nit's not mine. If he wants to send me a better commit message,\nthat'll be fine too.\n\nHave fun,\n\nAvery\n"},{"id":"75102","messageId":"alpine.DEB.1.00.0804232113480.20267@eeepc-johanness","threadId":"13233","inReplyTo":"32541b130804231312n50e86ce6g2b2331ca514ba982@mail.gmail.com","subject":"Re: [PATCH 1/3] Add --blob-filter option to filter-branch.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-04-23T20:14:16Z","receivedAt":"2008-04-23T20:14:16Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 23 Apr 2008, Avery Pennarun wrote:\n\n> On 4/23/08, Johannes Schindelin wrote:\n> > Not all of this belongs in the commit messaage.\n> >\n> > > Acked-by: Johannes Schindelin \n> >\n> > This does.\n> >\n> > A good general rule is: if you think it would be funny/strange to \n> > read this message in the output of \"git log\", it should be changed.\n> \n> I felt uncomfortable modifying the message attached to Jeff's patch\n> (which actually does pretty clearly explain what's going on), since\n> it's not mine. If he wants to send me a better commit message,\n> that'll be fine too.\n\nWell, I think that you would do a good job rephrasing this as a proper \ncommit message :-)\n\nCiao,\nDscho\n"},{"id":"75103","messageId":"1208981890-758-1-git-send-email-apenwarr@gmail.com","threadId":"13233","inReplyTo":"alpine.DEB.1.00.0804232113480.20267@eeepc-johanness","subject":"[PATCH 1/3 v2] Add --blob-filter option to filter-branch.","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-23T20:18:10Z","receivedAt":"2008-04-23T20:18:10Z","isPatch":true,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"From: Jeff King \n\nThis patch allows\n\n git filter-branch --blob-filter 'tr a-z A-Z'\n\nHowever, it's _still_ horribly slow. Shell script is nice and flexible,\nbut running a tight loop like this is just painful. I suspect\nfilter-branch in something like perl would be a lot faster and just as\nflexible.\n\nIt is still much better performance than a tree filter, though:\n\n $ cd git && time git filter-branch --tree-filter '\n find . -type f | while read f; do\n tr a-z A-Z <\"$f\" >tmp\n mv tmp \"$f\"\n done\n ' HEAD~10..HEAD\n\n real 4m38.626s\n user 1m32.726s\n sys 2m51.163s\n\n $ cd git && git filter-branch --blob-filter 'tr a-z A-Z' HEAD~10..HEAD\n real 1m40.809s\n user 0m36.822s\n sys 1m14.273s\n\nAcked-by: Johannes Schindelin \n---\n git-filter-branch.sh | 30 ++++++++++++++++++++++++++++++\n 1 files changed, 30 insertions(+), 0 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex ea59015..980c431 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -54,6 +54,23 @@ EOF\n \n eval \"$functions\"\n \n+munge_blobs() {\n+\twhile read mode sha1 stage path\n+\tdo\n+\t\tif ! test -r \"$workdir/../blob-cache/$sha1\"\n+\t\tthen\n+\t\t\tnew=`git cat-file blob $sha1 |\n+\t\t\t eval \"$filter_blob\" |\n+\t\t\t git hash-object -w --stdin`\n+\t\t\tprintf $new >$workdir/../blob-cache/$sha1\n+\t\tfi\n+\t\tprintf \"%s %s\\t%s\\n\" \\\n+\t\t\t\"$mode\" \\\n+\t\t\t$(cat \"$workdir/../blob-cache/$sha1\") \\\n+\t\t\t\"$path\"\n+\tdone\n+}\n+\n # When piped a commit, output a script to set the ident of either\n # \"author\" or \"committer\n \n@@ -105,6 +122,7 @@ tempdir=.git-rewrite\n filter_env=\n filter_tree=\n filter_index=\n+filter_blob=\n filter_parent=\n filter_msg=cat\n filter_commit='git commit-tree \"$@\"'\n@@ -150,6 +168,9 @@ do\n \t--index-filter)\n \t\tfilter_index=\"$OPTARG\"\n \t\t;;\n+\t--blob-filter)\n+\t\tfilter_blob=\"$OPTARG\"\n+\t\t;;\n \t--parent-filter)\n \t\tfilter_parent=\"$OPTARG\"\n \t\t;;\n@@ -227,6 +248,9 @@ ret=0\n # map old->new commit ids for rewriting parents\n mkdir ../map || die \"Could not create map/ directory\"\n \n+# cache rewritten blobs for blob filter\n+mkdir ../blob-cache || die \"Could not create blob-cache/ directory\"\n+\n case \"$filter_subdir\" in\n \"\")\n \tgit rev-list --reverse --topo-order --default HEAD \\\n@@ -295,6 +319,12 @@ while read commit parents; do\n \teval \"$filter_index\" < /dev/null ||\n \t\tdie \"index filter failed: $filter_index\"\n \n+\tif test -n \"$filter_blob\"; then\n+\t\tgit ls-files --stage |\n+\t\tmunge_blobs |\n+\t\tgit update-index --index-info\n+\tfi\n+\n \tparentstr=\n \tfor parent in $parents; do\n \t\tfor reparent in $(map \"$parent\"); do\n-- \n1.5.4.3\n"},{"id":"75104","messageId":"alpine.DEB.1.00.0804232122260.20267@eeepc-johanness","threadId":"13233","inReplyTo":"1208981890-758-1-git-send-email-apenwarr@gmail.com","subject":"Re: [PATCH 1/3 v2] Add --blob-filter option to filter-branch.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-04-23T20:22:58Z","receivedAt":"2008-04-23T20:22:58Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 23 Apr 2008, Avery Pennarun wrote:\n\n> From: Jeff King \n> \n> This patch allows\n> \n> git filter-branch --blob-filter 'tr a-z A-Z'\n> \n> However, it's _still_ horribly slow. Shell script is nice and flexible,\n> but running a tight loop like this is just painful. I suspect\n> filter-branch in something like perl would be a lot faster and just as\n> flexible.\n> \n> It is still much better performance than a tree filter, though:\n> \n> $ cd git && time git filter-branch --tree-filter '\n> find . -type f | while read f; do\n> tr a-z A-Z <\"$f\" >tmp\n> mv tmp \"$f\"\n> done\n> ' HEAD~10..HEAD\n> \n> real 4m38.626s\n> user 1m32.726s\n> sys 2m51.163s\n> \n> $ cd git && git filter-branch --blob-filter 'tr a-z A-Z' HEAD~10..HEAD\n> real 1m40.809s\n> user 0m36.822s\n> sys 1m14.273s\n> \n> Acked-by: Johannes Schindelin \n> ---\n\nThanks, I really appreciate it.\n\nCiao,\nDscho\n"},{"id":"75108","messageId":"20080423215501.GB30057@sigill.intra.peff.net","threadId":"13233","inReplyTo":"1208981890-758-1-git-send-email-apenwarr@gmail.com","subject":"Re: [PATCH 1/3 v2] Add --blob-filter option to filter-branch.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-04-23T21:55:01Z","receivedAt":"2008-04-23T21:55:01Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 23, 2008 at 04:18:10PM -0400, Avery Pennarun wrote:\n\n> From: Jeff King \n> \n> This patch allows\n> \n> git filter-branch --blob-filter 'tr a-z A-Z'\n\nThe commit message munging you did is fine.\n\nHowever, I think Johannes Sixt's question about providing the pathname\nneeds to be resolved. As it is now, the blob-filter is impossible to use\nin a mixed binary/text repository, short of the undocumented $path magic\nthat you described. And I am a little uncomfortable just adding the\n$path as he suggested because of the subtle bug it introduces.\n\n-Peff\n"},{"id":"75111","messageId":"32541b130804231507l4aff2ca3x962bf385a409685b@mail.gmail.com","threadId":"13233","inReplyTo":"20080423215501.GB30057@sigill.intra.peff.net","subject":"Re: [PATCH 1/3 v2] Add --blob-filter option to filter-branch.","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-23T22:07:03Z","receivedAt":"2008-04-23T22:07:03Z","isPatch":true,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/23/08, Jeff King wrote:\n> On Wed, Apr 23, 2008 at 04:18:10PM -0400, Avery Pennarun wrote:\n>\n> > From: Jeff King \n> >\n> > This patch allows\n> >\n> > git filter-branch --blob-filter 'tr a-z A-Z'\n>\n> The commit message munging you did is fine.\n>\n> However, I think Johannes Sixt's question about providing the pathname\n> needs to be resolved. As it is now, the blob-filter is impossible to use\n> in a mixed binary/text repository, short of the undocumented $path magic\n> that you described. And I am a little uncomfortable just adding the\n> $path as he suggested because of the subtle bug it introduces.\n\nIt is indeed a very subtle bug; so subtle, in fact, that I never\nexpect to experience it myself :)\n\nI think it would be fine to index into the cache using $path$sha1,\nwhich would seem to resolve this issue. The catch is that $path isn't\na very good cachefile name. I'd suggest doing an md5sum or something\non it, but that would result in an extra fork for every file, which\nbrings us back to our original level of slowness (or worse).\n\nHmm, I gues using a cachefile like $sha1/$path would work; it requires\na \"mkdir -p\", but only when *filling* the cache.\n\nAvery\n"},{"id":"75127","messageId":"20080424013259.GA31146@sigill.intra.peff.net","threadId":"13233","inReplyTo":"32541b130804231507l4aff2ca3x962bf385a409685b@mail.gmail.com","subject":"Re: [PATCH 1/3 v2] Add --blob-filter option to filter-branch.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-04-24T01:33:00Z","receivedAt":"2008-04-24T01:33:00Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 23, 2008 at 06:07:03PM -0400, Avery Pennarun wrote:\n\n> I think it would be fine to index into the cache using $path$sha1,\n> which would seem to resolve this issue. The catch is that $path isn't\n> a very good cachefile name. I'd suggest doing an md5sum or something\n> on it, but that would result in an extra fork for every file, which\n> brings us back to our original level of slowness (or worse).\n> \n> Hmm, I gues using a cachefile like $sha1/$path would work; it requires\n> a \"mkdir -p\", but only when *filling* the cache.\n\nKeep in mind that $path can have slashes. So you actually need to:\n\n mkdir -p `dirname $sha1/$path`\n echo $new >$sha1/$path\n\n-Peff\n"}]}