{"thread":{"id":"38072","subject":"git status / git diff -C not detecting file copy","startedAt":"2014-11-30T00:35:51Z","lastAt":"2014-12-03T16:03:18Z","messageCount":12,"participants":["Pol Online","Bryan Turner","Jeff King","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"252677","messageId":"CAJxwDJzzNV77cTP4nbzgCvFjjqp3C4X8d3j6uwhYvK4+g4r1YQ@mail.gmail.com","threadId":"38072","inReplyTo":null,"subject":"git status / git diff -C not detecting file copy","fromName":"Pol Online","fromEmail":"info@pol-online.net","sentAt":"2014-11-30T00:35:51Z","receivedAt":"2014-11-30T00:35:51Z","isPatch":false,"sender":{"key":"info@pol-online.net","avatar":null},"body":"Hi,\n\nThe documentation for git status at http://git-scm.com/docs/git-status\nimplies that it should be able to detect both renames and copies (with\nthe R and C states). The command git diff -C should do it as well.\n\nHowever I can't get either to detect copies in this simple test case -\nwhat is happening?\n\n\nmkdir test\ncd test/\ngit init\necho 'Hello World!' > hello.txt\necho 'Goodbye World!' > goodbye.txt\ngit add -A\ngit commit -m \"Initial commit\"\n\ncp hello.txt copied.txt\nmv goodbye.txt moved.txt\ngit add -A\n\n$ git status --short\nA copied.txt <------------ NO COPY DETECTED\nR goodbye.txt -> moved.txt\n\n$ git diff -M -C --summary --cached\n create mode 100644 copied.txt <------------ NO COPY DETECTED\n rename goodbye.txt => moved.txt (100%)\n\n$ git commit -m Test\n$ git diff -M -C --summary HEAD~\n create mode 100644 copied.txt <------------ NO COPY DETECTED\n rename goodbye.txt => moved.txt (100%)\n\n\n-Pol\n"},{"id":"252678","messageId":"CAGyf7-E_y8zRUKh5RWvAhPXzSgpnVab6e=e1v92rSVVxf+LNJg@mail.gmail.com","threadId":"38072","inReplyTo":"CAJxwDJzzNV77cTP4nbzgCvFjjqp3C4X8d3j6uwhYvK4+g4r1YQ@mail.gmail.com","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Bryan Turner","fromEmail":"bturner@atlassian.com","sentAt":"2014-11-30T01:03:35Z","receivedAt":"2014-11-30T01:03:35Z","isPatch":false,"sender":{"key":"bturner@atlassian.com","avatar":"https://gravatar.com/avatar/16bcf3167981c1ef7c804e502642366d888a35b0d0b0a4ca01fdc442aa1acb1e?d=mp&s=160"},"body":"Pol,\n\nBy default, -C only finds copies when the source file was also\nmodified in the same commit. Since you did not modify hello.txt in the\nsame commit where you copied it to copied.txt, it will not be\nconsidered.\n\nIf you pass -C -C (twice), or use --find-copies-harder, Git will\nconsider all files in the repository. Note that this can be slower,\nwhich is the reason why it's not the default.\n\nThe documentation for git diff describes the -C (--find-copies) and\n--find-copies-harder flags and their limitations.\n\nHope this helps,\nBryan Turner\n\nOn Sun, Nov 30, 2014 at 11:35 AM, Pol Online wrote:\n> Hi,\n>\n> The documentation for git status at http://git-scm.com/docs/git-status\n> implies that it should be able to detect both renames and copies (with\n> the R and C states). The command git diff -C should do it as well.\n>\n> However I can't get either to detect copies in this simple test case -\n> what is happening?\n>\n>\n> mkdir test\n> cd test/\n> git init\n> echo 'Hello World!' > hello.txt\n> echo 'Goodbye World!' > goodbye.txt\n> git add -A\n> git commit -m \"Initial commit\"\n>\n> cp hello.txt copied.txt\n> mv goodbye.txt moved.txt\n> git add -A\n>\n> $ git status --short\n> A copied.txt <------------ NO COPY DETECTED\n> R goodbye.txt -> moved.txt\n>\n> $ git diff -M -C --summary --cached\n> create mode 100644 copied.txt <------------ NO COPY DETECTED\n> rename goodbye.txt => moved.txt (100%)\n>\n> $ git commit -m Test\n> $ git diff -M -C --summary HEAD~\n> create mode 100644 copied.txt <------------ NO COPY DETECTED\n> rename goodbye.txt => moved.txt (100%)\n>\n>\n> -Pol\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"252679","messageId":"CAJxwDJzxUEd3czHpwDtKaERKDhvyCGOzGbKO4X9z44ugTJ2q4w@mail.gmail.com","threadId":"38072","inReplyTo":"CAGyf7-E_y8zRUKh5RWvAhPXzSgpnVab6e=e1v92rSVVxf+LNJg@mail.gmail.com","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Pol Online","fromEmail":"info@pol-online.net","sentAt":"2014-11-30T01:30:10Z","receivedAt":"2014-11-30T01:30:10Z","isPatch":false,"sender":{"key":"info@pol-online.net","avatar":null},"body":"Hi Bryan,\n\nOK that explains the behavior of git diff, but what about git status?\nThe doc implies it should be able to detect copies in the index /\nstaging area since it has a \"C\" state.\n\n- Pol\n\nOn Sun, Nov 30, 2014 at 10:03 AM, Bryan Turner wrote:\n> Pol,\n>\n> By default, -C only finds copies when the source file was also\n> modified in the same commit. Since you did not modify hello.txt in the\n> same commit where you copied it to copied.txt, it will not be\n> considered.\n>\n> If you pass -C -C (twice), or use --find-copies-harder, Git will\n> consider all files in the repository. Note that this can be slower,\n> which is the reason why it's not the default.\n>\n> The documentation for git diff describes the -C (--find-copies) and\n> --find-copies-harder flags and their limitations.\n>\n> Hope this helps,\n> Bryan Turner\n>\n> On Sun, Nov 30, 2014 at 11:35 AM, Pol Online wrote:\n>> Hi,\n>>\n>> The documentation for git status at http://git-scm.com/docs/git-status\n>> implies that it should be able to detect both renames and copies (with\n>> the R and C states). The command git diff -C should do it as well.\n>>\n>> However I can't get either to detect copies in this simple test case -\n>> what is happening?\n>>\n>>\n>> mkdir test\n>> cd test/\n>> git init\n>> echo 'Hello World!' > hello.txt\n>> echo 'Goodbye World!' > goodbye.txt\n>> git add -A\n>> git commit -m \"Initial commit\"\n>>\n>> cp hello.txt copied.txt\n>> mv goodbye.txt moved.txt\n>> git add -A\n>>\n>> $ git status --short\n>> A copied.txt <------------ NO COPY DETECTED\n>> R goodbye.txt -> moved.txt\n>>\n>> $ git diff -M -C --summary --cached\n>> create mode 100644 copied.txt <------------ NO COPY DETECTED\n>> rename goodbye.txt => moved.txt (100%)\n>>\n>> $ git commit -m Test\n>> $ git diff -M -C --summary HEAD~\n>> create mode 100644 copied.txt <------------ NO COPY DETECTED\n>> rename goodbye.txt => moved.txt (100%)\n>>\n>>\n>> -Pol\n>> --\n>> To unsubscribe from this list: send the line \"unsubscribe git\" in\n>> the body of a message to majordomo@vger.kernel.org\n>> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"252680","messageId":"CAGyf7-F9twCEUY-LN=xEf4=gfNW8oLEHJmTjHRQ2MncHZ2emZQ@mail.gmail.com","threadId":"38072","inReplyTo":"CAJxwDJzxUEd3czHpwDtKaERKDhvyCGOzGbKO4X9z44ugTJ2q4w@mail.gmail.com","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Bryan Turner","fromEmail":"bturner@atlassian.com","sentAt":"2014-11-30T01:54:53Z","receivedAt":"2014-11-30T01:54:53Z","isPatch":false,"sender":{"key":"bturner@atlassian.com","avatar":"https://gravatar.com/avatar/16bcf3167981c1ef7c804e502642366d888a35b0d0b0a4ca01fdc442aa1acb1e?d=mp&s=160"},"body":"Pol,\n\nIt's the same set of limitations. Git does not track renames or copies\nas such. It uses heuristics to compute a similarity index and then\ndecide. All the tree stores for your copy is a file add, and that's\nthe status you're seeing. I don't think there is any way to turn on\naggressive copy detection for git status. However, before you run git\ncommit, you could run git diff --find-copies-harder --cached instead\nand it should show the copy.\n\nI'll let someone a little more intimately familiar with the internals\nof git status comment on why the documentation for that mentions\ncopies.\n\nHope this helps,\nBryan Turner\n\nOn Sun, Nov 30, 2014 at 12:30 PM, Pol Online wrote:\n> Hi Bryan,\n>\n> OK that explains the behavior of git diff, but what about git status?\n> The doc implies it should be able to detect copies in the index /\n> staging area since it has a \"C\" state.\n>\n> - Pol\n>\n> On Sun, Nov 30, 2014 at 10:03 AM, Bryan Turner wrote:\n>> Pol,\n>>\n>> By default, -C only finds copies when the source file was also\n>> modified in the same commit. Since you did not modify hello.txt in the\n>> same commit where you copied it to copied.txt, it will not be\n>> considered.\n>>\n>> If you pass -C -C (twice), or use --find-copies-harder, Git will\n>> consider all files in the repository. Note that this can be slower,\n>> which is the reason why it's not the default.\n>>\n>> The documentation for git diff describes the -C (--find-copies) and\n>> --find-copies-harder flags and their limitations.\n>>\n>> Hope this helps,\n>> Bryan Turner\n>>\n>> On Sun, Nov 30, 2014 at 11:35 AM, Pol Online wrote:\n>>> Hi,\n>>>\n>>> The documentation for git status at http://git-scm.com/docs/git-status\n>>> implies that it should be able to detect both renames and copies (with\n>>> the R and C states). The command git diff -C should do it as well.\n>>>\n>>> However I can't get either to detect copies in this simple test case -\n>>> what is happening?\n>>>\n>>>\n>>> mkdir test\n>>> cd test/\n>>> git init\n>>> echo 'Hello World!' > hello.txt\n>>> echo 'Goodbye World!' > goodbye.txt\n>>> git add -A\n>>> git commit -m \"Initial commit\"\n>>>\n>>> cp hello.txt copied.txt\n>>> mv goodbye.txt moved.txt\n>>> git add -A\n>>>\n>>> $ git status --short\n>>> A copied.txt <------------ NO COPY DETECTED\n>>> R goodbye.txt -> moved.txt\n>>>\n>>> $ git diff -M -C --summary --cached\n>>> create mode 100644 copied.txt <------------ NO COPY DETECTED\n>>> rename goodbye.txt => moved.txt (100%)\n>>>\n>>> $ git commit -m Test\n>>> $ git diff -M -C --summary HEAD~\n>>> create mode 100644 copied.txt <------------ NO COPY DETECTED\n>>> rename goodbye.txt => moved.txt (100%)\n>>>\n>>>\n>>> -Pol\n>>> --\n>>> To unsubscribe from this list: send the line \"unsubscribe git\" in\n>>> the body of a message to majordomo@vger.kernel.org\n>>> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"252838","messageId":"20141202065550.GB1948@peff.net","threadId":"38072","inReplyTo":"CAGyf7-F9twCEUY-LN=xEf4=gfNW8oLEHJmTjHRQ2MncHZ2emZQ@mail.gmail.com","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-12-02T06:55:50Z","receivedAt":"2014-12-02T06:55:50Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Nov 30, 2014 at 12:54:53PM +1100, Bryan Turner wrote:\n\n> I'll let someone a little more intimately familiar with the internals\n> of git status comment on why the documentation for that mentions\n> copies.\n\nI don't think there is a good reason. git-status has used renames since\nmid-2005. The documentation mentioning copies was added much later,\nalong with the short and porcelain formats. That code handles whatever\nthe diff engine throws at it. I don't think anybody considered at that\ntime the fact that you cannot actually provoke status to look for\ncopies.\n\nInterestingly, the rename behavior dates all the way back to:\n\n commit 753fd78458b6d7d0e65ce0ebe7b62e1bc55f3992\n Author: Linus Torvalds \n Date: Fri Jun 17 15:34:19 2005 -0700\n\n Use \"-M\" instead of \"-C\" for \"git diff\" and \"git status\"\n \n The \"C\" in \"-C\" may stand for \"Cool\", but it's also pretty slow, since\n right now it leaves all unmodified files to be tested even if there are\n no new files at all. That just ends up being unacceptably slow for big\n projects, especially if it's not all in the cache.\n\nI suspect that the copy code may be much faster these days (it sounds\nlike we did not even have the find-copies-harder distinction then, and\nthese days we certainly take the quick return if there are no copy\ndestination candidates).\n\nTo get a rough sense of how much effort is entailed in the various\noptions, here are \"git log --raw\" timings for git.git (all timings are\nwarm cache, best-of-five, wall clock time):\n\n log --raw: 0m2.311s\n log --raw -M: 0m2.362s\n log --raw -C: 0m2.576s\n log --raw -C -C: 1m4.462s\n\nYou can see that rename detection adds a little, and copy detections\nadds a little more. That makes sense; it's rare for new files to appear\nat the same that old files are going away (renames), so most of the time\nit does nothing. Copies introduce a bit more work; we have to compare\nagainst any changed files, and there are typically several in each\ncommit. find-copies-harder is...well, very expensive.\n\nThese timings are of diffs between commits and their parents, of course.\nBut if we assume that \"git status\" will show diffs roughly similar to\nwhat gets committed, then this should be comparable. There are about 30K\nnon-merge commits we traversed there, so adding 200ms is an average of\nnot very much per commit. Of course the cost is disproportionately borne\nby diffs which have an actual file come into being. There are ~2000\ncommits that introduce a file, so it's probably accurate to say that it\neither adds nothing in most cases, or ~1/10th of a millisecond in\nothers.\n\nNote this is also doing inexact detection, which involves actually\nlooking at the contents of candidate blobs (whereas exact detection can\nbe done by comparing sha1s, which is very fast). If you set\ndiff.renamelimit to \"1\", then we do only exact detections. Here are\ntimings there:\n\n log --raw: 0m02.311s (for reference)\n log --raw -M: 0m02.337s\n log --raw -C: 0m02.347s\n log --raw -C -C: 0m24.419s\n\nThat speeds things up a fair bit, even for \"-C\" (we don't have to access\nthe blobs anymore, so I suspect the time is going to just accessing all\nof the trees; normally diff does not descend into subtrees that have the\nsame sha1). Of course, you probably wouldn't want to turn off inexact\nrenames completely. I suspect what you'd want is a --find-copies-moderately\nwhere we look for cheap copies using \"-C\", and then follow up with \"-C\n-C\" only using exact renames.\n\nSo from these timings, I'd conclude that:\n\n 1. It's probably fine to turn on copies for \"git status\".\n\n 2. It's probably even OK to use \"-C -C\" for some projects. Even though\n 22s looks scary there, that's only 11ms for git.git (remember,\n spread across 2000 commits). For linux.git, it's much, much worse.\n I killed my \"-C -C\" run after 10 minutes, and it had only gone\n through 1/20th of the commits. Extrapolating, you're looking at\n 500ms or so added to a \"git status\" run.\n\n So you'd almost certainly want this to be configurable.\n\nDoes either of you want to try your hand at a patch? Just enabling\ncopies should be a one-liner. Making it configurable is more involved,\nbut should also be pretty straightforward.\n\n-Peff\n"},{"id":"252866","messageId":"CAJxwDJxyOgH=NLAbPU8LvK5Wo5XR1t76kL3dPbwv_g5A-==gvQ@mail.gmail.com","threadId":"38072","inReplyTo":"20141202065550.GB1948@peff.net","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Pol Online","fromEmail":"info@pol-online.net","sentAt":"2014-12-02T14:15:10Z","receivedAt":"2014-12-02T14:15:10Z","isPatch":false,"sender":{"key":"info@pol-online.net","avatar":null},"body":"Jeff,\n\nThanks much for the detailed answer and analysis.\n\n> Does either of you want to try your hand at a patch? Just enabling\n> copies should be a one-liner. Making it configurable is more involved,\n> but should also be pretty straightforward.\n\nI'll have to pass on this. I'm absolutely not familiar with the Git\nsource code nor with the contributing guidelines and process at this\ntime. I'm currently working on libgit2 and trying to clone the git\nstatus behavior and that's how I discovered this file copy detection\nissue.\n\nIn the meantime, shouldn't we patch the docs? That should be trivial for sure.\n\n- Pol\n"},{"id":"252883","messageId":"xmqqvblukk98.fsf@gitster.dls.corp.google.com","threadId":"38072","inReplyTo":"20141202065550.GB1948@peff.net","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-12-02T17:57:07Z","receivedAt":"2014-12-02T17:57:07Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> Interestingly, the rename behavior dates all the way back to:\n>\n> commit 753fd78458b6d7d0e65ce0ebe7b62e1bc55f3992\n> Author: Linus Torvalds \n> Date: Fri Jun 17 15:34:19 2005 -0700\n>\n> Use \"-M\" instead of \"-C\" for \"git diff\" and \"git status\"\n> \n> The \"C\" in \"-C\" may stand for \"Cool\", but it's also pretty slow, since\n> right now it leaves all unmodified files to be tested even if there are\n> no new files at all. That just ends up being unacceptably slow for big\n> projects, especially if it's not all in the cache.\n> ...\n> To get a rough sense of how much effort is entailed in the various\n> options, here are \"git log --raw\" timings for git.git (all timings are\n> warm cache, best-of-five, wall clock time):\n\nThe rationale of the change talks about \"big projects\" and your\nanalysis and argument to advocate reversion of that change is based\non \"git.git\"? What is going on here?\n\nAlso our history is fairly unusual in that we do not have a lot of\nrenames (there was one large \"s|builtin-|builtin/|\" rename event,\nbut that is about it), which may not be a good example to base such\na design decision on.\n\nIt is a fine idea to make this configurable (\"status.renames = -C\"\nor something, perhaps?), though.\n"},{"id":"252887","messageId":"20141202200910.GB23461@peff.net","threadId":"38072","inReplyTo":"xmqqvblukk98.fsf@gitster.dls.corp.google.com","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-12-02T20:09:11Z","receivedAt":"2014-12-02T20:09:11Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Dec 02, 2014 at 09:57:07AM -0800, Junio C Hamano wrote:\n\n> > To get a rough sense of how much effort is entailed in the various\n> > options, here are \"git log --raw\" timings for git.git (all timings are\n> > warm cache, best-of-five, wall clock time):\n> \n> The rationale of the change talks about \"big projects\" and your\n> analysis and argument to advocate reversion of that change is based\n> on \"git.git\"? What is going on here?\n\nI find that git.git is often a useful and easy thing to time to\nextrapolate to other projects. It's 1/10th-1/20th the size of the kernel\n(both in tree size and commit depth), which I do consider a \"big\nproject\" (and I have a feeling is what Linus was talking about).\n\nOf course, performance numbers do not always scale linearly with repo\nsize. I didn't show the full numbers for the kernel, but they are:\n\n log --raw: 0m53.587s\n log --raw -M: 0m55.424s\n log --raw -C: 1m02.733s\n log --raw -C -C: \n\nThere are ~20K commits that introduce files in the kernel (about 10x\nwhat git.git had). So renames add well under a millisecond to each of\nthose diffs, and a single \"-C\" adds a third of a millisecond.\n\nWhich is pretty in-line with the git.git findings (it is not linear\nhere, but actually fairly constant. This makes sense, as it scales with\nthe size of the commit, not the size of the tree).\n\nAnd as I noted, \"-C -C\" is rather expensive (I gave some estimated\ntimings earlier; you could probably come up with something more accurate\nby doing smarter sampling).\n\n> Also our history is fairly unusual in that we do not have a lot of\n> renames (there was one large \"s|builtin-|builtin/|\" rename event,\n> but that is about it), which may not be a good example to base such\n> a design decision on.\n\nI think the work scales not with the number of actual renames, but with\nthe number of commits where we even bother to look at renames at all\n(i.e., ones with an 'A' diff-status). And my estimates assume that we\npay zero cost for other diffs, and attribute all of the extra time to\nthose diffs. So I think frequency of rename (or 'A') events does not\nimpact the estimate of the impact on a single \"git status\" run.\n\nWhat does impact it is the _size_ of each commit. If you quite\nfrequently add a new file while touching tens of thousands of other\nfiles, then the performance will be a lot more noticeable. And both\ngit.git and linux.git are probably better than some other projects about\nhaving small commits.\n\nStill, though. I stand by my earlier conclusions. Even with commits ten\ntimes as large as the kernel's, you are talking about 3ms added to a\n\"status\" run (and again, only when you hit such a gigantic commit _and_\nit has an 'A' file).\n\n> It is a fine idea to make this configurable (\"status.renames = -C\"\n> or something, perhaps?), though.\n\nI think it would be OK to move to \"-C\" as a default, but I agree it is\nnicer if it is configurable, as it gives a safety hatch for people in\npathological repos to drop back to the old behavior (or even turn off\nrenames altogether).\n\n-Peff\n"},{"id":"252893","messageId":"CAGyf7-EHBqZn5LCTYuA+07GSNOF0vVdszL6oTUKwY1ETRDKEwA@mail.gmail.com","threadId":"38072","inReplyTo":"20141202065550.GB1948@peff.net","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Bryan Turner","fromEmail":"bturner@atlassian.com","sentAt":"2014-12-02T21:40:47Z","receivedAt":"2014-12-02T21:40:47Z","isPatch":false,"sender":{"key":"bturner@atlassian.com","avatar":"https://gravatar.com/avatar/16bcf3167981c1ef7c804e502642366d888a35b0d0b0a4ca01fdc442aa1acb1e?d=mp&s=160"},"body":"On Tue, Dec 2, 2014 at 5:55 PM, Jeff King wrote:\n>\n> So from these timings, I'd conclude that:\n>\n> 1. It's probably fine to turn on copies for \"git status\".\n>\n> 2. It's probably even OK to use \"-C -C\" for some projects. Even though\n> 22s looks scary there, that's only 11ms for git.git (remember,\n> spread across 2000 commits). For linux.git, it's much, much worse.\n> I killed my \"-C -C\" run after 10 minutes, and it had only gone\n> through 1/20th of the commits. Extrapolating, you're looking at\n> 500ms or so added to a \"git status\" run.\n>\n> So you'd almost certainly want this to be configurable.\n>\n> Does either of you want to try your hand at a patch? Just enabling\n> copies should be a one-liner. Making it configurable is more involved,\n> but should also be pretty straightforward.\n\nI'm interested in taking a stab at a patch, but I'd like to confirm\nwhich way to go. Based on Junio's reply I'm not certain the simple\npatch could get accepted (assuming I do all the submission parts\nproperly and the implementation itself passes review). Does that mean\nthe only real option is the configurable patch?\n\n>\n> -Peff\n"},{"id":"252894","messageId":"20141202215013.GB25338@peff.net","threadId":"38072","inReplyTo":"CAGyf7-EHBqZn5LCTYuA+07GSNOF0vVdszL6oTUKwY1ETRDKEwA@mail.gmail.com","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-12-02T21:50:13Z","receivedAt":"2014-12-02T21:50:13Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Dec 03, 2014 at 08:40:47AM +1100, Bryan Turner wrote:\n\n> On Tue, Dec 2, 2014 at 5:55 PM, Jeff King wrote:\n> >\n> > So from these timings, I'd conclude that:\n> >\n> > 1. It's probably fine to turn on copies for \"git status\".\n> >\n> > 2. It's probably even OK to use \"-C -C\" for some projects. Even though\n> > 22s looks scary there, that's only 11ms for git.git (remember,\n> > spread across 2000 commits). For linux.git, it's much, much worse.\n> > I killed my \"-C -C\" run after 10 minutes, and it had only gone\n> > through 1/20th of the commits. Extrapolating, you're looking at\n> > 500ms or so added to a \"git status\" run.\n> >\n> > So you'd almost certainly want this to be configurable.\n> >\n> > Does either of you want to try your hand at a patch? Just enabling\n> > copies should be a one-liner. Making it configurable is more involved,\n> > but should also be pretty straightforward.\n> \n> I'm interested in taking a stab at a patch, but I'd like to confirm\n> which way to go. Based on Junio's reply I'm not certain the simple\n> patch could get accepted (assuming I do all the submission parts\n> properly and the implementation itself passes review). Does that mean\n> the only real option is the configurable patch?\n\nI think this is the part where you get to use your judgement, and decide\nwhat you think the maintainer will take. :)\n\nPersonally, I would probably go for the configurable version, as it is\nnot that much harder, and is a nicer end-point.\n\nJunio gave an example elsewhere in which the config option value would\nlook like \"-C -C\". I'd consider trying to match diff.renames instead,\nwhich takes false/true/copies for its three levels. It may make sense to\nteach both places \"copies-harder\" or something similar, for\ncompleteness.\n\n-Peff\n"},{"id":"252949","messageId":"xmqqa934lo2p.fsf@gitster.dls.corp.google.com","threadId":"38072","inReplyTo":"20141202200910.GB23461@peff.net","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-12-03T16:01:34Z","receivedAt":"2014-12-03T16:01:34Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> I find that git.git is often a useful and easy thing to time to\n> extrapolate to other projects. It's 1/10th-1/20th the size of the kernel\n> (both in tree size and commit depth), which I do consider a \"big\n> project\" (and I have a feeling is what Linus was talking about).\n>\n> Of course, performance numbers do not always scale linearly with repo\n> size.\n> ...\n> What does impact it is the _size_ of each commit. If you quite\n> frequently add a new file while touching tens of thousands of other\n> files, then the performance will be a lot more noticeable. And both\n> git.git and linux.git are probably better than some other projects about\n> having small commits.\n\nYes, the number of paths touched per-commit in git.git may not be\ntypical. If it is unusually lower, that would skew the result, as\nthe cost of rename detection is proportional to it, and the cost of\n-C -C is that number times the number of total paths in the project.\n\n> Still, though. I stand by my earlier conclusions. Even with commits ten\n> times as large as the kernel's, you are talking about 3ms added to a\n> \"status\" run (and again, only when you hit such a gigantic commit _and_\n> it has an 'A' file).\n>\n>> It is a fine idea to make this configurable (\"status.renames = -C\"\n>> or something, perhaps?), though.\n>\n> I think it would be OK to move to \"-C\" as a default, but I agree it is\n> nicer if it is configurable, as it gives a safety hatch for people in\n> pathological repos to drop back to the old behavior (or even turn off\n> renames altogether).\n\nYeah I am OK with that, too.\n"},{"id":"252950","messageId":"xmqq61dslnzt.fsf@gitster.dls.corp.google.com","threadId":"38072","inReplyTo":"20141202215013.GB25338@peff.net","subject":"Re: git status / git diff -C not detecting file copy","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-12-03T16:03:18Z","receivedAt":"2014-12-03T16:03:18Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> ... I'd consider trying to match diff.renames instead,\n> which takes false/true/copies for its three levels. It may make sense to\n> teach both places \"copies-harder\" or something similar, for\n> completeness.\n\nYeah, I think that is a very sensible thing to do.\n"}]}