{"thread":{"id":"3728","subject":"Following renames","startedAt":"2006-03-26T01:49:47Z","lastAt":"2006-03-27T21:59:56Z","messageCount":41,"participants":["Petr Baudis","Junio C Hamano","Linus Torvalds","Jakub Narebski","Ryan Anderson","Fredrik Kuivinen","Timo Hirvonen","Marco Costalba","Paul Jakma","David Lang","Johannes Schindelin","Andreas Ericsson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"17969","messageId":"20060326014946.GB18185@pasky.or.cz","threadId":"3728","inReplyTo":null,"subject":"Following renames","fromName":"Petr Baudis","fromEmail":"pasky@ucw.cz","sentAt":"2006-03-26T01:49:47Z","receivedAt":"2006-03-26T01:49:47Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":" Hi,\n\n so, now that I've put up with the fuzzy rename autodetection (for\nnow), I'd like to make cg-log auto-follow renames and I'm wondering\nabout the best implementation (it seems that I won't do without core\nGit cooperation). I think it should be possible to implement in a way\nso that it has minimal performance impact and therefore I can have it\nturned on by default.\n\n Now I'm using the notorious\n\n\tgit-rev-list listoffiles | git-diff-tree --stdin\n\npipeline in cg-log, and I'm wondering about the best way to add rename\ndetection there.\n\n In [1], Linus suggests a non-core solution. Unfortunately, it doesn't\nfly - it requires at least two git-ls-tree calls per revision which\nwould bog things down awfully (to roughly half of the original speed).\n\n But even if git-rev-list reported disappearing files, Cogito would\nhave to do a lot of complicated bookkeeping in order to properly track\nrenames in parallel branches - for each 'head' commit at any point of\nthe history traversal, you need to record a separate set of interesting\nfiles. It would also have to restart git-rev-list at any moment when a\nrename happens on any of the head commits. Scales well not.\n\n An obvious solution would be to have git-diff-tree --follow which\nupdates its interesting path set based on seen renames, and now that\nI've written about non-linear history, it's obvious that it's incorrect.\nThe other obvious way to go is then to add rename detection support to\ngit-rev-list, and it's less obvious that this is a dead end too - I\ndidn't inspect the code myself yet, but for now I trust Linus in [2]\n(I didn't quite understand the argument, I guess I need to sleep on it).\n\n So, any thoughts about how to approach this? Either git-diff-tree\nwould have to be taught about the heads bookkeeping, or the git-rev-list\nhurdles would have to be overcome, or we might have a\ngit-rev-rename-filter or something, but that feels quite redundant and\nmight meet with the same problems as git-rev-list.\n\n == References ==\n\n [1] Oct 21 \n [2] Oct 22 \n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"},{"id":"17972","messageId":"7virq1sywj.fsf@assigned-by-dhcp.cox.net","threadId":"3728","inReplyTo":"20060326014946.GB18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-03-26T02:49:48Z","receivedAt":"2006-03-26T02:49:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Petr Baudis writes:\n\n> An obvious solution would be to have git-diff-tree --follow which\n> updates its interesting path set based on seen renames, and now that\n> I've written about non-linear history, it's obvious that it's incorrect.\n> The other obvious way to go is then to add rename detection support to\n> git-rev-list, and it's less obvious that this is a dead end too - I\n> didn't inspect the code myself yet, but for now I trust Linus in [2]\n> (I didn't quite understand the argument, I guess I need to sleep on it).\n\nI'd have to sleep on how the core side can help Porcelains, but\nI think it is a good thing that you, one of the most vocal\nadvocate on the list for doing rename recording, are thinking\nabout this issue and probably would look into rev-list.c soon.\n\nLooking at the evolution of rev-list.c file itself was a good\nexercise to realize that rename tracking (more specifically,\nhaving whatchanged to follow renames) is not such a useful\nthing (at least for me).\n\nIf I am interested in rev-list.c's evolution from \"the set of\ncommand line flags it supported\" point of view, then whatchanged\nto show the history of rev-list.c file itself would be a very\ngood way to show that to me. rev-list_usage[] = \"...\" stayed\nthere almost from the beginning.\n\nHowever, if I am interested in the way how it traverses the\ncommits has changed over time, I would need to start from\nrevision.c and switch to rev-list.c when that part of the code\nwas split out from it, because the current rev-list.c does not\nhave the main part of the traversal logic at all.\n\nAnother example. Today's tar-tree updates have one interesting\nfunction I think should belong to strbuf.c, and before merging\nit to the mainline, I may move that function from tar-tree.c to\nstrbuf.c. After that happens, if I run \"whatchanged strbuf.c\"\nto see where that function came from, I would want it to notice\nit came from tar-tree.c, although it is not a rename at all.\nJust one function moved from a file to another.\n\nWhat this suggests is that switching the set of paths to follow\nwhile traversing ancestry chain needs to depend on which part of\nthe original file you are interested in. Marking \"this commit\nrenames (or copies) file A to file B\" is not that useful -- for\nthat matter, detecting at runtime like we currently do is not\nbetter either. If a file A and file B were cleaned up and\nmerged into a single file C, which is in the tip of the tree,\nwhich one you would want whatchanged to switch following depends\non which part of the C you were interested in.\n\nUnless you are interested in the _entire_ contents of the file,\nthat is. Then tracking or even recording renames becomes\nuseful, but that is a special case.\n\nThat is the reason I am not so enthused about recording renames.\nI think the time is better spent on enhancing what pickaxe tries\nto do (currently it does very little), which I hinted in a\nseparate message late last night.\n\nBut that does not have to stop you, and does not have to stop me\nfrom thinking about ways to help you either.\n"},{"id":"17975","messageId":"Pine.LNX.4.64.0603251919170.15714@g5.osdl.org","threadId":"3728","inReplyTo":"20060326014946.GB18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-26T03:19:50Z","receivedAt":"2006-03-26T03:19:50Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Mar 2006, Petr Baudis wrote:\n> \n> In [1], Linus suggests a non-core solution. Unfortunately, it doesn't\n> fly - it requires at least two git-ls-tree calls per revision which\n> would bog things down awfully (to roughly half of the original speed).\n\nNo it doesn't. It requires one git-ls-tree WHEN SOMETHING IS RENAMED.\n\nIn other words, basically never.\n\n\t\tLinus\n"},{"id":"17976","messageId":"e05354$cc9$1@sea.gmane.org","threadId":"3728","inReplyTo":"7virq1sywj.fsf@assigned-by-dhcp.cox.net","subject":"Re: Following renames","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-03-26T03:52:17Z","receivedAt":"2006-03-26T03:52:17Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Junio C Hamano wrote:\n\n> Petr Baudis writes:\n> \n>> An obvious solution would be to have git-diff-tree --follow which\n>> updates its interesting path set based on seen renames, and now that\n>> I've written about non-linear history, it's obvious that it's incorrect.\n>> The other obvious way to go is then to add rename detection support to\n>> git-rev-list, and it's less obvious that this is a dead end too - I\n>> didn't inspect the code myself yet, but for now I trust Linus in [2]\n>> (I didn't quite understand the argument, I guess I need to sleep on it).\n> \n> I'd have to sleep on how the core side can help Porcelains, but\n> I think it is a good thing that you, one of the most vocal\n> advocate on the list for doing rename recording, are thinking\n> about this issue and probably would look into rev-list.c soon.\n> \n> Looking at the evolution of rev-list.c file itself was a good\n> exercise to realize that rename tracking (more specifically,\n> having whatchanged to follow renames) is not such a useful\n> thing (at least for me).\n[...]\n> What this suggests is that switching the set of paths to follow\n> while traversing ancestry chain needs to depend on which part of\n> the original file you are interested in. Marking \"this commit\n> renames (or copies) file A to file B\" is not that useful -- for\n> that matter, detecting at runtime like we currently do is not\n> better either. If a file A and file B were cleaned up and\n> merged into a single file C, which is in the tip of the tree,\n> which one you would want whatchanged to switch following depends\n> on which part of the C you were interested in.\n> \n> Unless you are interested in the _entire_ contents of the file,\n> that is. Then tracking or even recording renames becomes\n> useful, but that is a special case.\n> \n> That is the reason I am not so enthused about recording renames.\n> I think the time is better spent on enhancing what pickaxe tries\n> to do (currently it does very little), which I hinted in a\n> separate message late last night.\n\nI think one of the better ideas/suggestions about *recording* filenames was\nin the \"impure renames / history tracking\" thread\n http://marc.theaimsgroup.com/?l=git&m=114122175216489&w=2\n \nabout adding *auxiliary* (helper) information about renames in commits. I'm\nnot sure about recording parts of the file that were moved or copied. That\nmight have been left for runtime detection in the likes of pickaxe.\n\nAs it would be helper-only information it would ensure backwards\ncompatibility (older versions would ignore additional information) and\nforward compatibility (newer version would fallback to current runtime\nrenames tracking/detection).\n\nTo be generic, I think that the command to record rename/copy or\ncopy'n'paste/cut'n'paste would take set of source files (one or more,\nunless we want to have an option to mark the file as new supressing any\nsuperficial similarities, in which case it would be zero or more), and set\nof destination files (one or more, with files which were in source repeated\nit was copy, not repeated if it was rename or cut'n'paste; unless we want\nto record deletions also, in which case it would be zero or more files).\nSuch information can be I guess easily entered by user... if one remembers\nto record rename/cut'n'paste/etc. that is. Perhaps if it were a way to easy\nadd such information later, for example confirming detected\nrenames/relationships during merge... It would be much more difficult for\nuser to enter the ranges unassisted.\n\nWhat worries me is that such information, recorded in \"own fields to the GIT\nrevision messages\" (in commits) can be used only if you track the ancestry;\nit doesn't help if you have only have two or more revisions and not build\nrelationship graph between them. But maybe I worry unnecessary...\n\nBTW. following renames is important not only in examining file [contents]\nhistory, in the likes of diff, whatchanged, annotate/blame, pickaxe but\nalso for merges.\n\nReferences:\n===========\n* http://marc.theaimsgroup.com/?l=linux-kernel&m=111314792424707\n* http://article.gmane.org/gmane.comp.version-control.git/217\n* http://marc.theaimsgroup.com/?l=git&m=114123702826251\n* http://marc.theaimsgroup.com/?l=git&m=114315795227271\n\n-- \nJakub Narebski\nWarsaw, Poland\n"},{"id":"17981","messageId":"44264426.8010608@michonline.com","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603251919170.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2006-03-26T07:35:02Z","receivedAt":"2006-03-26T07:35:02Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"Linus Torvalds wrote:\n> On Sun, 26 Mar 2006, Petr Baudis wrote:\n> \n>> In [1], Linus suggests a non-core solution. Unfortunately, it doesn't\n>> fly - it requires at least two git-ls-tree calls per revision which\n>> would bog things down awfully (to roughly half of the original speed).\n>> \n>\n> No it doesn't. It requires one git-ls-tree WHEN SOMETHING IS RENAMED.\n>\n> In other words, basically never.\n> \n\nA simple example is the first loop in git-annotate.perl. (Which was\nbasically written by Linus, I just translated it from a\nshell/pseudo-code example into Perl)\n\n\n-- \n\nRyan Anderson\n sometimes Pug Majere\n\n\n"},{"id":"17982","messageId":"20060326100717.GD18185@pasky.or.cz","threadId":"3728","inReplyTo":"44264426.8010608@michonline.com","subject":"Re: Following renames","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-03-26T10:07:17Z","receivedAt":"2006-03-26T10:07:17Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sun, Mar 26, 2006 at 05:19:50AM CEST, I got a letter\nwhere Linus Torvalds said that...\n> On Sun, 26 Mar 2006, Petr Baudis wrote:\n> > \n> > In [1], Linus suggests a non-core solution. Unfortunately, it doesn't\n> > fly - it requires at least two git-ls-tree calls per revision which\n> > would bog things down awfully (to roughly half of the original speed).\n> \n> No it doesn't. It requires one git-ls-tree WHEN SOMETHING IS RENAMED.\n> \n> In other words, basically never.\n\nHuh? I don't see that now (and caps don't help me see it better). That's\ncertainly not what is in [1], and I don't see _how_ to detect the\nrenames in this case, and what would I be actually doing git-ls-tree for\nwhen I've already detected the rename. Based on [1], I'd be doing\ngit-ls-tree merely to detect that a file _disappeared_ in the first\nplace, I have to do other stuff to detect the renames themselves.\n\nDear diary, on Sun, Mar 26, 2006 at 09:35:02AM CEST, I got a letter\nwhere Ryan Anderson said that...\n> A simple example is the first loop in git-annotate.perl. (Which was\n> basically written by Linus, I just translated it from a\n> shell/pseudo-code example into Perl)\n\nThanks for the hint. Unfortunately, this is precisely the thing I want\nto avoid, that is essentially reimplementing part of git-rev-list - to\ndo something good, I would have to do my own toposort and merge by date\nbetween parallel lines. OTOH, I might just construct a large revlist\ncommandline specifying all the segments I'm interested in and see what\nhappens when I run that.\n\nBesides, doing it in shell would be pretty ugly job (forcing me to\nfinally rewrite it in perl is not a bad thing but that'd be a somewhat\nlarger project since I share various common routines with other cg\ntools, etc).\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"},{"id":"17985","messageId":"20060326103411.GA4483@c165.ib.student.liu.se","threadId":"3728","inReplyTo":"20060326100717.GD18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Fredrik Kuivinen","fromEmail":"freku045@student.liu.se","sentAt":"2006-03-26T10:34:11Z","receivedAt":"2006-03-26T10:34:11Z","isPatch":false,"sender":{"key":"frekui@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13770967?v=4"},"body":"On Sun, Mar 26, 2006 at 12:07:17PM +0200, Petr Baudis wrote:\n> Dear diary, on Sun, Mar 26, 2006 at 05:19:50AM CEST, I got a letter\n> where Linus Torvalds said that...\n> > On Sun, 26 Mar 2006, Petr Baudis wrote:\n> > > \n> > > In [1], Linus suggests a non-core solution. Unfortunately, it doesn't\n> > > fly - it requires at least two git-ls-tree calls per revision which\n> > > would bog things down awfully (to roughly half of the original speed).\n> > \n> > No it doesn't. It requires one git-ls-tree WHEN SOMETHING IS RENAMED.\n> > \n> > In other words, basically never.\n> \n> Huh? I don't see that now (and caps don't help me see it better). That's\n> certainly not what is in [1], and I don't see _how_ to detect the\n> renames in this case, and what would I be actually doing git-ls-tree for\n> when I've already detected the rename. Based on [1], I'd be doing\n> git-ls-tree merely to detect that a file _disappeared_ in the first\n> place, I have to do other stuff to detect the renames themselves.\n> \n> Dear diary, on Sun, Mar 26, 2006 at 09:35:02AM CEST, I got a letter\n> where Ryan Anderson said that...\n> > A simple example is the first loop in git-annotate.perl. (Which was\n> > basically written by Linus, I just translated it from a\n> > shell/pseudo-code example into Perl)\n> \n> Thanks for the hint. Unfortunately, this is precisely the thing I want\n> to avoid, that is essentially reimplementing part of git-rev-list - to\n> do something good, I would have to do my own toposort and merge by date\n> between parallel lines. OTOH, I might just construct a large revlist\n> commandline specifying all the segments I'm interested in and see what\n> happens when I run that.\n> \n> Besides, doing it in shell would be pretty ugly job (forcing me to\n> finally rewrite it in perl is not a bad thing but that'd be a somewhat\n> larger project since I share various common routines with other cg\n> tools, etc).\n> \n\nIf you decide to modify rev-list to do rename tracking you might want\nto have a look at how this is done in blame.c. git-blame only tracks\none file (since that is what it needs) but I think it should be\npossible to track multiple files with a similar approach.\n\n- Fredrik\n"},{"id":"17986","messageId":"20060326105248.GG18185@pasky.or.cz","threadId":"3728","inReplyTo":"7virq1sywj.fsf@assigned-by-dhcp.cox.net","subject":"Re: Following renames","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-03-26T10:52:48Z","receivedAt":"2006-03-26T10:52:48Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"(Note that I do *not* want to raise the explicit vs. implicit rename\ntracking argument, in case anyone would misunderstood. I've accepted\nimplicit rename tracking as a fact of Git life for now. I just want to\nmake use of it now. ;-)\n\nDear diary, on Sun, Mar 26, 2006 at 04:49:48AM CEST, I got a letter\nwhere Junio C Hamano said that...\n> Looking at the evolution of rev-list.c file itself was a good\n> exercise to realize that rename tracking (more specifically,\n> having whatchanged to follow renames) is not such a useful\n> thing (at least for me).\n\nWell, noone argues that rename tracking cures all the woes of hackerkind\nand anything more precise than that is useless. I'm rather saying that\nrename tracking indeed _is_ a special case of something more general and\ntruly very interesting, but a special case so frequent that it's worth\ndoing even if we can't do the general case yet. Or at least people\n*think* it's very frequent and it gives them the warm fuzzy feeling\nknowing that the tool can handle it (at least somehow) - and the warm\nfuzzy feeling is important, especially if you're trusting your sources\nto the tool.\n\nSo, obviously, you'll find plenty of counter-examples where rename\ndetection won't help. I don't argue that. I merely say that there will\nstill be enough cases where following renames will help to warrant\ndoing it.\n\nNow, Git history has enough examples of where rename following would be\nuseful. When I'm digging into the history, I'm hitting the big tools\nrename barrier all the time, and just yesterday when wondering about\njdl's removal from git.txt I've hit 2cf565c53 - coming along any\nfile to that commit should make me follow Documentation/core-git.txt out\nof the commit (well, that's rather copy than rename detection).\n\n> Another example. Today's tar-tree updates have one interesting\n> function I think should belong to strbuf.c, and before merging\n> it to the mainline, I may move that function from tar-tree.c to\n> strbuf.c. After that happens, if I run \"whatchanged strbuf.c\"\n> to see where that function came from, I would want it to notice\n> it came from tar-tree.c, although it is not a rename at all.\n> Just one function moved from a file to another.\n\nA wild pickaxe - when the string disappears from file X, scan all the\nchanges in the commit and start following files where it reappears. This\nshould help, right?\n\nBut when you want to implement this, you hit the exact same problems as\nwhen you try to follow renames, only a different part of diffcore\ndetects it. So, what I'm trying to solve is actually not just following\nrenames but a more general problem.\n\n> If a file A and file B were cleaned up and merged into a single file\n> C, which is in the tip of the tree, which one you would want\n> whatchanged to switch following depends on which part of the C you\n> were interested in.\n\nIf in doubt (and the user does not use pickaxe to clarify it), you can\njust follow both. The user will get some extra stuff (or maybe even not\nif he wants to know about pieces from both), but we are at least trying\nto be useful and DTRT instead of doing nothing in case we would by any\nchance not do the very best.\n\n> Unless you are interested in the _entire_ contents of the file,\n> that is. Then tracking or even recording renames becomes\n> useful, but that is a special case.\n\nA frequent (and wanted) special case.\n\n> That is the reason I am not so enthused about recording renames.\n> I think the time is better spent on enhancing what pickaxe tries\n> to do (currently it does very little), which I hinted in a\n> separate message late last night.\n\nSure, pickaxe is cool, but as I said above, if you try to teach _it_\nfollowing around files, you'll hit the exact same problems as me. We're\njust trying to build something using lego blocks with different stuff\ninside but otherwise actually looking pretty much the same.\n\nThe thing with pickaxe is that frequently it would be simply more\nlaborous to dig for and construct the proper pickaxe string than just\nfiring up cg-log -s filename with greedy renames following and quickly\nscanning through the results.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"},{"id":"17987","messageId":"20060326105509.GH18185@pasky.or.cz","threadId":"3728","inReplyTo":"20060326105248.GG18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-03-26T10:55:09Z","receivedAt":"2006-03-26T10:55:09Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sun, Mar 26, 2006 at 12:52:48PM CEST, I got a letter\nwhere Petr Baudis said that...\n> Well, noone argues that rename tracking cures all the woes of hackerkind\n ^^^^^^^^^^\n\nOr is it hackerdom?\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"},{"id":"17993","messageId":"20060326190836.dbe95a72.tihirvon@gmail.com","threadId":"3728","inReplyTo":"7virq1sywj.fsf@assigned-by-dhcp.cox.net","subject":"Re: Following renames","fromName":"Timo Hirvonen","fromEmail":"tihirvon@gmail.com","sentAt":"2006-03-26T16:08:36Z","receivedAt":"2006-03-26T16:08:36Z","isPatch":false,"sender":{"key":"tihirvon@gmail.com","avatar":null},"body":"On Sat, 25 Mar 2006 18:49:48 -0800\nJunio C Hamano wrote:\n\n> Looking at the evolution of rev-list.c file itself was a good\n> exercise to realize that rename tracking (more specifically,\n> having whatchanged to follow renames) is not such a useful\n> thing (at least for me).\n\nIt would be useful for me. I had all files organized in subdirectories,\nbut then noticed it was not good idea because make does not play nicely\nwith subdirs, so I moved all files to top level directory.\n\nNow\n\n git-whatchanged -p file.c\n\nstops at the big rename. To continue I have to do\n\n git-whatchanged -p -- \n\n> Another example. Today's tar-tree updates have one interesting\n> function I think should belong to strbuf.c, and before merging\n> it to the mainline, I may move that function from tar-tree.c to\n> strbuf.c. After that happens, if I run \"whatchanged strbuf.c\"\n> to see where that function came from, I would want it to notice\n> it came from tar-tree.c, although it is not a rename at all.\n> Just one function moved from a file to another.\n\nYes in this case you can do\n\n$ git-whatchanged strbuf.c\n$ git-whatchanged tar-tree.c\n\nbut after rename...\n\n$ git-whatchanged old-file.c\nfatal: 'old-file.c': No such file or directory\n\n$ touch old-file.c\n$ git-whatchanged old-file.c\n\nHah, it worked!\n\n\nHmm... this works too without the touch-hack:\n\n$ git-whatchanged file.c old-file.c\n\nI wish I had known this before.\n\n> What this suggests is that switching the set of paths to follow\n> while traversing ancestry chain needs to depend on which part of\n> the original file you are interested in. Marking \"this commit\n> renames (or copies) file A to file B\" is not that useful -- for\n> that matter, detecting at runtime like we currently do is not\n> better either. If a file A and file B were cleaned up and\n> merged into a single file C, which is in the tip of the tree,\n> which one you would want whatchanged to switch following depends\n> on which part of the C you were interested in.\n\nOK, maybe following renames is not such a good idea. But for GUIs\n(gitk, qgit) following renames or even file merges (select a file to\nfollow by clicking it) would be big plus.\n\n-- \nhttp://onion.dynserv.net/~timo/\n"},{"id":"17994","messageId":"e06fl8$p9f$1@sea.gmane.org","threadId":"3728","inReplyTo":"7virq1sywj.fsf@assigned-by-dhcp.cox.net","subject":"Re: Following renames","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-03-26T16:31:52Z","receivedAt":"2006-03-26T16:31:52Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"I wonder what is the most common case in Linux kernel or git.\n\n1.) renaming the file in the same directory, old-file.c to new-file.c?\n2.) moving file to other directory (project reorganization), \n old-dir/file.c to new-dir/file.c?\n3.) splitting file into modules, huge-file.c to file1.c, file2.c?\n4.) copying fragment of one file to other?\n5.) moving fragment of code from one file to other?\n\n-- \nJakub Narebski\nWarsaw, Poland\n"},{"id":"17995","messageId":"Pine.LNX.4.64.0603260829550.15714@g5.osdl.org","threadId":"3728","inReplyTo":"20060326100717.GD18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-26T16:33:13Z","receivedAt":"2006-03-26T16:33:13Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Mar 2006, Petr Baudis wrote:\n> \n> Huh? I don't see that now (and caps don't help me see it better). That's\n> certainly not what is in [1], and I don't see _how_ to detect the\n> renames in this case, and what would I be actually doing git-ls-tree for\n> when I've already detected the rename. Based on [1], I'd be doing\n> git-ls-tree merely to detect that a file _disappeared_ in the first\n> place, I have to do other stuff to detect the renames themselves.\n\nNo, the point is that \"git-rev-list\" already does all of [1] in the core.\n\nIf you do\n\n\tgit-rev-list --parents --remove-empty $REV -- $filename\n\nthen you'll get the whole history for that filename. When it ends, you \nknow the file went away, and then you do basically _one_ \"where the hell \ndid it go\" thing.\n\nAnd yes, it's not git-ls-tree (unless you only want to follow pure \nrenames), it's actually one \"git-diff-tree -M $lastrev\". Then you just \ncontinue with the new filename (and do another \"git-rev-list\" until you \nhit the next rename).\n\n\t\tLinus\n"},{"id":"17996","messageId":"Pine.LNX.4.64.0603260836050.15714@g5.osdl.org","threadId":"3728","inReplyTo":"20060326190836.dbe95a72.tihirvon@gmail.com","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-26T16:43:10Z","receivedAt":"2006-03-26T16:43:10Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Mar 2006, Timo Hirvonen wrote:\n>\n> $ git-whatchanged old-file.c\n> fatal: 'old-file.c': No such file or directory\n> \n> $ touch old-file.c\n> $ git-whatchanged old-file.c\n> \n> Hah, it worked!\n\nIt worked even before:\n\n\tgit-whatchanged -- old-file.c\n\nalways works.\n\nIf you think of the \"filename spec\" as _always_ having to have a \"--\" to \nseparate the filenames from the other arguments, you're thinking the right \nway. Then, there's a _shorthand_ for existing files, where we allow users \nbeing lazy (because _I_ am very lazy indeed), which allows dropping of the \n\"--\", but then the code requires that the filenames are real filenames as \nof now.\n\n> Hmm... this works too without the touch-hack:\n> \n> $ git-whatchanged file.c old-file.c\n> \n> I wish I had known this before.\n\nActually, it -shouldn't- work. It's just that \"git-rev-parse\" isn't as \nanal as it should be.\n\nHere's a fix.\n\n\t\tLinus\n----\ndiff --git a/rev-parse.c b/rev-parse.c\nindex f90e999..104b1e2 100644\n--- a/rev-parse.c\n+++ b/rev-parse.c\n@@ -172,7 +172,9 @@ int main(int argc, char **argv)\n \t\tchar *dotdot;\n \t\n \t\tif (as_is) {\n-\t\t\tshow_file(arg);\n+\t\t\tif (show_file(arg) && as_is < 2)\n+\t\t\t\tif (lstat(arg, &st) < 0)\n+\t\t\t\t\tdie(\"'%s': %s\", arg, strerror(errno));\n \t\t\tcontinue;\n \t\t}\n \t\tif (!strcmp(arg,\"-n\")) {\n@@ -192,7 +194,7 @@ int main(int argc, char **argv)\n \n \t\tif (*arg == '-') {\n \t\t\tif (!strcmp(arg, \"--\")) {\n-\t\t\t\tas_is = 1;\n+\t\t\t\tas_is = 2;\n \t\t\t\t/* Pass on the \"--\" if we show anything but files.. */\n \t\t\t\tif (filter & (DO_FLAGS | DO_REVS))\n \t\t\t\t\tshow_file(arg);\n"},{"id":"17997","messageId":"Pine.LNX.4.64.0603260843250.15714@g5.osdl.org","threadId":"3728","inReplyTo":"e06fl8$p9f$1@sea.gmane.org","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-26T16:46:04Z","receivedAt":"2006-03-26T16:46:04Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Mar 2006, Jakub Narebski wrote:\n>\n> I wonder what is the most common case in Linux kernel or git.\n> \n> 1.) renaming the file in the same directory, old-file.c to new-file.c?\n\nThe kernel uses subdirectories extensively, and a lot of renames (most of \nthem, I'd say) is because of that subdirectory structure. \n\nSo the same-directory case is the unusual one, I'd say.\n\n> 3.) splitting file into modules, huge-file.c to file1.c, file2.c?\n> 4.) copying fragment of one file to other?\n> 5.) moving fragment of code from one file to other?\n\nI'd say that (5) is very common. And (4) happens a lot under certain \ncircumstances (new driver, new architecture, new filesystem..).\n\nDoing (3) happens, but probably less often that it should ;/\n\n\t\tLinus\n"},{"id":"17998","messageId":"e06hts$1ne$1@sea.gmane.org","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603260843250.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-03-26T17:10:35Z","receivedAt":"2006-03-26T17:10:35Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Linus Torvalds wrote:\n\n> On Sun, 26 Mar 2006, Jakub Narebski wrote:\n>>\n>> I wonder what is the most common case in Linux kernel or git.\n>> \n>> 1.) renaming the file in the same directory, old-file.c to new-file.c?\n>> 2.) moving file to other directory (project reorganization), \n>>     old-dir/file.c to new-dir/file.c?\n> The kernel uses subdirectories extensively, and a lot of renames (most of\n> them, I'd say) is because of that subdirectory structure.\n> \n> So the same-directory case is the unusual one, I'd say.\n\nIf (2) is common enough then discussed improvements to rename detection, \nnamely comparing basenames as a base for candidate selection is a good idea.\nI wonder how common is (2) compared to (1)+(2) i.e. move to other dir \nand rename, old-dir/old-file.c to new-dir/new-subdir/new-file.c\n\n>> 3.) splitting file into modules, huge-file.c to file1.c, file2.c?\n>> 4.) copying fragment of one file to other?\n>> 5.) moving fragment of code from one file to other?\n> \n> I'd say that (5) is very common. And (4) happens a lot under certain\n> circumstances (new driver, new architecture, new filesystem..).\n> \n> Doing (3) happens, but probably less often that it should ;/\n\nDetecting (4) and (5) fast (i.e. for merges) without auxilary (helper) \ninformation would probably be hard. For interrogation/porcellanish commands\n(like pickaxe) would probably be easier.\n\n-- \nJakub Narebski\nWarsaw, Poland\n"},{"id":"18002","messageId":"Pine.LNX.4.64.0603260947100.15714@g5.osdl.org","threadId":"3728","inReplyTo":"e06hts$1ne$1@sea.gmane.org","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-26T18:10:56Z","receivedAt":"2006-03-26T18:10:56Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Mar 2006, Jakub Narebski wrote:\n> \n> If (2) is common enough then discussed improvements to rename detection, \n> namely comparing basenames as a base for candidate selection is a good idea.\n\nBK had this \"renametool\" which got started automatically when you applied \na patch that removed one or more files and added one or more files, so \nthat you could then pair up the files manually.\n\nIt left the rename pairing 100% to the user, but it helped a bit by \nguessing what the pairing might be, and yes, it used the basenames to set \nup that initial guess.\n\nIt worked in many cases, but it also failed in many cases. I do think it \nwas a useful heuristic within the BK model (since the _real_ choice was \nleft to the user), but I don't think it's very useful for git.\n\nThe thing is, the fast rename detection that is in the \"next\" branch \nreally does a lot better, and it's fast enough.\n\n(If you wanted to make it even faster, but less precise, you could limit \nit to the first few kilobytes of file contents - still a _lot_ better \nheuristic than the actual filename, and it would make the worst-case \nbehaviour much better).\n\n> I wonder how common is (2) compared to (1)+(2) i.e. move to other dir \n> and rename, old-dir/old-file.c to new-dir/new-subdir/new-file.c\n\nI don't have any numbers, but from usign renametool for a few years, my \ngut feel/recollection is that about half of renames in the kernel were \nmoving to a new directory, and about half changed names (often in \n_addition_ to moving). But I didn't much think about it, so that's just a \nvery rough guess based on using a tool that helped you do these things \nmanually.\n\nFor example, one common case was a directory structure like\n\n\t..\n\ttype-file1.c\n\ttype-file2.c\n\totherfiles.c\n\tyet-more.c\n\t..\n\nbeing split up into a subdirectory\n\n\t..\n\ttype/file1.c\n\ttype/file2.c\n\totherfiles.c\n\tyet-more.c\n\t..\n\n(eg drivers/scsi/aic7xx-* being given a subdirectory of it's own, as \ndrivers/scsi/aic7xx/*). So the basename wouldn't stay the same, because it \ncontained some piece of data that became redundant with the move.\n\n> >> 3.) splitting file into modules, huge-file.c to file1.c, file2.c?\n> >> 4.) copying fragment of one file to other?\n> >> 5.) moving fragment of code from one file to other?\n> > \n> > I'd say that (5) is very common. And (4) happens a lot under certain\n> > circumstances (new driver, new architecture, new filesystem..).\n> > \n> > Doing (3) happens, but probably less often that it should ;/\n> \n> Detecting (4) and (5) fast (i.e. for merges) without auxilary (helper) \n> information would probably be hard. For interrogation/porcellanish commands\n> (like pickaxe) would probably be easier.\n\nYes. I don't think we necessarily want to merge automatically across \nthings like that, even if it sounds like something you'd want in a perfect \nworld. Stupid and obvious (and fails) is often better than smart and \ncomplex (and succeeds), because at least you _understand_ what happens. \n\nBut _following_ a particular change back is important, and should be both \nefficient and simple to do. Ie the example tool I talked about in\n\n http://article.gmane.org/gmane.comp.version-control.git/217\n\nis still relevant and important, I think.\n\nI literally think that people wouldn't even _want_ a \"git annotate\", if \nthey instead had more of a visual tool that showed the current state of \nthe file, and you could click on a line/set of lines to follow it back to \nthe previous change to that area. I'd argue that almost always when you \nwant \"annotate\", you already have the particular place that you want to \nlook at in mind (you're really not interested in the whole file).\n\nSo wouldn't it be _much_ nicer to have a \"graphical git-whatchanged\", \nwhere you just delve deeper (and you don't even look at the whole file \nlike git-whatchanged does, but you ask for a very particular region).\n\nIe, what I imagine would be something gitk/qgit like, where you see the \nfile content, select a line or two (or a whole function), and it goes back \nin history and shows you the last diff that changed that \nline/two/function. We can do that EFFICIENTLY. Much more efficiently than \ngit-annotate, in fact. And then when you see the diff, you might say \"I'm \nnot interested in this one, that was just a re-indent\" and then continue \nback. \n\nTHAT is the kind of graphical tool I'd want. And dammit, it should even be \n_easy_. I'm just a total clutz myself when it comes to doing things like\nQT or nice tcl/tk text-panes, and this really does have to be visual, \nsince the whole point is that \"select text\" and interactive part.\n\nSo if somebody wants to be a hero, and feels comfortable with those kinds \nof things, this really should be a fairly straightforward thing to do (it \nwould be useful even without rename detection or data movement detection, \nbut it's also something where you really _could_ do efficient data \nmovement detection by just looking at the \"whole diff\" when something \nchanged in that small area).\n\n\t\tLinus\n"},{"id":"18005","messageId":"20060326191445.GQ18185@pasky.or.cz","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603260829550.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-03-26T19:14:45Z","receivedAt":"2006-03-26T19:14:45Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sun, Mar 26, 2006 at 06:33:13PM CEST, I got a letter\nwhere Linus Torvalds said that...\n> If you do\n> \n> \tgit-rev-list --parents --remove-empty $REV -- $filename\n> \n> then you'll get the whole history for that filename. When it ends, you \n> know the file went away, and then you do basically _one_ \"where the hell \n> did it go\" thing.\n> \n> And yes, it's not git-ls-tree (unless you only want to follow pure \n> renames), it's actually one \"git-diff-tree -M $lastrev\". Then you just \n> continue with the new filename (and do another \"git-rev-list\" until you \n> hit the next rename).\n\nI wrote a long rant but then it all suddenly fit together and I have now\nan idea how to implement it reasonably elegantly.\n\nSo only a bugreport remains:\n\nMy current target is to support this tree (letters are filenames,\nnumbers are commit ids; I'll translate any git output to those digits):\n\n 2 4\n b -- d\n1 / \\ 6\na d\n \\ 3 5 /\n c -- d\n\nWith the commits created in the numerical order (so log shows\n1,2,3,4,5,6, and my target is cg-log d showing the same output). If\nanyone wants the sample history, it's at\n\n\thttp://pasky.or.cz/~xpasky/renametree1.git/\n\nCuriously, git-rev-list does something totally strange when trying to\nlist per-file history at this point:\n\n\t$ git-rev-list HEAD -- d\n\t4\n\nHuh? (It should list 6, 5, 4 instead.)\n\nI worked it around by recording a change in d in the merge 6:\n\n\thttp://pasky.or.cz/~xpasky/renametree2.git/\n\n\t$ git-rev-list --parents --remove-empty HEAD -- d\n\t6 4 5\n\t5\n\t4\n\nWhich is the expected output.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"},{"id":"18006","messageId":"e5bfff550603261122m5e680c62ye1290f3e601e947e@mail.gmail.com","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603260947100.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-03-26T19:22:06Z","receivedAt":"2006-03-26T19:22:06Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 3/26/06, Linus Torvalds wrote:\n>\n>\n> So wouldn't it be _much_ nicer to have a \"graphical git-whatchanged\",\n> where you just delve deeper (and you don't even look at the whole file\n> like git-whatchanged does, but you ask for a very particular region).\n>\n> Ie, what I imagine would be something gitk/qgit like, where you see the\n> file content, select a line or two (or a whole function), and it goes back\n> in history and shows you the last diff that changed that\n> line/two/function. We can do that EFFICIENTLY. Much more efficiently than\n> git-annotate, in fact. And then when you see the diff, you might say \"I'm\n> not interested in this one, that was just a re-indent\" and then continue\n> back.\n>\n> THAT is the kind of graphical tool I'd want. And dammit, it should even be\n> _easy_. I'm just a total clutz myself when it comes to doing things like\n> QT or nice tcl/tk text-panes, and this really does have to be visual,\n> since the whole point is that \"select text\" and interactive part.\n>\n> So if somebody wants to be a hero, and feels comfortable with those kinds\n> of things, this really should be a fairly straightforward thing to do (it\n> would be useful even without rename detection or data movement detection,\n> but it's also something where you really _could_ do efficient data\n> movement detection by just looking at the \"whole diff\" when something\n> changed in that small area).\n>\n\nI am a thousand miles away from being an hero (and glad of it), but....\n\nI really need a bit of feedback or comment about this because IMHO\nqgit annotate is *almost* very similar to what you would ask, so I\nneed to understand well the difference:\n\nFIRST WAY\n\nAfter annotating a file history (double click on a file name in\nbottom-right window or directly from tree view), you see the whole\nfile annotated. If you have the diff window open you see also the\ncorresponding patch (scrolled to selected file name).\n\nNow, double clicking on the chosen code line in file content makes\ncurrently two things:\n\n - Diff window is updated to show corresponding revision patch, i.e.\nthe last patch that modified that line of code.\n\n- File content, as well as file annotation, changes to show the\ncontent of the file just after the patch was applied, from there it is\nnormally possible to go back in the history of that code region in the\nsame way, i.e. double clicking on interesting lines.\n\nBiggest limitation of 'annotation browsing' is that 'code removing\nonly' patches are not annotated and you need to check them directly\nin diff window.\n\nSECOND WAY\n\nWithout opening the file viewer it is possible to select a file (or\nmore then one or one directory) from tree view and press magic wand\nbutton. This causes main view to be updated with git-rev-list --\n content, i.e. a filtered view.\n\nWith diff viewer window open you can browse across file patch history\nrelated to chosen file.\n\nBiggest limitation is that all the revisions who touch the file are\nshown, not only the ones limited to a selected region.\n\nIF I HAVE UNDERSTOOD...\n\nIf I have understood what you would like to see it something like the following:\n\n- From diff/file viewer window select a code region.\n\n- Press Magic wand button and feed git-rev-list with \n_and_ \n\n- Show git-rev-list output on main window as usual, but now selected\nrevisions are filtered out not only for path but also for region of\ncode touched.\n\n\nAm I guessing correctly?\n\nMarco\n"},{"id":"18008","messageId":"20060326203116.GR18185@pasky.or.cz","threadId":"3728","inReplyTo":"20060326191445.GQ18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-03-26T20:31:16Z","receivedAt":"2006-03-26T20:31:16Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sun, Mar 26, 2006 at 09:14:45PM CEST, I got a letter\nwhere Petr Baudis said that...\n> Curiously, git-rev-list does something totally strange when trying to\n> list per-file history at this point:\n> \n> \t$ git-rev-list HEAD -- d\n> \t4\n> \n> Huh? (It should list 6, 5, 4 instead.)\n\nObviously not 6 since the file was not changed in that revision, but I'd\nstill expect it to list 5.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"},{"id":"18009","messageId":"20060326210956.GS18185@pasky.or.cz","threadId":"3728","inReplyTo":"44264426.8010608@michonline.com","subject":"Re: Following renames","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-03-26T21:09:56Z","receivedAt":"2006-03-26T21:09:56Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sun, Mar 26, 2006 at 09:35:02AM CEST, I got a letter\nwhere Ryan Anderson said that...\n> Linus Torvalds wrote:\n> > On Sun, 26 Mar 2006, Petr Baudis wrote:\n> > \n> >> In [1], Linus suggests a non-core solution. Unfortunately, it doesn't\n> >> fly - it requires at least two git-ls-tree calls per revision which\n> >> would bog things down awfully (to roughly half of the original speed).\n> >> \n> >\n> > No it doesn't. It requires one git-ls-tree WHEN SOMETHING IS RENAMED.\n> >\n> > In other words, basically never.\n> > \n> \n> A simple example is the first loop in git-annotate.perl. (Which was\n> basically written by Linus, I just translated it from a\n> shell/pseudo-code example into Perl)\n\nOne case it does not handle:\n\n 2\n -- b --\n 1 / \\ 6\n a d\n \\ 3 5 /\n c --- d\n\ngit-rev-list at 6 will (understandably) show\n\n 6 5\n 5\n\nand you will never detect the d -> b rename leading to 2.\n\nThis is one reason why I'm actually not using --parents and pipe stuff\ndirectly to git-diff-tree --stdin -M and then read its output. This also\nlets me merge parallel lines of development based on date and I don't\nhave to fork per each file deletion.\n\nWith any luck I'll have the first draft of my (also perlish) script done\nthis evening yet. (BTW, it has the same output format as\n\n\tgit-rev-list | git-diff-tree --pretty=raw -M\n\nso with some tweaking it could also serve as a git-whatchanged backend.\nActually, it would be nice to have it in core Git in the long term so\nthat it gets all the portability tweaks and such.)\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"},{"id":"18012","messageId":"Pine.LNX.4.64.0603261415390.15714@g5.osdl.org","threadId":"3728","inReplyTo":"20060326191445.GQ18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-26T22:22:04Z","receivedAt":"2006-03-26T22:22:04Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Mar 2006, Petr Baudis wrote:\n> \n> My current target is to support this tree (letters are filenames,\n> numbers are commit ids; I'll translate any git output to those digits):\n> \n> 2 4\n> b -- d\n> 1 / \\ 6\n> a d\n> \\ 3 5 /\n> c -- d\n\nYeah, the problem with this is that you need to track separate names \nacross separate points. However:\n\n> Curiously, git-rev-list does something totally strange when trying to\n> list per-file history at this point:\n> \n> \t$ git-rev-list HEAD -- d\n> \t4\n> \n> Huh? (It should list 6, 5, 4 instead.)\n\nWhat it does is list the points where file \"d\" _changed_.\n\n\"d\" did not change in 6 - it had a parent commit (4) where \"d\" had the \nsame contents (in fact, it likely had _two_ parents where it had the same \ncontents, but git will pick the first one). So commit \"6\" is \nuninteresting, and commit \"5\" will never even be looked at, since we \ndecided that the history of \"d\" comes from the first parent with the same \ncontents.\n\nSo then it lists \"4\", because file \"d\" really did change in that commit \n(it went away).\n\nNow you need to look at \"4\" and find the rename (which gives you 2) and \nthen from there you do rename detection and get (1), and as a result your \nchange history should end up being\n\n (1)a -> (2)b -> (4)d (-> 6(d) which was your start point)\n\nwhich is correct (now, there are other histories _too_ that get us to the \nsame point, but the one you found this way was _a_ history).\n\n> I worked it around by recording a change in d in the merge 6:\n> \n> \thttp://pasky.or.cz/~xpasky/renametree2.git/\n> \n> \t$ git-rev-list --parents --remove-empty HEAD -- d\n> \t6 4 5\n> \t5\n> \t4\n> \n> Which is the expected output.\n\nNo, it's the expected output just because you expected merges to always \nshow up. Merges get ignored if any of the parents have the same content \nalready.\n\n\t\tLinus\n"},{"id":"18013","messageId":"Pine.LNX.4.64.0603261422280.15714@g5.osdl.org","threadId":"3728","inReplyTo":"e5bfff550603261122m5e680c62ye1290f3e601e947e@mail.gmail.com","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-26T22:23:57Z","receivedAt":"2006-03-26T22:23:57Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Mar 2006, Marco Costalba wrote:\n> \n> FIRST WAY\n> \n> After annotating a file history (double click on a file name in\n> bottom-right window or directly from tree view), you see the whole\n> file annotated. If you have the diff window open you see also the\n> corresponding patch (scrolled to selected file name).\n\nThe problem is that this step is already _way_ too expensive.\n\nI don't want to work with any tool that makes \"Step 1\" take a minute or \ntwo for a project that has a few years of history. Try it on the linux \nhistoric project with some file that gets lots of modifications.\n\nIn other words, starting off with \"annotate\" is MUCH too expensive. You \nshould start off basically with \"git-whatchanged\".\n\n\t\tLinus\n"},{"id":"18014","messageId":"20060326223154.GU18185@pasky.or.cz","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603261415390.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-03-26T22:31:54Z","receivedAt":"2006-03-26T22:31:54Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Mon, Mar 27, 2006 at 12:22:04AM CEST, I got a letter\nwhere Linus Torvalds said that...\n> So commit \"6\" is uninteresting, and commit \"5\" will never even be\n> looked at, since we decided that the history of \"d\" comes from the\n> first parent with the same contents.\n\nAnd this is the thing I have a problem with - this does not make much\nsense to me, why can't we just follow all parents instead of arbitrarily\nchoosing one of them?\n\n> which is correct (now, there are other histories _too_ that get us to the \n> same point, but the one you found this way was _a_ history).\n\nOk, in that case I want the _full_ history. :-)\n\n> No, it's the expected output just because you expected merges to always \n> show up. Merges get ignored if any of the parents have the same content \n> already.\n\nEek. Can I avoid that? What was the reason for choosing this behavior?\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"},{"id":"18015","messageId":"7vodzsq12g.fsf@assigned-by-dhcp.cox.net","threadId":"3728","inReplyTo":"20060326223154.GU18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-03-26T22:43:35Z","receivedAt":"2006-03-26T22:43:35Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Petr Baudis writes:\n\n>> No, it's the expected output just because you expected merges to always \n>> show up. Merges get ignored if any of the parents have the same content \n>> already.\n>\n> Eek. Can I avoid that? What was the reason for choosing this behavior?\n\nPerhaps rev-list --sparse?\n"},{"id":"18016","messageId":"Pine.LNX.4.64.0603261505230.15714@g5.osdl.org","threadId":"3728","inReplyTo":"20060326223154.GU18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-26T23:09:13Z","receivedAt":"2006-03-26T23:09:13Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 27 Mar 2006, Petr Baudis wrote:\n\n> Dear diary, on Mon, Mar 27, 2006 at 12:22:04AM CEST, I got a letter\n> where Linus Torvalds said that...\n> > So commit \"6\" is uninteresting, and commit \"5\" will never even be\n> > looked at, since we decided that the history of \"d\" comes from the\n> > first parent with the same contents.\n> \n> And this is the thing I have a problem with - this does not make much\n> sense to me, why can't we just follow all parents instead of arbitrarily\n> choosing one of them?\n\nSure, you can. It's _usually_ a huge waste of time, though. Why would you \nwant to do more work than you need, since clearly the other parent was \n_not_ interesting from the standpoint of the question \"where did this \ncontent come from\"?\n\n> > No, it's the expected output just because you expected merges to always \n> > show up. Merges get ignored if any of the parents have the same content \n> > already.\n> \n> Eek. Can I avoid that? What was the reason for choosing this behavior?\n\nHuge efficiency gains.\n\nLookie here. Do\n\n\tgitk -- rev-list.c\n\non the git archive with the current git-rev-list, and with your hacked-up \nversion.\n\nAnd tell me my version isn't a hell of a lot better. Because, I guarantee \nyou, it is. We're just not _interested_ in all those merges that didn't \nactually make any difference.\n\nRead up on what modern neuro-science thinks about the human brain, and \nwhat a lot of it is about. It's about ignoring irrelevant information.\n\nThe ability to throw stuff out that isn't interesting is the _real_ basis \nof true intelligence. I'd rather have git do the _intelligent_ history, \nthan show history that isn't relevant and workign harder doing so.\n\n\t\tLinus\n"},{"id":"18017","messageId":"Pine.LNX.4.64.0603261509320.15714@g5.osdl.org","threadId":"3728","inReplyTo":"7vodzsq12g.fsf@assigned-by-dhcp.cox.net","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-26T23:10:35Z","receivedAt":"2006-03-26T23:10:35Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Mar 2006, Junio C Hamano wrote:\n> Petr Baudis writes:\n> \n> >> No, it's the expected output just because you expected merges to always \n> >> show up. Merges get ignored if any of the parents have the same content \n> >> already.\n> >\n> > Eek. Can I avoid that? What was the reason for choosing this behavior?\n> \n> Perhaps rev-list --sparse?\n\nNo. \"--sparse\" still removes the uninteresting parents of merges. It just \ndoesn't then make the linear history any denser.\n\n\t\tLinus\n"},{"id":"18019","messageId":"20060326232649.GV18185@pasky.or.cz","threadId":"3728","inReplyTo":"20060326191445.GQ18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-03-26T23:26:49Z","receivedAt":"2006-03-26T23:26:49Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sun, Mar 26, 2006 at 09:14:45PM CEST, I got a letter\nwhere Petr Baudis said that...\n> Dear diary, on Sun, Mar 26, 2006 at 06:33:13PM CEST, I got a letter\n> where Linus Torvalds said that...\n> > If you do\n> > \n> > \tgit-rev-list --parents --remove-empty $REV -- $filename\n> > \n> > then you'll get the whole history for that filename. When it ends, you \n> > know the file went away, and then you do basically _one_ \"where the hell \n> > did it go\" thing.\n> > \n> > And yes, it's not git-ls-tree (unless you only want to follow pure \n> > renames), it's actually one \"git-diff-tree -M $lastrev\". Then you just \n> > continue with the new filename (and do another \"git-rev-list\" until you \n> > hit the next rename).\n> \n> I wrote a long rant but then it all suddenly fit together and I have now\n> an idea how to implement it reasonably elegantly.\n\nSo, this is what I have. Testing (I've gave it very little of that) and\nthoughts welcome. It is probably pretty efficient, at least in terms of\nfork()s it does only 2*N of them where N is the number of commits\ncontaining interesting renames. Actually, this should be even possible\nto reduce to N+1 if you do a single git-diff-tree call and multiplex\ndifferent git-rev-lists to it, but I'm too tired to do the trickery now.\n\nIt has 'cg' in the name but depends on no Cogito stuff; it should be in\nfact possible to trivially put it to git-whatchanged in place of the\nfinal pipeline (not that I'd be suggesting this to be done universally,\nbut perhaps git-whatchanged -f ...?). There are three downsides in this\nregard:\n\n(i) No -c support. I need the separate deltas coming out from\ngit-diff-tree but I think I can join them together pretty easily on my\nown, except that I have problems with -c (see\n<20060326102100.GF18185@pasky.or.cz>) so I'm not sure how exactly is it\nsupposed to behave.\n\n(ii) Only --pretty=raw output. It shouldn't be hard to add the\nreformatting code, but I'm personally not going to use it and kind of\nlazy, so I'll let someone else do that, I guess. :-)\n\n(iii) Raw deltas required. -p parsing support would be certainly useful\nand possible, but see (ii).\n\n\nTo quickly see what it does, you can try it e.g. on the git-log.sh file\nin the Git repository.\n\nThoughts? Opinions? Bugs? Patches?\n\n\nSigned-off-by: Petr Baudis \n\n\ndiff --git a/cg-Xfollowrenames b/cg-Xfollowrenames\nnew file mode 100755\nindex 0000000..fa5c552\n--- /dev/null\n+++ b/cg-Xfollowrenames\n@@ -0,0 +1,246 @@\n+#!/usr/bin/env perl\n+#\n+# git-rev-list | git-diff-tree --stdin following renames\n+# Copyright (c) Petr Baudis, 2006\n+# Uses bits of git-annotate.perl by Ryan Anderson.\n+#\n+# This script will efficiently show output as of the\n+#\n+#\tgit-rev-list --remove-empty ARGS -- FILE... |\n+#\tgit-diff-tree -M -r -m --stdin --pretty=raw ARGS\n+#\n+# pipeline, except that it follows renames of individual files listed\n+# in the FILE... set.\n+#\n+# Usage:\n+#\n+#\tcg-Xfollowrenames revlistargs -- difftreeargs -- revs -- files\n+\n+# TODO: Does not work on multiple files properly yet - most probably\n+# (I didn't test it!). We want git-rev-list to stop traversing the history\n+# when _any_ file disappears while now it probably stops traversing when\n+# _all_ files disappear.\n+\n+use warnings;\n+use strict;\n+\n+$| = 1;\n+\n+our (@revlist_args, @difftree_args, @revs, @files);\n+\n+{ # Load arguments\n+\tmy @argp = (\\@revlist_args, \\@difftree_args, \\@revs, \\@files);\n+\tmy $argi = 0;\n+\tfor my $arg (@ARGV) {\n+\t\tif ($arg eq '--' and $argi < $#argp) {\n+\t\t\t$argi++;\n+\t\t\tnext;\n+\t\t}\n+\t\tpush(@{$argp[$argi]}, $arg);\n+\t}\n+}\n+\n+\n+# The heads we watch (sorted by commit time)\n+our @heads;\n+# Each head is: {\n+#\t# Persistent for the whole line of development:\n+#\tpipe => $pipe,\n+#\tfiles => \\@files, # to watch for\n+#\n+#\tid => $sha1, # useful actually only for debugging\n+#\ttime => $timestamp,\n+#\tstr => $prettyoutput,\n+#\tparents => \\@sha1s,\n+#\n+#\t# When the commit is processed, spawn these extra heads:\n+#\trecurse => {$sha1id => \\@files, ...},\n+# }\n+\n+# To avoid printing duplicate commits\n+# FIXME: Currently, we will not handle merge commits properly since\n+# we hit them multiple times.\n+our %commits;\n+\n+\n+sub open_pipe($@) {\n+\tmy ($stdin, @execlist) = @_;\n+\n+\tmy $pid = open my $kid, \"-|\";\n+\tdefined $pid or die \"Cannot fork: $!\";\n+\n+\tunless ($pid) {\n+\t\tif (defined $stdin) {\n+\t\t\topen STDIN, \"<&\", $stdin or die \"Cannot dup(): $!\";\n+\t\t}\n+\t\texec @execlist;\n+\t\tdie \"Cannot exec @execlist: $!\";\n+\t}\n+\n+\treturn $kid;\n+}\n+\n+sub revlist($@) {\n+\tmy ($rev, @files) = @_;\n+\topen_pipe(undef, \"git-rev-list\", \"--remove-empty\",\n+\t @revlist_args, $rev, \"--\", @files)\n+\t\tor die \"Failed to exec git-rev-list: $!\";\n+}\n+\n+sub difftree($) {\n+\tmy ($revlist) = @_;\n+\topen_pipe($revlist, \"git-diff-tree\", \"-r\", \"-m\", \"--stdin\", \"-M\",\n+\t \"--pretty=raw\", @difftree_args)\n+\t\tor die \"Failed to exec git-diff-tree: $!\";\n+}\n+\n+sub revdiffpipe($@) {\n+\tmy ($rev, @files) = @_;\n+\tmy $pipe = difftree(revlist($rev, @files));\n+}\n+\n+\n+sub read_commit($$) {\n+\tmy ($head, $tolerant) = @_;\n+\tmy $pipe = $head->{'pipe'};\n+\tmy $against;\n+\tmy @oldset = @{$head->{'files'}};\n+\tmy @newset;\n+\tmy $rename;\n+\n+\t# Load header\n+\twhile (my $line = <$pipe>) {\n+\t\t$head->{'str'} .= $line;\n+\t\tchomp $line;\n+\t\t$line eq '' and goto header_loaded;\n+\n+\t\tif ($line =~ /^diff-tree (\\S+) \\(from (root|\\S+)\\)/) {\n+\t\t\t$head->{'id'} = $1;\n+\t\t\tif (not $tolerant and $commits{$1}++) {\n+\t\t\t\tclose $pipe;\n+\t\t\t\treturn undef;\n+\t\t\t}\n+\t\t\t# The 'root' case is harmless since there'll be no renames.\n+\t\t\t$against = $2;\n+\t\t} elsif ($line =~ /^parent (\\S+)/) {\n+\t\t\tpush (@{$head->{'parents'}}, $1);\n+\t\t} elsif ($line =~ /^committer .*?> (\\d+)/) {\n+\t\t\t$head->{'time'} = $1;\n+\t\t}\n+\t}\n+\treturn undef;\n+header_loaded:\n+\n+\t# Load message\n+\twhile (my $line = <$pipe>) {\n+\t\t$head->{'str'} .= $line;\n+\t\tchomp $line;\n+\t\t$line eq '' and goto message_loaded;\n+\t}\n+\treturn undef;\n+message_loaded:\n+\n+\t# Load delta\n+\twhile (my $line = <$pipe>) {\n+\t\t$head->{'str'} .= $line;\n+\t\tchomp $line;\n+\t\t$line eq '' and goto delta_loaded;\n+\n+\t\t$line =~ /^:/ or return undef;\n+\t\tmy ($info, $newfile, $oldfile) = split(\"\\t\", $line);\n+\t\tif ($info =~ /[RC]\\d*$/) {\n+\t\t\t# Behold, a rename!\n+\t\t\t# (Or a copy, it's all the same for us.)\n+\t\t\tmy $i;\n+\t\t\tfor ($i = 0; $i <= $#oldset; $i++) {\n+\t\t\t\t$oldfile eq $oldset[$i] or next;\n+\t\t\t\t$rename = 1;\n+\t\t\t\tsplice(@oldset, $i, 1);\n+\t\t\t\tpush(@newset, $newfile);\n+\t\t\t\tlast;\n+\t\t\t}\n+\t\t\t# In case of multiple candidates, follow\n+\t\t\t# all of them:\n+\t\t\t# (TODO: This might be a policy decision\n+\t\t\t# best left on the user.)\n+\t\t\tif ($i > $#oldset and grep { $oldfile eq $_ } @newset) {\n+\t\t\t\t$rename = 1;\n+\t\t\t\tpush(@newset, $newfile);\n+\t\t\t}\n+\t\t} elsif ($info =~ /D$/) {\n+\t\t\t# Not weeding out deleted files might cause bizarre\n+\t\t\t# results when following multiple files since\n+\t\t\t# git-rev-list weeds them out too (probably?).\n+\t\t\t@oldset = grep { $newfile ne $_ } @oldset;\n+\t\t\t@{$head->{'files'}} = grep { $newfile ne $_ } @{$head->{'files'}};\n+\t\t}\n+\t}\n+\t$head->{'str'} .= \"\\n\";\n+delta_loaded:\n+\n+\tif ($rename) {\n+\t\t$head->{'recurse'}->{$against} = [@newset, @oldset];\n+\t}\n+\treturn 1;\n+}\n+\n+sub load_commit($) {\n+\tmy ($head) = @_;\n+\t$head->{'time'} = undef;\n+\t$head->{'str'} = '';\n+\t$head->{'parents'} = ();\n+\n+\tread_commit($head, 0) or return undef;\n+\n+\t# In case there was a merge, the commit will be multiple times\n+\t# here, each time with a different delta section. Read them all.\n+\tfor (1 .. $#{$head->{'parents'}}) { # stupid vim syntax highlighting\n+\t\tread_commit($head, 1) or return undef;\n+\t}\n+\n+\treturn 1;\n+}\n+\n+\n+# Add head at the proper position\n+sub add_head($) {\n+\tmy ($head) = @_;\n+\tmy $i;\n+\tfor ($i = 0; $i <= $#heads; $i++) {\n+\t\tlast if ($head->{'time'} > $heads[$i]->{'time'})\n+\t}\n+\tsplice(@heads, $i, 0, $head);\n+}\n+\n+# Create new head\n+sub init_head($@) {\n+\tmy ($rev, @files) = @_;\n+\tmy $head = { files => \\@files, 'pipe' => revdiffpipe($rev, @files) };\n+\tload_commit($head) or return;\n+\tadd_head($head);\n+}\n+\n+\n+\n+{ # Seed the heads list\n+\tfor my $rev (@revs) {\n+\t\tinit_head($rev, @files);\n+\t}\n+}\n+\n+# Process the heads\n+{\n+\twhile (@heads) {\n+\t\tmy $head = splice(@heads, 0, 1);\n+\n+\t\tprint $head->{'str'};\n+\n+\t\tforeach my $parent (keys %{$head->{'recurse'}}) {\n+\t\t\tinit_head($parent, @{$head->{'recurse'}->{$parent}});\n+\t\t}\n+\t\t$head->{'recurse'} = undef;\n+\n+\t\tload_commit($head) or next;\n+\t\tadd_head($head);\n+\t}\n+}\n\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"},{"id":"18022","messageId":"e5bfff550603262147t3aec8da6p6bf2a333e2d35f1d@mail.gmail.com","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603261422280.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-03-27T05:47:04Z","receivedAt":"2006-03-27T05:47:04Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 3/27/06, Linus Torvalds wrote:\n>\n>\n> On Sun, 26 Mar 2006, Marco Costalba wrote:\n> >\n> > FIRST WAY\n> >\n> > After annotating a file history (double click on a file name in\n> > bottom-right window or directly from tree view), you see the whole\n> > file annotated. If you have the diff window open you see also the\n> > corresponding patch (scrolled to selected file name).\n>\n> The problem is that this step is already _way_ too expensive.\n>\n> I don't want to work with any tool that makes \"Step 1\" take a minute or\n> two for a project that has a few years of history. Try it on the linux\n> historic project with some file that gets lots of modifications.\n>\n\nHistoric Linux test (63428 revisions)\n\nFile: drivers/net/tg3.c\nRevisions that modify tg3.c : 292\n\nWith qgit\n15s to retrieve file history (git-rev-list)\n19.5s to annotate (git-diff-tree -p, current GNU algorithm, not new faster one)\n\nand...\n\n$ time git-whatchanged HEAD drivers/net/tg3.c > /dev/null\n98.01user 2.44system 1:46.19elapsed 94%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (797major+43033minor)pagefaults 0swaps\n\nNOTE: It seems that git-whatchanged asks for checked the out file to\nwork. It didn't work with no repository checked out.\n\n\nMarco\n"},{"id":"18023","messageId":"Pine.LNX.4.64.0603270642090.5276@sheen.jakma.org","threadId":"3728","inReplyTo":"e05354$cc9$1@sea.gmane.org","subject":"Re: Following renames","fromName":"Paul Jakma","fromEmail":"paul@clubi.ie","sentAt":"2006-03-27T06:00:45Z","receivedAt":"2006-03-27T06:00:45Z","isPatch":false,"sender":{"key":"paul@clubi.ie","avatar":null},"body":"On Sun, 26 Mar 2006, Jakub Narebski wrote:\n\n> I think one of the better ideas/suggestions about *recording* filenames was\n> in the \"impure renames / history tracking\" thread\n> http://marc.theaimsgroup.com/?l=git&m=114122175216489&w=2\n> \n\nFor the record, the responses I received were educational ;). \nSufficiently so I no longer think renames should be recorded. At \nleast, definitely not as renames.\n\nI now grok the reasoning for doing it by 'similarity' - it is indeed \na *much* more useful concept. (E.g. the 'pickaxe' idea people keep \nalluding though sounds amazingly useful).\n\nSo the question really is what, if any, weaknesses does the current \nsimilarity estimation have, and how to solve them. I can think of two \nweaknesses:\n\n1. the similarity algorithms can be expensive potentially, and they\n essentially get run a lot with the same inputs, to produce the\n same results - over and over as one works with a git repo. (there\n was a thread a while ago on this I think).\n\n2. Some 'similarities' are just not deducible by current software\n state of the art. E.g. where some code is rewritten in another\n language:\n\n \tfoo.X -> foo.Y\n\n The high-level algorithms may remain the exact same, but the code\n may be unrecognisable as similar except to a human. However,\n tracking history back across this rewrite probably would still be\n valuable to the human.\n\nSo I think what /might/ be interesting is to have a 'similarity \ncache', which would help 1, and to allow for manual injection of such \nhints (into a seperate and stronger cache most likely) - which would \nhelp 2.\n\nSomething to record the following information:\n\n(tree1,tree2)[1]:\n \tId1 <-> Id1'\n \t.\n \t.\n \t.\n \tIdn <-> Idn'\n\nThat would allow:\n\n1. Performance repercussions of similarity estimation to be one-time,\n cached there-after. (throw-away information, if a better\n similarity estimation heuristic comes along, you can rebuild this\n cache)\n\n2. The user to inject their own 'hints' into similarity estimation,\n particularly for cases that just aren't obvious and probably never\n will be to software estimators (e.g. the rewrite cases), but where\n the user sees value in being able to follow back the history.\n\nAvoids:\n\n- encoding anything permanently into the repository (which was\n something I was thinking of, and others before me apparently, but\n which I now accept would be an awful idea ;) ).\n\n1. I'm not sure if it should be indexed by (commit ID) or\n (tree1,tree2) tuple. ??\n\nregards,\n-- \nPaul Jakma\tpaul@clubi.ie\tpaul@jakma.org\tKey ID: 64A2FF6A\nFortune:\nMen take only their needs into consideration -- never their abilities.\n \t\t-- Napoleon Bonaparte\n"},{"id":"18024","messageId":"7vu09ko04w.fsf@assigned-by-dhcp.cox.net","threadId":"3728","inReplyTo":"e5bfff550603262147t3aec8da6p6bf2a333e2d35f1d@mail.gmail.com","subject":"Re: Following renames","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-03-27T06:46:39Z","receivedAt":"2006-03-27T06:46:39Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Marco Costalba\" writes:\n\n> NOTE: It seems that git-whatchanged asks for checked the out file to\n> work. It didn't work with no repository checked out.\n\nPerhaps,\n\n\t$ git-whatchanged HEAD -- drivers/net/tg3.c\n\nas Linus explained in a separate message today...\n"},{"id":"18025","messageId":"e0827k$7tk$1@sea.gmane.org","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603260947100.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-03-27T06:55:03Z","receivedAt":"2006-03-27T06:55:03Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Linus Torvalds wrote:\n\n> On Sun, 26 Mar 2006, Jakub Narebski wrote:\n>> \n>> If (2) is common enough then discussed improvements to rename detection,\n>> namely comparing basenames as a base for candidate selection is a good\n>> idea.\n> \n> BK had this \"renametool\" which got started automatically when you applied\n> a patch that removed one or more files and added one or more files, so\n> that you could then pair up the files manually.\n[...]\n> The thing is, the fast rename detection that is in the \"next\" branch\n> really does a lot better, and it's fast enough.\n\nI was thinking about the fast ename detection algorithm in \"next\" branch.\n\n\nThat is the question if recording additional (helper) information about\ncontents copying and moving like the mentioned \"renametool\" did is worth\nthe effort, both in coding it and from user's point of view. Or would\nbetter contents copying and moving detection (\"renames detection\") for\nwhatchanged and similar suffice.\n\nI am of opinion that voluntary information about contents moving and copying\nin the commits would help.\n\nPurposes:\n1.) Record contents moving and similarity information which cannot or cannot\nbe easily calculated; see Paul Jakma response in this thread\n MessageID: \nfor example copying fragment of code, small fragment of the whole file,\ncreating documentation or header file from code, or code skeleton from\ntemplate, or rewrite of code in different language (e.g. shell script to\nperl, script to compiled code e.g. Perl or Python to C).\n2.) Caching the results of similarity algorithm/rename detection tool (also\nPaul Jakma post), including remembering false positives and undetected\nrenames, for efficiency. Calculated automatically parts might be\nthrow-away.\n\nSources of information:\n1.) Manually entered information *at commit*, including *-rm, *-mv, *-cp\nlike commands (which nobody likes) and systematized (pseudolanguage?) for\ncopying and moving contents in the log messages.\n2.) Semi-manual tools like the mentioned \"renametool\" of BK.\n3.) Support from editor (remebering where copied and pasted, or cut and\npasted fragment came from, and providing prefilled command to record\ncontents moving (\"renames\") or prefilled commit log containing this\ninformation. Hard to get, probably most useful.\n4.) Information from resolved merges and results of diagnosis (pickaxe like)\ntools, especially recording \"renames\" which were not detected, and removing\n\"renames\" which were detected falsily. \n\nIs that the place where I should provide code (patch) for testing the\nidea :) ?\n\n>> I wonder how common is (2) compared to (1)+(2) i.e. move to other dir\n>> and rename, old-dir/old-file.c to new-dir/new-subdir/new-file.c\n>\n> For example, one common case was a directory structure like\n> \n> ..\n> type-file1.c\n> type-file2.c\n> otherfiles.c\n> yet-more.c\n> ..\n> \n> being split up into a subdirectory\n> \n> ..\n> type/file1.c\n> type/file2.c\n> otherfiles.c\n> yet-more.c\n> ..\n> \n> (eg drivers/scsi/aic7xx-* being given a subdirectory of it's own, as\n> drivers/scsi/aic7xx/*). So the basename wouldn't stay the same, because it\n> contained some piece of data that became redundant with the move.\n\nPerhaps fast rename detection algorithm needs some smart similarity estimate\nfor names, which would put more weight in the parts closer to basename, and\nwould detect */type-file1.c and */type/file1.c as similar.\n\n-- \nJakub Narebski\nWarsaw, Poland\n"},{"id":"18026","messageId":"7vodzsny4m.fsf@assigned-by-dhcp.cox.net","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603261509320.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-03-27T07:30:01Z","receivedAt":"2006-03-27T07:30:01Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> No. \"--sparse\" still removes the uninteresting parents of merges. It just \n> doesn't then make the linear history any denser.\n\nHmph, you are right. add_parents_to_list() calls prune_fn\nunconditionally while running limit_list().\n\nDisabling that with yet another flag might be a possibility but\nI suspect then it would not be much different from running\nrev-list without path limiter and having the caller process the\nresult.\n"},{"id":"18027","messageId":"Pine.LNX.4.62.0603262337580.26865@qynat.qvtvafvgr.pbz","threadId":"3728","inReplyTo":"e0827k$7tk$1@sea.gmane.org","subject":"Re: Following renames","fromName":"David Lang","fromEmail":"dlang@digitalinsight.com","sentAt":"2006-03-27T07:40:34Z","receivedAt":"2006-03-27T07:40:34Z","isPatch":false,"sender":{"key":"dlang@digitalinsight.com","avatar":null},"body":"On Mon, 27 Mar 2006, Jakub Narebski wrote:\n\n> 2.) Caching the results of similarity algorithm/rename detection tool (also\n> Paul Jakma post), including remembering false positives and undetected\n> renames, for efficiency. Calculated automatically parts might be\n> throw-away.\n\nthis sounds like it could easily devolve into a O(n!) situation where you \nare cacheing how everything is related (or not related) to everything \nelse. Paul was makeing the point that the purpose was to cache the data to \neliminate the time needed to calculate it, but if you don't store all the \nresults then you don't know if the result is not relavent, or unknown, so \nyou need to calculate it again.\n\nDavid Lang\n\n-- \nThere are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.\n -- C.A.R. Hoare\n"},{"id":"18028","messageId":"e085ka$ig9$1@sea.gmane.org","threadId":"3728","inReplyTo":"Pine.LNX.4.62.0603262337580.26865@qynat.qvtvafvgr.pbz","subject":"Re: Following renames","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-03-27T07:53:01Z","receivedAt":"2006-03-27T07:53:01Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"David Lang wrote:\n\n> On Mon, 27 Mar 2006, Jakub Narebski wrote:\n> \n>> 2.) Caching the results of similarity algorithm/rename detection tool\n>> (also Paul Jakma post), including remembering false positives and\n>> undetected renames, for efficiency. Calculated automatically parts might\n>> be throw-away.\n> \n> this sounds like it could easily devolve into a O(n!) situation where you\n> are cacheing how everything is related (or not related) to everything\n> else. Paul was makeing the point that the purpose was to cache the data to\n> eliminate the time needed to calculate it, but if you don't store all the\n> results then you don't know if the result is not relavent, or unknown, so\n> you need to calculate it again.\n\nFirst of all, you only remember non-trivial relations (i.e. file.c is always\nrelated to file.c). If the cache would be only for commits, it would be\nO(c*p*n), where c is number of commits, p is percentage of contents moving\n(\"renames\") times percent of files changed in the commit, and n is the\nnumber of files, probably O(c) practically. Even if we remember for all\n(tree1,tree2) pairs it would be O(c^2). Additionally cache can be limited\nin size (pruning oldest cache). \n\n-- \nJakub Narebski\nWarsaw, Poland\n"},{"id":"18029","messageId":"Pine.LNX.4.64.0603270005330.15714@g5.osdl.org","threadId":"3728","inReplyTo":"e5bfff550603262147t3aec8da6p6bf2a333e2d35f1d@mail.gmail.com","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-27T08:07:10Z","receivedAt":"2006-03-27T08:07:10Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 27 Mar 2006, Marco Costalba wrote:\n> \n> Historic Linux test (63428 revisions)\n> \n> File: drivers/net/tg3.c\n> Revisions that modify tg3.c : 292\n> \n> With qgit\n> 15s to retrieve file history (git-rev-list)\n> 19.5s to annotate (git-diff-tree -p, current GNU algorithm, not new faster one)\n\n.. and it does absolutely _nothing_ while it's doing that, does it?\n\n> $ time git-whatchanged HEAD drivers/net/tg3.c > /dev/null\n> 98.01user 2.44system 1:46.19elapsed 94%CPU (0avgtext+0avgdata 0maxresident)k\n> 0inputs+0outputs (797major+43033minor)pagefaults 0swaps\n\nIn contrast, git-whatchanged will start outputting the recent changes \nimmediately.\n\nAnd that's the point. Almost always, we're interested in the _recent_ \nstuff. The fact that it takes longer to get the old history is not very \nimportant. You generally don't ask \"what changed in this file\" for a file \nthat hasn't changed in five years.\n\n\t\tLinus\n"},{"id":"18030","messageId":"e5bfff550603270319w20796918wc8f8fe30a6c5627@mail.gmail.com","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603270005330.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-03-27T11:19:02Z","receivedAt":"2006-03-27T11:19:02Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 3/27/06, Linus Torvalds wrote:\n>\n>\n> On Mon, 27 Mar 2006, Marco Costalba wrote:\n> >\n> > Historic Linux test (63428 revisions)\n> >\n> > File: drivers/net/tg3.c\n> > Revisions that modify tg3.c : 292\n> >\n> > With qgit\n> > 15s to retrieve file history (git-rev-list)\n> > 19.5s to annotate (git-diff-tree -p, current GNU algorithm, not new faster one)\n>\n> .. and it does absolutely _nothing_ while it's doing that, does it?\n>\n\nyes, it's true.\n\n> > $ time git-whatchanged HEAD drivers/net/tg3.c > /dev/null\n> > 98.01user 2.44system 1:46.19elapsed 94%CPU (0avgtext+0avgdata 0maxresident)k\n> > 0inputs+0outputs (797major+43033minor)pagefaults 0swaps\n>\n> In contrast, git-whatchanged will start outputting the recent changes\n> immediately.\n>\n> And that's the point. Almost always, we're interested in the _recent_\n> stuff. The fact that it takes longer to get the old history is not very\n> important. You generally don't ask \"what changed in this file\" for a file\n> that hasn't changed in five years.\n>\n\nWe could run git-rev-list with a time range specifier (changes of last\nyear as example) by default so to have fast results and run all time\nhistory _only_ on request.\n\nThis perhaps could solve the fast output for recent revs problem, if\nthis is the problem.\n\nI still think the problem with annotation is that you don't see\npatches that _remove_ lines of code, you need the whole diff for this.\n\nMarco\n"},{"id":"18032","messageId":"Pine.LNX.4.63.0603271326540.19568@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"3728","inReplyTo":"e5bfff550603270319w20796918wc8f8fe30a6c5627@mail.gmail.com","subject":"Re: Following renames","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-03-27T11:30:32Z","receivedAt":"2006-03-27T11:30:32Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 27 Mar 2006, Marco Costalba wrote:\n\n> I still think the problem with annotation is that you don't see\n> patches that _remove_ lines of code, you need the whole diff for this.\n\nInteresting. You'd need a \"git-emalb\" (blame, but reverse), and you'd need \nto tell it a range \"rev1..rev2\" which is *not* to be interpreted as \"^rev1 \nrev2\" but as a direct path from rev1 to rev2.\n\nCiao,\nDscho\n"},{"id":"18033","messageId":"e5bfff550603270355s4b71c306hb4cb2b96eafd0f6e@mail.gmail.com","threadId":"3728","inReplyTo":"Pine.LNX.4.64.0603270005330.15714@g5.osdl.org","subject":"Re: Following renames","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-03-27T11:55:32Z","receivedAt":"2006-03-27T11:55:32Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 3/27/06, Linus Torvalds wrote:\n>\n> In contrast, git-whatchanged will start outputting the recent changes\n> immediately.\n>\n\nTo integrate git-whatchanged like functionality with filter on a\nspecific code region, the Linus original request, I am wondering about\nsomething like this:\n\nA new git-diff-tree option --range=a..b to delimit a region,\nidentified by code lines bounduaries.\n\nAs example\n\ngit-diff-tree --range=10..15 HEAD -- \n\nCoud give these answers, added to standard git-diff-tree output:\n\n* 10..25 --> modified region new region bounduaries are lines from 10 to 25\n\n 15..20 --> region _NOT_ modified but new region bounduaries are\nlines from 15 to 20 (perhaps patch added 5 lines _before_ the region)\n\n 10..15 ---> region _NOT_ modified and lines, if any, added/removed \n_after_ the region\n\n* 10..15 --> modified region with the same boundiaries (as example\nremoving trailing witespaces)\n\nWith this new option of git-diff-tree becames very simple to retrieve\na file history limited to a region, because the region bounduaries in\nouput from one rev are feed as input in parent rev.\n\nComments?\n\nMarco\n"},{"id":"18034","messageId":"4427DA3B.3070603@op5.se","threadId":"3728","inReplyTo":"e5bfff550603270355s4b71c306hb4cb2b96eafd0f6e@mail.gmail.com","subject":"Re: Following renames","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2006-03-27T12:27:39Z","receivedAt":"2006-03-27T12:27:39Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Marco Costalba wrote:\n> On 3/27/06, Linus Torvalds wrote:\n> \n>>In contrast, git-whatchanged will start outputting the recent changes\n>>immediately.\n>>\n> \n> \n> To integrate git-whatchanged like functionality with filter on a\n> specific code region, the Linus original request, I am wondering about\n> something like this:\n> \n> A new git-diff-tree option --range=a..b to delimit a region,\n> identified by code lines bounduaries.\n> \n\nMake it --line-range if you implement this. On a first glance I thought \nyou meant --commit-range.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n"},{"id":"18035","messageId":"Pine.LNX.4.64.0603270848200.15714@g5.osdl.org","threadId":"3728","inReplyTo":"e5bfff550603270319w20796918wc8f8fe30a6c5627@mail.gmail.com","subject":"Re: Following renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-27T16:52:52Z","receivedAt":"2006-03-27T16:52:52Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 27 Mar 2006, Marco Costalba wrote:\n> >\n> > And that's the point. Almost always, we're interested in the _recent_\n> > stuff. The fact that it takes longer to get the old history is not very\n> > important. You generally don't ask \"what changed in this file\" for a file\n> > that hasn't changed in five years.\n> \n> We could run git-rev-list with a time range specifier (changes of last\n> year as example) by default so to have fast results and run all time\n> history _only_ on request.\n\nYes.\n\nHowever, what I've been meaning to do (but just haven't had the time and \nenergy for so far) is to fix \"git-rev-list\" with a path limiter.\n\nRight now that always causes things to be totally serialized, and the \nrevision walking will first look up _all_ the history (well, it will prune \nout the merges) before starting to output stuff.\n\nSo right now in order for \"git-whatchanged\" to be fast and incremental, it \ndoesn't do any path limiting with git-rev-list at ALL, and does it all in \ngit-diff-tree. Which is horrid.\n\n> I still think the problem with annotation is that you don't see\n> patches that _remove_ lines of code, you need the whole diff for this.\n\nWell, that's just another reason \"annotate\" sucks.\n\nIf you select a range of lines, my suggested tool _would_ show you lines \nthat got removed there, and git-whatchanged does it quite well.\n\nI really think \"annotate\" is _fundamentally_ a broken operation. It's not \nwhat any sane developer actually wants, and it has serious limitations (ie \nit depends on whole history, and it cannot show removals well).\n\n\t\tLinus\n"},{"id":"18039","messageId":"20060327215956.GB27689@pasky.or.cz","threadId":"3728","inReplyTo":"20060326232649.GV18185@pasky.or.cz","subject":"Re: Following renames","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-03-27T21:59:56Z","receivedAt":"2006-03-27T21:59:56Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Mon, Mar 27, 2006 at 01:26:49AM CEST, I got a letter\nwhere Petr Baudis said that...\n> To quickly see what it does, you can try it e.g. on the git-log.sh file\n> in the Git repository.\n\nBy the way, the cg-log in master uses it now to automagically follow\nfile renames (in case you call it per-file like cg-log FILENAME). If you\nhate it, you can prevent it by cg-log --no-renames (cg-log -R).\n\nLooks pretty slick.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nRight now I am having amnesia and deja-vu at the same time. I think\nI have forgotten this before.\n"}]}