{"thread":{"id":"3544","subject":"On recording renames","startedAt":"2006-03-04T04:03:32Z","lastAt":"2006-03-04T17:36:09Z","messageCount":5,"participants":["Junio C Hamano","Paul Jakma","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"17131","messageId":"7vfylyx3e3.fsf@assigned-by-dhcp.cox.net","threadId":"3544","inReplyTo":null,"subject":"On recording renames","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-03-04T04:03:32Z","receivedAt":"2006-03-04T04:03:32Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Recently some people said \"I want to tell git that I renamed\nfileA to fileB\" on the list and #git channel, and I made some\nvague comments on the issue that might have confused people.\nThis message is to clarify where I stand.\n\nFirst of all, I understand some people \"want to tell git\", but\nat the same time, I know very well that \"git does not even want\nto hear\".  It does not care about names -- it only tracks\ncontents [*1*].\n\nHaving said that, there are at least two cases that renamed\nfiles matter in practice from the end user's point of view.\nDiff and merge.\n\nFor diff, it is often __very__ frustrating when you know you\nrenamed hello.c to world.c and then edited just a bit and \"git\ndiff -M hello.c world.c\" does not notice.  You can do one of two\nthings to help:\n\n (1) figure out why git (diffcore-rename) does not think they\n     are similar enough, and improve its similarity estimator.\n     Some of you who are paying attention to what is in my\n     \"next\" branch might have noticed that I have been working\n     in this area recently.\n\n (2) add a way to tell git-diff-files to compare hello.c in the\n     index with working tree file world.c:\n\n     \t$ git-diff-files -p 'hello.c->world.c'\n\nAnd people who \"want to tell git\" are after the second way.\nAlthough this can probably be implemented as an extension to\ndiffcore-rename [*2*], I have to say that is kludging around the\nreal problem.  Only as a workaround for pathological cases it\nmay be OK, but I am really reluctant to accept such a change\nwithout trying avenue (1) first.\n\nAbout merge.  Currently recursive merge strategy claims to\nhandle renames and I've seen it handle renames well in some\ncases.  However, it only uses three trees.  The rename between\nmerge base and one head, and the rename between merge base and\nthe other head are computed, compared and usual three-way merge\nrules are applied (e.g. if you kept it there while she moved it\nto somewhere else, result is to move it to somewhere else).  If\ntwo development tracks forked long time ago are being merged,\nand corresponding files deviated from each other beyond\nrecognition, there is no way for any heuristics to figure out\none is a rename-edit of the other only by looking at these three\ntrees:\n\n       a1--a2--a3--a4--a5--A\n      /                     \\\n  ---O---b1--b2--b3--b4--B---* \n\n    O has hello.c\n    a1 renames file hello.c to world.c and a2-a5-A modifies world.c\n    b1-b4-B modifies hello.c \n    we are about to merge A and B\n\n    comparing O and A may not notice O's hello.c and A's world.c\n    are similar!\n\nBut you are allowed to write a new merge strategy that is more\ncareful about renames.  There is no reason you can only look at\nthree trees.  Such a merge strategy, when given commit A and B,\nwould walk the history back, running \"diff-tree -M\" for each\ncommit along the way, and difference between O's hello.c and\na1's world.c would be hopefully *much* smaller than O's hello.c\nand A's world.c -- even the current similarity estimator may\nrecognize it is a rename.\n\nThat is the first thing I'd like to see.  I do not want to even\nthink about recording renames in commit objects before anybody\nexplores that avenue.\n\nEven with that, if O's hello.c and a1's world.c are _so_\ndifferent that if the changes are beyond recognition, you\n_might_ want to \"tell git\" about the rename, or even record such\na rename in the commit object a1.  But I personally doubt it\nwould help anything in practice.  After such a huge rewrite\nbetween O->a1, merging between A and B will be very hard anyway,\nand you would need some off-line method to extract the intention\nof the developer who originally did a1 commit while merging A\nand B.  And when you inspect that change yourself, you may\ndecide O's hello.c correspond to a1's world.c yourself.  At that\npoint you will be hand merging the mess, so your being able to tell\ngit about it would not help you much.\n\n\n[Footnotes]\n\n*1* This is by design, and I am not going to debate if that is a\ngood design or not here.  There is a \"Linus once said 'you say\nusers know better but users cannot be trusted -- trust me'\"\nfactor involved.  I am a trusting kind and somebody needs to\nconvince me not to trust Linus.\n\n*2* You would supply \"in what you are comparing, the source path\nhello.c and destination path world.c are similar with similarity\nindex 80% -- do not bother to estimate yourself, I am telling\nyou their similarity index so trust me\".\n\n\t$ git-diff-files -p -M --similarity='hello.c world.c 80%'\n"},{"id":"17132","messageId":"Pine.LNX.4.64.0603040405260.13612@sheen.jakma.org","threadId":"3544","inReplyTo":"7vfylyx3e3.fsf@assigned-by-dhcp.cox.net","subject":"Re: On recording renames","fromName":"Paul Jakma","fromEmail":"paul@clubi.ie","sentAt":"2006-03-04T04:09:44Z","receivedAt":"2006-03-04T04:09:44Z","isPatch":false,"sender":{"key":"paul@clubi.ie","avatar":null},"body":"Hi Junio,\n\nOn Fri, 3 Mar 2006, Junio C Hamano wrote:\n\n> Recently some people said \"I want to tell git that I renamed fileA \n> to fileB\" on the list and #git channel,\n\n> *1* This is by design, and I am not going to debate if that is a\n> good design or not here.\n\nThanks for your detailed email. Before I continue digesting it, I'd \nlike to revise my original proposal (having now read the \nmore of the 'design philosophy'):\n\n- I want to tell git that object A is related to object B between two\n   trees.\n\nI think at this stage a proof-of-concept would be an idea. I'll try \nget back with that before end of the month.\n\nregards,\n-- \nPaul Jakma\tpaul@clubi.ie\tpaul@jakma.org\tKey ID: 64A2FF6A\nFortune:\nwe just switched to Sprint.\n"},{"id":"17137","messageId":"7v64muvin9.fsf@assigned-by-dhcp.cox.net","threadId":"3544","inReplyTo":"7vfylyx3e3.fsf@assigned-by-dhcp.cox.net","subject":"Re: On recording renames","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-03-04T06:16:58Z","receivedAt":"2006-03-04T06:16:58Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <junkio@cox.net> writes:\n\n>\n>        a1--a2--a3--a4--a5--A\n>       /                     \\\n>   ---O---b1--b2--b3--b4--B---* \n>\n>     O has hello.c\n>     a1 renames file hello.c to world.c and a2-a5-A modifies world.c\n>     b1-b4-B modifies hello.c \n>     we are about to merge A and B\n>\n>     comparing O and A may not notice O's hello.c and A's world.c\n>     are similar!\n>\n> But you are allowed to write a new merge strategy that is more\n> careful about renames.  There is no reason you can only look at\n> three trees.  Such a merge strategy, when given commit A and B,\n> would walk the history back, running \"diff-tree -M\" for each\n> commit along the way, and difference between O's hello.c and\n> a1's world.c would be hopefully *much* smaller than O's hello.c\n> and A's world.c -- even the current similarity estimator may\n> recognize it is a rename.\n\nA bit more on merges and renames.  The thread that started on\nDec 16 2005 by Don Zickus is about a case that anybody\ninterested in renaming merge should think about.  Unfortunately\ngmane web interface says it is \"down for maintenance\" so I\ncannot give an URL, but the message IDs of key messages are:\n\n        <68948ca0512161205x3d5921bfm3bfcaa64f988eb99@mail.gmail.com>\n        <7vbqzgbcyv.fsf@assigned-by-dhcp.cox.net>\n\nThe whole thread is worth reading, but the punch line is:\n\n        The transition happened over time with multiple commits.\n        You cannot record \"this is the rename\" by attributing that\n        information to one particular commit.\n"},{"id":"17143","messageId":"7v8xrqqrdd.fsf@assigned-by-dhcp.cox.net","threadId":"3544","inReplyTo":"7v64muvin9.fsf@assigned-by-dhcp.cox.net","subject":"Re: On recording renames","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-03-04T13:19:42Z","receivedAt":"2006-03-04T13:19:42Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <junkio@cox.net> writes:\n\n> A bit more on merges and renames...\n> The whole thread is worth reading, but the punch line is:\n>\n>         The transition happened over time with multiple commits.\n>         You cannot record \"this is the rename\" by attributing that\n>         information to one particular commit.\n\nAfter re-reading that thread, and especially the analysis of the\nhistory of that part of the kernel source I did back then, I am\nagain convinced that Linus was right when he said \"file renames\ndo not matter\".  That real-life example shows how inadequate\nfile boundaries are when dealing with content changes.\n\nAn ideal merge strategy would handle the case where pieces of\ncode gradually moves around across file boundaries.  I do not\nthink this is something you can sensibly do by recording file\nrename history.  It would not help the situation a bit even if\nyou gave each file (or content or object or whatever you want to\ncall it) a persistent ID.\n\nOne way (now, it is my turn to handwave) to do such a merge\nmight be to take the whole tree as if it were a flat single file\n(think of it as a concatenation of all files in the tree) with\neach line tagged with the pathname.  You and your friend would\nstart from something like this.  A single file that describe\ntopics of interest to both of you:\n\n                    notes.txt:Kernel Topics\n                    notes.txt: - filesystem\n                    notes.txt: - scheduler\n                    notes.txt: - devices\n                    notes.txt:Cool Git Topics\n                    notes.txt: - git-cvsserver\n                    notes.txt: - Cogito\n\nAnd your friend splits this into two files and starts editing,\nwhile you edit the original file:\n\n        your friend                     you\n\n        linux.txt:Kernel Topics         notes.txt:Kernel Topics\n        linux.txt: - filesystem         notes.txt: - filesystem\n        linux.txt: - scheduler          notes.txt: - scheduler\n        linux.txt: - devices            notes.txt: - devices\n        linux.txt: - stable driver API  notes.txt: - mm\n        git.txt:Cool Git Topics         notes.txt:Cool Git Topics\n        git.txt: - git-cvsserver        notes.txt: - git-cvsserver\n        git.txt: - Cogito               notes.txt: - gitview\n                                        notes.txt: - Cogito\n                                        notes.txt: - StGIT\n                                        notes.txt: - diff --cc\n\nNow you would want to compare notes and merge them.  When\ncomparing these two \"trees\", the clever merge algorithm would\ntreat this two-column thingy and merge both labels\n(i.e. pathnames) and contents:\n\n                    linux.txt:Kernel Topics\n                    linux.txt: - filesystem\n                    linux.txt: - scheduler\n                    linux.txt: - devices\n                    linux.txt: - mm\n                    linux.txt: - stable driver API\n                    git.txt:Cool Git Topics\n                    git.txt: - git-cvsserver\n                    git.txt: - gitview\n                    git.txt: - Cogito\n                    git.txt: - StGIT\n                    git.txt: - diff --cc\n\nIt could even guess that the line you touched are related to the\nhunk your friend moved to another file (iow, your friend gave a\nnew label to the region you touched), and label your new line\nwith the same pathname as surrounding lines.\n\nI suspect this is weave merge taken to its extreme, but I am\nhandwaving so please do not ask me how I would propose to\nimplement it ;-).  The point really is that file is a poor unit\nof operation when dealing with changes.\n"},{"id":"17152","messageId":"Pine.LNX.4.64.0603040929540.22647@g5.osdl.org","threadId":"3544","inReplyTo":"7v8xrqqrdd.fsf@assigned-by-dhcp.cox.net","subject":"Re: On recording renames","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-03-04T17:36:09Z","receivedAt":"2006-03-04T17:36:09Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 4 Mar 2006, Junio C Hamano wrote:\n> \n> An ideal merge strategy would handle the case where pieces of\n> code gradually moves around across file boundaries.  I do not\n> think this is something you can sensibly do by recording file\n> rename history.  It would not help the situation a bit even if\n> you gave each file (or content or object or whatever you want to\n> call it) a persistent ID.\n\nActually, we have an absolutely perfect example of this much closer to \nhome.\n\nI originally did the \"rev-list split\" series on an older version of git, \nbefore you did the --objects-edge and the full pathname hashing \nimprovements. But when I was done, you'd merged that, and I needed to \nmerge my rev-list.c split with your improvements in order to send it to \nyou.\n\nNow, the whold file hadn't actually been renamed, but about 50% of that \nfile had been split into a new one. So effectively you had a merge where \npart of the new stuff had to be merged into another file.\n\nNow, I think this is actually more common than renames in many ways. It's \nnot a \"complete\" rename, but as far as _part_ of your changes were\nconcerned, it was one.\n\nAnd yes, such a split can be something that is done in stages, again \nexactly the same way about 85% of rev-list.c was moved into revision.c in \ntwo stages: the first stage was the argument parsing and setup, and the \nsecond stage was the actual revision walking logic.\n\n\t\tLinus\n"}]}