{"thread":{"id":"44909","subject":"Git: new feature suggestion","startedAt":"2017-01-18T10:48:07Z","lastAt":"2017-01-20T11:19:25Z","messageCount":15,"participants":["Joao Pinto","Stefan Beller","Konstantin Khomoutov","Junio C Hamano","Linus Torvalds","Jakub Narębski"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"309616","messageId":"4817eb00-6efc-e3c0-53d7-46f2509350d3@synopsys.com","threadId":"44909","inReplyTo":null,"subject":"Git: new feature suggestion","fromName":"Joao Pinto","fromEmail":"joao.pinto@synopsys.com","sentAt":"2017-01-18T10:40:52Z","receivedAt":"2017-01-18T10:48:07Z","isPatch":false,"sender":{"key":"joao.pinto@synopsys.com","avatar":"https://gravatar.com/avatar/96931bafcad4d7d044bc9b224a20ef0c71c6b6acc6803ca736b09b4ba2a6aabb?d=mp&s=160"},"body":"Hello,\n\nMy name is Joao Pinto, I work at Synopsys and I am a frequent Linux Kernel\ncontributor.\n\nLet me start by congratulate you for the fantastic work you have been doing with\nGit which is an excellent tool.\n\nThe Linux Kernel as all systems needs to be improved and re-organized to be\nbetter prepared for future development and sometimes we need to change\nfolder/files names or even move things around.\nI have seen a lot of Linux developers avoid this re-organization operations\nbecause they would lose the renamed file history, because a new log is created\nfor the new file, even if it is a renamed version of itself.\nI am sending you this e-mail to suggest the creation of a new feature in Git:\nwhen renamed, a file or folder should inherit his parent’s log and a “rename: …”\nwould be automatically created or have some kind of pointer to its “old” form to\nmake history analysis easier.\n\nI volunteer to help in the new feature if you find it useful. I think it would\nimprove log history analysis and would enable developers to better organize old\ncode.\n\nThank you for your attention.\n\nBest Regards,\nJoao Pinto\n"},{"id":"309642","messageId":"CAGZ79kYXQcUB+rVkboY9fMqu6R3RoHEJ7BTJn_+-RScFDjEduA@mail.gmail.com","threadId":"44909","inReplyTo":"4817eb00-6efc-e3c0-53d7-46f2509350d3@synopsys.com","subject":"Re: Git: new feature suggestion","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-01-18T18:50:40Z","receivedAt":"2017-01-18T18:52:01Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jan 18, 2017 at 2:40 AM, Joao Pinto wrote:\n> Hello,\n>\n> My name is Joao Pinto, I work at Synopsys and I am a frequent Linux Kernel\n> contributor.\n>\n> Let me start by congratulate you for the fantastic work you have been doing with\n> Git which is an excellent tool.\n>\n> The Linux Kernel as all systems needs to be improved and re-organized to be\n> better prepared for future development and sometimes we need to change\n> folder/files names or even move things around.\n> I have seen a lot of Linux developers avoid this re-organization operations\n> because they would lose the renamed file history, because a new log is created\n> for the new file, even if it is a renamed version of itself.\n\nWell there are a couple of things to help with digging in the logs.\n\ngit log:\n --follow\n Continue listing the history of a file beyond renames (works only\n for a single file).\n\n -M[], --find-renames[=]\n If generating diffs, detect and report renames for each commit. For\n following files across renames while traversing history, see\n --follow. If n is specified, it is a threshold on the similarity\n index (i.e. amount of addition/deletions compared to the file’s\n size). For example, -M90% means Git should consider a delete/add\n pair to be a rename if more than 90% of the file hasn’t changed.\n Without a % sign, the number is to be read as a fraction, with a\n decimal point before it. I.e., -M5 becomes 0.5, and is thus the\n same as -M50%. Similarly, -M05 is the same as -M5%. To limit\n detection to exact renames, use -M100%. The default similarity\n index is 50%.\n\n -C[], --find-copies[=]\n Detect copies as well as renames. See also --find-copies-harder. If\n n is specified, it has the same meaning as for -M.\n\n\n\n> I am sending you this e-mail to suggest the creation of a new feature in Git:\n> when renamed, a file or folder should inherit his parent’s log and a “rename: …”\n> would be automatically created or have some kind of pointer to its “old” form to\n> make history analysis easier.\n\nHow do you currently analyse history, which detailed feature is missing?\n\nMind that in the Git data model we deliberately do not record the rename\nat commit time, but rather want to identify the renames at log time.\nThis is because\nin the meantime between commit and log viewing someone could have written\na better rename detection, whereas at commit time we'd be stuck with ancient\ncruft forever. ;)\n\n>\n> I volunteer to help in the new feature if you find it useful. I think it would\n> improve log history analysis and would enable developers to better organize old\n> code.\n\nIMHO complete renames (i.e. git mv path/a/file.c path/b/thing.c) are already\ncovered quite well. Partial rename (e.g. moving code from one file into two\nseparate files or vice versa) is still a bit hard.\n\nI started such a new feature, see\nhttps://public-inbox.org/git/20160903033120.20511-1-sbeller@google.com/\nlatest code is at https://github.com/stefanbeller/git/commits/colored_diff12,\nbut the latest two commits are bogus and need rewriting.\n\nI think this feature is not 100% what you are aiming at, but is very close.\n\nThanks,\nStefan\n"},{"id":"309647","messageId":"21caed00-d103-5534-156c-0c19f25e0879@synopsys.com","threadId":"44909","inReplyTo":"CAGZ79kYXQcUB+rVkboY9fMqu6R3RoHEJ7BTJn_+-RScFDjEduA@mail.gmail.com","subject":"Re: Git: new feature suggestion","fromName":"Joao Pinto","fromEmail":"joao.pinto@synopsys.com","sentAt":"2017-01-18T19:04:43Z","receivedAt":"2017-01-18T19:14:53Z","isPatch":false,"sender":{"key":"joao.pinto@synopsys.com","avatar":"https://gravatar.com/avatar/96931bafcad4d7d044bc9b224a20ef0c71c6b6acc6803ca736b09b4ba2a6aabb?d=mp&s=160"},"body":"\nHi Stefan,\n\nÀs 6:50 PM de 1/18/2017, Stefan Beller escreveu:\n> On Wed, Jan 18, 2017 at 2:40 AM, Joao Pinto wrote:\n>> Hello,\n>>\n>> My name is Joao Pinto, I work at Synopsys and I am a frequent Linux Kernel\n>> contributor.\n>>\n>> Let me start by congratulate you for the fantastic work you have been doing with\n>> Git which is an excellent tool.\n>>\n>> The Linux Kernel as all systems needs to be improved and re-organized to be\n>> better prepared for future development and sometimes we need to change\n>> folder/files names or even move things around.\n>> I have seen a lot of Linux developers avoid this re-organization operations\n>> because they would lose the renamed file history, because a new log is created\n>> for the new file, even if it is a renamed version of itself.\n> \n> Well there are a couple of things to help with digging in the logs.\n> \n> git log:\n> --follow\n> Continue listing the history of a file beyond renames (works only\n> for a single file).\n> \n> -M[], --find-renames[=]\n> If generating diffs, detect and report renames for each commit. For\n> following files across renames while traversing history, see\n> --follow. If n is specified, it is a threshold on the similarity\n> index (i.e. amount of addition/deletions compared to the file’s\n> size). For example, -M90% means Git should consider a delete/add\n> pair to be a rename if more than 90% of the file hasn’t changed.\n> Without a % sign, the number is to be read as a fraction, with a\n> decimal point before it. I.e., -M5 becomes 0.5, and is thus the\n> same as -M50%. Similarly, -M05 is the same as -M5%. To limit\n> detection to exact renames, use -M100%. The default similarity\n> index is 50%.\n> \n> -C[], --find-copies[=]\n> Detect copies as well as renames. See also --find-copies-harder. If\n> n is specified, it has the same meaning as for -M.\n> \n> \n> \n>> I am sending you this e-mail to suggest the creation of a new feature in Git:\n>> when renamed, a file or folder should inherit his parent’s log and a “rename: …”\n>> would be automatically created or have some kind of pointer to its “old” form to\n>> make history analysis easier.\n> \n> How do you currently analyse history, which detailed feature is missing?\n> \n> Mind that in the Git data model we deliberately do not record the rename\n> at commit time, but rather want to identify the renames at log time.\n> This is because\n> in the meantime between commit and log viewing someone could have written\n> a better rename detection, whereas at commit time we'd be stuck with ancient\n> cruft forever. ;)\n> \n>>\n>> I volunteer to help in the new feature if you find it useful. I think it would\n>> improve log history analysis and would enable developers to better organize old\n>> code.\n> \n> IMHO complete renames (i.e. git mv path/a/file.c path/b/thing.c) are already\n> covered quite well. Partial rename (e.g. moving code from one file into two\n> separate files or vice versa) is still a bit hard.\n> \n> I started such a new feature, see\n> https://urldefense.proofpoint.com/v2/url?u=https-3A__public-2Dinbox.org_git_20160903033120.20511-2D1-2Dsbeller-40google.com_&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=BseICq5hy9UHxmX2XP8oPYLbn-HoEUlEuVUzqPHkX58&s=PybtKK0ELH3Nld_CQSYZnLqCQOWvnU4Fjj5iV_7EKqE&e= \n> latest code is at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_stefanbeller_git_commits_colored-5Fdiff12&d=DwIFaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=BseICq5hy9UHxmX2XP8oPYLbn-HoEUlEuVUzqPHkX58&s=pkTehcEmeHVLHdcNbUiU03meyH10cgUbGqLgOqXcL6w&e= ,\n> but the latest two commits are bogus and need rewriting.\n> \n> I think this feature is not 100% what you are aiming at, but is very close.\n> \n> Thanks,\n> Stefan\n> \n\nGreat info, helps a lot! I am going to analyse and get back to you ASAP.\n\nThanks\n\n"},{"id":"309712","messageId":"20170119093313.ea57832dfd1bc7e0b0f1e630@domain007.com","threadId":"44909","inReplyTo":"4817eb00-6efc-e3c0-53d7-46f2509350d3@synopsys.com","subject":"Re: Git: new feature suggestion","fromName":"Konstantin Khomoutov","fromEmail":"kostix+git@007spb.ru","sentAt":"2017-01-19T06:33:13Z","receivedAt":"2017-01-19T06:33:27Z","isPatch":false,"sender":{"key":"kostix+git@007spb.ru","avatar":null},"body":"On Wed, 18 Jan 2017 10:40:52 +0000\nJoao Pinto wrote:\n\n[...]\n> I have seen a lot of Linux developers avoid this re-organization\n> operations because they would lose the renamed file history, because\n> a new log is created for the new file, even if it is a renamed\n> version of itself. I am sending you this e-mail to suggest the\n> creation of a new feature in Git: when renamed, a file or folder\n> should inherit his parent’s log and a “rename: …” would be\n> automatically created or have some kind of pointer to its “old” form\n> to make history analysis easier.\n\nGit does not record renames because of its stance that what matters is\ncode _of the whole project_ as opposed to its location in a particular\nfile.\n\nHence with regard to renames Git \"works backwards\" by detecting them\ndynamically while traversing the history (such as with `git log`\netc). This detection uses certain heuristics which can be controlled\nwith knobs pointed to by Stefan Beller.\n\nStill, I welcome you to read the sort-of \"reference\" post by Linus\nTorvalds [1] in which he explains the reasoning behind this approach\nimplemented in Git. IMO, understanding the reasoning behind the idea\nis much better than just mechanically learning how to use it.\n\nThe whole thread (esp. Torvalds' replies) is worth reading, but that\nparticular mail summarizes the whole thing very well.\n\n(The reference link to it used to be [2], but Gmane is not fully\nrecovered to be able to display it.)\n\n1. http://public-inbox.org/git/Pine.LNX.4.58.0504150753440.7211@ppc970.osdl.org/\n2. http://thread.gmane.org/gmane.comp.version-control.git/27/focus=217\n"},{"id":"309744","messageId":"xmqqa8am3oee.fsf@gitster.mtv.corp.google.com","threadId":"44909","inReplyTo":"20170119093313.ea57832dfd1bc7e0b0f1e630@domain007.com","subject":"Re: Git: new feature suggestion","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-01-19T18:17:29Z","receivedAt":"2017-01-19T18:18:34Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Konstantin Khomoutov writes:\n\n> Still, I welcome you to read the sort-of \"reference\" post by Linus\n> Torvalds [1] in which he explains the reasoning behind this approach\n> implemented in Git. IMO, understanding the reasoning behind the idea\n> is much better than just mechanically learning how to use it.\n>\n> The whole thread (esp. Torvalds' replies) is worth reading, but that\n> particular mail summarizes the whole thing very well.\n>\n> (The reference link to it used to be [2], but Gmane is not fully\n> recovered to be able to display it.)\n>\n> 1. http://public-inbox.org/git/Pine.LNX.4.58.0504150753440.7211@ppc970.osdl.org/\n> 2. http://thread.gmane.org/gmane.comp.version-control.git/27/focus=217\n\nIndeed. Thanks for providing a link to it here ;-)\n\nThe message is the most important one in the early history of Git,\nand it still is one of the most important messages in the Git\nmailing-list archive. \"git log -S\" was designed to take a\nblock of text (even though people misuse it and feed a single line\nto it) exactly because it wanted to serve the \"tracking when that\nfile+line changed\" part in that vision. The rename detection in\n\"diff\" was meant to be used on the commit \"git log -S\" finds\nto see if the found change came from another file so that the user\ncan decide that \"digging further\" part needs to be done for another\nfile. \"git blame\" with -M and -C options were done to mostly\nautomate the \"drilling down\" process that finds the last commit that\ntouched each line in the above process, and when used with tools\nlike \"tig\", you can even peel one commit back and \"zoom down\" if the\nfound commit is an uninteresting one (e.g. a change with only code\nformatting).\n\nOne thing that is still missing in the current version of Git,\ncompared to the \"ideal SCM\" the message envisioned, is the part that\nnotices: \"oops, that line didn't even exist in the previous version,\nBUT I FOUND FIVE PLACES that matched almost perfectly in the same\ndiff, and here they are\".\n\n"},{"id":"309751","messageId":"d43abe2b-cd6a-9b08-272f-9dddbb8eccea@synopsys.com","threadId":"44909","inReplyTo":"20170119093313.ea57832dfd1bc7e0b0f1e630@domain007.com","subject":"Re: Git: new feature suggestion","fromName":"Joao Pinto","fromEmail":"joao.pinto@synopsys.com","sentAt":"2017-01-19T17:55:33Z","receivedAt":"2017-01-19T18:39:00Z","isPatch":false,"sender":{"key":"joao.pinto@synopsys.com","avatar":"https://gravatar.com/avatar/96931bafcad4d7d044bc9b224a20ef0c71c6b6acc6803ca736b09b4ba2a6aabb?d=mp&s=160"},"body":"\nHi,\n\nÀs 6:33 AM de 1/19/2017, Konstantin Khomoutov escreveu:\n> On Wed, 18 Jan 2017 10:40:52 +0000\n> Joao Pinto wrote:\n> \n> [...]\n>> I have seen a lot of Linux developers avoid this re-organization\n>> operations because they would lose the renamed file history, because\n>> a new log is created for the new file, even if it is a renamed\n>> version of itself. I am sending you this e-mail to suggest the\n>> creation of a new feature in Git: when renamed, a file or folder\n>> should inherit his parent’s log and a “rename: …” would be\n>> automatically created or have some kind of pointer to its “old” form\n>> to make history analysis easier.\n> \n> Git does not record renames because of its stance that what matters is\n> code _of the whole project_ as opposed to its location in a particular\n> file.\n> \n> Hence with regard to renames Git \"works backwards\" by detecting them\n> dynamically while traversing the history (such as with `git log`\n> etc). This detection uses certain heuristics which can be controlled\n> with knobs pointed to by Stefan Beller.\n> \n> Still, I welcome you to read the sort-of \"reference\" post by Linus\n> Torvalds [1] in which he explains the reasoning behind this approach\n> implemented in Git. IMO, understanding the reasoning behind the idea\n> is much better than just mechanically learning how to use it.\n> \n> The whole thread (esp. Torvalds' replies) is worth reading, but that\n> particular mail summarizes the whole thing very well.\n> \n> (The reference link to it used to be [2], but Gmane is not fully\n> recovered to be able to display it.)\n> \n> 1. https://urldefense.proofpoint.com/v2/url?u=http-3A__public-2Dinbox.org_git_Pine.LNX.4.58.0504150753440.7211-40ppc970.osdl.org_&d=DwIDaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=X0bQCOGTuZF-uq6smPwJDw4Q47qHgjWaewgTHCbhMnM&s=97U97toe9A6XOAJxbhxvWeYpzl-wPw9QvlhQfAEUTdI&e= \n> 2. https://urldefense.proofpoint.com/v2/url?u=http-3A__thread.gmane.org_gmane.comp.version-2Dcontrol.git_27_focus-3D217&d=DwIDaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=s2fO0hii0OGNOv9qQy_HRXy-xAJUD1NNoEcc3io_kx0&m=X0bQCOGTuZF-uq6smPwJDw4Q47qHgjWaewgTHCbhMnM&s=agYFOBCbLeaKAB6frWWzcwHkZyrMZLW4ExgDxzQyVlI&e= \n> \n\nThank you very much for the info!\n\nJoao\n"},{"id":"309752","messageId":"CA+55aFxAe8bH2xXkx1p5gYN+nc-D-vjNnfUeA_64Q3ttpbHq+w@mail.gmail.com","threadId":"44909","inReplyTo":"20170119093313.ea57832dfd1bc7e0b0f1e630@domain007.com","subject":"Re: Git: new feature suggestion","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2017-01-19T18:39:38Z","receivedAt":"2017-01-19T18:40:28Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Wed, Jan 18, 2017 at 10:33 PM, Konstantin Khomoutov\n wrote:\n>\n> Still, I welcome you to read the sort-of \"reference\" post by Linus\n> Torvalds [1] in which he explains the reasoning behind this approach\n> implemented in Git.\n\nIt's worth noting that that discussion was from some _very_ early days\nin git (one week into the whole thing), when none of those\nvisualization tools were actually implemented.\n\nEven now, ten years after the fact, plain git doesn't actually do what\nI outlined. Yes, \"git blame -Cw\" works fairly well, and is in general\nbetter than the traditional per-file \"annotate\". And yes, \"git log\n--follow\" does another (small) part of the outlined thing, but is\nreally not very powerful.\n\nSome tools on top of git do more, but I think in general this is an\narea that could easily be improved upon. For example, the whole\niterative and interactive drilling down in history of a particular\nfile is very inconvenient to do with \"git blame\" (you find a commit\nthat change the area in some way that you don't find interesting, so\nthen you have to restart git blame with the parent of that\nunintersting commit).\n\nYou can do it in tig, but I suspect a more graphical tool might be better.\n\n.. and we still end up having a lot of things where we simply just\nwork with pathnames. For example, when doing merges, it' absolutely\n_wonderful_ doing\n\n gitk --merge \n\nto see what happened to that filename that has a conflict during the\nmerge. But it's all based on the whole-file changes, and sometimes\nyou'd like to see just the commits that generate one particular\nconflict (in the kernel, things like the MAINTAINERS file can have\nquite a lot of changes, but they are all pretty idnependent, and what\nyou want to see is just \"changes to this area\").\n\nWe do have the \"-L\" flag to git log, but it doesn't actually work for\nthis particular case because of limitations.\n\nSo what I'm trying to say is that the argument from 10+ years ago that\n\"you can do better with intelligent tools after-the-fact\" is very much\ntrue, but it's also true that we don't actually have all those\nintelligent tools, and this is an area that could still be improved\nupon. Some of them are actually available as add-ons in various\ngraphical IDE's that use git.\n\n Linus\n"},{"id":"309759","messageId":"CA+55aFzGaxhRRHXUcfnUDcgyaAKy4jXLcKMXH8T61x8sxEJT+g@mail.gmail.com","threadId":"44909","inReplyTo":"991ef396-3fc3-27d6-283c-b8dffa10a7b7@synopsys.com","subject":"Re: Git: new feature suggestion","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2017-01-19T19:16:47Z","receivedAt":"2017-01-19T19:16:58Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Thu, Jan 19, 2017 at 10:54 AM, Joao Pinto wrote:\n>\n> I am currently facing some challenges in one of Linux subsystems where a rename\n> of a set of folders and files would be the perfect scenario for future\n> development, but the suggestion is not accepted, not because it's not correct,\n> but because it makes the maintainer life harder in backporting bug fixes and new\n> features to older kernel versions and because it is not easy to follow the\n> renamed file/folder history from the kernel.org history logs.\n\nHonestly, that's less of a git issue, and more of a \"patch will not\napply across versions\" issue.\n\nNo amount of rename detection will ever fix that, simply because the\nrename hadn't even _happened_ in the old versions that things get\nbackported to.\n\n(\"git cherry-pick\" can do a merge resolution and thus do \"backwards\"\nrenaming too, so tooling can definitely help, but it still ends up\nmeaning that even trivial patches are no longer the _same_ trivial\npatch across versions).\n\nSo renaming things increases maintainer workloads in those situations\nregardless of any tooling issues.\n\n(You may also be referring to the mellanox mess, where this issue is\nvery much exacerbated by having different groups working on the same\nthing, and maintainers having very much a \"I will not take _anything_\nfrom any of the groups that makes my life more complicated\" model,\nbecause those groups fucked up so much in the past).\n\nIn other words, quite often issues are about workflows rather than\ntools. The networking layer probably has more of this, because David\nactually does the backports himself, so he _really_ doesn't want to\ncomplicate things.\n\n Linus\n"},{"id":"309760","messageId":"991ef396-3fc3-27d6-283c-b8dffa10a7b7@synopsys.com","threadId":"44909","inReplyTo":"CA+55aFxAe8bH2xXkx1p5gYN+nc-D-vjNnfUeA_64Q3ttpbHq+w@mail.gmail.com","subject":"Re: Git: new feature suggestion","fromName":"Joao Pinto","fromEmail":"joao.pinto@synopsys.com","sentAt":"2017-01-19T18:54:41Z","receivedAt":"2017-01-19T19:19:57Z","isPatch":false,"sender":{"key":"joao.pinto@synopsys.com","avatar":"https://gravatar.com/avatar/96931bafcad4d7d044bc9b224a20ef0c71c6b6acc6803ca736b09b4ba2a6aabb?d=mp&s=160"},"body":"\nHi Linus,\n\nÀs 6:39 PM de 1/19/2017, Linus Torvalds escreveu:\n> On Wed, Jan 18, 2017 at 10:33 PM, Konstantin Khomoutov\n> wrote:\n>>\n>> Still, I welcome you to read the sort-of \"reference\" post by Linus\n>> Torvalds [1] in which he explains the reasoning behind this approach\n>> implemented in Git.\n> \n> It's worth noting that that discussion was from some _very_ early days\n> in git (one week into the whole thing), when none of those\n> visualization tools were actually implemented.\n> \n> Even now, ten years after the fact, plain git doesn't actually do what\n> I outlined. Yes, \"git blame -Cw\" works fairly well, and is in general\n> better than the traditional per-file \"annotate\". And yes, \"git log\n> --follow\" does another (small) part of the outlined thing, but is\n> really not very powerful.\n> \n> Some tools on top of git do more, but I think in general this is an\n> area that could easily be improved upon. For example, the whole\n> iterative and interactive drilling down in history of a particular\n> file is very inconvenient to do with \"git blame\" (you find a commit\n> that change the area in some way that you don't find interesting, so\n> then you have to restart git blame with the parent of that\n> unintersting commit).\n> \n> You can do it in tig, but I suspect a more graphical tool might be better.\n> \n> .. and we still end up having a lot of things where we simply just\n> work with pathnames. For example, when doing merges, it' absolutely\n> _wonderful_ doing\n> \n> gitk --merge \n> \n> to see what happened to that filename that has a conflict during the\n> merge. But it's all based on the whole-file changes, and sometimes\n> you'd like to see just the commits that generate one particular\n> conflict (in the kernel, things like the MAINTAINERS file can have\n> quite a lot of changes, but they are all pretty idnependent, and what\n> you want to see is just \"changes to this area\").\n> \n> We do have the \"-L\" flag to git log, but it doesn't actually work for\n> this particular case because of limitations.\n> \n> So what I'm trying to say is that the argument from 10+ years ago that\n> \"you can do better with intelligent tools after-the-fact\" is very much\n> true, but it's also true that we don't actually have all those\n> intelligent tools, and this is an area that could still be improved\n> upon. Some of them are actually available as add-ons in various\n> graphical IDE's that use git.\n> \n> Linus\n> \n\nI am currently facing some challenges in one of Linux subsystems where a rename\nof a set of folders and files would be the perfect scenario for future\ndevelopment, but the suggestion is not accepted, not because it's not correct,\nbut because it makes the maintainer life harder in backporting bug fixes and new\nfeatures to older kernel versions and because it is not easy to follow the\nrenamed file/folder history from the kernel.org history logs.\n\nLike nature shows us, the ability to adapt is the key for survival, so Linux\nwould gain a lot with some new features in git that can make maintainers life\neasier. Assisted-backporting would be an excellent feature for them.\n\nDid you ever thought about optimization backport operations through git or by an\nadd-on to it?\n\nI am available to help if this feature makes sense for git users.\n\nThanks,\nJoao\n"},{"id":"309793","messageId":"b96b71b9-f8a2-d039-6e8a-c64e7aac02a0@gmail.com","threadId":"44909","inReplyTo":"CA+55aFxAe8bH2xXkx1p5gYN+nc-D-vjNnfUeA_64Q3ttpbHq+w@mail.gmail.com","subject":"Re: Git: new feature suggestion","fromName":"Jakub Narębski","fromEmail":"jnareb@gmail.com","sentAt":"2017-01-19T21:48:08Z","receivedAt":"2017-01-19T21:48:27Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"W dniu 19.01.2017 o 19:39, Linus Torvalds pisze:\n> On Wed, Jan 18, 2017 at 10:33 PM, Konstantin Khomoutov\n> wrote:\n>>\n>> Still, I welcome you to read the sort-of \"reference\" post by Linus\n>> Torvalds [1] in which he explains the reasoning behind this approach\n>> implemented in Git.\n> \n> It's worth noting that that discussion was from some _very_ early days\n> in git (one week into the whole thing), when none of those\n> visualization tools were actually implemented.\n> \n> Even now, ten years after the fact, plain git doesn't actually do what\n> I outlined. Yes, \"git blame -Cw\" works fairly well, and is in general\n> better than the traditional per-file \"annotate\". And yes, \"git log\n> --follow\" does another (small) part of the outlined thing, but is\n> really not very powerful.\n\nIt is really a pity that \"git log --follow\" is so limited; it's\ndevelopment stopped at early 'good enough' implementation.\n\nFor example \"git log --follow gitweb/gitweb.perl\" would not show\nthe whole history of a file (which was once independent project),\nand \"git log --follow\" doesn't work for directories or multiple\nfiles.\n\n> \n> Some tools on top of git do more, but I think in general this is an\n> area that could easily be improved upon. For example, the whole\n> iterative and interactive drilling down in history of a particular\n> file is very inconvenient to do with \"git blame\" (you find a commit\n> that change the area in some way that you don't find interesting, so\n> then you have to restart git blame with the parent of that\n> unintersting commit).\n> \n> You can do it in tig, but I suspect a more graphical tool might be better.\n\nWell, we do have \"git gui blame\".\n\n[...]\n-- \nJakub Narębski\n"},{"id":"309795","messageId":"5207b04e-5e80-7100-4328-7e87ee619aea@synopsys.com","threadId":"44909","inReplyTo":"CA+55aFzGaxhRRHXUcfnUDcgyaAKy4jXLcKMXH8T61x8sxEJT+g@mail.gmail.com","subject":"Re: Git: new feature suggestion","fromName":"Joao Pinto","fromEmail":"joao.pinto@synopsys.com","sentAt":"2017-01-19T21:51:18Z","receivedAt":"2017-01-19T21:51:28Z","isPatch":false,"sender":{"key":"joao.pinto@synopsys.com","avatar":"https://gravatar.com/avatar/96931bafcad4d7d044bc9b224a20ef0c71c6b6acc6803ca736b09b4ba2a6aabb?d=mp&s=160"},"body":"Às 7:16 PM de 1/19/2017, Linus Torvalds escreveu:\n> On Thu, Jan 19, 2017 at 10:54 AM, Joao Pinto wrote:\n>>\n>> I am currently facing some challenges in one of Linux subsystems where a rename\n>> of a set of folders and files would be the perfect scenario for future\n>> development, but the suggestion is not accepted, not because it's not correct,\n>> but because it makes the maintainer life harder in backporting bug fixes and new\n>> features to older kernel versions and because it is not easy to follow the\n>> renamed file/folder history from the kernel.org history logs.\n> \n> Honestly, that's less of a git issue, and more of a \"patch will not\n> apply across versions\" issue.\n> \n> No amount of rename detection will ever fix that, simply because the\n> rename hadn't even _happened_ in the old versions that things get\n> backported to.\n> \n> (\"git cherry-pick\" can do a merge resolution and thus do \"backwards\"\n> renaming too, so tooling can definitely help, but it still ends up\n> meaning that even trivial patches are no longer the _same_ trivial\n> patch across versions).\n> \n> So renaming things increases maintainer workloads in those situations\n> regardless of any tooling issues.\n> \n> (You may also be referring to the mellanox mess, where this issue is\n> very much exacerbated by having different groups working on the same\n> thing, and maintainers having very much a \"I will not take _anything_\n> from any of the groups that makes my life more complicated\" model,\n> because those groups fucked up so much in the past).\n> \n> In other words, quite often issues are about workflows rather than\n> tools. The networking layer probably has more of this, because David\n> actually does the backports himself, so he _really_ doesn't want to\n> complicate things.\n\nI totally understand David' side! Synopsys is a well-known IP Vendor, and for a\nlong time its focus was the IP only. Knowadays the strategy has changed and\nSynopsys is very keen to help in Open Source, namelly Linux, developing the\ndrivers for new IP Cores and participating in the improvement of existing ones.\nI am part of the team that has that job.\n\nIn USB and PCI subystems developers created common Synopsys drivers (focused on\nthe HW IP) and so today they are massively used by all the SoC that use Synopsys\nIP.\n\nIn the network subsystem, there are some drivers that target the same IP but\nwere made by different companies. stmmac is an excelent driver for Synopsys MAC\n10/100/1000/QOS IPs, but there was another driver made by AXIS driver that also\ntargeted the QOS IP. We detected that issue and merged the AXIS specific driver\nops to stmmac, and nowadays, AXIS uses stmmac. So less drivers to maintain!\n\nThe idea that was rejected consisted of renaming stmicro/stmmac to dwc/stmmac\nand to have dwc (designware controllers) as the official driver spot for\nSynopsys Ethernet IPs.\nThere is another example of duplication, which is AMD' and Samsung' XGMAC\ndriver, targeting the same Synopsys XGMAC IP.\n\nI am giving this examples because although the refactor adds work for\nbackporting, it reduces the maintenance since we would have less duplicated\ndrivers as we have today.\n\nThanks,\nJoao\n\n\n> Linus\n> \n\n"},{"id":"309797","messageId":"CAGZ79kZ3g+J5=ZmP8zDCK8zBwMc7SwLdmgyB3Sab8qkTE=enhQ@mail.gmail.com","threadId":"44909","inReplyTo":"5207b04e-5e80-7100-4328-7e87ee619aea@synopsys.com","subject":"Re: Git: new feature suggestion","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-01-19T22:03:47Z","receivedAt":"2017-01-19T22:03:55Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Jan 19, 2017 at 1:51 PM, Joao Pinto wrote:\n> Às 7:16 PM de 1/19/2017, Linus Torvalds escreveu:\n>> On Thu, Jan 19, 2017 at 10:54 AM, Joao Pinto wrote:\n>>>\n>>> I am currently facing some challenges in one of Linux subsystems where a rename\n>>> of a set of folders and files would be the perfect scenario for future\n>>> development, but the suggestion is not accepted, not because it's not correct,\n>>> but because it makes the maintainer life harder in backporting bug fixes and new\n>>> features to older kernel versions and because it is not easy to follow the\n>>> renamed file/folder history from the kernel.org history logs.\n>>\n>> Honestly, that's less of a git issue, and more of a \"patch will not\n>> apply across versions\" issue.\n>>\n>> No amount of rename detection will ever fix that, simply because the\n>> rename hadn't even _happened_ in the old versions that things get\n>> backported to.\n>>\n>> (\"git cherry-pick\" can do a merge resolution and thus do \"backwards\"\n>> renaming too, so tooling can definitely help, but it still ends up\n>> meaning that even trivial patches are no longer the _same_ trivial\n>> patch across versions).\n>>\n>> So renaming things increases maintainer workloads in those situations\n>> regardless of any tooling issues.\n>>\n>> (You may also be referring to the mellanox mess, where this issue is\n>> very much exacerbated by having different groups working on the same\n>> thing, and maintainers having very much a \"I will not take _anything_\n>> from any of the groups that makes my life more complicated\" model,\n>> because those groups fucked up so much in the past).\n>>\n>> In other words, quite often issues are about workflows rather than\n>> tools. The networking layer probably has more of this, because David\n>> actually does the backports himself, so he _really_ doesn't want to\n>> complicate things.\n>\n> I totally understand David' side! Synopsys is a well-known IP Vendor, and for a\n> long time its focus was the IP only. Knowadays the strategy has changed and\n> Synopsys is very keen to help in Open Source, namelly Linux, developing the\n> drivers for new IP Cores and participating in the improvement of existing ones.\n> I am part of the team that has that job.\n>\n> In USB and PCI subystems developers created common Synopsys drivers (focused on\n> the HW IP) and so today they are massively used by all the SoC that use Synopsys\n> IP.\n>\n> In the network subsystem, there are some drivers that target the same IP but\n> were made by different companies. stmmac is an excelent driver for Synopsys MAC\n> 10/100/1000/QOS IPs, but there was another driver made by AXIS driver that also\n> targeted the QOS IP. We detected that issue and merged the AXIS specific driver\n> ops to stmmac, and nowadays, AXIS uses stmmac. So less drivers to maintain!\n>\n> The idea that was rejected consisted of renaming stmicro/stmmac to dwc/stmmac\n> and to have dwc (designware controllers) as the official driver spot for\n> Synopsys Ethernet IPs.\n> There is another example of duplication, which is AMD' and Samsung' XGMAC\n> driver, targeting the same Synopsys XGMAC IP.\n>\n> I am giving this examples because although the refactor adds work for\n> backporting, it reduces the maintenance since we would have less duplicated\n> drivers as we have today.\n\nThis sounds as if the code in question would only receive backports\nfor a specific\ntime (determined by HW lifecycle, maintenance life cycle and such).\n\nSo I wonder if this could be solved by not just renaming but\nadditionally adding a\nsymbolic link, such that the files in question seem to appear twice on\nthe file system.\nThen backports ought to be applicable (hoping git-am doesn't choke on symlinks),\nand after a while once the there no backports any more (due to life\ncycle reasons),\nremove the link?\n\nThis also sounds like a kind of problem, that others have run into before,\nhow did they solve it?\n\nThanks,\nStefan\n\n>\n> Thanks,\n> Joao\n>\n>\n>> Linus\n>>\n>\n"},{"id":"309811","messageId":"CA+55aFz5Rnt8U3bpvgoHQSfjPrnxnMfWUGBbHW2XKiagKXga5w@mail.gmail.com","threadId":"44909","inReplyTo":"b96b71b9-f8a2-d039-6e8a-c64e7aac02a0@gmail.com","subject":"Re: Git: new feature suggestion","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2017-01-20T00:26:41Z","receivedAt":"2017-01-20T00:36:37Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Thu, Jan 19, 2017 at 1:48 PM, Jakub Narębski wrote:\n> W dniu 19.01.2017 o 19:39, Linus Torvalds pisze:\n>>\n>> You can do it in tig, but I suspect a more graphical tool might be better.\n>\n> Well, we do have \"git gui blame\".\n\nDoes that actually work for people? Because it really doesn't for me.\n\nAnd I'm not just talking about the aesthetics of the thing, but the\nwhole experience, and the whole \"dig into parent\" which just gives me\nan error message.\n\n Linus\n"},{"id":"309817","messageId":"0b6e3682-9575-7b4c-6cde-7b914364abfc@synopsys.com","threadId":"44909","inReplyTo":"CAGZ79kZ3g+J5=ZmP8zDCK8zBwMc7SwLdmgyB3Sab8qkTE=enhQ@mail.gmail.com","subject":"Re: Git: new feature suggestion","fromName":"Joao Pinto","fromEmail":"joao.pinto@synopsys.com","sentAt":"2017-01-20T10:44:12Z","receivedAt":"2017-01-20T10:55:12Z","isPatch":false,"sender":{"key":"joao.pinto@synopsys.com","avatar":"https://gravatar.com/avatar/96931bafcad4d7d044bc9b224a20ef0c71c6b6acc6803ca736b09b4ba2a6aabb?d=mp&s=160"},"body":"\nHi Stefan,\n\nÀs 10:03 PM de 1/19/2017, Stefan Beller escreveu:\n> On Thu, Jan 19, 2017 at 1:51 PM, Joao Pinto wrote:\n>> Às 7:16 PM de 1/19/2017, Linus Torvalds escreveu:\n>>> On Thu, Jan 19, 2017 at 10:54 AM, Joao Pinto wrote:\n>>>>\n>>>> I am currently facing some challenges in one of Linux subsystems where a rename\n>>>> of a set of folders and files would be the perfect scenario for future\n>>>> development, but the suggestion is not accepted, not because it's not correct,\n>>>> but because it makes the maintainer life harder in backporting bug fixes and new\n>>>> features to older kernel versions and because it is not easy to follow the\n>>>> renamed file/folder history from the kernel.org history logs.\n>>>\n>>> Honestly, that's less of a git issue, and more of a \"patch will not\n>>> apply across versions\" issue.\n>>>\n>>> No amount of rename detection will ever fix that, simply because the\n>>> rename hadn't even _happened_ in the old versions that things get\n>>> backported to.\n>>>\n>>> (\"git cherry-pick\" can do a merge resolution and thus do \"backwards\"\n>>> renaming too, so tooling can definitely help, but it still ends up\n>>> meaning that even trivial patches are no longer the _same_ trivial\n>>> patch across versions).\n>>>\n>>> So renaming things increases maintainer workloads in those situations\n>>> regardless of any tooling issues.\n>>>\n>>> (You may also be referring to the mellanox mess, where this issue is\n>>> very much exacerbated by having different groups working on the same\n>>> thing, and maintainers having very much a \"I will not take _anything_\n>>> from any of the groups that makes my life more complicated\" model,\n>>> because those groups fucked up so much in the past).\n>>>\n>>> In other words, quite often issues are about workflows rather than\n>>> tools. The networking layer probably has more of this, because David\n>>> actually does the backports himself, so he _really_ doesn't want to\n>>> complicate things.\n>>\n>> I totally understand David' side! Synopsys is a well-known IP Vendor, and for a\n>> long time its focus was the IP only. Knowadays the strategy has changed and\n>> Synopsys is very keen to help in Open Source, namelly Linux, developing the\n>> drivers for new IP Cores and participating in the improvement of existing ones.\n>> I am part of the team that has that job.\n>>\n>> In USB and PCI subystems developers created common Synopsys drivers (focused on\n>> the HW IP) and so today they are massively used by all the SoC that use Synopsys\n>> IP.\n>>\n>> In the network subsystem, there are some drivers that target the same IP but\n>> were made by different companies. stmmac is an excelent driver for Synopsys MAC\n>> 10/100/1000/QOS IPs, but there was another driver made by AXIS driver that also\n>> targeted the QOS IP. We detected that issue and merged the AXIS specific driver\n>> ops to stmmac, and nowadays, AXIS uses stmmac. So less drivers to maintain!\n>>\n>> The idea that was rejected consisted of renaming stmicro/stmmac to dwc/stmmac\n>> and to have dwc (designware controllers) as the official driver spot for\n>> Synopsys Ethernet IPs.\n>> There is another example of duplication, which is AMD' and Samsung' XGMAC\n>> driver, targeting the same Synopsys XGMAC IP.\n>>\n>> I am giving this examples because although the refactor adds work for\n>> backporting, it reduces the maintenance since we would have less duplicated\n>> drivers as we have today.\n> \n> This sounds as if the code in question would only receive backports\n> for a specific\n> time (determined by HW lifecycle, maintenance life cycle and such).\n> \n> So I wonder if this could be solved by not just renaming but\n> additionally adding a\n> symbolic link, such that the files in question seem to appear twice on\n> the file system.\n> Then backports ought to be applicable (hoping git-am doesn't choke on symlinks),\n> and after a while once the there no backports any more (due to life\n> cycle reasons),\n> remove the link?\n> \n> This also sounds like a kind of problem, that others have run into before,\n> how did they solve it?\n\nI am currently involved in the PCI host/ refactor process and that will cause a\ntotal reorganization of the folder, because a new PCIe Endpoint is comming up\nfrom Texas Instruments. Bjorn (PCI Maintainer) is ok with it.\nThe network subsystem, is a very busy one, with lots of activity, and so I\nunderstand David Miller' point, because he already has work overload, but we\nhave to find a way to improve it and prepare for the future.\n\nI think this a hot topic, that should be discussed, since it might hold back the\nevolution of some subystems.\n\nThanks,\nJoao\n\n> \n> Thanks,\n> Stefan\n> \n>>\n>> Thanks,\n>> Joao\n>>\n>>\n>>> Linus\n>>>\n>>\n\n"},{"id":"309820","messageId":"e9e988b6-290f-3160-222f-2762865fe508@gmail.com","threadId":"44909","inReplyTo":"CA+55aFz5Rnt8U3bpvgoHQSfjPrnxnMfWUGBbHW2XKiagKXga5w@mail.gmail.com","subject":"Re: Git: new feature suggestion","fromName":"Jakub Narębski","fromEmail":"jnareb@gmail.com","sentAt":"2017-01-20T11:18:32Z","receivedAt":"2017-01-20T11:19:25Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"W dniu 20.01.2017 o 01:26, Linus Torvalds pisze:\n> On Thu, Jan 19, 2017 at 1:48 PM, Jakub Narębski wrote:\n>> W dniu 19.01.2017 o 19:39, Linus Torvalds pisze:\n>>>\n>>> You can do it in tig, but I suspect a more graphical tool might be better.\n>>\n>> Well, we do have \"git gui blame\".\n> \n> Does that actually work for people? Because it really doesn't for me.\n> \n> And I'm not just talking about the aesthetics of the thing, but the\n> whole experience, and the whole \"dig into parent\" which just gives me\n> an error message.\n\nStrange. I had been using \"git gui blame\" _because_ of its \"dig to parent\"\nfunctionality, and it worked for me just fine.\n\nThe other thing that I like about \"git gui blame\" is that it shows both\nthe commit that moved the fragment of code (via \"git blame\"), and the\ncommit that created the fragment of code (via \"git blame -C -C -w\", I think).\n\n\nAnyway, all of this (sub)discussion is about archeology, but what might\nbe more important is automatic rename handling when integrating changes,\nbe it git-am, git-merge, or something else...\n\n-- \nJakub Narębski\n\n"}]}