{"thread":{"id":"35737","subject":"git blame vs git log --follow performance","startedAt":"2014-01-26T21:10:52Z","lastAt":"2014-01-27T02:34:23Z","messageCount":4,"participants":["Joe Perches","Duy Nguyen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"233799","messageId":"1390770652.20150.25.camel@joe-AO722","threadId":"35737","inReplyTo":null,"subject":"git blame vs git log --follow performance","fromName":"Joe Perches","fromEmail":"joe@perches.com","sentAt":"2014-01-26T21:10:52Z","receivedAt":"2014-01-26T21:10:52Z","isPatch":false,"sender":{"key":"joe@perches.com","avatar":"https://avatars.githubusercontent.com/u/13122723?v=4"},"body":"Hi.\n\nIs there something that can be done about improving\ngit log --follow -- performance to be nearly\nequivalent speed to git blame -- ?\n\nThe overall cpu time taken for these 2 commands that\ntrack individual file history can be quite different.\n\n\tgit log --follow -- \nand\n\tgit blame -- \n\nIt seems that there can be a couple orders of magnitude\ndelta in the overall time taken.\n\nFor instance (using the Linus' linux kernel git):\n\n$ time git log --follow -- drivers/firmware/google/Kconfig > /dev/null\n\nreal\t0m42.329s\nuser\t0m40.984s\nsys\t0m0.792s\n\n$ time git blame -- drivers/firmware/google/Kconfig > /dev/null\n\nreal\t0m0.963s\nuser\t0m0.860s\nsys\t0m0.096s\n\nThis particular file has never been renamed.\n\nLooking at the output on screen, there does seem to\nbe 25+ seconds of cpu time consumed after the initial\n(last shown) commit that introduces this file.\n\nPerhaps adding a whole-file rename option to the\n\"git log\" history simplification mechanism could\nhelp?\n\nThoughts?\n"},{"id":"233808","messageId":"CACsJy8B4dkjae3tL+_JvD63HCoi_-nNWYGvT3xRTqi8UpYX95g@mail.gmail.com","threadId":"35737","inReplyTo":"1390770652.20150.25.camel@joe-AO722","subject":"Re: git blame vs git log --follow performance","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2014-01-27T01:33:11Z","receivedAt":"2014-01-27T01:33:11Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Jan 27, 2014 at 4:10 AM, Joe Perches wrote:\n> For instance (using the Linus' linux kernel git):\n>\n> $ time git log --follow -- drivers/firmware/google/Kconfig > /dev/null\n>\n> real 0m42.329s\n> user 0m40.984s\n> sys 0m0.792s\n>\n> $ time git blame -- drivers/firmware/google/Kconfig > /dev/null\n>\n> real 0m0.963s\n> user 0m0.860s\n> sys 0m0.096s\n>\n\nIt's not fair to compare blame and log. If you compare, compare it to\nnon follow version\n\n$ time git log --follow -- drivers/firmware/google/Kconfig > /dev/null\n\nreal 0m35.552s\nuser 0m35.120s\nsys 0m0.383s\n\n$ time git log -- drivers/firmware/google/Kconfig > /dev/null\n\nreal 0m4.366s\nuser 0m4.215s\nsys 0m0.144s\n\nAlthough because we need to detect rename, we can't really filter to\none path. So the base line is more like\n\n$ time git log > /dev/null\n\nreal 0m29.338s\nuser 0m28.485s\nsys 0m0.813s\n\nwith rename detection taking some more time.\n\n> Perhaps adding a whole-file rename option to the\n> \"git log\" history simplification mechanism could\n> help?\n>\n> Thoughts?\n\nI tested a version with rename detection logic removed. It did not\nchange the timing significantly. To improve --follow I think we need\nto do something about path filtering.\n-- \nDuy\n"},{"id":"233811","messageId":"1390788011.20150.32.camel@joe-AO722","threadId":"35737","inReplyTo":"CACsJy8B4dkjae3tL+_JvD63HCoi_-nNWYGvT3xRTqi8UpYX95g@mail.gmail.com","subject":"Re: git blame vs git log --follow performance","fromName":"Joe Perches","fromEmail":"joe@perches.com","sentAt":"2014-01-27T02:00:11Z","receivedAt":"2014-01-27T02:00:11Z","isPatch":false,"sender":{"key":"joe@perches.com","avatar":"https://avatars.githubusercontent.com/u/13122723?v=4"},"body":"On Mon, 2014-01-27 at 08:33 +0700, Duy Nguyen wrote:\n> On Mon, Jan 27, 2014 at 4:10 AM, Joe Perches wrote:\n> > For instance (using the Linus' linux kernel git):\n> >\n> > $ time git log --follow -- drivers/firmware/google/Kconfig > /dev/null\n> >\n> > real 0m42.329s\n> > user 0m40.984s\n> > sys 0m0.792s\n> >\n> > $ time git blame -- drivers/firmware/google/Kconfig > /dev/null\n> >\n> > real 0m0.963s\n> > user 0m0.860s\n> > sys 0m0.096s\n> >\n> \n> It's not fair to compare blame and log. If you compare, compare it to\n> non follow version\n\nPerhaps not, but git blame does follow renames.\n\n$ git blame --help\n[]\nThe origin of lines is automatically followed across\nwhole-file renames (currently there is no option to\nturn the rename-following off). To follow lines moved\nfrom one file to another, or to follow lines that were\ncopied and pasted from another file, etc., see the -C\nand -M options.\n\n> I tested a version with rename detection logic removed. It did not\n> change the timing significantly. To improve --follow I think we need\n> to do something about path filtering.\n\nPerhaps the log history could stop being read when\na commit is found that creates the file without\nanother file being deleted in the same commit.\n"},{"id":"233812","messageId":"CACsJy8A2zpOejtaCd8y8vAJW1LV6to0WRD+es7g-7o2hEUiBxA@mail.gmail.com","threadId":"35737","inReplyTo":"1390770652.20150.25.camel@joe-AO722","subject":"Re: git blame vs git log --follow performance","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2014-01-27T02:34:23Z","receivedAt":"2014-01-27T02:34:23Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Jan 27, 2014 at 4:10 AM, Joe Perches wrote:\n> Is there something that can be done about improving\n> git log --follow -- performance to be nearly\n> equivalent speed to git blame -- ?\n\nNot strictly about --follow, but there is room for improvement for\ndiff'ing in log in general. Right now we do \"diff HEAD HEAD~1\", \"diff\nHEAD~1 HEAD~2\" and so on (--follow needs diff to detect rename). At\neach step we load new tree objects and reparse. Notice after \"diff\nHEAD HEAD~1\" we may have \"HEAD~1\" and its subtrees read and parsed\n(not entirely). We could reuse that \"diff HEAD~1 HEAD~2\".\n\nOn git.git, \"git log --raw\" takes 10s and it seems tree object reading\nis about 2s.In ideal case we might be able to cut that to 1s. The tree\nparsing code (update_tree_entry) takes about 5s. We might be able to\ncut that in half, I'm not entirely sure. But there could be a lot of\nwork in caching \"HEAD~1\" and the overhead may turn out too high for\nany gain.\n-- \nDuy\n"}]}