{"thread":{"id":"47009","subject":"Ascertaining amount of \"original\" code across files/repo","startedAt":"2017-10-22T21:25:14Z","lastAt":"2017-10-23T02:04:57Z","messageCount":2,"participants":["Thomas Adam","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"330810","messageId":"20171022212505.wxdpfaevxw7fsn7e@laptop.local","threadId":"47009","inReplyTo":null,"subject":"Ascertaining amount of \"original\" code across files/repo","fromName":"Thomas Adam","fromEmail":"thomas@xteddy.org","sentAt":"2017-10-22T21:25:05Z","receivedAt":"2017-10-22T21:25:14Z","isPatch":false,"sender":{"key":"thomas@xteddy.org","avatar":"https://gravatar.com/avatar/e7256db4738e501e5d2e84f00bb0bd99503165729573848a03330301fc2adc4a?d=mp&s=160"},"body":"Hi all,\n\nI was recently left with an interesting problem of looking at a heuristic to\ndetermine how much original code was left in a repository.  Or to put another\nway, how much the code had changed since.  In my case \"original code\" means\n\"since the initial commit\", as this code base had been imported from CVS long\nago; and that was the correct starting point.\n\nWhat I came up with was the following heuristic.  What I'm curious to know is\nwhether there's an alternative way to look at this and/or if such tooling\nalready exists.\n\nWhat I did was first of all ascertain the number of original lines in each of\nthe files I was interested in:\n\n\tfor i in *.[ch]\n\tdo\n\t\tc=\"$(git --no-pager blame \"$i\" | grep -c '^\\^')\"\n\t\t[ $c -gt 0 ] && echo \"$i:$c\"\n\tdone | sort -t':' -k2 -nr\n\nGiven this, I then did some maths on the total lines from each of those files\nand to work out a percentage by file, and over all.\n\nWhat I'm curious to know is whether this approach of using \"git blame\" is a\ngood approach or not.\n\nThanks for your time.\n\n-- Thomas Adam\n"},{"id":"330816","messageId":"xmqqbmkyd3ge.fsf@gitster.mtv.corp.google.com","threadId":"47009","inReplyTo":"20171022212505.wxdpfaevxw7fsn7e@laptop.local","subject":"Re: Ascertaining amount of \"original\" code across files/repo","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-10-23T02:04:49Z","receivedAt":"2017-10-23T02:04:57Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thomas Adam <thomas@xteddy.org> writes:\n\n> What I did was first of all ascertain the number of original lines in each of\n> the files I was interested in:\n>\n> \tfor i in *.[ch]\n> \tdo\n> \t\tc=\"$(git --no-pager blame \"$i\" | grep -c '^\\^')\"\n> \t\t[ $c -gt 0 ] && echo \"$i:$c\"\n> \tdone | sort -t':' -k2 -nr\n\nAnother approach I've used when I was curious how many among 1244\nlines Linus originally wrote for Git in 2005 remains in today's\ncodebase goes the other way [*1*].\n\nThe \"reverse\" approach makes use of the -S option of blame to\nfabricate a hypothetical history where the very initial version of\nGit is today's version, and then there is another version that was\nbuilt on it (eh, rather reduced out of it) which is Linus's\noriginal.\n\n\t$ git tag initial e83c5163316f89\n\t$ cat >fake-history <<EOF\n\t$(git rev-parse initial) $(git rev-parse master)\n\t$(git rev-parse master)\n\tEOF\n\nThe list of files that Linus had in his original can befound out\nwith:\n\n\t$ git ls-tree -r --name-only initial\n\nand you can iterate over them with a command like this:\n\n\t$ git blame -Sfake-history -s -b initial -- cache.h\n\na brief commentary of the options:\n\n * \"-Sfake-history\" option points at a fake-history file, which uses\n   the same format as the \"graft\" file, to establish the fake\n   ancestry.  The first line claims that the Linus's 'initial'\n   version has only one parent, which is our current version\n   'master' (in reality, Linus's 'initial' version did not have any\n   parent, of course).  The second line claims that our current\n   version 'master' is a root commit without any parent.\n\n * \"-s\" squelches all metainformation other than commit object name\n   from the prefix of each line; \"-b\" further blanks out the commit\n   object name of the \"root\" commit---note that in this fake\n   history, our current state in 'master' is what is blanked out.\n\nThe output may start like so:\n\n                     1) #ifndef CACHE_H\n                     2) #define CACHE_H\n                     3) \n        e83c5163316  4) #include <stdio.h>\n        e83c5163316  5) #include <sys/stat.h>\n        e83c5163316  6) #include <fcntl.h>\n        e83c5163316  7) #include <stddef.h>\n\nThe idea is that a line that is blamed to the \"root\" commit\n(i.e. blank prefix) is what survived since Linus's version down to\nour current version.  In the fake world, Linus started from our\ntoday's version and ended up with the same result in his version for\nthese lines.  A line that is blamed to e83c516 is something we do\nnot have in our today's version that is \"added\" by Linus in this\nfake world---that in reality is what we \"lost\" from Linus's original\nover time.\n\nBy adding -M and -C on \"git blame\" command line, you'll find more\nlines that survived over time from Linus's original by getting moved\naround inside the same file and across file boundaries.  By adding -w,\nindentation-only changes would also be ignored.\n\nI am not judging which is more correct to go in the forward\ndirection like your approach does or to go in the reverse, as I\nhaven't thought about it deeply enough.\n\n\n[Reference]\n\n*1* https://docs.google.com/file/d/0Bw3FApcOlPDhMFR3UldGSHFGcjQ/view\n\n    Slide #11 was created using the above method.\n\n\n\n"}]}