{"thread":{"id":"23105","subject":"GSoC draft proposal: Line-level history browser","startedAt":"2010-03-20T09:18:22Z","lastAt":"2010-03-30T11:15:12Z","messageCount":54,"participants":["Bo Yang","Johannes Schindelin","Junio C Hamano","Alex Riesen","A Large Angry SCM","Jonathan Nieder","Jakub Narebski","Sverre Rabbelier","Peter Kjellerstedt","Thomas Rast","Michael J Gruber"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"137320","messageId":"41f08ee11003200218u59c45b6dl82a8eb56cc289256@mail.gmail.com","threadId":"23105","inReplyTo":null,"subject":"GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-20T09:18:22Z","receivedAt":"2010-03-20T09:18:22Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi,\n\nI am very interested in the project 'Line-level history browser',\nafter some days consideration, now I made up a draft of my proposal, I\nthink it is helpful to send it to the list before submitting it. Could\nyou please give me some advise?\n\n-----------------------------------------------\nDraft proposal: Line-level History Browser\n\n=====Purpose of this project=====\n\"git blame\" can tell us who is responsible for a line of code, but it\ncan't help if we want to get the detail of how the lines of code have\nevolved as what it is now.\nThis project will add a new utility for git called 'git line-log'. It\ncan trace the history of any line range of certain file at any\nrevision. For simplity, users can run the command like: ' git line-log\nbuiltin/diff.c 6..8 ', he will get the change history of code between\nline 6 and line 8 of the diff.c file. And for each history entry, it\nwill provide the commits, the diff block which contains changes of\nusers' interested lines.\nThis utility will trace all the modification history of interested\nlines and stop until it finds the root of the lines, which is a point\nwhere all the new code is added from scratch. Also, the users can\nspecify how deeply he wants this utility to trace. And this tool will\ntreat code move just like modification too, so it will follow the code\nmove inside one file.\nNote that, the history may not always be a single thread of commits.\nIf there are more than one commit which produce the specified line\nrange, the thread of history will split. And this utility will stop\nand provide all commits with its code changes to the user, let the\nuser to select which one to trace next.\n\n=====Work and technical issues=====\n==Command options==\nThis new tool should be used for exploring the history of changes for\ncertain line range of code in one file.\n\ngit line-log [options] \n\nOptions:\n1. Since it will output commit description, it will contain the option\nused to control whether we should show the whole commit message or\njust a short title.\n2. Option whether we should display only the 'user interested lines'\ndiff block [default] or display the whole diff with the interested\narea colorfully displayed.\n3. The max depth we trace into the commit history.\n4. The revision of the . This is very useful when the current\ninterested line range is produced by more than one commit. The user\ncan use this option to specify the file revision and trace down from\nthat revision and the line range.\n\n\nIts format should be .. or just a .\n\n==Design and implementation==\nGit store all the blobs instead of code delta, so we should traverse\nthe commit history and directly access the tree/blob objects to\ncompute the code delta and search for the diff which contains the\ninterested lines. Since git use libxdiff to format its diff file, we\nshould iterate through all xdiff's diff blocks and find what the code\nlooks like before the commit. Here, we will find a new line range\nwhich is the origin code before this commit. And then start another\nsearch from the current commit and the new line range. Recursively, we\ncan find all the modification history. We will stop when we find that\nthe current interested line range is added from scratch and is not\nmoved from other place of the file. We may also stop the traverse when\nwe reach the max search depth. Also, if the thread of change history\nsplit into two or more commits, we stop and provide the users all the\nrelated commits and corresponding line range.\n\nFor implementation related stuff, this tool heavily depends on\nlibxdiff. Because we will search our interested lines through xdiff's\noutput to find the right diff trunk to display and trace down. So, how\nwe search the xdiff's diff blocks is very important. After reading\nsome libxdiff document and code, I find that libxdiff output all the\ndiff blocks as string into a memory file. If we parse the diff block\nstring to find the changed lines, it is very inefficient. So, I\nsuggest changing xdiff's xdl_diff function to let it store some meta\ndata for each diff trunk. I think this will be very helpful for the\nperformance of this tool.\n\nGenerally,\n1. xdiff/xdiffi.c will get changed to make xdl_diff store some desired\nmeta data and pass it to caller.\n2. builtin/line-log.c will be added to complete most of the new\nfeatures, the most important function here may be cmd_linelog.\n3. git.c will be changed to add this new utility to the front end.\n4. Documents will be updated to introduce this new tool.\n\n=====About me=====\nI am Bo Yang, a Chinese graduate student majoring in Computer Science\nof NanKai University. I have touched some open source software since 5\nyears ago and began to contribute code to open source community from\nthree years ago. I have contributed to Mozilla/Mingw/Netsurf.\nTechnically, I am experienced in C/Bash Shell. I have attended last\nyear's GSoC with Netsurf project. In that project, I have completed\nmost of a DOM library in C.\nI begin to use git for source code revision from about two years ago.\nI use Git for track my Mozilla trunk source code. Because updating\nMozilla code by CVS in my school is very slow. So, I write one script\nto automatically updating the trunk with CVS at mid-night, when the\nnetwork flow is fast, on the server, and then use Git to maintain the\ncode. Then I use Git in my PC to clone/update the source code from my\nlocal server and that is very fast. I use Git to track my changes to\nthe code and some bug fixes. It is an excellent tool for\nbranch/history, I think.\nGit is my lovely daily tool for revision control. I have much\nexperience with it and have read \"Git Internals\" and also get some\nbasic knowledge about Git's code base. And I think the line-level\nhistory explorer is really suitable for me and I can make a good start\nwith this project in Git community.\n-----------------------------------------------\n\nAny feedback from you will be appreciated very much, thanks a lot!\n\nRegards!\nBo\n-- \nMy blog: http://blog.morebits.org\n"},{"id":"137325","messageId":"alpine.DEB.1.00.1003201209260.7596@pacific.mpi-cbg.de","threadId":"23105","inReplyTo":"41f08ee11003200218u59c45b6dl82a8eb56cc289256@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2010-03-20T11:30:02Z","receivedAt":"2010-03-20T11:30:02Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 20 Mar 2010, Bo Yang wrote:\n\n> I am very interested in the project 'Line-level history browser', after \n> some days consideration, now I made up a draft of my proposal, I think \n> it is helpful to send it to the list before submitting it. Could you \n> please give me some advise?\n\nI like it very much already! You obviously put in a substantial amount of \ntime to learn intricate details about the way Git operates, and what is \nalready available.\n\nAnd you also provided a patch (unrelated to line-level history browser), \nso you proved that you actually cloned Git, and that you can actually \npatch it and use Git itself to send a patch to this list.\n\nVery good.\n\nJust a few constructive criticisms (inlined):\n\n> This project will add a new utility for git called 'git line-log'. It \n> can trace the history of any line range of certain file at any revision.\n\nI think that that might be good for starters, but one could imagine that \nan integration into \"git log\" might be even better, so that gitk can use \nthis without any further changes.\n\n> For simplity, users can run the command like: ' git line-log \n> builtin/diff.c 6..8 ', he will get the change history of code between \n> line 6 and line 8 of the diff.c file.\n\nIt would be good if the code looked harder after failing with the simple \nstrategy, such as looking for code removed in other files, fuzzy matching \n(optional), and looking for code duplication (i.e. literal copying, or \nslightly modified copying).\n\nThe fuzzy matching might be necessary to catch things like a Java class \nmoving from one file into another (and changing its name): the first line \nchanges, but not completely.\n\n> After reading some libxdiff document and code, I find that libxdiff \n> output all the diff blocks as string into a memory file.\n\nAlmost.\n\nJust have a look at the word-level diff (--color-words):\n\nhttp://repo.or.cz/w/git/dscho.git/blob/bc1ed6aafd9ee4937559535c66c8bddf1864bec6:/diff.c#l382\n\nYou will see that there is a function fn_out_diff_words_aux(), which is \npassed to xdi_diff_outf(). That latter function calls xdiff such that the \nformer function receives a complete line at a time. And this is what I \nwould suggest doing in the line-level log, too.\n\nCiao,\nDscho\n"},{"id":"137328","messageId":"41f08ee11003200610n2c7c9684l6ca200cffdfdb434@mail.gmail.com","threadId":"23105","inReplyTo":"alpine.DEB.1.00.1003201209260.7596@pacific.mpi-cbg.de","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-20T13:10:55Z","receivedAt":"2010-03-20T13:10:55Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi Johannes,\n\n Thank you very much for your advice!\n\n>\n> I like it very much already! You obviously put in a substantial amount of\n> time to learn intricate details about the way Git operates, and what is\n> already available.\n>\n> And you also provided a patch (unrelated to line-level history browser),\n> so you proved that you actually cloned Git, and that you can actually\n> patch it and use Git itself to send a patch to this list.\n\nI am very happy you like it.\n\n>\n> I think that that might be good for starters, but one could imagine that\n> an integration into \"git log\" might be even better, so that gitk can use\n> this without any further changes.\n\nSo, I think add some new options to 'git log' is preferred.\n\n>\n> It would be good if the code looked harder after failing with the simple\n> strategy, such as looking for code removed in other files, fuzzy matching\n> (optional), and looking for code duplication (i.e. literal copying, or\n> slightly modified copying).\n>\n> The fuzzy matching might be necessary to catch things like a Java class\n> moving from one file into another (and changing its name): the first line\n> changes, but not completely.\n\nThat's really a good idea.\nSo, when the program reach the end of the history thread of some\nchanges of line range, it should not stop immediately. It then should\nmake a harder code search and try to find whether the new add lines of\ncode is moved to there or just copied from other place to there. And\nthese kind of search should use fuzzy matching instead of exact string\nmatching.\n\nBut notice that, detect code movement in one commit is much efficient\nthan detecting code copy. So, I think we should add an option to\ncontrol whether we detect such kind of code copy. By default, we\ndetect code move but not code copy. How do you think about this?\n\n> Just have a look at the word-level diff (--color-words):\n>\n> http://repo.or.cz/w/git/dscho.git/blob/bc1ed6aafd9ee4937559535c66c8bddf1864bec6:/diff.c#l382\n>\n> You will see that there is a function fn_out_diff_words_aux(), which is\n> passed to xdi_diff_outf(). That latter function calls xdiff such that the\n> former function receives a complete line at a time. And this is what I\n> would suggest doing in the line-level log, too.\n\nI have look over the function fn_out_diff_words_aux, this function\nparse each line of a memory diff. We can use it to detect the diff\nhunk head and find the line change. If you think the performance is\nacceptable, I think using this callback mechanism is all right.\n\nRegards!\nBo\n-- \nMy blog: http://blog.morebits.org\n"},{"id":"137329","messageId":"7vaau3ruhc.fsf@alter.siamese.dyndns.org","threadId":"23105","inReplyTo":"41f08ee11003200610n2c7c9684l6ca200cffdfdb434@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-03-20T13:30:23Z","receivedAt":"2010-03-20T13:30:23Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Bo Yang writes:\n\n> But notice that, detect code movement in one commit is much efficient\n> than detecting code copy. So, I think we should add an option to\n> control whether we detect such kind of code copy.\n\nIf you are hooking into \"git log\", it already has \"-M / -C / -C -C\" as a\nnotion to express \"different levels of digging\" to find code movement and\ncopies, and so does \"git blame\". You probably will save a lot of time if\nyou studied the current blame implementation thouroughly before designing\nor coding.\n\nTwo things that you need to think about carefully is why \"blame\" stops at\nthe commits it shows, and if you could \"peel\" these lines in its output to\npeek what are behind the lines, what you would see. This is not a rocket\nscience topic, but it is not entirely trivial.\n"},{"id":"137330","messageId":"alpine.DEB.1.00.1003201432410.7596@pacific.mpi-cbg.de","threadId":"23105","inReplyTo":"41f08ee11003200610n2c7c9684l6ca200cffdfdb434@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2010-03-20T13:36:25Z","receivedAt":"2010-03-20T13:36:25Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\n[please do not cull the Cc: list]\n\nOn Sat, 20 Mar 2010, Bo Yang wrote:\n\n> I (Johannes) wrote:\n>\n> > I think that that might be good for starters, but one could imagine \n> > that an integration into \"git log\" might be even better, so that gitk \n> > can use this without any further changes.\n> \n> So, I think add some new options to 'git log' is preferred.\n\nYes, I think that this should be the target for the user interface. \nHowever, the logic should be different enough to merit a completely new \nfile for the code (think \"git add --interactive\").\n\n> > It would be good if the code looked harder after failing with the \n> > simple strategy, such as looking for code removed in other files, \n> > fuzzy matching (optional), and looking for code duplication (i.e. \n> > literal copying, or slightly modified copying).\n> >\n> > The fuzzy matching might be necessary to catch things like a Java \n> > class moving from one file into another (and changing its name): the \n> > first line changes, but not completely.\n> \n> That's really a good idea.\n> So, when the program reach the end of the history thread of some\n> changes of line range, it should not stop immediately. It then should\n> make a harder code search and try to find whether the new add lines of\n> code is moved to there or just copied from other place to there. And\n> these kind of search should use fuzzy matching instead of exact string\n> matching.\n> \n> But notice that, detect code movement in one commit is much efficient\n> than detecting code copy. So, I think we should add an option to\n> control whether we detect such kind of code copy. By default, we\n> detect code move but not code copy. How do you think about this?\n\nYes, it is much more difficult, and it is more expensive. So: there are \nseveral steps in the project (you could also call them \"milestones\"), and \nfuzzy matching end lines would come later than simple code movement. And \nstill later than code movement between files.\n\n> > Just have a look at the word-level diff (--color-words):\n> >\n> > http://repo.or.cz/w/git/dscho.git/blob/bc1ed6aafd9ee4937559535c66c8bddf1864bec6:/diff.c#l382\n> >\n> > You will see that there is a function fn_out_diff_words_aux(), which \n> > is passed to xdi_diff_outf(). That latter function calls xdiff such \n> > that the former function receives a complete line at a time. And this \n> > is what I would suggest doing in the line-level log, too.\n> \n> I have look over the function fn_out_diff_words_aux, this function parse \n> each line of a memory diff. We can use it to detect the diff hunk head \n> and find the line change. If you think the performance is acceptable, I \n> think using this callback mechanism is all right.\n\nYes, I think that the performance is alright there, it works well enough \nfor --color-words.\n\nThanks,\nDscho\n"},{"id":"137368","messageId":"81b0412b1003201335g7b37c51mfa3e2280210ebb7e@mail.gmail.com","threadId":"23105","inReplyTo":"41f08ee11003200218u59c45b6dl82a8eb56cc289256@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2010-03-20T20:35:54Z","receivedAt":"2010-03-20T20:35:54Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On Sat, Mar 20, 2010 at 10:18, Bo Yang wrote:\n> \n> Its format should be .. or just a .\n\nYou might want to reconsider the line range syntax. Exactly the same syntax\nis already used to specify a commit range, so reusing it may lead to confusion.\n"},{"id":"137372","messageId":"7vbpeibtij.fsf@alter.siamese.dyndns.org","threadId":"23105","inReplyTo":"81b0412b1003201335g7b37c51mfa3e2280210ebb7e@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-03-20T20:57:56Z","receivedAt":"2010-03-20T20:57:56Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Alex Riesen writes:\n\n> On Sat, Mar 20, 2010 at 10:18, Bo Yang wrote:\n>> \n>> Its format should be .. or just a .\n>\n> You might want to reconsider the line range syntax. Exactly the same syntax\n> is already used to specify a commit range, so reusing it may lead to confusion.\n\nI would actually recommend you take a look at -L option from blame. What\nI use most often and find very handy myself is this pattern:\n\n\tblame -L '/^void some_function()/,/^}/' -- path\n\nas I do not have to count the line numbers.\n\nThere also was a discussion on allowing more than one -L to blame, which I\nthink is applicable to this feature. Check the list archive for the past\nfew months.\n"},{"id":"137379","messageId":"4BA544FC.7050007@gmail.com","threadId":"23105","inReplyTo":"81b0412b1003201335g7b37c51mfa3e2280210ebb7e@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"A Large Angry SCM","fromEmail":"gitzilla@gmail.com","sentAt":"2010-03-20T21:58:20Z","receivedAt":"2010-03-20T21:58:20Z","isPatch":false,"sender":{"key":"gitzilla@gmail.com","avatar":"https://gravatar.com/avatar/354625c442439908ff3dd99757dee330e29e9df7847472384faf7a00add247fb?d=mp&s=160"},"body":"Alex Riesen wrote:\n> On Sat, Mar 20, 2010 at 10:18, Bo Yang wrote:\n>> \n>> Its format should be .. or just a .\n> \n> You might want to reconsider the line range syntax. Exactly the same syntax\n> is already used to specify a commit range, so reusing it may lead to confusion.\n\nI, actually, think the proposed line range syntax works because it uses \nthe same _range_ notation. The issue is how to differentiate the _line_ \nrange(s) from the _commit_ range(s); and, yes, I would like multiple \nranges of each type as well as multiple files.\n"},{"id":"137409","messageId":"41f08ee11003202303h7f3743c0j7ed0c379e1acc97c@mail.gmail.com","threadId":"23105","inReplyTo":"7vaau3ruhc.fsf@alter.siamese.dyndns.org","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-21T06:03:07Z","receivedAt":"2010-03-21T06:03:07Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi Junio,\n\n Thank you very much for your advice.\n>\n> If you are hooking into \"git log\", it already has \"-M / -C / -C -C\" as a\n> notion to express \"different levels of digging\" to find code movement and\n> copies, and so does \"git blame\".  You probably will save a lot of time if\n> you studied the current blame implementation thouroughly before designing\n> or coding.\n\nYes, both blame and log has such '-M/-C/-C -C/' options. But the\nmeaning are not very same:\nFor 'git log': -M is used to detect file rename, -C is used to trace\ncode copy. Both options accept no argument.\nFor 'git blame': -M is used to trace code move, -C is used to trace\ncode copy. And both options accept a which specify the lower\nbound of the 'same code characters'.\nAnd, I think the line-level history tool act more like 'git blame'.\nSo, the '-C' option for 'git log' is exactly what we need but '-M' is\nnot. So, I think, maybe we should add another '-m' option to 'git log'\nfor line-level code movement detect.\n\nI have make a rough look over blame.c, it is really very helpful and I\nfind I can borrow some code from 'git blame' to make the line-level\nhistory browser.\n\nThanks a lot!\n\n>\n> Two things that you need to think about carefully is why \"blame\" stops at\n> the commits it shows, and if you could \"peel\" these lines in its output to\n> peek what are behind the lines, what you would see.  This is not a rocket\n> science topic, but it is not entirely trivial.\n\nI think blame's purpose is to find who is responsible for which line\nof code. So, it stop after it find the origin of the code. And\nline-level history browser will continue back into more history on\nwhat blame got, it will find what the line should be before this\ncommit, and go backward the history based on the origin line to get a\nmore old status and go on again. Simply, it is something like 'git\nblame' recursively. :)\n\nThanks again for your advice, I get too much from your feedback, thanks!\n\nRegards!\nBo\n"},{"id":"137410","messageId":"41f08ee11003202305k229466f9w6f137f759df4efa4@mail.gmail.com","threadId":"23105","inReplyTo":"alpine.DEB.1.00.1003201432410.7596@pacific.mpi-cbg.de","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-21T06:05:42Z","receivedAt":"2010-03-21T06:05:42Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":">>\n>> So, I think add some new options to 'git log' is preferred.\n>\n> Yes, I think that this should be the target for the user interface.\n> However, the logic should be different enough to merit a completely new\n> file for the code (think \"git add --interactive\").\n\nSo, a new file builtin/line-level.c will be added.\n\n>\n> Yes, it is much more difficult, and it is more expensive. So: there are\n> several steps in the project (you could also call them \"milestones\"), and\n> fuzzy matching end lines would come later than simple code movement. And\n> still later than code movement between files.\n\nOk, I will add some milestones on my next version proposal, thanks.\n\nRegards!\nBo\n"},{"id":"137411","messageId":"41f08ee11003202310l555cae23r23dbdfa476e8398f@mail.gmail.com","threadId":"23105","inReplyTo":"7vbpeibtij.fsf@alter.siamese.dyndns.org","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-21T06:10:05Z","receivedAt":"2010-03-21T06:10:05Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":">\n> I would actually recommend you take a look at -L option from blame.  What\n> I use most often and find very handy myself is this pattern:\n>\n>        blame -L '/^void some_function()/,/^}/' -- path\n>\n> as I do not have to count the line numbers.\n\nI have look at that options and I find it is very convenient and\nline-level browser will adopt that line syntax, too.\n\n> There also was a discussion on allowing more than one -L to blame, which I\n> think is applicable to this feature.  Check the list archive for the past\n> few months.\n\nI think it is rationale for 'git blame' to allow more than one -L to\nlet the users see more than one block of code. But for a tool which\nused to explore history, I think the user almost focus on one thread\nof history. If the history split on some point, we should ask user for\nchoose one to go on. So, I think the line-level browser need not to\nsupport such a thing. :)\n\nRegards!\nBo\n"},{"id":"137412","messageId":"41f08ee11003202316w2fddc5f4jebda47f325451577@mail.gmail.com","threadId":"23105","inReplyTo":"4BA544FC.7050007@gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-21T06:16:27Z","receivedAt":"2010-03-21T06:16:27Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"On Sun, Mar 21, 2010 at 5:58 AM, A Large Angry SCM wrote:\n> Alex Riesen wrote:\n>>\n>> On Sat, Mar 20, 2010 at 10:18, Bo Yang wrote:\n>>>\n>>> \n>>> Its format should be .. or just a .\n>>\n>> You might want to reconsider the line range syntax. Exactly the same\n>> syntax\n>> is already used to specify a commit range, so reusing it may lead to\n>> confusion.\n>\n> I, actually, think the proposed line range syntax works because it uses the\n> same _range_ notation. The issue is how to differentiate the _line_ range(s)\n> from the _commit_ range(s); and, yes, I would like multiple ranges of each\n> type as well as multiple files.\n\nAs what I said in previous post, I think we should adopt 'git blame'\nway. Use a '-L ,' to specify the line range. It\nsupport both line number and posix regex.\nFor multiple ranges stuff, I don't think it is very useful to support\nit for a history browser. Anyway, our users can only focus on one line\nof thread history. I am very willing to listen what is your use case\nfor a multiple ranges?\n\nThanks for your precious advice!\n\nRegards!\nBo\n"},{"id":"137444","messageId":"4BA61CF9.7040104@gmail.com","threadId":"23105","inReplyTo":"41f08ee11003202316w2fddc5f4jebda47f325451577@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"A Large Angry SCM","fromEmail":"gitzilla@gmail.com","sentAt":"2010-03-21T13:19:53Z","receivedAt":"2010-03-21T13:19:53Z","isPatch":false,"sender":{"key":"gitzilla@gmail.com","avatar":"https://gravatar.com/avatar/354625c442439908ff3dd99757dee330e29e9df7847472384faf7a00add247fb?d=mp&s=160"},"body":"Bo Yang wrote:\n[...]\n> For multiple ranges stuff, I don't think it is very useful to support\n> it for a history browser. Anyway, our users can only focus on one line\n> of thread history. I am very willing to listen what is your use case\n> for a multiple ranges?\n\nMore than one line range can be related and of interest to a \nforensics/archeology task.\n\nIn a simple multi range case, you'd have 2 line ranges in the same file \nthat you want to see the history and graph of. Such as 2 related macro \ndefinitions in a header file.\n\nIn a complex multi range case, you'd have many line ranges spread over \nmultiple blobs and some of the blobs have disjoint commit graphs.\n\nThe complex multi range case may be too much for a GSOC project, and the \nsimple multi range case may be also. However, the command syntax should \nbe general enough to handle them without being too ugly so that the \nimplementation could be improved and expanded later.\n"},{"id":"137510","messageId":"41f08ee11003212048q1950eb3s95f1caf7ba640874@mail.gmail.com","threadId":"23105","inReplyTo":"4BA61CF9.7040104@gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-22T03:48:38Z","receivedAt":"2010-03-22T03:48:38Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"On Sun, Mar 21, 2010 at 9:19 PM, A Large Angry SCM wrote:\n> Bo Yang wrote:\n> [...]\n>>\n>> For multiple ranges stuff, I don't think it is very useful to support\n>> it for a history browser. Anyway, our users can only focus on one line\n>> of thread history. I am very willing to listen what is your use case\n>> for a multiple ranges?\n>\n> More than one line range can be related and of interest to a\n> forensics/archeology task.\n>\n> In a simple multi range case, you'd have 2 line ranges in the same file that\n> you want to see the history and graph of. Such as 2 related macro\n> definitions in a header file.\n>\n> In a complex multi range case, you'd have many line ranges spread over\n> multiple blobs and some of the blobs have disjoint commit graphs.\n>\n> The complex multi range case may be too much for a GSOC project, and the\n> simple multi range case may be also. However, the command syntax should be\n> general enough to handle them without being too ugly so that the\n> implementation could be improved and expanded later.\n\nYeah, how do you think use the following syntax:\n\n@:, @:,\n\nThanks!\n\nRegards!\nBo\n"},{"id":"137512","messageId":"41f08ee11003212052p6b0a7495j2e38f24839541ffb@mail.gmail.com","threadId":"23105","inReplyTo":"4BA61CF9.7040104@gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-22T03:52:58Z","receivedAt":"2010-03-22T03:52:58Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi all,\n\n Thanks a lot for your precious advice and based on that, I have\nprepared a new version of my proposal, generally it provide a detailed\noptions which I want to add to 'git log' and a new syntax for\nsupporting multi line ranges in any file at any revision. Also, this\nversion provide a milestones and timeline for this project. Thanks\nagain for your advice and I appreciate your feedback very much for\nthis version.\n\n-----------------------------------------------------------------------\nDraft proposal(v2): Line-level History Browser\n\n=====Purpose of this project=====\n\"git blame\" can tell us who is responsible for a line of code, but it\ncan't help if we want to get the detail of how the lines of code have\nevolved as what it is now.\nThis project will add a new feature for 'git log' to display line\nlevel history. It can trace the history of any line range of certain\nfile at any revision. For simplity, users can run the command like: '\ngit log -L builtin/diff.c:6,8 ', he will get the change history of\ncode between line 6 and line 8 of the diff.c file. And for each\nhistory entry, it will provide the commits, the diff block which\ncontains changes of users' interested lines.\nThis utility will trace all the modification history of interested\nlines and stop until it finds the root of the lines, which is a point\nwhere all the new code is added from scratch. Also, the users can\nspecify how deeply he wants this utility to trace. And this tool will\ntreat code move just like modification too, so it will follow the code\nmove inside one commit.\nNote that, the history may not always be a single thread of commits.\nIf there are more than one commit which produce the specified line\nrange, the thread of history will split. And this utility will stop\nand provide all commits with its code changes to the user, let the\nuser to select which one to trace next.\n\n=====Work and technical issues=====\n==Command options==\nThis new feature should be used for exploring the history of changes\nfor certain line range of code in one file.\n\ngit log [-m] [-I] [-d depth] [--fuzzy] -L file1@rev1:, file2@rev2:,\n\nOptions:\n1. -m, option to control whether we should follow code movement.\nIf one -m is given, we follow code movement inside file, when more\nthan one '-m' is given, we follow the movement between files in one\ncommit. The is used to specify the lower bound for the number of\nlines of moved code. If it is not given, we set it as 1.\n2. -I, option to control whether we should display only the 'user\ninterested lines' diff block [default] or display the whole diff with\nthe interested area colorfully displayed.\n3. -d, option to control the max depth we trace into the commit history.\n4. --fuzzy, option to control whether fuzzy code copy mathing is used.\n5. '-L' to control whether we run a simple log or we want a line level log.\n6. Files and lines. I propose to use such a syntax to specify the\nfiles at revision and line range, @:,. This looks a little complex, but I think it is neccessary\nbecause we will support multiple file at any version and any line\nrange finally. The revision can be any revision format of Git and the\n can be a number, or a posix regex, just like what 'git blame'\ndo.\n7. And we will support code copy detect, too. The option which control\nwhether we trace code copy does exist in current 'git log', which is\nthe option '-C'. Similiarly, one '-C' is used to trace code copy of\nnew added code inside one commit. Two '-C' will trace any code copy\ninside commit tree.\n\n==Design and implementation==\nGit store all the blobs instead of code delta, so we should traverse\nthe commit history and directly access the tree/blob objects to\ncompute the code delta and search for the diff which contains the\ninterested lines. Since git use libxdiff to format its diff file, we\nshould iterate through all xdiff's diff blocks and find what the code\nlooks like before the commit. This will be done using the callback\nmechanism. Here, we will find a new line range which is the origin\ncode before this commit. And then start another search from the\ncurrent commit and the new line range. Recursively, we can find all\nthe modification history. We will stop when we find that the current\ninterested line range is added from scratch and is not moved from\nother place of the file. Here, if the user want to trace code copy,\nmore work will be done to find the possible code copy. We may also\nstop the traverse when we reach the max search depth. Also, if the\nthread of change history split into two or more commits, we stop and\nprovide the users all the related commits and corresponding line\nrange.\n\nGenerally,\n1. New callback for xdi_diff to parse the diff hunk and store line\nlevel history info.\n2. builtin/line-log.c will be added to complete most of the new features.\n3. builtin/log.c will be changed to add this new utility to the front end.\n4. Documents will be updated to introduce this new tool.\n\n=====Milestones and Timeline=====\nIn this summer, we will add support of line level history browser for\nonly one file. The multiple ranges support is currently not in this\nproject.\n\nThe milestones of the project are:\n1. Simple modification change history.\n2. Code movement inside one file detect.\n3. Code movement inside one commit but not a file.\n4. Code copy of modified file in one commit.\n5. Code copy of any place in one commit tree.\n6. Fuzzy matching support.\n\nAnd the timeline will be:\nApril 26 - May 23: Catch up with Git code base and study the\nimplementation of blame.c and log.c thouroughly.\n\nMay 24 - June 21 : Complete a version which supports code\nmodifcation trace but without code movement and code copy support.\n\nJune 22 - June 29: Complete a version which supports code movement\ninside one file.\n\nJune 30 - July 7: Complete a version which supports code movement\nbetween files inside one commit.\n\nJuly 8 - July 15: Complete a version which supports code copy of\nmodified file in one commit.\n\nJuly 16 - July 23: Complete a version which supports code copy of\nany file in one commit tree.\n\nJuly 24 - August 7: Complete fuzzy matching of code movement and copy detect.\n\n=====About me=====\nI am Bo Yang, a Chinese graduate student majoring in Computer Science\nof NanKai University. I have touched some open source software since 5\nyears ago and began to contribute code to open source community from\nthree years ago. I have contributed to Mozilla/Mingw/Netsurf.\nTechnically, I am experienced in C/Bash Shell. I have attended last\nyear's GSoC with Netsurf project. In that project, I have completed\nmost of a DOM library in C.\nI begin to use git for source code revision from about two years ago.\nI use Git for track my Mozilla trunk source code. Because updating\nMozilla code by CVS in my school is very slow. So, I write one script\nto automatically updating the trunk with CVS at mid-night, when the\nnetwork flow is fast, on the server, and then use Git to maintain the\ncode. Then I use Git in my PC to clone/update the source code from my\nlocal server and that is very fast. I use Git to track my changes to\nthe code and some bug fixes. It is an excellent tool for\nbranch/history, I think.\nGit is my lovely daily tool for revision control. I have much\nexperience with it and have read \"Git Internals\" and also get some\nbasic knowledge about Git's code base. And I think the line-level\nhistory explorer is really suitable for me and I can make a good start\nwith this project in Git community.\n\n---------------------------------------------------\nThank you very much!\n\nRegards!\nBo\n"},{"id":"137513","messageId":"7vfx3tvv9y.fsf@alter.siamese.dyndns.org","threadId":"23105","inReplyTo":"41f08ee11003212048q1950eb3s95f1caf7ba640874@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-03-22T04:24:09Z","receivedAt":"2010-03-22T04:24:09Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Bo Yang writes:\n\n> Yeah, how do you think use the following syntax:\n>\n> @:, @:,\n\nHorrible. That is not how we name things.\n\nWhat's wrong with bog standard:\n\n $ git log -L 10,20 master -- Documentation/git.txt\n\nwhich is exactly how \"blame\" does it?\n"},{"id":"137514","messageId":"41f08ee11003212134s586aa71cs23255d02f38d53d1@mail.gmail.com","threadId":"23105","inReplyTo":"7vfx3tvv9y.fsf@alter.siamese.dyndns.org","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-22T04:34:26Z","receivedAt":"2010-03-22T04:34:26Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"> Horrible.  That is not how we name things.\n>\n> What's wrong with bog standard:\n>\n>    $ git log -L 10,20 master -- Documentation/git.txt\n>\n> which is exactly how \"blame\" does it?\n\nThe 'blame' way is very good if we only support one line range. But if\nwe want to support multiple line ranges, I don't think it is suitable\nfor that case. Anyway, how can I specify multi-ranges which refers to\nmultiple files at multiple revision and multiple line ranges using\nabove syntax?\n\nExcept that, I still can't convince myself that we need multiple\nranges support. Anyway, how do we display such a result to our users?\n\nRegards!\nBo\n"},{"id":"137519","messageId":"7vhbo8syyf.fsf@alter.siamese.dyndns.org","threadId":"23105","inReplyTo":"41f08ee11003212134s586aa71cs23255d02f38d53d1@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-03-22T05:32:56Z","receivedAt":"2010-03-22T05:32:56Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Bo Yang writes:\n\n> The 'blame' way is very good if we only support one line range. But if\n> we want to support multiple line ranges, I don't think it is suitable\n> for that case. Anyway, how can I specify multi-ranges which refers to\n> multiple files at multiple revision and multiple line ranges using\n> above syntax?\n\nI would sort of see you may want to be able to say \"explain lines 10 thru\n15 of config.h and lines 100-115 of hello.c that appear in v1.2.0\", but I\nthink it is a total nonsense to ask for \"ll 10-15 of config.h in v1.2.0\nand ll 110-115 of hello.c in v1.0.0\". After all they never existed in the\nsame revision (otherwise you would have said \"ll 7-13 of config.h and ll\n110-115 of hello.c that appear in v1.0.0\"). So I would reject the\nSVN-like \"rev@\" in the first place.\n\nWhile I don't seriously buy \"multiple files\" either, if that is really\nneeded, I could be pursuaded with \"log -- path1:10-15 path2:1-7\", or\n\"log -L path1:10-15 -Lpath2:1-7 -- path1 path2\" or something similarly\nugly like these, but that is not how we generally name things, and it\nprobably shouldn't be a new option to \"log\" anymore.\n\nOn the other hand, multiple ranges in a single file is something that\nmay be quite reasonable, e.g.\n\n $ git log -L10-15 -L200-210 -- Makefile\n $ git log -L'*/^#ifdef WINDOWS/,/^#endif \\/\\* WINDOWS \\/\\*/' -- config.h\n\nAs I already said, I wouldn't be so worried about multiple-range feature,\nbut I would be worried about the usefulness of this feature, even for the\ncase to track a single range of a single file, starting from one given\nrevision. When you want to know where the first few lines of Makefile\ncame from, and if blame says the first line came from 2731d048, that\nreally means that between the revision you started digging from and the\nfound revision, there is no commit that touched that particular line, but\nequally importantly, that before that found revision, there wasn't a\ncorresponding line in that file---blame stopped exactly because there is\nnobody before that found revision that the line can be blamed on.\n\nSo implementing \"git log -L1,10 -- Makefile\" might be just the matter of\ndoing something like:\n\n 1. Run \"git blame -L1,10 -- Makefile\";\n 2. Note the commits that appear in the output;\n 3. Topologically sort these commits;\n 4. Run \"git show \"\n\nwhich is not very satisfying.\n\nAnd \"git log -L1 -- Makefile\" naturally degenerates into:\n\n 1. Run \"git blame -L1,1 -- Makefile\";\n 2. Note the commits that appear in the output;\n 3. Run \"git show \"\n\nwhich is not just unsatisfying, but is almost boring.\n\nI dunno.\n"},{"id":"137524","messageId":"41f08ee11003220031p6fda651eycff2641df10864da@mail.gmail.com","threadId":"23105","inReplyTo":"7vhbo8syyf.fsf@alter.siamese.dyndns.org","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-22T07:31:27Z","receivedAt":"2010-03-22T07:31:27Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"On Mon, Mar 22, 2010 at 1:32 PM, Junio C Hamano wrote:\n> Bo Yang writes:\n>\n>> The 'blame' way is very good if we only support one line range. But if\n>> we want to support multiple line ranges, I don't think it is suitable\n>> for that case. Anyway, how can I specify multi-ranges which refers to\n>> multiple files at multiple revision and multiple line ranges using\n>> above syntax?\n>\n> I would sort of see you may want to be able to say \"explain lines 10 thru\n> 15 of config.h and lines 100-115 of hello.c that appear in v1.2.0\", but I\n> think it is a total nonsense to ask for \"ll 10-15 of config.h in v1.2.0\n> and ll 110-115 of hello.c in v1.0.0\".  After all they never existed in the\n> same revision (otherwise you would have said \"ll 7-13 of config.h and ll\n> 110-115 of hello.c that appear in v1.0.0\").  So I would reject the\n> SVN-like \"rev@\" in the first place.\n>\n> While I don't seriously buy \"multiple files\" either, if that is really\n> needed, I could be pursuaded with  \"log -- path1:10-15 path2:1-7\", or\n> \"log -L path1:10-15 -Lpath2:1-7 -- path1 path2\" or something similarly\n> ugly like these, but that is not how we generally name things, and it\n> probably shouldn't be a new option to \"log\" anymore.\n>\n> On the other hand, multiple ranges in a single file is something that\n> may be quite reasonable, e.g.\n>\n>  $ git log -L10-15 -L200-210 -- Makefile\n>  $ git log -L'*/^#ifdef WINDOWS/,/^#endif \\/\\* WINDOWS \\/\\*/' -- config.h\n\nYeah, maybe one file multiple ranges is most rationale.\n\n> As I already said, I wouldn't be so worried about multiple-range feature,\n> but I would be worried about the usefulness of this feature, even for the\n> case to track a single range of a single file, starting from one given\n> revision.\n\nI am sorry, but I did not catch up you here. You worried about the\nusefulness of the multi-range feature or the line level history\nbrowser?\n\nI think tracking a single range of a single file, starting from one\ngiven revision is useful when the line of history split on some point.\nThis can let users focus on a single line of history using this\nfeature.\n\n>When you want to know where the first few lines of Makefile\n> came from, and if blame says the first line came from 2731d048, that\n> really means that between the revision you started digging from and the\n> found revision, there is no commit that touched that particular line, but\n> equally importantly, that before that found revision, there wasn't a\n> corresponding line in that file---blame stopped exactly because there is\n> nobody before that found revision that the line can be blamed on.\n>\n> So implementing \"git log -L1,10 -- Makefile\" might be just the matter of\n> doing something like:\n>\n>  1. Run \"git blame -L1,10 -- Makefile\";\n>  2. Note the commits that appear in the output;\n>  3. Topologically sort these commits;\n>  4. Run \"git show \"\n>\n> which is not very satisfying.\n\nYes, this is not satisfying. But as I understand, the line level\nhistory browser will do more than just this. It will not stop on 'step\n4', it can follow the change history recursively and deeply, to find\nmore. I think this is useful when we focus just one a range of code\nand want to know how it become into such a now condition.\n\nAnyway, it is not a bad thing too add a new convenient feature to a\ndaily tool. :)\n\nRegards!\nBo\n"},{"id":"137525","messageId":"7v7hp4n6qn.fsf@alter.siamese.dyndns.org","threadId":"23105","inReplyTo":"41f08ee11003220031p6fda651eycff2641df10864da@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-03-22T07:41:20Z","receivedAt":"2010-03-22T07:41:20Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Bo Yang writes:\n\n>> When you want to know where the first few lines of Makefile\n>> came from, and if blame says the first line came from 2731d048, that\n>> really means that between the revision you started digging from and the\n>> found revision, there is no commit that touched that particular line, but\n>> equally importantly, that before that found revision, there wasn't a\n>> corresponding line in that file---blame stopped exactly because there is\n>> nobody before that found revision that the line can be blamed on.\n> ...\n> Yes, this is not satisfying. But as I understand, the line level\n> history browser will do more than just this. It will not stop on 'step\n> 4', it can follow the change history recursively and deeply, to find\n> more.\n\nI am actually questioning the existence of \"recursively and deeply to find\nmore\"; the reason blame stopped at a particular commit is exactly because\nthere is no more---otherwise it wouldn't have stopped there but kept\ndigging deeper.\n\nThat is what I meant in the message you are responding to, quoted at the\ntop of this message.\n"},{"id":"137526","messageId":"41f08ee11003220052v26b1843ftcfbccec693971eec@mail.gmail.com","threadId":"23105","inReplyTo":"7v7hp4n6qn.fsf@alter.siamese.dyndns.org","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-22T07:52:58Z","receivedAt":"2010-03-22T07:52:58Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"On Mon, Mar 22, 2010 at 3:41 PM, Junio C Hamano wrote:\n> Bo Yang writes:\n>\n>>> When you want to know where the first few lines of Makefile\n>>> came from, and if blame says the first line came from 2731d048, that\n>>> really means that between the revision you started digging from and the\n>>> found revision, there is no commit that touched that particular line, but\n>>> equally importantly, that before that found revision, there wasn't a\n>>> corresponding line in that file---blame stopped exactly because there is\n>>> nobody before that found revision that the line can be blamed on.\n>> ...\n>> Yes, this is not satisfying. But as I understand, the line level\n>> history browser will do more than just this. It will not stop on 'step\n>> 4', it can follow the change history recursively and deeply, to find\n>> more.\n>\n> I am actually questioning the existence of \"recursively and deeply to find\n> more\"; the reason blame stopped at a particular commit is exactly because\n> there is no more---otherwise it wouldn't have stopped there but kept\n> digging deeper.\n\nI think an example may explain me well.\n\ncommit 1 of the file:\nline 1 rev 1\nline 2 rev 1\n\ncommit 2 of the file:\nline 1 rev 2\nline 2 rev 2\n\ncommit 3 of the file:\nline 1 rev 3\nline 2 rev 3\n\nIf we run, git blame file, it will show two lines are blamed on commit\n3. Line level utility will also show rev2 and rev1 to users as the\nformat like what git log provide. I think git blame focus on who\nproduce the current code range. And the line level browser will\nprovide more than that, it also answer, how the lines evolved into\ncurrent condition.\n\nI hope I explain everything clearly. :)\n\nRegards!\nBo\n"},{"id":"137528","messageId":"20100322081003.GA26535@progeny.tock","threadId":"23105","inReplyTo":"7v7hp4n6qn.fsf@alter.siamese.dyndns.org","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-03-22T08:10:03Z","receivedAt":"2010-03-22T08:10:03Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Junio C Hamano wrote:\n\n> I am actually questioning the existence of \"recursively and deeply to find\n> more\"; the reason blame stopped at a particular commit is exactly because\n> there is no more\n\nHmm, I can imagine some (mutually inconsistent) heuristics:\n\n - Suppose in the blamed commit a single isolated line changed. Then\n it is clear where to look next.\n\n - If the mystery code is at the beginning of the file (resp.\n beginning of a diff -C0 hunk), maybe it was based on the line at the\n same position within the previous commit.\n\n - Take the line with the lowest Levenshtein distance from the mystery\n code.\n\n - Expect certain common patterns of change: substituted words,\n whitespace changes, added arguments for a function, things like that.\n\nThat said, I still don’t have a clear picture of a basic strategy.\n\nInterested,\nJonathan\n"},{"id":"137534","messageId":"81b0412b1003220339n80ab38bh9e1a70c197538ac2@mail.gmail.com","threadId":"23105","inReplyTo":"7vhbo8syyf.fsf@alter.siamese.dyndns.org","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2010-03-22T10:39:36Z","receivedAt":"2010-03-22T10:39:36Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On Mon, Mar 22, 2010 at 06:32, Junio C Hamano wrote:\n> While I don't seriously buy \"multiple files\" either, if that is really\n\nyeah, _really_\n\n> needed, I could be pursuaded with  \"log -- path1:10-15 path2:1-7\", or\n> \"log -L path1:10-15 -Lpath2:1-7 -- path1 path2\" or something similarly\n> ugly like these, but that is not how we generally name things, and it\n> probably shouldn't be a new option to \"log\" anymore.\n\nBut then, how about putting the \"path\" last in the argument,\nso that the unambiguosly defined part of the format comes first?\nLess need for quoting of \":\" (or \"@\") in pathnames.\n"},{"id":"137557","messageId":"alpine.DEB.1.00.1003221602100.7596@pacific.mpi-cbg.de","threadId":"23105","inReplyTo":"81b0412b1003220339n80ab38bh9e1a70c197538ac2@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2010-03-22T15:05:12Z","receivedAt":"2010-03-22T15:05:12Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 22 Mar 2010, Alex Riesen wrote:\n\n> On Mon, Mar 22, 2010 at 06:32, Junio C Hamano wrote:\n> > While I don't seriously buy \"multiple files\" either, if that is really\n> \n> yeah, _really_\n\nYes. Besides, it is an easy fall-out of the common \"a Java class was split \ninto two\" case, where you follow line ranges in different files (at least \nat some stage) _anyway_.\n\nCiao,\nDscho\n"},{"id":"137559","messageId":"m3hbo8jr2e.fsf@localhost.localdomain","threadId":"23105","inReplyTo":"41f08ee11003212052p6b0a7495j2e38f24839541ffb@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-03-22T15:48:52Z","receivedAt":"2010-03-22T15:48:52Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Bo Yang writes:\n\n> This project will add a new feature for 'git log' to display line\n> level history. It can trace the history of any line range of certain\n> file at any revision. For simplity, users can run the command like: '\n> git log -L builtin/diff.c:6,8 ', he will get the change history of\n> code between line 6 and line 8 of the diff.c file. \n\nI think that, at least at first, line-level log should follow the\ngit-blame, i.e.\n\n git log -L , -- \n\nIf we want (in the future) to follow history of some lines from one\nfile, and other lines from other file together, we do not need to use\n\n -L :,\n\nsyntax. If parseopt allows, we can use posotion of parameters, i.e.\n\n -L , -L ,\n\n> And for each history entry, it will provide the commits, the diff\n> block which contains changes of users' interested lines.\n\nThe most important *new* algorithm you need to implement is, after\nfinding (blame-like) the commit that created given version of given\nline, what was previous version of given line and which line that was.\n\nYou can probably find some heuristic in existing merge tools, like\nemerge from GNU Emacs, or graphical diff tools.\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"137574","messageId":"alpine.DEB.1.00.1003221910520.7596@pacific.mpi-cbg.de","threadId":"23105","inReplyTo":"m3hbo8jr2e.fsf@localhost.localdomain","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2010-03-22T18:21:59Z","receivedAt":"2010-03-22T18:21:59Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 22 Mar 2010, Jakub Narebski wrote:\n\n> Bo Yang writes:\n> \n> > This project will add a new feature for 'git log' to display line \n> > level history. It can trace the history of any line range of certain \n> > file at any revision. For simplity, users can run the command like: ' \n> > git log -L builtin/diff.c:6,8 ', he will get the change history of \n> > code between line 6 and line 8 of the diff.c file.\n> \n> I think that, at least at first, line-level log should follow the\n> git-blame, i.e.\n> \n> git log -L , -- \n> \n> If we want (in the future) to follow history of some lines from one\n> file, and other lines from other file together, we do not need to use\n> \n> -L :,\n> \n> syntax. If parseopt allows, we can use posotion of parameters, i.e.\n> \n> -L , -L ,\n\nOh, is it bikeshedding time already? /me might have missed the start \nsignal.\n\n> > And for each history entry, it will provide the commits, the diff \n> > block which contains changes of users' interested lines.\n> \n> The most important *new* algorithm you need to implement is, after \n> finding (blame-like) the commit that created given version of given \n> line, what was previous version of given line and which line that was.\n> \n> You can probably find some heuristic in existing merge tools, like\n> emerge from GNU Emacs, or graphical diff tools.\n\nI do not think that these tools can help, as they never look further than \nidentical lines (and they mustn't, either).\n\nMore importantly, the first step really is about driving the libxdiff in \nsuch a way that you can recognize the exact same lines.\n\n(One point to note for the technical details: the algorithm has to expect \nopposite code moves, i.e. it must cope well when the diff shows the code \nin question removed in one hunk and added in another.)\n\nWe also should not get ahead of ourselves, but allow the student to get a \nfull understanding of the requirements, from which he can then make a \nproject plan (with milestones, Christian, no problem).\n\nBTW by \"requirements\" I do not mean something as technical as the syntax, \nbut rather a definition what people should be able to expect to do with \nthis at the end of the summer.\n\nAs to fuzzy matching of lines that could not be attributed otherwise, I \nthink that that will require a lot of playing around with different ideas. \nA simple Levenshtein-Damerau is highly unlikely to be enough.\n\nCiao,\nDscho\n"},{"id":"137575","messageId":"fabb9a1e1003221138n50e30adaxe91ac9503932c35a@mail.gmail.com","threadId":"23105","inReplyTo":"alpine.DEB.1.00.1003221910520.7596@pacific.mpi-cbg.de","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2010-03-22T18:38:02Z","receivedAt":"2010-03-22T18:38:02Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Mon, Mar 22, 2010 at 19:21, Johannes Schindelin\n wrote:\n> As to fuzzy matching of lines that could not be attributed otherwise, I\n> think that that will require a lot of playing around with different ideas.\n> A simple Levenshtein-Damerau is highly unlikely to be enough.\n\nI'd recommend making this either the last milestone, or not a\nmilestone at all. As I noticed with git-stats such metrics might not\nexist at all (or at least be too hard to find/implement), and it's\nquite a bummer to not be able to implement your primary milestone ;).\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"137577","messageId":"alpine.DEB.1.00.1003221924110.7596@pacific.mpi-cbg.de","threadId":"23105","inReplyTo":"41f08ee11003212052p6b0a7495j2e38f24839541ffb@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2010-03-22T19:24:54Z","receivedAt":"2010-03-22T19:24:54Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 22 Mar 2010, Bo Yang wrote:\n\n> Draft proposal(v2): Line-level History Browser\n> \n> =====Purpose of this project=====\n> \"git blame\" can tell us who is responsible for a line of code, but it\n> can't help if we want to get the detail of how the lines of code have\n> evolved as what it is now.\n> This project will add a new feature for 'git log' to display line\n> level history. It can trace the history of any line range of certain\n> file at any revision. For simplity, users can run the command like: '\n> git log -L builtin/diff.c:6,8 ', he will get the change history of\n> code between line 6 and line 8 of the diff.c file. And for each\n> history entry, it will provide the commits, the diff block which\n> contains changes of users' interested lines.\n\nI would not be too specific here about the exact syntax. I would rather \nhave an example where this might be useful.\n\nIn git.git, for example, you could point to pretty_print_commit() which \nwas split out from commit.c into pretty.c in 93fc05e(Split off the pretty \nprint stuff into its own file), and mention that it is hard to verify \nwithout much hassle that the code split was really only a code split, \nrather than a split with an evil change.\n\nOr you could point to 691f1a2(replace direct calls to unlink(2) with \nunlink_or_warn), where code was refactored, into a new function \n(unfortunately in two commits, so it might be a case not covered by your \nproject) and it might be somebody's task to find out the original author \nfor that function.\n\nBasically, I would like to have a structure in the proposal like this: \nwhat? why? how? when?\n\n> This utility will trace all the modification history of interested\n> lines and stop until it finds the root of the lines, which is a point\n> where all the new code is added from scratch. Also, the users can\n> specify how deeply he wants this utility to trace. And this tool will\n> treat code move just like modification too, so it will follow the code\n> move inside one commit.\n>\n> Note that, the history may not always be a single thread of commits.\n> If there are more than one commit which produce the specified line\n> range, the thread of history will split.\n\nDo not forget the case where there are more than one source of a code \nmove. Think \"refactoring\".\n\n> =====Work and technical issues=====\n> ==Command options==\n> This new feature should be used for exploring the history of changes\n> for certain line range of code in one file.\n> \n> git log [-m] [-I] [-d depth] [--fuzzy] -L file1@rev1: pos>, file2@rev2:,\n\nI would like this not to be specified too much here. For example, we do \nnot know yet, whether the matching will be fuzzy, or whether we find \nsomething cleverer than that.\n\nSo, I suggest to list not the command line options, but what you intend to \nsupport. I.e.:\n\n> Options:\n>\n> 1. -m, option to control whether we should follow code movement. \n> If one -m is given, we follow code movement inside file, when more \n> than one '-m' is given, we follow the movement between files in one \n> commit. The is used to specify the lower bound for the number \n> of lines of moved code. If it is not given, we set it as 1.\n\nHere you do not need to say that it is -m, but that you want to \nsupport following code movements both inside and between files, but only \noptionally, for performance reasons (or some such).\n\nIn any case, this would probably just reuse the -M option.\n\n> 2. -I, option to control whether we should display only the 'user \n> interested lines' diff block [default] or display the whole diff with \n> the interested area colorfully displayed.\n\nIt would be more in line with the diff options to use -U, but you do not \nhave to state that. Just talk about a configurable amount of context.\n\n> 3. -d, option to control the max depth we trace into the commit history.\n\nAgain, there are better options for \"git log\" already, but you do not need \nto be too explicit on the syntax side. Just say that you want to be able \nto use as many of \"git log\"s options as make sense in the context of \nline-level history.\n\n> 4. --fuzzy, option to control whether fuzzy code copy mathing is used.\n\nSee above.\n\n> 5. '-L' to control whether we run a simple log or we want a line level \n> log.\n\nSee above.\n\n> 6. Files and lines. I propose to use such a syntax to specify the files \n> at revision and line range, @:,. \n> This looks a little complex, but I think it is neccessary because we \n> will support multiple file at any version and any line range finally. \n> The revision can be any revision format of Git and the can be a \n> number, or a posix regex, just like what 'git blame' do.\n\nSee above.\n\n> 7. And we will support code copy detect, too. The option which control \n> whether we trace code copy does exist in current 'git log', which is \n> the option '-C'. Similiarly, one '-C' is used to trace code copy of \n> new added code inside one commit. Two '-C' will trace any code copy \n> inside commit tree.\n\nAgain, do not be too specific about details that have to be fleshed out \nwhile working on the project. For example, we do not know yet whether it \nwould make more sense to look for code movements automatically when we \ndetected a deletion, and maybe fall back automatically to detecting code \ncopies when we found an inter-file move.\n\n> ==Design and implementation==\n> Git store all the blobs instead of code delta, so we should traverse\n> the commit history and directly access the tree/blob objects to\n> compute the code delta and search for the diff which contains the\n> interested lines.\n\ns/ed/ing/\n\n> Since git use libxdiff to format its diff file, we should iterate \n> through all xdiff's diff blocks and find what the code looks like before \n> the commit. This will be done using the callback mechanism. Here, we \n> will find a new line range which is the origin code before this commit. \n> And then start another search from the current commit and the new line \n> range.\n>\n> Recursively, we can find all the modification history. We will stop when \n> we find that the current interested line range is added from scratch and \n> is not moved from other place of the file. Here, if the user want to \n> trace code copy, more work will be done to find the possible code copy. \n> We may also stop the traverse when we reach the max search depth.\n>\n> Also, if the thread of change history split into two or more commits, we \n> stop and provide the users all the related commits and corresponding \n> line range.\n\nGood.\n\n> Generally,\n> 1. New callback for xdi_diff to parse the diff hunk and store line\n> level history info.\n> 2. builtin/line-log.c will be added to complete most of the new features.\n> 3. builtin/log.c will be changed to add this new utility to the front end.\n> 4. Documents will be updated to introduce this new tool.\n\nGood.\n\n> =====Milestones and Timeline=====\n> In this summer, we will add support of line level history browser for\n> only one file. The multiple ranges support is currently not in this\n> project.\n> \n> The milestones of the project are:\n> 1. Simple modification change history.\n\nIMHO this should be split into\n\n\t1a) have an initial version which does nothing else than parse\n\t git-log options and a single additional -L, requiring exactly\n\t one file to be specified\n\n\t1b) implement the xdiff callback and identify the commits touching\n\t the line range (this is not completely trivial due to merges)\n\n> 2. Code movement inside one file detect.\n\nAgain, this has to be split a little bit. Code can split, and it can also \nunite. So, a single line range can easily become multiple ones.\n\n> 3. Code movement inside one commit but not a file.\n\ns/but not a file/between files/\n\n> 4. Code copy of modified file in one commit.\n\nYou mean code copy from somewhere in the same file?\n\n> 5. Code copy of any place in one commit tree.\n> 6. Fuzzy matching support.\n\nFor fuzzy matching support, I would add some ideas, such as trying to \nmatch alpha-numeric characters, or matching longest words or some such. \nAlso mention the possibility that this might be infeasible. In any case, \ngive an example what case this is trying to help with.\n\n> And the timeline will be:\n> April 26 - May 23: Catch up with Git code base and study the\n> implementation of blame.c and log.c thouroughly.\n\nHmm. Maybe it would be better to be more precise. Like: 1st week: follow \nthe bird's eye view on Git's source code. 2nd week, analyze the rev-list \nmachinery (probably first looking at the code of merge-base, for easier \nunderstanding), 3rd week, have a look at builtin/log.c, 4th week, \nunderstand blame.c\n\n> May 24 - June 21 : Complete a version which supports code\n> modifcation trace but without code movement and code copy support.\n> \n> June 22 - June 29: Complete a version which supports code movement\n> inside one file.\n> \n> June 30 - July 7: Complete a version which supports code movement\n> between files inside one commit.\n> \n> July 8 - July 15: Complete a version which supports code copy of\n> modified file in one commit.\n> \n> July 16 - July 23: Complete a version which supports code copy of\n> any file in one commit tree.\n> \n> July 24 - August 7: Complete fuzzy matching of code movement and copy detect.\n\nThis should probably adjusted a bit to my suggestions above.\n\nCiao,\nDscho\n"},{"id":"137579","messageId":"alpine.DEB.1.00.1003222025220.7596@pacific.mpi-cbg.de","threadId":"23105","inReplyTo":"fabb9a1e1003221138n50e30adaxe91ac9503932c35a@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2010-03-22T19:26:24Z","receivedAt":"2010-03-22T19:26:24Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 22 Mar 2010, Sverre Rabbelier wrote:\n\n> On Mon, Mar 22, 2010 at 19:21, Johannes Schindelin \n> wrote:\n> > As to fuzzy matching of lines that could not be attributed otherwise, \n> > I think that that will require a lot of playing around with different \n> > ideas. A simple Levenshtein-Damerau is highly unlikely to be enough.\n> \n> I'd recommend making this either the last milestone, or not a milestone \n> at all. As I noticed with git-stats such metrics might not exist at all \n> (or at least be too hard to find/implement), and it's quite a bummer to \n> not be able to implement your primary milestone ;).\n\nIndeed. TBH, I wanted to ask you to assist in that part of the project. \nYou probably can give a good overview over what does not work, and why.\n\nCiao,\nDscho\n"},{"id":"137583","messageId":"fabb9a1e1003221321o34c6c63n8bf874e4259f19f4@mail.gmail.com","threadId":"23105","inReplyTo":"alpine.DEB.1.00.1003222025220.7596@pacific.mpi-cbg.de","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2010-03-22T20:21:42Z","receivedAt":"2010-03-22T20:21:42Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Mon, Mar 22, 2010 at 20:26, Johannes Schindelin\n wrote:\n> Indeed. TBH, I wanted to ask you to assist in that part of the project.\n> You probably can give a good overview over what does not work, and why.\n\nBack then I think we even talked about teaching git log to find code\nmoves? I have some silly code online on repo.or.cz even. maybe.\nAnyway, my main problem there was finding a heuristic that would give\na sensible answer both in small _and_ large moves. It might be worth\ninvestigating two or more metrics instead, one that works for (very)\nsmall chunks of code, and thus require an almost exact match, then\nperhaps a somewhat linear function (the longer the block moved, the\nmore 'fuzz' you allow), and maybe after some size, say practical\nfull-file moves, use an algorithm similar to what rename detection\ndoes. \n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"137607","messageId":"41f08ee11003222301y569a5972q3c67d10c77abe27a@mail.gmail.com","threadId":"23105","inReplyTo":"20100322081003.GA26535@progeny.tock","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-23T06:01:04Z","receivedAt":"2010-03-23T06:01:04Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi,\n> Hmm, I can imagine some (mutually inconsistent) heuristics:\n>\n>  - Suppose in the blamed commit a single isolated line changed.  Then\n>   it is clear where to look next.\n>\n>  - If the mystery code is at the beginning of the file (resp.\n>   beginning of a diff -C0 hunk), maybe it was based on the line at the\n>   same position within the previous commit.\n>\n>  - Take the line with the lowest Levenshtein distance from the mystery\n>   code.\n>\n>  - Expect certain common patterns of change: substituted words,\n>   whitespace changes, added arguments for a function, things like that.\n>\n> That said, I still don’t have a clear picture of a basic strategy.\n\nI can't understand fully about your above strategy. I think we can\ncategory the code change into two cases:\n1. The diff looks like:\n\n@@ -1008,29 +1000,29 @@ int cmd_format_patch(int argc, const char\n**argv, const char *prefix)\n add_signoff = xmemdupz(committer, endpos - committer + 1);\n }\n\n- for (i = 0; i < extra_hdr_nr; i++) {\n- strbuf_addstr(&buf, extra_hdr[i]);\n+ for (i = 0; i < extra_hdr.nr; i++) {\n+ strbuf_addstr(&buf, extra_hdr.items[i].string);\n strbuf_addch(&buf, '\\n');\n }\n\n\nie: there is both deletion and addition in a change. And this means we\nmodify some lines of the code. So, what we do will be tracing the two\n'minus' lines and then find another diff. Start trace from that diff\nrecursively.\nYes, the new added code may also be moved or copied from other place.\nBut, I think here, we should focus on the lines before this changeset.\n\n2. The diff looks like:\n\n@@ -879,9 +885,12 @@ int cmd_grep(int argc, const char **argv, const\nchar *prefix)\n opt.regflags = REG_NEWLINE;\n opt.max_depth = -1;\n\n+ strcpy(opt.color_context, \"\");\n strcpy(opt.color_filename, \"\");\n+ strcpy(opt.color_function, \"\");\n strcpy(opt.color_lineno, \"\");\n strcpy(opt.color_match, GIT_COLOR_BOLD_RED);\n\nThis means, the code here is added from scratch. Here, I think we have\nthree options.\n1. Find if the new code is moved here from other place.\n2. Find if the new code is copied from other place.\n3. We find the end of the history, so stop here.\n\nThe problems remain how do we find the copied/moved code. The new\nadded code may be copied/moved from multiple place with little\nchanges.\n\nI hope I understand the requirement of the line-level browser, could\nyou please point it out if I have made some mistake?\n\nRegards!\nBo\n"},{"id":"137608","messageId":"41f08ee11003222308r63e2edabu5f352a50eea6e0a7@mail.gmail.com","threadId":"23105","inReplyTo":"alpine.DEB.1.00.1003221924110.7596@pacific.mpi-cbg.de","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-23T06:08:54Z","receivedAt":"2010-03-23T06:08:54Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi,\n\n>> Note that, the history may not always be a single thread of commits.\n>> If there are more than one commit which produce the specified line\n>> range, the thread of history will split.\n>\n> Do not forget the case where there are more than one source of a code\n> move. Think \"refactoring\".\n\nYeah, I really ignore such a condition. Thanks a lot!\nAnd any new added code can be moved/copied from multiple source. This\nwill really be a new problem for the fuzzy matching.\n\n>> =====Work and technical issues=====\n>> ==Command options==\n>> This new feature should be used for exploring the history of changes\n>> for certain line range of code in one file.\n>>\n>> git log [-m] [-I] [-d depth] [--fuzzy]  -L file1@rev1:> pos>,  file2@rev2:,\n>\n> I would like this not to be specified too much here. For example, we do\n> not know yet, whether the matching will be fuzzy, or whether we find\n> something cleverer than that.\n\nOk, I will focus on express what I will support instead of command line options.\n\n>\n>> =====Milestones and Timeline=====\n>> In this summer, we will add support of line level history browser for\n>> only one file. The multiple ranges support is currently not in this\n>> project.\n>>\n>> The milestones of the project are:\n>> 1. Simple modification change history.\n>\n> IMHO this should be split into\n>\n>        1a) have an initial version which does nothing else than parse\n>            git-log options and a single additional -L, requiring exactly\n>            one file to be specified\n>\n>        1b) implement the xdiff callback and identify the commits touching\n>            the line range (this is not completely trivial due to merges)\n>\n\nI will make a more specified milestones and timeline, thanks!\n\nRegards!\nBo\n"},{"id":"137609","messageId":"41f08ee11003222327u3a709249r7acf176fb31aeee3@mail.gmail.com","threadId":"23105","inReplyTo":"alpine.DEB.1.00.1003221924110.7596@pacific.mpi-cbg.de","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-23T06:27:10Z","receivedAt":"2010-03-23T06:27:10Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi,\n>> 4. Code copy of modified file in one commit.\n>\n> You mean code copy from somewhere in the same file?\n\nI am sorry not. I mean, lines copied from other files that were\nmodified in the same commit. Just what 'blame' means with one '-C'\noptions.\n\n>\n>> 5. Code copy of any place in one commit tree.\n>> 6. Fuzzy matching support.\n>\n> For fuzzy matching support, I would add some ideas, such as trying to\n> match alpha-numeric characters, or matching longest words or some such.\n> Also mention the possibility that this might be infeasible. In any case,\n> give an example what case this is trying to help with.\n>\n\nI think fuzzy matching is used to track multiple lines of\ncopy/movement, even with little change of the source.\nFor example, one C function is moved from file1 to file2 and get\nrenamed. In this case, most of the origin code of function body will\nremain unchanged except the function name. So, simply compare the new\nadded lines with original code line by line and permit some percent of\nmismatch will help to find this kind of movement.\n\n\nRegards!\nBo\n"},{"id":"137621","messageId":"m38w9jjqqd.fsf@localhost.localdomain","threadId":"23105","inReplyTo":"41f08ee11003222301y569a5972q3c67d10c77abe27a@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-03-23T10:08:28Z","receivedAt":"2010-03-23T10:08:28Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Bo Yang writes:\n> Jonathan Nieder writes:\n\n> > Hmm, I can imagine some (mutually inconsistent) heuristics:\n> >\n> > �- Suppose in the blamed commit a single isolated line changed. �Then\n> > � it is clear where to look next.\n> >\n> > �- If the mystery code is at the beginning of the file (resp.\n> > � beginning of a diff -C0 hunk), maybe it was based on the line at the\n> > � same position within the previous commit.\n> >\n> > �- Take the line with the lowest Levenshtein distance from the mystery\n> > � code.\n> >\n> > �- Expect certain common patterns of change: substituted words,\n> > � whitespace changes, added arguments for a function, things like that.\n> >\n> > That said, I still don’t have a clear picture of a basic strategy.\n> \n> I can't understand fully about your above strategy. I think we can\n> category the code change into two cases:\n>\n> 1. The diff looks like this:\n> \n> @@ -1008,29 +1000,29 @@ int cmd_format_patch(int argc, const char\n> **argv, const char *prefix)\n> add_signoff = xmemdupz(committer, endpos - committer + 1);\n> }\n> \n> - for (i = 0; i < extra_hdr_nr; i++) {\n> - strbuf_addstr(&buf, extra_hdr[i]);\n> + for (i = 0; i < extra_hdr.nr; i++) {\n> + strbuf_addstr(&buf, extra_hdr.items[i].string);\n> strbuf_addch(&buf, '\\n');\n> }\n\nErrr... how the first line in preimage differs from first line in\npostimage? The look as if they are the same:\n\n - for (i = 0; i < extra_hdr_nr; i++) {\n + for (i = 0; i < extra_hdr.nr; i++) {\n\n> \n> i.e. there is both deletion and addition in a change. And this means we\n> modify some lines of the code. So, what we do will be tracing the two\n> 'minus' lines and then find another diff. Start trace from that diff\n> recursively.\n>\n> Yes, the new added code may also be moved or copied from other place.\n> But, I think here, we should focus on the lines before this changeset.\n\nThe problem is when you are asking about tracking a subset of lines\nthat appear in postimage of a patch. For example if we ask for\nhistory of\n\n strbuf_addstr(&buf, extra_hdr.items[i].string);\n\nline, should we track history of\n\n for (i = 0; i < extra_hdr.nr; i++) {\n\nline which appears in relevant diff chunk? If not, how we should\ndetect which line in preimage (if any) corresponds to given line in\npostimage?\n\n> 2. The diff looks like:\n> \n> @@ -879,9 +885,12 @@ int cmd_grep(int argc, const char **argv, const\n> char *prefix)\n> opt.regflags = REG_NEWLINE;\n> opt.max_depth = -1;\n> \n> + strcpy(opt.color_context, \"\");\n> strcpy(opt.color_filename, \"\");\n> + strcpy(opt.color_function, \"\");\n> strcpy(opt.color_lineno, \"\");\n> strcpy(opt.color_match, GIT_COLOR_BOLD_RED);\n> \n> This means, the code here is added from scratch. Here, I think we have\n> three options.\n> 1. Find if the new code is moved here from other place.\n> 2. Find if the new code is copied from other place.\n> 3. We find the end of the history, so stop here.\n> \n> The problems remain how do we find the copied/moved code. The new\n> added code may be copied/moved from multiple place with little\n> changes.\n\nI guess that you could take a look at how git-blame does handle\nthis... but I think you would get something like generalization of\nordinary patch, where preimage of chunk can come from different place\n/ different file.\n\n\nP.S. I like it that you provide real-life examples. They really help\n with understanding what are you talking about.\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"137622","messageId":"41f08ee11003230338y5cf70a13xa649e67071bc2a72@mail.gmail.com","threadId":"23105","inReplyTo":"m38w9jjqqd.fsf@localhost.localdomain","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-23T10:38:42Z","receivedAt":"2010-03-23T10:38:42Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi,\n\n>\n> Errr... how the first line in preimage differs from first line in\n> postimage?  The look as if they are the same:\n>\n>  -       for (i = 0; i < extra_hdr_nr; i++) {\n>  +       for (i = 0; i < extra_hdr.nr; i++) {\n>\n\nMaybe some space... :)\n>\n> The problem is when you are asking about tracking a subset of lines\n> that appear in postimage of a patch.  For example if we ask for\n> history of\n>\n>                  strbuf_addstr(&buf, extra_hdr.items[i].string);\n>\n> line, should we track history of\n>\n>          for (i = 0; i < extra_hdr.nr; i++) {\n>\n> line which appears in relevant diff chunk?  If not, how we should\n> detect which line in preimage (if any) corresponds to given line in\n> postimage?\n\nIf I understand correctly, that is as following.\n\n@@ -1008,29 +1000,29 @@ int cmd_format_patch(int argc, const char\n**argv, const char *prefix)\n add_signoff = xmemdupz(committer, endpos - committer + 1);\n }\n\n- for (i = 0; i < extra_hdr_nr; i++) {\n- strbuf_addstr(&buf, extra_hdr[i]);\n+ for (i = 0; i < extra_hdr.nr; i++) {\n+ strbuf_addstr(&buf, extra_hdr.items[i].string);\n strbuf_addch(&buf, '\\n');\n }\n\nHere, the user only ask for tracking the strbuf_addstr line. And we\nfind the above diff hunk. I think we can then find what the line would\nbe in the preimage using @@ -1008,29 +1000,29 @@. The strbuf_addstr\nis located at\n1000(the postimage start line number)\n+3(the context number)\n+1(the number of lines '+' before this line) in the postimage,\nand we can calculate its line number in the preimage by the same way\n1008\n+3\n+1(the number of lines with '-' before this line).\n\nHow do you think about this method?\n\nRegards!\nBo\n"},{"id":"137626","messageId":"201003231222.40745.jnareb@gmail.com","threadId":"23105","inReplyTo":"41f08ee11003230338y5cf70a13xa649e67071bc2a72@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-03-23T11:22:38Z","receivedAt":"2010-03-23T11:22:38Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Tue, 23 Mar 2010, Bo Yang wrote:\n\nPlease do not forget to include attribution line, like the one I have\nadded below:\n \n> Jakub Narebski wrote:\n\n> > The problem is when you are asking about tracking a subset of lines\n> > that appear in postimage of a patch.  For example if we ask for\n> > history of\n> >\n> >                  strbuf_addstr(&buf, extra_hdr.items[i].string);\n> >\n> > line, should we track history of\n> >\n> >          for (i = 0; i < extra_hdr.nr; i++) {\n> >\n> > line which appears in relevant diff chunk?  If not, how we should\n> > detect which line in preimage (if any) corresponds to given line in\n> > postimage?\n> \n> If I understand correctly, that is as following.\n> \n> @@ -1008,29 +1000,29 @@ int cmd_format_patch(int argc, const char\n> **argv, const char *prefix)\n> add_signoff = xmemdupz(committer, endpos - committer + 1);\n> }\n> \n> - for (i = 0; i < extra_hdr_nr; i++) {\n> - strbuf_addstr(&buf, extra_hdr[i]);\n> + for (i = 0; i < extra_hdr.nr; i++) {\n> + strbuf_addstr(&buf, extra_hdr.items[i].string);\n> strbuf_addch(&buf, '\\n');\n> }\n> \n> Here, the user only ask for tracking the strbuf_addstr line. And we\n> find the above diff hunk. I think we can then find what the line would\n> be in the preimage using @@ -1008,29 +1000,29 @@. The strbuf_addstr\n> is located at\n> 1000(the postimage start line number)\n> +3(the context number)\n> +1(the number of lines '+' before this line) in the postimage,\n> and we can calculate its line number in the preimage by the same way\n> 1008\n> +3\n> +1(the number of lines with '-' before this line).\n> \n> How do you think about this method?\n\nThis would work with the simplest case, but not in more complicated\ncases, like for example preimage and postimage with different size.\n\nTake for example the following chunk (fragment):\n\ndiff --git a/run-command.c b/run-command.c\nindex 2feb493..3206d61 100644\n--- a/run-command.c\n+++ b/run-command.c\n@@ -67,19 +67,21 @@ static int child_notifier = -1;\n \n static void notify_parent(void)\n {\n-\twrite(child_notifier, \"\", 1);\n+\tssize_t unused;\n+\tunused = write(child_notifier, \"\", 1);\n }\n \n static NORETURN void die_child(const char *err, va_list params)\n\nIf you follow ssize_t line, it is created. If you follow line with\nwrite, which is 2nd line in postimage, its previous version is 1st\nline in preimage.\n\n\nAnother example would be reordering of lines, or reordering with\nsome change.\n\n-- \nJakub Narebski\nPoland\n"},{"id":"137627","messageId":"A612847CFE53224C91B23E3A5B48BAC73EC0B01FA0@xmail3.se.axis.com","threadId":"23105","inReplyTo":"41f08ee11003230338y5cf70a13xa649e67071bc2a72@mail.gmail.com","subject":"RE: GSoC draft proposal: Line-level history browser","fromName":"Peter Kjellerstedt","fromEmail":"peter.kjellerstedt@axis.com","sentAt":"2010-03-23T12:02:17Z","receivedAt":"2010-03-23T12:02:17Z","isPatch":false,"sender":{"key":"peter.kjellerstedt@axis.com","avatar":"https://gravatar.com/avatar/6d5a0182283c8eccd7b134a54dbfd5f30038f3ad4d38b96f424884b614a61ca2?d=mp&s=160"},"body":"> -----Original Message-----\n> From: git-owner@vger.kernel.org [mailto:git-owner@vger.kernel.org] On\n> Behalf Of Bo Yang\n> Sent: den 23 mars 2010 11:39\n> To: Jakub Narebski\n> Cc: Jonathan Nieder; Junio C Hamano; gitzilla@gmail.com; Alex Riesen;\n> git@vger.kernel.org\n> Subject: Re: GSoC draft proposal: Line-level history browser\n> \n> Hi,\n> \n> >\n> > Errr... how the first line in preimage differs from first line in\n> > postimage?  The look as if they are the same:\n> >\n> >  -       for (i = 0; i < extra_hdr_nr; i++) {\n> >  +       for (i = 0; i < extra_hdr.nr; i++) {\n> >\n> \n> Maybe some space... :)\n\nLook more closely. Hint: a _ is not the same as a . ;)\n\n//Peter\n"},{"id":"137629","messageId":"41f08ee11003230523h2c5f93b1l7e94d6c85a93a706@mail.gmail.com","threadId":"23105","inReplyTo":"201003231222.40745.jnareb@gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-23T12:23:36Z","receivedAt":"2010-03-23T12:23:36Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi,\n\nOn Tue, Mar 23, 2010 at 7:22 PM, Jakub Narebski wrote:\n>\n> This would work with the simplest case, but not in more complicated\n> cases, like for example preimage and postimage with different size.\n>\n> Take for example the following chunk (fragment):\n>\n> diff --git a/run-command.c b/run-command.c\n> index 2feb493..3206d61 100644\n> --- a/run-command.c\n> +++ b/run-command.c\n> @@ -67,19 +67,21 @@ static int child_notifier = -1;\n>\n>  static void notify_parent(void)\n>  {\n> -       write(child_notifier, \"\", 1);\n> +       ssize_t unused;\n> +       unused = write(child_notifier, \"\", 1);\n>  }\n>\n>  static NORETURN void die_child(const char *err, va_list params)\n>\n> If you follow ssize_t line, it is created.  If you follow line with\n> write, which is 2nd line in postimage, its previous version is 1st\n> line in preimage.\n>\n>\n> Another example would be reordering of lines, or reordering with\n> some change.\n\nAh, yes, you are right.\n\nAnd now, I really get the difference between the understanding about\nline level browser of us. :) When users want to browsing the history\nof some line or line range, you want to display only the related lines\nto them, but I want to display the minim diff hunk to them. :)\nAnd I think displaying the minimum diff hunk is sensible and feasible.\nCould you please tell me how do you think about this?\n\nRegards!\nBo\n"},{"id":"137633","messageId":"201003231449.42190.jnareb@gmail.com","threadId":"23105","inReplyTo":"41f08ee11003230523h2c5f93b1l7e94d6c85a93a706@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-03-23T13:49:40Z","receivedAt":"2010-03-23T13:49:40Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Tue, Mar 23, 2010, Bo Yang wrote:\n> On Tue, Mar 23, 2010 at 7:22 PM, Jakub Narebski wrote:\n> >\n> > This would work with the simplest case, but not in more complicated\n> > cases, like for example preimage and postimage with different size.\n> >\n> > Take for example the following chunk (fragment):\n> >\n> > diff --git a/run-command.c b/run-command.c\n> > index 2feb493..3206d61 100644\n> > --- a/run-command.c\n> > +++ b/run-command.c\n> > @@ -67,19 +67,21 @@ static int child_notifier = -1;\n> >\n> >  static void notify_parent(void)\n> >  {\n> > -       write(child_notifier, \"\", 1);\n> > +       ssize_t unused;\n> > +       unused = write(child_notifier, \"\", 1);\n> >  }\n> >\n> >  static NORETURN void die_child(const char *err, va_list params)\n> >\n> > If you follow ssize_t line, it is created.  If you follow line with\n> > write, which is 2nd line in postimage, its previous version is 1st\n> > line in preimage.\n> >\n> >\n> > Another example would be reordering of lines, or reordering with\n> > some change.\n> \n> Ah, yes, you are right.\n> \n> And now, I really get the difference between the understanding about\n> line level browser of us. :) When users want to browsing the history\n> of some line or line range, you want to display only the related lines\n> to them, but I want to display the minim diff hunk to them. :)\n> And I think displaying the minimum diff hunk is sensible and feasible.\n> Could you please tell me what do you think about this?\n\nThe problem is not what (part of) diff you would display. The problem\nis with following the history (with history simplification). *After*\ndisplaying diff / chunk / chunk fragment, do we further follow history\nof the whole preimage? Or do we follow history of line pre-change\nstarting from blamed commit?\n\nIf we *don't* follow the history, how line-level browser is different\nfrom (wrapped) git-blame?\n\n\nTry to come with the result of line-level history for some line in\ngit sources \"by hand\": this would help in discussion about what \nline-level history browser should do, and perhaps even be first test\nof it (see e.g. tests for git-blame).\n\n-- \nJakub Narebski\nPoland\n"},{"id":"137643","messageId":"41f08ee11003230823i1ca677b8q51b2413040c45c44@mail.gmail.com","threadId":"23105","inReplyTo":"201003231449.42190.jnareb@gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-23T15:23:39Z","receivedAt":"2010-03-23T15:23:39Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi,\n\nOn Tue, Mar 23, 2010 at 9:49 PM, Jakub Narebski wrote:\n> Try to come with the result of line-level history for some line in\n> git sources \"by hand\": this would help in discussion about what\n> line-level history browser should do, and perhaps even be first test\n> of it (see e.g. tests for git-blame).\n\nThanks for your advice of coming with a real example, Jakub! And I can\ngive a not too trivial one, :)\n\nIf you look at the pretty.c line 1032 line, you will find a line like:\n\nformat_commit_message(commit, user_format, sb, context);\n\nTake for example, we will trace the history of this line.\nWe will find that the first time this line appears:\n\n@@ -900,18 +900,18 @@ char *reencode_commit_message(const struct\ncommit *commit, const char **encoding\n...skipped...\n if (fmt == CMIT_FMT_USERFORMAT) {\n- format_commit_message(commit, user_format, sb, dmode);\n+ format_commit_message(commit, user_format, sb, context);\n return;\n }\nAnd we should trace the preimage, something like:\n if (fmt == CMIT_FMT_USERFORMAT) {\n format_commit_message(commit, user_format, sb, dmode);\n\nWe will find these below:\n@@ -770,7 +775,7 @@ void pretty_print_commit(enum cmit_fmt fmt, const struct com\n const char *encoding;\n\n if (fmt == CMIT_FMT_USERFORMAT) {\n- format_commit_message(commit, user_format, sb);\n+ format_commit_message(commit, user_format, sb, dmode);\n return;\n }\n\nAgain:\n+\n+ if (fmt == CMIT_FMT_USERFORMAT) {\n+ format_commit_message(commit, user_format, sb);\n+ return;\n+ }\n+\n\nHere, we find that the line is added from scratch and line level\nhistory browser will do a code movement and copy matching try to find\nwhether this line if moved from other files.\n\nAnd it is. In commit 93fc05eb9(Split off the pretty print stuff into\nits own file), some code is moved from commit.c to pretty.c and this\nline if from commit.c .\n\nOk, now, we will trace into commit.c for this line.\nAgain:\n char *reencoded;\n const char *encoding;\n- char *buf;\n\n- if (fmt == CMIT_FMT_USERFORMAT)\n- return format_commit_message(commit, user_format,\nbuf_p, space_p);\n+ if (fmt == CMIT_FMT_USERFORMAT) {\n+ format_commit_message(commit, user_format, sb);\n+ return;\n+ }\n\n encoding = (git_log_output_encoding\n ? git_log_output_encoding\n\nNow, we will trace the commit which produce the above preimage of the\ndiff hunk. And because there are four lines of the preimage in our\ntracing window. We should follow any commit which intersect with these\nfour lines. Fortunately, there is only one commit.\n\n@@ -1165,7 +1166,7 @@ unsigned long pretty_print_commit(enum cmit_fmt fmt,\n char *buf;\n\n if (fmt == CMIT_FMT_USERFORMAT)\n- return format_commit_message(commit, msg, buf_p, space_p);\n+ return format_commit_message(commit, user_format,\nbuf_p, space_p);\n\n encoding = (git_log_output_encoding\n ? git_log_output_encoding\n\n\nAgain, we find:\n\n if (fmt == CMIT_FMT_USERFORMAT)\n- return format_commit_message(commit, msg, buf, space);\n+ return format_commit_message(commit, msg, buf_p, space_p);\n\n encoding = (git_log_output_encoding\n\nAgain:\n char *encoding;\n\n+ if (fmt == CMIT_FMT_USERFORMAT)\n+ return format_commit_message(commit, msg, buf, space);\n+\n encoding = (git_log_output_encoding\n ? git_log_output_encoding\n\nAnd here, finally, we reach a place where the code is added from\nscratch and not copied/moved from other place.\n\nLine level history browser will just display all the related diff to\nusers and trace the code modification/move/copy.\n\nIt traces the preimage of the minimum related diff hunk carefully, if\nthere is any case that there are more than one commit intersect with\nthe preimage, we will stop and ask the users to select which way to go\non tracing.\n\nI hope this can help us to discuss the problem, thanks!\n\nRegards!\nBo\n"},{"id":"137661","messageId":"20100323185745.GA1382@progeny.tock","threadId":"23105","inReplyTo":"41f08ee11003222301y569a5972q3c67d10c77abe27a@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-03-23T18:57:45Z","receivedAt":"2010-03-23T18:57:45Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\n[reordering quoted text for convenience]\n\nBo Yang wrote:\n\n> I can't understand fully about your above strategy. I think we can\n> category the code change into two cases:\n\nThanks! What you said is much more coherent than the vague things I\nwrote.\n\n> 2. The diff looks like:\n[...]\n> This means, the code here is added from scratch. Here, I think we have\n> three options.\n> 1. Find if the new code is moved here from other place.\n> 2. Find if the new code is copied from other place.\n> 3. We find the end of the history, so stop here.\n\nIf the code is copied verbatim from elsewhere, this is something ‘git\nblame’ is already very good at. See [1].\n\nFuzzy matching is a big pain. ‘git blame’ knows how to ignore\nwhitespace. Dscho suggested counting common words. Maybe there are\nsome other ways. I think there is a real danger of getting lost in this\nproblem and wasting a lot of time, so although it is very interesting, I\nwould consider any progress in this area a bonus rather than a goal.\n\n> 1. The diff looks like:\n> \n> @@ -1008,29 +1000,29 @@ int cmd_format_patch(int argc, const char\n> **argv, const char *prefix)\n> add_signoff = xmemdupz(committer, endpos - committer + 1);\n> }\n> \n> - for (i = 0; i < extra_hdr_nr; i++) {\n> - strbuf_addstr(&buf, extra_hdr[i]);\n> + for (i = 0; i < extra_hdr.nr; i++) {\n> + strbuf_addstr(&buf, extra_hdr.items[i].string);\n> strbuf_addch(&buf, '\\n');\n> }\n> \n> \n> ie: there is both deletion and addition in a change. And this means we\n> modify some lines of the code. So, what we do will be tracing the two\n> 'minus' lines and then find another diff. Start trace from that diff\n> recursively.\n\nIf you can make a heuristic along these lines this work well, I think it\nwould be great. I imagine it might work very well for commits that made\nnice, small changes (like many of those in git.git). Jakub pointed out\nsome of the difficulties, and I like to hope your idea of “when in doubt,\ninclude more lines” may work well in many cases in git.git still.\n\nGood luck, and thank you for taking my crazy ideas seriously. :)\n\nRegards,\nJonathan\n\n[1] See v1.4.4-rc1~2 (Merge branch 'jc/pickaxe', 2006-11-07) and the\ncommits preceding it. About that series, Junio wrote:\n\n\tActually the plan is to make it do _true_ pickaxe,\n\talthough it will most likely end up either in dustbin or\n\treplace blame.\n\nIt replaced blame.\n\nI am not actually sure, but I assume “true pickaxe” refers to the\ngoals described in \nand the linked-to message.\n"},{"id":"137664","messageId":"20100323195724.GA1843@progeny.tock","threadId":"23105","inReplyTo":"41f08ee11003230823i1ca677b8q51b2413040c45c44@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-03-23T19:57:25Z","receivedAt":"2010-03-23T19:57:25Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Bo Yang wrote:\n\n> It traces the preimage of the minimum related diff hunk carefully, if\n> there is any case that there are more than one commit intersect with\n> the preimage, we will stop and ask the users to select which way to go\n> on tracing.\n\nThat might be necessary, but I will admit that I suspect it to be\nharder to make useful. One of the very nice things about ‘git log’ is\nthat it is easy to browse through history in a nonlinear way in a\npager (by using a pager’s search functionality). The “backend” ‘git\nrev-list’ is easy to write scripts with, also because of its simple\ninput and output.\n\nIf your program requires input from the user, how will it paginate its\noutput? Most pagers expect the standard input to be available for\ninput from the user.\n\nOne approach (I will not say it is a good one) to the problem of\nambiguous origins for a line is to blame _both_ parents. That is,\nstart following both lines of history in your revision walking.\nPerhaps higher-level tools like ‘git log --graph’ and gitk could\nvisually represent the branched history you are showing.\n\nAnother approach is to just choose one parent automatically: for\nexample, prefer the first parent, or assign some score representing\nthe relatedness of each parent and choose the most related one.\n\nJonathan\n"},{"id":"137672","messageId":"4BA937C9.9040004@gmail.com","threadId":"23105","inReplyTo":"20100323195724.GA1843@progeny.tock","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"A Large Angry SCM","fromEmail":"gitzilla@gmail.com","sentAt":"2010-03-23T21:51:05Z","receivedAt":"2010-03-23T21:51:05Z","isPatch":false,"sender":{"key":"gitzilla@gmail.com","avatar":"https://gravatar.com/avatar/354625c442439908ff3dd99757dee330e29e9df7847472384faf7a00add247fb?d=mp&s=160"},"body":"Jonathan Nieder wrote:\n> Bo Yang wrote:\n> \n>> It traces the preimage of the minimum related diff hunk carefully, if\n>> there is any case that there are more than one commit intersect with\n>> the preimage, we will stop and ask the users to select which way to go\n>> on tracing.\n> \n> That might be necessary, but I will admit that I suspect it to be\n> harder to make useful. One of the very nice things about ‘git log’ is\n> that it is easy to browse through history in a nonlinear way in a\n> pager (by using a pager’s search functionality). The “backend” ‘git\n> rev-list’ is easy to write scripts with, also because of its simple\n> input and output.\n> \n> If your program requires input from the user, how will it paginate its\n> output? Most pagers expect the standard input to be available for\n> input from the user.\n> \n> One approach (I will not say it is a good one) to the problem of\n> ambiguous origins for a line is to blame _both_ parents. That is,\n> start following both lines of history in your revision walking.\n> Perhaps higher-level tools like ‘git log --graph’ and gitk could\n> visually represent the branched history you are showing.\n> \n> Another approach is to just choose one parent automatically: for\n> example, prefer the first parent, or assign some score representing\n> the relatedness of each parent and choose the most related one.\n\nWhat I would like to see (and may be too much for a GSOC project) is the \n result to be a simplified commit graph with additional annotations of \nthe line range mappings that could be fed into something like a modified \ngitk to view the _history_ of the lines of interest.\n"},{"id":"137683","messageId":"41f08ee11003231930t3508994atb7cfc6bb354048@mail.gmail.com","threadId":"23105","inReplyTo":"20100323195724.GA1843@progeny.tock","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-24T02:30:06Z","receivedAt":"2010-03-24T02:30:06Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi,\n\nOn Wed, Mar 24, 2010 at 3:57 AM, Jonathan Nieder wrote:\n> Bo Yang wrote:\n>\n>> It traces the preimage of the minimum related diff hunk carefully, if\n>> there is any case that there are more than one commit intersect with\n>> the preimage, we will stop and ask the users to select which way to go\n>> on tracing.\n>\n> That might be necessary, but I will admit that I suspect it to be\n> harder to make useful.  One of the very nice things about ‘git log’ is\n> that it is easy to browse through history in a nonlinear way in a\n> pager (by using a pager’s search functionality).  The “backend” ‘git\n> rev-list’ is easy to write scripts with, also because of its simple\n> input and output.\n>\n> If your program requires input from the user, how will it paginate its\n> output?  Most pagers expect the standard input to be available for\n> input from the user.\n>\n> One approach (I will not say it is a good one) to the problem of\n> ambiguous origins for a line is to blame _both_ parents.  That is,\n> start following both lines of history in your revision walking.\n> Perhaps higher-level tools like ‘git log --graph’ and gitk could\n> visually represent the branched history you are showing.\n>\n> Another approach is to just choose one parent automatically: for\n> example, prefer the first parent, or assign some score representing\n> the relatedness of each parent and choose the most related one.\n\nBoth the approach is very precious for me. I think maybe I will\npropose the first one in my real proposal to Git, thanks a lot! You\nreally help my too much! Thanks!\n\nRegards!\nBo\n"},{"id":"137684","messageId":"41f08ee11003231939y390ea4cajdb73d09cf2dd6329@mail.gmail.com","threadId":"23105","inReplyTo":"20100323185745.GA1382@progeny.tock","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-24T02:39:33Z","receivedAt":"2010-03-24T02:39:33Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"HI,\n\nOn Wed, Mar 24, 2010 at 2:57 AM, Jonathan Nieder wrote:\n>\n> If you can make a heuristic along these lines this work well, I think it\n> would be great.  I imagine it might work very well for commits that made\n> nice, small changes (like many of those in git.git).  Jakub pointed out\n> some of the difficulties, and I like to hope your idea of “when in doubt,\n> include more lines” may work well in many cases in git.git still.\n>\n> Good luck, and thank you for taking my crazy ideas seriously. :)\n>\n> Regards,\n> Jonathan\n>\n> [1] See v1.4.4-rc1~2 (Merge branch 'jc/pickaxe', 2006-11-07) and the\n> commits preceding it.  About that series, Junio wrote:\n>\n>        Actually the plan is to make it do _true_ pickaxe,\n>        although it will most likely end up either in dustbin or\n>        replace blame.\n>\n> It replaced blame.\n>\n> I am not actually sure, but I assume “true pickaxe” refers to the\n> goals described in \n> and the linked-to message.\n\nI have looked over the article and the message from Linus, it really\nhelp me very much. The message and article pointed out most of the\nthings a line level tool should do, and I am happy to find that it is\nsimilar with my proposal. :) Thanks again for your precious advice and\nI think I can come up a better proposal, now. Thanks!\n\nRegards!\nBo\n"},{"id":"137685","messageId":"20100324040214.GA8755@progeny.tock","threadId":"23105","inReplyTo":"41f08ee11003231939y390ea4cajdb73d09cf2dd6329@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-03-24T04:02:15Z","receivedAt":"2010-03-24T04:02:15Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Bo Yang wrote:\n> On Wed, Mar 24, 2010 at 2:57 AM, Jonathan Nieder wrote:\n\n>> I am not actually sure, but I assume “true pickaxe” refers to the\n>> goals described in \n>> and the linked-to message.\n>\n> I have looked over the article and the message from Linus, it really\n> help me very much.\n\nOkay, so now I looked over that thread again. I found this [1]:\n\n \n\nIt’s for fuzzy matching of a certain kind. The latest version is under\nthe GPLv3, unfortunately for us. I would still like to reiterate my\nwarning to not get sidetracked on this, but maybe it would be pleasant\nreading.\n\nEnjoy,\nJonathan\n\n[1] Thanks, Linus.\nhttp://thread.gmane.org/gmane.comp.version-control.git/27/focus=225\n"},{"id":"138049","messageId":"41f08ee11003282114m34aa0f61w536b996dce6cecab@mail.gmail.com","threadId":"23105","inReplyTo":"201003282120.40536.trast@student.ethz.ch","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-29T04:14:27Z","receivedAt":"2010-03-29T04:14:27Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi Thomas,\nOn Mon, Mar 29, 2010 at 3:20 AM, Thomas Rast wrote:\n> Hi Bo\n>\n> I have one specific question about the draft project description:\n>\n> You wrote:\n>> And the timeline will be:\n>> April 26 - May 23:   Catch up with Git code base and study the\n>> implementation of blame.c and log.c thouroughly.\n>>\n>> May 24 - June 21 :   Complete a version which supports code\n>> modifcation trace but without code movement and code copy support.\n>>\n>> June 22 - June 29:   Complete a version which supports code movement\n>> inside one file.\n>>\n>> June 30 - July 7:    Complete a version which supports code movement\n>> between files inside one commit.\n>>\n>> July 8 - July 15:    Complete a version which supports code copy of\n>> modified file in one commit.\n>>\n>> July 16 - July 23:   Complete a version which supports code copy of\n>> any file in one commit tree.\n>>\n>> July 24 - August 7:  Complete fuzzy matching of code movement and copy detect.\n>\n> Where are you taking those numbers from?\n>\n> (I'm fine if the answer is \"I'm making them up from whole cloth\" but I\n> want to know anyway :-P)\n\nYou mean the dates? They are made up according on 'GSoC's timeline'\nand my estimation about the workload of each milestone.\n\nAnd this is the draft proposal, after a long thread of discussion, the\ntimeline and milestone change much. The fuzzy matching milestone will\nbecome a bonus milestone instead of a primary GSoC milestone. And I\nthink it may help that I provide a newest version of it, I paste it in\nthe end of the email.\n\nAnd I will appreciate any feedback from you. Especially about the\nimplementation section :)\n\n\nRegards!\nBo\n\n-------------------------------------------------------------------------\nDraft proposal(v3): Line-level History Browser\n\n=====Purpose of this project=====\n\"git blame\" can tell us who is responsible for a line of code, but it\ncan't help if we want to get the detail of how the lines of code have\nevolved as what it is now. For example, in Git, commit 93fc05e(Split\noff the pretty print stuff into its own file) split out\npretty_print_commit() from commit.c into pretty.c, and it is hard to\nverify without much hassle that the code split was really only a code\nsplit, rather than a split with an evil change.\n\nThis project will add a new feature for 'git log' to display line\nlevel history. It can trace the history of any line range of certain\nfile at any revision. And for each history entry, it will provide the\ncommits, the diff block which contains changes of users' interested\nlines.\n\nThis utility will trace all the modification history of interested\nlines and stop until it finds the root of the lines, which is a point\nwhere all the new code is added from scratch. Also, the users can\nspecify how deeply he wants this utility to trace. And this tool will\nalso follow the code movement and copy inside one commit, too.\n\nNote that, the history may not always be a single thread of commits.\nIf there are more than one commits which produce the specified line\nrange, or there are more than one source of code move/copy, the thread\nof history will split. And this utility may stop and provide all\ncommits with its code changes to the user, let the user to select\nwhich one to trace next. Or, it may also use 'git log --graph' way to\ndisplay the splitted history, we will provide options to control this.\n\n=====Work and technical issues=====\n==Scenario==\nFor how we use the line level browser and how the utility should act\nto us, here is an scenario:\nhttp://article.gmane.org/gmane.comp.version-control.git/143024/match=line+level+history+browser\nIt contains code movement between files but not code copy and fuzzy matching.\n\n==Features==\nThis new feature should be used for exploring the history of changes\nfor certain line range of code in one file. Following features will be\nsupported:\n1. Follow history of code modification of any single line range\nstarting from any revision. The above scenario provide a good example\nfor what this function used for and how it acts with users.\n\n2. Follow code movement inside one file. And follow code movement\nbetween files optionally for performance reason. With code movement\ndetect, we can find code refactoring easily just like what the above\nscenario do.\n\n3. Provide a configurable context to users, display only the 'user\ninterested lines' diff block or display the whole diff with the\ninterested area colorfully displayed.\n\n4. Detect code copy optionally. This may help us to understand why\nsome code is here and help on code refactoring. For example, we can\nalways make some 'usually copied code' a function.\n\n5. Simply fuzzy matching for code move/copy. Provide an option to\ncontrol whether we start a fuzzy matching for performance reason. This\ncan help us to find whether some code is really literally moved to\nhere or with some evil changes. And this may also help in some\nsituation like if we move some Java class to another file with only\nits class name changed. Anyway, fuzzy matching can help much on code\ndetection. And there can be many fuzzy detect strategies, but we will\nonly try to support the simplest one in this summer for time reason.\nMaybe a strategy like: 90% of the lines between two ranges of code are\nidentical or 90% of words are identical. This will be discussed again\nbefore coding I think.\n\n6. Provide a configurable way for how to display the history. A 'git\nlog --graph' way or stop to ask users when we meet history splitting.\n\n7. Reuse 'git log' existing options as many as possible.\n\n==Design and implementation==\nGit store all the blobs instead of code delta, so we should traverse\nthe commit history and directly access the tree/blob objects to\ncompute the code delta and search for the diff which contains the\ninteresting lines. Since git use libxdiff to format its diff file, we\nshould iterate through all xdiff's diff blocks and find what the code\nlooks like before the commit. This will be done using the callback\nmechanism. Here, we will find a new line range which is the origin\ncode before this commit. And then start another search from the\ncurrent commit and the new line range. Recursively, we can find all\nthe modification history. We will stop when we find that the current\ninterested line range is added from scratch and is not moved from\nother place of the file. Here, if the user want to trace code copy,\nmore work will be done to find the possible code copy. We may also\nstop the traverse when we reach the max search depth. Also, if the\nthread of change history split into two or more commits, we stop and\nprovide the users all the related commits and corresponding line\nrange.\n\nGenerally,\n1. New callback for xdi_diff to parse the diff hunk and store line\nlevel history info.\n2. builtin/line-log.c will be added to complete most of the new features.\n3. builtin/log.c will be changed to add this new utility to the front end.\n4. Documents will be updated to introduce this new tool.\n\n=====Milestones and Timeline=====\nIn this summer, we will add support of line level history browser for\nonly one file. The multiple ranges support is currently not in this\nproject.\n\nThe milestones of the project are:\n1. Simple modification change history.\n1a) Have an initial version which does nothing else than parse git-log\noptions and a single additional -L, requiring exactly one file to be\nspecified\n\n1b) Implement the xdiff callback and identify the commits touching the\nline range\n\n1c) Implement a workable line level log browser\n\n2. Code movement inside one file.\n2a) Support the whole section of code literally move.\n\n2b) Support code movement with splitting.\n\n3c) Support code movement with code uniting.\n\n3. Code movement inside one commit between files.\n\n4. Code lines copied from other files that were modified in the same commit.\n4a) Support the whole section of code literally copy.\n\n4b) Support code copy split and unite.\n\n5. Code copy of any place in one commit tree.\n\n6. Fuzzy matching support. Note that there is not a exact strategy for\nfuzzy matching and I would like this milestone a bonus one instead of\na primary milestone for GSoC. We will make a good support for this if\ntime allows.\n\n\nAnd the timeline will be:\nApril 26 - May 23:\n1st week, follow the bird's eye view on Git's source code.\n2nd week, have a look at the code of merge-base, analyze the rev-listmachinery\n3rd week, have a look at builtin/log.c,\n4th week, understand blame.c\n\nMay 24 - June 13 : Complete a version which supports code\nmodifcation trace but without code movement and code copy support. For\ndetail:\n1st week, milestone 1a, 1b\n2-3 week, milestone 1c\n\nJune 14 - July 11: Complete a version which supports code movement.\n1st week, milestone 2a\n2nd week, milestone 2b\n3rd week, milestone 2c\n4th week, milestone 3\n\nJuly 12 - August 1: Complete a version which supports code copy.\n1st week, milestone 4a\n2nd week, milestone 4b\n3rd week, milestone 5\n\nAugust 2 - August 14: Complete fuzzy matching of code movement and copy detect.\n\n\nAnd there is one milestone for each week nearly, so every week, I will\npost a stutas update to the list to let the community know the project\nprogress. And, patches will be sent for feature completion but not\nmilestone.\n\n=====About me=====\nI am Bo Yang, a Chinese graduate student majoring in Computer Science\nof NanKai University. I have touched some open source software since 5\nyears ago and began to contribute code to open source community from\nthree years ago. I have contributed to Mozilla/Mingw/Netsurf.\nTechnically, I am experienced in C/Bash Shell. I have attended last\nyear's GSoC with Netsurf project. In that project, I have completed\nmost of a DOM library in C.\nI begin to use git for source code revision from about two years ago.\nI use Git for track my Mozilla trunk source code. Because updating\nMozilla code by CVS in my school is very slow. So, I write one script\nto automatically updating the trunk with CVS at mid-night, when the\nnetwork flow is fast, on the server, and then use Git to maintain the\ncode. Then I use Git in my PC to clone/update the source code from my\nlocal server and that is very fast. I use Git to track my changes to\nthe code and some bug fixes. It is an excellent tool for\nbranch/history, I think.\nGit is my lovely daily tool for revision control. I have much\nexperience with it and have read \"Git Internals\" and also get some\nbasic knowledge about Git's code base. And I think the line-level\nhistory explorer is really suitable for me and I can make a good start\nwith this project in Git community.\n"},{"id":"138121","messageId":"201003292042.01549.trast@student.ethz.ch","threadId":"23105","inReplyTo":"41f08ee11003282114m34aa0f61w536b996dce6cecab@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2010-03-29T18:42:01Z","receivedAt":"2010-03-29T18:42:01Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Bo Yang wrote:\n> Draft proposal(v3): Line-level History Browser\n> \n> =====Purpose of this project=====\n> \"git blame\" can tell us who is responsible for a line of code, but it\n> can't help if we want to get the detail of how the lines of code have\n> evolved as what it is now. For example, in Git, commit 93fc05e(Split\n> off the pretty print stuff into its own file) split out\n> pretty_print_commit() from commit.c into pretty.c, and it is hard to\n> verify without much hassle that the code split was really only a code\n> split, rather than a split with an evil change.\n\nIs this really the right use-case? AFAICT the answer to the implied\nquestion is given by simply running 'git blame -M 93fc05e:pretty.c'.\n\n(Coming up with a better example should be easy; the way I currently\nthink of the feature means that it will mostly replace git-blame for\nme...)\n\n> Note that, the history may not always be a single thread of commits.\n> If there are more than one commits which produce the specified line\n> range, or there are more than one source of code move/copy, the thread\n> of history will split. And this utility may stop and provide all\n> commits with its code changes to the user, let the user to select\n> which one to trace next. Or, it may also use 'git log --graph' way to\n> display the splitted history, we will provide options to control this.\n\nI would, by far, prefer the latter. So far 'git log' has always been\nnoninteractive, and there's no really good way to make it interactive\nbecause it also goes through the pager. (In the case of blame this is\nsolved in 'git gui blame', which might also be a reasonable approach.)\n\nOTOH, if you can really fake a history walk, then just about any\nlog-oriented tool should be able to work with it. You'd get graphical\noutput for free with gitk and git log --graph. I haven't really\nthought through the ramifications, though.\n\n> =====Work and technical issues=====\n> ==Scenario==\n> For how we use the line level browser and how the utility should act\n> to us, here is an scenario:\n> http://article.gmane.org/gmane.comp.version-control.git/143024/match=line+level+history+browser\n> It contains code movement between files but not code copy and fuzzy matching.\n\nI would prefer if you could inline a short example, perhaps starting\nat your second diff snippet. Examples are good ;-)\n\nEven if not, please drop the /match= parameter since it is very\ndistracting.\n\n> 5. Simply fuzzy matching for code move/copy. Provide an option to\n> control whether we start a fuzzy matching for performance reason. This\n> can help us to find whether some code is really literally moved to\n> here or with some evil changes. And this may also help in some\n> situation like if we move some Java class to another file with only\n> its class name changed. Anyway, fuzzy matching can help much on code\n> detection. And there can be many fuzzy detect strategies, but we will\n> only try to support the simplest one in this summer for time reason.\n> Maybe a strategy like: 90% of the lines between two ranges of code are\n> identical or 90% of words are identical. This will be discussed again\n> before coding I think.\n> \n> 6. Provide a configurable way for how to display the history. A 'git\n> log --graph' way or stop to ask users when we meet history splitting.\n\nSee above.\n\n> 7. Reuse 'git log' existing options as many as possible.\n\nOne thing that IMO is missing from this list, is a plumbing mode that\njust feeds the raw data to a (presumed) frontend. It could be as\nsimple as supporting\n\n git log -L ... --pretty=raw --raw\n\nor similar, if this provides sufficient information. Compare 'git\nblame --porcelain'.\n\n> ==Design and implementation==\n> Git store all the blobs instead of code delta, so we should traverse\n> the commit history and directly access the tree/blob objects to\n> compute the code delta and search for the diff which contains the\n> interesting lines. Since git use libxdiff to format its diff file, we\n> should iterate through all xdiff's diff blocks and find what the code\n> looks like before the commit. This will be done using the callback\n> mechanism. Here, we will find a new line range which is the origin\n> code before this commit. And then start another search from the\n> current commit and the new line range. Recursively, we can find all\n> the modification history. We will stop when we find that the current\n> interested line range is added from scratch and is not moved from\n> other place of the file. Here, if the user want to trace code copy,\n> more work will be done to find the possible code copy. We may also\n> stop the traverse when we reach the max search depth. Also, if the\n> thread of change history split into two or more commits, we stop and\n> provide the users all the related commits and corresponding line\n> range.\n> \n> Generally,\n> 1. New callback for xdi_diff to parse the diff hunk and store line\n> level history info.\n> 2. builtin/line-log.c will be added to complete most of the new features.\n> 3. builtin/log.c will be changed to add this new utility to the front end.\n> 4. Documents will be updated to introduce this new tool.\n\nThis section is too handwavy for my taste. I think in most cases you\nsay \"we can\" when you really mean \"git-blame already does it, so we\ncan just use a similar algorithm\". Which is fine, but I'd rather see\nit spelled out so as to see what is not already covered by blame's code.\n\n> =====Milestones and Timeline=====\n> In this summer, we will add support of line level history browser for\n> only one file. The multiple ranges support is currently not in this\n> project.\n\nI agree with what Dscho pointed out earlier in the thread: multiple\nranges will be an easy exercise once you can follow a \"blame split\"\nwhere half the lines blame to some file and half the lines blame to\nanother.\n\nOther than that I think the milestones look sensible. As a theory\nguy, I'm not a huge believer in timelines, so lets hope someone else\ncomments on it.\n\n> And there is one milestone for each week nearly, so every week, I will\n> post a stutas update to the list to let the community know the project\n> progress. And, patches will be sent for feature completion but not\n> milestone.\n\nPush the code somewhere public as you go, even between feature\ncompletions. Post RFCs once you have workable features so people can\ncomment. Generally try to be visible.\n\nBonus points if you can think of something visible to do during the\nperiod where you look at code,\n\n> April 26 - May 23:\n> 1st week, follow the bird's eye view on Git's source code.\n> 2nd week, have a look at the code of merge-base, analyze the rev-listmachinery\n> 3rd week, have a look at builtin/log.c,\n> 4th week, understand blame.c\n\nwhether it be documenting your learnings in some way, improving docs\nas you go, or documenting the APIs you find.\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"138151","messageId":"41f08ee11003291952r467601b1o970ce3be802d8521@mail.gmail.com","threadId":"23105","inReplyTo":"201003292042.01549.trast@student.ethz.ch","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-30T02:52:25Z","receivedAt":"2010-03-30T02:52:25Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi Thomas,\n\nOn Tue, Mar 30, 2010 at 2:42 AM, Thomas Rast wrote:\n>\n> Is this really the right use-case?  AFAICT the answer to the implied\n> question is given by simply running 'git blame -M 93fc05e:pretty.c'.\n>\n> (Coming up with a better example should be easy; the way I currently\n> think of the feature means that it will mostly replace git-blame for\n> me...)\n\nI will cite the same example below in the scenario. :)\n\n> I would, by far, prefer the latter.  So far 'git log' has always been\n> noninteractive, and there's no really good way to make it interactive\n> because it also goes through the pager.  (In the case of blame this is\n> solved in 'git gui blame', which might also be a reasonable approach.)\n>\n> OTOH, if you can really fake a history walk, then just about any\n> log-oriented tool should be able to work with it.  You'd get graphical\n> output for free with gitk and git log --graph.  I haven't really\n> thought through the ramifications, though.\n\nOk, so let us try to abandon the interactive way totally.\n\n>> =====Work and technical issues=====\n>> ==Scenario==\n>> For how we use the line level browser and how the utility should act\n>> to us, here is an scenario:\n>> http://article.gmane.org/gmane.comp.version-control.git/143024/match=line+level+history+browser\n>> It contains code movement between files but not code copy and fuzzy matching.\n>\n> I would prefer if you could inline a short example, perhaps starting\n> at your second diff snippet.  Examples are good ;-)\n>\n> Even if not, please drop the /match= parameter since it is very\n> distracting.\n\nI put the example at the end of the proposal as a reference.\n\n>\n>> 7. Reuse 'git log' existing options as many as possible.\n>\n> One thing that IMO is missing from this list, is a plumbing mode that\n> just feeds the raw data to a (presumed) frontend.  It could be as\n> simple as supporting\n>\n>  git log -L ... --pretty=raw --raw\n>\n> or similar, if this provides sufficient information.  Compare 'git\n> blame --porcelain'.\n\nVery good feedback, I will add this, thanks a lot!\n\n>\n> This section is too handwavy for my taste.  I think in most cases you\n> say \"we can\" when you really mean \"git-blame already does it, so we\n> can just use a similar algorithm\".  Which is fine, but I'd rather see\n> it spelled out so as to see what is not already covered by blame's code.\n\nChanged in next version to make this clear. But only add some words to\nstate that 'blame does similar' :)\n\n>\n> Push the code somewhere public as you go, even between feature\n> completions.  Post RFCs once you have workable features so people can\n> comment.  Generally try to be visible.\n>\n> Bonus points if you can think of something visible to do during the\n> period where you look at code,\n\nYeah, really is a good point. And I have tried to play around on\ngithub.com and try to set up a http://github.com/byang/my_git for this\npurpose. :)\n\n>> April 26 - May 23:\n>> 1st week, follow the bird's eye view on Git's source code.\n>> 2nd week, have a look at the code of merge-base, analyze the rev-listmachinery\n>> 3rd week, have a look at builtin/log.c,\n>> 4th week, understand blame.c\n>\n> whether it be documenting your learnings in some way, improving docs\n> as you go, or documenting the APIs you find.\n\nThanks a lot for this good advice, I will do so.\n\nWith these feedback, I think I can make up a complete version of the\nproposal and submit it to Google. Thanks!\n\nRegards!\nBo\n"},{"id":"138171","messageId":"4BB1BF4B.2060604@drmicha.warpmail.net","threadId":"23105","inReplyTo":"41f08ee11003291952r467601b1o970ce3be802d8521@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2010-03-30T09:07:23Z","receivedAt":"2010-03-30T09:07:23Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Bo Yang venit, vidit, dixit 30.03.2010 04:52:\n> Hi Thomas,\n> \n> On Tue, Mar 30, 2010 at 2:42 AM, Thomas Rast wrote:\n>>\n>> Is this really the right use-case? AFAICT the answer to the implied\n>> question is given by simply running 'git blame -M 93fc05e:pretty.c'.\n>>\n>> (Coming up with a better example should be easy; the way I currently\n>> think of the feature means that it will mostly replace git-blame for\n>> me...)\n> \n> I will cite the same example below in the scenario. :)\n> \n>> I would, by far, prefer the latter. So far 'git log' has always been\n>> noninteractive, and there's no really good way to make it interactive\n>> because it also goes through the pager. (In the case of blame this is\n>> solved in 'git gui blame', which might also be a reasonable approach.)\n>>\n>> OTOH, if you can really fake a history walk, then just about any\n>> log-oriented tool should be able to work with it. You'd get graphical\n>> output for free with gitk and git log --graph. I haven't really\n>> thought through the ramifications, though.\n> \n> Ok, so let us try to abandon the interactive way totally.\n> \n>>> =====Work and technical issues=====\n>>> ==Scenario==\n>>> For how we use the line level browser and how the utility should act\n>>> to us, here is an scenario:\n>>> http://article.gmane.org/gmane.comp.version-control.git/143024/match=line+level+history+browser\n>>> It contains code movement between files but not code copy and fuzzy matching.\n>>\n>> I would prefer if you could inline a short example, perhaps starting\n>> at your second diff snippet. Examples are good ;-)\n>>\n>> Even if not, please drop the /match= parameter since it is very\n>> distracting.\n> \n> I put the example at the end of the proposal as a reference.\n> \n>>\n>>> 7. Reuse 'git log' existing options as many as possible.\n>>\n>> One thing that IMO is missing from this list, is a plumbing mode that\n>> just feeds the raw data to a (presumed) frontend. It could be as\n>> simple as supporting\n>>\n>> git log -L ... --pretty=raw --raw\n>>\n>> or similar, if this provides sufficient information. Compare 'git\n>> blame --porcelain'.\n> \n> Very good feedback, I will add this, thanks a lot!\n> \n>>\n>> This section is too handwavy for my taste. I think in most cases you\n>> say \"we can\" when you really mean \"git-blame already does it, so we\n>> can just use a similar algorithm\". Which is fine, but I'd rather see\n>> it spelled out so as to see what is not already covered by blame's code.\n> \n> Changed in next version to make this clear. But only add some words to\n> state that 'blame does similar' :)\n> \n>>\n>> Push the code somewhere public as you go, even between feature\n>> completions. Post RFCs once you have workable features so people can\n>> comment. Generally try to be visible.\n>>\n>> Bonus points if you can think of something visible to do during the\n>> period where you look at code,\n> \n> Yeah, really is a good point. And I have tried to play around on\n> github.com and try to set up a http://github.com/byang/my_git for this\n> purpose. :)\n\nYou may want to create your repo as a fork of gitster/git instead.\nThat's easier on github, they have a hard time anyways these days ;)\nSeriously, it helps making use of their network feature etc.\n\nI don't have anything to add to your proposal (I like it), but I'll be\nat NKU next week (Conference @ Chern Institute) so drop me a PM if you wish.\n\nCheers,\nMichael\n"},{"id":"138172","messageId":"m3vdcei3bb.fsf@localhost.localdomain","threadId":"23105","inReplyTo":"41f08ee11003291952r467601b1o970ce3be802d8521@mail.gmail.com","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-03-30T09:10:34Z","receivedAt":"2010-03-30T09:10:34Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Bo Yang writes:\n> On Tue, Mar 30, 2010 at 2:42 AM, Thomas Rast wrote:\n> >\n> > Is this really the right use-case?  AFAICT the answer to the implied\n> > question is given by simply running 'git blame -M 93fc05e:pretty.c'.\n> >\n> > (Coming up with a better example should be easy; the way I currently\n> > think of the feature means that it will mostly replace git-blame for\n> > me...)\n> \n> I will cite the same example below in the scenario. :)\n\nBy the way, it would be good to find an example with \"evil merge\",\nwhich means that the change to given line(s) is in the merge commit\nitself. Correctly simplifying history in such case might be\nnon-trivial.\n\nAnother example that it would be good to have is \"history split\"\nexample, which means the case where some lines were consolidated\n(e.g. after refactoring), and some of lines in \"preimage\" come\nfrom different lines of history.\n\nThis would help with writing tests for this feature (compare tests\nfor blame), although they are not in my opinion necessary for the\nproposal itself.\n \nI hope that all this cases would fall naturally from the\nimplementation.\n\n[...]\n> > Push the code somewhere public as you go, even between feature\n> > completions.  Post RFCs once you have workable features so people can\n> > comment.  Generally try to be visible.\n> >\n> > Bonus points if you can think of something visible to do during the\n> > period where you look at code,\n> \n> Yeah, really is a good point. And I have tried to play around on\n> github.com and try to set up a http://github.com/byang/my_git for this\n> purpose. :)\n\nmy_git is not very descriptive... well, unless you would do your work\non GSoC2010/line-level-history-browser branch, or something like that.\n\nIt might be good idea to have repo.or.cz as an additional repository,\nas a fork of git.git repo, and with SoC / GSoC labels. See\nhttp://repo.or.cz/w/git.git/forks?t=soc\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"138174","messageId":"4BB1C69D.7040009@drmicha.warpmail.net","threadId":"23105","inReplyTo":"4BB1BF4B.2060604@drmicha.warpmail.net","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2010-03-30T09:38:37Z","receivedAt":"2010-03-30T09:38:37Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Michael J Gruber venit, vidit, dixit 30.03.2010 11:07:\n\n> You may want to create your repo as a fork of gitster/git instead.\n\nActually, make this git/git, the other one isn't being updated... Sorry!\n\nMichael\n"},{"id":"138176","messageId":"41f08ee11003300410j512010a2wb24aa62857bdcdc3@mail.gmail.com","threadId":"23105","inReplyTo":"4BB1BF4B.2060604@drmicha.warpmail.net","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-30T11:10:32Z","receivedAt":"2010-03-30T11:10:32Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi Michael,\n\nOn Tue, Mar 30, 2010 at 5:07 PM, Michael J Gruber\n wrote:\n>\n> You may want to create your repo as a fork of gitster/git instead.\n> That's easier on github, they have a hard time anyways these days ;)\n> Seriously, it helps making use of their network feature etc.\n\nYeah, forked git/git. :)\n\n> I don't have anything to add to your proposal (I like it), but I'll be\n> at NKU next week (Conference @ Chern Institute) so drop me a PM if you wish.\n\nThat is really a big coincidence. :)\nI am very willing to meet you at NKU, and I think I can be your guide\nin NKU and some beautiful spots in Tianjin if you have spare time. :)\nAnyway, let us talk about this in personal email off the list. :-)\n\nRegards!\nBo\n"},{"id":"138177","messageId":"41f08ee11003300415y28608806je3e98b33367a6f3d@mail.gmail.com","threadId":"23105","inReplyTo":"m3vdcei3bb.fsf@localhost.localdomain","subject":"Re: GSoC draft proposal: Line-level history browser","fromName":"Bo Yang","fromEmail":"struggleyb.nku@gmail.com","sentAt":"2010-03-30T11:15:12Z","receivedAt":"2010-03-30T11:15:12Z","isPatch":false,"sender":{"key":"struggleyb.nku@gmail.com","avatar":"https://avatars.githubusercontent.com/u/233030?v=4"},"body":"Hi Jakub,\n\nOn Tue, Mar 30, 2010 at 5:10 PM, Jakub Narebski wrote:\n> By the way, it would be good to find an example with \"evil merge\",\n> which means that the change to given line(s) is in the merge commit\n> itself.  Correctly simplifying history in such case might be\n> non-trivial.\n\nIt is a little time consuming to find such a change in the history. I\nthink we can come up some ones at the start of the project manually\nand put them into the testcases. :)\n\n> Another example that it would be good to have is \"history split\"\n> example, which means the case where some lines were consolidated\n> (e.g. after refactoring), and some of lines in \"preimage\" come\n> from different lines of history.\n>\n> This would help with writing tests for this feature (compare tests\n> for blame), although they are not in my opinion necessary for the\n> proposal itself.\n>\n> I hope that all this cases would fall naturally from the\n> implementation.\n> [...]\n>> > Push the code somewhere public as you go, even between feature\n>> > completions.  Post RFCs once you have workable features so people can\n>> > comment.  Generally try to be visible.\n>> >\n>> > Bonus points if you can think of something visible to do during the\n>> > period where you look at code,\n>>\n>> Yeah, really is a good point. And I have tried to play around on\n>> github.com and try to set up a http://github.com/byang/my_git for this\n>> purpose. :)\n>\n> my_git is not very descriptive... well, unless you would do your work\n> on GSoC2010/line-level-history-browser branch, or something like that.\n>\n> It might be good idea to have repo.or.cz as an additional repository,\n> as a fork of git.git repo, and with SoC / GSoC labels.  See\n> http://repo.or.cz/w/git.git/forks?t=soc\n\nAh, a repo at http://github.com/byang/gsoc-line-browser is created\nand a mirror at http://repo.or.cz/w/gsoc-line-browser.git, I think\nthis is enough. :-)\n\nThanks!\nBo\n"}]}