{"thread":{"id":"2859","subject":"[RFC] faking cvs annotate","startedAt":"2005-12-16T01:13:16Z","lastAt":"2005-12-16T22:36:19Z","messageCount":10,"participants":["Martin Langhoff","Johannes Schindelin","Junio C Hamano","Eric Wong","Ryan Anderson","Linus Torvalds","Nicolas Pitre"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"13713","messageId":"46a038f90512151713i118e58acia466d0f65ff91383@mail.gmail.com","threadId":"2859","inReplyTo":null,"subject":"[RFC] faking cvs annotate","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2005-12-16T01:13:16Z","receivedAt":"2005-12-16T01:13:16Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"I know, I know, use pickaxe :-)\n\nAny suggestions as to how to implement a not-too-slow annotate that\nlooks reasonable enough so as to fool real cvs clients? Let's assume\nI'll have the \"per-file-version numbers\" to translate commit SHA1s\ninto cvs-ish version numbers in a magic hat, right next to the fluffy\nbunnies.\n\nSuggestions of GIT machinery that would shortcut the trip from\n\n git-rev-list HEAD $path\n\nto a annotate-ish output. Did I dream it or is qgit showing something\nannotate-ish in its screenshots?\n\nNote! Just told a white lie: I won't be actually using git-rev-list.\nInstead, I'll have a rev list that will gloss over branched\ndevelopment, picking only one \"path\" of the history every time\n(initially using the algorithm formerly known as rand() ) , and\nshowing merge commits as one.\n\ncheers,\n\n\nmartin\n"},{"id":"13714","messageId":"Pine.LNX.4.63.0512160229140.6323@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"2859","inReplyTo":"46a038f90512151713i118e58acia466d0f65ff91383@mail.gmail.com","subject":"Re: [RFC] faking cvs annotate","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-12-16T01:31:11Z","receivedAt":"2005-12-16T01:31:11Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 16 Dec 2005, Martin Langhoff wrote:\n\n> I know, I know, use pickaxe :-)\n> \n> Any suggestions as to how to implement a not-too-slow annotate that\n> looks reasonable enough so as to fool real cvs clients?\n\nFor starters, you could use my attempt:\n\nhttp://www.gelato.unsw.edu.au/archives/git/0508/7171.html\n\nIt does not fool any CVS user, because it does not fake version numbers. \nInstead, you can display SHA1s, short SHA1s, author names, or mixtures \nthereof.\n\nHowever, I learnt to use git-whatchanged in the meantime, and I'll \nprobably never go back.\n\nHth,\nDscho\n"},{"id":"13716","messageId":"7vpsnxlsw7.fsf@assigned-by-dhcp.cox.net","threadId":"2859","inReplyTo":"46a038f90512151713i118e58acia466d0f65ff91383@mail.gmail.com","subject":"Re: [RFC] faking cvs annotate","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-12-16T01:42:16Z","receivedAt":"2005-12-16T01:42:16Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Martin Langhoff writes:\n\n> Suggestions of GIT machinery that would shortcut the trip from\n>\n> git-rev-list HEAD $path\n>\n> to a annotate-ish output. Did I dream it or is qgit showing something\n> annotate-ish in its screenshots?\n\nI haven't actively done anything but one of the good things that\ncould happen is to split out the access routines for annotate\ndatabase qgit build when run the first time in the repository,\nand make them available to other Porcelains. There is no need\nto reinvent the wheel.\n"},{"id":"13718","messageId":"46a038f90512152037u66993aadg566a2a47abd279c9@mail.gmail.com","threadId":"2859","inReplyTo":"Pine.LNX.4.63.0512160229140.6323@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC] faking cvs annotate","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2005-12-16T04:37:32Z","receivedAt":"2005-12-16T04:37:32Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On 12/16/05, Johannes Schindelin wrote:\n> For starters, you could use my attempt:\n> http://www.gelato.unsw.edu.au/archives/git/0508/7171.html\n\nNot bad at all! I might simplify it a bit (won't need the commitmsg\nparsing), and see if how fast I can make it.\n\n> It does not fool any CVS user,\n\nAh, no. Those have been fooled already ;-)\n\n> However, I learnt to use git-whatchanged in the meantime, and I'll\n> probably never go back.\n\nOf course. I'm trying to meet a few the expectations of popular cvs\nclients (cli, TortoiseCVS, Eclipse, etc). If a workable git-annotate\nfalls out of this, oh well.\n\nOTOH, yours didn't get included. Any particular reasons? (other than\nstressing the point that you should use whatchanged/pickaxe?)\n\nAlso! On 12/16/05, Junio C Hamano wrote:\n> I haven't actively done anything but one of the good things that\n> could happen is to split out the access routines for annotate\n> database qgit build when run the first time in the repository,\n> and make them available to other Porcelains. There is no need\n> to reinvent the wheel.\n\nThat would be cool! Though my particular implementation may be a tad\nunconventional in its view of the \"file history\" as a strictly linear\nthing. Normal git porcelains don't even blink when dealing with\nbranching/merging development histories.\n\ncheers,\n\n\nmartin\n"},{"id":"13719","messageId":"20051216060953.GA22150@mail.yhbt.net","threadId":"2859","inReplyTo":"7vpsnxlsw7.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] faking cvs annotate","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2005-12-16T06:09:54Z","receivedAt":"2005-12-16T06:09:54Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano wrote:\n> Martin Langhoff writes:\n> \n> > Suggestions of GIT machinery that would shortcut the trip from\n> >\n> > git-rev-list HEAD $path\n> >\n> > to a annotate-ish output. Did I dream it or is qgit showing something\n> > annotate-ish in its screenshots?\n> \n> I haven't actively done anything but one of the good things that\n> could happen is to split out the access routines for annotate\n> database qgit build when run the first time in the repository,\n> and make them available to other Porcelains. There is no need\n> to reinvent the wheel.\n\nI started working on something using Algorithm::Annotate on CPAN during\nan ADD excursion a few weeks ago, but mostly forgot about it (thanks\nagain to ADD :). A few versions of git later and it still seems to\nwork as well as I remembered it.\n\nIt doesn't handle renames/copies yet, but it does let you customize the\nleft-hand side output unlike most/(all?) annotate implementations.\n\nI guess it's also quite useful when the output is piped to vim with a\nrepository browser like the one I brought up in\n<20051124093322.GA3899@mail.yhbt.net>\n\n---------- 8< ---------\n\n#!/usr/bin/env perl\n# Copyright (c) 2005, Eric Wong \n#\n# This file is licensed under the GPL v2, or a later version\n# at the discretion of Linus.\n#\n# Read git-whatchanged output and let all Algorithm::Annotate do all\n# the hard work for us. Thanks to Chia-liang Kao for the awesome \n# Algorithm::Annotate module.\n#\n# Formatting options of annotation output is available.\n# Dates can be formatted through strftime strings using the -D switch,\n# and annotation expansion can be done using the -F switch\n# on following variables:\n\n=cut\n\t%commit% %tree% %author_name% %author_email% %author_date%\n\t%committer_name% %committer_email% %commit_date% %message%\n=cut\nuse warnings;\nuse strict;\n\nsub usage ($) {\n\tprint '* git-annotate ',\n\t\t'[-w ] [-D ] [-F ] ',\n\t\t'[--localtime] [--line-prefix ] [--line-postfix ] ',\n\t\t'', \"\\n\";\n\texit $_[0];\n}\n\n# everybody with Perl 5 should have these, right?\nuse POSIX qw(strftime);\nuse Getopt::Long qw(:config gnu_getopt no_ignore_case no_auto_abbrev);\n\nmy %_o; # options\nusage(1) unless (GetOptions(\\%_o, qw(help|H|h width|w=i\n\t\tline_prefix|line-prefix:s line_postfix|line-postfix:s\n\t\tformat|F=s date_format|date-format|D=s localtime|l)));\nusage(0) if $_o{help};\n\n# ok, see if we can do more than show a help message:\nrequire Algorithm::Annotate or die\n\t\t\"Can't find Algorithm::Annotate, get it from CPAN ($!)\\n\";\n\nmy $file = shift or usage(1);\nunless (-f $file) {\n\tprint STDERR \"File does not exist or is not a file: $file\\n\";\n\texit 1;\n}\n\nmy $date_format = $_o{date_format} || '%Y-%m-%dT%H-%M-%SZ'; # ISO 8601\n\nmy $re_sha1 = qr/[a-f0-9]{40}/; # match 40 char hex strings\nmy @blobs; # we read from newest to oldest, but $ann needs it reversed\n\nsub t ($) {\n\treturn localtime($_[0]) if $_o{'localtime'};\n\treturn gmtime($_[0]);\n}\n\nif (my $pid = open my $child, '-|') {\n\tmy $meta;\n\twhile (<$child>) {\n\t\tchomp;\n\t\tif (/^diff-tree ($re_sha1) /o) {\n\t\t\t$meta = { commit => $1, message => [] };\n\t\t} elsif (/^tree ($re_sha1)$/) {\n\t\t\t$meta->{tree} = $1;\n\t\t} elsif (/^author (.*) <(\\S+)> (\\d+) [\\+\\-]?\\d+$/) {\n\t\t\t$meta->{author_name} = $1;\n\t\t\t$meta->{author_email} = $2;\n\t\t\t$meta->{author} = \"$1 <$2>\";\n\t\t\t$meta->{author_date} = strftime($date_format, t($3));\n\t\t} elsif (/^committer (.*) <(\\S+)> (\\d+) [\\+\\-]?\\d+$/) {\n\t\t\t$meta->{committer_name} = $1;\n\t\t\t$meta->{committer_email} = $2;\n\t\t\t$meta->{committer} = \"$1 <$2>\";\n\t\t\t$meta->{commit_date} = strftime($date_format, t($3));\n\t\t} elsif (/^\\s{4}(.*)$/) {\n\t\t\tpush @{$meta->{message}}, $1;\n\t\t} elsif (/^:\\d{6} \\d{6} $re_sha1 ($re_sha1)/o) {\n\t\t\tmy $blob = $1;\n\t\t\tif ($blob =~ /^0{40}$/) {\n\t\t\t\t# nonexistent (hopefully!) blob\n\t\t\t\t$meta = undef;\n\t\t\t\tnext;\n\t\t\t}\n\t\t\t\n\t\t\t$meta->{message} = join(\"\\n\",@{$meta->{message}});\n\t\t\tpush @blobs, { meta => $meta, blob => $blob };\n\t\t\t$meta = undef;\n\t\t}\n\t}\n} else {\n\texec('git-whatchanged','-m','--pretty=raw','--',$file) or die\n\t\t\t\"Unable to execute git-whatchanged $file: $? $!\";\n}\n\nmy $msg_format = $_o{format} || '%commit% %commit_date%';\nmy $max_len = 0;\nmy $ann = Algorithm::Annotate->new;\n\nforeach my $x (reverse @blobs) {\n\tmy @contents;\n\t\n\t# no need to do safe pipe opens since we know $x->{blob} is a hex sum\n\t@contents = `git-cat-file blob $x->{blob}`;\n\n\t# $msg is the left hand side of our final annotation output,\n\t# format it according to $msg_format\n\tmy $msg = $msg_format;\n\tmy $meta = $x->{meta};\n\n\t# our annotation message templating engine:\n\t$msg =~ s/\\%$_\\%/$meta->{$_}/g foreach keys %$meta;\n\t$msg =~ s/\\\\t/\\t/g;\n\t$msg =~ s/\\\\n/\\n/g;\n\n\t$ann->add($msg, \\@contents);\n\tif ($msg !~ /\\n/) {\n\t\t$max_len = length $msg if length $msg > $max_len;\n\t} else {\n\t\tforeach (split /\\n/, $msg) {\n\t\t\t$max_len = length $_ if length $_ > $max_len;\n\t\t}\n\t}\n}\n\n# see if we can annotate local changes, too\nmy $local_change;\nmy @diff_opts = qw(-z -C --find-copies-harder --name-only);\n{ # see if we've changed $file in the working tree\n\tif (my $pid = open my $child, '-|') {\n\t\t$local_change = (<$child>);\n\t} else {\n\t\texec('git-diff-files',@diff_opts,'--',$file);\n\t}\n}\n\nunless ($local_change) {\n\t# see if we've changed $file in the working tree\n\tif (my $pid = open my $child, '-|') {\n\t\t$local_change = (<$child>);\n\t} else {\n\t\texec('git-diff-index',@diff_opts,'--cached','HEAD','--',$file);\n\t}\n}\n\nopen my $fd,'<',$file or die \"Error opening file: $file: $!\\n\";\nmy @contents = <$fd>;\n\nif ($local_change) {\n\tmy $msg = '(uncommitted): ';\n\t$ann->add($msg, \\@contents);\n\t$max_len = length $msg if length $msg > $max_len;\n}\n\n# $result here is an array ref, each element corresponding to a line of\n# the annotated file contents\nmy $result = $ann->result or do {\n\tprint STDERR \"Unable to annotate $file. Is it tracked by git?\\n\";\n\texit 1;\n};\n\nmy $pre = defined $_o{line_prefix} ? $_o{line_prefix} : '';\nmy $post = defined $_o{line_postfix} ? $_o{line_postfix} : ': ';\nmy $width = $_o{width} || $max_len;\n\nforeach (@contents) {\n\tmy $msg = shift @$result;\n\tif ($msg !~ /\\n/) {\n\t\tprint ($pre, pack(\"A$width\", $msg), $post, $_);\n\t} else {\n\t\tmy @msgs = split /\\n/, $msg;\n\t\tprint ($pre, pack(\"A$width\", shift @msgs), $post, $_);\n\t\tforeach my $m (@msgs) {\n\t\t\tprint ($pre, pack(\"A$width\", $m), $post,\"\\n\");\n\t\t}\n\t}\n}\n\nclose $fd;\n\nexit 0;\n\n-- \nEric Wong\n"},{"id":"13726","messageId":"43A2D7D7.1060506@michonline.com","threadId":"2859","inReplyTo":"20051216060953.GA22150@mail.yhbt.net","subject":"Re: [RFC] faking cvs annotate","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2005-12-16T15:05:59Z","receivedAt":"2005-12-16T15:05:59Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"Eric Wong wrote:\n\n> #!/usr/bin/env perl\n> # Copyright (c) 2005, Eric Wong \n> #\n> # This file is licensed under the GPL v2, or a later version\n> # at the discretion of Linus.\n\nPlease use full names for this kind of authority delegation.\n\n> # Read git-whatchanged output and let all Algorithm::Annotate do all\n> # the hard work for us. Thanks to Chia-liang Kao for the awesome \n> # Algorithm::Annotate module.\n> #\n\nYou'll want to add libalgorithm-annotate-perl to the Debian/control\nDepends: line when you submit this as a patch. (I dont' have an\nRPM-based machine handy, so I can't doublecheck that.)\n\nI was going to comment on the rest, but I don't really have time to play\nwith it at the moment.\n\n\n-- \n\nRyan Anderson\n sometimes Pug Majere\n"},{"id":"13728","messageId":"Pine.LNX.4.64.0512160843450.3060@g5.osdl.org","threadId":"2859","inReplyTo":"46a038f90512152037u66993aadg566a2a47abd279c9@mail.gmail.com","subject":"Re: [RFC] faking cvs annotate","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-12-16T17:07:42Z","receivedAt":"2005-12-16T17:07:42Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 16 Dec 2005, Martin Langhoff wrote:\n>\n> On 12/16/05, Johannes Schindelin wrote:\n> > For starters, you could use my attempt:\n> > http://www.gelato.unsw.edu.au/archives/git/0508/7171.html\n> \n> Not bad at all! I might simplify it a bit (won't need the commitmsg\n> parsing), and see if how fast I can make it.\n\nNOTE! You can make it a whole lot faster these days by passing the path \ndown to git-rev-list. But as usual, I'm perl-incompatible, so I can't help \nyou with that modification.\n\n> OTOH, yours didn't get included. Any particular reasons? (other than\n> stressing the point that you should use whatchanged/pickaxe?)\n\nI'd love to see a git-annotate, but I want it to be efficient enough that \nit's not embarrassing.\n\nFor example, \"git-whatchanged\" is certainly efficient enough, and you \ncould just parse that output (possible without using -p and comparing file \ncontents on your own against the current HEAD as you progress backwards)\n\nOr, you could now use \"git-rev-list \", which is even more \nefficient, but the downside of that is that it computes the _full_ graph \nbefore it starts outputting the early ones. \n\nBut, for example:\n\n\t[torvalds@g5 linux-history]$ time git-whatchanged drivers/char/Makefile > /dev/null \n\treal 0m37.993s\n\tuser 0m41.912s\n\tsys 0m0.400s\n\n\t[torvalds@g5 linux-history]$ time git-rev-list HEAD drivers/char/Makefile > /dev/null \n\treal 0m6.713s\n\tuser 0m6.656s\n\tsys 0m0.056s\n\nthat's the old Linux history thing, with 60,000+ commit. The path-based \ngit-rev-list thing cuts it down to just 103 commits you need to look at \n(so the annotate itself should be really fast, and the cost really is all \nin that \"git-rev-list\" thing).\n\nOf course, with the current kernel tree, the git-rev-list takes a lot less \ntime, and only results in six commits, but that's a tree that only has a \nfew months of history, so when looking at performance, you really should \nlook at some bigger tree.\n\nNOTE! You can also decide to cut down on time by saying \"annotate back \nonly to a specific date\", ie something like\n\n\tgit-rev-list $(git-rev-parse --since=2.years.ago HEAD) drivers/char/Makefile\n\nwhich limits the thing by filename _and_ date (a good way to handle really \nreally big projects) where the rev-parse might otherwise be very \nexpensive.\n\nAnd, of course, a quality implementation will notice when a file \ndisappears, and use \"git-diff-tree -M\" on that commit to see if it was \nrenamed, and then continuing with the old name...\n\n\t\t\tLinus\n"},{"id":"13746","messageId":"Pine.LNX.4.64.0512161636150.26663@localhost.localdomain","threadId":"2859","inReplyTo":"Pine.LNX.4.64.0512160843450.3060@g5.osdl.org","subject":"Re: [RFC] faking cvs annotate","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2005-12-16T21:40:01Z","receivedAt":"2005-12-16T21:40:01Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 16 Dec 2005, Linus Torvalds wrote:\n\n> But, for example:\n> \n> \t[torvalds@g5 linux-history]$ time git-whatchanged drivers/char/Makefile > /dev/null \n> \treal 0m37.993s\n> \tuser 0m41.912s\n> \tsys 0m0.400s\n\nHmmm... I'd like to have a system like that too, where reality is \nsmaller than the sum of system and user time. ;-)\n\n\nNicolas\n"},{"id":"13749","messageId":"Pine.LNX.4.64.0512161412310.3698@g5.osdl.org","threadId":"2859","inReplyTo":"Pine.LNX.4.64.0512161636150.26663@localhost.localdomain","subject":"Re: [RFC] faking cvs annotate","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-12-16T22:12:46Z","receivedAt":"2005-12-16T22:12:46Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 16 Dec 2005, Nicolas Pitre wrote:\n\n> On Fri, 16 Dec 2005, Linus Torvalds wrote:\n> \n> > But, for example:\n> > \n> > \t[torvalds@g5 linux-history]$ time git-whatchanged drivers/char/Makefile > /dev/null \n> > \treal 0m37.993s\n> > \tuser 0m41.912s\n> > \tsys 0m0.400s\n> \n> Hmmm... I'd like to have a system like that too, where reality is \n> smaller than the sum of system and user time. ;-)\n\nIt's called \"smp\".\n\nYou can buy then anywhere.\n\n\t\tLinus\n"},{"id":"13751","messageId":"Pine.LNX.4.64.0512161723370.26663@localhost.localdomain","threadId":"2859","inReplyTo":"Pine.LNX.4.64.0512161412310.3698@g5.osdl.org","subject":"Re: [RFC] faking cvs annotate","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2005-12-16T22:36:19Z","receivedAt":"2005-12-16T22:36:19Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 16 Dec 2005, Linus Torvalds wrote:\n\n> \n> \n> On Fri, 16 Dec 2005, Nicolas Pitre wrote:\n> \n> > On Fri, 16 Dec 2005, Linus Torvalds wrote:\n> > \n> > > But, for example:\n> > > \n> > > \t[torvalds@g5 linux-history]$ time git-whatchanged drivers/char/Makefile > /dev/null \n> > > \treal 0m37.993s\n> > > \tuser 0m41.912s\n> > > \tsys 0m0.400s\n> > \n> > Hmmm... I'd like to have a system like that too, where reality is \n> > smaller than the sum of system and user time. ;-)\n> \n> It's called \"smp\".\n\nGah! Maybe I should have tried the same before posting, since I get \nthis on my P4 with HT:\n\n\treal 0m36.930s\n\tuser 0m41.971s\n\tsys 0m0.656s\n\n/me hides shamefully\n\n\nNicolas\n"}]}