{"thread":{"id":"43195","subject":"Re: [PATCH] (experimental) per-topic shortlog.","startedAt":"2006-11-27T00:44:18Z","lastAt":"2006-12-01T11:23:02Z","messageCount":18,"participants":["Jeff King","Junio C Hamano","Linus Torvalds","Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"297589","messageId":"7v8xhxsopp.fsf@assigned-by-dhcp.cox.net","threadId":"43195","inReplyTo":null,"subject":"[PATCH] (experimental) per-topic shortlog.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-27T00:44:18Z","receivedAt":"2006-11-27T00:44:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This implements an experimental \"git log-fpc\" command that shows\nshort-log style output sorted by topics.\n\nA \"topic\" is identified by going through the first-parent\nchains; this ignores the fast-forward case, but for a top-level\nintegrator it often is good enough.\n\nFor example, if the commit ancestry graph looks like this:\n\n         x---x---x---X---o---*---o---o---o HEAD\n          \\                 /\n           o---o---o---o---o\n\nand the command line asks for\n\n\tgit log-fpc --no-merges X..\n\nIt first finds all the commits 'o'.  Then it emits the four\ncommits on the upper line (assume the merge '*' has the commit\nthat is a child of X as its first parent in the picture).  When\nit does so, it the list of authors for these four commits on one\nline, followed by the title of these commits.  After that, it\ndoes the same for the five commits on the lower line.\n\n---\n\nI initially wanted to do this inside Johannes's enhanced\nshortlog, but ended up doing this as a pretty much independent\nthing, because the shortlog implementation stringifies the\ninformation from the commits too early to be easily enhanced for\nthis purpose.\n\nIf this turns out to be a better way to present shortlog,\nhowever, this should become an option to git-shortlog.\n\nA sample output from:\n\n\tgit log-fpc --no-merges v1.4.4.1..f64d7fd2\n\nlooks like this (f64d7fd2 was the tip of master when the last\n\"What's in\" message was sent out).  It shows that many \"fixes\"\nand git-svn enhancements were directly done on \"master\" (that is\nthe first group), while many gitweb enhancements, changing the\noutput from \"prune -n\", \"git branch\" enhancements, etc. were\nfirst cooked in separate topic branches and then later merged\ninto 'master'.\n\nTo this output, I can manually add a topic title to the\nbeginning of each group and it would make a better overview than\nwhat I currently send out in \"What's in\" message which is\ngenerated with shortlog.\n\n----------------------------------------------------------------\n\nEric Wong (6), Junio C Hamano (5), Lars Hjemli, Jakub Narebski,\n Iñaki Arenaza, Petr Baudis, Andy Parkins, and René Scharfe\n git-fetch: exit with non-zero status when fast-forward check fails\n git-svn: exit with status 1 for test failures\n git-svn: correctly access repos when only given partial read permissions\n git-branch -D: make it work even when on a yet-to-be-born branch\n Add -v and --abbrev options to git-branch\n git-clone: stop dumb protocol from copying refs outside heads/ and tags/.\n gitweb: (style) use chomp without parentheses consistently.\n gitweb: Replace SPC with &nbsp; also in tag comment\n git-svn: handle authentication without relying on cached tokens on disk\n git-cvsimport: add support for CVS pserver method HTTP/1.x proxying\n Make git-clone --use-separate-remote the default\n refs outside refs/{heads,tags} match less strongly.\n Increase length of function name buffer\n git-svn: preserve uncommitted changes after dcommit\n git-svn: correctly handle revision 0 in SVN repositories\n git-svn: error out from dcommit on a parent-less commit\n archive-zip: don't use sizeof(struct ...)\n\nJunio C Hamano and Andy Parkins\n Typefix builtin-prune.c::prune_object()\n Improve git-prune -n output\n\nPeter Baumann\n config option log.showroot to show the diff of root commits\n\nAndy Parkins\n Add support to git-branch to show local and remote branches\n\nJakub Narebski (7)\n gitweb: Finish restoring \"blob\" links in git_difftree_body\n gitweb: Refactor feed generation, make output prettier, add Atom feed\n gitweb: Add an option to href() to return full URL\n gitweb: New improved formatting of chunk header in diff\n gitweb: Default to $hash_base or HEAD for $hash in \"commit\" and \"commitdiff\"\n gitweb: Buffer diff header to deal with split patches + git_patchset_body refactoring\n gitweb: Protect against possible warning in git_commitdiff\n\n----------------------------------------------------------------\n\n builtin-log.c |  177 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n builtin.h     |    1 +\n git.c         |    1 +\n 3 files changed, 179 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin-log.c b/builtin-log.c\nindex 7acf5d3..1c2838c 100644\n--- a/builtin-log.c\n+++ b/builtin-log.c\n@@ -99,6 +99,183 @@ int cmd_log(int argc, const char **argv, const char *prefix)\n \treturn cmd_log_walk(&rev);\n }\n \n+/* bits #0..7 in revision.h, #8..11 in commit.c */\n+#define FPC_RESULT (1u<<12)\n+#define FPC_SHOWN  (1u<<13)\n+\n+struct author_record {\n+\tchar *name;\n+\tint count;\n+};\n+struct author_count {\n+\tint nr, alloc;\n+\tstruct author_record **au;\n+};\n+\n+static int cmp_count(const void *a_, const void *b_)\n+{\n+\tstruct author_record **a = (struct author_record **) a_;\n+\tstruct author_record **b = (struct author_record **) b_;\n+\treturn (*b)->count - (*a)->count;\n+}\n+\n+static void add_author(struct commit *c, struct author_count *ac)\n+{\n+\tconst char *buf = c->buffer;\n+\tchar *au = strstr(buf, \"\\nauthor \");\n+\tchar *eon;\n+\tstruct author_record *ar;\n+\tint i;\n+\n+\tif (!au)\n+\t\treturn; /* oops */\n+\tau += 7;\n+\twhile (*au && isspace(*au))\n+\t\tau++;\n+\tif (!*au)\n+\t\treturn; /* oops */\n+\teon = strchr(au, '<');\n+\tif (!eon)\n+\t\treturn; /* oops */\n+\twhile (au < --eon && isspace(*eon))\n+\t\t; /* back back back... */\n+\teon++;\n+\tfor (i = 0; i < ac->nr; i++)\n+\t\tif (!strncmp(ac->au[i]->name, au, eon-au) &&\n+\t\t    strlen(ac->au[i]->name) == eon - au) {\n+\t\t\t/* found it */\n+\t\t\tac->au[i]->count++;\n+\t\t\treturn;\n+\t\t}\n+\tif (ac->alloc <= ac->nr) {\n+\t\tac->alloc = alloc_nr(ac->alloc);\n+\t\tac->au = xrealloc(ac->au, sizeof(struct author_record *) *\n+\t\t\t\t  ac->alloc);\n+\t}\n+\tar = xcalloc(1, sizeof(struct author_record));\n+\tar->name = xmalloc(eon - au + 1);\n+\tmemcpy(ar->name, au, eon - au);\n+\tar->name[eon - au] = 0;\n+\tar->count = 1;\n+\tac->au[ac->nr++] = ar;\n+}\n+\n+static void show_fpc(struct object_array *list)\n+{\n+\tint i;\n+\tstruct author_count ac;\n+\n+\tif (!list->nr)\n+\t\treturn;\n+\tmemset(&ac, 0, sizeof(ac));\n+\tfor (i = 0; i < list->nr; i++)\n+\t\tadd_author((struct commit *) list->objects[i].item, &ac);\n+\tqsort(ac.au, ac.nr, sizeof(struct author_record *), cmp_count);\n+\n+\tfor (i = 0; i < ac.nr; i++) {\n+\t\tif (i) {\n+\t\t\tif (i < ac.nr - 1)\n+\t\t\t\tfputs(\", \", stdout);\n+\t\t\telse if (ac.nr != 2)\n+\t\t\t\tfputs(\", and \", stdout);\n+\t\t\telse\n+\t\t\t\tfputs(\" and \", stdout);\n+\t\t}\n+\t\tif (ac.au[i]->count < 2)\n+\t\t\tprintf(\"%s\", ac.au[i]->name);\n+\t\telse\n+\t\t\tprintf(\"%s (%d)\", ac.au[i]->name, ac.au[i]->count);\n+\t\tfree(ac.au[i]->name);\n+\t\tfree(ac.au[i]);\n+\t}\n+\tfree(ac.au);\n+\tputchar('\\n');\n+\n+\tfor (i = 0; i < list->nr; i++) {\n+\t\tstruct commit *c = (struct commit *) list->objects[i].item;\n+\t\tchar *buf = c->buffer;\n+\t\tchar *it = \"<unnamed>\";\n+\t\tint len = strlen(it);\n+\t\tbuf = strstr(buf, \"\\n\\n\");\n+\t\tif (buf) {\n+\t\t\tchar *lineend;\n+\t\t\twhile (*buf && isspace(*buf))\n+\t\t\t\tbuf++;\n+\t\t\tif (!*buf)\n+\t\t\t\tgoto emit;\n+\t\t\tlineend = strchr(buf, '\\n');\n+\t\t\tif (!lineend)\n+\t\t\t\tgoto emit;\n+\t\t\twhile (buf < lineend && isspace(*lineend))\n+\t\t\t\tlineend--;\n+\t\t\tlen = lineend - buf + 1;\n+\t\t\tit = buf;\n+\t\t}\n+\temit:\n+\t\tprintf(\" %.*s\\n\", len, it);\n+\t}\n+\tputchar('\\n');\n+}\n+\n+int cmd_log_fpc(int argc, const char **argv, const char *prefix)\n+{\n+\tstruct rev_info rev;\n+\tstruct commit *c;\n+\tstruct object_array result = { 0, 0, NULL };\n+\tint i;\n+\n+\tgit_config(git_log_config);\n+\tinit_revisions(&rev, prefix);\n+\trev.always_show_header = 1;\n+\tcmd_log_init(argc, argv, prefix, &rev);\n+\n+\tprepare_revision_walk(&rev);\n+\twhile ((c = get_revision(&rev)) != NULL)\n+\t\tadd_object_array(&(c->object), NULL, &result);\n+\n+\t/* clear flags and mark them \"relevant\" */\n+\tfor (i = 0; i < result.nr; i++)\n+\t\tresult.objects[i].item->flags |= FPC_RESULT;\n+\n+\tfor (;;) {\n+\t\tstruct object_array current;\n+\n+\t\tfor (i = 0; i < result.nr; i++) {\n+\t\t\tif (!(result.objects[i].item->flags & FPC_SHOWN))\n+\t\t\t\tbreak;\n+\t\t}\n+\t\tif (i >= result.nr)\n+\t\t\tbreak;\n+\n+\t\tmemset(&current, 0, sizeof(current));\n+\t\tc = (struct commit *) result.objects[i].item;\n+\t\twhile (c) {\n+\t\t\tint flags = c->object.flags;\n+\n+\t\t\tif ((flags & (FPC_RESULT|FPC_SHOWN)) == FPC_RESULT) {\n+\t\t\t\tadd_object_array(&(c->object), NULL, &current);\n+\t\t\t\tc->object.flags |= FPC_SHOWN;\n+\t\t\t}\n+\t\t\tif (!c->object.parsed)\n+\t\t\t\tparse_object(c->object.sha1);\n+\t\t\tif (!c->parents)\n+\t\t\t\tbreak;\n+\t\t\tc = c->parents->item;\n+\t\t}\n+\n+\t\t/* Finally, show the series. */\n+\t\tshow_fpc(&current);\n+\t}\n+\n+\t/* free them */\n+\tfor (i = 0; i < result.nr; i++) {\n+\t\tc = (struct commit *) result.objects[i].item;\n+\t\tfree(c->buffer);\n+\t\tfree_commit_list(c->parents);\n+\t}\n+\treturn 0;\n+}\n+\n static int istitlechar(char c)\n {\n \treturn (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||\ndiff --git a/builtin.h b/builtin.h\nindex 43fed32..a94540d 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -38,6 +38,7 @@ extern int cmd_grep(int argc, const char **argv, const char *prefix);\n extern int cmd_help(int argc, const char **argv, const char *prefix);\n extern int cmd_init_db(int argc, const char **argv, const char *prefix);\n extern int cmd_log(int argc, const char **argv, const char *prefix);\n+extern int cmd_log_fpc(int argc, const char **argv, const char *prefix);\n extern int cmd_ls_files(int argc, const char **argv, const char *prefix);\n extern int cmd_ls_tree(int argc, const char **argv, const char *prefix);\n extern int cmd_mailinfo(int argc, const char **argv, const char *prefix);\ndiff --git a/git.c b/git.c\nindex 1aa07a5..65d98bd 100644\n--- a/git.c\n+++ b/git.c\n@@ -243,6 +243,7 @@ static void handle_internal_command(int argc, const char **argv, char **envp)\n \t\t{ \"help\", cmd_help },\n \t\t{ \"init-db\", cmd_init_db },\n \t\t{ \"log\", cmd_log, RUN_SETUP | USE_PAGER },\n+\t\t{ \"log-fpc\", cmd_log_fpc, RUN_SETUP | USE_PAGER },\n \t\t{ \"ls-files\", cmd_ls_files, RUN_SETUP },\n \t\t{ \"ls-tree\", cmd_ls_tree, RUN_SETUP },\n \t\t{ \"mailinfo\", cmd_mailinfo },\n-- \n1.4.4.1.ge3fb\n\n"},{"id":"297946","messageId":"Pine.LNX.4.64.0611261652520.30076@woody.osdl.org","threadId":"43195","inReplyTo":"7v8xhxsopp.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-27T01:06:08Z","receivedAt":"2006-11-27T01:06:08Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Nov 2006, Junio C Hamano wrote:\n>\n> This implements an experimental \"git log-fpc\" command that shows\n> short-log style output sorted by topics.\n> \n> A \"topic\" is identified by going through the first-parent\n> chains; this ignores the fast-forward case, but for a top-level\n> integrator it often is good enough.\n\nUmm. May I suggest that you try this with the kernel repo too..\n\nThere, the \"first parent chain\" tends to be less interesting than a lot of \nother heuristics:\n\n - committer\n\n   If the committer changes, you should probably consider it a break, the \n   same way a second parent would be a break. You probably won't see this \n   in the git archive, because there tends to be a single committer, but \n   on something like the kernel where we really merge other peoples repos, \n   it's going to be as good (or better) than looking at \"other parents\".\n\n - subdirectory heuristics\n\n   Again, with git it's not very interesting, but I bet that you'd be able \n   to use heuristics like \"the bulk of the changes were contained within \n   this directory tree\" for projects like the kernel, and automatically \n   decide on \"topics\" like drivers/scsi, fs/ext3 etc.\n\nIn other words, I don't think the \"fpc\" decision is even very interesting. \nIf you _really_ want to do a cool shortlogger, I bet it can be done, but I \nsuspect that it would be a LOT cooler to do some automatic bayesian \nclustering based on committer, author and list of filenames changed.\n\nOf course, such a thing done well would probably be worthy of a doctoral \nthesis or something. Maybe somebody on this list who is into bayesian \nclustering and doesn't have a thesis subject...\n\n(Of course, since I haven't been in a University setting for the last ten \nyears, maybe bayesian clustering isn't the cool thing to work on any \nmore).\n\nAnyway, \"topics\" really should be something that is extremely open to \nvarious clustering models, bayesian or not ..\n\n"},{"id":"297046","messageId":"7vejrpr7mw.fsf@assigned-by-dhcp.cox.net","threadId":"43195","inReplyTo":"Pine.LNX.4.64.0611261652520.30076@woody.osdl.org","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-27T01:38:31Z","receivedAt":"2006-11-27T01:38:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@osdl.org> writes:\n\n> On Sun, 26 Nov 2006, Junio C Hamano wrote:\n>>\n>> This implements an experimental \"git log-fpc\" command that shows\n>> short-log style output sorted by topics.\n>> \n>> A \"topic\" is identified by going through the first-parent\n>> chains; this ignores the fast-forward case, but for a top-level\n>> integrator it often is good enough.\n>\n> Umm. May I suggest that you try this with the kernel repo too..\n\nHave you?\n\nI've compared \n\n\tgitk HEAD~40..HEAD\n\nand \n\n\tgit-log-fpc --no-merges HEAD~40..HEAD\n\nAdmittedly, the first group (\"from the tip of the master\") tends\nto be seriously mixed up without a fixed theme (well the theme\nappears to be \"fix trivial warnings and compilation breakages\nnot limited to any particular subsystem\"), but I find the other\ngroups quite a sane representation of what actually happened.\n\nMy copy of your tree is a bit old (HEAD is at 1abbfb412), but I\nsee:\n\n - a two-commit series on MIPS via Ralf Baechle,\n - a four-commit series on ARM via Russel King,\n - a three-commit series on POWERPC via Paul Mackerras,\n - a seventeen-commit series in net/ area via Dave Miller,\n - a three-commit series on x86_64 via Andi Kleen.\n ...\n\nAs you said, committer would be a good addition to break a\nfast-forward case to make it even better.\n"},{"id":"296156","messageId":"Pine.LNX.4.64.0611261747380.30076@woody.osdl.org","threadId":"43195","inReplyTo":"7vejrpr7mw.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-27T01:53:48Z","receivedAt":"2006-11-27T01:53:48Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Nov 2006, Junio C Hamano wrote:\n> \n>  - a two-commit series on MIPS via Ralf Baechle,\n>  - a four-commit series on ARM via Russel King,\n>  - a three-commit series on POWERPC via Paul Mackerras,\n>  - a seventeen-commit series in net/ area via Dave Miller,\n>  - a three-commit series on x86_64 via Andi Kleen.\n\nYou'll reasonably often see in the kernel:\n\n - a patch-series by Andrew (where nothing but filename clustering really \n   would help: the committer is me, and the thing is linear)\n\n - linearly on top of that, a git merge that was a fast-forward \n   (especially from the subset of people who actively rebase their trees: \n   that notably includes Dave Miller, but also for example the DVB people)\n\nso purely a first-parent logic would not catch that case at all (but the \ncommitter would at least catch the \"patch-series by Andrew\" -> \"Merge of \nnetwork tree by Davem\" break).\n\nBut especially with long patch-series through Andrew, it would be nice to \nhave some other heuristics (although they _tend_ to be fairly random, \nespecially at the end of the release cycle - at the beginning, I tend to \nhave series of 100-200 patches that often _could_ be clearly clustered \ninto a few clusters).\n\nAnyway, the real win of clusterign would likely be for big releases, ie \nsoemthing like \"v2.6.18..v2.6.19-rc1\", where there's definitely some \nclustering even apart from just merging (although the merge topology will \ndefinitely get some of it)\n\n"},{"id":"298182","messageId":"7vac2dr6ua.fsf@assigned-by-dhcp.cox.net","threadId":"43195","inReplyTo":"Pine.LNX.4.64.0611261652520.30076@woody.osdl.org","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-27T01:55:41Z","receivedAt":"2006-11-27T01:55:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@osdl.org> writes:\n\n> On Sun, 26 Nov 2006, Junio C Hamano wrote:\n>>\n>> This implements an experimental \"git log-fpc\" command that shows\n>> short-log style output sorted by topics.\n>> \n>> A \"topic\" is identified by going through the first-parent\n>> chains; this ignores the fast-forward case, but for a top-level\n>> integrator it often is good enough.\n\nAfter sending out a response, I re-read your message because I\ndid not quite get where bayesian would come into the picture.\n\nI think I should have used the word \"topic branch\" instead of\n\"topic\".  In other words, I was not interested in sifting the\nvarious totally unrelated linear commits into groups that deal\nwith distinct problems.\n\nBut again you are showing your superiour intelligence by setting\nthe problem in a much grander scheme ;-), where there is no such\ndeveloper discipline that would help the shortlogger (like use\nof topic branches).  In such a case, you would need a set of\nheuristics that you described.\n"},{"id":"298711","messageId":"Pine.LNX.4.64.0611261836250.30076@woody.osdl.org","threadId":"43195","inReplyTo":"7vac2dr6ua.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-27T02:52:57Z","receivedAt":"2006-11-27T02:52:57Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Nov 2006, Junio C Hamano wrote:\n> \n> I think I should have used the word \"topic branch\" instead of\n> \"topic\".  In other words, I was not interested in sifting the\n> various totally unrelated linear commits into groups that deal\n> with distinct problems.\n\nWell, I think you're grown slightly jaded by the fact that git has very \nactive \"normal\" development, that is actually done by you on the main \nbranch, and you do basically zero rebasing along the side branches.\n\nI think that's actually likely the exception rather than the rule. It's \nmuch more likely that people have almost _all_ active development done on \nside branches, and that - together with rebasing of the side branches - \ninevitably means that the \"main branch\" ends up not having such a clean \nset of \"topic branch\" merges.\n\nIn addition, on a more mature tree, a lot (probably _most_) of the commits \naren't really \"topics\" at all, but \"maintenance\", which exacerbates the \nproblem: you don't have a \"line of development of this feature\", you tend \nto have much more of a random \"fix this general area\", where the only \ncommon theme may be the fact that things are _related_ to some common \nsubsystem, but not that they are a \"topic branch\" in the _development_ \nsense.\n\nPut another way: bugs get fixed one by one, not in a nice linear fashion \nby \"topic\".\n\nSo I'm coming at it from a totally different project - where \"topic \nbranches\" simply aren't delineated as much, and even when they are, they \ntend to be merged in multiple steps (and they pull both ways when they \naren't re-based).\n\nSo that's why I don't think the pure branch topology is as interesting. A \nsingle line of development ends up being useful for you, and we'll \ncertainly see _some_ of that, but in the kernel, I pretty much guarantee \nthat you probably get better \"topic clustering\" by going simply by author, \nlike the old standard \"git shortlog\" does. Because that will tend to get \nthe clustering at a finer granularity (ie not just \"networking\", but \nthings like \"packet filtering\" etc).\n\nSo the \"sort by people\" actually works fairly well, but it's kind of an \n\"incidental\" thing, and it _would_ be potentially useful to have other \nways of grouping things.\n\nSee? It's not about \"superior intelligence\", it's about simply a totally \ndifferent development phase (and a less strictly defined problem space).\n\n"},{"id":"298637","messageId":"7vbqmtmlkv.fsf@assigned-by-dhcp.cox.net","threadId":"43195","inReplyTo":"Pine.LNX.4.64.0611261836250.30076@woody.osdl.org","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-27T06:48:32Z","receivedAt":"2006-11-27T06:48:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@osdl.org> writes:\n\n> I think that's actually likely the exception rather than the rule. It's \n> much more likely that people have almost _all_ active development done on \n> side branches, and that - together with rebasing of the side branches - \n> inevitably means that the \"main branch\" ends up not having such a clean \n> set of \"topic branch\" merges.\n\nYou are absolutely right about \"Andrew patchbomb\" which is\nlinear and does not have the series boundary.  Import from\nmostly linear foreign SCM would have the same issue.  Merge\ntopology would not help us at all in these cases.\n\n> In addition, on a more mature tree, a lot (probably _most_) of the commits \n> aren't really \"topics\" at all, but \"maintenance\", which exacerbates the \n> problem: you don't have a \"line of development of this feature\",\n> ...\n> Put another way: bugs get fixed one by one, not in a nice linear fashion \n> by \"topic\".\n\nAgain, you are right, but that only means topic based grouping\nis not for everybody, and certainly is not suitable for a long\nstretch of commits on the trunk of a mature project because they\ntend to touch everywhere and not all that clustered.  If those\nbugs were fixed by committing on separate topic branches and\nthen later merged, the topology based clustering would get the\ngrouping right, but I would imagine we would end up seeing\nhundreds of such short groups which would not be useful at all.\nIn such cases, it would be much more useful to have one huge\ngroup that says \"these are small fixes, each of which may touch\ndifferent areas -- they are not related but grouped together\nbecause they are all small, obviously correct and harmless\nfixes\".  So I suspect that is a slightly different issue -- it\njust illustrates the need for an \"ungrouped\" bin.\n\n> So I'm coming at it from a totally different project - where \"topic \n> branches\" simply aren't delineated as much, and even when they are, they \n> tend to be merged in multiple steps (and they pull both ways when they \n> aren't re-based).\n\nI agree multiple steps merge and merging both ways would happen\nin real life, but I had an impression that fpc handles that\ntopology reasonably well, unless that \"merge from upstream\" are\nof \"too frequent, automated and useless\" kind of merges.\n\n> ... but in the kernel, I pretty much guarantee \n> that you probably get better \"topic clustering\" by going simply by author, \n> like the old standard \"git shortlog\" does. Because that will tend to get \n> the clustering at a finer granularity (ie not just \"networking\", but \n> things like \"packet filtering\" etc).\n>\n> So the \"sort by people\" actually works fairly well, but it's kind of an \n> \"incidental\" thing, and it _would_ be potentially useful to have other \n> ways of grouping things.\n\nI think \"networking\" vs \"packet filtering\" largely depends on\nhow the networking subsystem you pull from is managed.  If\nnetfilter comes as e-mailed patches to DaveM and are applied\nonto the trunk of networking subsystem, we will face exactly the\nsame problem as we have with Andrew's patchbomb to your trunk.\n\nIf it were managed on a separate topic branch in the networking\nsubsystem repository (either DaveM manages them in his\nrepository as a topic, or DaveM pulls from netfilter git\nrepository -- I do not know how that part of the patchflow\nworks), I would imagine you would get the same \"per topic\"\ngrouping.\n\nAnother factor is that the author population of a wide and\nmature project like the kernel tends to be more diverse, and a\nsingle person tends to be focused on one thing at a time while\nothers work on different things.  There is enough work in one\nspecific area for one person to do, and the project is too wide\nfor one person to be everywhere.\n"},{"id":"296818","messageId":"Pine.LNX.4.64.0611270748300.30076@woody.osdl.org","threadId":"43195","inReplyTo":"7vbqmtmlkv.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-27T16:20:41Z","receivedAt":"2006-11-27T16:20:41Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Nov 2006, Junio C Hamano wrote:\n> \n> I think \"networking\" vs \"packet filtering\" largely depends on\n> how the networking subsystem you pull from is managed.  If\n> netfilter comes as e-mailed patches to DaveM and are applied\n> onto the trunk of networking subsystem, we will face exactly the\n> same problem as we have with Andrew's patchbomb to your trunk.\n\nMost of the subsystems end up using patches - they're simply better ways \nto move things around and have people comment on them than saying \"please \npull on this tree to see my suggestion\".  I do it myself: even when I \n_generate_ the diff in my tree, I will often just do a \n\n\tgit diff > ~/diff\n\nand then import the thing into my mailer, and say \"Maybe something like \nthis?\".\n\nSo I think patches are fundamentally the core way to get things in the \nperiphery into just about any system. Maybe we do it more than most just \nbecause we're so _used_ to them, but I actually think that if the kernel \ndoes it more than most (and I'm not sure it does), it's simply because the \nthing about patches is that they really _work_.\n\nSo yes, the network subsystem tends to be entirely linear by the time it \nhits me. That's true of a lot of other subsystems too (SCSI etc). There's \na _few_ subsystems that actually have real topic branches: ACPI and \nnetwork driver development comes to mind, but it seems to actually be the \nexception rather than the rule.\n\n(I think that a lot of people work like I occasionally do: they do have \ntheir own local branches for some stuff, but they end up re-linearizing \nand keeping them active with \"git rebase\", so the branches really are \npurely local, rather than something that is visible in the end result).\n\nBut the REAL reason I'd love to see a smarter \"data-mining\" git log \n(whether it does things by bayesian clustering or any other kind of \ngrouping technology) is that this is actually something that people ask \nfor: when I make my \"git shortlog\" for major releases, the thing is often \nthousands of lines long, and it would be _beautiful_ if that could be \ndata-mined somewhat more intelligently.\n\nSo, for example, do a simple\n\n\tgit shortlog v2.6.17..v2.6.18\n\n(with the shortlog in \"next\" that can do this - btw, why doesn't it \ndefault to using PAGER like \"git log\" does?), and realize that it's about \n8500 lines of stuff, and nobody can really be expected to read it. It's \nnot a \"shortlog\" in other words.\n\nSo what would a _nice_ \"shortlog\" do? I'd _love_ to see ways to make it \nmore concise, more \"short\" for something like this. Look at the output as \na _non_kernel_ person, and what does it tell you? Not a lot. It's just too \nbig.\n\nExamples of what I think would be _really_ useful (much more so than \ngoing by \"topic branches\", even if they existed):\n\n - Clustering.\n\n   The author-based clustering does work, but it would be even better to \n   cluster by other methods (\"subsystem\" - either by subdirectory, or by \n   noticing filename patters, or even patterns in the patches: there's a \n   lot of academic work on clustering human text, perhaps not as much on \n   clustering patches).\n\n - Shortening\n\n   The \"shortlog\" often isn't. It's wonderful for small things as-is, but \n   once it reaches a hundred lines or more, it's less so. It would often \n   be nice to be able to say \"only show the 100 biggest patches\" (or \n   preferably something smarter like \"the 25 biggest clusters, with a \n   short 4-line clustering explanation\", but even just the \"biggest \n   patches\" is useful in itself and much simpler)\n\n - External annotations (eventually)\n\n   One of the things that people like LWN editor Jonathan Corbet would \n   want is a way to say which patches are \"important\". But the thing is, \n   \"importance\" is (a) fleeting and (b) not necessarily as obvious when \n   the commit is made as it is afterwards. So you cannot (and must not) \n   mark things \"important\" at commit-time, and it thus can't really be \n   part of the repo itself, but at the same time, this is definitely \n   something that _could_ be somehow logged/annotated externally.\n\nNow, I realize that these are all pipe-dreams, but so was my old \"a better \nannotate than annotate\" a year or two ago. So I'm not saying that people \nshould work on this, I'm just saying that it's worth perhaps thinking \nabout, because I think the git model does actually give us the power to \n_do_ things like this. Eventually.\n\nAnd the reason? Performance! Git is fast enough that we really _can_ \nafford to do things like \"generate diffs for every single commit in the \nrange v2.6.17..v2.6.18\" and it takes me just 20 seconds to do on a \nreasonable machine with \"git log -p\". So good performance means that we \ncan _afford_ to do a diffstat for everything (or, just raw diffs to make \nit even cheaper - quite often you care more about _which_ files and how \nmany files something touched than the actual size of the diff in those \nfiles itself), and using that diffstat to some day generate shortlogs that \nare more useful for people like Jonathan Corbet and others that just want \nto get an overview of \"what happened\"?\n\n"},{"id":"298411","messageId":"Pine.LNX.4.63.0611280040480.30004@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43195","inReplyTo":"Pine.LNX.4.64.0611261652520.30076@woody.osdl.org","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-11-27T23:46:33Z","receivedAt":"2006-11-27T23:46:33Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sun, 26 Nov 2006, Linus Torvalds wrote:\n\n> Of course, such a thing done well would probably be worthy of a doctoral \n> thesis or something. Maybe somebody on this list who is into bayesian \n> clustering and doesn't have a thesis subject...\n\nFunny you should mention it... I recently was exposed to Formal Concept \nAnalysis, and immediately thought that this would have applications in the \nvisualization of source codes' histories.\n\nMaybe there is a way to apply Bayesian Inference to determine a subset \nwhich bears the highest information / subset size ratio.\n\nAs for reducing the number of lines in the shortlog: taking myself as an \nexample, I often touch the same code several times, just to fix bugs. So, \nif the same code was touched several times, just take the first oneline, \nand add \"(+fixes)\". Of course, this is more like a wedding between \nshortlog and annotate, and likely to be slow.\n\nCiao,\nDscho\n"},{"id":"294261","messageId":"7v7ixge8j2.fsf@assigned-by-dhcp.cox.net","threadId":"43195","inReplyTo":"Pine.LNX.4.63.0611280040480.30004@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-28T00:09:53Z","receivedAt":"2006-11-28T00:09:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> As for reducing the number of lines in the shortlog: taking myself as an \n> example, I often touch the same code several times, just to fix bugs. So, \n> if the same code was touched several times, just take the first oneline, \n> and add \"(+fixes)\". Of course, this is more like a wedding between \n> shortlog and annotate, and likely to be slow.\n\nInteresting.  While driving to work this morning I had the same\nthought.  A revision that does not appear in the output from\n\n\tfor file in $(list of files the commit touches)\n        do\n\t\tgit blame v2.6.17..v2.6.18 -- $file\n\tdone\n\ncan safely be omitted from the shortlog, because later changes\nfully supersedes it.\n\nI think the list of \"important\" changes is an interesting\nproblem, but the importance may not directly be related to the\nnumber of paths a patch touches (e.g. \"you reorder the members\nof a structure everybody uses in one include file and everything\nstarts performing faster due to better cache behaviour\" would be\na few lines of a single header file).  Also better clues to\njudge the importance would be found outside the repository.\n\"The patch discussed by many people on the list\" and \"the patch\nthat had very many iteration to get in the final shape\" would\ncertainly be interesting ones, but that information is often not\nfound in the repository.\n\n"},{"id":"296290","messageId":"20061128131139.GA10874@coredump.intra.peff.net","threadId":"43195","inReplyTo":"7v7ixge8j2.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2006-11-28T13:11:39Z","receivedAt":"2006-11-28T13:11:39Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Nov 27, 2006 at 04:09:53PM -0800, Junio C Hamano wrote:\n\n> Interesting.  While driving to work this morning I had the same\n> thought.  A revision that does not appear in the output from\n> \n> \tfor file in $(list of files the commit touches)\n>         do\n> \t\tgit blame v2.6.17..v2.6.18 -- $file\n> \tdone\n\nJust for fun, I took a look at what we might see by ordering commits by\ntheir \"amount of blamedness\". That is, the count of lines introduced by\na commit which were not later superseded. The script I used is below:\n\n#!/bin/sh\n\nstart=$1; shift\nend=$1; shift\n\nstart_sha1=`git-rev-parse $start^{}`\ngit-rev-list --parents $start..$end >revs\necho $start_sha1 >>revs\nfor i in `git-diff --raw -r $start $end | cut -f2`; do\n  echo blaming $i... >&2\n  git-blame -l -S revs $i | cut -d' ' -f1\ndone |\n  grep -v $start_sha1 |\n  sort | uniq -c | sort -rn |\n  while read count hash; do\n    echo \"$count `git-rev-list --max-count=1 --pretty=oneline $hash`\"\n  done\n\nThe top 15 for v1.4.3 to v1.4.4 are:\n\n1604 6973dcaee76ef7b7bfcabd2f26e76205aae07858 Libify diff-files.\n1100 9f613ddd21cbd05bfc139d9b1551b5780aa171f6 Add git-for-each-ref: helper for language bindings\n1050 cee7f245dcaef6dade28464f59420095a9949aac git-pickaxe: blame rewritten.\n700 58e60dd203362ecb9fdea765dcc2eb573892dbaf Add support for pushing to a remote repository using HTTP/DAV\n571 9f1afe05c3ab7228e21ba3666c6e35d693149b37 gitk: New improved gitk\n524 197e8951abd2ebf2c70d0847bb0b38b16b92175b http-push: support for updating remote info/refs\n504 83b5d2f5b0c95fe102bc3d1cc2947abbdf5e5c5b builtin-grep: make pieces of it available as library.\n462 aa1dbc9897822c8acb284b35c40da60f3debca91 Update http-push functionality\n344 a57a9493df00b6fbb3699fda8ceedf4ac0783ac6 Added Perl git-cvsimport-script\n343 f8b28a4078a29cbf93cac6f9edd8d5c203777313 gitk: Add a tree-browsing mode\n323 00449f992b629f7f7884fb2cf46ff411a2a4f381 Make git-fmt-merge-msg a builtin\n285 fd8ccbec4f0161b14f804a454e68b29e24840ad3 gitk: Work around Tcl's non-standard names for encodings\n283 9cf6d3357aaaaa89dd86cc156221b7b604e9358c Add git-index-pack utility\n277 e4fbbfe9eccd37c0f9c060eac181ce05988db76c Add git-zip-tree\n256 da7c24dd9c75d014780179f8eb843968919e4c46 gitk: Basic support for highlighting one view within another\n\nThe bottom 15 are:\n\n1 076b2324cdca9a2825c569cf9ec02d219c237e26 show-branch: make it work in a subdirectory.\n1 061303f0b50a648db8e0af23791fc56181f6bf93 cvsimport: always set $ENV{GIT_INDEX_FILE} to $index{$branch}\n1 057bc808b4aa2e7795f9bd395e68071301bc0b74 path-list: fix path-list-insert return value\n1 04c13d38772c77997d8789ee2067cc351b66e2aa Save the maxwidth setting in the ~/.gitk file.\n1 041a7308de3e6af36c5a6cc3412b542f42314f3f sha1_name.c: prepare to make get_tree_entry() reusable from others.\n1 0360e99d06acfbb0fcb72215cf6749591ee53290 [PATCH] Fix git-rev-parse --default and --flags handling\n1 02d3dca3bff6a67dead9f5b97dfe3576fe5b14e5 revision.c: fix \"dense\" under --remove-empty\n1 02c5cba2007856465710aa37cd41b404372ab95b find_unique_abbrev() with len=0 should not abbreviate\n1 02853588a48eddbaa42b58764960394e416d68bf Typofix in Makefile comment.\n1 024701f1d88d79f3777bf45c82437f40a80b6eaa Make pack-objects chattier.\n1 021b6e454944a4fba878651ebf9bfe0a3f6c3077 Make index file locking code reusable to others.\n1 01ff767a3266a1876ce24a200c45786083768fda Merge branch 'lt/refs' into next\n1 01385e275828c1116ea9bfcf827f82f450ee8f5f Comment fixes.\n1 013049c985e4095106e545559c17bc594d56468d revert/cherry-pick: handle single quote in author name.\n1 0086e2c854e3af3209915e4ec2f933bcef400050 Rename lost+found to lost-found.\n\nThis approach isn't without value; the top lines really _are_ important\nchanges, as they show where a lot of work (line-wise) went.  The bottom\nlines are relatively unimportant (oh boy, comment fixes!).  But there\nare obviously some one-liners that are very interesting. For example:\n  1 0abc0260fa3419de649fcc1444e3d256a17ca6c7 pager: default to LESS=FRSX not LESS=FRS\ngenerated quite a bit of discussion on the list, and end users would\ncare about it.\n\nI think it's clear that \"important commits\" is going to be something we\ndetermine through heuristics; blame-able lines is probably a heuristic\nworth considering.\n\n"},{"id":"297037","messageId":"Pine.LNX.4.63.0611281439500.30004@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43195","inReplyTo":"20061128131139.GA10874@coredump.intra.peff.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-11-28T13:43:49Z","receivedAt":"2006-11-28T13:43:49Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 28 Nov 2006, Jeff King wrote:\n\n> I think it's clear that \"important commits\" is going to be something we \n> determine through heuristics; blame-able lines is probably a heuristic \n> worth considering.\n\nI was surprised that not more of my stuff was in the top-15, since I \nsubmitted less-than-finished patches quite often. Especially \nmerge-recursive was quite a bit of work for Alex and me.\n\nBTW merge-recursive is a perfect example why this approach will break \ndown: most of the rewrite in C took place in a private repository with \nquite some commits. This does not show in the git repository.\n\nI fully expect the linux repository to behave similarly, since most of the \nfeatures are cooked elsewhere, and not all of them are pulled, but some \nare applied (i.e. they appear out of nowhere from the repository's \nviewpoint).\n\nCiao,\nDscho\n"},{"id":"294157","messageId":"20061128135614.GA29505@coredump.intra.peff.net","threadId":"43195","inReplyTo":"Pine.LNX.4.63.0611281439500.30004@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2006-11-28T13:56:14Z","receivedAt":"2006-11-28T13:56:14Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Nov 28, 2006 at 02:43:49PM +0100, Johannes Schindelin wrote:\n\n> BTW merge-recursive is a perfect example why this approach will break \n> down: most of the rewrite in C took place in a private repository with \n> quite some commits. This does not show in the git repository.\n> \n> I fully expect the linux repository to behave similarly, since most of the \n> features are cooked elsewhere, and not all of them are pulled, but some \n> are applied (i.e. they appear out of nowhere from the repository's \n> viewpoint).\n\nYes, I think this would be more useful in concert with some sort of\ngrouping. If we can make a group of commits related to merge-recursive,\nand score them as a single item, then they can be compared to other\ngroups (which may consist of a single commit or several).\n\n"},{"id":"295451","messageId":"7v1wnnysrn.fsf@assigned-by-dhcp.cox.net","threadId":"43195","inReplyTo":"20061128131139.GA10874@coredump.intra.peff.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-29T00:57:00Z","receivedAt":"2006-11-29T00:57:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> Just for fun, I took a look at what we might see by ordering commits by\n> their \"amount of blamedness\". That is, the count of lines introduced by\n> a commit which were not later superseded. The script I used is below:\n>\n> #!/bin/sh\n>\n> start=$1; shift\n> end=$1; shift\n>\n> start_sha1=`git-rev-parse $start^{}`\n> git-rev-list --parents $start..$end >revs\n> echo $start_sha1 >>revs\n> for i in `git-diff --raw -r $start $end | cut -f2`; do\n>   echo blaming $i... >&2\n>   git-blame -l -S revs $i | cut -d' ' -f1\n> done |\n>   grep -v $start_sha1 |\n>   sort | uniq -c | sort -rn |\n>   while read count hash; do\n>     echo \"$count `git-rev-list --max-count=1 --pretty=oneline $hash`\"\n>   done\n>\n> The top 15 for v1.4.3 to v1.4.4 are:\n>\n> 1604 6973dcaee76ef7b7bfcabd2f26e76205aae07858 Libify diff-files.\n\nSomething is SERIOUSLY wrong.\n\nThat commit is not even between v1.4.3 and v1.4.4.\n\n\n\n"},{"id":"297563","messageId":"20061201081117.GA20025@coredump.intra.peff.net","threadId":"43195","inReplyTo":"7v1wnnysrn.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2006-12-01T08:11:17Z","receivedAt":"2006-12-01T08:11:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Nov 28, 2006 at 04:57:00PM -0800, Junio C Hamano wrote:\n\n> > The top 15 for v1.4.3 to v1.4.4 are:\n> >\n> > 1604 6973dcaee76ef7b7bfcabd2f26e76205aae07858 Libify diff-files.\n> \n> Something is SERIOUSLY wrong.\n> \n> That commit is not even between v1.4.3 and v1.4.4.\n\nHmm, you're right. I haven't quite figured out what went wrong with the\nscript I posted. However, a somewhat simpler approach is to just use the\nrevision limiting in git-blame. The problem with this is that commits\nwhose parents aren't in the revision range end up getting blamed for a\nlot of lines they're not responsible for.\n\nAs a quick hack, I just threw out any revisions whose parents weren't in\nrange. This is wrong, since those revisions probably _do_ have some\ncorrectly blamed lines. It made me wonder about a possible feature for\ngit-blame: when we can't pass the blame up further, instead of taking\nresponsibility, output a \"no responsibility line\" (blaming on commit\n0{40}, or some other format). I think this should be more informative\nwhen there is a limit on the range of revisions.\n\nThe top of the \"blamedness\" list for v1.4.3..v1.4.4 is below. Important\nthings do seem to float to the top, but it would probably be much more\naccurate if we were scoring groups of commits (generated by some other\nanalysis).\n\n-Peff\n\n-- >8 --\n1050 cee7f245dcaef6dade28464f59420095a9949aac git-pickaxe: blame rewritten.\n223 fe142b3a4577a6692a39e2386ed649664ad8bd20 Rework cvsexportcommit to handle binary files for all cases.\n219 c31820c26b8f164433e67d28c403ca0df0316055 Make git-branch a builtin\n216 636171cb80255682bdfc9bf5a98c9e66d4c0444a make index-pack able to complete thin packs.\n182 b1f33d626501c3e080b324e182f1da76f49b5bf9 Swap the porcelain and plumbing commands in the git man page\n173 744d0ac33ab579845808b8b01e526adc4678a226 gitweb: New improved patchset view\n169 e30496dfcb98a305a57b835c248cbc3aa2376bfc gitweb: Support for 'forks'\n142 5b329a5f5e3625cdc204e3d274c89646816f384c t6022: ignoring untracked files by merge-recursive when they do not matter\n134 c0990ff36f0b9b8e806c8f649a0888d05bb22c37 Add man page for git-show-ref\n128 780e6e735be189097dad4b223d8edeb18cce1928 make pack data reuse compatible with both delta types\n121 2d477051ef260aad352d63fc7d9c07e4ebb4359b add the capability for index-pack to read from a stream\n116 576162a45f35e157427300066b0ff566ff698a0f remove .keep pack lock files when done with refs update\n110 e827633a5d7d627eb1170b2d0c71e944d0d56faf Built-in cherry\n"},{"id":"294554","messageId":"7vlklranrt.fsf@assigned-by-dhcp.cox.net","threadId":"43195","inReplyTo":"20061201081117.GA20025@coredump.intra.peff.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-12-01T10:55:34Z","receivedAt":"2006-12-01T10:55:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Tue, Nov 28, 2006 at 04:57:00PM -0800, Junio C Hamano wrote:\n>\n>> > The top 15 for v1.4.3 to v1.4.4 are:\n>> >\n>> > 1604 6973dcaee76ef7b7bfcabd2f26e76205aae07858 Libify diff-files.\n>> \n>> Something is SERIOUSLY wrong.\n>> \n>> That commit is not even between v1.4.3 and v1.4.4.\n>\n> Hmm, you're right. I haven't quite figured out what went wrong with the\n> script I posted. However, a somewhat simpler approach is to just use the\n> revision limiting in git-blame. The problem with this is that commits\n> whose parents aren't in the revision range end up getting blamed for a\n> lot of lines they're not responsible for.\n\nThe way you used \"-S rev\" was wrong.  It is a way to temporarily\ninstall grafts and nothing else; but your grafts introduced that\nway exactly matched the true parenthood except for the bottom\ncommit and side branches merged during the timeframe leaked\nright through your grafts.  The digger started from your HEAD\n(whatever that happened to be) along with the true parenthood\nand found an way ancient ancestor.\n\nA \"bit more correct\" script would have been something like this.\n\n-- >8 --\n#!/bin/sh\n#\n# Usage: sh ./run-me v1.4.3 v1.4.4\n#\nbottom=${1?bottom} top=${2?top}\nbottom=$(git rev-parse --verify \"$bottom^0\")\nrange=\"$bottom..$top\"\ntop=$(git rev-parse --verify \"$top^0\")\n\nfor path in $(git diff --name-only -r --diff-filter=AM \"$range\")\ndo\n\techo >&2 \"* $path\"\n\tgit blame -l -C \"$range\" -- \"$path\"\ndone | sed -e 's/ .*//' | sort | uniq -c | sort -n -r |\nwhile read num hash\ndo\n\ttest \"$hash\" = \"$bottom\" && continue\n\tit=$(git rev-list --pretty=oneline --abbrev --abbrev-commit -1 \"$hash\")\n\tprintf '%6d %s\\n' $num \"$it\"\ndone\n-- 8< --\n\nBut as you correctly observed, even the above script is wrong.\nThe top one blamed with the above script is this commit:\n\n  8301 808239a Merge branch 'sk/ftp'\n\nBut that is an ancestor of v1.4.3!\n\nWhat's wrong is that the ancestry graph around that commit\nroughly looks like this:\n\n         z---o---o---o\n        /             \\\n  808239a--v1.4.3--o---*---o---v1.4.4\n\nThe pickaxe passes the blame around to the parents but does not\nallow a \"boundary\" commits to pass the blame to their parents.\nAs the result, the blame at the commit marked with '*' are split\nalong both branches, and after the leftmost commit 'z' passes\nits blame to its parent, it stops there and ends up blaming\n808239a, which is an ancestor of the original \"boundary\" commit\nv1.4.3 given from the command line.  What's wrong with my script\nquoted above is that the filter that checks $hash with $bottom;\nit needs to check if $hash is an ancestor of $bottom.\n\nWith that change, the top commits are:\n\n  1109 9f613dd Add git-for-each-ref: helper for language bindings\n  1087 cee7f24 git-pickaxe: blame rewritten.\n   218 c31820c Make git-branch a builtin\n   209 636171c make index-pack able to complete thin packs.\n   200 fe142b3 Rework cvsexportcommit to handle binary files for all cases.\n\nwhich looks a bit more reasonable (I did not realize\nfor-each-ref was that big, but in fact it has its own\nmini-language).\n\nWhile what blame outputs is technically correct, it is not very\nuseful for this kind of application.  As you said, it probably\nmakes sense to gray-out the lines that are blamed on boundary\ncommits.\n\nSide note: one might be tempted to say \"then blame v1.4.3 for\nlines that 808239a is blamed for\", but that is a good\nworkaround.  The original command line could have more than one\nbottom commits, and the final blame might go to a common\nancestor of them, and we need to randomly choose between them,\nwhich is worse than telling the truth as we currently do.\n\nAnd here is an experimental patch to do that.\n\n-- >8 --\n[PATCH] git-blame: mark lines blamed on boundary commits.\n\nLines can be blamed on a commit that is older than the boundary\ncommit given on the command line when a merge with a branch that\nforked before the boundary is involved.  Mark them specially so\nthat later changes in the interested area can be easily\nidentified.\n\nIn porcelain format, their header line that describe the commit\ngets an extended attribute line \"boundary\".  In human format,\nthe commit SHA-1 are prefixed with a '-' character.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n\n---\n\n builtin-blame.c |   15 ++++++++++++++-\n 1 files changed, 14 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin-blame.c b/builtin-blame.c\nindex dc3ffea..46a9d0e 100644\n--- a/builtin-blame.c\n+++ b/builtin-blame.c\n@@ -1090,6 +1090,11 @@ static void assign_blame(struct scoreboard *sb, struct rev_info *revs, int opt)\n \t\tif (!(commit->object.flags & UNINTERESTING) &&\n \t\t    !(revs->max_age != -1 && commit->date  < revs->max_age))\n \t\t\tpass_blame(sb, suspect, opt);\n+\t\telse {\n+\t\t\tcommit->object.flags |= UNINTERESTING;\n+\t\t\tif (commit->object.parsed)\n+\t\t\t\tmark_parents_uninteresting(commit);\n+\t\t}\n \n \t\t/* Take responsibility for the remaining entries */\n \t\tfor (ent = sb->ent; ent; ent = ent->next)\n@@ -1273,6 +1278,8 @@ static void emit_porcelain(struct scoreboard *sb, struct blame_entry *ent)\n \t\tprintf(\"committer-tz %s\\n\", ci.committer_tz);\n \t\tprintf(\"filename %s\\n\", suspect->path);\n \t\tprintf(\"summary %s\\n\", ci.summary);\n+\t\tif (suspect->commit->object.flags & UNINTERESTING)\n+\t\t\tprintf(\"boundary\\n\");\n \t}\n \telse if (suspect->commit->object.flags & MORE_THAN_ONE_PATH)\n \t\tprintf(\"filename %s\\n\", suspect->path);\n@@ -1308,8 +1315,14 @@ static void emit_other(struct scoreboard *sb, struct blame_entry *ent, int opt)\n \tcp = nth_line(sb, ent->lno);\n \tfor (cnt = 0; cnt < ent->num_lines; cnt++) {\n \t\tchar ch;\n+\t\tint length = (opt & OUTPUT_LONG_OBJECT_NAME) ? 40 : 8;\n+\n+\t\tif (suspect->commit->object.flags & UNINTERESTING) {\n+\t\t\tlength--;\n+\t\t\tputchar('-');\n+\t\t}\n \n-\t\tprintf(\"%.*s\", (opt & OUTPUT_LONG_OBJECT_NAME) ? 40 : 8, hex);\n+\t\tprintf(\"%.*s\", length, hex);\n \t\tif (opt & OUTPUT_ANNOTATE_COMPAT)\n \t\t\tprintf(\"\\t(%10s\\t%10s\\t%d)\", ci.author,\n \t\t\t       format_time(ci.author_time, ci.author_tz,\n"},{"id":"298338","messageId":"7vhcwfanja.fsf@assigned-by-dhcp.cox.net","threadId":"43195","inReplyTo":"7vlklranrt.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-12-01T11:00:41Z","receivedAt":"2006-12-01T11:00:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <junkio@cox.net> writes:\n\n> What's wrong is that the ancestry graph around that commit\n> roughly looks like this:\n>\n>          z---o---o---o\n>         /             \\\n>   808239a--v1.4.3--o---*---o---v1.4.4\n> ...\n> While what blame outputs is technically correct, it is not very\n> useful for this kind of application.  As you said, it probably\n> makes sense to gray-out the lines that are blamed on boundary\n> commits.\n>\n> Side note: one might be tempted to say \"then blame v1.4.3 for\n> lines that 808239a is blamed for\", but that is a good\n\nGaah.  \"that is NOT a good workaround\".  Sorry about the wasted\nbandwidth.\n\n> workaround.  The original command line could have more than one\n> bottom commits, and the final blame might go to a common\n> ancestor of them, and we need to randomly choose between them,\n> which is worse than telling the truth as we currently do.\n"},{"id":"294015","messageId":"20061201112302.GA1910@coredump.intra.peff.net","threadId":"43195","inReplyTo":"7vlklranrt.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] (experimental) per-topic shortlog.","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2006-12-01T11:23:02Z","receivedAt":"2006-12-01T11:23:02Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Dec 01, 2006 at 02:55:34AM -0800, Junio C Hamano wrote:\n\n> [PATCH] git-blame: mark lines blamed on boundary commits.\n\nExcellent. This is exactly what I had in mind, and it seems to produce\nsensible results out of the box:\n\ngit-diff --raw -r --diff-filter=AM $1 | cut -f2 |\n  while read f; do\n    git-blame -l $1 -- $f | grep -v ^- | cut -d' ' -f1\n  done |\n  sort | uniq -c | sort -rn |\n  while read count hash; do\n    echo \"$count `git-rev-list --max-count=1 --pretty=oneline $hash`\"\n  done\n\nThanks!\n\n"}]}