{"thread":{"id":"13099","subject":"Re: Reporting bugs and bisection","startedAt":"2008-04-13T23:51:34Z","lastAt":"2008-04-17T21:01:55Z","messageCount":66,"participants":["david@lang.hm","Jakub Narebski","Willy Tarreau","Al Viro","Andrew Morton","David Miller","Christoph Hellwig","Andi Kleen","Adrian Bunk","Arjan van de Ven","James Morris","Roman Shaposhnik","Rene Herman","Ilpo Järvinen","Bill Fink","David Newall","Michael Kerrisk","jamal","Rafael J. Wysocki","Sverre Rabbelier","Stephen Clark","Alexey Dobriyan","Jesper Juhl","J. Bruce Fields","Ray Lee"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"74276","messageId":"alpine.DEB.1.10.0804131546370.9318@asgard","threadId":"13099","inReplyTo":"48028830.6020703@earthlink.net","subject":"Re: Reporting bugs and bisection","fromName":"","fromEmail":"david@lang.hm","sentAt":"2008-04-13T23:51:34Z","receivedAt":"2008-04-13T23:51:34Z","isPatch":false,"sender":{"key":"david@lang.hm","avatar":null},"body":"cross-posted to git for the suggestion at the bottom\n\nOn Sun, 13 Apr 2008, Stephen Clark wrote:\n\n> Evgeniy Polyakov wrote:\n>> On Sun, Apr 13, 2008 at 10:33:49PM +0200, Rafael J. Wysocki (rjw@sisk.pl) \n>> wrote:\n>>> Things like this are very disappointing and have a very negative impact on \n>>> bug\n>>> reporters. We should do our best to avoid them.\n>> \n>> Shit happens. This is a matter of either bug report or those who were in\n>> the copy list. There are different people and different situations, in\n>> which they do not reply.\n>> \n> Well less shit would happen if developers would take the time to at least \n> test their patches before they were submitted. It like we will just have the \n> poor user do our testing for us. What kind of testing do developers do. I \n> been a linux user and have followed the LKML for a number of years and have \n> yet to see\n> any test plans for any submitted patches.\n\nI've been reading LKML for 11 years now, I've tested kernels and reported \na few bugs along the way.\n\nthe expectation is that the submitter should have tested the patches \nbefore submitting them (where hardware allows). but that \"where hardware \nallows\" is a big problem. so many issues are dependant on hardwre that \nit's not possible to test everything.\n\nthere are people who download, compile and test the tree nightly (with \nfarms of machines to test different configs), but they can't catch \neverything.\n\nexpecting the patches to be tested to the point where there are no bugs is \nunreasonable.\n\nbisecting is a very powerful tool, but I do think that sometimes \ndevelopers lean on it a bit much. taking the attitude (as some have) that \n'if the reporter can't be bothered to do a bisection I can't be bothered \nto deal with the bug' is going way too far.\n\nif a bug can be reproduced reliably on a test system then bisecting it may \nreveal the patch that introduced or unmasked the bug (assuming that there \naren't other problems along the way), but if the bug takes a long time to \nshow up after a boot, or only happens under production loads, bisecting it \nmay not be possible. that doesn't mean that the bug isn't real, it just \nmeans that the user is going to have to stick with an old version until \nthere is a solution or work-around.\n\neven in the hard-to-test situations, the reporter is usually able to test \na few fixes, but there's a big difference between going to management and \nsaying \"the kernel guru's think that this will help, can we test it this \nweekend\" 2-3 times and doing a bisection that will take 10-15 cycles to \nfind the problem.\n\nit's very reasonable to ask the reporter if they can bisect the problem, \nbut if they say that they can't, declaring that they are out of luck is \nnot reasonable, it just means that it's going to take more thinking to \nfind the problem instead of being able to let the mechanical bisect \nprocess narrow things down for you. it may mean that the developer will \nneed to make a patch to instrament an old (working) kernel that has \nminimal impact on that kernel so that the reporter can run this to gather \ninformation about what the load is so that the developer can try to \nsimulate it on a new (non-working) kernel\n\nin theory everyone has a test environment that lets them simulate \neverything in their production envrionment. in practice this is only true \nat the very low end (where it's easy to do) and the very high end (where \nit's so critical that it's done no matter how much it costs). Everyone \nelse has a test environment that can test most things, but not everything. \nAs such when they run into a problem they may not be able to do lots of \nessentially random testing.\n\nelsewhere in this thread someone said that the pre-git way was to do a \nmanual bisect where the developer would send patches backing out specific \nchanges to find the problem. one big difference between tat and bisecting \nthe problem is that the manual process was focused on the changes in the \narea that is suspected of causing the problem, while the git bisect \nprocess goes after all changes. this makes it much more likely that the \ntester will run into unrelated problems along the way.\n\nI wonder if it would be possible to make a variation of git bisect that \nonly looked at a subset of the tree when picking bisect points (if you are \nlooking for a e1000 bug, testing bisect points that haven't changed that \ndriver won't help you for example). If this can be done it would speed up \nthe reporters efforts, but will require more assistance from the \ndevelopers (who would need to tell the reporters what subtrees to test) so \nit's a tradeoff of efficiancy vs simplicity.\n\nDavid Lang\n"},{"id":"74279","messageId":"m3ve2ls26p.fsf@localhost.localdomain","threadId":"13099","inReplyTo":"alpine.DEB.1.10.0804131546370.9318@asgard","subject":"Re: Reporting bugs and bisection","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-04-14T00:36:38Z","receivedAt":"2008-04-14T00:36:38Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"david@lang.hm writes:\n\n> cross-posted to git for the suggestion at the bottom\n\n[...]\n\n> Elsewhere in this thread someone said that the pre-git way was to do a\n> manual bisect where the developer would send patches backing out\n> specific changes to find the problem. one big difference between that\n> and bisecting the problem is that the manual process was focused on\n> the changes in the area that is suspected of causing the problem,\n> while the git bisect process goes after all changes. this makes it\n> much more likely that the tester will run into unrelated problems\n> along the way.\n> \n> I wonder if it would be possible to make a variation of git bisect\n> that only looked at a subset of the tree when picking bisect points\n> (if you are looking for a e1000 bug, testing bisect points that\n> haven't changed that driver won't help you for example). If this can\n> be done it would speed up the reporters efforts, but will require more\n> assistance from the developers (who would need to tell the reporters\n> what subtrees to test) so it's a tradeoff of efficiancy vs simplicity.\n\nErrr... the synopisis of git-bisect contains the following:\n\n git bisect start [ [...]] [--] [...]\n\nso you can limit bisection to commits affecting specified subsystem.\n\nP.S. Unfortunately git currently doesn't deal with directory renames,\nso if there was sime big code restructuring one has to provide all\nhistoric pathspecs.\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"74304","messageId":"20080414043939.GA6862@1wt.eu","threadId":"13099","inReplyTo":"alpine.DEB.1.10.0804131546370.9318@asgard","subject":"Re: Reporting bugs and bisection","fromName":"Willy Tarreau","fromEmail":"w@1wt.eu","sentAt":"2008-04-14T04:39:39Z","receivedAt":"2008-04-14T04:39:39Z","isPatch":false,"sender":{"key":"w@1wt.eu","avatar":"https://avatars.githubusercontent.com/u/8141789?v=4"},"body":"On Sun, Apr 13, 2008 at 04:51:34PM -0700, david@lang.hm wrote:\n> cross-posted to git for the suggestion at the bottom\n> \n> On Sun, 13 Apr 2008, Stephen Clark wrote:\n> \n> >Evgeniy Polyakov wrote:\n> >>On Sun, Apr 13, 2008 at 10:33:49PM +0200, Rafael J. Wysocki (rjw@sisk.pl) \n> >>wrote:\n> >>>Things like this are very disappointing and have a very negative impact \n> >>>on bug\n> >>>reporters. We should do our best to avoid them.\n> >>\n> >>Shit happens. This is a matter of either bug report or those who were in\n> >>the copy list. There are different people and different situations, in\n> >>which they do not reply.\n> >>\n> >Well less shit would happen if developers would take the time to at least \n> >test their patches before they were submitted. It like we will just have \n> >the poor user do our testing for us. What kind of testing do developers \n> >do. I been a linux user and have followed the LKML for a number of years \n> >and have yet to see\n> >any test plans for any submitted patches.\n> \n> I've been reading LKML for 11 years now, I've tested kernels and reported \n> a few bugs along the way.\n> \n> the expectation is that the submitter should have tested the patches \n> before submitting them (where hardware allows). but that \"where hardware \n> allows\" is a big problem. so many issues are dependant on hardwre that \n> it's not possible to test everything.\n> \n> there are people who download, compile and test the tree nightly (with \n> farms of machines to test different configs), but they can't catch \n> everything.\n> \n> expecting the patches to be tested to the point where there are no bugs is \n> unreasonable.\n[...]\n\nAgreed. The difficulty is that only the developer knows how confident\nhe is in his code. Even the subsystem maintainer does not know, which\nis the real issue since as long as the code is not identified, he does\nnot know whom to ping.\n\nAnd I think that it might help if we could add a \"Trust\" rating to the\npatches we submit, similarly to \"Tested-By\" or \"Signed-off-by\". We could\nuse 1 to 5. Basically, when the patch was completed at 3am and just builds,\nit's more likely 1/5. When it has been stressed for 1 week, it would be\n4/5. 5/5 would only be used in backports of known working code, for some\nwide-used external patches, or for trivial patches (eg: doc/whitespace\nfixes). The goal would clearly not be to just trust patches with a high\nrate (since they might break when associated with others), but for the\nsubsystem maintainer to quickly check if there are some of them the\nauthor does not 100% trust, in which case he could ping the author to\ncheck if his patch *may* cause the reported problem.\n\nWhat makes this rating system delicate is that the rate cannot be changed\nafterwards. But after all, that's not much of a problem. A bug may very\nwell reveal itself one year after the code was merged, so it's really the\ndeveloper's estimation which matters.\n\nFor this to be efficiently used, we would need git-commit to accept a\nnew \"-T \" argument with the following possible values :\n\n 0: untested (default)\n 1: builds\n 2: seems to be working\n 3: passed basic non-regression tests\n 4: survived stress testing at the developer's\n 5: known to be working for a long time somewhere else\n\nI'm sure many people would find this useless (or in fact reject the\nidea because it would show that most code will be rated 1 or 2),\nbut I really think it can help subsystem maintainers make the relation\nbetween a reported bug and a possible submitter.\n\nWilly\n\n"},{"id":"74307","messageId":"20080414053943.GU9785@ZenIV.linux.org.uk","threadId":"13099","inReplyTo":"20080414043939.GA6862@1wt.eu","subject":"Re: Reporting bugs and bisection","fromName":"Al Viro","fromEmail":"viro@zeniv.linux.org.uk","sentAt":"2008-04-14T05:39:43Z","receivedAt":"2008-04-14T05:39:43Z","isPatch":false,"sender":{"key":"viro@zeniv.linux.org.uk","avatar":null},"body":"On Mon, Apr 14, 2008 at 06:39:39AM +0200, Willy Tarreau wrote:\n\n[snip]\n\n> I'm sure many people would find this useless (or in fact reject the\n> idea because it would show that most code will be rated 1 or 2),\n> but I really think it can help subsystem maintainers make the relation\n> between a reported bug and a possible submitter.\n\nI have a related proposal: let us require all patches to be stamped\nwith Discordian *and* Eternal September dates. In triplicate. While\nwe are at it, why don't we introduce new mandatory headers like, say\nit,\n\nX-checkpatch: {Yes,No}\nX-checkpatch-why-not: \nX-pointless: \nX-arbitrary-rules-added-to-CodingStyle: (should be present if\nand only if X-pointless: 69 is present).\n\nCome to think of that, we clearly need a new file in Documentation/*,\ndocumenting such headers. Why don't we organize a subcommittee^Wnew maillist\ndevoted to that? That would provide another entry route for contributors,\nlowering the overall entry barriers even further...\n\n\nSeriously, looks like Andi is right - we've got ourselves a developing\nbeaurocracy. As in \"more and more ways of generating activity without\ndoing anything even remotely useful\". Complete with tendency to operate in\nthe ways that make sense only to beaurocracy in question and an ever-growing\nset of bylaws...\n"},{"id":"74310","messageId":"20080413232441.e216a02c.akpm@linux-foundation.org","threadId":"13099","inReplyTo":"20080414053943.GU9785@ZenIV.linux.org.uk","subject":"Re: Reporting bugs and bisection","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-04-14T06:24:41Z","receivedAt":"2008-04-14T06:24:41Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Mon, 14 Apr 2008 06:39:43 +0100 Al Viro wrote:\n\n> On Mon, Apr 14, 2008 at 06:39:39AM +0200, Willy Tarreau wrote:\n> \n> [snip]\n> \n> > I'm sure many people would find this useless (or in fact reject the\n> > idea because it would show that most code will be rated 1 or 2),\n> > but I really think it can help subsystem maintainers make the relation\n> > between a reported bug and a possible submitter.\n> \n> I have a related proposal: let us require all patches to be stamped\n> with Discordian *and* Eternal September dates. In triplicate. While\n> we are at it, why don't we introduce new mandatory headers like, say\n> it,\n> \n> X-checkpatch: {Yes,No}\n> X-checkpatch-why-not: \n> X-pointless: the way to \"68: aligns right ends of lines in comments\">\n> X-arbitrary-rules-added-to-CodingStyle: (should be present if\n> and only if X-pointless: 69 is present).\n> \n> Come to think of that, we clearly need a new file in Documentation/*,\n> documenting such headers. Why don't we organize a subcommittee^Wnew maillist\n> devoted to that? That would provide another entry route for contributors,\n> lowering the overall entry barriers even further...\n> \n\nNone of the above was particularly useful.\n\n> \n> Seriously, looks like Andi is right - we've got ourselves a developing\n> beaurocracy. As in \"more and more ways of generating activity without\n> doing anything even remotely useful\". Complete with tendency to operate in\n> the ways that make sense only to beaurocracy in question and an ever-growing\n> set of bylaws...\n\nNo. The problem we're discussing here is the apparently-large number of\nbugs which are in the kernel, the apparently-large number of new bugs which\nwe're adding to the kernel, and our apparent tardiness in addressing them.\n\nDo you agree with these impressions, or not?\n\nIf you do agree, what would you propose we do about it?\n"},{"id":"74312","messageId":"20080413.233959.217341225.davem@davemloft.net","threadId":"13099","inReplyTo":"20080413232441.e216a02c.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"David Miller","fromEmail":"davem@davemloft.net","sentAt":"2008-04-14T06:39:59Z","receivedAt":"2008-04-14T06:39:59Z","isPatch":false,"sender":{"key":"davem@davemloft.net","avatar":null},"body":"From: Andrew Morton \nDate: Sun, 13 Apr 2008 23:24:41 -0700\n\n> Do you agree with these impressions, or not?\n\nI think things are improving.\n\nI wrote or merged in ~10 bugs in the last hour, for example.\n\nAnd I also agree with Al's point, which was embedded in his humorous\nand obviously sarcastic suggestions, in that adding beurocracy isn't\nthe answer. We already have too much and it scares developers away.\n\nSure you don't want crap getting into the tree (for too long), but it\nis important to be careful to define crap properly. For example,\ninundating patch submitters with more requirements, especially ones\ninvolving automatons like checkpatch, is in the end bad.\n\nWe can improve the quality of stuff going in and be flexible at the\nsame time.\n"},{"id":"74313","messageId":"20080413.234324.153324549.davem@davemloft.net","threadId":"13099","inReplyTo":"20080413.233959.217341225.davem@davemloft.net","subject":"Re: Reporting bugs and bisection","fromName":"David Miller","fromEmail":"davem@davemloft.net","sentAt":"2008-04-14T06:43:24Z","receivedAt":"2008-04-14T06:43:24Z","isPatch":false,"sender":{"key":"davem@davemloft.net","avatar":null},"body":"From: David Miller \nDate: Sun, 13 Apr 2008 23:39:59 -0700 (PDT)\n\n> I wrote or merged in ~10 bugs in the last hour, for example.\n\nBug fixes! I meant \"fixes\" I swear!\n\nThat's quite a Freudian slip if I ever saw one.\n"},{"id":"74326","messageId":"20080414072328.GW9785@ZenIV.linux.org.uk","threadId":"13099","inReplyTo":"20080413232441.e216a02c.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Al Viro","fromEmail":"viro@zeniv.linux.org.uk","sentAt":"2008-04-14T07:23:28Z","receivedAt":"2008-04-14T07:23:28Z","isPatch":false,"sender":{"key":"viro@zeniv.linux.org.uk","avatar":null},"body":"On Sun, Apr 13, 2008 at 11:24:41PM -0700, Andrew Morton wrote:\n\n> No. The problem we're discussing here is the apparently-large number of\n> bugs which are in the kernel, the apparently-large number of new bugs which\n> we're adding to the kernel, and our apparent tardiness in addressing them.\n> \n> Do you agree with these impressions, or not?\n> \n> If you do agree, what would you propose we do about it?\n\nIn addition to obvious \"we need testing and something better than bugzilla\nto keep track of bugs\"? Real review of code in tree and patches getting into\nthe tree.\n\nAnd the latter part _must_ be done on each entry point. Any git tree\nthat acts as injection point really needs a working mechanism of some\nsort that would do that; afterwards it's too late, since review of\nthe stuff getting into mainline on a massive merge is sadly impractical.\n\nI don't know any formal mechanism that could take care of that; no more\nthan making sure that no backdoors are injected into the tree. It really\nhas to be a matter of trust for tree maintainers and community around\nthe subsystem.\n\nGit is damn good at killing the merge bottleneck. Too good, since it\nhides the review bottleneck. And we get equivalents of self-selected\ncommunities that had been problem for \"here's our CVS, here's monthly\ndump from it, apply\" kind of setups. It _is_ better, since one can\nget to commit history (modulo interesting issues with merge nodes and\nconflict resolution). But in practice it's not good enough - the patches\ngoing in during a merge (especially for a tree that collects from\nsecondaries) are not visible enough. And it's too late at that point,\nsince one has to do something monumentally ugly to get Linus revert\na large merge. On the scale of Great IDE Mess in 2.5...\n\nlinux-next might help with the last part, but I don't think it really\ndeals with the first one. It certainly helps to some extent, but...\n\nWe need higher S/N on l-k. We need people looking into the subsystem\ntrees as those grow and causing a stench when bad things are found,\nwith design issues getting brought to l-k if nothing else helps. We\nneed tree maintainers understanding that review, including out-of-community\none, is needed (the need of testing is generally better understood - I\n_hope_).\n\nWe need more people reading the fscking source. Subsystem by subsystem.\nWithout assumption that code is not broken. With mechanism collating\nthe questions asked and answers given. Ideally we need growing documentation\nof core subsystems and data structures, with explicit goal of helping\nreviewers new to an area to find their way around it. And yes, I'm\nguilty of procrastinating on that - several half-finished pieces on\nVFS-related stuff are sitting locally ;-/\n\nWe need gregkh to get real and stop assuming that two Signed-off-by are\nequivalent to \"reviewed at least twice\", while we are at it ;-)\n\nWe need people to realize that warnings are useful as triage tools -\nnot as \"Ug see warning. Warning bad. Ug fix that line. Warning go away.\nUg changeset count grow. Ug happy.\", but as machine-assisted part of\nfinding confused areas of code. With human combining signals from\ndifferent warnings to get statistically useful triage strategies (note\nthat aforementioned making gcc/sparse/whatnot to STFU by local change\nhas a lovely potential of distorting those signals and actually _hiding_\ncrap code).\n\nMaybe we need a list a-la linux-arch for tree maintainers to coordinate\nstuff - obviously open not only for those.\n\nWe really need to get around to doing triage of remaining stuff in -mm,\nBTW - again, guilty for not getting through such on VFS-related stuff\nin there. Hopefully linux-next trees will eventually vacuum most of the\npile in...\n\nAs for the bug that got this thread started... I'd say that asking to\nbisect was reasonable in this particular case. The following DSW mixed\ninto the thread very soon went the way of all DSW (OK, it hadn't godwinated\nyet, at least in the parts I've seen, so there's still way to go, but...)\n"},{"id":"74329","messageId":"20080414074329.GY9785@ZenIV.linux.org.uk","threadId":"13099","inReplyTo":"20080414072328.GW9785@ZenIV.linux.org.uk","subject":"Re: Reporting bugs and bisection","fromName":"Al Viro","fromEmail":"viro@zeniv.linux.org.uk","sentAt":"2008-04-14T07:43:29Z","receivedAt":"2008-04-14T07:43:29Z","isPatch":false,"sender":{"key":"viro@zeniv.linux.org.uk","avatar":null},"body":"On Mon, Apr 14, 2008 at 08:23:28AM +0100, Al Viro wrote:\n\n> And the latter part _must_ be done on each entry point. Any git tree\n> that acts as injection point really needs a working mechanism of some\n> sort that would do that; afterwards it's too late, since review of\n> the stuff getting into mainline on a massive merge is sadly impractical.\n\nPS: net/* is actually pretty sane in that respect - the huge volume\nbeing what it is, of course, but still, my impression is that it's\npretty far from the worst sources of crap. OTOH, I might be missing\nsecondary tree problems - e.g. net/sctp is much worse off in that\nrespect, AFAICT; there might very well be more of such areas.\n"},{"id":"74335","messageId":"20080414010412.c42dc560.akpm@linux-foundation.org","threadId":"13099","inReplyTo":"20080414072328.GW9785@ZenIV.linux.org.uk","subject":"Re: Reporting bugs and bisection","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-04-14T08:04:12Z","receivedAt":"2008-04-14T08:04:12Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Mon, 14 Apr 2008 08:23:28 +0100 Al Viro wrote:\n\n> On Sun, Apr 13, 2008 at 11:24:41PM -0700, Andrew Morton wrote:\n> \n> > No. The problem we're discussing here is the apparently-large number of\n> > bugs which are in the kernel, the apparently-large number of new bugs which\n> > we're adding to the kernel, and our apparent tardiness in addressing them.\n> > \n> > Do you agree with these impressions, or not?\n> > \n> > If you do agree, what would you propose we do about it?\n> \n> In addition to obvious \"we need testing and something better than bugzilla\n> to keep track of bugs\"?\n\nSwapping out bugzilla for something else wouldn't help. We'd end up with\nlots of people ignoring a good bug tracking system just like they were\nignoring a bad one.\n\n(And I don't think developers and maintainers _should_ spend time mucking\nin bug-tracking systems. They should have helpers who do all the\ntriaging/tracking/routing/closing work for them, and then provide other\ndevelopers with the results, letting them know what they should be spending\ntime on. But there's a manpower problem).\n\n> Real review of code in tree and patches getting into\n> the tree.\n> \n> And the latter part _must_ be done on each entry point. Any git tree\n> that acts as injection point really needs a working mechanism of some\n> sort that would do that; afterwards it's too late, since review of\n> the stuff getting into mainline on a massive merge is sadly impractical.\n> \n> I don't know any formal mechanism that could take care of that; no more\n> than making sure that no backdoors are injected into the tree. It really\n> has to be a matter of trust for tree maintainers and community around\n> the subsystem.\n> \n> Git is damn good at killing the merge bottleneck. Too good, since it\n> hides the review bottleneck. And we get equivalents of self-selected\n> communities that had been problem for \"here's our CVS, here's monthly\n> dump from it, apply\" kind of setups. It _is_ better, since one can\n> get to commit history (modulo interesting issues with merge nodes and\n> conflict resolution). But in practice it's not good enough - the patches\n> going in during a merge (especially for a tree that collects from\n> secondaries) are not visible enough. And it's too late at that point,\n> since one has to do something monumentally ugly to get Linus revert\n> a large merge. On the scale of Great IDE Mess in 2.5...\n> \n> linux-next might help with the last part, but I don't think it really\n> deals with the first one. It certainly helps to some extent, but...\n> \n> We need higher S/N on l-k. We need people looking into the subsystem\n> trees as those grow and causing a stench when bad things are found,\n> with design issues getting brought to l-k if nothing else helps. We\n> need tree maintainers understanding that review, including out-of-community\n> one, is needed (the need of testing is generally better understood - I\n> _hope_).\n> \n> We need more people reading the fscking source. Subsystem by subsystem.\n> Without assumption that code is not broken. With mechanism collating\n> the questions asked and answers given. Ideally we need growing documentation\n> of core subsystems and data structures, with explicit goal of helping\n> reviewers new to an area to find their way around it. And yes, I'm\n> guilty of procrastinating on that - several half-finished pieces on\n> VFS-related stuff are sitting locally ;-/\n> \n> We need gregkh to get real and stop assuming that two Signed-off-by are\n> equivalent to \"reviewed at least twice\", while we are at it ;-)\n> \n> We need people to realize that warnings are useful as triage tools -\n> not as \"Ug see warning. Warning bad. Ug fix that line. Warning go away.\n> Ug changeset count grow. Ug happy.\", but as machine-assisted part of\n> finding confused areas of code. With human combining signals from\n> different warnings to get statistically useful triage strategies (note\n> that aforementioned making gcc/sparse/whatnot to STFU by local change\n> has a lovely potential of distorting those signals and actually _hiding_\n> crap code).\n> \n> Maybe we need a list a-la linux-arch for tree maintainers to coordinate\n> stuff - obviously open not only for those.\n> \n> We really need to get around to doing triage of remaining stuff in -mm,\n> BTW - again, guilty for not getting through such on VFS-related stuff\n> in there. Hopefully linux-next trees will eventually vacuum most of the\n> pile in...\n\nThat all sounds good and I expect few would disagree. But if it is to\nhappen, it clearly won't happen by itself, automatically. We will need to\nforce it upon ourselves and the means by which we will do that is process\nchanges. The thing which is being disparaged as \"bureaucracy\".\n\nThe steps to be taken are:\n\na) agree that we have a problem\n\nb) agree that we need to address it\n\nc) identify the day-to-day work practices which will help address it (as\n you have done)\n\nd) identify the process changes which will force us to adopt those practices\n\ne) implement those process changes.\n\nI have thus far failed to get us past step a).\n"},{"id":"74340","messageId":"20080414.013058.149905948.davem@davemloft.net","threadId":"13099","inReplyTo":"20080414010412.c42dc560.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"David Miller","fromEmail":"davem@davemloft.net","sentAt":"2008-04-14T08:30:58Z","receivedAt":"2008-04-14T08:30:58Z","isPatch":false,"sender":{"key":"davem@davemloft.net","avatar":null},"body":"From: Andrew Morton \nDate: Mon, 14 Apr 2008 01:04:12 -0700\n\n> That all sounds good and I expect few would disagree. But if it is to\n> happen, it clearly won't happen by itself, automatically. We will need to\n> force it upon ourselves and the means by which we will do that is process\n> changes. The thing which is being disparaged as \"bureaucracy\".\n> \n> The steps to be taken are:\n> \n> a) agree that we have a problem\n ...\n> I have thus far failed to get us past step a).\n\nA lot of people, myself included, subconsciously don't want to\nget past step a) because the resulting \"bureaucracy\" or whatever\nyou want to call it is perceived to undercut the very thing\nthat makes the Linux kernel fun to work on.\n\nIt's still largely free form, loose, and flexible. And that's\na notable accomplishment considering how much things have changed.\nThat feeling is why I got involved in the first place, and I know\nit's what gets other new people in and addicted too.\n\nNobody is \"forced\" to do anything, and I notice you used the\nword \"force\" in d) :-)\n\nAnd I realize this relaxed attitude goes hand in hand with reduced\nquality and occaisionally more bugs. In many ways, I'm happy with\nthat tradeoff at least wrt. how that works out for the subsystems\nI'm responsible for.\n\nWe can ask more subsystem tree maintainers to run their trees more\nstrictly, review patches more closely, etc. But, be honest, good luck\ngetting that from the guys who do subsystem maintainence in their\nspare time on the weekends. The remaining cases should know better,\nor simply don't care.\n"},{"id":"74341","messageId":"20080414090634.GB15541@infradead.org","threadId":"13099","inReplyTo":"20080414.013058.149905948.davem@davemloft.net","subject":"Re: Reporting bugs and bisection","fromName":"Christoph Hellwig","fromEmail":"hch@infradead.org","sentAt":"2008-04-14T09:06:34Z","receivedAt":"2008-04-14T09:06:34Z","isPatch":false,"sender":{"key":"hch@infradead.org","avatar":null},"body":"On Mon, Apr 14, 2008 at 01:30:58AM -0700, David Miller wrote:\n> We can ask more subsystem tree maintainers to run their trees more\n> strictly, review patches more closely, etc. But, be honest, good luck\n> getting that from the guys who do subsystem maintainence in their\n> spare time on the weekends. The remaining cases should know better,\n> or simply don't care.\n\nActually my impression is that spare-time maitainer produce much better\ncode and subsystem trees than corporate-drones. But of course there's\na lot of shades between those two extremes.\n"},{"id":"74342","messageId":"878wzgwyyw.fsf@basil.nowhere.org","threadId":"13099","inReplyTo":"20080414.013058.149905948.davem@davemloft.net","subject":"Re: Reporting bugs and bisection","fromName":"Andi Kleen","fromEmail":"andi@firstfloor.org","sentAt":"2008-04-14T09:46:31Z","receivedAt":"2008-04-14T09:46:31Z","isPatch":false,"sender":{"key":"andi@firstfloor.org","avatar":null},"body":"David Miller writes:\n>\n> It's still largely free form, loose, and flexible. \n\nI think Al's point was that we need far more \"free form, loose and\nflexible\" work for reviewing code. As in people going over trees and\njust checking it for anything suspicious and going over existing code\nand checking it for anything suspicious and going also over mailing\nlist patch posts. And also maintainers who appreciate such review.\n\nAnd checking it for anything suspicious does not mean running\nonly checkpatch.pl or even just sparse, but actually reading it\nand trying to make sense of it.\n\nI don't see that really as conflicting with your goals.\n\nIt would be some more work for the maintainers to handle more such\nfeedback because they would need to process comments from such \"free\nform reviewers\". Some of them will undoutedly be wrong and that will\ntake some time away from processing features (and bugs) but I suspect\nit would be still worth it.\n\nOn the other hand it would also take some work away from\nprocessing bugs, but as Andrew mentions earlier it looks\nlike significant parts of the boring areas of bug reports \n(like getting basic information from reporter etc.) \ncould be \"out-sourced\" to bug masters. \n\nAnd I think being a bug master is an excellent way for someone who isn't\na great coder to contribute in excellent ways to Linux\n(far more than someone e.g. running checkpatch.pl ever could) \n\nThe challenging thing is also to make sure that the quality of\ncomments stays high. That means more focus on logic and functionality\nthan on form. If the reviewer just goes over the coding style or\ntrivialities I don't think that will improve Linux really. I think the\nproblem is often that people think kernel code must be very\ncomplicated and they don't even dare try to understand it. But\nfrankly a lot of the kernel code is not really that complicated logic\nwise and also doesn't need too specialized knowledge to understand.\nSo I am optimistic that there are a lot of people out there who would\nbe qualified to do some logic review.\n\nReally Linux needs a better \"reviewing culture\" and also\na better \"bug processing culture\"\n\n> We can ask more subsystem tree maintainers to run their trees more\n> strictly, review patches more closely, etc. But, be honest, good luck\n> getting that from the guys who do subsystem maintainence in their\n> spare time on the weekends. The remaining cases should know better,\n> or simply don't care.\n\nIn my experience weekend maintainers tend to be better at sharing\nout work. As in they usually (ok there are exceptions) more work\nincluding review work on the mailing lists, while my impression\nis that paid for maintainers tend to have tendency for more \ncentralized \"cathedral\" tree maintenance. That is with them trying to \nkeep everything under control and effectively much more stuff going on the \nbackground out of public view. But the sharing out of work and less\ncentralization is what we really want here I think.\n\nAnyways I'm not saying all paid-for maintainers are like this, but\nthere is certainly a trend I think.\n\nI admit I personally went through both phases in several projects.\n\nWhen you're really focussed on something it is tempting to do \nthe \"keep things under control\" central model, but in the end\nit is the wrong way to go.\n\n-Andi\n"},{"id":"74343","messageId":"20080414031530.2507660d.akpm@linux-foundation.org","threadId":"13099","inReplyTo":"20080414.013058.149905948.davem@davemloft.net","subject":"Re: Reporting bugs and bisection","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-04-14T10:15:30Z","receivedAt":"2008-04-14T10:15:30Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Mon, 14 Apr 2008 01:30:58 -0700 (PDT) David Miller wrote:\n\n> From: Andrew Morton \n> Date: Mon, 14 Apr 2008 01:04:12 -0700\n> \n> > That all sounds good and I expect few would disagree. But if it is to\n> > happen, it clearly won't happen by itself, automatically. We will need to\n> > force it upon ourselves and the means by which we will do that is process\n> > changes. The thing which is being disparaged as \"bureaucracy\".\n> > \n> > The steps to be taken are:\n> > \n> > a) agree that we have a problem\n> ...\n> > I have thus far failed to get us past step a).\n> \n> A lot of people, myself included, subconsciously don't want to\n> get past step a) because the resulting \"bureaucracy\" or whatever\n> you want to call it is perceived to undercut the very thing\n> that makes the Linux kernel fun to work on.\n> \n> It's still largely free form, loose, and flexible. And that's\n> a notable accomplishment considering how much things have changed.\n> That feeling is why I got involved in the first place, and I know\n> it's what gets other new people in and addicted too.\n> \n> Nobody is \"forced\" to do anything, and I notice you used the\n> word \"force\" in d) :-)\n\nOK, I was going to let this pass, but I changed my mind.\n\nYou carefully deleted my text so that you could misquote it, thereby\nflagrantly misrepresenting everything I said.\n\nHere it is again:\n\n: The steps to be taken are:\n: \n: a) agree that we have a problem\n: \n: b) agree that we need to address it\n: \n: c) identify the day-to-day work practices which will help address it (as\n: you have done)\n: \n: d) identify the process changes which will force us to adopt those practices\n: \n: e) implement those process changes.\n\nForcing a discipline upon oneself is totally different from having it\nforced upon you by someone else.\n\nEach step will need general agreement and buyin, otherwise none of it will\n(or should) work.\n\n"},{"id":"74344","messageId":"20080414.034116.24468363.davem@davemloft.net","threadId":"13099","inReplyTo":"20080414031530.2507660d.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"David Miller","fromEmail":"davem@davemloft.net","sentAt":"2008-04-14T10:41:16Z","receivedAt":"2008-04-14T10:41:16Z","isPatch":false,"sender":{"key":"davem@davemloft.net","avatar":null},"body":"From: Andrew Morton \nDate: Mon, 14 Apr 2008 03:15:30 -0700\n\n> You carefully deleted my text so that you could misquote it, thereby\n> flagrantly misrepresenting everything I said.\n\nNot the intention, but anyways:\n\n> Here it is again:\n> \n> : The steps to be taken are:\n> : \n> : a) agree that we have a problem\n> : \n> : b) agree that we need to address it\n> : \n> : c) identify the day-to-day work practices which will help address it (as\n> : you have done)\n> : \n> : d) identify the process changes which will force us to adopt those practices\n> : \n> : e) implement those process changes.\n> \n> Forcing a discipline upon oneself is totally different from having it\n> forced upon you by someone else.\n> \n> Each step will need general agreement and buyin, otherwise none of it will\n> (or should) work.\n\nThe \"force\" is to \"us\" which is a group.\n\nAnd I imagine that newcomers will be expected to adopt these\n\"practices\". So in effect, they will be \"forced\" into the process\nchanges as well.\n\nI'm getting more and more sensitive to issues on this level over time,\nbecause I realize that the fundamental issue in all human group issues\nis getting people to \"want\" to do things. And \"force\", in any form,\ntends to be incompatible with \"want\". And in particular, people will\noften even shun things they \"want\" when it is \"forced\" to them.\n"},{"id":"74348","messageId":"20080414120821.GA4625@cs181133002.pp.htv.fi","threadId":"13099","inReplyTo":"20080414010412.c42dc560.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Adrian Bunk","fromEmail":"bunk@kernel.org","sentAt":"2008-04-14T12:08:21Z","receivedAt":"2008-04-14T12:08:21Z","isPatch":false,"sender":{"key":"bunk@kernel.org","avatar":null},"body":"On Mon, Apr 14, 2008 at 01:04:12AM -0700, Andrew Morton wrote:\n>...\n> (And I don't think developers and maintainers _should_ spend time mucking\n> in bug-tracking systems. They should have helpers who do all the\n> triaging/tracking/routing/closing work for them, and then provide other\n> developers with the results, letting them know what they should be spending\n> time on. But there's a manpower problem).\n>...\n\nSpeaking as the one who was for a few years going again and again \nthrough all open bugs in the kernel Bugzilla:\n\nThe manpower problem isn't in handling the bugs in Bugzilla.\n\nI'd claim that even if all bugs in the kernel would be reported in the \nkernel Bugzilla I alone would be able to do all the handling of incoming \nbugs, bug forwarding and doing all the cleanup stuff like asking \nsubmitters whether a bug is still present in the latest kernel.\n\nThe manpower problem is at the developers and maintainers who could \nactually debug the problems.\n\nOne problem are unmaintained areas.\nDo we have anyone who would debug e.g. APM bugs?\nAnd if I want to be really nasty, I'll ask whether we have anyone who \nunderstands our floppy driver... ;)\n\nAnd who would debug problems with old and unmaintained drivers, e.g. \nsome old net or SCSI driver?\n\nNote that I do not blame James or Jeff or whoever else for the latter - \nthey might simply not have the time to spend a day or two for debugging \nsome obscure problem on some obscure hardware.\n\nAnd it could happen everywhere that maintainers simply don't have \nthe time to cope with all incoming bug reports.\n\nWe have many people who write new bugs^Wcode.\nBut too few people who review code.\nAnd too few people willing to maintain the existing code.\n\ncu\nAdrian\n\n-- \n\n \"Is there not promise of rain?\" Ling Tan asked suddenly out\n of the darkness. There had been need of rain for many days.\n \"Only a promise,\" Lao Er said.\n Pearl S. Buck - Dragon Seed\n\n"},{"id":"74355","messageId":"20080414074349.24fa90f8@laptopd505.fenrus.org","threadId":"13099","inReplyTo":"20080414010412.c42dc560.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Arjan van de Ven","fromEmail":"arjan@infradead.org","sentAt":"2008-04-14T14:43:49Z","receivedAt":"2008-04-14T14:43:49Z","isPatch":false,"sender":{"key":"arjan@infradead.org","avatar":"https://gravatar.com/avatar/42631f2338fc04830b31b08bc42b6575a3547fba469bd73ba7e1f237d650986a?d=mp&s=160"},"body":"On Mon, 14 Apr 2008 01:04:12 -0700\n> \n> The steps to be taken are:\n> \n> a) agree that we have a problem\n> \n\n\nI for one do not agree that we have a problem.\n\nBased on actual data on oopses (which very clearly excludes other kinds of bugs, so I know I only see part of the story)\nwe are doing reasonably well. Lets look at the 2.6.25 cycle. \nWe got a total of roughly 2700 reports of oopses/warn_ons from users. (This may sound high to those of you only reading\nlkml, but this includes automatically collected oopses from Fedora 9 beta testers).\nOut of these 2700, the top 20 issues account for 75% of the total reports.\n\nOut of these 20 issues, 9 were from still out of tree drivers (wireless.git and drm.git included in F9). These were\ncaught before they even got close to mainline.\nThe remaining 11 issues can be split in\n1) The ones we caught and fixed\n2) TCP/IP warnings that DaveM and co are chasing down hard (but have trouble finding reproducers)\n3) An EXT3 bug that in theory can cause data corruption, but in practice seems to happen after you yank out a USB stick\n with an EXT3 filesystem on (so it can't corrupt the disk data). Ted is working on this\n4) A bug (double free) that hits in the skb layer, probably caused by a bug in the ipv4 code\n (a first analysis + potential patch was mailed to netdev this weekend)\n5) sysfs \"existing file added\" warning, mostly in the USB stack\n (gregkh claims he fixed this recently, I'm not entirely sure he got all cases)\n\nAnd when I look beyond the first 20, the same pattern arises, we fixed the majority of the issues before -rc9.\nAt position 25 we have less than 20 reports per bug. At position 35 we have less than 10 reports per bug. \nAt position 50 we have less than 5 reports per bug. Conclusion there: the bugs people actually hit fall of dramatically;\nthere's a core set of issues that gets hit a lot, the rest quickly gets reduced to noise levels.\n\n\nTo me this does not sound like we have a huge quality problem because\n1) The distribution of the bugs is such that there is a relatively small set of core issues\n that are widely hit, and then there's a near exponential drop after that\n2) We are fixing the important bugs by and large before they hit a release\n (important as defined by the number of people actually hitting the bug)\n\n\n \nI'll be writing a report with more details about this soon with more analysis and statistics\n(I'll be looking at more detail around the top 25 issues, when they got introduced, when they got fixed etc)\n\n\n\n\n\n\n\n-- \nIf you want to reach me at my work email, use arjan@linux.intel.com\nFor development, discussion and tips for power savings, \nvisit http://www.lesswatts.org\n"},{"id":"74361","messageId":"Xine.LNX.4.64.0804150131300.4160@us.intercode.com.au","threadId":"13099","inReplyTo":"20080414072328.GW9785@ZenIV.linux.org.uk","subject":"Re: Reporting bugs and bisection","fromName":"James Morris","fromEmail":"jmorris@namei.org","sentAt":"2008-04-14T15:54:00Z","receivedAt":"2008-04-14T15:54:00Z","isPatch":false,"sender":{"key":"jmorris@namei.org","avatar":null},"body":"On Mon, 14 Apr 2008, Al Viro wrote:\n\n> Real review of code in tree and patches getting into the tree.\n\nThere is currently little incentive for developers to perform review. \n\nIt's difficult work, and is generally not rewarded or recognized, except \nin often quite negative ways. There is a small handful of people who do a \nlot of review, but they are exceptional in various ways.\n\nOTOH, writing code is relatively simple, and is much more highly rewarded:\n\n- People tend to get paid to write kernel code, but not so much to review \n it.\n\n- Things like \"who made the kernel\" statistics and related articles ignore \n code review.\n\n- Creating new features is perceived as the highest form of contribution \n for general developers, and likely important as career currency \n (similar to the publish or perish model in the academic world).\n\nI don't know how to solve this, but suspect that encouraging the use of \nreviewed-by and also including it in things like analysis of who is \ncontributing, selection for kernel summit invitations etc. would be a \nstart. At least, better than nothing.\n\n\n- James \n-- \nJames Morris\n\n"},{"id":"74372","messageId":"1208194521.25663.12.camel@work.sfbay.sun.com","threadId":"13099","inReplyTo":"20080414.034116.24468363.davem@davemloft.net","subject":"Re: Reporting bugs and bisection","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-14T17:35:21Z","receivedAt":"2008-04-14T17:35:21Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Mon, 2008-04-14 at 03:41 -0700, David Miller wrote:\n> I'm getting more and more sensitive to issues on this level over time,\n> because I realize that the fundamental issue in all human group issues\n> is getting people to \"want\" to do things. And \"force\", in any form,\n> tends to be incompatible with \"want\". And in particular, people will\n> often even shun things they \"want\" when it is \"forced\" to them.\n\nJust wanted to add my 2c by mentioning my favorite example of \n\"virtual Tom Sawyering\" as far as a tedious review process goes:\n http://en.wikipedia.org/wiki/Knuth_reward_check\n\nWhich is also quite cheap too -- AFAIK very few of those have ever\nbeen cashed.\n\nThanks,\nRoman.\n"},{"id":"74367","messageId":"20080414105152.9cc06fab.akpm@linux-foundation.org","threadId":"13099","inReplyTo":"20080414074349.24fa90f8@laptopd505.fenrus.org","subject":"Re: Reporting bugs and bisection","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-04-14T17:51:52Z","receivedAt":"2008-04-14T17:51:52Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Mon, 14 Apr 2008 07:43:49 -0700 Arjan van de Ven wrote:\n\n> On Mon, 14 Apr 2008 01:04:12 -0700\n> > \n> > The steps to be taken are:\n> > \n> > a) agree that we have a problem\n> > \n> \n> \n> I for one do not agree that we have a problem.\n> \n> Based on actual data on oopses (which very clearly excludes other kinds of bugs, so I know I only see part of the story)\n> we are doing reasonably well. Lets look at the 2.6.25 cycle. \n> We got a total of roughly 2700 reports of oopses/warn_ons from users. (This may sound high to those of you only reading\n> lkml, but this includes automatically collected oopses from Fedora 9 beta testers).\n> Out of these 2700, the top 20 issues account for 75% of the total reports.\n> \n> Out of these 20 issues, 9 were from still out of tree drivers (wireless.git and drm.git included in F9). These were\n> caught before they even got close to mainline.\n> The remaining 11 issues can be split in\n> 1) The ones we caught and fixed\n> 2) TCP/IP warnings that DaveM and co are chasing down hard (but have trouble finding reproducers)\n> 3) An EXT3 bug that in theory can cause data corruption, but in practice seems to happen after you yank out a USB stick\n> with an EXT3 filesystem on (so it can't corrupt the disk data). Ted is working on this\n> 4) A bug (double free) that hits in the skb layer, probably caused by a bug in the ipv4 code\n> (a first analysis + potential patch was mailed to netdev this weekend)\n> 5) sysfs \"existing file added\" warning, mostly in the USB stack\n> (gregkh claims he fixed this recently, I'm not entirely sure he got all cases)\n> \n> And when I look beyond the first 20, the same pattern arises, we fixed the majority of the issues before -rc9.\n> At position 25 we have less than 20 reports per bug. At position 35 we have less than 10 reports per bug. \n> At position 50 we have less than 5 reports per bug. Conclusion there: the bugs people actually hit fall of dramatically;\n> there's a core set of issues that gets hit a lot, the rest quickly gets reduced to noise levels.\n> \n> \n> To me this does not sound like we have a huge quality problem because\n> 1) The distribution of the bugs is such that there is a relatively small set of core issues\n> that are widely hit, and then there's a near exponential drop after that\n> 2) We are fixing the important bugs by and large before they hit a release\n> (important as defined by the number of people actually hitting the bug)\n> \n> \n> \n> I'll be writing a report with more details about this soon with more analysis and statistics\n> (I'll be looking at more detail around the top 25 issues, when they got introduced, when they got fixed etc)\n\nWell OK. But I don't think we can generalise from oops-causing bugs all\nthe way to all bugs. Very few bugs actually cause oopses, and oopses tend\nto be the thing which developers will zoom in on and pay attention to.\n\nIf we had metrics on \"time goes backwards\" or anything containing \"ASUS\",\nthings might be different.\n"},{"id":"74371","messageId":"20080414112444.2504b48d@laptopd505.fenrus.org","threadId":"13099","inReplyTo":"20080414105152.9cc06fab.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Arjan van de Ven","fromEmail":"arjan@infradead.org","sentAt":"2008-04-14T18:24:44Z","receivedAt":"2008-04-14T18:24:44Z","isPatch":false,"sender":{"key":"arjan@infradead.org","avatar":"https://gravatar.com/avatar/42631f2338fc04830b31b08bc42b6575a3547fba469bd73ba7e1f237d650986a?d=mp&s=160"},"body":"On Mon, 14 Apr 2008 10:51:52 -0700\nAndrew Morton wrote:\n\n\n> Well OK. But I don't think we can generalise from oops-causing bugs\n\nincluding all WARN_ON's and various other kernel backtrace-causing bugs.\n\n\n> all the way to all bugs. Very few bugs actually cause oopses, and\n> oopses tend to be the thing which developers will zoom in on and pay\n> attention to.\n\nmaybe.\n> \n> If we had metrics on \"time goes backwards\" or anything containing\n> \"ASUS\", things might be different.\n\nSounds really like we need to add more strategic WARN_ON's and other diagnostics in \nthe kernel to track these issues down.\n\n\nBecause another thing that I found so far is that what hits LKML is by far not representative\non what happens for users. The most obvious example was the whole input layer refcounting disaster\nin 2.6.25-rc; this was about 1/3rd of TOTAL reports for a few weeks in a row, but there\nwas hardly an LKML posting for it (in fact there was only 1 half one).\nWe need diagnostics and stuff the kernel spits out so that automated tools can detect these,\notherwise we'll very likely not get good information on what is actually wrong with the kernel.\n\n\nIn case you want to see the 2.6.25-rc data, the top 100 list is at\nhttp://www.kerneloops.org/twentyfive.html\n\n(I'm still working on annotating the individual items, but since there's 100\nthat does take time)\n\n-- \nIf you want to reach me at my work email, use arjan@linux.intel.com\nFor development, discussion and tips for power savings, \nvisit http://www.lesswatts.org\n"},{"id":"74373","messageId":"4803ACE5.4020502@keyaccess.nl","threadId":"13099","inReplyTo":"20080413232441.e216a02c.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Rene Herman","fromEmail":"rene.herman@keyaccess.nl","sentAt":"2008-04-14T19:13:41Z","receivedAt":"2008-04-14T19:13:41Z","isPatch":false,"sender":{"key":"rene.herman@keyaccess.nl","avatar":null},"body":"On 14-04-08 08:24, Andrew Morton wrote:\n\n> On Mon, 14 Apr 2008 06:39:43 +0100 Al Viro wrote:\n\n>> I have a related proposal: let us require all patches to be stamped\n>> with Discordian *and* Eternal September dates. In triplicate. While\n>> we are at it, why don't we introduce new mandatory headers like, say\n>> it,\n>>\n>> X-checkpatch: {Yes,No}\n>> X-checkpatch-why-not: \n>> X-pointless: > the way to \"68: aligns right ends of lines in comments\">\n>> X-arbitrary-rules-added-to-CodingStyle: (should be present if\n>> and only if X-pointless: 69 is present).\n>>\n>> Come to think of that, we clearly need a new file in Documentation/*,\n>> documenting such headers. Why don't we organize a subcommittee^Wnew maillist\n>> devoted to that? That would provide another entry route for contributors,\n>> lowering the overall entry barriers even further...\n>>\n> \n> None of the above was particularly useful.\n\nDoes that mean you're not going to take patches that align the right end of \nlines in comments? :-(\n\nRene.\n"},{"id":"74374","messageId":"Pine.LNX.4.64.0804142221480.7090@wrl-59.cs.helsinki.fi","threadId":"13099","inReplyTo":"20080414105152.9cc06fab.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Ilpo Järvinen","fromEmail":"ilpo.jarvinen@helsinki.fi","sentAt":"2008-04-14T19:30:55Z","receivedAt":"2008-04-14T19:30:55Z","isPatch":false,"sender":{"key":"ilpo.jarvinen@helsinki.fi","avatar":null},"body":"On Mon, 14 Apr 2008, Andrew Morton wrote:\n\n> On Mon, 14 Apr 2008 07:43:49 -0700 Arjan van de Ven wrote:\n> \n> > I'll be writing a report with more details about this soon with more analysis and statistics\n> > (I'll be looking at more detail around the top 25 issues, when they got introduced, when they got fixed etc)\n> \n> Well OK. But I don't think we can generalise from oops-causing bugs all\n> the way to all bugs. Very few bugs actually cause oopses, and oopses tend\n> to be the thing which developers will zoom in on and pay attention to.\n> \n> If we had metrics on \"time goes backwards\" or anything containing \"ASUS\",\n> things might be different.\n\nEven oopses have pitfalls, like in 25-rcs where those WARN_ON TCP \nbacktraces were due to three different bugs (there might be fourth one \nstill remaining). ...kerneloops.org didn't even make difference between \ndifferent WARN_ONs in a function though that would have helped only little \nin the case of 25-rc TCP because of different bugs causing failures in the \nsame invariant.\n\n-- \n i.\n"},{"id":"74379","messageId":"20080414133802.4535e4da.akpm@linux-foundation.org","threadId":"13099","inReplyTo":"4803ACE5.4020502@keyaccess.nl","subject":"Re: Reporting bugs and bisection","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-04-14T20:38:02Z","receivedAt":"2008-04-14T20:38:02Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Mon, 14 Apr 2008 21:13:41 +0200\nRene Herman wrote:\n\n> Does that mean you're not going to take patches that align the right end of \n> lines in comments? :-(\n\nerm, was that \":-(\" supposed to be a \":-)\"?\n\nI don't like to merge patches which fix typos and spellos and grammaros\nin comments, simply because I'd be buried in the things. I do take such\nfixes for user-visible text (Documentation/, kerneldoc comments and\nprintks).\n\nRight-justification of comments would fall rather a long way below spelling\nfixes.\n\n"},{"id":"74383","messageId":"20080414.150105.101568769.davem@davemloft.net","threadId":"13099","inReplyTo":"Xine.LNX.4.64.0804150131300.4160@us.intercode.com.au","subject":"Re: Reporting bugs and bisection","fromName":"David Miller","fromEmail":"davem@davemloft.net","sentAt":"2008-04-14T22:01:05Z","receivedAt":"2008-04-14T22:01:05Z","isPatch":false,"sender":{"key":"davem@davemloft.net","avatar":null},"body":"From: James Morris \nDate: Tue, 15 Apr 2008 01:54:00 +1000 (EST)\n\n> - Things like \"who made the kernel\" statistics and related articles ignore \n> code review.\n\nNote the apparent irony in that the person who ends up often on the\ntop of those lists, Al Viro, is also someone who also does a\nsignificant amount of code review.\n\nI think this is no accident.\n"},{"id":"74384","messageId":"4803D830.7000206@keyaccess.nl","threadId":"13099","inReplyTo":"20080414133802.4535e4da.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Rene Herman","fromEmail":"rene.herman@keyaccess.nl","sentAt":"2008-04-14T22:18:24Z","receivedAt":"2008-04-14T22:18:24Z","isPatch":false,"sender":{"key":"rene.herman@keyaccess.nl","avatar":null},"body":"On 14-04-08 22:38, Andrew Morton wrote:\n\n> On Mon, 14 Apr 2008 21:13:41 +0200\n> Rene Herman wrote:\n> \n>> Does that mean you're not going to take patches that align the right end of \n>> lines in comments? :-(\n> \n> erm, was that \":-(\" supposed to be a \":-)\"?\n\nThe \":-(\" was supposed to add to the implicitly obvious \":-)\". That is, was \nindeed joking (Al mentioned them) but with a slightly serious undertone:\n\n> I don't like to merge patches which fix typos and spellos and grammaros \n> in comments, simply because I'd be buried in the things. I do take such \n> fixes for user-visible text (Documentation/, kerneldoc comments and \n> printks).\n> \n> Right-justification of comments would fall rather a long way below\n> spelling fixes.\n\nYou, particularly, seem to be very good at picking up trivia. I've posted \ncompletely trivial patches from time to time for small things I encounter \nwhile looking at something else. Things at the \"are people going to look \nfunny at me for even bothering or...\" level but you picking them up means \nit's still useful to post, so I sometimes do.\n\nNow, in fact, Linux as a _whole_ doesn't seem bad at accepting that kind of \nsmall janitorial stuff but I have been noticing some backlash to it as well. \nI'm not sure it's worse or better than historically, but the \"checkpatch \nsyndrome\" certainly triggers more of it.\n\nAl specifically wanted more new eyes but the way to reward those new eyes is \naccepting their small changes. Al also specifically doesn't like those small \nchanges when at the level of the automated and semi-brainless checkpatch level.\n\nI believe the janitorial work has been over-organized, both through the \nkernel-janitors and checkpatch since while these are very useful in guiding \na newbie in _what_ to do they cause \"automated\" huge tree-wide trivia storms \nwhich people then don't react overly favourable to and the new eyes who did \nall that work of generating it all dim again...\n\nFrankly, the kernel really is fairly complex these days when starting at 0. \nMuch more complex certainly than, say, back in 2.0 or 2.2 days and while \nAl's scenario of per-subsystem reviews might be good, I don't believe it's \nvery realistic. Companies don't pay to have those done and for newbies it's \ngenerally too complex since understanding most parts of the kernel fully, \nrequires understanding most of the rest kernel rather well also.\n\nSo you get the really promising newbies? Yeah, that, or you don't get anyone \nand if some promising newbies are building up 137 part checkpatch inspired \npatchsets that don't help none.\n\nSo, what am I saying (what _am_ I saying?!?) ...\n\nI seemed to observe somewhat of an internal contradiction in Al's message \nabout new eyes and his dislike of the trivial stuff but the contradiction \nonly exists if the dislike wouldn't be limited to these kinds of huge trivia \nstorms. I believe it is, and I furthermore believe that yes, it's \nover-organization that causes many new eyes to focus on the brainless aspects.\n\nNow, do those new eyes have many other options when very few (to none) of \nthe core crowd ever does things like answer question on the kernelnewbies \nlist? From the established names, I only remember ever seeing Greg KH and \nAdrian Bunk there. And I'm _still_ pissed that noone would or could tell me \nwhat was wrong with the legacy CD-ROM driver I and Pekka Enberg were toying \naround with a while ago. Frankly, I care a whole lot less about a hundred \nsparse warning fixes.\n\nIn short -- the kernel in it's current state is already quite complex and if \nnew eyes are wanted they'll need to be coached more. I'm seeing very little \nof that.\n\nRene.\n"},{"id":"74385","messageId":"20080414160513.9f57e5ba.akpm@linux-foundation.org","threadId":"13099","inReplyTo":"20080414.150105.101568769.davem@davemloft.net","subject":"Re: Reporting bugs and bisection","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-04-14T23:05:13Z","receivedAt":"2008-04-14T23:05:13Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Mon, 14 Apr 2008 15:01:05 -0700 (PDT)\nDavid Miller wrote:\n\n> From: James Morris \n> Date: Tue, 15 Apr 2008 01:54:00 +1000 (EST)\n> \n> > - Things like \"who made the kernel\" statistics and related articles ignore \n> > code review.\n> \n> Note the apparent irony in that the person who ends up often on the\n> top of those lists, Al Viro, is also someone who also does a\n> significant amount of code review.\n> \n> I think this is no accident.\n\n\"who made the kernel\" was an interesting and useful exercise, but if you\nlike irony then...\n\n- The way to boost your commit count is to submit buggy patches and to\n then fix your own bugs.\n\n- The way to lower your commit count is to fix things in other people's\n patches, then fold your fix into the base patch. I've lost over 1000\n commits that way. Unless they are counting '^ [akpm' as a commit.\n"},{"id":"74414","messageId":"20080415045541.GA611@1wt.eu","threadId":"13099","inReplyTo":"20080414160513.9f57e5ba.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Willy Tarreau","fromEmail":"w@1wt.eu","sentAt":"2008-04-15T04:55:42Z","receivedAt":"2008-04-15T04:55:42Z","isPatch":false,"sender":{"key":"w@1wt.eu","avatar":"https://avatars.githubusercontent.com/u/8141789?v=4"},"body":"On Mon, Apr 14, 2008 at 04:05:13PM -0700, Andrew Morton wrote:\n> On Mon, 14 Apr 2008 15:01:05 -0700 (PDT)\n> David Miller wrote:\n> \n> > From: James Morris \n> > Date: Tue, 15 Apr 2008 01:54:00 +1000 (EST)\n> > \n> > > - Things like \"who made the kernel\" statistics and related articles ignore \n> > > code review.\n> > \n> > Note the apparent irony in that the person who ends up often on the\n> > top of those lists, Al Viro, is also someone who also does a\n> > significant amount of code review.\n> > \n> > I think this is no accident.\n> \n> \"who made the kernel\" was an interesting and useful exercise, but if you\n> like irony then...\n> \n> - The way to boost your commit count is to submit buggy patches and to\n> then fix your own bugs.\n> \n> - The way to lower your commit count is to fix things in other people's\n> patches, then fold your fix into the base patch. I've lost over 1000\n> commits that way. Unless they are counting '^ [akpm' as a commit.\n\nAnd if Dave speaks about these stats : http://lwn.net/Articles/237768/\nthen Al does not even appear in it, which proves your point.\n\nWilly\n"},{"id":"74415","messageId":"20080415012533.832db7ed.billfink@mindspring.com","threadId":"13099","inReplyTo":"878wzgwyyw.fsf@basil.nowhere.org","subject":"Re: Reporting bugs and bisection","fromName":"Bill Fink","fromEmail":"billfink@mindspring.com","sentAt":"2008-04-15T05:25:33Z","receivedAt":"2008-04-15T05:25:33Z","isPatch":false,"sender":{"key":"billfink@mindspring.com","avatar":null},"body":"On Mon, 14 Apr 2008, Andi Kleen wrote:\n\n> David Miller writes:\n> >\n> > It's still largely free form, loose, and flexible. \n> \n> I think Al's point was that we need far more \"free form, loose and\n> flexible\" work for reviewing code. As in people going over trees and\n> just checking it for anything suspicious and going over existing code\n> and checking it for anything suspicious and going also over mailing\n> list patch posts. And also maintainers who appreciate such review.\n> \n> And checking it for anything suspicious does not mean running\n> only checkpatch.pl or even just sparse, but actually reading it\n> and trying to make sense of it.\n\nIf you really want to get more such review, then it would be very\nuseful when someone asks about some obtuse portion of kernel code\nor makes a suggested improvement, that the reviewer then not be\nflamed as being dense for not understanding the code or some kernel\ncoding concept. It would be much better to treat it as an oppurtunity\nto educate rather than belittle, thus eventually enlarging the base\nof people who can assist with various aspects of kernel development.\nFor what's supposed to be an open, engaging community, and which\ngenerally is, there sometimes seems to be some level of dismissal\nof newcomers (not sure it's intended that way but nevertheless it\ncan tend to discourage newcomers from getting more involved).\n\n\t\t\t\t\t\t-Bill\n"},{"id":"74425","messageId":"4804765B.2070300@davidnewall.com","threadId":"13099","inReplyTo":"Xine.LNX.4.64.0804150131300.4160@us.intercode.com.au","subject":"Re: Reporting bugs and bisection","fromName":"David Newall","fromEmail":"davidn@davidnewall.com","sentAt":"2008-04-15T09:33:15Z","receivedAt":"2008-04-15T09:33:15Z","isPatch":false,"sender":{"key":"davidn@davidnewall.com","avatar":null},"body":"James Morris wrote:\n> I don't know how to solve this, but suspect that encouraging the use of \n> reviewed-by and also including it in things like analysis of who is \n> contributing, selection for kernel summit invitations etc. would be a \n> start. At least, better than nothing.\n\n\nWould it be hard to keep count of the number of errors introduced by\nauthor and reviewer?\n"},{"id":"74426","messageId":"517f3f820804150254w491cdf85s28f1d15696db8d96@mail.gmail.com","threadId":"13099","inReplyTo":"4804765B.2070300@davidnewall.com","subject":"Re: Reporting bugs and bisection","fromName":"Michael Kerrisk","fromEmail":"mtk.manpages@gmail.com","sentAt":"2008-04-15T09:54:18Z","receivedAt":"2008-04-15T09:54:18Z","isPatch":false,"sender":{"key":"mtk.manpages@gmail.com","avatar":null},"body":"On 4/15/08, David Newall wrote:\n> James Morris wrote:\n> > I don't know how to solve this, but suspect that encouraging the use of\n> > reviewed-by and also including it in things like analysis of who is\n> > contributing, selection for kernel summit invitations etc. would be a\n> > start. At least, better than nothing.\n>\n> Would it be hard to keep count of the number of errors introduced by\n> author and reviewer?\n\nI've found quite a few errors in kernel-userland APIs, but I'm not\nsure that this sort of negative statistic would be helpful -- e.g.,\nmore productive developers probably also introduce more errors.\n\n-- \nI'll likely only see replies if they are CCed to mtk.manpages at gmail dot com\n"},{"id":"74446","messageId":"1208265516.4419.125.camel@localhost","threadId":"13099","inReplyTo":"20080415045541.GA611@1wt.eu","subject":"Work WAS(Re: Reporting bugs and bisection","fromName":"jamal","fromEmail":"hadi@cyberus.ca","sentAt":"2008-04-15T13:18:36Z","receivedAt":"2008-04-15T13:18:36Z","isPatch":false,"sender":{"key":"hadi@cyberus.ca","avatar":null},"body":"On Tue, 2008-15-04 at 06:55 +0200, Willy Tarreau wrote:\n\n> And if Dave speaks about these stats : http://lwn.net/Articles/237768/\n> then Al does not even appear in it, which proves your point.\n\nStats such as those above, while useful, are flawed.\nIMO James Morris has (probably more than anybody else) hit on the core\nissue. To extend his view: theres more than just code review that\ndeserves respect. Testing is one. Commenting, not necessarily on code,\nbut on architecture is another. Documenting. Yes, running sparse or even\nLindent or checkpatch.\nIn the old/current Linux thinking (pun intended) work equates to\nchurning code. That thought process derives from Linus actually then\npropagates down stream to other folks.\nI think the Linus approach is still excellent - but its definition of\n\"work\" is no longer valid. Work must include all these other things\nand visible credit is important if the revolution is to continue.\n\nIf you look at it from a software engineering or production resource\nmanagement, the Linux development model has gotta be one of the most\ninefficient[1] - with a reward system geared to developers mostly.\nIf you want to look it from an investment of time (ROI perspective),\ndevelopers get way too much credit riding on everybody elses back.\nWhy should Mark Lord report another bug to us?\nPut yourself in his shoes:\n- he is a clever guy who has already worked around the bug. So a proper\nfix is only a convinience for him.\n- Blessed as he was - he got to do more and more work after reporting.\n- he got slapped for claiming he had to go and get lunch and therefore\ndidnt have time to do more bisect for a bug that wasnt just unique to\nhis setup.\n- he spent a gazillion electrons responding to people and justifying his\nstance\n- he got no credit for his time whatsoever when the bug was fixed (he\nwont be showing up on lwn list).\n\nI think perspective and credit for peoples time needs to change.\n\ncheers,\njamal\n\n[1] With current momentum, theres an infinite resources of developers\nand testers and documenters in Linux, i.e\nresource management is only valid as a metric if you had finite\nresources. So the point i am making is moot - but I do strongly believe\nthe momentum will dampen if current trend of defining work continues.\n"},{"id":"74448","messageId":"4804B5D5.4090404@davidnewall.com","threadId":"13099","inReplyTo":"517f3f820804150254w491cdf85s28f1d15696db8d96@mail.gmail.com","subject":"Re: Reporting bugs and bisection","fromName":"David Newall","fromEmail":"davidn@davidnewall.com","sentAt":"2008-04-15T14:04:05Z","receivedAt":"2008-04-15T14:04:05Z","isPatch":false,"sender":{"key":"davidn@davidnewall.com","avatar":null},"body":"Michael Kerrisk wrote:\n> On 4/15/08, David Newall wrote:\n> \n>> James Morris wrote:\n>> > I don't know how to solve this, but suspect that encouraging the use of\n>> > reviewed-by and also including it in things like analysis of who is\n>> > contributing, selection for kernel summit invitations etc. would be a\n>> > start. At least, better than nothing.\n>>\n>> Would it be hard to keep count of the number of errors introduced by\n>> author and reviewer?\n>> \n>\n> I've found quite a few errors in kernel-userland APIs, but I'm not\n> sure that this sort of negative statistic would be helpful -- e.g.,\n> more productive developers probably also introduce more errors.\n\nWe can already see which developers are more active. What we can't see\nis who is careless, which would be useful to know. It would also be\nuseful to know who is careless in approving changes, because they share\nresponsibility for those changes. It would be a good thing if this\nhighlighted that some people are behind frequent buggy changes.\n"},{"id":"74473","messageId":"200804152251.51308.rjw@sisk.pl","threadId":"13099","inReplyTo":"4804B5D5.4090404@davidnewall.com","subject":"Re: Reporting bugs and bisection","fromName":"Rafael J. Wysocki","fromEmail":"rjw@sisk.pl","sentAt":"2008-04-15T20:51:49Z","receivedAt":"2008-04-15T20:51:49Z","isPatch":false,"sender":{"key":"rjw@sisk.pl","avatar":null},"body":"On Tuesday, 15 of April 2008, David Newall wrote:\n> Michael Kerrisk wrote:\n> > On 4/15/08, David Newall wrote:\n> > \n> >> James Morris wrote:\n> >> > I don't know how to solve this, but suspect that encouraging the use of\n> >> > reviewed-by and also including it in things like analysis of who is\n> >> > contributing, selection for kernel summit invitations etc. would be a\n> >> > start. At least, better than nothing.\n> >>\n> >> Would it be hard to keep count of the number of errors introduced by\n> >> author and reviewer?\n> >> \n> >\n> > I've found quite a few errors in kernel-userland APIs, but I'm not\n> > sure that this sort of negative statistic would be helpful -- e.g.,\n> > more productive developers probably also introduce more errors.\n> \n> We can already see which developers are more active. What we can't see\n> is who is careless, which would be useful to know. It would also be\n> useful to know who is careless in approving changes, because they share\n> responsibility for those changes. It would be a good thing if this\n> highlighted that some people are behind frequent buggy changes.\n\nWell, even if someone introduces bugs relatively frequently, but then also\nworks with the reporters and fixes the bugs timely, it's about okay IMO.\n\nThe real problem is when patch submitters don't care for their changes any\nmore once the patches have been merged.\n\nThanks,\nRafael\n"},{"id":"74503","messageId":"480565D3.6000100@davidnewall.com","threadId":"13099","inReplyTo":"200804152251.51308.rjw@sisk.pl","subject":"Re: Reporting bugs and bisection","fromName":"David Newall","fromEmail":"davidn@davidnewall.com","sentAt":"2008-04-16T02:34:59Z","receivedAt":"2008-04-16T02:34:59Z","isPatch":false,"sender":{"key":"davidn@davidnewall.com","avatar":null},"body":"Rafael J. Wysocki wrote:\n> Well, even if someone introduces bugs relatively frequently, but then also\n> works with the reporters and fixes the bugs timely, it's about okay IMO.\n> \nThis really is not okay. Even if bugs are fixed a version or two later,\nthe impact those bugs have on users makes the system look bad and drives\nthem away. We do not, I believe, want Linux to top the list for \"most\nbugs\". It's unprofessional, unreliable and quite undesirable.\n"},{"id":"74496","messageId":"alpine.DEB.1.10.0804152042320.15483@asgard","threadId":"13099","inReplyTo":"480565D3.6000100@davidnewall.com","subject":"Re: Reporting bugs and bisection","fromName":"","fromEmail":"david@lang.hm","sentAt":"2008-04-16T03:53:27Z","receivedAt":"2008-04-16T03:53:27Z","isPatch":false,"sender":{"key":"david@lang.hm","avatar":null},"body":"On Wed, 16 Apr 2008, David Newall wrote:\n\n> Rafael J. Wysocki wrote:\n>> Well, even if someone introduces bugs relatively frequently, but then also\n>> works with the reporters and fixes the bugs timely, it's about okay IMO.\n>>\n> This really is not okay. Even if bugs are fixed a version or two later,\n> the impact those bugs have on users makes the system look bad and drives\n> them away. We do not, I believe, want Linux to top the list for \"most\n> bugs\". It's unprofessional, unreliable and quite undesirable.\n\ntimely frequently means the code was merged in -rc1/2 and was fixed before \nthe final release of the same version.\n\ngiven the huge variety of hardware and workloads, it's just too easy for \nthere to be cases where any trade-off you make (code size, performance, \nmemory usage, common case definitions) can turn around and bite you. In \naddition frequently hardware doesn't work quite the way the design specs \nsay that it should (completely ignoring the fact that many drivers are \nreverse engineered). what's most important is that when a case shows up it \ngets addressed promptly\n\nI'd rather have a developer/maintainer who introduces and fixed 100 bug, \nbut fixes them promptly, as opposed to one who only introduces one bug, \nbut refuses to consider fixing the code 'because they don't make mistakes \nlike that' (u\bsadly a common attitude from people who produce very \ngood code much of the time)\n\nbest of all is a developer/maintainer who writes very good code and is \nwilling to accept the fact that they make mistakes and fixes the code \npromptly, but those people are extremely rare, and usually they emerge \nfrom the pool of people who make more mistakes and fix them promptly, \nwhich is an added reason I'm more tolerant of that group.\n\nDavid Lang\n"},{"id":"74516","messageId":"20080416042920.GB25188@1wt.eu","threadId":"13099","inReplyTo":"480565D3.6000100@davidnewall.com","subject":"Re: Reporting bugs and bisection","fromName":"Willy Tarreau","fromEmail":"w@1wt.eu","sentAt":"2008-04-16T04:29:20Z","receivedAt":"2008-04-16T04:29:20Z","isPatch":false,"sender":{"key":"w@1wt.eu","avatar":"https://avatars.githubusercontent.com/u/8141789?v=4"},"body":"On Wed, Apr 16, 2008 at 12:04:59PM +0930, David Newall wrote:\n> Rafael J. Wysocki wrote:\n> > Well, even if someone introduces bugs relatively frequently, but then also\n> > works with the reporters and fixes the bugs timely, it's about okay IMO.\n> > \n> This really is not okay. Even if bugs are fixed a version or two later,\n> the impact those bugs have on users makes the system look bad and drives\n> them away. We do not, I believe, want Linux to top the list for \"most\n> bugs\". It's unprofessional, unreliable and quite undesirable.\n\nthat's what -rc are for, and it's unprofessional to use them in production :-)\n\n"},{"id":"74534","messageId":"4805C199.2090702@davidnewall.com","threadId":"13099","inReplyTo":"alpine.DEB.1.10.0804152042320.15483@asgard","subject":"Re: Reporting bugs and bisection","fromName":"David Newall","fromEmail":"davidn@davidnewall.com","sentAt":"2008-04-16T09:06:33Z","receivedAt":"2008-04-16T09:06:33Z","isPatch":false,"sender":{"key":"davidn@davidnewall.com","avatar":null},"body":"david@lang.hm wrote:\n> I'd rather have a developer/maintainer who introduces and fixed 100\n> bug, but fixes them promptly,\n\nAnd I'd rather be able to see that that person introduced 100 bugs than\nto have no idea. As has been said before, the current situation rewards\npeople for sloppy work.\n"},{"id":"74538","messageId":"87mynu5agq.fsf@basil.nowhere.org","threadId":"13099","inReplyTo":"4805C199.2090702@davidnewall.com","subject":"Re: Reporting bugs and bisection","fromName":"Andi Kleen","fromEmail":"andi@firstfloor.org","sentAt":"2008-04-16T11:02:29Z","receivedAt":"2008-04-16T11:02:29Z","isPatch":false,"sender":{"key":"andi@firstfloor.org","avatar":null},"body":"David Newall writes:\n>\n> And I'd rather be able to see that that person introduced 100 bugs than\n> to have no idea. As has been said before, the current situation rewards\n> people for sloppy work.\n\nA common issue in the kernel is code who works with a wide \nrange of hardware and firmware with varying quality. The original\ncode is written to spec but then in the real world the hardware\nand firmware has all kinds of interesting quirks not quite\nmatching the spec that need additional updates to handle. I don't think\nit's fair to say in this case the original developer was sloppy.\n\nThen there is also code which is just hard to tune. Examples for this\nare the CPU scheduler and the VM, but also other areas. They have to\nhandle a lot of different workloads with often subtle side effects.\nLots of people have put a lot of excellent work into tuning these\nsubsystems as users report issues with their workloads. Would you say\nthe original developers were sloppy? I don't think that would be a fair\ndescription. Some problems are just hard and need many \niterations to get right. And then often also the requirements change over \ntime and need further updates.\n\nThere are more such examples in kernel.\n\nGrading programers is a hard problem and I don't think the software\nindustry has really solved it so far, even though there was a lot of\neffort trying to do it over several decades. I doubt it will be solved\nfor the Linux kernel either.\n\n-Andi\n"},{"id":"74541","messageId":"200804161413.05869.rjw@sisk.pl","threadId":"13099","inReplyTo":"20080416042920.GB25188@1wt.eu","subject":"Re: Reporting bugs and bisection","fromName":"Rafael J. Wysocki","fromEmail":"rjw@sisk.pl","sentAt":"2008-04-16T12:13:04Z","receivedAt":"2008-04-16T12:13:04Z","isPatch":false,"sender":{"key":"rjw@sisk.pl","avatar":null},"body":"On Wednesday, 16 of April 2008, Willy Tarreau wrote:\n> On Wed, Apr 16, 2008 at 12:04:59PM +0930, David Newall wrote:\n> > Rafael J. Wysocki wrote:\n> > > Well, even if someone introduces bugs relatively frequently, but then also\n> > > works with the reporters and fixes the bugs timely, it's about okay IMO.\n> > > \n> > This really is not okay. Even if bugs are fixed a version or two later,\n> > the impact those bugs have on users makes the system look bad and drives\n> > them away. We do not, I believe, want Linux to top the list for \"most\n> > bugs\". It's unprofessional, unreliable and quite undesirable.\n> \n> that's what -rc are for, and it's unprofessional to use them in production :-)\n\nExactly.\n\nAnd BTW, by saying \"timely\" I meant \"in -rc\" or \"before the next major release\".\n\nThanks,\nRafael\n"},{"id":"74543","messageId":"bd6139dc0804160515s64a36748v49556c56d475dda4@mail.gmail.com","threadId":"13099","inReplyTo":"4804765B.2070300@davidnewall.com","subject":"Re: Reporting bugs and bisection","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-04-16T12:15:22Z","receivedAt":"2008-04-16T12:15:22Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"I'm not subscribed to the kernel mailing list, so please include me in\nthe cc if you don't reply to the git list (which I am subscribed to).\n\nGit is participating in Google Summer of Code this year and I've\nproposed to write a 'git statistics' command. This command would allow\nthe user to gather data about a repository, ranging from \"how active\nis dev x\" to \"what did x work on in the last 3 weeks\". It's main\nfeature however, would be an algorithm that ranks commits as being\neither 'buggy', 'bugfix' or 'enhancement'. (There are several clues\nthat can aid in determining this, a commit msg along the lines of\n\"fixes ...\" being the most obvious.)\nIn the light of this recent discussion, especially the part on\n\"keeping count of the number of errors introduced by\nauthor and reviewer?\", I thought it might for the kernel mailing list\nto be aware of this. Also mentioned in this thread was that reviewers\ndon't get enough credits. As long as patches are signed with, say,\n'reviewed-by:', 'acked-by:' or 'signed-off-by:' the command I suggest\nto implement would be able to give more accurate statistics on who\n\"works on the kernel\". This way reviewers get the credit they deserve.\nThe knife cuts on both sides of course, if someone reviews a patch\nthat is later determined to introduce a bug, they can be recorded to\nhave acked a buggy commit. This is especially interesting in\ndetermining who are the good reviewers, but also in determining who\nare the good contributors. A distinction could be made between parts\nof the source, say, a maintainer might excel in patches related to\ndriver foo, but when they submit a patch for driver bar it usually\ncontains bugs . Armed with these statistics reviewers might decide to\nbe more careful before acking a patch from that maintainer if it's on\ndriver bar, but when that same maintainer sends in a patch from driver\nbar it is probably ok and needs less attention.\nMy application, and a more extended description, can be found here:\nhttp://alturin.googlepages.com/gsoc2008\n\nI'm interested to know if the community is indeed as interested in my\nproposal as I hope and if I oversaw any obvious features that would\nmake it an even better command.\n\nCheers,\n\nSverre Rabbelier\n"},{"id":"74544","messageId":"4805F402.1020603@earthlink.net","threadId":"13099","inReplyTo":"alpine.DEB.1.10.0804152042320.15483@asgard","subject":"Re: Reporting bugs and bisection","fromName":"Stephen Clark","fromEmail":"sclark46@earthlink.net","sentAt":"2008-04-16T12:41:38Z","receivedAt":"2008-04-16T12:41:38Z","isPatch":false,"sender":{"key":"sclark46@earthlink.net","avatar":null},"body":"david@lang.hm wrote:\n> On Wed, 16 Apr 2008, David Newall wrote:\n> \n>> Rafael J. Wysocki wrote:\n>>> Well, even if someone introduces bugs relatively frequently, but then \n>>> also\n>>> works with the reporters and fixes the bugs timely, it's about okay IMO.\n>>>\n>> This really is not okay. Even if bugs are fixed a version or two later,\n>> the impact those bugs have on users makes the system look bad and drives\n>> them away. We do not, I believe, want Linux to top the list for \"most\n>> bugs\". It's unprofessional, unreliable and quite undesirable.\n> \n> timely frequently means the code was merged in -rc1/2 and was fixed \n> before the final release of the same version.\n> \n> given the huge variety of hardware and workloads, it's just too easy for \n> there to be cases where any trade-off you make (code size, performance, \n> memory usage, common case definitions) can turn around and bite you. In \n> addition frequently hardware doesn't work quite the way the design specs \n> say that it should (completely ignoring the fact that many drivers are \n> reverse engineered). what's most important is that when a case shows up \n> it gets addressed promptly\n> \n> I'd rather have a developer/maintainer who introduces and fixed 100 bug, \n> but fixes them promptly, as opposed to one who only introduces one bug, \n> but refuses to consider fixing the code 'because they don't make \n> mistakes like that' (u\bsadly a common attitude from people who produce \n> very good code much of the time)\n> \n> best of all is a developer/maintainer who writes very good code and is \n> willing to accept the fact that they make mistakes and fixes the code \n> promptly, but those people are extremely rare, and usually they emerge \n> from the pool of people who make more mistakes and fix them promptly, \n> which is an added reason I'm more tolerant of that group.\n> \n> David Lang\n> \nHaving been a Linux user since the late 90's the problem I see is that\ndevelopers decide to re-design stuff that is already working and then things\nthat used to work don't work anymore.\n\nLibata is a good example. I had an older laptop that eventually got working\nagain - but the old ide stuff wasn't studied enough to find out what had to be\nbrought forward and supported in libata.\n\nRegards,\nSteve\n-- \n\n\"They that give up essential liberty to obtain temporary safety,\ndeserve neither liberty nor safety.\" (Ben Franklin)\n\n\"The course of history shows that as a government grows, liberty\ndecreases.\" (Thomas Jefferson)\n"},{"id":"74546","messageId":"20080416132634.GA545@cs181133002.pp.htv.fi","threadId":"13099","inReplyTo":"bd6139dc0804160515s64a36748v49556c56d475dda4@mail.gmail.com","subject":"Re: Reporting bugs and bisection","fromName":"Adrian Bunk","fromEmail":"bunk@kernel.org","sentAt":"2008-04-16T13:26:34Z","receivedAt":"2008-04-16T13:26:34Z","isPatch":false,"sender":{"key":"bunk@kernel.org","avatar":null},"body":"On Wed, Apr 16, 2008 at 02:15:22PM +0200, Sverre Rabbelier wrote:\n> I'm not subscribed to the kernel mailing list, so please include me in\n> the cc if you don't reply to the git list (which I am subscribed to).\n> \n> Git is participating in Google Summer of Code this year and I've\n> proposed to write a 'git statistics' command. This command would allow\n> the user to gather data about a repository, ranging from \"how active\n> is dev x\" to \"what did x work on in the last 3 weeks\". It's main\n> feature however, would be an algorithm that ranks commits as being\n> either 'buggy', 'bugfix' or 'enhancement'. (There are several clues\n> that can aid in determining this, a commit msg along the lines of\n> \"fixes ...\" being the most obvious.)\n>...\n\nAt least with the data we have currently in git it's impossible to \nfigure that out automatically.\n\nE.g. if you look at commit f743d04dcfbeda7439b78802d35305781999aa11 \n(ide/legacy/q40ide.c: add MODULE_LICENSE), how could you determine \nautomatically that it is a bugfix, and the commit that introduced\nthe bug?\n\nYou can always get some data, but if you want to get usable statistics \nyou need explicit tags in the commits, not some algorithm that tries \nto guess.\n\n> Cheers,\n> \n> Sverre Rabbelier\n\ncu\nAdrian\n\n-- \n\n \"Is there not promise of rain?\" Ling Tan asked suddenly out\n of the darkness. There had been need of rain for many days.\n \"Only a promise,\" Lao Er said.\n Pearl S. Buck - Dragon Seed\n"},{"id":"74575","messageId":"20080416120247.c665859c.akpm@linux-foundation.org","threadId":"13099","inReplyTo":"20080416132634.GA545@cs181133002.pp.htv.fi","subject":"Re: Reporting bugs and bisection","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-04-16T19:02:47Z","receivedAt":"2008-04-16T19:02:47Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Wed, 16 Apr 2008 16:26:34 +0300\nAdrian Bunk wrote:\n\n> On Wed, Apr 16, 2008 at 02:15:22PM +0200, Sverre Rabbelier wrote:\n> > I'm not subscribed to the kernel mailing list, so please include me in\n> > the cc if you don't reply to the git list (which I am subscribed to).\n> > \n> > Git is participating in Google Summer of Code this year and I've\n> > proposed to write a 'git statistics' command. This command would allow\n> > the user to gather data about a repository, ranging from \"how active\n> > is dev x\" to \"what did x work on in the last 3 weeks\". It's main\n> > feature however, would be an algorithm that ranks commits as being\n> > either 'buggy', 'bugfix' or 'enhancement'. (There are several clues\n> > that can aid in determining this, a commit msg along the lines of\n> > \"fixes ...\" being the most obvious.)\n> >...\n\nSounds like an interesting project.\n\n> At least with the data we have currently in git it's impossible to \n> figure that out automatically.\n> \n> E.g. if you look at commit f743d04dcfbeda7439b78802d35305781999aa11 \n> (ide/legacy/q40ide.c: add MODULE_LICENSE), how could you determine \n> automatically that it is a bugfix, and the commit that introduced\n> the bug?\n> \n> You can always get some data, but if you want to get usable statistics \n> you need explicit tags in the commits, not some algorithm that tries \n> to guess.\n\nWell yes. One outcome of the project would be to tell us what changes we'd\nneed to make to our processes to make such data gathering more effective.\n\nOf course, we may not actually implement such changes. That would depend\nupon how useful the output is to us.\n"},{"id":"74578","messageId":"bd6139dc0804161239h17e79c70ta5e938619e5743c9@mail.gmail.com","threadId":"13099","inReplyTo":"20080416132634.GA545@cs181133002.pp.htv.fi","subject":"Re: Reporting bugs and bisection","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-04-16T19:39:41Z","receivedAt":"2008-04-16T19:39:41Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Wed, Apr 16, 2008 at 3:26 PM, Adrian Bunk wrote:\n> On Wed, Apr 16, 2008 at 02:15:22PM +0200, Sverre Rabbelier wrote:\n> At least with the data we have currently in git it's impossible to\n> figure that out automatically.\n\nI don't quite agree, as I explained in my proposal there are several\nways to detect that a commit was a bugfix. From thereon you can deduct\nthat if it was a bugfix, that the commit that introduced the fixed\nchange was a bug! From thereon you can start sifting and get more\nconfirmations. Junio has made several suggestions as to how this could\nbe implemented and I'm confident that and algorithm can be devised\nthat is at least capable of 'guessing' what type a commit is. Aside\nfrom the guessing part I think a lot of information can be gathered\nfrom commit msgs.\n\nOf course, some commits might not be able to be typed (as there might\nnot be any 'follow up' information on them). Those commits can be\nmarked as 'unknown' and be ignored. Agreed, should all commits be\n'unknown' then the command wouldn't be very useful, but especially on\nlarge repos there is a very large dataset. As the size of the dataset\nincreases I estimate that the correlation between commits increases\n(less commits that add new code which then is never changed\ntherafter). The higher the degree of correlation between individual\ncommits the more we can determine about the nature of a commit.\n\n\n> E.g. if you look at commit f743d04dcfbeda7439b78802d35305781999aa11\n> (ide/legacy/q40ide.c: add MODULE_LICENSE), how could you determine\n> automatically that it is a bugfix, and the commit that introduced\n> the bug?\n\nWell, a dead giveaway would be:\n\"http://bugzilla.kernel.org/show_bug.cgi?id=10124\"\n\n> You can always get some data, but if you want to get usable statistics\n> you need explicit tags in the commits, not some algorithm that tries\n> to guess.\n\nAs said above, I don't agree, you can 'guess' very reliably on a large\ndataset. Also, most commits are already 'tagged' in some way or\nanother. The trick is to find the pattern in this tagging and use it.\n\nI hope this clears things up a bit,\n\nCheers,\n\nSverre Rabbelier\n"},{"id":"74577","messageId":"bd6139dc0804161243i5e5c973aia0cf2d5ec19079fa@mail.gmail.com","threadId":"13099","inReplyTo":"20080416120247.c665859c.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-04-16T19:43:47Z","receivedAt":"2008-04-16T19:43:47Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Wed, Apr 16, 2008 at 9:02 PM, Andrew Morton\n wrote:\n> Sounds like an interesting project.\n\nThank you :).\n\n> Well yes. One outcome of the project would be to tell us what changes we'd\n> need to make to our processes to make such data gathering more effective.\n\nI defenitly agree here, the command's reliability could be increased\nby always specifying bugfixes in a certain way. 'fixed-bug:' for\nexample should be very recognizable.\n\n> Of course, we may not actually implement such changes. That would depend\n> upon how useful the output is to us.\n\nAh yes, free will and whatnot. Then again, everybody already does\n'signed-off-by:', if there's an easy command in git to mark a bugfix,\nit would increase the odds of people using it. Perhaps something like\n'git commit -b 10256\" which would then automagically append a\npredefined message to the commit users would feel more inclined?\n\nCheers,\n\nSverre Rabbelier\n"},{"id":"74579","messageId":"20080416195503.GR1677@cs181133002.pp.htv.fi","threadId":"13099","inReplyTo":"20080416120247.c665859c.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Adrian Bunk","fromEmail":"bunk@kernel.org","sentAt":"2008-04-16T19:55:03Z","receivedAt":"2008-04-16T19:55:03Z","isPatch":false,"sender":{"key":"bunk@kernel.org","avatar":null},"body":"On Wed, Apr 16, 2008 at 12:02:47PM -0700, Andrew Morton wrote:\n> On Wed, 16 Apr 2008 16:26:34 +0300\n> Adrian Bunk wrote:\n> \n> > On Wed, Apr 16, 2008 at 02:15:22PM +0200, Sverre Rabbelier wrote:\n> > > I'm not subscribed to the kernel mailing list, so please include me in\n> > > the cc if you don't reply to the git list (which I am subscribed to).\n> > > \n> > > Git is participating in Google Summer of Code this year and I've\n> > > proposed to write a 'git statistics' command. This command would allow\n> > > the user to gather data about a repository, ranging from \"how active\n> > > is dev x\" to \"what did x work on in the last 3 weeks\". It's main\n> > > feature however, would be an algorithm that ranks commits as being\n> > > either 'buggy', 'bugfix' or 'enhancement'. (There are several clues\n> > > that can aid in determining this, a commit msg along the lines of\n> > > \"fixes ...\" being the most obvious.)\n> > >...\n> \n> Sounds like an interesting project.\n> \n> > At least with the data we have currently in git it's impossible to \n> > figure that out automatically.\n> > \n> > E.g. if you look at commit f743d04dcfbeda7439b78802d35305781999aa11 \n> > (ide/legacy/q40ide.c: add MODULE_LICENSE), how could you determine \n> > automatically that it is a bugfix, and the commit that introduced\n> > the bug?\n> > \n> > You can always get some data, but if you want to get usable statistics \n> > you need explicit tags in the commits, not some algorithm that tries \n> > to guess.\n> \n> Well yes. One outcome of the project would be to tell us what changes we'd\n> need to make to our processes to make such data gathering more effective.\n> \n> Of course, we may not actually implement such changes. That would depend\n> upon how useful the output is to us.\n\nThat you can add this information through tags is clear, but according\nto his SoC application that's not what he wants to do.\n\nAccording to his application he wants to determine automatically whether \na commit was a fix or whether a commit introduced a bug by doing stuff \nlike tracking whether a changed line was modified again shortly after a \ncommit.\n\nThis plan of him will simply not result in accurate numbers.\n\nSure, you will get some numbers, but if anyone would e.g. wrongly accuse \nme that 2% of my commits last year introduced bugs I would get \n***really*** angry.\n\ncu\nAdrian\n\n-- \n\n \"Is there not promise of rain?\" Ling Tan asked suddenly out\n of the darkness. There had been need of rain for many days.\n \"Only a promise,\" Lao Er said.\n Pearl S. Buck - Dragon Seed\n\n"},{"id":"74580","messageId":"20080416195829.GB4762@martell.zuzino.mipt.ru","threadId":"13099","inReplyTo":"20080416120247.c665859c.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Alexey Dobriyan","fromEmail":"adobriyan@gmail.com","sentAt":"2008-04-16T19:58:29Z","receivedAt":"2008-04-16T19:58:29Z","isPatch":false,"sender":{"key":"adobriyan@gmail.com","avatar":null},"body":"On Wed, Apr 16, 2008 at 12:02:47PM -0700, Andrew Morton wrote:\n> On Wed, 16 Apr 2008 16:26:34 +0300\n> Adrian Bunk wrote:\n> \n> > On Wed, Apr 16, 2008 at 02:15:22PM +0200, Sverre Rabbelier wrote:\n> > > I'm not subscribed to the kernel mailing list, so please include me in\n> > > the cc if you don't reply to the git list (which I am subscribed to).\n> > > \n> > > Git is participating in Google Summer of Code this year and I've\n> > > proposed to write a 'git statistics' command. This command would allow\n> > > the user to gather data about a repository, ranging from \"how active\n> > > is dev x\" to \"what did x work on in the last 3 weeks\".\n\nThese are pointy-hairy questions.\n\n> > > It's main\n> > > feature however, would be an algorithm that ranks commits as being\n> > > either 'buggy', 'bugfix' or 'enhancement'. (There are several clues\n> > > that can aid in determining this, a commit msg along the lines of\n> > > \"fixes ...\" being the most obvious.)\n> > >...\n> \n> Sounds like an interesting project.\n\nThe interesting (and answerable) questions are:\n\n1) How many bugs one non-merge commit brings on average\n2) What is average time between buggy commit entering Linus's tree and\n fix entering the same tree.\n3) Graphs of #1 and #2 over time.\n4) rough division of bugs a-la refcounting, locking, hw, hw workaround.\n5) if other OS have such statistics, comparison with them\n (little finger for this)\n\n#1 alone can shred OSDL and LWN induced PDFs into innumerable pieces!\n"},{"id":"74583","messageId":"20080416130108.4eda6c2c@laptopd505.fenrus.org","threadId":"13099","inReplyTo":"20080416120247.c665859c.akpm@linux-foundation.org","subject":"Re: Reporting bugs and bisection","fromName":"Arjan van de Ven","fromEmail":"arjan@infradead.org","sentAt":"2008-04-16T20:01:08Z","receivedAt":"2008-04-16T20:01:08Z","isPatch":false,"sender":{"key":"arjan@infradead.org","avatar":"https://gravatar.com/avatar/42631f2338fc04830b31b08bc42b6575a3547fba469bd73ba7e1f237d650986a?d=mp&s=160"},"body":"On Wed, 16 Apr 2008 12:02:47 -0700\nAndrew Morton wrote:\n\n> \n> > At least with the data we have currently in git it's impossible to \n> > figure that out automatically.\n> > \n> > E.g. if you look at commit f743d04dcfbeda7439b78802d35305781999aa11 \n> > (ide/legacy/q40ide.c: add MODULE_LICENSE), how could you determine \n> > automatically that it is a bugfix, and the commit that introduced\n> > the bug?\n> > \n> > You can always get some data, but if you want to get usable\n> > statistics you need explicit tags in the commits, not some\n> > algorithm that tries to guess.\n> \n> Well yes. One outcome of the project would be to tell us what\n> changes we'd need to make to our processes to make such data\n> gathering more effective.\n\nalso.. \"what is a bugfix\" is an interesting thing... for some things it's very easy.\nFor others.. it's really hard to draw a solid line where bugs stop and features start.\n(for example, is a missing cpu id in oprofile a bugfix (\"oprofile doesn't work\") or \na feature (\"new cpu support\"). This one is one of the more simple ones even...)\n\n-- \nIf you want to reach me at my work email, use arjan@linux.intel.com\nFor development, discussion and tips for power savings, \nvisit http://www.lesswatts.org\n"},{"id":"74593","messageId":"20080416200408.GA7764@1wt.eu","threadId":"13099","inReplyTo":"20080416132634.GA545@cs181133002.pp.htv.fi","subject":"Re: Reporting bugs and bisection","fromName":"Willy Tarreau","fromEmail":"w@1wt.eu","sentAt":"2008-04-16T20:04:08Z","receivedAt":"2008-04-16T20:04:08Z","isPatch":false,"sender":{"key":"w@1wt.eu","avatar":"https://avatars.githubusercontent.com/u/8141789?v=4"},"body":"On Wed, Apr 16, 2008 at 04:26:34PM +0300, Adrian Bunk wrote:\n> On Wed, Apr 16, 2008 at 02:15:22PM +0200, Sverre Rabbelier wrote:\n> > I'm not subscribed to the kernel mailing list, so please include me in\n> > the cc if you don't reply to the git list (which I am subscribed to).\n> > \n> > Git is participating in Google Summer of Code this year and I've\n> > proposed to write a 'git statistics' command. This command would allow\n> > the user to gather data about a repository, ranging from \"how active\n> > is dev x\" to \"what did x work on in the last 3 weeks\". It's main\n> > feature however, would be an algorithm that ranks commits as being\n> > either 'buggy', 'bugfix' or 'enhancement'. (There are several clues\n> > that can aid in determining this, a commit msg along the lines of\n> > \"fixes ...\" being the most obvious.)\n> >...\n> \n> At least with the data we have currently in git it's impossible to \n> figure that out automatically.\n> \n> E.g. if you look at commit f743d04dcfbeda7439b78802d35305781999aa11 \n> (ide/legacy/q40ide.c: add MODULE_LICENSE), how could you determine \n> automatically that it is a bugfix, and the commit that introduced\n> the bug?\n> \n> You can always get some data, but if you want to get usable statistics \n> you need explicit tags in the commits, not some algorithm that tries \n> to guess.\n\nyes, and doing that would get back to the bureaucracy some people are\ntrying to reduce in order to save time to do the real work.\n\nHowever, in another project of mine, I've got used to systematically\nindicate the type of change in the subject line. It does not get any\nslower for the author, and it appears in shortlogs. And quite amazingly\nthe principle has immediately been adopted by several contributors :\n\n-----\nNote to contributors: it's very handy when patches comes with a properly\nformated subject. Try to put one of the following words between brackets\nto indicate the importance of the patch followed by a short description:\n\n[MINOR] minor fix, very low risk of impact\n[MEDIUM] medium risk, may cause unexpected regressions of low importance or\n which may quickly be discovered\n[MAJOR] major risk of hidden regression. This happens when I rearrange large\n parts of code, when I play with timeouts, with variable\n initializations, etc...\n[BUG] fix for a minor or medium-level bug.\n[CRITICAL] medium-term reliability or security is at risk, an upgrade is\n absolutely required.\n[RELEASE] release a new version\n[BUILD] fix build issues. If you could build, no upgrade required.\n[CLEANUP] code cleanup, silence of warnings, etc... theorically no impact\n[TESTS] added regression testing configuration files or scripts\n[DOC] documentation updates, no need to upgrade\n[LICENSE] licensing updates (may impact distro packagers)\n\nExample: \"[DOC] document options forwardfor to logasap\"\n-----\n\nNothing is mandatory, and I (as the maintainer) can still choose to\nadjust the prefix if I want. But in fact, I only had to to it when\ncontributors did not classify their patch themselves. Several other\ntags may be added for LKML, such as \"RFC\" which is already used,\netc...\n\nThe advantages of this usage are multiple. Nothing needs to be changed\nin the tools, no header needs to be added, it's still very compatible\nwith the mailing-list usages (and helps focusing on specific patches),\nit's absolutely not mandatory and easily tweakable.\n\nI'd like people in this thread not to forget that what we need is not\na fantastic tool to work around some developers' weaknesses, but cheap\n(if any) help from the developers to help reviewers. I think that such\na proposal falls exactly in this category.\n\nI'm quite ready to use it already (though I do not post often), and\nthink that it would still feel natural to many developers since most\nof them are already used to such a format. I think it just requires\na few starters to get most of us to progressively use such a scheme\nby default.\n\nRegards,\nWilly\n"},{"id":"74588","messageId":"20080416201606.GS1677@cs181133002.pp.htv.fi","threadId":"13099","inReplyTo":"bd6139dc0804161239h17e79c70ta5e938619e5743c9@mail.gmail.com","subject":"Re: Reporting bugs and bisection","fromName":"Adrian Bunk","fromEmail":"bunk@kernel.org","sentAt":"2008-04-16T20:16:06Z","receivedAt":"2008-04-16T20:16:06Z","isPatch":false,"sender":{"key":"bunk@kernel.org","avatar":null},"body":"On Wed, Apr 16, 2008 at 09:39:41PM +0200, Sverre Rabbelier wrote:\n> On Wed, Apr 16, 2008 at 3:26 PM, Adrian Bunk wrote:\n>...\n> > E.g. if you look at commit f743d04dcfbeda7439b78802d35305781999aa11\n> > (ide/legacy/q40ide.c: add MODULE_LICENSE), how could you determine\n> > automatically that it is a bugfix, and the commit that introduced\n> > the bug?\n> \n> Well, a dead giveaway would be:\n> \"http://bugzilla.kernel.org/show_bug.cgi?id=10124\"\n\nWhich could be \"There is no driver for my TV card in the kernel.\"\n\n> > You can always get some data, but if you want to get usable statistics\n> > you need explicit tags in the commits, not some algorithm that tries\n> > to guess.\n> \n> As said above, I don't agree, you can 'guess' very reliably on a large\n> dataset. Also, most commits are already 'tagged' in some way or\n> another. The trick is to find the pattern in this tagging and use it.\n> \n> I hope this clears things up a bit,\n\nI hope you are aware of the non-technical implications if the results \ndon't match reality?\n\nE.g. I am proud that my commits do virtually never introduce bugs, so \nany results someone publishes about what I do should better be right\nor my first thoughts are somewhere between \"fist\" and \"lawyer\". [1]\n\n> Cheers,\n> \n> Sverre Rabbelier\n\ncu\nAdrian\n\n[1] my actual reaction might only be an angry email, but I hope you\n get the point that wrong results can really piss off people\n\n-- \n\n \"Is there not promise of rain?\" Ling Tan asked suddenly out\n of the darkness. There had been need of rain for many days.\n \"Only a promise,\" Lao Er said.\n Pearl S. Buck - Dragon Seed\n"},{"id":"74595","messageId":"20080416205333.GT1677@cs181133002.pp.htv.fi","threadId":"13099","inReplyTo":"20080416201606.GS1677@cs181133002.pp.htv.fi","subject":"Re: Reporting bugs and bisection","fromName":"Adrian Bunk","fromEmail":"bunk@kernel.org","sentAt":"2008-04-16T20:53:33Z","receivedAt":"2008-04-16T20:53:33Z","isPatch":false,"sender":{"key":"bunk@kernel.org","avatar":null},"body":"On Wed, Apr 16, 2008 at 11:16:06PM +0300, Adrian Bunk wrote:\n>...\n> E.g. I am proud that my commits do virtually never introduce bugs, so \n> any results someone publishes about what I do should better be right\n> or my first thoughts are somewhere between \"fist\" and \"lawyer\". [1]\n>...\n\nTo avoid any misunderstandings:\n\nThis is not in any way meant against you personally.\n\nBut saying things like \" X% of your commits introduced bugs\" is not a\nfriendly thing, and wrong data could be quite hurting.\n\nEspecially in the open source world where much motivation comes from\npeople being proud of their work.\n\nEven correct data can do harm.\n\nAnd bad data can have really bad effects.\n\ncu\nAdrian\n\n-- \n\n \"Is there not promise of rain?\" Ling Tan asked suddenly out\n of the darkness. There had been need of rain for many days.\n \"Only a promise,\" Lao Er said.\n Pearl S. Buck - Dragon Seed\n"},{"id":"74597","messageId":"fu5p3i$96n$1@ger.gmane.org","threadId":"13099","inReplyTo":"20080416200408.GA7764@1wt.eu","subject":"Re: Reporting bugs and bisection","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-04-16T20:55:24Z","receivedAt":"2008-04-16T20:55:24Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Willy Tarreau wrote:\n\n> Note to contributors: it's very handy when patches comes with a properly\n> formated subject. Try to put one of the following words between brackets\n> to indicate the importance of the patch followed by a short description:\n> \n> [MINOR]    minor fix, very low risk of impact\n> [MEDIUM]   medium risk, may cause unexpected regressions of low importance or\n>            which may quickly be discovered\n\n[...]\n\nAnd git-am strips such prefixes because of [PATCH] and [PATCH n/m] which\nshould be stripped.\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n"},{"id":"74599","messageId":"bd6139dc0804161405j28470914u488568b565b68a0b@mail.gmail.com","threadId":"13099","inReplyTo":"20080416205333.GT1677@cs181133002.pp.htv.fi","subject":"Re: Reporting bugs and bisection","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-04-16T21:05:17Z","receivedAt":"2008-04-16T21:05:17Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Wed, Apr 16, 2008 at 10:53 PM, Adrian Bunk wrote:\n> To avoid any misunderstandings:\n>\n> This is not in any way meant against you personally.\n\nThanks for pointing it out, I wasn't quite sure, but assumed that :).\n\n> But saying things like \" X% of your commits introduced bugs\" is not a\n> friendly thing, and wrong data could be quite hurting.\n\nYes, it could be, and I agree that conclusions shouldn't be based on\nthe details, but on the bigger picture. Also, I think it should (at\nfirst) be used mainly as an indicator, of where attention might be\nrequired. I mean, if it points out that one contributor almost always\ncommits buggy code, you don't have to present them with those\nstatistics right away. Instead you can ask the program where it bases\nit's conclusions on, and research them yourself. If it does indeed\nturn out that they are slacking that much you have good ground to have\na talk with them.\n\n> Especially in the open source world where much motivation comes from\n> people being proud of their work.\n\nYes, that is very true, I very much agree with that, but on the other\nhand it might also point out contributors that are particularly\nskillful in a certain section that was previously not noted. As with\nall statistics, it's up to interpretation, misinterpreting statistics\ncould -always- have bad effects.\n\n> Even correct data can do harm.\n>\n> And bad data can have really bad effects.\n\nTrue, both, but as said, if properly interpreted it could be very useful.\n\nCheers,\n\nSverre Rabbelier\n"},{"id":"74601","messageId":"9a8748490804161417n4ad6c1den54ccd302831a66c6@mail.gmail.com","threadId":"13099","inReplyTo":"bd6139dc0804160515s64a36748v49556c56d475dda4@mail.gmail.com","subject":"Re: Reporting bugs and bisection","fromName":"Jesper Juhl","fromEmail":"jesper.juhl@gmail.com","sentAt":"2008-04-16T21:17:14Z","receivedAt":"2008-04-16T21:17:14Z","isPatch":false,"sender":{"key":"jesper.juhl@gmail.com","avatar":null},"body":"On 16/04/2008, Sverre Rabbelier wrote:\n...\n> Git is participating in Google Summer of Code this year and I've\n> proposed to write a 'git statistics' command. This command would allow\n> the user to gather data about a repository, ranging from \"how active\n> is dev x\" to \"what did x work on in the last 3 weeks\". It's main\n> feature however, would be an algorithm that ranks commits as being\n> either 'buggy', 'bugfix' or 'enhancement'.\n\nInterresting. Just be careful results are produced for the big picture\nand not used to point fingers at individuals.\n\n>(There are several clues\n> that can aid in determining this, a commit msg along the lines of\n> \"fixes ...\" being the most obvious.)\n\nOne thing I thought of is that the more \"Acked-by\", \"Reviewed-by\" and\n\"Signed-off-by\" lines a patch has, the better reviewed we can probably\nassume it to be and thus the probability of it having introduced a bug\nprobably drops slightly compared to other less-reviewed patches... or\nmaybe not, but at least it's something to think about :-)\n\n\n-- \nJesper Juhl \nDon't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html\nPlain text mails only, please http://www.expita.com/nomime.html\n"},{"id":"74603","messageId":"20080416212554.GV1677@cs181133002.pp.htv.fi","threadId":"13099","inReplyTo":"bd6139dc0804161405j28470914u488568b565b68a0b@mail.gmail.com","subject":"Re: Reporting bugs and bisection","fromName":"Adrian Bunk","fromEmail":"bunk@kernel.org","sentAt":"2008-04-16T21:25:54Z","receivedAt":"2008-04-16T21:25:54Z","isPatch":false,"sender":{"key":"bunk@kernel.org","avatar":null},"body":"On Wed, Apr 16, 2008 at 11:05:17PM +0200, Sverre Rabbelier wrote:\n> On Wed, Apr 16, 2008 at 10:53 PM, Adrian Bunk wrote:\n> > To avoid any misunderstandings:\n> >\n> > This is not in any way meant against you personally.\n> \n> Thanks for pointing it out, I wasn't quite sure, but assumed that :).\n\nSorry, I was a bit overreacting since I see too often people putting \nsome data into some statistics or graph and drawing conclusins without \npaying attention to whether their data allows these conclusions at all.\n\n> > But saying things like \" X% of your commits introduced bugs\" is not a\n> > friendly thing, and wrong data could be quite hurting.\n> \n> Yes, it could be, and I agree that conclusions shouldn't be based on\n> the details, but on the bigger picture. Also, I think it should (at\n> first) be used mainly as an indicator, of where attention might be\n> required. I mean, if it points out that one contributor almost always\n> commits buggy code,\n\nI would assume that in all projects the main maintainers already have an \nimpression of how good the quality of the patches of each main \ncontributor is.\n\nIn much more complex ways than a number could express.\n\n> you don't have to present them with those\n> statistics right away. Instead you can ask the program where it bases\n> it's conclusions on, and research them yourself.\n\nSooner or later someone will run the program for the Linux kernel, \nwrite a paper about the results, and publish his research somewhere.\n\n>...\n> Cheers,\n> \n> Sverre Rabbelier\n\ncu\nAdrian\n\n-- \n\n \"Is there not promise of rain?\" Ling Tan asked suddenly out\n of the darkness. There had been need of rain for many days.\n \"Only a promise,\" Lao Er said.\n Pearl S. Buck - Dragon Seed\n\n"},{"id":"74638","messageId":"20080417135013.GA2017@fieldses.org","threadId":"13099","inReplyTo":"20080416195503.GR1677@cs181133002.pp.htv.fi","subject":"Re: Reporting bugs and bisection","fromName":"J. Bruce Fields","fromEmail":"bfields@fieldses.org","sentAt":"2008-04-17T13:50:13Z","receivedAt":"2008-04-17T13:50:13Z","isPatch":false,"sender":{"key":"bfields@citi.umich.edu","avatar":null},"body":"On Wed, Apr 16, 2008 at 10:55:03PM +0300, Adrian Bunk wrote:\n> On Wed, Apr 16, 2008 at 12:02:47PM -0700, Andrew Morton wrote:\n> > On Wed, 16 Apr 2008 16:26:34 +0300\n> > Adrian Bunk wrote:\n> > \n> > > On Wed, Apr 16, 2008 at 02:15:22PM +0200, Sverre Rabbelier wrote:\n> > > > I'm not subscribed to the kernel mailing list, so please include me in\n> > > > the cc if you don't reply to the git list (which I am subscribed to).\n> > > > \n> > > > Git is participating in Google Summer of Code this year and I've\n> > > > proposed to write a 'git statistics' command. This command would allow\n> > > > the user to gather data about a repository, ranging from \"how active\n> > > > is dev x\" to \"what did x work on in the last 3 weeks\". It's main\n> > > > feature however, would be an algorithm that ranks commits as being\n> > > > either 'buggy', 'bugfix' or 'enhancement'. (There are several clues\n> > > > that can aid in determining this, a commit msg along the lines of\n> > > > \"fixes ...\" being the most obvious.)\n> > > >...\n> > \n> > Sounds like an interesting project.\n> > \n> > > At least with the data we have currently in git it's impossible to \n> > > figure that out automatically.\n> > > \n> > > E.g. if you look at commit f743d04dcfbeda7439b78802d35305781999aa11 \n> > > (ide/legacy/q40ide.c: add MODULE_LICENSE), how could you determine \n> > > automatically that it is a bugfix, and the commit that introduced\n> > > the bug?\n> > > \n> > > You can always get some data, but if you want to get usable statistics \n> > > you need explicit tags in the commits, not some algorithm that tries \n> > > to guess.\n> > \n> > Well yes. One outcome of the project would be to tell us what changes we'd\n> > need to make to our processes to make such data gathering more effective.\n> > \n> > Of course, we may not actually implement such changes. That would depend\n> > upon how useful the output is to us.\n> \n> That you can add this information through tags is clear, but according\n> to his SoC application that's not what he wants to do.\n> \n> According to his application he wants to determine automatically whether \n> a commit was a fix or whether a commit introduced a bug by doing stuff \n> like tracking whether a changed line was modified again shortly after a \n> commit.\n> \n> This plan of him will simply not result in accurate numbers.\n\nThey won't be completely accurate, but who knows, maybe they'd turn out\nto have a higher rate of accuracy than we'd expect. (I assume you could\ndo a closer manual study of a small random sample of the results to\nestimate the accuracy.) Seems worth a try.\n\n> Sure, you will get some numbers, but if anyone would e.g. wrongly accuse \n> me that 2% of my commits last year introduced bugs I would get \n> ***really*** angry.\n\nIt's just an experiment; reasonable people won't take it as the final\nword.\n\n--b.\n"},{"id":"74642","messageId":"20080417152633.GA12951@cs181133002.pp.htv.fi","threadId":"13099","inReplyTo":"20080417135013.GA2017@fieldses.org","subject":"Re: Reporting bugs and bisection","fromName":"Adrian Bunk","fromEmail":"bunk@kernel.org","sentAt":"2008-04-17T15:26:33Z","receivedAt":"2008-04-17T15:26:33Z","isPatch":false,"sender":{"key":"bunk@kernel.org","avatar":null},"body":"On Thu, Apr 17, 2008 at 09:50:13AM -0400, J. Bruce Fields wrote:\n> On Wed, Apr 16, 2008 at 10:55:03PM +0300, Adrian Bunk wrote:\n>...\n> > Sure, you will get some numbers, but if anyone would e.g. wrongly accuse \n> > me that 2% of my commits last year introduced bugs I would get \n> > ***really*** angry.\n> \n> It's just an experiment; reasonable people won't take it as the final\n> word.\n\nTake e.g. [1] as an example how git statistics about the Linux kernel \nare already used to \"prove\" things that aren't true.\n\n> --b.\n\ncu\nAdrian\n\n[1] http://digitalvampire.org/blog/index.php/2008/04/11/lies-d-oh-forget-it/\n\n-- \n\n \"Is there not promise of rain?\" Ling Tan asked suddenly out\n of the darkness. There had been need of rain for many days.\n \"Only a promise,\" Lao Er said.\n Pearl S. Buck - Dragon Seed\n"},{"id":"74651","messageId":"48078323.4010109@davidnewall.com","threadId":"13099","inReplyTo":"9a8748490804161417n4ad6c1den54ccd302831a66c6@mail.gmail.com","subject":"Re: Reporting bugs and bisection","fromName":"David Newall","fromEmail":"davidn@davidnewall.com","sentAt":"2008-04-17T17:04:35Z","receivedAt":"2008-04-17T17:04:35Z","isPatch":false,"sender":{"key":"davidn@davidnewall.com","avatar":null},"body":"Jesper Juhl wrote:\n> Interresting. Just be careful results are produced for the big picture\n> and not used to point fingers at individuals.\n> \n\nIf there are individuals at whom a finger needs to be pointed, this\nsystem will highlight them, and fingers will (and should) be pointed. \nContributors of poor-quality code need to be weeded-out. \nFinger-pointing, in these extreme cases, gives incentive to improve\nquality. It's a positive thing.\n"},{"id":"74658","messageId":"200804172109.35027.rjw@sisk.pl","threadId":"13099","inReplyTo":"48078323.4010109@davidnewall.com","subject":"Re: Reporting bugs and bisection","fromName":"Rafael J. Wysocki","fromEmail":"rjw@sisk.pl","sentAt":"2008-04-17T19:09:33Z","receivedAt":"2008-04-17T19:09:33Z","isPatch":false,"sender":{"key":"rjw@sisk.pl","avatar":null},"body":"On Thursday, 17 of April 2008, David Newall wrote:\n> Jesper Juhl wrote:\n> > Interresting. Just be careful results are produced for the big picture\n> > and not used to point fingers at individuals.\n> > \n> \n> If there are individuals at whom a finger needs to be pointed, this\n> system will highlight them, and fingers will (and should) be pointed. \n> Contributors of poor-quality code need to be weeded-out.\n\nDefine poor quality.\n \n> Finger-pointing, in these extreme cases, gives incentive to improve\n> quality. It's a positive thing.\n\nSorry, but I have to disagree. Negative finger-pointing is never a good thing.\nAlso, it doesn't give any incentive to anyone. It only makes people feel bad\nand finally discourages them from contributing anything.\n\nIf you want to give poeple incentives, reward them for doing things you'd like\nthem to do.\n\nThanks,\nRafael\n"},{"id":"74661","messageId":"2c0942db0804171235o49238b99u6cdbd3e5c8d6ebb7@mail.gmail.com","threadId":"13099","inReplyTo":"200804172109.35027.rjw@sisk.pl","subject":"Re: Reporting bugs and bisection","fromName":"Ray Lee","fromEmail":"ray-lk@madrabbit.org","sentAt":"2008-04-17T19:35:12Z","receivedAt":"2008-04-17T19:35:12Z","isPatch":false,"sender":{"key":"ray-lk@madrabbit.org","avatar":null},"body":"On Thu, Apr 17, 2008 at 12:09 PM, Rafael J. Wysocki wrote:\n> > Finger-pointing, in these extreme cases, gives incentive to improve\n> > quality. It's a positive thing.\n>\n> Sorry, but I have to disagree. Negative finger-pointing is never a good thing.\n\nCorrect, but let's be careful here. The original suggestion was,\neffectively, to get better metrics on the quality of contributions.\nThose metrics *could* be used for finger pointing, or (my preference)\nthey could be used to direct and allocate our scarce resources: code\nreviews and mentoring.\n\nThere's no way to know what the metrics will tell us until we have\nthem. Arguing against metrics because they *may* be used to point\nfingers at people is a silly argument; anything can be subverted to do\nthat.\n\nLet's get some measurements and see what they say. In the meantime,\ntry to believe that they could be put to good purposes, such as\nidentifying code areas that are tricky for contributors to get right\n(independent of contributor), or contributors that could benefit from\ncode reviews, etc.\n"},{"id":"74664","messageId":"bd6139dc0804171257l5f875694yb57105ba40170789@mail.gmail.com","threadId":"13099","inReplyTo":"2c0942db0804171235o49238b99u6cdbd3e5c8d6ebb7@mail.gmail.com","subject":"Re: Reporting bugs and bisection","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-04-17T19:57:06Z","receivedAt":"2008-04-17T19:57:06Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Thu, Apr 17, 2008 at 9:35 PM, Ray Lee wrote:\n> On Thu, Apr 17, 2008 at 12:09 PM, Rafael J. Wysocki wrote:\n> > > Finger-pointing, in these extreme cases, gives incentive to improve\n> > > quality. It's a positive thing.\n> >\n> > Sorry, but I have to disagree. Negative finger-pointing is never a good thing.\n>\n> Correct, but let's be careful here. The original suggestion was,\n> effectively, to get better metrics on the quality of contributions.\n> Those metrics *could* be used for finger pointing, or (my preference)\n> they could be used to direct and allocate our scarce resources: code\n> reviews and mentoring.\n\nExactly!\n\n> There's no way to know what the metrics will tell us until we have\n> them. Arguing against metrics because they *may* be used to point\n> fingers at people is a silly argument; anything can be subverted to do\n> that.\n\nThank you, that should have been said before, you worded it perfectly.\n\n> Let's get some measurements and see what they say. In the meantime,\n> try to believe that they could be put to good purposes, such as\n> identifying code areas that are tricky for contributors to get right\n> (independent of contributor), or contributors that could benefit from\n> code reviews, etc.\n\nThis especially is an area that I plan to focus on and should be very\nreliable when finished. As can be read in my application, I plan to\nlook at how often a piece of code is changed, in what timespan and by\nhow many different authors.\n\nThanks for the reply!\n\nCheers,\n\nSverre\n"},{"id":"74667","messageId":"20080417201657.GF27459@ZenIV.linux.org.uk","threadId":"13099","inReplyTo":"2c0942db0804171235o49238b99u6cdbd3e5c8d6ebb7@mail.gmail.com","subject":"Re: Reporting bugs and bisection","fromName":"Al Viro","fromEmail":"viro@zeniv.linux.org.uk","sentAt":"2008-04-17T20:16:57Z","receivedAt":"2008-04-17T20:16:57Z","isPatch":false,"sender":{"key":"viro@zeniv.linux.org.uk","avatar":null},"body":"On Thu, Apr 17, 2008 at 12:35:12PM -0700, Ray Lee wrote:\n> On Thu, Apr 17, 2008 at 12:09 PM, Rafael J. Wysocki wrote:\n> > > Finger-pointing, in these extreme cases, gives incentive to improve\n> > > quality. It's a positive thing.\n> >\n> > Sorry, but I have to disagree. Negative finger-pointing is never a good thing.\n> \n> Correct, but let's be careful here. The original suggestion was,\n> effectively, to get better metrics on the quality of contributions.\n\n\tThere already is one: reputation with people working on the tree,\nbe it actively modifying/reviewing/bug hunting/etc. _We_ _already_ _know_;\ngenerally one gets a decent idea of what to expect pretty soon.\n\n\tAnd frankly, that's the only thing that matters anyway; I suspect\nI'd do rather well by proposed criteria, but you know what? I don't give\na flying f*ck through the rolling doughnut for self-appointed PHBs and\ntheir idea of performance reviews.\n\n\tThink of it as a modified Turing test: convince me that you are\nnot a script piped through an Eng.Lit. wanker or an MBA, then I might care\nfor your opinion.\n\n\tAl, who never had problems with pointing fingers and laughing, but\nlikes an informed human brain to be the source of it...\n"},{"id":"74668","messageId":"2c0942db0804171338p7bc7d9f2u8079c2f8c8998e76@mail.gmail.com","threadId":"13099","inReplyTo":"20080417201657.GF27459@ZenIV.linux.org.uk","subject":"Re: Reporting bugs and bisection","fromName":"Ray Lee","fromEmail":"ray-lk@madrabbit.org","sentAt":"2008-04-17T20:38:18Z","receivedAt":"2008-04-17T20:38:18Z","isPatch":false,"sender":{"key":"ray-lk@madrabbit.org","avatar":null},"body":"On Thu, Apr 17, 2008 at 1:16 PM, Al Viro wrote:\n> On Thu, Apr 17, 2008 at 12:35:12PM -0700, Ray Lee wrote:\n> > On Thu, Apr 17, 2008 at 12:09 PM, Rafael J. Wysocki wrote:\n> > > > Finger-pointing, in these extreme cases, gives incentive to improve\n> > > > quality. It's a positive thing.\n> > >\n> > > Sorry, but I have to disagree. Negative finger-pointing is never a good thing.\n> >\n> > Correct, but let's be careful here. The original suggestion was,\n> > effectively, to get better metrics on the quality of contributions.\n>\n> There already is one: reputation with people working on the tree,\n> be it actively modifying/reviewing/bug hunting/etc. _We_ _already_ _know_;\n\nSigh. No, you already know. I don't. This is not a rhetorical point.\nI've just bid out another project that'd involve getting linux running\non another embedded hardware platform. If that happens, I get to spend\npaid time to work on the kernel, and as a by-product spend more time\nlooking at patches and code coming across the list.\n\nSo, where would it be best to spend my time? Or anyone else's?\n\n> generally one gets a decent idea of what to expect pretty soon.\n>\n> And frankly, that's the only thing that matters anyway; I suspect\n> I'd do rather well by proposed criteria, but you know what? I don't give\n> a flying f*ck through the rolling doughnut for self-appointed PHBs and\n> their idea of performance reviews.\n\n(Geez, conflate the issue much?) No one is saying you should. But\nalso, I haven't seen anyone saying it'd be used for performance\nreviews other than you.\n\n> Think of it as a modified Turing test: convince me that you are\n> not a script piped through an Eng.Lit. wanker or an MBA, then I might care\n> for your opinion.\n\n Shockingly enough, I actually don't care. I'm just trying to\nscratch my own itch, which is figure out where in the kernel (if\nanywhere!) it'd be best to donate my time.\n\nAnd your point is likely about the metrics, and yes, they'll be\ncomputer generated. So? Perhaps they'll be crap. Who knows until we\nlook at them and match them up with what everyone already knows? If,\nby some one in a thousand chance, they turn out to be good and useful,\nthen it'll either be a one-off eye-opener, or perhaps something useful\nmore than once.\n\nWho knows? And to the larger point, why put effort into stopping\nsomeone else from finding out?\n\n> Al, who never had problems with pointing fingers and laughing, but\n> likes an informed human brain to be the source of it...\n\n Shame and Guilt, two major motivators of human behavior, it's\ntrue. But, one last time, *you're* the one saying the stats would be\nused for finger pointing at people. Perhaps, instead, the stats will\nshow that we should all collectively point our fingers at some random\narea in the tree, where everyone, despite their track record, ends up\nmaking mistakes.\n\nLet the kid find out, that's all I'm saying.\n"},{"id":"74670","messageId":"20080417205301.GG27459@ZenIV.linux.org.uk","threadId":"13099","inReplyTo":"2c0942db0804171338p7bc7d9f2u8079c2f8c8998e76@mail.gmail.com","subject":"Re: Reporting bugs and bisection","fromName":"Al Viro","fromEmail":"viro@zeniv.linux.org.uk","sentAt":"2008-04-17T20:53:01Z","receivedAt":"2008-04-17T20:53:01Z","isPatch":false,"sender":{"key":"viro@zeniv.linux.org.uk","avatar":null},"body":"On Thu, Apr 17, 2008 at 01:38:18PM -0700, Ray Lee wrote:\n> > And frankly, that's the only thing that matters anyway; I suspect\n> > I'd do rather well by proposed criteria, but you know what? I don't give\n> > a flying f*ck through the rolling doughnut for self-appointed PHBs and\n> > their idea of performance reviews.\n> \n> (Geez, conflate the issue much?) No one is saying you should. But\n> also, I haven't seen anyone saying it'd be used for performance\n> reviews other than you.\n\n|| If there are individuals at whom a finger needs to be pointed, this \n|| system will highlight them, and fingers will (and should) be pointed.\n|| Contributors of poor-quality code need to be weeded-out.\n\nin this thread (From: David Newall).\n\n> Shame and Guilt, two major motivators of human behavior, it's\n> true. But, one last time, *you're* the one saying the stats would be\n> used for finger pointing at people.\n\nNot really. Unless you are trying to imply that David is my sock puppet, that\nis...\n"},{"id":"74672","messageId":"2c0942db0804171401r2696884bq22540deaab40ef9b@mail.gmail.com","threadId":"13099","inReplyTo":"20080417205301.GG27459@ZenIV.linux.org.uk","subject":"Re: Reporting bugs and bisection","fromName":"Ray Lee","fromEmail":"ray-lk@madrabbit.org","sentAt":"2008-04-17T21:01:55Z","receivedAt":"2008-04-17T21:01:55Z","isPatch":false,"sender":{"key":"ray-lk@madrabbit.org","avatar":null},"body":"On Thu, Apr 17, 2008 at 1:53 PM, Al Viro wrote:\n> On Thu, Apr 17, 2008 at 01:38:18PM -0700, Ray Lee wrote:\n> > > And frankly, that's the only thing that matters anyway; I suspect\n> > > I'd do rather well by proposed criteria, but you know what? I don't give\n> > > a flying f*ck through the rolling doughnut for self-appointed PHBs and\n> > > their idea of performance reviews.\n> >\n> > (Geez, conflate the issue much?) No one is saying you should. But\n> > also, I haven't seen anyone saying it'd be used for performance\n> > reviews other than you.\n>\n>\n> || If there are individuals at whom a finger needs to be pointed, this\n> || system will highlight them, and fingers will (and should) be pointed.\n> || Contributors of poor-quality code need to be weeded-out.\n>\n> in this thread (From: David Newall).\n\nAh, I failed reading comprehension, yet again. Well, sounds like you\nhave a beef to take up with David, then. That's still not an argument\nagainst trying to gather statistics and to see if they're worth\nanything.\n\n> > Shame and Guilt, two major motivators of human behavior, it's\n> > true. But, one last time, *you're* the one saying the stats would be\n> > used for finger pointing at people.\n>\n> Not really. Unless you are trying to imply that David is my sock puppet, that\n> is...\n\nMomentarily amusing to think so, but no :-).\n"}]}