Re: Only 27% of reviewed-by tags are explicit, and much more

From: Felipe Contreras <felipe.contreras@gmail.com>
Date: Jun 18, 2021, 16:00 UTC
Message-ID: <60ccc3332428c_5eaa2208df@natae.notmuch>
In-Reply-To: <CAPig+cTr2fEYN2y7FyKaspuaCpAq6wNtHgg_8RcrOmvsN5+XaA@mail.gmail.com>

Eric Sunshine wrote:

Show 15 quoted lines

> On Thu, Jun 17, 2021 at 11:47 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> > This prompted me to write a script [6] to programmatically find statistics
> > about these trailers. Obviously it isn't perfect (as all software); it
> > tries to avoid human fuzziness (like people pasting other patches with
> > scissors [-- >8 --], or just straight put pasting the patch [^From: ]), but
> > even so there are instances I manually had to skip [7].
> >
> > Here are the top 20 reviewers over the past 10 years with their
> > corresponding explicit over total Reviewed-by count:
> >    ...
> >   5. Eric Sunshine: 14% (17/116)
> 
> Does your script check cover letters? Based upon a quick glance at it,
> it doesn't seem to.

Not really, that's a good point.

Show 8 quoted lines

> Although I've reviewed thousands of patches over the years, I almost
> never give my Reviewed-by:; it is an exceedingly rare occurrence.
> However, when I do give it, it's almost always in response to the
> cover letter (saying "this entire series is reviewed by <me>"), not in
> response to individual patches. I've seen other reviewers do so, as
> well. So, if your script doesn't take cover letters into account, then
> you might want to revise it to do so in order to get a more accurate
> picture.

I've updated the script to consider all responses to the cover letter that start with 'Re: '.

> In fact, if my memory is correct, some reviewers give their
> Reviewed-by: to an entire series in response to one of the patches
> rather than to the cover letter, so perhaps you can come up with a
> heuristic to identify those cases too.

That's true. Depending whether or not that's the exception or the rule it might make sense to simply consider all Reviewed-by responses to apply to the entire series and make the heuristic match the cases where it's only for a single patch.

Anyway, with the updated script the explicit reviewed-bys are 40%, and here are the stats:

  1. Jonathan Nieder: 80% (254/314)
  2. Jeff King: 7% (18/248)
  3. Stefan Beller: 28% (54/190)
  4. Matthieu Moy: 64% (84/131)
  5. Eric Sunshine: 38% (45/116)
  6. Derrick Stolee: 11% (12/102)
  7. Taylor Blau: 46% (39/83)
  8. Michael Haggerty: 76% (42/55)
  9. Elijah Newren: 25% (12/47)
  10. Johannes Schindelin: 11% (4/35)
  11. Jonathan Tan: 28% (9/32)
  12. Nguyễn Thái Ngọc Duy: 20% (6/30)
  13. Ronnie Sahlberg: 100% (16/16)
  14. SZEDER Gábor: 0% (0/14)
  15. Luke Diamand: 7% (1/13)
  16. Felipe Contreras: 8% (1/12)
  17. Johannes Sixt: 40% (4/10)
  18. Ævar Arnfjörð Bjarmason: 22% (2/9)
  19. Stefano Lattarini: 37% (3/8)
  20. Torsten Bögershausen: 0% (0/7)

Jonathan Nieder, Matthieu Moy, Michael Haggerty and Ronnie Sahlberg got considerably more percentage, but others didn't.

The histogram [1] shows an increase in the 60%-100% range, in particular the 80% range (thanks to Jonathan Nieder), but there's still plenty below 50%.

You got considerably more, from 17 to 45, but still pretty far from a 100%.

I think the conclussion still stands: Reviewed-by isn't always expressly given, in fact, the majority of case it isn't.

Cheers.

[1] https://i.imgur.com/gr6YjsZ.png

-- 
Felipe Contreras