Measuring Community Involvement (was Re: Contributor Summit planning)

From: Derrick Stolee <stolee@gmail.com>
Date: Aug 14, 2018, 17:43 UTC
Message-ID: <3f0da3cd-0929-4aad-2c50-92c9d6c660e5@gmail.com>
In-Reply-To: <20180813215431.GB16006@sigill.intra.peff.net>

On 8/13/2018 5:54 PM, Jeff King wrote:

> So I try not to think too hard on metrics, and just use them to get a
> rough view on who is active.

I've been very interested in measuring community involvement, with the knowledge that any metric is flawed and we should not ever say "this metric is how we measure the quality of a contributor". It can be helpful, though, to track some metrics and their change over time.

Here are a few measurements we can make:

1. Number of (non-merge) commit author tag-lines.

     using git repo:

   > git shortlog --no-merges --since 2017 -sne junio/next | head -n 20
    284  Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
    257  Jeff King <peff@peff.net>
    206  Stefan Beller <stefanbeller@gmail.com>
    192  brian m. carlson <sandals@crustytoothpaste.net>
    159  Brandon Williams <bmwill@google.com>
    149  Junio C Hamano <gitster@pobox.com>
    137  Elijah Newren <newren@gmail.com>
    116  René Scharfe <l.s.r@web.de>
    112  Johannes Schindelin <Johannes.Schindelin@gmx.de>
    105  Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     96  Jonathan Tan <jonathantanmy@google.com>
     93  SZEDER Gábor <szeder.dev@gmail.com>
     78  Derrick Stolee <dstolee@microsoft.com>
     76  Martin Ågren <martin.agren@gmail.com>
     66  Michael Haggerty <mhagger@alum.mit.edu>
     61  Eric Sunshine <sunshine@sunshineco.com>
     46  Christian Couder <chriscool@tuxfamily.org>
     36  Phillip Wood <phillip.wood@dunelm.org.uk>
     35  Jonathan Nieder <jrnieder@gmail.com>
     33  Thomas Gummerer <t.gummerer@gmail.com>

2. Number of other commit tag-lines (Reviewed-By, Helped-By, 
Reported-By, etc.).

     Using git repo:

     $ git log --since=2018-01-01 junio/next|grep by:|grep -v 
Signed-off-by:|sort|uniq -c|sort -nr|head -n 20

      66     Reviewed-by: Stefan Beller <sbeller@google.com>
      22     Reviewed-by: Jeff King <peff@peff.net>
      19     Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
      12     Helped-by: Eric Sunshine <sunshine@sunshineco.com>
      11     Helped-by: Junio C Hamano <gitster@pobox.com>
       9     Helped-by: Jeff King <peff@peff.net>
       8     Reviewed-by: Elijah Newren <newren@gmail.com>
       7     Reported-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
       7     Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
       7     Acked-by: Brandon Williams <bmwill@google.com>
       6     Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>
       6     Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
       5     Mentored-by: Christian Couder <christian.couder@gmail.com>
       5     Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
       4     Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
       4     Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
       4     Helped-by: Stefan Beller <sbeller@google.com>
       4     Helped-by: René Scharfe <l.s.r@web.de>
       3     Reviewed-by: Martin Ågren <martin.agren@gmail.com>
       3     Reviewed-by: Lars Schneider <larsxschneider@gmail.com>

     (There does not appear to be enough density here to make a useful 
metric.)

3. Number of email messages sent.

     Using mailing list repo:

$ git shortlog --since 2017 -sne | head -n 20
   3749  Junio C Hamano <gitster@pobox.com>
   2213  Stefan Beller <sbeller@google.com>
   2112  Jeff King <peff@peff.net>
   1106  Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
   1028  Johannes Schindelin <Johannes.Schindelin@gmx.de>
    965  Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    956  Brandon Williams <bmwill@google.com>
    947  Eric Sunshine <sunshine@sunshineco.com>
    890  Elijah Newren <newren@gmail.com>
    753  brian m. carlson <sandals@crustytoothpaste.net>
    677  Duy Nguyen <pclouds@gmail.com>
    646  Jonathan Nieder <jrnieder@gmail.com>
    629  Derrick Stolee <stolee@gmail.com>
    545  Christian Couder <christian.couder@gmail.com>
    515  Jonathan Tan <jonathantanmy@google.com>
    425  Johannes Schindelin <johannes.schindelin@gmx.de>
    425  Martin Ågren <martin.agren@gmail.com>
    420  Jeff Hostetler <git@jeffhostetler.com>
    420  SZEDER Gábor <szeder.dev@gmail.com>
    363  Phillip Wood <phillip.wood@talktalk.net>

3. Number of threads started by user.

     (For this and the measurements below, I imported emails into a SQL 
table with columns [commit, author, date, message-id, in-reply-to, 
subject] and ran queries)

SELECT TOP 20
        COUNT(*) as NumSent
       ,[Author]
   FROM [git].[dbo].[mailing-list]
   WHERE [In-Reply-To] = ''
         AND CONVERT(DATETIME,[Date]) > CONVERT(DATETIME, '01-01-2018 
00:00')
GROUP BY [Author]
ORDER BY NumSent DESC

| NumSent | Author                     |
|---------|----------------------------|
| 76      | Junio C Hamano             |
| 64      | Stefan Beller              |
| 54      | Philip Oakley              |
| 50      | Nguyá»…n ThÃ¡i Ngá»c Duy   |
| 49      | Robert P. J. Day           |
| 47      | Christian Couder           |
| 36      | Ramsay Jones               |
| 34      | Elijah Newren              |
| 34      | SZEDER GÃ¡bor              |
| 33      | Johannes Schindelin        |
| 31      | Jeff King                  |
| 30      | Ã†var ArnfjÃ¶rÃ° Bjarmason |
| 24      | Jonathan Tan               |
| 22      | Alban Gruin                |
| 22      | brian m. carlson           |
| 18      | Randall S. Becker          |
| 15      | Paul-Sebastian Ungureanu   |
| 15      | Jeff Hostetler             |
| 15      | Brandon Williams           |
| 15      | Luke Diamand               |

4. Number of threads where the user participated

(This is measured by completing the transitive closure of In-Reply-To edges into a new 'BaseMessage' column.)

SELECT TOP 20
        COUNT(BaseMessage) as NumResponded
       ,Author
   FROM [git].[dbo].[mailing-list]
   WHERE [In-Reply-To] <> ''
         AND CONVERT(DATETIME,[Date]) > CONVERT(DATETIME, '01-01-2018 
00:00')
GROUP BY Author
ORDER BY NumResponded DESC

| NumResponded | Author                     |
|--------------|----------------------------|
| 2084         | Junio C Hamano             |
| 1596         | Stefan Beller              |
| 1211         | Jeff King                  |
| 1120         | Johannes Schindelin        |
| 1021         | Nguyá»…n ThÃ¡i Ngá»c Duy   |
| 799          | Eric Sunshine              |
| 797          | Ã†var ArnfjÃ¶rÃ° Bjarmason |
| 693          | Brandon Williams           |
| 654          | Duy Nguyen                 |
| 600          | Elijah Newren              |
| 593          | brian m. carlson           |
| 591          | Derrick Stolee             |
| 318          | SZEDER GÃ¡bor              |
| 299          | Jonathan Tan               |
| 286          | Christian Couder           |
| 263          | Jonathan Nieder            |
| 257          | Phillip Wood               |
| 256          | Derrick Stolee             |
| 238          | Taylor Blau                |
| 216          | Martin Ã…gren              |

(Note, some names have not been de-duplicated across multiple email addresses, but the email addresses are removed from these tables since I'm using a markdown generator that strips the emails in < >.)

If you have other ideas for fun measurements, then please let me know.

Thanks,

-Stolee