threads / discuss / 14707

Cleaning up log messages

Subject: Cleaning up log messages

## tl;dr

10 messages between Jul 27, 2008 and Jul 27, 2008.

replies: 9people: 4as markdown or json

Jon Smirl· Jul 27, 2008, 17:50 UTC · lore

I was playing around with git log for the kernel and observed that there is a lot of noise when trying to do statistics on the number of commits.

For example:

Author: Greg K-H <gregkh@suse.de>
Author: Greg KH <gregkh@suse.de>
Author: Greg KH <greg@kroah.com>
Author: Greg KH <greg@press.(none)>
Author: gregkh@suse.de <gregkh@suse.de>
Author: Greg Kroah-Hartman <gregkh@suse>
Author: Greg Kroah-Hartman <gregkh@suse.de>
Author: Greg Kroah-Hartman <greg@kroah.com>

I don't see an obvious way to do this with git, but it would be neat to have a 'clean' option on git log that would take each email address (author, signed-off, acked, etc) and map it through a table which would convert old email addresses in to the current one and also standardize the formatting of the names. A cleaned log would be altered on display, but just don't clean it if you want the original.

Of course this initial map would need to be built by hand. New commits could be checked against the map and the mapped updated if the person really has a new email address. Checking new commits against the map would help clean things up going forward. checkpatch.pl could also validate against the mapping file.

No pressing need to for this, it would just be a nice toy.

-- 
Jon Smirl
jonsmirl@gmail.com

Johannes Schindelin· Jul 27, 2008, 18:01 UTC · re: Jon Smirl · lore

Re: Cleaning up log messages

Hi,

On Sun, 27 Jul 2008, Jon Smirl wrote:

Show 19 quoted lines

> I was playing around with git log for the kernel and observed that there 
> is a lot of noise when trying to do statistics on the number of commits.
> 
> For example:
> 
> Author: Greg K-H <gregkh@suse.de>
> Author: Greg KH <gregkh@suse.de>
> Author: Greg KH <greg@kroah.com>
> Author: Greg KH <greg@press.(none)>
> Author: gregkh@suse.de <gregkh@suse.de>
> Author: Greg Kroah-Hartman <gregkh@suse>
> Author: Greg Kroah-Hartman <gregkh@suse.de>
> Author: Greg Kroah-Hartman <greg@kroah.com>
> 
> I don't see an obvious way to do this with git, but it would be neat
> to have a 'clean' option on git log that would take each email address
> (author, signed-off, acked, etc) and map it through a table which
> would convert old email addresses in to the current one and also
> standardize the formatting of the names.

Something like .mailmap?

And to show the mapped author name instead of the committed one, you would use "--pretty=format:%aN"? (Needs 1.6.0-rc0 at least, IIRC)

Ciao, Dscho

Jon Smirl· Jul 27, 2008, 18:16 UTC · re: Johannes Schindelin · lore

Re: Cleaning up log messages

On 7/27/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 30 quoted lines

> Hi,
>
>
>  On Sun, 27 Jul 2008, Jon Smirl wrote:
>
>  > I was playing around with git log for the kernel and observed that there
>  > is a lot of noise when trying to do statistics on the number of commits.
>  >
>  > For example:
>  >
>  > Author: Greg K-H <gregkh@suse.de>
>  > Author: Greg KH <gregkh@suse.de>
>  > Author: Greg KH <greg@kroah.com>
>  > Author: Greg KH <greg@press.(none)>
>  > Author: gregkh@suse.de <gregkh@suse.de>
>  > Author: Greg Kroah-Hartman <gregkh@suse>
>  > Author: Greg Kroah-Hartman <gregkh@suse.de>
>  > Author: Greg Kroah-Hartman <greg@kroah.com>
>  >
>  > I don't see an obvious way to do this with git, but it would be neat
>  > to have a 'clean' option on git log that would take each email address
>  > (author, signed-off, acked, etc) and map it through a table which
>  > would convert old email addresses in to the current one and also
>  > standardize the formatting of the names.
>
>
> Something like .mailmap?
>
>  And to show the mapped author name instead of the committed one, you would
>  use "--pretty=format:%aN"?  (Needs 1.6.0-rc0 at least, IIRC)

So we can already do this? Where is a .mailmap for the kernel tree?

-- 
Jon Smirl
jonsmirl@gmail.com

Petr Baudis· Jul 27, 2008, 18:33 UTC · re: Jon Smirl · lore

Re: Cleaning up log messages

On Sun, Jul 27, 2008 at 02:16:30PM -0400, Jon Smirl wrote:

Show 7 quoted lines

> On 7/27/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > Something like .mailmap?
> >
> >  And to show the mapped author name instead of the committed one, you would
> >  use "--pretty=format:%aN"?  (Needs 1.6.0-rc0 at least, IIRC)
> 
> So we can already do this? Where is a .mailmap for the kernel tree?

	http://repo.or.cz/w/linux-2.6.git?a=blob;f=.mailmap

...right there. :-)

				Petr "Pasky" Baudis

Jon Smirl· Jul 27, 2008, 19:07 UTC · re: Petr Baudis · lore

Re: Cleaning up log messages

On 7/27/08, Petr Baudis <pasky@suse.cz> wrote:

Show 14 quoted lines

> On Sun, Jul 27, 2008 at 02:16:30PM -0400, Jon Smirl wrote:
>  > On 7/27/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>
> > > Something like .mailmap?
>  > >
>  > >  And to show the mapped author name instead of the committed one, you would
>  > >  use "--pretty=format:%aN"?  (Needs 1.6.0-rc0 at least, IIRC)
>  >
>  > So we can already do this? Where is a .mailmap for the kernel tree?
>
>
>         http://repo.or.cz/w/linux-2.6.git?a=blob;f=.mailmap
>
>  ...right there. :-)

I updated to 1.6.0-rc0 and this is working. mailmap needs some cleanup. Errors are still in the list, but this is a lot better than it was. That made about 800 'contributors' disappear.

Is there a way to do short log and have it map the names? What about replacing the emails with their current email address?

Random missing entries.... Greg KH Greg Kroah-Hartman

Hans J Koch Hans J. Koch

Jean-Christophe Dubois Jean-Christophe DUBOIS

Miguel Boton Miguel Botón

-- 
Jon Smirl
jonsmirl@gmail.com

Johannes Schindelin· Jul 27, 2008, 19:20 UTC · re: Jon Smirl · lore

Re: Cleaning up log messages

Hi,

On Sun, 27 Jul 2008, Jon Smirl wrote:

Show 20 quoted lines

> On 7/27/08, Petr Baudis <pasky@suse.cz> wrote:
> > On Sun, Jul 27, 2008 at 02:16:30PM -0400, Jon Smirl wrote:
> >  > On 7/27/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> >
> > > > Something like .mailmap?
> >  > >
> >  > >  And to show the mapped author name instead of the committed one, you would
> >  > >  use "--pretty=format:%aN"?  (Needs 1.6.0-rc0 at least, IIRC)
> >  >
> >  > So we can already do this? Where is a .mailmap for the kernel tree?
> >
> >         http://repo.or.cz/w/linux-2.6.git?a=blob;f=.mailmap
> >
> >  ...right there. :-)
> 
> I updated to 1.6.0-rc0 and this is working. mailmap needs some
> cleanup. Errors are still in the list, but this is a lot better than
> it was. That made about 800 'contributors' disappear.
> 
> Is there a way to do short log and have it map the names?

Yes, as of v1.6.0-rc0~58 you can pass --pretty=format: to shortlog.

> What about replacing the emails with their current email address?

Nope, that was never meant to be done.

Ciao, Dscho

Jon Smirl· Jul 27, 2008, 19:31 UTC · re: Johannes Schindelin · lore

Re: Cleaning up log messages

On 7/27/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 28 quoted lines

> Hi,
>
>
>  On Sun, 27 Jul 2008, Jon Smirl wrote:
>
>  > On 7/27/08, Petr Baudis <pasky@suse.cz> wrote:
>  > > On Sun, Jul 27, 2008 at 02:16:30PM -0400, Jon Smirl wrote:
>  > >  > On 7/27/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>  > >
>  > > > > Something like .mailmap?
>  > >  > >
>  > >  > >  And to show the mapped author name instead of the committed one, you would
>  > >  > >  use "--pretty=format:%aN"?  (Needs 1.6.0-rc0 at least, IIRC)
>  > >  >
>  > >  > So we can already do this? Where is a .mailmap for the kernel tree?
>  > >
>  > >         http://repo.or.cz/w/linux-2.6.git?a=blob;f=.mailmap
>  > >
>  > >  ...right there. :-)
>  >
>  > I updated to 1.6.0-rc0 and this is working. mailmap needs some
>  > cleanup. Errors are still in the list, but this is a lot better than
>  > it was. That made about 800 'contributors' disappear.
>  >
>  > Is there a way to do short log and have it map the names?
>
>
> Yes, as of v1.6.0-rc0~58 you can pass --pretty=format: to shortlog.

How do you do it with git log? --pretty overrides the default of medium

--pretty[=<format>]

    Pretty-print the contents of the commit logs in a given format,
where <format> can be one of oneline, short, medium, full, fuller,
email, raw and format:<string>. When omitted, the format defaults to
medium.

Show 11 quoted lines

>
>
>  > What about replacing the emails with their current email address?
>
>
> Nope, that was never meant to be done.
>
>  Ciao,
>  Dscho
>
>

-- 
Jon Smirl
jonsmirl@gmail.com

Johannes Schindelin· Jul 27, 2008, 20:16 UTC · re: Jon Smirl · lore

Re: Cleaning up log messages

Hi,

On Sun, 27 Jul 2008, Jon Smirl wrote:

Show 8 quoted lines

> How do you do it with git log? --pretty overrides the default of medium
> 
> --pretty[=<format>]
> 
>     Pretty-print the contents of the commit logs in a given format,
> where <format> can be one of oneline, short, medium, full, fuller,
> email, raw and format:<string>. When omitted, the format defaults to
> medium.

You get it _almost_ with

$ f='commit %H%nAuthor: %aN <%ae>%nDate: %ad%n%n%s%n%n%b' $ git log --pretty="format:$f"

The only difference being that the commit message is not indented. If you really need that, it is easy to add.

But I rather doubt that you need it, as you want to make statistics, and therefore need to pipe the output into a script anyway.

Ciao, Dscho

Junio C Hamano· Jul 27, 2008, 18:47 UTC · re: Jon Smirl · lore

Re: Cleaning up log messages

"Jon Smirl" <jonsmirl@gmail.com> writes:

Show 10 quoted lines

> I was playing around with git log for the kernel and observed that
> there is a lot of noise when trying to do statistics on the number of
> commits.
>
> For example:
>
> Author: Greg K-H <gregkh@suse.de>
> Author: Greg KH <gregkh@suse.de>
> ...
> Author: Greg Kroah-Hartman <greg@kroah.com>

We have had .mailmap since a24e658 (git-shortlog: make the mailmap configurable., 2005-10-06); maybe the kernel tree wants a maintainer for the .mailmap file?

Jon Smirl· Jul 27, 2008, 20:52 UTC · re: Junio C Hamano · lore

Re: Cleaning up log messages

On 7/27/08, Junio C Hamano <gitster@pobox.com> wrote:

Show 18 quoted lines

> "Jon Smirl" <jonsmirl@gmail.com> writes:
>
>  > I was playing around with git log for the kernel and observed that
>  > there is a lot of noise when trying to do statistics on the number of
>  > commits.
>  >
>  > For example:
>  >
>  > Author: Greg K-H <gregkh@suse.de>
>  > Author: Greg KH <gregkh@suse.de>
>  > ...
>
> > Author: Greg Kroah-Hartman <greg@kroah.com>
>
>
> We have had .mailmap since a24e658 (git-shortlog: make the mailmap
>  configurable., 2005-10-06); maybe the kernel tree wants a maintainer for
>  the .mailmap file?

This seems to be the main problem. There are so many missing entries from the .mailmap file that I didn't think this feature was implemented. I'd guestimate that 300-400 needed entries are missing.

I've made a few attempts at writing a script to fix the easy ones but I don't have a good solution yet.

-- 
Jon Smirl
jonsmirl@gmail.com