git fsck, badDate: invalid author/committer line - bad date

While preparing changes for git://git.annexia.org/virt-top.git, git push to github complains:

remote: error: object c609265ccce27a902b850f5d62e6628710ee2928: badDate: invalid author/committer line - bad date

Also 'git fsck' is unhappy about a number of commits.

What is wrong with these commits, and how can the maintainer of virt-top.git repository fix his repo?

$  git  --no-pager  log -p -M --stat --pretty=fuller c609265ccce27a902b850f5d62e6628710ee2928^! | head
commit c609265ccce27a902b850f5d62e6628710ee2928
Author:     Richard W.M. Jones <rjones@redhat.com>
AuthorDate: Mon Oct 5 11:39:28 2009 +0100
Commit:     Richard W.M. Jones <rjones@redhat.com>
CommitDate: Mon Oct 5 11:39:28 2009 +0100

Olaf

On Thu, Oct 31, 2019 at 09:27:49AM +0100, Olaf Hering wrote:

'git log' doesn't show the commit object's headers as-is, but 'git cat-file' does:

  $ git cat-file -p c609265ccce27a902b850f5d62e6628710ee2928
  tree ea8e3bd64e67d159e706b8feccfc169f6ac0829d
  parent db4e80ee11a4e212a97efc1761ed237c7da72cb1
  author Richard W.M. Jones <rjones@redhat.com> <"Richard W.M. Jones <rjones@redhat.com>"> 1254739168 +0100
  committer Richard W.M. Jones <rjones@redhat.com> <"Richard W.M. Jones <rjones@redhat.com>"> 1254739168 +0100
  
  New translations.

And now you don't need me to tell you what triggers those errors.

You can tell 'git fsck' to ignore those badly formatted commits locally using the 'fsck'skipList' configuration variable:

  $ echo "# Commits with bogus author/committer lines" >.git-fsck-skiplist
  $ git fsck 2>&1 |cut -d' ' -f4 |sed -e 's/://' >>.git-fsck-skiplist
  $ git config fsck.skipList .git-fsck-skiplist
  $ git fsck
  Checking object directories: 100% (256/256), done.
  Checking objects: 100% (1803/1803), done.
  $ echo $?
  0

It might be worth to commit this file, so other devs can easily start using it as well, and don't have to maintain such a list themselves. (Note, however, that if this file is committed and the config variable is set, then attempting to run 'git fsck' while an older revision is checked out will result in a 'fatal: could not open object name list: .git-fsck-skiplist' error; I think that's an acceptable trade-off.)

However, I don't know how to tell about the skiplist file to GitHub, or any other Git hosting service for that matter.

Am Thu, 31 Oct 2019 11:15:39 +0100 schrieb SZEDER Gábor <szeder.dev@gmail.com>:

Thanks for all the details.

Is there a way to "replay" a git repository, so that all the commit contents and author/committer data are preserved? I think it is more important to have a clean repository than to preserve irrelevant commit hashes.

Olaf

On Thu, Oct 31, 2019 at 02:21:18PM +0100, Olaf Hering wrote:

Those commits can be fixed by simply transforming the fast-export stream, e.g.:

  $ git init new
  $ git -C virt-top/ fast-export --all |
    sed -e '/^\(author\|committer\) Richard W\.M\. Jones <rjones@redhat.com> </ s/<"Richard W\.M\. Jones <rjones@redhat.com>"> //' |
    git -C new fast-import

BUT!

All the usual warnings about rewriting already published history apply. The hash of a couple of commits from 2009 might seem irrelevant now, a decade later, but after correcting those author and committer lines the hashes of all subsequent commits will inherently change as well. This is, in general, upsetting for everyone who have cloned the repo and built their own work on top. Furthermore, some commit messages refer to older commits by their hash (e.g. in 431dbd98ba: "Simplifies and updates commit dbef8dd3bf00417e75a12c851b053e49c9e1a79e"); those references will go stale after rewriting history, unless you put in extra work to update them.

I would advise against rewriting history.

On Thu, Oct 31, 2019 at 12:56 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:

For completeness for future readers of this thread who have cases when rewriting history is a valid option, a tool such as Elijah's git-filter-repo[1] can not only fix the "broken" email addresses, but also can adjust hash references in commit messages so they don't go stale.

[1]: https://github.com/newren/git-filter-repo

On Thu, Oct 31, 2019 at 11:15:39AM +0100, SZEDER Gábor wrote:

You can contact GitHub support. The usual procedure is:

  1. Find out if rewriting is an option (i.e., if the history isn't
     already widely published), and walk the user through cleaning up
     the history if that's an option.

  2. For established history, we'll sometimes relax the fsck checks
     temporarily to allow it to be pushed (it depends on the exact
     failing check, but this one is mostly cosmetic and would be OK).

-Peff