Re: git-fast-export hg mutt (24M vs 184M)

Hello, I just tried to vendor track / import the mutt hg repository into git. git-fast-export-hg is quiet amazing but the resulting git repository blows up in size and I have not the slightes clue why.

        git clone git://repo.or.cz/fast-export.git
        hg clone http://dev.mutt.org/hg/mutt

Could someone have a look at this? I used Debian Etch with a git version I build myself using an ugly script. I had to upgrade mecurial as well because the version didn't had the cmdlog.py. I used the unstable debian package for that.

        (thinkpad) [~/work/mutt] git-init-db
        Initialized empty Git repository in .git/
        (thinkpad) [~/work/mutt] ~/work/fast-export/hg-fast-export.sh -r /tmp/mutt

git version 1.5.2.rc0.56.g6169a

        Thomas

Hello, git-repack -a -d -f got it down to 19M. I missed the -f parameter before. Sorry for the noise.

        Thomas

On Thu, May 03, 2007 at 09:17:16PM +0200, Thomas Glanzmann wrote:

  You may want to use git gc that does that (and a bit more) for you.

Pierre Habouzit <madcoder@debian.org> wrote:

Actually, in this case, no.

git-gc by default doesn't use the -f option. -f to git-repack means "no reuse deltas". That particular feature of git-repack is basically required to be used after running git-fast-import with anything sizeable.

The reason you need -f is git-fast-import does not write optimally compressed blobs (file revisions) when it creates the packfile. Instead it does a reasonable best effort while using a minimum amount of memory. The Git packfiles get most of their compression benefits from being able to see all of a project's data at once; this is impossible in fast-import as we're only seeing a small part of the incoming data stream at any single point in time.

If you had a lot of tags imported you might want to also use `git pack-refs` (one of the chores that git-gc does), or `git pack-refs --all` if you have a lot of dangling branches imported. The other chores in git-gc aren't actually useful after running fast-import (reflog expire, prune, rerere gc).

On Thu, May 03, 2007 at 05:18:24PM -0400, Shawn O. Pearce wrote:

  okay, so why git fast-import does not let some note somewhere (to be
picked by git gc later) "a fast-import has been run, use -f for next
repack if you want best compression" ?

  I'd think that would make a lot of sense, and that users that now
naively (like me) think git-gc would always be enough would not be
dramatically wrong ? :)

  I mean it's nothing *very* important but some 
  `touch $GIT_DIR/info/unpacked-fast-import` in fast-import then:
  if test -f $GIT_DIR/info/unpacked-fast-import; then
      REPACK_OPTIONS=$REPACK_OPTIONS\ -f
  fi
  // do the repack
  rm -f $GIT_DIR/info/unpacked-fast-import

  would do the trick, wouldn't it ?

On Fri, 4 May 2007, Pierre Habouzit wrote:

Nah.

The conversion script should do it itself directly after it is done with fast-import.

Nicolas