# Re: git-fast-export hg mutt (24M vs 184M) 6 messages from 2007-05-03 to 2007-05-04. Participants: Thomas Glanzmann, Pierre Habouzit, Shawn O. Pearce, Nicolas Pitre. Thread: https://gitlist.dev/t/7958 ## Thomas Glanzmann, 2007-05-03 18:56 Subject: git-fast-export hg mutt (24M vs 184M) Message-ID: <20070503185623.GA11817@cip.informatik.uni-erlangen.de> URL: https://gitlist.dev/e/20070503185623.GA11817%40cip.informatik.uni-erlangen.de ``` Hello, I just tried to vendor track / import the mutt hg repository into git. git-fast-export-hg is quiet amazing but the resulting git repository blows up in size and I have not the slightes clue why. git clone git://repo.or.cz/fast-export.git hg clone http://dev.mutt.org/hg/mutt Could someone have a look at this? I used Debian Etch with a git version I build myself using an ugly script. I had to upgrade mecurial as well because the version didn't had the cmdlog.py. I used the unstable debian package for that. (thinkpad) [~/work/mutt] git-init-db Initialized empty Git repository in .git/ (thinkpad) [~/work/mutt] ~/work/fast-export/hg-fast-export.sh -r /tmp/mutt git version 1.5.2.rc0.56.g6169a Thomas ``` ## Thomas Glanzmann, 2007-05-03 19:17 Subject: Re: git-fast-export hg mutt (24M vs 184M) Message-ID: <20070503191716.GB11817@cip.informatik.uni-erlangen.de> URL: https://gitlist.dev/e/20070503191716.GB11817%40cip.informatik.uni-erlangen.de In-Reply-To: <20070503185623.GA11817@cip.informatik.uni-erlangen.de> ``` Hello, git-repack -a -d -f got it down to 19M. I missed the -f parameter before. Sorry for the noise. Thomas ``` ## Pierre Habouzit, 2007-05-03 21:01 Subject: Re: git-fast-export hg mutt (24M vs 184M) Message-ID: <20070503210112.GE3260@artemis> URL: https://gitlist.dev/e/20070503210112.GE3260%40artemis In-Reply-To: <20070503191716.GB11817@cip.informatik.uni-erlangen.de> ``` On Thu, May 03, 2007 at 09:17:16PM +0200, Thomas Glanzmann wrote: > Hello, > git-repack -a -d -f got it down to 19M. I missed the -f parameter > before. Sorry for the noise. You may want to use git gc that does that (and a bit more) for you. -- ·O· Pierre Habouzit ··O madcoder@debian.org OOO http://www.madism.org ``` ## Shawn O. Pearce, 2007-05-03 21:18 Subject: Re: git-fast-export hg mutt (24M vs 184M) Message-ID: <20070503211824.GB16538@spearce.org> URL: https://gitlist.dev/e/20070503211824.GB16538%40spearce.org In-Reply-To: <20070503210112.GE3260@artemis> ``` Pierre Habouzit wrote: > On Thu, May 03, 2007 at 09:17:16PM +0200, Thomas Glanzmann wrote: > > Hello, > > git-repack -a -d -f got it down to 19M. I missed the -f parameter > > before. Sorry for the noise. > > You may want to use git gc that does that (and a bit more) for you. Actually, in this case, no. git-gc by default doesn't use the -f option. -f to git-repack means "no reuse deltas". That particular feature of git-repack is basically required to be used after running git-fast-import with anything sizeable. The reason you need -f is git-fast-import does not write optimally compressed blobs (file revisions) when it creates the packfile. Instead it does a reasonable best effort while using a minimum amount of memory. The Git packfiles get most of their compression benefits from being able to see all of a project's data at once; this is impossible in fast-import as we're only seeing a small part of the incoming data stream at any single point in time. If you had a lot of tags imported you might want to also use `git pack-refs` (one of the chores that git-gc does), or `git pack-refs --all` if you have a lot of dangling branches imported. The other chores in git-gc aren't actually useful after running fast-import (reflog expire, prune, rerere gc). -- Shawn. ``` ## Pierre Habouzit, 2007-05-03 22:29 Subject: Re: git-fast-export hg mutt (24M vs 184M) Message-ID: <20070503222946.GH3260@artemis> URL: https://gitlist.dev/e/20070503222946.GH3260%40artemis In-Reply-To: <20070503211824.GB16538@spearce.org> ``` On Thu, May 03, 2007 at 05:18:24PM -0400, Shawn O. Pearce wrote: > Pierre Habouzit wrote: > > On Thu, May 03, 2007 at 09:17:16PM +0200, Thomas Glanzmann wrote: > > > Hello, > > > git-repack -a -d -f got it down to 19M. I missed the -f parameter > > > before. Sorry for the noise. > > > > You may want to use git gc that does that (and a bit more) for you. > > Actually, in this case, no. > > git-gc by default doesn't use the -f option. -f to git-repack > means "no reuse deltas". That particular feature of git-repack is > basically required to be used after running git-fast-import with > anything sizeable. okay, so why git fast-import does not let some note somewhere (to be picked by git gc later) "a fast-import has been run, use -f for next repack if you want best compression" ? I'd think that would make a lot of sense, and that users that now naively (like me) think git-gc would always be enough would not be dramatically wrong ? :) I mean it's nothing *very* important but some `touch $GIT_DIR/info/unpacked-fast-import` in fast-import then: if test -f $GIT_DIR/info/unpacked-fast-import; then REPACK_OPTIONS=$REPACK_OPTIONS\ -f fi // do the repack rm -f $GIT_DIR/info/unpacked-fast-import would do the trick, wouldn't it ? -- ·O· Pierre Habouzit ··O madcoder@debian.org OOO http://www.madism.org ``` ## Nicolas Pitre, 2007-05-04 01:11 Subject: Re: git-fast-export hg mutt (24M vs 184M) Message-ID: URL: https://gitlist.dev/e/alpine.LFD.0.99.0705032109070.24220%40xanadu.home In-Reply-To: <20070503222946.GH3260@artemis> ``` On Fri, 4 May 2007, Pierre Habouzit wrote: > On Thu, May 03, 2007 at 05:18:24PM -0400, Shawn O. Pearce wrote: > > Pierre Habouzit wrote: > > > On Thu, May 03, 2007 at 09:17:16PM +0200, Thomas Glanzmann wrote: > > > > Hello, > > > > git-repack -a -d -f got it down to 19M. I missed the -f parameter > > > > before. Sorry for the noise. > > > > > > You may want to use git gc that does that (and a bit more) for you. > > > > Actually, in this case, no. > > > > git-gc by default doesn't use the -f option. -f to git-repack > > means "no reuse deltas". That particular feature of git-repack is > > basically required to be used after running git-fast-import with > > anything sizeable. > > okay, so why git fast-import does not let some note somewhere (to be > picked by git gc later) "a fast-import has been run, use -f for next > repack if you want best compression" ? Nah. The conversion script should do it itself directly after it is done with fast-import. Nicolas ```