From: Kevin Ballard Date: Wed, 23 Jan 2008 02:58:09 GMT Subject: Re: git filter-branch should run git gc --auto Message-ID: <60D6C8F5-EF88-48F0-92CA-8E49838C0CB9@sb.org> In-Reply-To: <7v1w89qmw3.fsf@gitster.siamese.dyndns.org> On Jan 22, 2008, at 9:46 PM, Junio C Hamano wrote: > Kevin Ballard writes: > >> I just glanced at git-filter-branch.sh (and I must say I was >> incredibly surprised to find out it was a shell script) and it seems >> it never runs git-gc or git-repack. Doesn't that end up with the same >> problems as git-svn sans git-repack when filtering a large number of >> commits? I was just thinking, if I were to git-filter-branch on my >> massive repo (in fact, the same repo that started this thread, with >> over 33000 commits in the upstream svn repo), even if I just do >> something as simple as change the commit msg wont I end up with >> thousands of unreachable objects? I shudder to think how many >> unreachable objects I would have if I pruned the entire dports >> directory off of the tree. >> >> Am I missing something, or does git-filter-branch really not do any >> garbage collection? I tried reading the source, but complex bash >> scripts are almost as bad as perl in terms of readability. > > Theoretically yes, and it largely depends on what you do, but > filter-branch goes over the objects that already exists in your > repository, and hopefully you won't be rewriting majority of > them. > > So the impact of not repacking is probably much less painful in > practice. > > But again as I said, it largely depends on what you do in your > filter. If you are upcasing (or convert to NFD ;-)) the > contents of all of your blob objects, you would certainly want > to repack every once in a while. I'm actually considering what the cost would be of switching macports to git (not that it will ever happen - too many anonymous people pull from svn trunk). Right now the svn trunk contains a subfolder for the source code and another subfolder for all ~4400+ Portfiles. In such a theoretical move, I'd want to split that up, probably into two unrelated branches. Doing so would mean running git-filter-branch over a linear commit history that's 31580 objects long, with a tree filter to prune the dports directory away and a msg filter to remove the svn- id stuff that git-svn left behind. This means that every single commit objects would be changed, as well as the root tree object for every single commit. That would be about 63160 objects. I'd also have to figure out some way to remove the commit objects entirely that only reference the dports directory. Then I'd have to do it again with the opposite tree filter (to prune everything but the dports directory and move the contents of the dports directory up one level) and same msg filter. Granted, if I do the first action in a branch, that leaves no unreachable objects (since the originals are still referenced), but the second operation definitely would leave unreachable objects, and were I to clone the repository instead and do the operations in the different repos (which is perfectly legitimate - otherwise I'd have to clone it after everything else and then delete branches) then both actions would leave thousands of objects unreachable. I'd suggest a patch to run git gc --auto, but it looks like you just did in a subsequent email. As for your comments about the reflogs, can't I disable recording those, at least temporarily? I'd rather clean up after myself as I work rather than balloon the repository and collapse it in a single operation at the end. -Kevin Ballard -- Kevin Ballard http://kevin.sb.org kevin@sb.org http://www.tildesoft.com