Removing redundant packs

  Hi,

  before exporting whole repo.or.cz over rsync for mirroring, I would
like to clean up the worst repositories (esp. linux-2.6 repositories)
whose objects database looks absolutely horrible now.

  At the same time, I have to be careful not to prune all unreferenced
objects since they can be referenced in forks.

  How would you go about it? My idea was to do a fresh cummulative pack
of all referenced objects with

	git repack -a

and then remove packs that contain _only_ objects that are already in the fresh pack. But I have trouble with the second step; a very naive grep loop ran for about a day when I stopped it and it was nowhere near the half of the packs yet. I didn't figure out how to make use of git-pack-redundant either.

  Does anyone have any hints?

  Thanks!

Petr Baudis <pasky@ucw.cz> writes:

How about...

 (1) In each repository, run "repack -a -d".  That would ensure that
     everybody has the necessary objects that they themselves need.  By
     doing this for a repository that borrows from another makes sure
     pruning the latter would not break the former, so you start from
     leaves and move on to the repositories they borrow from.

 (2) In each repository, run "repack -a -l -d".  By doing this for a
     repository that borrows from another repacks the former without
     objects it borrows from the latter, so the former repository will
     retain what it needs but will borrow what are common from the
     latter, so you start from the repository that does not borrow from
     anybody else, and move on to the repositories that borrow from the
     one you already run this step.

On Sat, Dec 08, 2007 at 01:37:44PM -0800, Junio C Hamano wrote:

I'm not sure this works. Try this:

# make parent repo with two commits, fully packed mkdir parent && cd parent && git init && echo content >file && git add file && git commit -m added && echo more >>file && git commit -m more -a && git repack -a -d

# clone child repo with alternates git clone -s . ../child

# and now forget about the second commit in the parent git reset --hard HEAD^ rm -rf .git/logs

# at this point the "parent" repo doesn't know anything about the second # commit, but contains the objects; the child repo does know about it, # but relies on the pack in the parent for the objects

# so your advice is to "repack -a -d" first in the child, then in the # parent. # child first; output will be: "Nothing new to pack." cd ../child && git repack -a -d # and then parent, which will lose the objects for the second commit cd ../parent && git repack -a -d

# and confirm that child is broken # output is: "fatal: bad object HEAD" cd ../child && PAGER=cat git show

You seem to think that repacking without "-l" will pull objects from alternates, but it doesn't (I have also seen "repack -a -d" recommended for "how do I break alternates dependencies", but it obviously also doesn't work for that).

-Peff

On Sat, Dec 08, 2007 at 06:44:15PM -0500, Jeff King wrote:

And by "you seem to think" I obviously don't mean "stupid n00b, you don't understand git" but "I'm unclear on what the behavior should be. You seem to be saying this, which means this is a serious bug."

-Peff