Re: Is there a way to trim old SHAs from a git tree (so it's not so large)?

After doing a "make mrproper" in my Linux git tree, the result is still 1.1GB of files. Compare that with just the tarball, which is just one-forth the size.

Is there a way to "trim away" old commits from the repository, so that it just 
doesn't take up that much space?  I don't care about any commits made in 2005. 
  As long as I can still do "git pull" from the source repo to update mine, 
that's good enough.

Timur Tabi <timur@freescale.com> wrote:

No.  However...

Have you tried "git repack -a -d" to repack the loose objects into a pack file? Doing this every so often should reduce your disk space consumed by a HUGE amount.

Shawn Pearce wrote:

Woah!  It shrunk that bad boy down to 420M!  That should do it, thanks!

I wonder why the powerpc tree shrank so much. Do you think the maintainer just needs to run git-repack on his tree?

Timur Tabi <timur@freescale.com> wrote:

Possible, yes.

However published repositories don't tend to repack as often as it makes things harder for people who clone/fetch over HTTP rather than the native git protocol.

The reason is that HTTP can fetch individual loose objects that you don't have yet, but if the object is only available in a pack file then you need to fetch the entire pack file. But you might already have most of that pack file, so now you are downloading lots of data you already have. :-(

-------- Original-Nachricht --------
Datum: Wed, 15 Nov 2006 16:11:57 -0600
Von: Timur Tabi <timur@freescale.com>
An: git@vger.kernel.org
Betreff: Is there a way to trim old SHAs from a git tree (so it\'s not so large)?

Is it possible to do this with shallow clone? Shallow clone the local repository my.git (which should be trimmed) starting from the last needed commit to a new local repository my_trimmed.git. And then remove my.git (with something like rm -rf my.git) and rename my_trimmed.git to my.git?

Thomas

Thomas Kolejka wrote:

Maybe. How do you do a shallow clone? I tried "git clone" followed by "git-repack", and that helped a lot, but the result was still twice the size of a normal tarball. I don't see any "shallow" option to the clone command, and git-shallow-pack doesn't exist on my installation.

Timur Tabi <timur@freescale.com> wrote:

Twice the size of a normal tarball isn't too bad, considering that you have the _complete_ history in the pack and yet the normal tarball has no history at all.

Shallow clone is a development feature still being working on in Junio's 'pu' branch of git.git. It has a few issues still to be worked out so it hasn't been made part of one of the more stable branches yet (like 'next', 'master', or 'maint').

Shawn Pearce wrote:

That is true. However, for the particular project I'm working on, double the size is not really acceptable.

My goal is to provide a source tree that is "git enabled", so that user can use git command to fetch, apply, and create patches. Currently, we're just handing out tarballs, so we want to move people to the 21st century.

Well, until it's available on an official git release, it doesn't help me.

Timur Tabi <timur@freescale.com> wrote:

One of the reasons its hanging out in 'pu' still is that there is a lack of people who are interested in the feature, and thus not enough people are testing it. Perhaps you might be able to lend a hand in that regard?

Shawn Pearce wrote:

Sure, I can do that! Can you give me some pointers? I've never done development on git itself, so I don't know Junio or his pu (sorry, I couldn't resist :-)).

Timur Tabi <timur@freescale.com> wrote:

Junio C Hamano is the Git maintainer. His published Git repository is here:

	http://www.kernel.org/pub/scm/git/git.git/

though it oddly has pack files from Jun 2006 in the wrong directory. Weird. Anyway...

If you clone from that URL, or better though the native Git protocol:

	git://www.kernel.org/pub/scm/git/git.git

you will get a branch called `pu`, which is the set of "Proposed Updates" to Git that Junio and others are currently working on. You can then checkout a branch off that, and build it:

	git checkout -b pu-build pu
	make

finally you can either run from that directory (see INSTALL file) or you can install the binary somewhere else. We don't really recommend using `pu` for production level work, so make sure you have a backup of any repository you run it on. :)

Shawn Pearce wrote:

So how do I make a shallow clone? I've set it all up, but there is no git-shallow-clone command, and git help clone doesn't have anything either.

Hi,

On Tue, 21 Nov 2006, Timur Tabi wrote:

Try "git clone --depth 1 <url>". This will cut each ancestor chain after one ancestor (IIRC).

Ciao, Dscho

Johannes Schindelin wrote:

I think you mean git-clone.sh instead of git-clone. If I do the above command, I get:

$ ./git clone --depth 1 git://127.0.0.1/temp/u-boot-83xx/
Usage: /home/b04825/bin/git-clone [--template=<template_directory>] 
[--use-separate-remote] [--reference <reference-repo>] [--bare] [-l [-s]] [-q] 
[-u <upload-pack>] [--origin <name>] [-n] <repo> [<dir>]

However, git-clone.sh is not quite working either. I had to run git-daemon on my machine, because git-clone.sh doesn't like the http protocol, and my firewall blocks everything but that. So I cloned a repo, started git-daemon, and I tried this:

$ ./git-clone.sh --depth 1 git://127.0.0.1/temp/u-boot-83xx usage: git-fetch-pack [--all] [-q] [-v] [-k] [--thin] [--exec=upload-pack] [host:]directory <refs>... fetch-pack from 'git://127.0.0.1/temp/u-boot-83xx/' failed.

A regular git-clone of git://127.0.0.1/temp/u-boot-83xx works, so I think there's something wrong with git-clone.sh or my invocation thereof.

Timur Tabi wrote:

Erm, you have to compile git with "make bindir=$(pwd)" to run it from working directory. Or just install it somewhere not over git, like /usr/local/ or /home/local/.

Hi,

On Tue, 21 Nov 2006, Timur Tabi wrote:

No. If "git clone" does not work for you, the compilation failed. Do you run git in-place? Then you _have_ to set

	bindir=$(pwd)

in config.mak. Otherwise, the git wrapper (you wrote "git-clone", but then you used "git clone") will look in $(HOME)/bin for your executables, and I bet that you have the other (official) git installed there.

For now, shallow clones only work with the git protocol, i.e. via git-daemon and ssh.

Hth,

Johannes Schindelin wrote:

When I tried that, make spits out this:

GIT_VERSION = 1.4.4.ge1173-dirty
     * new build flags or prefix
(cd perl && /usr/bin/perl Makefile.PL \
         PREFIX='/home/b04825')
Writing Makefile for Git

So I tried this:

make configure ./configure --prefix=$PWD make

and got this:

     * new build flags or prefix
(cd perl && /usr/bin/perl Makefile.PL \
         PREFIX='/temp/git')
Writing Makefile for Git

which is better, but git-clone still doesn't work:

$ ./git clone --depth 1 git://127.0.0.1/temp/u-boot-83xx Failed to run command 'clone': Success

$ ./git-clone --depth 1 git://127.0.0.1/temp/u-boot-83xx usage: git-fetch-pack [--all] [-q] [-v] [-k] [--thin] [--exec=upload-pack] [host:]directory <refs>... fetch-pack from 'git://127.0.0.1/temp/u-boot-83xx' failed.

In both of these cases, the git-daemon process doesn't log anything.

I even tried "export GIT_DIR=$PWD", but that didn't do anything.

Timur Tabi wrote:

make configure ./configure --prefix=$PWD --bindir=$PWD make

Hi,

On Tue, 21 Nov 2006, Timur Tabi wrote:

Well, if you help it, it will be available on an official git release soon!

Ciao, Dscho