Re: Dump http servers still slow?

From: Junio C Hamano <junkio@cox.net>
Date: Jul 31, 2005, 06:51 UTC
Message-ID: <7v1x5ftqn5.fsf@assigned-by-dhcp.cox.net>
In-Reply-To: <1122645821.4263.6.camel@localhost.localdomain>

Darrin Thompson <darrint@progeny.com> writes:

> 1. Pack files should reduce the number of http round trips.
> 2. What I'm seeing when I check out mainline git is the acquisition of a
> single large pack, then 600+ more recent objects. Better than before,
> but still hundreds of round trips.

I've packed the git.git repository, by the way. It has 43 unpacked objects totalling 224 kilobytes, so cloning over dumb http should go a lot faster until we accumulate more unpacked objects.

Some of you may have noticed that in the proposed updates queue ("pu" branch) I have a couple of commits related to pulling from a packed dumb http server. There are two "git fetch http://" commits to let you pull from such, and another stupid "count objects" script that you can use to see how many unpacked objects you have in your repository; the latter is to help you decide when to repack.

Brave souls may want to try out the dumb http fetch. For example, it _should_ do the right thing even if you do the following:

 $ git clone http://www.kernel.org/pub/scm/git/git.git/ newdir
 $ cd newdir
 $ mv .git/objects/pack/pack-* . ;# even if you unpack packs on your
 $ rm -f pack-*.idx		 ;# end, it should do the right thing.
 $ for pack in pack-*.pack; do
     git-unpack-objects <$pack
     rm -f "$pack"
   done
 $ rm -f .git/refs/heads/pu
 $ git prune ;# lose objects in "pu" but still not in "master"
 $ git pull origin pu
 $ git ls-remote origin |
   while read sha1 refname
   do
       case "$refname" in
       refs/heads/master) echo $sha1 >".git/$refname" ;;
       esac
   done ;# revert master to upstream master
 $ old=$(git-rev-parse master^^^^^^^^^^)
 $ echo "$old" >.git/refs/heads/master ;# rewind further
 $ git checkout -f master
 $ git prune ;# try losing a bit more objects.
 $ git pull origin master
 $ git ls-remote ./.		;# show me my refs
 $ git ls-remote origin		;# show me his refs

Unlike my other shell scripts I usually write in my e-mail buffer, I have actually run the above ;-).

-jc