Re: dumb transports not being welcomed..

From: Junio C Hamano <junkio@cox.net>
Date: Sep 13, 2005, 22:11 UTC
Message-ID: <7vpsrcwrc1.fsf@assigned-by-dhcp.cox.net>
In-Reply-To: <7vacig1wrb.fsf@assigned-by-dhcp.cox.net>

Junio C Hamano <junkio@cox.net> writes:

Show 8 quoted lines

> Sam Ravnborg <sam@ravnborg.org> writes:
>
>> Whats wrong using cogito?
>> In other words. Why does you feel like that when we use cogito to do
>> cg-update.
>
> Using cogito is not a problem at all.  The mechanism to prepare
> trees to serve wider audience not being used widely is.

I need to clarify what I meant by 'not welcoming dumb transport' a bit better. Namely, those (~80 - 23) = ~57 repositories lack support for 'git ls-remote' over http, which means you cannot discover what refs the repository has.

Some people argued that it can be done via recursive wget on refs/ hierarchy. Here is what you would get if you do that against kernel.org:

  $ wget -r -np -nH --cut-dirs=4 http://kernel.org/pub/scm/git/git.git/refs/.
  $ ls -R refs
  refs:
  ./      index.html          index.html?C=N;O=A  index.html?C=S;O=D
  ../     index.html?C=M;O=A  index.html?C=N;O=D  tags/
  heads/  index.html?C=M;O=D  index.html?C=S;O=A

  refs/heads:
  ./          index.html?C=M;O=A  index.html?C=N;O=D  master  todo
  ../         index.html?C=M;O=D  index.html?C=S;O=A  pu
  index.html  index.html?C=N;O=A  index.html?C=S;O=D  rc

  refs/tags:
  ./                  index.html?C=M;O=D  index.html?C=S;O=D  v0.99.2  v0.99.6
  ../                 index.html?C=N;O=A  junio-gpg-pub       v0.99.3
  index.html          index.html?C=N;O=D  v0.99               v0.99.4
  index.html?C=M;O=A  index.html?C=S;O=A  v0.99.1             v0.99.5

Of course, I do not have a branch called index.html there, and this also means I will not be able to have a branch with that name even if I wanted to.

Also some webservers are configured not to even allow directory index, and they may use different formatting for directory index even when they do support it, so excluding anything that matches index.html* would work well but that is only heuristics.

The file $GIT_DIR/info/refs was introduced to solve this by listing the available refs for discovery, and hooks/post-update, when enabled, runs update-server-info to update the file (among other things) whenever you push into the repository. info/refs is not strictly necessary for repositories at kernel.org because people tend to know what refs are available for pulling and you can always visit there via gitweb to find it out.

I just felt that it is a good habit to get into to prepare your repositories in a shape usable even when served by an HTTP server that is less forgiving than what kernel.org runs -- that was what I felt "discouraging" about.

Another thing is that the missing info/refs file means the repository is not prepared with update-server-info, so it is likely that it lacks objects/info/packs to describe what packs are in the object database. I believe cogito uses git-http-pull after you tell which ref to pull, and this step would break if the repository is packed, objects/info/packs is not available, and if the downloader does not have an object that is already prune-packed in the repository. This means either people are not packing their repository (hence nobody complained), or public are pulling over rsync transport (which slurps everything in sight). Both are good reasons to feel discouraged about.