threads / discuss / 3781

HTTP repo referencing stale heads (can't clone)

Subject: HTTP repo referencing stale heads (can't clone)

## tl;dr

11 messages between Apr 3, 2006 and Apr 5, 2006.

replies: 10people: 5as markdown or json

Daniel Drake· Apr 3, 2006, 16:01 UTC · lore
Hi,

I maintain a small git repo. I upload it over ssh (with git-push) to a machine where it is distributed over http:

http://dsd.object4.net/git/zd1211.git/
For some reason it is no longer possible to clone this repo over http:

walk 35afe6b3a859242a18812e7485ea8b211e24abaf walk 93d9a9f469282e1e392c16ce571da4c08805e8bb error: Couldn't get http://dsd.object4.net/git/zd1211.git/refs/heads/softmac-old for heads/softmac-old The requested URL returned error: 404 error: Could not interpret heads/softmac-old as something to pull

"softmac-old" is an old branch, which I have recently deleted. I deleted it by removing the .git/refs/heads/softmac-old file, and relying on git-prune to clear out old objects.

Even on the server-side, there is no obvious reference to this old head:

$ find -name '*softmac*' $ grep -R softmac * (no results for either)

"git-fsck-objects" reports nothing, "git-fsck-objects --full" reports: dangling commit 7cc423c942975005f96f308186537ad6e7808c2e dangling commit b36378de6231f1b5100b1517b9c8c243a21090fd

I have tried running git-prune and git-update-server-info, but that doesn't help.

Any ideas? I'm still new to git. I am running git-1.2.4

Thanks, Daniel

Junio C Hamano· Apr 3, 2006, 17:23 UTC · re: Daniel Drake · lore

Re: HTTP repo referencing stale heads (can't clone)

Daniel Drake <dsd@gentoo.org> writes:
Show 9 quoted lines
> I maintain a small git repo. I upload it over ssh (with git-push) to a
> machine where it is distributed over http:
>
> http://dsd.object4.net/git/zd1211.git/
>...
> I have tried running git-prune and git-update-server-info, but that
> doesn't help.
>
> Any ideas?
My standard answer would be

http-server$ cd /var/www/git/zd1211.git/ ;# or whereever http-server$ GIT_DIR=. git-update-server-info

but you said you have run it already... Can you try to see if there is some caching proxy involved that still serves stale info/refs file?

client$ wget http://dsd.object4.net/git/zd1211.git/info/refs
Nick Hengeveld· Apr 3, 2006, 18:09 UTC · re: Junio C Hamano · lore

Re: HTTP repo referencing stale heads (can't clone)

On Mon, Apr 03, 2006 at 10:23:22AM -0700, Junio C Hamano wrote:
> My standard answer would be
> 
> http-server$ cd /var/www/git/zd1211.git/ ;# or whereever
> http-server$ GIT_DIR=. git-update-server-info

Is there any interest in making the HTTP transport slighly less dumb by using DAV?

I have a working patch to http-fetch that tries to use PROPFIND to get a remote pack list and falls back to using objects/info/packs. It's feasible to do something similar to get a remote ref list when cloning, although that's a bit more work as all refs would have to be fetched into a local repo and parsed to determine the object type.

Long term, this could give a repo admin the choice of either making sure git-update-server-info has been run after every ref/pack change or enabling DAV once. Assuming they need to use HTTP.

-- 
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.
Junio C Hamano· Apr 4, 2006, 07:03 UTC · re: Nick Hengeveld · lore

Re: HTTP repo referencing stale heads (can't clone)

Nick Hengeveld <nickh@reactrix.com> writes:
> Is there any interest in making the HTTP transport slighly less dumb by
> using DAV?

I personally feel PROPFIND is the right way to do "wget -r", and very much welcome a patch to replace objects/info/packs with it when able.

Show 5 quoted lines
> I have a working patch to http-fetch that tries to use PROPFIND to get a
> remote pack list and falls back to using objects/info/packs.  It's
> feasible to do something similar to get a remote ref list when cloning,
> although that's a bit more work as all refs would have to be fetched
> into a local repo and parsed to determine the object type.

Faking info/refs with PROPFIND, if we do not have to peel the onion ^{}, should be relatively cheap operation, and could be done as an enhancement to git-ls-remote.sh. If your faked info/refs file lacks ^{} entries, git-fetch cannot auto-follow tags, but git-clone should work as before.

A clever sysadmins could mod_rewrite requests to info/refs and objects/info/packs with a custom CGI, but then probably they would be running git-daemon ;-).

Petr Baudis· Apr 4, 2006, 10:00 UTC · re: Nick Hengeveld · lore

Re: HTTP repo referencing stale heads (can't clone)

Dear diary, on Mon, Apr 03, 2006 at 08:09:30PM CEST, I got a letter where Nick Hengeveld <nickh@reactrix.com> said that...

> Long term, this could give a repo admin the choice of either making sure
> git-update-server-info has been run after every ref/pack change or
> enabling DAV once.  Assuming they need to use HTTP.

Well, what is the actual advantage of DAV compared to git-update-server-info? Why would I prefer enabling DAV?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Right now I am having amnesia and deja-vu at the same time.  I think
I have forgotten this before.
Nick Hengeveld· Apr 4, 2006, 12:10 UTC · re: Petr Baudis · lore

Re: HTTP repo referencing stale heads (can't clone)

On Tue, Apr 04, 2006 at 12:00:35PM +0200, Petr Baudis wrote:
> Well, what is the actual advantage of DAV compared to
> git-update-server-info? Why would I prefer enabling DAV?

In theory, things should work the same either way. It seems that in practice though, the server info files continue to surface as a source of fetch problems.

-- 
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.
Petr Baudis· Apr 4, 2006, 15:27 UTC · re: Nick Hengeveld · lore

Re: HTTP repo referencing stale heads (can't clone)

Dear diary, on Tue, Apr 04, 2006 at 02:10:57PM CEST, I got a letter where Nick Hengeveld <nickh@reactrix.com> said that...

Show 8 quoted lines
> On Tue, Apr 04, 2006 at 12:00:35PM +0200, Petr Baudis wrote:
> 
> > Well, what is the actual advantage of DAV compared to
> > git-update-server-info? Why would I prefer enabling DAV?
> 
> In theory, things should work the same either way.  It seems that in
> practice though, the server info files continue to surface as a source
> of fetch problems.

Because people do not know they have to set up their post-update hook. When they are already going at lengths to find out how to set up DAV for git fetch, they would discover the post-update hook way as well.

So, it really seems rather redundant to me.
-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Right now I am having amnesia and deja-vu at the same time.  I think
I have forgotten this before.
Nick Hengeveld· Apr 4, 2006, 17:56 UTC · re: Petr Baudis · lore

Re: HTTP repo referencing stale heads (can't clone)

On Tue, Apr 04, 2006 at 05:27:37PM +0200, Petr Baudis wrote:
> Because people do not know they have to set up their post-update hook.
> When they are already going at lengths to find out how to set up DAV for
> git fetch, they would discover the post-update hook way as well.

There are other reasons that a client can end up with a stale copy of the server info files - in this case the problem was an intermediate proxy out of the control of the repo admin.

While we should be able to fix that particular problem, it seems safer to go straight to the source if possible.

> So, it really seems rather redundant to me.

If I've already enabled DAV for pushing to a repo, I'd find it nice to be able to use it for fetches as well.

-- 
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.
Daniel Drake· Apr 3, 2006, 18:28 UTC · re: Junio C Hamano · lore

Re: HTTP repo referencing stale heads (can't clone)

Junio C Hamano wrote:
> client$ wget http://dsd.object4.net/git/zd1211.git/info/refs
Ah, should have known. I am behind a (lame) transparent proxy on port 80.

I opened that file in my web browser and it showed the old heads. After a force-refresh (ctrl+F5, which sends some additionally http headers to refresh the page from the real server), the old heads disappeared, and git now clones successfully.

git-http-fetch should probably send those extra headers too. I'll try to find some time to look at this next week.

Thanks! Daniel

Nick Hengeveld· Apr 4, 2006, 18:01 UTC · re: Daniel Drake · lore

Re: HTTP repo referencing stale heads (can't clone)

On Mon, Apr 03, 2006 at 07:28:28PM +0100, Daniel Drake wrote:
Show 9 quoted lines
> Ah, should have known. I am behind a (lame) transparent proxy on port 80.
> 
> I opened that file in my web browser and it showed the old heads. After 
> a force-refresh (ctrl+F5, which sends some additionally http headers to 
> refresh the page from the real server), the old heads disappeared, and 
> git now clones successfully.
> 
> git-http-fetch should probably send those extra headers too. I'll try to 
> find some time to look at this next week.

git-http-fetch uses the "Pragma: no-cache" header when requesting objects that shouldn't be cached. Is this the additional header you're referring to?

This patch adds the header to git-ls-remote for the info/refs request.
git-ls-remote: send no-cache header when fetching info/refs

Proxies should not cache this file as it can cause a client to end up with a stale version, as reported here:

http://marc.theaimsgroup.com/?l=git&m=114407944125389
Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
---
 git-ls-remote.sh |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
da9b6fa01f1a8bd6ab5f6d4346584f3f032584aa
diff --git a/git-ls-remote.sh b/git-ls-remote.sh
index 2c9a588..b6882a9 100755
--- a/git-ls-remote.sh
+++ b/git-ls-remote.sh
@@ -53,7 +53,7 @@ http://* | https://* )
         if [ -n "$GIT_SSL_NO_VERIFY" ]; then
             curl_extra_args="-k"
         fi
-	curl -nsf $curl_extra_args "$peek_repo/info/refs" ||
+	curl -nsf $curl_extra_args --header "Pragma: no-cache" "$peek_repo/info/refs" ||
 		echo "failed	slurping"
 	;;
 

-- 
1.3.0.rc1.g9aef-dirty
Radoslaw Szkodzinski· Apr 5, 2006, 12:23 UTC · re: Nick Hengeveld · lore

Re: HTTP repo referencing stale heads (can't clone)

Nick Hengeveld wrote:
Show 16 quoted lines
> On Mon, Apr 03, 2006 at 07:28:28PM +0100, Daniel Drake wrote:
> 
>> Ah, should have known. I am behind a (lame) transparent proxy on port 80.
>>
>> I opened that file in my web browser and it showed the old heads. After 
>> a force-refresh (ctrl+F5, which sends some additionally http headers to 
>> refresh the page from the real server), the old heads disappeared, and 
>> git now clones successfully.
>>
>> git-http-fetch should probably send those extra headers too. I'll try to 
>> find some time to look at this next week.
> 
> git-http-fetch uses the "Pragma: no-cache" header when requesting
> objects that shouldn't be cached.  Is this the additional header you're
> referring to?
> 
As per HTTP 1.1, it should also send:
Cache-Control: no-cache
Pragma: no-cache is the deprecated extension.
It's safe to send both.
-- 
GPG Key id:  0xD1F10BA2
Fingerprint: 96E2 304A B9C4 949A 10A0  9105 9543 0453 D1F1 0BA2

AstralStorm

← back to recent threads