threads / discuss / 32538

Trying to understand the web dav details

Subject: Trying to understand the web dav details

## tl;dr

5 messages between Jan 6, 2013 and Jan 6, 2013.

replies: 4people: 3as markdown or json

Jason Pyeron· Jan 6, 2013, 01:32 UTC · lore
When doing a clone by https (reverse proxied to http) the first request is
GET /git/project/info/refs?service=git-upload-pack

How does the ?service=xxxx get translated in to the action performed on the web server?

I ask because I have 2 projects, one works the other does not.
I am using httpd-2.0.52-49.ent.centos4 and git-1.7.9.6-1.
I am not even sure what to tell more about or where to look next.
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-                                                               -
- Jason Pyeron                      PD Inc. http://www.pdinc.us -
- Principal Consultant              10 West 24th Street #100    -
- +1 (443) 269-1555 x333            Baltimore, Maryland 21218   -
-                                                               -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
Jeff King· Jan 6, 2013, 04:19 UTC · re: Jason Pyeron · lore

Re: Trying to understand the web dav details

On Sat, Jan 05, 2013 at 08:32:09PM -0500, Jason Pyeron wrote:
Show 6 quoted lines
> When doing a clone by https (reverse proxied to http) the first request is
> 
> GET /git/project/info/refs?service=git-upload-pack
> 
> How does the ?service=xxxx get translated in to the action performed on the web
> server?

If you are using the git-http-backend CGI, it will interpret the service tag and start smart-http. See "git help http-backend" for details on plugging it into Apache.

Cloning/fetching does not use DAV at all; it is only for non-smart pushing (and I would not recommend setting it up; the smart protocol spoken by git-http-backend does pushing much more efficiently, and is better maintained).

Show 5 quoted lines
> I ask because I have 2 projects, one works the other does not.
> 
> I am using httpd-2.0.52-49.ent.centos4 and git-1.7.9.6-1.
> 
> I am not even sure what to tell more about or where to look next.

If you haven't set up git-http-backend, then git is just going to fetch the remote repo's data directly over http. So the usual advice for accessing something via http would apply (check the server's access and error logs, try hitting it with a web browser, etc).

If you set GIT_CURL_VERBOSE=1 in your environment, git will spew a lot of debugging information about what http requests it is making. That might give you a clue (you haven't said anything about what does not work, so I can't be more specific).

-Peff
Pyeron, Jason J CTR (US)· Jan 6, 2013, 04:49 UTC · re: Jeff King · lore

RE: Trying to understand the web dav details

Show 17 quoted lines
> -----Original Message-----
> From: Jeff King
> Sent: Saturday, January 05, 2013 11:20 PM
> 
> On Sat, Jan 05, 2013 at 08:32:09PM -0500, Jason Pyeron wrote:
> 
> > When doing a clone by https (reverse proxied to http) the first
> request is
> >
> > GET /git/project/info/refs?service=git-upload-pack
> >
> > How does the ?service=xxxx get translated in to the action performed
> on the web
> > server?
> 
> If you are using the git-http-backend CGI, it will interpret the
> service
No, using plain jane http and webdav. This server is not "allowed" to use cgi processes.
Show 9 quoted lines
> tag and start smart-http. See "git help http-backend" for details on
> plugging it into Apache.
> 
> Cloning/fetching does not use DAV at all; it is only for non-smart
> pushing (and I would not recommend setting it up; the smart protocol
> spoken by git-http-backend does pushing much more efficiently, and is
> better maintained).
> 
> > I ask because I have 2 projects, one works the other does not.
Should have said I get a 404 versus 200, see below.
Show 11 quoted lines
> >
> > I am using httpd-2.0.52-49.ent.centos4 and git-1.7.9.6-1.
> >
> > I am not even sure what to tell more about or where to look next.
> 
> If you haven't set up git-http-backend, then git is just going to fetch
> the remote repo's data directly over http. So the usual advice for
> accessing something via http would apply (check the server's access and
> error logs, try hitting it with a web browser, etc).
> 
> If you set GIT_CURL_VERBOSE=1 in your environment, git will spew a lot
That’s what I did before mailing out the first time. I tried to summarize the contents, but I will paste the relevant portions below.
> of debugging information about what http requests it is making. That
> might give you a clue (you haven't said anything about what does not
> work, so I can't be more specific).
Below are two traces, the first one is the one that does not work, the second does. I cannot see any differences on the web server for the two directory trees.

************************************************************************** ************************************************************************** ************************************************************************** **************************************************************************

jason.pyeron@localhost ~/tmp
$ GIT_TRACE=1 GIT_CURL_VERBOSE=1 git clone https://server/git/project-x
trace: built-in: git 'clone' 'https://server/git/project-x'
Cloning into 'project-x'...
trace: run_command: 'git-remote-https' 'origin' 'https://server/git/project-x'
* Couldn't find host server in the .netrc file; using defaults
* About to connect() to proxy 214.36.0.135 port 8080 (#0)
*   Trying 214.36.0.135...
* 0x80076a48 is at send pipe head!
* STATE: CONNECT => WAITCONNECT handle 0x8007f3c0; (connection #0)
* Connected to 214.36.0.135 (214.36.0.135) port 8080 (#0)
* Connected to 214.36.0.135 (214.36.0.135) port 8080 (#0)
* Establish HTTP proxy tunnel to server:443
> CONNECT server:443 HTTP/1.1
Host: server:443
User-Agent: git/1.7.9
Proxy-Connection: Keep-Alive
Pragma: no-cache
* STATE: WAITCONNECT => WAITPROXYCONNECT handle 0x8007f3c0; (connection #0)
* Multi mode finished polling for response from proxy CONNECT
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* successfully set certificate verify locations:
*   CAfile: /usr/ssl/certs/ca-bundle.crt
  CApath: none
* STATE: WAITPROXYCONNECT => WAITCONNECT handle 0x8007f3c0; (connection #0)
* STATE: WAITCONNECT => PROTOCONNECT handle 0x8007f3c0; (connection #0)
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*        subject: C=US; ST=Maryland; O=PD Inc; OU=Intranet; CN=server; emailAddress=security@pdinc.us
*        start date: 201
*        expire date: 201
*        issuer: C=U
*        SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* STATE: PROTOCONNECT => DO handle 0x8007f3c0; (connection #0)
> GET /git/project-x/info/refs?service=git-upload-pack HTTP/1.1
User-Agent: git/1.7.9
Host: server
Accept: */*
Pragma: no-cache
* STATE: DO => DO_DONE handle 0x8007f3c0; (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x8007f3c0; (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0x8007f3c0; (connection #0)
* additional stuff not fine /usr/src/ports/curl/curl-7.27.0-1/src/curl-7.27.0/lib/transfer.c:1037: 0 0
* additional stuff not fine /usr/src/ports/curl/curl-7.27.0-1/src/curl-7.27.0/lib/transfer.c:1037: 0 0
* The requested URL returned error: 404 Not Found
* Closing connection #0
* Couldn't find host server in the .netrc file; using defaults
* About to connect() to proxy 214.36.0.135 port 8080 (#0)
*   Trying 214.36.0.135...
* 0x80076a48 is at send pipe head!
* STATE: CONNECT => WAITCONNECT handle 0x80139640; (connection #0)
* Connected to 214.36.0.135 (214.36.0.135) port 8080 (#0)
* Connected to 214.36.0.135 (214.36.0.135) port 8080 (#0)
* Establish HTTP proxy tunnel to server:443
> CONNECT server:443 HTTP/1.1
Host: server:443
User-Agent: git/1.7.9
Proxy-Connection: Keep-Alive
Pragma: no-cache
* STATE: WAITCONNECT => WAITPROXYCONNECT handle 0x80139640; (connection #0)
* Multi mode finished polling for response from proxy CONNECT
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* successfully set certificate verify locations:
*   CAfile: /usr/ssl/certs/ca-bundle.crt
  CApath: none
* SSL re-using session ID
* STATE: WAITPROXYCONNECT => WAITCONNECT handle 0x80139640; (connection #0)
* STATE: WAITCONNECT => PROTOCONNECT handle 0x80139640; (connection #0)
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*        subject: C=US; ST=Maryland; O=PD Inc; OU=Intranet; CN=server; emailAddress=security@pdinc.us
*        start date: 201
*        expire date: 201
*        issuer: C=U
*        SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* STATE: PROTOCONNECT => DO handle 0x80139640; (connection #0)
> GET /git/project-x/info/refs HTTP/1.1
User-Agent: git/1.7.9
Host: server
Accept: */*
Pragma: no-cache
* STATE: DO => DO_DONE handle 0x80139640; (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x80139640; (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0x80139640; (connection #0)
* additional stuff not fine /usr/src/ports/curl/curl-7.27.0-1/src/curl-7.27.0/lib/transfer.c:1037: 0 0
* additional stuff not fine /usr/src/ports/curl/curl-7.27.0-1/src/curl-7.27.0/lib/transfer.c:1037: 0 0
* additional stuff not fine /usr/src/ports/curl/curl-7.27.0-1/src/curl-7.27.0/lib/transfer.c:1037: 0 0
* additional stuff not fine /usr/src/ports/curl/curl-7.27.0-1/src/curl-7.27.0/lib/transfer.c:1037: 0 0
* additional stuff not fine /usr/src/ports/curl/curl-7.27.0-1/src/curl-7.27.0/lib/transfer.c:1037: 0 0
* The requested URL returned error: 404 Not Found
* Closing connection #0
fatal: https://server/git/project-x/info/refs not found: did you run git update-server-info on the server?

************************************************************************** ************************************************************************** ************************************************************************** **************************************************************************

jason.pyeron@localhost ~/tmp
$ GIT_TRACE=1 GIT_CURL_VERBOSE=1 git clone https://server/git/test
trace: built-in: git 'clone' 'https://server/git/test'
Cloning into 'test'...
trace: run_command: 'git-remote-https' 'origin' 'https://server/git/test'
* Couldn't find host server in the .netrc file; using defaults
* About to connect() to proxy 214.36.0.135 port 8080 (#0)
*   Trying 214.36.0.135...
* 0x80076968 is at send pipe head!
* STATE: CONNECT => WAITCONNECT handle 0x8007f2c8; (connection #0)
* Connected to 214.36.0.135 (214.36.0.135) port 8080 (#0)
* Connected to 214.36.0.135 (214.36.0.135) port 8080 (#0)
* Establish HTTP proxy tunnel to server:443
> CONNECT server:443 HTTP/1.1
Host: server:443
User-Agent: git/1.7.9
Proxy-Connection: Keep-Alive
Pragma: no-cache
* STATE: WAITCONNECT => WAITPROXYCONNECT handle 0x8007f2c8; (connection #0)
* Multi mode finished polling for response from proxy CONNECT
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* successfully set certificate verify locations:
*   CAfile: /usr/ssl/certs/ca-bundle.crt
  CApath: none
* STATE: WAITPROXYCONNECT => WAITCONNECT handle 0x8007f2c8; (connection #0)
* STATE: WAITCONNECT => PROTOCONNECT handle 0x8007f2c8; (connection #0)
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*        subject: C=US; ST=Maryland; O=PD Inc; OU=Intranet; CN=server; emailAddress=security@pdinc.us
*        start date: 201
*        expire date: 201
*        issuer: C=U
*        SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* STATE: PROTOCONNECT => DO handle 0x8007f2c8; (connection #0)
> GET /git/test/info/refs?service=git-upload-pack HTTP/1.1
User-Agent: git/1.7.9
Host: server
Accept: */*
Pragma: no-cache
* STATE: DO => DO_DONE handle 0x8007f2c8; (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x8007f2c8; (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0x8007f2c8; (connection #0)
* additional stuff not fine /usr/src/ports/curl/curl-7.27.0-1/src/curl-7.27.0/lib/transfer.c:1037: 0 0
* additional stuff not fine /usr/src/ports/curl/curl-7.27.0-1/src/curl-7.27.0/lib/transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< Date: Sun, 06 Jan 2013 04:26:14 GMT
< Server: Apache/2.0.52 (CentOS)
< Last-Modified: Fri, 02 Nov 2012 05:05:55 GMT
< ETag: "714064-3b-172852c0"
< Accept-Ranges: bytes
< Content-Length: 59
< Content-Type: text/plain; charset=UTF-8
< Connection: close
<
* STATE: PERFORM => DONE handle 0x8007f2c8; (connection #0)
* Closing connection #0
* Couldn't find host server in the .netrc file; using defaults
* About to connect() to proxy 214.36.0.135 port 8080 (#0)
*   Trying 214.36.0.135...
* 0x80076968 is at send pipe head!
* STATE: CONNECT => WAITCONNECT handle 0x801395e8; (connection #0)
* Connected to 214.36.0.135 (214.36.0.135) port 8080 (#0)
* Connected to 214.36.0.135 (214.36.0.135) port 8080 (#0)
* Establish HTTP proxy tunnel to server:443
> CONNECT server:443 HTTP/1.1
Host: server:443
User-Agent: git/1.7.9
Proxy-Connection: Keep-Alive
Pragma: no-cache
* STATE: WAITCONNECT => WAITPROXYCONNECT handle 0x801395e8; (connection #0)
* Multi mode finished polling for response from proxy CONNECT
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* successfully set certificate verify locations:
*   CAfile: /usr/ssl/certs/ca-bundle.crt
  CApath: none
* SSL re-using session ID
* STATE: WAITPROXYCONNECT => WAITCONNECT handle 0x801395e8; (connection #0)
* STATE: WAITCONNECT => PROTOCONNECT handle 0x801395e8; (connection #0)
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*        subject: C=US; ST=Maryland; O=PD Inc; OU=Intranet; CN=server; emailAddress=security@pdinc.us
*        start date: 201
*        expire date: 201
*        issuer: C=U
*        SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* STATE: PROTOCONNECT => DO handle 0x801395e8; (connection #0)
> GET /git/test/HEAD HTTP/1.1
Jeff King· Jan 6, 2013, 05:38 UTC · re: Pyeron, Jason J CTR (US) · lore

Re: Trying to understand the web dav details

On Sun, Jan 06, 2013 at 04:49:57AM +0000, Pyeron, Jason J CTR (US) wrote:
Show 8 quoted lines
> > > How does the ?service=xxxx get translated in to the action
> > > performed on the web server?
> > 
> > If you are using the git-http-backend CGI, it will interpret the
> > service
> 
> No, using plain jane http and webdav. This server is not "allowed" to
> use cgi processes.

Then the service parameter should be ignored by your webserver, and it should just serve the info/refs file from the repository on the filesystem. And you are stuck using WebDAV for push.

> > GET /git/project-x/info/refs HTTP/1.1
> [...]
> * The requested URL returned error: 404 Not Found
Does the info/refs file exist in the project-x repository?
> fatal: https://server/git/project-x/info/refs not found: did you run git update-server-info on the server?
Did you?

If you can't run any git programs on the server at all (and it sounds like that may be the case), you'll need to run it locally before putting the repository data on the server.

Once you have WebDAV set up for pushing, it will update the info/refs file for each push. But if you are initially seeding the server with rsync or a tarfile, you'll want to make sure it has an up-to-date info/refs file.

-Peff
Jason Pyeron· Jan 6, 2013, 06:20 UTC · re: Jeff King · lore

RE: Trying to understand the web dav details

Ignore everything below, it was a case sensitive typo. It always worked it.
Show 27 quoted lines
> -----Original Message-----
> From: Jeff King
> Sent: Sunday, January 06, 2013 0:38
> 
> On Sun, Jan 06, 2013 at 04:49:57AM +0000, Pyeron, Jason J CTR 
> (US) wrote:
> 
> > > > How does the ?service=xxxx get translated in to the action 
> > > > performed on the web server?
> > > 
> > > If you are using the git-http-backend CGI, it will interpret the 
> > > service
> > 
> > No, using plain jane http and webdav. This server is not 
> "allowed" to 
> > use cgi processes.
> 
> Then the service parameter should be ignored by your 
> webserver, and it should just serve the info/refs file from 
> the repository on the filesystem. And you are stuck using 
> WebDAV for push.
> 
> > > GET /git/project-x/info/refs HTTP/1.1
> > [...]
> > * The requested URL returned error: 404 Not Found
> 
> Does the info/refs file exist in the project-x repository?
Yes.
Show 6 quoted lines
> 
> > fatal: https://server/git/project-x/info/refs not found: 
> did you run git update-server-info on the server?
> 
> Did you?
> 
Many times.
Show 7 quoted lines
> If you can't run any git programs on the server at all (and 
> it sounds like that may be the case), you'll need to run it 
> locally before putting the repository data on the server.
> 
> Once you have WebDAV set up for pushing, it will update the 
> info/refs file for each push. But if you are initially 
> seeding the server with rsync or a tarfile, you'll want to 

Seeding it seems to work, it is the bare init that seems to be failing. Might be on to something there.

> make sure it has an up-to-date info/refs file.
Here is the create script:
#!/bin/bash

if [ $# != 1 ]; then exit 1; fi

if [ -e "$1" ]; then exit 2; fi

mkdir "$1" cd "$1" git init --bare cp hooks/post-update.sample hooks/post-update chmod +x hooks/post-update git update-server-info

--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-                                                               -
- Jason Pyeron                      PD Inc. http://www.pdinc.us -
- Principal Consultant              10 West 24th Street #100    -
- +1 (443) 269-1555 x333            Baltimore, Maryland 21218   -
-                                                               -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
 

← back to recent threads