threads / discuss / 42449

Why is my git-http-backend solution using WebDAV on push?

Subject: Why is my git-http-backend solution using WebDAV on push?

## tl;dr

5 messages between May 25, 2016 and May 25, 2016.

replies: 4people: 3as markdown or json

Luke Madhanga· May 25, 2016, 20:28 UTC · lore

I've implemented a PHP wrapper for git http backend which works well. I've done this to give me advanced control of who has access to repositories on my server. You can see the implementation on http://stackoverflow.com/questions/36998492/channel-git-on-the-server-calls-through-php/37242591#37242591. I can pull from the server okay and all works well. However, I cannot push. When I read my trace code to see where it fails, I see that the last request is a PROPFIND request. The URL for this request does not have any of the usual 'info/refs' etc. that one usually gets on git calls.

During reading of the documentation and reviewing how git-http-backend should be implemented, I can see that it should not be making a WebDAV request. I have also come across posts where people implementing http-backend the 'normal' way have also had this problem, e.g. http://serverfault.com/questions/390864/git-push-over-http-using-git-http-backend-and-apache-is-not-working

So the question is, why is WebDAV being used on push?
Trace code from the push:
[2016-05-25 09:49:35] ==========================
[2016-05-25 09:49:35] REQUEST: {
    "q": "p\/git-backend\/run\/1\/info\/refs",
    "service": "git-receive-pack"
}
[2016-05-25 09:49:35] SERVER: {
    "REDIRECT_STATUS": "200",
    "HTTP_HOST": "...",
    "HTTP_USER_AGENT": "git\/2.7.4",
    "HTTP_ACCEPT": "*\/*",
    "HTTP_ACCEPT_ENCODING": "gzip",
    "HTTP_ACCEPT_LANGUAGE": "en-GB, en;q=0.9, *;q=0.8",
    "HTTP_PRAGMA": "no-cache",
    "PATH": "...",
    "SERVER_SIGNATURE": "<address>Apache\/2.4.18 (Ubuntu) Server at
... Port 80<\/address>\n",
    "SERVER_SOFTWARE": "Apache\/2.4.18 (Ubuntu)",
    "SERVER_NAME": "...",
    "SERVER_ADDR": "...",
    "SERVER_PORT": "80",
    "REMOTE_ADDR": "...",
    "DOCUMENT_ROOT": "...",
    "REQUEST_SCHEME": "http",
    "CONTEXT_PREFIX": "",
    "CONTEXT_DOCUMENT_ROOT": "...",
    "SERVER_ADMIN": "...",
    "SCRIPT_FILENAME": "...",
    "REMOTE_PORT": "49630",
    "REDIRECT_URL": "\/p\/git-backend\/run\/1\/info\/refs",
    "REDIRECT_QUERY_STRING":
"q=p\/git-backend\/run\/1\/info\/refs&service=git-receive-pack",
    "GATEWAY_INTERFACE": "CGI\/1.1",
    "SERVER_PROTOCOL": "HTTP\/1.1",
    "REQUEST_METHOD": "GET",
    "QUERY_STRING":
"q=p\/git-backend\/run\/1\/info\/refs&service=git-receive-pack",
    "REQUEST_URI":
"\/p\/git-backend\/run\/1\/info\/refs?service=git-receive-pack",
    "SCRIPT_NAME": "\/index.php",
    "PHP_SELF": "\/index.php",
    "REQUEST_TIME_FLOAT": 1464162575.091,
    "REQUEST_TIME": 1464162575
}
[2016-05-25 09:49:35] Path: /core.git/info/refs?service=git-receive-pack
[2016-05-25 09:49:35] Cleaned, result only:
[2016-05-25 09:49:35] f4648182f5f8eee082c37a83a0072cfc4210e5c5 refs/heads/master
8c4efcd77809bc9b94a59cf94653add8007c6b7d refs/heads/zztest
[2016-05-25 09:49:35] ==========================
[2016-05-25 09:49:35] REQUEST: {
    "q": "p\/git-backend\/run\/1\/HEAD"
}
[2016-05-25 09:49:35] SERVER: {
    "REDIRECT_STATUS": "200",
    "HTTP_HOST": "...",
    "HTTP_USER_AGENT": "git\/2.7.4",
    "HTTP_ACCEPT": "*\/*",
    "HTTP_ACCEPT_ENCODING": "gzip",
    "HTTP_ACCEPT_LANGUAGE": "en-GB, en;q=0.9, *;q=0.8",
    "HTTP_PRAGMA": "no-cache",
    "PATH": "...",
    "SERVER_SIGNATURE": "<address>Apache\/2.4.18 (Ubuntu) Server at
... Port 80<\/address>\n",
    "SERVER_SOFTWARE": "Apache\/2.4.18 (Ubuntu)",
    "SERVER_NAME": "...",
    "SERVER_ADDR": "...",
    "SERVER_PORT": "80",
    "REMOTE_ADDR": "...",
    "DOCUMENT_ROOT": "...",
    "REQUEST_SCHEME": "http",
    "CONTEXT_PREFIX": "",
    "CONTEXT_DOCUMENT_ROOT": "...",
    "SERVER_ADMIN": "...",
    "SCRIPT_FILENAME": "...",
    "REMOTE_PORT": "49630",
    "REDIRECT_URL": "\/p\/git-backend\/run\/1\/HEAD",
    "REDIRECT_QUERY_STRING": "q=p\/git-backend\/run\/1\/HEAD",
    "GATEWAY_INTERFACE": "CGI\/1.1",
    "SERVER_PROTOCOL": "HTTP\/1.1",
    "REQUEST_METHOD": "GET",
    "QUERY_STRING": "q=p\/git-backend\/run\/1\/HEAD",
    "REQUEST_URI": "\/p\/git-backend\/run\/1\/HEAD",
    "SCRIPT_NAME": "\/index.php",
    "PHP_SELF": "\/index.php",
    "REQUEST_TIME_FLOAT": 1464162575.266,
    "REQUEST_TIME": 1464162575
}
[2016-05-25 09:49:35] Path: /core.git/HEAD
[2016-05-25 09:49:35] Cleaned, result only:
[2016-05-25 09:49:35] ref: refs/heads/master
[2016-05-25 09:49:35] ==========================
[2016-05-25 09:49:35] REQUEST: {
    "q": "p\/git-backend\/run\/1\/"
}
[2016-05-25 09:49:35] SERVER: {
    "REDIRECT_STATUS": "200",
    "HTTP_HOST": "...",
    "HTTP_USER_AGENT": "git\/2.7.4",
    "HTTP_ACCEPT": "*\/*",
    "HTTP_DEPTH": "0",
    "CONTENT_TYPE": "text\/xml",
    "CONTENT_LENGTH": "167",
    "HTTP_EXPECT": "100-continue",
    "PATH": "...",
    "SERVER_SIGNATURE": "<address>Apache\/2.4.18 (Ubuntu) Server at
... Port 80<\/address>\n",
    "SERVER_SOFTWARE": "Apache\/2.4.18 (Ubuntu)",
    "SERVER_NAME": "...",
    "SERVER_ADDR": "...",
    "SERVER_PORT": "80",
    "REMOTE_ADDR": "...",
    "DOCUMENT_ROOT": "...",
    "REQUEST_SCHEME": "http",
    "CONTEXT_PREFIX": "",
    "CONTEXT_DOCUMENT_ROOT": "...",
    "SERVER_ADMIN": "...",
    "SCRIPT_FILENAME": "...",
    "REMOTE_PORT": "49632",
    "REDIRECT_URL": "\/p\/git-backend\/run\/1\/",
    "REDIRECT_QUERY_STRING": "q=p\/git-backend\/run\/1\/",
    "GATEWAY_INTERFACE": "CGI\/1.1",
    "SERVER_PROTOCOL": "HTTP\/1.1",
    "REQUEST_METHOD": "PROPFIND",
    "QUERY_STRING": "q=p\/git-backend\/run\/1\/",
    "REQUEST_URI": "\/p\/git-backend\/run\/1\/",
    "SCRIPT_NAME": "\/index.php",
    "PHP_SELF": "\/index.php",
    "REQUEST_TIME_FLOAT": 1464162575.711,
    "REQUEST_TIME": 1464162575
}
[2016-05-25 09:49:35] Path: /core.git/
[2016-05-25 09:49:35] <?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:prop xmlns:R="http://.../p/git-backend/run/1/">
<D:supportedlock/>
</D:prop>
</D:propfind>
[2016-05-25 09:49:35] Exception: Request not supported: '/path/to/git/repo.git/'
When run using curl verbose, the output is
* Couldn't find host xxx in the .netrc file; using defaults
*   Trying xxx...
* Connected to xxx (xxx) port 80 (#0)
> GET /p/git-backend/run/1/info/refs?service=git-receive-pack HTTP/1.1
Host: xxx
User-Agent: git/2.7.4
Accept: */*
Accept-Encoding: gzip
Accept-Language: en-GB, en;q=0.9, *;q=0.8
Pragma: no-cache
< HTTP/1.1 200 OK
< Date: Wed, 25 May 2016 19:00:25 GMT
< Server: Apache/2.4.18 (Ubuntu)
< Set-Cookie: PHPSESSID=yyy; path=/
< Expires: Fri, 01 Jan 1980 00:00:00 GMT
< Cache-Control: no-cache, max-age=0, must-revalidate
< Pragma: no-cache
< Vary: Accept-Encoding
< Content-Encoding: gzip
< Content-Length: 109
< Content-Type: text/plain;charset=UTF-8
<
* Connection #0 to host madhan.ga left intact
* Couldn't find host xxx in the .netrc file; using defaults
* Found bundle for host xxx: 0x9bade0 [can pipeline]
* Re-using existing connection! (#0) with host xxx
* Connected to xxx (xxx) port 80 (#0)
> GET /p/git-backend/run/1/HEAD HTTP/1.1
Host: xxx
User-Agent: git/2.7.4
Accept: */*
Accept-Encoding: gzip
Accept-Language: en-GB, en;q=0.9, *;q=0.8
Pragma: no-cache
< HTTP/1.1 200 OK
< Date: Wed, 25 May 2016 19:00:25 GMT
< Server: Apache/2.4.18 (Ubuntu)
< Set-Cookie: PHPSESSID=zzz; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate
< Pragma: no-cache
< Content-Length: 23
< Content-Type: text/plain;charset=UTF-8
<
* Connection #0 to host xxx left intact
* Couldn't find host xxx in the .netrc file; using defaults
*   Trying xxx...
* Connected to xxx (xxx) port 80 (#0)
> PROPFIND /p/git-backend/run/1/ HTTP/1.1
Host: xxx
User-Agent: git/2.7.4
Accept: */*
Depth: 0
Content-Type: text/xml
Content-Length: 167
Expect: 100-continue
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* HTTP 1.0, assume close after body
< HTTP/1.0 100 Continue
< Date: Wed, 25 May 2016 19:00:25 GMT
< Server: Apache/2.4.18 (Ubuntu)
< Set-Cookie: PHPSESSID=aaa; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate
< Pragma: no-cache
< Content-Length: 0
< Connection: close
< Content-Type: text/json; charset=utf-8
* Empty reply from server
* Closing connection 0
error: Cannot access URL http://xxx/p/git-backend/run/1/, return code 52
fatal: git-http-push failed
error: failed to push some refs to 'http://xxx/p/git-backend/run/1'
-- 
Yours,
Luke Madhanga
Jeff King· May 25, 2016, 21:38 UTC · re: Luke Madhanga · lore

Re: Why is my git-http-backend solution using WebDAV on push?

On Wed, May 25, 2016 at 09:28:21PM +0100, Luke Madhanga wrote:
Show 9 quoted lines
> I've implemented a PHP wrapper for git http backend which works well.
> I've done this to give me advanced control of who has access to
> repositories on my server. You can see the implementation on
> http://stackoverflow.com/questions/36998492/channel-git-on-the-server-calls-through-php/37242591#37242591.
> I can pull from the server okay and all works well. However, I cannot
> push. When I read my trace code to see where it fails, I see that the
> last request is a PROPFIND request. The URL for this request does not
> have any of the usual 'info/refs' etc. that one usually gets on git
> calls.

If the git client detects that the server doesn't implement the smart-http protocol, it will automatically downgrade to the older, "dumb" protocol which is less efficient.

For fetching/cloning, this might make your test appear to work, even though it is downgrading behind the scenes. You should be able to check GIT_CURL_VERBOSE output to tell the difference; the smart protocol will POST to .../git-upload-pack, whereas the dumb protocol will download the individual packfiles and objects directly.

For pushing, the dumb protocol uses WebDAV, which is what you're seeing.

So the question is: why doesn't the client think your server is a smart server?

Just skimming your output, I'd guess:
Show 20 quoted lines
> > GET /p/git-backend/run/1/info/refs?service=git-receive-pack HTTP/1.1
> Host: xxx
> User-Agent: git/2.7.4
> Accept: */*
> Accept-Encoding: gzip
> Accept-Language: en-GB, en;q=0.9, *;q=0.8
> Pragma: no-cache
> 
> < HTTP/1.1 200 OK
> < Date: Wed, 25 May 2016 19:00:25 GMT
> < Server: Apache/2.4.18 (Ubuntu)
> < Set-Cookie: PHPSESSID=yyy; path=/
> < Expires: Fri, 01 Jan 1980 00:00:00 GMT
> < Cache-Control: no-cache, max-age=0, must-revalidate
> < Pragma: no-cache
> < Vary: Accept-Encoding
> < Content-Encoding: gzip
> < Content-Length: 109
> < Content-Type: text/plain;charset=UTF-8
> <
The content-type here should be:
  application/x-git-receive-pack-advertisement

The body content should also include "# service=git-receive-pack" on the first line, but if you are successfully calling into git-http-backend, I think it should take care of that detail.

-Peff
Luke Madhanga· May 25, 2016, 21:54 UTC · re: Jeff King · lore

Re: Why is my git-http-backend solution using WebDAV on push?

Hmmmm. Interesting.
When you look at the PHP code, you'll see the following
    $res = self::proc_open("{$gitcoredir}/git-http-backend", [],
$gitdir, true, [...]);
    ...
    $resbits = explode("\n", $res);
    foreach ($resbits as $index => $header) {
        if ($header && strpos($header, ':') !== false) {
            // Headers
            header($header);
            unset($resbits[$index]);
        } else {
            // First blank line is the space between the headers and
the start of the response from Git
            break;
        }
    }
    echo ltrim(implode("\n", $resbits));
    exit;
Everything being returned is from a direct call to the git-http-backend.

A manual CLI call to git-http-backend doesn't include 'application/x-git-receive-pack-advertisement'

REQUEST_METHOD=GET GIT_PROJECT_ROOT=/path/to/core/ PATH_INFO=/repo.git/info/refs /usr/lib/git-core/git-http-backend

The above command outputs
Expires: Fri, 01 Jan 1980 00:00:00 GMT
Pragma: no-cache
Cache-Control: no-cache, max-age=0, must-revalidate
Content-Length: 118
Content-Type: text/plain

f4648182f5f8eee082c37a83a0072cfc4210e5c5 refs/heads/master 8c4efcd77809bc9b94a59cf94653add8007c6b7d refs/heads/zztest

Luke Madhanga· May 25, 2016, 21:54 UTC · re: Luke Madhanga · lore

Re: Why is my git-http-backend solution using WebDAV on push?

Thanks for the response btw
On 25 May 2016 at 22:54, Luke Madhanga <lmadhanga@gmail.com> wrote:
Show 41 quoted lines
> Hmmmm. Interesting.
>
> When you look at the PHP code, you'll see the following
>
>     $res = self::proc_open("{$gitcoredir}/git-http-backend", [],
> $gitdir, true, [...]);
>     ...
>     $resbits = explode("\n", $res);
>     foreach ($resbits as $index => $header) {
>         if ($header && strpos($header, ':') !== false) {
>             // Headers
>             header($header);
>             unset($resbits[$index]);
>         } else {
>             // First blank line is the space between the headers and
> the start of the response from Git
>             break;
>         }
>     }
>     echo ltrim(implode("\n", $resbits));
>     exit;
>
>
> Everything being returned is from a direct call to the git-http-backend.
>
> A manual CLI call to git-http-backend doesn't include
> 'application/x-git-receive-pack-advertisement'
>
> REQUEST_METHOD=GET GIT_PROJECT_ROOT=/path/to/core/
> PATH_INFO=/repo.git/info/refs /usr/lib/git-core/git-http-backend
>
> The above command outputs
>
> Expires: Fri, 01 Jan 1980 00:00:00 GMT
> Pragma: no-cache
> Cache-Control: no-cache, max-age=0, must-revalidate
> Content-Length: 118
> Content-Type: text/plain
>
> f4648182f5f8eee082c37a83a0072cfc4210e5c5 refs/heads/master
> 8c4efcd77809bc9b94a59cf94653add8007c6b7d refs/heads/zztest
-- 
Yours,
Luke Madhanga
Junio C Hamano· May 25, 2016, 22:16 UTC · re: Luke Madhanga · lore

Re: Why is my git-http-backend solution using WebDAV on push?

Luke Madhanga <lmadhanga@gmail.com> writes:
Show 5 quoted lines
>> A manual CLI call to git-http-backend doesn't include
>> 'application/x-git-receive-pack-advertisement'
>>
>> REQUEST_METHOD=GET GIT_PROJECT_ROOT=/path/to/core/
>> PATH_INFO=/repo.git/info/refs /usr/lib/git-core/git-http-backend

The request client makes to probe is (taking it from Peff's message that is quoting from your trace):

> > GET /p/git-backend/run/1/info/refs?service=git-receive-pack HTTP/1.1

Your manual CLI call seems not to have "?service=git-receive-pack" anywhere. Where did it go? QUERY_STRING, perhaps?

Here is what I am observing:
    $ GIT_HTTP_EXPORT_ALL=Yes \
    > REQUEST_METHOD=GET \
    > GIT_PROJECT_ROOT=$(pwd)/.git \
    > PATH_INFO='/info/refs' \
    > QUERY_STRING=service=git-receive-pack \
    > git -c http.receivepack=yes http-backend 2>&1 | sed -e '/^.$/q'
    Expires: Fri, 01 Jan 1980 00:00:00 GMT
    Pragma: no-cache
    Cache-Control: no-cache, max-age=0, must-revalidate
    Content-Type: application/x-git-receive-pack-advertisement
Show 11 quoted lines
>>
>> The above command outputs
>>
>> Expires: Fri, 01 Jan 1980 00:00:00 GMT
>> Pragma: no-cache
>> Cache-Control: no-cache, max-age=0, must-revalidate
>> Content-Length: 118
>> Content-Type: text/plain
>>
>> f4648182f5f8eee082c37a83a0072cfc4210e5c5 refs/heads/master
>> 8c4efcd77809bc9b94a59cf94653add8007c6b7d refs/heads/zztest

← back to recent threads