{"thread":{"id":"27388","subject":"Is there a debug mode for git-clone https traffic?","startedAt":"2011-05-17T23:08:06Z","lastAt":"2011-05-23T08:21:55Z","messageCount":7,"participants":["Ævar Arnfjörð Bjarmason","Richard Peterson","Tay Ray Chuan","Junio C Hamano","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"168088","messageId":"BANLkTi=-i063MqrboT1MHrQy-vM0yBTUGg@mail.gmail.com","threadId":"27388","inReplyTo":null,"subject":"Is there a debug mode for git-clone https traffic?","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2011-05-17T23:08:06Z","receivedAt":"2011-05-17T23:08:06Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"I''m trying to debug a problem where over a https proxy I have\n\"warning: remote HEAD refers to nonexistent ref, unable to checkout.\"\nwhen doing git-clone.\n\nI suspect that it might be an overzealous security scanner sitting in\nthe middle.\n\nIs there some debugging mode for git-clone where it'll dump the\ntraffic being sent over the wire that I could use to confirm this?\n"},{"id":"168094","messageId":"-3808464152500901395@unknownmsgid","threadId":"27388","inReplyTo":"BANLkTi=-i063MqrboT1MHrQy-vM0yBTUGg@mail.gmail.com","subject":"Re: Is there a debug mode for git-clone https traffic?","fromName":"Richard Peterson","fromEmail":"bigtheta@gmail.com","sentAt":"2011-05-18T01:38:39Z","receivedAt":"2011-05-18T01:38:39Z","isPatch":false,"sender":{"key":"bigtheta@gmail.com","avatar":null},"body":"On May 17, 2011, at 19:08, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:\n\n> I''m trying to debug a problem where over a https proxy I have\n> \"warning: remote HEAD refers to nonexistent ref, unable to checkout.\"\n> when doing git-clone.\n\nLast time I ran into that error, I was cloning from a bundle where the\nremote head was simply not in the bundle. The branches I needed were\nall there. Running \"git branch -a\" and then checking out one of the\nexistent branches worked fine.\n\nNot sure if that's related to your situation at all.\n\nRichard\n"},{"id":"168103","messageId":"BANLkTinmoJA6tZDaGD9K9cBm_dbmti=P=w@mail.gmail.com","threadId":"27388","inReplyTo":"BANLkTi=-i063MqrboT1MHrQy-vM0yBTUGg@mail.gmail.com","subject":"Re: Is there a debug mode for git-clone https traffic?","fromName":"Tay Ray Chuan","fromEmail":"rctay89@gmail.com","sentAt":"2011-05-18T05:29:27Z","receivedAt":"2011-05-18T05:29:27Z","isPatch":false,"sender":{"key":"rctay89@gmail.com","avatar":"https://avatars.githubusercontent.com/u/61553?v=4"},"body":"On Wed, May 18, 2011 at 7:08 AM, Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n>\n> I''m trying to debug a problem where over a https proxy I have\n> \"warning: remote HEAD refers to nonexistent ref, unable to checkout.\"\n> when doing git-clone.\n>\n> I suspect that it might be an overzealous security scanner sitting in\n> the middle.\n>\n> Is there some debugging mode for git-clone where it'll dump the\n> traffic being sent over the wire that I could use to confirm this?\n\nRun the process with GIT_CURL_VERBOSE=1. There's also another option\nto debug communication between transport helpers but I forget what it\nis.\n\n--\nCheers,\nRay Chuan\n"},{"id":"168104","messageId":"7v4o4sefos.fsf@alter.siamese.dyndns.org","threadId":"27388","inReplyTo":"BANLkTinmoJA6tZDaGD9K9cBm_dbmti=P=w@mail.gmail.com","subject":"Re: Is there a debug mode for git-clone https traffic?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-05-18T05:33:07Z","receivedAt":"2011-05-18T05:33:07Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Tay Ray Chuan <rctay89@gmail.com> writes:\n\n> On Wed, May 18, 2011 at 7:08 AM, Ævar Arnfjörð Bjarmason\n> <avarab@gmail.com> wrote:\n>>\n>> I''m trying to debug a problem where over a https proxy I have\n>> \"warning: remote HEAD refers to nonexistent ref, unable to checkout.\"\n>> when doing git-clone.\n>>\n>> I suspect that it might be an overzealous security scanner sitting in\n>> the middle.\n>>\n>> Is there some debugging mode for git-clone where it'll dump the\n>> traffic being sent over the wire that I could use to confirm this?\n>\n> Run the process with GIT_CURL_VERBOSE=1. There's also another option\n> to debug communication between transport helpers but I forget what it\n> is.\n\nGIT_TRACE_PACKET\n"},{"id":"168122","messageId":"20110518085932.GI27482@sigill.intra.peff.net","threadId":"27388","inReplyTo":"BANLkTi=-i063MqrboT1MHrQy-vM0yBTUGg@mail.gmail.com","subject":"Re: Is there a debug mode for git-clone https traffic?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-05-18T08:59:32Z","receivedAt":"2011-05-18T08:59:32Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 18, 2011 at 01:08:06AM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> I''m trying to debug a problem where over a https proxy I have\n> \"warning: remote HEAD refers to nonexistent ref, unable to checkout.\"\n> when doing git-clone.\n> \n> I suspect that it might be an overzealous security scanner sitting in\n> the middle.\n> \n> Is there some debugging mode for git-clone where it'll dump the\n> traffic being sent over the wire that I could use to confirm this?\n\nOthers have mentioned ways to dump data from clone. But you can more or\nless see what's going on with \"git ls-remote <remote>\". The sha1 for\n\"HEAD\" will try to match with the sha1 for some other ref. If it\ndoesn't, then we give up.\n\n-Peff\n"},{"id":"168142","messageId":"7vmxikc92w.fsf@alter.siamese.dyndns.org","threadId":"27388","inReplyTo":"20110518085932.GI27482@sigill.intra.peff.net","subject":"Re: Is there a debug mode for git-clone https traffic?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-05-18T15:38:47Z","receivedAt":"2011-05-18T15:38:47Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> Others have mentioned ways to dump data from clone. But you can more or\n> less see what's going on with \"git ls-remote <remote>\". The sha1 for\n> \"HEAD\" will try to match with the sha1 for some other ref. If it\n> doesn't, then we give up.\n\nOne thing noticed by somebody at $work is that we perform poorly when\nfetching from or pushing to a remote, which has an alternate, and when\nthat alternate has a dangling HEAD that does not point at a valid commit.\n\nI don't recall the exact details, but the story goes like...\n\nThe remote in the middle would want to mark the objects that are available\nfrom the alternate by internally running ls-remote against its alternate,\nwhich returns with a failure status, and then the entire serving process\nis aborted, and the original client does not get what it wanted from the\nremote.\n\n... or something like that.\n\nAs the internal discovery of what additional objects are available to the\nremote from its alternate is primarily for improving network performance\nby avoiding to transfer unnecessary objects, it is tempting to argue that\nthe remote could ignore the failure in ls-remote against the alternate.\n\nThe remote however _is_ relying on the alternate for some (and possibly\nlarge part of) objects it _thinks_ it has via its own refs, and an\nalternate that fails ls-remote because it has its refs messed up is a bad\nsign that does not give us confidence that the remote itself is healthy,\nso from that point of view, we might be doing the right thing by refusing\nto serve the original requestor to prevent the damage from spreading.\n"},{"id":"168487","messageId":"BANLkTinqKSwOYJOC93w-yyG79O9za7Z4yA@mail.gmail.com","threadId":"27388","inReplyTo":"BANLkTi=-i063MqrboT1MHrQy-vM0yBTUGg@mail.gmail.com","subject":"Re: Is there a debug mode for git-clone https traffic?","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2011-05-23T08:21:55Z","receivedAt":"2011-05-23T08:21:55Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Wed, May 18, 2011 at 01:08, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:\n> I''m trying to debug a problem where over a https proxy I have\n> \"warning: remote HEAD refers to nonexistent ref, unable to checkout.\"\n> when doing git-clone.\n>\n> I suspect that it might be an overzealous security scanner sitting in\n> the middle.\n>\n> Is there some debugging mode for git-clone where it'll dump the\n> traffic being sent over the wire that I could use to confirm this?\n\nI've debugged this, and the results are strange. Here it is on a\nmachine that doesn't work (proxy hostname manually munged):\n\n    Cloning into magit...\n    * Couldn't find host github.com in the .netrc file; using defaults\n    * About to connect() to proxy proxy-BROKEN.example.net port 3128 (#0)\n    *   Trying 10.146.207.60... * Connected to\nproxy-BROKEN.example.net (10.146.207.60) port 3128 (#0)\n    * Establish HTTP proxy tunnel to github.com:443\n    > CONNECT github.com:443 HTTP/1.1\n    Host: github.com:443\n    User-Agent: git/1.7.4.4\n    Proxy-Connection: Keep-Alive\n    Pragma: no-cache\n\n    < HTTP/1.0 200 Connection established\n    <\n    * Proxy replied OK to CONNECT request\n    * found 158 certificates in /etc/ssl/certs/ca-certificates.crt\n    > GET /magit/magit.git/info/refs?service=git-upload-pack HTTP/1.1\n    User-Agent: git/1.7.4.4\n    Host: github.com\n    Accept: */*\n    Pragma: no-cache\n\n    * Connection #0 to host proxy-BROKEN.example.net left intact\n    * Couldn't find host github.com in the .netrc file; using defaults\n    * Connection #0 seems to be dead!\n    * Closing connection #0\n    * About to connect() to proxy proxy-BROKEN.example.net port 3128 (#0)\n    *   Trying 10.146.207.60... * Connected to\nproxy-BROKEN.example.net (10.146.207.60) port 3128 (#0)\n    * Establish HTTP proxy tunnel to github.com:443\n    > CONNECT github.com:443 HTTP/1.1\n    Host: github.com:443\n    User-Agent: git/1.7.4.4\n    Proxy-Connection: Keep-Alive\n    Pragma: no-cache\n\n    < HTTP/1.0 200 Connection established\n    <\n    * Proxy replied OK to CONNECT request\n    * found 158 certificates in /etc/ssl/certs/ca-certificates.crt\n    > GET /magit/magit.git/HEAD HTTP/1.1\n    User-Agent: git/1.7.4.4\n    Host: github.com\n    Accept: */*\n    Pragma: no-cache\n\n    * Connection #0 to host proxy-BROKEN.example.net left intact\n    warning: remote HEAD refers to nonexistent ref, unable to checkout.\n\nAnd here on a box with a different proxy where this does work:\n\n    Cloning into magit...\n    * Couldn't find host github.com in the .netrc file, using defaults\n    * About to connect() to proxy proxy-OK.example.net port 3128\n    *   Trying 10.147.82.1... * connected\n    * Connected to proxy-OK.example.net (10.147.82.1) port 3128\n    * Establish HTTP proxy tunnel to github.com:443\n    > CONNECT github.com:443 HTTP/1.0\n    Host: github.com:443\n    User-Agent: git/1.7.2.1\n    Proxy-Connection: Keep-Alive\n    Pragma: no-cache\n\n    < HTTP/1.0 200 Connection established\n    <\n    * Proxy replied OK to CONNECT request\n    * successfully set certificate verify locations:\n    *   CAfile: /etc/pki/tls/certs/ca-bundle.crt\n      CApath: none\n    * SSL connection using AES256-SHA\n    * Server certificate:\n    *        subject: /O=*.github.com/OU=Domain Control\nValidated/CN=*.github.com\n    *        start date: 2009-12-11 05:02:36 GMT\n    *        expire date: 2014-12-11 05:02:36 GMT\n    *        subjectAltName: github.com matched\n    *        issuer: /C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com,\nInc./OU=http://certificates.godaddy.com/repository/CN=Go Daddy Secure\nCertification Authority/serialNumber=07969287\n    * SSL certificate verify ok.\n    > GET /magit/magit.git/info/refs?service=git-upload-pack HTTP/1.1\n    User-Agent: git/1.7.2.1\n    Host: github.com\n    Accept: */*\n    Pragma: no-cache\n\n    < HTTP/1.1 200 OK\n    < Server: nginx/0.7.67\n    < Date: Mon, 23 May 2011 08:14:22 GMT\n    < Content-Type: application/x-git-upload-pack-advertisement\n    < Transfer-Encoding: chunked\n    < Connection: keep-alive\n    < Expires: Fri, 01 Jan 1980 00:00:00 GMT\n    < Pragma: no-cache\n    < Cache-Control: no-cache, max-age=0, must-revalidate\n    * Connection #0 to host proxy-OK.example.net left intact\n    * Couldn't find host github.com in the .netrc file, using defaults\n    * Re-using existing connection! (#0) with host proxy-OK.example.net\n    * Connected to proxy-OK.example.net (10.147.82.1) port 3128\n    > POST /magit/magit.git/git-upload-pack HTTP/1.1\n    User-Agent: git/1.7.2.1\n    Host: github.com\n    Accept: */*\n    Accept-Encoding: deflate, gzip\n    Content-Type: application/x-git-upload-pack-request\n    Accept: application/x-git-upload-pack-result\n    Content-Length: 828\n\n    0073want 761d6ad09bfce0d354c8fe958f5fc1b0fdde0a9a\nmulti_ack_detailed side-band-64k thin-pack no-progress ofs-delta\n    0032want 292c83d70df3735ba3809dd946d99cf7aa49f6bd\n    0032want 1984ce646f25ec073afe77f9d90c11f422659c60\n    0032want fd6f6d5ea7b209f8968dfcd999a20f84e6e63edc\n    0032want b60cb2bdb31d78670690e22a7fa9eb1a5fb6a40c\n    0032want eb0da0582e6635e8b89636170a0de847a71d975e\n    0032want 387969a8734377e4baabaf2533dfad8f427dce5f\n    0032want cff8d04ef1fc0cce10a274e3737fbbe3ae9be43c\n    0032want 8be54f67577478ce05cf68a44a377cab893e9ea0\n    0032want 7b3473865309c8a6bc41d0e674ee648e124bf106\n    0032want 01aa8d5874b4ad19ffd0423e2c0e0123c5393051\n    0032want 00579d3e79504ce0be173b9cd0aec4100cbe8a76\n    0032want be85320fabef7af16fb1a27fa74908f54a1f2403\n    0032want b4b0a8bffc393137d3a52d62bc92c762a77067e9\n    0032want e4d766306adbda64a974b5ea1daa9b7fffdc29d6\n    00000009done\n    < HTTP/1.1 200 OK\n    < Server: nginx/0.7.67\n    < Date: Mon, 23 May 2011 08:14:22 GMT\n    < Content-Type: application/x-git-upload-pack-result\n    < Transfer-Encoding: chunked\n    < Connection: keep-alive\n    < Expires: Fri, 01 Jan 1980 00:00:00 GMT\n    < Pragma: no-cache\n    < Cache-Control: no-cache, max-age=0, must-revalidate\n    * Connection #0 to host proxy-OK.example.net left intact\n\nWhich is curious, because I can get that file manually with curl on\nboth of those boxes, i.e. the BROKEN and OK one, respectively:\n\n    $ curl -v https://github.com/magit/magit.git/info/refs?service=git-upload-pack\n| sha1sum\n    * About to connect() to proxy proxy-BROKEN.example.net port 3128 (#0)\n    *   Trying 10.146.207.60...   % Total    % Received % Xferd\nAverage Speed   Time    Time     Time  Current\n                                     Dload  Upload   Total   Spent\nLeft  Speed\n      0     0    0     0    0     0      0      0 --:--:-- --:--:--\n--:--:--     0connected\n    * Connected to proxy-BROKEN.example.net (10.146.207.60) port 3128 (#0)\n    * Establish HTTP proxy tunnel to github.com:443\n    > CONNECT github.com:443 HTTP/1.1\n    > Host: github.com:443\n    > User-Agent: curl/7.21.6 (i486-pc-linux-gnu) libcurl/7.21.6\nOpenSSL/1.0.0d zlib/1.2.3.4 libidn/1.20 libssh2/1.2.8 librtmp/2.3\n    > Proxy-Connection: Keep-Alive\n    >\n    < HTTP/1.0 200 Connection established\n    <\n    * Proxy replied OK to CONNECT request\n    * successfully set certificate verify locations:\n    *   CAfile: none\n      CApath: /etc/ssl/certs\n    * SSLv3, TLS handshake, Client hello (1):\n    } [data not shown]\n    * SSLv3, TLS handshake, Server hello (2):\n    { [data not shown]\n    * SSLv3, TLS handshake, CERT (11):\n    { [data not shown]\n    * SSLv3, TLS handshake, Server finished (14):\n    { [data not shown]\n    * SSLv3, TLS handshake, Client key exchange (16):\n    } [data not shown]\n    * SSLv3, TLS change cipher, Client hello (1):\n    } [data not shown]\n    * SSLv3, TLS handshake, Finished (20):\n    } [data not shown]\n    * SSLv3, TLS change cipher, Client hello (1):\n    { [data not shown]\n    * SSLv3, TLS handshake, Finished (20):\n    { [data not shown]\n    * SSL connection using AES256-SHA\n    * Server certificate:\n    *        subject: O=*.github.com; OU=Domain Control Validated;\nCN=*.github.com\n    *        start date: 2009-12-11 05:02:36 GMT\n    *        expire date: 2014-12-11 05:02:36 GMT\n    *        subjectAltName: github.com matched\n    *        issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com,\nInc.; OU=http://certificates.godaddy.com/repository; CN=Go Daddy\nSecure Certification Authority; serialNumber=07969287\n    *        SSL certificate verify ok.\n    > GET /magit/magit.git/info/refs?service=git-upload-pack HTTP/1.1\n    > User-Agent: curl/7.21.6 (i486-pc-linux-gnu) libcurl/7.21.6\nOpenSSL/1.0.0d zlib/1.2.3.4 libidn/1.20 libssh2/1.2.8 librtmp/2.3\n    > Host: github.com\n    > Accept: */*\n    >\n      0     0    0     0    0     0      0      0 --:--:-- --:--:--\n--:--:--     0< HTTP/1.1 200 OK\n    < Server: nginx/0.7.67\n    < Date: Mon, 23 May 2011 08:19:19 GMT\n    < Content-Type: application/x-git-upload-pack-advertisement\n    < Transfer-Encoding: chunked\n    < Connection: keep-alive\n    < Expires: Fri, 01 Jan 1980 00:00:00 GMT\n    < Pragma: no-cache\n    < Cache-Control: no-cache, max-age=0, must-revalidate\n    <\n    { [data not shown]\n    100  1531    0  1531    0     0   2773      0 --:--:-- --:--:--\n--:--:--  2877\n    * Connection #0 to host proxy-BROKEN.example.net left intact\n    * Closing connection #0\n    * SSLv3, TLS alert, Client hello (1):\n    } [data not shown]\n    5b7eb0b5c25a8700bfc8376a5a38da78724dc1dd  -\n\n    $ curl -v https://github.com/magit/magit.git/info/refs?service=git-upload-pack\n| sha1sum\n    * About to connect() to proxy proxy-OK.example.net port 3128\n    *   Trying 10.147.82.5... connected\n    * Connected to proxy-OK.example.net (10.147.82.5) port 3128\n    * Establish HTTP proxy tunnel to github.com:443\n    > CONNECT github.com:443 HTTP/1.0\n    > Host: github.com:443\n    > User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5\nOpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5\n    > Proxy-Connection: Keep-Alive\n    >\n    < HTTP/1.0 200 Connection established\n    <\n    * Proxy replied OK to CONNECT request\n    * successfully set certificate verify locations:\n    *   CAfile: /etc/pki/tls/certs/ca-bundle.crt\n      CApath: none\n    * SSLv2, Client hello (1):\n    SSLv3, TLS handshake, Server hello (2):\n    SSLv3, TLS handshake, CERT (11):\n    SSLv3, TLS handshake, Server finished (14):\n    SSLv3, TLS handshake, Client key exchange (16):\n    SSLv3, TLS change cipher, Client hello (1):\n    SSLv3, TLS handshake, Finished (20):\n    SSLv3, TLS change cipher, Client hello (1):\n    SSLv3, TLS handshake, Finished (20):\n    SSL connection using AES256-SHA\n    * Server certificate:\n    *        subject: /O=*.github.com/OU=Domain Control\nValidated/CN=*.github.com\n    *        start date: 2009-12-11 05:02:36 GMT\n    *        expire date: 2014-12-11 05:02:36 GMT\n    *        subjectAltName: github.com matched\n    *        issuer: /C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com,\nInc./OU=http://certificates.godaddy.com/repository/CN=Go Daddy Secure\nCertification Authority/serialNumber=07969287\n    * SSL certificate verify ok.\n    > GET /magit/magit.git/info/refs?service=git-upload-pack HTTP/1.1\n    > User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5\nOpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5\n    > Host: github.com\n    > Accept: */*\n    >\n    < HTTP/1.1 200 OK\n    < Server: nginx/0.7.67\n    < Date: Mon, 23 May 2011 08:18:47 GMT\n    < Content-Type: application/x-git-upload-pack-advertisement\n    < Transfer-Encoding: chunked\n    < Connection: keep-alive\n    < Expires: Fri, 01 Jan 1980 00:00:00 GMT\n    < Pragma: no-cache\n    < Cache-Control: no-cache, max-age=0, must-revalidate\n      % Total    % Received % Xferd  Average Speed   Time    Time\nTime  Current\n                                     Dload  Upload   Total   Spent\nLeft  Speed\n    100  1531    0  1531    0     0   3173      0 --:--:-- --:--:--\n--:--:--     0* Connection #0 to host proxy-OK.example.net left intact\n\n    * Closing connection #0\n    * SSLv3, TLS alert, Client hello (1):\n    5b7eb0b5c25a8700bfc8376a5a38da78724dc1dd  -\n\nThe SSL part seems different though, but since curl(1) can get the\ncontent perhaps there's some use of the curl library in Git that's\ntripping me up?\n"}]}